BEgridClientv5

From Begrid Wiki
Jump to navigationJump to search

PageOutline

BEgrid Quattor Client

I. Description

This pages describes in detail how to setup a 'BEgrid Quattor Client', which is a server (!) that will allow installation and configuration of one or more clusters on a particular site, for integration in BEgrid. The actual configuration of the site, as well as all necessary software packages, are retrieved from the 'Centralised BEgrid (CB) Repository'.

This 'BEgrid Quattor Client' can be installed on a dedicated server, but can also be run as a (Xen) virtual server. One option, for which setup instructions are provided, is to use one machine for NAT and Local DNS (for the worknodes), and have the BEgrid Quattor Client as a virtual server on that same box.

The 'BEgrid Quattor Client' will fulfill the following tasks:

  • Pxe-boot server
  • Reversed proxy + cache webserver
  • Fetch templates + search and replace + build
    • machine templates
    • ks+pxe profiles
  • AII Server: automatically contact all cluster nodes to get new profiles after build, so that all nodes always have an uptodate configuration.

I.1. Requirements

  • Some diskspace for the cache (15GB)
  • A certificate from a user that can connect to the CB
  • Firewall settings should be pretty tight: this server contains all host certificates ... !

I.2. Current situation

Origin:

  • quattor.begrid.be is the official Centralised BEgrid server.

II. Practical

II.1. Managing the BEgrid client

II.1.1 Access for admins

  • To get access to the Centralised BEgrid (CB), send your IP(range) to the contact person.
  • To get access to the CB-SCDB and to the SWREP-repository, send your BEGRID-cert DN to contact person.

Use begrid@belnet.be as contact person or begrid-tech@lists.belnet.beNOSPAM as technical mailing list.

II.1.2. Admin tools

To be installed on your normal pc/laptop.

Eclipse support

Also see Eclipse setup/config for BEgrid: Configure_Eclipse

SCDB client setup in official guide here.
Specific info for the new Panc v7 here.


Setup steps to checkout the centralised repository:

  1. go to Window -> Show view -> Other -> SVN -> SVN Repository
  2. rightclick -> New -> Repository Location
    1. Url: https://quattor.begrid.be/repos/centralised-begrid-v5/trunk
    2. Root: https://quattor.begrid.be/repos/centralised-begrid-v5
    3. Press Finish (no login/passwd needed, only your certificate.)
    4. If there's an error, make sure that you are allowed to access the repository and that your .subversion/servers file is correctly set.
  3. rightclick on https://quattor.begrid.be/repos/centralised-begrid-v5/trunk -> Checkout As -> Simple -> Project
    1. Give it a name: eg centralised-begrid-v5
    2. Finish
  4. In Navigator, there should be a folder called centralised-begrid-v5
    1. This folder contains a file .project. Rightclick -> Team -> add to: svn ignore
      1. With newer version of eclipse hidden files are not shown. So to be able to access them, Windows -> Navigation -> Show View Menu -> Filters
      2. Check the box called .* resources and press OK to confirm
    2. Create a new folder build (this one will contain all locally build xml-files, and these should never be uploaded). Rightclick -> Team -> add to: svn ignore

Script

  • svncheck: python script to help fetch/build etc

Quattor Configuration changes

Private info

Don't add passwords or any other form of secret information in the repository. In every cluster-configuration there's one directory called private that will be overwritten with files on the final build machine (ie the client machine) that can contain this private information. Instead of this, place files with eg dummy global variables, so that you can build and test the profiles locally.

  • e.g.: cfg/sites/begrid/private/passwd.tpl.
  • Notice that the values to assign to ROOT_PASSWD and AII_OSINSTALL_ROOTPW variables are MD5 hashes. Set the output of the following command for both of them. Choose preferably a different password for each (since AII_OSINSTALL_ROOTPW is added to the ks file, and the ks file is served through plain http).
openssl passwd -1
  • Other variables are set with plaintext passwords.
New files with dummy private information should be added in cfg/clusters/name_of_cluster/private !!


III. Installation instructions

III.1. Base install SL5

  • Get latest SL5
    • Get image from:
 wget http://linuxsoft.cern.ch/scientific/50/i386/images/boot.iso
    • Burn to CD (check with -scanbus):
 cdrecord dev=x,x,x boot.iso
    • Boot and install using http installation. Take eg fast CERN mirror:
 linuxsoft.cern.ch
 scientific/50/i386/
    • choose server, no firewall (to avoid complications (set it later!))
    • if you are not using the XEN based setup, make sure that the /var (either through separate partition or as part of /) has enough diskpace available for the rpm caching. (At least 15GB of free space needed for that).
    • Complete the install; choose a proper name/network config. This will depend on the way you want to use this server: as a 'BEgrid Client', or as a 'Xen master', on which you'll install a Xen client that is to become the 'BEgrid Client'


III.2. Base SL5 post-installation

  • Install the cb-client-forge rpm:
rpm -Uvh http://quattor.begrid.be/begrid/Central_BEGrid_Repository/i386_homebrew_el5/cb-client-forge-5.0.1-1.sl5.noarch.rpm
    • This provides access to a quattor begrid mirror of DAG, quattor and some cb-client homebuild rpms.
  • Change yum default repositories to CERN ones (faster and more reliable connection). Just run this line:
for i in <tt>ls /etc/yum.repos.d/</tt>;do sed -i 's#ftp://ftp.scientificlinux.org/linux#http://linuxsoft.cern.ch#' /etc/yum.repos.d/$i; done
  • Stop 'nightly yum update':
  service yum stop
  chkconfig --del yum
  • Install ntp (if not yet done)
  yum install ntp
  chkconfig --level 345 ntpd on 
  echo "server ntp.belnet.be" >> /etc/ntp.conf
  echo "restrict ntp.belnet.be mask 255.255.255.255 nomodify notrap noquery" >> /etc/ntp.conf
  service ntpd start
    • Change ntp.belnet.be to whatever timeserver you prefer.

Using Xen ?

  • If you are running your BEgrid Client on a Xen Virtual Machine, follow the instructions in this link; if you're installing the BEGrid CLient itself, continue here ...

III.3. CB-client

  • Check that your hostname is correct: the command hostname -f gives a FQDN, ie a hostname with domainname!). If it doesn't, fix this first.
  • Install basics. Now meta-package for the CB-client installation. It installs
    • httpd rpms
    • AII rpms
    • SINDES rpms
    • svncheck rpms
    • swrep-soap-client rpms
yum install cb-client
    • this might complain about Package perl-LC-1.0.11-1.noarch.rpm is not signed. Set gpgcheck=0 in /etc/yum.conf and change it after the installation.


svncheck

  • svncheck is installed through an rpm cb-client-svncheck and is part of the cb-client install
  • files are located in /opt/CB5
  • The Centralised-begrid (/opt/CB5) folder has the following structure:
    • /opt/CB5/keys: this one holds the begrid CA certificate and a valid user .p12 file. (This is used to connect to the SCDB-server.)
    • /opt/CB5/subversion: some subversion specific parameters. edit the servers file:
      • correct full path to key (.p12 file)
      • plaintext passwd for the key (it does not prompt for the passwd)
    • /opt/CB5/tmp: will contain the checkout and build files.
    • /opt/CB5/private: here you can put private files (such as passwords and certificate in the templates. passwd.tpl; pub_key.tpl)
      • svncheck does this by simple copy from this directory into cfg/clusters. So keep that structure.
      • remove the template cluster given as example, otherwise runcheck will try to build it later ...
      • /opt/CB5/private/<clustername-glite-version>/passwd.tpl
          • This file contains the passwords that will be used for your site.
          • You can pick any password you like.
          • (Unless certain nodes are not configured with Quattor, in that case they must match which the non Quattor nodes).
      • /opt/CB5/private/<clustername-glite-version>/local_users.tpl
          • ???
          • Not needed for a CE or a WN.
      • /opt/CB5/private/<clustername-glite-version>/pub_key.tpl
      • /opt/CB5/private/<clustername-glite-version>/<your_ce_fqdn>.tpl
    • /opt/CB5/svncheck: this is the code written by Jean-François Roche (jfroche@jfroche.be):
      • copy config.conf.orig to config.conf
      • in config.conf you can specify most needed parameters.
      • svn_repos: point it to the trunk of the centralised-begrid repository. building tags relies on this!
        • default should be fine
      • cluster_regexp: the ant task to build the cluster without the compile.profiles part.
        • it has to start with a ., eg for IIHE this is .iihe-glite
      • DON'T FORGET to change the email section
  • ./runcheck -h for more info


SINDES

  • Rpms and files provided by cb-client-sindes rpm and is part of the cb-client install
  • More info and detailed full instructions here
  • httpd config should be in /etc/httpd/conf.d/sindes-ssl.conf.begrid
mv /etc/httpd/conf.d/sindes-ssl.conf /etc/httpd/conf.d/sindes-ssl.conf-orig
cp /etc/httpd/conf.d/sindes-ssl.conf.begrid /etc/httpd/conf.d/sindes-ssl.conf
  • edit /etc/sindes/ca.config, make sure to set O, OU and CN.
    • the CN of the CA certificate should be different from the certificate of the host of the CA.
    • meaning: don't use only the machine fqdn, add eg CA
    • example
 [ req_distinguished_name ]
 O                      = IIHE
 OU                     = GRID
 CN                     = Local BEgrid client CA
  • edit /etc/sindes/sindesrc
    • in [MISC], set correctly the domain (and use it correctly in sindessh !)
    • use the longest common domain between your nodes
    • in SINDES, all machines will be known as their fqdn without this string.
  • generate the CA certificates and check the output for potential errors
sindes-bootstrap-ca -a
    • This also generates a rpm with the CA certifiacte called eg SINDES-ca-certificate-<machine>-0.1-1.noarch.rpm
    • you need to provide a passphrase for the apache.key.
    • upload the SINDES-ca-certificate-<machine>-0.1-1.noarch.rpm to the quattor-repository (or ask someone else to do it).
    • instructions on how to do it yourself
  • quick check: try to login as sindes user and run acl -print
# su - sindes
SINDESsh  > acl -print
�----------------------------------------------�
|       hostname          TTL     Request Right|
�----------------------------------------------�
�----------------------------------------------�

webservice

  • Configuration for the reverse proxy + cache:
    • Taken from mod_cache http://httpd.apache.org/docs/2.2/mod/mod_cache.html and mod_proxy http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
    • Reverse proxy is the only one supported by Quattor: your profiles will point to the rpm repository at quattor.begrid.be, but in fact your local BEgrid Client will get the rpms, (in theory optionally) cache them, and provide them to node that is being installed.
    • Using a disk cache is preferred to lower the load on the CB and the network (and it should be faster)
    • generated by cb-client-extra
    • There should be file in /etc/httpd/conf.d/cb-cache.conf with the necessary config settings
    • There should also be a cronjob /etc/cron.hourly/htcacheclean-cron.sh to clean up the cache
  • restart httpd and watch the output:
  service httpd restart
    • This will prompt for passwd. To get rid of this, see howto remove passwd from apache.key in /etc/sindes/keys)
    • Output 1: [warn] module <modul name> is already loaded, skipping.
      • This means that the modules were already loaded in httpd.conf (or elsewhere). This error can be ignored or cleaned up by removing the duplicate LoadModule entries.
    • Output 2: Stopping httpd: [FAILED].
      • This means that httpd was not running by default and should be added to the default startup processes:
  chkconfig --add httpd
  chkconfig --level 3 httpd on
    • [error] VirtualHost _default_:443 -- mixing * ports and non-* ports with a NameVirtualHost address is not supported, proceeding with undefined results
      • ignore?

AII

  • Rpms and files provided by cb-client-aii rpm are part of the cb-client install
  • There should be a basic confguration file at /etc/aii-shellfe.conf
    • generated by cb-client-extra
  • modify /usr/share/doc/aii-<version>/eg/dhcpd.conf and copy it to /etc/dhcpd.conf
    • Example for IIHE
# 
# DHCPD Config far AII
#
# Uncommnent this line if ISC DHCP ver. 3
ddns-update-style ad-hoc;
# write here your network name
shared-network iihe.ac.be {
    deny unknown-clients;
    not authoritative;
    # Write here your domain name
    option domain-name "iihe.ac.be";
    # Parameters for the installation via PXE using pxelinux
    filename                           "pxelinux.0";
    # Uncommnent this line if ISC DHCP ver. 2
    # option dhcp-class-identifier       "PXEClient";
    # Uncommnent this line if ISC DHCP ver. 3
    option vendor-class-identifier       "PXEClient";
    option vendor-encapsulated-options 01:04:00:00:00:00:ff;
    # Complete with (at least) the gateway + DNS.
    # Hosts entries will be inserted
    # automatically by AII in this section
    subnet 193.190.246.0 netmask 255.255.255.0 {
      option routers 193.190.246.65;
      option domain-name-servers 193.190.246.229;
    }
    # remove the following subnet if you are not using 
    # private network otherwise keep it and adapt it 
    # your site
    subnet 192.168.0.0 netmask 255.255.0.0 {
    option routers 192.168.10.100;
    option domain-name-servers 192.168.10.100;
    }
  }
  • add the dhcp deamon at the boot:
 chkconfig --add dhcpd
 chkconfig --level 345 dhcpd on
  • configure syslinux and tftp-server (last one uses hosts.* for acl):
    • download tarball with default images
    • locate the localboot.cfg file from the distribution and copy it to /osinstall/nbp/pxelinux.cfg
    • manual: download for each os distribution supported by CB the pxeimages and put them in a correctly named directory
    • directory names should be identical as what is found under cfg/os/ BUT with the - replaced with _
  mkdir /osinstall/nbp/slc450_i386
  cd /osinstall/nbp/slc450_i386
  wget http://linuxsoft.cern.ch/cern/slc450/i386/images/pxeboot/vmlinuz
  wget http://linuxsoft.cern.ch/cern/slc450/i386/images/pxeboot/initrd.img
  ln -s /osinstall/ks /var/www/html/ks
  • in /etc/xinetd.d/tftp modify the following options
  server_args             = -s /osinstall/nbp
  disable                 = no
  • restart the corresponding service
  service xinetd restart
  • the default firewall settings of SL5 block tftp traffic (and probably also eg http to port 444 for SINDES).
    • Either configure the firewall properly or disbales iptables altogether.
/etc/init.d/iptables stop
chkconfig iptables off
chkconfig --del iptables


  • allow acknowledgment script to do its work:
    • copy the script to the default cgi directory:
  cp /usr/sbin/aii-installack.cgi /var/www/cgi-bin
  chmod o+rx /var/www/cgi-bin/aii-installack.cgi
    • add apache to /etc/sudoers
    • MUST be done for private interfaces (with private fqdn) as well!! (not done by this line of code):
  echo "apache  $(hostname -f)=(ALL)     NOPASSWD: /usr/sbin/aii-shellfe" >> /etc/sudoers
    • Also comment in /etc/sudoers
Defaults    requiretty
    • a quick check for parsing errors can be done with running command sudo
      • if all is fine, you will get something like 


usage: sudo -K | -L | -V | -h | -k | -l | -v
usage: sudo [-HPSb] [-p prompt] [-u username|#uid]
            { -e file [...] | -i | -s | <command> }



      • else, you will see


>>> sudoers file: syntax error, line 89 <<<
sudo: parse error in /etc/sudoers near line 89



    • quick test for the functionality


wget -q --output-document=- "http://$(hostname -f)/cgi-bin/aii-installack.cgi"



    • output should be an error like (since the CB client is not setup by quattor)


[ERROR] No node matches gridy(.begrid.be)?
[INFO] aii-installack: host 'gridy.begrid.be' configured to boot from local disk



      • if something goes wrong you will get an error like


[ERROR] aii-installack: error while executing command: /usr/bin/sudo /usr/sbin/aii-shellfe --boot gridy.begrid.be --nodhcp --noosinstall



SWREP


To be able to the run the python script used for uploading rpms and CA cert to the central repository follow this procedure.



  • Install pyOpenSSL package


yum install pyOpenSSL



  • Install the correct perl-Net-SSLeay package


rpm -e --nodeps perl-Net_SSLeay-1.25-sl3
yum install perl-Net-SSLeay



  • Example of usage for CA cert update (for more detail use the --help option)


/opt/CB5/tmp/src/begrid/cb-client/cb-client-swrep/swrep.py --mode ca










 Back to BEgrid_And_Quattor page




Template:TracNotice









Retrieved from "https://wiki.begrid.be/index.php?title=BEgridClientv5&oldid=20"