AddNewWN

From Begrid Wiki
Revision as of 09:11, 9 June 2021 by Maintenance script (talk | contribs) (Created page with " ==== Copy/Edit a new WN profile ==== *Located in <tt>cfg/sites/yoursite-glite-31/profiles/</tt> *Name of the WN should be profile_nodeXX-YY.yourdomain.be **Example : <tt>c...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Copy/Edit a new WN profile

  • Located in cfg/sites/yoursite-glite-31/profiles/
  • Name of the WN should be profile_nodeXX-YY.yourdomain.be
    • Example : cfg/sites/kuleuven-glite-31/profiles/profile_node11-1.wn.cc.kuleuven.ac.be
  • Type :
include machine-type/wn;
  • Package repository :
include { PKG_REPOSITORY_CONFIG };

Add a hardware template

  • Located in cfg/sites/yoursite-production/site/hardware/machine/
    • Example : cfg/sites/ulb-vub/hardware/machine/sunfire_v20z_s1
  • Set the correct values for (at least):
    • CPU arch
    • harddisk (type+size)
    • memory
    • Ethernet interface(s) driver
    • MAC address(es)

Adapt Databases

  • In cfg/sites/yoursite-production/site/databases.tpl :
    • Associate the WN with its IP address
    • Associate the WN with its hardware template
  • In cfg/sites/yoursite-production/site/os_version_db.tpl :
    • Associate the WN with its OS version

Adapt config-grid.tpl

  • Located in cfg/sites/yoursite-production/site/config-grid.tpl
  • Add the WN hostname in the existing list:
variable WORKER_NODES = list(
...
  "nodeXX-YY.yourdomain.be",
...
);
  • If you have nodes with more than one CPU, add also in the following list the WN hostname together with the number of CPU (separated by a coma) :
variable WN_CPUS = list(
...
  "nodeXX-YY.yourdomain.be",2,
...
);

Bulk generation

Because several files need to be created/adapted, we have delevoped a tool to generate most of templates. Ofcourse this is only possible when enough information is provided to the tool.

The script starts from a hardware template and a descrition file with at least the macaddresses. Further configuration can be obtained by either completing the description file, or by providing a 'generator' for the values. The generator in this case is a string ending with an integer. This integer is increased with 1 for every (correct) line in the description file.

It is also possible to generate the mapping between the machine FQDN and IPaddress and between FQDN and the hardware template. To get the machine name, one can put it in the description file, use the generator or when the IP address can be determined, use reverse DNS lookup. The ip is obtained either from the description file, a generator or DNS lookup.

And if the machinename can be resolved, it is also possible to generate the object templates, given the machine-type.

Usage

Download the script: 

wget -O hardware_template_generator.py 'http://quattor.begrid.be/trac/centralised-begrid-v4/attachment/wiki/AddNewWN/hardware_template_generator.py?format=raw' && chmod +x hardware_template_generator.py 


Usage :
        --template: name of the hardware template example
        --desc: name of file with the descriptions
                FORMAT: (NAME)? (MAC_ADDRESS)+ (IP_ADDRESS)? (SERIAL)? (LOCATION)?
        --gen_digit: the generator will create names with (at least) this number of digits (eg to make the node001 names).
        --ip_gen: ipaddress to start the ip generation from (+1 for each line)
        --name_gen: the naming generator. (Has to end in an integer.) (+1 for each line)
        --serial_gen: the serialnumber generator. (Has to end in an integer.) (+1 for each line.)
        --location_gen: the location generator. (Has to end in an integer.) (+1 for each line.)
        --templatename_gen: the hardware template name generator. (Has to end in an integer.) (+1 for each line.)

        --domain: domain to use for nslookup

        --leftright: add left or right to the template name, the location and the serialnumber. The value indicates wheter to start with left or right. (also supports a or b)

        --object: will create the object templates if the name of the machine can be determined: value is the name of teh machine-type (eg wn).
        --object_prefix: the name of the object template is made of this prefix and the fqdn (default prefix: profile_ ).

        --os: OS description string (to be used in the os_version_db)
        --wn_cpu: the number of jobs/cpu (to be used with variable WN_CPUS)

        --outdir: output basedir (default /tmp/quattor)
        --indir: input dir (default $PWD)

        --help: print this

Example: using a description file called test_1 

00:00:00:00:00:00
00:00:00:00:00:01
00:00:00:00:00:02

and a hardware template test_hardware_s1.tpl 

structure template hardware/machine/test_hardware_s1;

"smp" = false;

"architecture" = nlist(
  "glibc","i686",
  "kernel","i686",
  "kernel_smp","i686",
  "openssl","i686"
);

"location" = "rack3_s1";
"serialnumber" = "s1";

"cpu" = list(create("hardware/cpu/very_fast"));

"harddisks" = nlist("sda", create("hardware/harddisk/big"));

"ram" = list(create("hardware/ram/a_lot_of_ram"));

"cards/nic" = nlist("eth0",create("hardware/nic/special"));

"cards/nic/eth0/hwaddr"              = "00:30:48:79:C2:08";

"cards/nic/eth0/boot" = true;

the following commandline 

./hardware_template_generator.py --template=test_hardware_s1.tpl --desc=test_1 --name_gen=node11-1 --domain=wn.iihe.ac.be --serial=s1 --location=rack1_p1 --templatename_gen=hardware_t1 --object=wn

will print (NOTE: all nodes node11-1.wn.iihe.ac.be, node11-2.wn.iihe.ac.be and node11-3.wn.iihe.ac.be are in the DNS!) 

[stdweird@localhost quatt]$ ./hardware_template_generator.py --template=test_hardware_s1.tpl --desc=test_1 --name_gen=node11-1 --domain=wn.iihe.ac.be --serial=s1 --location=rack1_p1 --templatename_gen=hardware_t1 --object=wn
Wrote /tmp/quattor/profiles/profile_node11-1.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t1.tpl
Wrote /tmp/quattor/profiles/profile_node11-2.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t2.tpl
Wrote /tmp/quattor/profiles/profile_node11-3.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t3.tpl

To add to the databases:
FQDN - IP
   escape("node11-1.wn.iihe.ac.be"),"192.168.11.1",
   escape("node11-2.wn.iihe.ac.be"),"192.168.11.2",
   escape("node11-3.wn.iihe.ac.be"),"192.168.11.3",

FQDN - hardware template
   escape("node11-1.wn.iihe.ac.be"),"hardware/machine/hardware_t1",
   escape("node11-2.wn.iihe.ac.be"),"hardware/machine/hardware_t2",
   escape("node11-3.wn.iihe.ac.be"),"hardware/machine/hardware_t3",

List of WNs (to add to WORKER_NODES)
"node11-1.wn.iihe.ac.be","node11-2.wn.iihe.ac.be","node11-3.wn.iihe.ac.be",

and create 

[stdweird@localhost quatt]$ find /tmp/quattor/
/tmp/quattor/
/tmp/quattor/site
/tmp/quattor/profiles
/tmp/quattor/profiles/profile_node11-1.wn.iihe.ac.be.tpl
/tmp/quattor/profiles/profile_node11-3.wn.iihe.ac.be.tpl
/tmp/quattor/profiles/profile_node11-2.wn.iihe.ac.be.tpl
/tmp/quattor/hardware
/tmp/quattor/hardware/hardware_t1.tpl
/tmp/quattor/hardware/hardware_t3.tpl
/tmp/quattor/hardware/hardware_t2.tpl

with 

[stdweird@localhost quatt]$ cat /tmp/quattor/profiles/profile_node11-3.wn.iihe.ac.be.tpl
object template profile_node11-3.wn.iihe.ac.be;




include machine-types/wn;

#
# software repositories (should be last)
#
include { PKG_REPOSITORY_CONFIG };

and 

[stdweird@localhost quatt]$ cat /tmp/quattor/hardware/hardware_t3.tpl 

structure template hardware/machine/hardware_t3;

"smp" = false;

"architecture" = nlist(
  "glibc","i686",
  "kernel","i686",
  "kernel_smp","i686",
  "openssl","i686"
);


"location" = "rack1_p3";

"serialnumber" = "s3";

"cpu" = list(create("hardware/cpu/very_fast"));

"harddisks" = nlist("sda", create("hardware/harddisk/big"));

"ram" = list(create("hardware/ram/a_lot_of_ram"));

"cards/nic" = nlist("eth0",create("hardware/nic/special"));

"cards/nic/eth0/hwaddr" = "00:00:00:00:00:02";

"cards/nic/eth0/boot" = true;

Slightly more advanced example (makes only sense for WNs) (identical description file and hardware template): command line 

./hardware_template_generator.py --template=test_hardware_s1.tpl --desc=test_1 --name_gen=node11-1 --domain=wn.iihe.ac.be --serial=s1 --location=rack1_p1 --templatename_gen=hardware_t1 --object=wn  --leftright=b --gen_digit=3 --os=x86_64_slc4_45 --wn_cpu=5

gives as output 


Wrote /tmp/quattor/profiles/profile_node11-001.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t001_b.tpl
Wrote /tmp/quattor/profiles/profile_node11-002.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t002_a.tpl
Wrote /tmp/quattor/profiles/profile_node11-003.wn.iihe.ac.be.tpl
Wrote /tmp/quattor/hardware/hardware_t003_b.tpl

To add to the databases:
FQDN - IP
   escape("node11-001.wn.iihe.ac.be"),"192.168.11.1",
   escape("node11-002.wn.iihe.ac.be"),"192.168.11.2",
   escape("node11-003.wn.iihe.ac.be"),"192.168.11.3",

FQDN - hardware template
   escape("node11-001.wn.iihe.ac.be"),"hardware/machine/hardware_t001_b",
   escape("node11-002.wn.iihe.ac.be"),"hardware/machine/hardware_t002_a",
   escape("node11-003.wn.iihe.ac.be"),"hardware/machine/hardware_t003_b",

FQDN - os_version
   escape("node11-001.wn.iihe.ac.be"),"x86_64_slc4_45",
   escape("node11-002.wn.iihe.ac.be"),"x86_64_slc4_45",
   escape("node11-003.wn.iihe.ac.be"),"x86_64_slc4_45",

List of WNs (to add to WORKER_NODES)
"node11-001.wn.iihe.ac.be","node11-002.wn.iihe.ac.be","node11-003.wn.iihe.ac.be",

List of WNs and number of jobs/cpu (to add to WN_CPUS)
"node11-001.wn.iihe.ac.be",5,"node11-002.wn.iihe.ac.be",5,"node11-003.wn.iihe.ac.be",5,


Description file

Everything defined in the description file has preference over eg the generators.

Each machine has one line of like 

(NAME)? (MAC_ADDRESS)+ (IP_ADDRESS)? (SERIAL)? (LOCATION)?

The ? and + signs mean optional and at least one. The following are correct all descriptions: 

00:00:00:00:00:00
machine1 00:00:00:00:00:01
machine2 00:00:00:00:00:02 00:00:00:00:00:03
00:00:00:00:00:04 192.168.1.1
00:00:00:00:00:05 192.168.1.2 seri_1 rack_5_p2


Back to BEgrid_And_Quattor page



Template:TracNotice

Retrieved from "https://wiki.begrid.be/index.php?title=AddNewWN&oldid=3"