Users gLite WMS

Overview

Using glite-wms-* commands, your jobs will go through WMS servers.

The Workload Management System (WMS) is responsible for distributing and managing tasks across computing and storage resources available on a Grid. WMS assigns user jobs to CEs and SEs belonging to a Grid environment in a convenient fashion, so that:

- jobs are always executed on resources that match the job requirements

- grid-wide load balance is maintained, i.e. jobs are evenly and efficiently distributed across the entire Grid.

Certificate Usage

We assume here that you followed these steps, so:

• you are provided with a certificate,
• you belong to a VO (eg. VO beapps),
• you have access to a UI (eg m0.iihe.ac.be, ui.begrid.be,...),
• you have generated your usercert.pem and userkey.pem files in directory .globus.

Before sending any command to the grid, you will need to generate a grid proxy certificate for the VO you wish to operate in; run this:

 $ voms-proxy-init --voms beapps

Then you will be prompted to type your userkey.pem password. This generates a proxy (limited certificate).

To see the validity time of your proxy and extra information, issue this:

 $ voms-proxy-info --all

If you want to run jobs that will take more than the default time (12 hours), you can generate it with a longer validity. This will make your proxy certificate valid for 72 hours:

 $ voms-proxy-init -hours 72

JDL

Jobs submitted to the grid are described using a specific language, the Job Description Language (JDL). Here we provide a very basic sample :

[
 VirtualOrganisation = "beapps";
                                 # the VO you belong to
 JobType = "normal";
                                 # in general this is normal or mpich
 Type = "Job";
                                 # this is a job :-)
 Executable = "cat";
                                 # name of the command without arguments
 Arguments = "/proc/cpuinfo";
                                 # arguments if necessary
 StdOutput = "output";
                                 # to put the standard output (display) in the file we call output
 StdError = "error";
                                 # to put the standard error in the file we call error
 InputSandbox = "/bin/cat";
                                 # to send the binary (or the script) and all necessary input files
 OutputSandbox = {
     "output",
     "error"     };
                                 # to receive the output files
]

More options for the JDL file can be found on this site: https://wiki.italiangrid.it/twiki/bin/view/CREAM/JdlGuide.

Remark: Don't send/receive input/output sandboxes greater than 20 MB. For greater files usage, look at section Advanced data access with gridftp.

Submission

To submit a job, just use this:

 glite-wms-job-submit -a <your jdl file>

This should display the following lines:

 $ glite-wms-job-submit -a GetInfo.jdl
 
 Connecting to the service https://wms01.begrid.be:7443/glite_wms_wmproxy_server
 
 
 ====================== glite-wms-job-submit Success ======================
 
 The job has been successfully submitted to the WMProxy
 Your job identifier is:
 
 https://wms01.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 
 ==========================================================================

This gives you the jobID:

 https://wms01.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA

Normally, this url also works as a website: opening this url in your browser (equipped with your certificate) shows you the status of that job. Read on to know what you can do with it...

Monitoring

You can follow the different steps of your job execution with:

 glite-wms-job-status <jobID>

Example of output:

 $ glite-wms-job-status https://wms01.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 
 
*************************************************************
 BOOKKEEPING INFORMATION:
 
 Status info for the Job : https://wms01.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 Current Status:     Scheduled
 Status Reason:      Job successfully submitted to Globus
 Destination:        kg-ce01.cc.kuleuven.be:2119/jobmanager-pbs-beapps
 Submitted:          Mon Jun 23 22:59:01 2008 CEST
*************************************************************

You can collect several jobID's in a file called jobslist, using the following command repeatedly:

 glite-wms-job-submit -a --nomsg -o jobslist <your jdl file>

Then use this file like this with a lot of glite commands:

 $ glite-wms-job-status -i jobslist
 
 ------------------------------------------------------------------
 1 : https://wms01.begrid.be:9000/h4F7qonPxc2NVzdqDVl6ZQ
 2 : https://wms01.begrid.be:9000/WVAFM1FEjar96V1HzpOebA
 3 : https://wms01.begrid.be:9000/5kF_xAQWge0Y8VJvdsttkA
 a : all
 q : quit
 ------------------------------------------------------------------
 
 Choose one or more jobId(s) in the list - [1-3]all:

Now you can choose which jobs you want to get the status of (or all).

Fetching Outputs

When your job enters the "Done (Success)" status, you are able to retrieve the output files (the ones you specified in the ouput sandbox of the jdl file). Use this command:

 glite-wms-job-output --dir <path to where you want your outputs> <jobID>

or if you have collected your jobIDs in a file:

 glite-wms-job-output --dir <path to where you want your outputs> -i jobslist

In this directory, all the files you specified in the output sandbox will be available:

 $ ls -lrta
 total 9
 drwxr-xr-x  19 jo users  992 Jun 23 23:21 ../
 -rw-r--r--   1 jo users 4824 Jun 23 23:21 output
 -rw-r--r--   1 jo users    0 Jun 23 23:21 error
 drwxr-xr-x   2 jo users   96 Jun 23 23:21 ./