Start your first jobs with wms
Certificate Usage
We assume here that you followed these steps, so:
- you are provided with a certificate,
- you belong to a VO (eg. VO beapps),
- you have access to a UI (eg login.vic.cc.kuleuven.be),
- you have copied your usercert.pem and userkey.pem files in directory .globus.
Before being able to use the grid commands, you need to load the BEgrid environment; on the K.U.Leuven User Interface 'vic', this can be done with this command:
$ module load begrid
Having that, before sending any command to the grid, you will need to generate a grid proxy certificate for the VO you wish to operate in; run this:
$ glite-voms-proxy-init --voms beapps
Then you will be prompted to type your userkey.pem password. This generates a proxy (limited certificate).
To see the validity time of your proxy and extra information, issue this:
$ glite-voms-proxy-info --all
If you want to run jobs that will take more than the default time (12 hours), you can generate it with a longer validity. This will make your proxy certificate valid for 72 hours:
$ glite-voms-proxy-init -hours 72
JDL
Jobs submitted to the grid are described using a specific language, the Job Description Language (JDL). Here we provide a very basic sample :
VirtualOrganisation = "beapps";
# the VO you belong to
JobType = "normal";
# in general this is normal or mpich
Type = "Job";
# this is a job :-)
Executable = "cat";
# name of the command without arguments
Arguments = "/proc/cpuinfo";
# arguments if necessary
StdOutput = "output";
# to put the standard output (display) in the file we call output
StdError = "error";
# to put the standard error in the file we call error
InputSandbox = "/bin/cat";
# to send the binary (or the script) and all necessary input files
OutputSandbox = {
"output",
"error" };
# to receive the output files
When testing a new job, it is preferred that you execute it in the site of your institute. To specify this to your JDL file, just add the following line to the others:
Requirements = RegExp(".*.kuleuven.be", other.GlueCEUniqueId);
(in this case a site within the kuleuven.be domain will be selected)
More options for the JDL file can be found on this site: https://edms.cern.ch/file/454439//LCG-2-UserGuide.pdf.
Remark: Don't send/receive input/output sandboxes greater than 20 MB. For greater files usage, look at section Advanced data access with gridftp.
Submission
Before submitting you can list the sites that can execute your job according to your requirements. This is done by issuing the following command:
glite-wms-job-list-match -a <your jdl file>
This returns only the K.U.Leuven site because of the requirement we specified:
$ glite-wms-job-list-match -a GetInfo.jdl
Connecting to the service https://wms.begrid.be:7443/glite_wms_wmproxy_server
==========================================================================
COMPUTING ELEMENT IDs LIST
The following CE(s) matching your job requirements have been found:
*CEId*
- kg-ce01.cc.kuleuven.be:2119/jobmanager-pbs-beapps
==========================================================================
To submit a job, just use this:
glite-wms-job-submit -a <your jdl file>
This should display the following lines:
$ glite-wms-job-submit -a GetInfo.jdl Connecting to the service https://wms.begrid.be:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA ==========================================================================
This gives you the jobID:
https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
Since recently, this url also works as a website: opening this url in your browser (equiped with your certificate) shows you the status of that job. Read on to know what you can do with it...
Monitoring
You can follow the different steps of your job execution with:
glite-wms-job-status <jobID>
Example of output:
$ glite-wms-job-status https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA Current Status: Scheduled Status Reason: Job successfully submitted to Globus Destination: kg-ce01.cc.kuleuven.be:2119/jobmanager-pbs-beapps Submitted: Mon Jun 23 22:59:01 2008 CEST *************************************************************
You can collect several jobID's in a file called jobslist, using the following command repeatedly:
glite-wms-job-submit -a --nomsg -o jobslist <your jdl file>
Then use this file like this with a lot of glite commands:
$ glite-wms-job-status -i jobslist ------------------------------------------------------------------ 1 : https://wms.begrid.be:9000/h4F7qonPxc2NVzdqDVl6ZQ 2 : https://wms.begrid.be:9000/WVAFM1FEjar96V1HzpOebA 3 : https://wms.begrid.be:9000/5kF_xAQWge0Y8VJvdsttkA a : all q : quit ------------------------------------------------------------------ Choose one or more jobId(s) in the list - [1-3]all:
Now you can choose which jobs you want to get the status of (or all).
Fetching Outputs
When your job enters the "Done (Success)" status, you are able to retrieve the output files (the ones you specified in the ouput sandbox of the jdl file). Use this command:
glite-wms-job-output --dir <path to where you want your outputs> <jobID>
or if you have collected your jobIDs in a file:
glite-wms-job-output --dir <path to where you want your outputs> -i jobslist
In this directory, all the files you specified in the output sandbox will be available:
$ ls -lrta total 9 drwxr-xr-x 19 jo users 992 Jun 23 23:21 ../ -rw-r--r-- 1 jo users 4824 Jun 23 23:21 output -rw-r--r-- 1 jo users 0 Jun 23 23:21 error drwxr-xr-x 2 jo users 96 Jun 23 23:21 ./
