View source for Start your first jobs with wms


=== Certificate Usage ===

We assume here that you [[Access_to_BEgrid| followed these steps]], so:
*you are provided with a certificate,
*you belong to a VO (eg. VO beapps),
*you have access to a UI (eg login.vic.cc.kuleuven.be),
*you have copied your <tt>usercert.pem</tt> and <tt>userkey.pem</tt> files in directory <tt>.globus</tt>.

Before being able to use the grid commands, you need to load the BEgrid environment; on the K.U.Leuven User Interface 'vic', this can be done with this command:
<pre>
 $ module load begrid
</pre>
Having that, before sending any command to the grid, you will need to generate a grid proxy certificate for the VO you wish to operate in; run this:
<pre>
 $ glite-voms-proxy-init --voms beapps
</pre>
Then you will be prompted to type your <tt>userkey.pem</tt> password. This generates a ''proxy'' (limited certificate).

To see the validity time of your proxy and extra information, issue this:
<pre>
 $ glite-voms-proxy-info --all
</pre>
If you want to run jobs that will take more than the default time (12 hours), you can generate it with a longer validity. This will make your proxy certificate valid for 72 hours:
<pre>
 $ glite-voms-proxy-init -hours 72
</pre>
=== JDL ===

Jobs submitted to the grid are described using a specific language, the Job Description Language (JDL).
Here we provide a very basic sample :
<pre>
 VirtualOrganisation = "beapps";
                                 # the VO you belong to
 JobType = "normal";
                                 # in general this is normal or mpich
 Type = "Job";
                                 # this is a job :-)
 Executable = "cat";
                                 # name of the command without arguments
 Arguments = "/proc/cpuinfo";
                                 # arguments if necessary
 StdOutput = "output";
                                 # to put the standard output (display) in the file we call output
 StdError = "error";
                                 # to put the standard error in the file we call error
 InputSandbox = "/bin/cat";
                                 # to send the binary (or the script) and all necessary input files
 OutputSandbox = {
     "output",
     "error"     };
                                 # to receive the output files
</pre>
When '''testing''' a new job, it is preferred that you execute it in the site of your institute. To specify this to your JDL file, just add the following line to the others:
<pre>
 Requirements = RegExp(".*.kuleuven.be", other.GlueCEUniqueId);
</pre>
(in this case a site within the ''kuleuven.be'' domain will be selected)

More options for the JDL file can be found on this site: https://edms.cern.ch/file/454439//LCG-2-UserGuide.pdf.

'''Remark''': Don't send/receive input/output sandboxes greater than 20 MB.
For greater files usage, look at section [[Advanced_data_access_with_gridftp| Advanced data access with gridftp]].

=== Submission ===

Before submitting you can list the sites that can execute your job according to your requirements. This is done by issuing the following command:
<pre>
 glite-wms-job-list-match -a <your jdl file>
</pre>
This returns only the K.U.Leuven site because of the requirement we specified:
<pre>
 $ glite-wms-job-list-match -a GetInfo.jdl
 
 Connecting to the service https://wms.begrid.be:7443/glite_wms_wmproxy_server
 
 ==========================================================================
 
                      COMPUTING ELEMENT IDs LIST
  The following CE(s) matching your job requirements have been found:
 
         *CEId*
  - kg-ce01.cc.kuleuven.be:2119/jobmanager-pbs-beapps
 
 ==========================================================================
</pre>
To submit a job, just use this:
<pre>
 glite-wms-job-submit -a <your jdl file>
</pre>
This should display the following lines:
<pre>
 $ glite-wms-job-submit -a GetInfo.jdl
 
 Connecting to the service https://wms.begrid.be:7443/glite_wms_wmproxy_server
 
 
 ====================== glite-wms-job-submit Success ======================
 
 The job has been successfully submitted to the WMProxy
 Your job identifier is:
 
 https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 
 ==========================================================================
</pre>
This gives you the jobID:
<pre>
 https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
</pre>
Since recently, this url also works as a website: opening this url in your browser (equiped with your certificate) shows you the status of that job. Read on to know what you can do with it...

=== Monitoring ===

You can follow the different steps of your job execution with:
<pre>
 glite-wms-job-status <jobID>
</pre>
Example of output:
<pre>
 $ glite-wms-job-status https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 
 
*************************************************************
 BOOKKEEPING INFORMATION:
 
 Status info for the Job : https://wms.begrid.be:9000/hwJR_qq9PWtKfLONZPgHnA
 Current Status:     Scheduled
 Status Reason:      Job successfully submitted to Globus
 Destination:        kg-ce01.cc.kuleuven.be:2119/jobmanager-pbs-beapps
 Submitted:          Mon Jun 23 22:59:01 2008 CEST
*************************************************************
</pre>
You can collect several jobID's in a file called <tt>jobslist</tt>, using the following command repeatedly:
<pre>
 glite-wms-job-submit -a --nomsg -o jobslist <your jdl file>
</pre>
Then use this file like this with a lot of glite commands:
<pre>
 $ glite-wms-job-status -i jobslist
 
 ------------------------------------------------------------------
 1 : https://wms.begrid.be:9000/h4F7qonPxc2NVzdqDVl6ZQ
 2 : https://wms.begrid.be:9000/WVAFM1FEjar96V1HzpOebA
 3 : https://wms.begrid.be:9000/5kF_xAQWge0Y8VJvdsttkA
 a : all
 q : quit
 ------------------------------------------------------------------
 
 Choose one or more jobId(s) in the list - [1-3]all:
</pre>
Now you can choose which jobs you want to get the status of (or all).

=== Fetching Outputs ===

When your job enters the "Done (Success)" status, you are able to retrieve the output files (the ones you specified in the ouput sandbox of the jdl file). Use this command:
<pre>
 glite-wms-job-output --dir <path to where you want your outputs> <jobID>
</pre>
or if you have collected your jobIDs in a file:
<pre>
 glite-wms-job-output --dir <path to where you want your outputs> -i jobslist
</pre>
In this directory, all the files you specified in the output sandbox will be available:
<pre>
 $ ls -lrta
 total 9
 drwxr-xr-x  19 jo users  992 Jun 23 23:21 ../
 -rw-r--r--   1 jo users 4824 Jun 23 23:21 output
 -rw-r--r--   1 jo users    0 Jun 23 23:21 error
 drwxr-xr-x   2 jo users   96 Jun 23 23:21 ./
</pre>



{{TracNotice|{{PAGENAME}}}}