User General user info job glite

From Begrid Wiki
Revision as of 09:14, 9 June 2021 by Maintenance script (talk | contribs) (Created page with " == Log in on a BEgrid user interface == There are several user interfaces named the following way : <pre> m<x>.iihe.ac.be with x=[0-9] </pre> If you have carefully follo...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Log in on a BEgrid user interface

There are several user interfaces named the following way : 

m<x>.iihe.ac.be with x=[0-9]

If you have carefully followed this procedure, you should be able to log in on one of them. As an example, let's try to log in on m6 : 

ssh <your_login>@m6.iihe.ac.be


Creation of a VOMS proxy

If you are member of beapps

Prior to using the resources of the grid, you need a valid personal grid proxy. This proxy is kind of access token that will be used by the grid software to check your identity and your authorizations. It not only contains the DN of your personal certificate, but also the name of the virtual organization your are registered to.

To create such a proxy for the belgian VO "beapps", issue the following command : 

voms-proxy-init --voms beapps

Grid proxies are limited in time. By default, it is valid for 12 hours. For longer duration, use the "--valid" option. Below, we request a proxy valid for 24 hours : 

voms-proxy-init --voms beapps --valid 24:00

If you want to check a proxy you've just created, type the following command : 

voms-proxy-info --all

If you can't register to beapps

Issue the following command : 

/bin/createproxy.pl

And now, the magic of PUSP will show up with the command : 

voms-proxy-info --all

that should result in something like this : 

subject   : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:sgerard/CN=1598008192
issuer    : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be/CN=user:sgerard
identity  : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
type      : RFC3820 compliant impersonation proxy
strength  : 1024
path      : /tmp/x509up_u20533
timeleft  : 11:59:49
key usage : Digital Signature, Key Encipherment
==== VO beapps extension information ====
VO        : beapps
subject   : /DC=org/DC=terena/DC=tcs/C=BE/O=Vrije Universiteit Brussel/CN=Robot - STEPHANE GERARD stgerard@vub.ac.be
issuer    : /DC=org/DC=terena/DC=tcs/C=BE/ST=Brussels/L=Brussels/O=Le reseau telematique belge de la recherche/CN=voms01.begrid.be
attribute : /beapps/Role=NULL/Capability=NULL
timeleft  : 11:59:49
uri       : voms01.begrid.be:18004

You've got a valid beapps proxy, you are ready for the next step...

Job submission

This section describes how to manage your jobs the glite-ce-* commands. With these commands, you must know beforehand on which CE you will send your jobs.

Choosing a Computing Elemement

First, you need to get the list of the CEs that are supporting the VO you belong to. You can do it like that : 

[sgerard@m6 ~]$ lcg-infosites --vo beapps CE
#   CPU	   Free	Total Jobs	Running	Waiting	ComputingElement
----------------------------------------------------------------
   5947	      0	         0	      0	      0	cream02.iihe.ac.be:8443/cream-pbs-beapps
   5832	    226	      2231	   1986	    245	creamce.gina.sara.nl:8443/cream-pbs-medium
   5832	    226	         0	      0	      0	creamce.gina.sara.nl:8443/cream-pbs-short
   5832	    237	      2239	   1982	    257	creamce2.gina.sara.nl:8443/cream-pbs-medium
   5832	    237	         0	      0	      0	creamce2.gina.sara.nl:8443/cream-pbs-short
   5832	    237	      2239	   1982	    257	creamce3.gina.sara.nl:8443/cream-pbs-medium
   5832	    237	         0	      0	      0	creamce3.gina.sara.nl:8443/cream-pbs-short

Compose/edit a JDL file to describe your job

Jobs submitted to the grid are described using a specific language, the Job Description Language (JDL). Here we provide a very basic sample : 

[
 VirtualOrganisation = "beapps";
                                 # the VO you belong to
 JobType = "normal";
                                 # in general this is normal or mpich
 Type = "Job";
                                 # this is a job :-)
 Executable = "test_job.sh";
                                 # name of the command without arguments
 Arguments = "/proc/cpuinfo";
                                 # arguments if necessary
 StdOutput = "output";
                                 # to put the standard output (display) in the file we call output
 StdError = "error";
                                 # to put the standard error in the file we call error
 InputSandbox = "test_job.sh";
                                 # to send the binary (or the script) and all necessary input files
 OutputSandbox = {"output","error"};
                                 # to receive the output files
 OutputSandboxBaseDestURI = "gsiftp://localhost";
                                 # to specify how the outputsandbox files will be transfered back to the user
]

More options for the JDL file can be found on this site: https://wiki.italiangrid.it/twiki/bin/view/CREAM/JdlGuide.

Remark: Don't send/receive input/output sandboxes greater than 20 MB. For greater files usage, look at section Advanced data access with gridftp.

Submit your job

To submit a job, just use this: 

 glite-ce-job-submit -a -r <target_resource> <your_jdl_file>

The <target_resource> is specified like this : 

<full_hostname>:<port_number>/cream-<service>-<queue_name>

where <service> identifies the underlying resource management system (e.g. lsf, pbs, etc.). For example : 

 $ glite-ce-job-submit -a -r cream02.iihe.ac.be:8443/cream-pbs-beapps job.jdl
2016-02-09 14:44:44,459 WARN - VirtualOrganisation specified in the JDL but overriden with [beapps]
https://cream02.iihe.ac.be:8443/CREAM979231935

The output gives you the jobID: 

https://cream02.iihe.ac.be:8443/CREAM979231935

You will need this jobID in the following steps to monitor your job and get its ouput when it's finished.

Monitor your job

You can follow the different steps of your job execution with: 

 glite-ce-job-status <jobID>

Example of output: 

 $ glite-ce-job-status https://cream02.iihe.ac.be:8443/CREAM979231935
 
******  JobID=https://cream02.iihe.ac.be:8443/CREAM979231935
	Status        = [IDLE]

Retrieve the results

When your job has reached the "DONE-OK" status, you are able to retrieve the output files (the ones you specified in the ouput sandbox of the jdl file). Use this command: 

 glite-ce-job-output [--dir <path to where you want your outputs>] <jobID>

If you don't specify the path for your outputs with the --dir option, then a directory containing your output sandbox files will be automatically created in the current directory.

Example : 

 $ glite-ce-job-output https://cream02.iihe.ac.be:8443/CREAM979231935

2018-03-08 15:26:41,256 INFO - For JobID https://cream02.iihe.ac.be:8443/CREAM979231935 output will be stored in the dir ./cream02.iihe.ac.be_8443_CREAM979231935

Data Management

Other information


Template:TracNotice

Retrieved from "https://wiki.begrid.be/index.php?title=User_General_user_info_job_glite&oldid=119"