Job description format

UCC uses a simple format that allows you to specify the application or executable you want to run, arguments and environment settings, any files to stage in from remote servers or the local machine and any result files to stage out.

A number of sample files can be found in the "samples" directory of your UCC distribution.

The format used is called JSON, and contains comma-separated key-value mappings, where the values can be simple strings, or lists of values, or maps. String values should be placed in "quotes". Comments are possible using the "#" hash character, as in shell scrips.

Each JSON file must begin and end with curly braces "{ ... }". Several complete job samples can be found in the UCC_HOME/samples directory of the distribution.

Note: quotes "" are needed around the keys and values in case special characters (like ':' or '/'") appear, if in doubt use quotes!

To view an example job showing all available options, simply run

ucc run -H

(most of the options shown are not mandatory, of course)

NOTE You may alternatively specify jobs in the JSDL format that is used natively in UNICORE 6. To do this, run UCC with the "-j" option.

Specifying the application or executable

You can specify a UNICORE application by name and version, or using a (machine dependent) path to an executable file.

#using application name and version

{

   ApplicationName: "Date",
   ApplicationVersion: "1.0",
   
}

note the comma-separation and the curly braces. To call an executable,

#using an executable

{

   Executable: "/bin/date",
   
}

Arguments and Environment settings

Arguments and environment settings are specified using a list of String values. Here is an example.

{

   Executable: "/bin/ls",
   
   Arguments: ["-l", "-t"],
   
   Environment: ["PATH=/bin", "FOO=bar"],
   
}

Importing files into the job workspace

To import files from your local computer or from remote sites to the job's working directory on the remote UNICORE server, there's the "Imports" keyword. Here is an example Import section that specifies two imports:

{

   Imports: [ 
   
   #import a local file
    { From: "/work/data/fileName", To: "uspaceFileName" },
    
    #import a remote file from a UNICORE storage
    { From: "u6://DEMO-SITE/Home/testfile", To: "otherUspaceFile" },
   ]
   
}

Exporting result files from the job workspace

To export files from the job's working directory to your local machine or to some remote storage, use the "Exports" keyword. Here is an example Exports section that specifies two exports:

{

   Exports: [ 
    #this exports to a local file
    { From: "image.png", To: "/home/me/image.png" },
    
    #this exports to a UNICORE storage
    { From: "stdout", To: "u6://DEMO-SITE/Home/results/myjob/stdout" },
   
   ]
   
}

The remote location can be given as a full UNICORE 6 URI, or using the more user friendly (but slower) "u6://" notation. Read more on remote locations.

Local files can be given as an absolute or relative path; in the latter case the configured output directory will be used as base directory.

The protocol to be used for imports and exports can be chosen using the "Preferred Protocols" entry, containing a space-separated list of protocols:

{

   Preferred protocols: "BFT RBYTEIO",

}

If not specified, BFT will be used.

Redirecting standard input

If you want to have your application or executable read its standard input from a file, you can use the following

  Stdin: filename,

then the standard input will come from the file named "filename" in the job working directory.

Site name

You can optionally specify on which site (if available) the job should be run.

  Site: "DEMO-SITE",
  

Resources

A job definition can have a Resources section specifying the resources to request on the remote system.

  Resources: {
  
    #memory per CPU (bytes, you may use the common "k","M" or "G")
    Memory: 268435456 ,
    
    #time per CPU (seconds, use "h" or "m" for hours or minutes)
    Runtime: 86400 ,
    
    #Total number of requested CPUs
    CPUs: 64 ,
    
    #you may optionally give the number of nodes
    #Nodes: 2 ,
    #together with the CPUs per node
    #CPUsPerNode: 32,

    #Custom resources (site-dependent!)
    StackLimitPerThread : 262144,
  
    #Operating system
    Operating system: LINUX,   #MACOS, WINNT, ...
        
    #Resource reservation reference
    Reservation: job1234,
        
  }

Note that you can also specify a reservation reference if your batch system supports this and you have made a resource reservation.

Selecting the remote login

In case you have multiple logins on the remote site mapped to the same credential, you can select the user name to use as follows

  User name: yourlogin

Hint: you can get a list of your logins on the site by executing

  ucc list-sites -s SITENAME -a | grep -i xlogin

Specifying the user email for batch system notifications

Some batch systems support sending email upon completion of jobs. To specify your email, use

  User email: foo@bar.org

Hint: if you want to explicitely switch off the email notification, use "NONE" as email value. This might be necessary because UNICORE uses the email address from your certificate (if present).

Specifying the job name

The job name can be set simply by

  Name: Test job

Specifying the status check interval for batch mode

Once a job is started, it is often not useful to check its status every few seconds, because the job might be running several minutes or more. Especially in batch mode it can reduce the load on the servers if the update interval is chosen longer. This can be achieved by using the following setting (this only affects batch mode!):

  Update interval: 60,   #only check once a minute (default is one second)