Running jobs

Introduction

The UCC can run jobs specified in a simple job description format. In the following it is assumed that you have installed UCC and tried the quick start examples.

For example, assume the file "myjob.u" looks as follows

#sample job file
{
 ApplicationName="Date", ApplicationVersion="1.0"
}

To run this through UNICORE, use

ucc run [options] myjob.u

This will submit the job, wait for completion, download the stdout and stderr files, and place them in your default output directory.

Controlling the output location and file names

Output files will be placed in the directory given by the "-o" option, if not given, the current directory is used. Also, file names will be prefixed with the job id, to assure unique file names. This behaviour can be changed using the "-b" option. When "-b" is given on the command line, files will keep their original names as specified in the job description file.

Specifying the site

In the example above, a random site will be chosen to execute the job. To control it, you can use the "-s" option. This will accept the name of a target system. The target systems available to you can be listed by

ucc list-sites [options]

Options overview

The following options are available when running jobs (see also the general options overview}

option (short and long form) description
-s,--sitename SITE Site where the job shall be run
-o,--output Output_dir Directory for any output produced (default is the current directory)
-b,--brief Do not prefix the output files with the job id
-O,--stdout specify a name for the exported standard out (by default: 'stdout')
-E,--stderr specify a name for the exported standard error (by default: 'stderr')
-a,--asynchronous Run asynchronously
-y,--with-timing Timing mode

Processing jobs asynchronously

In case of long-running jobs, you will want to run the job asynchronously, i.e. just submit the job, stage in any files and start it, in order to get the results later. UCC supports this, of course. The basic idea is that when submitting a job in asynchronous mode, a job descriptor file is written that contains the job's address, and any information about export files.

Asynchronous submission

Use the "-a" flag when submitting a job

ucc run [options] -a <job file>

this will submit the job, stage-in any local files, start the job and exit. A job descriptor file (ending in ".job") will be written to your configured output directory.

Get the status of a particular job

ucc get-status [options] <job_desc>

will retrieve the job status. If not given on the command line, the job id will be read from the console.

Download results

To get stdout, stderr and other files you have marked for export in your job description, do

ucc get-output [options] -o <outdir> <job_desc>

Here, the option "-o" specifies the directory where to put the output, by default the current directory is used. As before, the job address can also be read from the console.

Referencing a job by its EPR (Endpoint reference)

In case you want to check on a job not submitted through UCC, or in case you do not have the job descriptor file any more, you can also refer to a job given its EPR. For example, the "list-jobs" command will produce a list of all job EPRs that you can access.

Note that in this case UCC will only retrieve stdout and stderr files. To download other result files, you'll have to use the data movement functions.

Uploading and executing an executable

To upload and execute a file on a remote server, you'll need a small helper script to make the uploaded file executable and run it:

#!/bin/sh
chmod +x myapp
./myapp

Your ucc job description would then look as follows

{
  Executable: "/bin/sh",
  Arguments: ["helper.sh"],
  Imports: [
    {From: "helper.sh", To: "helper.sh"},
    {From: "myapp", To: "myapp"},
  ],
}