Workflow extensions

Introduction

Starting with UCC 6.2.1, the UCC commands to access the workflow system are integrated into the core UCC distribution.

As the workflow system itself, these UCC commands were developed in the Chemomentum project (http://www.chemomentum.org)

Command overview

The following commands are provided. More details and examples follow.

  • system-info : list services that are necessary for executing workflows
  • workflow-submit : submit a workflow file
  • workflow-info : list information about workflows
  • workflow-trace : gather performance data from the workflow execution

Basic use

To check the availability of workflow services, issue the following command

ucc system-info

The distribution contains some example workflow files in the samples/workflows directory that you can edit and submit.

ucc workflow-submit yourworkflow.swf

which will submit the workflow and print the address of the workflow to standard output. To get the workflow status,

ucc workflow-info <workflow_address>

To list all your workflows, you can use the workflow-info command without an explicit workflow address

ucc workflow-info -l

Staging in data files needed for the workflow

If you have local files that need to be imported before starting the workflow, you have to specify this using a normal UCC job file that contains only an "Imports" section:

{
#stage-in specification

  Imports: [
          {From: local-file.sh, To: "c9m:${WORKFLOW_ID}/input.sh"}
  ],
}

This will cause UCC to copy the local file "local-file.sh" to the workflow storage space. You can refer to this file in your workflow using the "global" name "c9m:...", say in a script activity:

....
        <jsdl:DataStaging>
        <jsdl:CreationFlag>overwrite</jsdl:CreationFlag>
        <jsdl:FileName>input.sh</jsdl:FileName>
          <jsdl:Source>
            <jsdl:URI>c9m:${WORKFLOW_ID}/input.sh</jsdl:URI>
          </jsdl:Source>
        </jsdl:DataStaging>

....

The workflow system will resolve the name at runtime and your file will be used. The "$WORKFLOW_ID" will be replaced by UCC by the real workflow ID, when submitting the job. This allows you to group your files by workflow ID.

Getting output

You can use the usual get-file command to download files using the "global IDs" used by the workflow engine. Hint: the workflow-info command will list the files that are produced by the workflow.

More

Tracing

The trace functionality of the workflow engine allows to retrieve some performance data, try

ucc workflow-trace <your_workflow_address>