Extend workflow manager to accommodate data acquisition and file management

[lewismc]

Post 0.1 release, once we have stable stand alone services for data acquisition, file management and workflow management (running pycoal tasks), our workflow management should be extended to automate data acquisition and file management. This is a huge move towards the creation of an end-to-end science data system processing capability.

[lewismc]

This issue requires us to investigate the following

  Action:
    Id: TriggerPostIngestWorkflow
    Description: Triggers workflow event with the name [ProductType]Ingest
    Phases: [postIngestSuccess]

Essentially what we want to do here is add the action to crawlctl as follows

--actionIds DeleteDataFile TriggerPostIngestWorkflow

We also need to add the workflow manager URL e.g.

--workflowMgrUrl http://localhost:9001

Once we configure the workflow manager, this will send an event and will enable us to kick off a workflow :)

[lewismc]

This is actually working the other way around where the crawler_launcher kicks off the workflow. All of this will be covered in #9