Use case: Run multiple, overlapping, independent jobs

There are circumstances where you will want to have an Agent on which you run multiple, independent, potentially overlapping jobs or batches of jobs via API with different inputs.

This steps outside the default Mozenda workflow, which is optimized around Agents and Sequences that run synchronously, one batch at a time. It can be accomplished, however.

Setup

• Create a collection for the Agent’s inputs. Add an additional column (e.g. InputBatch) which will be used to identify the set of inputs you wish to run in a particular batch.

• Create a View with a parameterized filter criterion that filters on that additional column (e.g. InputBatch Equals %DataListInputBatch%)

• Create an Agent with a DataList which pulls from the View you created

• Add a View Parameter to the DataList settings with the same Name (e.g. DataListInputBatch) as your View criterion parameter name and a parameter value (e.g. %APIInputBatch%). Make sure to click (+) Add and then SAVE.
  

• Configure the Agent to Store item history. Do not set unique fields on the Agent Collection.

Using the API to create independent Job/Jobs that pull from specific inputs

Here you will do four things:

• Use the API to add inputs to the input Collection
• Use the API to run the Agent with these particular inputs
• Use the API to monitor the Jobs as they run
• Use the API to export the data collected

Use the API to add inputs to the input Collection

To do this, use the Collection.AddItem API endpoint. Make sure to provide a value for InputBatch (or whatever you decided to call this field).

Use the API to run the Agent with these particular inputs

To do this, use the Agent.Run API endpoint. You will use the AgentParameter.<AgentParameterName> URL query parameter to supply the value for the parameter you set in the DataList settings (e.g. APIInputBatch).

For example, if you added the following row to your input collection and wanted to run the agent with this input row:

[
    {
        "EndDate": "2025-01-01", 
        "StartDate": "2025-02-01",
        "ZipCode": "90210",
        "InputBatch": "BatchNum8675309"
   }
]

Your Agent.Run API request might look something like:

curl -X GET "https://api.mozenda.com/rest?WebServiceKey=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX&Service=Mozenda10&Operation=Agent.Run&AgentID=1001&AgentParameter.APIInputBatch=BatchNum8675309"

If you added more than one row and want to split those rows among multiple jobs, you would add the &SplitJobs URL parameter.

In both cases, a successful request will return either a single job or array of the WebPageHarvest Jobs created.

{
    "AgentID": 1001,
    "Job": [
        {
            "JobID": 1001,
            "JobType": "WebPageHarvest"
        },
        {
            "JobID": 1002,
            "JobType": "WebPageHarvest"
        },
        {
            "JobID": 1003,
            "JobType": "WebPageHarvest"
        },
        {
            "JobID": 1004,
            "JobType": "ReportRefreshData"
        }
    ],
    "JsonResult": {
        "Command": "Agent.Run",
        "AutoLoggedIn": false,
        "WebConsoleKey": "",
        "Result": "Success",
        "ErrorCode": "",
        "ErrorDescription": "",
        "ErrorList": []
    }
}

Use the API to monitor the Jobs as they run

To monitor the jobs as they run, use the Job.Get API endpoint. Alternatively, you can use the &Job.StatusURL API endpoint to notify your endpoint when a job's status changes.

Use the API to export the data collected by a given Job

When a Job is finished, use View.Export (https://help.mozenda.com/docs/viewexport) with:

• The Job’s ID to pull the data that that job collected
• BookmarkID set to -1 (this is equivalent to setting the Agent’s default bookmark to “All items ever found”)

curl -X GET "https://api.mozenda.com/rest?WebServiceKey=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX&Service=Mozenda10&Operation=View.Export&ViewID=XXXX&ViewParameter.JobID=1001&BookmarkID=-1"

Cleanup

The cleanup schedule that is best depends on your needs. Both the input collections and the Agents’ output collections need to have their data deleted from time to time. Here are your options:

• Collection.Clear
  • Executes a TRUNCATE command on the underlying table. Fast and destructive.
  • Good to use in cases where the Agent has no jobs running.
• View.DeleteItems
  • Executes a DELETE FROM command on the underlying table. Somewhat slower and more limited.
  • Can be run while Jobs are running
    • Pass in the DataListBatchInputBatch (or whatever parameter name you set in your View (&ViewParameter.DataListInputBatch parameter)