Create a sequence

A sequence is a workflow for automating and scheduling common web scraping tasks, including clearing collections and views, running agents, and publishing data. Sequences become especially powerful when used to automatically run chains of agents that feed into each other. Additional benefits include continuing to run a project if an error ocrrurs and optimizing workflows.

You might decide to use a sequence to continue to run a project even if an error occurs or to consolidate notifications, scheduling, and publishing.

Create a sequence

1. Open the Web Console.

2. Select Sequences in the top navigation bar.

3. Select New Sequence.

4. Name your sequence.

5. Select Add Step.

6. Select the step you want to add. Repeat to add more steps to the sequence. These are the possible steps you can choose from:

   • Run Agent - The agent will begin to scrape data according to its configuration.
     • If applicable, you can divide an agent's Data List inputs among multiple concurrent jobs and add additional error handling.
   • Clear Collection - Clear the contents of a collection.
   • Delete View Items - Specify certain data from a scrape and delete it.
   • Publish Collection Data - Send your data to a specific location and format
   • Update Field Value - Update the values of a field.
     
     
     

7. Select multiple steps to run concurrently by selecting the steps, clicking More > Run the selected x steps concurrently.

8. After your sequence is configured, select Ready > Run to start it.

Creating a multi-step workflow (sequence)

The Expert's Corner