Load a new page

An agent can scrape more than one web page using the Load New Page action. This action allows an agent to navigate through URLs without clicking a link by simulating the process of manually entering one or more URLs.

Enter a specific URL

The most common way to start building an agent is to simply load the website you want to scrape.

In the Agent Builder:

1. Select Start a new Agent from this page.

2. Select Add Action > Load New Page.
3. Enter a URL into the URL field.

4. Select ADD PAGE.

This action creates a new page in the Action List and loads the new URL.

Input a list of URLs

Use Load New Page to create an agent that scrapes a list of pages with a similar layout. You can input a list of URLs from an existing collection or upload a CSV file from your computer then reference the field from the Input List action.

In a spreadsheet:

1. Create a column with an appropriate name. The column name will be used as your field reference when using Load New Page action.
2. Enter the list of URLs into the same column.
3. Save the document as a CSV UTF-8 format. This is the only format Mozenda uses as input. See Data file formats.

In the Agent Builder:

4. Select Add Action > Input List.
5. Select:
   a) Import File if it is the first time importing using the data list > Select the list of URLs you want to scrape.
   b) Collection if you have previously used the data list > Select the list of URLs you want to scrape.

Begin Data List displays on the Actions List.

6. Select Add Action.
7. Select Load New Page.
8. Enter the name of your URLs collection column in the URL textbox inside two %%. (For example, %StartingURL%).

You can find the name by double-clicking the Begin Data List action and identifying the field that stores the URLs.

9. Select ADD PAGE.

This action creates a new page in the Action List and loads the new URL.

When you run the agent, the list of URLs in your collection will load consecutively and the agent will scrape the information you have specified.