Scrape pages that use AJAX

Many modern websites use a set of techniques and technologies referred to as AJAX (Asynchronous JavaScript and XML) to actively update parts of a web page without reloading the entire page. This requires less network bandwidth and provides the user a more dynamic, responsive experience.

Mozenda agents can react to these changes so you can consume the data from these sites just as you would from a static or CMS-generated site.

1. In the Agent Builder, open the agent.
2. Right-click the action that changes the page, and then select Properties.

This might be a Click Item, Set User Input, or Page List action.

3. Click After Performing Action.
4. Check the box next to Wait for AJAX to alter the page.
5. Click SAVE.

6. Test the agent to ensure that it waits for the new content to load before it proceeds to the next action.