The unique fields feature allows you to automatically remove duplicate records from your data set.
- In the Web Console, open the agent or collection.
- Select Data > Fields from the toolbar.
- Select the fields.
Choose fields that contain unique data, such as an SKU, address, or ISBN. The data gathered between agent jobs will be compared based on the field(s) you've selected. Avoid fields that are subject to change, such as price or quantity.
- Click Set unique fields.
- Click Continue to delete duplicate records matching the unique field(s).
- If you remove the unique fields, duplicates will reappear the next time you run the agent.
- You can't add unique fields that are at the agent group-level on the agent level. You need to do that in the agent group.