When agent jobs run, it makes requests to the target website’s host servers to collect data, download images, etc. An INet error occurs when one of these requests fails.
Diagnosis Agent Jobs
- Open the Web Console.
- Open an agent or collection.
- Click Data > Jobs.
Common INET errors
The two most common causes of INet errors are:
- Blocked agents
- Premium harvesting proxy
Some website’s host servers attempt to block data gathering. When a host server detects an agent browsing the website it hosts, it might produce a warning page or respond with an HTTP error. When a host server stops responding to an agent, the harvesting server produces an INet error.
Premium Harvesting proxy nodes
Agents running via premium harvesting use a range of IP addresses. This sends requests through the target website's host servers multiple times. In some cases this creates more opportunities for a request to fail or timeout before reaching the target website’s host server.
Optimize the agent to avoid detection by the target website’s host server to prevent request failures. The three most common solutions are to:
- Resume the agent job
- Change the Premium Harvesting location
- Optimize the agent
Resume the agent job
While it's difficult to identify the exact cause of an INet error event, you can frequently be resolved by resuming the agent job.
The agent job can be resumed by opening the agent in the Web Console and clicking Resume.
Change the Premium Harvesting location
If the agent is running on premium harvesting and continues to experience INet errors, the website’s host server may have detected the agent despite the proxy routing. When this happens, you have the option of changing the geolocation of the requests you are sending to the host server which may temporarily unblock the agent.
Optimize the agent
The agent’s behavior may clash with the way the website behaves or is causing the agent to be detected by the website. An agent becomes more stable and less noticeable when the number of requests made to the target website is decreased.
There are a few methods for decreasing the number of requests an agent job makes to a website:
- Use Load Page actions in place of click items when navigating from one page to another.
- Instead of navigating from the home page to the target data, start the Agent directly on the page where the data is located.
- Look for ways to reduce the number of web pages that are loaded into the agent.
Possible INet Errors
The following is a select list of the INet errors an Agent job might encounter. A complete list of all INet error codes is available on Microsoft’s Error Codes page.
Authentication is needed to access the object.
The attempt to connect to the Internet has failed.
The Internet connection has timed out.
The download has failed (the connection was interrupted).
A security problem was encountered.
The server or proxy was not found.
WinInet cannot redirect.