Resolve INet errors

When agent jobs run, it makes requests to the target website’s host servers to collect data, download images, etc. An INet error occurs when one of these requests fails.

Diagnosis Agent Jobs
In the Web Console:

1. Select an agent or collection.
2. Select Data > Jobs.

Common INET errors
The two most common causes of INet errors are:

• Blocked agents
• Premium Harvesting proxy

• Blocked agent
  Some website’s host servers attempt to block data gathering. When a host server detects an agent browsing the website it hosts, it might produce a warning page or respond with an HTTP error. When a host server stops responding to an agent, the harvesting server produces an INet error.

• Premium Harvesting proxy nodes
  Agents running via Premium Harvesting use a range of IP addresses. This sends requests through the target website's host servers multiple times. In some cases this creates more opportunities for a request to fail or timeout before reaching the target website’s host server.

Solutions
Optimize the agent to avoid detection by the target website’s host server to prevent request failures. The three most common solutions are to:

• Resume the agent job
• Change the Premium Harvesting location
• Optimize the agent

Resume the agent job
While it's difficult to identify the exact cause of an INet error event, you can frequently be resolved by resuming the agent job.

The agent job can be resumed by selecting the agent in the Web Console and selecting Resume.

Change the Premium Harvesting location
If the agent is running on premium harvesting and continues to experience INet errors, the website’s host server might have detected the agent despite the proxy routing. When this happens, you have the option of changing the geolocation of the requests you are sending to the host server which might temporarily unblock the agent.

Optimize the agent
The agent’s behavior might clash with the way the website behaves or is causing the agent to be detected by the website. An agent becomes more stable and less noticeable when the number of requests made to the target website is decreased.

There are a few methods for decreasing the number of requests an agent job makes to a website:

• Use Load Page actions in place of click items when navigating from one page to another.
  * Instead of navigating from the home page to the target data, start the agent directly on the page where the data is located.
  * Look for ways to reduce the number of web pages that are loaded into the agent.

Possible INet Errors
The following is a select list of the INet errors an agent job might encounter. A complete list of all INet error codes is available on Microsoft’s Error Codes page.

• INetAuthenticationRequired
  Authentication is needed to access the object.
• INetCannotConnect
  The attempt to connect to the Internet has failed.
• INetConnectionTimeout
  The Internet connection has timed out.
• INetDownloadFailure
  The download has failed (the connection was interrupted).
• INetSecurityProblem
  A security problem was encountered.
• INetResourceNotFound
  The server or proxy was not found.
• INetRedirectFailed
  WinInet cannot redirect.