Ask FetchFox to figure out which proxy to use
proxy
parameter, but it can become difficult and time consuming to know which proxy to use for which site. Instead, you can ask FetchFox to automatically suggest a proxy to use for any given site.
auto
as a value for the proxy
parameter. This tells FetchFox to automatically pick the most appropriate proxy on a per domain basis.
Below is an example of how to call the crawl endpoint using the automatic proxy picker.
auto
as the proxy value, it will check its historical data for the domain it is visiting. If there is historical data, then it will use that to pick the appropriate proxy. If there isn’t, then FetchFox will run some trials to generate initial data on that domain. Thus, the first time you make a call with auto
for particular domain, it may be a little slower than usual.
url
. This is the URL you are trying to access.
Below is an example of how to call this endpoint.
results.best
. This proxy is the cheapeset one that we found to reliably access the target domain. We also return some historical statistics on historical visits to that domain, which you will see in results.stats
. This shows how many requests from each proxy succeeded or failed.
Note that the suggest endpoint will incur cost the first time it runs for any domain. This is because FetchFox needs to visit a sampling of URLs on the target domain, and evaluate the results using AI.
force_run
and count
parameters. These will tell FetchFox to make count
new visits to the target domain using each proxy, regardless of historical data.