Skip to main content
By default, FetchFox connects to sites from a datacenter. These datacenters have well known IP addresses, and some sites will block access to requests coming from a known dataenter IP address. To access these sites, you can use a proxy server. FetchFox makes it easy to use proxies. We integrate with several pre-selected proxy providers, and continuously monitor their reliability. To use a proxy, all you need to do is specify which one you would like to use. You can specify which proxy to use using the proxy option on any request. Below is an example of using the residential_cdp proxy for a crawl. Different domains will need different proxies. You’ll typically want to use the cheapest proxy that can reliably access a site. Below is a table showing the proxies available, and their costs.
ProxyCost per GBLoads assets?
none$0.01 / GBNo
datacenter$0.01 / GBNo
mobile$0.50 / GBNo
residential_cdp$8.00 / GBNo
residential_cdp_assets$8.50 / GBYes
We continuously evaluate new proxies for cost and reliability. If you are unable to access a site, or need to access a site at lower cost, please email help@fetchfoxai.com.

​
Try multiple proxies

You can ask FetchFox to try multiple proxies at once. Simply pass in an array of strings as the proxy parameter. FetchFox will launch concurrent visits for each proxy you pass in, and use the first valid response it gets. Below is an example of running an extraction with multiple proxies. 
curl -X POST https://api.fetchfox.ai/api/extract \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $FETCHFOX_API_KEY" \
-d '{
    "urls": ["https://pokemondb.net/pokedex/pikachu"],
    "template": "pokemon name, number, and basic stats",
    "proxy": ["datacenter", "residential_cdp"]
}'
The response will indicate which proxy was used.

​
Automatically pick a proxy

It can be time consuming to decide which proxy to use for each scrape. If you’d like FetchFox to automatically pick a proxy for you, just pass in auto as the proxy parameter. Below is an example of how to run a crawl that automatically picks a proxy. The same parameter can be used for all other endpoints like scrape and extract. 
curl -X POST https://api.fetchfox.ai/api/crawl \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $FETCHFOX_API_KEY" \
-d '{
    "pattern":"https://pokemondb.net/pokedex/*",
    "proxy": "auto"
}'
Read our guide on automatic proxy selection for more information on how this works.
⌘I