Increasing scraping speed
Last updated
Was this helpful?
FAQ
Last updated
Was this helpful?
When you send a request to the API we will route it through our proxy pools, check the response for CAPTCHAs, bans, etc. and then either return the valid HTML response to you or keep retrying the request with different proxies for up to 60 seconds before returning you a 500 status error.
Frequently, the API has higher latencies than sending requests directly to a normal proxy as average latencies typically range from 4-12 seconds (depending on website) and single requests can sometimes take up to 60 seconds. However, this is compensated by the fact that our average success rate is around 98%.
If you would like to increase the volume of successful requests you can make in a given time period then we can increase your number of concurrent threads. to enquiry about increasing your concurrency limit.
If you like to reduce the latency of each request or reduce the longtail of some requests taking 20-30 seconds then you can use our premium proxy pools by adding premium=true to your request, or by to see if they can increase the speed of your requests.