When working with cUrl and proxies in PHP and a lot of request, first of all it can of course help to run multiple process at the same time, but besides that how you use the proxy itself can be a huge performance boost if done like below:
$curlResource = curl_init($this->sourceUrl);
+ if (isset($this->getProxy())) {
+ $headerData[] = 'X-Forwarded-For: ' . $this->getProxy();
+ }
+ curl_setopt($curlResource, CURLOPT_SSL_VERIFYHOST, false);
+ curl_setopt($curlResource, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curlResource, CURLOPT_URL, $this->sourceUrl);
curl_setopt($curlResource, CURLOPT_HEADER, 1);
curl_setopt($curlResource, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curlResource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curlResource, CURLOPT_HTTPHEADER, $headerData);
- // Set proxy if applicable
- if (NULL !== $proxyIp && FALSE !== $proxyIp) {
- curl_setopt($curlResource, CURLOPT_PROXY, $proxyIp);
- curl_setopt($curlResource, CURLOPT_HTTPPROXYTUNNEL, 1);
- }
```
`
This is a small diff from an example. The most important part is that instead of using the `CURLOPT_PROXY` and `CURLOPT_HTTPPROXYTUNNEL` then use the `X-Forward-For` header do the trick for you.
The win here is due to they will be no network latency what so ever, as the proxy is only simulated and not actually used.
I'm using this test for GEO IP redirects etc. and when testing e.g. 300 redirects, then it does matter that you gain a four times faster job than you had before. With this said this of course depends on the proxy speed, but i'm close to guarantee that this will speed up you job, if you do this change.
Top comments (0)