- Alternative HTTP Client, new version
0.9, API for Humans. - Added Multi-Thread web scraper Built-in one-liner.
- Added Multi-Thread file downloader Built-in one-liner.
- 1 file, 0 dependency, ~100 Lines of code,
2.7to3.8, Alpine & ARM. - GitHub Actions CI building from scratch.
- GitHub Actions CI running Unittests from scratch.
- Examples for web scraper and file downloader.
- Extras for Data Science, Web Scrapping, HTTP REST JSON APIs.
- Examples, Dockerfile, tests, FAQ, CoC, Debug helpers, JSON helpers.
- Docs has all functions with detailed arguments and returns with types.
| Library | Speed | Files | LOC | Dependency | Devs | Scraper |
|---|---|---|---|---|---|---|
| PyWGET | 152.39 |
1 | 338 | Wget | >17 | No |
| Requests | 15.58 |
>20 | 2558 | >=7 | >527 | No |
| Urllib | 4.00 |
??? | 1200 | 0(std lib) | ??? | No |
| Urllib3 | 3.55 |
>40 | 5242 | >5(SSL) | >188 | No |
| PyCurl | 0.75 |
>15 | 5932 | Curl,LibCurl | >50 | No |
| FTR | 0.45 |
1 | 99 | 0 | 1 | Yes, 2 |
Hello World
requests.get("http://httpbin.org/get")
-
GET,POST,PATCH,PUT,DELETEand more.
Multi-Thread Web Scraper Built-in
requests.scrapper(["http://example.org", "http://example.io"], threads=True)
- Theres 2 ready-made Web Scrapers built-in, easy to use one-liner.
Multi-Thread File Downloader Built-in
requests.download2([("http://example.org/foo.jpg", "output.jpg"), ], threads=True)
-
delay=1000for 1 Second delay sleep between downloads.
Multi-Thread Bulk GET
requests.get2str2(["http://example.org", "http://example.io"], threads=True)
-
threads=Falsefor No Multi-Thread.
GitHub
🐍😼👍
Top comments (5)
I keep seeing the double-p in
scrapperand think, "Like, a scrappy fighter?" Is it possible you intended to call the methodscraper?I'm passing headers to faster_than_requests but it gives 400 Bad request.
this is my code sample:
import faster_than_requests as requests
headers = [("Host", "api.sample.com"), ("Connection", "Keep-Alive"), ("Accept-Encoding", "gzip")]
response = requests.post(url=self.offer_url, body=self.offer_data, http_headers=headers, proxy_url=self.proxyURL)
print(response["status"])
any help would be greatly appreciated. I currently use the requests library but is too slow.
Cool, I like the color scheme of your text editor...animus, I gonna put attention to th is librarie...animus
Great Post Juan!
If you think this is interesting, checkout Async for HTTP requests, I bet it will blow your mind! 🦄
Nice, I see you implemented it in Nim eheh!
Have you tried httpx's async support as well?