Теги: http  crawlers  pip 

http-библиотечка на python для запросов/ответов.


>>> r = requests.get('', auth=('user', 'pass'))
>>> r.status_code
>>> r.headers['content-type']
'application/json; charset=utf8'
>>> r.encoding
>>> r.text
>>> r.json()
{'private_gists': 419, 'total_private_repos': 77, ...}

Смотри еще:

  • [http-requests-errors]
  • grequests GRequests allows you to use Requests with Gevent to make asynchronous HTTP Requests easily.
  • requests-futures Asynchronous Python HTTP Requests for Human
  • requests-threads This repo contains a Requests session that returns the amazing Twisted’s awaitable Deferreds instead of Response objects