Proxies with Python Requests module

Proxies with Python Requests module

The proxies dict syntax is {protocol:ip:port, ...}. With it you can specify different (or the same) proxie(s) for requests using http, https, and ftp protocols:

http_proxy  =
https_proxy =
ftp_proxy   =

proxyDict = { 
              http  : http_proxy, 
              https : https_proxy, 
              ftp   : ftp_proxy

r = requests.get(url, headers=headers, proxies=proxyDict)

Deduced from the requests documentation:

method – method for the new Request object.
url – URL for the new Request object.

proxies – (optional) Dictionary mapping protocol to the URL of the proxy.

On linux you can also do this via the HTTP_PROXY, HTTPS_PROXY, and FTP_PROXY environment variables:

export HTTP_PROXY=
export FTP_PROXY=

On Windows:

set http_proxy=
set https_proxy=
set ftp_proxy=

Thanks, Jay for pointing this out:
The syntax changed with requests 2.0.0.
Youll need to add a schema to the url:

You can refer to the proxy documentation here.

If you need to use a proxy, you can configure individual requests with the proxies argument to any request method:

import requests

proxies = {

requests.get(, proxies=proxies)

To use HTTP Basic Auth with your proxy, use the http://user:[email protected]/ syntax:

proxies = {
    http: http://user:[email protected]:3128/

Proxies with Python Requests module

I have found that urllib has some really good code to pick up the systems proxy settings and they happen to be in the correct form to use directly. You can use this like:

import urllib

r = requests.get(, proxies=urllib.request.getproxies())

It works really well and urllib knows about getting Mac OS X and Windows settings as well.

Leave a Reply

Your email address will not be published. Required fields are marked *