Python Requests

Requests takes a proxy per request or per session through a proxies dictionary. This page shows a working setup for NyxonProxies and the details that catch people out: sessions that keep one IP, environment variables that override your code, SOCKS5 DNS and passwords with special characters.

Install#

pip install requests

Basic request#

import requests

proxy = "http://USERNAME:[email protected]:8080"
proxies = {"http": proxy, "https": proxy}

r = requests.get("https://api.ipify.org?format=json", proxies=proxies, timeout=30)
print(r.json())

Both keys point to the same http:// URL. The key says which sites use the proxy, and the scheme of the URL says how to talk to the gateway. Leave out the https key and HTTPS requests go straight to the site, from your own IP and without any error.

Always pass a timeout. Without one, a request can wait forever.

Use a session#

A Session applies the proxy to every request and reuses connections:

import requests

proxy = "http://USERNAME:[email protected]:8080"

s = requests.Session()
s.proxies.update({"http": proxy, "https": proxy})

for _ in range(3):
    print(s.get("https://api.ipify.org", timeout=30).text)

The session keeps its connection to the gateway open, so all three requests leave from the same IP, even without a sticky session. That suits a login or a checkout. For rotation, see the next section.

A new IP on every request#

Without a session in the username, every new connection gets a new IP. requests.get opens a fresh connection each time:

import requests

proxy = "http://USERNAME:[email protected]:8080"

for _ in range(3):
    r = requests.get("https://api.ipify.org", proxies={"http": proxy, "https": proxy}, timeout=30)
    print(r.text)

Each line usually prints a different address. More in Connections and tunnels.

Country and sticky session#

Targeting and sessions go in the username. This helper builds a proxy that keeps one IP in a chosen country:

import secrets
import requests


def sticky(country, minutes=10):
    sid = secrets.token_hex(4)
    url = f"http://USERNAME-country-{country}-session-{sid}-time-{minutes}:[email protected]:8080"
    return {"http": url, "https": url}


proxies = sticky("de")
for _ in range(2):
    print(requests.get("https://api.ipify.org", proxies=proxies, timeout=30).text)

Both requests print the same German IP, for up to 10 minutes. Call sticky() again for a new session and a new IP. All parameters are listed in Username parameters.

SOCKS5#

SOCKS support needs an extra package:

pip install "requests[socks]"

import requests

proxy = "socks5h://USERNAME:[email protected]:1080"
r = requests.get("https://api.ipify.org", proxies={"http": proxy, "https": proxy}, timeout=30)
print(r.text)

Use socks5h://, so the gateway resolves the hostname. With socks5://, Requests looks up the name on your machine and sends only the IP.

Passwords with special characters#

Characters such as @, :, / and # break the proxy URL. Percent-encode the password first:

from urllib.parse import quote

password = quote("p@ss:w/rd#1", safe="")
proxy = f"http://USERNAME:{password}@geo.gw.nyxon.sh:8080"

Or change the proxy password to letters and digits only.

Environment variables#

Requests reads HTTPS_PROXY, HTTP_PROXY and NO_PROXY by default, so a script with no proxy code can still use the gateway:

export HTTPS_PROXY="http://USERNAME:[email protected]:8080"
python scraper.py

On Windows and macOS, when no variable is set, Requests can also pick up the system proxy settings. trust_env = False on a session turns both off. See Environment variables.

Errors#

More in Error codes and Troubleshooting.