Python aiohttp
Use NyxonProxies with aiohttp and asyncio: a proxy per request or session, concurrency limits, rotation, sticky sessions, SOCKS5 with aiohttp-socks and errors.
aiohttp is an asyncio HTTP client built for many requests at once. It takes the proxy on each request or once on the session, and speaks HTTP proxies natively. SOCKS5 needs one extra package.
Install
pip install aiohttpBasic request
import asyncio
import aiohttp
async def main():
async with aiohttp.ClientSession() as session:
async with session.get(
"https://api.ipify.org?format=json",
proxy="http://USERNAME:[email protected]:8080",
) as r:
print(await r.json())
asyncio.run(main())A proxy for the whole session
Pass proxy= to the session and every request uses it:
import asyncio
import aiohttp
async def main():
async with aiohttp.ClientSession(proxy="http://USERNAME:[email protected]:8080") as session:
async with session.get("https://api.ipify.org") as r:
print(await r.text())
asyncio.run(main())Many requests at once
Cap the number of requests in flight with a semaphore. Requests that run in parallel each open their own connection, so on a rotating username they leave from different IPs:
import asyncio
import aiohttp
PROXY = "http://USERNAME:[email protected]:8080"
limit = asyncio.Semaphore(10)
async def fetch(session, url):
async with limit:
async with session.get(url, proxy=PROXY) as r:
return await r.text()
async def main():
timeout = aiohttp.ClientTimeout(total=60)
async with aiohttp.ClientSession(timeout=timeout) as session:
ips = await asyncio.gather(*(fetch(session, "https://api.ipify.org") for _ in range(20)))
print(ips)
asyncio.run(main())A new IP on every request
The session keeps connections open and reuses them, so requests sent one after another can share an IP. A connector with force_close=True closes each connection after its request, and every request gets a new one:
import asyncio
import aiohttp
async def main():
connector = aiohttp.TCPConnector(force_close=True)
async with aiohttp.ClientSession(connector=connector) as session:
for _ in range(3):
async with session.get("https://api.ipify.org", proxy="http://USERNAME:[email protected]:8080") as r:
print(await r.text())
asyncio.run(main())Sticky sessions per task
To keep one IP per task, give each task its own session ID in the username:
import asyncio
import secrets
import aiohttp
def sticky(country, minutes=10):
sid = secrets.token_hex(4)
return f"http://USERNAME-country-{country}-session-{sid}-time-{minutes}:[email protected]:8080"
async def check(session, country):
proxy = sticky(country)
for _ in range(2):
async with session.get("https://api.ipify.org", proxy=proxy) as r:
print(country, await r.text())
async def main():
async with aiohttp.ClientSession() as session:
await asyncio.gather(*(check(session, c) for c in ["us", "de", "gb"]))
asyncio.run(main())Each country prints the same IP twice. See Rotating and sticky sessions.
SOCKS5
aiohttp's proxy= speaks HTTP only. For the SOCKS5 port, install aiohttp-socks and use its connector:
pip install aiohttp-socksimport asyncio
import aiohttp
from aiohttp_socks import ProxyConnector
async def main():
connector = ProxyConnector.from_url("socks5://USERNAME:[email protected]:1080")
async with aiohttp.ClientSession(connector=connector) as session:
async with session.get("https://api.ipify.org") as r:
print(await r.text())
asyncio.run(main())The connector sends the hostname to the gateway, so DNS is resolved on the proxy side. Write socks5://: it does not accept socks5h://.
Passwords with special characters
Percent-encode the password before you put it in the URL:
from urllib.parse import quote
password = quote("p@ss:w/rd#1", safe="")
proxy = f"http://USERNAME:{password}@geo.gw.nyxon.sh:8080"Environment variables
aiohttp ignores HTTPS_PROXY and the other variables unless you create the session with trust_env=True. See Environment variables.
Errors
| Error | Cause |
|---|---|
ClientHttpProxyError with 407 | Wrong username or password. |
ClientProxyConnectionError | Wrong host or port, or a firewall. |
ServerDisconnectedError | A socks5:// URL passed to proxy=. Use aiohttp-socks. |
InvalidURL | Special characters in the credentials were not encoded. |
Last updated
Still stuck?
People answer the live chat around the clock. Send the proxy line you used (without the password) and the error you see.