Python aiohttp

Use NyxonProxies with aiohttp and asyncio: a proxy per request or session, concurrency limits, rotation, sticky sessions, SOCKS5 with aiohttp-socks and errors.

aiohttp is an asyncio HTTP client built for many requests at once. It takes the proxy on each request or once on the session, and speaks HTTP proxies natively. SOCKS5 needs one extra package.

Install

bash
pip install aiohttp

Basic request

python
import asyncio
import aiohttp


async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get(
            "https://api.ipify.org?format=json",
            proxy="http://USERNAME:[email protected]:8080",
        ) as r:
            print(await r.json())


asyncio.run(main())

A proxy for the whole session

Pass proxy= to the session and every request uses it:

python
import asyncio
import aiohttp


async def main():
    async with aiohttp.ClientSession(proxy="http://USERNAME:[email protected]:8080") as session:
        async with session.get("https://api.ipify.org") as r:
            print(await r.text())


asyncio.run(main())

Many requests at once

Cap the number of requests in flight with a semaphore. Requests that run in parallel each open their own connection, so on a rotating username they leave from different IPs:

python
import asyncio
import aiohttp

PROXY = "http://USERNAME:[email protected]:8080"
limit = asyncio.Semaphore(10)


async def fetch(session, url):
    async with limit:
        async with session.get(url, proxy=PROXY) as r:
            return await r.text()


async def main():
    timeout = aiohttp.ClientTimeout(total=60)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        ips = await asyncio.gather(*(fetch(session, "https://api.ipify.org") for _ in range(20)))
        print(ips)


asyncio.run(main())

A new IP on every request

The session keeps connections open and reuses them, so requests sent one after another can share an IP. A connector with force_close=True closes each connection after its request, and every request gets a new one:

python
import asyncio
import aiohttp


async def main():
    connector = aiohttp.TCPConnector(force_close=True)
    async with aiohttp.ClientSession(connector=connector) as session:
        for _ in range(3):
            async with session.get("https://api.ipify.org", proxy="http://USERNAME:[email protected]:8080") as r:
                print(await r.text())


asyncio.run(main())

Sticky sessions per task

To keep one IP per task, give each task its own session ID in the username:

python
import asyncio
import secrets
import aiohttp


def sticky(country, minutes=10):
    sid = secrets.token_hex(4)
    return f"http://USERNAME-country-{country}-session-{sid}-time-{minutes}:[email protected]:8080"


async def check(session, country):
    proxy = sticky(country)
    for _ in range(2):
        async with session.get("https://api.ipify.org", proxy=proxy) as r:
            print(country, await r.text())


async def main():
    async with aiohttp.ClientSession() as session:
        await asyncio.gather(*(check(session, c) for c in ["us", "de", "gb"]))


asyncio.run(main())

Each country prints the same IP twice. See Rotating and sticky sessions.

SOCKS5

aiohttp's proxy= speaks HTTP only. For the SOCKS5 port, install aiohttp-socks and use its connector:

bash
pip install aiohttp-socks
python
import asyncio
import aiohttp
from aiohttp_socks import ProxyConnector


async def main():
    connector = ProxyConnector.from_url("socks5://USERNAME:[email protected]:1080")
    async with aiohttp.ClientSession(connector=connector) as session:
        async with session.get("https://api.ipify.org") as r:
            print(await r.text())


asyncio.run(main())

The connector sends the hostname to the gateway, so DNS is resolved on the proxy side. Write socks5://: it does not accept socks5h://.

Passwords with special characters

Percent-encode the password before you put it in the URL:

python
from urllib.parse import quote

password = quote("p@ss:w/rd#1", safe="")
proxy = f"http://USERNAME:{password}@geo.gw.nyxon.sh:8080"

Environment variables

aiohttp ignores HTTPS_PROXY and the other variables unless you create the session with trust_env=True. See Environment variables.

Errors

ErrorCause
ClientHttpProxyError with 407Wrong username or password.
ClientProxyConnectionErrorWrong host or port, or a firewall.
ServerDisconnectedErrorA socks5:// URL passed to proxy=. Use aiohttp-socks.
InvalidURLSpecial characters in the credentials were not encoded.

Last updated

Still stuck?

People answer the live chat around the clock. Send the proxy line you used (without the password) and the error you see.