# Changelog

All notable changes to this project are listed here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/).

## [Unreleased]

## [1.34.0] – 2026-11-12

### Changed
- **A proxy whose second or content request broke off in transport gets a second chance at the end of the run.** Free proxies drop connections for a while: in every hourly run about 350 confirms and 280 content checks ended in `connect_timeout`, `timeout`, `connection_closed` or `no_answer` right after the proxy had passed its basic check (and after the retry 1.5 s later), and those proxies were lost. Now they get both requests once more at the end of the run, at least 20 seconds after their try, and count with whatever those end as. `checks.second_chance` in `run.json` says how the second chances ended
- **The hourly run gives a request 10 seconds instead of 6** (`--timeout 10`; the connect stays at 4 s). Free proxies are slow – the median latency of the list is above 2 seconds – and ~10,000 candidates a run connected but answered after the 6 seconds. The check phase takes about 5–6 minutes instead of 3–5
- Measured in A/B runs on 12 Nov 2026 (runners started at the same time, 150,000 candidates each): the hourly list grew from 768 proxies (as before) to 986 with the second chance and to 1,177 with both. A 15-second timeout gave 6 % more than 10 seconds with a slower list; six new self-checking sources found by a source search gave 3 more proxies – their proxies are in the existing lists already – and were not added. The hourly list still swings with one backconnect gateway (45.74.31.40–47, ~230,000 listed addresses): when it is up, a run lists 2,000–2,500 proxies, otherwise around 1,000–1,200. Checking all its addresses instead of 150,000 candidates gave 537 instead of 323 on 12 Nov, and a run without the 150,000 limit 1,111 instead of 1,095 – the limit isn't what holds the list back

### Added
- The README of the list repository has Java (`java.net.http`, the `http://` lines of `https.txt`) and Rust (reqwest, `socks5.txt`) quick starts next to Node.js and Go (#388, thanks @tylrcc)

## [1.33.1] – 2026-11-11

### Changed
- A GitHub release now shows its version's section of this changelog. Before, it listed the merged pull requests, the release pull request included; the generated list is only the fallback for a version without a section
- **A second or content request that httpbin.org's rate limit held back goes to a fallback target.** After v1.33.0, 14.8 % of the confirm checks and 19.1 % of the content checks in 12 hourly runs ended `unavailable:target_rate_limited` – proxies on shared exits and backconnect pools, about 290 working proxies a run held back from the published list. In a measurement on 11 Nov (440 proxies; 39 parallel requests through 13 proxies sharing one exit) httpbin.org's `/get` and `/html` answered 78 of 78 with 429, while `http://ipv4.icanhazip.com/` returned the right IP 78 of 78 times (median 0.47 s) and `http://example.com/` the same 577 bytes 78 of 78 times (byte-identical across 176 exits), never a 429. Now such a confirm runs against `ipv4.icanhazip.com` (the same exit IP, or `failed:exit_ip_mismatch`) and such a content check against `example.com`, with `info.cern.ch` in reserve (byte for byte as fetched directly at the start of the run, or `failed:content_mismatch`). The check's `target` names the fallback, and `run.json` lists the run's fallback targets under `targets`. A proxy confirmed via the IP echo has no anonymity level – the echo shows no headers – so it isn't in `elite.txt`, and its check's `limits` say so. Without an answer from the echo or a reference page at the start of a run there is no fallback, and the check stays `unavailable:target_rate_limited`. Only the overflow goes there, with its own pacing of 20 requests per second per exit. How many proxies this brings back to the list will show in the next runs
- The PyPI project links point at the changelog and the docs in the repository and add the live list and the benchmark on proxy-scraper.de. The development status says Beta instead of Production/Stable

### Fixed
- **An exit IP with hundreds of proxies no longer stalls the check phase.** Once an exit was booked more than 30 s ahead, every further second or content request on it waited the full 30 s and then all went at once into the target's 429 (run of 11 Nov 08:24 UTC: a backconnect pool on one exit, a 2.5 times longer check phase, 1,457 proxies kept without a second request). Now a request whose slot is more than 30 s away is not sent and the check is `unavailable:target_rate_limited` at once
- **`run.json` keeps the protocol fallback apart from the basic checks.** The extra checks of another protocol on the same address were counted in `checks.basic`, so `checks.basic` no longer added up to `checked`; they are now under `checks.fallback`, and a new `fallback` field says how many ran and how many found a proxy (`{"tried": n, "found": n}`). In the per-protocol numbers of the live view and the summary, a proxy found this way now counts as checked for the protocol it was found as, not only as working
- When `-o -` output went to a pipe that closed early, the devnull descriptor used to quiet it stayed open; it is now closed. A dead assignment in the second-request code is gone (both found by CodeQL)

### Documentation
- The README starts with the lists: the four list URLs to copy, live counts per protocol and the time of the last run as badges, and the one-line `uvx proxy-scraper-cli --pick 3 --verify`. A new "Why this list" section shows the hourly benchmark chart of the popular GitHub lists and what each check requires. The video, the demo and the comparison of the three ways to get proxies moved to a section on the tool further down; nothing was dropped. The translations are unchanged

## [1.33.0] – 2026-11-11

### Added
- New source: proxyelite.info's free list (mixed protocols, updated every 15 min). A search of 432 GitHub repositories and
  31 public endpoints found no other new list that wasn't a mirror of one already read, blocked by robots.txt or stale.

### Changed
- **The second and the content request recover from a lost connection.** In the hourly runs about 1,300 of ~2,000 proxies that passed the first request failed the second, 380–930 of them with `connect_timeout` right after the same address had answered. Most of those are lost SYN packets: Linux sends a SYN again after 1 s and 3 s, and the run's 3 s connect timeout left room for little more than the first; 58 % of these proxies had already needed more than 0.9 s to connect in the first request (20 % of the confirmed ones). Now both later requests connect with at least 6 s (the time after the connect is unchanged, so their limit grows from 6 to 8 s in the hourly run), and a request that failed in transport (`connect_timeout`, `timeout`, `connection_closed`, `no_answer`) is tried once more after 1.5 s, through the same per-exit pacing and 429 handling. A wrong exit IP, a bad status or a changed page is not tried again. In an experiment, a failed second request retried after ~3 s passed in 29 % of cases (43 of 144 for `connect_timeout`); how much of that shows up in the published list is an estimate until the next runs. Worst case, one proxy's second request can take about 28.5 s instead of 15 in the hourly run (plus its pacing wait), and the content request the same; the check records only how the last try ended
- **An address that answers in another protocol than its list says is tried in that protocol.** When an address listed as HTTP answers, but not as HTTP (`no_answer`, `connection_closed`, `invalid_response`, `handshake_refused`), it is checked as SOCKS5 and then as SOCKS4 on the same `ip:port`; a SOCKS5 or SOCKS4 address that refuses the handshake is checked as the other SOCKS version. Only for the types the run checks, and not when that `type ip:port` is a candidate of the run anyway or was tried already, so nothing is checked twice. A proxy found this way is listed and remembered under the protocol it speaks; the listed entry still counts as a failure for its source. In an experiment, 34 % of HTTP-listed addresses that failed with `no_answer` worked as SOCKS, and SOCKS4/5 were swapped in 2 % of SOCKS failures; how many proxies that adds to the published list is an estimate until the next runs. The extra basic checks are counted in `run.json` under `checks.basic` like any attempt. Lists without a type ("auto") are still read as HTTP and SOCKS5 only, not SOCKS4
- **The hourly run gives the first request's TCP connect 4 s instead of 3** (`--connect-timeout 4`). Linux sends a lost SYN again after 1 s and 3 s, so 3 s cut the second retransmission off as it went out. Proxies that are really dead mostly fail with `connect_refused` or long before, so the run should take only somewhat longer; the gain in passes is an estimate until the next runs show it. The docs that describe the hourly run say so; [the numbers from 28–29 Sep](docs/free-proxies-in-numbers.md) were measured with 3 s and still say so

## [1.32.1] – 2026-11-10

### Changed
- **Back to 150,000 candidates per run.** The hourly runs with 300,000 on 10 Nov passed the basic check about as
  often as before (1,682–2,701 vs 1,795–2,578): the second 150,000 are lists whose proxies almost never answer. The
  share for barely checked sources stays.
- An unchanged list is only skipped as outdated when its proxies also stopped passing (under 1 % of at least 50
  recent checks). Static lists that still deliver, e.g. one with a 25 % hit rate, kept being skipped after a week.

### Fixed
- **Proxies that share an exit IP were failed for httpbin.org's rate limit, not for anything they did.** Since about 2026-10-29 httpbin.org, the second check target, answers HTTP 429 above 17 requests per second per client IP. Free proxies share exit IPs heavily (one hourly list: 350 proxies on 217 exits, up to 28 on one), and the run sent the second request and the content request of all proxies on one exit at once, so it tripped the limit itself; the 429 was recorded as `failed:bad_status`. Since then the share of first-check passes that also passed the second request fell from about 60 % to 30–40 %. (The published list fell from about 1,500 proxies to about 450 in the same weeks; most of that was one SOCKS5 gateway with many ports, each counted as a proxy, that shrank around 29 Oct – this limit was the smaller part.) In the run that was measured, proxies with an exit of their own passed the second request 71 % of the time, proxies on an exit shared by three or more 27 %. Now requests to the second target are spaced to at most 8 per second per exit IP (the wait for a slot doesn't count against the proxy's timeout and is capped at 30 s); a 429 is asked once more after its `Retry-After` (at most 3 s); and a target that still refuses with its rate-limit headers makes the check `unavailable:target_rate_limited` – held back from the published list, never counted as a dead proxy. How much of the list comes back will show in the next runs; a bare 429 without rate-limit headers, and every 503, still count against the proxy
- `bad_status` now means only "an HTTP status other than 200". A connection that ends before a complete response head arrives is the new reason `no_answer` (first, second and content request). Before, both were `bad_status`, so counts of `bad_status` from earlier runs include what is now `no_answer`

## [1.32.0] – 2026-11-10

### Added
- A benchmark page, https://proxy-scraper.de/benchmark/: "Which free proxy list on GitHub actually works?" The popular free proxy lists on GitHub side by side, rewritten with every hourly run – proxies listed, checked, passed every check and their share in recent runs, plus when each list last changed. The numbers are the ones of the sources ranking; the page explains how they are measured and why proxy-scraper's own list is not in it. Also as `benchmark.json` and as a chart to share or embed (`benchmark-light.svg`, `benchmark-dark.svg`), linked from the sources page, the sitemap and the README of the `proxy-list` branch

### Security
- **The pool API of the rotating server is guarded like its status page.** Without `--serve-password` it answered any `GET`: a website that points its own name at `127.0.0.1` (DNS rebinding) could read `/all`, and an `<img src="http://127.0.0.1:8899/delete?proxy=…">` on any page could empty or change the pool through `/delete`, `/pop` and `/report`. Now, without a password, the API only answers requests addressed to an IP or `localhost` (`403` for a host name); `/pop`, `/delete` and `/report` refuse requests a browser marks as coming from another website (`Sec-Fetch-Site`, or a foreign `Origin`) with a password too; and logins of upstream proxies (`user:password@`) are masked in `/get`, `/pop`, `/all` and `/delete` (as on the status page) unless the API is protected by `--serve-password`. `curl` and scripts work as before
- An `Authorization` header (the GitHub token of the source search) or a cookie is no longer sent on to another host – or from `https` to `http` – when a download is redirected
- The page fetcher of the MCP server (`fetch_url`) refused private addresses but accepted them inside an IPv6 wrapper such as NAT64 (`http://[64:ff9b::7f00:1]/`); it now uses the same address check as the source downloads
- `proxies.csv` (each run and the published one): text cells that start with `=`, `+`, `-`, `@`, a tab or a carriage return get a leading `'`, so a spreadsheet doesn't run a provider name from the ASN database as a formula. Numbers are unchanged
- Discord bot: provider names and other fields from the list are escaped, so `[text](url)` can't become a masked link and a backtick can't close the code block
- The list repository push pins GitHub's published SSH host keys instead of trusting the first key it sees (`StrictHostKeyChecking=accept-new`)
- The live list (MCP tools, Python API) and the monthly country/provider databases are only downloaded over verified TLS; the unverified retry for TLS-inspecting firewalls is kept for public proxy lists, whose proxies are all checked before use

### Changed
- Website: the first screen now shows what visitors come for. Next to a smaller headline are the count from the last run with its HTTP/SOCKS4/SOCKS5 split, the links to `all.txt`, `http.txt`, `socks4.txt` and `socks5.txt` with a copy button each, and "Star on GitHub" with the live star count. Below it, a quick start with tabs for curl, pipx, uvx and Python. All numbers come from `stats.json` and space is reserved for them, so nothing moves while they load
- The website moved from GitHub Pages to its own domain, https://proxy-scraper.de/ (own server). The hourly run uploads it there; canonical links, sitemap, feeds, the bookmarklet and the docs point to the new address. The `proxy-list` branch and its raw files stay as they are
- The Discord/Telegram bot links to and reads from https://proxy-scraper.de; `CITATION.cff` and the PyPI documentation link point there too
- The README of the list repository (`free-proxy-list`, written by `mirror.py`) starts with the raw URLs to copy, a "checked every hour" badge (from `run_hours`) and the repository's stars. Every list links to GitHub raw, jsDelivr and proxy-scraper.de; usage has `curl`, `wget` and Python up front, Node.js and Go folded away. A new section explains each check, which lists need it and how many proxies passed it in the run (`run.json`), the share of checked candidates that made the list replaces the fixed "150,000", and a star-history chart sits next to the star request. Without `run.json` those numbers are left out
- **The hourly run checks 300,000 candidates instead of 150,000** (about 4–5 minutes of checking instead of 2). The texts on the website, the source pages, the weekly report, the Hugging Face card and the FAQs say so
- **A limited run samples lists that were barely checked.** With `--limit`, 15 % of the slots after the known working proxies go to sources with fewer than 10 checks in the learned statistics, one random proxy from each in turn and a different one every run. Before, a list whose proxies never made it under the limit was never checked, so it kept its starting estimate and never moved up. Slots that no such source needs go back to the usual order; without `--limit` nothing changes

### Fixed
- The weekly report feed got a second entry for the same week after the move, because entries were matched by their full id (which contained the old address). Entries are matched by week now and only one per week is kept
- The live list for the MCP tools and the Python API came only from the website, which is uploaded separately and may lag behind; when the website's copy is out of date the `proxy-list` branch is asked too and the newer list wins
- The website upload gave every unchanged file a new timestamp each hour, so browsers and crawlers never got a `304`; unchanged files keep their time now
- Docs checked against the current code and the live data of 10 Nov 2026: outdated numbers replaced by dated ones (sources, candidates, datacenter and blocklist shares, reason-code totals, test count), the GitHub Action example now uses a tag that exists, the options table lists every option, the website is described as its own server instead of GitHub Pages, and withdrawn wording is gone from the FAQ, the background pages and the translations. `--completion` help names PowerShell too

## [1.31.0] – 2026-10-23

### Security
- **Lists found automatically are only downloaded from public addresses.** The GitHub search took any host name with a dot, and neither the addresses it resolved to nor redirects were checked: a config naming `127.0.0.1.nip.io`, `0x7f.1` or `[64:ff9b::7f00:1]` made the scan send a request to this machine or the local network. Mined URLs, the lists saved by earlier searches and the URLs in third-party meta lists are now checked on every redirect hop (every resolved address, all IPv4 spellings, IPv4-mapped and NAT64 IPv6), and the connection goes to the address that was checked, with Host header, SNI and certificate check on the name. Curated lists and `--source` are unchanged (#356)
- **The rotating server no longer sends a request twice when that could do something twice.** A small `POST` was repeated through the next proxy if the first one delivered it and then dropped the connection. After a request may have gone out, only `GET`, `HEAD`, `OPTIONS` and `TRACE` go to another proxy; everything else gets a `502` with `X-Proxy-Scraper: no-retry-after-send`. Before sending, failover works for every method as before (#352)

### Fixed
- Rotating server: a request body that stops arriving ends with `408` instead of holding the connection forever; at most 1,024 clients at once (`503`, `Retry-After: 1`); a proxy's interim answers (`100`, `103`) are capped in number, size and time; a cancelled connection always gives its proxy back to the pool (it stayed counted as busy). Tunnels keep no idle limit (#352)
- A country lookup still running when the last check ended was cancelled, so the proxy kept no country and `--country` dropped it (#353)
- The check target's health check asked a fresh DNS answer instead of the address the checker was pinned to, so a healthy new backend could hide a dead one; the checker now follows a moved backend together with the watch (#353)
- Two processes or threads writing the same state file at once could fail with `FileNotFoundError` (one fixed temporary name); each write now has its own temporary file and is fsynced. With overlapping runs the last one to save wins – all such files are caches or ordering hints (#353)
- `ProxyRotator.close()` twice raised `RuntimeError: Event loop is closed`; closing is now idempotent and final, and using a closed rotator says so (#353)
- A chunked response that wasn't followed by the connection closing ran into the timeout; bodies end at the last chunk now, and cut-off or malformed ones are an error instead of a silent partial body (#356)
- Parallel blocklist lookups of the same exit IP each asked DNS; they share one lookup now (#356)

### Changed
- README cut to the essentials (install, first command, the check states, quick starts); recipes, options, FAQ, how it works and the full server, MCP and Python docs moved to [docs/](docs/README.md). All old anchors still work (#351)
- CI: the merge checks no longer depend on free proxies being up (fixtures and the localhost lab instead); the real internet checks run daily as `live-smoke`. New jobs build a wheel from the sdist and install it outside the checkout, and run the Parquet round-trip with PyArrow for real. Python 3.9 tests run on Ubuntu 24.04 (#354, #357)

## [1.30.0] – 2026-10-16

### Added
- **Every check has an outcome you can read.** Each kept proxy carries `checks`: `basic`, `confirm`, `content` and `https`, each `passed`, `failed`, `unavailable` or `skipped`, with a reason code, the target, the time (`at`, and `checked_at` for the proxy) and what was measured. In `proxies.json`, as columns at the end of `proxies.csv`, in the Python API (`result.checks`) and in the MCP results. [docs/check-states.md](docs/check-states.md) has the states, the reason codes and what each list requires
- `--pick N --verify` checks the candidates of the hourly list from your machine first and prints only the ones that pass now (with `--https-only` the HTTPS test too). It tries 5 candidates per proxy asked for (10 to 60), up to 8 at a time, for 90 seconds at most. Exit code 3 when fewer than N passed; they are still printed. What happened to each candidate goes to stderr
- `--diagnose PROXY` checks one proxy and shows what each check found: reachable or not and why, exit IP, second request, content, HTTPS, latency and time, and what could not be checked. `--json` for a report scripts can read. Passwords are masked in all of it, also in the error for an address that can't be read. Unlike a scan it takes any address; a proxy on your own machine or network will usually end as `own_ip_exposed`, because its exit IP is yours
- `--pick --json` prints when the list was published, how old it is and whether it is overdue, next to each proxy's own check time
- Every run writes `run.json` next to its results: how many candidates there were, how many got a verdict, how many didn't, each check by state and reason, and how many of the changed test pages contain a `<script` tag. The hourly list publishes it and keeps the counts in `history.json`
- `docs/analysis/`: scripts that reproduce the numbers of [the numbers article](docs/free-proxies-in-numbers.md) from the workflow logs and from the list's own files, with the per-run data of 28–29 Sep 2026 as a CSV
- `docs/demo/`: a lab of simulated proxies on localhost and the recording made with it

### Changed
- **A check that didn't run is no longer reported like one that passed.** Without a reachable second target (httpbin.org) a run used to keep every proxy as if it had been confirmed, and without a reference page a rewritten page looked like an unchanged one. Both are `unavailable` now in the files; the run says so in a note (at the start for a missing reference page, at the end for hits kept without a second request)
- **A check target that goes down no longer fails the proxies checked against it.** The second target was only asked once, at the start: if it went down mid-run, every proxy after that was counted as a failed second request and learned as dead. Now a failed second request or HTTPS test makes the tool ask the target directly (every 3 seconds at most), and if it doesn't answer either, the proxy's check is `unavailable`. `--diagnose` and `--pick --verify` do the same for the first request
- The hourly list only lists proxies whose basic check, second request and content check were measured and passed, `https.txt` also the HTTPS test. `stats.json` says so in `requires` and counts the rest in `held_back`. If the second target was down for part of a run the list is shorter; a run with fewer than 20 measured passes publishes nothing, and the previous list stays online
- `https` in the results is `null`, not `false`, when the HTTPS test could not run (its target was down). A proxy whose content request got no page back is `unavailable` for that check and therefore not on the published list; before, it counted as unchanged
- Words: a failed second request is no longer called a "fake" or a "honeypot", a changed test page no longer "tampering" or "injected scripts", in the terminal, on the website, in the list's README and in the package description. The checks measure what they measure; why a proxy fails them, they don't know
- The numbers article says what each number counts and what it can't tell. Corrected: 3.6 million are checks, not proxies; the page differed for 5.2 % of the proxies that reached that check (not "1 in 40 working proxies"), for reasons nobody recorded; "85 % gone within an hour" was a share of ended stretches, and about a quarter of a list is on the list again a day later
- README: the top says in one screen what the tool is for, with a real terminal recording and the three ways to get proxies side by side. The generated screenshots are labelled as illustrations with sample data. The comparison is by task and includes two tools it left out, and where they are the better choice
- The website's "typical free list" column is gone (no other list was measured) and its animated run is labelled as an illustration

- Docker image: base image is now `python:3.14-slim` (was 3.12), still pinned by digest. CI dependencies bumped together: ruff 0.17.0 (lock file regenerated, the separate Dependabot PR failed on the stale `lint.txt`), CodeQL action 4.38.3 for init, analyze and upload-sarif at once (the analyze-only bump failed on a version mismatch – Dependabot now groups them), Scorecard action 2.5.0, upload/download-artifact 7.0.2/8.0.2

### Fixed
- On Windows with Python before 3.13, a control answer from just before a check began could count as fresh (the event loop's clock ticks every ~15.6 ms there), so `--diagnose` and `--pick --verify` could compare a changed test page against an old copy or blame a proxy for a target that was down. Control answers and checks are now ordered by `time.perf_counter()`
- A proxy whose TCP connect used up the whole time limit was reported as `timeout` ("accepted the connection but did not answer") instead of `connect_timeout`, and with a control set (`--diagnose`, `--pick --verify`) it could even become "target unreachable". On Windows, which retries a refused local port for about 2 s, this was the normal case
- `--pick` when the list can't be loaded suggested `check_proxies`, which is a tool of the MCP server, not a command
- `--recheck` with only local or private addresses said "No proxies to check" without saying that such addresses are skipped
- Rows of the downloaded list are only used when they are `ip:port` of a public address, by `--pick`, the Python API, the MCP tools and the rotating fetcher alike. A row with a line break, a login or a local address was passed on as it came
- A proxy answering the second request with deeply nested JSON ended the whole run with a `RecursionError` on Python 3.9
- Running out of sockets during a check was learned as a dead proxy

## [1.29.0] – 2026-10-14

### Security
- The rotating proxy server no longer starts as an open proxy by accident: `--serve-host` on any address but `127.0.0.1` / `::1` / `localhost` now needs a password (`PROXY_SCRAPER_SERVE_PASSWORD`) or the explicit `--serve-open`. Before, this was only a warning – and Docker publishes ports past host firewalls like ufw, so `-p 8899:8899` without a password served anyone on the internet. The check runs before the scan, exits with code 2 and says what to set. `compose.yaml` already uses a password and is unaffected. If you run the image with `--serve-host 0.0.0.0 -p 127.0.0.1:8899:8899` and no password, add `--serve-open` ([#328](https://github.com/maximilianfeix/proxy-scraper/pull/328))
- Supply chain: every CI install from the hash-locked requirement files now runs with `--require-hashes`, so a line without a hash fails the build instead of installing unchecked, and the Docker base image is pinned by digest (multi-arch index) with a weekly Dependabot update that keeps the base image's security fixes coming. The release job checks `mcp-publisher` against the sha256 from its release before running it ([#329](https://github.com/maximilianfeix/proxy-scraper/pull/329))

### Fixed
- `test_finds_add_up_and_old_ones_expire` depended on today's date and failed from 2026-10-12 on ([#327](https://github.com/maximilianfeix/proxy-scraper/pull/327))

## [1.28.0] – 2026-10-11

The first tag since 1.27.0 – everything listed under 1.27.1 below ships with this release too.

### Added
- `--recheck https://example.com/proxies.txt` checks a list straight from the web – downloaded and read like stdin (plain lines, `type://`, logins, JSON, CSV, tables). A download error is one line, not a traceback; `--serve-refill` re-downloads the list and falls back to the history when it's gone ([#267](https://github.com/maximilianfeix/proxy-scraper/issues/267))
- `--export proxifier` writes `proxifier.ppx`, a profile for Proxifier 4 on Windows and macOS: the best 50 proxies in a redundancy chain, localhost direct, logins XML-escaped ([#268](https://github.com/maximilianfeix/proxy-scraper/issues/268))
- A Selenium recipe: Chrome and Firefox through the local rotating server, since neither can send a proxy login ([#266](https://github.com/maximilianfeix/proxy-scraper/issues/266))
- READMEs in German, Russian, Brazilian Portuguese and French (`README.de.md`, `README.ru.md`, `README.pt-BR.md`, `README.fr.md`), condensed like the Chinese one, with one language bar across all READMEs and a test that keeps it complete ([#294](https://github.com/maximilianfeix/proxy-scraper/issues/294), [#265](https://github.com/maximilianfeix/proxy-scraper/issues/265))
- Which free proxy lists actually work: the website has a [ranking of the sources](https://maximilianfeix.github.io/proxy-scraper/sources/), rewritten every hour from what the runs learned. Every source with the proxies it lists, how many were checked, how many passed every check and the share, plus whether the list is still maintained. The files of one repository count as one source, and the numbers are those of an average recent run. All of it is in `sources.json` too

### Fixed
- The MCP server now starts after a plain `uv sync`: the MCP SDK is also a default uv dependency group, not only the `mcp` extra. Glama builds the server that way, and without the SDK its build found no tools, so its page stayed on 1.25.0. CI now builds the server the same way ([#303](https://github.com/maximilianfeix/proxy-scraper/issues/303))
- An IPv6 literal as a check target or `--target` went out without brackets: `CONNECT 2606:4700::1:443`, and the same in the `Host` header and the absolute URL for HTTP proxies. The checks, the target sites and the steps after a run now build these through the same helper as the rotating server ([#302](https://github.com/maximilianfeix/proxy-scraper/issues/302))

## [1.27.1] – 2026-10-08

### Added
- A Spanish README (`README.es.md`) ([#270](https://github.com/maximilianfeix/proxy-scraper/issues/270))
- GitHub Marketplace-ready Action metadata and a complete workflow example with all inputs and outputs.

### Fixed
- Malformed URLs in third-party source files no longer abort discovery or the hourly publishing pipeline. Valid IPv6 source URLs retain their brackets, with regression and generated-input tests ([#297](https://github.com/maximilianfeix/proxy-scraper/pull/297)).
- The Action description now fits GitHub Marketplace's length limit.

### Fixed
- Requests to a list or API on a non-default port sent the `Host` header without the port; the rotating server now brackets IPv6 literals in the `Host` header and the forwarded request line too ([#300](https://github.com/maximilianfeix/proxy-scraper/issues/300))
- A malformed target name (`example..com`, a label over 63 characters, spaces or line breaks) made the rotating server blame healthy proxies: a few such requests could disable them. It is now refused up front with a 502, before any proxy is picked ([#300](https://github.com/maximilianfeix/proxy-scraper/issues/300))

## [1.27.0] – 2026-10-06

### Added
- README: the MCP server's Glama rating (A) ([#288](https://github.com/maximilianfeix/proxy-scraper/issues/288))

### Security
- Releases are signed with Sigstore: the wheel and sdist on the GitHub release come with `.sigstore.json` bundles, made keylessly by the release workflow and verified in the same run. Check a download with `sigstore verify identity --cert-identity https://github.com/maximilianfeix/proxy-scraper/.github/workflows/release.yml@refs/tags/v1.27.0 --cert-oidc-issuer https://token.actions.githubusercontent.com <file>` ([#290](https://github.com/maximilianfeix/proxy-scraper/issues/290))
- The workflows install Python packages only from hash-locked files (`.github/requirements/`), one universal lock per use; a test keeps each lock in line with its source ([#292](https://github.com/maximilianfeix/proxy-scraper/issues/292))

### Fixed
- A test that failed now and then on a busy machine: it read only the first part of the dashboard page ([#286](https://github.com/maximilianfeix/proxy-scraper/issues/286))

## [1.26.0] – 2026-10-06

### Added
- Logins in the `ip:port:user:pass` form that paid proxy lists often use, with or without `type://`, in the lists you hand over: `--recheck file.txt` and `--recheck -`. The bare `ip:port` is tried too (after the login), since `ip:port:US:elite` looks the same, and the password ends at a paste's punctuation (`,;|"'` and brackets). Scraped sources and `--source` leave the form alone, so public lists don't sprout made-up logins ([#254](https://github.com/maximilianfeix/proxy-scraper/issues/254))
- README: who already uses the hourly list – it's a built-in source of monosans/proxy-scraper-checker and gfpcom/free-proxy-list ([#263](https://github.com/maximilianfeix/proxy-scraper/issues/263))
- `CITATION.cff`, so GitHub shows "Cite this repository" ([#275](https://github.com/maximilianfeix/proxy-scraper/issues/275))
- OpenSSF Scorecard runs weekly; results in the Security tab and on scorecard.dev ([#273](https://github.com/maximilianfeix/proxy-scraper/issues/273))

### Fixed
- The PyPI page: none of the README's images loaded there and 33 links led nowhere, because they were relative. The release build now makes them absolute for the tag and fails if a Markdown parser still finds a relative one ([#280](https://github.com/maximilianfeix/proxy-scraper/issues/280))
- `load_dataset` on the Hugging Face snapshots failed ("column names don't match"): every day now has the same columns, the site results are `works_on` / `blocked_on` lists, unknown values are null, and all earlier days were rewritten. The dataset card explains every field ([#271](https://github.com/maximilianfeix/proxy-scraper/issues/271))

### Security
- Workflows start read-only; write access only on the jobs that need it ([#277](https://github.com/maximilianfeix/proxy-scraper/issues/277))
- Every GitHub Action is pinned to a commit SHA, kept current by dependabot ([#282](https://github.com/maximilianfeix/proxy-scraper/issues/282))

## [1.25.0] – 2026-10-05

### Added
- The Docker image is an MCP server too: `docker run -i --rm ghcr.io/maximilianfeix/proxy-scraper --mcp` speaks MCP over stdio with `get_proxies`, `check_proxies` and `fetch_url` – no Python needed. The docker workflow talks MCP to every built image and fails unless the three tools are there. The README shows it as a client config, and `glama.json` names the maintainer for MCP directories that build servers from the Dockerfile ([#256](https://github.com/maximilianfeix/proxy-scraper/issues/256))
- README: the PyPI downloads next to the version ([#257](https://github.com/maximilianfeix/proxy-scraper/issues/257))

## [1.24.0] – 2026-10-05

### Added
- `--recheck -` checks a list piped in on stdin: `ip:port`, `type://ip:port` anywhere in the text (several on a line, inside JSON), logins, JSON, CSV exports and HTML or Markdown tables – in the order they came in, so `--limit` takes the first ones. Untyped proxies are tried as HTTP and SOCKS5 (as SOCKS4 with `--types socks4` alone), and every one of them counts. Read as bytes, so a stray non-UTF-8 byte or a BOM doesn't spoil it; a terminal on stdin gives a clear message instead of waiting ([#248](https://github.com/maximilianfeix/proxy-scraper/issues/248))
- The site check tries Discord too: `works-with/discord.txt` on the live list, the website filter, `--pick --works-on discord`, `live_proxies(works_on="discord")`, the MCP tools and the action. Measured on 80 live proxies: 32 through, 15 turned away by Cloudflare. YouTube, X, Netflix, GitHub and Bing let all of them through, so they'd say nothing and aren't added ([#247](https://github.com/maximilianfeix/proxy-scraper/issues/247))

### Changed
- README: a no-install line at the top (`uvx proxy-scraper-cli --pick 5` – five checked proxies in about five seconds), the animated demo opens on the live dashboard instead of a mostly empty box, the check diagram moved to *How it works*, and the first FAQ entry answers who runs free proxies and what that means for using them ([#249](https://github.com/maximilianfeix/proxy-scraper/issues/249)). The contributors are on it too ([#144](https://github.com/maximilianfeix/proxy-scraper/issues/144))

## [1.23.0] – 2026-10-02

### Added
- Python 3.14 support: tested in CI on Linux, macOS and Windows ([#229](https://github.com/maximilianfeix/proxy-scraper/issues/229)) – thanks [@sameer-dhande](https://github.com/sameer-dhande)
- The hourly lists in a repository of their own: [maximilianfeix/free-proxy-list](https://github.com/maximilianfeix/free-proxy-list) – every protocol, HTTPS, elite, stable, the big sites and now one list per country (`countries/de.txt`), plus JSON and CSV, with the numbers of the last run on its README. `proxyscraper/mirror.py` builds it after `publish.py`; the workflow pushes one commit per run and squashes the history at the start of every month. Without the `LIST_REPO_KEY` secret (forks) the step does nothing
- The README on the `proxy-list` branch links to it
- Quick start examples for Node.js, Go and curl with SOCKS5 in the free-proxy-list README ([#230](https://github.com/maximilianfeix/proxy-scraper/issues/230)) – thanks [@sameer-dhande](https://github.com/sameer-dhande)
- The daily snapshots also go to a Hugging Face dataset as Parquet, one file per UTC day with a `snapshot_date` column, when `HF_DATASET_REPO` and `HF_TOKEN` are set ([#188](https://github.com/maximilianfeix/proxy-scraper/issues/188)) – thanks [@arm21-afk](https://github.com/arm21-afk)
- A new source: the hproxy-com HTTP list, 1.3 % of 3,000 candidates working in a test run – thanks [@arm21-afk](https://github.com/arm21-afk)
- Country names for 22 more countries, so their proxies get a page on the website – thanks [@sujaljadhav14](https://github.com/sujaljadhav14)
- A live dashboard for the rotating server at `/__proxy-scraper/`: usable proxies, success rate, requests, traffic and uptime, a chart of requests per second, countries, the best proxies and the latest connections, refreshed every two seconds. Self-contained (no CDN, works offline), client data rendered only as text, a strict Content-Security-Policy with the script pinned by hash, and `--serve-password` protects it. `/__proxy-scraper/status` now also lists the latest connections (without who sent them and with upstream passwords masked) and the countries of the pool. Without `--serve-password`, the latest connections are only in answers to requests made to an IP address or `localhost` – a website using DNS rebinding can't read them ([#240](https://github.com/maximilianfeix/proxy-scraper/issues/240))
- [README.zh-CN.md](README.zh-CN.md): a Simplified Chinese README – install (with a PyPI mirror), the checks, the hourly list with GitHub Pages and jsDelivr for networks where raw.githubusercontent is unreliable, the proxy_pool-compatible API, the rotating server, Python, MCP and an FAQ. The English README links to it
- `brew install maximilianfeix/tap/proxy-scraper` in the install section of both READMEs. The formula in [maximilianfeix/homebrew-tap](https://github.com/maximilianfeix/homebrew-tap) updates itself within about a day of each release
- `--export pac` writes `proxy.pac` for browsers, FoxyProxy and the OS proxy settings: the best 30 HTTP and SOCKS5 proxies in order, local names and private IPv4 addresses direct, no `DIRECT` fallback, no DNS lookups and no SOCKS4 (which would resolve names locally), ES5 so Windows' PAC engine runs it, failing closed when there are no proxies. The live list publishes it every hour as `proxy.pac` ([#242](https://github.com/maximilianfeix/proxy-scraper/issues/242))
- Recipe: the rotating server as the upstream of Burp Suite, OWASP ZAP and mitmproxy, with where to set it in each and why ZAP needs its HTTP (not SOCKS) setting; tested end to end with mitmproxy ([#244](https://github.com/maximilianfeix/proxy-scraper/issues/244))

### Changed
- "Best first" now counts how likely a proxy is still up. Over a week of hourly runs, 25 % of new proxies were listed again an hour later, 99 % of those listed for a day. The published lists, the pages, the website, `--pick`, `live_proxies()`, `ProxyRotator` and the MCP tools rank by page time divided by that chance, so the first lines of a file hold proxies that last: on the list of 2026-09-29 the share of the top 50 expected to still work went from 67 % to 88 %, with the median page time from 1.1 to 1.4 s. Own scans have no streaks and rank by speed as before

### Fixed
- README and ARCHITECTURE quoted early measurements ("one in five" proxies inject scripts, "5 out of 6" hits are honeypots, 450+ tests) – now the numbers from 3.6 million checks (one in 40, 54 %) and 760+ tests

## [1.22.0] – 2026-09-29

### Added
- Website: a format picker next to "Copy filtered" and the downloads – `type://ip:port`, plain `ip:port`, JSON or CSV. The choice is remembered in the browser
- A one-time hint after the second run that found proxies, pointing to the GitHub repo. Only in an interactive terminal, never in CI (`CI` set) or with `PROXY_SCRAPER_NO_STAR_HINT=1`
- [docs/free-proxies-in-numbers.md](docs/free-proxies-in-numbers.md): what 24 hourly runs (3.6 million checks) and a week of lists found

### Fixed
- Website: the proxy table fits the page from 360 px phones to wide screens. Before, the provider column sat behind a sideways scroll even at 1440 px, and on phones the latency was cut off. Narrower screens drop the least needed columns (provider, then HTTPS and "Gets through", then speed), and "Gets through" shows two sites and a "+N" count so rows stay one line high

## [1.21.0] – 2026-09-28

### Added
- Proxy pool API on the `--serve` port: `/get`, `/pop`, `/all`, `/count`, `/delete` and `/report`, with filters for country, protocol, HTTPS, anonymity and latency, and `format=txt` for plain URLs. Endpoints and JSON fields follow jhao104/proxy_pool, so code written for it works unchanged. With `--serve-password` it wants the password as Basic auth, like the status page
- `--source URL|FILE` (repeatable) adds your own proxy lists – URLs or local files, any text with ip:port, `socks5=…` fixes the type. They are never skipped by the learned source ranking. `--only-sources` loads nothing else, and the history then only reorders proxies from these lists

## [1.20.0] – 2026-09-28

### Added
- `ProxyRotator.proxy_url()`: one local proxy address for requests, httpx, Playwright, Scrapy or curl that rotates through the live list behind the scenes – best first, with failover, HTTPS only through verified-TLS proxies, country and protocol from the rotator. `playwright_proxy()` gives the same as Playwright's proxy settings. It takes over each new hourly list by itself ([#215](https://github.com/maximilianfeix/proxy-scraper/issues/215))

## [1.19.0] – 2026-09-28

### Changed
- The published lists (all.txt, http.txt, socks5.txt, …, proxies.json), the country and protocol pages and the website now come best first too: by first answer plus the download speed of the last check, like `--pick` since 1.18. Whoever takes the first lines of a file gets the ones that load pages. The website has a "Best first" button to go back after sorting by a column ([#212](https://github.com/maximilianfeix/proxy-scraper/issues/212))

## [1.18.0] – 2026-09-28

### Changed
- "Fastest first" now counts the download too, not just the first answer: `--pick`, `live_proxies()`, `ProxyRotator`, the MCP server's fetch_url and the action rank proxies from the live list by the speed measured in the last check. With real pages, the first 25 HTTPS proxies loaded 26 of 50 pages (median 2.8 s) instead of 4 of 50 (20 s). Own scans without speed data rank by latency as before ([#186](https://github.com/maximilianfeix/proxy-scraper/issues/186))

### Added
- `--pick --min-speed KBPS`, `live_proxies(min_speed=…)` already had it, and `min-speed` for the GitHub Action ([#186](https://github.com/maximilianfeix/proxy-scraper/issues/186))

## [1.17.0] – 2026-09-28

### Added
- The country and protocol pages (like /country/de/ or /socks5/) got a real overview: how many work right now, tunnel HTTPS, stayed on the list all week and get through to Google, the median speed, speed/uptime/sites per proxy in the table, the ready-made `--pick` and curl commands for exactly that selection, and answers to the usual questions ([#207](https://github.com/maximilianfeix/proxy-scraper/issues/207))

## [1.16.0] – 2026-09-28

### Added
- `ProxyRotator` in the Python API: `rotator.get(url)` loads a page through the hourly list and switches to the next proxy when one fails – the loop every scraper writes by hand, on the same failover as `--serve` and the MCP server's fetch_url. Sync and async ([#204](https://github.com/maximilianfeix/proxy-scraper/issues/204))

## [1.15.0] – 2026-09-27

### Added
- A Telegram bot: `/proxy de socks5` answers with a fast proxy and a curl line to try it, `/proxies 10 us https` with a short list, `/stats` with the last run. It runs in the same process as the Discord bot, needs only a `TELEGRAM_TOKEN` secret, and uses long polling – no open port ([#187](https://github.com/maximilianfeix/proxy-scraper/issues/187))

## [1.14.0] – 2026-09-27

### Added
- `proxy-scraper --pick [N]`: proxies from the hourly checked list in about half a second, no scan – with the usual filters plus `--min-uptime` and `--works-on`, only the URLs on stdout, so `curl -x "$(proxy-scraper --pick)"` works ([#199](https://github.com/maximilianfeix/proxy-scraper/issues/199))

## [1.13.0] – 2026-09-27

### Added
- Website: "Why this list, not another one" – for each thing you'd want to know about a proxy, what a typical free list tells you and what this one measured in the last run, with live numbers. And an FAQ (safety, why free proxies die, updates, license, Python and AI agents) with FAQPage structured data for search engines ([#196](https://github.com/maximilianfeix/proxy-scraper/issues/196))

## [1.12.0] – 2026-09-27

### Added
- A bookmarklet: drag "Get a proxy" from the website to the bookmarks bar, and one click on any page copies a proxy that was on the list in 90 %+ of this week's checks. Plus "copy a reliable one now" right on the site ([#189](https://github.com/maximilianfeix/proxy-scraper/issues/189))

### Changed
- The steps after a run (site check, speed) share their plumbing in `runsteps.py`, and nothing outside the checker touches its private TLS tunnel anymore ([#192](https://github.com/maximilianfeix/proxy-scraper/issues/192))

### Fixed
- Website: long country names wrapped in the Countries chart, the labels share one column now

## [1.11.0] – 2026-09-27

### Added
- An Atom feed for the weekly report (report/feed.xml): one entry per week, to follow it in a feed reader or pipe it into Slack, Discord or Mastodon ([#185](https://github.com/maximilianfeix/proxy-scraper/issues/185))
- Instagram and TikTok in the check of which sites let a proxy through – on the website, in works-with/, the MCP server and `live_proxies()`. Measured first: ChatGPT, eBay and Indeed block every proxy and LinkedIn and Twitch let all of them through, so those would say nothing
- Daily snapshots: the first run of each UTC day keeps its proxies.json for good, gzipped, as an asset of that year's `snapshots-YYYY` release ([#143](https://github.com/maximilianfeix/proxy-scraper/issues/143))
- Download speed per proxy: after every run each HTTPS-capable proxy downloads 100 KB from Cloudflare's speed test, 12 s at most, timed from the request to the last byte. `speed_kbps` in proxies.json and the CSV, a sortable Speed column on the website (the Anonymity column made room – the Elite filter is still there), on the proxy pages and in the weekly report, `min_speed` in `live_proxies()` and `min_speed_kbps` in the MCP server ([#140](https://github.com/maximilianfeix/proxy-scraper/issues/140))

### Fixed
- The same GitHub file through jsDelivr or githack counted as a separate source and was downloaded twice – mirrors are mapped to the original raw URL now
- Website: sorting by a column with gaps (speed, uptime) put the proxies without a value in between – they always go last now

## [1.10.0] – 2026-09-27

### Added
- A GitHub Action: `uses: maximilianfeix/proxy-scraper@v1` gives any workflow working proxies – same filters as `live_proxies()`, optionally rechecked from the runner, outputs `proxy`, `file` and `count` ([#178](https://github.com/maximilianfeix/proxy-scraper/issues/178))
- Website: a world map of where the working proxies are – hover for the count, click a country to filter the list. And the page stays live: it shows when the next check is due and loads the new list by itself when it lands ([#176](https://github.com/maximilianfeix/proxy-scraper/issues/176))
- Tab completion for PowerShell: `proxy-scraper --completion powershell | Out-String | Invoke-Expression`. Options with their help, the choices after an option, paths where a file goes. Tested against a real pwsh via TabExpansion2 ([#114](https://github.com/maximilianfeix/proxy-scraper/issues/114))
- `--export singbox`: a sing-box config with a local HTTP+SOCKS proxy on 127.0.0.1:2080 and a urltest group over all proxies, SOCKS4 included. Checked with `sing-box check` and a real run ([#113](https://github.com/maximilianfeix/proxy-scraper/issues/113))
- `--list-sources --json` prints the source ranking as JSON for scripts ([#115](https://github.com/maximilianfeix/proxy-scraper/issues/115))

### Fixed
- A result file line with a Unicode digit ("1.2.3.4:²") crashed the parser – found by the new fuzz tests
- Proxy server: when a free proxy hung up in the middle of an upload, uvloop reported it as RuntimeError, the handler died and the failure was never counted against that proxy
- The Amazon check counted nearly every proxy as "through": Amazon now answers bots with a 2 KB stub page and status 200, even without a proxy. It checks the search page now and only counts a 200 with a real page behind it

### Tests
- Property-based tests for the parser with hypothesis: random bytes and list-like text never crash it, every key it returns is well-formed with a public address, and every public address is found in every list format ([#120](https://github.com/maximilianfeix/proxy-scraper/issues/120))

## [1.9.0] – 2026-09-27

### Added
- A weekly report, rebuilt with every run: how long free proxies last (most are gone within the hour), how many get through to Google, Reddit and Amazon – datacenter exits vs the rest – and where they are. As a page (report/), as markdown to post and as a short post that fits on X
- A page per proxy on the website: protocol, country, provider, uptime, which big sites let it through, a timeline of every check this week and copy-ready curl/Python commands to test it from your own machine. Linked from the table; only proxies on the list for half the week or more are in the sitemap, the rest are noindex
- README recipes for httpx, aiohttp and Scrapy (a rotating downloader middleware), each tried against the live list ([#118](https://github.com/maximilianfeix/proxy-scraper/issues/118), [#119](https://github.com/maximilianfeix/proxy-scraper/issues/119))
- Lists that don't say their protocol (plain ip:port in a generic proxies.txt) are tried as HTTP and as SOCKS5. On 20,000 such entries 44 worked as HTTP and 32 as SOCKS5 – those 32 were missed before ([#44](https://github.com/maximilianfeix/proxy-scraper/issues/44))

## [1.8.1] – 2026-09-27

### Fixed
- Generic lists (proxies.txt, all.txt …) with plain ip:port lines gave nothing and were paused as "unreachable" – they're read as HTTP now: 64 more sources deliver
- The download cache stores parsed proxies, so a parser improvement never reached unchanged lists. It now reloads every list once when the parser changes
- Source statistics written by another version are read instead of thrown away

### Added
- Website: the filters live in the address, so a view can be shared or bookmarked (plus a Share view button), and the pages per protocol and country open the list with their filter. Countries show their names ([#116](https://github.com/maximilianfeix/proxy-scraper/issues/116), [#117](https://github.com/maximilianfeix/proxy-scraper/issues/117))
- Discovery also reads the source lists of other proxy scrapers (sources.json, config.toml, urls.txt …) and keeps every URL in them that really holds 20+ proxies – about 250 more sources in a test run

## [1.8.0] – 2026-09-27

### Added
- Which big sites let each proxy through: after every run the HTTPS-capable proxies are tried on Google, Reddit and Amazon, each with its own rule (Google's captcha redirect counts as blocked). `sites` in proxies.json, `works-with/google.txt` and friends, a filter and column on the website, `works_on` in the MCP server and in `live_proxies()` ([#139](https://github.com/maximilianfeix/proxy-scraper/issues/139))
- `live_proxies()` in the Python API: the hourly checked list with the same filters as `find_proxies`, plus `min_uptime` – no scan needed ([#141](https://github.com/maximilianfeix/proxy-scraper/issues/141))
- Website: the theme switch grows the new theme out of the button, and the rows settle in when filters or sorting change instead of snapping. The scroll-driven route only does its layout work while it's on screen (about 60 % fewer layout reads while scrolling)
- Uptime per proxy on the live list: `uptime_24h`, `uptime_7d` and `first_seen` in proxies.json, an Uptime column on the website, `stable.txt` with everything listed in 90 %+ of this week's runs, and `min_uptime_percent` in the MCP server's get_proxies ([#138](https://github.com/maximilianfeix/proxy-scraper/issues/138))
- Charts in the README, redrawn with every run: working proxies over the last week by protocol, and the top countries. Plus GitHub Pages and jsDelivr links for every list file ([#142](https://github.com/maximilianfeix/proxy-scraper/issues/142))

## [1.7.1] – 2026-09-26

### Changed
- On PyPI now: `pipx install proxy-scraper-cli` (the plain name is blocked by an older, similar package – the commands stay `proxy-scraper` and `proxy-scraper-mcp`). `proxy-scraper --mcp` starts the MCP server too, and each release publishes it to the official MCP registry as `io.github.maximilianfeix/proxy-scraper` ([#42](https://github.com/maximilianfeix/proxy-scraper/issues/42))

## [1.7.0] – 2026-09-26

### Added
- MCP server for AI agents: `proxy-scraper-mcp` gives Claude Code, Claude Desktop, Cursor, VS Code, Codex and other MCP clients three tools – `get_proxies` (instant, from the hourly list, with the website's filters), `check_proxies` (fresh from your own network, with progress) and `fetch_url` (a page through a verified proxy as readable text, with failover, verified TLS for HTTPS, and local or private targets refused, also after redirects). `pip install "proxy-scraper[mcp]"` on Python 3.10+, setup for every client in the README ([#129](https://github.com/maximilianfeix/proxy-scraper/issues/129))

### Changed
- More sources: 28 new curated lists (Databay, the full proxifly list, vakhov, freeproxy.world and 19 GitHub lists that bring proxies nobody else had), and the GitHub search now runs daily, keeps what earlier runs found for three weeks, asks 23 queries instead of 9 and skips VPN-config repos ([#127](https://github.com/maximilianfeix/proxy-scraper/issues/127))
- Website: calmer trend chart (no spikes from runs minutes apart), less empty space, the footer links no longer run into the wordmark ([#131](https://github.com/maximilianfeix/proxy-scraper/issues/131))

### Fixed
- Proxy server: stopping it ends open tunnels instead of leaving them running (on Python 3.12+ a tunnel whose target never hangs up could keep Ctrl+C waiting), its own 502 and 400 are marked with an `X-Proxy-Scraper` header, and IPv6 targets work in absolute `http://` URLs
- Result files are placed when they're written, not when the program starts, so embedding proxy-scraper (like the MCP server does) can move them

## [1.6.0] – 2026-09-26

### Changed
- The live list is refreshed every hour instead of every 6 hours (a full run takes about 3 minutes). "Stable" still means 24 hours, `stats.json` now says `run_hours`, and the Discord bot still posts a summary at most every 6 hours (`PROXYBOT_POST_EVERY_HOURS`) ([#111](https://github.com/maximilianfeix/proxy-scraper/issues/111))
- Terminal: the final report shows the datacenter and blocklist shares next to HTTPS and anonymity, the wizard panel no longer repeats the name from the banner ([#91](https://github.com/maximilianfeix/proxy-scraper/issues/91))
- Website: WCAG 2.1 AA fixes (3:1 borders on controls, a pause button for the ticker, keyboard-accessible copying, sort buttons, table caption, skip link), link previews with an image, a touch icon, a section documenting the JSON files, and `/` to jump to the search ([#79](https://github.com/maximilianfeix/proxy-scraper/issues/79), [#80](https://github.com/maximilianfeix/proxy-scraper/issues/80))
- README redesigned to match the website: brand-colored badges and buttons, the five checks as a diagram, headings without emojis, a section for the Discord bot. The German README is gone, everything is English now ([#73](https://github.com/maximilianfeix/proxy-scraper/issues/73))
- Everything is in English now: terminal UI, CLI help, messages, code comments, tests and the generated proxy-list README. Numbers use English formatting (12,345 and 1.5%), and the terminal uses the lime brand color ([#72](https://github.com/maximilianfeix/proxy-scraper/issues/72))
- New website design and a real logo: the route through a proxy that forms a check mark. Big live numbers, a ticker with the fastest proxies, a scroll scene through the five checks, smooth scrolling, light and dark theme. The README banner uses the same look

### Added
- The website gets a static page per protocol (HTTP, SOCKS4, SOCKS5, HTTPS, elite) and per country, each with a `proxies.txt`, plus a `sitemap.xml` and dataset markup, so search engines find the list ([#121](https://github.com/maximilianfeix/proxy-scraper/issues/121))
- `--serve-refill HOURS`: the running proxy server checks fresh proxies in the background and adds the hits, so it no longer runs dry over days; `compose.yaml` uses it ([#109](https://github.com/maximilianfeix/proxy-scraper/issues/109))
- `-o -` prints the hits to stdout for pipes, the interface goes to stderr then ([#105](https://github.com/maximilianfeix/proxy-scraper/issues/105))
- `--serve-password` / `PROXY_SCRAPER_SERVE_PASSWORD`: the rotating server can require a password (HTTP, SOCKS5 and the status page), plus a `compose.yaml` that runs it in Docker ([#103](https://github.com/maximilianfeix/proxy-scraper/issues/103))
- Discord bot: `/proxies not_blocklisted:true` and the blocklist status in `/proxy`
- Release workflow can publish to PyPI with trusted publishing (no token in the repo), switched on with the repository variable `PYPI_PUBLISH`; Dependabot also watches the Python dependencies of the tool and the bot ([#42](https://github.com/maximilianfeix/proxy-scraper/issues/42))
- Blocklist info for every exit IP (SpamCop, one cached DNS lookup each) and `--no-blocklisted` to skip listed ones; about 29 % of working proxies are listed. Skipped with a note when the DNS resolver is refused. The website shows a BL tag and a "Not blocklisted" filter ([#66](https://github.com/maximilianfeix/proxy-scraper/issues/66))
- Discord bot (`bot/`): posts every run of the live list with the fastest proxies and the full lists as files, sets up its own read-only channels, and answers `/proxies`, `/proxy`, `/stats` and `/about`. A GitHub Action tests it and deploys it to a server as a hardened systemd service ([#74](https://github.com/maximilianfeix/proxy-scraper/issues/74), [#75](https://github.com/maximilianfeix/proxy-scraper/issues/75))

### Fixed
- Learning: proxies still being confirmed when `--want` is reached or Ctrl+C is pressed no longer count as dead, a list with none of the requested `--types` is no longer paused as unreachable, and outdated lists get another look every 3 days instead of never ([#83](https://github.com/maximilianfeix/proxy-scraper/issues/83))
- Proxy server: a target that is down no longer gets working proxies disabled (a refused tunnel only counts against a proxy if another proxy reaches the same target), a client leaving in the middle of a request body no longer prints a traceback, a slow TLS ClientHello keeps the part that already arrived, and IPv6 targets work over HTTP CONNECT and SOCKS5 ([#82](https://github.com/maximilianfeix/proxy-scraper/issues/82))
- A `--recheck` file that is missing now gives a message instead of a traceback, `http://` lines without an address no longer crash the parser, your own IP must match a whole address before a proxy counts as transparent, `gh` is only asked for a token when discovery can run, and the last German strings are gone ([#84](https://github.com/maximilianfeix/proxy-scraper/issues/84))

## [1.5.0] – 2026-09-25

### Added
- Tab completion for bash, zsh and fish: `proxy-scraper --completion zsh` prints the script. It is generated from the options, so it never goes stale ([#62](https://github.com/maximilianfeix/proxy-scraper/issues/62))
- Python API: `find_proxies()` and `check_proxies()` (plus async versions) run the same checks as the CLI and return the hits; `docs/ARCHITECTURE.md` explains how the modules fit together ([#50](https://github.com/maximilianfeix/proxy-scraper/issues/50))
- Proxy server: `/__proxy-scraper/metrics` in the Prometheus text format (requests, bytes, pool size and median latency per type), no extra dependency ([#67](https://github.com/maximilianfeix/proxy-scraper/issues/67))
- Stable proxies: the live list remembers how many runs in a row each proxy worked (`streaks.json`); the website shows it and filters for 24 h+ ([#61](https://github.com/maximilianfeix/proxy-scraper/issues/61))
- `--recheck live` starts from the public live list and checks it again from your own network: about 30 s instead of a full scan ([#60](https://github.com/maximilianfeix/proxy-scraper/issues/60))
- `--serve-host` to make the proxy server reachable from outside a Docker container, with a warning when it's not a loopback address ([#43](https://github.com/maximilianfeix/proxy-scraper/issues/43))
- Provider (ASN and name) for every proxy from the offline DB-IP ASN database, a guess whether it's a datacenter, and `--no-datacenter` to skip those. About 45 % of working proxies exit from datacenters ([#49](https://github.com/maximilianfeix/proxy-scraper/issues/49))
- Proxy server: `--rotate weighted|random|round-robin|fastest`, `--sticky SEC`, per-request choice through the username (`country-de`, `type-socks5`, `session-NAME`), SOCKS5 on the same port, a JSON status endpoint, and disabled proxies are re-checked every 5 minutes ([#48](https://github.com/maximilianfeix/proxy-scraper/issues/48))
- The live list has a website on GitHub Pages: search, filters, copy and download, country and latency charts and a trend over the last runs ([#47](https://github.com/maximilianfeix/proxy-scraper/issues/47))
- Proxies that tamper with content are filtered out: a static HTML page has to arrive unchanged. On 270 working proxies, 54 injected something, mostly a `<script src="http://…">` ([#46](https://github.com/maximilianfeix/proxy-scraper/issues/46))

## [1.4.0] – 2026-09-25

### Added
- Proxies with credentials (`socks5://user:pass@…`) are checked with their login instead of losing it: Basic auth for HTTP, RFC 1929 for SOCKS5, user ID for SOCKS4. They're never published to the public live list ([#7](https://github.com/maximilianfeix/proxy-scraper/issues/7))
- Fallback check targets: before a run all targets are probed (Cloudflare-hosted ones are rejected), during the run a watchdog switches when the current one goes down. Proxies checked during the outage are re-checked and don't count for the source statistics ([#8](https://github.com/maximilianfeix/proxy-scraper/issues/8))
- Docker image (`ghcr.io/maximilianfeix/proxy-scraper`, amd64 + arm64) with volumes for learned state and results; CI runs a real scan inside the container ([#5](https://github.com/maximilianfeix/proxy-scraper/issues/5))
- `--export proxychains,clash,curl` (or `all`) writes ready-made configs next to the results ([#4](https://github.com/maximilianfeix/proxy-scraper/issues/4))

### Changed
- Countries are looked up offline in the DB-IP Lite database (downloaded once a month, 2 µs per lookup) instead of waiting for ip-api.com, which is now only a fallback. On 943 IPs it agreed with ip-api on 96 % ([#2](https://github.com/maximilianfeix/proxy-scraper/issues/2))
- Unchanged lists are no longer downloaded again: ETag / Last-Modified with a local cache of the parsed proxies. A second run right after the first went from 161 MB in 22 s to 0 MB in 9 s. `--no-cache` forces a full download ([#9](https://github.com/maximilianfeix/proxy-scraper/issues/9))
- The rotating server skips upstream proxies that answer `407 Proxy Authentication Required`, even after `100 Continue`

## [1.3.0] – 2026-09-24

### Added
- Installable as a real command-line tool: `pipx install git+https://github.com/maximilianfeix/proxy-scraper.git` gives you `proxy-scraper`, plus `python -m proxyscraper` and `--version` ([#26](https://github.com/maximilianfeix/proxy-scraper/issues/26))
- Release workflow: every version tag is tested, built, smoke-tested and published with a wheel
- “Next steps” panel after a run with ready-to-copy commands ([#27](https://github.com/maximilianfeix/proxy-scraper/issues/27))
- English README (German version in `README.de.md`), contributing guide, security policy, issue forms ([#28](https://github.com/maximilianfeix/proxy-scraper/issues/28))

### Changed
- Calmer, more consistent terminal UI: one colour palette, framed sections, clearer panel titles ([#27](https://github.com/maximilianfeix/proxy-scraper/issues/27))
- When installed, learned state lives in the user data folder (`PROXY_SCRAPER_HOME` overrides it) and results go to `./results`
- `sources.json` moved into the package

## [1.2.0] – 2026-09-24

### Added
- Setup wizard: started without arguments, the tool asks what you are looking for – presets or step by step – and shows the matching command line ([#12](https://github.com/maximilianfeix/proxy-scraper/issues/12))
- Rotating proxy server with `--serve` and silent failover; HTTPS only through proxies with verified TLS ([#15](https://github.com/maximilianfeix/proxy-scraper/issues/15))
- `--target` keeps only proxies that really reach a given site ([#14](https://github.com/maximilianfeix/proxy-scraper/issues/14))
- Live proxy list published every 6 hours by GitHub Actions on the `proxy-list` branch ([#18](https://github.com/maximilianfeix/proxy-scraper/issues/18))
- Windows support ([#6](https://github.com/maximilianfeix/proxy-scraper/issues/6))

### Changed
- Honeypots are filtered out: every hit must fetch a second, independent page with the same exit IP ([#20](https://github.com/maximilianfeix/proxy-scraper/issues/20))
- Filters make checking faster – `--max-latency` becomes the timeout, unneeded HTTPS tests are skipped ([#13](https://github.com/maximilianfeix/proxy-scraper/issues/13))
- Cleaner architecture: `RunOptions`, a run in phases, the UI as its own package ([#11](https://github.com/maximilianfeix/proxy-scraper/issues/11))
- Lint (ruff), CodeQL and Dependabot

## [1.0.0] – 2026-09-24

First public version.

- 700+ sources: curated list, meta sources and automatic GitHub discovery
- Hand-written HTTP, SOCKS4 and SOCKS5 handshakes, 2000+ checks in parallel
- HTTPS test, anonymity level and country for every hit
- Learns the hit rate per source and checks known proxies first
- Filters (`--country`, `--https-only`, `--anonymity`, `--max-latency`) and `--want N`
- Results as txt, json and csv under `results/`
- Live dashboard in the terminal

[Unreleased]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.34.0...HEAD
[1.34.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.33.1...v1.34.0
[1.33.1]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.33.0...v1.33.1
[1.33.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.32.1...v1.33.0
[1.32.1]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.32.0...v1.32.1
[1.32.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.31.0...v1.32.0
[1.31.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.30.0...v1.31.0
[1.30.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.29.0...v1.30.0
[1.29.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.28.0...v1.29.0
[1.28.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.27.0...v1.28.0
[1.27.1]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.27.0...29b8fb6
[1.27.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.26.0...v1.27.0
[1.26.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.25.0...v1.26.0
[1.25.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.24.0...v1.25.0
[1.24.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.23.0...v1.24.0
[1.23.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.22.0...v1.23.0
[1.22.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.21.0...v1.22.0
[1.21.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.20.0...v1.21.0
[1.20.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.19.0...v1.20.0
[1.19.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.18.0...v1.19.0
[1.18.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.17.0...v1.18.0
[1.17.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.16.0...v1.17.0
[1.16.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.15.0...v1.16.0
[1.15.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.14.0...v1.15.0
[1.14.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.13.0...v1.14.0
[1.13.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.12.0...v1.13.0
[1.12.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.11.0...v1.12.0
[1.11.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.10.0...v1.11.0
[1.10.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.9.0...v1.10.0
[1.9.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.8.1...v1.9.0
[1.8.1]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.8.0...v1.8.1
[1.8.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.7.1...v1.8.0
[1.7.1]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.7.0...v1.7.1
[1.7.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.6.0...v1.7.0
[1.6.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.5.0...v1.6.0
[1.5.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.4.0...v1.5.0
[1.4.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.3.0...v1.4.0
[1.3.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.2.0...v1.3.0
[1.2.0]: https://github.com/maximilianfeix/proxy-scraper/compare/v1.0.0...v1.2.0
[1.0.0]: https://github.com/maximilianfeix/proxy-scraper/releases/tag/v1.0.0
