# @citisen/dsh-github-accelerator

[English](README.md) | [中文](README.zh.md)

A [DeepSeek Harness](https://github.com/deepseek-ai) (`dsh`) bundle that keeps the
`web_fetch` tool working when GitHub is reached through a **hosts-file accelerator** —
SteamCommunity302, Watt Toolkit / Steam++, or anything else that maps domains to
`127.0.0.1` and serves them from a local reverse proxy.

Without it, every `web_fetch` of GitHub fails with:

```
Error: URL hostname "github.com" resolves to a non-public IP address
```

## Why it fails

`dsh`'s built-in local fetch provider (`http`) resolves the request hostname, rejects the
whole answer set if any address is not public unicast, and pins a validated address into
the connection. That is the right default for an anonymous fetcher — and it is exactly
what a hosts-file accelerator breaks, because `127.0.0.1 github.com` makes the host
resolve to a non-public address by construction.

There is a second wall right behind it. An accelerator that intercepts TLS signs an
on-the-fly certificate with a CA it installed into the **operating system** certificate
store. Browsers and `curl` trust it; Node does not read that store, so even with the
address check out of the way the request dies with
`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`.

## What this plugin does

It registers one extra fetch provider, id `http-accelerated`, that makes exactly one
decision — is the request hostname on the configured allowlist?

- **On the allowlist** (GitHub by default): the request is sent to whatever the system
  resolver returns, so the accelerator's loopback endpoint is used, and the operating
  system's trust anchors are added to the TLS handshake so the accelerator's
  interception CA is accepted.
- **Everything else**: handed to the built-in `http` provider unchanged. Public-address
  validation, the redirect policy, byte caps, timeouts and error codes stay exactly as
  shipped.

Ordinary fetching therefore behaves as before. A private destination that is *not* on
the allowlist (say `http://127.0.0.1:8080/`) is still refused by the built-in provider.

## Requirements

- DeepSeek Harness with the `web` service and the built-in `http` fetch provider
  (any stock profile). Tested on dsh `0.2.1-rc.2` (desktop, Electron 44 / Node 24.21).
- Node ≥ 24.9 in the `dsh` host for the operating-system trust store
  (`tls.getCACertificates`). On older runtimes the plugin falls back to Node's bundled
  roots, and an accelerator's own CA needs `insecureTLS: true`.

## Install

Through the Web UI: **Plugins → install**, enter `@citisen/dsh-github-accelerator`, then
enable it. Or from a terminal:

```sh
dsh plugin --profile desktop add @citisen/dsh-github-accelerator
```

The same package straight from GitHub:

```sh
dsh plugin --profile desktop add github:caparisen/dsh-github-accelerator
```

`dsh plugin` forwards to pnpm inside the profile and then reconciles
`dsh.profile.bundles`; because this package declares `dsh.bundle`, enabling it needs no
hand-edited YAML.

Use `--profile web` instead of `--profile desktop` if you run the browser profile from
the CLI. `dsh` hot-reloads the profile, but fully quit and reopen the application if the
provider does not take effect.

### From a local checkout

```sh
cd "$DSH_HOME/profiles/desktop"
pnpm add /absolute/path/to/dsh-github-accelerator
```

then add `"@citisen/dsh-github-accelerator"` to `dsh.profile.bundles` in that profile's
`package.json`. On Windows a link across drives is unreliable in pnpm: it can resolve to
a non-existent path, after which the bundle reconciliation silently skips the package.
Fix the link by hand (`rmdir node_modules\@citisen\dsh-github-accelerator`, then
`mklink /J` the real directory) and add the bundle entry yourself.

## Configure

The bundle ships a working default (GitHub's own hosts). To change it, override the row
in your profile's `cordis.patch.yml` — your layer wins over the bundle layer:

```yaml
- id: web-fetch-accelerated
  name: '@citisen/dsh-github-accelerator'
  config:
    hosts:
      - github.com
      - '*.github.com'
      - '*.githubusercontent.com'
      - huggingface.co
      - '*.huggingface.co'
```

| Key | Default | Meaning |
| --- | --- | --- |
| `hosts` | GitHub hosts | Hostnames allowed to resolve to non-public addresses. `*.example.com` matches `example.com` and every subdomain; `*` matches everything (not recommended) |
| `trustSystemCertificates` | `true` | Add the OS certificate store to the TLS trust anchors for allowlisted hosts. This is what accepts a local accelerator's CA |
| `insecureTLS` | `false` | Skip certificate verification for allowlisted hosts. Last resort only, for a CA that cannot be installed in the store |
| `timeoutMs` | `30000` | Deadline for one fetch, including the body read |
| `maxResponseBytes` | `5000000` | Body byte cap; a declared oversize body fails with `WEB_FETCH_TOO_LARGE` |
| `maxBodyChars` | `100000` | Decoded character cap; longer bodies are truncated and flagged |
| `maxRedirects` | `5` | Same-origin redirect hops |
| `userAgent` | dsh's default UA | Request header |
| `providerId` / `baseProviderId` | `http-accelerated` / `http` | Provider ids; change only if you know why |
| `selfTestUrl` | empty | When set, the plugin fetches this URL once at startup through `ctx.web.fetch` and logs the result — a one-line health check |

Most accelerators cover far more than GitHub (`huggingface.co`, `cdn.jsdelivr.net`,
`registry-1.docker.io`, … — see the `#S302` block in your hosts file). Add only hosts
whose resolution you are willing to trust, because allowlisting a hostname disables the
public-address check for it.

## Verify

Set `selfTestUrl` and read the host log at startup:

```
[web-fetch-accelerated] registered provider "http-accelerated" for 5 host pattern(s): github.com, *.github.com, ...
[web-fetch-accelerated] self-test https://github.com/ -> HTTP 200 html 57684 chars in 812 ms
```

A `web_fetch` of `https://github.com/` should then return page content, and
`https://api.github.com/rate_limit` should return JSON.

## Security notes

- Allowlisted hosts no longer get the "must be public" check. That is the point of the
  plugin, and it is the cost: if an allowlisted name is pointed at an internal service,
  `web_fetch` will reach it. Keep the list small.
- TLS for allowlisted hosts trusts everything the operating system trusts. That matches
  browser behaviour, and is broader than Node's bundled roots.
- Everything off the allowlist is still handled by the built-in provider, unchanged.
- Only the model-facing `web_fetch` path is affected. `git`, `curl` and package managers
  in the shell use their own configuration and were never blocked by this error.

## Troubleshooting

| Symptom | Cause and fix |
| --- | --- |
| Still `resolves to a non-public IP address` | The provider is not selected, or the host is not allowlisted. Check the startup log line; confirm `web`'s `fetchProvider` is `http-accelerated` and the bundle is enabled |
| `the TLS certificate for … is not trusted` | The accelerator's CA is missing from the OS store, or the runtime is older than Node 24.9. Reinstall the accelerator's certificate, or set `insecureTLS: true` |
| `web fetch failed: … ECONNREFUSED … may not be running` | The accelerator is not running, or no longer owns that host |
| `WEB_PROVIDER_CONFIGURED_MISSING` | The bundle is enabled but the package is not installed (or its files are gone). Reinstall it |
| `WEB_PROVIDER_UNAVAILABLE` / `not registered` | The built-in `http` provider was disabled. This plugin delegates to it and cannot stand alone |

## Uninstall

Disable or remove the bundle in the Plugins page, or:

```sh
dsh plugin --profile desktop remove @citisen/dsh-github-accelerator
```

then restart. `web`'s `fetchProvider` returns to `http` from `dsh-base`, and the original
error comes back with it.

## How it works

`cordis.patch.yml` pins the `web` service to `http-accelerated` and inserts one row. The
plugin's `apply` registers a `WebFetchProvider` that:

1. parses the URL and matches the hostname against `hosts`;
2. on a match, performs the request with `node:https`/`node:http` against the system
   resolver's answer, following same-origin redirects and applying the same body caps,
   charset decoding and `WEB_*` error codes as the built-in provider;
3. otherwise calls the provider registered under `http` and returns its result verbatim.

The provider is a plain ESM module with no `@deepseek-ai/*` imports, so it can be loaded
and tested outside `dsh`.

## Development

```sh
npm test              # local HTTP server: redirects, caps, charsets, delegation, errors
npm run test:network  # additionally fetches https://github.com/ and https://api.github.com/
```

`npm test` needs nothing but Node ≥ 24; `npm run test:network` needs an accelerator (or
any path to GitHub) on the machine.

## License

MIT