# Browser runtime architecture

The current implementation is a JavaScript/POSIX compatibility runtime with
CPython compiled to WebAssembly. It is not a Linux kernel or an x86 emulator.
Keep this distinction in the public API and product claims.

## Existing execution paths

- JavaScript: the module engine and worker pods implement a subset of Node APIs.
- Python: CPython runs in a process worker and calls the shared virtual kernel.
- WASI: compiled wasm32-wasi programs use the existing WASI host.
- Browser preview: a service worker routes HTTP into the virtual network.
- Native Linux ELF executables and native Node addons: unsupported by these paths.

Browser execution needs suitable worker URLs and, for blocking worker calls,
SharedArrayBuffer and cross-origin isolation. Static hosting is still needed to
serve application and runtime assets. It is not a computation backend.
Python build-on-miss is different: its local builder requires Node/toolchains,
and its remote builder uses another machine. Prebuilt compatible wheels avoid
that backend. Not every PyPI package has such a wheel.

## Changes in this pass

The npm installer now overlaps up to six sibling package downloads while keeping
filesystem mutations ordered. Installers created through `forCwd` share their
metadata, archive and platform-detail caches within that installer family.
Transient metadata/archive failures are evicted so retries can recover.
An extracted package no longer hides an incomplete dependency tree, and optional
dependencies override duplicate required entries.

Python startup now cleans resources after worker creation or message-send
failures, and honors cancellation that arrives before or during worker creation.

Caches remain in memory; there is no persistent offline package store, lockfile
implementation, global hoisting, or measured real-project performance claim.
Browser bundling and the static import guard are checks of build compatibility,
not substitutes for interactive browser acceptance testing.

## Native Linux direction (not implemented)

Use a separately loaded browser machine emulator with its own Linux kernel and
disk image if running unchanged Linux packages is required. Keep it optional:
ordinary JavaScript/Python projects should not download an entire Linux image.
Expose the same high-level exec, filesystem transfer and preview operations, but
report the selected backend and its limitations explicitly. The emulator owns
its filesystem; transfer files explicitly instead of pretending it shares the
JavaScript runtime's synchronous in-memory volume.

This requires choosing the CPU architecture, emulator, kernel/root filesystem,
network bridge and image distribution terms. Native addon compatibility then
comes from running actual Node inside the guest Linux system. It brings larger
assets and different startup/performance tradeoffs, and does not promise every
Linux package will work. No such backend is included in this change.

MIT licensing of this project can avoid a runtime subscription. It does not
promise free CDN hosting, registry availability or third-party services forever.
Audit the separate distribution terms of any kernel, disk image and packages
before redistributing them.

## Extension implementation

The subsequent implementation adds ELF dispatch, external WASI command packs,
three ordered backend tiers, a translator output cache, and optional local Git
and embedded SQL adapters. See [Developer tool packs](developer-tool-packs.md)
for the working APIs and explicit limits. Translation and emulator engines
remain external and are not included.

## Verified behaviour, and the gaps that remain

A verification harness in the browser test project (`npm run verify`, driving
real Chrome over the published package) exercises the container boot, Node,
CPython, an npm install from the registry, both x86-64 ELF tiers, WASI command
packs, Git, PostgreSQL, an in-container HTTP server, and the frontend
automation subset. All of it passes in a real browser.

One gap is closed and one remains.

## Outbound HTTP for Python

A browser cannot open a raw TCP socket, so nothing inside the container can
speak TLS to a real server: outbound traffic has to leave at the HTTP layer,
through the page's `fetch`. The container therefore serves a loopback HTTP
endpoint, advertised to the guest as `SBX_HTTP_EGRESS`, which takes an
*absolute* URL in the request line the way a proxy does and performs the
transfer on the guest's behalf. `https://` targets are named in that request
line rather than tunnelled with CONNECT, precisely so the guest never begins a
TLS handshake it cannot finish; the real request leaves the page over HTTPS.

The startup hook points `urllib.request` and `http.client` at it, so the
standard library reaches the network without changes to a program. The same
outbound policy every other exit applies is applied here, and because a refusal
comes back as a real HTTP response it carries the reason -- a guest is told that
outbound access is disabled and where to enable it, rather than being handed an
errno it renders as "connection refused".

Libraries that manage their own TLS rather than going through `http.client` --
`urllib3`, and so `requests` -- still cannot egress. Their sockets would need to
carry a TLS handshake this transport cannot terminate. A page's `fetch` is also
subject to CORS, so a cross-origin target must send the headers that let the
page read the response; that is a browser rule, not a container policy.

The bundled startup hook supplies an HTTPX transport when the package is
installed, so `httpx.get()` and `httpx.AsyncClient` use the same egress endpoint
in a browser. This is an integration for HTTPX rather than a claim that every
third-party socket or TLS library is browser-compatible; a library that opens
its own raw socket still needs the native Node path or a general WISP/TCP relay.

## Python thread offloads

The bundled pthread interpreter supports standard `asyncio.to_thread`,
`run_in_executor`, and AnyIO's thread pool, including synchronous FastAPI routes.
These execute on real Python worker threads, preserving context propagation,
exceptions, and cancellation of the awaiting task.

Native socket calls release the GIL and send requests directly from each pthread
to the process supervisor over a private BroadcastChannel. Every outstanding
request owns a shared response buffer, so a waiting interpreter cannot prevent
another thread from completing network I/O. The supervisor still applies the
same descriptor ownership and outbound network policy.

Emscripten filesystem operations still proxy through the interpreter thread.
While other Python threads exist, the asyncio selector limits each wait to
10 ms and briefly yields, allowing those queued operations to run. This prevents
the old deadlock between a sleeping selector and a worker trying to wake it.
It adds polling overhead while the thread pool is alive.

Deploy the matching `python-worker.js` and the complete `python/` runtime directory
together. The worker detects the native bridge before enabling this behavior.
Older custom interpreter images retain the previous explicit `to_thread` error
and inline AnyIO fallback rather than silently entering a known deadlock.

Regression coverage includes context variables, exceptions, cancellation, file
I/O, simultaneous thread HTTP calls into an asyncio server, executor shutdown,
and FastAPI synchronous endpoints. The browser fixture also offloads a file read
and an outbound HTTPX request from a FastAPI streaming endpoint. This does not
certify arbitrary native extensions or indefinite blocking filesystem operations;
thread churn and long-running cleanup soak remain release gates. Browser egress
restrictions still apply independently of threading.
