# pmtiles-swarm changelog ## master ### ✨ Features and improvements - _...Add new stuff here..._ ### 🐞 Bug fixes - _...Add new stuff here..._ ## 0.106.2 ### ✨ Features and improvements - _...Add new stuff here..._ ### 🐞 Bug fixes - **A restart forgot when a watched location was last polled.** The schedule lived in memory, so every start began with none β€” `isDue` saw no last run and said yes, and every source polled at once whatever interval it asked for. Usually that costs one wasted HEAD, because a poll finds the URL it already holds and stops. It is not usually for a date template: the URL moves with the date, so the poll does not recognise what it has β€” it finds the next build and fetches it. A node restarted daily downloaded a weekly planet daily. Now written to `/source-schedule.json`, the same write-then-rename the export schedule uses, and read before the first poll. A record that will not parse is discarded rather than kept: it would compare as `NaN`, which reads as *never* due, and would stop a source being polled at all. The first start after upgrading still polls once, having nothing to read yet. To skip even that, write the file before starting β€” `{"": ""}`, keyed by the name in the config. ## 0.106.1 ### ✨ Features and improvements - _...Add new stuff here..._ ### 🐞 Bug fixes - **A touchpad without separate buttons spun the map instead of panning it.** MapLibre binds rotate and pitch to the right mouse button, and a clickpad reports a press with a second finger resting on it as exactly that β€” so pressing down to drag rotated rather than panned. Every map this node draws now drops button 2 before MapLibre sees it, in the capture phase on the container. Rotation stays on ctrl-drag and on shift with the arrow keys; `dragRotate: false` would have fixed the accident and taken the deliberate gesture with it, since one handler covers both. - **Contour tiles wrote one protobuf field out of order.** A feature carried `type` (tag 3) ahead of `tags` (tag 2). Protobuf allows any order and most readers do not care, but maplibre-native's rejects it β€” the same defect maplibre-contour#412 fixed in that project's own writer, one message further up. Our layer already ascended; the feature did not. The bytes change, and `bakeRevision` does not hash the encoder, so a contour export resumed across this upgrade would splice both orders into one archive. Both decode the same, but start a stopped contour export fresh rather than resuming it. ## 0.106.0 ### ✨ Features and improvements - **Contours are drawn past where the ground stops.** A raster endpoint should not do this; a contour one should. The interval gets finer as the zoom does, so z16 over a z12 DEM draws 1 m lines that z12's own level never drew β€” traced from the z12 tiles split down to the square and scaled back up before tracing, which is what `maplibre-contour` does when it overzooms. The endpoint used to stop at the source's maxzoom and under-advertise even that: a stack over a z8 archive served contours to z14 while its `tiles.json` claimed z8. Now the document says how far the intervals actually reach, and the archive and category endpoints overzoom too β€” before this they answered 404 one level past the DEM. It stops where the intervals stop changing, since past the deepest level a table names the lines would be the same ones smoothed, and is capped at what splitting can carry. The deep end is also cheaper than the middle: nine squares out of a few parent tiles rather than nine merges. ### 🐞 Bug fixes - _...Add new stuff here..._ ## 0.105.0 ### ✨ Features and improvements - **The built-in contour intervals are contour-generator's now.** They were maplibre-contour's README example with the gaps filled in, which drew far coarser lines and nothing at all below z9. The table is adopted whole β€” `1:[600,3000]` through `16:[1,5]` β€” so a pyramid baked by contour-generator and a stack traced live draw the same lines at the same heights, the same reason the merge maths is kept in step with the offline merger. Two consequences. Contours are drawn from **z1** rather than z9, and a contour tile is nine merged terrain tiles β€” at low zoom each covering most of the world β€” so the merged-heights cache is doing more work than before. And the lines are finer everywhere: z14 draws every 5 m where it drew every 50 m. A recipe naming only the deep end, `{"12": [10, 50]}`, declines the shallow zooms and the cost of tracing them. A bare `?interval=` follows the same zooms, so it too now draws from z1 β€” which is what contour-generator's `--increment` does. ### 🐞 Bug fixes - _...Add new stuff here..._ ## 0.104.0 ### ✨ Features and improvements - **An MBTiles archive is served under its own name.** `/archives//archive.mbtiles`, and `/latest//archive.mbtiles`, rather than `.pmtiles` for everything. It is the same endpoint and the same bytes β€” a web seed hands over the file on disk, which is all the piece hashes care about β€” but a URL is read by people too, and `.pmtiles` on a SQLite file says it is something it is not. Asking for the wrong extension answers 404 and names the right one. An MBTiles archive still answers to `archive.pmtiles`, unadvertised. That is the URL every torrent published before this distinction existed carries in its `url-list`, and nothing rewrites a web seed once it is in one β€” withdrawing it would strand those swarms. ### 🐞 Bug fixes - **A hand-added web seed was checked for its scheme and nothing else.** The self-published seed got the full reachability judgement; one added through the console or `POST /api/torrents//webseeds` did not, so a loopback address went into the .torrent unremarked β€” and nothing rewrites a web seed once it is in one. Both paths now run the same check, before anything is written, so a bad URL in a batch does not leave half of it published. The check also notices a URL ending in a slash. BEP 19 reads that as a directory and appends the torrent name, which works only where the host serves the file that way β€” and webtorrent does not implement the rule for single-file torrents, so it requests the directory and fails every piece. Allowed, since it is legitimate against a host arranged for it, but reported as a warning. ## 0.103.0 ### ✨ Features and improvements - **Elevation at a coordinate.** `GET /stacks//elevation?lon=&lat=&zoom=` answers the height under a point, and `POST` of `{"points": [...]}` answers up to a thousand of them in the order asked. The same two under `/archives//` and `/latest//`, since an archive of terrain is already heights. The shape follows tileserver-gl's endpoint deliberately, so a client written against that keeps working β€” see [NOTICE.md](NOTICE.md). What differs is that `"elevation": null` is an answer: the height is read from the `Float32Array` a stack merges, where a hole is `NaN`, so "there is no data here" and "this is at sea level" stay different answers. Anything reading encoded pixels has to invent a height for ground nothing covers, because every triple of bytes in a terrain tile is one. Points are grouped by the tile they fall in and each tile is read once, so a track down one valley costs the few merges its tiles are worth rather than one per point. - **Contour endpoints describe themselves.** `GET /stacks//contours/tiles.json`, and the same under `/archives/` and `/latest/`. The zoom range is the thresholds' rather than the source's: a stack with ground from z0 draws its first line at z9, and a client told otherwise spends eight zooms of requests on tiles that can only answer 404. - **Hillshade and contours can be switched off in the preview.** Two boxes in the header, for the layers that page actually drew. Not only cosmetic for the contours: MapLibre stops fetching from a source once no visible layer draws from it, so unticking the box really does stop the nine merges each contour tile costs. The contour link is gone from the console and the catalogue with it β€” a hidden layer never asks its source for a tile, so there was nothing left for a separate page to save. `?contours=1` still works, now as the box's starting state, and the box keeps it current so the lines survive the raw/terrain switch. - **Terrain has its own document.** [docs/terrain.md](docs/terrain.md) covers the encodings, contours and elevation across all three ways terrain is served. They were under tile stacks, which stopped being true once the same endpoints answered for an archive and a category. ### 🐞 Bug fixes - **An archive or category elevation reading ignored the archive's zoom range.** The summary spells it `minZoom`/`maxZoom` and everything downstream reads the TileJSON spelling, so passing the summary straight through left both undefined and every reading clamped to a hardcoded 0-14. An archive stopping at z8 was read at z14, where it has no tile, and answered `null` for ground it covers. Caught before release; the stack path was never affected, since it takes its range from the resolved recipe. ## 0.102.0 ### ✨ Features and improvements - **Hillshade and contours can be switched off in the preview.** Two boxes in the header, shown only for the layers that page actually drew. Not only a cosmetic switch for the contours: MapLibre stops fetching from a source once no visible layer draws from it, so unticking the box really does stop the nine tile merges each contour tile costs. 3D stays where it was, on MapLibre's own terrain button. ### 🐞 Bug fixes - **The contour preview asked for a tile at the literal coordinates `{z}`.** Its source URL was built with `new URL()`, which percent-encodes braces β€” so the template went out as `contours/%7Bz%7D/%7Bx%7D/%7By%7D.pbf`, MapLibre never substituted it, and every request came back `400: z, x and y must be integers`. Joined to the origin instead, since the path was already absolute and there was nothing to resolve. ## 0.101.0 ### ✨ Features and improvements - **Contours, traced from what a stack merges.** `GET /stacks//contours/{z}/{x}/{y}.pbf` returns a gzipped vector tile of contour lines. The interval comes from the recipe's `contours.thresholds` or from `?interval=` on the request β€” contours are a view of a stack rather than a property of one, and the same terrain is wanted at 10 m on a walking map and 100 m on an atlas. Drawn here rather than from an archive for one reason. A tool reading archives has to answer "what is the elevation?" for ground no archive covers, and an encoded terrain tile cannot say "nothing" β€” every triple of bytes is a height. So it invents one, and a constant beside real terrain is a cliff, which a tracer renders as lines packed tight along the seam. A stack has `NaN` and hands its holes over unfilled, so a line stops at the edge of the data instead. A level may name several intervals. `[100, 500]` draws every hundred metres and marks every fifth, and each feature carries `level` β€” how many intervals its height divides by β€” so a style draws the major lines thicker and labels only those, from one layer. That is `maplibre-contour`'s convention, so a style written for its tiles works against these. Worth knowing what it costs: a contour tile is traced from its own tile **plus its eight neighbours**, because a line crossing an edge has to be traced from the ground on both sides or it will not meet the line next door. Roughly nine merged terrain tiles each. See [terrain.md](docs/terrain.md) β€” "Contours". A **Contours** button sits beside Preview and Terrain on any stack that draws as terrain, and the preview draws the lines over the hillshade rather than instead of it β€” a contour on its own says nothing about which side of it is uphill. Major lines draw themselves thicker from `level`, with no second layer and no filter. There are no labels: text needs a glyph server, and a preview that fetched fonts from elsewhere would be a preview of that as much as of the archive. `ele` is on every feature for a style that has fonts to spend. **An export can write them too.** The export dialog's *What to make* list has its second entry, and `POST /api/stacks//bake` takes `kind: "contours"` with an optional `thresholds`. The archive is written as gzipped MVT with the right header, and the run narrows itself to the zooms its thresholds actually draw at β€” without that it walks every tile the sources hold at z0–z8 to trace nothing, which on a planet is hours spent producing silence. The interval is part of the export's revision, so a checkpoint cannot be resumed across a change to it: a 20 m run continued into a 100 m one would splice two sets of lines into an archive nothing downstream could tell apart. **Merged heights are cached in memory**, under `stacks.heightsCacheBytes` (64 MiB, zero to turn it off). A different cache from `stacks.cacheBytes` and for a different shape of reuse: that one holds encoded tiles so a second request for one tile is free, this holds the numbers a tile was merged from, because several *different* tiles are built out of the same neighbours. An NΓ—N contour run needs (N+2)Β² merges and asks for 9NΒ² β€” measured at 4Γ— fewer source reads and twice the speed on a 4Γ—4 block, approaching 9Γ— on a large one. A feathered source benefits more mildly, since four sibling tiles share the parents its ramp is measured against. Each contour export gets its own, which goes when the job does. A bake deliberately does not touch the disk cache β€” a planet export would evict the serving node's entire cache with tiles nobody will ask for again β€” and that is exactly why it can have one of its own. **Contours from an archive or a category too**, at `/archives//contours/{z}/{x}/{y}.pbf` and `/latest//contours/{z}/{x}/{y}.pbf`. An archive is already terrain β€” its pixels are a packed height and it states the packing in its own metadata β€” so these decode the tile and do nothing else: no recipe to resolve, no sources to merge, no masks or clips, because there is no recipe saying to. Nine reads a tile against the stack path's nine merges. They also do not climb to a parent where the archive has no tile at that zoom, which the stack path would: for contours that is the wrong favour, since a line traced from an upscaled parent is the parent's line drawn twice as thick rather than detail the zoom has. An infohash is immutable and cached as such; a category revalidates, and is tagged by the build it resolved to so a rebuild lands as a new tag rather than the same one with different lines behind it. The **Contours** button now appears anywhere terrain does β€” a stack's row, an archive's panel, and the public catalogue β€” rather than on stacks alone. The preview rewrites its own path into the contour endpoint, which is the same rewrite for all three. - **An export can write part of a stack.** A zoom range, an area, or both, from the export dialog or as `minzoom` / `maxzoom` / `bounds` on `POST /api/stacks//bake`. Absent still means all of it. An export reads every tile its sources hold, which for a planet is hours and a file nobody wanted all of. The two narrow differently. PMTiles orders tile ids by zoom and then along a Hilbert curve, so a zoom range is a contiguous run of ids and the scan **ends** at the deepest zoom asked for rather than filtering past it β€” which matters, because not enumerating unwanted zooms is why the export iterates coverage in the first place. A box is not contiguous, so it is a test per tile; it runs before the merge, so what it saves is the merge. A tile is written when it **overlaps** the box, not when it sits inside it, so an archive reaches its own edge instead of stopping up to a tile short. The dialog can also fill the box in from a `z/x/y` tile, which is how a planet is usually split: regions that tile evenly, never overlap, and have a name to agree on. Three things move with the selection, each a silent fault otherwise. The **revision**, so a checkpoint taken under one selection is not resumed under another β€” the stream of ids is different and resuming would skip whatever the new selection adds below the mark. Both **names**, since two exports of one recipe over different ground are two archives and a date does not tell them apart. And the header's **bounds**, which the writer defaults to the whole world: an archive claiming a planet and holding one country is one a client keeps asking for tiles that were never written. The zooms are still read off the tiles actually written, which is more honest than the request. ### 🐞 Bug fixes ## 0.100.0 ### ✨ Features and improvements - **A nested stack takes `maskColors`, and fades into what is under it.** Two things a stack could not do that every other kind of source could, both for the same reason: it is evaluated rather than stored, so it has no bytes. That is a fact about storage, not about meaning, and it was showing up in the recipe as a source with a different set of options. A colour is now decoded into the height it names, under the encoding the inner stack packs its own output in, and masked as a height β€” through `decodeHeights` rather than arithmetic written a second time. `maskColors: ["#0186a0"]` and `maskValues: [0]` say the same thing to a mapbox-encoded stack, so a source keeps its mask when it is swapped between an archive and a stack. `encoding`, `baseVal`, `interval` and the custom factors stay refused: those describe how to unpack channels into a number, and the number arrived already made. Feathering a nested source now works rather than silently doing nothing. A ramp is measured in pixels and the pixels that say how far a hole reaches are partly in the next tile β€” which for an archive means reading its parent and for a stack means evaluating it again, which was never implemented. Validation also stopped asking a nested source for a mask before it would accept a feather: its holes are already an edge, and the mask being demanded would have made a second one. The representation is unchanged. A nested stack is still merged as heights, because a hole is `NaN` and no encoding has one β€” encoding it would turn every hole into a sentinel the recipe above had to mask back out, which is the problem nesting avoids by construction. ### 🐞 Bug fixes ## 0.99.2 ### ✨ Features and improvements ### 🐞 Bug fixes - **The stack editor named a pinned archive by its infohash.** The list of saved stacks was fixed in 0.98.3, but the editor has its own row and kept showing `33394361a935…` β€” where the menu that offered the archive a moment earlier had shown its filename. The editor works on the recipe, and a pinned source in a recipe is an infohash and nothing else, so the row had nothing else to print. It remembers the names the source menu already fetched now, and the infohash is on hover. ## 0.99.1 ### ✨ Features and improvements ### 🐞 Bug fixes - **The sidecar stops segfaulting, which is what left a library half loaded after a restart.** `pmtiles-torrent` 0.11.1. The traceback 0.99.0 made possible named the same frame every time: the alert pump, inside `session.wait_for_alert()`. That call hands back a borrowed pointer to an alert which the next `pop_alerts()` β€” the following statement β€” invalidates, and nothing ever read the return value; it was only a way to sleep. The pump waits on its own stop event now. This is the cause behind the `not loaded` archives, not another symptom of them. Each crash took whatever restore had handed over before it, because the replacement sidecar holds none of it, so the archives restored first were the ones that came back missing. The node stops that leaving a mark either way β€” it hands them back β€” but they should now not be dropped in the first place. Sidecar shutdown is quicker too: the pump answers the stop at once, where the old wait had to be sat through to the end of its timeout. ## 0.99.0 ### ✨ Features and improvements - **A sidecar crash now names the frame it happened in.** `pmtiles-torrent` 0.11.0 arms `faulthandler` in the sidecar, so a segfault writes a Python traceback to stderr on its way out and this node forwards it into the log line by line, beside the `sidecar killed by SIGSEGV` that used to be the whole story. Nothing to configure; it costs nothing until the process faults. Worth having because that signal is otherwise the only evidence. On the node this came from it appeared several times per start for eleven days, and each crash takes the archives handed over before it β€” which is what leaves a library reading `not loaded` after a restart. ### 🐞 Bug fixes - **An archive the engine took and then did not keep is handed back, rather than only reported.** The seeding check has been able to spot this for a while and its own message admitted the rest: "nothing will start it before the next restart". It was right, which is why the cure was always another restart. The usual cause is a sidecar that dies partway through a restore β€” the replacement holds nothing, so the archives handed over before it died are absent while the ones after it are fine, and `add` resolving is no evidence that anything was kept. Those are now handed back once and re-checked, and the log says whether it took. Once, not in a loop: an engine that refuses twice will not be talked round by a third try. And only when the engine is holding *some* of the library β€” one holding none of it, or one that could not be listed at all, is not suffering a per-archive fault, and re-adding everything on the strength of that answer is how a node spends its start hashing what it already had. - **A pinned archive in a stack was listed by its infohash.** Forty characters of hex, where the picker that offered it had shown a filename. The row leads with the archive's name now and shows the infohash as what it resolves to, which for a pinned source is exactly what it means: this build and no later one. ## 0.98.3 ### ✨ Features and improvements ### 🐞 Bug fixes - **A stack could not be pointed at a completed MBTiles archive.** The source picker offered only PMTiles, so an MBTiles the node was perfectly able to serve tiles from was simply absent from the menu. It offers a complete one now, by the same rule the tile route enforces β€” and still hides one that is arriving, since a stack naming that would answer 503 for every tile until the download finished. - **A planet dump was offered as a stack source.** The same filter read an archive with no recorded kind as PMTiles, which is not a harmless default: a mirrored `.osm.pbf` appeared in the menu, and naming it would have produced a recipe that failed at its first tile. The kind now falls back to the filename before anything is assumed, which is how `prewarm.js` resolves it after the identical mistake sent every mirrored dump round the header-read backoff for ever. Such an archive is also labelled `unknown Β· not servable` in the list, where it previously said nothing at all. ## 0.98.2 ### ✨ Features and improvements ### 🐞 Bug fixes - **The console called a complete MBTiles archive "not servable".** It is, and has been since the tile store learned to open one: PMTiles always, MBTiles from a complete local copy. The console never followed. It tested `kind === 'pmtiles'` in two places, so an MBTiles that had finished hashing was labelled `mbtiles Β· not servable` in the list and had its TileJSON, preview and tile endpoint hidden in the panel β€” all of which the node was answering perfectly well. One rule now, matching what the tile route actually enforces, and an archive still arriving reads as `servable once complete` rather than as one that never will be. Warming stays PMTiles-only: it fetches the pieces a region's tiles live in, and an MBTiles is read whole or not at all, so there is never a region of one to warm. ## 0.98.1 ### ✨ Features and improvements - **The two archive readers no longer import each other.** Summarising an MBTiles archive reached into `pmtiles-probe.js` for `summarize`, which put one format's reader inside the other's module and dragged the `pmtiles` package in behind it. The half neither format owns β€” the summary shape, its version, the metadata readers and the tile-type table β€” is now `archive-summary.js`, which imports nothing and opens nothing: it takes a header and a metadata document and returns a summary. `pmtiles-probe.js` keeps the PMTiles half and `mbtiles.js` the SQLite half, and neither reaches the other. Nothing about what the node serves changes; this is where the code lives. ### 🐞 Bug fixes - **An MBTiles archive holding MLT tiles was summarised as an unknown format.** The table mapping a format name onto a tile type had been written twice, in opposite directions, and the copy beside the MBTiles reader was written by hand and had no entry for `mlt` β€” so an archive declaring it came back as `unknown`, which is what the catalog then reported and what a stack over it read. There is one table now, with the reverse lookup derived from it, so the two halves cannot drift apart again. ## 0.98.0 ### ✨ Features and improvements - **A finished MBTiles archive works as a stack layer, and now says what it holds.** It always read: the tile store opens one from a complete local copy, and a stack asks the store for tiles like anything else. What it could not do was describe itself. Only PMTiles was ever probed for a summary β€” right when only PMTiles could be served, and never revisited once MBTiles became servable β€” so those entries carried none, and a summary is what `stackCoverage` reads. A stack naming one advertised the fallbacks in its TileJSON, z0–z14 over the whole world, however narrow the archive actually was. Nothing refused it and nothing warned: the recipe was valid, the tiles were right, and only the coverage was a fiction. Both formats that can be served are read for a summary now, through the same summariser β€” `probeMbtiles` off the adapter's `getHeader` and `getMetadata`, which is what that adapter exists for. Most of the MBTiles spec is optional, so where an archive states nothing the reader derives what it can: the zoom range comes from the tiles table when `minzoom` and `maxzoom` are absent. The metadata reader also carries `encoding` through, with the four factors a custom packing is unreadable without. An elevation stack decodes the pixels rather than passing them through, so an archive that lost its encoding was not a tile that failed but a tile of wrong heights. Archives already in the catalog are summarised by the head warmer on its next pass, once complete β€” an MBTiles has no head to pull out of a swarm, so there is nothing to be due until the file is whole, and then the read is local and instant. Unchanged: `/latest/` still offers no tile endpoint for an MBTiles, because a category is a promise to every node and this one is only keepable on a node holding the whole file. See [tile-stacks.md](docs/tile-stacks.md) β€” "An MBTiles archive as a source". ### 🐞 Bug fixes ## 0.97.0 ### ✨ Features and improvements - **A stack can be told to forget what it has merged, and forgets it by itself when its recipe changes.** Editing a stack never served a stale tile β€” the cache key covers the recipe's revision, so the old tiles simply stop being asked for β€” but they stayed on the disk, spending the budget until eviction happened to reach them. Under a stack nobody is actively panning, that is a long time, and a node whose stacks are edited often could end up holding mostly tiles no request can reach. Merged tiles now carry the stack they belong to in their filename, after the digest so the sharding still spreads and in the name so it survives a restart. `StackStore` compares revisions on every reload and says which recipes are no longer what they were; the node clears those. It covers every way a recipe moves β€” the console's Save, `PUT /api/stacks/`, an import, a delete, a stack feed, and an operator editing `stacks.json` by hand β€” and announces nothing when a rewrite leaves the recipes the same, so a restart does not read as an edit and throw away the last run's work. Alongside it, a **Clear cache** button on each stack in the console, showing what it would free, and `DELETE /api/stacks//cache` behind it. That is for what a revision cannot see: an archive rewritten under an address that did not change, a cutline redrawn, a codec upgraded, or simply wanting the disk back now. `/api/stacks` reports `cache: {entries, bytes}` per stack. Both take the stacks **nesting** the cleared one with them, however many levels up. An outer stack's tiles were merged from the inner one's, so clearing one level and not the other leaves the older answer being served from above. Tiles written by an earlier version have no stack in their name; they are evicted as before and `DELETE /api/storage/merged-tiles` still empties everything. See [tile-stacks.md](docs/tile-stacks.md) β€” "Clearing what a stack has merged". ### 🐞 Bug fixes - **A directory the configuration named but nobody had made stopped the unit dead.** `status=226/NAMESPACE`, six milliseconds of CPU, and nothing in the journal β€” because the program never ran: with `ProtectSystem=strict`, systemd builds the mount namespace first, and a `ReadWritePaths=` entry that does not exist makes that fail. Every path is now prefixed with `-`, systemd's "ignore this if it does not exist", so the node starts and a write that has nowhere to go fails where it is attempted, naming the path. A missing directory should be a message, not a silence. `init --systemd` also lists the directories the configuration names that are not there yet, with the `install -d` line for each. See [running-as-a-service.md](docs/running-as-a-service.md) β€” "status=226/NAMESPACE". ## 0.96.0 ### ✨ Features and improvements - **The swarm-read limit is memory now, not a count.** `tiles.maxOpenSwarmArchives: 16` becomes `tiles.swarmCacheBytes: 1 GiB`, because a count was the wrong unit for it. What is expensive about a cache-mode reader is its piece cache, which is **RAM** β€” a map of whole pieces in the node's own heap β€” and how big it is depends on the torrent: `max(64 MiB, 8 Γ— pieceLength)`. Sixteen readers is a gigabyte against the 4 MiB pieces this project creates and two against the 16 MiB pieces a planet torrent usually has, which is not a limit anybody chose. Each reader is asked what its cache costs rather than assumed, so the count now follows from the memory: halve `pieceCacheBytes` and twice as many archives stay open for the same gigabyte. The last reader is never closed, whatever the budget says β€” the archive just opened is the one being read. `maxOpenSwarmArchives` remains for a node that wants a hard count as well; unset by default. Complete archives are unaffected: they hold a file descriptor and no piece cache, and are bounded by `maxOpenArchives`. - **`init --systemd` regenerates the unit without touching the configuration.** The unit is derived from the configuration and from how many archives the library holds, and both move β€” a watched folder added later belongs in `ReadWritePaths`, and a grown library needs longer to write its resume data than the unit allows. Running it again used to be refused, which sent people to `--force`, which replaces `swarm.config.json` outright: tokens, stacks, feeds and all, to regenerate a file beside it. It now reads the configuration, writes only the unit, and says so β€” along with the `diff` to run before replacing what is installed, since `ReadWritePaths` is derived and a path added to the installed unit by hand will not be in the new one. ### 🐞 Bug fixes - **The unit's `ReadWritePaths` left out where an export writes.** `stackExports[].savePath` and `.publishDir`, and a watched folder's `publishDir`, were not among the directories derived from the configuration β€” so a nightly bake to a directory named nowhere else ran for an hour and was refused the write at the end, with that directory's permissions perfect. Which is the exact failure the derived list exists to prevent. Re-run `init --systemd` to pick them up. ## 0.95.0 ### ✨ Features and improvements - **The defaults now assume a library of hundreds, not a handful.** `tiles.maxOpenArchives` goes from 16 to 128 and `tiles.directoryCacheEntries` from 200 to 2000. Sixteen was set for the most expensive kind of open archive and then applied to every kind, which is wrong for the node this is increasingly used to build: a stack assembled from a provider's file index names several hundred sources, and a bake walks every one of them. At sixteen such a run spent most of its time reopening archives it had just closed, and each reopen re-reads a header and a directory. So the limit is now three limits, because the handles cost different things. A complete archive is a file descriptor and the unit allows 65535 of them. A **cache-mode** archive carries a piece cache sized from the torrent's piece length β€” at 16 MiB pieces a hundred of them is gigabytes β€” so `tiles.maxOpenSwarmArchives` keeps the old ceiling of 16 and is counted separately. A **URL or bucket** archive holds an HTTP reader and nothing else, and costs a network round trip to reopen, so `tiles.maxOpenRemoteArchives` is 64. A node serving only cache-mode archives behaves as before. - **The unit's stop timeout is derived from the library it was written for.** Stopping writes resume data for every archive and the node allows two seconds apiece, so a fixed five minutes covered a library of 145 and no more β€” silently, because outgrowing it produces no error, just a library that comes back at 0% and re-hashes for hours. `init --systemd` now counts the catalog and sizes `TimeoutStopSec` to fit, and a node whose library has outgrown a default unit says so at startup. ### 🐞 Bug fixes - **A sidecar left running by a previous start is now killed rather than fought.** This is the restart that has to be done two or three times before it takes. The sidecar exits when its pipe closes, which covers an orderly stop β€” but not one in the middle of hashing, since libtorrent does not hand control back until the check finishes, and not a node killed outright. What is left holds the listen port and the resume directory, the next start fails against it, and since a failed start leaves its own sidecar they accumulate. Stopping now insists: the shutdown request goes first so resume data is still saved, and a sidecar that has not gone within a few seconds is killed rather than left behind. And a start reaps what a previous run left before spawning anything β€” by pid, but only where `/proc` confirms that pid is really a sidecar, since pids are reused and killing whatever inherited one would be worse than the problem. ## 0.94.0 ### 🐞 Bug fixes - **Saving settings rewrote a proxy list nobody had touched.** A trusted-proxy list may be stored as a string β€” `"loopback, 10.0.0.0/8"` is what the documentation shows β€” and the box rendered that as one line while reading it back as an array. The two never compared equal, so the field was sent on every Save whether or not anybody had looked at it, and before the previous release that rewrite was the thing that stopped the node from starting. Opening the settings page and pressing Save was enough to do it. The box now shows one entry per line whichever way the config wrote them, and records what a save will read back rather than what the config holds β€” so an untouched field is untouched. ## 0.93.0 ### 🐞 Bug fixes - **A trusted-proxy list typed with commas stopped the node from starting.** The settings field split what was typed on newlines only, so one line reading `172.16.1.2, 172.16.1.3` was saved as an array holding both addresses in one string. Express splits a comma list when it is handed a bare string and never inside an array, so proxy-addr was given `172.16.1.2, 172.16.1.3` as a single address and threw -- while the app was being built, before the listener binds. The node would not start, could not be reached, and could not be corrected from the console that had written the value; on the node that found this it was 155 restarts. Three things were wrong and all three are fixed. The field now splits on commas as well as newlines. Every shape the setting can be written in -- a string, an array, commas, spaces, newlines -- is flattened to what Express wants. And an entry that is not an address is ignored and logged rather than thrown: this setting is not worth a node that will not boot, and trusting nobody is the safe end of being wrong about it. - **A failed restore took the whole node down, console included.** Handing the library back to the engine at startup already tolerates a failure per archive; the call coming apart as a whole was unguarded, and it happens before the listener binds β€” so under `Restart=always` the result is a crash loop with no console to look at and no way to see why. It is now reported and the node starts anyway, where every archive shows as **not loaded** until it is fixed. - **Nothing tested that the node starts at all.** Every other test builds the pieces `src/index.js` wires together and never runs the wiring, so an import cycle or a step that throws before the listener binds was a failure only a real start could find. There is now a boot test that runs the entry point the way the service does and asks it for a page. ## 0.92.0 ### 🐞 Bug fixes - **A stack's TileJSON published the addresses its sources are read from.** A URL source is named by its URL and an S3 source by its bucket and key, and the document naming them is served to anybody who can load the map β€” while the archives at the other end are read with credentials nobody else has. The sources are gone from it entirely, which was the right answer anyway: they are the ingredients of one endpoint, not a list for a client to join, and listing them invited fetching those instead of the tiles. What is left is a count and a `revision` fingerprint covering the recipe and what every source resolved to β€” the one thing the list was good for, in twenty bytes rather than tens of kilobytes. The document for a 459-source stack is now 336 bytes. - **Signing in threw away the view you asked for.** A link to `#stacks` on a guarded node asked for a password and then showed the archives, with the address still reading `#stacks` β€” which is what made it look broken rather than like a redirect. Both ways into the console now land where the address says. - **An archive the engine has no record of was drawn as 0%.** That is a different fact from "none of it is here", and it is the more alarming one to get wrong: a library the engine failed to take back after a restart read as a library that had lost its data. The row now says **not loaded**, and says where the reason is logged and that nothing on disk has been touched. ## 0.91.0 ### ✨ Features and improvements - **A source at a URL can be added by hand, not only imported.** **Add source β†’ an address you type…** in the stack editor, writing the same source the importer does. The card asks for the address and for the zoom range the archive holds β€” a catalog archive states its range in its own header and this has no header anybody has read, so leaving it empty means every tile asks it. Anything published as a plain HTTPS download works, an S3 object or presigned URL included, and so does `s3://bucket/key.pmtiles` for a private bucket β€” see below. - **A stack's source list folds up past five sources.** An imported list is several hundred rows, which buried every other stack on the page under one of them. The fold says how many there are and names the base; a stack of a base and a layer or two stays open, since folding that hides nothing worth a click. - **A stack source may be an object in a private S3 bucket.** `s3://bucket/terrain.pmtiles`, read with a signed request per byte range. Only for a bucket that is not public β€” a public object or a presigned URL is an ordinary HTTPS address and always worked. `endpoint` is what makes it S3-compatible rather than S3: MinIO, Ceph, Garage, R2, Wasabi and B2 all answer the same protocol at their own address, and path-style addressing is the default because it is what they speak. Credentials go under **Settings β†’ Feeds β†’ S3 buckets**, per bucket or once for an account; with none configured the standard `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION` and `AWS_S3_ENDPOINT` variables are read, so a machine already set up to reach a bucket needs nothing typed in. Signed here rather than by an SDK: SigV4 for a GET is a hash, four HMACs and a string, all of which `node:crypto` has. An AWS client would be a large dependency for one signature, and an optional one would make reading a bucket work on some installs and not on others for no reason the operator could see β€” so it is neither optional nor a dependency. The signer is checked against AWS's own published test vectors rather than against itself. ### 🐞 Bug fixes - **A stack of remote sources could be served but not exported.** The bake refused any stack whose sources resolved to no archive on this node, which is every stack read from URLs β€” refusing exactly the export worth having, since baking is how terrain that lives somewhere else becomes something this node holds and can seed. It also had no way to walk a remote archive's directories, which is how a bake finds which tiles exist at all. - **An archive whose deepest tiles repeat said it stopped a zoom short.** Identical tiles collapse into one directory entry covering a range of ids, so the last entry's own id can sit at a shallower zoom than the tiles it addresses β€” and the header was read off that id. A client asks for nothing past a header's `maxZoom`, so the deepest zoom of such an archive was unreachable. Found baking a stack over flat terrain, where every tile of the last zoom was the same tile. - **A stack used as a source of another stack was reported as unresolved.** It resolves to a recipe rather than to bytes, and the console's reader looked only for a catalog entry or an address β€” so a working nested stack showed "does not resolve", with no zoom range and no box, and put a "sources missing" badge on the stack above it. It now says what it stands for and how deep it reaches, worked out from the sources that stack would serve. - **A stack's advertised box ignored any source that stated none.** The union was taken over whichever sources had a box, so a stack of a global base plus regional patches β€” an imported list, exactly β€” advertised the patches as its extent and left the base out of it. A source with no box is not one covering nothing, so it is now every box or none, and none means the world. ## 0.90.0 ### ✨ Features and improvements - **A stack source may name a URL instead of a category, archive or stack.** `{ "url": "https://…" }`, read straight over HTTP with no torrent involved β€” for an archive published as a plain download, like Mapterhorn's terrain: a global base plus hundreds of regional patches, 11.8 TiB in total and never meant to be downloaded whole. `FetchSource`, from the `pmtiles` package this project already depends on, asks for byte ranges the way a swarm-backed source does; a tile costs the header once, the directory once, and the tile itself. Parent climbing works the same way it does for a category source β€” a shallow archive upscales for a deeper request through the same code, since reading one is now only a question of which store method answers. New per-source `minzoom` and `maxzoom` fields, checked before anything is opened, are what make hundreds of these practical: a tile request outside a source's stated box or zoom range skips it without a request leaving this node, which for a stack built from a provider's whole file list is most of them, on every tile. A URL source always decodes rather than passing through raw, having no infohash to answer that question with, and is not seeded, retired or rebuilt β€” none of the mechanisms built for an archive this node actually holds apply to one it does not. - **A provider's list of PMTiles URLs can be imported as sources.** **Stacks β†’ Import URL list…**, or `POST /api/stacks//import`. Mapterhorn's `download_urls.json` names 458 files with the box and zoom range of each; naming those by hand is not work anybody should do once, let alone again when the provider adds one. An index with an `items` list is read, and so is a plain list of addresses for a provider that publishes no index β€” which shape it is is worked out from the document, since somebody pasting an address has no reason to know. The global file becomes `sources[0]` and `required`: a stack is painted bottom-first, so the thing covering everywhere has to sit under everything patching it, and the index does not list it first. Every other entry keeps its box and zoom range, which is what lets a tile outside one skip it without a request leaving the node. The encoding is asked for rather than read β€” an index rarely states it, Mapterhorn's files are all terrarium and its JSON never says so, and a terrain source read with the wrong one is a cliff face. The list is fetched **by the node**, not the browser: the console is often on a different network, and what matters is that the machine which will read the archives can reach them. **Check** shows what an import would write before it writes anything. In the editor an imported batch is one row rather than several hundred cards, with **Re-import** and **Remove all**. A re-import replaces only what came from that same address β€” anything typed by hand is left alone β€” and puts the batch back **where it already was** rather than on the end: painting order is the whole meaning of a stack, and a batch that moved each time would quietly bury a local override, a day later, on a schedule, with nothing to say why the map had changed. - **A stack can follow a provider's file list on a timer.** A stack feed row with **Into stack** set reads a URL list rather than another node's recipes, and keeps the named stack level with it. Mapterhorn's index has grown through several versions, and a node that imported it once is a node serving whenever that was. The row says which of the two it is by whether **Into stack** is filled in β€” a feed of recipes names its own stacks and an index cannot, being a list of files with no opinion about what they are for, so the row supplies the stack and the encoding as well. It reconciles exactly as **Re-import** does: hand-written sources untouched, the batch back where it was, a withdrawn file dropped rather than left to 404 per tile. A poll where nothing changed writes nothing at all β€” not the same recipe again, which would move its revision and with it every tile cached against it. A stack that does not exist yet is created; one that does keeps everything it had. ### 🐞 Bug fixes - **A stack reported no encoding unless its recipe stated one outright.** Reading `output.encoding` alone reports null for the ordinary recipe β€” the one that re-encodes to nothing and writes whatever its base source is written in, which is every imported stack, since an import sets the encoding per source. Three places read it that way and all three were wrong about terrain: the console offered no terrain preview, the TileJSON told clients nothing so terrarium heights rendered through the mapbox formula, and a baked archive was written with no encoding metadata at all β€” an export that had to be corrected by hand afterwards to be readable. - **A stack's TileJSON listed every source.** For a stack imported from a provider's index that is 458 addresses and 55 KB, in a document every map load fetches, describing files a client cannot use and will never ask for β€” it reads tiles from the XYZ endpoint. It now names 25 with a count of the rest. The extension it advertises is the one the endpoint actually serves, too: a stack whose sources state no format has no coverage format to read, and the document said `.bin` for tiles answered as webp. - **Cancel asked you to fill the form in first.** A `