{
  "$schema": "hasna.station_template.v1",
  "name": "station",
  "version": "1.8.0",
  "description": "Hasna station contract v1. Every item traces to a measured 2026-07-28 failure on station01 (designs/station-contract-design.md §8). Layers: base + dgx-spark (121G physical Spark) + ec2 (32G Graviton cloud station). Owner ruling 2026-07-30: AWS stations carry NO tailscale and are not connected to the physical fleet — tailscale lives only in physical overlays, and SSM (the ec2 accessFloor) is the whole access path for cloud stations. 1.3.0 adds the machines-roster user service (roster reconciler) to the dgx-spark overlay. 1.4.0 adds the base-layer non-interactive PATH profile (station17 build 3: SSM automation could not resolve bun-installed CLIs). 1.5.0 adds the base-layer ~/.bashrc non-interactive block (station17 2026-07-30: `mosh station17 codewith` could not resolve bun CLIs — ssh/mosh remote commands are non-login non-interactive, profile.d never runs for them, and sshd-spawned bash sources ~/.bashrc instead of BASH_ENV, so the PATH must be spliced above the stock interactive guard). 1.6.0 adds the base-layer journald size cap (station17 build 2 died of disk exhaustion with journald unable to create its journal, yet the rebuilt boxes carried the accidental min(10%-of-filesystem, 4G) default and no drift item covered the axis). 1.7.0 bounds the apt archive cache with an APT::Periodic::AutocleanInterval drop-in (station17 accumulated 97,818,544 bytes of .deb across 90 files in 16 hours, because Download-Upgradeable-Packages \"1\" pairs with Unattended-Upgrade \"0\" and the AutocleanInterval default is never, while the drift check read clean 42/42). 1.8.0 makes the check itself able to fail: every bun global carries a minVersion FLOOR (12 of 42 items reported ok at any version, pinned by a test that asserted a 9.9.9 fixture was ok), and the ec2 overlay declares tailscale ABSENT as a checkable item (the 2026-07-30 ruling removed tailscale from the ec2 render but nothing asserted its absence — an ec2 render emitted no tailscale item at all, and station18 running a live tailscale read clean 42/42).",
  "base": {
    "files": [
      {
        "id": "sysctl-reclaim",
        "source": "files/base/etc/sysctl.d/99-zz-hasna-station.conf",
        "target": "/etc/sysctl.d/99-zz-hasna-station.conf",
        "mode": "0644",
        "kind": "sysctl",
        "lesson": "2026-07-28 kernel panic: vm.swappiness=1 in vendor 99-dgx-spark.conf caused direct-reclaim livelock; the fix had to be renamed 99-zz-* to win lexicographic ordering."
      },
      {
        "id": "tmpfiles-mglru",
        "source": "files/base/etc/tmpfiles.d/99-zz-hasna-station-mglru.conf",
        "target": "/etc/tmpfiles.d/99-zz-hasna-station-mglru.conf",
        "mode": "0644",
        "kind": "tmpfiles",
        "lesson": "2026-07-28: MGLRU min_ttl_ms=1000 was runtime-only and would have died on reboot; tmpfiles rule is the persistence."
      },
      {
        "id": "path-profile",
        "source": "files/base/etc/profile.d/99-zz-hasna-station-path.sh",
        "target": "/etc/profile.d/99-zz-hasna-station-path.sh",
        "mode": "0644",
        "kind": "plain",
        "lesson": "2026-07-30 station17 build 3: sudo -iu hasna bash -lc 'machines --version' exited 127 — Ubuntu's stock ~/.bashrc early-returns for non-interactive shells before bun's PATH export, so SSM-driven automation could not find any bun-installed CLI and command:aws-cli went red under a PATH lacking /usr/local/bin. A converged box that automation cannot drive is not a station; profile.d covers every login shell regardless of interactivity."
      },
      {
        "id": "bashrc-noninteractive-path",
        "source": "files/base/home/bashrc-block-noninteractive-path.sh",
        "target": "~/.bashrc",
        "mode": "0644",
        "kind": "bashrc-block",
        "lesson": "2026-07-30 station17: `mosh station17 codewith` (and every `ssh stationN <cmd>`) found no bun-installed CLI — those shells are NON-LOGIN and NON-INTERACTIVE, so the 1.4.0 profile.d fix never runs for them, and the sshd-spawned bash sources ~/.bashrc INSTEAD of $BASH_ENV (measured: with ~/.bashrc hidden and sshd SetEnv pointing BASH_ENV at the PATH profile, resolution still failed). Ubuntu's stock ~/.bashrc early-returns before its own PATH exports, so the block must be spliced ABOVE the interactive guard — the only hook those shells read."
      },
      {
        "id": "journald-cap",
        "source": "files/base/etc/systemd/journald.conf.d/99-zz-hasna-station.conf",
        "target": "/etc/systemd/journald.conf.d/99-zz-hasna-station.conf",
        "mode": "0644",
        "kind": "journald-dropin",
        "lesson": "2026-07-29 station17 build 2: the box died of disk exhaustion and journald logged 'Failed to create new system journal: No space left on device' — yet the rebuilt stations shipped with NO journal cap (stock journald.conf, empty conf.d), leaving the bound at the accidental min(10%-of-filesystem, 4G) default ('max 4.0G' measured in both boxes' journald startup lines) with no drift item covering the axis (found by the 2026-07-30 item-4 box audit). SystemMaxUse=2G is a chosen bound two orders of magnitude above measured usage (16M); SystemKeepFree=8G mirrors disk.minFreeGb so journald backs off before the drift-checked free-space floor is breached."
      },
      {
        "id": "apt-cache-autoclean",
        "source": "files/base/etc/apt/apt.conf.d/99-zz-hasna-station-apt-cache.conf",
        "target": "/etc/apt/apt.conf.d/99-zz-hasna-station-apt-cache.conf",
        "mode": "0644",
        "kind": "plain",
        "lesson": "2026-07-30 item-4 station17 box audit: the `linux-update-downloads` setup step pairs Download-Upgradeable-Packages \"1\" with Unattended-Upgrade \"0\" — download daily, install never — and never sets AutocleanInterval, whose APT default is \"0\" (never autoclean). /var/cache/apt/archives therefore grows monotonically with .deb files that by policy can never be installed: 97,818,544 bytes across 90 .deb after 16 hours on station17, while the drift check read clean 42/42 because no item covered the axis. Every Linux station inherits it; station21 would too. 7 days keeps a re-install window without unbounded growth."
      },
      {
        "id": "earlyoom-dropin",
        "source": "files/base/etc/systemd/system/earlyoom.service.d/50-hasna-station.conf",
        "target": "/etc/systemd/system/earlyoom.service.d/50-hasna-station.conf",
        "mode": "0644",
        "kind": "systemd-dropin",
        "lesson": "2026-07-28: the userspace memory guard computed swap depth and ignored it, printing healthy 3.5 minutes before the panic; earlyoom trips on the swap condition. Kill-drill proven on station02 2026-07-28."
      },
      {
        "id": "slice-tests",
        "source": "files/base/home/user-systemd/hasna-tests.slice",
        "target": "~/.config/systemd/user/hasna-tests.slice",
        "mode": "0644",
        "kind": "systemd-user-unit",
        "lesson": "Aggregate local tests require one active cgroup-v2 controller installed by normal station setup; package-derived limits live in its managed drop-in and earlyoom remains only the host backstop."
      }
    ],
    "packages": {
      "apt": ["earlyoom", "git", "curl", "unzip", "build-essential", "tmux", "sqlite3", "jq"],
      "bun": [
        { "name": "@hasna/machines", "minVersion": "0.2.38", "lesson": "2026-07-30: package:bun:* reported ok at ANY version, so 12 of 42 drift items were version-blind \u2014 a station carrying a CLI from before a fix read identical to one updated an hour ago, and a test asserted a 9.9.9 fixture was ok. Each floor is a version that is PUBLISHED: clean means 'not running an older CLI than the one we shipped with'. It is a FLOOR, never a pin \u2014 a newer version is ok, and the renders install latest. A floor ABOVE the newest published version would be the same defect mirrored: a check that cannot pass instead of one that cannot fail, failing every station on a version nobody can install. @hasna/machines is floored at 0.2.38, the package version prepared for this release, which is what 'the shipped template floors globals at the version it shipped with' means when the template ships inside that package." },
        { "name": "@hasna/todos", "minVersion": "0.13.3", "lesson": "Task state is fleet-shared: an old todos CLI on one station writes state every other station reads." },
        { "name": "@hasna/conversations", "minVersion": "0.5.13", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/mementos", "minVersion": "0.14.69", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/secrets", "minVersion": "0.2.9", "lesson": "Secret handling is the one CLI where running a stale build is a security question, not only a feature question." },
        { "name": "@hasna/monitor", "minVersion": "0.1.25", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/loops", "minVersion": "0.4.30", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/repos", "minVersion": "0.1.37", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/projects", "minVersion": "0.1.96", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/snapshots", "minVersion": "0.1.3", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/accounts", "minVersion": "0.2.27", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." },
        { "name": "@hasna/knowledge", "minVersion": "0.2.92", "lesson": "Version floor for the shipped CLI set (published 2026-07-31)." }
      ]
    },
    "services": [
      { "name": "earlyoom", "scope": "system", "expectEnabled": true, "expectActive": true },
      { "name": "hasna-tests.slice", "scope": "user", "expectEnabled": false, "expectActive": true }
    ],
    "workstationTestProfile": true,
    "sysctls": {
      "vm.swappiness": "60",
      "vm.min_free_kbytes": "1048576",
      "vm.watermark_scale_factor": "150",
      "kernel.panic": "10"
    },
    "runtimeValues": [
      {
        "path": "/sys/kernel/mm/lru_gen/min_ttl_ms",
        "value": "1000",
        "lesson": "2026-07-28: MGLRU anti-thrash knob must survive reboot; checked live, not just as a file."
      }
    ],
    "unitConventions": {
      "match": "hasna-*.service",
      "startLimitIntervalSec": 300,
      "startLimitBurst": 5,
      "onFailureUnit": "hasna-unit-failure-notify@%n.service",
      "requireAbsoluteExecStart": true,
      "lesson": "2026-07-28: bare ExecStart PATH bug caused 203/EXEC (snapshots-agent); 0/40 hasna units had StartLimit keys; a missing-binary unit looped ~290k restarts because the default 10s/5 window resets at >2s spacing."
    },
    "secretsBootstrap": {
      "envSecretName": "stations/prod/station-env",
      "optional": true
    },
    "swap": { "sizeGb": 0 }
  },
  "overlays": {
    "dgx-spark": {
      "files": [
        {
          "id": "slice-agents",
          "source": "files/dgx-spark/user-systemd/hasna-agents.slice",
          "target": "~/.config/systemd/user/hasna-agents.slice",
          "mode": "0644",
          "kind": "systemd-user-unit",
          "lesson": "2026-07-28: the slice existed but agents ran in the session scope, so MemoryHigh/Max were decoration; the reconciler launches into this slice."
        },
        {
          "id": "slice-hq",
          "source": "files/dgx-spark/user-systemd/hasna-hq.slice",
          "target": "~/.config/systemd/user/hasna-hq.slice",
          "mode": "0644",
          "kind": "systemd-user-unit",
          "lesson": "2026-07-28: hq/coordinator seats need MemoryLow protection when agent load spikes."
        },
        {
          "id": "machines-roster-service",
          "source": "files/dgx-spark/user-systemd/machines-roster.service",
          "target": "~/.config/systemd/user/machines-roster.service",
          "mode": "0644",
          "kind": "systemd-user-unit",
          "lesson": "2026-07-29: station01 had only the obsolete /opt/fixture unit and a stale Jul 8 heartbeat; this unit runs the packaged SQLite-leased reconciler."
        }
      ],
      "services": [
        { "name": "machines-roster.service", "scope": "user", "expectEnabled": true, "expectActive": true },
        { "name": "tailscaled", "scope": "system", "expectEnabled": true, "expectActive": true }
      ],
      "tailscale": {
        "join": true,
        "authKeySecretName": "stations/prod/tailscale/authkey",
        "hostnameFromStation": true,
        "ssh": true
      }
    },
    "ec2": {
      "files": [
        {
          "id": "slice-agents",
          "source": "files/ec2/user-systemd/hasna-agents.slice",
          "target": "~/.config/systemd/user/hasna-agents.slice",
          "mode": "0644",
          "kind": "systemd-user-unit",
          "lesson": "Scaled from dgx-spark 54G/60G-on-121G to 20G/24G-on-32G; a 32G station only works if agent memory is actually bounded."
        },
        {
          "id": "slice-hq",
          "source": "files/ec2/user-systemd/hasna-hq.slice",
          "target": "~/.config/systemd/user/hasna-hq.slice",
          "mode": "0644",
          "kind": "systemd-user-unit",
          "lesson": "hq seat protection on the 32G class."
        }
      ],
      "packages": {
        "apt": [],
        "bun": []
      },
      "absences": [
        {
          "id": "tailscale",
          "command": "tailscale",
          "service": "tailscaled",
          "paths": [
            "/usr/sbin/tailscaled",
            "/usr/bin/tailscale",
            "/var/lib/tailscale",
            "/etc/default/tailscaled",
            "/etc/apt/sources.list.d/tailscale.list"
          ],
          "lesson": "Owner ruling 2026-07-30: AWS stations do not run tailscale, and AWS is not connected to the physical fleet at all — SSM is the whole access path, not a floor beneath a mesh. The first implementation of that ruling removed tailscale from the ec2 render and stopped, which left the ruling CLAIMED and UNCHECKED: check.ts guarded its whole tailscale block on `effective.tailscale?.join`, so an ec2 render emitted NO tailscale item (measured 2026-07-30 22:02Z: tailscale_items=[] on station17) and station18 — running a live tailscale with BackendState=Running — still read clean 42/42. This is a declared ABSENCE, not a `tailscale:join` check: asking whether the tailnet is healthy on a box that must not be on one is noise, and noise is how real drift gets ignored. It asks only 'is it here', and any yes is a violation, because `setup --apply` does not uninstall. The narrow future exception the ruling allows — a single AWS box granted tailscale as an access path — is expressed by giving that box its own overlay, which then fails to load alongside this absence rather than silently contradicting it."
        }
      ],
      "accessFloor": {
        "service": "snap.amazon-ssm-agent.amazon-ssm-agent",
        "ensure": "( snap list amazon-ssm-agent >/dev/null 2>&1 || snap install amazon-ssm-agent --classic ) && systemctl enable --now snap.amazon-ssm-agent.amazon-ssm-agent.service",
        "lesson": "2026-07-29 station17: the tailscale join died at boot (runcmd: aws: not found) on a box with zero SG ingress, no keypair, and no authorized SSH keys — SSM, authorized purely by the instance profile stations-prod-ec2-profile with no boot-time secret, was the only way back in. Owner ruling: nothing that requires fetching a secret at boot may sit on the critical path to a machine's reachability; the floor must be explicit and drift-checked, never incidental."
      },
      "commands": [
        {
          "id": "aws-cli",
          "command": "aws",
          "install": "curl -fsSL https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip -o /tmp/awscliv2.zip && unzip -q -o /tmp/awscliv2.zip -d /tmp && /tmp/aws/install --update && rm -rf /tmp/aws /tmp/awscliv2.zip",
          "lesson": "2026-07-29 station17: the ec2 overlay declared apt package 'awscli', which has NO installation candidate on Ubuntu 24.04 — noble dropped the deb (apt-cache policy awscli -> Candidate: (none), while htop resolves from the same sources, so it is the package and not the apt config). Two measured consequences: cloud-init's package_update_upgrade_install module FAILED on every launch (cloud-init status: error), and the tailscale-join runcmd — which shells out to 'aws secretsmanager get-secret-value' — ran with no aws on PATH and left station17 off the tailnet. AWS CLI v2 ships only as the official installer, never as a distro package, so the requirement has to be expressed as a command, not an apt name."
        }
      ],
      "swap": { "sizeGb": 8 },
      "disk": {
        "rootMinGb": 64,
        "minFreeGb": 8,
        "lesson": "2026-07-29 station17 build 2 (i-0f522f0138a0411e1): launched with no BlockDeviceMappings, so the AMI-default 8G gp3 root volume shipped. Measured over SSM: / 6.8G total with 364K available (100%), /swapfile a PARTIAL 4,439,506,944-byte file — fallocate -l 8G allocated extents until ENOSPC and the old test -f guard then skipped it forever — swapon --show empty, journald 'Failed to create new system journal: No space left on device', cloud-final FAILED 43.8s into modules-final. The launch must request at least this root volume size explicitly; setup cannot converge an undersized volume, so the drift check reports it as a violation, not drift."
      }
    }
  }
}