Skip to content

fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) - #234

Draft
scuttletoby wants to merge 19 commits into
mainfrom
fork/pc-agents-kit
Draft

scuttletoby wants to merge 19 commits into
mainfrom
fork/pc-agents-kit

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Part 2 of 2 of the pc-agents kit, stacked on part 1 (#232): pc-agents setup, which makes a fresh WSL2 distro on a Windows PC into the wall around T3's agents, the bootstrap that starts it, and the owner steps. Review part 1 first; this PR's base is part 1's branch (merged in after part 1's fix round 1).

What is in it

  • sbin/pc-agents setup --source DIR --sha SHA (as root):
    1. reads the machine file and refuses a distro that is not fresh before anything is written (another login user, with a hint when Ubuntu's first-launch setup made one; an agent with another uid or in another group; a file the kit would write that it did not write or that was edited since; a cron/at/ssh unit where the kit masks one; /, /opt, /opt/pc-agents, /etc, /etc/systemd, /etc/systemd/system, /etc/pc-agents or /etc/t3code not root's alone);
    2. adds agent (no sudo, its own group only, login shell /usr/sbin/nologin; an older kit's agent gets it with usermod); if nft is missing, apt-get update then apt-get install -y nftables;
    3. copies the kit's release into /opt/pc-agents/<sha>/ (root-owned) and switches the current link;
    4. writes every file of part 1 (each whole: new file, then one rename; the record holds old and new hash during the write), then loads the firewall at once, reloads systemd and masks WSL's binfmt handler, cron, at, ssh and the agent's user manager;
    5. installs the configured T3 version as root under /opt/pc-agents/t3 with upstream's scripts/install.sh (T3CODE_HOME, T3CODE_VERSION, T3CODE_RELEASE_BASE_URL, clean environment, 30-minute limit);
    6. runs pc-agents check; only when every wall holds does it enable T3 and the check timer. It clears the start limit (systemctl reset-failed) first. If anything changed (the restart marker is there) it restarts T3, then watches for 15 s that T3 stays up (active, the same main process, no automatic restart; on failure it points at journalctl -u t3code-agents.service), and only then removes the marker. On a failed wall it disables both and says what to do (the first run always ends here: wsl --terminate so wsl.conf applies, then run again).
  • pc-agents as-agent -- CMD (as root): runs CMD as agent through systemd-run (a transient system unit named pc-agents-as-agent-<pid>, which check --enforce stops with T3) with the same sandbox lines as T3's unit, after a passing check. CMD runs through /usr/bin/env --, so a bare name is found in the agent's PATH (~/.local/bin first), not root's. The owner steps use it for the Claude Code install and the Claude, Codex and gh logins, so no step opens a login shell or starts the agent's user manager. The distro's default user is root.
  • bootstrap.sh: fetches a commit that is on the fork's protected ci-release branch and runs the setup from it (the heavy kit's bootstrap, adapted; curl has connect and total time limits). The README fetches the bootstrap itself from ci-release, so the script that verifies the sha is a reviewed one.
  • README: the 10 owner steps (fresh distro and the first-launch note, machine file, setup, restart, setup again, Tailscale with a tag and --accept-dns=false, Claude and gh logins through as-agent, linking with Full access, a logon task that keeps the distro running), the Tailscale tag and ACL, the state answers, the limits.
  • fork-kit/common/machine.example.json: a pc-agents section (made-up values, peers included; a test keeps it equal to the README's example and valid).

Decisions in fix round 1

  • Restart marker (B2): /etc/pc-agents/restart-pending, root-owned, written before the first changing write and removed only after T3 restarted and is active. Any stopped run, failed install or T3 that did not start leaves it, so the next good run restarts T3 even with nothing left to write.
  • Firewall before the download (B3): the new table is loaded right after its file is written; a slow or failed T3 install never leaves the old table in place. Part 1's firewall wall then agrees with the file.
  • Owner steps through as-agent (B1): systemd-run --uid=agent with every --property= of the unit's sandbox (lib sandbox(), shared with the unit), not runuser: the logins run inside the same walls as T3. Part 1 masks user@<uid>.service and fails on linger and user units.

Runs again, stops half-way, meets old or empty state

Case What happens Test
Runs again, nothing changed no file touched (mode, owner, mtime, inode, content), no restart, T3 downloaded once test_second_run_changes_nothing
Stopped at any write T3 never enabled by the stopped run; the next run ends with exactly the uninterrupted state test_stopped_half_way_then_run_again (every write in turn)
Killed right after policy.json is written the old T3 still runs the old policy; check fails on it (the 2-minute check stops it); the marker makes the next run restart it, once test_killed_after_the_policy_is_written_still_restarts_t3, RestartPending.test_killed_after_the_policy_is_written_check_fails
Upgrade stopped at any write every stop that changed a file leaves check failing on restart until setup runs again; then check passes RestartPending.test_upgrade_killed_at_any_write_fails_check_until_run_again (every write in turn)
Killed during setup's own restart or its 15 s wait the restarted T3 started after the marker: check passes; the marker stays and the next run restarts T3 once more RestartPending.test_check_passes_on_its_own_restart_and_a_kill_while_watching_restarts_again
SIGTERM or SIGHUP (closed window) during the download setup ends (exit 143/129) and the download's whole process group is killed Signals.test_sigterm_or_sighup_kills_the_running_download_too
Killed at a write, then the machine file edited the record has the old and new hash, so the half-written file is not taken for a hand edit; the next run writes the newest values test_killed_at_a_write_then_the_machine_file_edited (before and after the write)
T3 install fails on an upgrade setup says so (exit 1); the new firewall is already loaded; check fails on restart while the old T3 runs, --enforce stops it and as-agent refuses; the next run installs and restarts T3 test_failed_t3_install_on_upgrade_still_restarts_t3_on_the_next_run, RestartPending.test_interrupted_upgrade_keeps_old_t3_and_check_fails, StaleFirewall.test_tightened_allow_with_failed_install
Loaded table differs from the file check fails and --enforce stops T3 StaleFirewall.test_a_loaded_table_that_is_not_the_files_fails
T3 does not start after the restart exit 1 naming journalctl -u t3code-agents.service; the marker stays test_t3_that_does_not_start_is_reported
T3 starts, then exits a moment later the 15 s watch reports it (exit 1, no "done"); the marker stays; the next run works test_t3_that_dies_a_moment_after_the_restart_is_reported
Rerun within the start limit (5 starts in 10 min) reset-failed before enable/restart, so T3 starts test_rerun_within_the_start_limit_still_restarts_t3
A step hangs (download, apt) time limit, the whole process group is killed, setup says so (part 1 run()) part 1 Run test
No nft, apt offline apt-get update then install; on failure the message names the command to run test_nftables_installed_after_a_package_list_update
Marker left behind by an old run the next good run restarts T3 and removes it the B2 tests
Marker from an older kit (no time in it) check fails on restart while T3 runs; the next setup run restarts T3 and removes it RestartPending.test_marker_of_an_older_kit_fails_until_setup_runs_again, Restart.test_marker_of_an_older_kit_fails (check.test.py)
T3 started before this version, no marker nothing stale is known: the wall passes Walls.test_all_hold
No ExecMainStartTimestampMonotonic, unreadable marker, systemctl show fails check fails on restart Restart.test_no_start_time_fails, test_unreadable_marker_fails, test_systemctl_show_fails
Marker of an earlier boot; T3 stopped passes (every T3 now started after it; nothing stale runs) Restart.test_marker_of_an_earlier_boot_passes, test_t3_started_before_the_marker_fails_and_enforce_stops_it
Agent made by an older kit with a login shell usermod --shell /usr/sbin/nologin, once test_agent_of_an_older_kit_gets_no_login_shell
Existing distro without the logon task runs while a WSL window is open, as before; add the task any time (README step 10), nothing in the distro changes README step 10, Limits
Old kit (T3 in the agent's home) the old unit file is in the record, so it is replaced; the old runtime in the home is left and no longer run README "Old releases"
Hand-made or first-launched distro refused before any write, with the reason (and the userdel -r hint) Refused.*, test_a_distro_with_its_own_user, test_folders_that_are_not_roots_alone
No or broken machine file refused naming the key, nothing written test_bad_config
A wall fails when as-agent runs nothing runs as the agent AsAgent.test_refused_when_a_wall_does_not_hold

Decisions in fix round 2

  • Restart wall (B1, both lenses' must-fix): the marker now holds this boot's id and the CLOCK_MONOTONIC time, and is rewritten before and after every change (and after a T3 install, and right before the restart). check gets a restart wall: it fails while the marker is there and T3's main process started before that time (systemctl show -p MainPID,ExecMainStartTimestampMonotonic); an unreadable marker, start time or systemctl show fails too. No T3 process (stopped, or in its ExecStartPre check) and a marker of an earlier boot pass. Setup's own first check skips the wall. Monotonic time, not the wall clock: WSL's clock jumps on resume and time sync, and the monotonic one is what systemd reports.
  • Settle wait (B4): SETTLE = 15 s, polled every second; the marker is removed only after it.
  • Logon task (B3): an owner step with the same task as the desk starter on main (fork-kit/desk/windows/start-desk.ps1, inline there, so not reusable as a file), conhost.exe --headless, at the owner's logon, no administrator.

What can be locked

Nothing had to be left unlocked: the managed policy locks any T3 setting, the fork's included (part 1's server test). Since part 1's round 1, T3 runs from root's copy, so the locks bind the agent; the limits that are WSL2's, not the kit's, are in the README's Limits.

General vs project

General: any Windows PC with WSL2 and any fork of this repo. The fork's OWNER/NAME, the commit, the distro name and every machine fact (T3 version and release URL, port, uid, peers, child ceiling and other locked settings, caps, nameservers, Tailscale tag) are the owner's input: the bootstrap's arguments and the machine file. Nothing project-specific is in the kit; the examples use made-up values.

Upstream fit

The T3 install is upstream's scripts/install.sh (run by root with T3CODE_HOME); the locks are upstream's managed policy; T3 runs with upstream's T3CODE_* variables. No upstream file changes. The unit is the kit's own root system unit rather than t3 service install's user unit, so the agent cannot override it.

Tests

python3 -I -B fork-kit/pc-agents/tests/setup.test.py: 35 tests on a fake distro (fake useradd/usermod, env as T3's installer, apt-get, systemctl that knows which ExecStart and policy a running unit has, its start time and restarts, systemd-run that looks commands up in the caller's PATH, nft; never the real system). check.test.py: 36 tests, with the restart wall's 8 cases. Round 2: the new tests fail on the round-1 code (setup 29, check 10), and each fix taken out alone fails its own test (9 of 9).

Size

Non-test code added over part 1: about 410 lines (sbin/pc-agents +250/-6, bootstrap.sh 51, lib +98/-33 for sandbox(), the marker path and the restart wall, the machine file example 13). Part 1 has 493; together over 400, hence the split.

Follow-up (in the README's Limits): no hash manifest of the kit's code yet, and the bootstrap duplicates the heavy kit's.

🤖 Generated with Claude Code

scuttletoby and others added 2 commits November 15, 2026 19:24
A WSL2 distro on a Windows PC as the wall around T3's agents: the files
the kit owns (wsl.conf, resolv.conf, managed policy, uid-matched nft
table, a root system unit with User=agent and a sandbox, a daily check
timer), rendered from the pc-agents section of the machine file, and
`pc-agents check [--enforce]`: the fleet's wsl2 walls (run, not copied)
plus the kit's own. The unit runs the check before every start.

A server test pins that the managed policy locks the fork's settings
(agentSlots, linkedEnvironmentsEnabled, automaticUpdates).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`pc-agents setup` makes a fresh WSL2 distro into the wall: refuses a
distro that is not fresh with nothing changed, adds the agent user,
installs the kit's release under /opt/pc-agents/<sha>, writes and
records every file atomically, masks interop/cron/at/ssh, installs the
configured T3 version for the agent, loads the firewall, and enables
T3 only after every wall holds. A second run changes nothing; a run
stopped at any write finishes on the next.

The bootstrap fetches a ci-release commit; the README has the owner
steps, the Tailscale tag and ACL, and the state answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Two lenses at 9d46a34: security lens234s NOT SAFE, correctness lens234c NOT SAFE (Opus 5.5 high, read-only). setup.test.py 16 OK, check.test.py 4 OK. The walls hold on a fresh install; the gaps are in updates, the owner steps and the check.

Must fix:

  1. Owner steps run agent-written files outside the sandbox (README steps 7-8, lib/pc_agents.py:148 default user agent). wsl -d <distro> -u agent (or plain wsl -d <distro>) starts a login shell and the agent's systemd user manager as agent with none of the unit's walls: the agent's ~/.bashrc or a user unit then gets AF_VSOCK, io_uring, namespaces, /run/WSL, WSLg. Fix: owner steps via wsl -u root -- runuser -u agent -- env -i HOME=/home/agent ... (no login, no rc files), mask user@<uid>.service, a wall on /var/lib/systemd/linger/agent, reconsider default=agent.
  2. A restart is lost after a stopped or failed update (sbin/pc-agents:101-129, proven). changed lives in one run only: unit and policy.json are written before the T3 install and the restart. Install fails or the run is killed before the restart -> the next run sees nothing changed, only enable --now, the old binary and the looser policy keep running, setup says done, the check passes. Fix: a root-owned restart-pending marker written before the first change, removed only after a good restart.
  3. An old firewall stays loaded and check passes (proven). firewall.nft is written ~:111 but loaded :124 after the install; the firewall wall (:175) only looks for skuid != uid and drop, never compares the loaded table with the file. Owner narrows allow, install fails -> LAN stays open, check --enforce exits 0. Fix: load right after writing; check compares nft list table with the file.

Should fix:
4. Any other distro shares the network namespace and can drop table inet pc_agents (e.g. a flush ruleset at boot); the daily check leaves up to 24 h open. Check every few minutes; say it in Limits.
5. README step 6: tailscale up without --accept-dns=false overwrites the kit's resolv.conf (fails closed, breaks setup); /run/tailscale/tailscaled.sock is reachable from the unit (InaccessiblePaths=-/run/tailscale).
6. k.run timeout=120 with no TimeoutExpired handling (lib/pc_agents.py:218-219, used at sbin:99, :113): a slow download or apt gives a traceback and leaves curl running.
7. apt-get install -y nftables without apt-get update on a fresh image; the error should name the fix.

Nits: setup says "done" while T3 crash-loops on a rejected lockedSettings (check is-active or point at journalctl); a killed run plus a machine.json edit is refused as "not written by this kit" (README "next run finishes the job" is not true then); step 2 -u root on a --no-launch distro may run first-launch user setup; the managed policy binds an honest T3 only (binary in the agent's home; say it in Limits); the bootstrap runs before its own ci-release check; owner of /opt/pc-agents and /etc/systemd/system not checked.

Both proof tests (lost restart, stale firewall) go into the fix round. A fix round follows; two lenses re-check.

scuttletoby and others added 2 commits November 16, 2026 07:10
- Agent code runs only in T3's unit: default distro user root, the agent's
  user manager masked, walls for the mask, linger and user units.
- T3's runtime is root's under /opt/pc-agents/t3; the unit runs it from
  there; a wall checks its owner, mode and recorded hash.
- A check that cannot run or crashes stops T3 (ExecStopPost on the walls
  service); a failed stop is reported.
- Second network wall in the unit (IPAddressDeny/IPAddressAllow with the
  new peers key); the firewall wall compares the loaded table with the
  file; check timer every 2 minutes.
- Units wall asks systemctl show; the file scan covers system.control,
  system.attached, generators, /run, /usr/lib and prefix drop-ins.
- Gateway wall fails when allow or peers cover the gateway.
- lockedSettings takes whole objects and record entries; blank
  organization refused; StartLimit for a policy T3 cannot apply.
- run() kills the whole process group on a timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merges fork/pc-agents-walls (fix round 1: masks, linger and user-unit walls, root-owned T3 runtime, loaded-table
firewall wall, ExecStopPost stop, systemd's own view of the units) and fixes the setup and owner steps:

- B1: owner steps never open a shell as agent; `pc-agents as-agent -- CMD` runs logins as the agent through
  systemd-run under the unit's own walls (lib sandbox()); default distro user is root.
- B2: root-owned /etc/pc-agents/restart-pending written before the first change, removed only after T3
  restarted and is active, so a stopped run or a failed install still restarts T3 next time.
- B3: the firewall is loaded right after its file is written, before T3's download.
- B5/B6: step 6 --accept-dns=false; 30-minute install and 15-minute apt timeouts; bootstrap curl timeouts.
- B7: apt-get update before installing nftables; the error names the fix.
- B8: is-active after the restart (points at journalctl); the record keeps old and new hashes during a write;
  setup refuses unsafe /opt/pc-agents, /etc/systemd/system and parents; first-launch user hint; README steps
  fetch the bootstrap from ci-release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1 (head 6f840d6; merges part 1's round 1 first, no force-push)

  • B1 owner steps never open a shell as agent: pc-agents as-agent -- CMD runs it through systemd-run --uid=agent with the unit's own sandbox (fork-kit/pc-agents/sbin/pc-agents:207, fork-kit/pc-agents/lib/pc_agents.py:123); README steps 7-8 use it (fork-kit/pc-agents/README.md:115). Default user root and the user-manager mask are part 1. Tests: AsAgent (tests/setup.test.py:266).
  • B2 restart marker /etc/pc-agents/restart-pending: written before the first change (sbin/pc-agents:37, :63), removed only after restart + is-active (sbin/pc-agents:193). Tests: test_failed_t3_install_on_upgrade_still_restarts_t3_on_the_next_run (tests/setup.test.py:156), test_killed_after_the_policy_is_written_still_restarts_t3 (:189); fake systemctl now keeps the ExecStart and policy a running unit has.
  • B3 firewall loaded right after its file is written, before T3's download (sbin/pc-agents:178). Tests: StaleFirewall (tests/setup.test.py:245), including a loaded table that differs from the file.
  • B4 timer every 2 minutes and Limits text: part 1.
  • B5 step 6 tailscale up ... --accept-dns=false (README.md:111); -/run/tailscale in part 1.
  • B6 timeouts: 30 min for T3's install, 15 min for apt (sbin/pc-agents:114, :155); bootstrap curl --connect-timeout 30 --max-time 900 (bootstrap.sh:32); process-group kill in part 1's run().
  • B7 apt-get update before apt-get install -y nftables, error names the fix (sbin/pc-agents:155). Test test_nftables_installed_after_a_package_list_update (tests/setup.test.py:220).
  • B8:
    • is-active after the restart, pointing at journalctl (sbin/pc-agents:196; test :232);
    • record keeps old and new hash during a write, so a kill plus a machine file edit is not taken for a hand edit (sbin/pc-agents:171; test test_killed_at_a_write_then_the_machine_file_edited :205; README :150);
    • first-launch note in step 2 (README.md:99) and the preflight hint (sbin/pc-agents:90; test :298);
    • the README fetches bootstrap.sh from ci-release, whose bootstrap refuses a sha not on that branch before running anything (README.md:103, bootstrap.sh:6);
    • setup refuses /opt/pc-agents, /etc/systemd/system and their parents unless root's alone (sbin/pc-agents:92; test test_folders_that_are_not_roots_alone :304).

Each new test was checked by taking its fix out: it fails. setup.test.py 26 OK, check.test.py 23 OK. The PR body has the failure table.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check after fix round 1 (lenspcac1), read at: 364edb6 (#232) and 6f840d6 (#234) — SAFE WITH NITS (both)

The kernel walls, the root-owned runtime, the crash stop and the firewall re-check hold. Every round-0 must-fix and should-fix is closed or honestly documented. Tests: setup.test.py 26 OK, check.test.py 23 OK. Three removed fixes were each caught by the new tests (no mark(): 4 fail; record keeps only the new hash: both killed_at_a_write subtests fail; no ExecStopPost: both Crash tests fail).

Round 0: 232-1 FIXED (wsl -u agent documented, README:115); 232-2..7 FIXED (RestartPreventExitStatus documented as impossible); 232 nits FIXED; 234-1 PARTLY (see 1); 234-2 FIXED for the next run (see 3); 234-3..7 FIXED; 234 nits FIXED, apart from the first-launch user (DOCUMENTED) and "done" while T3 crash-loops (PARTLY, see 4).

Findings (lines at 6f840d6)

  1. (234, should fix) Owner step 7 -- claude fails on a real distro (sbin/pc-agents:218-222, README:118). systemd-run resolves a bare command in the caller's (root's) PATH on the client side, and claude lives in /home/agent/.local/bin, so it fails with "Failed to find executable". The fake systemd-run only logs, so the tests pass. Risk: the owner falls back to wsl -u agent. Fix: ['/usr/bin/env', '--'] + cmd.
  2. (232/234, should fix, plausible) Nothing keeps the distro running. WSL stops a distro soon after its last wsl.exe session closes and does not start it at boot, so T3 stops when the window closes and after every PC restart. Fix: an owner step for a logon/boot task running wsl -d <distro> -u root --exec sleep infinity (or a .wslconfig idle setting), plus a Limits line.
  3. (232 check / 234 marker, should fix, proven) check passes while a restart is pending (no wall for the marker, sbin:239-309). Setup killed after writing policy.json/unit or a T3 download timeout leaves old T3 running with the old IPAddressAllow/policy, and check --enforce reports PASS. Proof test:
    self.ready(); self.machine({**CONFIG, 'maxRuntimeMode': 'approval-required'})
    self.stop_after('/etc/t3code/policy.json')
    self.assertNotIn('approval-required', self.read('/log/units/t3code-agents.service.active'))
    self.assertNotEqual(self.check('--enforce')[0], 0)   # FAILS: exit 0, all PASS
    Fix: FAIL when the marker exists and T3's ExecMainStartTimestamp is older than the marker's mtime.
  4. (234) is-active straight after restart races T3's policy load (sbin:194-199): setup can print "done" and T3 exits 1 a second later. Fix: poll the port or NRestarts=0 for ~15 s.
  5. (232/234) StartLimitBurst=5/600 s with RestartSec=30 (lib:177-178) refuses a rerun of setup within 10 min. Fix: systemctl reset-failed before the restart.
  6. (232, plausible) Boot race: first check 1 min after timer start (lib:209-211); a slow network-online.target makes it fail on "table not loaded", and its systemctl stop cancels T3's queued start every time. Fix: After=t3code-agents.service on pc-agents-walls.service.
  7. Nits: README step 6 says add peers then, but they are required at step 2 (lib:80); say in the README that nothing restarts T3 after a wall stopped it; walls oneshot has no TimeoutStartSec; a hand-made sudoers.d entry for agent is not refused (harmless under NoNewPrivileges); give agent nologin or add a "no agent process outside T3's/as-agent's cgroup" wall.

State: killed at any write → marker first, record keeps both hashes; killed in load-firewall → stale note fails closed; leftover marker → one extra restart; missing machine file and hand-edited policy refused; Tailscale logged out → T3 serves, link waits; older kit → files accepted, a running user@ only caught via linger (nit above).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check after fix round 1 (lenspcas1), read at: 364edb6 (#232) and 6f840d6 (#234) — #232 SAFE WITH NITS, #234 NOT SAFE

Tests at both heads: check.test.py 23 OK, setup.test.py 26 OK. Mutation check: without the user@ mask, the firewall compare or the restart marker, their new tests fail.

Round 0: #232 A1-A5 FIXED; A6 FIXED for the gateway (Windows host LAN address inside allow only DOCUMENTED, README:76); A7 PARTLY/DOCUMENTED (StartLimitBurst=5 replaces RestartPreventExitStatus); A8 FIXED (system D-Bus still reachable, nit). #234 B1 FIXED (as-agent refuses on a failed wall; rejected --property fails closed); B2 PARTLY (finding 1); B3-B7 FIXED; B8 FIXED with a nit (is-active right after restart, sbin:196; fails closed). Round-1 choices hold: bootstrap needs protected:true + merge-base == sha (bootstrap.sh:39); old+new record hash not forgeable by the agent; unsafe folders refused via lstat (sbin:92-95); every step has a time limit.

Findings

  1. [fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234, must fix] Interrupted upgrade leaves the old T3 running and the 2-minute check passes (sbin:283-289, sbin:343-356; check never reads the marker). Owner tightens maxRuntimeMode + new t3Version → download fails, setup exits 1 → current, unit, policy and daemon-reload are new, the old process keeps old binary/policy/sandbox → runtime wall passes ("not installed (yet)"), check --enforce exits 0 every run. An upgrade that closes a hole leaves it open with a passing check. Fix: a restart wall failing while /etc/pc-agents/restart-pending exists (skipped when before_enable). Proof (fails at 6f840d6):
    class PendingIgnoredByCheck(Case):
        def test_interrupted_upgrade_keeps_old_t3_and_check_passes(self):
            self.ready()
            self.machine({**CONFIG, 't3Version': '1.2.4', 'maxRuntimeMode': 'approval-required'})
            self.write('/log/install-fail', '')
            self.assertEqual(self.setup_run()[0], 1)
            running = self.read('/log/units/t3code-agents.service.active')
            self.assertIn('versions/1.2.3/t3 serve', running); self.assertIn('full-access', running)
            code, out = self.check('--enforce')
            self.assertNotEqual(code, 0, out)   # FAILS: exit 0, every wall PASS
  2. [fork-kit: pc-agents walls and their check (part 1 of 2) #232, should fix] Loopback open both ways (lib:167 oifname "lo" accept; no 127/8 in IPAddressDeny, lib:138). Outward (documented, README:174): other distros' localhost servers, one with interop = path to Windows. Inward (undocumented): no SocketBindAllow, so the agent can listen on any port and WSL localhost forwarding shows it on Windows localhost; the owner opens it, the page runs in a localhost origin and reads localhost cookies (not split by port). Fix: SocketBindDeny=any + SocketBindAllow=tcp:<port> (fixed dev range) and an nft drop of skuid agent to lo outside those ports; at least a Limits line + "no localhost servers in other distros".
  3. [fork-kit: pc-agents walls and their check (part 1 of 2) #232, should fix] LAN reachable via the home's public IP (hairpin NAT): router admin page or port-forwarded devices. README:14 says the home network is out of reach. Say it in Limits, or block the current public address.
  4. [fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234, nit] run() uses start_new_session (lib:319): SIGHUP/SIGTERM kills pc-agents but not the installer/apt group; a second install can race it (small harm). The docstring holds only for Ctrl-C.
  5. Nits: timer ActiveState not checked, only UnitFileState (sbin:271-272); --enforce does not stop a running as-agent unit; resolved stub can resolve LAN names via Windows; NETLINK_SOCK_DIAG lists other distros' sockets; agent output via --pty can use OSC 52 (Windows clipboard); /dev/shm shared with root.

State: again → same() (sbin:26) + marker (sbin:193); half-way → new file + rename (sbin:37-60), record keeps both hashes (sbin:167-177), marker first (sbin:42-43), check gap = finding 1, killed install leaves a root-owned .staging-*; hand-made distro/older kit/edited policy refused by preflight (sbin:86-111); missing machine file refused (sbin:344-348); /run marker after reboot rewritten by ExecStartPre (lib:185).

scuttletoby and others added 11 commits November 16, 2026 07:44
Loopback only to T3's port and a devPorts range (unit SocketBindDeny/Allow, nft drop
on lo); walls service ordered after T3 with a start timeout; agent login shell nologin,
sudoers and running-timer walls; README limits (hairpin NAT, D-Bus, shm, sock_diag,
resolved stub, OSC 52) and how to restart T3 after a wall stopped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SocketBindDeny/Allow move into sandbox(), so as-agent gets them too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ute limit

The serial run had grown to about 500 s of tests on a 4-core runner and
was cancelled at the job limit. Test files now run FORK_CI_TEST_JOBS at
once (default: the CPU count), each one's output printed whole in its
own group. A process a test leaves behind holding its output no longer
keeps the run waiting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runAll test drives controlled promises and checks start order and the
most running at once; the script test has two files that each wait for the
other's marker file, so they pass only when run side by side (and fail with
FORK_CI_TEST_JOBS=1). No timers or Date.now in the tests. Header cut to 3 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`sed ... | grep -q` under pipefail fails when grep exits at the match
while sed still writes: a race (5 in 2000 runs idle, 45 under load) that
the parallel runner made show up in fork-follow.test.sh. Ten such pipes
(grep -q, head) now read the whole output first; two of them failed open
(`! ... | grep -q`). fork-ci-tests.test.cjs fails on any new one in a
listed shell test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- restart wall: setup marks T3 for a restart (boot id + monotonic time)
  before and after every change; check fails, and --enforce stops T3 and
  as-agent units, while a T3 that started before the marker runs.
- after the restart setup watches T3 for 15 s before "done" and removes
  the marker only then; reset-failed first, so a rerun within the start
  limit still starts T3.
- as-agent runs the command through /usr/bin/env with the agent's PATH
  (~/.local/bin first); the agent gets /usr/sbin/nologin (usermod for an
  agent an older kit made).
- SIGTERM/SIGHUP kill the running child's process group too.
- README: the restart wall, Windows logon task to keep the distro up,
  limits, step 6 wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 2 (head 4f70d86; part 1 merged in at d6f9e20)

Each fix below was also checked by taking it out of the new code alone: the tests named fail every time (9 of 9).

  • B1 MUST, restart wall: FIXED.
    • Setup writes root's marker /etc/pc-agents/restart-pending (this boot's id and CLOCK_MONOTONIC) before and after every change, after a T3 install, and right before its restart: fork-kit/pc-agents/sbin/pc-agents:38-70, :119, :205-207. It is removed only after T3 restarted and stayed up (:210-215).
    • New restart wall: fails while T3's main process started before the marker (ExecMainStartTimestampMonotonic; another boot passes; no start time, an unreadable marker, an older kit's marker or a failed systemctl show fail): fork-kit/pc-agents/lib/pc_agents.py:335-376, wall at sbin/pc-agents:316-318. It is skipped only for setup's own check before its enable/restart.
    • So --enforce stops the stale T3 and as-agent refuses. Both lens proofs are tests: tests/setup.test.py:282 RestartPending (interrupted upgrade; killed after the policy is written; killed at every write; setup's own restart passes; older kit marker) and tests/check.test.py:242 Restart (8 cases).
  • B2, as-agent -- claude: FIXED. sbin/pc-agents:252-255: the command runs as /usr/bin/env -- <cmd> with the unit's PATH (/home/agent/.local/bin first). The fake systemd-run now resolves a bare command in the caller's PATH like the real one, so the old code fails AsAgent (tests/setup.test.py:450).
  • B3, keep the distro running: DOCS. README step 10 (README.md:144-155): a logon task, no administrator needed (conhost.exe --headless wsl.exe -d <distro> -u root --exec sleep infinity, Register-ScheduledTask at logon for the current user; the unregister line too), pointing to fork-kit/desk/windows/start-desk.ps1 for its logon-task piece. A Limits line, README.md:244-246: it runs only while you are logged in and the PC is awake.
  • B4, T3 stays up: FIXED. sbin/pc-agents:221-236 stays_up(): for 15 s after a (re)start, T3 must stay active with the same main process and no automatic restart, or setup says so and keeps the marker. Test: tests/setup.test.py:246 (the fake's "dies a moment after" mode).
  • B5, start limit: FIXED. sbin/pc-agents:202: systemctl reset-failed before enable/restart. Test: tests/setup.test.py:260.
  • B6 nits:
    • SIGTERM/SIGHUP kill the running child's process group, then exit 128+signal: lib/pc_agents.py:379 stop_on_signals(), called at sbin/pc-agents:418. Test: tests/setup.test.py:383 Signals (a hanging download, then the signal; the download is gone). FIXED.
    • --enforce also stops running as-agent units (pc-agents-as-agent-<pid>): sbin/pc-agents:395-399; the walls unit's ExecStopPost stops them too (lib/pc_agents.py:231-232). Tests: tests/check.test.py:36 Walls.test_enforce_stops_t3, :214 Crash. FIXED.
    • README step 6 peers wording: they are required at step 2 (README.md:126). DOCS.
  • Also, from part 1's A4 (no login shell): setup makes agent with /usr/sbin/nologin, and an agent an older kit made gets it by usermod (sbin/pc-agents:156-163). Test: tests/setup.test.py:270.
  • Changed old test: StaleFirewall.test_tightened_allow_with_failed_install expected check to pass after a failed install. With B1, the old T3 still runs then, so only the restart wall fails; the firewall still agrees with its file.

State: the README's "Runs again, stops half-way" (README.md:175-198) and the PR body's failure table have the new cases: killed at any write, during the restart or the 15 s wait; a marker from an older kit or an earlier boot; no start time; a rerun within the start limit; an older kit's agent; a distro without the logon task.

Tests (fake distro only): check.test.py 36 OK, setup.test.py 35 OK. The new tests on the code before this round: check.test.py 10 fail, setup.test.py 29 fail.

scuttletoby and others added 3 commits November 16, 2026 08:51
Main's fork-run-alert.test.sh had the same early-exit pipe the new guard
refuses; read the whole output first, as in the other shell tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pc-agents-kit

# Conflicts:
#	fork-kit/README.md
#	fork-kit/common/machine.example.json
@scuttletoby
scuttletoby changed the base branch from fork/pc-agents-walls to main November 16, 2026 09:09
# Conflicts:
#	.github/fork-ci-tests.txt
#	fork-kit/pc-agents/README.md
#	fork-kit/pc-agents/lib/pc_agents.py
#	fork-kit/pc-agents/sbin/pc-agents
#	fork-kit/pc-agents/tests/check.test.py
#	fork-kit/pc-agents/tests/fake_distro.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant