Skip to content

fork-kit: pc-agents walls and their check (part 1 of 2) - #232

Merged
scuttletoby merged 9 commits into
mainfrom
fork/pc-agents-walls
Nov 16, 2026
Merged

scuttletoby merged 9 commits into
mainfrom
fork/pc-agents-walls

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Part 1 of 2 of the pc-agents kit (fork-kit/pc-agents/): run T3's agents on a Windows PC inside a walled WSL2 distro, and link to it with Full access. This part is the walls and their check. Part 2 (#234, stacked on this one) is pc-agents setup, the bootstrap and the owner steps.

What is in it

  • lib/pc_agents.py holds the config: the pc-agents section of the machine file /etc/fork-kit/machine.json, read with fork-kit/common/machine_file.py. It also renders every file the kit owns from that config:
    • /etc/wsl.conf:
      • automount off and mountFsTab off;
      • interop off and appendWindowsPath off;
      • systemd on;
      • its own resolv.conf;
      • default user root.
    • /etc/resolv.conf: the public nameservers from the config. WSL's own nameserver is on the blocked Windows host.
    • /etc/t3code/policy.json (the managed policy):
      • turns the beta (linkedEnvironmentsEnabled) on;
      • turns automaticUpdates, Connect, webhook tasks and browser cookie import off;
      • locks maxRuntimeMode and the config's lockedSettings.
    • /etc/pc-agents/firewall.nft: for the agent's uid only, it drops private, link-local, CGNAT and multicast ranges, and localhost traffic other than to T3's port and devPorts.
    • t3code-agents.service, a root system unit with User=agent:
      • ExecStart runs T3 from /opt/pc-agents/t3/runtime/versions/<v>/t3 (root's);
      • RestrictAddressFamilies without AF_VSOCK, and io_uring off;
      • InaccessiblePaths=/mnt /run/WSL /tmp/.X11-unix /usr/lib/wsl /run/tailscale;
      • IPAddressDeny= the blocked ranges, IPAddressAllow= the peers and allow;
      • SocketBindDeny=any, SocketBindAllow= T3's port and devPorts (new machine-file key, default 4100-4199);
      • SHELL=/bin/bash (the agent's login shell is /usr/sbin/nologin);
      • MemoryMax, CPUQuota, ProtectSystem=strict, no namespaces;
      • StartLimitBurst=5 in 10 minutes;
      • ExecStartPre loads the firewall, then runs the check.
    • A timer that runs pc-agents check --enforce every 2 minutes. Its service has ExecStopPost, which stops T3 whenever the check did not succeed, including when it cannot start or crashes. It is ordered After=t3code-agents.service (a check never cancels T3's start at boot) and has TimeoutStartSec=5min (a hung check fails, and so stops T3).
  • sbin/pc-agents check [--enforce] runs the fleet's merged WSL2 walls plus the kit's own walls:
    • every file is root-owned, byte for byte, and recorded (record);
    • masks, with the agent's user@<uid>.service now among them;
    • linger and user-units;
    • drop-ins: every folder systemd reads, including prefix drop-ins;
    • units: systemctl show, and the timer is running (ActiveState), not only enabled;
    • agent: the only login user, no other group, login shell /usr/sbin/nologin;
    • sudoers: no rule in /etc/sudoers or /etc/sudoers.d names the agent, its group, its uid or ALL users;
    • kit-code: /, /opt, /opt/pc-agents, the link and the release folder;
    • runtime: T3 is root's, and its hash is in the record;
    • automount, with a trailing-slash prefix;
    • firewall: the loaded table is the file's;
    • gateway: blocked, and no allow or peers entry covers it.
  • pc-agents load-firewall loads the file. It then saves the file's sha256 and what nft list shows in /run/pc-agents/firewall.loaded, so the check can compare against both.
  • apps/server/src/policy/managedSettings.forkKeys.test.ts proves that the managed policy locks fork settings.

A2: the locks bind the agent (decision: root-owned runtime)

T3 is installed by root under /opt/pc-agents/t3. Part 2 does this with upstream scripts/install.sh and T3CODE_HOME=/opt/pc-agents/t3; the script checks the release's SHA256SUMS and runs t3 --version as root. The unit runs it from there. The runtime wall checks two things:

  • the folders and every file are root's and writable by root alone;
  • t3's sha256 is the one setup recorded.

The agent cannot swap in a T3 that ignores the policy, so the locks are not merely advisory. The README Limits say what is left: an agent can still start its own programs inside the unit (even its own copy of T3). Those ignore the policy but stay inside every other wall.

What can be locked

lockedSettings now takes:

  • one field per dotted path;
  • a whole setting as an object (agentSlots);
  • one record entry as an object (providerInstances.codex-work; ids may contain -).

The kit refuses what is cheap to tell:

  • a field of a provider instance (T3 refuses those);
  • an object deeper than a record entry;
  • the kit's own two settings;
  • a blank organization.

T3 refuses the rest (a partial object, a field inside a nullable setting) and does not start. T3 exits 1 on a policy it cannot apply, the same code as any other failure, so RestartPreventExitStatus= cannot single it out. Instead the unit gives up after 5 starts in 10 minutes (StartLimitBurst=5).

Runs again, stops half-way, meets old or empty state

Case What happens
Runs again The check is read-only, except that --enforce stops the unit and load-firewall rewrites its saved file only when it changed. Two runs give the same answer.
The check crashes, cannot import its lib, or current is broken The walls service fails. ExecStopPost stops T3 (test Crash.test_missing_lib_stops_t3). At T3's start, ExecStartPre fails, so T3 does not start.
systemctl stop fails The check says "stopping ... FAILED" and exits 1. The timer tries again 2 minutes later.
Killed between nft -f and saving firewall.loaded The saved file is old or missing, so the firewall wall fails and T3 is stopped or not started. The next load-firewall (at T3's start) fixes it.
Another distro drops or changes the global nft table The firewall wall fails within 2 minutes and T3 is stopped. The unit's IPAddressDeny= holds meanwhile.
The firewall file is newer than the loaded table (setup stopped before loading) The firewall wall fails until load-firewall (test test_firewall_file_newer_than_the_loaded_table).
T3's version folder is missing (install not done or failed) The runtime wall passes, because there is nothing to wall: ExecStart does not exist, so T3 cannot start. A folder that is present but changed fails.
Old kit: the unit still runs T3 from the agent's home file:t3code-agents.service differs, so T3 does not start until part 2's setup runs again. The old runtime in the agent's home is left alone and unused.
Old machine file without peers The config is refused, so the check fails (FAIL check) until peers is added.
No machine file, or a broken one FAIL check.
Hand-made units: a copy in /usr/lib, a system.control or prefix drop-in, a disabled timer, a pending reload drop-ins or units fails.
The agent lingers or has user units linger or user-units fails. The agent's user manager is masked anyway.
The timer is enabled but stopped units fails (ActiveState), seen at T3's next start.
A distro from round 1: the agent's shell is /bin/bash agent fails until part 2's setup runs again (it sets nologin).
Old machine file without devPorts The default 4100-4199 applies; the unit and firewall change, so setup restarts T3 once.
A check started while T3 is still starting at boot (slow network) It waits for T3's start (After=), then checks; it no longer fails on a table not loaded yet and cancels T3's start.
The check hangs TimeoutStartSec=5min fails the service, and ExecStopPost stops T3.
A wall stopped T3 Nothing starts it again by itself (README: fix, then systemctl start t3code-agents.service).

General vs project

General: any WSL2 PC, any repo. Every machine fact comes from the machine file the using project's owner writes:

  • the organization name;
  • the T3 version and release URL;
  • the port and the uid;
  • the runtime mode ceiling and the locked settings;
  • the memory and CPU caps;
  • the nameservers;
  • the extra allowed networks and the linking peers;
  • the Tailscale tag.

The README and the tests use made-up values.

The CI runner change (.github/scripts/fork-ci-tests.cjs runs the fork's test files in parallel) is general: it has no project facts, and FORK_CI_TEST_JOBS sets how many run at once.

Upstream fit

Built on upstream mechanisms:

  • T3's managed policy for the locks;
  • scripts/install.sh (T3CODE_HOME, T3CODE_VERSION, T3CODE_RELEASE_BASE_URL) for the root install in part 2;
  • the server's own T3CODE_PORT/T3CODE_LAN.

It uses its own system unit instead of t3 service install, because that makes a user unit the agent could override.

Tests

  • python3 -I -B fork-kit/pc-agents/tests/check.test.py: 28 tests, on a fake distro. The fake systemctl keeps which units are enabled and running and answers show and is-active.
  • managedSettings.forkKeys.test.ts: 5 tests, unchanged.
  • node --test .github/scripts/fork-ci-tests.test.cjs: 17 tests, none of them timed (they use marker files and promises, not clocks); one fails when a listed shell test pipes into grep -q, grep -m or head, which races with SIGPIPE under pipefail. The fork's script tests now run in parallel, because one after another they no longer fit the CI job's 10-minute limit.

Size and split

Non-test code here is now 555 lines (lib/pc_agents.py 380, sbin/pc-agents 175), which is over the 400-line guide for one PR. Fix rounds 1 and 2 added about 220 lines of walls the reviews asked for. A further split would cut the walls apart from their check, so it stays one PR; part 2 is separate.

🤖 Generated with Claude Code

A WSL2 distro on a Windows PC as the wall around T3's agents: the files
the kit owns (wsl.conf, resolv.conf, managed policy, uid-matched nft
table, a root system unit with User=agent and a sandbox, a daily check
timer), rendered from the pc-agents section of the machine file, and
`pc-agents check [--enforce]`: the fleet's wsl2 walls (run, not copied)
plus the kit's own. The unit runs the check before every start.

A server test pins that the managed policy locks the fork's settings
(agentSlots, linkedEnvironmentsEnabled, automaticUpdates).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Two lenses at b842a49: security lens232s NOT SAFE, correctness lens232c NOT SAFE (Opus 5.5 high, read-only). check.test.py 4 OK. The kernel walls inside the unit are sound; agent code can run outside the unit, and the check has gaps. 9 proof tests (scratch, not committed) fail on this head.

Must fix:

  1. Agent code runs outside the unit (lib/pc_agents.py:148 [user] default=agent; unit ReadWritePaths=/home/agent). The agent writes ~/.bashrc or a user unit + default.target.wants link; the owner runs wsl -d <distro> (or fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234's owner steps as agent); user@<uid>.service and the shell run unwalled: AF_VSOCK, user namespaces (the fleet userns wall is dropped at pc_agents.py:22 because "the unit closes them"), /mnt/wsl, no MemoryMax. Linger keeps it across boots. check exits 0 with a user unit and /var/lib/systemd/linger/agent. Fix: default user root (or an owner user), mask user@<agent uid>.service, walls for the mask and an empty linger dir, owner steps run in the unit's sandbox.
  2. Policy locks are enforced by a binary the agent can rewrite (ExecStart runs T3 from /home/agent/.t3/runtime/..., inside ReadWritePaths). Patch it, crash it, Restart=on-failure starts the patched one; check never hashes the runtime. Fix: root-owned runtime under /opt with a hash wall, or say plainly the locks are advisory against the agent.
  3. --enforce doesn't stop T3 when the check crashes before check() (sbin/pc-agents:233 imports first: broken or missing current -> ModuleNotFoundError, T3 keeps running); a failed systemctl stop is reported as stopped (:321). Fix: OnFailure=/ExecStopPost= on pc-agents-walls.service that stops T3 when the check did not succeed; check the stop's return code.

Should fix:
4. Firewall is one global table, checked at start and daily; any root in any distro (nftables.service flush ruleset) removes it for up to 24 h; the wall only greps skuid/ drop. Fix: a second wall in the unit, IPAddressDeny= the private ranges + IPAddressAllow= the linking peer (cgroup BPF, already proven by the cgroup-bpf wall); compare the loaded table with the file.
5. Drop-in wall misses system.control (systemctl set-property), system.attached, prefix drop-ins (t3code-.service.d), /usr/lib service.d, /run/systemd/generator{,.late}, the timer's drop-ins; never checks the timer is enabled. Fix: ask systemd (systemctl show -p FragmentPath,DropInPaths,UnitFileState for all three units).
6. gateway wall passes when allow contains the Windows host (172.16.0.0/12): fail when an allow entry covers the gateway.
7. lockedSettings refuses every object value, so whole nullable settings, providerInstances.<id>, record entries and ids with - cannot be locked, while the README says "lock the whole setting". Kit checks are shallower than T3's (blank organization passes the kit, T3 refuses, restart loop every 30 s): add RestartPreventExitStatus= for the policy exit code.

Limits / nits: abstract unix sockets are shared across distros (Limits); /run/tailscale/tailscaled.sock and the system bus are reachable (InaccessiblePaths=-/run/tailscale); kit-code wall misses /opt/pc-agents, the version folder and COMMON (prepended to sys.path before stdlib, sbin:231); automount skips /usr/lib/wsl* without a trailing slash (:273); record file owner unchecked; fake nft never parses firewall.nft.

A fix round for #232 and #234 together follows; two lenses re-check each.

- Agent code runs only in T3's unit: default distro user root, the agent's
  user manager masked, walls for the mask, linger and user units.
- T3's runtime is root's under /opt/pc-agents/t3; the unit runs it from
  there; a wall checks its owner, mode and recorded hash.
- A check that cannot run or crashes stops T3 (ExecStopPost on the walls
  service); a failed stop is reported.
- Second network wall in the unit (IPAddressDeny/IPAddressAllow with the
  new peers key); the firewall wall compares the loaded table with the
  file; check timer every 2 minutes.
- Units wall asks systemctl show; the file scan covers system.control,
  system.attached, generators, /run, /usr/lib and prefix drop-ins.
- Gateway wall fails when allow or peers cover the gateway.
- lockedSettings takes whole objects and record entries; blank
  organization refused; StartLimit for a policy T3 cannot apply.
- run() kills the whole process group on a timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1 (head 364edb6)

  • A1 (agent code only in T3's unit):
    • default user root: fork-kit/pc-agents/lib/pc_agents.py:187;
    • user@<uid>.service masked: lib/pc_agents.py:109;
    • walls masks, linger and user-units: fork-kit/pc-agents/sbin/pc-agents:50-58;
    • tests: class Holes in tests/check.test.py (user unit plus linger, system.control drop-in, timer drop-in, prefix drop-in) and test_agent_user_manager_masked.
  • A2 (decided: root-owned runtime):
  • A3 (a crash still stops T3):
    • ExecStopPost on the walls service: lib/pc_agents.py:198;
    • the stop's return code is checked: sbin/pc-agents:150-153;
    • tests: Crash.test_missing_lib_stops_t3, test_enforce_says_when_the_stop_fails.
  • A4 (second network wall):
    • IPAddressDeny/IPAddressAllow with the new required peers: lib/pc_agents.py:167-168, :79;
    • load-firewall saves the file's hash and the listing: lib/pc_agents.py:252;
    • the firewall wall compares them: sbin/pc-agents:99-107;
    • timer every 2 minutes: lib/pc_agents.py:201;
    • tests: test_firewall_changed_after_it_was_loaded, test_firewall_file_newer_than_the_loaded_table.
  • A5 (units as systemd sees them):
    • units wall with systemctl show: sbin/pc-agents:62-72;
    • file scan of every unit folder and prefix drop-in: lib/pc_agents.py:29, :115, sbin/pc-agents:59-61;
    • tests: test_units_as_systemd_loaded_them, test_unit_copies_and_prefix_drop_ins.
  • A6 (gateway): sbin/pc-agents:108-121; test test_allow_or_peers_covering_the_gateway.
  • A7 (lockedSettings):
    • locked_problem: lib/pc_agents.py:84;
    • blank organization: lib/pc_agents.py:67;
    • RestartPreventExitStatus is not possible, because T3 exits 1 on a bad policy like on any failure. StartLimitBurst=5 instead: lib/pc_agents.py:143;
    • test: Unit.test_config.
  • A8 (nits):
    • -/run/tailscale: lib/pc_agents.py:166;
    • kit-code covers /, /opt, /opt/pc-agents and the release tree: sbin/pc-agents:74-80;
    • automount trailing slash: sbin/pc-agents:94;
    • record wall: sbin/pc-agents:49;
    • README Limits: abstract sockets, another distro dropping the table, an honest T3;
    • tests: test_kit_code_folders, test_automount_only_under_usr_lib_wsl, the record subtest.
  • B6 (part 1 side): run() runs in its own process group and kills the whole group on a timeout (exit 124 with a message): lib/pc_agents.py:306. Test: Run.test_timeout_kills_the_whole_group.

Tests: check.test.py 23 OK (was 4). The PR body has the failure table and the A2 decision.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check after fix round 1 (lenspcac1), read at: 364edb6 (#232) and 6f840d6 (#234) — SAFE WITH NITS (both)

The kernel walls, the root-owned runtime, the crash stop and the firewall re-check hold. Every round-0 must-fix and should-fix is closed or honestly documented. Tests: setup.test.py 26 OK, check.test.py 23 OK. Three removed fixes were each caught by the new tests (no mark(): 4 fail; record keeps only the new hash: both killed_at_a_write subtests fail; no ExecStopPost: both Crash tests fail).

Round 0: 232-1 FIXED (wsl -u agent documented, README:115); 232-2..7 FIXED (RestartPreventExitStatus documented as impossible); 232 nits FIXED; 234-1 PARTLY (see 1); 234-2 FIXED for the next run (see 3); 234-3..7 FIXED; 234 nits FIXED, apart from the first-launch user (DOCUMENTED) and "done" while T3 crash-loops (PARTLY, see 4).

Findings (lines at 6f840d6)

  1. (234, should fix) Owner step 7 -- claude fails on a real distro (sbin/pc-agents:218-222, README:118). systemd-run resolves a bare command in the caller's (root's) PATH on the client side, and claude lives in /home/agent/.local/bin, so it fails with "Failed to find executable". The fake systemd-run only logs, so the tests pass. Risk: the owner falls back to wsl -u agent. Fix: ['/usr/bin/env', '--'] + cmd.
  2. (232/234, should fix, plausible) Nothing keeps the distro running. WSL stops a distro soon after its last wsl.exe session closes and does not start it at boot, so T3 stops when the window closes and after every PC restart. Fix: an owner step for a logon/boot task running wsl -d <distro> -u root --exec sleep infinity (or a .wslconfig idle setting), plus a Limits line.
  3. (232 check / 234 marker, should fix, proven) check passes while a restart is pending (no wall for the marker, sbin:239-309). Setup killed after writing policy.json/unit or a T3 download timeout leaves old T3 running with the old IPAddressAllow/policy, and check --enforce reports PASS. Proof test:
    self.ready(); self.machine({**CONFIG, 'maxRuntimeMode': 'approval-required'})
    self.stop_after('/etc/t3code/policy.json')
    self.assertNotIn('approval-required', self.read('/log/units/t3code-agents.service.active'))
    self.assertNotEqual(self.check('--enforce')[0], 0)   # FAILS: exit 0, all PASS
    Fix: FAIL when the marker exists and T3's ExecMainStartTimestamp is older than the marker's mtime.
  4. (234) is-active straight after restart races T3's policy load (sbin:194-199): setup can print "done" and T3 exits 1 a second later. Fix: poll the port or NRestarts=0 for ~15 s.
  5. (232/234) StartLimitBurst=5/600 s with RestartSec=30 (lib:177-178) refuses a rerun of setup within 10 min. Fix: systemctl reset-failed before the restart.
  6. (232, plausible) Boot race: first check 1 min after timer start (lib:209-211); a slow network-online.target makes it fail on "table not loaded", and its systemctl stop cancels T3's queued start every time. Fix: After=t3code-agents.service on pc-agents-walls.service.
  7. Nits: README step 6 says add peers then, but they are required at step 2 (lib:80); say in the README that nothing restarts T3 after a wall stopped it; walls oneshot has no TimeoutStartSec; a hand-made sudoers.d entry for agent is not refused (harmless under NoNewPrivileges); give agent nologin or add a "no agent process outside T3's/as-agent's cgroup" wall.

State: killed at any write → marker first, record keeps both hashes; killed in load-firewall → stale note fails closed; leftover marker → one extra restart; missing machine file and hand-edited policy refused; Tailscale logged out → T3 serves, link waits; older kit → files accepted, a running user@ only caught via linger (nit above).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check after fix round 1 (lenspcas1), read at: 364edb6 (#232) and 6f840d6 (#234) — #232 SAFE WITH NITS, #234 NOT SAFE

Tests at both heads: check.test.py 23 OK, setup.test.py 26 OK. Mutation check: without the user@ mask, the firewall compare or the restart marker, their new tests fail.

Round 0: #232 A1-A5 FIXED; A6 FIXED for the gateway (Windows host LAN address inside allow only DOCUMENTED, README:76); A7 PARTLY/DOCUMENTED (StartLimitBurst=5 replaces RestartPreventExitStatus); A8 FIXED (system D-Bus still reachable, nit). #234 B1 FIXED (as-agent refuses on a failed wall; rejected --property fails closed); B2 PARTLY (finding 1); B3-B7 FIXED; B8 FIXED with a nit (is-active right after restart, sbin:196; fails closed). Round-1 choices hold: bootstrap needs protected:true + merge-base == sha (bootstrap.sh:39); old+new record hash not forgeable by the agent; unsafe folders refused via lstat (sbin:92-95); every step has a time limit.

Findings

  1. [fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234, must fix] Interrupted upgrade leaves the old T3 running and the 2-minute check passes (sbin:283-289, sbin:343-356; check never reads the marker). Owner tightens maxRuntimeMode + new t3Version → download fails, setup exits 1 → current, unit, policy and daemon-reload are new, the old process keeps old binary/policy/sandbox → runtime wall passes ("not installed (yet)"), check --enforce exits 0 every run. An upgrade that closes a hole leaves it open with a passing check. Fix: a restart wall failing while /etc/pc-agents/restart-pending exists (skipped when before_enable). Proof (fails at 6f840d6):
    class PendingIgnoredByCheck(Case):
        def test_interrupted_upgrade_keeps_old_t3_and_check_passes(self):
            self.ready()
            self.machine({**CONFIG, 't3Version': '1.2.4', 'maxRuntimeMode': 'approval-required'})
            self.write('/log/install-fail', '')
            self.assertEqual(self.setup_run()[0], 1)
            running = self.read('/log/units/t3code-agents.service.active')
            self.assertIn('versions/1.2.3/t3 serve', running); self.assertIn('full-access', running)
            code, out = self.check('--enforce')
            self.assertNotEqual(code, 0, out)   # FAILS: exit 0, every wall PASS
  2. [fork-kit: pc-agents walls and their check (part 1 of 2) #232, should fix] Loopback open both ways (lib:167 oifname "lo" accept; no 127/8 in IPAddressDeny, lib:138). Outward (documented, README:174): other distros' localhost servers, one with interop = path to Windows. Inward (undocumented): no SocketBindAllow, so the agent can listen on any port and WSL localhost forwarding shows it on Windows localhost; the owner opens it, the page runs in a localhost origin and reads localhost cookies (not split by port). Fix: SocketBindDeny=any + SocketBindAllow=tcp:<port> (fixed dev range) and an nft drop of skuid agent to lo outside those ports; at least a Limits line + "no localhost servers in other distros".
  3. [fork-kit: pc-agents walls and their check (part 1 of 2) #232, should fix] LAN reachable via the home's public IP (hairpin NAT): router admin page or port-forwarded devices. README:14 says the home network is out of reach. Say it in Limits, or block the current public address.
  4. [fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234, nit] run() uses start_new_session (lib:319): SIGHUP/SIGTERM kills pc-agents but not the installer/apt group; a second install can race it (small harm). The docstring holds only for Ctrl-C.
  5. Nits: timer ActiveState not checked, only UnitFileState (sbin:271-272); --enforce does not stop a running as-agent unit; resolved stub can resolve LAN names via Windows; NETLINK_SOCK_DIAG lists other distros' sockets; agent output via --pty can use OSC 52 (Windows clipboard); /dev/shm shared with root.

State: again → same() (sbin:26) + marker (sbin:193); half-way → new file + rename (sbin:37-60), record keeps both hashes (sbin:167-177), marker first (sbin:42-43), check gap = finding 1, killed install leaves a root-owned .staging-*; hand-made distro/older kit/edited policy refused by preflight (sbin:86-111); missing machine file refused (sbin:344-348); /run marker after reboot rewritten by ExecStartPre (lib:185).

Loopback only to T3's port and a devPorts range (unit SocketBindDeny/Allow, nft drop
on lo); walls service ordered after T3 with a start timeout; agent login shell nologin,
sudoers and running-timer walls; README limits (hairpin NAT, D-Bus, shm, sock_diag,
resolved stub, OSC 52) and how to restart T3 after a wall stopped it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

scuttletoby commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner Author

Fix round 2 (heads b6ae1ea, then 315aa3b, 519449e, f446ea1 and d6f9e20 for the CI fixes)

  • A1 loopback both ways (lenspcas1 fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2): FIXED.
    • new machine-file key devPorts (LOW-HIGH, default 4100-4199, validated): fork-kit/pc-agents/lib/pc_agents.py:82, :86-98;
    • nft: the agent's localhost traffic only to T3's port and devPorts, the rest dropped: lib/pc_agents.py:150;
    • unit: SocketBindDeny=any + SocketBindAllow=tcp: T3's port and devPorts: lib/pc_agents.py:186-187;
    • checked through the byte-for-byte unit file wall and the loaded-table compare; tests Unit.test_loopback, Unit.test_loopback_rules_are_checked (tests/check.test.py:258, :279);
    • README (docs): wall row README.md:15, Limits "Run no localhost servers in other distros", WSL forwarding to Windows' localhost and shared localhost cookies, servers outside devPorts.
  • A2 hairpin NAT (lenspcas1 fork: CI check that refuses database schema and migration changes #3): DOCS. README.md:14 no longer says the whole home network is out of reach; Limits line on the home's public address (README.md:117-121).
  • A3 boot race (lenspcac1 fork: fast CI on GitHub-hosted runners (fork-tests.yml) #6): FIXED. After=t3code-agents.service and TimeoutStartSec=5min on the walls oneshot: lib/pc_agents.py:217-218; test in Unit.test_unit (tests/check.test.py:255).
  • A4 nits:
    • timer ActiveState checked: sbin/pc-agents:65, :71-72; test test_timer_stopped_but_enabled: FIXED;
    • agent login shell /usr/sbin/nologin (+ agent wall), T3 keeps SHELL=/bin/bash from its unit: lib/pc_agents.py:15, :166, sbin/pc-agents:76-79; test test_agent_has_no_login_shell (setup sets the shell in fork-kit: pc-agents setup, bootstrap and owner steps (part 2 of 2) #234): FIXED;
    • sudoers wall (a rule naming the agent, %agent, #uid, ALL users, a User_Alias): lib/pc_agents.py:323, sbin/pc-agents:80; test test_sudo_rules_that_cover_the_agent and the sudoers case of test_each_wall: FIXED;
    • README: nothing restarts T3 after a wall stopped it, and how to start it (README.md:95-97): DOCS;
    • Limits lines for the system D-Bus, /dev/shm, NETLINK_SOCK_DIAG, the resolved stub (now dropped by the loopback rule) and OSC 52: README.md:125-131: DOCS.

Tests: check.test.py 28 OK (was 23). Every new test fails on the round-1 code (16 failures, run with the old code and the new tests). The PR body's failure table has the new cases.

  • CI: "Fork script tests" cancelled at the job's 10-minute limit (run 43825995959): FIXED in 315aa3b, format and lint in 519449e and f446ea1.
    • Not a hang: check.test.py had run 14 s when the job limit hit. The step ran every test file one after another, about 500 s of tests on a 4-core runner, plus setup.
    • .github/scripts/fork-ci-tests.cjs:124 runAll runs the files FORK_CI_TEST_JOBS at once (default: the CPU count, :137); each file's output is printed whole in its own group with its time (:143). A process a test leaves behind holding its output no longer keeps the run waiting (:175).
    • Tests: .github/scripts/fork-ci-tests.test.cjs 15 OK (was 10); 4 of the new ones fail on the old runner (the fifth checks FORK_CI_TEST_JOBS=1 runs one at a time). A full local run with 4 jobs passes; the slowest file took 59 s.
    • Lint (f446ea1): the tests prove the parallel run without clocks. runAll is driven by promises the test resolves, checking start order and the most running at once; in the script test two files each wait for the other's marker file, so they pass only side by side. No timers or Date.now. The runner's header is 3 lines. vp check on both files: no errors or warnings. The 9 CI lint warnings are all in upstream files this PR does not touch.
  • CI: fork-follow.test.sh "workflow: the last sweep reports runs since the run began" failed at f446ea1 (run 43831854073): FIXED in d6f9e20.
    • Root cause: a SIGPIPE race, not shared state. The check was sed ... | grep -q ... under set -o pipefail. grep -q exits at the first match, sed is then killed while it writes the rest, and pipefail reads that as a failed check. Reproduced locally: 5 failures in 2000 runs idle, 45 in 2000 with the CPU busy, so parallel runs made a race that was already there show up. The Refusing to call gh text in the log was left over from an earlier check's output file. fork-secret-scan.test.sh:73 already noted the same trap.
    • The same pattern was in 10 places. Each now reads the whole output first (grep -q X <<< "$(...)"): .github/scripts/fork-follow.test.sh:444, 473, 509, 1057, 1059, 1070, 1072, .github/scripts/fork-interaction-limit.test.sh:110-111, .github/scripts/fork-secret-scan.test.sh:64 (head -n 1 became sed -n 1p). Two of them failed open: ! ... | grep -q (fork-follow.test.sh:1070, the pinned-actions check, and not_called) turned a SIGPIPE into a pass.
    • Guard: .github/scripts/fork-ci-tests.test.cjs:311 fails when any listed shell test pipes into grep -q, grep -m or head (it lists file:line); :294 checks the pattern. On the old files it names all 10 lines. A few non-test scripts pipe one short printf or a small gh answer into grep -q/head -n1; those are outside this PR and were left alone.
    • Tests: fork-ci-tests.test.cjs 17 OK. The whole runner locally with FORK_CI_TEST_JOBS=4: all 48 files pass (slowest 57 s); the three changed shell tests also pass on their own.

scuttletoby and others added 6 commits November 16, 2026 08:06
…ute limit

The serial run had grown to about 500 s of tests on a 4-core runner and
was cancelled at the job limit. Test files now run FORK_CI_TEST_JOBS at
once (default: the CPU count), each one's output printed whole in its
own group. A process a test leaves behind holding its output no longer
keeps the run waiting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runAll test drives controlled promises and checks start order and the
most running at once; the script test has two files that each wait for the
other's marker file, so they pass only when run side by side (and fail with
FORK_CI_TEST_JOBS=1). No timers or Date.now in the tests. Header cut to 3 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`sed ... | grep -q` under pipefail fails when grep exits at the match
while sed still writes: a race (5 in 2000 runs idle, 45 under load) that
the parallel runner made show up in fork-follow.test.sh. Ten such pipes
(grep -q, head) now read the whole output first; two of them failed open
(`! ... | grep -q`). fork-ci-tests.test.cjs fails on any new one in a
listed shell test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's fork-run-alert.test.sh had the same early-exit pipe the new guard
refuses; read the whole output first, as in the other shell tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby
scuttletoby marked this pull request as ready for review November 16, 2026 09:06
@scuttletoby
scuttletoby merged commit d4ac8ee into main Nov 16, 2026
33 checks passed
@scuttletoby
scuttletoby deleted the fork/pc-agents-walls branch November 16, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant