Repository navigation
fork-kit: pc-agents walls and their check (part 1 of 2) - #232
Conversation
A WSL2 distro on a Windows PC as the wall around T3's agents: the files the kit owns (wsl.conf, resolv.conf, managed policy, uid-matched nft table, a root system unit with User=agent and a sandbox, a daily check timer), rendered from the pc-agents section of the machine file, and `pc-agents check [--enforce]`: the fleet's wsl2 walls (run, not copied) plus the kit's own. The unit runs the check before every start. A server test pins that the managed policy locks the fork's settings (agentSlots, linkedEnvironmentsEnabled, automaticUpdates). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Two lenses at b842a49: security lens232s NOT SAFE, correctness lens232c NOT SAFE (Opus 5.5 high, read-only). Must fix:
Should fix: Limits / nits: abstract unix sockets are shared across distros (Limits); A fix round for #232 and #234 together follows; two lenses re-check each. |
- Agent code runs only in T3's unit: default distro user root, the agent's user manager masked, walls for the mask, linger and user units. - T3's runtime is root's under /opt/pc-agents/t3; the unit runs it from there; a wall checks its owner, mode and recorded hash. - A check that cannot run or crashes stops T3 (ExecStopPost on the walls service); a failed stop is reported. - Second network wall in the unit (IPAddressDeny/IPAddressAllow with the new peers key); the firewall wall compares the loaded table with the file; check timer every 2 minutes. - Units wall asks systemctl show; the file scan covers system.control, system.attached, generators, /run, /usr/lib and prefix drop-ins. - Gateway wall fails when allow or peers cover the gateway. - lockedSettings takes whole objects and record entries; blank organization refused; StartLimit for a policy T3 cannot apply. - run() kills the whole process group on a timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fix round 1 (head 364edb6)
Tests: |
Lens: correctness re-check after fix round 1 (lenspcac1), read at: 364edb6 (#232) and 6f840d6 (#234) — SAFE WITH NITS (both)The kernel walls, the root-owned runtime, the crash stop and the firewall re-check hold. Every round-0 must-fix and should-fix is closed or honestly documented. Tests: Round 0: 232-1 FIXED ( Findings (lines at 6f840d6)
State: killed at any write → marker first, record keeps both hashes; killed in |
Lens: security re-check after fix round 1 (lenspcas1), read at: 364edb6 (#232) and 6f840d6 (#234) — #232 SAFE WITH NITS, #234 NOT SAFETests at both heads: Round 0: #232 A1-A5 FIXED; A6 FIXED for the gateway (Windows host LAN address inside Findings
State: again → |
Loopback only to T3's port and a devPorts range (unit SocketBindDeny/Allow, nft drop on lo); walls service ordered after T3 with a start timeout; agent login shell nologin, sudoers and running-timer walls; README limits (hairpin NAT, D-Bus, shm, sock_diag, resolved stub, OSC 52) and how to restart T3 after a wall stopped it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fix round 2 (heads b6ae1ea, then 315aa3b, 519449e, f446ea1 and d6f9e20 for the CI fixes)
Tests:
|
…ute limit The serial run had grown to about 500 s of tests on a 4-core runner and was cancelled at the job limit. Test files now run FORK_CI_TEST_JOBS at once (default: the CPU count), each one's output printed whole in its own group. A process a test leaves behind holding its output no longer keeps the run waiting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runAll test drives controlled promises and checks start order and the most running at once; the script test has two files that each wait for the other's marker file, so they pass only when run side by side (and fail with FORK_CI_TEST_JOBS=1). No timers or Date.now in the tests. Header cut to 3 lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`sed ... | grep -q` under pipefail fails when grep exits at the match while sed still writes: a race (5 in 2000 runs idle, 45 under load) that the parallel runner made show up in fork-follow.test.sh. Ten such pipes (grep -q, head) now read the whole output first; two of them failed open (`! ... | grep -q`). fork-ci-tests.test.cjs fails on any new one in a listed shell test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's fork-run-alert.test.sh had the same early-exit pipe the new guard refuses; read the whole output first, as in the other shell tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part 1 of 2 of the pc-agents kit (
fork-kit/pc-agents/): run T3's agents on a Windows PC inside a walled WSL2 distro, and link to it with Full access. This part is the walls and their check. Part 2 (#234, stacked on this one) ispc-agents setup, the bootstrap and the owner steps.What is in it
lib/pc_agents.pyholds the config: thepc-agentssection of the machine file/etc/fork-kit/machine.json, read withfork-kit/common/machine_file.py. It also renders every file the kit owns from that config:/etc/wsl.conf:mountFsTaboff;appendWindowsPathoff;resolv.conf;root./etc/resolv.conf: the public nameservers from the config. WSL's own nameserver is on the blocked Windows host./etc/t3code/policy.json(the managed policy):linkedEnvironmentsEnabled) on;automaticUpdates, Connect, webhook tasks and browser cookie import off;maxRuntimeModeand the config'slockedSettings./etc/pc-agents/firewall.nft: for the agent's uid only, it drops private, link-local, CGNAT and multicast ranges, and localhost traffic other than to T3's port anddevPorts.t3code-agents.service, a root system unit withUser=agent:ExecStartruns T3 from/opt/pc-agents/t3/runtime/versions/<v>/t3(root's);RestrictAddressFamilieswithoutAF_VSOCK, and io_uring off;InaccessiblePaths=/mnt /run/WSL /tmp/.X11-unix /usr/lib/wsl /run/tailscale;IPAddressDeny=the blocked ranges,IPAddressAllow=thepeersandallow;SocketBindDeny=any,SocketBindAllow=T3's port anddevPorts(new machine-file key, default4100-4199);SHELL=/bin/bash(the agent's login shell is/usr/sbin/nologin);MemoryMax,CPUQuota,ProtectSystem=strict, no namespaces;StartLimitBurst=5in 10 minutes;ExecStartPreloads the firewall, then runs the check.pc-agents check --enforceevery 2 minutes. Its service hasExecStopPost, which stops T3 whenever the check did not succeed, including when it cannot start or crashes. It is orderedAfter=t3code-agents.service(a check never cancels T3's start at boot) and hasTimeoutStartSec=5min(a hung check fails, and so stops T3).sbin/pc-agents check [--enforce]runs the fleet's merged WSL2 walls plus the kit's own walls:record);user@<uid>.servicenow among them;lingeranduser-units;drop-ins: every folder systemd reads, including prefix drop-ins;units:systemctl show, and the timer is running (ActiveState), not only enabled;agent: the only login user, no other group, login shell/usr/sbin/nologin;sudoers: no rule in/etc/sudoersor/etc/sudoers.dnames the agent, its group, its uid orALLusers;kit-code:/,/opt,/opt/pc-agents, the link and the release folder;runtime: T3 is root's, and its hash is in the record;automount, with a trailing-slash prefix;firewall: the loaded table is the file's;gateway: blocked, and noalloworpeersentry covers it.pc-agents load-firewallloads the file. It then saves the file's sha256 and whatnft listshows in/run/pc-agents/firewall.loaded, so the check can compare against both.apps/server/src/policy/managedSettings.forkKeys.test.tsproves that the managed policy locks fork settings.A2: the locks bind the agent (decision: root-owned runtime)
T3 is installed by root under
/opt/pc-agents/t3. Part 2 does this with upstreamscripts/install.shandT3CODE_HOME=/opt/pc-agents/t3; the script checks the release's SHA256SUMS and runst3 --versionas root. The unit runs it from there. Theruntimewall checks two things:t3's sha256 is the one setup recorded.The agent cannot swap in a T3 that ignores the policy, so the locks are not merely advisory. The README Limits say what is left: an agent can still start its own programs inside the unit (even its own copy of T3). Those ignore the policy but stay inside every other wall.
What can be locked
lockedSettingsnow takes:agentSlots);providerInstances.codex-work; ids may contain-).The kit refuses what is cheap to tell:
organization.T3 refuses the rest (a partial object, a field inside a nullable setting) and does not start. T3 exits 1 on a policy it cannot apply, the same code as any other failure, so
RestartPreventExitStatus=cannot single it out. Instead the unit gives up after 5 starts in 10 minutes (StartLimitBurst=5).Runs again, stops half-way, meets old or empty state
--enforcestops the unit andload-firewallrewrites its saved file only when it changed. Two runs give the same answer.currentis brokenExecStopPoststops T3 (testCrash.test_missing_lib_stops_t3). At T3's start,ExecStartPrefails, so T3 does not start.systemctl stopfailsnft -fand savingfirewall.loadedfirewallwall fails and T3 is stopped or not started. The nextload-firewall(at T3's start) fixes it.firewallwall fails within 2 minutes and T3 is stopped. The unit'sIPAddressDeny=holds meanwhile.firewallwall fails untilload-firewall(testtest_firewall_file_newer_than_the_loaded_table).runtimewall passes, because there is nothing to wall:ExecStartdoes not exist, so T3 cannot start. A folder that is present but changed fails.file:t3code-agents.servicediffers, so T3 does not start until part 2's setup runs again. The old runtime in the agent's home is left alone and unused.peersFAIL check) untilpeersis added.FAIL check./usr/lib, asystem.controlor prefix drop-in, a disabled timer, a pending reloaddrop-insorunitsfails.lingeroruser-unitsfails. The agent's user manager is masked anyway.unitsfails (ActiveState), seen at T3's next start./bin/bashagentfails until part 2's setup runs again (it setsnologin).devPorts4100-4199applies; the unit and firewall change, so setup restarts T3 once.After=), then checks; it no longer fails on a table not loaded yet and cancels T3's start.TimeoutStartSec=5minfails the service, andExecStopPoststops T3.systemctl start t3code-agents.service).General vs project
General: any WSL2 PC, any repo. Every machine fact comes from the machine file the using project's owner writes:
peers;The README and the tests use made-up values.
The CI runner change (
.github/scripts/fork-ci-tests.cjsruns the fork's test files in parallel) is general: it has no project facts, andFORK_CI_TEST_JOBSsets how many run at once.Upstream fit
Built on upstream mechanisms:
scripts/install.sh(T3CODE_HOME,T3CODE_VERSION,T3CODE_RELEASE_BASE_URL) for the root install in part 2;T3CODE_PORT/T3CODE_LAN.It uses its own system unit instead of
t3 service install, because that makes a user unit the agent could override.Tests
python3 -I -B fork-kit/pc-agents/tests/check.test.py: 28 tests, on a fake distro. The fakesystemctlkeeps which units are enabled and running and answersshowandis-active.managedSettings.forkKeys.test.ts: 5 tests, unchanged.node --test .github/scripts/fork-ci-tests.test.cjs: 17 tests, none of them timed (they use marker files and promises, not clocks); one fails when a listed shell test pipes intogrep -q,grep -morhead, which races with SIGPIPE under pipefail. The fork's script tests now run in parallel, because one after another they no longer fit the CI job's 10-minute limit.Size and split
Non-test code here is now 555 lines (
lib/pc_agents.py380,sbin/pc-agents175), which is over the 400-line guide for one PR. Fix rounds 1 and 2 added about 220 lines of walls the reviews asked for. A further split would cut the walls apart from their check, so it stays one PR; part 2 is separate.🤖 Generated with Claude Code