Skip to content

fork fleet: desk setup pack kit (D8) - #225

Merged
scuttletoby merged 10 commits into
mainfrom
fork/desk-d8-setup-pack
Nov 16, 2026
Merged

scuttletoby merged 10 commits into
mainfrom
fork/desk-d8-setup-pack

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Row D8 of the desktop pool plan: a setup pack for a desktop that runs fork fleet jobs over the desk link (D1-D7).

Why

Today a desktop joins a desk pool through about twenty hand steps in the runbook (launcher, walls, desk agent, gate
hook, runner users, settings files, key, units). That is slow and easy to get subtly wrong (a missing wall, a
foreign launcher, a lingering user). This kit does all of it from one release commit, checks every part with one
PASS/FAIL list, and removes it again.

  • fork-kit/desk/bootstrap.sh: same pattern as the other kits. It checks the sha is on the protected ci-release
    branch, downloads that commit and runs the installer from it.
  • fork-kit/desk/sbin/desk-install:
    • install: reads the desk section of the machine file (pool, always-on host and its host key, slots,
      reserve). It makes fork-runner-N users with no login, no linger, and no cron or at. It stages the release's
      fleet files, tests them, and moves them into /opt/fork-fleet. Then it writes the settings files and makes the
      desk key once. Last it writes the units, starts the refresh timer (the first template is built in the
      background) and starts the agent.
    • check: one PASS/FAIL line per part, including every wall of this OS (fork-fleet-walls.py describe). It
      only reads.
    • uninstall: stops every runner (launcher, then systemctl stop); while one still runs it stops there and keeps
      the nft table, users and record. Then it removes every fork-fleet-* namespace and ffh-* link, the nft
      table, and only what the kit made: its units, the files of its manifest, template, settings, key, its users and
      the deny files it created. Anything else in its folders is kept and named; the record goes last.
  • The desk section in the example machine file, a row in the kit table, a pointer in the runbook.

Not here: the Windows starter (D9), the "free" script (D10), seeds (D11), the fork's own pool switch (D12).

What happens when this runs again, stops half-way, or meets existing, old or empty state?

  • Runs again: files, settings and units that are the same are not rewritten, no user is made, the key is
    kept, and the agent is neither stopped nor restarted (a test checks no systemctl restart is run). After a change
    to a file, setting, key or unit the agent is restarted. A test compares the whole fake root before and after.
    The agent reads its settings only when it starts, so the record gets a restart-pending mark before the first
    change and loses it after the restart: a run stopped in between (or refused by the agent's settings check)
    restarts the agent on the next run, and check fails until then.
  • Stops half-way: a test stops the install after each of its six steps on a fresh root and then runs it again.
    The result equals a full install every time.
    • A new release is staged in /opt/fork-fleet/.kit-stage-* and tested first: the walls know this OS, the gate
      loads with the fleet config, the pool is in it, bash -n, and the agent's --help. A release that fails is
      never moved in. The next run removes a leftover stage folder.
    • Changed files are renamed in while the agent and the refresh timer are stopped. MANIFEST.sha256 and then SOURCE are written last,
      so check shows a half-done swap.
    • Users are written to the record before useradd, so uninstall still removes a user from a run that stopped
      right there.
    • The key is made under a temporary name and renamed (public half first). A private key without its public half
      is refused with a fix.
    • The fleet config choice (and the project's copy, /etc/fork-fleet/fleet-config.json) is recorded before the
      swap, so a run stopped right after the swap keeps what was asked for.
    • A stopped uninstall keeps its record until the very end (removed after everything else), so running it again
      finishes; a runner that will not stop keeps the nft table, users and record in place.
  • Existing / old state: a launcher, launch.conf, desk.conf, desk key, unit or systemd drop-in for one of
    the kit's units that the kit did not write (an older hand install) is refused before anything changes; check
    fails on a drop-in that appears later, and on an installed unit that is not the release's (header plus
    /opt/fork-fleet/units/UNIT). The kit never adopts or overwrites it. Type-wide service.d//timer.d/ drop-ins
    are not checked: they are the distribution's defaults for every unit and only root can write them. Users that
    already exist are used but not recorded, so uninstall leaves them (their namespaces and links are still removed);
    the cron/at deny lines the kit added for them are recorded and removed. A user that a smaller slots no longer
    uses stays (no login, unused) and check names it; uninstall removes it.
    A record of another shape is refused, not trusted. An upgrade removes files the old release had
    and the new one has not.
  • Empty state: check with nothing installed lists FAILs and writes nothing. uninstall with no record does
    nothing and exits 0. An OS with no walls file (walls exit 3) is refused before anything is written.

General vs project

General. The kit names no project, person or machine. Pool, host, host key and slots come from the private
machine file on that desktop. The pools come from the release's fleet config or a project's --fleet-config,
which is remembered for later upgrades (--release-fleet-config goes back to the release's own). A
--fleet-config in a folder that others or a runner user may write is refused. Example values are made up (fleet-host.example).

Upstream fit

This touches only fork-only folders (fork-kit/, .github/fork-*). Nothing in upstream T3 changes.

Size

After fix round 1 the PR has about 765 added lines that are not tests (installer, bootstrap, README, example),
over the 400 guideline; round 1 added about 180 installer lines (uninstall that removes only its own, drop-ins,
restart only on change, settings check). Splitting off check or
uninstall would leave a PR that cannot be tested on its own state rules.

Layout differs from the other kits on purpose (README, "Why one flat folder"). The desk agent refuses a launcher or
walls path with a link in it, so there is no "versions + one link". Files are renamed into /opt/fork-fleet and
recorded in a manifest instead.

Tests

fork-kit/desk/tests/install.test.py (listed in .github/fork-ci-tests.txt) has 43 tests (round 4). They run on a fake
root in a temporary folder, with fake useradd, systemctl, ssh-keygen, ip, nft and so on, and stub fleet
files. Nothing touches the real system, and no root is needed. Beyond the first 19 (fresh install, re-run, a stop
after each step, upgrade, refusals, bad releases, keys, existing users, check, uninstall, bootstrap arguments),
round 1 added: a changed setting restarts the agent (and a plain re-run does not), drop-ins refused at install and
failed by check, settings that the machine file no longer gives, uninstall keeps what the kit did not make, removes
the deny files it made, keeps the table while a runner still runs, removes every runner namespace, removes its
record last, refuses a record of another shape, --release-fleet-config, and bootstrap flags after --fleet-config.

Run with python3 -I -B -W error::ResourceWarning fork-kit/desk/tests/install.test.py.

Round 2 added: a change written by a killed run still restarts the agent (and check fails until then), a refused
agent check keeps the restart, the fleet config choice outlives a run killed right after the swap (both ways),
check fails on a unit that is not the release's, check names a user left by fewer slots, uninstall removes the
deny lines it added for users that were there, a fleet config others may write is refused, and the bootstrap test
now gives a good --sha/--repo before --fleet-config.

Local run (Python 3.12) at round 2: 33 tests, OK (round 3: 39; round 4: 43); fork-kit/common/tests/machine_file.test.py 17 OK.

Red-checks

Each rule below was broken on purpose in a copy of the installer, one at a time, and the named test failed (all red;
the full run is OK again after the restore):

  • the user is recorded before useradd -> test_a_user_made_by_a_run_that_died_is_removed_by_uninstall
  • unchanged files are left alone -> test_runs_again_and_changes_nothing
  • a foreign install is refused -> test_refusals_change_nothing
  • a bad release is never moved in -> test_bad_release_is_not_installed
  • uninstall removes only its own users -> test_existing_user_is_used_not_recorded
  • a half key is refused -> test_key_is_made_once_and_a_half_key_is_refused
  • the OS check comes before any write -> test_refusals_change_nothing
  • an agent that refuses its settings starts nothing -> test_agent_refusing_its_settings_starts_nothing

Round 1: the new test file run against the round-0 installer: 15 tests fail, among them every round-1 MUST test
(drop-ins, re-run without restart, uninstall keeps what it did not make, deny files, table kept while a runner runs,
every namespace, record last, record shape, bootstrap flags). The record-last test lists folders by name, so a
whole-folder removal fails it whatever order the file system gives.

Round 2: the new test file run against the round-1 installer: 8 tests fail (restart after a killed run, restart
after a refused agent check, unit check, fewer slots, fleet config others may write, the fleet config choice both
ways, deny lines for users that were there); the bootstrap test fails on the round-0 bootstrap.

Round 3

  • A --fleet-config link: the folders of the link and of the file it names are both checked.
  • A hand-made fork-runner-N is used only when it is locked (no login shell, locked password, not in an allow file): install refuses otherwise, check FAILs.
  • check looks at file, folder and desk-key modes; install puts a changed mode or owner back for files in /opt/fork-fleet and folder modes (round 4 adds settings, units and the refusals below).
  • One install or uninstall at a time (a lock; round 4 moves it to /etc/fork-fleet).
  • A kill between saving a new fleet-config choice and the swap now fails check; an old record naming a project copy that is gone gets a readable refusal.
  • Still documented, not changed: type-wide service.d//timer.d/ drop-ins; a user left by a smaller slots until uninstall.
  • Tests: install.test.py 39 OK.

Round 4

  • A /opt/fork-fleet, /opt/fork-fleet/template, /etc/fork-fleet, desk key or its public half that is a link or not owned by root is refused before anything changes (lstat; not followed, not fixed with chown, which would follow a link). check says which fix.
  • A hand-made runner user must also have a uid other than 0, its own group only (id -G equals id -g, as the launcher's check_users; round 5 also wants the main group to be the user's own) and no home that is there (/nonexistent, empty or missing).
  • The lock is on /etc/fork-fleet (0700, opened with O_NOFOLLOW): no local user can hold it.
  • Settings and units: a link, a changed mode or an owner other than root fails check, and install writes the file again as root's.
  • Tests: install.test.py 43 OK. The round-4 test file against the round-3 installer: 15 failures (every new test and refusal case, plus the moved lock and the reworded mode lines).

Round 5

  • A folder inside /opt/fork-fleet (units, fork-fleet-walls, and any other the installed manifest names) that is a link or not owned by root is refused before anything changes, like the top folders.
  • "Its own group only" now means it: the main group (passwd gid) must be the group named after the user (getent group USER), and no other group. A user made with useradd -g 0 is refused by install and FAILs check. The launcher's check_users still checks uid 0 and one group only; the gid-0 case is caught by install and check.
  • check also looks at desk-key.pub (link, owner, mode 0644); install sets its mode to 0644.
  • A unit or setting path that is a FIFO is never opened (it hung install and check): check FAILs, install writes a setting again; a FIFO at a unit path is refused as not the kit's.
  • Not checked, documented in the README: the system folders above (/opt, /etc, /etc/systemd/system). They are root's on a stock system, and a user who could write them could already change what root runs.
  • Tests: install.test.py 47 OK (round 6: subfolder closed before files go in). The round-5 test file against the round-4 installer: 8 failures (subfolder link and owner for units and fork-fleet-walls, the two new refusal cases for subfolders, gid 0 in check and in install, linked public key, FIFO).

🤖 Generated with Claude Code

fork-kit/desk: bootstrap, install, PASS/FAIL check and uninstall for a
desktop's side of a desk pool (runner users, launcher, walls, desk agent,
gate hook, settings, key, units), from one release commit. Safe to run
again and to stop at any step; refuses an install it did not make;
uninstall removes only what it made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Review round 1 at 04a398d: two independent lenses (security, correctness), both SAFE WITH NITS. Fix round to follow on this branch.

Security (lens225s): no way out for a hostile job as fork-runner-N; bootstrap trust, root-owned paths, nologin users, machine-file patterns and the stop-half-way cases all hold. Findings:

  1. desk-install:302,414,435-437: check misses systemd drop-ins and uninstall leaves them. An old fork-fleet-desk-agent.service.d/override.conf with its own ExecStart= gives ALL PASS while systemd runs the override. Refuse at install, or compare systemctl cat with the kit's unit text. (proven)
  2. :441-442,449: uninstall deletes all of /etc/fork-fleet and /var/lib/fork-fleet, including files the kit did not make (e.g. a token). The PR body says it removes only its own. (proven)
  3. :242-247,446-448: at.deny is created and never removed. With no at.deny and no at.allow, only root may use at; creating at.deny opens at to every other local user, and that stays after uninstall. Remember which deny files the kit created and delete them. (proven)
  4. :428-434: uninstall ignores active/stop failures and deletes the fork_fleet nft table anyway; a runner that did not stop keeps routed network with no filter (ip_forward stays 1). Stop fork-fleet-* units via systemctl as a fallback; keep the table until none is active.
  5. :212: the refresh timer keeps running during the file swap (only the agent is stopped).
  6. :181-184: a malformed pools value (dict) hits the suppressed error types.
  7. :305: no way back to the release's own fleet config once a project config was used.

Correctness (lens225c): install/check/uninstall agree; 19 tests pass and CI lists them. Findings:

  1. :338: a plain re-run always runs systemctl restart, though README and body say the agent is not stopped. With KillMode=control-group this kills an in-flight start (host gets 75). Restart only when files, settings or units changed. (proven)
  2. :432-433: uninstall removes namespaces only for createdUsers; a hand-made fork-runner-1 keeps fork-fleet-fork-runner-1 and its ffh-* link. (proven)
  3. :449: rmtree(ETC) can delete desk-kit.json before launch.conf; killed there, the next install refuses launch.conf as foreign and uninstall says "nothing installed". Remove the rest, then the record, then rmdir.
  4. Same as security 2 (README vs whole-folder removal).
  5. bootstrap.sh:163,197: args after --fleet-config pass through unchecked, so a later --sha wins and skips the ci-release check; --fleet-config before --repo gives a misleading error. Parse it like the other flags.
  6. machine.example.json/README use pool desktop, which the release config does not have; use pc or say the pool must exist.
  7. :305,432: a record of the wrong shape crashes with KeyError instead of a Refused.
  8. :374-377: check does not notice the machine file and the written settings disagree.

Both lenses ran the state question (runs again, stopped at each step, old/foreign/empty state); everything not listed above is handled.

scuttletoby and others added 3 commits November 15, 2026 20:45
- install refuses a systemd drop-in it did not write for its units; check fails on one
- re-run restarts the agent only when a file, setting or key changed
- check compares the settings files with the machine file
- uninstall stops every runner, keeps the nft table, users and record while one
  still runs, removes every fork-fleet-* namespace and ffh- link, removes only
  what the kit made (deny files it created, its users' deny lines), names what
  it leaves, and removes its record last
- the record is read only when it has this kit's shape
- --release-fleet-config goes back to the release's fleet config
- bootstrap parses --fleet-config like the other flags

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e ip (D8)

Uninstall now deletes the namespaces ip netns list shows, not one per
recorded user; the older test listed none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A whole-folder removal took the record first only for some directory
orders, so the test passed on the old code by luck. Listing by name puts
the record first, which the old order fails every time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1 (commits 5f64eb9, 7e73543, 99bf235; head 99bf235). Tests: install.test.py 27 OK, machine_file.test.py 17 OK.

Security (lens225s):

  1. Drop-ins: fixed (5f64eb9). Install refuses a drop-in it did not write for any of its units (UNIT.d/*.conf, also the dash-prefix forms); check fails on one that appears later. Tests: the refusal cases in test_refusals_change_nothing, test_check_fails_on_a_drop_in_and_on_settings_the_machine_file_no_longer_gives.
  2. Uninstall deletes whole folders: fixed (5f64eb9). It removes the files of its manifest, template, runner copies, settings and key only; anything else (a token, a drop-in) is kept and named. Test: test_uninstall_keeps_what_the_kit_did_not_make.
  3. at.deny left behind: fixed (5f64eb9). The record names the deny files the kit created; uninstall deletes those, and only removes its users' lines from ones that were there. Test: test_uninstall_removes_the_deny_files_it_made.
  4. nft table removed while a runner runs: fixed (5f64eb9). Uninstall stops through the launcher, then systemctl stop for any fork-fleet-* unit left; while one is still active it stops there and keeps the table, users and record. Test: test_uninstall_keeps_the_table_while_a_runner_still_runs.
  5. Refresh timer during the swap: fixed (5f64eb9): the timer and refresh service are stopped with the agent.
  6. Malformed pools: fixed (5f64eb9): a pools that is not a list of objects is a clear refusal.
  7. No way back to the release's config: fixed (5f64eb9): --release-fleet-config (tested in test_a_changed_setting_restarts_the_agent).

Correctness (lens225c):

  1. Re-run always restarts: fixed (5f64eb9). Restart only after a file, setting or key changed. Tests: test_runs_again_and_changes_nothing (no systemctl restart), test_a_changed_setting_restarts_the_agent.
  2. Namespaces only for created users: fixed (5f64eb9). Every fork-fleet-* namespace and ffh-* link is removed. Test: test_uninstall_removes_every_runner_namespace.
  3. Record removed before the rest: fixed (5f64eb9): record last, then the empty folders. Test: test_uninstall_removes_its_record_last.
  4. Same as security 2: fixed; README says what uninstall removes and keeps.
  5. bootstrap --fleet-config: fixed (5f64eb9): parsed like the other flags; a later --sha is checked. Tests in the bootstrap argument test.
  6. Pool desktop in the example: fixed (5f64eb9): pc, and README says the pool must be in the installed config.
  7. Record of another shape: fixed (5f64eb9): refused, not trusted. Test: test_uninstall_refuses_a_record_of_another_shape.
  8. check vs machine file: fixed (5f64eb9): a check row compares the written settings with the machine file (same test as 1).

Fails before the fix: the new test file against the round-0 installer: 15 tests fail (every MUST test above). Two follow-up test commits: 7e73543 (the uninstall test gives its namespace through the fake ip), 99bf235 (the record-last test lists folders by name, so it fails on the old order every time).

PR body updated (uninstall, re-run, drop-ins, size, test count). Still draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Re-check lens 225c r1 (correctness) at 99bf235: SAFE WITH NITS

Earlier findings: S1, S2/C4, S4, S5, S6, S7, C1, C2, C3, C5, C6, C7, C8 FIXED (new tests fail on round-0: 14 fail + 3 errors); S3 at.deny PARTLY (finding 2). 27 tests pass at head; the new test file is in .github/fork-ci-tests.txt:45.

New:

  1. SHOULD FIX, round-1 regression: a stopped run loses the restart (desk-install:373-392). changed lives only in memory; the agent reads desk.conf only at start (fork-fleet-desk-agent.py:441). The machine file's host changes, install writes desk.conf, then stops before :392 (Ctrl-C, failing sysctl -p at :375, agent refusing settings at :383, failed daemon-reload). The re-run sees the settings unchanged and only runs systemctl start (a no-op): the agent keeps dialing the old host and check still says ALL PASS. Fix: save a pending-restart flag in the record before writing, clear it after the restart.
self.assertEqual(self.install()[0], 0)
self.machine({'version': 1, 'desk': {**MACHINE['desk'], 'host': 'other-host.example'}})
self.install(stop_after='settings')
self.clear_calls(); self.assertEqual(self.install()[0], 0)
self.assertIn('systemctl restart fork-fleet-desk-agent.service', self.calls())   # fails
  1. Nit (security-shaped): at.deny stays after uninstall when a runner user already existed (:292-294 vs :562-565): install creates /etc/at.deny with both runner users, uninstall strips only the user it made, the file stays, so at is open to every other local user. Remove the lines the kit added for all runner users.
  2. Nit: the fleet-config choice is saved after the swap (:369-371); killed after --release-fleet-config swaps but before save_state, the record still says project and the next plain install keeps the stale release config forever. Save fleetConfig before the swap.
  3. Nit: the bootstrap test case at install.test.py:613 passes on round-0 too (sha stays empty there); give a valid --sha before --fleet-config and a bad one after.

State question: re-run (:140 skip), half-way (:189, :276, :313-321, record last :571, stuck runner :533; gap = finding 1), old/foreign state (:347-355, :131), empty (:466, :522). Fakes can't show a real systemctl show DropInPaths on WSL, nor that a running agent keeps its start-time settings (what hides finding 1).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Re-check lens 225s r1 (security) at 99bf235: SAFE WITH NITS

No way out of fork-runner-N for a hostile job and no way in from the network. S1-S7 and C1-C8 FIXED (round-1 tests on the round-0 installer: 14 fail + 3 errors; head: install 27 OK, machine_file 17 OK). The part of S1 asking check to compare unit text is NOT FIXED (finding 2).

New:

  1. A killed install loses the restart (same as lens 225c r1 finding 1): :369-392 sets changed only from this run's writes; the agent reads desk.conf once at start. Fix: a restart-pending flag in the record, or restart when the agent's ActiveEnterTimestamp is older than settings, key or units.
  2. check passes with a stale or edited unit (:482 only tests the kit header). An upgrade killed after settings leaves release B in /opt and A's units in /etc/systemd/system; after a reboot check says ALL PASS for release B while systemd runs A's unit. Fix: compare each unit with HEADER + /opt/fork-fleet/units/<unit> (optionally systemctl show -p FragmentPath).
  3. Nit: drop_ins skips the type-wide service.d/ and timer.d/ (:183); root-only paths.
  4. Nit: when slots shrinks, the extra fork-runner-N stays and check stops looking at it (:438); uninstall still removes it.
  5. Nit: --fleet-config ownership is not checked (bootstrap.sh:37, :356); only matters if the owner picks a job-writable path.

Probe tests (fail at head):

def test_a_change_written_by_a_killed_run_still_restarts_the_agent(self):
    self.assertEqual(self.install()[0], 0)
    self.machine({'version': 1, 'desk': {**MACHINE['desk'], 'host': 'new-host.example'}})
    self.assertIn('stopped after', str(self.install(stop_after='settings')[0]))
    self.clear_calls(); self.assertEqual(self.install()[0], 0)
    self.assertIn('systemctl restart fork-fleet-desk-agent.service', self.calls())
def test_check_fails_when_a_unit_is_not_the_release_unit(self):
    self.assertEqual(self.install()[0], 0); self.template()
    new = self.release('release2', {'.github/scripts/fork-fleet-desk-agent.service': '# v2\n[Service]\n'})
    self.install(sha=SHA2, source=new, stop_after='settings')
    self.assertEqual(self.run_main('check')[0], 1)

Checked and holding: bootstrap runs only curl + stdlib before the sha check (merge base, protected branch, no redirects, python3 -I); /etc/fork-fleet 0700, key 0600; runner users with no shell, no linger, cron/at denied, and the launcher refuses uid 0/extra groups/linger; symlink-safe rmtree after runners stop.

…et config) (D8)

- A restart-pending mark in the record, saved before the first change the
  agent reads (files, settings, key, units) and cleared after the restart: a
  run stopped in between restarts the agent on the next run; check names it.
- check compares each installed unit with the header plus the release's
  /opt/fork-fleet/units/UNIT.
- The record lists every deny line the kit added (also for runner users that
  were there before); uninstall removes them all.
- The fleet config choice (and the project's copy in
  /etc/fork-fleet/fleet-config.json) is saved before the swap.
- --fleet-config is refused in a folder others or a runner user may write.
- check names a kit-made runner user that a smaller slots no longer uses.
- The bootstrap test gives a good --sha/--repo before --fleet-config, so it
  fails on the round-0 bootstrap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 2 at 4a13be5 (one commit on top of 99bf235)

Lens 225c r1 (correctness) + 225s r1 (security):

  • 225c 1 / 225s 1, a killed install loses the restart -> fixed (4a13be5). The record gets restartPending: true before the first change the agent reads (swapped files, settings, key, units) and loses it only after systemctl restart worked. check fails while it is set. Tests: test_a_change_written_by_a_killed_run_still_restarts_the_agent (the lens probe, plus the check line and "not again"), test_a_refused_agent_check_keeps_the_restart_for_the_next_run.
  • 225s 2, check passes with a stale or edited unit -> fixed (4a13be5): each unit must equal the header plus /opt/fork-fleet/units/UNIT. Test: test_check_fails_when_a_unit_is_not_the_release_unit (the lens probe, then a full install passes again).
  • 225c 2, at.deny stays for a runner user that already existed -> fixed (4a13be5): the record lists every [deny file, user] line the kit added, saved before the write; uninstall removes them all (older records: the users it made, as before). Test: test_uninstall_removes_the_deny_lines_it_added_for_users_that_were_there.
  • 225c 3, fleet-config choice saved after the swap -> fixed (4a13be5): the choice is saved before the swap, and the project's config is kept in /etc/fork-fleet/fleet-config.json (written from the staged copy, so a refused config changes nothing), so a run killed right after the swap keeps the choice both ways. Test: test_the_fleet_config_choice_outlives_a_run_killed_after_the_swap (two subtests).
  • 225c 4, the bootstrap test passes on round 0 -> fixed (4a13be5): a good --sha/--repo before --fleet-config and a bad one after; fails on the round-0 bootstrap ("run this as root" instead of "--sha must be").
  • 225s 3 nit, type-wide service.d//timer.d/ -> documented (README, PR body): they hold the distribution's defaults for every unit (refusing them would refuse a stock Fedora), and only root can write them.
  • 225s 4 nit, slots shrinking -> fixed in part (4a13be5): check names a kit-made user that is no longer a slot ("no longer a slot ... uninstall removes it"). Install does not remove it, because a runner may still be running as it; documented. Test: test_check_names_a_runner_user_left_by_fewer_slots.
  • 225s 5 nit, --fleet-config ownership -> fixed (4a13be5): refused when the file or a folder above it is writable by others (sticky folders like /tmp are fine) or by a runner user's group, or owned by a runner user. Test: refusal case "a fleet config others may write".

State question: runs again (a cleared mark gives start, not restart; test), stops half-way (mark saved before the first write; choice saved before the swap), old state (a round-1 record without addedDenyLines falls back to the users it made; a round-1 project record without the copy reads /opt/fork-fleet/fork-fleet.json), empty (fresh record has the new keys).

Red-check: the new test file against the round-1 installer: 8 fail (both restart tests, unit check, fewer slots, open fleet config, fleet config both ways, deny lines). Head: install.test.py 33 OK.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check r2 (lens225cr2), read at: 4a13be5 — SAFE WITH NITS

Earlier findings

Tests (python only): install.test.py 33 OK, machine_file.test.py 17 OK at head; head tests against the round-1 installer give exactly the 8 claimed failures. .github/fork-ci-tests.txt:45 runs the new file. machine.example.json is neutral.

New findings (nits)

  1. Symlink passes the ownership check (desk-install:101): it walks os.path.abspath, the install reads the real file (:400). A root-made link /root/f.json -> /srv/open/f.json (0777, not sticky) passes. Fix: os.path.realpath.
    def test_probe_symlink_to_open_folder(self):
        os.makedirs(os.path.join(self.tmp, 'open'), exist_ok=True); os.chmod(os.path.join(self.tmp, 'open'), 0o777)
        self.write(os.path.join(self.tmp, 'open', 'f.json'), json.dumps({'pools': [{'name': 'desktop'}]}))
        os.makedirs(os.path.join(self.tmp, 'safe'), exist_ok=True)
        os.symlink(os.path.join(self.tmp, 'open', 'f.json'), os.path.join(self.tmp, 'safe', 'link.json'))
        self.assertEqual(self.install('--fleet-config', os.path.join(self.tmp, 'safe', 'link.json'))[0], 1)  # 0 at head
  2. check passes after a run killed between saving the choice and the swap (:431 -> :434): --fleet-config A, then --fleet-config B killed before the swap leaves A in place with no restart mark; check says ALL PASS until the next install. Fix: when the record says project, check compares /etc/fork-fleet/fleet-config.json with /opt/fork-fleet/fork-fleet.json.
  3. The runner-user branches of the ownership check (:95-100, :107) read the real user list; fakes only model id, so they are untested.
  4. Old record project with no project copy and no installed copy (:403 -> stage :237) fails with a raw Python error, not a message naming the fix (also true in round 1).

State question: runs again: no mark means start (:457), unchanged files not rewritten (:285, :423). Stops half-way: users, deny files and lines recorded before made (:312, :325, :330); choice before swap (:431); mark before any write; staging folder swept (:224); record removed last on uninstall (:655); only gap is nit 2. Old state: new keys filled (:150-151), wrong shape refused (:166), foreign units/drop-ins refused (:395-399). Empty: check before install writes nothing (:538), uninstall without a record does nothing (:604). Fakes can't show the real agent reading desk.conf only at start, or real DropInPaths output.

verdict SAFE WITH NITS, PR #225, head 4a13be5

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check r2 (lens225sr2), read at: 4a13be5 — SAFE WITH NITS

Tests at head: install.test.py 33 OK, machine_file.test.py 17 OK; the head test file against the round-1 installer (99bf235) fails exactly the 8 tests fix round 2 names.

Earlier findings

  • Restart lost after a killed run (225c 1 / 225s 1): FIXED. restartPending saved before any change (desk-install:415-420), set before the swap (:288), each changed setting/unit (:422-425) and a new key (:352); cleared only after the restart works (:457-458); check fails while set (:553). Both tests fail on round 1.
  • check passed with a stale unit (225s 2): FIXED (:555-564, unit file checked against the manifest; unloaded unit caught by the mark). Test fails on round 1.
  • at.deny lines for pre-existing users (225c 2): FIXED (:330-333, removed :645-649). Test fails on round 1.
  • Fleet config choice after the swap (225c 3): FIXED (:427-433, copied from the checked staging copy). Both subtests fail on round 1.
  • Bootstrap test passing on round 0 (225c 4): FIXED per the PR (not re-run against round 0).
  • Distribution-wide service.d/ / timer.d/ (225s 3): documented; root-only, acceptable.
  • Users left when slots shrinks (225s 4): PARTLY; check names it (:515-518), uninstall removes it; no-login until then, acceptable.
  • --fleet-config writer check (225s 5): PARTLY, see 1.

New findings

  1. Nit, proven: a --fleet-config symlink skips the write check (desk-install:101 vs :400). job_writable walks abspath folders; the installer reads the realpath. /root/f.json -> /srv/open/f.json (world-writable, no sticky bit) passes, and the file can be swapped before stage() copies it (:412). Owner-chosen path, low risk. Fix: job_writable(os.path.realpath(a.fleet_config)).
    def test_probe_fleet_config_link_into_an_open_folder_is_refused(self):
        open_dir, safe = os.path.join(self.tmp, 'open'), os.path.join(self.tmp, 'safe')
        os.makedirs(open_dir); os.chmod(open_dir, 0o777)
        self.write(os.path.join(open_dir, 'f.json'), slurp(os.path.join(self.source, '.github/fork-fleet.json')))
        os.makedirs(safe); os.symlink(os.path.join(open_dir, 'f.json'), os.path.join(safe, 'f.json'))
        code, _, err = self.install('--fleet-config', os.path.join(safe, 'f.json'))
        self.assertEqual(code, 1)   # head: 0, the config is accepted
  2. Nit: a pre-existing runner user is used unchecked (:308-311). The launcher checks groups, linger and uid (fork-fleet-launch.sh:618-631), not shell or password; with cron.allow/at.allow present the deny step is skipped (:321) and allow-list membership is not checked. An old hand-made fork-runner-1 with /bin/bash and a password can still log in while check says ALL PASS. Fix: refuse, or FAIL in check, on a non-nologin shell, a usable password or an allow-file line.
  3. Nit: check ignores file modes (:541 hash + root owner only; not folders, the key's 0600, /etc/fork-fleet 0700). A root chmod 0666 on /opt/fork-fleet/fork-fleet-launch.sh lets a runner change root-run code and check passes; install does not reset modes of unchanged files (:285).
  4. Nit: two installs at once are not locked out; stage() deletes every .kit-stage-* (:224-226) including another run's, which then crashes mid-swap (next run repairs).

State question: runs again: unchanged files skipped (:285, :423), start without a mark (:457). Stops half-way: record first (:409), users before useradd (:312), deny lines (:330), mark (:415), choice before swap (:430), staging swept (:224), refused agent check keeps the mark (:448), uninstall removes the record last (:655) and stops if a runner is running (:615). Old state: hand install refused (:390-392), foreign units/drop-ins refused (:395-399), round-1 record works (:151, :403), other shapes refused (:157-166). Empty: check says not installed (:537), uninstall changes nothing (:604).

Holding: no way out found for a fork-runner-N job (nologin, no linger, cron/at denied, own group only; /etc/fork-fleet 0700, key 0600; uninstall stops runners first and does not follow symlinks; bootstrap uses only curl + stdlib until the sha is confirmed on protected ci-release; no redirects, Python -I).

verdict SAFE WITH NITS, PR #225, head 4a13be5

…ock, stopped choice) (D8)

- --fleet-config: the folders of the link and of the file it names are both checked.
- A runner user made by hand must have no login shell, a locked password and no
  cron.allow/at.allow line: install refuses otherwise, check FAILs.
- check looks at modes (files, folders, desk key); install puts a changed mode or
  owner back from the release.
- One install or uninstall at a time (flock on /opt/fork-fleet); a second is refused
  and does not sweep the first one's staging folder.
- The restart mark is set before a new fleet config choice is saved, and check
  compares the project's copy with the installed one: a run killed between the
  choice and the swap fails check.
- An old record that names a project config that is gone: a readable refusal.
- Tests for the runner-user branches of the ownership check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 3

Commit 8c7524f (head). Findings from lens225cr2 and lens225sr2:

  • cr2 1 / sr2 1, a --fleet-config link skips the write check: fixed (8c7524f). The folders of the link and of the file it names (realpath) are both checked; a link into an open folder is refused (test).
  • cr2 2, check passes after a kill between saving the choice and the swap: fixed (8c7524f). The restart mark is set before a new choice is saved, and check compares the project's copy with the installed /opt/fork-fleet/fork-fleet.json (test).
  • cr2 3, runner-user branches of the ownership check untested: fixed (8c7524f), tests with a fake user list.
  • cr2 4, old project record with no copy left: fixed (8c7524f), a readable refusal that names the fix (test).
  • sr2 2, a hand-made runner user used unchecked: fixed (8c7524f). It must have no login shell, a locked password and no cron.allow/at.allow line: install refuses before changing anything, check FAILs (tests).
  • sr2 3, check ignores modes: fixed (8c7524f). check looks at file, folder and desk-key modes; install puts a changed mode or owner back from the release (tests).
  • sr2 4, two installs at once: fixed (8c7524f). One install or uninstall at a time (a lock on /opt/fork-fleet); a second is refused and does not sweep the first one's staging folder (test).
  • Earlier "documented" items stay documented: type-wide service.d//timer.d/ drop-ins; a user left by a smaller slots (named by check, removed by uninstall).

State question: runs again: the lock is taken and let go per run; modes already right are left alone. Stops half-way: the mark is set before the choice is saved, so check FAILs until the next install finishes. Old state: an old record naming a missing project copy gets a message; a pre-existing user that is not locked is refused before any change. Empty: no lock folder before install means nothing to lock.

Tests (local, at 8c7524f): install.test.py 39 OK.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security + correctness re-check r3 (lens225r3), read at: 8c7524f — SAFE WITH NITS

No way found for a hostile job as fork-runner-N to leave its user, and no way in from the network. Tests at head: install.test.py 39 OK. Head tests against the round-2 installer (4a13be5): 13 failures + 1 error — every new test except the cr2 3 ownership-branch test (coverage-only, code unchanged, expected).

Earlier findings

  • cr2 1 / sr2 1, --fleet-config link skips the write check: FIXED (desk-install:125 checks the folders of the link and its target; test fails on round 2). Small gap: the folder of a middle link in a chain is not checked; only the owner picks that path, negligible.
  • cr2 2, run killed between choice and swap: FIXED (restart mark set first :488-489; check compares the project's copy with the installed config :625-630; all three subtests fail on round 2).
  • cr2 3, runner-user branches untested: FIXED (coverage-only, install.test.py:421).
  • cr2 4, old project record with no copy left: FIXED (readable refusal :456-460; test errors on round 2).
  • sr2 2, hand-made runner user used unchecked: FIXED (shell, locked password, allow-file at :92-111, enforced at install :461-463 and in check :578); gaps in nit 3.
  • sr2 3, check ignored modes: FIXED (files :604-613, folders and key :615-623; install puts back files :319-333, folders :466-467, key mode :505). Owner only partly, nit 1.
  • sr2 4, two installs at once: FIXED (flock at :406-414 taken before the staging sweep; :468 before :256).
  • Documented items still hold: type-wide service.d/ drop-ins; a user left behind when slots shrinks.

New findings (all nits)

  1. Install neither fixes nor refuses a non-root or linked /opt/fork-fleet, /etc/fork-fleet or desk key (proven), so check gives a fix that never works. :466-467 only makedirs+chmod (follows a link, no chown); :505 only sets the key's mode. Bad sequence: /opt/fork-fleet left as a link by a hand setup, or owned by uid 1000: install OK twice, the root-run launcher sits in a user-writable folder, check says FAIL "not root-owned … install again" forever. Detection works; fix: refuse at install (lstat: not a link, uid 0) or chown. Probe (fails at head):
def test_probe_opt_link_is_refused_or_fixed(self):
    other = os.path.join(self.tmp, 'elsewhere'); os.makedirs(other)
    os.makedirs(os.path.dirname(self.path('/opt/fork-fleet')), exist_ok=True)
    os.symlink(other, self.path('/opt/fork-fleet'))
    if self.install()[0] == 0:
        self.template(); self.assertEqual(self.install()[0], 0)
        self.assertNotIn('FAIL', self.run_main('check')[1])  # head: FAIL kit /opt/fork-fleet ... install again
  1. PR body overstates and is stale: line 133 "install puts a changed mode or owner back" is true only for files under /opt (nit 1); lines 88 and 105 say 33 tests, real count 39.
  2. user_problem (:92-111) is weaker than the launcher's check_users (fork-fleet-launch.sh:619-635): no uid 0, extra-group or home check. A hand-made fork-runner-1 in wheel or docker: install accepts, check ALL PASS, every job start dies "in other groups". A hand-made user with a real home and ~/.ssh/authorized_keys also passes; with UsePAM yes sshd ignores the ! lock, so ssh -N forwarding works (jobs cannot plant the key under ProtectHome in their own root, so only a gap when the owner set it up that way). README's "as locked as one the kit makes" overstates. Fix: also require its own group only, uid not 0, home /nonexistent or missing.
  3. Any local user can hold the lock: :409 opens world-readable /opt/fork-fleet (0755) and takes flock, so any local user can block install and uninstall forever. Jobs cannot (not mounted in their root). Locking /etc/fork-fleet (0700) closes it.
  4. Units checked for text and mode, not owner (:648, same_text); only root can change that, same class as sr2 3.

State question

  • Runs again: unchanged files skipped (:319-323), modes put back (:332-333, :467, :505), agent restarted only when the mark is set (:521), lock per run (:468).
  • Stops half-way: mark set before the choice (:488), choice saved before the swap (:493), record saved before users are made (make_users); staging sweep only under the lock (:256); uninstall removes the record last (:742).
  • Old or foreign state: an install the kit did not make is refused (:441-444); old project record with no copy refused with the fix (:456); unlocked hand-made user refused (:462). Gaps: linked or foreign-owned folder (nit 1), user in other groups (nit 3).
  • Empty: fresh record (:445); check before install writes nothing; uninstall without a record changes nothing (:689); no lock when /opt/fork-fleet is gone (:692).

Still holding: the bootstrap runs only curl and the standard library until the sha is confirmed on protected ci-release (merge base equals the sha, no redirects, python3 -I). Jobs run under their own RootDirectory, ProtectHome, ProtectSystem=strict, with /etc/fork-fleet 0700 and the key 0600. Uninstall stops the runners before removing anything.

verdict SAFE WITH NITS, PR #225, head 8c7524f

…, runner users as strict as the launcher, lock in /etc/fork-fleet, owner of units and settings) (D8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 4

Commit 1ccced0 (head). Findings from lens225r3:

  • Nit 1, a linked or non-root /opt/fork-fleet, /etc/fork-fleet or desk key: fixed (1ccced0). Install lstats each of them (and /opt/fork-fleet/template, the key's public half) before changing anything and refuses a link, another kind or an owner other than root, with the fix. Refused, not fixed: chown would follow a link. check now says "a link or not owned by root: remove the link (or chown root)". Tests: the lens probe as test_a_linked_or_foreign_owned_folder_or_key_is_refused (link and, with a faked lstat, another owner), plus three refusal cases (linked /opt/fork-fleet, /etc/fork-fleet, desk key) that check the tree is unchanged.
  • Nit 2, stale PR body: fixed (body: test counts per round, the "mode or owner back" line says which files).
  • Nit 3, user_problem weaker than the launcher's check_users: fixed (1ccced0). Also uid not 0, id -G equal to id -g (its own group only, as the launcher) and a home that is /nonexistent, empty or not there. Install refuses, check FAILs (tests: three refusal cases, test_a_hand_made_user_in_other_groups_or_with_a_home_fails_check). README no longer says "as locked as".
  • Nit 4, any local user can hold the lock: fixed (1ccced0). The lock is on /etc/fork-fleet (0700), opened with O_NOFOLLOW; uninstall always takes it (the record lives there). Test test_the_lock_is_in_the_root_only_folder (a user's lock on /opt/fork-fleet no longer blocks).
  • Nit 5, units checked for text and mode, not owner: fixed (1ccced0). One same_file (lstat: a plain file, root's, mode, text) serves write_file and check, so a linked or foreign-owned setting or unit FAILs check and install writes it again as root's (test test_a_unit_or_setting_with_another_owner_fails_check_and_install_puts_it_back).
  • Middle links in a --fleet-config chain: still not checked (lens: negligible, only the owner picks that path). Documented here.

State question: runs again: the new checks only read; a file already root's with the right text is left alone. Stops half-way: the refusals come before the first write, so a refused run changes nothing (refusal cases compare the whole tree). Old state: a hand setup with a linked folder or a user in other groups is refused with the fix instead of installed. Empty: nothing there passes the lstat checks (absent is fine).

Tests (local, at 1ccced0): install.test.py 43 OK. The round-4 test file against the round-3 installer: 15 failures.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security+correctness re-check r4 (lens225r4), read at: 1ccced0 — SAFE WITH NITS

No way found for a hostile job running as fork-runner-N to get out of its user, and no way in from the network. All 43 tests in install.test.py pass at head. Against the round-3 installer (8c7524f) the head tests give exactly the 15 failures fix round 4 claims, so every new test fails on the old code.

Earlier findings (lens225r3)

  • Nit 1, linked or non-root /opt/fork-fleet, /etc/fork-fleet or desk key: PARTLY. The five paths are checked with lstat before the first write (desk-install:499-502, helper :123-136); tests fail on round 3. The folders inside /opt/fork-fleet are not checked (finding 1).
  • Nit 2, stale PR body: FIXED.
  • Nit 3, user_problem weaker than the launcher: FIXED (:103-109); matches check_users (fork-fleet-launch.sh:627,630). Wording gap: finding 2.
  • Nit 4, any local user can hold the lock: FIXED. Lock on /etc/fork-fleet (0700), O_NOFOLLOW (:441-448); uninstall always takes it (:735). No released kit took the old /opt lock, so no stale old lock.
  • Nit 5, units not checked for owner: FIXED. One same_file check (:139-147) used by write_file (:241) and check (:690); its test fails on round 3.
  • Middle links in a --fleet-config chain: documented; negligible.

New findings

  1. SHOULD FIX (completes the round-4 MUST): a link or foreign owner one level down is neither refused nor fixed (proven). :499-502 checks /opt/fork-fleet and /template, not /opt/fork-fleet/units or /opt/fork-fleet/fork-fleet-walls. swap (:354-357) follows such a link (makedirs, os.replace, chmod all go to the target); check (:656) flags it, so the "install OK, check FAILs forever" loop round 4 closed for the top folder is back one level down.
    • Bad sequence: a hand setup leaves /opt/fork-fleet/units pointing at a folder another user can write. Install says OK twice. The root-run walls programs and unit texts now live there. On the next install put (:528-530) copies whatever text is there into /etc/systemd/system as a root unit. Only root can create the link, so a CI job cannot.
    • Fix: add every manifest subfolder (units, fork-fleet-walls) to the list at :499.
    def test_probe_linked_units_subfolder_is_refused_or_fixed(self):   # same with 'fork-fleet-walls'
        other = os.path.join(self.tmp, 'elsewhere'); os.makedirs(other)
        os.makedirs(self.path('/opt/fork-fleet')); os.symlink(other, self.path('/opt/fork-fleet/units'))
        if self.install()[0] == 0:   # head: 0, unit texts written into `other`
            self.template(); self.assertEqual(self.install()[0], 0)
            self.assertNotIn('FAIL', self.run_main('check')[1])   # head: FAIL kit /opt/fork-fleet/units: a link ...
  2. Nit: "its own group only" really means "one group only" (:105-106; launcher :630). A hand-made user from useradd -g 0 has id -g = id -G = 0; install accepts it, check says PASS, jobs run with gid 0. Small impact (NoNewPrivileges, own RootDirectory). Fix: require the passwd gid to be a group named after the user, or reword README/check/body. Probe: patch run so id -g/id -G print 0, then self.assertIsNotNone(self.m.user_problem('fork-runner-1')).
  3. Nit: check does not look at desk-key.pub. Install refuses a linked .pub (:500), but the check list at :653 has KEY only. Proven: a linked .pub still gives ALL PASS. Root only.
  4. Nit, tiny, not new: same_file opens the file before testing it is a plain file (:142-143); a FIFO at a unit or settings path hangs install or check. Root only.
  5. Nit: parents not checked (/opt, /etc, /etc/systemd/system). After a chown -R user /opt that user could swap /opt/fork-fleet; normally the owner, who has sudo anyway; runner users cannot write there. Mention in README or check with job_writable.

State question

  • Runs again: new checks only read (:499-506); a root-owned file with the right text is left alone (:241); lock per run (:518).
  • Stops half-way: every refusal (:499, users :503-506) comes before done('check') and the first write (:508). Restart mark and fleet-config choice as in round 3 (:524, :535-541). Uninstall takes the lock, removes the record last (:791).
  • Old or foreign state: linked top folder, /etc/fork-fleet or key refused (:499); a pre-existing user checked like a kit-made one (:503-506); linked subfolder is finding 1; a round-3 record needs nothing new.
  • Empty state: a missing path passes lstat (:127); uninstall without a record returns before the lock (:731-733).

Still holding: bootstrap unchanged (curl + stdlib check until the sha is confirmed as merge base on protected ci-release, bootstrap.sh:47-58, redirects off, Python -I). Jobs: NoNewPrivileges, own RootDirectory, ProtectSystem=strict, ProtectHome; key 0600 in 0700 /etc/fork-fleet.

verdict SAFE WITH NITS, PR #225, head 1ccced0

…main group named after the user, public key in check, no FIFO opened) (D8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 5 (head d4184a9)

Work list: lens225r4 (SAFE WITH NITS).

  • Finding 1 (SHOULD FIX), linked or foreign-owned subfolder: fixed (d4184a9). The lstat refusal list now also has every folder inside /opt/fork-fleet: units, fork-fleet-walls, and any other folder the installed manifest names. Tests: test_a_linked_or_foreign_owned_subfolder_is_refused (link and foreign owner, both folders, first and later install) and two new test_refusals_change_nothing cases. Both fail on round 4.
  • Nit 2, "own group only": fixed (d4184a9). user_problem also wants the passwd gid to be the group named after the user (getent group USER), so a useradd -g 0 user is refused by install and FAILs check. The check line, README and PR body now say this. The launcher's check_users is unchanged (uid 0 and one group only); install and check catch gid 0. Tests: a new subtest in test_a_hand_made_user_in_other_groups_or_with_a_home_fails_check and a new refusal case. Both fail on round 4.
  • Nit 3, check and desk-key.pub: fixed (d4184a9). check looks at the public half (link, owner, mode 0644), and install sets it to 0644 so check and install agree. Test: test_check_names_a_linked_public_key, fails on round 4.
  • Nit 4, FIFO hang: fixed (d4184a9). same_file lstats first and opens only a plain root file with the right mode. ours does not open a path that is not a plain file after following links. Test: test_a_fifo_at_a_unit_or_setting_does_not_hang (alarm guard), which hangs, so fails, on round 4.
  • Nit 5, parent folders: documented in the README (install section). /opt, /etc and /etc/systemd/system are the system's and root-owned on a stock system. A user who can write them can already change what root runs. They are not checked, to keep this round small.

State question: the new checks only read (lstat, getent), and all of them come before done('check') and the first write. Run again: same answers. Stopped half-way: nothing was written before the refusal. Old state: a subfolder from an older install that is a link is refused with the fix. Empty: a missing folder passes lstat.

Tests: install.test.py 46 OK (was 43). The round-5 test file against the round-4 installer gives 8 failures, all in the new tests and cases.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security+correctness (lens225r5), read at: d4184a9 — SAFE WITH NITS

No way found for a hostile job (fork-runner-N) to leave its user, and none from the network. install.test.py: 46 OK at head; against the round-4 installer (1ccced0) exactly the 8 claimed failures, all in new tests/cases.

Earlier findings (lens225r4)

  • 1 linked/foreign-owned subfolder: FIXED. desk-install:509-514 checks units, fork-fleet-walls and every manifest folder before done('check') and the first write (:520). Staging makes only those two (:306-308). Missing folder passes lstat (:134), then made by root. No TOCTOU for non-root (root-owned 0755 /opt/fork-fleet). Remaining gap: new nit 1.
  • Nit 2 main group: FIXED (:107-110: group exists, same name, gid = passwd gid). useradd -g 0 and -g users refused; only a root-made groupadd -o -g 0 fork-runner-1 passes.
  • Nit 3 check and desk-key.pub: FIXED (:668; install sets 0644 at :561).
  • Nit 4 FIFO hang: FIXED (same_file lstat + plain root-owned file + mode before open, :148-151; ours same via os.stat, :273).
  • Nit 5 parent folders: documented in README and PR body.
  • README, PR body and code agree. Red "Test Web 1": this diff touches only fork-kit/, one Python test line in .github/fork-ci-tests.txt and docs; it cannot reach the web suite.

New (nits)

  1. (proven) A root-owned but world-writable subfolder is accepted, and release files are moved in before its mode is fixed. not_root_owned (:129-142) checks link/kind/owner, not mode; swap renames at :376 and sets 0755 at :378 (top folders are set earlier, :520-522). A local non-job user racing renames over units/fork-fleet-desk-agent.service between :376 and :378 gets their text copied by put (:565) into /etc/systemd/system as a root unit (same for walls programs). check does flag "mode 777" beforehand. Fix: set the subfolders 0755 at :520-522 before the swap, or refuse g/o-writable folders in not_root_owned. Probe (fails at head with [511, 511, 511]):
    def test_probe_world_writable_subfolder_is_refused_before_files_go_in(self):
        os.makedirs(self.path('/opt/fork-fleet/units')); os.chmod(self.path('/opt/fork-fleet/units'), 0o777)
        seen, real = [], os.replace
        def spy(a, b, *k, **kw):
            if '/opt/fork-fleet/units/' in b: seen.append(stat.S_IMODE(os.stat(os.path.dirname(b)).st_mode))
            return real(a, b, *k, **kw)
        with unittest.mock.patch.object(self.m.os, 'replace', spy):
            code = self.install()[0]
        if code == 0:
            self.assertEqual([x for x in seen if x & 0o022], [], 'unit files placed while units/ was 0777')
  2. (root-only) check passes with desk-key.pub missing (skips missing, :670-673) while install refuses that state ("remove both", make_key :443-444). Nothing breaks; they disagree.
  3. (root-only, not new) read_manifest (:352-355), now called at :509, opens MANIFEST.sha256 without a type test (FIFO hangs install). uninstall remove of units/X (:771-773) follows a linked subfolder (manifest-named files only).

State question: runs again: new checks only read (:107, :509-518); right root-owned file left alone (:148); pub mode reset each run (:561). Stops half-way: every new refusal before done('check')/first write (:519); test_refusals_change_nothing compares the whole tree; restart mark/choice/swap order as round 4 (:536-547); uninstall removes the record last (:791). Old/foreign: linked subfolder refused (:514); other main group refused (:108-110); 0777 root subfolder = nit 1; older .pub mode set right (:561). Empty: no manifest -> subdirs = SUBDIRS; missing paths pass; uninstall without record changes nothing (:745-747).

Still holding: bootstrap.sh unchanged (curl + stdlib until sha confirmed as merge base on protected ci-release, no redirects, -I). Jobs: NoNewPrivileges, own RootDirectory, ProtectSystem=strict, ProtectHome; key 0600 in 0700 /etc/fork-fleet; lock on that folder.

A root-owned subfolder left writable by others (a hand setup) let a local
user swap a file in between the rename and the chmod, and put copies that
text into /etc/systemd/system as a root unit. The swap now makes and
chmods every release subfolder 0755 first, then renames the files in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 6 (orchestrator), head 827bd5b

  • lens225r5 nit 1 (proven): FIXED. swap now makes and chmods every release subfolder 0755 before any file is renamed in, so a root-owned folder left writable by others is closed before the unit and walls files land. New test test_a_subfolder_others_can_write_is_closed_before_files_go_in (spies on os.replace, asserts no file goes into units/ while it is g/o-writable, and the folder ends 0755). It fails on d4184a9 and passes at head; install.test.py 47 OK.
  • Nit 2 (check passes with desk-key.pub missing) and nit 3 (read_manifest FIFO, uninstall following a linked subfolder): not fixing in this PR. Both need root to cause them and break nothing.
  • Red "Test Web 1" on the previous head: not touched by this diff (fork-kit only); this push runs CI again.

# Conflicts:
#	fork-kit/common/machine.example.json
@scuttletoby
scuttletoby marked this pull request as ready for review November 16, 2026 07:27
@scuttletoby
scuttletoby merged commit 565e4e9 into main Nov 16, 2026
33 checks passed
@scuttletoby
scuttletoby deleted the fork/desk-d8-setup-pack branch November 16, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant