Skip to content

fork fleet: seed verb and seeds in the fleet config (D11) - #236

Draft
scuttletoby wants to merge 46 commits into
mainfrom
fork/desk-d11-seed
Draft

scuttletoby wants to merge 46 commits into
mainfrom
fork/desk-d11-seed

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Row D11 of the desktop pool plan, part 1 of 2: the host side of seeds, the warm caches for a desk pool's jobs. It adds seeds to a pool's fleet config entry and a seeds / seed NAME request on the desk link's forced command. Stacked on #230 (D10). Critical: it adds a second request to the desk link's forced command.

The D11 row came to 518 non-test lines, over the 400 limit, so it is split:

  • this PR: the config, the host's offer and the forced command's divert (183 non-test lines)
  • D11b, stacked on this one: the desktop fetching, its timer, the install kit and jobs reading the seeds

Why

A job's copy starts empty, so every job on a desktop downloads its tools and dependencies again. That makes it slow and wastes the PC's network. With seeds, the always-on host offers ready-made caches and the desktop keeps a recent copy that its jobs start from.

  • Config: pools[].seeds is a list of {"name", "every_hours", "max_age_hours", "max_mb"}. check_config refuses bad names, values that are not positive whole numbers (bools too), max_age_hours below every_hours, max_mb above 16384, missing fields and duplicate names.
  • Host offer: the new fork-fleet-desk-seed.py (standard library only) answers two requests.
    • seeds prints the pool's list as one JSON object: for each seed, bytes and made when it is offered now, and why when it is not.
    • seed NAME streams /var/lib/fork-fleet/seed-files/POOL/NAME.tar (a folder per pool: two pools never share a
      file), followed by a trailer line on stderr: seed NAME BYTES SHA256 MADE.
    • The fleet makes nothing itself. The project's own timer or job writes the tar (temp file and rename).
  • Forced command: fork-fleet-desk.py serve POOL already runs as the desk-link user for the desk key. When SSH_ORIGINAL_COMMAND starts with seed (so seeds, seed NAME, and also seeds x, seed, seedsx), it hands the request to the seed module instead of serving a session; the module refuses anything but seeds and seed NAME with exit 2. Every other request keeps the old path.

Safety (critical)

  • No paths from the desktop: only a name that is in the pool's own seeds list is served. The name never becomes part of a path unless it is on that list, and names match [a-z0-9_-] (the POOL pattern). seed ../m2, seed /etc/passwd, seed M2, seed m2 x, seeds x and the like all exit 2 and send nothing (tested, also through the real forced command: seeds x, seed, seedsx and seeds exit 2 and make no socket). A command that does not start with seed keeps the old path (a session request).
  • What may be sent: the file is opened with O_NOFOLLOW. It must be a plain file that no group or other user may write, in a pool folder and a files folder (not links) that no group or other user may write, at most max_mb, younger than max_age_hours, and not dated more than 5 minutes in the future. Otherwise it exits 4 and sends nothing. The folders' owner is not checked, only their mode (documented).
  • A changed file is caught: at most the checked size is read. A file changed in place while it was sent (size, mtime or ctime differ) exits 4, and the desktop then throws its copy away (D11b). A same-size rewrite with its old date put back is caught by ctime. A file replaced by a rename while it is sent is fine: the open file then has no links left (st_nlink 0), so only its ctime (changed by the unlink) is not compared, and the old whole file is sent (tested). The trailer's SHA-256 is of exactly the bytes sent.
  • Never as root, and a fleet config that will not load exits 2.
  • One stream at a time per pool: a lock in the desk link's own socket folder (POOL.seed.lock, under safe_dir). A second stream exits 3, so a misbehaving desktop cannot open many. A desktop that takes less than 64 KiB in 120 s, or not the whole file within 600 s plus one second per 256 KiB, is cut off (exit 1), so it cannot hold the lock for long. A lock file that cannot be opened or that another user owns is a clear refusal (exit 1, tested).
  • The docs say: never put a secret in a seed, because every job of the pool reads it.

What happens when this runs again, stops half-way, or meets existing, old or empty state?

Case What happens
Runs again Both requests are read-only on the host, apart from the lock file, which is reused. Asking twice gives the same answer while the file is unchanged.
Stops half-way When the ssh session dies mid-stream, the desktop gets no trailer or a short stream and keeps its old copy (D11b). The lock is an flock, so it goes with the process, and the next stream gets it (tested).
Existing / old state A file older than max_age_hours is not offered. A file replaced by rename while being sent is fine, because the open fd still reads the old whole file. A file written in place is refused. A fleet config without seeds means an empty list. Older desktops that never ask are unaffected.
Empty state A seed not made yet shows why: not made yet in the list, and seed NAME exits 4. A pool with no seeds lists [].

General vs project

General. The verb, the config check and the offer name no project. Which caches a pool has, and how the host makes each tar, are the project's own config and its own timer. The doc example (gradle) is only an example.

Upstream fit

Only the fork-only .github/scripts/fork-fleet* files, .github/fork-fleet.md and the fork CI test list change. Nothing in upstream T3 changes.

Tests

.github/scripts/fork-fleet-desk-seed.test.py (new, in .github/fork-ci-tests.txt). It uses temp folders only: no root, no ssh, no network.

  • Config: the seeds check, with a good list, an empty list, max_mb at its cap and 11 bad ones.
  • Host:
    • The list says what is offered, and why not.
    • A seed streams whole, with the right trailer.
    • Nine refused commands send nothing.
    • Nine files that may not be sent: missing, a link, group-writable, too big, too old, dated in the future, a folder, a pool folder others may write, and a files folder others may write. Each must be refused for its own reason, read from the pool's folder.
    • A second stream is refused, and the lock goes with its holder.
    • A file changed while it is sent is refused, and no more than its checked size is sent; a same-size rewrite with its old date put back is refused.
    • A file replaced by rename while it is sent is sent whole, with its trailer.
    • A lock file of another user is refused.
    • A desktop that stops reading lets the lock go; one that does not finish by the deadline is cut off; one that reads slowly but steadily is kept.
    • Two pools never share a seed file.
    • It never runs as root, and a bad config is refused.
    • The forced command, run as a real process from a copy of the host's install folder, diverts seeds, seed NAME and every other seed-prefixed command. It serves no session for them.

The existing fork-fleet-desk.test.py and fork-fleet.test.py were run too.

Local run (round 2):

  • .github/scripts/fork-fleet-desk-seed.test.py: 16 OK
  • .github/scripts/fork-fleet-desk.test.py: 42 OK
  • .github/scripts/fork-fleet.test.py: 201 OK (7 skipped)
  • Round 1: the seed tests run against the round-0 code fail 11 times (per-pool path, future date, cut length, ctime rewrite, the seed/seeds divert, the stall deadline). Some of these fail on the round-0 code only because the per-pool path changed where the test puts the tar.
  • Round 2: the seed tests run against the round-1 code fail 5 times (rename while sent, the deadline, a slow steady reader, the stall message, the max_mb cap). Mutations of the round-2 code: without the future-date check, without the lock-owner check, and with round-0's path (no pool folder), the matching tests fail (the future-dated case now says why it fails: "not made yet" instead of "dated in the future").

Red-checks

Each rule was broken in a copy, one at a time, and the named test failed (all 11 red):

  • only a listed name is sent -> test_other_commands_and_names_are_refused_and_send_nothing
  • a link is never followed (O_NOFOLLOW) -> test_a_file_that_may_not_be_sent_is_exit_4
  • a file others may write is not sent -> the same test
  • a file older than max_age_hours is not sent -> the same test
  • a file changed while it is sent is refused -> test_a_file_changed_while_sent_is_exit_4
  • one stream per pool -> test_a_second_stream_for_the_pool_is_exit_3
  • a lock of another user is refused (round 2) -> test_a_lock_of_another_user_is_refused
  • a file dated in the future is not sent (round 2) -> test_a_file_that_may_not_be_sent_is_exit_4
  • never as root -> test_never_as_root_and_a_bad_config_is_exit_2
  • config: max_age_hours at least every_hours -> Config
  • the forced command diverts seed requests -> test_the_forced_command_diverts_seed_requests

The first run found one gap: with O_NOFOLLOW removed the link case still passed, because the link's target was
group-writable and refused for that reason. The test now makes the target 0644, so only the link can be the reason.

Round 3

  • A last stall alarm after the last write of the whole file is not a stall (it gave a traceback); test with a fake signal.
  • A request-level test for a pool without seeds ([]).
  • Documented in fork-fleet.md "Seeds": make a seed only from trusted code, never in a job that runs PR code; a hard link kept to the old tar means exit 4; at least about 256 KiB/s is needed (a slower link is backed off by fork fleet: a desktop fetches its desk pool's seeds (D11b) #237); the pool lock can be held about 18 h for 16 GiB at the slowest rate.
  • Not fixed, documented: no host-side rate limit (the key is the pool's own desktop, one stream per pool at a time; fork fleet: a desktop fetches its desk pool's seeds (D11b) #237's backoff limits a well-behaved desktop).
  • Tests: fork-fleet-desk-seed.test.py 18 OK, fork-fleet-desk.test.py 42 OK, fork-fleet.test.py 201 OK (7 skipped).

Round 4

  • Test only: test_a_desktop_that_stops_reading_a_one_piece_seed_is_exit_1 covers the wrote == sent half of the stall guard (fork-fleet-desk-seed.py:172): a seed under one piece has sent == size at the stall, so only that half refuses the cut stream. It passes at head and fails when that half is removed (checked in a copy). The red-check list above does not include it; this line does.
  • D10's round 4 (and through it D8's and D9's) merged in first.
  • Tests: fork-fleet-desk-seed.test.py 19 OK, fork-fleet-desk.test.py 42 OK; the merged kit tests install.test.py 44 OK, windows.test.py 24 OK (4 skipped), free_contract.test.py 17 OK.

🤖 Generated with Claude Code

scuttletoby and others added 4 commits November 15, 2026 19:56
fork-kit/desk: bootstrap, install, PASS/FAIL check and uninstall for a
desktop's side of a desk pool (runner users, launcher, walls, desk agent,
gate hook, settings, key, units), from one release commit. Safe to run
again and to stop at any step; refuses an install it did not make;
uninstall removes only what it made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
start-desk.ps1 sets up a WSL2 distro for the desk kit from Windows
PowerShell 5.1: a memory and swap cap in .wslconfig, NAT, the WSL2
walls' settings inside the distro, the packages, the machine file, then
the desk kit's bootstrap from one release commit, a logon task that
keeps the distro running, and a PASS/FAIL check. -Uninstall removes it.
Every step skips what is already right; WSL restarts only on a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One small contract for every tool that asks whether a machine may take more
work: no arguments, 3 s, exit 0 free / 75 busy, one JSON object on stdout.
fork-kit/common/free_contract.py is the one reader (judge, ask, and a CLI for
shell callers); free-fixtures.json holds the cases every reader must agree on.
Two examples: memory (pressure, swap) and busy-while-running (named programs).

The launcher's fits also asks FREE_COMMAND when launch.conf names one (default
none: the memory rule alone). The desk kit installs the reader and the memory
example and writes FREE_COMMAND from the machine file's desk.freeCommand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A desk pool may list warm caches ("seeds") in its fleet config entry. The
always-on host's forced command answers a desktop's `seeds` (the pool's list)
and `seed NAME` (that one ready-made tar, with a size and SHA-256 trailer) in
an ssh session beside the agent's. Only names of the list, never a path;
exit 3 while another stream for the pool runs, 4 when the file may not be
sent. The desktop's fetcher follows in the next change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Review round 1 at 57e1cb5: two independent lenses (security, correctness), both SAFE WITH NITS. Nothing serves a path the desktop picks; the existing serve request is unchanged; 9 seed + 42 desk tests pass. Fix round to follow.

  1. Shared across pools (fork-fleet-desk-seed.py:33,58): the file path is FILES/NAME.tar with no pool, so a second desk pool b listing gradle streams the tar pool A's timer made. Use FILES/POOL/NAME.tar or document the sharing (fork-fleet.md:704 understates it).
  2. A file dated in the future is offered forever (:70): now - mtime is negative, so the age check never fails. Proven: a 10-years-ahead mtime with max_age_hours 48 is sent with exit 0; it should be exit 4.
  3. The stream is not capped at the checked size (:111,123): send() reads to EOF, so a file growing in place streams past max_mb and only then exits 4. Read at most st.st_size.
  4. The changed-file check skips ctime (:111): a same-size rewrite with the old mtime put back is sent with exit 0. Add st_ctime_ns.
  5. PR body overstates refusals (fork-fleet-desk.py:412-414): seed, seeds x, serve x exit 2 only when offer() is called directly (the test does). Through the forced command they fall through to serve (old behaviour, not a regression; a session lock is made, exit 1). Either divert every seed/seeds-prefixed command, or correct the body; D11b must treat a session reply to seeds as "no seeds".
  6. No deadline on send (:101-106): a desktop that stops reading holds the pool's seed lock until TCP gives up (no ClientAliveInterval by default); other seed requests get exit 3 meanwhile.
  7. Nits: the lock file's owner is not checked (:95; a root-owned leftover gives exit 1 with a terse OSError instead of the clear refusal serve gives); the seed folder's owner is not checked, only its mode (:59-61).

State question: runs again (read-only, lock reused), killed mid-stream (flock dies, no trailer), broken pipe (exit 1, files closed), missing/broken config (exit 2), pool not in config (exit 2), no seeds (empty list), missing folder/tar ("not made yet", exit 4), missing socket folder (made): all handled.

scuttletoby and others added 15 commits November 15, 2026 20:45
- install refuses a systemd drop-in it did not write for its units; check fails on one
- re-run restarts the agent only when a file, setting or key changed
- check compares the settings files with the machine file
- uninstall stops every runner, keeps the nft table, users and record while one
  still runs, removes every fork-fleet-* namespace and ffh- link, removes only
  what the kit made (deny files it created, its users' deny lines), names what
  it leaves, and removes its record last
- the record is read only when it has this kit's shape
- --release-fleet-config goes back to the release's fleet config
- bootstrap parses --fleet-config like the other flags

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- .wslconfig is read and written as UTF-8 without a BOM (never ASCII), by a
  temp file and a rename, with a dated backup; only [wsl2] keys are judged
- the restart decision reads the live state (PID 1, interop, wsl.conf newer
  than boot, memory, networking mode, .wslconfig newer than the boot)
- a distro the starter did not make is named and must be typed to take over
- MemoryGB above the PC's RAM less 2 GB is refused
- exit 1 on a failed check; -Uninstall needs no -Repo/-Sha; UNC paths work

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ask() runs the file unsafe() judged (the link's target), never the link
- NaN and Infinity are not JSON: not available (two fixture cases)
- a child that left the group with setsid and still holds stdout is ended;
  one that also closed stdout is the documented bound
- poll() instead of select() (descriptors above 1024)
- busy-while-running matches names longer than 15 characters
- FREE.md: the bound, links, ACLs, the memory example's pressure caveat

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontract

# Conflicts:
#	fork-kit/desk/sbin/desk-install
- seeds live in FILES/POOL/NAME.tar: pools never share a file
- a file dated more than five minutes in the future is refused (exit 4)
- the stream stops at the size that was checked; ctime and inode join the
  changed-file check
- the forced command sends every command that starts with `seed` to the seed
  handler, so `seeds x` and `seed` exit 2 and never reach a session
- a desktop that takes nothing for STALL seconds is cut off (exit 1) and the
  pool's lock let go
- the lock file must be this user's; a clear refusal otherwise

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e ip (D8)

Uninstall now deletes the namespaces ip netns list shows, not one per
recorded user; the older test listed none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A whole-folder removal took the record first only for some directory
orders, so the test passed on the old code by luck. Listing by name puts
the record first, which the old order fails every time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… flag (D9)

The text check that no ASCII encoding is left matched the comment that
explains the bug.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-1 fix resolved the link again after the check, so a link
repointed between the two still changed what ran. The new test caught it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ool (D11)

The lock check after a stall sent 1 MB into a pipe nobody read until the
call returned, and the second pool reused the first one's label and beacon,
which the config check refuses. The pool test also puts a tar at the old
shared path, so the old code fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1 (commits 7026afd and 099f438, the second a test-only fix; D10's round 1 merged in; head 099f438). Tests: desk-seed 12 OK, desk 42 OK.

  1. Shared across pools: fixed (7026afd). The path is now FILES/POOL/NAME.tar (a folder per pool); D11b and the runbook follow. Test: test_pools_never_share_a_seed_file.
  2. A file dated in the future: fixed (7026afd). More than 5 minutes ahead is refused (exit 4, "dated in the future"). Test: a case in test_a_file_that_may_not_be_sent_is_exit_4.
  3. Stream not capped: fixed (7026afd). send() reads at most the checked size. Test: test_a_file_changed_while_sent_is_exit_4 checks the cut length.
  4. ctime: fixed (7026afd). The changed-file check compares size, mtime, ctime and inode. Test: test_a_same_size_rewrite_with_its_old_date_put_back_is_exit_4.
  5. Forced-command divert: fixed (7026afd). Every command starting with seed goes to the seed module, so seeds x, seed, seedsx, seeds exit 2 and make no socket. Test: test_the_forced_command_diverts_seed_requests. Body corrected.
  6. No send deadline: fixed (7026afd). A desktop that takes nothing for 120 s is cut off (exit 1) and the lock goes. Test: test_a_desktop_that_stops_reading_lets_the_lock_go.
  7. Nits: lock owner: fixed (an open error or another owner is a clear refusal, exit 1). Seed folder owner: documented (only the mode is checked; fork-fleet.md Seeds).

Fails before the fix: the seed tests run against the round-0 code give 10 failures and 1 error (every test above). Some fail there partly because the per-pool path moved the tar; test_pools_never_share_a_seed_file also puts a tar at the old shared path, so it fails on the old code for the real reason.

PR body updated. Still draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 2 at 3fb8617 (D10 round 2 merged in first)

Lens 236s r1 (security):

  1. Regression, MUST: a seed replaced by rename while it streams. Fixed in 3fb8617.
    • When the open file has no links left (st_nlink 0), it was replaced, not rewritten. Its ctime is then not compared; size and mtime still are.
    • The lens test test_a_rename_while_sent_is_fine sends the old file whole, with its trailer. It fails on round 1.
  2. Nit: the per-write alarm lets a very slow reader hold the lock. Fixed in 3fb8617.
    • Each stream now has an overall deadline: 600 s plus one second per 256 KiB.
    • The stall wait restarts per 64 KiB written, not per 1 MiB.
    • Tests: test_a_slow_desktop_meets_the_deadline, and test_a_desktop_reading_slowly_but_steadily_is_not_dropped. Both fail on round 1.
  3. Nit: the st_ino check never differs. Fixed: the check is removed, and the PR body no longer claims an inode compare.
  4. Nit: max_mb had no upper bound. Fixed: at most 16384 (fork-fleet.py SEED_MAX_MB). There is a Config test case for it.
  5. Earlier note: the future-date test passed on round 0. Fixed.
    • Every refused file must now say its own reason (assertIn(name, err)), and the tar is read from the pool's folder.
    • With round 0's path, the future-dated case fails ("not made yet" instead of "dated in the future"). With the future-date check removed, it fails too.

Lens 236c r1 (correctness):

  1. Regression: the rename case. This is the same as security finding 1 above: fixed.
  2. Nit: the stall alarm restarts per 1 MiB, but the docs say "takes nothing for two minutes". Fixed: the alarm is per 64 KiB. The docs (fork-fleet.md, the module header, the PR body) now say "less than 64 KiB in two minutes, or not the whole file by the deadline".
  3. Nit: no test for the lock-owner refusal. Fixed: test_a_lock_of_another_user_is_refused. With the check replaced by if False:, it fails.
  4. Note for D11b: an older host's seeds reply. Handled in D11b (fork fleet: a desktop fetches its desk pool's seeds (D11b) #237): a non-JSON reply or exit 1 will mean "no seeds".

State question:

  • Runs again: nothing new is stored, and the lock file is reused.
  • Stops half-way:
    • A rename mid-stream now completes.
    • A killed stream gives no trailer, and the desktop keeps its old copy.
    • A slow or stalled desktop gives up the lock within the deadline.
  • Old state: a fleet config with max_mb above 16384 is now refused by check_config. No shipped config has one.

Local run (round 2):

Test Result
desk-seed 16 OK
desk 42 OK
desk-agent 31 OK
desk-key 20 OK
fork-fleet 201 OK (7 skipped)

Red-check: the new seed tests run against round-1 code give 3 failures and 2 errors.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check r2 (lens236sr2), read at: 3fb8617 — SAFE WITH NITS

Earlier findings (new tests run against round-1 099f438: 3 failures + 2 errors, as the PR claims)

New findings (nits)

  1. Slow links never get large seeds (:152): the deadline assumes >= 256 KiB/s; at 100 KiB/s a seed over about 96 MiB is always cut off (exit 1) and re-read from the start every every_hours. Fix: document the minimum rate; D11b backs off after a deadline exit.
  2. Cache poisoning not documented (fork-fleet.md Seeds): if the seed is built by a job running PR code, it can plant an artifact every later pool job starts from. Add: "make a seed only from trusted code (main), never in a job that runs PR code".
  3. No host-side rate limit: the pool's desk key can re-pull a file of up to 16 GiB in a loop (host disk/network). Fix: host-side "at most once per every_hours".
  4. Lock hold for the largest seed: a reader just above the stall rate (~546 B/s) holds the pool lock up to 600 + 65536 s (~18 h) for 16 GiB. Pool-local.
  5. A writer that keeps a hard link to the old tar then renames the new one over leaves st_nlink >= 1, so the stream in progress gets exit 4. Availability only.

Checked safe: exact command parse (:89,104; seed, seed ../x, seed m2 x, seed m2\n, seedsx exit 2; long input cut to 60 chars; non-UTF-8 escaped); pool from the forced command through pool_ok (fork-fleet-desk.py:398); only FILES/POOL read, folders lstat-checked, O_NOFOLLOW|O_NONBLOCK, plain file only (:66-78); launch path byte-identical, divert only on startswith('seed') (fork-fleet-desk.py:412); config refuses null/bool/missing/max_age_hours < every_hours/duplicates/max_mb > 16384 (fork-fleet.py:143-151); broken pipe exits 1 (fork-fleet-desk.py:427).

State question: runs again: read-only, lock reused (:108). Killed: kernel releases the flock; no trailer unless every check passed (:129-131); no signal handler, SIGHUP ends cleanly. Leftover lock of another user refused (:112); symlinked lock ELOOP exit 1 (:110). Socket folder made or refused with the tmpfiles hint (fork-fleet-desk.py:402). Missing/broken config exit 2 (:54), unknown pool exit 2 (:59), old config without seeds = empty (:58). Half-installed host: FileNotFoundError exit 1 (:427), D11b reads it as "no seeds". Old-kit host answers with a session, handled by D11b. Empty cache: "not made yet" exit 4 (:72). Monotonic deadline; future dates capped by AHEAD (:78).

Tests: desk-seed 16 OK, desk 42 OK (no PowerShell).

verdict SAFE WITH NITS, PR #236, head 3fb8617

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check r2 (lens236cr2), read at: 3fb8617 — SAFE WITH NITS

Round-1 findings (round-1 fork-fleet-desk-seed.py swapped in under the head tests)

  1. Rename while streaming: FIXED (fork-fleet-desk-seed.py:128, skip ctime only when st_nlink == 0). Test fails on round 1; same-size rewrite with old date still caught (mutating the check turns its test red).
  2. Slow reader holding the lock: FIXED (deadline :152, :160-163; stall wait per 64 KiB :164). Both tests fail on round 1.
  3. st_ino check: FIXED, removed (:128).
  4. max_mb upper bound: FIXED (fork-fleet.py:58, :147; cap and cap + 1 tested).
  5. Future-date test passing on round 0: FIXED, each refused case asserts its own reason (test:141).
  6. Lock-owner refusal test: FIXED (test_a_lock_of_another_user_is_refused).
  7. Stall-alarm wording: FIXED (fork-fleet.md:716-717, module header).
    Round 1 against head tests: 3 failures + 1 error (head fork-fleet.py kept, so the config case does not error).

Focus: matches the PR body and fork-fleet.md. Old configs without seeds = empty list (fork-fleet.py:143); only max_mb > 16384 newly refused. The desktop agent sends serve POOL (fork-fleet-desk-agent.py:166), so fork-fleet-desk.py:414-417 runs as before. Source changing mid-stream: in-place write exits 4, growth cut at the checked size, rename sends the old file whole, trailer SHA-256 is of the bytes sent. fork-ci-tests.txt:12 runs the file.

Nits

  1. "A pool with no seeds lists []" has no request-level test (behaviour checked: {"v":1,"seeds":[]} exit 0; seed m2 exit 2):
    class Extra(Base):
        def test_a_pool_without_seeds_lists_nothing(self):
            cfg = json.loads(json.dumps(CONFIG)); cfg['pools'][0].pop('seeds', None)
            with open(os.path.join(self.t, 'fork-fleet.json'), 'w') as f: json.dump(cfg, f)
            code, out, err = self.offer('seeds')
            self.assertEqual((code, json.loads(out)), (0, {'v': 1, 'seeds': []}), err)
            self.assertEqual(self.offer('seed m2')[:2], (2, b''))
  2. Rename exemption needs link count 0 (:128): a tar also kept under a second hard link exits 4 after a rename over it. One sentence in the docs.
  3. Alarm race (:163, :171): a last alarm between loop end and signal.alarm(0) lets Stalled escape; fork-fleet-desk.py:422-428 catches only Refused/OSError, so traceback + exit 1. Lock released, copy discarded, no harm.
  4. For D11b: the 256 KiB/s floor (:41) means ~1 Mbit/s never finishes a 4 GiB seed (~4.7 h per attempt holding the lock); D11b should back off after a deadline cut.

State question: runs again: read-only, lock reused (:108). Killed: flock released, no trailer, desktop keeps its old copy. Broken pipe: exit 1 (fork-fleet-desk.py:426). Missing/broken config or unknown pool: exit 2 (:52-59). Empty cache: "not made yet" exit 4 (:72). Leftover lock of another user: exit 1 (:110, :113). No socket folder: made 0700 like serve (fork-fleet-desk.py:59-75). Seed module missing after a half install: exit 1. Older desktops never send seed.

Tests: desk-seed 16 OK, desk 42 OK, fork-fleet OK (7 skipped).

verdict SAFE WITH NITS, PR #236, head 3fb8617

scuttletoby and others added 7 commits November 16, 2026 00:57
…ock, stopped choice) (D8)

- --fleet-config: the folders of the link and of the file it names are both checked.
- A runner user made by hand must have no login shell, a locked password and no
  cron.allow/at.allow line: install refuses otherwise, check FAILs.
- check looks at modes (files, folders, desk key); install puts a changed mode or
  owner back from the release.
- One install or uninstall at a time (flock on /opt/fork-fleet); a second is refused
  and does not sweep the first one's staging folder.
- The restart mark is set before a new fleet config choice is saved, and check
  compares the project's copy with the installed one: a run killed between the
  choice and the swap fails check.
- An old record that names a project config that is gone: a readable refusal.
- Tests for the runner-user branches of the ownership check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ges listed, PID 1 start) (D9)

- networkingMode/dnsTunneling under [experimental] (the pre-2.0 form that can still
  win over [wsl2]) are removed by the merge, listed in the question, and the
  Windows-side check FAILs while one is there.
- The question lists each .wslconfig key it changes, old -> new.
- The owed-shutdown mark is one for every distro, like .wslconfig; uninstall keeps it.
- wsl.conf is compared with PID 1's start (/proc/1/stat field 22 + btime), not the
  mtime of /proc/1; unreadable means restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontract

# Conflicts:
#	fork-kit/desk/sbin/desk-install
#	fork-kit/desk/tests/install.test.py
…ailing-/ link case

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…no-seeds test; document trust, rate and links

The stall alarm could go off between the end of the send loop and alarm(0), so Stalled escaped as a traceback.
Now alarm(0) runs inside the try and a late alarm after everything read was written counts as done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 3

Commit e5e0000 (head; D10 merged in as e8bdc7c). Findings from lens236sr2 (security) and lens236cr2 (correctness):

  • cr2 3, alarm race (a last alarm between the loop end and alarm(0)): fixed (e5e0000). alarm(0) runs in an inner finally; a Stalled that arrives after the last write of the whole file is not a stall. Test with a fake signal whose alarm(0) fires SIGALRM; it errors on the round-2 code.
  • cr2 1, no request-level test for a pool without seeds: fixed (e5e0000), test_a_pool_without_seeds_lists_nothing.
  • sr2 2, cache poisoning: documented (e5e0000, fork-fleet.md "Seeds"): make a seed only from trusted code (the default branch), never in a job that runs PR code.
  • sr2 5 / cr2 2, a hard link kept to the old tar means exit 4: documented (e5e0000).
  • sr2 1 / cr2 4, slow links: documented here (at least about 256 KiB/s; at 100 KiB/s a seed over about 96 MiB is always cut off) and fixed in fork fleet: a desktop fetches its desk pool's seeds (D11b) #237: a torn seed is tried again after 1, 2, 4, 8, 16, then 24 h.
  • sr2 4, lock hold of up to ~18 h for 16 GiB: documented (e5e0000). Pool-local: only that pool's own desktop waits.
  • sr2 3, no host-side rate limit: documented, not fixed (e5e0000). The desk key is one pool's own desktop; it can already hold that pool's lock, and the stream is one at a time per pool. A host-side "once per every_hours" would need state the read-only host side does not keep, and the desktop's backoff (fork fleet: a desktop fetches its desk pool's seeds (D11b) #237) now limits a well-behaved desktop.

State question: runs again: read-only, the lock is reused. Stops half-way: flock let go by the kernel, no trailer, the desktop keeps its old copy; the race fix changes nothing here. Old/empty: a pool without seeds lists [] (now tested).

Tests (local, at e5e0000): fork-fleet-desk-seed.test.py 18 OK, fork-fleet-desk.test.py 42 OK, fork-fleet.test.py 201 OK (7 skipped), fork-gate-trust-docs.test.py all ok.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security + correctness re-check r3 (lens236r3), read at: e5e0000 — SAFE WITH NITS

Round-2 findings (round-2 head 3fb8617; round-2 fork-fleet-desk-seed.py run under the head tests):

  • cr2 3, alarm race: FIXED (fork-fleet-desk-seed.py:169-173, alarm(0) in an inner finally, so a late alarm lands in except Stalled). test_an_alarm_that_comes_after_the_last_write_is_not_a_stall fails on round 2 with a Stalled traceback, as claimed. On Python 3.12 the handler runs at the check right after alarm(0), so it cannot slip past the except.
  • cr2 1, request-level test for a pool without seeds: FIXED (test_a_pool_without_seeds_lists_nothing). It also passes on round 2: added coverage, the behaviour was already right.
  • sr2 2, cache poisoning: DOCUMENTED (fork-fleet.md:708-710).
  • sr2 5 / cr2 2, a hard link kept to the old tar: DOCUMENTED (fork-fleet.md:711-713).
  • sr2 1 / cr2 4, slow links: DOCUMENTED (fork-fleet.md:723-725); the numbers check out (256 KiB/s = about 2.1 Mbit/s; the 100 KiB/s limit gives 96.2 MiB). The backoff is in fork fleet: a desktop fetches its desk pool's seeds (D11b) #237.
  • sr2 4, lock held up to about 18 h: DOCUMENTED (:725-727; 600 s + 65536 s is right).
  • sr2 3, no host-side rate limit: DOCUMENTED, not fixed (:727-730); the reason is honest and the risk stays within one pool.

New finding

  1. Nit, test gap in the round's own guard (fork-fleet-desk-seed.py:172). Drop wrote == sent and and all 18 tests still pass. In that mutated code, a desktop that stops reading a seed of one piece (1 MiB or less, so sent == size at once) gets exit 0 and a full trailer for a cut stream (proven: 0 != 1 : seed m2 532480 0f36… 1794792582). The head code is correct (exit 1); the existing stall test uses a tar over 1 MiB, so the other half of the condition still refuses it. A second mutation, if wrote == sent: (dropping the size check), also survives; its only effect is exit 4 instead of exit 1 when a read stalls between pieces. This test (class Host) passes at head and fails on mutation 1:
def test_a_desktop_that_stops_reading_a_one_piece_seed_is_exit_1(self):
    self.make('m2', {'a': os.urandom(512 << 10)})   # under PIECE: sent == size at the stall
    r, w = os.pipe()   # nobody reads r
    err = io.StringIO()
    try:
        with mock.patch.object(seed, 'STALL', 1):
            code = seed.offer(POOL, 'seed m2', os.path.join(self.t, 'run'), out=w, err=err,
                              config=os.path.join(self.t, 'fork-fleet.json'), files=os.path.join(self.t, 'files'))
    finally:
        os.close(r); os.close(w)
    self.assertEqual(code, 1, err.getvalue())
    self.assertNotIn('seed m2 ', err.getvalue())   # no trailer for a cut stream

The PR body's red-check list ("all 11 red") does not cover this guard.

What the round could have broken (checked, fine)

  • An earlier exception (such as a broken pipe) with an alarm pending becomes Stalled; the stall branch gives exit 1, the same code.
  • A deadline or stall mid-piece always has wrote < sent, so it is still refused.
  • A file cut short in place reaches pass, then :129 gives exit 4.
  • Forced-command divert (fork-fleet-desk.py:412), name parsing (:89,104), O_NOFOLLOW and folder checks (:66-78), config checks (fork-fleet.py:143-151): unchanged; the round-2 checks still hold.

State question

  • Runs again: read-only; the lock is reused (:108).
  • Killed mid-stream: the divert runs before serve's signal handlers are set, so SIGTERM/SIGHUP kill it; the kernel releases the flock, no trailer is sent, the desktop keeps its old copy.
  • Late alarm: handled (:170-173).
  • Leftover lock of another user: refused (:110,113).
  • Missing or broken config, unknown pool: exit 2 (:54,59).
  • Config without seeds: empty list (:58, now tested).
  • Empty cache: "not made yet", exit 4 (:72).
  • Half-installed host: exit 1 (fork-fleet-desk.py:427).

Tests at head: desk-seed 18 OK, desk 42 OK. Round-2 seed code under the head tests: 1 error (the race test). Two mutations of :172 survive; the test above catches the first. PowerShell tests skipped.

verdict SAFE WITH NITS, PR #236, head e5e0000

scuttletoby and others added 7 commits November 16, 2026 01:53
…, runner users as strict as the launcher, lock in /etc/fork-fleet, owner of units and settings) (D8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, question and merge decide the same way) (D9)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontract

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd 4)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 4

Commit 70822b5 (head; D10's round 4 merged in first as e56ad75). Findings from lens236r3:

  • Nit 1, the wrote == sent half of the guard at fork-fleet-desk-seed.py:172 untested: fixed (70822b5, test only). The lens's test as test_a_desktop_that_stops_reading_a_one_piece_seed_is_exit_1 (a 512 KiB seed, a pipe nobody reads, STALL 1: exit 1 and no trailer). It passes at head and fails in a copy with wrote == sent and removed (1 failure of 19). The second mutation (dropping the size check) only changes exit 4 to exit 1 and is left untested, as the lens judged. The PR body's red-check note now names the new test.
  • Earlier documented items stay documented (cache poisoning, a kept hard link, slow links, the long lock, no host-side rate limit).

State question: test only; the code did not change. Runs again: read-only. Stopped half-way: a cut one-piece stream gets no trailer, so the desktop keeps its old copy. Old or empty state: unchanged from round 3.

Tests (local, at 70822b5): fork-fleet-desk-seed.test.py 19 OK, fork-fleet-desk.test.py 42 OK, install.test.py 44 OK, windows.test.py 24 OK (4 skipped), free_contract.test.py 17 OK.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: re-check r4 (lensd8minr4), read at: 70822b5 — SAFE

  • The untested half of the stall check: FIXED with a test only (fork-fleet-desk-seed.test.py:223-237).
  • Does the new test fail on the round-3 code? No, and it should not: this round did not change the code, so it passes on both. The mutation the earlier lens meant does fail it: with wrote == sent and removed at fork-fleet-desk-seed.py:172, the test gives AssertionError: 0 != 1 (exit 0 and a full trailer for a cut stream).
  • Tests: fork-fleet-desk-seed.test.py 19 OK, fork-fleet-desk.test.py OK.
  • PR body: accurate; it names the new test outside the "all 11 red" list.
  • State: a cut one-piece stream gets no trailer, so the desktop keeps its old copy. Old and empty state as in round 3.

verdict SAFE, PR #236, head 70822b5

@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 5

SAFE at round 4; nothing to fix. Merged #230 round 5 (which carries #225's round-5 fixes) in 22a6433.

Tests: install.test.py 47 OK.

Still a draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: re-check (lensd8minr5), read at: 22a6433 — SAFE

Round-5 merge is clean: git show --remerge-diff prints nothing.

  • Round 4 was SAFE; merge-only confirmed: this PR's own files (fork-fleet-desk-seed.py/.test.py, fork-fleet-desk.py, fork-fleet.py, its fork-fleet.md section) unchanged since 70822b5.
  • State unchanged: a cut stream gets no trailer, so the desktop keeps its old copy. fork-fleet-desk-seed.test.py 63 OK.

@scuttletoby
scuttletoby changed the base branch from fork/desk-d10-free-contract to main November 16, 2026 08:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant