Skip to content

fork fleet: a desktop fetches its desk pool's seeds (D11b) - #237

Draft
scuttletoby wants to merge 13 commits into
fork/desk-d11-seedfrom
fork/desk-d11b-seed-fetch
Draft

scuttletoby wants to merge 13 commits into
fork/desk-d11-seedfrom
fork/desk-d11b-seed-fetch

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Row D11 of the desktop pool plan, part 2 of 2: a desktop fetches its desk pool's seeds (warm caches) from the always-on host and its jobs read them. It is stacked on #236 (D11), the host side; the D11 row was over 400 non-test lines, so it is split in two. Critical: this runs as root on the desktop, unpacks data from the network, and changes what every job can see.

Why

D11 lets the host offer seeds. This PR makes a desktop take them, keep a recent copy and let jobs start from it, so jobs stop downloading the same tools and dependencies each time.

  • fork-fleet-desk-seed.py run (desktop, root, hourly):
    • It asks the host for the list with the agent's own ssh settings (pinned host key, desk key) and the command seeds. Then, for each seed that is new and due, it runs seed NAME.
    • The limits are the desktop's own: which seeds, every_hours, max_age_hours and max_mb come from the
      desktop's own installed fleet config (/opt/fork-fleet/fork-fleet.json, its pool's seeds). max_mb is the
      smaller of the host's and its own. A name the host lists that its own config does not is ignored. The host
      only says what is offered now (bytes, made time).
    • The list's output is read up to 1 MB (more is torn); only the last 4 KB of ssh's stderr is kept.
    • The stream is counted, hashed and cut at max_mb. It is unpacked with Python's tar data filter, which refuses absolute paths, .., links leading out, devices, and group/other write and setuid bits, and a guard of its own on what the tar would make on disk: each member's size plus 4 KiB for the member and for each folder it makes, all within max_mb; at most 200,000 members and 200,000 folders; no member more than 64 folders deep or with a name or link target over 1 KiB (round 2).
    • Then a walk with lstat refuses a link that resolves outside the seed and anything that is not a folder, file or link, and makes every file and folder readable by jobs (o+r, folders o+rx), so a 0600 file from a cache is readable.
    • The trailer must match the bytes and SHA-256 that were received, and its bytes and made time must be the list's (round 2: a host cannot date a copy so that the next run drops it and fetches again).
    • Any failure of one seed (a torn stream, a full disk, an error this code did not foresee) is named in the log and the run goes on with the next seed; old versions are pruned in every run, before any fetch (round 3) and in a finally (round 2). Folders are removed one by one without recursion, so a tree of any depth (one an older release left) never stops a run.
    • The new version is flushed to disk and renamed into seeds/.v/NAME.STAMP. Then the link seeds/NAME is swapped to it in one rename.
  • Units:
    • t3code-fleet-seed.service: oneshot, sandboxed. It may write only its own state folder, and it has memory, CPU and task limits. Its sandbox lines are marked safety-critical.
    • t3code-fleet-seed.timer: hourly with a random delay; Persistent= catches up after sleep.
  • Desk kit (D8):
    • It installs the script and both units and enables the timer.
    • It adds ALLOW_PATHS=/var/lib/fork-fleet/seeds to launch.conf, so jobs see the seeds read-only.
    • check has a PASS/FAIL row for the timer, and one for the seed service's last result (round 2), so a seed run that fails every hour shows.
    • Uninstall disables the timer, and the seeds go with the kit's work folder.
  • Docs: the desktop paragraph in fork-fleet.md, the README, and the script header's state table.

Safety (critical)

  • Nothing from the host becomes a path except a name matching [a-z0-9][a-z0-9_-]{0,39} that the desktop's own config also lists. The list is checked field by field (NaN/Infinity refused); anything malformed is torn and nothing is fetched. The host never sets a size, age or pace.
  • The tar data filter plus a fresh empty folder per version means a hostile tar cannot write outside seeds/.new/NAME.STAMP/data (tested, one tar each: a ../ path, a device, an absolute link, a link up and out, an absolute hard link). The lstat walk after it is a second line for a Python whose filter has a known bypass: a link out of the seed is refused even with the filter patched to pass everything (tested).
  • Seeds are root's (folders 0755, no group/other write). Jobs see them read-only through the launcher's existing ALLOW_PATHS bind, so no job can change a seed another job reads.
  • A size cap (the smaller max_mb, on the stream and on what the tar would make on disk, 4 KiB counted per member and per folder), member, folder, depth and name caps, a time cap (30 min per stream, 30 s for the list), capped list output and stderr, and one run at a time (lock).
  • Runs with only CAP_DAC_OVERRIDE, ProtectSystem=strict, ProtectHome, no new privileges, and only IP and Unix sockets.
  • It refuses to run on a Python without the tar data filter. It does not check the Python patch level (3.12.0-3.12.10 and 3.13.0-3.13.3 have filter bypasses unless the distro patched them); the lstat walk and ProtectSystem=strict are the backstop.

What happens when this runs again, stops half-way, or meets existing, old or empty state?

Case What happens
Runs again The same seed on the host (same bytes and made time) is not fetched again. A newer one waits until every_hours after the last fetch. A second run while one runs exits at once (lock). Tested: fetch, then a second run fetches nothing.
Stops half-way Unpacking happens in seeds/.new/, and the next run empties it (folder by folder: a 1,500-deep tree an older release left is removed, tested). The link is swapped by one rename, so it names either the old version or the new one, never a half one. The new version is flushed (os.sync) before any link names it. The replaced version's time is touched before the link moves, so a stop there never lets it be pruned early. Tested: leftovers in .new, a bad link, a missing label.
Torn download A cut stream, a trailer that does not match the stream or the list, or is odd (²), over max_mb (folders and empty files counted), too many members, too deep, a name too long, a link out, a bad tar, a member with a size below 0 (or a non-file with a size), a link whose path is already there or that cannot be made, or any other error: the copy is removed, the old seed stays, the run goes on to the next seed, old versions are still pruned, and it is tried again after about 1, 2, 4, 8, 16, then 24 h; another copy on the host skips the wait, but only once per wait, so a host that remakes the seed every hour still meets the back-off (each tested). A leftover .NAME.link from a killed swap is removed at the start of every run.
Existing / old state A seed older than its max_age_hours, made more than an hour in the future (the clocks disagree), or no longer in the desktop's own config: the link is removed and jobs run cold. A seed the host stops listing is kept until its own age drops it. A replaced version is kept 24 h for a job still reading it (a job runs at most 1 h by default), then pruned by the next hourly run, so it lives up to about 25 h. A label that is missing or unreadable: the link is removed and the seed is fetched again in the same run.
Host down, or a host on an older kit No list (ssh fails, a reply that is not the list, exit 1 of a half-done install) means no new seeds: each seed stays until its own age drops it, and .torn.json is left as it is, so a lost list never resets the back-off (both tested). Such a run exits 0, so a host down for good shows in check only as seeds that age out (documented, see Round 5).
Its own config broken A broken own fleet config, or one without its pool, fetches nothing and exits 2 (the check row shows it), but still drops each seed past the max_age_hours it was fetched with (kept in its label; tested).
Empty state No seeds ever fetched, or a pool without seeds: nothing in seeds/, and jobs run cold. Uninstall removes the seeds with the work folder (tested).

Disk use per seed: up to max_mb x (25 / every_hours, rounded up, + 2): the copy in use, the copies replaced in the last ~25 h, and one in .new while it unpacks (a host cannot force more: the trailer's date must be the list's, and a torn copy is never kept). There is no free-space check before unpacking; a full disk tears that seed cleanly, and a .torn.json that cannot be written makes the run exit 1. Not changed (documented in fork-fleet.md): jobs can see .new/ and .v/ under the seeds folder (read-only); the 24 h keep is fixed.

General vs project

General. The fetcher, the units and the kit change name no project. Which seeds exist, their sizes and how the host makes them are the project's fleet config and its own host timer. How a job uses a seed (for example, copying it into its cache folder) is up to the project's workflow.

Upstream fit

Only the fork-only .github/scripts/fork-fleet* files, .github/fork-fleet.md and fork-kit/desk/ change. Nothing in upstream T3 changes.

Tests

  • .github/scripts/fork-fleet-desk-seed.test.py gains a Desktop class. It uses a fake ssh script that runs the real host side as a process, with modes for host down, a cut stream and a wrong trailer, and fake roots only:
    • a fetch that is checked and swapped in, then not fetched again
    • every_hours pacing, with the old version kept for running jobs and pruned after
    • a cut stream, a trailer that does not match, and over max_mb: each leaves the old seed
    • five tars that try to leave their folder: each is refused, the old seed stays, and nothing is written outside
    • a seed dropped when its own config no longer lists it, too old, or from the future
    • round 1: max_mb, every_hours and max_age_hours are the desktop's, names its config does not list are
      ignored; stderr is never kept whole and a huge list is refused; a 0600 file is readable by jobs; a link out of
      the seed is refused even past the filter; an odd trailer keeps the old seed and the run goes on; too many members
      keep the old seed
    • host down: each seed is kept until its own age drops it
    • round 2: a deep path does not stop the run; any error in one seed (and a path deeper than Python's recursion) never stops the others and old versions are still pruned; a 1,500-deep tree left by an older run is removed; folders, empty files and long names count against max_mb; the trailer's bytes and made time must be the list's; a broken or pool-less own config still drops a too old seed; an older or half-installed host means no seeds
    • a run that stopped half-way, bad links and bad labels
    • one run at a time
    • bad settings exit 2
  • fork-kit/desk/tests/install.test.py: the seed script and units installed, ALLOW_PATHS in launch.conf, the stop and enable order, the check rows for the timer and the seed service's last result, uninstall disabling the timer and removing the seeds.

Local run (round 2, D11 round 2 merged in):

  • .github/scripts/fork-fleet-desk-seed.test.py: 41 OK
  • .github/scripts/fork-fleet-desk.test.py: 42 OK
  • .github/scripts/fork-fleet-desk-agent.test.py: 31 OK
  • fork-kit/desk/tests/install.test.py: 34 OK
  • Round 2: the new seed tests against the round-1 code give 4 failures and 4 errors (deep path, any error in one seed, a deep tree left by an older run, folders against max_mb, the trailer against the list, a broken own config). The new check row test fails on round-1 desk-install. The older-host test passes on round 1 too: that behaviour needed no code change, only a test.
  • Round 1: the seed tests run against the round-0 code (test helpers adapted to its old shared path) give 14 failures and 7 errors. All seven new desktop tests are among them (host limits, every/max age, stderr cap, 0600, link past the filter, odd trailer, too many members).

Red-checks

Each rule was broken in a copy, one at a time, and the named test failed (all 12 red):

  • the tar data filter -> test_a_tar_that_would_leave_its_folder_writes_nothing_outside
  • the trailer is checked -> test_a_trailer_that_does_not_match_leaves_the_old_seed
  • max_mb cut on the desktop -> test_more_than_max_mb_leaves_the_old_seed
  • a replaced version is kept for running jobs -> test_a_new_seed_waits_for_every_hours_and_the_old_stays_for_running_jobs
  • a dropped version is kept for running jobs -> test_dropped_when_no_longer_listed_or_too_old_or_from_the_future
  • every_hours paces fetches -> test_a_new_seed_waits_for_every_hours_and_the_old_stays_for_running_jobs
  • a too old seed is dropped -> test_dropped_when_no_longer_listed_or_too_old_or_from_the_future
  • host down keeps each seed -> test_host_down_keeps_each_seed_until_its_own_age_drops_it
  • leftovers of a stopped run are emptied -> test_a_run_that_stopped_half_way_and_bad_links_or_labels
  • one run at a time -> test_one_run_at_a_time
  • the host clock guard -> the Desktop tests
  • jobs see the seeds (ALLOW_PATHS) -> fork-kit/desk/tests/install.test.py

The first run found one gap: with the keep for a replaced version removed, the test still passed. The keep time was
set from the real clock while the prune read the run's clock, so the test's later swap stayed inside 24 h by
accident. Both keeps (replaced and dropped) now use the run's clock, the test swaps more than 24 h after the old
version came, and a new check covers the dropped version.

Round 3

  • Prune runs right after the start-of-run clean, before any fetch, and again in the finally: a run killed in every fetch (memory, timeout) cannot fill .v.
  • Unpack memory is bounded in-process: members are let go one by one; extended (pax, GNU long-name) headers at most 1 MiB; sparse members refused (make seeds without --sparse); folder-name bytes capped at 16 MiB; a hard link must name a file unpacked before it. MEMBERS stays 200,000. Not done: unpacking in a child process (the bound plus prune-first cover it).
  • A deeply nested list is no list (RecursionError); a folder that cannot be listed tears the seed.
  • One run fetches for at most 2 h (RUN_WAIT), each seed at most the time left, inside the unit's 3 h.
  • seeds/.torn.json: a torn seed (or a fetch the run was killed in) is tried again after 1, 2, 4, 8, 16, then 24 h; torn on every try for more than a day makes each run exit 1, so desk-install check FAILs the seed service row. Jobs can read the file (it holds the host's reason, at most 300 characters).
  • Disk budget per seed documented (round 4 corrects it to max_mb x (25 / every_hours, rounded up, + 2)).
  • Tests: fork-fleet-desk-seed.test.py 52 OK (all 9 new tests fail on the round-2 script), install.test.py 40 OK, fork-fleet-desk.test.py 42 OK, fork-fleet.test.py 201 OK (7 skipped).

Round 4

  • A link member whose path is already there is refused, and a TarFile subclass (Tar) turns tarfile's fallback for a link it cannot make (_find_link_target, which reads the whole rest of the stream into memory) into a torn seed: a hostile tar (a hard link over a file, or a symlink over a folder, then ~108 MB of pax headers) now stays far under 512 MB (it went over 512 MB before).
  • A member with a size below 0, or a member that is not a file with a size, is refused: a negative size can no longer lower the disk count.
  • A run without the host's list exits 0 and leaves .torn.json as it is (it used to wipe it, which reset the back-off and hid a seed torn for days).
  • A .torn.json that cannot be written (a full disk) never stops the fetch: the try counts in memory, the run says so and exits 1, so check FAILs.
  • A torn seed is tried at once when the host offers another copy (the record keeps the [bytes, made] it last tried); old records without it wait as before. Round 5 limits this to once per wait.
  • Docs: retry times are approximate (the timer's 10 min random delay and a 5 min margin can add up to an hour); a replaced copy lives about 25 h, and the disk budget is corrected.
  • A test fails loud if tarfile's private _proc_member, _proc_sparse, _proc_gnusparse_* or TarFile._find_link_target go away in a later Python.
  • Tests: fork-fleet-desk-seed.test.py 59 OK (the 6 new tests give 7 failures and 1 error on the round-3 script), install.test.py 44 OK, fork-fleet-desk.test.py 42 OK, windows.test.py 24 OK (4 skipped: no PowerShell here).

Round 5

  • An extended header (pax or GNU long name) with a size below 0 is refused before it is counted (Info._proc_member). Pythons without the CVE-2025-8194 fix (3.12.0-3.12.11 and others) let it lower the header count and read a long name whole.
  • A new copy on the host skips the back-off only once per wait. The record keeps early (the last try skipped its wait); a second new copy in the same wait waits. Each try still counts, so the wait still doubles: a host that remakes a torn seed every hour gets tries at hours 0, 1, 2, 6, 7, 23, 24, then two a day: one when the 24 h wait ends, one for the next new copy (round 4: every hour). An old record without early may skip once.
  • The log line after a torn fetch says "tried again in about N h", and adds "at once if the host makes another copy before then" only when that is true.
  • A link member named with a trailing / (a/) is checked against the path a, so it cannot replace a member already there.
  • At run time the desktop refuses to fetch (exit 2, check shows it) when its Python's tarfile lacks any private part the limits rely on (TarInfo._proc_member, _proc_sparse, _proc_gnusparse_00/01/10, TarFile._find_link_target), the same as a missing data filter; seeds past their age are still dropped.
  • Not changed (documented in the script header, the README and fork-fleet.md): a run without the host's list exits 0, so a host down for good shows in check only as seeds that age out. Exiting 1 after a day without a list would be a new rule; left for later.
  • Docs: the script header, TornLog, the README and fork-fleet.md say the once-per-wait rule.
  • Tests: fork-fleet-desk-seed.test.py 63 OK (the 4 new tests give 7 failures on the round-4 script), install.test.py 47 OK, fork-fleet-desk.test.py 42 OK.

🤖 Generated with Claude Code

fork-fleet-desk-seed.py run, hourly from t3code-fleet-seed.timer (root,
sandboxed), asks the always-on host for the pool's seed list over the desk
link's key, fetches a seed only when it changed and every_hours passed,
unpacks it with tarfile's data filter, checks the trailer's size and SHA-256
and swaps it in by rename. Jobs see /var/lib/fork-fleet/seeds read-only
(ALLOW_PATHS). Torn downloads keep the old seed; old, future-dated or
unlisted seeds are dropped; replaced versions stay a day for running jobs.
The desk kit installs the script and the timer and removes the seeds on
uninstall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby
scuttletoby force-pushed the fork/desk-d11b-seed-fetch branch from fbf95e3 to ec41bd8 Compare November 15, 2026 20:02
@scuttletoby

Copy link
Copy Markdown
Owner Author

Review round 1 at ec41bd8: two independent lenses. Security: NOT SAFE. Correctness: SAFE WITH NITS. Fix round to follow; do not merge before it.

  1. NOT SAFE, the host chooses the desktop's size cap (fork-fleet-desk-seed.py:232, checked only at :203): Summed(p.stdout, item['maxMb'] << 20) takes maxMb from the host's own seeds listing; maxAgeHours and everyHours too (:329,343,345). The desktop already has its own /opt/fork-fleet/fork-fleet.json (installed by desk-install) but never reads it. A hostile host lists maxMb: 10000000 and streams a valid 200 GB tar; it is swapped in and the old copy kept 24 h; on WSL2 the virtual disk grows on the Windows drive. Both lenses found it. Fix: take all three values from the desktop's own config (pool_seeds(conf['POOL'], CONFIG)), and ignore names it does not list. Proven:
    def test_max_mb_is_the_desktops_not_the_hosts(self):   # in class Desktop
        self.config([{**SEEDS[0], 'max_mb': 3}, SEEDS[1]])  # host lists 3; the desktop's own copy says 2
        self.make('m2', {'a': os.urandom(5 << 19)})         # 2.5 MB
        self.run_seed()
        self.assertIsNone(self.files('m2'), self.said)
  2. ssh's stderr and the list's stdout have no limit (:187-189,226-228,244): stderr goes to a TemporaryFile in the unit's private /tmp (on disk); a host writing stderr for 30 min fills the disk whatever max_mb is, then err.read() hits MemoryMax=512M. Keep the last 4 KB of stderr; stop reading the list after a small limit.
  3. A 0600 file stays root-only, so jobs cannot read it (:234): the data filter never adds o+r (pip's HTTP cache writes 0600). Add o+r after unpacking (or document tar --mode=a+rX). Proven: a 0600 member comes out 0o100600.
  4. Python check too weak (:302): hasattr(tarfile, 'data_filter') passes on 3.12.0-3.12.10 / 3.13.0-3.13.3 with the June-2025 filter bypasses (CVE-2025-4517 and others) unless the distro patched them. ProtectSystem=strict limits an escape to the seeds folder. After unpacking, walk the new folder with lstat and refuse a link that resolves outside it; mark the unit's sandbox lines as safety-critical.
  5. An odd trailer crashes the whole run (:245-248): '²'.isdigit() is true but int('²') raises ValueError, not caught at :351; later seeds and prune are skipped. Use re.fullmatch(r'[0-9]{1,20}', ...) and catch ValueError.
  6. Nits: jobs see .new/ and .v/ through ALLOW_PATHS; a leftover .NAME.link (killed between :256 and :261) is cleaned only if that name is fetched again; sparse members have unbounded apparent size; very many tiny members can exhaust memory (only stops the run); KEEP_OLD fixed at 24 h; no free-space check before unpacking (a full disk is handled cleanly).

State question: lock on re-run, .new emptied after a kill mid-unpack, orphan versions pruned after 24 h, hand-made folder at seeds/NAME removed and refetched, missing/broken desk.conf exit 2, launcher skips a missing ALLOW_PATHS entry, uninstall of an older kit: all handled.

scuttletoby and others added 4 commits November 15, 2026 20:47
# Conflicts:
#	.github/scripts/fork-fleet-desk-seed.py
#	fork-kit/desk/README.md
#	fork-kit/desk/sbin/desk-install
#	fork-kit/desk/tests/install.test.py
- the desktop takes only the names its own installed fleet config lists for
  its pool, and every_hours, max_age_hours and max_mb from it (max_mb: the
  smaller of its own and the host's); the host can no longer raise them
- ssh's stderr keeps only its last 4 KB; the host's list is read up to 1 MB
- after unpacking, a walk refuses a link that leads out of the seed and any
  entry that is not a folder, file or link, and makes everything readable by
  jobs (0600 cache files)
- an odd trailer (a MADE int() cannot read) is a torn copy, never a crash
- members' sizes and count limited (sparse members, many tiny ones)
- a leftover .NAME.link is removed at the start of each run
- seeds come from the host's per-pool folder (D11); SKEW names the desktop's
  clock tolerance
- the unit's sandbox lines are marked safety-critical

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	fork-kit/desk/tests/install.test.py
…s the seed units (D11b)

The fake ssh template wrote a real newline inside a bytes literal, so every
desktop run got a syntax error instead of a list. D8's upgrade test now
names the stop call, which on this branch also stops the seed timer and
service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1 (commits bfe2987 and 8b68764, the second a test-only fix; D11's round 1 merged in at 29d9534 and 54bf612; head 8b68764). Tests: desk-seed 30 OK, desk 42 OK, install 28 OK.

  1. NOT SAFE, host picks the limits: fixed (bfe2987). Which seeds, every_hours, max_age_hours and max_mb come from the desktop's own installed fleet config (pool_seeds of its POOL); max_mb is the smaller of host and own; names its own config does not list are ignored; a bad own config exits 2. Tests: test_max_mb_is_the_desktops_not_the_hosts (the lens's test), test_every_and_max_age_hours_are_the_desktops_and_unknown_names_are_ignored.
  2. stderr/stdout without limit: fixed (bfe2987). A thread keeps only the last 4 KB of ssh's stderr (no temp file); the list is read up to 1 MB, more is torn. Test: test_stderr_is_never_kept_whole_and_a_huge_list_is_refused.
  3. 0600 stays root-only: fixed (bfe2987). After unpacking, files get o+r and folders o+rx. Test: test_a_0600_file_is_readable_by_jobs.
  4. Python check too weak: fixed (bfe2987) with an lstat walk after unpacking that refuses a link resolving outside the seed and anything not a file, folder or link; the unit's sandbox lines are marked safety-critical. Test: test_a_link_out_of_the_seed_is_refused_even_past_the_filter (filter patched to pass everything). The patch-level check itself is not added (distros backport the fix without changing the version); body says so.
  5. Odd trailer: fixed (bfe2987). re.fullmatch(r'[0-9]{1,20}'), and ValueError is caught so the run goes on. Test: test_an_odd_trailer_keeps_the_old_seed_and_the_run_goes_on.
  6. Nits: leftover .NAME.link: fixed (removed at the start of every run, tested). Many tiny members / sparse apparent size: fixed (a member count cap and a cap on summed member sizes; test test_too_many_members_keep_the_old_seed). .new/ and .v/ visible to jobs, KEEP_OLD fixed at 24 h, no free-space check: documented in fork-fleet.md and the body (read-only for jobs; a full disk tears the seed cleanly).

Fails before the fix: the seed tests run against the round-0 code (helpers adapted to its old shared path) give 14 failures and 7 errors; every new desktop test above is among them. 8b68764 fixes the test file itself: the noisy fake ssh did not compile, so desktop runs got no list. It also brings D8's upgrade test up to date with the seed units.

PR body updated. Still draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Re-check lens 237c r1 (correctness) at 8b68764: SAFE WITH NITS

Earlier findings 1-6 all FIXED, each new test failing on round-0 for its own reason (round-0 desktop side with today's host side: 8 failures + 1 error, all 7 new tests among them): host-chosen limits (fork-fleet-desk-seed.py:403-418, :436-437, :449), stderr/list caps (Tail :220-235, 1 MB list :251/:260), 0600 files (settle :313-315), Python-side checks (:309-316; hard links out blocked by ProtectSystem=strict; the patch-level check knowingly left out per the body), odd trailer (:346-347, :471), nits (.NAME.link cleanup :431-433, member/size caps :286-297, docs). Tests: seed 30, desk 42, install 28, all OK. Docs match the code.

New (both nits):

  • N1: :415-418 exits 2 on a broken or pool-less desktop config before the age drop at :441, so a fetched seed stays linked past max_age_hours while the config is broken. Fall back to the label's own maxAgeHours.
def test_a_broken_own_config_still_drops_a_too_old_seed(self):   # class Desktop
    self.make('m2', {'a': b'one'}); self.run_seed()
    with open(os.path.join(self.t, 'own.json'), 'w') as f: f.write('{')
    with self.assertRaises(agent.Bad): self.run_seed(now=lambda: time.time() + 1001 * 3600)
    self.assertIsNone(self.files('m2'))
  • N2: the check row at desk-install:439 looks only at the timer; a seed service failing every hour still shows PASS.

State question answered line by line: rerun (:461, :463, lock :422-427), killed mid-stream/unpack (:428-430, :472-474), between rename and swap (:481-487, :431), disk full / host down (:471, :438-451), seed removed from config (:445, :399), hand-made folder or bad link (:443), empty cache (:420-421), broken desk.conf (:497-500); desk-install install/upgrade/check/uninstall cover the new units (:394, :307, :261, :361, :439, :537, :569).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Re-check lens 237s r1 (security) at 8b68764: NOT SAFE

Earlier findings 1-6 FIXED (own_seeds :403-410, items :436-437, Tail :220-235, list cap :251/:260, settle :309-315, trailer :346-347, .NAME.link :431-433, member/size caps :286-297). Nit: the 0600 test passes when nothing is fetched (os.walk of a missing folder checks nothing). Tests: desk-seed 30, desk 42, install 28, all OK.

New:

  • A (HIGH): folders are not counted against max_mb. guarded :291-296 sums member.size only; a member dN/x/x/.../f makes ~900 folders at 4 KB each. Proven: a ~470 KB seed (under max_mb 2) made 135,155 folders (~527 MB); a 624 KB stream made 704 MB. A hostile or poisoned host streaming 100 MB of such members fills /var (inodes too; on WSL it grows the virtual disk), then it is swapped in as a valid seed.
  • B (HIGH): a deep path crashes the whole run. os.makedirs recurses per folder; ~1,000 levels raise RecursionError, which the except at :471 does not catch, so later seeds and prune (:477) never run. With an earlier seed changed every every_hours, .v grows by up to max_mb per swap without end.
  • C (MEDIUM): the trailer's MADE is never checked against the list's. :350 stores int(last[4]) after a digits check only; MADE=1 makes the next run drop the seed as too old (:450-453) and fetch it again. Proven: 5 fetches in 5 hourly runs with every_hours 6, each replaced copy kept 24 h (about 24 copies of max_mb, not "about two").
  • D (nits): a broken own config exits 2 (:416-418) before the age drop (:441-454), same as lens 237c r1 N1; p.stderr never closed.

Suggested fixes: refuse members deeper than ~64 folders or names over ~1 KB and charge 4 KB per member and per folder level against most; catch Exception per seed and run prune in a finally; require the trailer's bytes and MADE to equal the list's.

Failing tests (class Desktop):

def deep_tar(self, name, depth, count=1):
    tmp = os.path.join(self.t, 'files', POOL, f'.{name}.tmp')
    with tarfile.open(tmp, 'w', format=tarfile.GNU_FORMAT) as tar:
        for i in range(count):
            tar.addfile(tarfile.TarInfo(f'd{i}/' + 'x/' * depth + 'f'))
    os.utime(tmp, (time.time() + 5,) * 2); os.replace(tmp, self.tar(name))

def test_a_deep_path_does_not_stop_the_run(self):
    self.make('npm', {'a': b'n'}); deep_tar(self, 'm2', 1500)
    self.assertEqual(self.run_seed(), 0)
    self.assertEqual(self.files('npm'), {'a': b'n'})

def test_folders_count_against_max_mb(self):
    deep_tar(self, 'm2', 900, count=150); self.run_seed()
    self.assertLess(sum(len(d) for _, d, _ in os.walk(self.root)), 100000)

def test_the_trailers_made_cannot_skip_every_hours(self):
    # self.conf['SSH'] = a wrapper that runs the fake ssh and rewrites the trailer's MADE to 1
    self.make('m2', {'a': b'one'}); t0 = time.time()
    for h in range(5): self.run_seed(now=lambda: t0 + h * 3600)
    self.assertEqual(len(self.fetches()), 1)

State checks all fine: rerun (:461-464, lock :422-427), killed mid-unpack (:428-430), between rename and swap (:481-487, :431-433), during a drop (:399), hand-made seeds/NAME (:441-453), empty cache (:420-421), broken desk.conf (:497-500); ssh pinned host key, -F none, desk key only; jobs get a read-only root-owned bind.

scuttletoby and others added 2 commits November 15, 2026 22:57
# Conflicts:
#	.github/scripts/fork-fleet-desk-seed.py
#	.github/scripts/fork-fleet-desk-seed.test.py
#	fork-kit/desk/README.md
#	fork-kit/desk/sbin/desk-install
…d never stops the run (D11b)

- guarded(): each member's size plus 4 KiB for it and each folder it makes, within max_mb; at most
  200,000 members and folders; no member over 64 folders deep or with a name/link over 1 KiB
- run(): any error in one seed is said and the run goes on; prune runs in a finally, per version
- remove(): folder by folder through dir fds, no recursion (a tree of any depth)
- the trailer's bytes and MADE must be the list's
- a broken or pool-less own config still drops seeds past their label's maxAgeHours, then exits 2
- an older or half-installed host (no list) means no seeds: tested
- desk-install check: a row for the seed service's last result
- ssh's stderr pipe closed; the 0600 test fails when nothing was fetched

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 2 (head 223795c; D11 #236 round 2 merged in first)

Lens 237s r1 (security):

  • A, folders count against max_mb: fixed (223795c). guarded() now charges each member's size plus 4 KiB for the member and for each folder it makes, all within max_mb; at most 200,000 members and 200,000 folders (kept as (parent, name) pairs, so memory grows per folder name, not per full path); a member more than 64 folders deep, or with a name or link target over 1 KiB, is refused. Tests: test_folders_count_against_max_mb (your 900-deep x 150 tar now makes no folders at all; also 20 members 60 deep = 1,220 folders, 600 empty files, and a 1.2 KB name, each refused against max_mb 2 with the old seed kept).
  • B, one seed never stops the run: fixed (223795c). Each seed's age check and fetch runs in its own try (any Exception, RecursionError included); a failed fetch is said and .new emptied; prune runs in a finally, one version at a time. remove() no longer uses shutil.rmtree (it raised RecursionError on a 1,500-deep tree here): it goes folder by folder through dir fds with O_NOFOLLOW, no recursion. Tests: test_a_deep_path_does_not_stop_the_run (yours), test_any_failure_of_one_seed_never_stops_the_run_and_old_versions_are_still_pruned (a RuntimeError from one fetch, then a 1,500-deep tar with the depth/name/entry limits patched off, so RecursionError really is hit: the other seed is fetched, an old version is pruned, .new is empty), test_a_tree_deeper_than_recursion_left_by_an_older_run_is_removed (old state: 1,500-deep trees in .new and .v).
  • C, the trailer's bytes and MADE must be the list's: fixed (223795c). Test test_the_trailers_bytes_and_made_must_be_the_lists: a trailer with MADE 1 (right size and sha256) keeps the old seed, and five hourly runs after it leave one version in .v; a list whose bytes differ from the stream keeps the old seed too. Your lens test counted fetches; with the fix every hour still tries again (the copy is torn each time, nothing is kept), so the test counts kept copies instead of fetches.
  • D, nits: fixed (223795c). Broken own config: as N1 below. p.stderr is closed in listing and fetch once its reader thread ends (a child that holds it past 5 s keeps it open, and the thread).
  • 0600 test nit: fixed (223795c). It now checks the fetched files first, so it fails when nothing was fetched.

Lens 237c r1 (correctness):

  • N1, a broken or pool-less own config still drops a too old seed: fixed (223795c). The run still locks, cleans .new, and drops each seed past the maxAgeHours in its label (the desktop's own max_age_hours when it was fetched); it fetches nothing and then exits 2. Test test_a_broken_or_pool_less_own_config_still_drops_a_too_old_seed (your test, plus a config without the pool, plus "kept while under its age").
  • N2, check shows the seed service's last result: fixed (223795c). A new row t3code-fleet-seed.service last run: RESULT (from systemctl show -p Result), FAIL when it is not success. Test: test_check_names_each_failure (the fake systemctl gives exit-code).

From #236 c r1 note 4:

  • An older or half-installed host means "no seeds": already the behaviour, now tested (223795c). A reply that is not the list, or exit 1, was already caught as "no list" with each seed kept until its age drops it; no code change was needed, so this test also passes on round-1 code. Test test_an_older_or_half_installed_host_means_no_seeds (a reply that is not JSON with exit 0, and exit 1).

State question for each fix: runs again (the lock; a failed seed tries again next hour, nothing piles up in .v), stops half-way (.new emptied by the next run, a tree of any depth), old state (deep trees from an older release are removed; a label without maxAgeHours counts as unreadable and is dropped then fetched again), empty state (no own config: nothing fetched, nothing to drop, exit 2).

Red-check: the new seed tests run against the round-1 code (this branch before the fix, D11 round 2 merged in): 4 failures and 4 errors, every MUST test among them (A, B, B old state, C, N1). The install test against round-1 desk-install: test_check_names_each_failure fails (N2).

Tests at the head: desk-seed 41 OK, desk 42 OK, desk-agent 31 OK, install 34 OK.

Docs: the script's header and state table, fork-fleet.md "Seeds" and the desk README (the check row) match. PR body edited (the guard, the trailer, per-seed errors, the check row, the new state rows, tests).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check r2 (lens237sr2), read at: 223795c — NOT SAFE

A hostile host can still stop old copies from ever being pruned, so the desktop's disk fills. Small fix: also prune at the start of each run.

Earlier findings (round-1 head 8b68764)

  • 237s A, folders count against max_mb: FIXED (fork-fleet-desk-seed.py:316-343, 4 KiB per member/new folder :338, depth 64 :326, name 1 KiB :328-330). Test fails on round 1.
  • 237s B, one seed stops the run: FIXED (each :535-541 catches every Exception, prune in finally :530-531, remove :435-470 not recursive). Three tests fail on round 1.
  • 237s C, trailer MADE: FIXED (:395). Test fails on round 1.
  • 237s D and 237c N1: FIXED (broken/pool-less own config drops by label age :516-519, :559; p.stderr closed :256-260). Test fails on round 1.
  • 237c N2: FIXED (check row reads Result of t3code-fleet-seed.service).
  • Round-0 items 1-6: still fixed. Tests at head: desk-seed 41 OK, install 34 OK; against round 1: 7 failures + 5 errors, all 8 new Desktop tests among them.

New findings

  1. HIGH: a small tar uses too much memory, the service is SIGKILLed, and prune never runs (:380-381, :530-531; unit MemoryMax=512M, MemorySwapMax=0). tarfile holds a whole pax/GNU long-name header in memory and expands it about 18x; a 36 MB seed (one global pax header of 2.5M tiny records, max_mb 64) peaked at 644 MB. finally does not run on SIGKILL. Bad sequence: pool lists m2 and npm; the host offers a new m2 every every_hours and keeps npm as the memory bomb. Each run swaps in m2 (old copy to .v), then is killed on npm; .v gains a full copy (up to max_mb) every every_hours, never pruned. Same harm as 237s B by a new route. TimeoutStartSec=3h can cause the same kill (7+ seeds each streamed slowly for 30 min). Fix: prune(root, now, say) after clean at :512, before any fetch; optionally unpack in a child process or refuse pax/long-name headers over about 1 MB.
    def test_a_run_killed_in_every_fetch_still_prunes(self):          # class Desktop; needs import signal
        os.makedirs(os.path.join(self.root, '.v', 'm2.1', 'data')); os.utime(os.path.join(self.root, '.v', 'm2.1'), (1, 1))
        self.make('m2', {'a': b'm'}); self.make('npm', {'a': b'n'}); real = seed.fetch
        def oom(a, c, item, r, n):
            if item['name'] == 'npm': os.kill(os.getpid(), signal.SIGKILL)
            return real(a, c, item, r, n)
        pid = os.fork()
        if pid == 0:
            with mock.patch.object(seed, 'fetch', oom): self.run_seed()
            os._exit(0)
        os.waitpid(pid, 0)
        self.assertFalse(os.path.exists(os.path.join(self.root, '.v', 'm2.1')))
    # second test: a 36 MB tar with one global pax header of 2.5M tiny records (max_mb 64, mode 0644);
    # a forked run_seed() must end with ru_maxrss under 512 MB. It reaches 644 MB.
  2. Nit: settle can miss a link out of the seed (:351): os.walk skips a folder it cannot list (ENAMETOOLONG); links can push real paths past PATH_MAX while each member stays under 64 folders. Matters only if the data filter is bypassed (ProtectSystem=strict still limits writes). Fix: os.walk(..., onerror=...) raising Torn.
  3. Nit: disk budget per seed is up to max_mb x (24 / every_hours + 1) (25 copies for every_hours 1); write it in the "Seeds" section of fork-fleet.md.

Checked and fine: ssh (pinned ed25519 host key, -F none, IdentitiesOnly, desk key only, name checked :74); traversal/absolute/devices/links (data filter, re-checked in settle :355-363; hard links can't cross the mount); setuid stripped (:359-361), root-owned read-only; max_mb on the stream (:233) and per member (:339); atomic rename swap (:412).

State question: runs again: lock (:505-510). Killed mid-unpack: clean (:512, :544-548). Killed between rename and swap: .NAME.link removed (:513-515), orphan pruned only once a prune runs (finding 1). Killed during a drop: timestamp first (:475). Hand-made seeds/NAME: dropped (:553, :563) without following links. Broken own config: drop by label age then exit 2 (:518, :559); broken desk.conf: exit 2 (:612-616). Empty cache: folders made (:503-504). Older host: "no list" (:524-526). Install upgrade stops the seed units; uninstall disables them and removes SEEDS.

verdict NOT SAFE, PR #237, head 223795c

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check r2 (lens237cr2), read at: 223795c — SAFE WITH NITS

Tests at head: desk-seed 41 OK, install 34 OK. Against the pre-round code (c6131e4, D11 r2 merged in): seed tests 4 failures + 4 errors, install 1 failure (test_check_names_each_failure), as the fix comment says.

Earlier findings

  • 237s A, folders count against max_mb: FIXED (guarded fork-fleet-desk-seed.py:322-341, caps and depth/name limits :325-330). Test fails on old code.
  • 237s B, one seed never stops the run: FIXED (each :535-541, take catches all :582-587, prune in finally :530-531, remove not recursive :435-470). Three tests error on old code.
  • 237s C, trailer size and date: FIXED (:395-397).
  • 237s D nits: FIXED (Tail.close :256-260, called :287, :390; 0600 test checks fetched files, test :526).
  • 237c N1, broken own config still drops a too-old seed: FIXED (:559, :516-519). Both subtests fail on old code.
  • 237c N2, check shows the seed service's last result: PARTLY. Row added (desk-install:510-512, test fails on old code), but per-seed failures now exit 0 (:528, :584-587), so a seed torn every hour still shows PASS; only a broken config, no data filter or an uncaught error turns it FAIL.

New (nits)

  1. Hostile host can still run the unit out of memory (MEMBERS = 200_000 :77 vs MemoryMax=512M fork-fleet-seed.service:43): streaming tarfile keeps every member in t.members, extractall keeps folder members. Measured, members only: 199k files with ~1 KB names 307 MB peak; 199k folders 446 MB; real unpacking adds kernel dentries to the same cgroup. Needs max_mb >= ~800. Unit SIGKILLed, finally skipped, prune not run, every hour. Estimate, not proven. Fix: ~50k members or a cap on total name bytes. (See lens237sr2 fork: the server and CLI update from the desktop app's repository #1 for the proven pax-header route.)
  2. Deeply nested list crashes the run (listing :294-305): json.loads('[' * 200000) raises RecursionError, uncaught at :304/:524, escapes run (prune still runs) and main (:614) as a traceback; docs say exit 0. Fix: catch RecursionError at :304.
    def test_a_deeply_nested_list_means_no_list(self):
        self.make('m2', {'a': b'one'}); self.run_seed()
        with mock.patch.object(seed.json, 'loads', side_effect=RecursionError('deep')):
            self.assertEqual(self.run_seed(now=lambda: time.time() + 7 * 3600), 0)
        self.assertEqual(self.files('m2'), {'a': b'one'})
  3. A host that stalls on purpose can use up TimeoutStartSec=3h: FETCH_WAIT 30 min per seed (:76); more than 6 crawling seeds get the unit killed before prune.

State question: runs again: lock (:505-510), unchanged seed skipped (:575), every_hours respected (:577). Killed mid-stream/unpack: .new emptied next run (:512, :544-548). Killed after rename into .v before the link swap: .NAME.link removed (:513-515), unlinked version pruned after 24 h (:591-603). Killed between marking and swap: old one still linked, not pruned early (:409-412, :597). Disk full / host down / older host: torn, .new cleaned (:528-529); no list keeps each seed until its age drops it (:524-526). Seed removed from config: dropped :555, pruned after 24 h. Hand-made folder or label without maxAgeHours: dropped and refetched (:428-432, :553). Empty cache: :503-504. Broken own config: age drops then exit 2 (:498-501, :518-519). desk-install: upgrade stops seed units (:297), install enables the timer (:459), check covers timer and last result (:506-512), uninstall disables both (:622) and removes seeds (:654; rmtree safe at ~68 levels). Docs match apart from nits 1-3.

verdict SAFE WITH NITS, PR #237, head 223795c

scuttletoby and others added 2 commits November 16, 2026 01:05
# Conflicts:
#	fork-kit/desk/README.md
#	fork-kit/desk/sbin/desk-install
- Prune old versions right after the start-of-run clean, before any
  fetch, so a run killed in every fetch (MemoryMax, TimeoutStartSec)
  cannot let .v fill up.
- Unpack member by member and let each go; extended (pax, GNU long
  name) headers at most 1 MiB per member and globally; sparse members
  refused; folder-name bytes capped; a hard link must name a regular
  file already unpacked.
- A deeply nested list is no list (RecursionError); a folder that
  cannot be listed tears the seed.
- One run fetches for at most RUN_WAIT (2 h), each seed at most the
  time left, well inside the unit's 3 h TimeoutStartSec.
- .torn.json: a torn seed (or a fetch the run was killed in) is tried
  again after 1, 2, 4, 8, 16, then 24 h; torn on every try for over a
  day makes the run exit 1, so desk-install check shows FAIL.
- Docs: memory limits, prune order, run deadline, backoff, disk budget
  per seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 3

Commit 4812b17 (head; D11 merged in as 9b40182, with desk-install and README conflicts joined by hand). Findings from lens237sr2 (security, NOT SAFE) and lens237cr2 (correctness). All new tests fail on the round-2 script (red-check in a copy of the tree: 10 failures + 2 errors, all 9 new tests).

  • sr2 1 (HIGH, MUST), a hostile host can stop prune: fixed (4812b17).
    • prune(root, now, say) now runs right after clean, before any fetch, as well as in the finally. A run that is SIGKILLed in every fetch can no longer fill .v. Test test_a_run_killed_in_every_fetch_still_prunes (forked child, SIGKILL in the npm fetch).
    • Unpack memory is bounded: unpack takes members one at a time and lets each go (t.members is cleared). Extended headers (pax, GNU long name or link) are at most 1 MiB per member and in all for global ones. Old GNU and pax sparse members are refused. Folder-name bytes are capped at 16 MiB. A hard link must name a regular file already unpacked.
    • Tests: test_a_pax_header_seed_stays_under_the_units_memory (the 36 MB global-pax tar from the lens; peak RSS checked under 512 MB in a forked run), test_extended_headers_and_sparse_members_are_held_to_limits, test_members_are_let_go_as_they_are_unpacked and test_a_hard_link_must_name_a_file_unpacked_before_it.
  • cr2 1 (MUST), 200k members vs MemoryMax: fixed (4812b17) by not keeping members and capping folder-name bytes, so MEMBERS stays at 200,000. Not done: a child process for unpacking. The memory is bounded in-process instead, and prune-first already covers a kill.
  • cr2 2 (MUST), a deeply nested list: fixed (4812b17). listing catches RecursionError: no list, exit 0, the old seed stays. Test test_a_deeply_nested_list_means_no_list uses a fake ssh that prints [ x 200,000; a global json.loads patch would also break the config reads.
  • sr2 2 (MUST), os.walk can skip a folder: fixed (4812b17). settle walks with onerror raising Torn ("cannot be listed"). Test test_a_folder_that_cannot_be_listed_tears_the_seed.
  • sr2 3 (MUST), disk budget: documented (4812b17, fork-fleet.md "Seeds"). It is up to max_mb x (24 / every_hours + 2): the copy in use, the copies replaced in the last 24 h, and one being unpacked. With every_hours 1 and max_mb 4096 that is about 104 GiB; with every_hours 6, about 24 GiB.
  • cr2 3 / sr2 1 (MUST), 3 h TimeoutStartSec vs many slow seeds: fixed (4812b17). A run fetches for at most RUN_WAIT (2 h), and each seed gets at most the time left (and at most 30 min). The seeds left wait for the next hour. Test test_a_run_stops_fetching_when_its_time_is_used.
  • 237c N2 (MUST), check PASSes while a seed is torn every run: fixed (4812b17).
    • seeds/.torn.json records each torn seed. A try is counted before the fetch, so a kill counts as torn.
    • A torn seed is tried again after 1, 2, 4, 8, 16, then 24 h. This also covers a slow link that is always cut off (fork fleet: seed verb and seeds in the fleet config (D11) #236).
    • A seed torn on every try for more than 24 h makes each run exit 1, so desk-install check shows the seed service's last run as FAIL.
    • Test test_a_seed_torn_every_time_backs_off_then_fails_the_run_after_a_day: tries at h 0, 1, 3, 7, 15 and 31; exit 1 from h 25; back to 0 after a good fetch.

State question:

  • Runs again: prune runs first and last, and the torn log is read and rewritten (tmp file + rename).
  • Stops half-way: a kill in a fetch is already counted as torn. .new is emptied and old versions are pruned by the next run before any fetch. A leftover .torn.json.tmp is overwritten.
  • Old state: a missing or corrupt .torn.json is an empty record. An older kit ignores the file, because its name starts with . and is not a seed name. Seeds no longer in the config are dropped from the record.
  • Empty: no seeds means an empty record and nothing saved.
  • .torn.json is readable by jobs (read-only, like the seeds); it holds the host's reason, cut to 300 characters. The README says so.

Tests (local, at 4812b17): fork-fleet-desk-seed.test.py 52 OK, install.test.py 40 OK, fork-fleet-desk.test.py 42 OK, fork-fleet.test.py 201 OK (7 skipped), fork-gate-trust-docs.test.py all ok.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check r3 (lens237cr3), read at: 4812b17 — SAFE WITH NITS

Tests at head: desk-seed 52 OK, install 40 OK. Against the round-2 script (223795c) in a copy of the tree: desk-seed 10 failures + 3 errors; all 10 new tests fail there (the third error is D11's Host stall test, from the merge).

Earlier findings

New findings (nits, no data loss)

  1. One lost list wipes the torn record. When listing fails, items is {} (:599); torn.keep(items) (:602) drops every record and torn.save (:606) writes it. Bad sequence: a slow link cuts every fetch and loses the 30 s list now and then; each loss resets the backoff to hourly and resets first. Proven over 48 h with the list lost twice a day: tries at h 0, 1, 3, 7, 12, 13, 15, 19, 24, 25, … and the run never exits 1 — the fork fleet: seed verb and seeds in the fleet config (D11) #236 slow-link case N2 and the backoff were meant to catch. Fix: keep only when the list was read.
  2. Disk full never reported. trying() saves before the fetch (:651); on a full /var the save raises inside each, the seed is never fetched or counted, the run exits 0 and check PASSes forever. Fix: count in memory and let late() fire even when the save fails.
  3. A fixed seed can wait up to 24 h: the backoff ignores a new host copy (different bytes/made, :725); check FAILs meanwhile. Fix: store the tried (bytes, made) and retry at once when it changes.
  4. Retry times drift: RandomizedDelaySec=10min allows runs 50 min apart, and the 300 s margin in due (:642) then makes the "1 h" retry about 2 h now and then. Docs' "1, 2, 4…" is approximate.
def test_one_lost_list_does_not_reset_the_torn_record(self):   # class Desktop
    self.make('m2', {'a': b'one'}); start, codes, real = int(time.time()), {}, seed.listing
    def torn(*a, **k): raise seed.Torn('cut off')
    with mock.patch.object(seed, 'fetch', torn):
        for h in range(49):
            lost = lambda a, c, h=h: (_ for _ in ()).throw(seed.Torn('no list')) if h % 12 == 11 else real(a, c)
            with mock.patch.object(seed, 'listing', lost):
                codes[h] = self.run_seed(now=lambda h=h: start + h * 3600)
    self.assertTrue(any(codes[h] for h in range(25, 49)))   # fails at 4812b17d85

State question

  • Runs again: lock (:578-583), prune first and last (:589, :612), torn record rewritten by tmp + rename (:664-670).
  • Stops half-way: a kill during a fetch was already counted (:732); .new emptied (:585, :682-686); leftover .NAME.link removed (:586-588); a version renamed but not swapped is kept 24 h then pruned (:745-757); a leftover .torn.json.tmp is overwritten.
  • Old, broken, empty: missing or corrupt .torn.json counts as empty (:626-632), an older kit ignores it (:590); a broken or pool-less own config still drops seeds by age, then exits 2 (:571-574, :592-593, :697); an empty cache gets its folders (:576-577); an older host means no list (:598-600, but see finding 1).
  • Torn fetch, disk full, host down: copy torn, old seed stays (:735-740); disk full during the torn-record save escapes late() (finding 2).
  • Seed removed from config: dropped (:693-694) with its record (:635).
  • desk-install: unchanged this round; upgrade stops the seed units (desk-install:336), install enables both timers (:523), check reads the timer and the service's last result (:570-576), uninstall disables and removes seeds (:709, :741).

Docs: README and fork-fleet.md match the code apart from findings 1-4.

verdict SAFE WITH NITS, PR #237, head 4812b17

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check r3 (lens237sr3), read at: 4812b17 — NOT SAFE

Two small hostile tars still get past limits round 3 says it enforces. Both are small fixes.

All 9 new tests fail against the round-2 script 223795c (10 failures + 3 errors; one error is a D11 host-side test from the merge). Tests at head: desk-seed 52 OK, install 40 OK.

Earlier findings

  • sr2 1, a run killed in every fetch stops prune: FIXED (prune before any fetch fork-fleet-desk-seed.py:589 and in the finally :612).
  • sr2 1 / cr2 1, unpack memory under MemoryMax=512M: PARTLY. Header limits (:336-358) and letting members go (:378-380) hold for the tested cases; finding 1 gets around them.
  • sr2 2, a folder os.walk cannot list: FIXED (onerror :422-424).
  • sr2 3, disk budget: documented in fork-fleet.md; finding 2 breaks it.
  • cr2 2, deeply nested list: FIXED (:320).
  • cr2 3, the 3 h unit limit: FIXED (at most 2 h of fetching :82, :601; each seed gets the time left :728-734).
  • N2, check PASS while a seed is torn every run: FIXED (tries counted before the fetch :646-651, back-off in due :637-644, a seed torn over a day makes the run exit 1 :607-610); but see finding 3.
  • Round-1 A-D and N1: still fixed, except finding 2 reopens part of A.

New findings

  1. HIGH: a link that cannot be made makes tarfile read the whole rest of the stream into memory (unpack :361-380; linked only checks hard links, and only that the target is a regular file). When os.link/os.symlink fails, tarfile falls back to _find_link_target, which loads every remaining member, with its pax headers, with no limit from guarded. Example tars: a, b, then a hard link b → a (os.link fails, b exists); or a symlink d over a folder d/ (unlink fails). Either, followed by 80 members each with a ~0.9 MB pax header (~78 MB stream, max_mb 128), peaked at 577 MB: the unit would be OOM-killed. Prune-first and the torn count limit the damage to that seed's tries, but the round-2 memory requirement is not met. Fix: a TarFile subclass whose _find_link_target raises Torn, and refuse any link member whose path already exists (os.path.lexists).
  2. MEDIUM: a negative size bypasses the max_mb disk count (guarded :407, seen[1] += member.size + ...). A folder member with a negative base-256 size (0xff prefix) parses and its offset does not move, subtracting a huge amount from the counter so every later member and folder is free. Proven with max_mb 2: 1,500 empty files (6 MB by the 4 KiB-per-entry count) swapped in. Only the 200k member/folder caps remain: about 1.6 GB of entries per copy, times up to 26 copies with every_hours 1. Fix: in check, refuse member.size < 0 and any non-regular member with size not 0.
  3. Nit: when the list fails (host down), torn.keep({}) (:602, :606) wipes .torn.json; one failed list a day resets the back-off and check never shows FAIL. Fix: skip keep and save when there is no list. (Same as lens237cr3 finding 1.)
  4. Nit: Info relies on tarfile's private _proc_member and _proc_gnusparse_*; if a future Python renames them, the header caps stop working silently. Add a test that these methods exist.

Failing tests (class Desktop, round-3 helpers raw and block; forked() runs run_seed() in a child and returns 1 when peak RSS reached 512 MB, like the existing pax test):

def test_a_link_that_cannot_be_made_never_loads_the_stream(self):
    big = [{**SEEDS[0], 'max_mb': 128}, SEEDS[1]]; self.config(big); self.own(big)
    hard = tarfile.TarInfo('b'); hard.type, hard.linkname = tarfile.LNKTYPE, 'a'
    x = tarfile.TarInfo('pax'); x.type = tarfile.XHDTYPE
    body = b''.join(b'15 a.%07d=x\n' % i for i in range(65_000))           # ~0.98 MB, under HEADER_MOST
    rest = b''.join(self.block(x, body) + self.block(tarfile.TarInfo(f'f{i}')) for i in range(80))
    self.raw('m2', self.block(tarfile.TarInfo('a'), b'1') + self.block(tarfile.TarInfo('b'), b'2') + self.block(hard) + rest)
    self.assertEqual(self.forked(), 0)   # peak 577 MB now (same with a symlink 'd' over folder 'd/x')

def test_a_negative_size_never_lowers_the_disk_count(self):
    d = bytearray(tarfile.TarInfo('z').tobuf(format=tarfile.GNU_FORMAT)); d[156:157] = tarfile.DIRTYPE
    d[124:136] = b'\xff' + b'\0' * 11; d[148:156] = b' ' * 8; d[148:156] = b'%06o\0 ' % sum(d[:512])
    self.raw('m2', bytes(d) + b''.join(self.block(tarfile.TarInfo(f'e{i}')) for i in range(1500)))
    self.run_seed()
    self.assertIsNone(self.files('m2'))   # now: 1,500 files swapped in under max_mb 2

Checked and fine

  • Hostile tar: traversal, absolute paths, links out, devices and setuid blocked (data filter plus settle :416-436); a negative-size extended header is rejected by tarfile ("invalid offset"); the stream is cut at max_mb (:249); trailer bytes, SHA-256 and the list's made time must match before anything is used (:465-470); seeds are root's and jobs mount them read-only, so a job cannot poison the next; the swap is one atomic rename (:485).
  • ssh: pinned host key, desk key only.

State question

  • Runs again: lock (:578-583).
  • Killed during a fetch: the try is already counted (:732); the next run empties .new (:585) and prunes first (:589).
  • Killed between rename and swap: the half-made .NAME.link is removed (:586-588).
  • Killed during a drop: the version's time is touched first (:548).
  • Old or empty: missing or corrupt .torn.json = empty record (:631); a broken own config drops seeds by label age, then exits 2 (:571-574, :592-593); an empty cache gets its folders (:576-577); an older host = no list (:598-600).

verdict NOT SAFE, PR #237, head 4812b17

# Conflicts:
#	fork-kit/desk/README.md
#	fork-kit/desk/sbin/desk-install
…, torn record kept

- A link member whose path is already there is refused, and a Tar subclass turns
  tarfile's fallback (_find_link_target, which reads the rest of the stream into
  memory) into Torn: a link that cannot be made never copies its target.
- A member with a size below 0, or a non-file with a size, is refused: the disk
  count can no longer be lowered.
- A run without the host's list leaves .torn.json as it is (exit 0).
- A torn record that cannot be written (full disk) never stops the fetch: the try
  counts in memory and the run exits 1.
- A torn seed is tried at once when the host offers another copy.
- Docs: retry times are approximate; a replaced copy lives about 25 h; budget.
- Test that tarfile's private parts the limits rely on are there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 4 (head c68366b)

Merged D11 round 4 (c1cbe8d302; conflicts in the desk README and the uninstall unit list resolved: the seed units stay in the list, the lock is always taken). Fixes in c68366bb30.

Security re-check r3 (lens237sr3, NOT SAFE)

  1. HIGH, a link that cannot be made loads the stream: fixed (c68366bb30). A link member whose path is already there is refused (os.path.lexists), and the Tar subclass's _find_link_target raises Torn, so tarfile never reads the rest of the stream to copy a target. Test test_a_link_that_cannot_be_made_never_loads_the_stream: a hard link over a file, and a symlink over a folder, each followed by ~108 MB of pax headers, run in a forked child (1 GB address limit): under 512 MB now, over 512 MB on round 3 for the hard link; plus a hard link whose os.link fails for another reason.
  2. MEDIUM, a negative size lowers the disk count: fixed (c68366bb30). guarded refuses a size below 0 and any member that is not a file with a size. Test test_a_negative_size_never_lowers_the_disk_count (the lens tar, and a folder with a size).
  3. Nit, a failed list wipes .torn.json: fixed, see cr3 1.
  4. Nit, private tarfile parts: fixed. test_the_private_tarfile_parts_the_limits_rely_on_are_there checks _proc_member, _proc_sparse, _proc_gnusparse_* and TarFile._find_link_target.

Correctness re-check r3 (lens237cr3, SAFE WITH NITS)

  1. One lost list wipes the torn record: fixed (c68366bb30). A run without the list exits 0 before the record is read, so nothing is kept or saved. Test test_one_lost_list_does_not_reset_the_torn_record: 48 h with the list lost twice a day, tries at h 0, 1, 3, 7, 15, 31, and exit 1 from h 25 on (except in the two lost-list runs). Round 3 tried at 0, 1, 3, 7, 12, 13, ... and never exited 1.
  2. Disk full is never reported: fixed (c68366bb30). TornLog.save keeps an OSError in unsaved and does not raise. The try counts in memory, the fetch goes on, and the run says .torn.json cannot be written and exits 1. Test test_a_torn_record_that_cannot_be_saved_still_counts_and_fails_the_run; on round 3 the seed was never fetched and the run exited 0.
  3. A fixed seed waits up to 24 h: fixed (c68366bb30). The record keeps the [bytes, made] it last tried, and another copy on the host is tried at once. Test test_a_new_copy_on_the_host_is_tried_at_once_after_torn_tries. Old records without copy wait as before.
  4. Retry times drift: documented in the script header, fork-fleet.md and the README (about 1, 2, 4, ... h; up to an hour longer). Also documented: a replaced copy lives about 25 h, not 24, and the disk budget is now max_mb x (25 / every_hours, rounded up, + 2).

State question for this round

  • Runs again: a refused tar is torn (removed from .new, back-off). A lost list leaves the record. An unwritable record makes every run exit 1 until it can be written.
  • Stops half-way: unchanged (the try is saved before the fetch, .new is emptied, prune runs first).
  • Old or empty state: a round-3 .torn.json without copy is read as before. An old seed tar with two members at one link path is now refused (a tar -c never makes one).

Tests at c68366b: fork-fleet-desk-seed.test.py 59 OK (the 6 new tests give 7 failures and 1 error on the round-3 script), install.test.py 44 OK, fork-fleet-desk.test.py 42 OK, windows.test.py 24 OK (4 skipped: no PowerShell). The memory test's child stayed under 512 MB.

PR body updated: the Round 4 section; the disk-use line (it said "about two copies", wrong for every_hours below 24); the torn and host-down rows; 25 h.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness re-check r4 (lens237cr4), read at: c68366b — SAFE WITH NITS

Round 4 fixes every earlier finding. One new nit that matters (a seed the host remakes often skips the back-off) and one small one.

Tests at head: desk-seed 59 OK, install 44 OK. With the round-3 script (c1cbe8d302) in this tree: 7 failures and 1 error, all in the 6 new tests, as the fix comment says.

Earlier findings

  • sr3 1 (HIGH), a link that cannot be made loads the rest of the stream: FIXED. Tar._find_link_target override raises Torn (fork-fleet-desk-seed.py:367-371); a link whose path is already there is refused (:383-384). The test fails on round 3 for both cases (hard link, symlink over a folder).
  • sr3 2 (MEDIUM), a negative size lowers the disk count: FIXED (:409-410); test fails on round 3.
  • sr3 3 / cr3 1, a lost list wipes the torn record: FIXED. The run returns 0 before the record is read (:615-617); test fails on round 3.
  • sr3 4, private tarfile parts: FIXED; the new test checks they exist (errors on round 3, as expected).
  • cr3 2, a full disk is never reported: FIXED. save keeps the error in unsaved (:692-704), the run exits 1 (:624-631); test fails on round 3.
  • cr3 3, a fixed seed waits up to 24 h: FIXED. A record keeps the copy it last tried (:665, :677-678); test fails on round 3.
  • cr3 4, retry times drift: FIXED (documented in the script header, fork-fleet.md and the README).

Real tars: a GNU-format and a pax tar from tar -c -C dir . (hard links, nested hard links, relative symlinks, a 0600 empty file) both unpack through Tar/Info/unpack; the new "path already there" check does not refuse them.

New findings (nits, no data loss)

  1. A host that remakes a seed often never meets the back-off (due :665). A new [bytes, made] means "try at once", but the host may remake the seed every hour. Bad sequence: a slow link that always cuts a 500 MB seed gets a 30-minute fetch every hour, for ever. late() still turns check to FAIL after 24 h because first is kept. The log line is wrong too: it says "tried again in 24 h" (:769), then tries next hour.
    def test_a_host_that_remakes_each_hour_never_meets_the_back_off(self):   # class Desktop
        start, tried = int(time.time()) - 13 * 3600, []
        def torn(agent_, conf, item, root, now, **kw):
            tried.append(round((now() - start) / 3600)); raise seed.Torn('cut off (slow link)')
        with mock.patch.object(seed, 'fetch', torn):
            for h in range(12):
                self.make('m2', {'a': b'%d' % h}, mtime=start + h * 3600 - 60)
                self.run_seed(now=lambda h=h: start + h * 3600)
        self.assertEqual(tried, [0, 1, 3, 7])   # now [0, 1, 2, ..., 11]
    Fix: cap the wait for a new copy at every_hours (or skip the wait for a new copy only once per back-off step) instead of dropping it to zero.
  2. A host that stays down never shows in check. A run without the list exits 0 even when a seed in .torn.json is late (:617). Bad sequence: the host's key or address changes for good; every run exits 0, check PASSes, seeds age out and jobs run cold. Documented, unchanged from earlier rounds. Optional fix: exit 1 when there has been no list for more than TORN_DAY.

State question

  • Runs again: lock (:595-600); prune first and last (:606, :634); record written as temp + rename (:697-702).
  • Stops half-way: a kill during a fetch was already counted (trying saves first, :672-679); next run empties .new (:602) and removes a leftover .NAME.link (:603-605); a leftover .torn.json.tmp is overwritten later and is not a seed name.
  • Old, broken or empty state: a round-3 record without copy keeps the back-off (:665); a missing or corrupt record counts as empty (:651-657); a broken own config still drops seeds by age, then exits 2 (:609-610); an empty cache gets its folders (:593-594); an older or down host gives no list: exit 0, record left as is (:615-617).
  • Disk full: the try is kept in memory, exit 1 (:624-631).
  • Seeds removed from config: dropped (age_out), record goes on the next run that reads the list (:659-660).
  • desk-install: upgrade stops both seed units (:371); install enables the timer (:566); check covers the timer and the service's last result (:609-615); uninstall disables both (:751) and removes the seeds (:783); the merge kept the seed units in the uninstall list.
  • Docs match the code apart from nit 1; disk-budget figures (108 GiB, 28 GiB) are right.

verdict SAFE WITH NITS, PR #237, head c68366b

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security re-check r4 (lens237sr4), read at: c68366b — SAFE WITH NITS

Every round-3 MUST is fixed, and each new test fails on the round-3 code. One new medium issue that needs an older Python; the fix is one line.

Tests: at head all pass (fork-fleet-desk-seed.test.py 59, install.test.py, fork-fleet-desk.test.py). Round-4 tests against the round-3 script (4812b17): 7 failures and 1 error, all from the 6 new tests, as the fix comment says.

Earlier findings

  • sr3 1 (HIGH), a link that cannot be made loads the stream: FIXED. Tar._find_link_target override raises Torn (fork-fleet-desk-seed.py:367-372); link members whose path exists are refused (:385-386). Attacked: hard link b/ over file b gets past the existence check (lexists('b/') is False) but the override raises Torn (two layers hold); symlink d/ over folder d refused; folder d then hard link d refused; hard link to a symlink refused (:387-393). Python 3.12 calls the override from makelink_with_filter, and the "os.link fails" subtest exercises that path.
  • sr3 2 (MEDIUM), negative size: FIXED for members (:409-410); an unknown type or FIFO with a size is refused too.
  • cr3 1 / sr3 3, a lost list wipes the torn record: FIXED. return 0 before the record is read (:618); the finally still prunes (:632).
  • cr3 2, a torn save that fails: FIXED. save keeps the error in unsaved (:692-704); the try counts in memory and late() still runs (:622-629).
  • cr3 3, a new host copy: FIXED (:662-670, :676-678).
  • sr3 4, private tarfile parts: PARTLY. The test checks they exist, but only on CI's Python; the desktop's own Python is never checked at run time (finding 3).

New findings

  1. MEDIUM: a negative-size extended header lowers the header-memory count on Pythons without the CVE-2025-8194 fix. Info._proc_member adds self.size before tarfile looks at it (:348-352). The script relies on tarfile's 2025 _block refusal of sizes below 0, which only patched Pythons have (3.12.12+, 3.13.6+, or a distro backport).
    • Bad sequence on an older Python that passes the data_filter check (3.12.0-3.12.11, 3.11.4+ before mid-2025, unpatched): a GNU long-name header of size -1 MB, then a 2 MB (or 400 MB) long name, read whole. Proven by putting back the old _block: a 2 MB name got through and used ended at [0, 0]. With max_mb 4096 that kills the unit at MemoryMax (the round-2 requirement again).
    • Fix: if self.size < 0: raise Torn(...) at the top of Info._proc_member.
    def test_a_negative_extended_header_never_lowers_the_header_count(self):
        def hdr(name, typ, size):
            d = bytearray(tarfile.TarInfo(name).tobuf(format=tarfile.GNU_FORMAT)); d[156:157] = typ; d[124:136] = size
            d[148:156] = b' ' * 8; d[148:156] = b'%06o\0 ' % sum(d[:512]); return bytes(d)
        neg = b'\xff' + ((1 << 88) - (1 << 20)).to_bytes(11, 'big')          # -1 MB, base-256
        body = b'A' * ((2 << 20) - 1) + b'\0'                                # 2 MB long name, over HEADER_MOST
        blob = hdr('n', tarfile.GNUTYPE_LONGNAME, neg) + hdr('b', tarfile.GNUTYPE_LONGNAME, b'%011o\0' % len(body)) \
            + body + tarfile.TarInfo('f').tobuf(format=tarfile.GNU_FORMAT) + b'\0' * 1024
        old_block = lambda self, n: -(-n // 512) * 512                      # tarfile before the 2025 fix
        with mock.patch.object(tarfile.TarInfo, '_block', old_block), self.assertRaises((seed.Torn, tarfile.ReadError)):
            with seed.Tar.open(fileobj=io.BytesIO(blob), mode='r|', tarinfo=seed.Info) as t:
                t.next()
  2. Nit: a link name ending in / gets past the existence check (:385). A symlink a/ -> x replaces the file a and the tar settles, which the docstring says cannot happen. It stays inside the seed (content harm only). Fix: got.name.rstrip('/').
  3. Nit: the desktop checks only hasattr(tarfile, 'data_filter') at run time (:597). If a future Python drops _find_link_target or _proc_member, the limits are lost silently. Fix: refuse to fetch (exit 2) when those parts are missing, like the existing check.
  4. Nit: a new host copy skips the back-off (:666). A host that makes a bad copy every hour gets it tried every hour, and a fetch that kills the run means seeds sorted after it are never tried. Memory is now bounded, so unlikely, and check would show the kill.

Checked and fine: traversal, absolute paths, links out (incl. a/ -> ../..), devices, FIFOs, setuid (stripped), ownership (data_filter leaves files as root) refused or neutralised. The stream is cut at max_mb (:252); trailer size and SHA-256 must match the list before settle and the swap (:478-485). Swap is one rename (:497). Jobs see seeds read-only. Unit sandboxed (ProtectSystem=strict, MemoryMax=512M). ssh: pinned host key, desk key only.

State question

  • Runs again: lock (:600-603); a refused tar counts as torn and backs off (:771-777).
  • Stops half-way: try saved before the fetch (:676-679); next run empties .new (:605), removes a leftover .NAME.link (:606-608), prunes first (:609); a half-written .torn.json.tmp is overwritten (:697).
  • Old, broken or empty state: round-3 record without copy waits as before (:666); missing or corrupt record = empty (:648-655); full disk -> exit 1 (:622-629); broken own config drops seeds by age, exit 2 (:610-613); empty cache gets its folders (:596-597); no list -> exit 0, record kept (:618); an old tar with two members at one link path is now refused (documented).

verdict SAFE WITH NITS, PR #237, head c68366b

scuttletoby and others added 2 commits November 16, 2026 02:33
…tive extended header, slash links, tarfile parts at run time (D11b)

- Info._proc_member refuses a size below 0 before it is counted (Pythons
  without the CVE-2025-8194 fix).
- A new copy on the host skips the back-off only once per wait (`early`
  in .torn.json); the log line says "about N h" and the at-once note only
  when it holds.
- A link name ending in / is checked against its path without the slash.
- run() refuses to fetch (exit 2) when tarfile lacks a private part the
  limits rely on.
- Docs: the once-per-wait rule; a host down for good shows only as seeds
  that age out (documented, not changed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 5

Merged #236 (round 5 of the stack below, #225's fixes) first, then fixed the round-4 re-checks (lens237cr4, lens237sr4) in 5be550a.

lens237sr4

  1. MEDIUM, a negative-size extended header lowers the header count: fixed (5be550a). Info._proc_member raises Torn on a size below 0 before it counts it. Test test_a_negative_extended_header_never_lowers_the_header_count (the lens test, with tarfile's old _block put back) fails on round 4.
  2. Nit, a link name ending in /: fixed (5be550a). The existence check uses got.name.rstrip('/'). Test test_a_link_named_with_a_slash_over_a_file_is_refused fails on round 4.
  3. Nit, private tarfile parts only checked in CI: fixed (5be550a). run() refuses to fetch (exit 2) when any part in PRIVATE is missing, like the data-filter check; seeds past their age are still dropped. Test test_a_python_without_the_private_tarfile_parts_fetches_nothing fails on round 4.
  4. Nit, a new host copy skips the back-off: fixed, see cr4 1.

lens237cr4

  1. A host that remakes a seed often never meets the back-off: fixed (5be550a). A new copy skips the wait only once per back-off step (early in the record); each try still counts, so the wait still doubles. Lens test test_a_host_that_remakes_each_hour_never_meets_the_back_off: tries at hours [0, 1, 2, 6, 7] in 12 h (round 4: 0 to 11, so it fails there). The log line now says "tried again in about N h" and adds "at once if the host makes another copy before then" only when that can happen (tested in the same test).
  2. A host down for good never shows in check: documented, not changed (script header, README, fork-fleet.md, PR body). An exit 1 after a day without a list is a new rule; this round adds none.

State question

  • Runs again: early is saved in .torn.json with the try, before the fetch.
  • Stops half-way: a run killed in an early try has already saved early and the higher try count, so the next run waits.
  • Old state: a round-4 record has no early, so it may skip once, then follows the new rule. A record without copy waits as before.

Tests fork-fleet-desk-seed.test.py 63 OK (the 4 new tests give 7 failures on the round-4 script), install.test.py 47 OK, fork-fleet-desk.test.py 42 OK.

Still a draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: security (lens237sr5), read at: 5be550a — SAFE WITH NITS

Security re-check after fix round 5. fork-fleet-desk-seed.test.py: 63 OK at head; the round-4 script in the same tree gets 7 failures (the 4 new tests).

Earlier findings

  • sr4 1 (MEDIUM) negative-size extended header: FIXED. Info._proc_member raises Torn before counting (fork-fleet-desk-seed.py:356-357). Attacked with base-256 negative size (refused :357), huge base-256 long-name size (header cap :362), pax size=-1048576 / folder size=-5 (member check :409), pax size=2^60 and GNU.sparse.realsize=2^40 (disk count :422).
  • sr4 2 trailing-/ link name: FIXED (:393-394), test fails on round 4.
  • sr4 3 private tarfile parts: FIXED (:345, :599-601). Exit 2, nothing fetched, aged seeds still dropped (:617-619). _proc_sparse listed but not overridden (harmless).
  • sr4 4 / cr4 1 new host copy skips back-off: FIXED (:676-702). 72 h simulation: hourly remake -> tries at h [0,1,2,6,7,23,24,48,49] (2 per 24 h); 3-hourly -> [0,1,3,6,14,15,39,42,...]. Log line truthful (:798-799); the next new copy does skip (due, :691).
  • cr4 2 host down for good: documented only. Acceptable (seeds age out, jobs run cold), but nobody is told; an optional TORN_DAY exit 1 is still worth adding later.

New (nits)

  1. A real fix can wait up to ~24 h: hourly broken remakes, h7 early try torn (tries=5), h8 good copy -> due gives 15 h, tried at h23 (round 4: at once). Bounded by the 24 h cap; late() still FAILs check (:707). Worth one doc line.
  2. The parts check proves the names exist, not that tarfile still calls them. A future _proc_gnusparse_NN handler would not be overridden. Limited harm (disk count uses real size :420; an endless map hits MemoryMax = torn try). Optional: refuse any _proc_gnusparse_* on tarfile.TarInfo that Info does not override.
  3. A bad header after the first ends the unpack quietly: pax size<0 makes next() return None; fetch reads the rest (:478-479), so a short/empty seed is swapped in. Host's own data only, nothing written outside the seed. Optional: after end-of-tar accept only zero bytes.

Checked, fine: ../absolute paths, links leading out, devices/FIFOs, setuid and g/o write bits, ownership; stream cut at max_mb and disk count before every extract; trailer size+SHA-256 checked before settle and the one-rename swap (:481-507); seeds read-only to jobs (ALLOW_PATHS bind, root-owned); ssh_argv (fork-fleet-desk-agent.py:159-167) pinned ed25519 host key, own known-hosts, desk key only, no agent.

State question: runs again: lock (:610-614), prune first and last (:620, :645), record via temp+rename (:724-731). Stops half-way: trying saves early and try count before fetch (:696-704); next run empties .new (:616), removes leftover .NAME.link (:617-619). Old/empty: round-4 record without early may skip once; no copy never skips; missing/corrupt record = empty (:664-670); broken config / missing data filter or parts = exit 2 but still age-drops (:597-605, :621-623); no host list = exit 0, record kept (:629-631); full disk = exit 1 (:635-643); empty cache folders made (:607-608).

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens: correctness (lens237cr5), read at: 5be550a — SAFE WITH NITS

Round 5 fixes every round-4 finding; each of the 4 new tests fails on round 4. fork-fleet-desk-seed.test.py 63 OK, install.test.py 47 OK at head; round-4 script (c68366b) in this tree: 7 failures, all from the new tests.

Earlier findings

  • sr4 1 negative extended-header size: FIXED (fork-fleet-desk-seed.py:356-357). With and without tarfile's 2025 fix: base-256 <0 on a member and a global pax header, base-256 2^85 on a long name and pax header, pax size=-5 and a 30-digit size on a file, pax size on a symlink: all refused before any read (:357, header cap :362, max_mb, non-file with size :418).
  • sr4 2 trailing-/ link: FIXED (:393-394). Also tried a//, ./a, hard link b//, a/. over a file, d/ over a folder: all refused (this check, tarfile, or the data filter).
  • sr4 3 private tarfile parts: FIXED (:345-346, :599-601); exit 2, aged seeds still dropped (:613-615). TarFile.members (:400) not checked; semi-public, a missing one only fails fetches cleanly.
  • cr4 1 / sr4 4 hourly remake skips back-off: FIXED (:685-702). 96 h simulation: hourly remake -> h 0,1,2,6,7,23,24,48,49,73,74 (~2/day); one copy -> 0,1,3,7,15,31,55,79. Log line truthful ("about N h" matches; "at once if..." only when true).
  • cr4 2 host down for good exits 0: documented (script header :49-51, fork-fleet.md, README). Acceptable: the desk agent's own check rows show that host's state.

New (nits)

  1. A real fix can wait up to ~24 h behind the once-per-wait skip (:690): bad copies hourly at the 24 h step, one bad new copy uses the skip at h24, a fix at h30 is tried at h48. Intended trade-off; suggest one doc line: "a fix made right after another new copy waits for the step".
  2. Each early try pushes the next later (:697-699: resets last, raises the count). Bounded at 24 h; late() keeps first.

State question: runs again: lock (:609-613); early flag saved before fetch (:697-702). Stops half-way: higher count + early already saved, next run waits (:686); empties .new (:615), removes .NAME.link (:616-618), prunes first (:619); half-written .torn.json.tmp overwritten (:732). Old/broken/empty: round-4 record without early skips once; no copy never skips; non-bool early only removes the skip; missing/corrupt = empty (:663-669); full disk exit 1 (:633-638); broken config or missing tarfile part: age-drop then exit 2; no host list exit 0, record kept (:626-628). Seeds removed from config: age_out (:620-621), record goes next run (:671-672). desk-install (merged base): upgrade stops both seed units (:381), install enables the timer (:579), check rows for timer and last result (:622-628), uninstall disables both (:764) and removes seeds (:796). Docs match the code.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant