Repository navigation
feat: Grok Build host surface + host-agnostic team bus (ADR-402) - #3512
Conversation
fef3d8d to
9f0f8f4
Compare
9f0f8f4 to
a0f8cd7
Compare
| }; | ||
|
|
||
| function readPrompt(path) { | ||
| return spawnSync('/bin/cat', [path], { encoding: 'utf8' }).stdout || ''; |
Extend the host-agnostic team bus from ruvnet#3512 beyond Grok. - Host-adapter seam (mcp-tools/team-hosts/): one TeamHostAdapter per host (grok, claude, codex, command) supplies the spawn plan, the protocol lines and stop-identity parsing. Grok's plan output is unchanged. - Codex host: team_spawn returns a `codex exec` plan (prompt on stdin, --json events, -o last message, no bypass flags). `ruflo team run` executes it; turn.completed with exit 0 is done, turn.failed, a non-zero exit or a timeout is failed. No hook trust is needed on this path. - Generic command host: any agent CLI declared in .claude-flow/team-hosts.json (argv only, never a shell, strictly validated) can be a teammate. - `ruflo team <verb> --params`, `team run` and `team hook-stop`. The runner drains the member's inbox into a "Messages for you" block in the prompt, so a child without Ruflo MCP still gets queued work, and re-queues them if the run fails. A failed run reports to the lead only and does not advance the plan. - team_on_stop takes an outcome, runId (duplicates are no-ops) and reason, and normalizes `role:agent` names. - team.json gets schemaVersion 1: v0 data is read and upgraded, newer versions are refused, writes use an exclusive lock plus tmp+rename. The team tools are the only writer; the Grok bus script becomes a shim over `ruflo team`. - init --codex adds an Agent Teams section to AGENTS.md, an agent-teams skill, and a SubagentStop hook in .codex/hooks.json (on by default, --no-team-hooks to skip). Hooks call the local CLI (`npx --no-install ruflo`), never ruflo@latest. - @claude-flow/codex: extract runHeadlessProcess and buildWorkerEnvironment; version 3.0.4. - Host conformance bench covers codex and command hosts; the live probe spends model turns only with --execute / --live. - ADR-402 rewritten as one ADR: bus, format, adapter seam, command host, runner and stop signal, then Codex, Grok and Claude. Co-Authored-By: RuFlo <ruv@ruv.net>
0dee8bb to
a0f8cd7
Compare
|
Added
|
proffesor-for-testing
left a comment
There was a problem hiding this comment.
Thanks Mat — the team-bus direction is right, and the core is in good shape: pnpm install --frozen-lockfile, the cli build and tsc --noEmit are clean (0 errors), the new team-tools / ollama-local-default-model tests pass (9/9), and 250 related existing tests pass on this head and on main (b14c79e) alike. The spawn plan does carry only prompt, description, background, isolation, and path traversal in team names is rejected.
Requesting changes for the items below. Each (R) was reproduced on head 0dee8bb81; (C) is from reading the code, cross-checked by a second reviewer (Codex).
Blocking
init --grokcan corrupt a valid~/.grok/config.toml(R).grok-generator.ts:136-149detects an existing status line only by the literal header line[ui.status_line], then appends a new block. Existing valid configs using an inline table ([ui]+status_line = {…}), dotted keys (status_line.type = …),[ ui.status_line ]or[ui."status_line"]all become invalid TOML (tomllib: Cannot declare ('ui','status_line') twice). Grok would then presumably fail to read the user config.- Silent write to the user's home (R).
grok-generator.ts:245-254writes (or creates)~/.grok/config.tomlon everyinit --grok, with no opt-out, no dry-run, and the path is not listed in the init summary. Suggest: make it opt-in (e.g.--grok-statusline) or print the snippet for the user to paste; if writing, parse and merge with a TOML library. - Mailboxes are per agent name, not per team (R).
team-tools.ts:148,253,450-458—team_inboxvalidatesteamand then ignores it. A message sent tocoderon team A was returned byteam_inboxforcoderon team B.team_statusalso counts other teams' mail. - Lost updates on
team.json(R).team-tools.ts:169-180,341-366andscripts/grok-team-bus.mjs:94-96do read-modify-write with no lock and no atomic rename. 30 parallelteam_spawncalls left 13/30 members registered. Parallel SubagentStop hooks can drop plan advances the same way. - The SubagentStop hook never advances the plan (R). The spawn plan sets
description: "role:agent"(team-tools.ts:183-188); the hook falls back todescriptionas the agent name (grok-subagent-stop-hook.mjs:51-72), andon-stoprejectscoder:alice(:is not allowed). Stdio is ignored, so it fails silently — the plan stayed at['ready','pending'].
Major
- The advertised status row can't be reached (R).
host-statusline.mjs:120expects two skills (agent-teams-grok,handoff) but onlyagent-teams-grokships, so afterinitthe best output isRuFlo │ missing skills 1/2. - User-level status command runs a repo-relative script in every repo (C).
grok-generator.ts:123installssh -c 'test -f scripts/host-statusline.mjs && node scripts/host-statusline.mjs'into the user config, so any repo opened in Grok that shipsscripts/host-statusline.mjsgets it executed — unless Grok gates status-line commands on folder trust. Please pin it to an absolute path or aruflosubcommand. team_shutdowndoesn't close the team, andmaxAgentsisn't enforced (C).team_shutdownonly setsstatus(:650-661), but spawn and send check only that the team exists (:341,:390).maxAgentsis stored (:305) and never checked at spawn.- Fresh install can't use what init advertises (C). The generated MCP config launches
ruflo@latest(templates/grok/config.toml:12-28, which itself notes a local build is needed until a release shipsteam_*), whileinit.ts:449printsteam_create / team_spawnas next steps. - Windows (C). The status command requires
sh, and a Windows home path inserted into a TOML basic string keeps backslashes (\U…), which is an invalid escape (grok-generator.ts:115-135).
Minor
- Broadcasts sent before any member exists go to
_broadcast/(:413-426), whichteam_inboxcan never read (names must start alphanumeric) (R). priorityis unvalidated:"abc"producedNaN_msg_….json; filenames sort lexically, so10_sorts before2_(R).- A mailbox directory that is a symlink is followed on write (
:417-431) (C). - Two concurrent inbox drains throw
ENOENTonrenameSync;activeTeam()picks the first active team when several are active (C).
Scope
- The memory tag-filter / export v1.1 changes (
memory-bridge.ts,memory-initializer.ts,memory-tools.ts) and the provider-precedence / Ollama default-model changes (agent-execute-core.ts:190,201,508-523) aren't Grok-related and change behaviour for existing users: with an explicit provider set, a strayOPENROUTER_API_KEYno longer wins, andclaude-*model IDs now map to the Ollama default. Could those go in their own PR? (The tag filter also loads up to 100k rows into memory.) scripts/*are byte-identical copies oftemplates/grok/scripts/*, and the repo-root.grok/anddocs/grok/README.mdhave already drifted from the templates. The 1,785-line benchmark JSON, 1,101-line bench script and 464-lineREADME-GROK.mdwould be easier to review separately.
CI / merge order
- No checks appear in the PR UI because this is a fork PR: 28 workflow runs on
0dee8bb8areaction_required, waiting for maintainer approval. - #3513 is stacked on
a0f8cd71ewithout0dee8bb81; the only rebase conflict isscripts/probe-host-live.mjs(see the review there).
Review method: three agent reviewers built each PR in isolated worktrees and ran the new and related suites plus live CLI/MCP probes, compared against a main baseline; Codex CLI then cross-checked the findings and added the (C) items. — Dragan (QE)
0dee8bb to
6f5badc
Compare
|
Thanks, Dragan. That was a thorough review, and the reproductions made the fixes easy to test. I force-updated the branch with a single commit on main (fa61df2). Point by point: Blocking
Major
Minor
Scope
Verification (fork head vs. main fa61df2): One thing I couldn't verify: whether Grok runs hook commands through a shell on Windows. The SubagentStop hook still uses |
6f5badc to
19c1afe
Compare
|
One follow-up to the reply above: I force-updated to
|
Extend the host-agnostic team bus from ruvnet#3512 beyond Grok and Claude, built on its single store (templates/grok/scripts/grok-team-bus.mjs + grok-team-store.mjs). No second store: the CLI loads those files for every read and write. - Exec hosts (mcp-tools/team-hosts/): a Codex adapter and a generic command host declared in .claude-flow/team-hosts.json (argv only, never a shell, strictly validated). team_spawn takes `hosts` and `model` and adds a host.<label> entry per exec host next to the store's grok and claude entries (spawnMember gains `hostPlans` and `model`). Unknown team_spawn keys are rejected (`task` suggests `prompt`). - `ruflo team run` runs one headless turn and records the stop: - The command is rebuilt from the built-in adapter or the validated team-hosts.json entry at run time; the plan stored in team.json is never executed. - A command host runs only after `ruflo team trust-host <label>`, recorded in ~/.claude-flow/ and tied to a digest of the entry. Shells, command wrappers and path-like commands also need --allow-unsafe-command. passEnv can never re-add secret-named or CLAUDE_FLOW_* variables. - Refuses a shut-down team, a plan step that is not ready, and a second concurrent run of the same agent (claimed under the store lock; a dead runner's claim is replaced). - Drains only this team's mailbox for the agent into the prompt and re-queues to the same team on failure. - Result file reads are capped and symlinks ignored; --dry-run shows argv, passed env names and trust state. - onStop takes outcome, runId (repeats are no-ops) and reason; a failed stop marks the step failed and does not advance. - `ruflo team <verb> --params` and `team hook-stop`. hook-stop reads stdin to EOF, reports an unparseable payload or ambiguous team on stderr instead of recording `{}`, and resolves the team from the payload, a `role:agent@team` description, TEAM_NAME, or the only active team listing the agent. - @claude-flow/codex: runHeadlessProcess, killProcessTree and buildWorkerEnvironment in dual-mode/process.ts. The child runs in its own process group; a timeout or interrupt sends SIGTERM to the group, then SIGKILL after a grace period (taskkill /T on Windows), and the call returns only after exit. Captured output keeps the head and the tail (so turn.completed survives) and decodes UTF-8 once. The orchestrator's stopAll kills process trees. No version bump. - init --codex adds an Agent Teams section to AGENTS.md, an agent-teams skill, and a SubagentStop hook in .codex/hooks.json (--no-team-hooks to skip). - init --grok never writes through a symlink inside the project and never follows a symlinked template entry. - probe-host-live: codex/command team round trips; model turns only with --execute / --live; no /bin/cat. - ADR-402 amendment: exec hosts, the trust model (including that team_send `from` and hook-stop are unauthenticated on a file bus), the runner, and the Codex host. Co-Authored-By: RuFlo <ruv@ruv.net>
proffesor-for-testing
left a comment
There was a problem hiding this comment.
Round 2 on 19c1afe17. Thanks Mat, this is a big improvement: all ten round-1 items are fixed, and we reproduced each fix. One new packaging issue blocks, and it's a one-line fix; once that's in, I'm happy to approve with nits.
| Round-1 item | Result |
|---|---|
| 1. TOML corruption | Fixed. Run from a CLI installed out of an npm pack tarball, with a temp HOME: inline table, dotted keys, [ ui.status_line ] and [ui."status_line"] stay byte-identical. Inline ui = {…}, [[ui]] and unparsable files fall back to the snippet. Merged files parse with python tomllib. |
| 2. Home write | Fixed. Plain init --grok and --dry-run write nothing under ~/.grok. |
| 3. Per-team mailbox | Fixed. Mail sent to coder on A doesn't show in B's team_inbox. |
| 4. Lost updates | Fixed. 15 CLI + 15 MCP parallel spawns register 30/30, and a stale lock is broken. See N2/N3 for what's left. |
| 5. SubagentStop | Fixed. coder:alice@H1 advances the plan; ambiguous, missing and shut-down teams print to stderr and exit 1; non-team descriptions exit 0. |
| 6–9 | Fixed. The status row is reachable once the folder is trusted. The status script lives at an absolute path under ~/.grok/ruflo/ and is refreshed on re-run. Shutdown closes spawn/send/broadcast/plan/on-stop. maxAgents is enforced. The MCP pin is ruflo@<version>. |
| 10. Windows | Read only, not run: values are escaped as TOML strings and sh is gone. |
| Minors | Fixed. Empty broadcasts are refused. Priority abc/-1/1000/1.5 is refused and filenames are zero-padded. Whitespace-only messages are refused. |
Blocking
N1: the claude-flow package doesn't ship the templates the team tools now load at runtime (R). The root package.json files list includes v3/@claude-flow/cli/dist/** but not v3/@claude-flow/cli/templates/**, and main has no templates directory at all today, so this PR is the first to depend on it. From an npm pack of the root package:
mcp exec -t team_createfails withCannot find module …/templates/grok/scripts/grok-team-bus.mjs;init --grokfails with "Grok templates not found".
The @claude-flow/cli tarball (which ruflo depends on) does include templates and works end to end, so this only hits npx claude-flow users, a published entry point (3.48.0). Fix: add v3/@claude-flow/cli/templates/** to the root files list.
Strongly suggested
- N2: stale-lock break can still admit two holders (R, narrow).
grok-team-store.mjs:163-185: waiter W1 reads the stale token; meanwhile W2 breaks the lock and C takes a fresh one; W1 then renames C's live lock aside, D takes the free slot withwx, and restoring C's lock fails. A scripted replay of that ordering left C and D both holding the lock. It needs a crashed holder plus three waiters. Suggest: if the token changed, back off and retry instead of renaming. - N3: lock wait (10 s) is shorter than the stale window (30 s) (R). After a
kill -9of a holder, spawns in the next ~20 s time out at 10 s, and the SubagentStop hook's own timeout is 5 s (subagent-stop-team.json:9). The wait also uses a synchronousAtomics.waitinside the MCP server, which blocks it (store:16-17,115).
Minor / nits
--grok-statuslinereplaces a symlinked~/.grok/config.toml(dotfiles setups) with a regular file at mode 0600 (grok-generator.ts:216-221) (R).- An integer above 2^53 in the user config crashes init with "Do not know how to serialize a BigInt" after the project files are written. The user config is untouched (
grok-generator.ts:162,250) (R). - The status row matches text instead of parsing:
trusted = truein a comment reads as trusted, and an MCP server withenabled = falsestill reads asruflo(host-statusline.mjs:54-76) (R). @iarna/tomlis TOML 0.5, so some valid TOML 1.0 (mixed-type arrays) falls back to the snippet. That's safe.- Re-creating a team with
forcekeeps its old mailbox (grok-team-bus.mjs:152-158) (C). - Priority accepts
1e2/0x10. - Stray changes:
.claude/helpers/.helpers-versionbump androuter.jsmode 644→755.
Build: pnpm install --frozen-lockfile passes, the cli build passes, and tsc --noEmit shows 0 errors. The swarm/shared/memory/neural/cli-core deps need to be built first. team-tools, grok-init and guidance-brain pass 42/42. Merges cleanly onto current main.
Round-2 method: an agent reviewer rebuilt the PR, re-ran the round-1 reproductions (including from an npm packed install), and Codex CLI cross-checked; new Codex claims were verified before inclusion. (R) = reproduced, (C) = read from code. — Dragan (QE)
N1 (blocking): root package.json "files" listed cli/dist but not cli/templates, so npm pack (and npx claude-flow) shipped a build that team_create / init --grok couldn't find grok-team-bus.mjs / grok-team-store.mjs from. Added v3/@claude-flow/cli/templates/** and verified with a real npm pack + tar extract (files present with the fix, absent without it). N2: breakStaleLock renamed a stale lock without re-checking it was still the lock it first observed. A waiter that read a stale token could have that exact lock broken by someone else and re-acquired fresh in between, and the rename would steal that live lock out from under its holder, opening a window for a fourth party to grab the vacated slot — two holders at once. Now re-verifies token + mtime immediately before renaming and backs off (no rename at all) if either changed. Added a direct regression test (monkeypatched fs to force the exact interleaving) plus a concurrent-CLI-process test with a pre-seeded stale lock. N3: LOCK_WAIT_MS (10s) was shorter than LOCK_STALE_MS (30s), so any acquire attempted in the ~20s gap between "waiter gives up" and "lock considered stale" failed even with nothing actually stuck — and the SubagentStop hook's own 5s command timeout couldn't survive either value. Lowered stale/wait to 4s/4.5s (wait >= stale, both comfortably under the hook's 5s budget with real critical sections being sub-millisecond JSON r-m-w). Also replaced the synchronous Atomics.wait lock-wait loop with an async setTimeout-based poll (sleepMsAsync), so an MCP server keeps serving other requests while a caller waits for the lock; threaded async/await through withTeamLock/createTeam/spawnMember/runCli/main and the SubagentStop hook and team-tools.ts's dispatcher. sleepMs (sync) stays only for writeJsonAtomic's bounded Windows rename retry. Minor 1: --grok-statusline replaced a symlinked ~/.grok/config.toml with a plain file, destroying the symlink. writeFileAtomic now resolves an existing symlink and writes through it instead. Minor 2: a TOML integer above 2^53 in the user's config parses to a BigInt (@iarna/toml) and crashed JSON.stringify inside the status-line merge, which ran as the LAST step of init — leaving project files already written. Now validated up front, before any file is touched, naming the offending key; init fails clean with nothing written instead of half-done. Minor 3: host-statusline.mjs matched `trusted = true` / `[mcp_servers.x]` via substring/regex, so a commented-out trusted=true read as trusted and an `enabled = false` server still counted as loaded. Can't add a runtime dependency on @iarna/toml here (this script ships as a zero-dependency copy outside any node_modules tree — project scripts/ and ~/.grok/ruflo/host-statusline.mjs). Added a small comment-aware table/key parser instead and used it for grok, and for codex's mcp_servers check too (same bug class). Minor 4: team_create --force reopened a team but left the old incarnation's mailboxes on disk. Now clears mailbox/ when recreating. Minor 5: priority accepted "1e2" and "0x10" (Number() coerces both). Now requires a plain base-10 integer literal string. Minor (stray changes): reverted an incidental .helpers-version bump and a router.js mode change (644->755) picked up from a concurrent session's helper auto-refresh — not part of this PR. Verified: cli build + tsc --noEmit clean; grok-init.test.ts and team-tools.test.ts (41 tests, including 8 new ones) and guidance-brain.test.ts pass; full `npm test` for @claude-flow/cli is green except 20 pre-existing failures across 16 files, all from @claude-flow/memory / @claude-flow/mcp / @claude-flow/codex not being built in this environment (none touch grok/team/statusline code). Co-Authored-By: RuFlo <ruv@ruv.net>
…n claims (ruvnet#3513) MAJOR A — secrets reaching command hosts through the inherited environment. buildWorkerEnvironment's base-environment strip (codex/dual-mode/process.ts) kept a narrower, separately maintained regex than command.ts's passEnv validation, and didn't generalize CLAUDE_FLOW_*. A trusted host's base env still leaked PGPASSWORD, DATABASE_URL, SSH_AUTH_SOCK, GITHUB_PAT, MYSQL_PWD, SESSION_SECRET_X, MY_AUTH, SLACK_WEBHOOK_URL and every CLAUDE_FLOW_*. New shared source of truth: codex/src/dual-mode/env-policy.ts's isProtectedEnvName, imported by both process.ts (buildWorkerEnvironment) and cli's team-hosts/command.ts (passEnv validation). Regression tests: unit-level strip check plus an end-to-end spawn of a real `printenv` host with every poisoned name from the repro. MAJOR E — SIGKILL escalation cancelled when the leader exits first. runHeadlessProcess's settle() cleared every pending timer, including a just-armed SIGKILL escalation, as soon as the group leader's `close` event fired — which can happen before a descendant that trapped SIGTERM and detached its own stdio (`exec >/dev/null 2>&1`) actually dies. The escalation is no longer tracked in the timers settle() clears; it now fires unconditionally after the grace period. Regression test reproduces the exact trap/detach scenario and asserts no descendant survives past the grace period. MAJOR F — re-spawning a member mid-run drops its concurrency claim. spawnMember (grok-team-bus.mjs) overwrote a member's whole record, dropping the status/runId/runPid/runHost a live `ruflo team run` had set, letting a second run start concurrently. Re-spawn now carries a live claim forward. onStop now validates the runId it's given against the member's *current* runId before releasing the claim, so a stop report with no runId or a stale one can't clobber a live one. Tests: respawn-mid-run still blocks a second run; onStop with missing/stale runId leaves a live claim untouched. MEDIUM B — unsafe-command list missed interpreters; trust recorded before shown. trust.ts's UNSAFE_COMMANDS now includes node, python/python3, perl, ruby, npx, git, find, and other general-purpose interpreters that run code via a flag. `ruflo team trust-host` now prints the resolved host entry before persisting the trust record, not after. Tests: interpreter classification, an end-to-end `node -e` host refused then allowed, and a spy on stdout/fs ordering proving the preview print precedes the write. MEDIUM D — SIGHUP not forwarded; drained messages lost on runner death. team-runner.ts forwards SIGHUP the same way it already forwards SIGINT/SIGTERM (a closed terminal/SSH session no longer orphans the child untracked). Because an aborted run already takes the failed-outcome requeue path, this also requeues drained-but-undelivered inbox messages for the SIGHUP case. SIGKILL of the runner itself can't be caught by definition; documented as a known limitation in docs/adr/ADR-402-host-agnostic-agent-teams.md. MINOR fixes: 1. trust.ts's recordTrust takes a cross-process lock (same O_EXCL/stale-break pattern as team.json's withTeamLock), fixing lost records and a racing --revoke getting clobbered. Regression test spawns 12 real `node` processes against the built dist (a single process can't reproduce this — recordTrust is synchronous) and asserts all 11 trusts and 1 revoke land correctly. 2. codex/initializer.ts's mergeTeamStopHook refuses a symlinked .codex directory, mirroring the Grok-side init path's existing check. 3. team-run-helpers.ts's readResultFile opens with O_NOFOLLOW and fstats the descriptor instead of lstat-then-open, closing a TOCTOU window. 4. team-runner.ts wraps run-directory/result-file/run-file handling in a try/catch: any exception after the inbox has been drained now requeues the drained messages (unless the normal failed-outcome path already did) and releases the run claim, instead of stranding the member "running". Also: root package.json's "files" array was missing v3/@claude-flow/cli/templates/** even though the cli package's own files array already includes templates — added (shared one-liner with ruvnet#3512). Build + typecheck + tests green for both packages (codex: 26/26 new-file tests, cli: 107/107 team-related tests, all existing tests still pass). Pre-existing, unrelated failures in this large monorepo (codex worktrees.test.ts macOS /tmp symlink resolution; assorted cli memory/native-binding tests) are untouched by this change and were failing before it. UNRESOLVED: @claude-flow/codex's own package.json declares 3.0.3, but npm's published `@claude-flow/codex` latest is still 3.0.2 — a @claude-flow/cli release right now would resolve the older, published 3.0.2 and miss runHeadlessProcess/env-policy and everything else added in this branch and ruvnet#3512. This needs an actual `npm publish` of @claude-flow/codex before (or as part of) shipping this PR; out of scope for a worktree change. Co-Authored-By: RuFlo <ruv@ruv.net>
|
This addresses the blocker plus everything under "strongly suggested" and "minor." Blocking
Strongly suggested
Minor
Not touched, both intentionally: the Build: |
proffesor-for-testing
left a comment
There was a problem hiding this comment.
Round 3 on 37147fa99: approving, with nits. Thanks Mat.
Verified by reproduction
- N1 (packaging): fixed. We ran
npm pack --ignore-scriptsat the repo root (the "Exit handler" error didn't occur here), and the tarball includes all fourtemplates/grok/scripts/*.mjs. We installed it into a temp project with a sandboxed HOME:init --grok --grok-statusline,mcp exec -t team_createandteam_spawnall succeed. - Minor items: a symlinked
~/.grok/config.tomlis written through and the link is kept. An integer above 2^53 is reported (naming the key) before any project file is written. The status line now parses TOML:# trusted = truedoesn't count andenabled = falseshows as missing.create --forceclears the mailbox (3 files → 0).1e2and0x10are rejected through both the CLI and MCP. - N3 wiring: the lock wait is async, and every caller awaits it (
team-tools.ts, the stop hook, the CLI). - Build and tests:
tsc --noEmitpasses, grok-init / team-tools / guidance-brain pass 49/49, and the branch merges cleanly onto currentmain. - Stray files: nothing under
.claude/differs frommain.
Follow-up suggestions (not blocking)
- The lock is never refreshed while held, and it counts as stale after 4 s (
grok-team-store.mjs:30-31). A holder that stalls for more than 4 s (SIGSTOP, laptop sleep, or the hook's 5 s timeout killing it mid-update) gets its lock broken by the next writer, and one of the two updates is lost. We reproduced this with a 6 s holder insideupdateTeam: both writers readcounter=0, and the final value was1. The N2 race is also still reachable, but only with pauses injected between the token re-check and the rename (:219-229). 100 natural runs with 3 and 6 waiters showed no overlap. Checking that the pid in the lock token is still alive (process.kill(pid, 0)) before breaking the lock would close both on a single host. - The >2^53 check at
grok-generator.ts:435also rejects the case where the user already has their own[ui.status_line], where nothing would be serialized. Falling back to printing the snippet would keep that case working as before. - Nit: the header regex at
host-statusline.mjs:111rejects]inside a quoted key, so a trusted folder like/x/a]breads as untrusted.
Round-3 method: an agent reviewer rebuilt the head, re-ran the round-2 reproductions (including a root npm pack install), and ran timing-injected lock races; Codex CLI cross-checked, and we verified its claims before including them. — Dragan (QE)
Extend the host-agnostic team bus from ruvnet#3512 beyond Grok and Claude, built on its single store (templates/grok/scripts/grok-team-bus.mjs + grok-team-store.mjs). No second store: the CLI loads those files for every read and write. - Exec hosts (mcp-tools/team-hosts/): a Codex adapter and a generic command host declared in .claude-flow/team-hosts.json (argv only, never a shell, strictly validated). team_spawn takes `hosts` and `model` and adds a host.<label> entry per exec host next to the store's grok and claude entries (spawnMember gains `hostPlans` and `model`). Unknown team_spawn keys are rejected (`task` suggests `prompt`). - `ruflo team run` runs one headless turn and records the stop: - The command is rebuilt from the built-in adapter or the validated team-hosts.json entry at run time; the plan stored in team.json is never executed. - A command host runs only after `ruflo team trust-host <label>`, recorded in ~/.claude-flow/ and tied to a digest of the entry. Shells, command wrappers and path-like commands also need --allow-unsafe-command. passEnv can never re-add secret-named or CLAUDE_FLOW_* variables. - Refuses a shut-down team, a plan step that is not ready, and a second concurrent run of the same agent (claimed under the store lock; a dead runner's claim is replaced). - Drains only this team's mailbox for the agent into the prompt and re-queues to the same team on failure. - Result file reads are capped and symlinks ignored; --dry-run shows argv, passed env names and trust state. - onStop takes outcome, runId (repeats are no-ops) and reason; a failed stop marks the step failed and does not advance. - `ruflo team <verb> --params` and `team hook-stop`. hook-stop reads stdin to EOF, reports an unparseable payload or ambiguous team on stderr instead of recording `{}`, and resolves the team from the payload, a `role:agent@team` description, TEAM_NAME, or the only active team listing the agent. - @claude-flow/codex: runHeadlessProcess, killProcessTree and buildWorkerEnvironment in dual-mode/process.ts. The child runs in its own process group; a timeout or interrupt sends SIGTERM to the group, then SIGKILL after a grace period (taskkill /T on Windows), and the call returns only after exit. Captured output keeps the head and the tail (so turn.completed survives) and decodes UTF-8 once. The orchestrator's stopAll kills process trees. No version bump. - init --codex adds an Agent Teams section to AGENTS.md, an agent-teams skill, and a SubagentStop hook in .codex/hooks.json (--no-team-hooks to skip). - init --grok never writes through a symlink inside the project and never follows a symlinked template entry. - probe-host-live: codex/command team round trips; model turns only with --execute / --live; no /bin/cat. - ADR-402 amendment: exec hosts, the trust model (including that team_send `from` and hook-stop are unauthenticated on a file bus), the runner, and the Codex host. Co-Authored-By: RuFlo <ruv@ruv.net>
…n claims (ruvnet#3513) MAJOR A — secrets reaching command hosts through the inherited environment. buildWorkerEnvironment's base-environment strip (codex/dual-mode/process.ts) kept a narrower, separately maintained regex than command.ts's passEnv validation, and didn't generalize CLAUDE_FLOW_*. A trusted host's base env still leaked PGPASSWORD, DATABASE_URL, SSH_AUTH_SOCK, GITHUB_PAT, MYSQL_PWD, SESSION_SECRET_X, MY_AUTH, SLACK_WEBHOOK_URL and every CLAUDE_FLOW_*. New shared source of truth: codex/src/dual-mode/env-policy.ts's isProtectedEnvName, imported by both process.ts (buildWorkerEnvironment) and cli's team-hosts/command.ts (passEnv validation). Regression tests: unit-level strip check plus an end-to-end spawn of a real `printenv` host with every poisoned name from the repro. MAJOR E — SIGKILL escalation cancelled when the leader exits first. runHeadlessProcess's settle() cleared every pending timer, including a just-armed SIGKILL escalation, as soon as the group leader's `close` event fired — which can happen before a descendant that trapped SIGTERM and detached its own stdio (`exec >/dev/null 2>&1`) actually dies. The escalation is no longer tracked in the timers settle() clears; it now fires unconditionally after the grace period. Regression test reproduces the exact trap/detach scenario and asserts no descendant survives past the grace period. MAJOR F — re-spawning a member mid-run drops its concurrency claim. spawnMember (grok-team-bus.mjs) overwrote a member's whole record, dropping the status/runId/runPid/runHost a live `ruflo team run` had set, letting a second run start concurrently. Re-spawn now carries a live claim forward. onStop now validates the runId it's given against the member's *current* runId before releasing the claim, so a stop report with no runId or a stale one can't clobber a live one. Tests: respawn-mid-run still blocks a second run; onStop with missing/stale runId leaves a live claim untouched. MEDIUM B — unsafe-command list missed interpreters; trust recorded before shown. trust.ts's UNSAFE_COMMANDS now includes node, python/python3, perl, ruby, npx, git, find, and other general-purpose interpreters that run code via a flag. `ruflo team trust-host` now prints the resolved host entry before persisting the trust record, not after. Tests: interpreter classification, an end-to-end `node -e` host refused then allowed, and a spy on stdout/fs ordering proving the preview print precedes the write. MEDIUM D — SIGHUP not forwarded; drained messages lost on runner death. team-runner.ts forwards SIGHUP the same way it already forwards SIGINT/SIGTERM (a closed terminal/SSH session no longer orphans the child untracked). Because an aborted run already takes the failed-outcome requeue path, this also requeues drained-but-undelivered inbox messages for the SIGHUP case. SIGKILL of the runner itself can't be caught by definition; documented as a known limitation in docs/adr/ADR-402-host-agnostic-agent-teams.md. MINOR fixes: 1. trust.ts's recordTrust takes a cross-process lock (same O_EXCL/stale-break pattern as team.json's withTeamLock), fixing lost records and a racing --revoke getting clobbered. Regression test spawns 12 real `node` processes against the built dist (a single process can't reproduce this — recordTrust is synchronous) and asserts all 11 trusts and 1 revoke land correctly. 2. codex/initializer.ts's mergeTeamStopHook refuses a symlinked .codex directory, mirroring the Grok-side init path's existing check. 3. team-run-helpers.ts's readResultFile opens with O_NOFOLLOW and fstats the descriptor instead of lstat-then-open, closing a TOCTOU window. 4. team-runner.ts wraps run-directory/result-file/run-file handling in a try/catch: any exception after the inbox has been drained now requeues the drained messages (unless the normal failed-outcome path already did) and releases the run claim, instead of stranding the member "running". Also: root package.json's "files" array was missing v3/@claude-flow/cli/templates/** even though the cli package's own files array already includes templates — added (shared one-liner with ruvnet#3512). Build + typecheck + tests green for both packages (codex: 26/26 new-file tests, cli: 107/107 team-related tests, all existing tests still pass). Pre-existing, unrelated failures in this large monorepo (codex worktrees.test.ts macOS /tmp symlink resolution; assorted cli memory/native-binding tests) are untouched by this change and were failing before it. UNRESOLVED: @claude-flow/codex's own package.json declares 3.0.3, but npm's published `@claude-flow/codex` latest is still 3.0.2 — a @claude-flow/cli release right now would resolve the older, published 3.0.2 and miss runHeadlessProcess/env-policy and everything else added in this branch and as part of) shipping this PR; out of scope for a worktree change. Co-Authored-By: RuFlo <ruv@ruv.net>
37147fa to
a1d30ab
Compare
N1 (blocking): root package.json "files" listed cli/dist but not cli/templates, so npm pack (and npx claude-flow) shipped a build that team_create / init --grok couldn't find grok-team-bus.mjs / grok-team-store.mjs from. Added v3/@claude-flow/cli/templates/** and verified with a real npm pack + tar extract (files present with the fix, absent without it). N2: breakStaleLock renamed a stale lock without re-checking it was still the lock it first observed. A waiter that read a stale token could have that exact lock broken by someone else and re-acquired fresh in between, and the rename would steal that live lock out from under its holder, opening a window for a fourth party to grab the vacated slot — two holders at once. Now re-verifies token + mtime immediately before renaming and backs off (no rename at all) if either changed. Added a direct regression test (monkeypatched fs to force the exact interleaving) plus a concurrent-CLI-process test with a pre-seeded stale lock. N3: LOCK_WAIT_MS (10s) was shorter than LOCK_STALE_MS (30s), so any acquire attempted in the ~20s gap between "waiter gives up" and "lock considered stale" failed even with nothing actually stuck — and the SubagentStop hook's own 5s command timeout couldn't survive either value. Lowered stale/wait to 4s/4.5s (wait >= stale, both comfortably under the hook's 5s budget with real critical sections being sub-millisecond JSON r-m-w). Also replaced the synchronous Atomics.wait lock-wait loop with an async setTimeout-based poll (sleepMsAsync), so an MCP server keeps serving other requests while a caller waits for the lock; threaded async/await through withTeamLock/createTeam/spawnMember/runCli/main and the SubagentStop hook and team-tools.ts's dispatcher. sleepMs (sync) stays only for writeJsonAtomic's bounded Windows rename retry. Minor 1: --grok-statusline replaced a symlinked ~/.grok/config.toml with a plain file, destroying the symlink. writeFileAtomic now resolves an existing symlink and writes through it instead. Minor 2: a TOML integer above 2^53 in the user's config parses to a BigInt (@iarna/toml) and crashed JSON.stringify inside the status-line merge, which ran as the LAST step of init — leaving project files already written. Now validated up front, before any file is touched, naming the offending key; init fails clean with nothing written instead of half-done. Minor 3: host-statusline.mjs matched `trusted = true` / `[mcp_servers.x]` via substring/regex, so a commented-out trusted=true read as trusted and an `enabled = false` server still counted as loaded. Can't add a runtime dependency on @iarna/toml here (this script ships as a zero-dependency copy outside any node_modules tree — project scripts/ and ~/.grok/ruflo/host-statusline.mjs). Added a small comment-aware table/key parser instead and used it for grok, and for codex's mcp_servers check too (same bug class). Minor 4: team_create --force reopened a team but left the old incarnation's mailboxes on disk. Now clears mailbox/ when recreating. Minor 5: priority accepted "1e2" and "0x10" (Number() coerces both). Now requires a plain base-10 integer literal string. Minor (stray changes): reverted an incidental .helpers-version bump and a router.js mode change (644->755) picked up from a concurrent session's helper auto-refresh — not part of this PR. Verified: cli build + tsc --noEmit clean; grok-init.test.ts and team-tools.test.ts (41 tests, including 8 new ones) and guidance-brain.test.ts pass; full `npm test` for @claude-flow/cli is green except 20 pre-existing failures across 16 files, all from @claude-flow/memory / @claude-flow/mcp / @claude-flow/codex not being built in this environment (none touch grok/team/statusline code). Co-Authored-By: RuFlo <ruv@ruv.net>
…us line (ruvnet#3512) A lock whose holder pid is still alive is no longer broken after 4s, so a stalled writer does not lose an update. An existing [ui.status_line] no longer fails init when another value is a BigInt. Quoted trusted-folder keys may contain ]. Co-Authored-By: RuFlo <ruv@ruv.net>
…uvnet#3512) grokTemplatesRoot moves to init/grok-templates.ts, which has no dependencies. The team_* tools import it from there, so loading the MCP tool table no longer pulls in grok-generator and @iarna/toml. On current main, analyze-json-output.test.ts runs the analyze command from a copied source tree with only cli-core and security beside it, and failed 20 of 20 with "Cannot find package '@iarna/toml'" before this change. grok-generator re-exports grokTemplatesRoot, so its callers are unchanged. Co-Authored-By: RuFlo <ruv@ruv.net>
a1d30ab to
c6472d0
Compare
Extend the host-agnostic team bus from ruvnet#3512 beyond Grok and Claude, built on its single store (templates/grok/scripts/grok-team-bus.mjs + grok-team-store.mjs). No second store: the CLI loads those files for every read and write. - Exec hosts (mcp-tools/team-hosts/): a Codex adapter and a generic command host declared in .claude-flow/team-hosts.json (argv only, never a shell, strictly validated). team_spawn takes `hosts` and `model` and adds a host.<label> entry per exec host next to the store's grok and claude entries (spawnMember gains `hostPlans` and `model`). Unknown team_spawn keys are rejected (`task` suggests `prompt`). - `ruflo team run` runs one headless turn and records the stop: - The command is rebuilt from the built-in adapter or the validated team-hosts.json entry at run time; the plan stored in team.json is never executed. - A command host runs only after `ruflo team trust-host <label>`, recorded in ~/.claude-flow/ and tied to a digest of the entry. Shells, command wrappers and path-like commands also need --allow-unsafe-command. passEnv can never re-add secret-named or CLAUDE_FLOW_* variables. - Refuses a shut-down team, a plan step that is not ready, and a second concurrent run of the same agent (claimed under the store lock; a dead runner's claim is replaced). - Drains only this team's mailbox for the agent into the prompt and re-queues to the same team on failure. - Result file reads are capped and symlinks ignored; --dry-run shows argv, passed env names and trust state. - onStop takes outcome, runId (repeats are no-ops) and reason; a failed stop marks the step failed and does not advance. - `ruflo team <verb> --params` and `team hook-stop`. hook-stop reads stdin to EOF, reports an unparseable payload or ambiguous team on stderr instead of recording `{}`, and resolves the team from the payload, a `role:agent@team` description, TEAM_NAME, or the only active team listing the agent. - @claude-flow/codex: runHeadlessProcess, killProcessTree and buildWorkerEnvironment in dual-mode/process.ts. The child runs in its own process group; a timeout or interrupt sends SIGTERM to the group, then SIGKILL after a grace period (taskkill /T on Windows), and the call returns only after exit. Captured output keeps the head and the tail (so turn.completed survives) and decodes UTF-8 once. The orchestrator's stopAll kills process trees. No version bump. - init --codex adds an Agent Teams section to AGENTS.md, an agent-teams skill, and a SubagentStop hook in .codex/hooks.json (--no-team-hooks to skip). - init --grok never writes through a symlink inside the project and never follows a symlinked template entry. - probe-host-live: codex/command team round trips; model turns only with --execute / --live; no /bin/cat. - ADR-402 amendment: exec hosts, the trust model (including that team_send `from` and hook-stop are unauthenticated on a file bus), the runner, and the Codex host. Co-Authored-By: RuFlo <ruv@ruv.net>
…n claims (ruvnet#3513) MAJOR A — secrets reaching command hosts through the inherited environment. buildWorkerEnvironment's base-environment strip (codex/dual-mode/process.ts) kept a narrower, separately maintained regex than command.ts's passEnv validation, and didn't generalize CLAUDE_FLOW_*. A trusted host's base env still leaked PGPASSWORD, DATABASE_URL, SSH_AUTH_SOCK, GITHUB_PAT, MYSQL_PWD, SESSION_SECRET_X, MY_AUTH, SLACK_WEBHOOK_URL and every CLAUDE_FLOW_*. New shared source of truth: codex/src/dual-mode/env-policy.ts's isProtectedEnvName, imported by both process.ts (buildWorkerEnvironment) and cli's team-hosts/command.ts (passEnv validation). Regression tests: unit-level strip check plus an end-to-end spawn of a real `printenv` host with every poisoned name from the repro. MAJOR E — SIGKILL escalation cancelled when the leader exits first. runHeadlessProcess's settle() cleared every pending timer, including a just-armed SIGKILL escalation, as soon as the group leader's `close` event fired — which can happen before a descendant that trapped SIGTERM and detached its own stdio (`exec >/dev/null 2>&1`) actually dies. The escalation is no longer tracked in the timers settle() clears; it now fires unconditionally after the grace period. Regression test reproduces the exact trap/detach scenario and asserts no descendant survives past the grace period. MAJOR F — re-spawning a member mid-run drops its concurrency claim. spawnMember (grok-team-bus.mjs) overwrote a member's whole record, dropping the status/runId/runPid/runHost a live `ruflo team run` had set, letting a second run start concurrently. Re-spawn now carries a live claim forward. onStop now validates the runId it's given against the member's *current* runId before releasing the claim, so a stop report with no runId or a stale one can't clobber a live one. Tests: respawn-mid-run still blocks a second run; onStop with missing/stale runId leaves a live claim untouched. MEDIUM B — unsafe-command list missed interpreters; trust recorded before shown. trust.ts's UNSAFE_COMMANDS now includes node, python/python3, perl, ruby, npx, git, find, and other general-purpose interpreters that run code via a flag. `ruflo team trust-host` now prints the resolved host entry before persisting the trust record, not after. Tests: interpreter classification, an end-to-end `node -e` host refused then allowed, and a spy on stdout/fs ordering proving the preview print precedes the write. MEDIUM D — SIGHUP not forwarded; drained messages lost on runner death. team-runner.ts forwards SIGHUP the same way it already forwards SIGINT/SIGTERM (a closed terminal/SSH session no longer orphans the child untracked). Because an aborted run already takes the failed-outcome requeue path, this also requeues drained-but-undelivered inbox messages for the SIGHUP case. SIGKILL of the runner itself can't be caught by definition; documented as a known limitation in docs/adr/ADR-402-host-agnostic-agent-teams.md. MINOR fixes: 1. trust.ts's recordTrust takes a cross-process lock (same O_EXCL/stale-break pattern as team.json's withTeamLock), fixing lost records and a racing --revoke getting clobbered. Regression test spawns 12 real `node` processes against the built dist (a single process can't reproduce this — recordTrust is synchronous) and asserts all 11 trusts and 1 revoke land correctly. 2. codex/initializer.ts's mergeTeamStopHook refuses a symlinked .codex directory, mirroring the Grok-side init path's existing check. 3. team-run-helpers.ts's readResultFile opens with O_NOFOLLOW and fstats the descriptor instead of lstat-then-open, closing a TOCTOU window. 4. team-runner.ts wraps run-directory/result-file/run-file handling in a try/catch: any exception after the inbox has been drained now requeues the drained messages (unless the normal failed-outcome path already did) and releases the run claim, instead of stranding the member "running". Also: root package.json's "files" array was missing v3/@claude-flow/cli/templates/** even though the cli package's own files array already includes templates — added (shared one-liner with ruvnet#3512). Build + typecheck + tests green for both packages (codex: 26/26 new-file tests, cli: 107/107 team-related tests, all existing tests still pass). Pre-existing, unrelated failures in this large monorepo (codex worktrees.test.ts macOS /tmp symlink resolution; assorted cli memory/native-binding tests) are untouched by this change and were failing before it. UNRESOLVED: @claude-flow/codex's own package.json declares 3.0.3, but npm's published `@claude-flow/codex` latest is still 3.0.2 — a @claude-flow/cli release right now would resolve the older, published 3.0.2 and miss runHeadlessProcess/env-policy and everything else added in this branch and as part of) shipping this PR; out of scope for a worktree change. Co-Authored-By: RuFlo <ruv@ruv.net>
|
Rebased onto 3.56.9 (55e2c90). The three reviewed commits applied cleanly. One new commit, c6472d0: |
Adds `ruflo init --grok` and nine team_* MCP tools. Together they let a lead agent on Grok Build (or Codex, or Claude) run a named agent team without Claude's SendMessage. The team state lives in the project under .claude-flow/teams/. Team bus (team_create, team_spawn, team_send, team_broadcast, team_inbox, team_plan, team_status, team_on_stop, team_shutdown) - One implementation: templates/grok/scripts/grok-team-bus.mjs (ops, CLI) plus grok-team-store.mjs (storage). The MCP tools import these files, the SubagentStop hook imports them, and init --grok copies them into a project as a CLI with no dependencies. The repo root has no second copy. - Mail is scoped per team: teams/<team>/mailbox/<agent>/. team_inbox requires the team, and team_status counts only that team's mail. - team.json updates take a cross-process lock (team.lock, an O_EXCL file holding an owner token) and are written through a temp file and a rename. A stale lock is broken by renaming it aside and checking the token, never by a blind delete, and release removes only the holder's own token. 30 concurrent spawns from separate processes register all 30. - team_shutdown closes the team. After it, spawn, send, plan and on-stop refuse. team_spawn enforces maxAgents. - A broadcast with no members is refused. priority is an integer from 0 to 999, stored as a zero-padded filename prefix. Symlinked mailbox directories are refused. Concurrent drains claim each message by rename, so no message is delivered twice and no drain throws ENOENT. - The spawn description is <role>:<agent>@<team>. The SubagentStop hook parses it to advance that team's plan. When the hook cannot record a team subagent, or the team is ambiguous, it writes the reason to stderr and exits 1. init --grok - Writes .grok/ (config, rules, four agent profiles, a skill, the SubagentStop hook), scripts/ and docs/grok/README.md in the project, and lists every path it wrote. --dry-run writes nothing. - Pins the project MCP server to the running release (ruflo@<version>, which ships team_*) instead of @latest. From a source checkout it points at that checkout's bin/cli.js. Inserted values are escaped as TOML strings, so Windows paths are valid. - Writes nothing outside the project unless --grok-statusline is passed. With the flag, it installs ~/.grok/ruflo/host-statusline.mjs and merges [ui.status_line] into ~/.grok/config.toml using @iarna/toml. The merge leaves an existing status row alone in any TOML form. It writes only when the re-parsed result keeps every other value, and otherwise prints the snippet to paste. The command is `node "<absolute path>"`, which needs no shell and runs no script from the repo. - The status row expects only the agent-teams-grok skill. Also classifies team_* in the capability brain (swarm-coordination) and adds ADR-402. Tests: __tests__/team-tools.test.ts, __tests__/grok-init.test.ts. Co-Authored-By: RuFlo <ruv@ruv.net>
N1 (blocking): root package.json "files" listed cli/dist but not cli/templates, so npm pack (and npx claude-flow) shipped a build that team_create / init --grok couldn't find grok-team-bus.mjs / grok-team-store.mjs from. Added v3/@claude-flow/cli/templates/** and verified with a real npm pack + tar extract (files present with the fix, absent without it). N2: breakStaleLock renamed a stale lock without re-checking it was still the lock it first observed. A waiter that read a stale token could have that exact lock broken by someone else and re-acquired fresh in between, and the rename would steal that live lock out from under its holder, opening a window for a fourth party to grab the vacated slot — two holders at once. Now re-verifies token + mtime immediately before renaming and backs off (no rename at all) if either changed. Added a direct regression test (monkeypatched fs to force the exact interleaving) plus a concurrent-CLI-process test with a pre-seeded stale lock. N3: LOCK_WAIT_MS (10s) was shorter than LOCK_STALE_MS (30s), so any acquire attempted in the ~20s gap between "waiter gives up" and "lock considered stale" failed even with nothing actually stuck — and the SubagentStop hook's own 5s command timeout couldn't survive either value. Lowered stale/wait to 4s/4.5s (wait >= stale, both comfortably under the hook's 5s budget with real critical sections being sub-millisecond JSON r-m-w). Also replaced the synchronous Atomics.wait lock-wait loop with an async setTimeout-based poll (sleepMsAsync), so an MCP server keeps serving other requests while a caller waits for the lock; threaded async/await through withTeamLock/createTeam/spawnMember/runCli/main and the SubagentStop hook and team-tools.ts's dispatcher. sleepMs (sync) stays only for writeJsonAtomic's bounded Windows rename retry. Minor 1: --grok-statusline replaced a symlinked ~/.grok/config.toml with a plain file, destroying the symlink. writeFileAtomic now resolves an existing symlink and writes through it instead. Minor 2: a TOML integer above 2^53 in the user's config parses to a BigInt (@iarna/toml) and crashed JSON.stringify inside the status-line merge, which ran as the LAST step of init — leaving project files already written. Now validated up front, before any file is touched, naming the offending key; init fails clean with nothing written instead of half-done. Minor 3: host-statusline.mjs matched `trusted = true` / `[mcp_servers.x]` via substring/regex, so a commented-out trusted=true read as trusted and an `enabled = false` server still counted as loaded. Can't add a runtime dependency on @iarna/toml here (this script ships as a zero-dependency copy outside any node_modules tree — project scripts/ and ~/.grok/ruflo/host-statusline.mjs). Added a small comment-aware table/key parser instead and used it for grok, and for codex's mcp_servers check too (same bug class). Minor 4: team_create --force reopened a team but left the old incarnation's mailboxes on disk. Now clears mailbox/ when recreating. Minor 5: priority accepted "1e2" and "0x10" (Number() coerces both). Now requires a plain base-10 integer literal string. Minor (stray changes): reverted an incidental .helpers-version bump and a router.js mode change (644->755) picked up from a concurrent session's helper auto-refresh — not part of this PR. Verified: cli build + tsc --noEmit clean; grok-init.test.ts and team-tools.test.ts (41 tests, including 8 new ones) and guidance-brain.test.ts pass; full `npm test` for @claude-flow/cli is green except 20 pre-existing failures across 16 files, all from @claude-flow/memory / @claude-flow/mcp / @claude-flow/codex not being built in this environment (none touch grok/team/statusline code). Co-Authored-By: RuFlo <ruv@ruv.net>
…us line (ruvnet#3512) A lock whose holder pid is still alive is no longer broken after 4s, so a stalled writer does not lose an update. An existing [ui.status_line] no longer fails init when another value is a BigInt. Quoted trusted-folder keys may contain ]. Co-Authored-By: RuFlo <ruv@ruv.net>
…uvnet#3512) grokTemplatesRoot moves to init/grok-templates.ts, which has no dependencies. The team_* tools import it from there, so loading the MCP tool table no longer pulls in grok-generator and @iarna/toml. On current main, analyze-json-output.test.ts runs the analyze command from a copied source tree with only cli-core and security beside it, and failed 20 of 20 with "Cannot find package '@iarna/toml'" before this change. grok-generator re-exports grokTemplatesRoot, so its callers are unchanged. Co-Authored-By: RuFlo <ruv@ruv.net>
c6472d0 to
6aaaf82
Compare
Extend the host-agnostic team bus from ruvnet#3512 beyond Grok and Claude, built on its single store (templates/grok/scripts/grok-team-bus.mjs + grok-team-store.mjs). No second store: the CLI loads those files for every read and write. - Exec hosts (mcp-tools/team-hosts/): a Codex adapter and a generic command host declared in .claude-flow/team-hosts.json (argv only, never a shell, strictly validated). team_spawn takes `hosts` and `model` and adds a host.<label> entry per exec host next to the store's grok and claude entries (spawnMember gains `hostPlans` and `model`). Unknown team_spawn keys are rejected (`task` suggests `prompt`). - `ruflo team run` runs one headless turn and records the stop: - The command is rebuilt from the built-in adapter or the validated team-hosts.json entry at run time; the plan stored in team.json is never executed. - A command host runs only after `ruflo team trust-host <label>`, recorded in ~/.claude-flow/ and tied to a digest of the entry. Shells, command wrappers and path-like commands also need --allow-unsafe-command. passEnv can never re-add secret-named or CLAUDE_FLOW_* variables. - Refuses a shut-down team, a plan step that is not ready, and a second concurrent run of the same agent (claimed under the store lock; a dead runner's claim is replaced). - Drains only this team's mailbox for the agent into the prompt and re-queues to the same team on failure. - Result file reads are capped and symlinks ignored; --dry-run shows argv, passed env names and trust state. - onStop takes outcome, runId (repeats are no-ops) and reason; a failed stop marks the step failed and does not advance. - `ruflo team <verb> --params` and `team hook-stop`. hook-stop reads stdin to EOF, reports an unparseable payload or ambiguous team on stderr instead of recording `{}`, and resolves the team from the payload, a `role:agent@team` description, TEAM_NAME, or the only active team listing the agent. - @claude-flow/codex: runHeadlessProcess, killProcessTree and buildWorkerEnvironment in dual-mode/process.ts. The child runs in its own process group; a timeout or interrupt sends SIGTERM to the group, then SIGKILL after a grace period (taskkill /T on Windows), and the call returns only after exit. Captured output keeps the head and the tail (so turn.completed survives) and decodes UTF-8 once. The orchestrator's stopAll kills process trees. No version bump. - init --codex adds an Agent Teams section to AGENTS.md, an agent-teams skill, and a SubagentStop hook in .codex/hooks.json (--no-team-hooks to skip). - init --grok never writes through a symlink inside the project and never follows a symlinked template entry. - probe-host-live: codex/command team round trips; model turns only with --execute / --live; no /bin/cat. - ADR-402 amendment: exec hosts, the trust model (including that team_send `from` and hook-stop are unauthenticated on a file bus), the runner, and the Codex host. Co-Authored-By: RuFlo <ruv@ruv.net>
…n claims (ruvnet#3513) MAJOR A — secrets reaching command hosts through the inherited environment. buildWorkerEnvironment's base-environment strip (codex/dual-mode/process.ts) kept a narrower, separately maintained regex than command.ts's passEnv validation, and didn't generalize CLAUDE_FLOW_*. A trusted host's base env still leaked PGPASSWORD, DATABASE_URL, SSH_AUTH_SOCK, GITHUB_PAT, MYSQL_PWD, SESSION_SECRET_X, MY_AUTH, SLACK_WEBHOOK_URL and every CLAUDE_FLOW_*. New shared source of truth: codex/src/dual-mode/env-policy.ts's isProtectedEnvName, imported by both process.ts (buildWorkerEnvironment) and cli's team-hosts/command.ts (passEnv validation). Regression tests: unit-level strip check plus an end-to-end spawn of a real `printenv` host with every poisoned name from the repro. MAJOR E — SIGKILL escalation cancelled when the leader exits first. runHeadlessProcess's settle() cleared every pending timer, including a just-armed SIGKILL escalation, as soon as the group leader's `close` event fired — which can happen before a descendant that trapped SIGTERM and detached its own stdio (`exec >/dev/null 2>&1`) actually dies. The escalation is no longer tracked in the timers settle() clears; it now fires unconditionally after the grace period. Regression test reproduces the exact trap/detach scenario and asserts no descendant survives past the grace period. MAJOR F — re-spawning a member mid-run drops its concurrency claim. spawnMember (grok-team-bus.mjs) overwrote a member's whole record, dropping the status/runId/runPid/runHost a live `ruflo team run` had set, letting a second run start concurrently. Re-spawn now carries a live claim forward. onStop now validates the runId it's given against the member's *current* runId before releasing the claim, so a stop report with no runId or a stale one can't clobber a live one. Tests: respawn-mid-run still blocks a second run; onStop with missing/stale runId leaves a live claim untouched. MEDIUM B — unsafe-command list missed interpreters; trust recorded before shown. trust.ts's UNSAFE_COMMANDS now includes node, python/python3, perl, ruby, npx, git, find, and other general-purpose interpreters that run code via a flag. `ruflo team trust-host` now prints the resolved host entry before persisting the trust record, not after. Tests: interpreter classification, an end-to-end `node -e` host refused then allowed, and a spy on stdout/fs ordering proving the preview print precedes the write. MEDIUM D — SIGHUP not forwarded; drained messages lost on runner death. team-runner.ts forwards SIGHUP the same way it already forwards SIGINT/SIGTERM (a closed terminal/SSH session no longer orphans the child untracked). Because an aborted run already takes the failed-outcome requeue path, this also requeues drained-but-undelivered inbox messages for the SIGHUP case. SIGKILL of the runner itself can't be caught by definition; documented as a known limitation in docs/adr/ADR-402-host-agnostic-agent-teams.md. MINOR fixes: 1. trust.ts's recordTrust takes a cross-process lock (same O_EXCL/stale-break pattern as team.json's withTeamLock), fixing lost records and a racing --revoke getting clobbered. Regression test spawns 12 real `node` processes against the built dist (a single process can't reproduce this — recordTrust is synchronous) and asserts all 11 trusts and 1 revoke land correctly. 2. codex/initializer.ts's mergeTeamStopHook refuses a symlinked .codex directory, mirroring the Grok-side init path's existing check. 3. team-run-helpers.ts's readResultFile opens with O_NOFOLLOW and fstats the descriptor instead of lstat-then-open, closing a TOCTOU window. 4. team-runner.ts wraps run-directory/result-file/run-file handling in a try/catch: any exception after the inbox has been drained now requeues the drained messages (unless the normal failed-outcome path already did) and releases the run claim, instead of stranding the member "running". Also: root package.json's "files" array was missing v3/@claude-flow/cli/templates/** even though the cli package's own files array already includes templates — added (shared one-liner with ruvnet#3512). Build + typecheck + tests green for both packages (codex: 26/26 new-file tests, cli: 107/107 team-related tests, all existing tests still pass). Pre-existing, unrelated failures in this large monorepo (codex worktrees.test.ts macOS /tmp symlink resolution; assorted cli memory/native-binding tests) are untouched by this change and were failing before it. UNRESOLVED: @claude-flow/codex's own package.json declares 3.0.3, but npm's published `@claude-flow/codex` latest is still 3.0.2 — a @claude-flow/cli release right now would resolve the older, published 3.0.2 and miss runHeadlessProcess/env-policy and everything else added in this branch and as part of) shipping this PR; out of scope for a worktree change. Co-Authored-By: RuFlo <ruv@ruv.net>
|
Rebased onto main
Gates ran in clean Linux containers (node 22 for V3 Packages, node 20 for the root suite), compared against the same job on main in the same environment. Main shows some failing files there that pass in real CI, so the claim is only that this branch adds none. #3513 is stacked on this one, so this can merge first. |
Minor release (new backward-compatible flags: Grok Build CLI host surface ruvnet#3512, memory list/search tag filter ruvnet#3526) bundled with a round of correctness fixes merged since 3.58.1: memory delete-sync (ruvnet#4138), real AgentDB HNSWIndex replacing the dead getController('hnsw') path with safe exact-search fallback (ruvnet#4249/ruvnet#4322), harness live-process reporting (ruvnet#3521), RaBitQ on-demand index build (ruvnet#4318/ruvnet#4319), metaharness ADR-152 cost-dimension and verdict-band fix (ruvnet#3520), swarm idle-agent heartbeat false-positive (ruvnet#4320), ephemeral-agent termination idempotency (ruvnet#4323), status/task_list field-mapping and total-count fixes (ruvnet#4321/ruvnet#4324). Also fixes the actual cause of broken images/links on npmjs.com/package/ruflo: `ruflo/package.json`'s `repository.directory: "ruflo"` field added an extra path layer on top of the README content (the root README.md, copied in verbatim by `prepublishOnly` at publish time, which is already written with repo-root-relative paths) — every relative image/link resolved to e.g. `raw/main/ruflo/ruflo/assets/...` instead of `raw/main/ruflo/assets/...`. Removed the `directory` field to match claude-flow's own (directory-less) repository entry. A prior PR (ruvnet#4325) fixed `ruflo/README.md`'s own relative paths for anyone browsing that file directly on GitHub, but that file is overwritten by the root README at publish time and was never the actual npm-rendered content. Published to npm: @claude-flow/cli@3.59.0, claude-flow@3.59.0, ruflo@3.59.0, and standalone @claude-flow/memory@3.0.8 (the ruvnet#4249 fix — not part of the 3-package train, published separately per its own leaf's npm presence). All three umbrella packages' latest/alpha/v3alpha dist-tags verified pointing at 3.59.0. Co-Authored-By: RuFlo <ruv@ruv.net>
What this adds
ADR-402 makes Ruflo's Agent Teams work as a host-agnostic MCP surface instead of a Claude Code-only feature built on
SendMessage/Task. The team bus (create/spawn/send/inbox/broadcast/plan/status/on_stop/shutdown) now lives in Ruflo asteam_*MCP tools backed by a filesystem mailbox under.claude-flow/teams/{teamId}/, withteam_spawnreturning a spawn plan keyed by host (host.grok.spawn,host.claude.taskType, ...) so any host adapter can drive it. Claude Code's teammate-plugin stays as one adapter rather than the only one.The first non-Claude adapter is Grok Build.
npx ruflo init --grokscaffolds.grok/config.toml,.grok/rules/,.grok/agents/*.md,.grok/skills/, and aSubagentStophook wired toteam_on_stop, mirroring the existinginit --codexpattern. On Grok Build 1.0.41 the lead callsspawn_subagentdirectly with the plan Ruflo hands back; it doesn't need Grok to have its own SendMessage-equivalent.Also included:
docs/grok/README.mdandREADME-GROK.md: the operator guide and the monorepo/pre-publish guide for using Ruflo from Grok Build, including running it against an external project rather than the ruflo monorepo itself.scripts/bench-grok-host-conformance.mjs: a no-LLM conformance bench that checks the MCP/CLI surface Grok is supposed to call (host surface, tool inventory, team lifecycle, swarm, hive-mind, learning loop, neural, CLI parity, init product) — this is what backs the ADR-402 Phase 4 claim, not a live Grok run.scripts/probe-host-live.mjs/scripts/probe-grok-host-live.mjs: live, single-machine probes of an actually-installed Grok/Claude/Codex binary and its MCP config. These aren't part of the conformance claim and their sample output isn't included in this PR (see "left out" below).scripts/grok-team-bus.mjs,scripts/grok-subagent-stop-hook.mjs: the hook-side implementation the templates install.v3/@claude-flow/cli/src/mcp-tools/team-tools.tsand the memory-tools/memory-initializer changes needed to support it (tag-preservingmemory export/importround-trips,team_*MCP tool registration).Two merge conflicts came up bringing this current with
upstream/main(v3/@claude-flow/cli/src/mcp-tools/memory-tools.tsand.../src/memory/memory-initializer.ts), both in code this branch didn't originally touch that upstream had since changed independently. Both were resolved to keep upstream's newer behavior (thedbPath-awarestoreEntrycall with a real success check, and the TTL-awareliveMemoryRowSql()helper) plus this branch's tag-preservation on import — not by picking one side wholesale.The design record is ADR-402 (
v3/docs/adr/ADR-402-host-agnostic-agent-teams.md). It was drafted as ADR-320 on the fork; 320, 400 and 401 are already taken upstream, so it is renumbered here.Relationship to #3373
#3373 (open, fixes #3372) makes Grok Build a detected host: a row in
defaultHostRegistry(),maybeAutoDetectGrok, and the trust instruction. This PR is complementary: it adds the team bus asteam_*MCP tools and an explicitinit --grokscaffold for Grok's agents, rules, skills andSubagentStophook. It follows the same line on folder trust.init --grokprints the/hooks-trust/grok --truststep and never writes~/.grok/trusted_folders.toml. Both PRs editv3/@claude-flow/cli/src/commands/init.ts, so whichever lands second needs a rebase there; I'm happy to do that one.How this was tested
All of this was run locally against the merged tree (
upstream/main+feat/grok-host, after the fork-local cleanup below), node 22.23.1 / pnpm 11.13.0:pnpm install --frozen-lockfileinv3/— clean, no lockfile changes needed.pnpm build(pnpm -r build, same as the CIbuildjob) — all 25 workspace packages built with no errors.pnpm typecheck(pnpm -r typecheck, same as CI) — exit 0, no errors.pnpm test(vitest run, same as CI'stestjob) — full v3 suite. Failures seen: two memory-command tests in@claude-flow/cli/__tests__/commands.test.ts("should search memory" / "should list memory entries") that need a real memory DB the test file never seeds (memory storeisit.skip'd there as "requires live memory service"), plus a handful ofgraph-analyzer.test.tsandprovider-integration.test.ts(Ollama-not-running) failures that vary between the pnpm-hoisted duplicate copies of the same spec file. I reproduced the same twocommands.test.tsfailures, deterministically, on a cleanupstream/maincheckout with no grok-host changes at all — they're pre-existing and unrelated to this branch, not something this PR introduces.testjob runs fromv3/@claude-flow/cli:node scripts/test-mcp-policy-stdio.mjs(10/10 pass),node scripts/smoke-flywheel-proof.mjs(PASSED),node scripts/replay-generation.mjs(PASSED, offline clean-room replay).node scripts/bench-grok-host-conformance.mjs(the feature's own bench) — 91/92 checks pass, 0 critical failures, 1 warning. The report ADR-402 points to (docs/benchmarks/grok-host-conformance-latest.md) is this run.witness-verifyjob's check:node plugins/ruflo-core/scripts/witness/verify.mjs --manifest verification/macos/manifest.md.json --jsonagainst the built tree returnsok: true,regressed: 0(89 pass, 28 drift, 0 missing) with upstream's manifest unchanged. This PR doesn't touchverification/.What was deliberately left out of this branch, and why
The fork branch (weave-logic-ai/ruflo
feat/grok-host) carried some things that only make sense there:data/clone-data.ledger.json/.proof.json/.rvf: the fork's own clone-tracker bot state, unrelated to this feature. Restored to upstream's version.docs/handoff.md: a personal session handoff note. Removed; doesn't exist upstream.handoffGrok skill (.grok/skills/handoff/and itstemplates/grok/skills/handoff/copy). It wrote session notes todocs/handoff.md, a personal workflow rather than default Ruflo behavior, andinit --grokwould have installed it into every project. Removed along with its references in the bench file list, theinit --grokoutput, and both Grok READMEs.docs/benchmarks/{grok-host,host}-live-latest.{json,md}: one-machine output from the live probe scripts (installed binary paths, local MCP config detection). Not referenced by any doc or script, so removed rather than scrubbed.docs/benchmarks/grok-host-conformance-latest.{json,md}: this one is referenced (ADR-402, README-GROK.md), so it's kept but regenerated fresh and with the CLI/repo path made repo-relative instead of this machine's absolute worktree path.verification/macos/: the fork had a regenerated witness manifest and a local witness run inhistory.jsonl, both keyed to the fork branch. Upstream's versions are kept; they verify clean against this tree.agentdb.rvf/agentdb.rvf.lock(root andv3/@claude-flow/cli/) and the related.gitignoreedits: the fork untracked these as a side cleanup. Out of scope here, so they're left exactly as upstream has them..grok/config.toml(the ruflo repo's own dev config, not the shipped template underv3/@claude-flow/cli/templates/grok/, which was already portable): switched the local-CLI MCP override to a path relative to the repo root and commented out theruvnet-brainblock instead of hardcoding this machine's home directory, matching the template's{{HOME}}-placeholder convention.None of this touches the feature's actual surface — the MCP tools, the CLI, the templates, the ADR, or the operator docs are unchanged by the cleanup.
Follow-ups
probe-host-live.mjs's installed-binary detection into the conformance bench (or clarifying the split in the scripts' own doc comments) rather than keeping them as three separate mjs entry points.verification/macos/manifest.md.jsoncurrently carries 28 "drift" entries (content changed since it was last signed, marker still present, not a regression) — worth a regen pass with real release versions once this lands, rather than as part of this PR.🤖 Generated with RuFlo