Repository navigation
dream(intelligence): #3159 LearningBridge.consolidate() reward-blind fix (evaluated, ACCEPT) - #3160
Conversation
Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_014pTYiv93Ft6xAN8FAaWH5s
|
All 7 failing checks on this PR ( Root cause, confirmed not caused by this PR:
Related but insufficient existing fix: #3104 ("chore: sync pnpm lockfile for mcp alpha.10") is still open and touches Proposed patch (out of scope for this PR — a monorepo dependency-resolution fix, not an intelligence-surface correctness fix): either (a) add Not re-running — this isn't a flake (the version genuinely doesn't exist on the registry; re-running changes nothing). Keeping this PR watched until it's green. Generated by Claude Code |
ruvnet
left a comment
There was a problem hiding this comment.
Exact-head Dream review (0a6b0b46).
The focused 56→62 reward-resolution tests and the correction from get(id) to getByKey(namespace,key) are useful evidence. The repository-level acceptance gate is not satisfied: CodeQL passes, but CI/CD, CVE Audit, Cross-Agent Integration, Verification, and V3 CI all fail. Several mandatory jobs are skipped downstream of setup failures, so 464/465 local package evidence cannot establish workspace compatibility or security acceptance.
Frozen acceptance threshold: exact-head install succeeds; all six workflows complete; the stored-entry path proves entry.id !== entry.key; two trajectories with distinct confidences produce distinct completeTask rewards; lookup failure preserves the documented 1.0 fallback; and no ledger history is backfilled without independent provenance.
INCONCLUSIVE
…10 never published) Root package.json pinned an exact @claude-flow/mcp version that was never published to the npm registry (only alpha.9 exists there), and the root workspaces array doesn't include v3/@claude-flow/mcp, so npm resolves this dependency externally rather than as a local workspace link. This broke `npm ci` at the repo root for every PR, including ones that never touch this dependency (confirmed identical on main HEAD before this fix). Reverts the pin to 3.0.0-alpha.9, the last version actually published. Verified: `npm install --package-lock-only --ignore-scripts` and a full `npm ci --legacy-peer-deps` both succeed after this change. Requested by PR review on #3160 (exact-head install as an acceptance criterion) — this fix is scoped to the version pin only, no other changes. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_014pTYiv93Ft6xAN8FAaWH5s
|
Addressing the acceptance threshold from the review:
Re-running is unnecessary on my end since I've reproduced the install fix locally against the exact same command CI uses; the next CI run on Generated by Claude Code |
Records tonight's hive-mind_consensus Sybil-vote fix, verified GitHub fates for the trailing nights (#3139/#3152/#3160 still open/draft; #3169/#3184/#3221/#3243 merged), and 2026-09-10's completed-but- unmerged performance night (#3265/#3266). Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01DXHWKVWLjLv6wHkx6bBDF1
…fix (evaluated, ACCEPT) LearningBridge.consolidate() reported every completed trajectory's reward as a hardcoded 1.0, regardless of the underlying insight's actual quality. Now resolves the real confidence via backend.getByKey(namespace, key) — the entryId flowing through this bridge is the caller's key, not the backend's internal id, a distinction the round-1 fix got wrong and an adversarial critic caught before this PR. Baseline 56/56 -> candidate 62/62 tests; full package 464/465 (1 pre-existing environmental failure, unrelated). Also recovers 9 nights (2026-08-24..2026-09-01) of missing ledger rows — pipeline ran to completion each night but the ledger-append step silently failed, same pattern as the 08-14..08-19 gap recovered on 08-19. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_014pTYiv93Ft6xAN8FAaWH5s
57c0a1c to
1813c7f
Compare
|
This branch was rebased onto current Fixed in 7afb6df:
Confirmed pre-existing and NOT fixed here (unrelated to this PR, root-caused not just asserted):
Local Generated by Claude Code |
…ion gap (evaluated, ACCEPT-scoped) (#3291) * dream(security): #3290 fix hive-mind_consensus Sybil-vote authentication gap hive-mind_consensus's vote action recorded proposal.votes[voterId] for any caller-supplied voterId string, with no check against state.workers (the roster hive-mind_join populates). Since calculateRequiredVotes() derives the quorum threshold from state.workers.length, a single caller could cross any strategy's threshold (raft/bft/quorum) by voting under fabricated identities -- a Sybil attack on consensus, not merely a double-vote. The existing double-vote guard and Byzantine-flip detector only catch one identity contradicting its own prior vote, not many distinct forged identities each voting once. Fixed with a fail-closed roster-membership check before a vote is recorded. Stash-isolated: 2/3 new tests fail against real baseline (forged votes silently accepted; a BFT proposal reaches "approved" from 3 forged identities against 3 real workers), 3/3 pass against candidate. Full @claude-flow/cli suite: 2863/3036 passing, 46 pre-existing/unrelated failures (workspace-resolution gaps), zero "hive" failures. Independent adversarial critique: CONFIRMED-WITH-CAVEATS -- hive-mind_join/leave remain fully unauthenticated, so this closes forged-vote Sybil but not forged-join Sybil (disclosed follow-up). Closes #3290 (full research: 5-role parallel fan-out, ledger check, intelligence/swarm scan findings, competitor comparison, adversarial critique). Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01DXHWKVWLjLv6wHkx6bBDF1 * dream(security): update ledger for #3290/#3291 (2026-09-11) Records tonight's hive-mind_consensus Sybil-vote fix, verified GitHub fates for the trailing nights (#3139/#3152/#3160 still open/draft; unmerged performance night (#3265/#3266). Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01DXHWKVWLjLv6wHkx6bBDF1 * dream(security): capability-bind hive-mind join/leave/vote, fix CI cold-install Addresses ruvnet's review REJECT on PR #3291 at b275fdd: 1. hive-mind_join/hive-mind_leave were themselves unauthenticated, so the roster-membership check alone (the original fix) only closed forged-vote Sybil, not forged-join Sybil -- an attacker could just legitimately join under fake names. hive-mind_init now mints a 32-byte hiveToken; join, leave, and vote all require it (constant-time comparison), fail-closed: a denied call makes zero state change (no roster write, no vote write), verified by reloading state.json fresh off disk after each denial (standing in for a restart/reopen). The CLI's own hive-mind subcommands are unaffected -- a new getHiveTokenForCli() reads the token directly off the same-machine state file (never exposed via hive-mind_status, which is MCP-reachable) and the CLI wires it through automatically. 2. The required "Test Suite" CI workflow failed twice on the exact head: @claude-flow/security (a root npm-workspace member) has no prepare/postinstall script, only prepublishOnly (which npm ci never runs), so a fresh checkout's dist/ never gets built, causing ERR_MODULE_NOT_FOUND in doctor.js -> policy-runtime.ts. Root-caused via a real root `npm ci` (confirmed dist/index.js is missing without a prepare script, present with one) and cross-checked against PR #3266's identical first-run failure. Added "prepare": "npm run build" -- standard npm lifecycle, fixes every future checkout, not just this PR. Full @claude-flow/cli suite re-run after both fixes: 28 failed / 2884 passed (was 46/2863) -- the ~18-test improvement is exactly the @claude-flow/security-dependent tests the prepare script now fixes; the remainder are pre-existing, unrelated @claude-flow/neural/@claude-flow/mcp workspace-resolution gaps. Zero "hive" failures. 6/6 new/updated tests pass against the candidate, 6/6 fail appropriately against a stash-isolated baseline. tsc --noEmit clean on every touched file. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01DXHWKVWLjLv6wHkx6bBDF1 --------- Co-authored-by: Claude <noreply@anthropic.com>
1. Hypothesis
2. Candidate
v3/@claude-flow/memory/src/learning-bridge.ts:consolidate()now resolves each trajectory's reward via a new privateresolveConsolidationReward(entryId), which callsthis.backend.getByKey(this.config.insightNamespace, entryId)(new config field, default'learnings') and readsentry.metadata.confidence, falling back to1.0when the entry is missing, non-numeric, or the lookup throws.Why
getByKey, notget:entryIdhere is the caller-assignedkeystring an insight was stored under (AutoMemoryBridge.storeInsightInAgentDBreturnskey, e.g.insight:debugging:<ts>:<n>), not the backend's internalentry.id(an independently-generated UUID minted bycreateDefaultEntry).backend.get(id)is indexed byid;backend.getByKey(namespace, key)is indexed by thekeyIndexpopulated atstore()time. A round-1 version of this fix usedget(entryId)and would have silently missed in production every time — see §8.3. Evaluation Receipt
Real evaluator:
vitest runonv3/@claude-flow/memory/src/learning-bridge.test.ts, deterministic, $0 LLM cost.git stash-isolated): 56/56 passing.store()→onInsightRecorded(insight, entry.key)→consolidate()path whereentry.id !== entry.key, differentiation across two trajectories, 3 fallback-to-1.0cases, and a custom-insightNamespacecase).auto-memory-bridge.test.ts, a read-only-file-permission test) reproduces identically against the stashed baseline — confirmed environmental (tests run as root in this sandbox, which bypasses thechmodthe test relies on), not caused by this diff.4. Baseline Comparison
Baseline: every consolidated trajectory reports
reward=1.0to the neural system regardless of the insight's actual confidence — a reward-blind learning signal, withpatternsLearnedcounted unconditionally. Candidate: reward now reflects the insight's current confidence (which itself reflects access boosts viaonInsightAccessedand time decay viadecayConfidences), verified directly via mockcompleteTaskcall-argument assertions, not inferred.5. Darwin Lineage
Not run — deliberately. Binary correctness fix (pass the real signal, or don't), no continuous parameter for Darwin to tune. Same reasoning as last night's own precedent (PR #3152) for this class of finding.
6. Flywheel Evidence
No
.claude-flow/flywheel/state in this repo. Evidence retained as: this diff, issue #3159, anddocs/dream-cycle/dream-gist-2026-09-02.md.7. Reward Hack Check
Manual checklist (no generic diff/benchmark-scanning CLI reachable this session —
@metaharness/weight-eftdoctor-reported as available but not fit for this class of check per last night's own note too). No test weakened — diffed the full test file; every change adds coverage, nothing removed or loosened (confirmed by the clean 56→62 comparison). No gold/expected data touched. No seed manipulation (no randomness involved). No cherry-picked runs. Minor, disclosed cost: onebackend.getByKey()call per active trajectory inconsolidate(), gated byconsolidationThreshold(default 10) — O(threshold) reads per cycle, not unbounded.8. Security Review
Not security-sensitive: internal learning-signal correctness fix, no new attack surface, no credential/network/filesystem exposure change.
Independent adversarial-critic subagent (separate context), two rounds:
backend.get(entryId)would almost always miss —get()is indexed byentry.id, butentryIdis actually thekeystring. The fix would pass unit tests (the mockget()had zero id/key validation) while doing nothing in production — silently relocating the exact bug it claimed to fix.backend.getByKey(namespace, key); added configurableinsightNamespace(default'learnings', matching the only real caller,AutoMemoryBridge); rewrote the mock backend'sgetByKeyto perform a genuine namespace+key lookup against stored entries (previously a bare stub); rewrote all reward-resolution tests to go throughbackend.store()withentry.id !== entry.key, proving the fix resolves through the correct index rather than an unvalidated mock.agentdb-backend.ts,sqlite-backend.ts,hybrid-backend.ts) and confirmedgetByKeycorrectly resolves the realkeyvaluesAutoMemoryBridgeactually produces. One non-blocking nit:controller-registry.tsinstantiates a second, separateLearningBridgethat doesn't threadinsightNamespacethrough — but that instance is never fed any insights (dead/unwired in that registry path), a pre-existing, out-of-scope gap this diff doesn't introduce or need to fix.9. Regression Analysis
464/465 across the
@claude-flow/memorypackage; the 1 failure is pre-existing and environmental (confirmed via stashed-baseline re-run), not a regression from this diff.10. ADR
None created — matches the explicit exclusion criteria (a correctness/wiring fix, not an architectural decision).
11. Research Gist
docs/dream-cycle/dream-gist-2026-09-02.md— 5 parallel research roles (deep researcher, 2 scan roles on capabilities/memory, competitor analyst, architecture reviewer) on tonight's DEEP=intelligence surface. Headline external finding: ReasoningBank/FORGE (2025-26) converge on failure-derived memory as the SOTA direction for self-learning agents — Ruflo'sLearningBridgewas reward-blind before this fix, a prerequisite gap. Separately, a strong competitor-comparison finding: every framework checked (LangGraph, AutoGen, CrewAI, OpenAI) has either never shipped human-gated evolutionary self-improvement or is actively retreating from it (OpenAI's Agent Builder/Evals wind-down, announced 2026-06-03; CrewAI's issue #3015 declined with no rebuttal) — both citing/evidencing safety concerns (DGM reward-hacking, documented "capability erosion under self-evolution") that Ruflo's Darwin/Flywheel governance shape is specifically built to address. No gist-creation tool was reachable in this execution environment (GitHub MCP tools cover issues/PRs/repos, not gists) — committing the report todocs/dream-cycle/instead, matching the repo's own established convention for these reports.Also surfaced tonight, not part of this candidate: PR #3110 (2026-08-27, EWC-gate wiring, already evaluated ACCEPT-scoped) was never merged to
main— that bug is still live in production. Recommend prioritizing its merge. And: 9 nights (2026-08-24..2026-09-01) of Dream Cycle ledger rows were missing despite the pipeline completing each night (branch+issue+PR all exist, verified not inferred) — same ledger-append failure mode recovered once before on 08-19. Backfilled in this PR'sdocs/dream-cycle/LEDGER.mdchange.12. Issue
Closes evidence trail documented in #3159.
13. Witness
4d0134e59b4fa5e8552cb7b98c6b9846f08b0c8229c9189c833c3d56c378f9a1329b3baf6f16543493ad28e4e62ec76b14c0fdb9bead6402cb018f0e57ec6695c82e007365b5f2ae91d530a74c82a2eb8a1d362b14. Merge Policy
Human review required. Do not self-merge. Do not autonomously promote Flywheel state.
🤖 Generated with RuFlo
https://claude.ai/code/session_014pTYiv93Ft6xAN8FAaWH5s
Generated by Claude Code