[Dream Cycle 2026-10-08] memory: dedup() eager-removal recall hypothesis INCONCLUSIVE, fix REJECTED (+ nightlyLearner.run() wiring fix) + plugins,automation scan #3908

See the correction comment below — §5 and §14 here are the corrected, final versions; the original first-pass evaluation (which wrongly reported a clean REJECT for the hypothesis itself) is preserved in the comment for the audit trail.

1. Tonight's Rotation

DATE=2026-10-08
DEEP=memory
SCAN=plugins,automation
SLOT=3
SESSION_COMMIT=6051f6702b610e1101e727ccffd5f379f75ea6fc

2. Ledger Check

docs/dream-cycle/LEDGER.md has no row past 2026-09-30. Verified via git ls-remote --heads origin "dream/*" and mcp__github__search_pull_requests (not inferred from the sparse table): the pipeline ran every night from 10-01 through 10-07 except 10-04 (a genuine no-run gap, matching the swarm slot). PRs found: #3599 (10-01, MERGED 10-07), #3622 (10-02, open, intelligence ACCEPT), #3650 (10-03, open, memory ACCEPT — HNSW quantizer serialize/deserialize fix), #3754 (10-05, closed not merged — duplicate of #3517 which shipped in v3.55.0), #3835 (10-06, open, security ACCEPT-scoped), #3879 (10-07, open, intelligence ACCEPT). None of the last 7 nights' candidate PRs are stale yet (<14 days old). This is the same ledger-append gap already elevated as high-priority in August — see Scan Findings: automation below.

Prior memory-surface nights (08-18, 08-28, 09-03, 09-08, 09-18, 09-28, 10-03) each addressed a distinct finding — no 3x-repeated direction to reject under STEP 1.1.

3. Deep Dive Findings

See Hypothesis/Evaluation below. Summary: MemoryConsolidator.dedup()'s near-duplicate pass calls HNSWIndex.removePoint() eagerly, mid-loop, inside a search-dependent multi-round convergence algorithm — exactly the shape of the literature's "unreachable points phenomenon" (arXiv 2407.07871). Investigated directly rather than assumed real, then re-investigated again after an adversarial critic caught a flaw in the first measurement (see correction comment below). Found: fix REJECT (solid) + hypothesis INCONCLUSIVE (corrected), plus one unrelated small ACCEPT fix found during the same code read (nightlyLearner.run() wiring).

4. Hypothesis

Given MemoryConsolidator.dedup()'s pass-2 loop, when HNSWIndex.removePoint() is called eagerly per dropped entry inside that loop, then HNSW search() recall@10 for unrelated background vectors should measurably degrade relative to a freshly rebuilt index, subject to: deterministic $0 evaluation, no modification of gold/ground-truth data, reproducible across repeated runs. Frozen before evaluation; not modified after.

5. Evaluation Receipt (CORRECTED — see comment below for the full correction narrative)

evaluated: rejected (the candidate FIX, solid and reproducible — unaffected by the correction): deferring index.removePoint() to a single batch after dedup()'s rounds converge was implemented in full in consolidator.ts, then broke a real pre-existing test (consolidator.test.ts's cluster-convergence test: merged dropped from 14 to 7 for n=15 identical vectors). Root cause: deferred removal means index.search() returns the same top-K set every round (nothing shrinks), so later rounds never resurface the rest of the cluster. Eager removal is load-bearing for the algorithm's completeness. Diff reverted, not shipped. Independent critic re-ran this regression test 17x with zero failures.

evaluated: inconclusive (the underlying HYPOTHESIS, corrected from an initial, wrong "rejected"): direct recall measurement (v3/@claude-flow/memory/src/dedup-eager-removal-recall-probe.test.ts, committed) now runs 24 independent trials per invocation (fresh index + fresh random HNSW graph topology each time — HNSWIndex.getRandomLevel() uses unseeded Math.random()). Five re-runs tonight (120 trials): mean recall delta consistently positive (+0.10 to +0.14), but 1/24 trials degraded (>0.05 drop) in 3/5 runs, worst single-trial drop -0.12. An independent adversarial critic separately ran ~46 single-trial probes and found one full recall collapse to 0. Net: a single experiment cannot reliably distinguish "degradation happens" from "it doesn't" here — INCONCLUSIVE by definition.

evaluated: accepted (secondary fix, bridgeSessionEnd()'s nightlyLearner wiring in v3/@claude-flow/cli/src/memory/memory-bridge.ts): the session-end consolidation trigger checked typeof nightlyLearner.consolidate === 'function', but neither backend (agentdb's real NightlyLearner class, or the ADR-125 MemoryConsolidator wrapper) has ever had that method — both expose run()/runAll() instead. Unconditionally false since introduction; consolidation never actually ran at session end, silently, on either backend. Fixed to check/call .run(). New test (3 cases) proves baseline-fails/candidate-passes. Independent critic confirmed: only caller of this controller, no other code depends on the old guard, 3/3 tests pass, no reward-hacking signal.

Full suite: 565/566 @claude-flow/memory (1 pre-existing unrelated environmental failure, same as prior PRs e.g. #3650); relevant @claude-flow/cli tests green; tsc shows the same pre-existing unbuilt-sibling-package errors as before (none in touched files).

6. Darwin Results

Skipped for both. The rejected candidate was a correctness-class algorithm change (reverted) with no continuous tunable parameter. The accepted fix is a method-name correction with no fitness gradient. Same skip class as #3110/#3160/.../#3754.

7. Flywheel Evidence

No formal .harness/bench.json corpus applies to either change — evaluated via real Vitest (deterministic, $0, zero LLM calls), matching the established pattern for this class of dream-cycle candidate (e.g. #3650, #3754). Classification: the recall-probe trial statistics are MEASUREMENT; "eager removal is load-bearing for convergence" is INFERENCE from a reproduced code-level regression; the REJECT (fix) and INCONCLUSIVE (hypothesis) verdicts are DECISIONs; the nightlyLearner fix's correctness is MEASUREMENT (baseline-fails/candidate-passes).

npx ruvector harness flywheel gate/verify exist (reachable this session via npx --yes ruvector@0.3.3) but expect a signed-evidence-bundle schema not producible from a plain Vitest run — promotion-gate checklist done manually, same disclosed gap as prior nights.

8. Reward Hack Check

npx ruvector/npx metaharness reward-hack-specific subcommands were not reachable as standalone commands this session (this fresh checkout's @claude-flow/cli-core is unbuilt). Manual checklist instead — and this time including the correction itself as a reward-hack-adjacent check:

• No test/benchmark weakened to force a pass — the opposite happened: the first version's assertion was replaced with a stricter, more honest statistical measurement after an adversarial critic found it was unreliable, and the verdict was walked back from REJECT to INCONCLUSIVE rather than defended.
• No gold-data tampering — bruteForceTopK ground truth computed independently of the index under test.
• No cherry-picking — the correction explicitly reports the unfavorable numbers (1/24 trials degraded in most runs) rather than hiding them behind a passing aggregate assertion.
• No cost hiding — $0 deterministic evaluation throughout.
• No error suppression added — the pre-existing catch { non-fatal } in memory-bridge.ts predates this change and is preserved as-is.

9. Security Review

Neither change touches a security-sensitive surface. The reverted candidate never shipped. The nightlyLearner.run() fix changes which internal method gets called at session end on an already-trusted, already-reachable internal controller — no new input validation surface, no new external trust boundary. Independent critic noted one UX-relevant (not security-relevant) side effect: the legacy AgentDB NightlyLearner.run() path now actually fires at session end (previously silent dead code) and prints several console banner lines — non-fatal, wrapped in try/catch, flagged for whoever owns session-end UX.

10. Scan Findings: plugins

Prior-night lead (2026-10-07 gist) confirmed by direct code read: v3/@claude-flow/cli/src/plugins/manager.ts only shells to npm and reads/writes a JSON manifest — zero import()/require() call sites anywhere in v3/@claude-flow/cli/src/plugins/. trust-policy.ts's real withhold/register decision on a plugin's declared hooks/commands is therefore bookkeeping with no runtime effect, for trusted and untrusted plugins alike, since nothing ever loads plugin code to execute them. Testable tonight as a $0 static-grep CI check; building the real sandboxed loader is separate, larger design work — not attempted tonight.

11. Scan Findings: automation

The dream-cycle's own ledger-append step (STEP 25) has been silently missing for 7 consecutive nights now (2026-10-01 through 2026-10-07; confirmed real branches/PRs exist for all but a genuine 2026-10-04 no-run gap via git ls-remote + PR search, not inferred). No .github/workflows/*.yml anywhere references "dream", LEDGER.md, or the documented external trigger id (trig_01HpEqAcEP7wzrxy3TzakrQ2) — the schedule lives entirely outside this repo's version control, and nothing asserts a ledger row was appended after a run. This is the same gap the ledger itself already elevated to "first-class, high-priority candidate" after its 3rd occurrence in August 2026 — now recurring for a 4th+ time, still with no CI-level guard attempted.

12. Competitors Reviewed

Qdrant, Weaviate, Milvus, LanceDB, Vespa, pgvector/pgvectorscale (vector-DB deletion/compaction mechanics); Mem0, LangGraph Store, CrewAI, AutoGen (agent-memory consolidation behavior). Full table and sourcing in the gist below.

13. Gist

No gh gist create available this session (GitHub CLI token invalid — gh auth status reports "Failed to log in... invalid token"; GitHub MCP tools used instead, which have no gist-equivalent). Full report committed to this branch at docs/dream-cycle/dream-gist-2026-10-08.md, including the correction.

14. Witness (CORRECTED — recomputed after the gist was corrected)

Session commit:  6051f6702b610e1101e727ccffd5f379f75ea6fc
Candidate commit: (see PR — filled in docs/dream-cycle/dream-gist-2026-10-08.md on this branch)
Gist SHA-256 (pre-witness, mutable fields stripped to PENDING): 0e578f58305ad7c37301c678acc89095a8c34e323d6316533dd3c754c8ec6012
Witness stamp: c1edadf7b1f435fc188b4b179f958844b93f861f69e79eb83d480e9b60ea6fe7

Verifier: fetch docs/dream-cycle/dream-gist-2026-10-08.md from the branch, strip its Witness block's filled-in values back to PENDING, SHA-256 the file, concatenate with the session commit above, SHA-256 again — must equal the witness stamp.

15. Recommendation

Human review recommended for the small nightlyLearner.run() wiring fix (low risk, well-tested, restores a long-silently-broken feature, one UX note about new session-end console output). No action needed on the primary hypothesis beyond reading the committed, corrected probe test before anyone re-attempts the "defer removal" direction or tries to re-settle the INCONCLUSIVE verdict without first seeding HNSWIndex.getRandomLevel(). The two scan findings (plugin permission enforcement, ledger-append reliability) are both good candidates for dedicated future capabilities/automation nights rather than action tonight.

Merge policy: human review required. Not self-merged. No Flywheel state autonomously promoted.