Repository navigation
fix(memory): build the RaBitQ index on demand and use it on the vector path - #4318
Merged
Merged
Conversation
…r path (ruvnet#4203) shouldRebuildRabitq() had no callers and the index was only ever populated by the explicit embeddings_rabitq_build tool, so searchRabitq() always returned null and the sql.js pre-filter was dead. ensureRabitqIndex() drives the existing thresholds from the search entry points (60s throttle, 10-vector floor, 20% drift, disabled after a missing-WASM failure, bound to the source dbPath and dimensions). The bridge cosine path now scans RaBitQ candidates and reranks them with the same exact cosine, falling back to the unchanged exhaustive scan on any error.
…uvnet#4203) Six tests against a real sqlite store: build on demand, bridge uses the index and reranks with exact cosine, no rebuild per search, below-floor negative control, exhaustive-vs-indexed parity when the corpus fits the candidate window, and recall plus exact-rerank on a 60-row corpus larger than the window (>=80% top-5 recall).
Contributor
Author
|
CI is waiting for approval as a first-time contribution. Could a maintainer approve the workflow run? thanks. |
Owner
|
Reviewed the diff for correctness against the PR description's own claims (not just taking them at face value):
Minor gap: no test exercises the WASM-unavailable permanent-disable path or genuinely concurrent No blocking issues. CI is running now (just approved the pending first-contribution workflow runs) — will merge once green. |
3 tasks done
ruvnet
pushed a commit
that referenced
this pull request
Oct 20, 2026
…-build (#4203) (#4319) Review follow-up to #4318, which merged without coverage for two branches that ensureRabitqIndex() relies on. A missing @ruvector/rabitq-wasm now has a test: the first attempt fails and sets autoBuildDisabled, and a second call ten minutes later (past the 60s throttle) performs no further import, so the flag - not the throttle - is what stops the retries. Four concurrent ensureRabitqIndex() calls on a cold index: exactly one starts the build, the index ends up with the right vector count, and it serves searches afterwards.
Merged
meefs
pushed a commit
to meefs/claude-code-flow
that referenced
this pull request
Oct 20, 2026
Minor release (new backward-compatible flags: Grok Build CLI host surface ruvnet#3512, memory list/search tag filter ruvnet#3526) bundled with a round of correctness fixes merged since 3.58.1: memory delete-sync (ruvnet#4138), real AgentDB HNSWIndex replacing the dead getController('hnsw') path with safe exact-search fallback (ruvnet#4249/ruvnet#4322), harness live-process reporting (ruvnet#3521), RaBitQ on-demand index build (ruvnet#4318/ruvnet#4319), metaharness ADR-152 cost-dimension and verdict-band fix (ruvnet#3520), swarm idle-agent heartbeat false-positive (ruvnet#4320), ephemeral-agent termination idempotency (ruvnet#4323), status/task_list field-mapping and total-count fixes (ruvnet#4321/ruvnet#4324). Also fixes the actual cause of broken images/links on npmjs.com/package/ruflo: `ruflo/package.json`'s `repository.directory: "ruflo"` field added an extra path layer on top of the README content (the root README.md, copied in verbatim by `prepublishOnly` at publish time, which is already written with repo-root-relative paths) — every relative image/link resolved to e.g. `raw/main/ruflo/ruflo/assets/...` instead of `raw/main/ruflo/assets/...`. Removed the `directory` field to match claude-flow's own (directory-less) repository entry. A prior PR (ruvnet#4325) fixed `ruflo/README.md`'s own relative paths for anyone browsing that file directly on GitHub, but that file is overwritten by the root README at publish time and was never the actual npm-rendered content. Published to npm: @claude-flow/cli@3.59.0, claude-flow@3.59.0, ruflo@3.59.0, and standalone @claude-flow/memory@3.0.8 (the ruvnet#4249 fix — not part of the 3-package train, published separately per its own leaf's npm presence). All three umbrella packages' latest/alpha/v3alpha dist-tags verified pointing at 3.59.0. Co-Authored-By: RuFlo <ruv@ruv.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ensureRabitqIndex(): builds/rebuilds on demand using the existingshouldRebuildRabitq()thresholds (10-vector floor, 20% drift), with a 60sthrottle, a process-level disable when the WASM backend is unavailable, and
binding to the source
dbPathand dimensions.vector path, and
embeddings_rabitq_search, which no longer needs a manualbuild first.
bridgeSearchBruteForceCosine()reranks RaBitQ's Hamming candidates with thesame exact cosine, reading only the candidate rows by primary key; any failure
falls back to the unchanged exhaustive scan.
Closes #4203.
Behavior
The exhaustive path pre-ranks
LIMIT 10000, so on corpora larger than 10k theindexed path no longer drops rows outside that window. This is an intended
improvement, not a ranking change. The bridge hybrid path (
bridgeSearchEntries,1000-row BM25+cosine) is untouched, so
describeBackend()continues to reportbrute-force cosinethere.Test plan
__tests__/issue-4203-rabitq-auto-build.test.ts— 6 tests against a realsqlite store: build on demand, exact rerank, no rebuild per search,
below-floor negative control, indexed-vs-exhaustive parity within the
candidate window, recall + exact rerank on a 60-row corpus above the
window (>=80% top-5 recall).
issue-2922-hnsw-status-honesty — 9 files / 32 tests pass.
tsc --noEmiton@claude-flow/cli: no new errors (14 pre-existing, nonein touched files).
Risks
dropped. A
rowCount > 10000fallback can be added if strict parity isrequired.
the window are not candidates until a rebuild fires.
rabitq.meta.jsonis written but never read, so a new process rebuilds on itsfirst search. Restoring from meta needs version/dims/sourcePath/count
validation and is out of scope.
memory-bridge.tsalready exceeds 500 lines; the change is additive ratherthan a refactor.