Repository navigation
Conversation
… columns
A point query on a rowid the table does not hold returns no rows on a flat
column: vec0_get_vector_data() looks the rowid up in _rowids first, and
vec0Filter_point() reads its SQLITE_EMPTY as the end of the results. A
rescore column read the rowid's row of _rescore_vectors, and a DiskANN
column its row of _vectorsNN, before that lookup, so the same query failed:
create virtual table t using vec0(
embedding float[8] indexed by rescore(quantizer=bit));
insert into t(rowid, embedding) values (1, ?);
select rowid from t where rowid = 999;
-- Could not fetch vector data for 999 from rescore vectors
A rowid deleted from the table fails the same way, and a DiskANN column
reports "Could not fetch vector data for 999 from DiskANN vectors table".
An IVF column already returns no rows.
Every index type registers its rows in _rowids, an all-DiskANN table
included, so vec0_get_vector_data() now looks the rowid up there before
reading any index's vectors. tests/test-rescore.py, tests/test-diskann.py
and tests/test-ivf.py check a never-inserted and a deleted rowid on each;
the rescore and DiskANN tests fail on main.
This was referenced Oct 23, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A point query on a rowid the table does not hold returns no rows on a flat column, and fails on a rescore or DiskANN column:
The same happens with
quantizer=int8, for a rowid that was deleted, and on a DiskANN column (Could not fetch vector data for 999 from DiskANN vectors table). An IVF column already returns no rows.vec0_get_vector_data()looks the rowid up in_rowidswithvec0_get_chunk_position(), whoseSQLITE_EMPTYvec0Filter_point()reads as the end of the results, but only on the flat path; the DiskANN and rescore branches come first and read their own tables by rowid, where a missing row is an error. Every index type registers its rows in_rowids(the insert's first step, an all-DiskANN table included, which skips only the chunk placement), so the lookup now comes before every index's read. Its other callers,vec0Column_fullscan()andvec0Column_knn(), read rowids taken from_rowidsor from a KNN result.Test.
tests/test-rescore.py(both quantizers),tests/test-diskann.pyandtests/test-ivf.pyeach check a never-inserted and a deleted rowid. On main the rescore and DiskANN tests fail with the errors above; on this branch they pass, and the IVF one passes on both. The Python tests pass: 242 passed and 67 skipped asmake loadablebuilds, 305 passed and 4 skipped with-DSQLITE_VEC_EXPERIMENTAL_IVF_ENABLE=1. It merges cleanly with #327 to #331 and #336 to #341.Origin. The rescore port to photostructure/sqlite-vec, photostructure#8 (cd4bb09), lists this among its differences from upstream, with the lookup first in
vec0_get_vector_data()as here, and the DiskANN port, photostructure#11 (cd53413), tests it for DiskANN columns; the change and its tests are written here for asg017's main. No open or closed PR or issue here makes the same change.