Skip to content

Decode the whole file schema when a file opens - #11578

Open
connortsui20 wants to merge 1 commit into
developfrom
ct/lazy-struct-field-errors
Open

connortsui20 wants to merge 1 commit into
developfrom
ct/lazy-struct-field-errors

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Summary

A file whose schema has a struct field dtype that fails to decode, such as one with an unknown type tag, panics when it opens instead of returning an error. Struct field dtypes are decoded lazily, and StructFields::fields() can't return an error, so it calls vortex_expect on the decode. While the file opens, the footer reads the file statistics against the schema, which reaches that accessor.

Changes

The file reader now decodes the whole schema once when the file opens, before the statistics and the layout, and returns the first decode error. The alternative was to make the field accessors fallible, which changes an accessor with many callers and leaves each caller to handle an error that only a corrupt file can produce. Making only the statistics path fallible isn't enough either, since the footer drops statistics errors on purpose, so a broken schema would get through a successful open and fail later in a scan. The other users of DType::from_flatbuffer (IPC, Python, and CUDA) stay lazy, and a dtype or footer that the caller supplies through with_dtype or with_footer still skips the embedded schema.

Lazy field dtypes exist for wide schemas (#1826), so the open time of files with 10,000 i64 columns was measured before and after. A nested schema opened 0.31 ms slower without file statistics and 0.29 ms faster with them, and the flat schema showed no clear change. The full numbers are below. The eager decoder also reads straight from the footer bytes, so the copy of the dtype buffer is gone.

The new tests open a file whose field tag was changed after it was written, once with file statistics and once without, and expect the decode error in both cases.

Wide-file open timings

Apple M4 Max, 128 GB RAM, macOS 27.0.1, rustc 1.100.0, bench profile. Each file has one row and 10,000 non-nullable i64 leaves. The flat schema has 10,000 root fields, and the nested schema has 100 root structs with 100 fields each. The timing uses VortexOpenOptions::open_buffer with fresh options on every iteration and no cached footer. Each process warmed up with 40 opens per shape, then ran nine batches of 80 opens. Each cell gives the medians of two processes, run in before, after, after, before order. File generation, disk IO, and scans are outside the timed region.

Schema File statistics Bytes Before (ms) After (ms)
Flat Omitted 6,157,656 2.394, 2.371 2.369, 2.341
Flat Legacy 6,677,744 7.050, 7.066 7.885, 6.614
Nested Omitted 6,130,456 1.929, 1.879 2.216, 2.220
Nested Legacy 6,133,368 2.588, 2.525 2.255, 2.283

The flat file with statistics varied between the two runs after the change, so that row is inconclusive.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20 connortsui20 mentioned this pull request Nov 17, 2026
17 of 57 tasks
@connortsui20
connortsui20 marked this pull request as ready for review November 18, 2026 02:25
@connortsui20

Copy link
Copy Markdown
Member Author

@claude review this, and also summarize the actual logical changes here because theres a bunch of refactoring in here as well

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @connortsui20's task in 47s —— View job


Review of #11578

I read the whole diff and checked every caller of DType::from_flatbuffer. I ran no builds or tests.

What actually changes in behaviour

Most of the 228 changed lines are a refactor. Only two things change behaviour:

  1. Opening a file now decodes the whole schema up front (vortex-file/src/footer/deserializer.rs, parse_dtype). It used to call DType::from_flatbuffer, which leaves struct, union and map field dtypes undecoded until something reads them. It now calls the new DType::from_flatbuffer_eager, which decodes every nested dtype before returning. A bad field tag now returns an error when the file opens. Before, it panicked later in StructFields::fields() through vortex_expect.
  2. The schema bytes are no longer copied. The eager decoder reads straight from the footer bytes (&[u8]), so the FlatBuffer::copy_from of the dtype segment is gone.

Everything else gives the same result as before:

  • Refactor: StructFields::from_fb, UnionVariants::from_fb, MapDType::from_fb and the body of TryFrom<ViewedDType> for DType are merged into one private DTypeDecoder { lazy_buffer: Option<&FlatBuffer>, session }.
    • When lazy_buffer is Some, struct, union and map fields become ViewedDType views, exactly as before.
    • When it is None, those fields are decoded straight away into owned dtypes.
  • List elements and extension storage dtypes were already decoded straight away. They used to go through DType::try_from(ViewedDType::from_fb_loc(..)); they now use self.decode(..) with the same lazy_buffer. Any struct nested inside them is still lazy on the lazy path, so the result is the same.
  • DType::from_flatbuffer (used by IPC, Python serde and CUDA) and TryFrom<ViewedDType> now just build a lazy decoder. Their behaviour is unchanged.
  • Smaller changes:
    • Self:: became DType:: because the code moved out of impl DType.
    • The ViewedDType::buffer() accessor was removed.
    • A SAFETY comment was added on the follow call.
    • roundtrip_dtype now checks both the eager and the lazy decoder.
  • New test: open_rejects_unknown_struct_field_dtype (with and without file statistics). It writes a file, sets the first field's union tag to 0xFF and expects Unknown DType variant when the file opens.

Review

The change looks correct, and I found no bugs. Details:

  • The lazy path keeps its semantics. I compared each branch of the old code with the new one. Struct, union and map fields still become ViewedDType when lazy_buffer is Some. List, fixed-size list and extension storage dtypes were decoded straight away before and still are. The extension registry and allows_unknown lookups now go through self.session, which is the same session.
  • The recursion depth is bounded. The eager path recurses through decode → struct_fields → field → decode. The flatbuffer verifier inside root::<fbd::DType> already rejects anything deeper than its default max_depth (64), so a malicious schema can't overflow the stack.
  • The test sets the tag correctly.
    • field._tab.loc() is relative to dtype_bytes, and the vtable entry is relative to the table, so segment.offset + loc + voffset points at the absolute byte of type_type.
    • The generated union verifier accepts unknown tags, so verification passes and the error really comes from decode's _ => branch. That is the code path the PR claims to cover.
  • Remaining scope (not a blocker): IPC (vortex-ipc/src/stream.rs:55 and iterator.rs:36), vortex-python/src/serde/mod.rs:115 and vortex-python-cuda still decode lazily, so a corrupt dtype there can still panic in fields(). The PR description says this, but if Panic in StructFields::fields() on malformed file: unwraps a fallible DType decode #8848 covers IPC as well, keep the issue open or open a follow-up instead of letting Fixes #8848 close it. Python's vortex.open goes through vortex-file, so file reads from Python are fixed.

Nits, all optional:

  • DTypeDecoder::fields could be written as dtypes.map(|dt| self.field(dt)).collect::<VortexResult<Vec<_>>>(). The ExactSizeIterator bound and manual loop exist only to pre-size the Vec, and the collect version gets that from size_hint anyway.
  • Doc comment on from_flatbuffer_eager: it says file readers use it, but it is now public API for any crate. One sentence on the trade-off would help other callers choose: the eager decoder allocates every nested dtype up front and returns errors instead of panicking, while the lazy decoder is cheaper for wide schemas.
  • Test location: open_rejects_unknown_struct_field_dtype opens a complete file (write, corrupt, open_options().open), so it may fit better under vortex-file/tests/ than in the footer deserializer's unit tests. Keeping it here is fine, because it needs checked_segment_slice, which is private to this module.
  • Open-time cost: the nested schema without statistics got 0.31 ms (about 15%) slower to open. That seems an acceptable price for turning a panic into an error, and the PR description already reports it.

Overall it looks good to merge.

@codspeed

codspeed Bot commented Nov 18, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 0.82%

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 1973 untouched benchmarks
⏩ 53 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime threshold_avx512[80] 2.1 µs 2.4 µs -11.24%
⚡ Walltime decode_blocked_bitpacked_chunked_neon[u32, 524288] 29.9 µs 27 µs +10.83%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/lazy-struct-field-errors (702a5b4) with develop (3ebf68d)

Open in CodSpeed

Footnotes

  1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@connortsui20 connortsui20 added the changelog/fix A bug fix label Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panic in StructFields::fields() on malformed file: unwraps a fallible DType decode

1 participant