Repository navigation
Conversation
Keep native layout lowering, segment decoding, and legacy statistics conversion in vortex-layout while consolidating plans, preparation, execution, caches, and the default driver in vortex-scan. Preserve existing paths through compatibility exports and update engine consumers to the new ownership boundaries. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Construct source-independent plans for node, optimizer, pipeline, and scan execution tests in vortex-scan. Keep native lowering and file-backed comparisons in vortex-layout, and remove the cross-crate scan test harness and its public inspection APIs. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Contributor
|
This is quite a lot more than a refactor |
Move record-batch scan output to an ArrowScanExt trait in vortex-arrow so the scan engine has no direct Arrow dependencies. Expose generic worker-side batch mapping to preserve parallel conversion and decompression. Signed-off-by: Nicholas Gates <nick@nickgates.com>
The scan engine does not use the sequence encoding, paste, or termtree. Remove the stale dependencies left by the extraction so the engine only declares the lower-level primitives and utilities it actually uses. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Validate native legacy rewrites through rule sets, decode delivered bytes in scan fixtures, and restore checked raw statistics fixtures and driver concurrency. Keep internal construction and source inspection APIs crate-private without changing crate dependencies. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Restore the original legacy-schema constructor and tests beside the native schema helper. Retain generic evaluator tests in vortex-scan and remove the duplicated native schema validation from its fixtures. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Restore the native ZoneMap representation and schema helper implementations without wrappers or test-only fields. Keep generic scan evaluation independent and replace qualified body paths introduced by the reorganization with module imports. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Restore the native builder to scan::v2 and the existing public scan definitions to the crate root. Keep legacy field-name lookup in the native zone map, where it is still needed for older files. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Restore limits, metrics, and byte-range partitioning to their original modules. Share the existing row block size across the crate boundary and keep internal scan helpers private. Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Keep native layout lowering, segment decoding, and legacy statistics conversion in vortex-layout while consolidating plans, preparation, execution, caches, and the default driver in vortex-scan. Preserve existing paths through compatibility exports and update engine consumers to the new ownership boundaries.