Repository navigation
[opt](inverted index) Share index runs among the scanners of a split segment - #70374
Draft
airborne12 wants to merge 1 commit into
Draft
airborne12 wants to merge 1 commit into
airborne12 wants to merge 1 commit into
Conversation
…segment ### Background ParallelScannerBuilder splits a large segment into row ranges, one per scanner, and each scanner evaluates the segment's index predicates over the whole segment before it keeps its own range. TextIndexReader can make concurrent identical full-segment queries share one run (apache#68758); only SNII turns that on. ### What problem does this PR solve? Issue Number: None Related PR: apache#68758 (added the shared run for SNII) PR Type: Performance Problem Summary: With the default 2M rows per scanner, an index predicate on a 6.3M-row segment runs three times: MATCH, keyword and BKD predicates on CLucene indexes and every search() call. CLucene text readers now share runs as SNII does, the BKD reader shares them through the same helper, moved into InvertedIndexReader, and search(), which reads indexes without the readers' query path, shares its whole-segment result the same way. ### Details - **Why it was slow:** each scanner of a split segment repeats the segment's index work. Over 100M rows in 6.3M-row segments, split scanners took 1.6 to 3.1 times the CPU of whole-segment scanners for MATCH on CLucene and 1.4 to 2.3 times for BKD ranges; search() repeated its work on both formats. - **Optimization:** a run is shared by concurrent identical queries under the result cache's key, whether or not the result cache is on. Scoring runs and runs restricted to candidate rows run alone. The leader publishes on every exit path without allocating, and a follower whose leader ended without a result runs for itself. search() hands every follower its own copy, since SQL composition mutates its input. - **Results:** a branch-4.2 build with these changes, 100M rows in 6.3M-row segments, one client, alternating runs on one BE: CLucene MATCH_REGEXP 'cach.*' CPU 18.4 to 6.0 s and MATCH_PHRASE 1.98 to 0.77 s, both equal to an unsplit scan; search() `*cache*` 14.8 to 5.3 s (V3) and 11.3 to 4.8 s (SNII). At 64 clients, with search() runs shared only within a query, its workload served 57 to 139 queries per second (V3) and 77 to 166 (SNII), p99 3315 to 1046 ms and 1625 to 707 ms; sharing across queries as well, as here, measured up to 7% more. Rows identical. - **Applicability and cost:** segments larger than the rows per scanner. Followers wait for the leader instead of repeating its work; concurrent identical queries of different sessions share a run as well, as they already did on SNII. Scanner layout, writing and storage are unchanged. - **Behavior change:** CLucene readers now share concurrent identical queries; `FulltextIndependentQueries`, which asserted they did not, becomes `FulltextConcurrentQueriesShareRuns`. ### Testing Local ASAN build: - `InvertedIndexReaderTest.FulltextIdenticalQueriesShareOneRun` and `BkdIdenticalQueriesShareOneRun` hold the leader of two identical queries until the follower joins, then check that both get the same rows from one run; `FulltextConcurrentQueriesShareRuns` checks every one of three concurrent queries leads, joins or hits the cache. - `SearchFlightTest`: scanners of one search share a run and each gets its own copy; a different search runs alone and completes while the first is held; a follower whose leader fails runs for itself; a scoring run never joins. - With each shared run switched off in turn, every sharing test above fails. - 2832 inverted-index, search, match, SNII and BKD unit tests pass; clang-tidy reports nothing on the changed lines. - Regression `inverted_index_p0/test_split_scanners_share_index_runs` on a local ASAN cluster: one 16384-row CLucene segment split among 8 scanners (NumScanners 8 in the profile) gives the same rows as a whole-segment scan for MATCH, keyword, BKD, search() and compound predicates. ### Release note Queries with inverted index predicates on large segments use less CPU when the segment is read by several scanners. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: local ASAN unit and regression tests only; CI not run ### AI Disclosure - Tool: Claude Code (Claude Opus 5.5) - Generated: code, tests, this description - Human review: not yet reviewed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
run buildall |
Member
Author
|
/review |
Contributor
|
Codex automated review failed and did not complete. Error: Review setup stopped before context preparation ran; inspect the first failed or cancelled setup step. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: #68758 (added the shared run for SNII)
Problem Summary: Every scanner of a split segment evaluates the segment's index predicates over the whole segment, so a 6.3M-row segment cut at 2M rows per scanner runs them three times. CLucene and BKD readers now share concurrent identical queries as SNII readers already do, and search() shares its whole-segment result the same way. At 64 clients a search() workload served 139 instead of 57 queries per second.
Release note
Queries with inverted index predicates use less CPU when a large segment is read by several scanners.
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)
🤖 Generated with Claude Code