[improvement](datasource) Bound and fairly schedule external scan planning by 924060929 · Pull Request #69298 · apache/doris

924060929 · 2026-10-19T17:41:44Z

What problem does this PR solve?

Issue Number: None

Problem Summary: External scans can monopolize shared FIFO workers, block on metadata submitted to the same pool, repeat path/backend-assignment work, and load cold Iceberg manifests serially. This change uses query-owned bounded task groups, separate split and metadata workers, cooperative continuations and bounded completion-order manifest prefetch. Early stop and exceptions close query-owned tasks and resources, including planning failures before a scan can register its finish callback. Path reuse, bounded consistent-hash candidate lookup and compatible range compaction stay with scheduling to avoid moving the bottleneck into scan bookkeeping. File-format selection and snapshot-specific partition type adaptation preserve per-file semantics without duplicate SDK planning or changes to shared cache values.

Each backend's bounded Split channel wakes on data, completion, cancellation and failure, eliminating the empty terminal request's previous 100 ms polling delay. An earlier four-query JFR reproduction found 78 delayed EOF requests in 400 SQL executions; the fixed path has none in the subsequent replay.

Initial RPC delivery is separate from background lookahead: split_initial_batch_size (default 64) limits the first response per backend, additionally capped by its request size. Further responses use the BE request size. Producers continue before BE consumption, limited by the configured queues and cooperative fairness. The response bound counts real splits before transport-envelope compaction. There is no producer-side wait for a first BE request.

The query-level enable_external_table_batch_mode switch is checked at the engine entry before either partition-batch capability checks or file-streaming estimates. Setting it to false consistently selects synchronous planning; previously the partition-batch path could bypass the switch. Provider implementations still determine whether asynchronous planning is supported. Partition/file thresholds and producer/queue defaults are unchanged.

Scan-local partition adaptation preserves delete matching after numeric promotion and source-schema cleanup, even when data and delete Manifest entries have different cache warmth. UNKNOWN partition targets use consistent numeric Java representations before delete-key comparison, without rewriting shared cache entries. Regression tests exercise both cache orders, identity/truncate partitions, concurrent historical/current projections and different-partition exclusion.

Split channels and their capacity waiters are indexed by backend ID. This avoids repeatedly hashing and boxing a Backend's endpoint/health fields for each producer batch, which was visible in CPU/alloc flamegraphs at split_batch_size=1. The immutable ID also keeps one bounded channel when a backend handle's metadata changes. Batch generation size, FIFO delivery, backpressure, wakeup ordering, cancellation and fair scheduling are retained. A focused queue/lifecycle suite has 63 passing tests, including three-backend streaming and the same-ID/different-handle capacity-wakeup boundary.

Main benefits and independent SQL measurements

This PR addresses two different concerns: bounded scheduling and query lifecycle correctness, and overlapping cold metadata I/O. Query groups rotate ready work and bound queued tasks; separate SPLIT and METADATA workers avoid parent/child pool starvation; cancellation prevents queued work and cache-to-SDK fallback from restarting a finished query. These are scheduling/resource guarantees, not a claim that every SQL statement becomes faster.

The clearest standalone latency gain is Iceberg cold full scans: approximately 32–34% less complete-SQL time. Hive already has partition-parallel listing, so warm Hive single-query latency remains approximately flat. With the same Linux AArch64 Release BE, one backend, six CPUs, warm BE footer/data cache, split_batch_size=1 and split_queue_capacity=4096:

Single-query complete SQL median, ms	Unoptimized master baseline runs	This PR alone	Interpretation
Hive, 1,000 partitions, warm LIMIT 10	8.60–9.28	8.81	Approximately flat
Hive, 10,000 partitions, warm full scan	752.84–783.66	748.83	Small difference; no large speedup claimed
Iceberg, 1,000 manifests, cold LIMIT 10	925.44–933.77	920.35	Approximately flat; partition-view work remains
Iceberg, 1,000 manifests, cold full scan	2109.54–2168.54	1425.94	32–34% less elapsed time

Baseline: frozen unoptimized master e0bf4874070. The standalone measurement package records 9b6d902ffa1 (built source d61d21b943a before its equivalent EOF-fix amendment). Each single-query cell has 40 measured samples across forward/reverse runs after prewarm. Cold means FE catalog/Manifest invalidation, not cold JVM or BE data. These historical measurements predate the subsequent initial-delivery refinements, rebase and review repairs; they establish the standalone implementation's measured benefit and are not a fresh A/B of the current head. The total PR gain includes Manifest prefetch and scan-path improvements; it does not isolate the fairness mechanism itself.

The focused initial-delivery experiments below compare two already-optimized FE versions. For example, Hive four-query LIMIT group median improves 15.38 → 11.82 ms (23.1%), while the controlled slow-listing/delayed-BE full scan stays 388.52 → 386.96 ms. These incremental results must not be added to, or substituted for, the standalone master comparison above.

Ready split batches are drained together within each BE request, capped at 64 producer batches and the number that fits the remaining request. This amortizes queue operations and per-batch channel checks without waiting to fill a response or changing producer buffering. Capacity callbacks execute after the queue lock is released; callback-published errors and cancellation are checked before returning the drained data. The scalar path still handles empty queues and partial batches.

A four-phase fixed-Release-BE comparison on a warm 10,000-file Iceberg table narrows single-query batch-one overhead against each phase's batch128 control from 5.8–6.4% to 0.7–2.8%. Four-query full scans show no stable change. The initial four-query LIMIT difference of approximately 1 ms does not reproduce in a separate 3,200-SQL reverse-order repetition; sub-millisecond differences remain noisy. CPU samples inside getNextBatch are essentially unchanged (392 versus 389), while allocation samples fall from 444 to 174, mostly boxing and iterators. These are statistical samples, not exact invocation/byte counts. Across both formal comparisons, 4,800 SQL statements pass result checks; separate cache-hit profiles, Hive correctness and Iceberg cold LIMIT startup checks also pass. Both isolated PR and integration worktrees pass 66 focused unit tests and full build.sh --fe including Checkstyle. This comparison changes only SplitSource.class; plugins and the Release BE are identical.

Release note

External Hive and Iceberg scans support bounded fair task scheduling, query cancellation and concurrent cold Manifest loading. The new split_initial_batch_size session setting controls first-response size independently of split_batch_size and split_queue_capacity. File formats, pinned snapshots and full-scan results are preserved.

Check List (For Author)

Current-head repair validation at 28fb25d01bd: StmtExecutorTest and IcebergManifestReadContextTest passed (69 tests); build.sh --fe -j4 and Checkstyle passed. Earlier review-repair validation at 17859e3ff14: 1,521 Iceberg package tests (zero failures/errors, one environment-dependent skip), two query-lifecycle tests, and build.sh --fe -j4 including Checkstyle passed. Coverage includes cancellation before first scheduler acquisition, all four cache-to-SDK fallback paths, shared query read admission and identity/truncate data/delete manifests after source-schema cleanup. Previously reported live SQL/regression and performance experiments are historical and were not rerun after every review repair. CI status remains tracked by the checks on this PR.

Test: Clean build.sh --fe, including Checkstyle; 750 scoped FE/SPI/Hive/Iceberg tests in 53 classes, zero failures/errors/skips. Updated Hive streaming regression and Iceberg scheduled-split regression both passed without skips. Matched Release-BE experiment: 1,200 four-query LIMIT SQL and 150 full scans, all result checked. Separate 400-query RPC/EOF diagnostic; footer/page-cache profile verification.
Behavior changed: Yes, external scan scheduling, bounded metadata prefetch, transport compaction and initial delivery size.
Does this need documentation: Yes, developer scheduling documentation included.

Supporting experiments: initial delivery, full-scan overlap and RPC diagnostics

Focused initial-delivery experiment

Same Linux AArch64 Release BE, one backend, six engine CPUs, warm BE data/footer cache; num_partitions_in_batch_mode=1, split_batch_size=1, split_queue_capacity=4096. Hive fixture: 1,000 partitions, one file per partition. The reference FE already includes the other scheduling/scan changes. It is the exact FE before the initial-delivery change, not unoptimized master or SR.

Each fresh FE follows identical 45-second full-scan/LIMIT prewarm. Four-query LIMIT results have 100 groups / 400 SQL per version, without agents, profilers, builds or unit tests during timing. Every LIMIT length and full COUNT/SUM result is checked. Full scans use 10 measured SQL per cell; cold cells run forward/reverse order.

Complete SQL milliseconds	Before initial-delivery change	Independent initial delivery
Four-query LIMIT, group median	15.38	11.82
Four-query LIMIT, group mean	16.57	11.94
Four-query LIMIT, group p95	26.94	14.10
Four-query LIMIT, per-SQL median	13.95	11.15
Warm full scan	57.57	57.24
Cold directory listing, local S3	104.68	101.17
Cold directory listing, first fetch delayed 300 ms	379.52	377.03
Cold listing with additional 20 ms/request, no fetch delay	344.54	345.52
Cold listing with additional 20 ms/request and 300 ms first-fetch delay	388.52	386.96

The cold-listing catalog disables only the file-list cache over the same table, with warmed partition metadata and BE data/footer caches. First-fetch delay is injected at SplitSource.getNextBatch entry before demand callbacks; the slower-listing case injects 20 ms before each real directory request. These controlled delay-injection measurements include the sleeps and do not represent ordinary production latencies or all resource effects of a slow BE startup.

A rejected producer startup barrier had the same LIMIT group median (11.85 ms), but the slower-listing/delayed-fetch full scan took 651.26 ms. Independent initial delivery restores this to 386.96 ms. During the same 300 ms delay, both the pre-change reference and final implementation complete another 896 directory listings (median); the barrier completed zero. The final source contains no startup barrier. The barrier's per-SQL LIMIT median was 10.77 ms versus 11.15 ms now; group mean, p95 and maximum all improve. Both distributions are retained in validation rather than claiming uniform per-SQL improvement.

Separate diagnostic replay of 400 LIMIT SQL has 400 Split RPCs: 399 first responses contain 64 real file ranges and one contains 25. Every first response satisfies the bound, with no extra RPC needed in these LIMIT cases and no delayed EOF request. Diagnostic query timings are excluded from the table. A full-scan profile records 1,000 footer hits / zero footer reads and 1,000 page hits / zero page misses.

Backend candidate lookup was independently measured with production ConsistentHash/Backend classes, 256 virtual nodes, JDK17/macOS ARM64: 0.21–0.26 microseconds at 2–128 backends, including four threads. That is a function benchmark, not multi-backend complete SQL. No SR performance matrix was rerun. The subsequent Iceberg matrix is recorded below with its own fixed BE identity.

Iceberg first-delivery follow-up

Matched previous/current FE (3b97eee547c / 3e94facccfb), keeping the same Linux AArch64 Release BE throughout the timed runs (SHA256 prefix 26a52681f1ed). Only initial-delivery behavior differs between these FE variants; the rest of the implementation is held fixed. This BE differs from the one used in the Hive table above, so absolute values should not be mixed between the tables.

Doris Manifest Cache enabled; SDK manifest cache disabled. Both use producer batch1, queue4096, manifest concurrency8, pruningfalse. Cold means catalog/manifest invalidation with hot BE data/footer caches and warm JVM. Before each cold four-query group, invalidate exactly once, outside timing; all four then run concurrently. Previous groups' background tasks drain first. Every COUNT/SUM and LIMIT length is checked. Single-query40 samples/cell; four-query30 groups/cell, forward/reverse order.2560 measured SQL.

Manifests	Cache	SQL	Single before ms	Single after ms	4-query GROUP before ms	4-query GROUP after ms
100	warm	limit	8.36	6.65	15.52	11.41
100	cold	limit	16.77	16.40	17.67	17.72
100	warm	full	78.31	81.03	192.53	195.17
100	cold	full	111.02	107.41	219.79	222.71
1000	warm	limit	8.17	6.78	13.96	9.78
1000	cold	limit	15.52	15.91	16.62	16.73
1000	warm	full	79.51	78.68	190.70	196.50
1000	cold	full	244.96	241.17	292.82	301.91

Warm LIMIT improves17–20% single-query and26–30% group completion. Cold LIMIT is near-flat. Some full-scan points increase1–3.5%, which are disclosed rather than calling all cases faster.

An additional same-JVM parameter-only ABBA comparison (cap64 versus unlimited,20 samples/groups per mode/cell;1600 SQL) shows full scans mostly within-1.2% to+2.1%. Its1000-manifest cold single full scan had one13.6% median increase; a focused random-order repetition of50 samples/mode did not reproduce it (unlimited223.60ms versus64's223.25ms). The median-difference bootstrap95% interval is-12.90 to+8.28ms. These runs do not establish zero overhead or equal tails; they did not reproduce a consistent large full-scan regression. No product code change is made from the small mixed differences.

Separate diagnostics directly count actual Manifest-cache loader executions: cold100/1000, warm0/0. Both full-scan profiles show10000 footer hits/zero footer reads and10000 page hits/zero misses. No agent, profiler, build or UT runs during the4260 measured SQL. Supplementary RPC diagnostics are excluded from the timing claims.

Focused multi-backend small-batch follow-up

Balanced assignments now consult the existing weight-count index before constructing redistribution heaps. Successful offers and consumers with no capacity waiter bypass the query-wide capacity monitor. On a failed offer, callback registration precedes the retry under that monitor, preserving wakeups when capacity is released before registration. Listener-set mutations and error fanout remain protected, and callbacks execute outside the monitor.

A four-phase previous/current FE experiment holds the remaining code fixed and alternates reference/current/current/reference packages. The fixed Linux AArch64 Release BE uses three backends, two configured cores each, a shared six-CPU budget, 4 GiB per BE and footer-cache capacity 20000. Both variants use partition/file thresholds 1, initial response 64, producer batch1 and queue4096. All 3200 measured SQL results pass; each table cell has 80 queries/groups. Separate profiles verify zero footer reads and zero page misses on every backend. No profiler, agent, build or unit test overlaps timing.

Complete SQL median, ms	Previous implementation	Current implementation
Hive 10000 files, single LIMIT 10	5.464	5.131
Hive 10000 files, four-query LIMIT group	11.815	11.155
Hive 10000 files, single COUNT/SUM	55.577	55.486
Hive 10000 files, four-query COUNT/SUM group	159.172	160.824

The demonstrated benefit is reduced FE resource/lock cost: equal-workload sampling of 240 full scans reduces active CPU samples from 6385 to 4373 (31.5%) and allocation samples from 39174 to 25396 (35.2%). Recorded accumulated waits at capacity notification fall from 295.112 ms to zero above the 10-us recording threshold. Samples and accumulated parallel waits are not complete-SQL time. Full scans remain approximately flat, with the four-query median increasing 1.0%; this experiment does not establish an end-to-end full-scan speedup. The current same-version batch128/queue32 control takes 152.504 ms for four-query full scans, leaving 5.5% batch1 overhead. LIMIT is preserved. Batch128 LIMIT controls also improve in this experiment, so the full observed LIMIT difference should not be attributed solely to batch1 handling.

Both isolated PR and integration worktrees pass 120 focused policy/split/concurrency tests and full FE build/Checkstyle. Added tests cover redistribution across cumulative weight thresholds, monitor-independent uncontended paths and deterministic capacity release before waiter registration. Source and modified-class identities match between the independently built worktrees. Configuration defaults remain unchanged.

hello-stephen · 2026-10-19T17:41:51Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

924060929 · 2026-10-19T17:41:53Z

run buildall

924060929 · 2026-10-19T17:41:55Z

/review

github-actions

Reviewed #69298 at 7de569daeb9521790075380d85a36e81968bf400 against the supplied authoritative aggregate diff. All 77 changed paths were covered by the main review and two reviewers with complementary scopes; an additional reviewer checked four cross-boundary risks.

The scheduling and scan hot-path changes broadly implement their stated goal. Four P2 issues are reported inline: retained manifests fail after source-schema cleanup, recovered schema names can collide with partition names, detached prefetch can outlive its FileIO owner, and one existing producer test retains malformed scan envelopes. No existing inline threads or independently confirmed existing P0/P1 comments were present. No additional user focus was supplied.

The invocation chain reviewed was:

statement -> captured query groups -> SPLIT producer -> Hive/Iceberg source
                                           |             |
                                           |       METADATA workers
                                           v             |
                                  assignment + BE queues <-+
                                           |
                                  SplitSource / compaction
                                           |
                                  BE descriptor iterator -> per-file reader
finish/cancel -> stop groups/producers -> statement/read-lease cleanup

Applicable code-review skill checkpoints:

Checkpoint	Conclusion
Goal and proof	Fair task groups, separate metadata workers, bounded completion-order prefetch and smaller transport envelopes are implemented. Checked-in tests cover their main behavior; the four reported cases need fixes.
Scope and abstraction	Broad but related changes span scheduling, lifetime and scan transport. Shared helpers and SPI keep generic planning connector-agnostic.
Concurrency and protected state	Worker, planner, cancellation and RPC threads use captured sessions, atomics/concurrent queues, group admission locks, producer scheduleLock and assignment/readiness/consumer locks. The detached-load ownership issue is reported inline.
Work under locks	External reads, future waits and user callbacks are outside registry locks; backend-policy mutation remains serialized separately from readiness publication.
Lock order and deadlock	Reviewed admission/listener/queue/cleanup paths showed no additional concrete lock cycle.
Lifecycle and initialization	Captured groups reject resurrection, queued work cancels, and source cleanup is cooperative. Running metadata callables still need ownership through actual exit. No C++ static-initialization change.
Configuration	Three FE worker/capacity settings are restart-scoped and captured at creation. The forwarded session manifest concurrency is validated to 1..64 and shared across query scans.
Symbol/protocol/storage compatibility	Connector API major advances 18 to 19 with surface/version baselines; no new persisted format. Existing BE readers support nested range descriptors.
Thrift/protobuf definitions	None changed. Existing optional per-range fields and query-option defaults were checked at consumption sites.
Parallel paths	Both Coordinator and ThriftPlansBuilder compact transport copies; streaming enforces the original split count before compaction. Native/JNI and row-id paths consume per-range formats.
Special conditions	Backpressure/cancellation conditions have concrete triggers and comments. URI-prefix reuse and backend candidate clamping preserve reviewed semantics. Dropped-source recovery has the two inline defects.
Test coverage	Positive/negative, cancellation, failure, queue-capacity, single-worker, race, serialization, path and schema-promotion cases exist. The stale producer fixture introduces an intermittent test failure.
Changed expected results	Hive fixture totals and Iceberg count/sum/union/limit outputs are consistent by inspection. Output-generation provenance was not independently reproduced.
Observability	Fair-executor pool/active/queue/group/completion metrics and existing generation/cache profiles and exception context cover the new paths. No separate proven gap found.
Transactions, persistence and failover	Read planning and write-format inference are changed; no EditLog, committed-data visibility, rowset version or MoW delete-bitmap protocol change. Read resource ownership was traced through statement cleanup.
Writes and atomicity	No new data-write or commit protocol. Write-side ORC inference remains; no additional atomicity defect identified.
FE-to-BE values	Formats, credentials, partitions and connector descriptors remain in existing fields and survive both transport paths.
Performance and memory	Producer work, task queues, prefetch results and query read permits are bounded. Warm-cache reads avoid futures; path reuse and envelope compaction avoid repeated wrapper work. Scanner concurrency is preserved by the wrapper floor.
Other issues and final recheck	Retry/Flight/planning-only cleanup, continuation admission, schema evolution/delete matching, mixed formats/JNI and test fixtures were rechecked. Every candidate and initial risk has an explicit ledger disposition.

Validation was static only, as required: no builds, tests, cluster execution or source edits. Author-reported performance and test results were not independently rerun. There are no unresolved candidates or uncovered changed paths within this static scope. The review converged after 2 rounds: every assigned reviewer returned NO_NEW_VALUABLE_FINDINGS on the same frozen set.

Review converged after 2 round(s).

hello-stephen · 2026-10-19T18:53:53Z

TPC-H: Total hot run time: 28431 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7de569daeb9521790075380d85a36e81968bf400, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17584	5241	5149	5149
q2	2213	296	248	248
q3	10180	1381	820	820
q4	4678	463	345	345
q5	7455	794	522	522
q6	182	188	154	154
q7	753	805	604	604
q8	9245	1333	1066	1066
q9	5394	4164	4180	4164
q10	6831	1281	1024	1024
q11	451	249	228	228
q12	643	400	280	280
q13	18075	3576	3022	3022
q14	272	272	249	249
q15	q16	731	706	653	653
q17	1193	746	586	586
q18	6980	6143	6085	6085
q19	1101	958	613	613
q20	377	327	226	226
q21	4989	2137	2323	2137
q22	353	297	256	256
Total cold run time: 99680 ms
Total hot run time: 28431 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5731	5695	5673	5673
q2	723	606	585	585
q3	5216	5343	4910	4910
q4	2181	2295	1772	1772
q5	5229	5157	5710	5157
q6	258	193	158	158
q7	2344	2072	1878	1878
q8	3094	2766	2745	2745
q9	7872	7970	7798	7798
q10	3734	3681	3291	3291
q11	628	590	455	455
q12	735	780	556	556
q13	3553	4069	3236	3236
q14	308	314	304	304
q15	q16	709	719	670	670
q17	8407	7630	7085	7085
q18	13092	12318	13092	12318
q19	955	830	831	830
q20	2195	2170	1974	1974
q21	5959	5052	5203	5052
q22	512	488	420	420
Total cold run time: 73435 ms
Total hot run time: 66867 ms

hello-stephen · 2026-10-19T19:03:38Z

TPC-DS: Total hot run time: 150261 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7de569daeb9521790075380d85a36e81968bf400, data reload: false

query5	4302	583	425	425
query6	423	208	193	193
query7	4799	448	238	238
query8	312	160	144	144
query9	8822	3092	3072	3072
query10	473	300	256	256
query11	5926	3855	3546	3546
query12	144	100	86	86
query13	1249	430	316	316
query14	6042	3929	3647	3647
query14_1	3408	3396	3403	3396
query15	213	192	176	176
query16	944	439	402	402
query17	845	697	524	524
query18	2428	409	293	293
query19	185	154	136	136
query20	81	81	81	81
query21	208	120	100	100
query22	13037	12997	12818	12818
query23	13388	12744	12414	12414
query23_1	12587	12575	12418	12418
query24	7399	837	455	455
query24_1	463	471	474	471
query25	508	401	362	362
query26	1260	248	135	135
query27	2801	429	267	267
query28	4621	1923	1918	1918
query29	1576	588	457	457
query30	305	215	181	181
query31	1066	908	785	785
query32	147	88	92	88
query33	514	282	234	234
query34	907	835	496	496
query35	742	779	652	652
query36	849	807	706	706
query37	119	91	86	86
query38	1801	1702	1670	1670
query39	788	798	717	717
query39_1	725	726	735	726
query40	214	110	96	96
query41	65	67	95	67
query42	88	82	82	82
query43	338	360	304	304
query44	1283	670	681	670
query45	176	177	158	158
query46	820	961	551	551
query47	3034	2912	2943	2912
query48	293	296	215	215
query49	575	395	294	294
query50	671	273	210	210
query51	10640	10416	10440	10416
query52	79	80	69	69
query53	187	207	154	154
query54	265	200	173	173
query55	75	68	66	66
query56	226	208	213	208
query57	1646	1630	1567	1567
query58	280	259	253	253
query59	2063	2088	1914	1914
query60	290	223	220	220
query61	162	140	141	140
query62	393	327	282	282
query63	201	153	154	153
query64	2735	908	821	821
query65	3302	3262	3252	3252
query66	1801	436	323	323
query67	20101	20086	19917	19917
query68	2988	1008	622	622
query69	417	291	259	259
query70	861	771	764	764
query71	301	248	211	211
query72	2797	2524	2270	2270
query73	515	536	292	292
query74	4608	4485	4339	4339
query75	2316	2277	1892	1892
query76	2317	1008	577	577
query77	341	379	254	254
query78	9794	9666	9148	9148
query79	887	851	508	508
query80	519	460	359	359
query81	534	322	271	271
query82	578	138	102	102
query83	304	193	176	176
query84	309	124	103	103
query85	799	424	365	365
query86	327	239	224	224
query87	1979	1974	1811	1811
query88	3634	2661	2660	2660
query89	314	282	243	243
query90	1848	163	160	160
query91	156	155	126	126
query92	95	87	88	87
query93	990	994	568	568
query94	445	305	269	269
query95	574	341	352	341
query96	677	480	236	236
query97	2414	2425	2293	2293
query98	163	154	147	147
query99	764	750	649	649
Total cold run time: 233742 ms
Total hot run time: 150261 ms

hello-stephen · 2026-10-19T19:08:30Z

ClickBench: Total hot run time: 24.35 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7de569daeb9521790075380d85a36e81968bf400, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.60	0.10	0.10
query5	0.26	0.24	0.23
query6	1.19	0.69	0.68
query7	0.04	0.00	0.00
query8	0.06	0.03	0.04
query9	0.41	0.32	0.35
query10	0.58	0.59	0.60
query11	0.21	0.14	0.14
query12	0.18	0.14	0.14
query13	0.50	0.51	0.49
query14	1.05	1.01	1.02
query15	0.62	0.60	0.59
query16	0.36	0.35	0.35
query17	1.13	1.10	1.14
query18	0.22	0.19	0.20
query19	2.11	1.96	1.97
query20	0.02	0.01	0.02
query21	15.43	0.20	0.14
query22	4.95	0.06	0.05
query23	16.26	0.33	0.12
query24	3.06	0.63	0.45
query25	0.13	0.10	0.06
query26	1.48	0.22	0.14
query27	0.05	0.05	0.04
query28	3.51	0.73	0.35
query29	12.52	4.21	3.30
query30	0.30	0.14	0.10
query31	2.77	0.56	0.31
query32	3.21	0.60	0.52
query33	3.14	3.23	3.19
query34	15.40	3.98	3.38
query35	3.34	3.33	3.37
query36	0.58	0.45	0.42
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 97.57 s
Total hot run time: 24.35 s

924060929 · 2026-10-19T19:32:40Z

run buildall

924060929 · 2026-10-19T19:32:42Z

/review

github-actions

Static review of all 77 changed paths at eadfc6f identified one additional P2: an outer cloud replan retry can permanently retain newly allocated scheduler groups after repopulating a closed statement scope. The inline comment describes the trigger, ownership chain and fix direction.

The four existing P2 reports remain applicable and are not duplicated: retained-manifest schema cleanup (4317059423), recovery-schema name collisions (4317059437), metadata callable/table-owner lifetime (4317059444), and malformed producer-test envelopes (4317059450). There are no existing P0/P1 findings to carry as blocking IDs. No additional user-provided focus was supplied.

Goal and proof: the code implements query-owned fair scheduling, separate metadata workers, cooperative split production, bounded completion-ordered prefetch and smaller transport envelopes. Tests cover these mechanisms; the new retry leak and known P2 cases qualify the correctness conclusion.
Scope and clarity: the changes are broad but follow the documented scheduling/hot-path goal and connector SPI boundary. Per-file format replaces read-side format inference; writer inference remains available.
Concurrency: traced planning, SPLIT/METADATA workers, BE-fetch RPCs and cancellation/finalization. CAS/queues own task publication; admission, producer, readiness, backend-policy and per-backend wait locks protect their respective state. Tasks, IO and callbacks execute outside scheduler registry locks. No additional deadlock, lost wakeup or concurrent source iteration was verified.
Lifecycle: captured groups reject resurrection through an old session; normal cleanup, planning failure, inner retries and deferred Flight were traced. Outer replan scope reuse is the new inline finding; running FileIO owner retention is the existing report. No static initialization issue applies.
Configuration: three FE thread/queue settings are nonmutable and captured at executor creation; no dynamic-resize promise is made. Session manifest concurrency is forwarded, checked in [1,64] and shared through a query-scoped semaphore.
Compatibility: connector API advances 18.0 to 19.0 with its version assertion and plugin surface baseline. New SPI defaults/forwarding are present. ConnectorMetadata methods, persisted formats and existing wire descriptors are unchanged.
Thrift/protobuf definitions: none changed; cross-branch field-ID comparison is inapplicable.
Parallel paths: checked Hive/Iceberg, cache-on/off, synchronous/streaming/COUNT, metadata tables, both FE dispatch builders and BE local/remote split readers. Per-file formats precede reader/schema setup, including ORC and row-ID fetch.
Special conditions: startup demand bypasses the single-lane case; terminal changes wake consumers; admission rechecks cover early dequeues. Partition adaptation uses field IDs, preserves nulls and rejects lossy narrowing. Known schema-recovery cases remain reported above.
Test coverage: reviewed changed unit and regression cases for queue pressure, fairness, cancellation, failure, schema promotions, deletes, mixed formats, serialization and LIMIT/full/multiple scans. Outer replan group cleanup needs coverage; the existing malformed fixture and three Iceberg edge cases also remain gaps.
Expected results: Hive expectations match the added scheduling configuration. Iceberg IDs 0..191 imply 192 rows/distinct IDs and sum 18336; UNION gives 384 and 36672, and LIMIT counts are 10. Actual execution and output-generation provenance were not independently established.
Observability: pool size, active/queued tasks, group count and completions have metrics; existing scan profiles, atomic manifest counters and contextual errors remain. No additional necessary observability omission was verified.
Transactions/persistence: no EditLog, committed-data visibility or persisted-version protocol changes. Read-transaction cleanup remains classloader-pinned; the lifetime issues are identified separately.
Writes/atomicity: no new data mutation or commit protocol; write-format inference preserves its previous resolution/failure behavior. No additional crash/atomicity regression was verified.
FE/BE propagation: no new wire variable. Compaction preserves all inner descriptors, parameter/volume/location boundaries and separate split sources; both BE readers drain every inner range and remote EOF requires an empty response.
Performance/resources: bounded task queues, one pending producer batch, manifest windows/read budgets, capped hash candidates and per-producer path reuse address the intended costs. Scanner parallelism is retained. The new permanent group retention is the resource exception; reported benchmarks were not rerun.
Further review: independently challenged continuation overlap, callback reentry, option defaults, historical cache load orders, delete matching, ORC initialization and both retry loops. Only the outer-replan scheduler leak adds a distinct finding beyond the existing threads.

Review converged after 2 rounds: all three latest reviewers returned NO_NEW_VALUABLE_FINDINGS on the unchanged one-comment set. All changed paths, candidates and risk focuses have explicit dispositions; no unresolved review question or uncovered changed path remains. Validation was static only: no builds, tests, runtime stress or benchmark reruns, and no repository source changes.

Review converged after 2 round(s).

hello-stephen · 2026-10-19T20:03:14Z

FE UT Coverage Report

Increment line coverage 85.95% (477/555) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-10-19T20:52:45Z

TPC-H: Total hot run time: 28445 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eadfc6f447199375305fb429e1235d45f8beda75, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17570	5106	5063	5063
q2	2116	352	255	255
q3	10205	1384	828	828
q4	4690	466	336	336
q5	7435	801	537	537
q6	184	183	154	154
q7	750	792	598	598
q8	9254	1333	1097	1097
q9	5530	4222	4220	4220
q10	6819	1286	1028	1028
q11	444	261	227	227
q12	636	403	275	275
q13	18098	3517	2992	2992
q14	280	268	245	245
q15	q16	730	722	657	657
q17	1197	741	581	581
q18	7022	6167	6117	6117
q19	1127	954	589	589
q20	391	344	228	228
q21	4944	2160	2300	2160
q22	339	289	258	258
Total cold run time: 99761 ms
Total hot run time: 28445 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5620	5611	5639	5611
q2	654	570	522	522
q3	5234	5388	4947	4947
q4	1997	2157	1554	1554
q5	5289	5187	5437	5187
q6	257	202	153	153
q7	2300	2098	1892	1892
q8	3044	2796	2807	2796
q9	7994	8072	8019	8019
q10	3732	3669	3227	3227
q11	635	606	440	440
q12	695	680	510	510
q13	3589	3897	3251	3251
q14	328	331	299	299
q15	q16	716	714	653	653
q17	8311	7300	6897	6897
q18	12980	12214	12828	12214
q19	918	828	831	828
q20	2164	2155	1900	1900
q21	5678	4582	4677	4582
q22	491	475	435	435
Total cold run time: 72626 ms
Total hot run time: 65917 ms

hello-stephen · 2026-10-19T21:02:31Z

TPC-DS: Total hot run time: 150530 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eadfc6f447199375305fb429e1235d45f8beda75, data reload: false

query5	4318	577	440	440
query6	436	204	191	191
query7	4800	428	226	226
query8	316	165	146	146
query9	8773	3080	3072	3072
query10	469	308	276	276
query11	6056	3834	3563	3563
query12	146	89	89	89
query13	1238	448	332	332
query14	6167	3944	3648	3648
query14_1	3384	3467	3464	3464
query15	205	193	182	182
query16	979	453	407	407
query17	856	645	534	534
query18	2423	424	312	312
query19	201	160	136	136
query20	85	85	85	85
query21	211	130	104	104
query22	12940	12967	12801	12801
query23	13346	12829	12505	12505
query23_1	12674	12561	12620	12561
query24	7375	851	459	459
query24_1	477	478	466	466
query25	534	426	381	381
query26	1261	255	140	140
query27	2770	442	262	262
query28	4642	1922	1949	1922
query29	1569	600	466	466
query30	291	212	186	186
query31	1008	930	836	836
query32	144	96	92	92
query33	535	302	244	244
query34	976	841	501	501
query35	742	750	630	630
query36	807	788	731	731
query37	128	110	85	85
query38	1769	1721	1704	1704
query39	817	797	747	747
query39_1	721	729	734	729
query40	217	119	97	97
query41	72	68	68	68
query42	87	82	83	82
query43	351	365	317	317
query44	1305	694	743	694
query45	179	169	161	161
query46	880	950	558	558
query47	3000	2681	2808	2681
query48	313	303	214	214
query49	566	396	291	291
query50	661	266	205	205
query51	10460	10274	10137	10137
query52	75	78	65	65
query53	181	203	149	149
query54	238	193	211	193
query55	68	68	64	64
query56	220	210	220	210
query57	1675	1623	1560	1560
query58	286	267	247	247
query59	2107	2184	1940	1940
query60	275	239	222	222
query61	145	145	143	143
query62	380	321	299	299
query63	193	154	156	154
query64	2763	906	741	741
query65	3303	3249	3255	3249
query66	1768	425	309	309
query67	20212	20262	19942	19942
query68	2936	998	607	607
query69	384	306	234	234
query70	830	769	763	763
query71	298	234	214	214
query72	2690	2547	2346	2346
query73	511	516	294	294
query74	4624	4489	4340	4340
query75	2324	2284	1911	1911
query76	2291	1011	619	619
query77	350	375	270	270
query78	9715	9749	9211	9211
query79	946	898	517	517
query80	517	405	337	337
query81	527	317	267	267
query82	582	130	110	110
query83	299	193	185	185
query84	305	122	95	95
query85	819	432	363	363
query86	339	238	230	230
query87	2000	1963	1778	1778
query88	3678	2706	2682	2682
query89	321	286	238	238
query90	1884	168	168	168
query91	153	155	123	123
query92	92	88	88	88
query93	1028	985	585	585
query94	446	311	275	275
query95	586	389	317	317
query96	676	489	242	242
query97	2444	2437	2293	2293
query98	162	149	155	149
query99	767	774	642	642
Total cold run time: 234734 ms
Total hot run time: 150530 ms

hello-stephen · 2026-10-19T21:07:22Z

ClickBench: Total hot run time: 24.26 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eadfc6f447199375305fb429e1235d45f8beda75, data reload: false

query1	0.01	0.01	0.01
query2	0.10	0.05	0.05
query3	0.25	0.09	0.09
query4	1.61	0.11	0.11
query5	0.23	0.24	0.24
query6	1.17	0.68	0.70
query7	0.04	0.01	0.01
query8	0.05	0.04	0.04
query9	0.40	0.35	0.37
query10	0.57	0.61	0.58
query11	0.20	0.14	0.15
query12	0.19	0.15	0.16
query13	0.48	0.49	0.50
query14	1.03	1.03	1.02
query15	0.64	0.61	0.61
query16	0.35	0.35	0.36
query17	1.19	1.13	1.15
query18	0.20	0.18	0.19
query19	2.15	1.93	2.02
query20	0.01	0.01	0.02
query21	15.48	0.19	0.13
query22	4.94	0.05	0.06
query23	16.26	0.32	0.12
query24	3.14	0.48	0.38
query25	0.10	0.07	0.05
query26	1.08	0.21	0.16
query27	0.05	0.03	0.05
query28	3.28	0.77	0.38
query29	12.47	4.17	3.26
query30	0.29	0.13	0.09
query31	2.77	0.54	0.32
query32	3.23	0.59	0.51
query33	3.15	3.16	3.28
query34	15.42	4.01	3.39
query35	3.33	3.32	3.33
query36	0.54	0.42	0.42
query37	0.10	0.07	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.92 s
Total hot run time: 24.26 s

924060929 · 2026-10-19T21:17:22Z

run buildall

924060929 · 2026-10-19T21:17:27Z

/review

github-actions

Reviewed all 78 changed paths at 8b6086f, including production code, deleted code, tests, documentation and expected results. No additional distinct inline findings were substantiated. This is a static review, not an approval of the unresolved issues already discussed.

Five existing P2 findings still apply and are not reposted: outer replan scope/task-group leak, producer test envelopes incompatible with compaction, prefetch callable outliving its table/FileIO owner, recovery-schema partition-name collision, and retained manifests after historical-schema cleanup. No existing P0/P1 finding was present or independently confirmed, so existing_blocking_comment_ids is empty.

Critical checkpoint conclusions:

Goal and evidence: separate SPLIT/METADATA workers, cooperative producer continuations, bounded manifest prefetch and transport compaction implement the intended scheduling design. Added unit and regression tests exercise the intended behavior; the five known defects above remain.
Scope and reuse: scheduling, per-producer path reuse, backend candidate bounds and transport helpers form a coherent scan-planning change. The removed read-format cache is replaced by existing per-file format metadata; write-side format resolution remains intact.
Concurrency and locking: traced request, producer, metadata, RPC and cancellation threads through queue admission, listener registration, readiness and terminal notifications. Remote loads and future waits occur outside the manifest pending lock; callback and queue lock ordering did not establish another defect. Scheduling is cooperative and cannot preempt an SDK call that does not return.
Lifecycle: checked queued/running futures, backpressure suspension, early LIMIT, planning failure, EXPLAIN, retry and deferred Arrow Flight cleanup. Existing owner-lifetime and outer-retry findings cover the verified remaining failures; no additional lifetime issue was substantiated.
Configuration: worker counts are startup settings; the bounded manifest concurrency and initial batch size are validated session settings and forwarded where required. Initial delivery limits real splits before compaction and does not suspend producer lookahead.
Compatibility: connector API major version advances to 19.0 with the changed public surface baseline; ConnectorMetadata's baseline surface is unchanged. Both FE plan builders preserve existing FE/BE descriptor fields, and BE already flattens multiple ranges per envelope. No Thrift/protobuf definitions or persisted format changes occur.
Parallel paths: reviewed synchronous/streaming/COUNT planning, Hive partition lanes, native/JNI metadata reads, V1/V2 scanners, mixed ORC/Parquet files, row-id fetches and write-format callers. Per-range physical formats and Iceberg field-id mappings remain authoritative.
Conditions and errors: examined admission/full/closed states, cancellation during permit waits, empty or filtered scans, unsupported formats and lossy historical conversions. Failures propagate through their established error channels; known schema-recovery defects are covered by the existing threads.
Test coverage and expected results: inspected admission, fairness, cancellation, wakeup, bounded-window, promotion/delete, transport serialization and path-identity tests, plus Hive and Iceberg regression cases. Expected aggregate results are internally consistent. The remaining malformed fixture is the existing test finding linked above.
Observability: new pool/active/queued/completed/group metrics complement existing scan profiles and cache counters; no additional observability defect was established.
Persistence and failover: no EditLog, internal partition-version, rowset or delete-bitmap changes. Snapshot pinning and delete-file metadata were traced through cache adaptation before residual/delete matching; the two known historical-spec recovery issues remain.
Writes, transactionality and crashes: write-format property/inference behavior remains, and no write commit or atomicity protocol changes were introduced. Cancellation/resource failure analysis is covered by the lifecycle conclusions above.
FE/BE variable passing: transport compaction retains per-file descriptors and existing fields in both plan builders; new session controls are consumed in FE rather than requiring new BE wire fields.
Performance and memory: inspected bounded task queues/read windows, short-held locks, per-producer path reuse and physical-backend candidate clamping. Synchronous compaction retains the scanner concurrency upper bound. No additional static performance regression was substantiated; performance claims were not independently benchmarked.
Further issues and user focus: no additional focus was supplied. The final missed-path sweep found no unresolved static candidate or uncovered changed path.

Converged in 1 review round: all three assigned reviewers, including the separate risk reviewer, returned NO_NEW_VALUABLE_FINDINGS on the unchanged empty new-comment set. Builds, tests and runtime/remote-I/O timing validation were not performed under this review-only contract. Delivery verification is separate from convergence.

Review converged after 1 round(s).

hello-stephen · 2026-10-19T22:01:54Z

FE UT Coverage Report

Increment line coverage 86.45% (504/583) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-10-19T22:32:49Z

TPC-H: Total hot run time: 28212 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b6086f26c73ca8e693f10a8e886c9e976d5e684, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17567	5186	5012	5012
q2	2091	316	252	252
q3	10258	1379	808	808
q4	4685	456	329	329
q5	7428	792	531	531
q6	181	182	151	151
q7	756	795	605	605
q8	9244	1325	1056	1056
q9	5595	4196	4164	4164
q10	6813	1278	1032	1032
q11	414	248	230	230
q12	646	395	279	279
q13	18063	3504	2999	2999
q14	271	263	237	237
q15	q16	716	708	655	655
q17	1193	742	574	574
q18	7056	6149	6083	6083
q19	1103	943	591	591
q20	370	337	228	228
q21	4354	2132	2262	2132
q22	346	301	264	264
Total cold run time: 99150 ms
Total hot run time: 28212 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5606	5571	5579	5571
q2	706	583	540	540
q3	5207	5312	4871	4871
q4	1984	2150	1541	1541
q5	5243	5153	5299	5153
q6	272	209	161	161
q7	2360	2016	1855	1855
q8	3172	2699	2731	2699
q9	7994	8023	7850	7850
q10	3745	3664	3202	3202
q11	612	446	424	424
q12	867	720	538	538
q13	3549	3902	3267	3267
q14	304	316	296	296
q15	q16	723	701	631	631
q17	8248	7519	6846	6846
q18	13027	12215	12796	12215
q19	901	838	779	779
q20	2135	2141	1919	1919
q21	5657	4684	4658	4658
q22	512	437	440	437
Total cold run time: 72824 ms
Total hot run time: 65453 ms

hello-stephen · 2026-10-19T22:42:35Z

TPC-DS: Total hot run time: 150104 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b6086f26c73ca8e693f10a8e886c9e976d5e684, data reload: false

query5	4299	584	445	445
query6	423	209	188	188
query7	4809	447	217	217
query8	317	164	148	148
query9	8737	3053	3068	3053
query10	482	312	244	244
query11	5998	3832	3556	3556
query12	146	96	87	87
query13	1246	433	307	307
query14	6124	3939	3652	3652
query14_1	3408	3398	3375	3375
query15	208	197	180	180
query16	938	427	351	351
query17	841	653	573	573
query18	2418	411	333	333
query19	183	155	129	129
query20	82	80	77	77
query21	205	120	102	102
query22	13092	12920	12843	12843
query23	13335	12788	12352	12352
query23_1	12547	12442	12556	12442
query24	7379	820	450	450
query24_1	452	452	473	452
query25	493	396	324	324
query26	1255	273	137	137
query27	2769	432	263	263
query28	4650	1958	1909	1909
query29	1512	571	411	411
query30	291	216	181	181
query31	986	918	816	816
query32	129	89	91	89
query33	526	282	233	233
query34	891	846	486	486
query35	746	737	647	647
query36	842	854	746	746
query37	122	97	75	75
query38	1760	1708	1684	1684
query39	808	808	744	744
query39_1	732	722	738	722
query40	213	123	94	94
query41	67	63	60	60
query42	84	83	79	79
query43	345	364	305	305
query44	1291	669	673	669
query45	175	175	157	157
query46	852	959	556	556
query47	3017	2917	2913	2913
query48	326	302	215	215
query49	578	388	293	293
query50	669	270	202	202
query51	10458	10274	10237	10237
query52	78	78	72	72
query53	183	201	151	151
query54	231	188	169	169
query55	69	63	61	61
query56	243	216	226	216
query57	1648	1622	1539	1539
query58	278	259	249	249
query59	2129	2196	2047	2047
query60	298	243	220	220
query61	139	145	141	141
query62	371	334	301	301
query63	189	158	164	158
query64	2712	931	762	762
query65	3283	3245	3259	3245
query66	1759	421	313	313
query67	20137	20086	20024	20024
query68	3231	1002	613	613
query69	381	294	240	240
query70	851	806	774	774
query71	304	243	214	214
query72	2815	2705	2414	2414
query73	509	520	292	292
query74	4615	4497	4322	4322
query75	2313	2303	1910	1910
query76	2325	1011	575	575
query77	342	383	267	267
query78	9885	9683	9102	9102
query79	975	937	536	536
query80	614	421	339	339
query81	521	319	271	271
query82	581	134	103	103
query83	326	197	186	186
query84	314	127	110	110
query85	814	422	367	367
query86	317	231	228	228
query87	1994	1980	1838	1838
query88	3626	2663	2616	2616
query89	330	283	239	239
query90	1894	156	159	156
query91	170	151	127	127
query92	98	86	88	86
query93	1017	986	573	573
query94	434	315	255	255
query95	588	387	297	297
query96	669	478	230	230
query97	2410	2464	2281	2281
query98	162	170	144	144
query99	778	763	638	638
Total cold run time: 234181 ms
Total hot run time: 150104 ms

hello-stephen · 2026-10-19T22:47:27Z

ClickBench: Total hot run time: 24.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b6086f26c73ca8e693f10a8e886c9e976d5e684, data reload: false

query1	0.01	0.01	0.01
query2	0.09	0.05	0.05
query3	0.26	0.09	0.08
query4	1.62	0.10	0.10
query5	0.24	0.24	0.23
query6	1.19	0.68	0.68
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.40	0.34	0.36
query10	0.61	0.58	0.59
query11	0.20	0.14	0.14
query12	0.19	0.15	0.15
query13	0.49	0.49	0.49
query14	1.02	1.01	1.02
query15	0.65	0.61	0.60
query16	0.34	0.36	0.34
query17	1.09	1.12	1.15
query18	0.21	0.20	0.19
query19	2.12	2.00	1.99
query20	0.02	0.01	0.01
query21	15.42	0.20	0.13
query22	4.80	0.06	0.05
query23	16.27	0.33	0.12
query24	3.09	0.67	0.35
query25	0.14	0.06	0.06
query26	1.24	0.21	0.15
query27	0.04	0.04	0.04
query28	3.77	0.79	0.36
query29	12.48	4.11	3.30
query30	0.28	0.14	0.10
query31	2.77	0.56	0.32
query32	3.23	0.59	0.51
query33	3.18	3.23	3.24
query34	15.45	3.99	3.39
query35	3.35	3.35	3.35
query36	0.55	0.45	0.44
query37	0.09	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.15	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.42 s
Total hot run time: 24.42 s

hello-stephen · 2026-10-20T01:21:59Z

FE Regression Coverage Report

Increment line coverage 65.61% (372/567) 🎉
Increment coverage report
Complete coverage report

924060929 · 2026-10-20T04:01:42Z

run buildall

1

github-actions · 2026-10-27T09:22:57Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2026-10-27T09:31:56Z

FE Regression Coverage Report

Increment line coverage 58.60% (402/686) 🎉
Increment coverage report
Complete coverage report

…69718) ### What problem does this PR solve? Issue Number: None Related PR: #69298, #69566, #66530 Problem Summary: #### In short A statement's connector state could outlive the statement in two ways on master. A split producer still inside a read when the statement ends went on to load values into the already-closed connector statement scope, such as an Iceberg table borrowed from the table cache, and nothing ever closed them. `EXPLAIN REFRESH MATERIALIZED VIEW` plans on a context of its own that has no query id, so its scans registered their query-finish cleanup under `""` and its connector scope was never closed: each such EXPLAIN over an external base table left one registry entry and one open scope behind, without bound. This PR makes a closed scope refuse to load, and finishes the explain context's own statement when the EXPLAIN ends. This was found while checking master against branch-4.x's #66473 (scan reuse, ported as #66530). branch-4.x has neither the connector statement scope nor the query-finish registry, and its statement resources already refuse registration once closed. #### Background - Every statement has a connector statement scope (`ConnectorStatementScopeImpl`, hung on the `StatementContext`). Connectors keep what the statement shares in it: the `ConnectorMetadata` of each catalog, Iceberg table leases borrowed from the table cache (returning one is what lets the cache close the table's FileIO when it evicts it), Iceberg tracked tables, write transactions, and the query's task groups since #69298. `closeAll()` releases them when the statement ends. - A scan registers two query-finish callbacks under the query id when it starts planning splits: one stops its split producers, the other (`QueryFinishCleanup`) releases a read transaction and closes the scope. #69298 runs them at the end of every attempt, also for EXPLAIN and for plans that fail; #69566 does the same for prepared EXECUTE. - Batch and streaming scans plan on split producers. A producer calls into the connector off the statement thread, for example Iceberg's `streamSplits`, which resolves its table through the scope. #### The problem, and what it cost 1. **Late producers load into the closed scope.** Stopping a scan only flags its producers (`SplitProducer.close()`), and closing the query's task groups cancels queued work but deliberately does not interrupt metadata IO. A producer already inside `getNextBatch` keeps going after `closeAll()`. `closeAll()` had also cleared the scope, so the producer found nothing and loaded again: Iceberg borrowed a new table lease, Hive partition batches built a new `ConnectorMetadata`. Both stayed in the closed scope forever, so the lease was never returned. A value whose load was in flight while `closeAll()` ran was missed the same way. This happens whenever a statement ends while a producer is still reading: an EXPLAIN of a large table (EXPLAIN also starts batch scans), a LIMIT that finishes early, a KILL, a timeout. Nothing fails and no result is wrong; the FE just keeps the leases, metadata and FileIO alive. 2. **`EXPLAIN REFRESH MATERIALIZED VIEW` leaks its own statement.** It plans on an MTMV context of its own (`RefreshMTMVCommand.getExplainConnectContext`), and that context had no query id, so the connector sessions of its scans had `""` as their query id and registered their cleanup under `""`. The end of the outer statement finishes only the outer query id; nothing finishes `""`, and it could not be finished at the end of one statement anyway, since every context without a query id shares it. The explain context's `StatementContext` was never closed either. | Before this PR | Effect | |---|---| | A producer reading when the statement ends | Borrows a new table lease (or builds a new metadata) into the closed scope; never released | | `EXPLAIN REFRESH MATERIALIZED VIEW` over an external base table | One permanent registry entry under `""` and one open connector scope per execution | #### How this PR fixes it - `ConnectorStatementScopeImpl`: after `closeAll()` the statement is over, so `computeIfAbsent` (and with it `computeIfAbsentForPlanning` and `getOrCreateMetadata`) does not run the loader and throws `CancellationException`, as the query's task groups already do once closed; Iceberg's planning paths treat it as "the query stopped" and do not fall back to the SDK. A load that raced the close is released by its caller, because `closeAll()` could not see it. `closeAll()` now removes each entry before releasing its value, still finalizing transactions first, so every value is released exactly once. `getIfPresent` returns nothing once closed. - `StatementContext.getConnectorStatementScopeForNewSession()`: a session built on a statement whose scope is already closed (the lineage of an INSERT, after the insert finished its query, say) captures `ConnectorStatementScope.NONE` and the direct task scheduler, as offline work does, instead of failing in or loading into the closed scope. Sessions captured before the close keep the closed scope, so a producer holding one stops at its next load. - `ExplainCommand`: an explain context of its own gets a query id, and when the EXPLAIN ends, whether planning succeeded or not, it runs that query id's query-finish callbacks and closes the context's `StatementContext`, as `MTMVTask` does for a refresh. The thread-local context is still restored if that cleanup throws. - Comments that said EXPLAIN never registers a query-finish cleanup and that `closeAll()` runs after the producers have quiesced are corrected; the SPI javadoc of `computeIfAbsent` states the closed-scope behavior. #### Results | | Before | After | |---|---|---| | Producer resolving its table after the statement closed the scope | New lease borrowed, never returned | `CancellationException`, nothing borrowed | | Value whose load raced `closeAll()` | Kept in the closed scope, never closed | Closed once by its caller | | Session built after the scope closed | Loads task groups into the closed scope | Offline session (no scope, direct scheduler) | | `EXPLAIN REFRESH MATERIALIZED VIEW` | Entry under `""` and open scope left behind | Its callbacks run once, its scope closed | New tests: - `ConnectorStatementScopeTest`: `closedScopeRefusesToLoadAndHandsOutNothing`, `valueLoadedWhileTheScopeClosesIsReleasedOnceByItsCaller` (the loader blocks on another thread while `closeAll()` runs), `sessionBuiltAfterTheScopeClosesRunsAsOfflineWork`. - `SplitProducerTest.readThatOutlivesItsStatementLoadsNothingIntoTheClosedScope`: a producer is stopped inside `getNextBatch`, the scope closes, then the read resolves its table. - `ExplainRefreshMTMVCommandTest.testExplainCommandFinishesTheStatementOfItsOwnContext`: planning registers a callback under the explain context's query id and keeps a closeable in its scope; after the EXPLAIN, both succeeding and failing, the callback ran once and the value was closed. Each new test was checked against a mutation of the code it guards; every mutation turns its test red: | Mutation | Red test | |---|---| | Drop the closed check before loading | `closedScopeRefusesToLoadAndHandsOutNothing` (the loader ran twice), `readThatOutlivesItsStatementLoadsNothingIntoTheClosedScope` (the late read loaded once) | | Drop the closed check after loading | `valueLoadedWhileTheScopeClosesIsReleasedOnceByItsCaller` (the caller got the value) | | Let a new session capture the closed scope | `sessionBuiltAfterTheScopeClosesRunsAsOfflineWork` (building the session throws: the scope cannot load `engine:query-task-groups`) | | Do not finish the explain context | `testExplainCommandFinishesTheStatementOfItsOwnContext` (callback never ran) | | Give the explain context no query id | `testExplainCommandFinishesTheStatementOfItsOwnContext` (planned without a query id) |

…hema (#69488) ### What problem does this PR solve? Issue Number: None Related PR: #66538, #57531, #65851 Problem Summary: **In short.** On an Iceberg table, `DEFAULT(column)` anywhere except `UPDATE ... SET` and `MERGE ... VALUES` reads the default from the read schema, which carries no Iceberg write-default. This PR resolves it against the write schema that the statement pinned for its target, as branch-4.1 and branch-4.2 do. **Background.** An Iceberg `write-default` applies to rows written after it is set. So it lives in the writer schema that a write statement pins (`IcebergWriteSchemaContext`), not in the columns of the read schema. #65851 resolves `DEFAULT(column)` against that pinned schema in `UPDATE ... SET`, `MERGE ... UPDATE SET` and `MERGE ... INSERT VALUES` (`ConnectorWriteSchemaUtils`). Every other `DEFAULT(column)` either fails at binding or reaches `RewriteDefaultExpression` (#57531), which reads the default from the bound column. branch-4.1 #66538, also in branch-4.2, resolves these against the pinned write target too. It rejects `DEFAULT(column)` on an Iceberg column in a statement that does not write the table. **The problem, and what it cost.** Reproduced on a local cluster with a `file://` format-version 3 table: `CREATE TABLE t (id INT, c INT)`, `ALTER TABLE t ADD COLUMN d INT DEFAULT 42`, one row `(0, 0)`, so `d = 42`. | Statement | Master today | After this PR (same as branch-4.2) | |---|---|---| | `INSERT INTO t (id, c, d) VALUES (1, 1, DEFAULT(d))` | `Unknown column 'd' in 'table list'` | writes 42 | | `INSERT INTO t (id, c, d) SELECT 2, 2, DEFAULT(d) FROM t WHERE id = 0` | writes NULL | writes 42 | | `INSERT INTO t (id, c, d) SELECT 3, 3, DEFAULT(d)` | `Unknown column 'd'` | writes 42 | | `UPDATE t SET c = 100 WHERE d = DEFAULT(d)` | updates nothing: `d = NULL` makes the plan an empty set | updates every row, `d = 42` is pushed down to the scan | | `MERGE INTO t USING (SELECT 0 AS id) s ON t.id = s.id AND t.d = DEFAULT(t.d) WHEN MATCHED THEN UPDATE SET c = 300` | matches nothing | updates row 0 | | `SELECT DEFAULT(d) FROM t` | NULL | `DEFAULT(d) must reference a column of the table that this statement writes` | The `UPDATE` and `MERGE` cases fail silently: the statement succeeds and changes nothing. **How this PR fixes it.** - fe-core cannot tell from a column whether its connector keeps defaults in the read schema. Hive does: an HMS default shows up in both schemas. Iceberg does not. The new connector capability `SUPPORTS_WRITE_ONLY_DEFAULTS` says which kind a table is. The Iceberg connector declares it. The Hive connector passes it on to iceberg-on-HMS tables, like the other per-table capabilities, but only through the two-argument `getTableSchema`. Since #69299, a statement that pins the table at its latest snapshot reads the schema from the snapshot overload, which reflects none of them, so on an iceberg-on-HMS table such a statement keeps master's results (for example, `UPDATE t SET c = 100 WHERE d = DEFAULT(d)` still changes no row). Reflecting the sibling's capabilities there changes planning for all of them and is left to a follow-up PR. - For a target that declares it, `InsertUtils.normalizePlan` resolves `DEFAULT(column)` in `INSERT ... VALUES` and in the select lists of `INSERT ... SELECT`, including the replacements of `SELECT * REPLACE(...)`, against the pinned write schema. It does this by name, before the source relations are bound, so the name always refers to a target column, as in branch-4.2. In MySQL, `DEFAULT(column)` in a `SELECT` refers to a source column instead. A derived table in `FROM` is a query of its own and is left alone: `DEFAULT(column)` in it still names a column of its own `FROM` clause. - `RewriteDefaultExpression` takes the default for a `DEFAULT(column)` bound to such a table from the pinned write column with the same field id. That covers the `WHERE` of `UPDATE` and the `ON` of `MERGE`. A MERGE resolves it whatever its WHEN branches, also when they all delete, as branch-4.2 does. If the statement pinned no write schema for that table, it fails. A `DELETE` pins the writer schema too, for its delete sink, but it writes no row values, so `RowLevelDmlCommand` marks its target and `DEFAULT(column)` on it fails with `DEFAULT(d) cannot be used in a DELETE: the defaults of this table apply only to the rows that a statement writes`, as in branch-4.2, whose DELETE installs no write schema context. Master reads NULL there, so `DELETE FROM t WHERE d = DEFAULT(d)` deletes nothing; resolved to the write-default it would delete every row with `d = 42`. - A resolved default is cast to the type of its writer column before the expression around it is bound, as `RewriteDefaultExpression` already does for a bound `DEFAULT(column)`. The parsed literal `-1` is a TINYINT, so without the cast `bit_count(DEFAULT(c))` on an INT column would count 8 bits instead of 32. The external `UPDATE`, `MERGE` and changelog paths use the same helper. `RewriteDefaultExpression` rejects a bound `DEFAULT(column)` whose writer column has another type than the read column the statement was bound with, which happens only when the table changed outside Doris after Doris cached it, and asks to refresh the table: the expressions around it were bound against the read type. - Hive, Paimon and internal tables do not declare the capability, so they behave as before. - The new enum constant changes the connector plugin API surface, so `connector.plugin.api.version` goes from 26.0 to 27.0, as for the other recent changes to that surface (#69299, #69588, #69298, #69683 and #69835 took 22.0 to 26.0). **Where `DEFAULT(column)` resolves on an Iceberg target (direct Iceberg catalogs).** | Statement | Position of `DEFAULT(column)` | Result | branch-4.2 | |---|---|---|---| | `INSERT` / `INSERT OVERWRITE` | `VALUES` rows | the target column's write-default, typed as the column | same | | | the INSERT's own select lists (with or without `FROM`, set operation branches, `SELECT * REPLACE`) | the target column's write-default, by name, before binding | same, except `* REPLACE`, which branch-4.2 binds to the source | | | `GROUP BY` / `ROLLUP` outputs, a derived table in `FROM`, `WITH` definitions, subqueries, `WHERE` / `ON` | bound like any `DEFAULT(column)` (last row) | same, except derived tables, which branch-4.2 resolves against the target | | | a `VALUES` branch of a set operation | fails with an unknown column, as on master | same | | | a qualifier other than the target (`DEFAULT(src.c)`, `DEFAULT(ctl.db.t.c)`) in the positions above | fails: `column in assignment list is invalid` | same | | `UPDATE` | `SET` / `WHERE` | write-default | same | | `MERGE` | `ON` and `WHEN ... AND` conditions, whatever the WHEN branches, also when they all delete | write-default | same | | | `UPDATE SET` / `INSERT VALUES` | write-default | same | | `DELETE` (also `DELETE ... USING`) | anywhere | fails: `cannot be used in a DELETE` | fails: its DELETE has no write schema context | | any | a bound `DEFAULT(x.c)` | `x` is the written table: its write-default by field id; an OLAP or Hive table: that column's default; another Iceberg table: fails with `must reference a column of the table that this statement writes` | same | A bound `DEFAULT(column)` whose writer column has another type than the read column fails and asks to refresh the table. Where `DEFAULT(column)` is bound rather than resolved by name (the third row), it keeps its meaning from before this PR and names a column of its own `FROM` clause, as on master and branch-4.2: for example in `INSERT INTO t SELECT id, DEFAULT(d) FROM src GROUP BY id, d` with an OLAP or Hive `src`, it takes the default of `src.d`. **Results.** - Local cluster, with the table above: the table shows what each statement did before and after. `EXPLAIN UPDATE t SET c = 5 WHERE d = DEFAULT(d)` goes from an empty set to a scan with `d = 42` pushed down to Iceberg. - Unit tests: 6 new tests in `InsertUtilsTest`, 6 in the new `RewriteDefaultExpressionTest` and 1 in `RowLevelDmlCommandTest`. They cover VALUES, and INSERT ... SELECT both from a staging relation whose own column is also `name` and without FROM: each resolves to the pinned default, and an unknown column fails. `DEFAULT(column)` inside a derived table, qualified or not and also under a UNION branch, is left to binding. A `SELECT * REPLACE(DEFAULT(name) AS name)` replacement resolves like the rest of the select list and keeps the star's qualifier and `EXCEPT` list. The pinned write column wins over the read column. A statement that writes nothing fails. A table without the capability is unchanged. With master's `InsertUtils` and `RewriteDefaultExpression`, the 5 tests of the fix fail and the 2 tests of the unchanged path pass. The `SELECT * REPLACE` test fails with this PR's previous `InsertUtils`, where the replacement stays `DEFAULT(name)`. `IcebergConnectorTest`, `HiveConnectorMetadataSiblingDelegationTest` and `ConnectorPluginSurfaceTest` pass. So do the fe-core tests around the write schema: `ConnectorWriteSchemaUtilsTest`, `ExternalRowLevelUpdatePlanBuilderTest`, `ExternalRowLevelMergePlanBuilderTest`, `ChangelogRowLevelDmlTransformTest` and `PluginDrivenExternalTableTest`. Two of the tests pin the typing: `bit_count(DEFAULT(bits))` gets the default as an INT in VALUES and in SELECT, and fails without the cast; a BIGINT writer column over an INT read column is rejected, directly and under an IF bound to INT. `DEFAULT(column)` on the target of a DELETE fails even with a pinned write-default, and only a DELETE marks its target (an UPDATE does not); skipping the check, or marking UPDATE instead of DELETE, makes exactly these two tests fail. `PositionDeleteRowLevelDmlTransformTest`, `PhysicalPlanTranslatorIcebergRowLevelDmlTest`, `DeleteFromCommandTest` and `UpdateCommandTest` pass. - Regression: `test_iceberg_initial_defaults` now writes rows 20 to 22 with `DEFAULT(default_int)` in VALUES and in INSERT ... SELECT with and without FROM. It then updates them with `WHERE default_int = DEFAULT(default_int)` and `MERGE ... ON ... = DEFAULT(...)`, for Parquet and ORC, and checks the rows in the `.out`. A plain `SELECT DEFAULT(default_int)` must fail, and so must a `DELETE ... WHERE default_int = DEFAULT(default_int)` on rows 20 to 22, which hold the write-default. The suite passed locally before the DELETE block was added, and the whole suite, DELETE block included, has since passed in two External regression runs on CI. With master's FE it fails at the first of these statements: `Unknown column 'default_int' in 'table list'`.

### What problem does this PR solve? Issue Number: None Related PR: apache#69298 Problem Summary: Cold Hive scans now plan file splits through the bounded streaming path. With hive.ignore_absent_partitions=false, a missing partition raises a UserException wrapped by the file cache. The streaming completion callback exposed only the outer cache message, losing the missing-partition reason previously shown by synchronous planning. Reuse one Hive scan error conversion in both paths to retain the table name, underlying reason and original cause. The missing-partition policy and error status are unchanged. ### Release note Preserve actionable Hive file-listing errors during streaming scan planning. ### Check List (For Author) - Test: Unit Test: the new cold-streaming initialization test reproduced the reported message loss before the fix; all 94 selected FE tests passed afterward, including synchronous parity, streaming, cancellation and delivery cases. Full build.sh --fe package build and Checkstyle passed. The external-service regression was not run locally and will be re-triggered on the updated PR. - Behavior changed: Yes, asynchronous Hive split-planning errors retain the underlying cause text and table context, matching the existing synchronous path. - Does this need documentation: No

### What problem does this PR solve? Issue Number: None Related PR: apache#69298 Problem Summary: The active-read cancellation test for external-file split planning waited for the source-close latch and then immediately required the producer completion counter to be one. SplitProducer closes the source before invoking its completion callback, so the test can legitimately observe zero completions and fail FE CI. Hold the test read active across cancellation, force a controlled close/completion gap, and wait for a callback-completed latch before checking terminal counts. This deterministically verifies exactly one close and one completion without depending on thread scheduling. ### Release note None ### Check List (For Author) - Test: Unit Test - Reproduced the exact CI assertion failure with a deterministic close barrier before fixing the completion wait. - All 62 related FE tests pass: SplitProducerTest, SplitAssignmentTest, SplitSourceTest, HiveStreamingSplitTest and FairTaskExecutorTest. - Full FE packaging and Checkstyle pass through build.sh --fe -j4 (UI build disabled). - Behavior changed: No (test synchronization only) - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298 Problem Summary: Statement cleanup can stop a Hudi split assignment before its filesystem-view owner publishes the sync thread. Reject an already-stopped owner before remote sync and preserve terminal lease cleanup. Iceberg streaming scans adapt historical identity partitions to the selected schema, but DATE-to-nanosecond timestamp conversion was missing and dropped promoted fields could lose equality deletes. Preserve physical partition types in delete manifests, convert temporal values exactly, and normalize dropped partition keys in query-local views. ### Release note Fix cancelled Hudi planning and Iceberg scans after DATE partition columns are promoted to nanosecond timestamps or removed. ### Check List (For Author) - Test: Unit Test: 339 related FE tests; full FE packaging and Checkstyle via build.sh --fe - Behavior changed: Yes; reject cancelled Hudi sync and preserve historical temporal partitions and equality deletes - Does this need documentation: No

…views ### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298 Problem Summary: Historical DATE identity partitions rebound to ordinary microsecond timestamps fail during lazy manifest planning, and dropped DATE/timestamp partition keys use incompatible units and Java representations that can omit equality deletes. Add exact DATE/microsecond conversions and a common lossless dropped temporal-key representation while rejecting overflow, lossy narrowing and unsupported zoned promotions. Adapted content-file views also lack Serializable, causing non-batch statement task reuse to fail and fall back to SDK planning. Make the static data/delete wrappers serializable without capturing query resources, and cover actual serialized reuse across equivalent relations. ### Release note Fix Iceberg scans and equality-delete matching after DATE partition evolution to microsecond timestamps, and preserve statement task reuse for adapted manifest files. ### Check List (For Author) - Test: Unit Test: 253 related FE tests; full FE packaging and Checkstyle via build.sh --fe - Behavior changed: Yes; read historical temporal partitions correctly and reuse serialized adapted scan tasks - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: Hive streaming batches may span several partitions and a single partition may span many batches. Using the outgoing batch size as a per-partition split estimate therefore undercounts large partitions and overcounts several small ones. Accumulate generated splits locally for the current selected partition, publish the existing global maximum at yields and partition boundaries, and reset only when advancing partitions. Keep lazy enumeration, the ten-split floor and int saturation. The three new external scan scheduler sizes also accepted nonpositive startup values, failing later in scheduler construction or the first query group. Add a pure positive-integer validator to ordinary and custom startup configuration loading while preserving empty defaults, immutability and the exclusion of runtime callbacks with side effects. ### Release note Correct Hive split estimates during streaming planning and reject nonpositive external scan scheduler sizes at startup. ### Check List (For Author) - Test: Unit tests and packaging - Reproduced both configuration load failures and both split-estimate failures before repair. - All 147 related FE tests passed: 13 fe-common and 134 fe-core tests. - Full FE packaging and Checkstyle passed; git diff --check passed. - No local complete P0 or external-service regression in this cycle. - Behavior changed: Yes. Partition estimates accumulate independently of outgoing batch sizes, and nonpositive scheduler sizes are rejected during startup before the invalid field is updated. Lazy enumeration and existing scheduling/cancellation behavior are preserved. - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: Cancelled Iceberg planning can retain an independent table-generation lease when task-group lookup throws before producer cleanup is registered. Resolve that executor before acquiring the lease. Hive and Iceberg also eagerly allocate a configured maximum batch capacity, so a large valid setting can fail with OOM even for empty work; share a bounded initial allocation while preserving the configured batch maximum. Hive streaming COUNT(*) decides whether to split one partition at a time, producing excessive ranges and footer reads when the selected scan already has enough files. Use the existing concurrent cancellable producers for bounded metadata lookahead, make the scan-wide decision after all lookahead workers finish, and reuse captured metadata before lazily listing further partitions. Preserve singleton listing, fair task admission, interruption, range backpressure and the eager COUNT policy. ### Release note Avoid excessive Hive COUNT(*) scan ranges, premature memory allocation for large split batch settings, and leaked Iceberg planning references after cancellation. ### Check List (For Author) - Test: Unit tests and packaging - Reproduced the Iceberg lease leak and Hive COUNT range regression before repair; isolated the allocation failure in a separate 64MB JDK process. - All 218 selected FE tests passed with no failures, errors or skips, using run-fe-ut.sh. - Covered both scanner versions, Parquet/ORC, metadata reuse without file caching, few-file splitting, maximum batch settings, metadata failure and lookahead cancellation. - Full FE packaging and Checkstyle passed using build.sh --fe -j4; git diff --check passed. - Complete P0 and external-service regression were not run locally in this cycle. - Behavior changed: Yes. Streaming COUNT(*) uses the selected scan's split decision, large batch limits allocate only for actual work, and failed admission acquires no independent generation reference. Scheduling and cancellation continue through existing producers. - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: Hudi batch planning registers its filesystem-view owner directly as a statement resource. Closing the statement after GetFlightInfo therefore stops SplitAssignment and cancels its producer before a deferred Arrow Flight query can fetch splits during DoGet. The existing scan-level deferred cleanup guard cannot protect this independently registered owner. Register a statement-only cleanup wrapper that captures the query executor and checks its deferred flag at close time, while keeping the actual owner as an unconditional SplitAssignment resource for coordinator finalization and query cancellation. Preserve duplicate registration failure cleanup, task accounting, interruption and exact terminal lease release. The captured executor remains valid even if connection cleanup clears its current executor reference. ### Release note Keep deferred Hudi Arrow Flight batch scans alive until query finalization or cancellation. ### Check List (For Author) - Test: Unit tests and packaging - Reproduced the premature assignment stop through real HudiScanNode.startSplit and StatementContext.close before repair. - Added deferred, ordinary and absent-executor controls; verified registered source lifetime, forced scan stop, queued producer cancellation and one lease release. - All 90 selected FE tests passed with no failures, errors or skips using run-fe-ut.sh. - Full FE packaging and Checkstyle passed using build.sh --fe -j4; git diff --check and final source hashes passed. - No live Arrow Flight/Hudi external-service regression was run locally in this cycle. - Behavior changed: Yes. Deferred Hudi scans survive statement cleanup, while non-deferred cleanup and explicit query cancellation still stop the batch owner. - Does this need documentation: No

…nning ### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: External metadata planning can monopolize shared workers and retain eagerly planned split graphs. Backport bounded lazy production and fair scheduling to branch-4.2, retaining generation ownership and previous review fixes. Integrate with the release branch's individual scan-range queues, per-backend cancellation, live empty-batch protocol, and Iceberg partition/cache versioning. Transfer Hudi partition admission and filesystem-view ownership through asynchronous delivery so queue backpressure neither blocks workers nor permits unbounded pending partitions. ### Release note Bound and fairly schedule external scan planning. ### Check List (For Author) - Test: 386 focused FE unit tests passed through run-fe-ut.sh; full FE packaging and integrated Checkstyle passed through build.sh --fe -j4. External-service regression was not rerun locally. - Behavior changed: Yes. External scan planning uses bounded lazy production and fair scheduling while preserving branch-4.2 delivery semantics. - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: HiveSplitEstimateTest still mocks the former addToQueue delivery path. The fair producer reads a zero batch size from that mock and emits no files, so four current-head FE tests fail before verifying their estimates. Use a real SplitAssignment and SplitSource with deterministic backend assignment, retain the existing file-count and estimate assertions, and stop the assignment during cleanup. ### Release note None ### Check List (For Author) - Test: Reproduced four failures locally; all 35 focused FE tests passed through run-fe-ut.sh. Full FE packaging and integrated Checkstyle passed through build.sh --fe -j4. - Behavior changed: No - Does this need documentation: No

### What problem does this PR solve? Issue Number: N/A Related PR: apache#69298, apache#69986 Problem Summary: The cold file-count regression requests EXPLAIN VERBOSE while asserting approximate streaming split counts. Branch-4.2 deliberately disables batch mode for VERBOSE so it can enumerate exact file details. Use normal EXPLAIN for this streaming-selection assertion, matching the suite's other streaming checks, and retain all expected data results. ### Release note None ### Check List (For Author) - Test: 21 focused Hive FE tests passed through run-fe-ut.sh, covering exact VERBOSE mode and cold streaming selection; git diff --check. External Hive regression is not available locally and must be verified by remote CI. - Behavior changed: No - Does this need documentation: No

### What problem does this PR solve? Issue Number: None Related PR: apache#69298 Problem Summary: External split production and Iceberg manifest reads moved from the registered scheduleExecutor to FairTaskExecutor instances constructed directly by ExternalMetaCacheMgr. The existing thread_pool metrics therefore no longer showed the migrated workers, queued work or progress. A production-entry unit test reproduced zero registered NotCheckpointExternalSplit pool-size metrics before the fix. Create these schedulers through ThreadPoolManager and include their existing counters in its startup metric registration, using distinct normal and checkpoint names. Latch-controlled real scheduler tests verify active, queued, completed and group values through MetricRepo. Scheduling and delivery behavior are unchanged. ### Release note External split and metadata schedulers now expose thread_pool metrics for worker count, active tasks, queued tasks, completed tasks and task groups. ### Check List (For Author) - Test: Unit Test and build - 45 focused FE tests passed with zero failures, errors or skips via run-fe-ut.sh; the new production-entry metric test failed before the fix and passes after it. - build.sh --fe -j4 including Checkstyle passed. - Behavior changed: Yes, export metrics for the replacement schedulers; query scheduling and scan delivery are unchanged. - Does this need documentation: No, existing thread_pool metric labels are reused.

…nal scan planning (#69298) (#69986) cherry pick from #69298

924060929 requested review from CalvinKirs and morrySnow as code owners October 19, 2026 17:41

github-actions Bot reviewed Oct 19, 2026

View reviewed changes

924060929 force-pushed the codex/lake-pr-split-scheduling branch from 7de569d to eadfc6f Compare October 19, 2026 19:32

github-actions Bot reviewed Oct 19, 2026

View reviewed changes

Comment thread fe/fe-core/src/main/java/org/apache/doris/connector/ConnectorSessionBuilder.java Resolved

924060929 marked this pull request as draft October 19, 2026 20:35

924060929 force-pushed the codex/lake-pr-split-scheduling branch from eadfc6f to 8b6086f Compare October 19, 2026 21:17

924060929 marked this pull request as ready for review October 19, 2026 21:17

github-actions Bot reviewed Oct 19, 2026

View reviewed changes

924060929 force-pushed the codex/lake-pr-split-scheduling branch from 8b6086f to ff44a87 Compare October 20, 2026 04:00

924060929 added dev/4.2.x and removed dev/4.2.x labels Oct 27, 2026

morrySnow approved these changes Oct 27, 2026

View reviewed changes

924060929 merged commit ba3a660 into apache:master Oct 27, 2026
39 of 40 checks passed

924060929 deleted the codex/lake-pr-split-scheduling branch October 27, 2026 09:22

github-actions Bot added the approved Indicates a PR has been approved by one committer. label Oct 27, 2026

github-actions Bot added the dev/4.2.x-conflict label Oct 27, 2026

This was referenced Oct 27, 2026

[fix](fe) Resolve DEFAULT(column) on Iceberg tables from the write schema #69488

Merged

[fix](fe) Stop leaking connector scope state after a statement ends #69718

Merged

924060929 mentioned this pull request Nov 2, 2026

branch-4.2: [improvement](datasource) Bound and fairly schedule external scan planning (#69298) #69986

Merged

yiguolei pushed a commit that referenced this pull request Nov 4, 2026

branch-4.2: [improvement](datasource) Bound and fairly schedule exter…

ef6ab82

…nal scan planning (#69298) (#69986) cherry pick from #69298

yiguolei added dev/4.2.0-merged and removed dev/4.2.x dev/4.2.x-conflict labels Nov 4, 2026

Conversation

924060929 commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Main benefits and independent SQL measurements

Release note

Check List (For Author)

Focused initial-delivery experiment

Iceberg first-delivery follow-up

Focused multi-backend small-batch follow-up

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hello-stephen commented Oct 19, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

924060929 commented Oct 19, 2026

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Oct 19, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 19, 2026

Uh oh!

hello-stephen commented Oct 20, 2026

FE Regression Coverage Report

Uh oh!

924060929 commented Oct 20, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

hello-stephen commented Oct 27, 2026

FE Regression Coverage Report

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

924060929 commented Oct 19, 2026 •

edited

Loading