Skip to content

[Opt](compaction) Add time requirement conf to control whether to prune rows with delete sign=1 - #59441

Closed
bobhan1 wants to merge 1 commit into
apache:masterfrom
bobhan1:base-compaction-prune-del-sign-conf
Closed

bobhan1 wants to merge 1 commit into
apache:masterfrom
bobhan1:base-compaction-prune-del-sign-conf

Conversation

@bobhan1

@bobhan1 bobhan1 commented Dec 29, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas

Thearas commented Dec 29, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

Copy link
Copy Markdown
TPC-H: Total hot run time: 34356 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6abe583721045e23576307a2aa9fc2a49f236416, data reload: false

------ Round 1 ----------------------------------
q1	17589	4302	4051	4051
q2	2003	348	233	233
q3	10214	1279	735	735
q4	10222	949	323	323
q5	7726	2191	1887	1887
q6	211	168	136	136
q7	925	805	655	655
q8	9290	1421	1175	1175
q9	7061	5161	5196	5161
q10	6846	1816	1436	1436
q11	517	309	283	283
q12	724	751	590	590
q13	17794	3839	3034	3034
q14	304	290	279	279
q15	584	504	505	504
q16	716	676	639	639
q17	724	741	636	636
q18	7553	7438	7434	7434
q19	1111	972	597	597
q20	416	375	252	252
q21	4186	3944	3310	3310
q22	1051	1022	1006	1006
Total cold run time: 107767 ms
Total hot run time: 34356 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4057	4030	4029	4029
q2	341	389	310	310
q3	2109	2594	2240	2240
q4	1298	1731	1286	1286
q5	4052	4053	4905	4053
q6	233	176	131	131
q7	2047	1966	1751	1751
q8	2562	2353	2317	2317
q9	7200	7116	7204	7116
q10	2489	2701	2287	2287
q11	591	502	463	463
q12	739	775	687	687
q13	3814	4101	3229	3229
q14	305	339	291	291
q15	533	503	515	503
q16	691	694	670	670
q17	1151	1344	1433	1344
q18	8018	7680	7600	7600
q19	916	1013	912	912
q20	1950	2074	1878	1878
q21	4574	4235	4179	4179
q22	1127	1053	977	977
Total cold run time: 50797 ms
Total hot run time: 48253 ms

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 174200 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6abe583721045e23576307a2aa9fc2a49f236416, data reload: false

query5	4425	568	423	423
query6	328	226	208	208
query7	4199	451	260	260
query8	330	250	238	238
query9	8740	2575	2597	2575
query10	520	360	297	297
query11	15330	15203	14909	14909
query12	169	111	109	109
query13	1284	504	382	382
query14	5851	2984	2703	2703
query14_1	2619	2552	2578	2552
query15	207	194	177	177
query16	1005	473	378	378
query17	1101	696	589	589
query18	2423	439	351	351
query19	229	231	191	191
query20	124	119	115	115
query21	215	136	118	118
query22	4173	4127	4335	4127
query23	16183	15545	15388	15388
query23_1	15354	15504	15472	15472
query24	7463	1580	1229	1229
query24_1	1238	1194	1210	1194
query25	573	476	425	425
query26	1256	268	163	163
query27	2749	458	294	294
query28	4570	2174	2150	2150
query29	815	541	457	457
query30	307	238	215	215
query31	771	627	558	558
query32	75	70	65	65
query33	546	352	317	317
query34	895	868	515	515
query35	754	785	694	694
query36	878	857	831	831
query37	132	92	82	82
query38	2774	2705	2663	2663
query39	760	753	729	729
query39_1	705	695	741	695
query40	210	129	115	115
query41	66	61	61	61
query42	101	99	102	99
query43	470	500	408	408
query44	1316	743	742	742
query45	189	182	171	171
query46	849	956	599	599
query47	1369	1503	1453	1453
query48	307	331	253	253
query49	605	407	323	323
query50	630	278	206	206
query51	3731	3746	3781	3746
query52	103	106	101	101
query53	289	320	268	268
query54	279	258	248	248
query55	77	75	67	67
query56	278	289	280	280
query57	972	1038	983	983
query58	268	246	246	246
query59	2083	2140	1946	1946
query60	318	297	295	295
query61	165	173	154	154
query62	398	363	311	311
query63	294	260	272	260
query64	5070	1316	981	981
query65	3817	3736	3821	3736
query66	1434	431	300	300
query67	15318	15096	15376	15096
query68	7174	983	721	721
query69	504	344	303	303
query70	1066	961	921	921
query71	361	301	271	271
query72	6125	4738	4726	4726
query73	683	583	310	310
query74	8733	8776	8520	8520
query75	2883	2852	2501	2501
query76	3796	1052	660	660
query77	535	364	274	274
query78	9795	9880	9216	9216
query79	1506	914	595	595
query80	656	595	470	470
query81	500	264	226	226
query82	209	142	112	112
query83	255	249	228	228
query84	253	114	103	103
query85	904	554	441	441
query86	333	319	319	319
query87	2921	2847	2719	2719
query88	3201	2249	2284	2249
query89	386	365	349	349
query90	2019	152	157	152
query91	171	160	142	142
query92	66	66	61	61
query93	1101	936	561	561
query94	569	296	308	296
query95	557	379	294	294
query96	584	453	213	213
query97	2308	2380	2320	2320
query98	220	198	195	195
query99	594	563	469	469
Total cold run time: 252753 ms
Total hot run time: 174200 ms

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 26.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6abe583721045e23576307a2aa9fc2a49f236416, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.12	0.11
query5	0.26	0.26	0.25
query6	1.14	0.66	0.65
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.55	0.49	0.50
query10	0.54	0.56	0.54
query11	0.16	0.11	0.11
query12	0.16	0.13	0.12
query13	0.62	0.60	0.60
query14	0.98	0.99	0.99
query15	0.79	0.79	0.80
query16	0.39	0.40	0.41
query17	1.05	0.99	1.05
query18	0.23	0.21	0.21
query19	1.93	1.78	1.83
query20	0.01	0.01	0.02
query21	15.43	0.27	0.14
query22	4.90	0.05	0.05
query23	15.94	0.28	0.11
query24	2.14	0.94	0.22
query25	0.11	0.07	0.06
query26	0.15	0.13	0.13
query27	0.07	0.05	0.08
query28	5.07	1.05	0.87
query29	12.59	3.90	3.16
query30	0.27	0.13	0.14
query31	2.82	0.62	0.38
query32	3.23	0.56	0.47
query33	3.03	3.00	2.99
query34	16.56	5.08	4.44
query35	4.46	4.46	4.47
query36	0.64	0.51	0.48
query37	0.11	0.07	0.06
query38	0.07	0.04	0.03
query39	0.04	0.03	0.04
query40	0.16	0.14	0.12
query41	0.10	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 98.88 s
Total hot run time: 26.75 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 47.83% (11/23) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.41% (18966/35508)
Line Coverage 39.30% (176076/448086)
Region Coverage 33.83% (136140/402418)
Branch Coverage 34.78% (58838/169152)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 47.83% (11/23) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.30% (25096/34713)
Line Coverage 59.03% (263816/446894)
Region Coverage 53.85% (218927/406581)
Branch Coverage 55.47% (94133/169714)

@github-actions

Copy link
Copy Markdown
Contributor

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants