Skip to content

[Opt](mow) Forbid time_series compaction policy on unique table - #49905

Merged
zhannngchen merged 4 commits into
apache:masterfrom
bobhan1:mow-forbid-time-series
Apr 17, 2025
Merged

[Opt](mow) Forbid time_series compaction policy on unique table#49905
zhannngchen merged 4 commits into
apache:masterfrom
bobhan1:mow-forbid-time-series

Conversation

@bobhan1

@bobhan1 bobhan1 commented Apr 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1
bobhan1 force-pushed the mow-forbid-time-series branch 2 times, most recently from c2387c4 to fb79316 Compare April 9, 2025 08:33
@bobhan1

bobhan1 commented Apr 9, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@bobhan1
bobhan1 force-pushed the mow-forbid-time-series branch from 5c088fd to a35583e Compare April 9, 2025 08:57
@bobhan1

bobhan1 commented Apr 9, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@bobhan1 bobhan1 changed the title [Opt](mow) Forbid time_series compaction policy on mow table [Opt](mow) Forbid time_series compaction policy on unique table Apr 9, 2025
zhannngchen
zhannngchen previously approved these changes Apr 9, 2025
View reviewed changes

@zhannngchen zhannngchen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Apr 9, 2025
@github-actions

github-actions Bot commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@bobhan1

bobhan1 commented Apr 10, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Apr 10, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34017 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a35583ee6359a5e15003e0405d16676d37085346, data reload: false

------ Round 1 ----------------------------------
q1	25779	5092	5023	5023
q2	2056	287	184	184
q3	10389	1287	692	692
q4	10237	1027	526	526
q5	7557	2335	2363	2335
q6	191	165	134	134
q7	926	734	600	600
q8	9304	1285	1094	1094
q9	6849	5047	5078	5047
q10	6830	2304	1886	1886
q11	490	296	265	265
q12	349	355	235	235
q13	17777	3694	3096	3096
q14	226	217	211	211
q15	547	481	472	472
q16	643	604	583	583
q17	614	857	379	379
q18	7421	7093	6972	6972
q19	1508	942	574	574
q20	337	350	230	230
q21	4309	3294	2504	2504
q22	1050	1049	975	975
Total cold run time: 115389 ms
Total hot run time: 34017 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5155	5197	5154	5154
q2	252	346	237	237
q3	2227	2705	2344	2344
q4	1440	1832	1437	1437
q5	4541	4476	4407	4407
q6	215	168	128	128
q7	1985	1922	1745	1745
q8	2584	2583	2790	2583
q9	7297	7160	7205	7160
q10	2977	3174	2768	2768
q11	572	515	504	504
q12	701	759	601	601
q13	3486	3855	3378	3378
q14	281	294	268	268
q15	522	473	478	473
q16	668	690	654	654
q17	1147	1516	1413	1413
q18	7786	7565	7343	7343
q19	842	828	867	828
q20	1968	1926	1827	1827
q21	5277	4857	4727	4727
q22	1119	1057	1038	1038
Total cold run time: 53042 ms
Total hot run time: 51017 ms

@doris-robot

doris-robot commented Apr 10, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 193940 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a35583ee6359a5e15003e0405d16676d37085346, data reload: false

query1	1425	1064	1055	1055
query2	6175	1953	1956	1953
query3	11151	4626	4562	4562
query4	25391	23854	23341	23341
query5	5230	614	448	448
query6	293	196	193	193
query7	3985	492	283	283
query8	302	250	247	247
query9	8542	2577	2575	2575
query10	512	326	293	293
query11	15527	15010	14809	14809
query12	162	114	109	109
query13	1557	541	402	402
query14	8934	6208	6154	6154
query15	220	190	178	178
query16	7671	638	463	463
query17	1173	789	633	633
query18	2027	414	306	306
query19	195	188	165	165
query20	130	122	111	111
query21	207	125	109	109
query22	4562	4514	4297	4297
query23	34577	33817	33559	33559
query24	8626	2437	2430	2430
query25	535	469	400	400
query26	1229	279	157	157
query27	2867	509	335	335
query28	4942	2461	2426	2426
query29	713	559	465	465
query30	285	233	198	198
query31	892	881	869	869
query32	78	60	66	60
query33	567	371	324	324
query34	794	872	517	517
query35	815	869	749	749
query36	951	1002	910	910
query37	122	99	78	78
query38	4393	4225	4247	4225
query39	1548	1438	1488	1438
query40	221	123	120	120
query41	54	52	52	52
query42	115	105	111	105
query43	516	531	494	494
query44	1376	824	827	824
query45	176	177	168	168
query46	873	1025	647	647
query47	1842	1889	1836	1836
query48	387	431	305	305
query49	768	551	433	433
query50	685	709	411	411
query51	4317	4239	4232	4232
query52	107	107	99	99
query53	234	273	191	191
query54	606	595	511	511
query55	86	78	86	78
query56	314	307	304	304
query57	1185	1166	1071	1071
query58	269	268	283	268
query59	2811	2858	2774	2774
query60	326	325	334	325
query61	138	128	124	124
query62	764	738	687	687
query63	227	190	183	183
query64	3992	1243	839	839
query65	4428	4403	4383	4383
query66	966	479	307	307
query67	16113	15607	15309	15309
query68	9060	893	522	522
query69	510	303	275	275
query70	1213	1134	1126	1126
query71	473	316	299	299
query72	5565	4797	5033	4797
query73	754	667	351	351
query74	9241	9092	9068	9068
query75	4287	3214	2666	2666
query76	3831	1207	754	754
query77	814	379	285	285
query78	9977	10289	9301	9301
query79	4290	794	553	553
query80	686	527	441	441
query81	483	250	223	223
query82	369	123	95	95
query83	336	252	233	233
query84	294	102	88	88
query85	748	354	317	317
query86	345	321	276	276
query87	4441	4514	4406	4406
query88	2879	2228	2203	2203
query89	431	313	281	281
query90	2064	215	220	215
query91	145	142	110	110
query92	79	61	55	55
query93	2734	975	594	594
query94	693	420	321	321
query95	378	299	293	293
query96	496	555	270	270
query97	3110	3278	3130	3130
query98	225	227	196	196
query99	1421	1407	1290	1290
Total cold run time: 285573 ms
Total hot run time: 193940 ms

@doris-robot

doris-robot commented Apr 10, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.16 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a35583ee6359a5e15003e0405d16676d37085346, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.28	0.19	0.19
query4	1.59	0.19	0.19
query5	0.58	0.56	0.59
query6	1.19	0.72	0.71
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.51	0.50
query10	0.59	0.59	0.57
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.61
query14	2.81	2.69	2.80
query15	0.91	0.85	0.84
query16	0.38	0.38	0.38
query17	1.02	1.01	1.02
query18	0.21	0.20	0.19
query19	1.85	1.90	1.92
query20	0.01	0.01	0.02
query21	15.36	0.89	0.55
query22	0.76	1.17	0.70
query23	14.91	1.39	0.66
query24	7.65	0.77	0.73
query25	0.45	0.19	0.08
query26	0.71	0.16	0.13
query27	0.05	0.05	0.05
query28	9.26	0.90	0.46
query29	12.52	3.98	3.30
query30	0.26	0.08	0.06
query31	2.84	0.59	0.39
query32	3.24	0.56	0.48
query33	3.01	3.04	3.03
query34	15.85	5.14	4.48
query35	4.55	4.52	4.63
query36	0.66	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.02	0.02	0.02
query40	0.16	0.13	0.14
query41	0.09	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.66 s
Total hot run time: 31.16 s

@github-actions github-actions Bot removed the approved Indicates a PR has been approved by one committer. label Apr 10, 2025
@bobhan1

bobhan1 commented Apr 10, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@bobhan1
bobhan1 force-pushed the mow-forbid-time-series branch from 2d110b2 to 58e35be Compare April 10, 2025 09:21
@bobhan1

bobhan1 commented Apr 10, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@bobhan1
bobhan1 force-pushed the mow-forbid-time-series branch from 58e35be to 04ccdf5 Compare April 11, 2025 02:00
@doris-robot

doris-robot commented Apr 11, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04ccdf5c7932f06b30a8adf4c660e68b13fef148, data reload: false

query1	0.03	0.04	0.03
query2	0.11	0.11	0.10
query3	0.25	0.19	0.19
query4	1.59	0.19	0.19
query5	0.59	0.57	0.60
query6	1.20	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.52
query10	0.57	0.57	0.58
query11	0.16	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.61	0.60
query14	2.65	2.70	2.71
query15	0.93	0.85	0.84
query16	0.38	0.38	0.39
query17	0.99	1.06	1.03
query18	0.20	0.19	0.20
query19	1.91	1.90	1.86
query20	0.01	0.01	0.02
query21	15.35	0.92	0.57
query22	0.74	1.28	0.65
query23	14.89	1.41	0.63
query24	7.22	1.33	1.22
query25	0.47	0.13	0.07
query26	0.65	0.16	0.14
query27	0.05	0.05	0.05
query28	9.52	0.89	0.44
query29	12.54	3.96	3.26
query30	0.25	0.08	0.06
query31	2.82	0.60	0.38
query32	3.23	0.56	0.47
query33	3.07	3.06	3.09
query34	15.86	5.12	4.56
query35	4.52	4.52	4.53
query36	0.67	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.03	0.04	0.03
Total cold run time: 105.29 s
Total hot run time: 31.63 s

@zhannngchen
zhannngchen merged commit 44f093d into apache:master Apr 17, 2025
github-actions Bot pushed a commit that referenced this pull request Apr 17, 2025
@bobhan1 Your Name
[Opt](mow) Forbid time_series compaction policy on unique table (#49905)
3606bbf 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions Bot mentioned this pull request Apr 17, 2025
Merged
bobhan1 added a commit to bobhan1/doris that referenced this pull request Apr 17, 2025
@bobhan1
[Opt](mow) Forbid time_series compaction policy on unique table (apac…
a21695a 
…he#49905)

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@bobhan1 bobhan1 mentioned this pull request Apr 17, 2025
Merged
yiguolei pushed a commit that referenced this pull request Apr 18, 2025
@bobhan1
branch-2.1-pick: [Opt](mow) Forbid time_series compaction policy on u…
4146cbb 
…nique table (#49905) (#50132)

pick #49905
@bobhan1 bobhan1 mentioned this pull request Apr 19, 2025
Merged
16 tasks
dataroaring pushed a commit that referenced this pull request Apr 21, 2025
@bobhan1
[Fix](case) Fix case that use time series compaction on unique table (#…
2cca481 
…50194)

some cases in p1/p2 are not modified in
#49905
github-actions Bot pushed a commit that referenced this pull request Apr 21, 2025
@bobhan1 Your Name
[Fix](case) Fix case that use time series compaction on unique table (#…
f398d90 
…50194)

some cases in p1/p2 are not modified in
#49905
bobhan1 added a commit to bobhan1/doris that referenced this pull request Apr 21, 2025
@bobhan1
[Fix](case) Fix case that use time series compaction on unique table (a…
b949be4 
…pache#50194)

some cases in p1/p2 are not modified in
apache#49905
dataroaring pushed a commit that referenced this pull request Apr 25, 2025
@bobhan1 @dataroaring
[Opt](mow) Forbid time_series compaction policy on unique table (#49905)
18f43b0 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
dataroaring pushed a commit that referenced this pull request Apr 27, 2025
@github-actions @bobhan1
branch-3.0: [Opt](mow) Forbid time_series compaction policy on unique…
a6a9f99 
… table #49905 (#50128)

Cherry-picked from #49905

Co-authored-by: bobhan1 <baohan@selectdb.com>
@yiguolei yiguolei mentioned this pull request May 13, 2025
Closed
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@bobhan1 koarz
[Opt](mow) Forbid time_series compaction policy on unique table (apac…
93d1b43 
…he#49905)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [x] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [x] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@bobhan1 koarz
[Fix](case) Fix case that use time series compaction on unique table (a…
cb015f9 
…pache#50194)

some cases in p1/p2 are not modified in
apache#49905
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Closed
suxiaogang223 pushed a commit to suxiaogang223/doris that referenced this pull request Jul 15, 2025
@bobhan1
(selectdb-cloud-4.0.4): pick [Opt](mow) Forbid time_series compaction…
9d065fb 
… policy on unique table (apache#49905) (apache#3916)

pick apache#49905
Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Jul 21, 2025
@bobhan1 Chen Zhang
(selectdb-cloud-4.0.4): pick [Opt](mow) Forbid time_series compaction…
e807db9 
… policy on unique table (apache#49905) (apache#3916)

pick apache#49905
@coderabbitai coderabbitai Bot mentioned this pull request Nov 6, 2026
Open
4 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhannngchen zhannngchen zhannngchen approved these changes
@dataroaring dataroaring dataroaring approved these changes
@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner
@w41ter w41ter Awaiting requested review from w41ter

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.6-merged kind/behavior-changed p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@bobhan1 @hello-stephen @doris-robot @zhannngchen @dataroaring @yiguolei