Skip to content

[refactor](binlog) Resolve before columns by unique ID - #67050

Closed
TsukiokaKogane wants to merge 1 commit into
apache:masterfrom
TsukiokaKogane:refactor_binlog_before_column
Closed

[refactor](binlog) Resolve before columns by unique ID#67050
TsukiokaKogane wants to merge 1 commit into
apache:masterfrom
TsukiokaKogane:refactor_binlog_before_column

Conversation

@TsukiokaKogane

@TsukiokaKogane TsukiokaKogane commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #65418

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@TsukiokaKogane

TsukiokaKogane commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 17376 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cbaa2c77f10addb7477a2ba0edeff8428e69a605, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17570	3063	3046	3046
q2	1882	241	158	158
q3	10285	889	513	513
q4	4658	251	205	205
q5	7674	618	390	390
q6	142	140	95	95
q7	547	515	409	409
q8	9242	949	898	898
q9	3480	2454	2467	2454
q10	6499	870	708	708
q11	445	270	239	239
q12	688	394	326	326
q13	17860	1554	1161	1161
q14	162	157	144	144
q15	q16	449	406	367	367
q17	821	815	782	782
q18	3154	2334	2273	2273
q19	2186	836	794	794
q20	671	526	499	499
q21	5673	1676	1881	1676
q22	337	265	239	239
Total cold run time: 94425 ms
Total hot run time: 17376 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3495	3420	3396	3396
q2	211	216	159	159
q3	2301	2379	2177	2177
q4	1221	1196	926	926
q5	2256	2176	2170	2170
q6	178	122	86	86
q7	1077	959	882	882
q8	1644	1470	1438	1438
q9	3191	3191	3163	3163
q10	1852	1804	1645	1645
q11	362	281	263	263
q12	469	445	344	344
q13	1500	1540	1182	1182
q14	172	170	175	170
q15	q16	394	399	367	367
q17	1066	1050	1046	1046
q18	4967	4424	4814	4424
q19	849	841	863	841
q20	967	935	839	839
q21	3947	3231	3312	3231
q22	409	356	330	330
Total cold run time: 32528 ms
Total hot run time: 29079 ms

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 84399 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cbaa2c77f10addb7477a2ba0edeff8428e69a605, data reload: false

query5	4284	420	338	338
query6	402	167	152	152
query7	4868	447	266	266
query8	292	131	120	120
query9	8683	3026	3032	3026
query10	392	263	223	223
query11	5373	1065	943	943
query12	113	71	71	71
query13	1191	444	331	331
query14	6081	2228	2131	2131
query14_1	2002	2003	2006	2003
query15	172	126	113	113
query16	927	404	380	380
query17	892	465	402	402
query18	2336	330	238	238
query19	158	140	115	115
query20	88	69	67	67
query21	209	117	99	99
query22	5481	5403	5451	5403
query23	6726	6177	6213	6177
query23_1	6064	6129	6036	6036
query24	7158	1090	768	768
query24_1	789	792	777	777
query25	419	282	251	251
query26	1262	281	162	162
query27	2708	470	294	294
query28	4546	1505	1508	1505
query29	923	441	347	347
query30	269	182	154	154
query31	854	428	353	353
query32	103	50	48	48
query33	469	210	174	174
query34	982	858	481	481
query35	416	397	338	338
query36	557	558	528	528
query37	119	80	67	67
query38	1017	865	802	802
query39	496	509	494	494
query39_1	484	458	478	458
query40	220	128	111	111
query41	53	52	51	51
query42	83	80	76	76
query43	244	247	212	212
query44	1032	544	545	544
query45	118	106	104	104
query46	785	831	518	518
query47	777	768	749	749
query48	304	312	226	226
query49	535	239	185	185
query50	782	318	257	257
query51	8307	8204	8362	8204
query52	87	77	74	74
query53	210	220	171	171
query54	238	211	189	189
query55	74	64	60	60
query56	265	235	252	235
query57	694	664	667	664
query58	252	205	216	205
query59	1276	1235	1129	1129
query60	291	220	213	213
query61	157	150	135	135
query62	352	212	178	178
query63	186	160	160	160
query64	2854	711	594	594
query65	1672	1653	1592	1592
query66	2026	313	277	277
query67	9857	9773	9962	9773
query68	2958	1225	754	754
query69	340	222	195	195
query70	681	635	617	617
query71	317	267	240	240
query72	2302	1809	1603	1603
query73	660	621	356	356
query74	1988	1230	1152	1152
query75	1246	1162	1018	1018
query76	2308	757	539	539
query77	256	249	206	206
query78	3920	3546	3142	3142
query79	2748	838	584	584
query80	1622	384	342	342
query81	515	200	175	175
query82	604	126	96	96
query83	316	260	240	240
query84	285	119	102	102
query85	879	457	378	378
query86	467	172	165	165
query87	1001	966	910	910
query88	2970	2121	2142	2121
query89	315	232	201	201
query90	2040	142	138	138
query91	150	143	130	130
query92	57	51	44	44
query93	1990	1238	796	796
query94	633	266	210	210
query95	609	364	410	364
query96	798	559	280	280
query97	1062	1050	1034	1034
query98	177	144	137	137
query99	428	363	305	305
Total cold run time: 180078 ms
Total hot run time: 84399 ms

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 16.62 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cbaa2c77f10addb7477a2ba0edeff8428e69a605, data reload: false

query1	0.00	0.01	0.00
query2	0.12	0.07	0.07
query3	0.32	0.21	0.21
query4	1.61	0.17	0.21
query5	0.26	0.25	0.25
query6	1.16	0.42	0.40
query7	0.04	0.01	0.01
query8	0.08	0.07	0.06
query9	0.42	0.29	0.29
query10	0.40	0.39	0.40
query11	0.29	0.16	0.16
query12	0.28	0.16	0.16
query13	0.38	0.39	0.37
query14	0.46	0.46	0.46
query15	0.49	0.42	0.41
query16	0.29	0.28	0.29
query17	0.69	0.65	0.66
query18	0.26	0.25	0.24
query19	1.23	1.12	1.14
query20	0.02	0.01	0.02
query21	15.39	0.29	0.25
query22	4.88	0.11	0.10
query23	15.93	0.40	0.24
query24	2.49	0.51	0.34
query25	0.13	0.07	0.07
query26	0.73	0.21	0.17
query27	0.08	0.08	0.07
query28	3.46	0.68	0.36
query29	12.43	3.34	2.67
query30	0.33	0.21	0.21
query31	2.74	0.41	0.20
query32	3.44	0.37	0.28
query33	1.47	1.38	1.41
query34	15.35	2.40	1.93
query35	1.90	1.92	1.86
query36	0.51	0.39	0.36
query37	0.09	0.07	0.07
query38	0.07	0.06	0.06
query39	0.05	0.05	0.04
query40	0.14	0.11	0.10
query41	0.10	0.06	0.06
query42	0.07	0.05	0.04
query43	0.05	0.05	0.05
Total cold run time: 90.63 s
Total hot run time: 16.62 s

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 78.32% (112/143) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 35.88% (61/170) 🎉
Increment coverage report
Complete coverage report

@morningman morningman mentioned this pull request Sep 1, 2026
Open
35 of 69 tasks
@morningman morningman closed this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei Awaiting requested review from yiguolei yiguolei is a code owner
@liaoxin01 liaoxin01 Awaiting requested review from liaoxin01 liaoxin01 is a code owner
@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner
@luwei16 luwei16 Awaiting requested review from luwei16 luwei16 is a code owner
@morrySnow morrySnow Awaiting requested review from morrySnow morrySnow is a code owner
@924060929 924060929 Awaiting requested review from 924060929 924060929 is a code owner
@englefly englefly Awaiting requested review from englefly englefly is a code owner
@starocean999 starocean999 Awaiting requested review from starocean999 starocean999 is a code owner
@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner
@CalvinKirs CalvinKirs Awaiting requested review from CalvinKirs CalvinKirs is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@TsukiokaKogane @hello-stephen @morningman