[morningman]

What problem does this PR solve?

Issue Number: None

Related PR: #66913, #66914

Problem Summary:

In short. This ports from branch-4.x #66913 the cleanup of an Iceberg HiveCatalog whose initialization fails after it has created its FileIO: on master, nothing closes that FileIO. With the FileIOs master configures, the leaked object holds no native resource today, so this is hygiene that keeps a FileIO with real resources from leaking once per failed catalog creation.

Background. DorisHiveCatalog (fe-connector-iceberg, used for Iceberg on HMS) extends Iceberg's HiveCatalog and owns the FileIO the catalog creates (#66914). In Iceberg 1.12, HiveCatalog#initialize creates the FileIO first, then the Hive client pool, and only then the CloseableGroup that closes the FileIO, the KMS client and the metrics reporter. #66913, the branch-4.1 port of #66914, closes the FileIO when initialization fails; master's DorisHiveCatalog does not.

The problem, and what it cost. When CachedClientPool fails (an unknown element in client-pool-cache-keys, a clients that is not a number) or the metrics reporter fails to load (metrics-reporter-impl), the catalog has a FileIO but no group: HiveCatalog#close closes nothing, and DorisHiveCatalog has not captured the FileIO yet. The catalog properties reach Iceberg as the user gave them, and IcebergConnector caches only a catalog that initialized, so each metadata access on such a catalog creates and drops another FileIO. Today that is only garbage: master's HMS catalogs use HadoopFileIO, whose close does nothing, and ResolvingFileIO and S3FileIO create their clients lazily.

How this PR fixes it. When HiveCatalog#initialize throws, DorisHiveCatalog#initialize closes the FileIO the catalog created and rethrows; a failure to close is added to the initialization failure as suppressed. The FileIO is null when loading the FileIO itself failed, and then there is nothing to close. Not ported from #66913: its close() changes, since Iceberg 1.12 (#69031) closes the FileIO in HiveCatalog#close itself, and its DorisHadoopCatalog. The hadoop flavor has the same order (HadoopCatalog creates the FileIO before the lock manager), but master uses Iceberg's HadoopCatalog, and a subclass would change the catalog-impl mapping without freeing anything today.

Results.

client-pool-cache-keys with an unknown element	Before	After
DorisHiveCatalog#initialize	throws IllegalArgumentException, the FileIO stays open	throws IllegalArgumentException, the FileIO is closed

Original author: @924060929

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

  New DorisHiveCatalogTest#closesFileIOWhenHiveClientPoolInitializationFails (from branch-4.1: [fix](metacache) release Iceberg FileIO and Hudi fsView on cache eviction #66914 #66913) has the catalog load a FileIO that counts its closes, through io-impl, and fail on an unknown cache key element: the FileIO is closed once. Without the change it is never closed (expected 1 but was 0). The other case of the class passes, and fe-connector-iceberg Checkstyle reports nothing.

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[morningman]

run buildall

[morningman]

/review

[github-actions]

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41768177444

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

[morningman]

/review

[hello-stephen]

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

[github-actions]

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41777729614

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

[hello-stephen]

TPC-H: Total hot run time: 28826 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 327978a44bc2fe8a973973831531196c598e435d, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17580	5153	5027	5027
q2	2176	309	258	258
q3	10232	1389	826	826
q4	4683	458	333	333
q5	7456	781	533	533
q6	181	183	149	149
q7	739	794	599	599
q8	9249	1332	1187	1187
q9	5521	4241	4165	4165
q10	6826	1295	1030	1030
q11	430	251	231	231
q12	637	403	280	280
q13	18089	3682	3057	3057
q14	277	278	254	254
q15	q16	734	707	640	640
q17	1205	751	583	583
q18	7040	6166	6110	6110
q19	1120	955	611	611
q20	389	340	235	235
q21	5546	2583	2426	2426
q22	378	333	292	292
Total cold run time: 100488 ms
Total hot run time: 28826 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	5950	5978	6090	5978
q2	755	607	586	586
q3	5668	5779	5257	5257
q4	2014	2142	1573	1573
q5	5886	5620	5723	5620
q6	239	189	142	142
q7	2218	1993	1828	1828
q8	2996	2655	2609	2609
q9	7684	7465	7452	7452
q10	3616	3571	3170	3170
q11	570	410	389	389
q12	676	664	504	504
q13	3315	3569	3075	3075
q14	286	291	261	261
q15	q16	656	664	608	608
q17	7747	6974	6911	6911
q18	12961	12187	12900	12187
q19	926	808	797	797
q20	2131	2154	1928	1928
q21	5607	4528	4662	4528
q22	527	467	420	420
Total cold run time: 72428 ms
Total hot run time: 65823 ms

[hello-stephen]

TPC-DS: Total hot run time: 151467 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 327978a44bc2fe8a973973831531196c598e435d, data reload: false

query5	4338	598	454	454
query6	427	216	215	215
query7	4817	444	234	234
query8	320	166	152	152
query9	8774	3211	3214	3211
query10	464	327	269	269
query11	5926	3861	3595	3595
query12	147	92	91	91
query13	1266	450	342	342
query14	6155	3984	3624	3624
query14_1	3456	3427	3378	3378
query15	211	195	180	180
query16	962	408	418	408
query17	882	659	542	542
query18	2430	422	352	352
query19	199	168	136	136
query20	93	87	83	83
query21	207	127	107	107
query22	13079	12961	12859	12859
query23	13691	12879	12412	12412
query23_1	12568	12540	12466	12466
query24	7377	862	474	474
query24_1	484	483	490	483
query25	543	430	369	369
query26	1256	261	141	141
query27	2754	442	280	280
query28	4604	2033	2038	2033
query29	1579	608	452	452
query30	315	222	190	190
query31	1013	943	821	821
query32	162	132	89	89
query33	511	295	234	234
query34	865	852	511	511
query35	760	762	638	638
query36	840	806	711	711
query37	120	93	88	88
query38	1788	1716	1632	1632
query39	795	795	740	740
query39_1	736	742	751	742
query40	219	110	96	96
query41	65	64	63	63
query42	91	85	82	82
query43	348	360	310	310
query44	1390	704	727	704
query45	184	179	166	166
query46	828	954	603	603
query47	2969	2881	2792	2792
query48	301	323	221	221
query49	582	393	291	291
query50	696	268	211	211
query51	10736	10578	10486	10486
query52	84	81	70	70
query53	189	209	151	151
query54	228	189	209	189
query55	72	66	61	61
query56	244	213	216	213
query57	1689	1636	1562	1562
query58	290	264	258	258
query59	2153	2176	2002	2002
query60	275	247	224	224
query61	151	142	150	142
query62	395	329	281	281
query63	191	161	157	157
query64	2755	939	779	779
query65	3335	3260	3299	3260
query66	1777	414	305	305
query67	20147	20301	19985	19985
query68	2975	1023	613	613
query69	393	282	257	257
query70	847	767	776	767
query71	304	228	215	215
query72	2729	2585	2318	2318
query73	521	531	301	301
query74	4640	4540	4331	4331
query75	2352	2252	1904	1904
query76	2316	1020	621	621
query77	355	352	272	272
query78	9745	9684	9140	9140
query79	1001	867	535	535
query80	529	430	394	394
query81	540	319	283	283
query82	750	133	106	106
query83	311	195	187	187
query84	315	125	104	104
query85	807	440	379	379
query86	321	236	228	228
query87	2045	1967	1809	1809
query88	3791	2769	2759	2759
query89	328	282	258	258
query90	1889	178	172	172
query91	162	153	126	126
query92	100	78	91	78
query93	1005	1002	586	586
query94	453	298	269	269
query95	577	393	300	300
query96	686	508	243	243
query97	2457	2436	2312	2312
query98	161	158	156	156
query99	773	764	648	648
Total cold run time: 235220 ms
Total hot run time: 151467 ms

[hello-stephen]

ClickBench: Total hot run time: 24.25 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 327978a44bc2fe8a973973831531196c598e435d, data reload: false

query1	0.01	0.00	0.01
query2	0.09	0.05	0.05
query3	0.26	0.09	0.08
query4	1.60	0.10	0.10
query5	0.23	0.24	0.24
query6	1.18	0.68	0.68
query7	0.04	0.01	0.01
query8	0.05	0.04	0.04
query9	0.40	0.35	0.32
query10	0.57	0.57	0.58
query11	0.19	0.14	0.13
query12	0.18	0.14	0.15
query13	0.48	0.48	0.47
query14	1.01	1.01	1.03
query15	0.63	0.60	0.60
query16	0.35	0.33	0.36
query17	1.13	1.16	1.09
query18	0.20	0.19	0.19
query19	2.13	2.00	2.00
query20	0.02	0.01	0.02
query21	15.51	0.18	0.15
query22	4.85	0.06	0.05
query23	16.27	0.33	0.12
query24	2.72	0.92	0.43
query25	0.11	0.04	0.08
query26	1.70	0.21	0.15
query27	0.04	0.05	0.04
query28	3.50	0.74	0.38
query29	12.52	4.13	3.22
query30	0.28	0.13	0.10
query31	2.77	0.56	0.33
query32	3.22	0.60	0.49
query33	3.22	3.20	3.27
query34	15.32	4.01	3.39
query35	3.31	3.31	3.35
query36	0.57	0.44	0.43
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.04	0.03
query40	0.18	0.15	0.15
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.05	0.04	0.03
Total cold run time: 97.19 s
Total hot run time: 24.25 s

[morningman]

/review

[github-actions]

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41795404090

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

[morningman]

run feut

[morningman]

run cloud_p0

[hello-stephen]

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

[morningman]

/review

[github-actions]

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41847083400

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

[morningman]

/review

[github-actions]

Codex automated review failed and did not complete.

Error: Selected model is at capacity. Please try a different model.
Workflow run: https://github.com/apache/doris/actions/runs/41986376873

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

[morningman]

Local pipeline review — ✅ PASS

schema: doris-repo-review/v1
status: PASS
pr: apache/doris#70164
commit: 327978a44bc2fe8a973973831531196c598e435d
base: c0274bfb4accd3f013b320704d55919f7ae46c2e
reviewed_at: 2026-11-06T09:00+08:00
reviewer: morningman
model: gpt-6.1-sol
effort: xhigh
findings: {blocker: 0, major: 0, minor: 0, nit: 0}
rounds: 1
converged: true

Notes for maintainers

None.

_{Reviewed locally with the doris-repo-review pipeline. Repository policy may accept this receipt for the matching commit; it is not a human Apache approval.}