[u70b3]

What problem does this PR solve?

Issue: #66497 (design v5.1, slice 3). Target: branch-4.1.

This is PR3C, building on #67201 (PR3A, DDL surface) and #67235 (PR3B, durable jobs), both already merged into branch-4.1. It adds the FE admission path for top-level Lance CREATE, CREATE OR REPLACE, and DROP INDEX, returns durable job IDs, and exposes job inspection SQL.

enable_lance_index_mutation remains false by default. This slice does not execute index mutations: admitted jobs remain PENDING until dispatch/worker support and FORCE_RELEASE arrive in later slices. Enabling it now leaves unresolved jobs that prevent target-changing ALTER CATALOG and DROP CATALOG.

Behavior

• Admission reads dataset version, schema fields, logical indexes, and physical index families from one opened latest Dataset. It does not call countRows() or getIndexStatistics(). Schema contract v1 preserves Arrow field identity, nullability, vector dimensions/element types, and relevant type parameters for later worker revalidation. For fixed-size lists the pinned SDK reconstructs no child field — the element type lives only in the manifest logical type and element nullability is synthesized as nullable (a manifest-format fact that also binds existing datasets) — so the contract sources element facts from the reconstructed Arrow view and is pinned by a real on-disk dataset fixture.
• Reserved names and ambiguous case-only collisions are rejected (reserved names fail before any metadata read). REPLACE and DROP persist the stored display name of a unique case-insensitive match. Column identities that are ambiguous under the table's case-insensitive lookup relation are rejected before anything is journaled. CREATE IF NOT EXISTS is a no-op only when algorithm, physical family, column (compared in the same path-segment representation on both sides), and exposed whitelist properties match — an omitted num_bits compares as the persisted 8; DROP IF EXISTS is a no-op when the authoritative name is absent.
• Metric comparison ignores case. Numeric properties use strict parsed-long equality for numeric primitives or integer strings; fractions, exponent notation, overflow, and malformed values fail closed. Unexposed properties are skipped, and num_partitions is not compared.
• An admitted statement returns one JobId row after the job and fence are durable; an IF no-op returns the same column with zero rows. The response works for direct and forwarded connections.
• SHOW LANCE INDEX JOBS [FROM [catalog.]db] [WHERE TableName = "t" [AND State = "PENDING"]] and SHOW LANCE INDEX JOB <id> authorize each persisted target. Orphans — including targets whose provider lookup fails (credential expiry, outage) and targets whose currently resolved dataset locator no longer matches the job's persisted locator after a legal identity change — require global ADMIN; unauthorized rows are omitted, and unauthorized/missing job IDs share error 5103, so provider failures never leak through the authorization boundary. Locators, credentials, properties JSON, and schema-contract contents are not exposed. WHERE values accept what the parser actually produces for string literals, including VarcharLiteral; the FROM database name is canonicalized through the selected catalog's resolution semantics.

Concurrency and configuration

Catalog identity is captured under the CatalogMgr lock before the remote snapshot read. After the read, admission rechecks catalog existence, identity properties, and a local identity-change generation under the same lock used by catalog DDL, then allocates the ID and creates the job. This closes both the guard-check/create race and A → B → A identity changes. IF no-ops also revalidate the target. Identity changes invalidate in-flight reads even when a tentative ALTER is rolled back; same-value rewrites with unchanged key spelling and credential-only changes remain allowed.

Metadata I/O runs outside the DDL lock. Final admission uses the lock order CatalogMgr → job manager → journal. DROP and changes to target-identity properties — lance.catalog.type, warehouse, namespace parent/delimiter/root_database, and the storage-routing endpoint/region keys the Lance property chain actually consumes — reject unresolved jobs; access keys and tokens stay rotatable. Renames and replay are not blocked by the unresolved-job guard.

The gate and unresolved-job quotas (table/catalog/global defaults 8/64/256) are mutable and masterOnly. ADMIN SET changes runtime values on the master; it does not persist them to the custom configuration file or automatically synchronize other FEs. Restart loads configured file values/defaults, while promotion uses the promoted FE's own configuration. The two static bounds (num_partitions 4096, num_sub_vectors 256) are masterOnly — validation runs on the master — so a single-FE ADMIN SET takes effect only there; use ADMIN SET ALL FRONTENDS CONFIG to update the cluster.

ADMIN SET callbacks validate positive quotas/bounds. File-loaded quotas are checked again before ID allocation; the job manager independently rejects non-positive limits. Gate-off admission uses error 5102.

Job SHOW commands use FORWARD_NO_SYNC: follower requests execute on the master, without waiting for follower journal replay. This does not make SHOW a follower-local stale read.

Validation

• FE reactor compilation passed (JDK 17, Maven 3.9.9).
• 262 tests passed across 16 focused FE test classes, with zero failures/errors/skips: configuration, catalog guard, admission, snapshots, index families, static validation, schema contracts, job queries, loader defenses, parser, command responses, and job authorization. Beyond the admission/concurrency cases (concurrent DROP and identity ALTER versus admission, A → B → A, failed ALTER rollback, replay with null old properties, target changes during IF no-ops, strict numeric parsing, real command-to-CatalogMgr integration), coverage includes WHERE predicate shapes through the real parser (LIKE, reversed literals, VarcharLiteral values, which the parser produces for regular string literals), reserved-name rejection without a snapshot read, half-orphan row hiding for non-ADMIN, gate-off persistence assertions, the exactly-one-operation precondition, multi-column stored indexes in the IF preflight, unparsable snapshot properties, the loader's physical-entry defenses (bounds, duplicate UUIDs recorded before the system-entry filter, system-entry filtering, ordering), malformed provider schema facts (negative field ids, non-positive dimensions), the catalog's identity epoch, failure sanitization and REST early rejection, storage-routing key guarding, SHOW locator revalidation and provider-failure non-disclosure, and a real on-disk Lance dataset fixture pinning the fixed-size-list contract semantics.
• JaCoCo new-code line coverage: 93% on the new classes (each ≥ 86%), 81% on the new lines of modified files, 90% overall. The remainder is the JNI dataset-open path exercised by the external regression suite.
• The full FE test suite (fe-common + fe-core + reactor dependencies, ~9.6k tests) ran to completion on both the pre-review and the review-fix heads: the only failures are this machine's documented timezone-, network- and native-library-related environmental tests — 38 failing methods across 15 classes, verified identical (same classes, same counts) by running the same classes on the merge base.
• Maven validate/Checkstyle passed with zero violations; git diff --check passed.
• Groovy compilation and a harness exercising the actual regression suite's original-value restoration and cleanup-failure propagation passed.
• The external MinIO/REST regression was not run locally; the corresponding Docker environment is not running. BE compilation was not run.

The external regression suite runs as nonConcurrent, captures the original master gate/quota values, and restores them even on failure. Cleanup failures are reported. Admitted PENDING jobs/catalogs remain durable in this slice; per-run names avoid same-name collisions, but repeated runs still consume the global unresolved-job quota.

Deferred

Dispatch, BE selection, worker execution, possible-live slot configuration, and the local/file operator assertion belong to subsequent dispatch/worker slices. FORCE_RELEASE/RESOLVE and retention GC belong to PR3E (#67754). The gate default remains unchanged. REST catalog mutations, ALTER TABLE ADD/DROP INDEX, and BUILD INDEX remain unsupported.

Release note

Experimental, default-disabled FE admission and job inspection for Lance index mutations; actual index execution remains deferred.

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[u70b3]

run buildall

[u70b3]

/review

[u70b3]

/review

[u70b3]

run buildall

[Gabriel39]

/review

[Gabriel39]

run buildall

[github-actions]

Requesting changes after a full three-round review of the authoritative 38-file diff at exact head dc54320b7501fe99e1f8a6fede3aeb16290b3f74 against verified base 14d3d957c02f79e5505879f9de60b3a4a2a05f1c.

The six P1 issues are: incomplete storage-routing identity can redirect an unresolved mutation; SHOW authorization can disclose provider failures and can authorize historical jobs against a different current dataset; the production mutation gate is enableable before the required isolation/possible-live/release boundary exists; the pinned Lance SDK cannot supply the primitive fixed-list contract as modeled; and case-distinct provider fields can resolve to and journal the wrong physical field.

The seven P2 issues cover system/user UUID collision handling, quoted top-level IF matching, database-name normalization in SHOW filtering, follower-local ANN admission limits, the implicit fixed num_bits=8 comparison, malformed numeric schema facts entering durable contracts, and irreversible regression-suite consumption of global unresolved-job quota. Inline comments contain the reproductions and narrow remediation requirements.

Critical checkpoints reviewed: parser and command routing, catalog target capture/ALTER/DROP guards, admission and IF semantics, job durability/quota/fence/replay behavior, SHOW authorization and confidentiality, pinned Java/JNI/Rust schema compatibility, configuration ownership, and unit/regression coverage. No additional user focus was supplied.

This is a static review only: the runner instructions prohibit local builds/tests. At submission time the GitHub-hosted style/license/dependency/formatter checks and reported BE/Cloud/coverage/performance checks pass (with expected skipped extension/third-party jobs); TeamCity COMPILE and FE UT remain pending.

[github-actions]

[P1] Keep this slice impossible to enable in production. The accepted v5.1 contract (sections 9.1 and 9.7) says this is disabled FE mutation control and mutation cannot be enabled until hard worker isolation, finite possible-live limits, and the remaining evidence gates exist. mutable=true (and ordinary fe.conf loading) instead permits admission now; the warning confirms every resulting job/fence is durable but unresolvable because dispatch and FORCE_RELEASE are absent. Please use a test-only/internal seam for admission coverage and expose an enableable runtime setting only with the complete safety boundary.

[u70b3]

Keeping mutable=true for now: this is the same seam the dispatch and FORCE_RELEASE slices use as the production gate (roadmap: #66497 (comment)), and the regression suite enables it via ADMIN SET. A hard lock can land together with FORCE_RELEASE instead of freezing the seam twice — see #67630 (comment).

[github-actions]

[P2] These admission limits should route to the admission owner. Lance mutation validation runs only after AlterTableCommand is forwarded to the master, but masterOnly=false lets ADMIN SET FRONTEND CONFIG on a follower succeed locally without changing what the master enforces. Mark both ANN bounds master-only (while retaining the explicit all-FE form when desired), or otherwise make the update cluster-wide, and test follower routing instead of locking in the false annotation.

[u70b3]

Fixed in 4fba663 — both ANN bounds are now masterOnly=true, matching where the validation actually runs.

[github-actions]

[P1] Include storage-routing properties in the target identity guard. s3.endpoint (and the OSS equivalent) is not a credential: changing it can make the same s3://bucket/path resolve on a different MinIO/S3 service. Jobs persist the URI but not the endpoint, and later reads/workers rebuild options from the current catalog, while this five-key fingerprint also misses the change during in-flight admission. Please fingerprint/guard the normalized effective non-secret routing identity (including every accepted endpoint alias, provider selector, and region-derived endpoint), while leaving access keys/tokens rotatable.

[u70b3]

Fixed in 4fba663 — the guard now fingerprints the 18 storage-routing keys the Lance property chain actually consumes (endpoint/region families across s3/oss/cos/obs/minio/gs/ozone); credentials stay rotatable.

[github-actions]

[P1] Reject ambiguous top-level field identities in this pinned snapshot before the IF preflight or contract selection. Lance accepts distinct A and a, but ExternalTable.getColumn() returns the first equalsIgnoreCase match; static validation and storedColumnName() can therefore select and journal the wrong field ID/type, while CREATE IF may falsely match before the builder runs. Detect collisions under the actual Doris lookup relation as well as the persisted normalization (ROOT-lowercasing alone misses some Unicode equalsIgnoreCase pairs), fail closed, and cover ASCII and quoted-Unicode cases.

[u70b3]

Fixed in de4daa4 — top-level field names colliding under the table's equalsIgnoreCase lookup relation now fail closed before the IF preflight and contract selection.

[github-actions]

[P2] Compare the same field-name representation on both sides. Logical metadata escapes even a top-level name as a path segment (for example, a field named a.b becomes a backtick-quoted path), while the parser strips identifier quoting before putting raw a.b in def.getCols(). Thus a valid CREATE INDEX IF NOT EXISTS is rejected as a different definition although table lookup and schema-contract construction resolve the same field. Compare field identity/raw stored names, or canonicalize the request with the same path formatter, and test dots, spaces, and embedded backticks.

[u70b3]

Fixed in de4daa4 — the request column is converted to the same loader path-segment representation before comparison, so a name the parser unquotes compares equal to its escaped logical form.

[github-actions]

[P2] Validate provider-supplied numeric schema facts before journaling this contract. The pinned manifest-read path reconstructs protobuf fields without calling Schema.validate(), so a corrupt or historical manifest can reach Java with field.getId() < 0 or a fixed-list size <= 0; neither this builder nor IndexedField.validateForAdmission() rejects them. Reject negative IDs (zero is valid) and non-positive dimensions here, map the failure to the bounded Lance admission error, and add manifest-shaped cases.

[u70b3]

Fixed in d192e4d — negative field ids and non-positive fixed-list dimensions are rejected before the contract is journaled.

[github-actions]

[P2] Apply the catalog's database-name semantics to this filter. Admission persists the resolved local database name, but FROM keeps the user's identifier spelling and this exact equality runs before resolution. With lower_case_database_names=1 or 2, SHOW ... FROM DB1 can return no rows even though the catalog resolves DB1 to stored db1. Resolve/canonicalize the requested database through the selected catalog without weakening authorization, and cover both case modes.

[u70b3]

Fixed in a53722f — the FROM database name is now resolved through the catalog's own name semantics before the equality filter.

[github-actions]

[P1] Authorize against the job's durable target, not just today's catalog/db/table names. After a terminal job releases the unresolved guard, a legal warehouse, namespace, or routing ALTER can repoint the same catalog id and names from locator A to dataset B. This helper then succeeds, so a user with SHOW on B can read A's historical job message, executor, and FORCE audit fields. Freshly resolve and revalidate the provider plus normalized effective target (including non-secret routing identity) against the job; treat mismatches or missing identity as ADMIN-only orphans while keeping credential rotation allowed.

[u70b3]

Fixed in a53722f — authorization additionally revalidates the job's persisted locator against the catalog's currently resolved dataset target; a mismatch degrades to the orphan/ADMIN-only path.

[github-actions]

[P1] Keep provider lookup failures inside the non-disclosing authorization boundary. getTableNullable() can initialize, list, or load external metadata and throw on expired credentials or an outage. For detail, that makes a valid job ID return a provider-specific error while a missing or unauthorized ID returns fixed 5103; for list, one bad target aborts every row. Treat lookup failure as an unresolved/orphan target through an exception-safe lookup boundary, so non-ADMIN callers learn nothing and ADMIN can still inspect the orphan.

[u70b3]

Fixed in a53722f — target resolution is exception-safe: any provider lookup failure counts as an unresolved target (fixed 5103 for detail, row omitted for list), so provider errors never leak through the authorization boundary.

[github-actions]

[P2] This test is not repeatable on the shared FE state it explicitly anticipates. A successful intended run leaves this catalog plus three permanently PENDING jobs (normal CREATE, quota-case CREATE, and DROP); the timestamp avoids name/fence collisions but every run still consumes the global unresolved quota. With the default 256 limit, 85 runs leave 255 entries and the next run cannot finish, with partial failures leaking sooner. Run this scenario only in disposable/resettable FE state or defer it until an authorized cleanup/release path exists.

[u70b3]

Not taking for this slice — the regression pipeline FE is per-run disposable, so CI runs don't accumulate against the global quota; the suite adopts RESOLVE-based cleanup when PR3E (#67754) lands — see #67630 (comment).

[hello-stephen]

FE UT Coverage Report

Increment line coverage 78.88% (631/800) 🎉
Increment coverage report
Complete coverage report

[u70b3]

Response plan per point:

P1 – gate mutability (Config): keeping mutable=true for now. This flag is the same control the dispatch and FORCE_RELEASE slices will exercise as the production gate (roadmap: #66497 (comment)). For this slice the boundary is default-off + EXPERIMENTAL + the admission-time positive-quota assertion, and the external regression suite enables it via ADMIN SET FRONTEND CONFIG. If a hard lock is still preferred, mutable=false can land together with the FORCE_RELEASE slice instead of freezing the seam twice.

P1 – storage-routing identity (CatalogMgr): agreed — the endpoint-family keys the Lance catalog property chain actually consumes will join the target identity guard; access keys and tokens stay rotatable.

P1 – SHOW authorization (both comments): agreed — target resolution becomes exception-safe (any provider lookup failure counts as an unresolved target, so non-ADMIN callers learn nothing and ADMIN still sees the orphan row), and authorization will additionally revalidate the job's persisted locator against the catalog's currently resolved dataset target.

P1 – fixed-list contract vs pinned SDK: verifying empirically against a real local dataset on the pinned 9.1.0-beta.3 before choosing between narrowing the contract facts and an SDK-side path; findings will follow here.

P2s: column case-collision fail-closed under the table-lookup relation, num_bits=8 as the request-side default in the IF comparison, same path-segment representation on both sides of the column comparison, UUID check before the system-entry filter, numeric schema-fact validation (negative ids, non-positive dimensions), FROM database-name canonicalization, and masterOnly for the two ANN bounds — all accepted, landing in the next push.

P2 – regression quota consumption: the regression pipeline FE is per-run disposable, so CI runs don't accumulate; the suite will adopt the RESOLVE-based cleanup as soon as PR3E (#67754) lands.

[u70b3]

P1 – fixed-list contract vs pinned SDK: findings and fix. Verified empirically on the pinned 9.1.0-beta.3 with a real on-disk dataset through the JNI path: getLanceSchema() returns empty children for every FixedSizeList variant (f32/f16 × element nullability) — the element type is folded into the manifest logical-type string (fixed_size_list:float:4). This is fixed-size-list-specific: struct and variable-size-list children round-trip losslessly, including not-null flags. Element nullability has no manifest slot at all; it is synthesized as nullable on every read path, and datasets written today cannot be backfilled by an SDK upgrade.

The contract now sources element facts from the reconstructed Arrow view (asArrowField()'s synthesized child): element type from the child's Arrow type, element nullability copied from the synthesized flag (true today; improves automatically if a future SDK preserves the fact), failing closed when the view yields no child. This is pinned by a real JNI dataset fixture — a schema-only dataset created via Dataset.create, reopened, and built through the admission snapshot path — alongside unit fixtures that now mirror the real empty-children shape.

Landed in 4f887ac, together with the fixture migration and a real-dataset contract test.

[u70b3]

/review

[u70b3]

run buildall

[hello-stephen]

FE UT Coverage Report

Increment line coverage 78.35% (666/850) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 40.09% (344/858) 🎉
Increment coverage report
Complete coverage report

[u70b3]

run feut

[u70b3]

run external

[u70b3]

run external

[u70b3]

run p0

[u70b3]

run external

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 50.35% (428/850) 🎉
Increment coverage report
Complete coverage report

[u70b3]

run buildall

[u70b3]

@Gabriel39 @zhangstar333 @yiguolei This is ready for another pass when you have time. It builds on #67201 and #67235 (both merged); the feedback from the review rounds is addressed, and CI is green across the board (FE UT, P0, cloud_p0, external regression — including the Lance index admission suite). Given the branch-4.1 window, another review round or an approval would help land this.

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 47.50% (428/901) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 47.50% (428/901) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 47.50% (428/901) 🎉
Increment coverage report
Complete coverage report

[u70b3]

/review

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 47.50% (428/901) 🎉
Increment coverage report
Complete coverage report

np

[github-actions]

PR approved by anyone and no changes requested.

[github-actions]

PR approved by at least one committer and no changes requested.