[sambhav37]

What changes were proposed in this pull request?

In InMemoryCatalog.createPartitions, skip partitions that already exist when ignoreIfExists = true, instead of overwriting their metadata. Previously ignoreIfExists only disabled the duplicate check, and the subsequent loop still replaced every existing partition (including its location) with the new definition.

This aligns InMemoryCatalog with HiveExternalCatalog, which passes ifNotExists to the Hive metastore and leaves existing partitions untouched.

Why are the changes needed?

ALTER TABLE .. RECOVER PARTITIONS calls createPartitions(..., ignoreIfExists = true) for every partition directory it finds under the table root. With the in-memory catalog, after the table root is changed via ALTER TABLE .. SET LOCATION, recovering partitions silently repoints already-registered partitions to the new root, so queries return different data:

CREATE TABLE t (id INT, value STRING) USING PARQUET PARTITIONED BY (p STRING)
  LOCATION '/tmp/spark_t_old';
INSERT INTO t VALUES (1, 'old-a', 'a'), (2, 'old-b', 'b');

CREATE TABLE t_new (id INT, value STRING) USING PARQUET PARTITIONED BY (p STRING)
  LOCATION '/tmp/spark_t_new';
INSERT INTO t_new VALUES (3, 'new-b', 'b');

ALTER TABLE t SET LOCATION '/tmp/spark_t_new';
ALTER TABLE t RECOVER PARTITIONS;
SELECT id, value, p FROM t ORDER BY id;
-- 1  old-a  a
-- 3  new-b  b    <- partition p=b now points to /tmp/spark_t_new/p=b

Does this PR introduce any user-facing change?

Yes, it fixes a bug. With the in-memory catalog (spark.sql.catalogImplementation=in-memory), ALTER TABLE .. RECOVER PARTITIONS no longer changes the location of existing partitions. For the example above, the last query now returns:

1  old-a  a
2  old-b  b

This matches the behavior of the Hive catalog.

How was this patch tested?

Added new tests:

• ExternalCatalogSuite: "create partitions with ignoreIfExists does not alter existing partitions". It runs for both InMemoryCatalogSuite and HiveExternalCatalogSuite.
• v1 AlterTableRecoverPartitionsSuiteBase: "SPARK-60129: keep locations of existing partitions". It runs for the in-memory (sequential and parallel) and Hive v1 suites.

Both new tests fail without the fix and pass with it:

build/sbt 'catalyst/testOnly *InMemoryCatalogSuite'
build/sbt 'sql/testOnly org.apache.spark.sql.execution.command.v1.AlterTableRecoverPartitions*'
build/sbt 'hive/testOnly *HiveExternalCatalogSuite org.apache.spark.sql.hive.execution.command.AlterTableRecoverPartitionsSuite'

Was this patch authored or co-authored using generative AI tooling?

Co-authored: (claude-opus-5-5)