Repository navigation
[SPARK-57500][SQL] Fix wrong results and SQL injection in MySQL JDBC pushdown of string values containing backslashes - #56558
Open
jiwen624 wants to merge 6 commits into
Conversation
jiwen624
force-pushed
the
SPARK-mysql-backslash-literal-pushdown
branch
4 times, most recently
from
September 19, 2026 00:21
35839ad to
83fecd4
Compare
jiwen624
force-pushed
the
SPARK-mysql-backslash-literal-pushdown
branch
2 times, most recently
from
October 24, 2026 06:22
677c6f3 to
96dcebf
Compare
… literals in comparison/IN predicates
…lash as sql_mode-independent hex strings
…down works on TiDB
…tern access from MiMa
jiwen624
force-pushed
the
SPARK-mysql-backslash-literal-pushdown
branch
from
October 24, 2026 23:04
92e1c42 to
ddc790b
Compare
jiwen624
marked this pull request as ready for review
October 25, 2026 05:54
Contributor
Author
|
Hi @cloud-fan since you fixed the LIKE side of this in SPARK-57332, could you take a look? This extends the same fix to =/IN (where a \ silently drops rows, or lets a value inject SQL) using hex literals, so it holds regardless of sql_mode. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Extends the SPARK-57332 fix for MySQL backslash escaping from LIKE to all pushed-down string values, and makes it independent of
sql_mode. With this fixMySQLDialectpushes strings that contain\as hex literals (_utf8mb4 X'...'), which MySQL reads the same in everysql_mode. Escaping inside a quoted literal can't do that, because\is an escape character only whenNO_BACKSLASH_ESCAPESis off, and Spark does not know the mode when it compiles the filter.STARTS_WITH/ENDS_WITH/CONTAINSare now built from the raw pattern value, with!as MySQL's LIKE escape character, so the MySQL override ofescapeStringLiteralForLikePatternis no longer needed. Patterns that are not string literals are no longer pushed down.Not changed here (as follow-ups instead):
COMMENTaccepts only a quoted literal, so it has nosql_mode-independent form.V2ExpressionSQLBuilder.visitStartsWith/visitEndsWith/visitContainsstill strip the quotes from the rendered pattern, which affects non-JDBC DSv2 sources.Why are the changes needed?
MySQL reads
\in a pushed literal as an escape:df.filter($"path" === "C:\\temp")pushes'C:\temp', which MySQL reads asC:<TAB>emp, so the row is silently not returned.df.filter($"c" === "x\\' OR 1=1) #")closes the literal early and runsOR 1=1, returning every row. An application that passes untrusted input into such a filter lets that input run SQL in the pushedWHEREclause.Also, on all dialects,
startswith(a, b)with a column pattern was pushed asa LIKE 'b%'and returned no rows.Does this PR introduce any user-facing change?
Yes. These filters now return correct results, and string values can no longer inject SQL. Known limitation: on MySQL/MariaDB with a non-default connection collation, a comparison with no column operand against a value containing
\can fail withIllegal mix of collations. Comparisons against columns are unaffected.How was this patch tested?
New tests in
JDBCSuite,JDBCV2SuiteandMySQLIntegrationSuite(bothsql_modes, MySQL and MariaDB drivers); all fail without the fix. Also verified manually on TiDB v8.5.1 and MariaDB 12.2.2.Was this patch authored or co-authored using generative AI tooling?
Yes.