Skip to content

Support column value replacement in SQL exports - #821

Open
adamziel wants to merge 1 commit into
trunkfrom
codex/sql-export-column-values
Open

adamziel wants to merge 1 commit into
trunkfrom
codex/sql-export-column-values

Conversation

@adamziel

Copy link
Copy Markdown
Collaborator

Allows a SQL dump to contain a replacement column value while leaving the stored source value unchanged.

The row reader accepts trusted column_read_expressions from server code. Normal reads, exact-row reloads, and oversized-value reads use the same expression. Text replacements keep the source column's character set so chunk boundaries do not split multibyte characters. Columns without a replacement keep their existing values. The HTTP endpoint does not accept these expressions from clients.

This is the first step of the Reprint exclusion stack. #817 uses it to remove Reprint from exported activation lists and adds the opt-in exclusion flag. #818 enables that flag by default for new pulls. This PR adds no Reprint exclusion behavior or CLI flags.

Testing

Export a table with a conditional column replacement and a small statement-size limit. Resume between SQL fragments, import the dump, and check that the selected value changed while the other row values and the source stayed unchanged. The regression test covers UTF-8, latin1, and binary values.

@github-actions

Copy link
Copy Markdown
Contributor

Pull pipeline performance — large-directory

Site: large-directory · 2,000+ plus targeted file-transfer scenarios files · 10,000 posts · 25,000 postmeta · PHP 8.5.10

No changes above 5%.

Changes of 5% or less (25)
Stage PR trunk Δ Status Details
playground-sqlite-db-pull 2.81 s 2.85 s ⚪ -39 ms (-1.4%) ✓
MetricsPR:
condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
trunk:
condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
playground-sqlite-db-apply 23.88 s 23.15 s ⚪ +736 ms (+3.2%) ✓
MetricsPR:
condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
trunk:
condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
url-rewrite-html 199 ms 197 ms ⚪ +1 ms (+0.7%) ✓
MetricsPR:
rows=128
input_MiB=0.29
MiB_per_second=1.44
sample_ms=200.1, 199.2, 198.4, 198.8, 198.1
peak_MiB=24
trunk:
rows=128
input_MiB=0.29
MiB_per_second=1.45
sample_ms=199.7, 197.8, 197.1, 197.2, 197.4
peak_MiB=24
url-rewrite-style-elements 46 ms 46 ms ⚪ +0 ms (+0.5%) ✓
MetricsPR:
rows=128
input_MiB=0.32
MiB_per_second=7.01
sample_ms=46.7, 46.6, 45.6, 45.9, 46
peak_MiB=12
trunk:
rows=128
input_MiB=0.32
MiB_per_second=7.04
sample_ms=46.6, 45.7, 45.7, 45.8, 45.9
peak_MiB=12
url-rewrite-style-large-value 765 ms 743 ms ⚪ +21 ms (+2.9%) ✓
MetricsPR:
rows=1
input_MiB=0.65
MiB_per_second=0.85
sample_ms=770.1, 764.4, 764.8, 764.9, 763.7
peak_MiB=10
trunk:
rows=1
input_MiB=0.65
MiB_per_second=0.87
sample_ms=745.9, 739.3, 743.4, 744.6, 741.8
peak_MiB=10
url-rewrite-blocks-literal-urls 69 ms 69 ms ⚪ +0 ms (+0.1%) ✓
MetricsPR:
rows=128
input_MiB=0.65
MiB_per_second=9.53
sample_ms=69.6, 68.6, 68.3, 68.6, 68.3
peak_MiB=24
trunk:
rows=128
input_MiB=0.65
MiB_per_second=9.55
sample_ms=69.5, 68.5, 68.3, 68.6, 68.5
peak_MiB=24
url-rewrite-blocks-nested-html 70 ms 70 ms ⚪ +0 ms (+0.2%) ✓
MetricsPR:
rows=128
input_MiB=0.82
MiB_per_second=11.79
sample_ms=70.4, 69.5, 69.7, 69.5, 69.6
peak_MiB=28
trunk:
rows=128
input_MiB=0.82
MiB_per_second=11.81
sample_ms=70.3, 69.1, 69.7, 69.5, 69.5
peak_MiB=28
url-rewrite-blocks-repeated-urls 70 ms 69 ms ⚪ +0 ms (+0.6%) ✓
MetricsPR:
rows=128
input_MiB=0.81
MiB_per_second=11.65
sample_ms=70.6, 69.6, 69.5, 69.4, 69.4
peak_MiB=28
trunk:
rows=128
input_MiB=0.81
MiB_per_second=11.72
sample_ms=70.1, 69.1, 69.1, 69.2, 69.1
peak_MiB=28
url-rewrite-blocks-escaped-quotes 302 ms 302 ms ⚪ -0 ms (-0.1%) ✓
MetricsPR:
rows=128
input_MiB=0.85
MiB_per_second=2.83
sample_ms=304.2, 301.7, 301.1, 301.2, 301.6
peak_MiB=40
trunk:
rows=128
input_MiB=0.85
MiB_per_second=2.82
sample_ms=303.6, 301.8, 302.1, 301.4, 301.8
peak_MiB=40
url-rewrite-blocks-encoded-shortcodes 504 ms 505 ms ⚪ -1 ms (-0.2%) ✓
MetricsPR:
rows=128
input_MiB=1
MiB_per_second=1.98
sample_ms=506.8, 504.6, 503.9, 504.3, 504
peak_MiB=54
trunk:
rows=128
input_MiB=1
MiB_per_second=1.98
sample_ms=506.2, 506.6, 505.1, 504.7, 504.9
peak_MiB=54
url-rewrite-blocks-no-source-urls 2 ms 2 ms ⚪ -0 ms (-0.5%) ✓
MetricsPR:
rows=128
input_MiB=0.67
MiB_per_second=376.85
sample_ms=2.4, 1.8, 1.7, 1.7, 1.9
peak_MiB=14
trunk:
rows=128
input_MiB=0.67
MiB_per_second=375.13
sample_ms=2.4, 1.8, 1.8, 1.7, 1.9
peak_MiB=14
url-rewrite-serialized-options 34 ms 34 ms ⚪ -0 ms (-1.2%) ✓
MetricsPR:
rows=128
input_MiB=0.38
MiB_per_second=11.19
sample_ms=34.9, 33.8, 34, 34, 34.2
peak_MiB=20
trunk:
rows=128
input_MiB=0.38
MiB_per_second=11.06
sample_ms=35.4, 34.6, 34.4, 34.3, 34.2
peak_MiB=20
url-rewrite-selected-site-html 109 ms 109 ms ⚪ +0 ms (+0.3%) ✓
MetricsPR:
rows=128
input_MiB=0.29
MiB_per_second=2.63
sample_ms=111.7, 109.6, 109, 109.1, 108.6
peak_MiB=24
trunk:
rows=128
input_MiB=0.29
MiB_per_second=2.64
sample_ms=110.4, 108.9, 108.3, 108.7, 108.8
peak_MiB=24
url-rewrite-selected-site-style-elements 185 ms 185 ms ⚪ +0 ms (+0.1%) ✓
MetricsPR:
rows=128
input_MiB=0.32
MiB_per_second=1.74
sample_ms=187.3, 185.7, 185.4, 185.3, 184.9
peak_MiB=24
trunk:
rows=128
input_MiB=0.32
MiB_per_second=1.74
sample_ms=187.7, 185.5, 185.2, 184.8, 185.1
peak_MiB=24
url-rewrite-selected-site-style-large-value 512 ms 511 ms ⚪ +1 ms (+0.3%) ✓
MetricsPR:
rows=1
input_MiB=0.65
MiB_per_second=1.26
sample_ms=528.8, 512.4, 513.8, 512.3, 511.2
peak_MiB=26
trunk:
rows=1
input_MiB=0.65
MiB_per_second=1.26
sample_ms=513.3, 511.1, 511.4, 510.1, 509.2
peak_MiB=26
url-rewrite-selected-site-blocks-literal-urls 84 ms 84 ms ⚪ +0 ms (+0.3%) ✓
MetricsPR:
rows=128
input_MiB=0.65
MiB_per_second=7.81
sample_ms=84.7, 83.7, 84.1, 83.8, 83.8
peak_MiB=30
trunk:
rows=128
input_MiB=0.65
MiB_per_second=7.83
sample_ms=84.8, 84.3, 83.6, 83.1, 83.4
peak_MiB=30
url-rewrite-selected-site-blocks-nested-html 212 ms 212 ms ⚪ +1 ms (+0.3%) ✓
MetricsPR:
rows=128
input_MiB=0.82
MiB_per_second=3.87
sample_ms=213.8, 212.3, 212.3, 212.5, 212.3
peak_MiB=40
trunk:
rows=128
input_MiB=0.82
MiB_per_second=3.88
sample_ms=212.8, 211.6, 211.6, 212, 211.7
peak_MiB=40
url-rewrite-selected-site-blocks-repeated-urls 159 ms 159 ms ⚪ -0 ms (-0.2%) ✓
MetricsPR:
rows=128
input_MiB=0.81
MiB_per_second=5.11
sample_ms=159.6, 158.3, 158.3, 158.6, 158.8
peak_MiB=28
trunk:
rows=128
input_MiB=0.81
MiB_per_second=5.1
sample_ms=159.6, 158.3, 159.2, 158.9, 159
peak_MiB=28
url-rewrite-selected-site-blocks-escaped-quotes 212 ms 213 ms ⚪ -2 ms (-0.8%) ✓
MetricsPR:
rows=128
input_MiB=0.85
MiB_per_second=4.02
sample_ms=213.4, 211.8, 211.8, 212.1, 211.7
peak_MiB=40
trunk:
rows=128
input_MiB=0.85
MiB_per_second=3.99
sample_ms=214.4, 213.5, 213.3, 213.2, 213.4
peak_MiB=40
url-rewrite-selected-site-blocks-encoded-shortcodes 401 ms 401 ms ⚪ +0 ms (+0.1%) ✓
MetricsPR:
rows=128
input_MiB=1
MiB_per_second=2.49
sample_ms=402.1, 399.4, 400.8, 401.1, 403.2
peak_MiB=54
trunk:
rows=128
input_MiB=1
MiB_per_second=2.49
sample_ms=402, 399.9, 399.4, 400.7, 400.7
peak_MiB=54
url-rewrite-selected-site-blocks-no-source-urls 2 ms 2 ms ⚪ -0 ms (-0.9%) ✓
MetricsPR:
rows=128
input_MiB=0.67
MiB_per_second=361.01
sample_ms=2.5, 1.9, 1.8, 1.8, 2
peak_MiB=14
trunk:
rows=128
input_MiB=0.67
MiB_per_second=357.59
sample_ms=2.5, 1.9, 1.8, 1.8, 2
peak_MiB=14
url-rewrite-selected-site-serialized-options 35 ms 35 ms ⚪ +0 ms (+0.7%) ✓
MetricsPR:
rows=128
input_MiB=0.38
MiB_per_second=10.87
sample_ms=35.8, 34.8, 35, 35.1, 35
peak_MiB=20
trunk:
rows=128
input_MiB=0.38
MiB_per_second=10.94
sample_ms=35.7, 34.7, 34.5, 34.8, 36.4
peak_MiB=20
url-rewrite-selected-site-child-paths-style-elements 188 ms 189 ms ⚪ -1 ms (-0.4%) ✓
MetricsPR:
rows=128
input_MiB=0.32
MiB_per_second=1.71
sample_ms=190.2, 188.2, 187.9, 187.8, 187.9
peak_MiB=24
trunk:
rows=128
input_MiB=0.32
MiB_per_second=1.71
sample_ms=190.4, 188.2, 188.2, 188.7, 188.6
peak_MiB=24
url-rewrite-selected-site-child-paths-blocks-nested-html 213 ms 215 ms ⚪ -2 ms (-1.0%) ✓
MetricsPR:
rows=128
input_MiB=0.82
MiB_per_second=3.86
sample_ms=213.6, 213, 212.8, 212.8, 212.8
peak_MiB=40
trunk:
rows=128
input_MiB=0.82
MiB_per_second=3.82
sample_ms=215.9, 215, 214.9, 216.5, 214.5
peak_MiB=40
url-rewrite-selected-site-child-paths-blocks-repeated-urls 159 ms 159 ms ⚪ +0 ms (+0.2%) ✓
MetricsPR:
rows=128
input_MiB=0.81
MiB_per_second=5.1
sample_ms=159.9, 158.6, 159, 158.9, 159.5
peak_MiB=28
trunk:
rows=128
input_MiB=0.81
MiB_per_second=5.11
sample_ms=159.6, 158.7, 158.6, 158.7, 158.7
peak_MiB=28

Total: PR 31.22 s · trunk 30.51 s · ⚪ +717 ms (+2.4%)

Numbers carry runner noise; treat single-run deltas as directional, not authoritative.

URL rows show the median of five samples, using the same inputs and PHP binary for both builds. Each sample starts with fresh caches and reuses one rewriter across 128 distinct values. Input generation and output checks are outside the timer. Every output must pass before a speed is reported.

📈 Trunk performance history — commit-by-commit timeline.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant