Skip to content

evm: Leave the memory a write overwrites unzeroed - #1788

Open
chfast wants to merge 1 commit into
masterfrom
evm/grow-memory-writes
Open

chfast wants to merge 1 commit into
masterfrom
evm/grow-memory-writes

Conversation

@chfast

@chfast chfast commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

MSTORE and the copies (CALLDATACOPY, CODECOPY, EXTCODECOPY and RETURNDATACOPY) overwrite the memory they check right after it grows, so zeroing it first is wasted work. check_memory() now takes how the instruction overwrites the memory as a template parameter. MSTORE grows with grow_memory_for_store(), which leaves the new word unzeroed when the store starts at the memory end. A copy grows with grow_memory_for_copy(). It zeroes a one-word growth whole and sends any other to grow_memory_general_for_copy(), which zeroes only the gap below the first word the copy covers whole, and the last word. Reads, MSTORE8, MCOPY (its source may lie in the new memory) and the calls zero the whole growth as before. This resolves the TODO in grow_memory().

On 1000 recent mainnet blocks, run through zilkworm's evmone fork, the memory grew 16.0M times, and 64.6% of the 1,025M bytes zeroed for it were overwritten right after by the instruction that grew it. An MSTORE starting exactly at the memory end makes 42% of the growths. An MSTORE straddling the end makes 28%, but it covers only part of the new word. The copies make 5% of the growths and 30% of the zeroed bytes, nearly all of them overwritten.

Memory::grow() takes the words to leave unzeroed, within the capacity and past it. It sets the size first and zeroes the last word before the gap, so that nothing is needed after the memset of the gap. In grow_memory_for_store(), a store straddling the memory end and one starting at it take separate paths, so that the straddling store still compares new_size only once, as grow_memory() does. grow_memory_for_copy() takes gas_left last: as the first argument, it made GCC allocate the registers of the out-of-line returndatacopy() differently, with one more instruction on every execution.

Instructions per growth, from the growth call to its return with memset included, counted by single-stepping (x86-64, GCC 15 and Clang 21):

Growth GCC master GCC this PR Clang master Clang this PR
MSTORE at the memory end 23 20 23 19
MSTORE straddling the end 23 23 23 23
mstore(0x40, 0x80) on empty memory 65 62 61 55
MLOAD, one word 23 23 23 23
MLOAD, two words 58 53 54 47
copy of 20 bytes at the end 23 23 23 23
copy of 100 bytes straddling the end 65 43 61 39
copy of 600 bytes at the end 96 43 92 39
copy of 600 bytes, 72 past the end 103 70 99 63
copy of 3000 bytes to empty memory 229 43 225 39

Two kinds of growth take more instructions than on master. A copy within the last word of a growth by more than a word zeroes that word twice: a copy of 20 bytes at offset 1000 of empty memory takes 129 instead of 124 with GCC, and 122 instead of 120 with Clang. A growth by one word past the capacity takes 224–227 instructions instead of 219 with GCC and 217–218 instead of 215 with Clang, realloc() included. That happens once per doubling of the memory buffer, and a copy of more than a word past the capacity takes fewer: 225 instead of 500 for 5000 bytes with GCC.

On the paths that do not grow the memory, the handlers and the interpreter loops keep their instructions, except GCC's EXTCODECOPY, which gets 2–3 instructions shorter. GCC's three interpreter loops are unchanged, and Clang's differ only in the arguments of RETURNDATACOPY's growth call. In the Advanced interpreter, GCC's RETURNDATACOPY takes one more instruction on every execution, a register exchange at its return data bounds check, which no order of the growth call's arguments avoids. In callgrind, the evmone-bench cases take 0.22% (GCC) and 0.27% (Clang) fewer instructions in total, structarray_alloc 1.4% and 1.8% fewer. The EEST benchmark fixtures of the memory and copy instructions take 0.015% and 0.011% fewer.

In cycles, measured with perf on an AMD Zen 3 together with #1793 and #1796, against the master before them, the change stays within code-layout noise: with the interpreter loops at the same addresses in both builds, the main benchmarks change by -0.22% (GCC) and -0.13% (Clang) in geomean, while shifting the loops by a few hundred bytes alone changes single benchmarks by up to 2.4%, and one micro-benchmark by 7%.

On rv64im with the SP1 guest flags (GCC 15), an MSTORE at the memory end takes 17 instructions instead of 22, and a straddling one 22 as before. Growths by more than a word for the reads and stores take 39–40 instead of 44, plus the same memset. A copy at the memory end takes 41 instructions instead of 44 plus a memset of its size. A one-word copy takes 23 instead of 22, from gas_left going last. A copy within the last word of a longer growth takes 49 instead of 44, plus the same memset, and a one-word growth past the capacity 77–79 instead of 74, plus realloc() and memset.

@codspeed

codspeed Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 1 benchmark

1 warning

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 34 improved benchmarks
❌ 1 regressed benchmark
✅ 864 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ baseline/execute/main/swap_math/spent 37.2 µs 37.6 µs -1.1%
⚡ baseline/execute/micro/memory_grow_mstore/by32 596.9 µs 571.8 µs +4.4%
⚡ test_create[fork_Osaka-blockchain_test-max code size with non-zero data-opcode_CREATE-benchmark-gas-value_1M] 3.8 ms 3.7 ms +2.84%
⚡ baseline/execute/main/structarray_alloc/nfts_rank 4.6 ms 4.5 ms +2.75%
⚡ test_create[fork_Osaka-blockchain_test-max code size with non-zero data-opcode_CREATE2-benchmark-gas-value_1M] 4.5 ms 4.4 ms +2.49%
⚡ test_create[fork_Osaka-blockchain_test-0.75x max code size with non-zero data-opcode_CREATE-benchmark-gas-value_1M] 3.5 ms 3.4 ms +2.31%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-non_zero_topic-1_MiB_non_zero_data-opcode_LOG4-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.1%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-non_zero_topic-1_MiB_non_zero_data-opcode_LOG0-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.1%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-zeros_topic-1_MiB_non_zero_data-opcode_LOG0-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.1%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-zeros_topic-1_MiB_non_zero_data-opcode_LOG2-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.1%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_non_zero_data-opcode_LOG4-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.1%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-zeros_topic-1_MiB_non_zero_data-opcode_LOG3-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-non_zero_topic-1_MiB_non_zero_data-opcode_LOG1-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-zeros_topic-1_MiB_non_zero_data-opcode_LOG2-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_non_zero_data-opcode_LOG1-benchmark-gas-value_1M] 2.7 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-non_zero_topic-1_MiB_non_zero_data-opcode_LOG3-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_False-non_zero_topic-1_MiB_non_zero_data-opcode_LOG2-benchmark-gas-value_1M] 2.6 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_non_zero_data-opcode_LOG2-benchmark-gas-value_1M] 2.7 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_non_zero_data-opcode_LOG0-benchmark-gas-value_1M] 2.7 ms 2.6 ms +2.09%
⚡ test_log[fork_Osaka-blockchain_test-fixed_offset_True-non_zero_topic-1_MiB_non_zero_data-opcode_LOG3-benchmark-gas-value_1M] 2.7 ms 2.6 ms +2.09%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing evm/grow-memory-writes (fe8c553) with master (fc9264a)

Open in CodSpeed

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.25%. Comparing base (fc9264a) to head (fe8c553).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1788   +/-   ##
=======================================
  Coverage   98.24%   98.25%           
=======================================
  Files         186      186           
  Lines       17175    17264   +89     
  Branches     3921     3939   +18     
=======================================
+ Hits        16873    16962   +89     
  Misses        233      233           
  Partials       69       69           
Flag Coverage Δ
eest-develop-gmp 27.02% <50.46%> (+0.06%) ⬆️
eest-legacy 17.33% <45.79%> (+0.10%) ⬆️
eest-libsecp256k1 29.17% <50.46%> (+0.05%) ⬆️
eest-stable 84.26% <91.52%> (+0.02%) ⬆️
evmone-unittests 94.13% <99.06%> (+0.02%) ⬆️
riscv64 94.84% <99.06%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.65% <100.00%> (+0.02%) ⬆️
tooling 94.87% <ø> (ø)
tests 99.82% <100.00%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
lib/evmone/execution_state.hpp 95.65% <100.00%> (+0.49%) ⬆️
lib/evmone/instructions.hpp 100.00% <100.00%> (ø)
test/unittests/evm_memory_test.cpp 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chfast
chfast force-pushed the evm/grow-memory-writes branch 3 times, most recently from fbed073 to ae491d8 Compare October 28, 2026 18:13
@chfast
chfast marked this pull request as ready for review October 28, 2026 18:15
@chfast
chfast force-pushed the evm/grow-memory-writes branch 2 times, most recently from 6afb0d1 to f8fc9f8 Compare October 28, 2026 20:15
chfast added a commit that referenced this pull request Oct 29, 2026
`Memory::grow()` grows the capacity with `realloc()` only once per
doubling of the buffer, but that code sat inline in every growth. It
moves to `Memory::grow_capacity()`, out of line and under
`[[unlikely]]`, which then zeroes the extent with
`grow_within_capacity()`, as `grow()` does within the capacity.

This is split out of #1788, where `grow_within_capacity()` leaves the
words the instruction overwrites unzeroed. There, zeroing after a call
that returns to `grow()` would keep the range of those words across the
call: on x86 that costs every growth by more than a word 2–3 more
instructions with GCC and 5–8 with Clang. On master alone, that shape
would take as many instructions within the capacity and 5 fewer past it.

Instructions per growth, from the growth call to its return with memset
and `realloc()` included, counted by single-stepping (x86-64, GCC 15 and
Clang 21): a growth by more than a word, in `grow_memory_general()`,
takes 2 fewer with both compilers, e.g. 65 instead of 67 with GCC and 61
instead of 63 with Clang for `mstore(0x40, 0x80)` on empty memory. It no
longer loads the capacity and the data pointer for `realloc()` before
the capacity check, and it falls through to the memset. A growth past
the capacity takes 13 more: 219 instead of 206 with GCC and 215 instead
of 202 with Clang. The one-word growths in `grow_memory()` are
unchanged, and so is the rest of the library, but for the numbering of
Clang's constant-pool labels. In callgrind, the evmone-bench cases take
0.007% fewer instructions in total with either compiler, from the
general growths, and the EEST benchmark fixtures of the memory and copy
instructions change by less than 0.001%.

On rv64im with the SP1 guest flags, the growths within the capacity take
the same instructions as before with GCC 15 and Clang 21, and a growth
past the capacity takes 17–18 more.
@chfast
chfast force-pushed the evm/grow-memory-writes branch from f8fc9f8 to 7c0c17f Compare October 29, 2026 10:04
MSTORE and the copies (CALLDATACOPY, CODECOPY, EXTCODECOPY and
RETURNDATACOPY) overwrite the memory they check right after it grows.
check_memory() takes how the instruction overwrites it as a template
parameter, and the growth leaves the words overwritten whole unzeroed:
MSTORE grows with grow_memory_for_store(), which leaves the new word
unzeroed when the store starts at the memory end, and a copy growing the
memory by more than a word goes to grow_memory_general_for_copy(), which
zeroes only the gap up to the first word the copy covers whole, and the
last word. Any other growth is zeroed as before. Memory::grow() takes
the range of the words left unzeroed and sets the size before zeroing,
within the capacity and past it.

On 1000 mainnet blocks, 64.6% of the 1,025M bytes the growths zeroed
were overwritten right after by the instruction that grew the memory. An
MSTORE starting at the memory end makes 42% of the growths. The copies
make 5% but zero 30% of the bytes, nearly all of them overwritten.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant