Skip to content

perf: find best-fit free chunk via a size-ordered index instead of a full scan - #320

Open
sargunv wants to merge 1 commit into
Traverse-Research:mainfrom
sargunv:free-list-size-index
Open

sargunv wants to merge 1 commit into
Traverse-Research:mainfrom
sargunv:free-list-size-index

Conversation

@sargunv

@sargunv sargunv commented Nov 16, 2026

Copy link
Copy Markdown

FreeListAllocator::allocate checks every free chunk in a block to find the best fit, so allocation slows down as a long-running app frees buffers. In one Vulkan map-rendering benchmark, about 1,300 free chunks made this scan 41% of our render thread's instructions.

This keeps free chunks in a BTreeSet<(size, chunk_id)> and walks up from the requested size. The first chunk that passes the existing alignment and granularity checks is the best fit.

free chunks before after speedup
1 80 ns 91 ns 0.9×
100 1.8 µs 167 ns 11×
1,000 17.1 µs 111 ns 154×
10,000 147 µs 109 ns 1,350×

Measured with a release build on Apple Silicon. One block was fragmented into N free chunks of 4–64 KiB; I timed allocate+free pairs and took the best of 5 runs.

USE_BEST_FIT is removed because first fit and best fit now pick the same chunk.

AI-assisted — Tool: Claude Code; model: anthropic/claude-opus-5-5

FreeListAllocator::allocate scanned every free chunk in the block on each
allocation. Keep free chunks in a BTreeSet ordered by (size, chunk id) and
walk up from the requested size, taking the first chunk that fits after
alignment and granularity padding. Ties between equal-size chunks now go to
the lowest chunk id instead of HashSet order.
@Jasper-Bekkers

Copy link
Copy Markdown
Member

Can you let claude build and report a simulation of various allocation patterns (mixes of big and small allocation such as constant buffers and render targets) and report both fragmentation as well as memory high water marks.

I suspect overall this is a good change but the remark about the 4-64KB chunks makes it so I want to investigate that a bit more.

@sargunv

sargunv commented Nov 16, 2026

Copy link
Copy Markdown
Author

Will do

the remark about the 4-64KB chunks makes it so I want to investigate that a bit more.

to clarify, that was outlining the method for measuring performance. it's not an outcome of the change. The old and new code pick the same chunk (except tie breaking between equal size chunks was previously random and is now the lower chunk id).

So fragmentation and peak memory use should match, and I'll show that with the simulation

@sargunv

sargunv commented Nov 16, 2026

Copy link
Copy Markdown
Author

Ran it: fragmentation and peak memory are unchanged across five simulated workloads (constant buffers, staging buffers, render targets up to 132 MiB, streamed textures, 64 KiB granularity), and allocation is 1.6–17× faster. That's expected, since both versions pick the same chunk; only the tie-break between equal-sized chunks changed. The 4–64 KiB range was just how the microbenchmark fragmented its block.

Simulator and raw results: https://gist.github.com/sargunv/96776b49e18aa454c426fdd4653a43b6

Details

Why the result matches. The old loop skipped chunks smaller than the request and kept the smallest one that passed the alignment and granularity checks. The new loop starts at the requested size and stops at the first chunk that passes, which is the same chunk. Ties between equal-sized chunks used to go by HashSet iteration order, which is randomly seeded per process; now they go by lowest chunk ID.

Setup. The simulator calls FreeListAllocator directly, with no GPU. Its block layer copies MemoryType::allocate/free: 256 MiB blocks, newest block tried first, a personal block for anything larger than a block, and empty blocks released except the last one. Each workload is a fixed, seeded trace of 3,000 frames:

  • frame-loop: 300 long-lived meshes (64 KiB–8 MiB), plus 20–60 constant buffers (256 B–64 KiB) and 2–6 staging buffers (64 KiB–4 MiB) per frame, each freed 3 frames later.
  • resize-storm: 10 render targets (up to 132 MiB, 64 KiB-aligned, non-linear) recreated at a random resolution between 720p and 4K every 50 frames. It alternates destroy-then-create with create-then-destroy, over lighter per-frame traffic.
  • texture-streaming: mip-mapped BC7/RGBA8 textures from 64² to 4096², streamed under a 600 MiB budget. Small buffers that live 500–3,000 frames are added in between and pin memory.
  • granularity-64k: linear buffers (256 B–1 MiB) mixed with non-linear images (64 KiB–16 MiB), with bufferImageGranularity set to 64 KiB.
  • random-churn: sizes from 256 B to 32 MiB (log-uniform), alignment of 256 B, 4 KiB or 64 KiB, mixed linear and non-linear, random lifetimes.

Each workload ran 50 times per version in a release build on Apple Silicon. Values are the median, with the range in parentheses when runs differed. "Stranded free" is the share of free bytes that sit outside each block's largest free chunk, sampled every 10 frames.

workload peak live peak capacity, old peak capacity, new stranded free, old stranded free, new ns/alloc, old ns/alloc, new
frame-loop 508 MiB 768 MiB 768 MiB 47.1% (44.7–48.1) 44.3% 645 146
resize-storm 842 MiB 1024 MiB 1024 MiB 8.0% 8.0% 134 81
texture-streaming 758 MiB 1536 MiB (1536–1792) 1536 MiB 50.4% (45.9–54.6) 49.1% 686 111
granularity-64k 4104 MiB 4864 MiB (4864–5120) 4864 MiB 68.5% (63.6–71.2) 67.9% 3019 178
random-churn 2783 MiB 3328 MiB 3328 MiB 54.9% (52.7–55.9) 53.0% 2938 362

Mean capacity over mean live memory and the number of blocks created also stay inside the old range. The old code produced a different layout on 43–50 of 50 runs in four workloads, and twice occasionally needed an extra 256 MiB block because of it. The new code gives the same layout every run and always matches the old best case.

One worst case is left. If many big-enough chunks fail the alignment or granularity check, the new loop still walks past all of them. That costs at most what the old full scan did on every allocation.

AI-assisted — Tool: Claude Code; model: anthropic/claude-opus-5-5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants