Skip to content

Enable shared NVCC PCH caching for Modal submissions - #534

Merged
msaroufim merged 4 commits into
mainfrom
feat/nvcc-pch-cache
Nov 15, 2026
Merged

msaroufim merged 4 commits into
mainfrom
feat/nvcc-pch-cache

Conversation

@msaroufim

@msaroufim msaroufim commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Enable NVCC PCH by default for compatible load_inline submissions. Warm 17 profiles from five observed flag combinations for Torch/CUDA headers (excluding CUTLASS), then mount the shared cache read-only. Misses compile normally; KERNELBOT_NVCC_PCH=0 disables it.

CPU compile medians: two fresh 4-CPU/8-GiB workers per mode, same NVCC 13.4.99 and host PCH. Times include import, compilation, dependency capture, and artifact packing.

Workload / CUDA flags PCH off PCH on Speedup
T4 vector add / default 55.18s 40.79s 1.35×
T4 vector add / -O3 57.27s 50.30s 1.14×
T4 vector add / -O3 --use_fast_math 58.59s 35.01s 1.67×
T4 CUTLASS GEMM / -O3 48.11s 48.97s 0.98×
B200 vector add / -O3 --use_fast_math -arch=sm_100a 82.41s 52.16s 1.58×

All 20 timed runs passed correctness and GPU artifact reuse. CUTLASS showed no clear benefit. Measurements predate the final catalog/metadata fixes; compiler, toolkit, and framework headers are unchanged.

CI passes. Overall, 33 instrumented cases passed, including five B200 cases. Final canary passed on T4/A100/H100; B200 confirmed CPU PCH consumption but timed out awaiting GPU capacity. L4 was compile/warm tested only. Excluded a C++17 profile that also fails on stock CUDA without PCH. Benchmark harnesses and raw data stay outside this patch.

Thanks to Michael Buch at NVIDIA for the compiler support and PCH guidance.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/libkernelbot
  inline_artifacts.py 36, 52, 68
  python_precompile.py
  utils.py
Project Total  

This report was generated by python-coverage-comment-action

@msaroufim msaroufim changed the title Add an opt-in NVCC framework PCH cache for Modal submissions Enable shared NVCC PCH caching for Modal submissions Nov 15, 2026
@msaroufim
msaroufim marked this pull request as ready for review November 15, 2026 23:06
@msaroufim
msaroufim merged commit 7f720fd into main Nov 15, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant