Repository navigation
coverage: Switch over to hybrid HIR-aware coverage spans - #161517
Conversation
|
Some changes occurred in match lowering cc @Nadrieril Some changes occurred to MIR optimizations cc @rust-lang/wg-mir-opt Some changes occurred in match checking cc @Nadrieril |
|
r? @BoxyUwU rustbot has assigned @BoxyUwU. Use Why was this reviewer chosen?The reviewer was selected based on:
|
34344b6 to
8f05f77
Compare
This comment has been minimized.
This comment has been minimized.
This comment was marked as resolved.
This comment was marked as resolved.
8f05f77 to
7b6d193
Compare
This comment has been minimized.
This comment has been minimized.
7b6d193 to
828d5b7
Compare
|
This PR changes MIR cc @oli-obk, @RalfJung, @JakobDegen, @vakaras |
828d5b7 to
1cb3598
Compare
This comment has been minimized.
This comment has been minimized.
|
After some more investigation I found #145569 (comment), which suggests that for That might avoid the need to store a separate HirId in |
1cb3598 to
5dd6297
Compare
This comment has been minimized.
This comment has been minimized.
632cf3b to
a3fa4da
Compare
|
I dont really know much about how coverage is implemented and dont have the time to do so. r? @davidtwco as he's doing other coverage stuff atm afaik |
This comment has been minimized.
This comment has been minimized.
a3fa4da to
b2aca25
Compare
This comment has been minimized.
This comment has been minimized.
b2aca25 to
ee6b0a2
Compare
|
I'll land them separately, so that it's easier to look back at this PR if something goes wrong. |
This variant will be used for marker statements that are injected when a HIR expression is about to be lowered to MIR, and at other points relevant to coverage instrumentation. Adding this variant in its own commit keeps the mundane plumbing steps separate from later changes that will perform injection and analysis.
In order to make coverage instrumentation HIR-aware, we need to have MIR building inject marker statements at key locations, so that instrumentation can accurately reconstruct the relationship between HIR constructs and MIR-level control flow. The most important of these is `PointKind::Expr`, which associates a HIR expression with the start of the MIR that will evaluate that expression. There isn't a single convenient place to inject these, but we can get close by observing that THIR building wraps every expression in a `thir::ExprKind::Scope` node, which also contains the `HirId` of that expression. If we inject a marker whenever `thir::ExprKind::Scope` is lowered, that results in an accurate marker for every expression. Other markers are also injected at key points that don't correspond directly to a HIR node, to avoid major regressions in coverage-map quality.
Instead of trying to heuristically recover source-code spans from MIR soup, we can now take advantage of the `CoverageKind::Point` statements injected for specific HIR expressions. This is a transitional implementation that lets us remove the old MIR-soup code, but still relies heavily on the existing span-refinement heuristics, and does not take advantage of the full possibilities of HIR-aware coverage instrumentation. In order to avoid annoying never-executed spans in code that contains assertions, a heuristic has been added that ignores any expression within the arguments of a macro-expanded function call that returns `!`. Some coverage tests now require `//@ min-llvm-version: 23`, due to changes in how LLVM sorts the coverage mapping entries.
Now that we have a span for every HIR expression, we don't need these markers.
8ba456b to
de8e5d7
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
Rollup merge of #161517 - Zalathar:coverage-point, r=davidtwco coverage: Switch over to hybrid HIR-aware coverage spans --- One of the deep flaws in coverage instrumentation at the moment is that the instrumentor doesn't have actual knowledge of source-level code structure. Instead, it resorts to guessing source spans based on scanning through MIR statements and terminators, and applying a few crude heuristics that can never give truly accurate results. (The decision to use MIR-based spans was made during the original implementation of `-Zinstrument-coverage`. I was not able to find any discussion of this design decision or its consequences in the stabilization thread at #90132.) This PR is a partial step towards a better way of doing things. During THIR-to-MIR lowering, we now inject special `CoverageKind::Point` marker statements that represent a specific connection between some part of the source code (in HIR form) and a specific point in MIR control-flow. Instead of trying to recover coverage spans from arbitrary MIR statements/terminators, we now only extract spans from those dedicated marker statements. Having accurate HIR-to-MIR correlations unlocks a lot of exciting possibilities for better coverage instrumentation. This PR mostly sets the stage for potential future improvements, but does demonstrate the ability to exclude specific HIR expressions from the set of coverage spans (to make assertions less noisy), which was previously unthinkable. --- The key to making this all work is being able to accurately inject marker statements during MIR building. There is no single point in MIR building that all THIR expressions pass through, and most THIR expression nodes do not keep track of their corresponding HirId. However, it turns out that THIR building already wraps every expression node in a `thir::ExprKind::Scope` node that *does* store the original HirId, and the number of places in MIR building that handle those scopes is relatively small. So if we simply inject an appropriate marker statement at those places, with the HirId that is already available, that takes care of marking every HIR-level expression node in the built MIR. To avoid major regressions in coverage-span quality, we also need to inject markers in a few extra places that don't correspond to a HIR node, because they represent implied code such as the `else {}` of a one-sided `if`, or the automatic `return` at the end of a function body. --- There are necessarily some changes to coverage output as a result of the new approach. I have tried to keep these minimal, but I don't think it's worth trying to chase 1:1 compatibility with the old spans, which were very much a product of several layers of implementation details and heuristics. The differences seem reasonable overall, and in some cases are a clear improvement.
rust-lang/rust#161517 (part of 2026-09-07) changed the coverage output slightly, this accepts both versions.
rust-lang/rust#161517 (part of 2026-09-07) changed the coverage output slightly, this accepts both versions.
rust-lang/rust#161517 (part of 2026-09-07) changed the coverage output slightly, this accepts both versions.
The results changed because of rust-lang/rust#161517
rust-lang/rust#161517 (part of 2026-09-07) changed the coverage output slightly, this accepts both versions.
…rgo-llvm-cov 0.9.1, and the Rust line floor drops from 93 to 91, since 1.100.0 no longer counts a method name as covering its line (#243) `rust-toolchain.toml` pins `channel = "1.100.0"` instead of `stable`, with the same components and target. CI's `test` job installs cargo-llvm-cov 0.9.1 instead of 0.8.7, the same `x86_64-unknown-linux-musl` tarball, pinned by its sha256 `3fca950394a3c49457657c158b1619cec8dfd2647ae5b48746734c0ab969a522`. The Rust aggregate line floor goes from 93 to 91 in `--fail-under-lines` in `ci.yml` and `scripts/check-all.sh`, whose six coverage lines still hash identically once their prefixes are stripped, and in the value `scripts/coverage-floor.mjs` mirrors. **Why.** CI ran whatever `stable` was. Under Rust 1.100.0 the same commit, `main` at 28b18b6, with the same tests and commands, reads 29822 lines, 27519 covered, 92.28%, against 31158 lines, 29359 covered, 94.23% on 1.99.1, so every pull request failed `--fail-under-lines 93`. The cause is upstream, "coverage: Switch over to hybrid HIR-aware coverage spans" (rust-lang/rust#161517, milestone 1.100.0): a method name no longer gets a coverage region of its own. So `.map_err(|e| format!("reading work {work_id}: {e}"))?;` at `crates/taiyaku-library/src/librarydoc.rs:310`, which ran 609 times, is credited only with its closure's and its `?`'s regions, both 0, and reads uncovered. Crediting such lines back gives 94.21%: the counting changed, not the testing. **The owner's rulings.** The floor is 91, which keeps about the margin 93 had under 1.99.1. The toolchain is pinned to 1.100.0. CI moves to cargo-llvm-cov 0.9.1, the version the local gate runs. **Bumping the toolchain** is now a pull request of its own: `docs/ci.md` says to measure the same commit on both toolchains, then move the pin, and re-set the floor only if the counting changed. **Docs.** `docs/ci.md` gains "Why the Rust floor is 91, and the toolchain is pinned" under the coverage gates, with the two measurements, the upstream change and its example line, the 94.21% figure, the rulings and how a bump is done. Its present-tense statements of the floor and the toolchain now say 91 and pinned: the coverage gates table, "The numbers, and where they came from", whose Rust row now carries the 1.100.0 reading with a note on what it read before, "Why the Rust per-file floor exists", "Toolchain note" and "The pedantic lints, and why they are advisory". Dated measurements stay as they were. `docs/status.md` gains a settled bullet pointing at the new section, and its coverage and pedantic bullets say the floor and the toolchain as they now stand. This opens and closes no O-number. **Comments.** Those in `ci.yml`, `check-all.sh`, the `Dockerfile` and `advisories.yml` that said the toolchain floats now say it is pinned, and say the `rust:1-bookworm` tag still floats where that matters. **For review: the pedantic decision's reason.** That `clippy::pedantic` is an advisory step and never in `[workspace.lints]` is unchanged. Its stated reason was that a floating toolchain would turn an unrelated pull request red; under the pin a new clippy arrives only with the pull request that moves the pin, so that reason no longer holds as written. The docs and comments now keep the decision and say what it does under the pin: a bump is not held red by the lints its clippy adds in files it does not otherwise touch. Whether that still justifies the decision is the owner's call; this pull request does not reopen it. **Gate.** `scripts/check-all.sh` on 7f6d5c9 exited 0: "all green, except NOT MEASURED: - standing corpus checks (TAIYAKU_CORPUS unset)", and "coverage-floor: 29822 lines, 27520 covered, 92.28% — a slack of 381 covered lines under the 91% aggregate, and 80 of 100 file(s) at or under it". rustc there is 1.100.0 through the toolchain file: `rustup show active-toolchain` reads `1.100.0-x86_64-unknown-linux-gnu (overridden by '…/rust-toolchain.toml')`. With `aggregateFloor` in `coverage-floor.mjs` put back to 93 against `check-all.sh`'s 91, the floor script exited 2 naming both figures; restored, it exits 0. The one commit after the gate, 340d0b4, changes three sentences in `docs/ci.md` and `docs/status.md` and no code; `status-test-count` and `check-comments` pass on it. **Only CI can show** that the 0.9.1 tarball passes `sha256sum -c` in the runner. The digest was computed here from the downloaded asset and matches the `digest` GitHub's release API reports for it; the release publishes no checksum file. CI will also show whether `taiyaku-serve/build.rs`, which 0.8.7 listed in CI's report, leaves it under 0.9.1, as the local report's 100 files do not include it.
View all comments
One of the deep flaws in coverage instrumentation at the moment is that the instrumentor doesn't have actual knowledge of source-level code structure. Instead, it resorts to guessing source spans based on scanning through MIR statements and terminators, and applying a few crude heuristics that can never give truly accurate results.
(The decision to use MIR-based spans was made during the original implementation of
-Zinstrument-coverage. I was not able to find any discussion of this design decision or its consequences in the stabilization thread at #90132.)This PR is a partial step towards a better way of doing things. During THIR-to-MIR lowering, we now inject special
CoverageKind::Pointmarker statements that represent a specific connection between some part of the source code (in HIR form) and a specific point in MIR control-flow. Instead of trying to recover coverage spans from arbitrary MIR statements/terminators, we now only extract spans from those dedicated marker statements.Having accurate HIR-to-MIR correlations unlocks a lot of exciting possibilities for better coverage instrumentation. This PR mostly sets the stage for potential future improvements, but does demonstrate the ability to exclude specific HIR expressions from the set of coverage spans (to make assertions less noisy), which was previously unthinkable.
The key to making this all work is being able to accurately inject marker statements during MIR building. There is no single point in MIR building that all THIR expressions pass through, and most THIR expression nodes do not keep track of their corresponding HirId. However, it turns out that THIR building already wraps every expression node in a
thir::ExprKind::Scopenode that does store the original HirId, and the number of places in MIR building that handle those scopes is relatively small. So if we simply inject an appropriate marker statement at those places, with the HirId that is already available, that takes care of marking every HIR-level expression node in the built MIR.To avoid major regressions in coverage-span quality, we also need to inject markers in a few extra places that don't correspond to a HIR node, because they represent implied code such as the
else {}of a one-sidedif, or the automaticreturnat the end of a function body.There are necessarily some changes to coverage output as a result of the new approach. I have tried to keep these minimal, but I don't think it's worth trying to chase 1:1 compatibility with the old spans, which were very much a product of several layers of implementation details and heuristics. The differences seem reasonable overall, and in some cases are a clear improvement.