Skip to content

coverage: Switch over to hybrid HIR-aware coverage spans - #161517

Merged
rust-bors[bot] merged 5 commits into
rust-lang:mainfrom
Zalathar:coverage-point
Sep 6, 2026
Merged

rust-bors[bot] merged 5 commits into
rust-lang:mainfrom
Zalathar:coverage-point

Conversation

@Zalathar

@Zalathar Zalathar commented Aug 22, 2026 •

Copy link
Copy Markdown
Member

View all comments


One of the deep flaws in coverage instrumentation at the moment is that the instrumentor doesn't have actual knowledge of source-level code structure. Instead, it resorts to guessing source spans based on scanning through MIR statements and terminators, and applying a few crude heuristics that can never give truly accurate results.

(The decision to use MIR-based spans was made during the original implementation of -Zinstrument-coverage. I was not able to find any discussion of this design decision or its consequences in the stabilization thread at #90132.)

This PR is a partial step towards a better way of doing things. During THIR-to-MIR lowering, we now inject special CoverageKind::Point marker statements that represent a specific connection between some part of the source code (in HIR form) and a specific point in MIR control-flow. Instead of trying to recover coverage spans from arbitrary MIR statements/terminators, we now only extract spans from those dedicated marker statements.

Having accurate HIR-to-MIR correlations unlocks a lot of exciting possibilities for better coverage instrumentation. This PR mostly sets the stage for potential future improvements, but does demonstrate the ability to exclude specific HIR expressions from the set of coverage spans (to make assertions less noisy), which was previously unthinkable.


The key to making this all work is being able to accurately inject marker statements during MIR building. There is no single point in MIR building that all THIR expressions pass through, and most THIR expression nodes do not keep track of their corresponding HirId. However, it turns out that THIR building already wraps every expression node in a thir::ExprKind::Scope node that does store the original HirId, and the number of places in MIR building that handle those scopes is relatively small. So if we simply inject an appropriate marker statement at those places, with the HirId that is already available, that takes care of marking every HIR-level expression node in the built MIR.

To avoid major regressions in coverage-span quality, we also need to inject markers in a few extra places that don't correspond to a HIR node, because they represent implied code such as the else {} of a one-sided if, or the automatic return at the end of a function body.


There are necessarily some changes to coverage output as a result of the new approach. I have tried to keep these minimal, but I don't think it's worth trying to chase 1:1 compatibility with the old spans, which were very much a product of several layers of implementation details and heuristics. The differences seem reasonable overall, and in some cases are a clear improvement.

@Zalathar Zalathar added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Aug 22, 2026
@rustbot

rustbot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Some changes occurred in match lowering

cc @Nadrieril

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred in match checking

cc @Nadrieril

@rustbot rustbot added A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 22, 2026
@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Aug 22, 2026
@rustbot

rustbot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

r? @BoxyUwU

rustbot has assigned @BoxyUwU.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: compiler
  • compiler expanded to 75 candidates
  • Random selection from 21 candidates

@rust-log-analyzer

This comment has been minimized.

@Zalathar

This comment was marked as resolved.

@rust-log-analyzer

This comment has been minimized.

@rustbot

rustbot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

This PR changes MIR

cc @oli-obk, @RalfJung, @JakobDegen, @vakaras

@rustbot

This comment has been minimized.

@Zalathar

Copy link
Copy Markdown
Member Author

After some more investigation I found #145569 (comment), which suggests that for thir::Expr nodes that are not Scope, we can also recover the corresponding node from thir::Expr::temp_scope_id, since it's just the local part of the corresponding HIR node's HirId.

That might avoid the need to store a separate HirId in thir::ExprKind::If, for example.

@rustbot

This comment has been minimized.

@Zalathar
Zalathar force-pushed the coverage-point branch 2 times, most recently from 632cf3b to a3fa4da Compare August 26, 2026 05:37
@BoxyUwU

BoxyUwU commented Aug 27, 2026

Copy link
Copy Markdown
Member

I dont really know much about how coverage is implemented and dont have the time to do so.

r? @davidtwco

as he's doing other coverage stuff atm afaik

@rust-bors

This comment has been minimized.

@rustbot

This comment has been minimized.

@Zalathar

Zalathar commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

I'll land them separately, so that it's easier to look back at this PR if something goes wrong.

This variant will be used for marker statements that are injected when a HIR
expression is about to be lowered to MIR, and at other points relevant to
coverage instrumentation.

Adding this variant in its own commit keeps the mundane plumbing steps separate
from later changes that will perform injection and analysis.
In order to make coverage instrumentation HIR-aware, we need to have MIR
building inject marker statements at key locations, so that instrumentation can
accurately reconstruct the relationship between HIR constructs and MIR-level
control flow.

The most important of these is `PointKind::Expr`, which associates a HIR
expression with the start of the MIR that will evaluate that expression. There
isn't a single convenient place to inject these, but we can get close by
observing that THIR building wraps every expression in a
`thir::ExprKind::Scope` node, which also contains the `HirId` of that
expression. If we inject a marker whenever `thir::ExprKind::Scope` is lowered,
that results in an accurate marker for every expression.

Other markers are also injected at key points that don't correspond directly to
a HIR node, to avoid major regressions in coverage-map quality.
Instead of trying to heuristically recover source-code spans from MIR soup, we
can now take advantage of the `CoverageKind::Point` statements injected for
specific HIR expressions.

This is a transitional implementation that lets us remove the old MIR-soup
code, but still relies heavily on the existing span-refinement heuristics, and
does not take advantage of the full possibilities of HIR-aware coverage
instrumentation.

In order to avoid annoying never-executed spans in code that contains
assertions, a heuristic has been added that ignores any expression within the
arguments of a macro-expanded function call that returns `!`.

Some coverage tests now require `//@ min-llvm-version: 23`, due to changes in
how LLVM sorts the coverage mapping entries.
Now that we have a span for every HIR expression, we don't need these markers.
@rustbot

rustbot commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@Zalathar

Zalathar commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Rebased over the merge of #161397. PR CI is green, and my local tests have succeeded.

@bors r=davidtwco

@rust-bors

rust-bors Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

📌 Commit de8e5d7 has been approved by davidtwco

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 6, 2026
rust-bors Bot pushed a commit that referenced this pull request Sep 6, 2026
Rollup of 3 pull requests

Successful merges:

 - #161517 (coverage: Switch over to hybrid HIR-aware coverage spans )
 - #162030 (Prevent `--test` to be used in `rustdoc-html` testsuite)
 - #162329 (Suggest extra borrows for nested reference arguments)
@rust-bors
rust-bors Bot merged commit 211c330 into rust-lang:main Sep 6, 2026
13 checks passed
@rustbot rustbot added this to the 1.100.0 milestone Sep 6, 2026
rust-bors Bot pushed a commit that referenced this pull request Sep 6, 2026
Rollup merge of #161517 - Zalathar:coverage-point, r=davidtwco

coverage: Switch over to hybrid HIR-aware coverage spans

---

One of the deep flaws in coverage instrumentation at the moment is that the instrumentor doesn't have actual knowledge of source-level code structure. Instead, it resorts to guessing source spans based on scanning through MIR statements and terminators, and applying a few crude heuristics that can never give truly accurate results.

(The decision to use MIR-based spans was made during the original implementation of `-Zinstrument-coverage`. I was not able to find any discussion of this design decision or its consequences in the stabilization thread at #90132.)

This PR is a partial step towards a better way of doing things. During THIR-to-MIR lowering, we now inject special `CoverageKind::Point` marker statements that represent a specific connection between some part of the source code (in HIR form) and a specific point in MIR control-flow. Instead of trying to recover coverage spans from arbitrary MIR statements/terminators, we now only extract spans from those dedicated marker statements.

Having accurate HIR-to-MIR correlations unlocks a lot of exciting possibilities for better coverage instrumentation. This PR mostly sets the stage for potential future improvements, but does demonstrate the ability to exclude specific HIR expressions from the set of coverage spans (to make assertions less noisy), which was previously unthinkable.

---

The key to making this all work is being able to accurately inject marker statements during MIR building. There is no single point in MIR building that all THIR expressions pass through, and most THIR expression nodes do not keep track of their corresponding HirId. However, it turns out that THIR building already wraps every expression node in a `thir::ExprKind::Scope` node that *does* store the original HirId, and the number of places in MIR building that handle those scopes is relatively small. So if we simply inject an appropriate marker statement at those places, with the HirId that is already available, that takes care of marking every HIR-level expression node in the built MIR.

To avoid major regressions in coverage-span quality, we also need to inject markers in a few extra places that don't correspond to a HIR node, because they represent implied code such as the `else {}` of a one-sided `if`, or the automatic `return` at the end of a function body.

---

There are necessarily some changes to coverage output as a result of the new approach. I have tried to keep these minimal, but I don't think it's worth trying to chase 1:1 compatibility with the old spans, which were very much a product of several layers of implementation details and heuristics. The differences seem reasonable overall, and in some cases are a clear improvement.
@Zalathar
Zalathar deleted the coverage-point branch September 6, 2026 09:28
nicolas-guichard added a commit to nicolas-guichard/grcov that referenced this pull request Sep 8, 2026
rust-lang/rust#161517 (part of 2026-09-07)
changed the coverage output slightly, this accepts both versions.
nicolas-guichard added a commit to nicolas-guichard/grcov that referenced this pull request Sep 8, 2026
rust-lang/rust#161517 (part of 2026-09-07)
changed the coverage output slightly, this accepts both versions.
nicolas-guichard added a commit to nicolas-guichard/grcov that referenced this pull request Sep 9, 2026
rust-lang/rust#161517 (part of 2026-09-07)
changed the coverage output slightly, this accepts both versions.
marco-c added a commit to mozilla/grcov that referenced this pull request Sep 9, 2026
marco-c pushed a commit to mozilla/grcov that referenced this pull request Sep 9, 2026
rust-lang/rust#161517 (part of 2026-09-07)
changed the coverage output slightly, this accepts both versions.
Davey-Hughes added a commit to Davey-Hughes/taiyaku that referenced this pull request Nov 13, 2026
…rgo-llvm-cov 0.9.1, and the Rust line floor drops from 93 to 91, since 1.100.0 no longer counts a method name as covering its line (#243)

`rust-toolchain.toml` pins `channel = "1.100.0"` instead of `stable`, with the same components and target. CI's `test` job installs cargo-llvm-cov 0.9.1 instead of 0.8.7, the same `x86_64-unknown-linux-musl` tarball, pinned by its sha256 `3fca950394a3c49457657c158b1619cec8dfd2647ae5b48746734c0ab969a522`. The Rust aggregate line floor goes from 93 to 91 in `--fail-under-lines` in `ci.yml` and `scripts/check-all.sh`, whose six coverage lines still hash identically once their prefixes are stripped, and in the value `scripts/coverage-floor.mjs` mirrors.

**Why.** CI ran whatever `stable` was. Under Rust 1.100.0 the same commit, `main` at 28b18b6, with the same tests and commands, reads 29822 lines, 27519 covered, 92.28%, against 31158 lines, 29359 covered, 94.23% on 1.99.1, so every pull request failed `--fail-under-lines 93`.

The cause is upstream, "coverage: Switch over to hybrid HIR-aware coverage spans" (rust-lang/rust#161517, milestone 1.100.0): a method name no longer gets a coverage region of its own. So `.map_err(|e| format!("reading work {work_id}: {e}"))?;` at `crates/taiyaku-library/src/librarydoc.rs:310`, which ran 609 times, is credited only with its closure's and its `?`'s regions, both 0, and reads uncovered. Crediting such lines back gives 94.21%: the counting changed, not the testing.

**The owner's rulings.** The floor is 91, which keeps about the margin 93 had under 1.99.1. The toolchain is pinned to 1.100.0. CI moves to cargo-llvm-cov 0.9.1, the version the local gate runs.

**Bumping the toolchain** is now a pull request of its own: `docs/ci.md` says to measure the same commit on both toolchains, then move the pin, and re-set the floor only if the counting changed.

**Docs.** `docs/ci.md` gains "Why the Rust floor is 91, and the toolchain is pinned" under the coverage gates, with the two measurements, the upstream change and its example line, the 94.21% figure, the rulings and how a bump is done. Its present-tense statements of the floor and the toolchain now say 91 and pinned: the coverage gates table, "The numbers, and where they came from", whose Rust row now carries the 1.100.0 reading with a note on what it read before, "Why the Rust per-file floor exists", "Toolchain note" and "The pedantic lints, and why they are advisory". Dated measurements stay as they were. `docs/status.md` gains a settled bullet pointing at the new section, and its coverage and pedantic bullets say the floor and the toolchain as they now stand. This opens and closes no O-number.

**Comments.** Those in `ci.yml`, `check-all.sh`, the `Dockerfile` and `advisories.yml` that said the toolchain floats now say it is pinned, and say the `rust:1-bookworm` tag still floats where that matters.

**For review: the pedantic decision's reason.** That `clippy::pedantic` is an advisory step and never in `[workspace.lints]` is unchanged. Its stated reason was that a floating toolchain would turn an unrelated pull request red; under the pin a new clippy arrives only with the pull request that moves the pin, so that reason no longer holds as written. The docs and comments now keep the decision and say what it does under the pin: a bump is not held red by the lints its clippy adds in files it does not otherwise touch. Whether that still justifies the decision is the owner's call; this pull request does not reopen it.

**Gate.** `scripts/check-all.sh` on 7f6d5c9 exited 0: "all green, except NOT MEASURED: - standing corpus checks (TAIYAKU_CORPUS unset)", and "coverage-floor: 29822 lines, 27520 covered, 92.28% — a slack of 381 covered lines under the 91% aggregate, and 80 of 100 file(s) at or under it". rustc there is 1.100.0 through the toolchain file: `rustup show active-toolchain` reads `1.100.0-x86_64-unknown-linux-gnu (overridden by '…/rust-toolchain.toml')`. With `aggregateFloor` in `coverage-floor.mjs` put back to 93 against `check-all.sh`'s 91, the floor script exited 2 naming both figures; restored, it exits 0. The one commit after the gate, 340d0b4, changes three sentences in `docs/ci.md` and `docs/status.md` and no code; `status-test-count` and `check-comments` pass on it.

**Only CI can show** that the 0.9.1 tarball passes `sha256sum -c` in the runner. The digest was computed here from the downloaded asset and matches the `digest` GitHub's release API reports for it; the release publishes no checksum file. CI will also show whether `taiyaku-serve/build.rs`, which 0.8.7 listed in CI's report, leaves it under 0.9.1, as the local report's 100 files do not include it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants