Skip to content

openvmm: pin the policy check of a restored host CPU profile - #1585

Merged
Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
agents/issue-1357-host-profile-policy
Nov 14, 2026
Merged

Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
agents/issue-1357-host-profile-policy

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1357

What and why

This pins microsoft/openvmm#5094 at 7b205c0a0fe7a40cc08fd46951d724bc293bc703. With the pin, restoring a snapshot of a host CPU profile (--cpu-profile host, or the fallback of auto) refuses a profile document that the derivation's policy would change, with E_PROFILE_DIGEST. Before, a snapshot that carried a document with VMX or SVM set restored, and on KVM its guest could use them.

At NVX's previous pin, efe6bc267:

  • Cause. A snapshot of a host profile carries the only copy of the profile's document, and host_profile_for_restore accepted any document that decodes to a valid host profile with the ID and digest that the snapshot recorded. Whoever crafts a snapshot writes the digest of an edited document and the effective CPUID that it gives. Nothing compared the document with the derivation's policy (apply_policy, clear_stateless_features, and the ALLOWED_* XSAVE constants of vmm_core/cpu_profile/src/derive.rs), which clears the features that a VM must not see. A document with CPUID.1:ECX[5], VMX, or CPUID.0x80000001:ECX[2], SVM, therefore restored: the effective CPUID, the backend's support check, and VP 0's CPUID all follow the document.
  • Reach. On KVM, a time ABI partition's CPUID is the profile's effective CPUID, so the VMX or SVM strip that new_partition adds for !nested_virt does not reach it. The guest could enable VMX or SVM and reach KVM's nested-virtualization code, which nested_virt false keeps closed. On MSHV and WHP, the backend's support check refuses the edited profile (E_PROFILE_UNSUPPORTED), since those partitions lack VMX. Only a crafted or foreign snapshot carries such a document: OpenVMM derives the profile of its own captures with the same policy, so they never do.

What the OpenVMM pull request changes

microsoft/openvmm#5094 adds one signed commit on efe6bc267, the head of feature/microvm/dev. It has a good signature (openvmm_pin.py source).

Position Commit Change
276 7b205c0a0 derive::policy_violations applies the derivation's own apply_policy and clear_stateless_features to a profile's CPUID table, with the profile's XCR0, compares the result with the table, and checks the XSAVE features against ALLOWED_XCR0, ALLOWED_XSS, and ALLOWED_XSAVE_EXTENSIONS. host_profile_for_restore returns E_PROFILE_DIGEST for a profile that it finds outside the policy, naming at most eight of the differences, and both the controller's restore preflight (preflight_time_abi_restore) and the VM worker's profile selection (select_on_host) call it, so a crafted snapshot fails before the worker starts, and again if a worker is ever handed one. A profile that the derivation produced is a fixed point of its policy, which only sets and clears bits and zeroes leaves, so the check accepts it, and a rule that the derivation gains is checked without a second copy of it. The derivation, the pinned profiles, the policy label POLICY, and the error codes are unchanged. Three tests of the restore fail at efe6bc267, since each restore succeeds, and the derive tests check that every pinned profile, the host profile of every recorded host, and the host profile of random mutations of each host's CPUID follow the policy (an offline run of 18,000 mutations found no violation), and that each feature that the policy withholds is a violation alone.

The patch that Ruize Tang (@tangruize) attached to #1357 proposes a cpu_profile::policy module of declarative rule tables that CpuProfile::effective_cpuid checks at every start. This takes a smaller route to the same refusal: it reuses the derivation's own rules, so no second copy of them can fall out of step with the derivation, and it checks only where a profile enters from a snapshot, so a cold boot and a pinned profile, which are constants that OpenVMM verifies, pay nothing.

What this pull request changes in NVX

  • dca32976276 documents the check where the documentation describes a host profile's snapshot: the limits of host profiles in doc/usage.md, the CPU profile selection and verification of doc/design/2.2-cpu-and-topology.md, and the restore compatibility of doc/design/1.5-compatibility-and-versioning-model.md.
  • 0928ba504 moves the openvmm gitlink to the pin, and nothing else.
  • 106dcd449 maps position 276 in the OpenVMM roadmap.

The consequence for older host-profile snapshots

The issue flags that a host-profile snapshot that an OpenVMM with another policy captured can no longer be restored once the policy changes. No version is created or bumped to handle it: no policy version, profile revision, schema or format version, capability or generation, or error code. Restore applies the rules of its own build, and does not read the policy label that a profile's provenance names. A host-profile snapshot that another policy derived therefore restores only if this build's policy leaves its profile unchanged, and otherwise fails with E_PROFILE_DIGEST, whose message says that the snapshot needs a recapture. Host profiles are for development and are not pinned, and a build that computes another effective CPUID already rejects their snapshots (E_CPU_SURFACE), so the documentation states the behavior and the recapture rather than a compatibility promise.

Pin details

  • Roadmap. The pin is 152 commits past the mapped tip.
    • Position 276 is not yet assigned to a PR.
    • It builds on the chain: upstream main, 7d90c4104, lacks the cpu_profile crate and the time ABI's modules of openvmm_helpers and openvmm_core, which its change and its tests are in, and the section of the snapshots page that it edits. Replaying it there conflicts in all seven of its files, so it follows the time ABI series upstream. It changes nothing that a mapped PR adds.
  • Prerequisites. None. The pull request's base, efe6bc267, is NVX's previous pin and the head of feature/microvm/dev.
  • Equivalence. None claimed; this pin replaces no reviewed revision.
  • Coordinated changes. None. The control-session vectors, the source excerpts, the contract versions, the activation bound, and the CPU profiles that nvx-rs copies from OpenVMM all match the revision's (openvmm_vectors and openvmm_cpu_profiles, below).

Scope and limits

  • What the check bounds. It holds a restored host profile's feature bits and XSAVE features to the policy. What the derivation fixes besides them, the XSAVE layout, the pinned MSRs, and the physical address width, stays with the destination backend's support check, which bounds a profile by what the backend presents; the generation by check_generation; and the brand and provenance are informative only.
  • AMD. No AMD host ran the reproduction. The AMD rules, SVM among them, are covered by recorded AMD fingerprints and a crafted AMD profile in the unit tests.
  • Where the hosts ran. The hosts ran OpenVMM built from 71b358bce, an earlier version of the pinned commit whose tree, fee5fcc1c, is that of 7b205c0a0 (they differ in the message only), with NVX at 6e7a13200, before the rebase onto 507ceeea4. The rebase moves no OpenVMM or cpu_profile code, so the exact head's CI below shows the same scenarios on the rebased base.

Landing order

microsoft/openvmm#5094 landed first, by fast-forwarding feature/microvm/dev from efe6bc267 to 7b205c0a0 (python scripts\openvmm_pin.py land 7b205c0a0fe7a40cc08fd46951d724bc293bc703 --remote origin --push), and it shows as merged at that commit. This pull request merged after that, through the merge queue. OpenVMM's merge button would squash the commit and leave the pinned revision off the branch, so it wasn't used.

Validation

NVX

Run on Windows:

  • python scripts\openvmm_pin.py source 7b205c0a0fe7a40cc08fd46951d724bc293bc703 --pr 5094: passed. The revision is the pull request's head, with a good signature.
  • python scripts\openvmm_pin.py check-roadmap --worktree: passed, with 276 positions and 287 references.
  • python scripts\openvmm_pin.py check-branch --topic: passed, with three signed commits and no coordinated files.
  • python scripts\check_markdown_links.py and git diff --check: passed.
  • python scripts\nvx.py verify: passed.
  • cargo test --locked --manifest-path nvx-rs/Cargo.toml --no-default-features --test openvmm_vectors -- --ignored (4 passed) and --test openvmm_cpu_profiles -- --ignored (1 passed).
  • python scripts\local_gates.py --base origin/dev selects seven gates: test-changes, markdown-links, test-layout, openvmm-vectors, openvmm-cpu-profiles, nvx-cli, and openvmm-ci. I ran the first six, and all passed; nvx-cli, as python scripts\local_gates.py --base origin/dev --run --only nvx-cli with NVX_RS set to a release nvx-rs built from this branch, ran 2199 tests (88 skipped), then scripts\test_adversarial.py ran 84 (11 skipped). test-changes found no changes to existing tests, so the pull request needs no tests-modified label. The seventh, openvmm-ci, is nvx.py check-openvmm-ci, under CI below.

OpenVMM

For cpu_profile, openvmm_helpers, and openvmm_core, with this change, with Rust 1.97.1 and cargo-nextest 0.9.133:

  • Windows x64: cargo clippy --all-targets, cargo doc --no-deps, and cargo nextest run --profile agent (132, 138, and 130 passed) passed, and cargo xtask fmt --fix changed nothing.
  • Linux x64, on nvx-kvm-test: cargo nextest run --profile agent passed with 132 cpu_profile tests and 276 openvmm_helpers and openvmm_core tests.
  • Regression: at efe6bc267, with the new tests, the restore test of cpu_profile, the preflight test of openvmm_helpers, and the selection test of openvmm_core fail, since each restore succeeds. With the pinned revision, disabling the stateless clear, the XSAVE check, or the derivation's VMX clear makes tests fail.

Hosts

The checkouts were temporary worktrees. On each host, nvx-rs download installed the release of the nearest first-parent ancestor of the checkout, v0.1.0-dev.6e7a13200472, whose OpenVMM is the packaged efe6bc267 binary. OpenVMM was then built from 71b358bce in a release build without LTO or debug information: on each Linux host in its worktree, and for WHP on Windows x64, from where openvmm.exe was copied to the host.

A scratch script, issue1357_repro.py, which is not part of this change, cold boots a microVM with --cpu-profile host, captures its snapshot with a post-restore probe that reads IA32_VMX_BASIC through /dev/cpu/0/msr, restores the snapshot as captured (the control), then restores a copy whose host profile document has CPUID.1:ECX[5] set, recorded with the edited document's own digest and the effective CPUID record that it gives. The control restores on every host, and its guest's read faults, since the guest has no VMX.

Host Platform Host type The edited snapshot at efe6bc267 (packaged) The edited snapshot with the pin's tree
nvx-kvm-test (Intel Xeon Platinum 8573C, Linux 6.18.57) linux-kvm virtual-machine restores, and the guest reads IA32_VMX_BASIC, 0x0098100011e57ed0, without a #GP [E_PROFILE_DIGEST]: "CPUID 0x1 ECX bit 5 is set, which the policy clears"
nvx-mshv-test (Intel Xeon Platinum 8573C) linux-mshv virtual-machine E_PROFILE_UNSUPPORTED E_PROFILE_DIGEST
nvx-whp-test (Intel Xeon Platinum 8370C, Windows Server 2025) windows-whp virtual-machine E_PROFILE_UNSUPPORTED E_PROFILE_DIGEST

On KVM, a build of efe6bc267 that I built on the host restores it the same way as the packaged one.

nvx.py test-microvm --backend BACKEND --scenario cpu-profile-fallback --scenario guest-boot --scenario snapshot-core ran these scenarios, which capture and restore a real host profile, on each host with the pin's tree, and all passed.

CI

  • OpenVMM's CI of 7b205c0a0: run 43565576664. The six jobs that NVX requires passed, as python scripts\nvx.py check-openvmm-ci --revision 7b205c0a0fe7a40cc08fd46951d724bc293bc703 reports: unit tests [x64-linux] and [x64-windows], and run vmm-tests for x64-linux-amd-kvm, x64-linux-intel-mshv, x64-windows-intel, and x64-windows-amd. 33 of the run's 34 jobs passed. The one that failed, run vmm-tests [x64-windows-amd-snp], isn't one that NVX requires, and it fails at the base too: run 43527602221 of efe6bc267 failed that job and no other. It wasn't rerun.
  • NVX CI: run 43566211802 of the head 106dcd449 succeeded: 38 checks passed and 8 were skipped, Publish development release, Publish nvx crate, Package nvx crate, Record performance results, Persist performance baselines, Cancel superseded merge group runs, and the KVM and WHP Debug kernel jobs. It ran what I left to it:
    • NVX CLI / Linux, whose gate I ran on Windows only, and NVX CLI / Windows.
    • NVX microVM tests on KVM, MSHV, and WHP, with their shards, their guests and sandboxes, and WHP's runtime adapter, with the OpenVMM that the jobs built from the pin.
    • The benchmarks, Platform / Linux / KVM, Platform / Linux / MSHV, and Platform / Windows / WHP, all on virtual machines, and Performance regression gate.
    • OpenVMM CI of the pinned revision, which waited 1 h for OpenVMM's jobs above, and Required status check.
  • Review: Copilot reviewed this head and found no issues. The pull request has no review threads.
  • Merge queue: run 43571273767 of the merge group, this head on 507ceeea4, succeeded: 35 jobs passed and 8 were skipped.

A snapshot of a host profile carries the only copy of the profile's
document, and the digest that it records proves nothing about what the
document says, since whoever crafts a snapshot writes both. OpenVMM now
also holds the document to its derivation policy when it restores the
snapshot (E_PROFILE_DIGEST), so that an edited host profile cannot give
the guest what every derived profile withholds, such as VMX or SVM on
KVM, and refuses a profile that the policy would change.

No version is created or bumped, and OpenVMM applies the policy of its
own build whatever policy a profile's provenance names. A snapshot of a
host profile that an OpenVMM with another policy derived therefore
restores only if the restoring OpenVMM's policy leaves it unchanged.
Host profiles are for development, are not pinned, and a build that
computes another effective CPUID already rejects their snapshots, so the
documents that describe a host profile's snapshot say that it can need a
recapture: the usage guide's limits of host profiles, the CPU profile
selection and verification of the design, and the restore compatibility
of the compatibility model, whose bullet for host profiles now names the
policy. The verification's report of every violation notes that a
profile outside the policy names its first eight differences.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A snapshot of a host CPU profile (--cpu-profile host, or the fallback of
auto) carries the only copy of the profile's document, and restoring it
accepted any document that decodes to a valid host profile with the
recorded ID and digest, which whoever crafts a snapshot writes along
with the effective CPUID that the document gives. Nothing compared the
document with the derivation's policy, so a document with VMX
(CPUID.1:ECX[5]) or SVM (CPUID.0x80000001:ECX[2]) restored, and on KVM,
where the time ABI partition's CPUID is the profile's effective CPUID,
the guest could enable VMX or SVM and reach KVM's nested-virtualization
code, which nested_virt false keeps closed (#1357).

Pin 7b205c0a0fe7a40cc08fd46951d724bc293bc703, the head of
microsoft/openvmm#5094, one commit on efe6bc267, the head of
feature/microvm/dev:

- 7b205c0a0 cpu_profile: refuse a restored host profile outside the
  derivation policy

host_profile_for_restore now also refuses a host profile that the
policy would change (E_PROFILE_DIGEST), naming at most eight of the
differences, and both the controller's restore preflight and the VM
worker's profile selection call it. The check applies the derivation's
own rules to the profile's CPUID table, so a derived profile passes it,
and it checks the XSAVE features against the policy's allowances. The
derivation, the pinned profiles, and the policy label are unchanged,
and no version is created or bumped: restore applies the rules of its
own build, so a host profile's snapshot that an OpenVMM with another
policy captured restores only if this policy leaves its profile
unchanged, and otherwise needs a recapture, as the error says.

Tests: with OpenVMM built from the pin's tree, a host profile's
snapshot restores, and a copy of it whose profile has VMX set, recorded
with its own digest and effective CPUID, is refused with
E_PROFILE_DIGEST on nvx-kvm-test, nvx-mshv-test, and nvx-whp-test. The
unchanged OpenVMM restores that copy on nvx-kvm-test, where the guest
reads IA32_VMX_BASIC through /dev/cpu/0/msr without a #GP, and refuses
it on nvx-mshv-test and nvx-whp-test with E_PROFILE_UNSUPPORTED, since
their partitions lack VMX. The microVM scenarios guest-boot,
cpu-profile-fallback, and snapshot-core, which capture and restore a
real host profile, pass on all three. The revision's three tests of the
restore fail at efe6bc267, since each restore succeeds, and its 132
cpu_profile, 138 openvmm_helpers, and 130 openvmm_core tests pass on
Windows.

NVX needs no coordinated change: the control-session vectors, the
source excerpts, the contract versions, the activation bound, and the
CPU profiles that nvx-rs copies from OpenVMM match the revision's.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The NVX pin moves to 7b205c0a0, position 276, 152 commits past the
mapped tip. Position 276, the one commit of microsoft/openvmm#5094 on
efe6bc267, the head of feature/microvm/dev, is not yet assigned to a
PR.

Position 276 makes the restore of a host CPU profile refuse a profile
that the derivation policy would change, which let a snapshot's own
copy of its profile give the guest the VMX or SVM that every derived
profile withholds. It adds policy_violations to the cpu_profile
crate's derivation, which applies the policy of position 131 to the
profile, and has host_profile_for_restore of position 142 refuse a
profile that differs. It extends position 142's host profile tests of
the restore preflight and of the worker's selection of the profile, and
describes the refusal on the Guide's snapshots page. It changes nothing
that a mapped PR adds. Replaying it on upstream main conflicts in all
seven of its files, since upstream main lacks the cpu_profile crate,
the time ABI's modules of openvmm_helpers and openvmm_core, and the
snapshots page's section on time and CPU compatibility, so it follows
the time ABI series upstream.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive snapshot-policy change needs maintainer review because exact-pin platform CI is not fully verified and the required openvmm#5094 change remains unmerged.

No open issues 🎉

What changed in this PR

This PR pins OpenVMM’s restored host CPU profile policy check to address #1357 and documents its compatibility implications in NVX. The check rejects snapshot profiles that violate the restoring build’s derivation policy, including profiles that enable VMX or SVM.

Changes:

  • Document policy validation and the E_PROFILE_DIGEST rejection.
  • Explain when snapshots derived under another policy require recapture.
  • Map OpenVMM commit 7b205c0a0 at roadmap position 276.
File Description
doc/​usage.md Explains host-profile restore restrictions and recapture requirements.
doc/​openvmm-upstream-roadmap.md Updates the pin, chain counts, and position 276 mapping.
doc/​design/​2.2-cpu-and-topology.md Describes policy validation and bounded violation reporting.
doc/​design/​1.5-compatibility-and-versioning-model.md Adds policy compliance to host-profile restore requirements.

Review effort: Balanced

@ppenna

Copy link
Copy Markdown
Contributor Author

No change needed here. Copilot's overview, which has no findings, notes that exact-pin platform CI was not yet verified and that microsoft/openvmm#5094 was unmerged when it reviewed. Both are now settled, or will be before this merges:

  • OpenVMM's CI of the exact pin, 7b205c0a0, passed the six jobs that NVX requires (run 43565576664: unit tests on x64-linux and x64-windows, and run vmm-tests on x64-linux-amd-kvm, x64-linux-intel-mshv, x64-windows-intel, and x64-windows-amd), and this head's OpenVMM CI of the pinned revision job passed on them.
  • This head's NVX CI, run 43566211802, succeeded, with the microVM tests and benchmarks on KVM, MSHV, and WHP.
  • [DO NOT REVIEW] cpu_profile: refuse a restored host profile outside the derivation policy openvmm#5094 lands first, by fast-forwarding feature/microvm/dev from efe6bc267 to 7b205c0a0 (python scripts\openvmm_pin.py land 7b205c0a0fe7a40cc08fd46951d724bc293bc703 --remote origin --push), and this pull request merges after that, through the merge queue.

@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 14, 2026
Merged via the queue into dev with commit c738721 Nov 14, 2026
48 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1357-host-profile-policy branch November 14, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

openvmm: a snapshot's host CPU profile can give the guest VMX or SVM

2 participants