Repository navigation
openvmm: pin the XSAVE restore's bounds check - #1642
Conversation
Restoring a KVM VP whose saved XSAVE area was shorter than its legacy area and header, or whose header selected a component that ended past the end of the area, panicked the VM worker, in debug and release builds: Xsave::write_standard unwrapped its reads and sliced the area without checking either bound. A save never returns such an area, so only a crafted or damaged snapshot reaches this (#1381). Pin f9cc4163bc5fe03a578a07064ec2fc212268d9bf, the head of microsoft/openvmm#5119, one commit on 23fb67887, the head of feature/microvm/dev: - f9cc4163b virt_kvm: refuse to restore a short or malformed XSAVE area write_standard now returns a typed XsaveError, and checks the area's words and each selected component's place before it writes a byte, so the restore fails with the xsave element's StateError before any KVM_SET_XSAVE changes the VP. The issue proposed a predicate that set_xsave would call before a write_standard that still panics, and the check of #1380 refuses with a fixed InvalidState message. The typed result has one definition of the places, so that the check and the copy cannot disagree, and it says which word count or component is wrong. No saved state, saved-state format, or version changes, and none is created or bumped. An area that a save returns restores as before. Tests: five unit tests of write_standard in virt and one in virt_kvm, which OpenVMM's CI runs, and a KVM-only backend contract test that restores shortened and malformed areas into a real VP, which runs in the vmm-tests jobs of the KVM hosts. The contract test fails at the base, with a panic, and passes with the pin. NVX needs no coordinated change: the control-session vectors, the source excerpts, the contract versions, the activation bound, and the CPU profiles that nvx-rs copies from OpenVMM match the revision's. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
The roadmap inaccurately describes position 302 as changing only chain-introduced code, omitting changes to the existing upstream Xsave::write_standard signature and component-copy loop that must also be carried during upstreaming.
1 open issue
What changed in this PR
This PR updates NVX’s OpenVMM pin to address #1381: malformed XSAVE state should fail KVM restore with an error rather than panic the VM worker. The reviewed documentation records the revision and its upstream integration context.
Changes:
- Records pin
f9cc4163bat position 302 and updates the chain counts. - Documents XSAVE bounds validation, error propagation, and regression tests.
- Describes upstream conflicts and prerequisite chain positions.
| File | Description |
|---|---|
doc/openvmm-upstream-roadmap.md |
Updates the pin mapping and documents the XSAVE restore fix and upstream dependencies. |
Review effort: Balanced
The NVX pin moves to f9cc4163b, position 302, 178 commits past the mapped tip. Position 302, the one commit of microsoft/openvmm#5119 on 23fb67887, the head of feature/microvm/dev, is not yet assigned to a PR. Position 302 makes a restore of a short or malformed XSAVE area fail with an error, where it panicked the VM worker: Xsave::write_standard, which Buffers::new calls on the area of the saved state, unwrapped its reads of the legacy area and of the header, and sliced the area at each selected component without checking either bound. It now returns a typed XsaveError, checking the area and the buffer before it writes a byte, and KvmError gains Xsave. It changes no line that a mapped PR adds: it rewrites write_standard, which upstream main has and no commit of the chain changes, changes lines that positions 281 and 301 added, which no PR has yet, and adds the rest. Replaying it on upstream main conflicts in xsave.rs, which does not exist there since position 281 adds it, in lib.rs beside position 284's GifClear, and in vp.rs beside the tests of positions 132 and 296, never in write_standard. Its contract test needs the kvm_only form of backend_test! that position 279 adds, and upstream main's other caller of write_standard, in set_xsave of vp_state.rs, which position 281 replaces, would have to handle the result. So it follows positions 132, 279, 281, 284, 296, and 301 upstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2d97fae to
f053407
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
The KVM saved-state pin needs maintainer confirmation because exact-pin AMD/KVM contract testing is still running and OpenVMM #5119 has not landed on feature/microvm/dev.
No open issues 🎉
1 resolved since last review
Review effort: Balanced

Fixes #1381
What and why
On KVM, restoring a saved VP state whose XSAVE element is shorter than 72 words, or whose header selects a component that ends past the area, panicked the VM worker, in debug and release builds. A save never returns such an area, so only a crafted or damaged snapshot reaches it.
Root cause.
xsave::restorebuilds theKVM_SET_XSAVEbuffers withBuffers::new, which callsXsave::write_standard(vmm_core/virt/src/x86/vp.rs) on the area of the saved state.write_standardunwrapped its reads of the legacy area (fxsave) and of the header (xsave_header), and sliced the area at the compact place of each component that both XCOMP_BV and XSTATE_BV select, without checking either bound. The checks thatrestorealready made do not cover it:check_xstate_bv(#1380, landed as #1589) reads XSTATE_BV only, withget, andcheck_xsave_layout(#1382) checks the caps' layout against KVM's buffer, not a restored area. MSHV and WHP pass the compact area to the hypervisor as it is, and never callwrite_standard.Change, in OpenVMM's
virtandvirt_kvm(microsoft/openvmm#5119, one commit onfeature/microvm/dev, headf9cc4163bc5fe03a578a07064ec2fc212268d9bf):write_standardreturnsResult<(), XsaveError>. Before it writes a byte, it checks that the area holds the legacy area and the header, that the buffer holds them, and that each component that both masks select lies in the area at its compact place and in the buffer at its standard offset, and it fails withAreaTooShort,ComponentPastArea, orBufferTooShortotherwise, so a refusal writes nothing. The check and the copy take the components' places from one iterator, so they cannot disagree.Buffers::new, the only caller, returns the error, andrestorepropagates it before anyKVM_SET_XSAVE.KvmErrorgainsXsave, which wraps it, sorestore_allfails with theStateErrorof the xsave element and the VP's XSAVE state stays as it was.write_standardnever read. Only areas that panicked are refused, with an error. The guest sees nothing different.Why this is not the issue's patch, nor #1589's
InvalidStateThe patch of #1381 adds a predicate,
Xsave::fits_standard(len, caps), whichset_xsavecalls beforewrite_standard. It stays a second description of where each component lies, which must agree with a function that panics on any area it does not accept, and its test catches the panics ofwrite_standardto compare the two. The check of #1380 that #1589 landed refuses withKvmError::InvalidStateand a fixed message, with no new type, which cannot say what is wrong, and leaveswrite_standardpanicking on its own for any other caller.This change makes the conversion itself fallible and typed instead:
write_standardwith its specification directly.KvmErrorvariant.The shapes, at the base and with this change
An area that a saved state holds, restored through
restore_allorset_xsaveon a real KVM VP (the contract test), or throughwrite_standard(thevirttests):called Result::unwrap() on an Err value: SizeError, infxsaveAreaTooShort { words }called Result::unwrap() on an Err value: Size(SizeError), inxsave_headerAreaTooShort { words }range end index 832 out of range for slice of length 576ComponentPastArea { component: 2, words: 72 }range end index 832 out of range for slice of length 824ComponentPastArea { component: 2, words: 103 }BufferTooShort { len }(KVM's 4096-byte buffer, whichcheck_xsave_layoutsizes for the caps, cannot reach this)No version
Nothing is versioned, and no reader refuses a snapshot: no saved-state field, format, schema, capability, or version changes, and none is created or bumped. No profile, contract, or time ABI version, digest, or identity changes either.
XsaveErrorandKvmError::Xsaveare in-process types. A snapshot of an earlier build restores exactly as before, except that an area that panicked the worker is refused with an error.Validation
The change is OpenVMM's, so most of the evidence is OpenVMM's tests, which microsoft/openvmm#5119 lists. On the KVM host
nvx-kvm-test(an Azure virtual machine: Linux 6.18.57.1-1.azl3 on an Intel Xeon Platinum 8573C):23fb67887, fails:called Result::unwrap() on an Err value: SizeError, atvmm_core/virt/src/x86/vp.rs:863, infxsave, and the other 11 KVM contract tests pass. A scratch variant of it, not kept, restored each shape above through a real VP at9d64daad3and showed the panics of the table's second column;write_standardis the same at23fb67887.f9cc4163b: the 12 KVM contract tests pass, the new one included, and the 255 tests ofvirt,virt_kvm, andkvm, the ignored/dev/kvmones included, pass. Each of four mutants ofwrite_standardfails 2 to 4 of the five newvirttests.openvmmrelease binary built at the pin (source_revisionf9cc4163bc5fe03a578a07064ec2fc212268d9bf, clean),test-microvmpasses the scenarios that restore real VPs' XSAVE areas throughrestore.Checks that ran
python scripts/nvx.py verifypython scripts/local_gates.py --base origin/dev --run --keep-going --only GATE..., in two runs, at the headtest-changes,markdown-links,test-layout,openvmm-vectors, andopenvmm-cpu-profilespassed, andnvx-clipassed (2210 tests with 88 skipped, and the adversarial suite's 84 with 11 skipped).openvmm-ci, the one other gate that the change selects, was not run: it cannot read the pin's OpenVMM runs from here (GitHub's rate limit), so CI doespython scripts/openvmm_pin.py check-roadmap --worktree,python scripts/openvmm_pin.py check-branch --topic,python scripts/check_markdown_links.py,python scripts/check_test_layout.py,python scripts/check_test_changes.py --base origin/dev(0 changes),git diff --checkcargo test --locked --manifest-path nvx-rs/Cargo.toml --no-default-features --test openvmm_vectors -- --ignored, and--test openvmm_cpu_profiles -- --ignored, at the pinopenvmm/at the pin:cargo test -p virt(126 tests),cargo clippy --locked --all-targets -p virt -- -D warnings,cargo doc --no-deps -p virt, andcargo clippy --locked --all-targets -p virt --target aarch64-apple-darwin -- -D warningsvirtthere, where OpenVMM's CI also runs it)nvx-kvm-testopenvmm/at the pin:cargo nextest run --profile agent --run-ignored all -p virt -p virt_kvm -p kvm, andcargo nextest run -p vmm_tests --test virt_backend_testsnvx-kvm-testopenvmm/at the pin:cargo clippy --locked --all-targets -p virt -p virt_kvm -p vmm_tests -- -D warnings(glibc, andvirt_kvmwith--target x86_64-unknown-linux-musl),cargo doc --no-deps -p virt -p virt_kvm,cargo xtask fmt --pass rustfmt --pass lintsnvx-kvm-testpython3 scripts/nvx.py test-microvm --backend kvm --processors 1 2with--scenariosmp,smp-snapshot,snapshot-core,snapshot-rollback,restore-downtime,restore-processors,restore-memory,snapshot-tiers, andtime-abi-conformance, with theopenvmmrelease binary built at the pin (clean) in a checkout of this branchLeft to CI:
openvmm-ciand OpenVMM's own CI of the pinned revision, whichopenvmm-ciwaits for, among it the first run of the new contract test on AMD (x64-linux-amd-kvm); the MSHV and WHP jobs (the change is confined tovirt_kvmand the cross-platformvirt, whose new code is tested on Windows too); and the benchmarks.CI of this head,
f05340789df, whose OpenVMM isf9cc4163b:OpenVMM CI of the pinned revision, the microVM tests on KVM, MSHV, and WHP, the performance regression gate, andRequired status checkincluded, with no rerun.f9cc4163b: the six jobs thatopenvmm-cirequires passed, the unit tests on x64-linux and x64-windows, and the vmm-tests onx64-linux-amd-kvm,x64-linux-intel-mshv,x64-windows-intel, andx64-windows-amd. The new contract testxsave::kvm::refuses_a_short_or_malformed_xsave_areapassed onx64-linux-amd-kvm(job 132930492056), its first run on AMD.x64-windows-amd-snpfailed in Hyper-V OpenHCL SNP guest tests and is not required. No job was rerun.2d97fae11ff) had one thread: the roadmap remark said that position 302 changes only lines of positions 281 and 301, where it also rewrites upstream'swrite_standard.f05340789dffixes the remark and the commit message, and the thread is resolved. Its review of this head has no open issue.Tests modified
python scripts/check_test_changes.py --base origin/devlists no change to an existing test asset, so this pull request has notests-modifiedlabel. The tests are OpenVMM's, listed in microsoft/openvmm#5119: five unit tests ofwrite_standardinvirt,a_short_or_malformed_area_gets_no_buffersinvirt_kvm, and the KVM-only contract testxsave::kvm::refuses_a_short_or_malformed_xsave_area. The existing tests ofxsave.rschanged only in unwrapping the result ofBuffers::new.OpenVMM pin
f9cc4163bc5fe03a578a07064ec2fc212268d9bf, a signed commit.23fb67887, the head offeature/microvm/dev, which holds [DO NOT REVIEW] virt_kvm: restore the init components of an XSAVE area as init openvmm#5118 (openvmm: KVM's set_xsave keeps old PKRU or MXCSR after a VM reset on some hosts #1360). It changesvmm_core/virt/src/x86/vp.rs,vmm_core/virt_kvm/src/arch/x86_64/xsave.rs,vmm_core/virt_kvm/src/lib.rs,vmm_tests/vmm_tests/tests/virt_backend_tests/main.rs, and the newvmm_tests/vmm_tests/tests/virt_backend_tests/xsave.rs. No prerequisites.write_standard, which upstream main has and no commit of the chain changes, changes lines that positions 281 and 301 added, and adds the rest; and that it follows positions 132, 279, 281, 284, 296, and 301 upstream, where it conflicts with upstream main in three files, none of them inwrite_standard, and needs thekvm_onlyform ofbackend_test!and a handling of the new result in upstream's other caller ofwrite_standard.openvmm-vectorsandopenvmm-cpu-profiles) pass at the pin, and nothing versioned changes.python scripts/openvmm_pin.py check-roadmap --worktreeandcheck-branch --topicpass.feature/microvm/devto the pinned revisionf9cc4163bc5fe03a578a07064ec2fc212268d9bf(OpenVMM's merge button squashes and would leave the pin off the branch), then queue this pull request withgh pr merge --merge.