Repository navigation
host: let a restore attach a listener to a disconnected boot console - #1744
Merged
Pedro Henrique Penna (ppenna) merged 2 commits intoNov 18, 2026
Merged
Conversation
A program that restores a snapshot through the nvx library could not read the boot console of the restored VM unless the capture had attached a listener to it. Snapshot::accepts required the restore's console to keep the captured attachment policy, so vm::create refused a ConsoleAttachment::Listener for a console that the capture left disconnected, as NVX's snapshot capture leaves it so that a capture waits for no reader: "snapshot boot console attachment policy does not match the requested console". What a restored guest's kernel and init write to the console, such as a panic, an oops, or an out-of-memory kill, then reached no one. OpenVMM lets a restore attach a listener to a disconnected boot console since microsoft/openvmm#4991 (#996), which NVX pins, and design 3.3 says that NVX's restores use it. NVX's own use of it left with the template clone path that the slimming workstream (#1076) replaced, and the workstream's restore check kept the disposition that it recorded before OpenVMM could do so: a listener only for a console captured as one. Snapshot::accepts now takes a listener or none for a console captured disconnected, and still only a listener for one captured as a listener. vm::create then renders --virtio-console listen= at the console endpoint of the VM directory, beside the restore's view of the snapshot on Linux, where OpenVMM requires a restore's console socket, or at a fresh pipe in OpenVMM's console namespace on Windows. The console keeps the captured device's policy: it discards the guest's output while no client is connected, so the guest never waits for a reader. No format, key, or contract changes: the snapshot's key holds no console attachment. The rustdoc of Snapshot::accepts and ConsoleAttachment, the crate's README, the library guide, and design 3.3 say so. Evidence, at dev e2c662b with release v0.1.0-dev.e2c662b58ae0 on nvx-kvm-test (KVM), nvx-mshv-test (MSHV), and nvx-whp-test (WHP), all virtual machines: a program on the public API restored a snapshot of nvx snapshot capture with a listener and an activation record. Before the change, vm::create refused it on all three. After it, two restores with a listener and one whose first reader left and a second connected read the guest's console through Console::connect, among it nvx-init's NVX-INIT-ACTIVATION line when the restore's deadline passed, and each VM exited with guest-exit status 50 and a complete teardown; a disconnected restore was unchanged. Tests, of which the first two fail without the change, with the refusal above: - snapshot_accepts_a_listener_or_none_on_a_boot_console_captured_ disconnected; - a_restore_attaches_a_listener_to_a_boot_console_captured_ disconnected, which also checks the endpoint and the restore's snapshot path; - snapshot_accepts_requires_the_boot_console_attachment_not_the_ control_listener_or_device drops its refusal of that listener; - the console case of snapshot_creation_checks_eligibility_and_real_ host_time_before_creating_outputs refuses a disconnected restore of a console captured as a listener instead, and RestoreFixture::console_listener finds the boot console's attachment by its stable ID, so that it also serves a fixture with a NIC. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
live_restored_init_activation, the ignored test that restores a snapshot through nvx-init on a real host, read the restored guest's console from a loopback TCP listener: the restore attached the console with --virtio-console connect=tcp:ADDRESS, at the address that the snapshot's capture recorded. Since #551, OpenVMM refuses TCP endpoints for microVM consoles, and a snapshot whose console recorded one, because guests reach loopback TCP listeners through the host-loopback mapping of their gateways, so the fixture could no longer restore its snapshot. The restore now attaches a listener to the snapshot's disconnected boot console, as a capture with --virtio-console none, such as nvx snapshot capture, leaves it: a Unix socket beside the control socket in the snapshot's parent directory, where OpenVMM requires both, or a pipe in OpenVMM's console namespace on Windows. A reader connects through nvx::host::vm::Console, which checks that OpenVMM's process serves the endpoint, as soon as OpenVMM listens, before the fixture waits for the restore-ready event, since the console discards what the guest writes while no client is connected, and it reads until the VM closes the console; a VM that ends before the reader connects leaves it an empty log, and its outcome report tells how it ended. The NVX_SL77B_LIVE_CONSOLE input, the file that named the TCP address, goes, and the fixture's documentation names the snapshot and the distro layer that it takes. Evidence, at dev e2c662b with release v0.1.0-dev.e2c662b58ae0 on nvx-mshv-test (MSHV) and nvx-whp-test (WHP), virtual machines, with the openvmm/ directory of nvx snapshot capture --memory-mib 128 --net 10.88.0.2/24 and the release's Ubuntu distro layer: every case restored with its console on the listener, which delivered the guest's output. The missing and deadline cases passed, with guest-exit status 52 and 50. The published and unpublished cases failed only at the workload's check that the distro holds no /sbin/nvx-ingress, which the release's Ubuntu layer holds; without that check, both passed on MSHV. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
This pull request changes existing testsIt modifies or removes 5 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests. Test code
Relocated unchanged: 0; added: 2. Compared the merge base |
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change enabling boot-console listeners during snapshot restore.
No open issues 🎉
What changed in this PR
Allows library callers to attach a boot-console listener when restoring a snapshot captured with the console disconnected, restoring diagnostic access after the VM API refactor.
Changes:
- Relax snapshot compatibility checks while preserving discard-on-disconnect behavior and existing listener-only restrictions.
- Add regression coverage and migrate the live restore fixture from TCP to local console listeners.
- Document restore-time listeners without changing the public API, snapshot format, or OpenVMM pin.
| File | Description |
|---|---|
| nvx/src/host/vm/tests.rs | Test restore listener arguments and endpoint placement; preserve rejection coverage. |
| nvx/src/host/snapshot/tests.rs | Test accepted console combinations and locate fixture attachments by stable ID. |
| nvx/src/host/snapshot.rs | Accept listeners for boot consoles captured disconnected. |
| nvx/src/host/restore/tests/live_init.rs | Replace the TCP console with a local listener read through Console. |
| nvx/src/host/config/mod.rs | Clarify listener output behavior after restore. |
| nvx/README.md | Describe restore listeners and console compatibility rules. |
| doc/library.md | Explain reading restored guests’ console output. |
| doc/design/3.3-consoles.md | Update console design terminology for the current snapshot API. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-1179-microvm-interfaces
branch
November 18, 2026 06:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1179
What does this pull request change?
A program that restores a snapshot through the
nvxlibrary can attach a listener to the restored VM's boot console when the snapshot's capture left that console disconnected, asnvx snapshot captureleaves it so that a capture waits for no reader. Before,vm::createrefused that restore withUnsupported: "snapshot boot console attachment policy does not match the requested console", so what a restored guest's kernel and init write to the console, such as a panic, an oops, an out-of-memory kill, or nvx-init'sNVX-INIT-ACTIVATION:diagnostics, reached no one. This is the last change of the umbrella #1179: the audit of its objective below found no other gap.Start with
nvx/src/host/snapshot.rs.Snapshot::acceptstakesConsoleAttachment::ListenerorDisconnectedfor a boot console captureddiscard-while-disconnected, and still onlyListenerfor one capturedrecreate-listener; it refuses every other combination, as before.vm::createthen renders--virtio-console listen=at the VM directory's console endpoint: a Unix socket beside the restore's view of the snapshot on Linux, where OpenVMM requires a restore's console socket, or a fresh pipe in OpenVMM's console namespace on Windows. OpenVMM accepts that listener since [DO NOT REVIEW] microvm: let a restore attach a listener to a disconnected boot console openvmm#4991 (nvx-rs: let an adapter read the boot console of a template's clones #996), and the console keeps the captured device's policy: it discards the guest's output while no client is connected, so the guest never waits for a reader.Snapshot::acceptsandConsoleAttachment::Listener, thenvx::host::vmandnvx::host::snapshotsections ofnvx/README.md,doc/library.md, anddoc/design/3.3-consoles.md, whose sentence about the clones of templates no longer described NVX.live_restored_init_activation, the ignored test innvx/src/host/restore/tests/live_init.rsthat restores a snapshot through nvx-init on an MSHV or WHP host, attached the restored console with--virtio-console connect=tcp:ADDRESS, which OpenVMM refuses for microVM consoles since microVM: keep guests from reaching loopback-TCP console listeners through host loopback #551, so it could no longer restore its snapshot. It now attaches a listener to the snapshot's disconnected boot console, reads it throughnvx::host::vm::Console, and drops itsNVX_SL77B_LIVE_CONSOLEinput; its doc comment names the snapshot and the distro layer that it takes.No version, ABI, protocol, snapshot format, snapshot key, or public API changes: a snapshot's key holds no console attachment, and
scripts/public_api.pyfinds the API snapshots unchanged. Theopenvmmgitlink does not move.Limitation: the live fixture's
publishedandunpublishedcases need a distro layer without/sbin/nvx-ingress, which the release's Ubuntu layer has, so they fail with it at that check of the restored workload, after the restore and the activation succeeded. The fixture's doc comment says so; this pull request leaves the workload's checks as they are.Why is this change needed?
Root cause. #1651, the slimming workstream #1076, replaced the template and clone paths that held #996's NVX part (#1407: an adapter's console listener on a template's clones) with
host::vm::create. Its new restore check,Snapshot::accepts(d2e6c81dd13), kept #1076's disposition "#996: a restored VM has a console listener when its snapshot was captured with one", andf6e313756b1returned the modules that held #996's NVX part to their base. That disposition is superseded: #1076's body last changed on 2026-11-04, before #1407 landed #996 on 2026-11-09, when OpenVMM could not yet attach a listener to a disconnected console. OpenVMM can at the pin, and design 3.3 says that NVX's restores use it.Rejected alternative. Capturing snapshots with a console listener instead would change what
nvx snapshot capturerecords and leave every snapshot already captured without a readable console after a restore, while OpenVMM already supports the restore-time listener, so only NVX's check had to follow it.Audit of #1179. The umbrella asks that the host's interfaces to a microVM, its consoles, the management RPC, activation delivery, and the errors that NVX matches, be complete, safe, and stable. Each child is closed as completed by a merged pull request on
dev, and its OpenVMM change is in the pinned chain,6e5303f94, the tip offeature/microvm/dev:d6a8aec2792b36bc9312andf2f7d7fe8(microsoft/openvmm#4886, microsoft/openvmm#4887)a1b2485c0f731fd7ee96(microsoft/openvmm#4951)7714de163888f72e9426(microsoft/openvmm#4988)94584ae51f39e0e725af(microsoft/openvmm#4989)62b53254c1f; NVX part restored here7348c3775(microsoft/openvmm#4991)a386c22e48ab52a65518(microsoft/openvmm#4992)e2c662b58aea6f10f3ba(microsoft/openvmm#4996)c7122cd20, the pin at which this audit started, change neither the consoles, nor portb, nor the RPC.--microvm-lifecycle managedwith--microvm-workload-identity), which needs a live control console, stays on the command line, as do--microvm-report,--net-tap, which a microVM refuses, and the hidden time-ABI test hooks. NVX itself launches OpenVMM by its command line and uses no RPC.devpush run of nvx: hand a restored clone its activation record in its restore packet #1743's merge, 44245636537, passed every NVX microVM job on KVM, MSHV, and WHP, among them the guest and sandbox jobs, whosesnapshot-core,snapshot-tiers, anduntiered-network-policyscenarios restore snapshots through the CLI, which hands each restored clone its activation record; the host runs below delivered records too.nvx/srcmatches no OpenVMM error text, andscripts/match only bracketed codes,[E_PORT_FORWARD_BIND](OpenVMM: give a port forward that cannot be bound at restore a stable error code #972),[E_PROFILE_*],[E_TSC_*],[E_LAPIC_RATE_MISMATCH], and[E_DOWNTIME_EXCESSIVE], each of which OpenVMM defines at the pin.doc/acceptance-case-matrix.mdrecords the library's forward-bind attribution as rewritten to the policy-free evidence classes, withplatform-network-policymatching the code.How was this change validated?
Checks and results:
snapshot_accepts_a_listener_or_none_on_a_boot_console_captured_disconnectedanda_restore_attaches_a_listener_to_a_boot_console_captured_disconnected, which also checks the endpoint and the restore's snapshot path.python scripts\local_gates.py --base origin/dev --run --keep-goingwithnvxbuilt: all 9 selected gates passed: test-changes, markdown-links, test-layout, nvx-format, nvx-lint, nvx-test, nvx-doc, nvx-msrv, and nvx-cli (2309 tests OK, 89 skipped; 84 OK, 11 skipped). The quick gates passed again on the final head, whose only further change was the fixture's doc comment.nvx/:cargo fmt --all --check,cargo clippy --locked --all-targets --all-features -- -D warnings,cargo test --locked --all-features --lib(1826 passed, 59 ignored), andcargo test --locked --lib --no-default-features(191 passed); andpython3 scripts/public_api.py: all passed.python3 scripts/check_test_changes.py --base origin/dev: 5 changes to existing tests, explained below, and 2 added tests.deve2c662b58and its releasev0.1.0-dev.e2c662b58ae0: a program on the public API captured a snapshot withnvx snapshot capture, then restored it withvm::createand an activation record (71 bytes) supplied throughPrepared::supply_activation_record:vm::createrefused the listener restoreNVX-INIT-ACTIVATION: the activation failed in its applied state: the deadline passed before the activation was ready, since the program asks for no Ready; each VM ended with guest-exit status 50 and a complete teardown in 30.1 s, and a disconnected restore was unchangedconsole.sockin each VM directory//./pipe/openvmm-microvm-<128-bit hex>openvmm/directory ofnvx snapshot capture sl77b --memory-mib 128 --net 10.88.0.2/24and the release's Ubuntu distro layer: on MSHV and WHP, every case restored with its console on the listener, which delivered the guest's output.missingpassed (status 52, the console showing "the restore packet carried no record") anddeadlinepassed (status 50).publishedandunpublishedreached the workload with their activations applied and failed only at itstest ! -e /sbin/nvx-ingress; without that check, both passed on MSHV with status 0.Backend, host OS, host type, and guest: KVM and MSHV on Azure Linux 3.0, WHP on Windows, all virtual machines, with the release's kernel, Alpine initramfs, and Ubuntu distro layer.
Left to CI: the macOS check, the feature-alone Clippy runs and the MSRV check on Linux, and the jobs that the change selects on the KVM, MSHV, and WHP runners.
Checklist
python3 scripts/check_test_layout.pychecks.python3 scripts/check_test_changes.py --base origin/devlists no changeto an existing test, or a Changes to existing tests section below
explains each one.
.github/copilot-instructions.mdand the skills when a convention orworkflow changes.
nvxupdatesnvx-rs/public-api/withpython3 scripts/public_api.py --update.description says how the change versions or migrates them.
Changes to existing tests
snapshot_accepts_requires_the_boot_console_attachment_not_the_control_listener_or_device(nvx/src/host/snapshot/tests.rs) drops its assertion thatacceptsrefuses a listener for a console captured disconnected, which this pull request makes valid on purpose; its other refusals stay, and the newsnapshot_accepts_a_listener_or_none_on_a_boot_console_captured_disconnectedcovers the new combinations.RestoreFixture::console_listener(nvx/src/host/snapshot/tests.rs) finds the boot console's attachment by its stable ID,console:microvm-virtio0, rather than by its position, so that it also serves a fixture with a NIC; it returns the same attachment for the existing fixtures.consolecase ofsnapshot_creation_checks_eligibility_and_real_host_time_before_creating_outputs(nvx/src/host/vm/tests.rs) needed a console mismatch thatvm::createrefuses before it creates any output. Its old mismatch, a listener for a console captured disconnected, is now accepted, so it captures a listener withconsole_listener()and requests a disconnected restore, whichacceptsstill refuses; the case still checks that a refused restore creates no output.live_init.rs's imports andrunchange as described above: the fixture attaches a listener to the restored console instead of the TCP connection that OpenVMM refuses since microVM: keep guests from reaching loopback-TCP console listeners through host loopback #551, and reads it throughConsole; its cases and their checks are unchanged.