Skip to content

host: let a restore attach a listener to a disconnected boot console - #1744

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1179-microvm-interfaces
Nov 18, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
agents/issue-1179-microvm-interfaces

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Fixes #1179

What does this pull request change?

A program that restores a snapshot through the nvx library can attach a listener to the restored VM's boot console when the snapshot's capture left that console disconnected, as nvx snapshot capture leaves it so that a capture waits for no reader. Before, vm::create refused that restore with Unsupported: "snapshot boot console attachment policy does not match the requested console", so what a restored guest's kernel and init write to the console, such as a panic, an oops, an out-of-memory kill, or nvx-init's NVX-INIT-ACTIVATION: diagnostics, reached no one. This is the last change of the umbrella #1179: the audit of its objective below found no other gap.

Start with nvx/src/host/snapshot.rs.

  • Snapshot::accepts takes ConsoleAttachment::Listener or Disconnected for a boot console captured discard-while-disconnected, and still only Listener for one captured recreate-listener; it refuses every other combination, as before. vm::create then renders --virtio-console listen= at the VM directory's console endpoint: a Unix socket beside the restore's view of the snapshot on Linux, where OpenVMM requires a restore's console socket, or a fresh pipe in OpenVMM's console namespace on Windows. OpenVMM accepts that listener since [DO NOT REVIEW] microvm: let a restore attach a listener to a disconnected boot console openvmm#4991 (nvx-rs: let an adapter read the boot console of a template's clones #996), and the console keeps the captured device's policy: it discards the guest's output while no client is connected, so the guest never waits for a reader.
  • Documentation: the rustdoc of Snapshot::accepts and ConsoleAttachment::Listener, the nvx::host::vm and nvx::host::snapshot sections of nvx/README.md, doc/library.md, and doc/design/3.3-consoles.md, whose sentence about the clones of templates no longer described NVX.
  • Live restore fixture (the second, test-only commit): live_restored_init_activation, the ignored test in nvx/src/host/restore/tests/live_init.rs that restores a snapshot through nvx-init on an MSHV or WHP host, attached the restored console with --virtio-console connect=tcp:ADDRESS, which OpenVMM refuses for microVM consoles since microVM: keep guests from reaching loopback-TCP console listeners through host loopback #551, so it could no longer restore its snapshot. It now attaches a listener to the snapshot's disconnected boot console, reads it through nvx::host::vm::Console, and drops its NVX_SL77B_LIVE_CONSOLE input; its doc comment names the snapshot and the distro layer that it takes.

No version, ABI, protocol, snapshot format, snapshot key, or public API changes: a snapshot's key holds no console attachment, and scripts/public_api.py finds the API snapshots unchanged. The openvmm gitlink does not move.

Limitation: the live fixture's published and unpublished cases need a distro layer without /sbin/nvx-ingress, which the release's Ubuntu layer has, so they fail with it at that check of the restored workload, after the restore and the activation succeeded. The fixture's doc comment says so; this pull request leaves the workload's checks as they are.

Why is this change needed?

Root cause. #1651, the slimming workstream #1076, replaced the template and clone paths that held #996's NVX part (#1407: an adapter's console listener on a template's clones) with host::vm::create. Its new restore check, Snapshot::accepts (d2e6c81dd13), kept #1076's disposition "#996: a restored VM has a console listener when its snapshot was captured with one", and f6e313756b1 returned the modules that held #996's NVX part to their base. That disposition is superseded: #1076's body last changed on 2026-11-04, before #1407 landed #996 on 2026-11-09, when OpenVMM could not yet attach a listener to a disconnected console. OpenVMM can at the pin, and design 3.3 says that NVX's restores use it.

Rejected alternative. Capturing snapshots with a console listener instead would change what nvx snapshot capture records and leave every snapshot already captured without a readable console after a restore, while OpenVMM already supports the restore-time listener, so only NVX's check had to follow it.

Audit of #1179. The umbrella asks that the host's interfaces to a microVM, its consoles, the management RPC, activation delivery, and the errors that NVX matches, be complete, safe, and stable. Each child is closed as completed by a merged pull request on dev, and its OpenVMM change is in the pinned chain, 6e5303f94, the tip of feature/microvm/dev:

Child Closed by OpenVMM commit at the pin
#549: portb without host input, consoles on the boot console #1248, d6a8aec2792 positions 202-203, b36bc9312 and f2f7d7fe8 (microsoft/openvmm#4886, microsoft/openvmm#4887)
#972: a stable code for a forward that cannot bind #1306, a1b2485c0f7 position 206, 31fd7ee96 (microsoft/openvmm#4951)
#551: no loopback TCP console endpoints #1404, 7714de16388 position 220, 8f72e9426 (microsoft/openvmm#4988)
#1370: keep undelivered portb output #1405, 94584ae51f3 position 221, 9e0e725af (microsoft/openvmm#4989)
#996: a listener on a clone's boot console #1407, 62b53254c1f; NVX part restored here position 222, 7348c3775 (microsoft/openvmm#4991)
#760: the command line's microVM settings over the management RPC #1413, a386c22e48a position 223, b52a65518 (microsoft/openvmm#4992)
#544: a clone's activation record in its restore packet #1743, e2c662b58ae position 228, a6f10f3ba (microsoft/openvmm#4996)

How was this change validated?

Checks and results:

  • New tests fail before the fix, with the refusal above, and pass after it: snapshot_accepts_a_listener_or_none_on_a_boot_console_captured_disconnected and a_restore_attaches_a_listener_to_a_boot_console_captured_disconnected, which also checks the endpoint and the restore's snapshot path.
  • Windows, python scripts\local_gates.py --base origin/dev --run --keep-going with nvx built: all 9 selected gates passed: test-changes, markdown-links, test-layout, nvx-format, nvx-lint, nvx-test, nvx-doc, nvx-msrv, and nvx-cli (2309 tests OK, 89 skipped; 84 OK, 11 skipped). The quick gates passed again on the final head, whose only further change was the fixture's doc comment.
  • Linux, on nvx-kvm-test, in nvx/: cargo fmt --all --check, cargo clippy --locked --all-targets --all-features -- -D warnings, cargo test --locked --all-features --lib (1826 passed, 59 ignored), and cargo test --locked --lib --no-default-features (191 passed); and python3 scripts/public_api.py: all passed.
  • python3 scripts/check_test_changes.py --base origin/dev: 5 changes to existing tests, explained below, and 2 added tests.
  • A code-review pass found no issue.
  • Host runs, with dev e2c662b58 and its release v0.1.0-dev.e2c662b58ae0: a program on the public API captured a snapshot with nvx snapshot capture, then restored it with vm::create and an activation record (71 bytes) supplied through Prepared::supply_activation_record:
Backend Before After
KVM vm::create refused the listener restore 2 listener restores and 1 whose first reader left and a second connected read the console (264, 264, and 118 bytes), including NVX-INIT-ACTIVATION: the activation failed in its applied state: the deadline passed before the activation was ready, since the program asks for no Ready; each VM ended with guest-exit status 50 and a complete teardown in 30.1 s, and a disconnected restore was unchanged
MSHV the same refusal the same, at console.sock in each VM directory
WHP the same refusal the same, at pipes //./pipe/openvmm-microvm-<128-bit hex>
  • Live fixture, with the openvmm/ directory of nvx snapshot capture sl77b --memory-mib 128 --net 10.88.0.2/24 and the release's Ubuntu distro layer: on MSHV and WHP, every case restored with its console on the listener, which delivered the guest's output. missing passed (status 52, the console showing "the restore packet carried no record") and deadline passed (status 50). published and unpublished reached the workload with their activations applied and failed only at its test ! -e /sbin/nvx-ingress; without that check, both passed on MSHV with status 0.

Backend, host OS, host type, and guest: KVM and MSHV on Azure Linux 3.0, WHP on Windows, all virtual machines, with the release's kernel, Alpine initramfs, and Ubuntu distro layer.

Left to CI: the macOS check, the feature-alone Clippy runs and the MSRV check on Linux, and the jobs that the change selects on the KVM, MSHV, and WHP runners.

Checklist

  • Tests cover the new or fixed behavior, in the layout that
    python3 scripts/check_test_layout.py checks.
  • python3 scripts/check_test_changes.py --base origin/dev lists no change
    to an existing test, or a Changes to existing tests section below
    explains each one.
  • The documentation that the change affects is updated, including
    .github/copilot-instructions.md and the skills when a convention or
    workflow changes.
  • A change to the public API of nvx updates nvx-rs/public-api/ with
    python3 scripts/public_api.py --update.
  • Snapshots, protocols, and on-disk formats stay compatible, or the
    description says how the change versions or migrates them.

Changes to existing tests

  • snapshot_accepts_requires_the_boot_console_attachment_not_the_control_listener_or_device (nvx/src/host/snapshot/tests.rs) drops its assertion that accepts refuses a listener for a console captured disconnected, which this pull request makes valid on purpose; its other refusals stay, and the new snapshot_accepts_a_listener_or_none_on_a_boot_console_captured_disconnected covers the new combinations.
  • RestoreFixture::console_listener (nvx/src/host/snapshot/tests.rs) finds the boot console's attachment by its stable ID, console:microvm-virtio0, rather than by its position, so that it also serves a fixture with a NIC; it returns the same attachment for the existing fixtures.
  • The console case of snapshot_creation_checks_eligibility_and_real_host_time_before_creating_outputs (nvx/src/host/vm/tests.rs) needed a console mismatch that vm::create refuses before it creates any output. Its old mismatch, a listener for a console captured disconnected, is now accepted, so it captures a listener with console_listener() and requests a disconnected restore, which accepts still refuses; the case still checks that a refused restore creates no output.
  • live_init.rs's imports and run change as described above: the fixture attaches a listener to the restored console instead of the TCP connection that OpenVMM refuses since microVM: keep guests from reaching loopback-TCP console listeners through host loopback #551, and reads it through Console; its cases and their checks are unchanged.

A program that restores a snapshot through the nvx library could not
read the boot console of the restored VM unless the capture had
attached a listener to it. Snapshot::accepts required the restore's
console to keep the captured attachment policy, so vm::create refused
a ConsoleAttachment::Listener for a console that the capture left
disconnected, as NVX's snapshot capture leaves it so that a capture
waits for no reader: "snapshot boot console attachment policy does not
match the requested console". What a restored guest's kernel and init
write to the console, such as a panic, an oops, or an out-of-memory
kill, then reached no one.

OpenVMM lets a restore attach a listener to a disconnected boot console
since microsoft/openvmm#4991 (#996), which NVX pins, and design 3.3
says that NVX's restores use it. NVX's own use of it left with the
template clone path that the slimming workstream (#1076) replaced, and
the workstream's restore check kept the disposition that it recorded
before OpenVMM could do so: a listener only for a console captured as
one.

Snapshot::accepts now takes a listener or none for a console captured
disconnected, and still only a listener for one captured as a
listener. vm::create then renders --virtio-console listen= at the
console endpoint of the VM directory, beside the restore's view of the
snapshot on Linux, where OpenVMM requires a restore's console socket,
or at a fresh pipe in OpenVMM's console namespace on Windows. The
console keeps the captured device's policy: it discards the guest's
output while no client is connected, so the guest never waits for a
reader. No format, key, or contract changes: the snapshot's key holds
no console attachment.

The rustdoc of Snapshot::accepts and ConsoleAttachment, the crate's
README, the library guide, and design 3.3 say so.

Evidence, at dev e2c662b with release v0.1.0-dev.e2c662b58ae0 on
nvx-kvm-test (KVM), nvx-mshv-test (MSHV), and nvx-whp-test (WHP), all
virtual machines: a program on the public API restored a snapshot of
nvx snapshot capture with a listener and an activation record. Before
the change, vm::create refused it on all three. After it, two restores
with a listener and one whose first reader left and a second
connected read the guest's console through Console::connect, among it
nvx-init's NVX-INIT-ACTIVATION line when the restore's deadline passed,
and each VM exited with guest-exit status 50 and a complete teardown;
a disconnected restore was unchanged.

Tests, of which the first two fail without the change, with the
refusal above:
- snapshot_accepts_a_listener_or_none_on_a_boot_console_captured_
  disconnected;
- a_restore_attaches_a_listener_to_a_boot_console_captured_
  disconnected, which also checks the endpoint and the restore's
  snapshot path;
- snapshot_accepts_requires_the_boot_console_attachment_not_the_
  control_listener_or_device drops its refusal of that listener;
- the console case of snapshot_creation_checks_eligibility_and_real_
  host_time_before_creating_outputs refuses a disconnected restore of a
  console captured as a listener instead, and
  RestoreFixture::console_listener finds the boot console's attachment
  by its stable ID, so that it also serves a fixture with a NIC.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
live_restored_init_activation, the ignored test that restores a
snapshot through nvx-init on a real host, read the restored guest's
console from a loopback TCP listener: the restore attached the console
with --virtio-console connect=tcp:ADDRESS, at the address that the
snapshot's capture recorded. Since #551, OpenVMM refuses TCP endpoints
for microVM consoles, and a snapshot whose console recorded one,
because guests reach loopback TCP listeners through the host-loopback
mapping of their gateways, so the fixture could no longer restore its
snapshot.

The restore now attaches a listener to the snapshot's disconnected
boot console, as a capture with --virtio-console none, such as nvx
snapshot capture, leaves it: a Unix socket beside the control socket in
the snapshot's parent directory, where OpenVMM requires both, or a pipe
in OpenVMM's console namespace on Windows. A reader connects through
nvx::host::vm::Console, which checks that OpenVMM's process serves the
endpoint, as soon as OpenVMM listens, before the fixture waits for the
restore-ready event, since the console discards what the guest writes
while no client is connected, and it reads until the VM closes the
console; a VM that ends before the reader connects leaves it an empty
log, and its outcome report tells how it ended. The
NVX_SL77B_LIVE_CONSOLE input, the file that named the TCP address,
goes, and the fixture's documentation names the snapshot and the distro
layer that it takes.

Evidence, at dev e2c662b with release v0.1.0-dev.e2c662b58ae0 on
nvx-mshv-test (MSHV) and nvx-whp-test (WHP), virtual machines, with
the openvmm/ directory of nvx snapshot capture --memory-mib 128 --net
10.88.0.2/24 and the release's Ubuntu distro layer: every case
restored with its console on the listener, which delivered the guest's
output. The missing and deadline cases passed, with guest-exit status
52 and 50. The published and unpublished cases failed only at the
workload's check that the distro holds no /sbin/nvx-ingress, which the
release's Ubuntu layer holds; without that check, both passed on MSHV.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 18, 2026 06:22
@github-actions github-actions Bot added the tests-modified Changes existing tests, test data, or test wiring label Nov 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request changes existing tests

It modifies or removes 5 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

Test code

Change What Before After
modified helper fn run in host::restore::tests::live_init nvx/src/host/restore/tests/live_init.rs:163 nvx/src/host/restore/tests/live_init.rs:167
modified helper fn console_listener in host::snapshot::tests, impl RestoreFixture`` nvx/src/host/snapshot/tests.rs:1484 nvx/src/host/snapshot/tests.rs:1484
modified imports the imports of host::restore::tests::live_init nvx/src/host/restore/tests/live_init.rs:1 nvx/src/host/restore/tests/live_init.rs:1
modified test fn snapshot_accepts_requires_the_boot_console_attachment_not_the_control_listener_or_device in host::snapshot::tests nvx/src/host/snapshot/tests.rs:1935 nvx/src/host/snapshot/tests.rs:1954
modified test fn snapshot_creation_checks_eligibility_and_real_host_time_before_creating_outputs in host::vm::tests nvx/src/host/vm/tests.rs:922 nvx/src/host/vm/tests.rs:922

Relocated unchanged: 0; added: 2. Compared the merge base e2c662b58ae0 with efd08606c32b.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change enabling boot-console listeners during snapshot restore.

No open issues 🎉

What changed in this PR

Allows library callers to attach a boot-console listener when restoring a snapshot captured with the console disconnected, restoring diagnostic access after the VM API refactor.

Changes:

  • Relax snapshot compatibility checks while preserving discard-on-disconnect behavior and existing listener-only restrictions.
  • Add regression coverage and migrate the live restore fixture from TCP to local console listeners.
  • Document restore-time listeners without changing the public API, snapshot format, or OpenVMM pin.
File Description
nvx/​src/​host/​vm/​tests.rs Test restore listener arguments and endpoint placement; preserve rejection coverage.
nvx/​src/​host/​snapshot/​tests.rs Test accepted console combinations and locate fixture attachments by stable ID.
nvx/​src/​host/​snapshot.rs Accept listeners for boot consoles captured disconnected.
nvx/​src/​host/​restore/​tests/​live_init.rs Replace the TCP console with a local listener read through Console.
nvx/​src/​host/​config/​mod.rs Clarify listener output behavior after restore.
nvx/​README.md Describe restore listeners and console compatibility rules.
doc/​library.md Explain reading restored guests’ console output.
doc/​design/​3.3-consoles.md Update console design terminology for the current snapshot API.

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 18, 2026
Merged via the queue into dev with commit b10722c Nov 18, 2026
56 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1179-microvm-interfaces branch November 18, 2026 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests-modified Changes existing tests, test data, or test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenVMM/microVM umbrella: consoles, the management RPC, activation delivery, and stable error codes

2 participants