Repository navigation
openvmm: keep undelivered portb output - #1405
Merged
Pedro Henrique Penna (ppenna) merged 3 commits intoNov 9, 2026
Merged
Conversation
Pins microsoft/openvmm#4989 at 881619911d1a6ec2fec7bebdad7cfe69d09018f1, one commit on 99aac19b3, the head of feature/microvm/dev: - 881619911 chipset: keep undelivered microVM portb output The byte channel (portb) dropped all the guest output that it had buffered when a write to its host endpoint failed with an error other than a broken pipe, such as a connection reset. Its stop then wrote the output that it had kept to an endpoint without a peer, which the socket, named-pipe, and file backends discard and report as written. A snapshot taken after such a failure, or while no peer was connected, therefore saved none of that output, and the restored VM never wrote it, although the byte channel's specification saves pending output so that capture does not silently lose it (#1370). NVX's own endpoint, the pair that OpenVMM relays to stderr, also disconnects at its first failed write, so once that relay was gone, a snapshot lost the output that the guest wrote meanwhile. The pinned revision keeps the output that the endpoint has not accepted, whatever the error, for the next write, the next peer, and the snapshot, and its stop writes only to a connected endpoint. The protocol vectors, the excerpts and contract versions that nvx-rs copies from OpenVMM, and its CPU profiles match the revision's. On nvx-kvm-test, nvx-mshv-test, and nvx-whp-test (KVM, MSHV, and WHP virtual machines), the guest of release v0.1.0-dev.733a7031e655 wrote a marker to hvc0 after OpenVMM's stderr was closed, so that the byte channel's relay had exited, then requested a snapshot, which was restored. The release's own OpenVMM, at 90f9e6180, lost the marker in 3 of 3 runs on each backend. The change on 90f9e6180, 65a8e2c50, and the pinned revision both had the restored VM write it, in 3 of 3 runs each on each backend. Tests: OpenVMM's chipset tests pass, 100 of 100, on Linux and Windows, and the 5 new and changed ones fail without the change. The openvmm_vectors and openvmm_cpu_profiles comparisons pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The byte channel's specification saves pending output, so that capture does not silently lose it, but did not say which output is pending. The pinned OpenVMM revision keeps the output that the host endpoint has not accepted, while no peer is connected and after a write to the endpoint fails, whatever the error, where it used to drop the output after most write failures and to lose it at the device's stop when no peer was connected (#1370). Say so in 3.1, with the restored VM writing the saved output to its endpoint. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Map position 226, 881619911, the commit of microsoft/openvmm#4989 that the previous commit pins, after position 225, 99aac19b3, the head of feature/microvm/dev. It keeps the output that portb's host endpoint has not accepted, after a failed write with any error and at the device's stop while no peer is connected, for the next peer and the snapshot. It changes the output of the portb console of PR15's position 060, the CLI guide's sentence on that output, which PR25's position 125 adds, and position 208's test endpoint, so it is unassigned, as the other commits after the mapped tip are, and the remarks say which mapped work it revisits. Replaying it on upstream main conflicts in both of its files, which upstream main lacks or lacks the changed part of, so it follows PR15 and PR25 upstream. The pin is 95 commits past the mapped tip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block approval of the documentation updates.
No open issues 🎉
What changed in this PR
The reviewed documentation accompanies the OpenVMM fix for #1370, clarifying how undelivered microVM byte-channel output survives snapshots and recording the fix’s upstream placement.
Changes:
- Clarifies that output remains pending without a connected peer or after any write failure, and is saved for delivery after restore.
- Updates the roadmap to pin
881619911at position 226 and records its dependencies on earlier upstream work.
| File | Description |
|---|---|
doc/openvmm-upstream-roadmap.md |
Records the new pin, commit position, and upstream dependencies. |
doc/design/3.1-platform-control-device.md |
Clarifies output retention and delivery after restore. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-1370-portb-write-error
branch
November 9, 2026 03:53
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1370
What and why
The byte channel (portb, the microVM's raw output console at port
0xe9) lostguest output that a snapshot should keep, in two ways:
BrokenPipe, such asConnectionReset, cleared all the output that portb hadbuffered, up to 1 MiB. A snapshot's console flush and the device's stop both
write that output, so such a failure there saved an empty buffer, and the
restored VM never wrote those bytes, although the controller logs that
undelivered bytes stay in the snapshot. The output was lost for the
endpoint's next peer too.
The socket, named-pipe, and file backends drop their connection at any failed
write and then discard writes while reporting them as written, so stop
emptied the buffer of an endpoint that no peer had joined, or that a failed
write had disconnected, even after a broken pipe, which openvmm: a console write error drops microVM output a snapshot should keep #1370 took for safe.
NVX's own endpoint, the pair that OpenVMM relays to its stderr, is such a
backend: once that relay is gone, a snapshot lost the output that the guest
wrote meanwhile (see Validation).
microsoft/openvmm#4989,
chipset: keep undelivered microVM portb output, keepsthe output that the endpoint has not accepted, whatever the error, for the next
write, the next peer, and the snapshot, and its stop writes only to a connected
endpoint. Of the retention policies that #1370 weighs, it takes the first,
keeping the bytes on every error:
that a frontend such as portb keeps what a write fails to send, for the next
peer.
written since the last failure, and still lose it for the next peer.
channel's specification (design 3.1), the CLI guide, and the controller's
warning, under which pending output is saved.
The 1 MiB bound and its overflow policy are unchanged: a full buffer drops the
newest bytes, and guest writes never block. A failed write is retried at the
next guest write, drain request, or peer, never in a loop. A failure other than
a broken pipe now logs a rate-limited warning rather than an error, since no
output is lost.
Commits:
openvmm: pin keeping undelivered portb outputmoves the gitlink from99aac19b3, the head offeature/microvm/dev, to881619911d1a6ec2fec7bebdad7cfe69d09018f1, the head of[DO NOT REVIEW] chipset: keep undelivered microVM portb output openvmm#4989, one commit on
99aac19b3. The protocol vectors andthe CPU profiles that nvx-rs copies are unchanged.
doc: say when the byte channel's output stays pendingsays, in design 3.1,which output stays pending, and that the restored VM writes it.
doc: map keeping undelivered portb output in the OpenVMM roadmapmapsposition 226, unassigned, 95 commits past the mapped tip, with the mapped
work that it revisits: the output of PR15's portb console (position 060),
PR25's sentence on it in the CLI guide (position 125), and position 208's
test endpoint. It conflicts with upstream main in both of its files, so it
follows PR15 and PR25 upstream.
881619911is patch-equivalent to65a8e2c50, the same commit on90f9e6180,which the reproduction below tested (
openvmm_pin.py equivalence: identicalrange-diff).
OpenVMM's test changes (this pull request changes no NVX test):
Listenertest endpoint now discards writes while no peer is connectedand reports them as written, as the socket and named-pipe backends do, where
it recorded them as delivered, and it can report a reset as
ConnectionReset. The existing tests that use it never write without a peer.portb_output_error_preserves_the_next_write_wakeupnow expects the failedwrite's bytes before the next write's (
b"kept next write"), where itexpected them dropped; it still checks that the next guest write wakes the
device.
Landing order: land microsoft/openvmm#4989 first, by fast-forwarding
feature/microvm/devto881619911d1a6ec2fec7bebdad7cfe69d09018f1(OpenVMM'smerge button squashes, which would leave the pin off the branch); then merge
this pull request.
Validation
Reproduction, before and after:
90f9e6180)65a8e2c50)881619911)cargo nextest run --profile agent -p chipsetcargo nextest run --profile agent -p chipsetThe end-to-end check boots the guest of release v0.1.0-dev.733a7031e655 with
its boot console on OpenVMM's standard streams and closes OpenVMM's stderr, so
the byte channel's relay fails at the guest's next write,
NVX-1370-A, andexits. The guest then writes
NVX-1370-Bto/dev/hvc0, which finds the relaygone, requests a snapshot, and, once restored, writes
NVX-1370-C. Therelease's own OpenVMM (
90f9e6180) restored a VM that wrote onlyC; with thechange, the restored VM wrote the tail of
Athat the relay never read, thenB, thenC.With this branch's head (OpenVMM built at the pin on each host),
python3 scripts/nvx.py test-microvm --backend BACKEND --scenario console-exit --scenario console-snapshot --scenario snapshot-core --scenario snapshot-rollback --scenario lifecycle --scenario run-console-restore --scenario smp-snapshot --processors 1 2passed on nvx-kvm-test (kvm),nvx-mshv-test (mshv), and nvx-whp-test (whp), all virtual machines, as it did
before the rebase.
Three of the first five runs of console-exit with 2 vCPUs on nvx-whp-test, a
host that other sessions share, timed out: after 60 s, the restored guest had
written only 8-33 KiB of its 64 KiB to hvc0, and the 1-vCPU console-exit run
just before the first of them took 29 s, against 11 s later. Every later run
passed: 5 runs of the scenario with the change, 6 at origin/dev (
99aac19b3),and 10 replays of its restore with the change. The change alters only what a
failed write keeps and what stop writes to an endpoint without a peer; a
failed write there would end the restore with an exit error, within the exit
drain's 5 s, not stall it.
OpenVMM package checks for
chipset, on Linux and Windows:cargo check,cargo clippy --all-targets, andcargo doc --no-depspass without warnings,and nextest passes 100 of 100;
cargo xtask fmt --fixpasses.Local gates (
python scripts/local_gates.py --base origin/dev --run, Windows):test-changes (no changes to existing tests), markdown-links, test-layout,
openvmm-vectors, and openvmm-cpu-profiles pass, and
python scripts/openvmm_pin.py check-branch --topicandcheck-roadmappass.nvx-cli passes (2187 and 84 tests). openvmm-ci waits for OpenVMM's CI of the
pin, the run of microsoft/openvmm#4989.
Left to CI: the full microVM matrix, the other scenarios and processor counts,
benchmarks, and OpenVMM's CI of the pin.