You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add an opt-in agent backend over a native sandbox library - #418
Replaces #395, and now also carries #419, #420, #421, and #422, which this revision folds in and closes. The sandbox lifecycle moves out of this repository: the opt-in agent backend becomes a thin client of a separately supplied native library, aci_edge_agent, that owns the lifecycle, and the data model moves into its own crate so that both sides of the library's ABI compile the same definitions. See "Changes since the previous revision".
Summary
Model crate. The edge sandbox crate's serializable requests, results, capabilities, errors, sandbox IDs, and request validation move into a model crate, in the crate's model directory, with no backend. The crate re-exports every type, so its public API is unchanged, and its CI checks now cover both packages.
Setup, sandbox spec, and native ABI types. A native host takes a SetupConfig once, when it opens: the runtime bundle or files, the state root, the hypervisor, the CPU profile, timeouts, the guest session policy, flow control of streamed executions, diagnostics, and the defaults that sandboxes inherit. A SandboxSpec carries the per-sandbox settings that the provision request does not define: the image, resources, guest network, hostname, and host loopback forwards. The provision request gains runtimeConfig.networkProxy. The client and Backend gain provision_with, and capabilities report which spec fields and network features a backend honors, so validation rejects what a backend does not support. A backend that advertises no spec support, such as the direct OpenVMM backend, accepts only an empty spec.
agent backend (opt-in). A thin client of aci_edge_agent.dll (libaci_edge_agent.so on Linux), which the caller supplies with an independently approved SHA-256. It loads the library once per process after checking that digest and the sandbox ABI version, and it loads the bytes that it checked, so a writer cannot swap the file in between. It opens a host with a SetupConfig, and forwards each Backend operation through the sandbox ABI, whose aci_edge_* exports pass the model's JSON. The library owns the state root and its records, the OpenVMM processes and boot consoles, the guest sessions, and the image registry. Later backends for a loaded library and setup open without touching a file, so a caller may create one per operation.
Streamed executions. Output arrives as raw bytes while the command runs, with no size limit; one pump thread copies it into the sinks, and a failed sink discards only its own stream. Piped stdin, cancellation, and timeouts in whole milliseconds up to an hour are supported, and cancellation and timeouts end the whole process tree. A stream that the caller holds without reading cannot delay a timeout. If reading the caller's stdin fails, the workload is killed and the execution fails. Up to 16 executions run side by side on one sandbox.
Linux process liveness. A zombie OpenVMM main thread counts as running until the process's other threads exit, as a Windows process runs until its last thread exits, so a stop no longer returns while those threads still hold OpenVMM's descriptors.
Direct backend. Its guest network check is now the model's, which also rejects the network, broadcast, and gateway addresses that OpenVMM refuses. Otherwise its behavior is unchanged.
Tests and docs. Guest tests run the backend end to end against a built library. Unit tests run executions against a test double of the library, including a failed event, a failed stdin read, and an undrained stream at a timeout. An ignored library test checks, without a hypervisor, that a separately built library binds the sandbox ABI. An example walks through the lifecycle, and the README documents the backend, its setup, images, host paths and network policies, the proxy and forwarded ports, working directories and environments, and streamed executions.
The native library and its guest are built separately and are not included in this public repository. A caller-supplied digest alone is not an authenticated installation: callers must approve and protect the installed library independently.
Changes since the previous revision
Rebased onto dev. The openvmm pin moves to nvx: prototype nested KVM guests #121 rebased onto dev's pin, and the crate's CI action keeps dev's toolchain selection while checking the whole workspace, which now includes the model crate. The crate's tree is unchanged at every commit.
The lifecycle moves out of the crate into the native library: the state store, the OpenVMM processes and consoles, the guest sessions, the image registry, host paths, egress rules, the proxy, forwarded ports, and CPU profiles. The direct backend's files return to their dev form, apart from the liveness fix, the guest network check, and Hypervisor, which moves into the model.
AgentConfig holds the setup, the library path, and the library's approved SHA-256, instead of the OpenVMM configuration and the image.
Executions stream instead of returning buffered output. Piped stdin, cancellation, and millisecond timeouts are supported, and the output limit is gone.
Review fixes since 7ed52d8:
A failed event or stdin read now kills the workload before the input worker is joined, so the execution ends.
Flow-control windows whose sum overflows are rejected.
{"resources": {}} is an empty spec.
Image references reject any whitespace or control character.
The example stops a sandbox whose start failed.
The README states that resources.vcpus may only be 1.
Since 6c5d614, the backend and its library are renamed, with no change in behavior or in the ABI's JSON:
The feature and module are agent, the types AgentBackend and AgentConfig, and the constructor AciEdgeSandbox::agent.
The library is aci_edge_agent.dll (libaci_edge_agent.so), and the backend binds its aci_edge_* exports.
The guest tests and the example are agent_guest and agent_lifecycle, and they read EDGE_AGENT_TEST_* variables.
A guest test now checks the library's actual sandbox directory after deprovision.
Since 6bebbe1, the example checks the guest log for the exec: line that a streamed execution writes. It looked for an execute: line that the guest no longer writes, so it reported an error after the command had run.
Review fixes since fc06c99:
The backend loads the bytes that it checked. On Linux it loads them from a sealed memfd, and the load fails if the kernel refuses to create or execute one. On Windows, a handle that denies writes and deletes keeps the file and its directories in place until the library is loaded. Other hosts cannot load the library.
If a timed execution's outcome has not arrived when its timeout elapses, wait closes the output streams, as cancellation does. A stream that the caller holds without reading therefore cannot delay the outcome, and an execution that loses output this way ends as TimedOut. The deadline uses the whole milliseconds that the library receives.
A malformed sandbox token from the library is a backend_error, not the caller's malformed_id.
The provision and exec envelopes borrow the request, so they copy nothing. They stop compiling when the model's wire inputs gain a field, and a test checks that each one deserializes as the model's type.
Review fixes since 199aa7a:
exec_next returns a backend_error when the library reports an execution event with a nonzero length and a null data pointer, instead of building a slice from it.
A capability set without network.networkProxy parses as it does without spec, with the proxy unsupported, so older payloads still deserialize.
The Copilot Autofix commit d66e6db is folded into commit 4 in the form that cargo fmt expects, which its separate commit did not use. image.path must now be valid UTF-8, as well as absolute and free of NUL characters.
Validation
Each commit that changes the crate (2–5) passes its CI checks on Windows with Rust 1.95.0, the release that dev pins: fmt; Clippy with -D warnings on all targets with --all-features, and on the library with --no-default-features; tests with --all-features (163, 163, 183, and 193 tests across commits 2–5) and with default features; docs with -D warnings; and cargo +1.89 check.
The head also passes the bundled artifact tests on Windows, and on Linux the same checks plus the macOS build check.
The ignored library test passes against freshly built aci_edge_agent.dll and libaci_edge_agent.so.
Windows/WHP on an AMD EPYC 7763, which amd.milan.v1 serves: in 125 s.
Linux/MSHV on an AMD EPYC 9V74, with the host CPU profile: in 105 s.
openvmm_e2e (7 tests) passes with dev's release guest (v0.1.0-dev.a399b7b192b7) on WHP, and on MSHV. On MSHV it ran with a local launch patch for the host CPU profile, because auto selects amd.genoa.v1, which that hypervisor cannot present; and as root with a world-writable umask, because the suite expects an ordinary caller and /dev/mshv is root-only on that host.
The example runs a command end to end with the same builds: on WHP, and on MSHV with --hypervisor mshv --cpu-profile host. On MSHV, its process maps the library only from the sealed memfd.
Interleaved benchmark rounds on MSHV against fc06c99, 30 lifecycles a side, pass the performance gate. Medians: /bin/true exec 1.48 ms against 1.43 ms, and stop 168 ms against 161 ms; stop varies by about ±40 ms with the OpenVMM process's exit. The first backend of a process still takes about 0.73 s.
Interleaved benchmark rounds against the previous revision's in-crate lifecycle show no regression:
Median /bin/true exec latency drops from 5.1 to 2.6 ms on WHP and from 1.8 to 1.3 ms on MSHV.
Creating a backend for each call adds about 18 µs on WHP and 118 µs on MSHV.
100 MiB of output streams in 3.0–3.9 s on WHP and in about 1.9 s on MSHV.
Notes
AgentConfig is #[non_exhaustive]: construct it with AgentConfig::new and its builder methods.
The library does not honor image references or hostnames in a SandboxSpec yet, and Capabilities::spec reports them as unsupported.
Every guest has one virtual processor, so the library accepts resources.vcpus only as 1.
Enrique Saurez (esaurez)
changed the title
Add opt-in native-host image-backed edge lifecycle
Add an opt-in nvxhost backend over a native sandbox library
Oct 7, 2026
Enrique Saurez (esaurez)
changed the title
Add an opt-in nvxhost backend over a native sandbox library
Add an opt-in agent backend over a native sandbox library
Oct 8, 2026
The edge runtime boots a sandbox from one read-only image with a RAM-backed
overlay upper and no scratch disk. OpenVMM accepts that layout from
nanvix/openvmm#121, rebased here onto the previous pin.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux the main thread of OpenVMM can exit and leave a zombie while its other
threads still exit. Liveness checks treated that zombie as gone, so a stop
could return while those threads still held OpenVMM's descriptors, and an
immediate restart could collide with them. A zombie now counts as running
until its other threads exit, as a Windows process runs until its last thread
exits.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The edge sandbox crate's serializable requests, results, capabilities,
errors, sandbox IDs, and request validation move into a model crate with
no backend. A backend implemented outside the crate can then read and
write the same JSON from the same definitions. The crate re-exports every
type, so its public API is unchanged, and it becomes a workspace whose
checks now cover both packages.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Parsing from the left treats every IPv6 colon as a field separator, so a documented rule such as --egress-allow 2001:db8::/32 is rejected even though the policy model supports IPv6. Parse an optional protocol/port suffix from the right and otherwise keep the full CIDR.
A native host takes a SetupConfig once, when it opens: the runtime bundle
or files, the state root, the hypervisor, the CPU profile, timeouts, the
guest session policy, flow control of streamed executions, diagnostics,
and the defaults that sandboxes inherit. A SandboxSpec carries the
per-sandbox creation settings that the provision request does not define:
the image, resources, guest network, hostname, and host loopback forwards.
The provision request gains runtimeConfig.networkProxy, which routes the
guest's traffic through a proxy on host loopback. The client and the
Backend trait gain provision_with, which takes the spec, and capabilities
report which spec fields and network features a backend honors, so
validation rejects a request or spec that the backend does not support.
The image types, execution outcomes, and Hypervisor move into the model
with serde forms, and the wire module defines the envelopes of the native
ABI.
The direct OpenVMM backend checks its guest network with the model's check,
which also rejects the network, broadcast, and gateway addresses that
OpenVMM refuses.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The agent feature adds a backend that is a thin client of a separately
supplied native library, aci_edge_agent, which owns the whole sandbox
lifecycle: the state root and its records, the OpenVMM processes and boot
consoles, the guest sessions, and the image registry. The backend loads the
library once per process after checking its SHA-256 and sandbox ABI version,
opens a host with a SetupConfig, and forwards each Backend operation through
the sandbox ABI, whose aci_edge_* exports pass the JSON of the model crate.
It loads the bytes that it checked: on Linux from a sealed in-memory copy,
and on Windows while its handle keeps the file from being changed, renamed,
or deleted; other hosts cannot load it. A later backend for a loaded library
finds it by its requested path without touching the file.
Executions stream: output arrives as raw bytes while the command runs, with
no size limit, and one pump thread copies it into the sinks; a failed sink
discards only its own stream. A stdin worker feeds piped input once the
library accepts it; a failed read of that input kills the workload and fails
the execution. Cancellation and timeouts in whole milliseconds end the whole
process tree. Once a timeout elapses, waiting closes the output streams, so
that a stream that the caller holds without reading cannot delay the
outcome; an execution that loses output that way ends as TimedOut.
AgentBackend takes AgentConfig {setup, library, library_sha256}, and images,
guest logs, and diagnostics stay available through it.
Guest tests run the backend end to end against a built library, and unit
tests drive executions through input and library failures and an undrained
stream at a timeout with a test double of the library. An ignored library
test checks that a separately built library binds the sandbox ABI, and an
example walks through the lifecycle.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #395, and now also carries #419, #420, #421, and #422, which this revision folds in and closes. The sandbox lifecycle moves out of this repository: the opt-in
agentbackend becomes a thin client of a separately supplied native library,aci_edge_agent, that owns the lifecycle, and the data model moves into its own crate so that both sides of the library's ABI compile the same definitions. See "Changes since the previous revision".Summary
modeldirectory, with no backend. The crate re-exports every type, so its public API is unchanged, and its CI checks now cover both packages.SetupConfigonce, when it opens: the runtime bundle or files, the state root, the hypervisor, the CPU profile, timeouts, the guest session policy, flow control of streamed executions, diagnostics, and the defaults that sandboxes inherit. ASandboxSpeccarries the per-sandbox settings that the provision request does not define: the image, resources, guest network, hostname, and host loopback forwards. The provision request gainsruntimeConfig.networkProxy. The client andBackendgainprovision_with, and capabilities report which spec fields and network features a backend honors, so validation rejects what a backend does not support. A backend that advertises no spec support, such as the direct OpenVMM backend, accepts only an empty spec.agentbackend (opt-in). A thin client ofaci_edge_agent.dll(libaci_edge_agent.soon Linux), which the caller supplies with an independently approved SHA-256. It loads the library once per process after checking that digest and the sandbox ABI version, and it loads the bytes that it checked, so a writer cannot swap the file in between. It opens a host with aSetupConfig, and forwards eachBackendoperation through the sandbox ABI, whoseaci_edge_*exports pass the model's JSON. The library owns the state root and its records, the OpenVMM processes and boot consoles, the guest sessions, and the image registry. Later backends for a loaded library and setup open without touching a file, so a caller may create one per operation.Commits
Dependencies
96e2363, the OpenVMM revision thatdevpins and nanvix/openvmm'smain. This branch pinsf12dcf5e8f2ea60cc15899dd16852320530670bd.Changes since the previous revision
dev. Theopenvmmpin moves to nvx: prototype nested KVM guests #121 rebased ontodev's pin, and the crate's CI action keepsdev's toolchain selection while checking the whole workspace, which now includes the model crate. The crate's tree is unchanged at every commit.devform, apart from the liveness fix, the guest network check, andHypervisor, which moves into the model.runtimeConfig.networkProxy, host loopback forwards, the guest network, and the CPU profile), and their guest tests are part of the end-to-end suite.AgentConfigholds the setup, the library path, and the library's approved SHA-256, instead of the OpenVMM configuration and the image.7ed52d8:{"resources": {}}is an empty spec.resources.vcpusmay only be 1.6c5d614, the backend and its library are renamed, with no change in behavior or in the ABI's JSON:agent, the typesAgentBackendandAgentConfig, and the constructorAciEdgeSandbox::agent.aci_edge_agent.dll(libaci_edge_agent.so), and the backend binds itsaci_edge_*exports.agent_guestandagent_lifecycle, and they readEDGE_AGENT_TEST_*variables.6bebbe1, the example checks the guest log for theexec:line that a streamed execution writes. It looked for anexecute:line that the guest no longer writes, so it reported an error after the command had run.fc06c99:waitcloses the output streams, as cancellation does. A stream that the caller holds without reading therefore cannot delay the outcome, and an execution that loses output this way ends asTimedOut. The deadline uses the whole milliseconds that the library receives.backend_error, not the caller'smalformed_id.199aa7a:exec_nextreturns abackend_errorwhen the library reports an execution event with a nonzero length and a null data pointer, instead of building a slice from it.network.networkProxyparses as it does withoutspec, with the proxy unsupported, so older payloads still deserialize.openvmmpin moves to microVM: opt in to read-only distro with RAM overlay upper nanvix/openvmm#121 atf12dcf5, which also rejects a barenvx_overlay_uppertoken and documents the RAM-backed overlay layout in the OpenVMM Guide.d66e6dbis folded into commit 4 in the form thatcargo fmtexpects, which its separate commit did not use.image.pathmust now be valid UTF-8, as well as absolute and free of NUL characters.Validation
devpins: fmt; Clippy with-D warningson all targets with--all-features, and on the library with--no-default-features; tests with--all-features(163, 163, 183, and 193 tests across commits 2–5) and with default features; docs with-D warnings; andcargo +1.89 check.bundledartifact tests on Windows, and on Linux the same checks plus the macOS build check.aci_edge_agent.dllandlibaci_edge_agent.so.--release --test-threads=1) passes at the head against a freshly built library and an OpenVMM built from microVM: opt in to read-only distro with RAM overlay upper nanvix/openvmm#121:amd.milan.v1serves: in 125 s.openvmm_e2e(7 tests) passes withdev's release guest (v0.1.0-dev.a399b7b192b7) on WHP, and on MSHV. On MSHV it ran with a local launch patch for the host CPU profile, becauseautoselectsamd.genoa.v1, which that hypervisor cannot present; and as root with a world-writable umask, because the suite expects an ordinary caller and/dev/mshvis root-only on that host.--hypervisor mshv --cpu-profile host. On MSHV, its process maps the library only from the sealed memfd.fc06c99, 30 lifecycles a side, pass the performance gate. Medians:/bin/trueexec 1.48 ms against 1.43 ms, and stop 168 ms against 161 ms; stop varies by about ±40 ms with the OpenVMM process's exit. The first backend of a process still takes about 0.73 s./bin/trueexec latency drops from 5.1 to 2.6 ms on WHP and from 1.8 to 1.3 ms on MSHV.Notes
AgentConfigis#[non_exhaustive]: construct it withAgentConfig::newand its builder methods.SandboxSpecyet, andCapabilities::specreports them as unsupported.resources.vcpusonly as 1.