Skip to content

Planned restricted tiers (isolated browser, typed app control, custom policies, VM): acceptance criteria #45

Open

Description

@chaodu-agent

This issue holds the acceptance criteria for the restricted tiers listed under "Planned (not yet available)" in docs/tool-profiles.md. None of these tiers may be called a boundary until its criteria hold.

The original "Now" plan (rename sandbox → desktop documented as shell-equivalent, observe profile, browser_evaluate removed from the desktop browser list, tool classification with boundary tests, docs) is done in #46, #48, #49 and #50. The original body is kept at the end. The "Visibility: menu-bar indicator + one-click revoke" item moved to #57. Profile naming is done in #59: observe and owner (see docs/tool-profiles.md).

The browser tier now sits behind the BrowserBackend interface (see #118 and docs/adr/browser-tiers.md): the router picks a backend, and an isolated tier would be one more backend. The managed Playwright backend still uses a persistent profile, so a per-grant throwaway profile is still unmet.

browser tier: isolated browser

A fresh, per-grant throwaway browser profile removes the owner's cookies but not the owner's network position: localhost web terminals (Jupyter, code-server, ttyd, dev servers with eval endpoints), services on the private network that trust a network-level identity header, and file://. Chromium's URLBlocklist has no CIDR support, so it cannot block private ranges on its own.

  • The browser's traffic goes through an egress proxy that refuses loopback, link-local, RFC 1918, CGNAT 100.64.0.0/10 and overlay-network names, and file://, or the browser runs in a VM / separate machine with no route to them.
  • A test navigates to http://127.0.0.1:<port>, a private-overlay address and file:///etc/hosts under browser and expects refusal.
  • Docs say what the tier still reaches (the public internet, as the computer's public IP).
  • The tier's browser_* tools join the same "unclassified fails" test as local tools (localToolClass / LOCAL_TOOL_CLASS), so a new Playwright tool cannot ship unclassified. browser_evaluate stays excluded.

Typed app control (replacing generic osascript for restricted tiers)

Many apps run code: Terminal, iTerm, Script Editor, Finder (.command files), Automator, Shortcuts, IDEs with integrated terminals, anything with a plugin or script host. Xcode matters most: an agent that can push a repository with a Run Script build phase and then trigger Product → Build has code execution as the desktop user. The same holds for any build or test tool pointed at a repository the agent controls (npm scripts, Makefiles, Gradle, swift build plugins).

  • The app list is a pure allowlist of bundle IDs. Nothing is allowed by being absent from a deny list.
  • Each allowed app / action is classified with the same observe / act / shell classes as local tools; "can build or run a project whose contents the agent controls" is shell, so Xcode build/run and any IDE task runner land in shell.
  • The boundary tests fail if a profile that is not shell-equivalent allows any shell-class app action.

Custom policies

  • No criteria recorded yet. Any custom policy must use the same tool classes and fail the boundary tests if it is not declared shell-equivalent but allows a shell-class tool.

VM

  • Lend a disposable macOS VM (Virtualization.framework / Tart) instead of the host. Criteria to be written when this is picked up.

Applies to every tier: egress

"No egress" in the pod means no overlay-network egress; ordinary internet egress is open by default (the network policy excludes only the overlay ranges, and the ECS tasks have public IPs). A shell-equivalent profile, or an injected observe agent, can exfiltrate to any host. Deployments that need more use a documented egress allowlist (filed on openab-pty).

Original body (2026-09-30)

Summary

The sandbox tool profile is presented as a restricted mode, but it is not a security boundary. GUI control is shell access. An agent that can type, click or run AppleScript can open a terminal, or call do shell script, and so run anything the desktop user can run. Lending a computer under sandbox therefore hands the agent the desktop user's shell for the whole lease (up to 24 h).

External review, 2026-09-30, confirmed against main:

Claimed Actual
macOS ToolProfile.sandbox "no exec*, because exec is a full shell as the desktop user" only exec* is removed; osascript (do shell script), key (type into Terminal) and mouse remain
Linux node sandbox profile names are shared with macOS identical to owner, including bash (mcp.rs: "Both profiles get everything")
sandboxBrowserTools README: "not arbitrary code in the browser process" the list includes browser_evaluate
Playwright upstream — persistent shared profile (--user-data-dir pw-profile --shared-browser-context): the agent browses with the user's logged-in cookies, and from the host's network (localhost + tailnet)

A name-filtered tool list inside one desktop session cannot draw a boundary. osascript in particular cannot be made safe by filtering script text: JXA doShellScript, ObjC.import → NSTask, run script, Terminal do script, System Events keystrokes all bypass it.

Plan

Now (this PR):

  • Drop browser_evaluate from sandboxBrowserTools (Swift + Linux). This is a bug: it contradicts the README.
  • Rename the profile to desktop. Keep accepting sandbox as a wire / persisted alias, so existing Connect / Remote builds and stored grants keep working. The doc comments must say plainly that it is shell-equivalent.
  • Adversary test: every profile declares whether it is shell-equivalent, and a test fails if a profile that says "no shell" exposes any shell-capable tool (exec*, osascript, key, mouse, bash). Today this forces desktop to be declared shell-equivalent; later it guards any new restricted profile.
  • README + reverse-attach ADR: replace "the agent already has a shell, so no exec" with the honest statement, and recommend lending a dedicated machine (e.g. a Linux hands node) rather than a primary Mac when full control is granted.
  • Linux node: accept desktop, and stop describing sandbox as narrower.

Next (separate PRs):

  • observe profile (sys_info + screenshot only) as a real boundary, and the suggested default in the grant UIs.
  • browser profile: Playwright only, with a per-grant throwaway browser profile, no browser_evaluate, and the host-network reach documented.
  • Replace generic osascript for restricted tiers with typed tools (open app / click menu item / list windows) and a bundle-ID allowlist that excludes Terminal, iTerm, Script Editor and System Settings.
  • Visibility: menu-bar indicator of the active grant + live tool-call log + one-click revoke. Optional Remote approval for osascript, or for keys sent while the front app is a terminal. Document these as friction, not a boundary.
  • Long term: lend a disposable macOS VM (Virtualization.framework / Tart) instead of the host.

Client-side copy: oablab/oab-pty-mac (linked issue).

Original body (before 2026-11-16 cleanup)

This issue holds the acceptance criteria for the restricted tiers listed under "Planned (not yet available)" in docs/tool-profiles.md. None of these tiers may be called a boundary until its criteria hold.

The original "Now" plan (rename sandbox → desktop documented as shell-equivalent, observe profile, browser_evaluate removed from the desktop browser list, tool classification with boundary tests, docs) is done in #46, #48, #49 and #50. The original body is kept at the end. The "Visibility: menu-bar indicator + one-click revoke" item moved to #57. Profile naming will follow #59 (observe / owner).

browser tier: isolated browser

A fresh, per-grant throwaway browser profile removes the owner's cookies but not the owner's network position: localhost web terminals (Jupyter, code-server, ttyd, dev servers with eval endpoints), services on the private network that trust a network-level identity header, and file://. Chromium's URLBlocklist has no CIDR support, so it cannot block private ranges on its own.

  • The browser's traffic goes through an egress proxy that refuses loopback, link-local, RFC 1918, CGNAT 100.64.0.0/10 and overlay-network names, and file://, or the browser runs in a VM / separate machine with no route to them.
  • A test navigates to http://127.0.0.1:<port>, a private-overlay address and file:///etc/hosts under browser and expects refusal.
  • Docs say what the tier still reaches (the public internet, as the computer's public IP).
  • The tier's browser_* tools join the same "unclassified fails" test as local tools (localToolClass / LOCAL_TOOL_CLASS), so a new Playwright tool cannot ship unclassified. browser_evaluate stays excluded.

Typed app control (replacing generic osascript for restricted tiers)

Many apps run code: Terminal, iTerm, Script Editor, Finder (.command files), Automator, Shortcuts, IDEs with integrated terminals, anything with a plugin or script host. Xcode matters most: an agent that can push a repository with a Run Script build phase and then trigger Product → Build has code execution as the desktop user. The same holds for any build or test tool pointed at a repository the agent controls (npm scripts, Makefiles, Gradle, swift build plugins).

  • The app list is a pure allowlist of bundle IDs. Nothing is allowed by being absent from a deny list.
  • Each allowed app / action is classified with the same observe / act / shell classes as local tools; "can build or run a project whose contents the agent controls" is shell, so Xcode build/run and any IDE task runner land in shell.
  • The boundary tests fail if a profile that is not shell-equivalent allows any shell-class app action.

Custom policies

  • No criteria recorded yet. Any custom policy must use the same tool classes and fail the boundary tests if it is not declared shell-equivalent but allows a shell-class tool.

VM

  • Lend a disposable macOS VM (Virtualization.framework / Tart) instead of the host. Criteria to be written when this is picked up.

Applies to every tier: egress

"No egress" in the pod means no overlay-network egress; ordinary internet egress is open by default (the network policy excludes only the overlay ranges, and the ECS tasks have public IPs). A shell-equivalent profile, or an injected observe agent, can exfiltrate to any host. Deployments that need more use a documented egress allowlist (filed on openab-pty).

Original body (2026-09-30)

Summary

The sandbox tool profile is presented as a restricted mode, but it is not a security boundary. GUI control is shell access. An agent that can type, click or run AppleScript can open a terminal, or call do shell script, and so run anything the desktop user can run. Lending a computer under sandbox therefore hands the agent the desktop user's shell for the whole lease (up to 24 h).

External review, 2026-09-30, confirmed against main:

Claimed Actual
macOS ToolProfile.sandbox "no exec*, because exec is a full shell as the desktop user" only exec* is removed; osascript (do shell script), key (type into Terminal) and mouse remain
Linux node sandbox profile names are shared with macOS identical to owner, including bash (mcp.rs: "Both profiles get everything")
sandboxBrowserTools README: "not arbitrary code in the browser process" the list includes browser_evaluate
Playwright upstream — persistent shared profile (--user-data-dir pw-profile --shared-browser-context): the agent browses with the user's logged-in cookies, and from the host's network (localhost + tailnet)

A name-filtered tool list inside one desktop session cannot draw a boundary. osascript in particular cannot be made safe by filtering script text: JXA doShellScript, ObjC.import → NSTask, run script, Terminal do script, System Events keystrokes all bypass it.

Plan

Now (this PR):

  • Drop browser_evaluate from sandboxBrowserTools (Swift + Linux). This is a bug: it contradicts the README.
  • Rename the profile to desktop. Keep accepting sandbox as a wire / persisted alias, so existing Connect / Remote builds and stored grants keep working. The doc comments must say plainly that it is shell-equivalent.
  • Adversary test: every profile declares whether it is shell-equivalent, and a test fails if a profile that says "no shell" exposes any shell-capable tool (exec*, osascript, key, mouse, bash). Today this forces desktop to be declared shell-equivalent; later it guards any new restricted profile.
  • README + reverse-attach ADR: replace "the agent already has a shell, so no exec" with the honest statement, and recommend lending a dedicated machine (e.g. a Linux hands node) rather than a primary Mac when full control is granted.
  • Linux node: accept desktop, and stop describing sandbox as narrower.

Next (separate PRs):

  • observe profile (sys_info + screenshot only) as a real boundary, and the suggested default in the grant UIs.
  • browser profile: Playwright only, with a per-grant throwaway browser profile, no browser_evaluate, and the host-network reach documented.
  • Replace generic osascript for restricted tiers with typed tools (open app / click menu item / list windows) and a bundle-ID allowlist that excludes Terminal, iTerm, Script Editor and System Settings.
  • Visibility: menu-bar indicator of the active grant + live tool-call log + one-click revoke. Optional Remote approval for osascript, or for keys sent while the front app is a terminal. Document these as friction, not a boundary.
  • Long term: lend a disposable macOS VM (Virtualization.framework / Tart) instead of the host.

Client-side copy: oablab/oab-pty-mac (linked issue).

Activity

  1. chaodu-agent commented on Sep 30, 2026

    @chaodu-agent
    ContributorAuthor

    Follow-up from an external review of docs/tool-profiles.md (2026-09-30). Point 1 (hand-maintained danger list) and point 4 (observe and prompt injection) are fixed / documented in #48. Points 2 and 3 concern tiers that are still planned here, so recording them as acceptance criteria — neither tier may be called a boundary until these hold.

    browser tier: a throwaway profile is necessary, not sufficient

    A fresh profile removes the owner's cookies. It does not remove the owner's network position, and on this computer that can itself be a shell:

    • localhost web terminals. Jupyter, code-server, ttyd, a dev server with an eval endpoint — browser_navigate to one of them is a shell as the desktop user.
    • Tailnet identity. Requests from this computer's browser to any service behind tailscale serve arrive stamped with the owner's Tailscale-User-Login. A service that authenticates on that header alone treats the agent as the owner. instance-mcp itself is unaffected because it requires identity and a bearer token, but other services on the tailnet may not.
    • file:// reads local files, and local HTML can run script.

    Chromium's managed URLBlocklist can block file:// and localhost, but it has no CIDR support, so it cannot block 100.64.0.0/10. Acceptance:

    • The browser's traffic goes through an egress proxy that refuses loopback, link-local, RFC 1918, CGNAT 100.64.0.0/10 / tailnet names (*.ts.net), and file:// — or the browser runs in a VM / separate machine with no route to them.
    • A test that navigates to http://127.0.0.1:<port>, a tailnet address and file:///etc/hosts under browser and expects refusal.
    • Docs say what the tier still reaches (the public internet, as the computer's public IP).

    Typed app control: a pure allowlist, and build tools count as shell

    "The allowlist must exclude Terminal, iTerm, Script Editor and System Settings" reads like a denylist inside an allowlist. Many more apps run code: Finder (opening a .command file), Automator, Shortcuts, VS Code / Cursor (integrated terminal), and anything with a plugin or script host.

    The one that matters most here is Xcode, because Xcode builds are a primary use of this tool. An agent in the pod can push a repository containing a Run Script build phase; if it can then trigger Product → Build on this Mac, that is arbitrary code execution as the desktop user. The same holds for any build or test tool the agent can point at a repository it controls (npm scripts, Makefiles, Gradle, swift build plugins).

    • The app list is a pure allowlist of bundle IDs. Nothing is allowed by being absent from a deny list.
    • Each allowed app / action is classified with the same observe / act / shell classes as local tools; "can build or run a project whose contents the agent controls" is shell, so Xcode build/run, and any IDE task runner, lands in shell.
    • The boundary tests fail if a profile that is not shell-equivalent allows any shell-class app action.
  2. chaodu-agent commented on Sep 30, 2026

    @chaodu-agent
    ContributorAuthor

    Two corrections/additions from a second review pass (docs fixed in the PR above):

    • 'No egress' means no tailnet egress; internet egress is open by default. The userspace sidecar is inbound-only, so the shell has no tailnet path (ADR fact 2), but ordinary internet egress is open — networkpolicy-no-tailnet-egress.yaml excludes only the tailnet ranges, and the ECS tasks set assignPublicIp: true. So a shell-equivalent profile (desktop) or an injected observe agent can exfiltrate to any host. The real mitigation for deployments that need it is a documented egress allowlist (model API, git, registries) — filed on openab-pty.
    • browser tool classification. localToolClass / LOCAL_TOOL_CLASS covers local tools; upstream browser_* are gated by a separate allowlist (correct today: none under observe). Acceptance for the browser tier: its tools must join the same 'unclassified fails' test, so a new Playwright tool cannot ship unclassified.
  3. chaodu-agent commented on Sep 30, 2026

    @chaodu-agent
    ContributorAuthor

    Operational follow-up on the tailnet-egress caveat: our own homelab node p1 was in exactly that shape — host tailscaled running, no NetworkPolicy — and the openab-pty session shells there could reach other tailnet machines as p1 (measured: macmini/black/rpi1 :22 reachable from both pods). Applied openab-pty's networkpolicy-no-tailnet-egress.yaml on 2026-09-30; re-measured: tailnet blocked from both pods, internet/DNS unaffected, reverse attach (inbound through the pod sidecar) still attaches, and the p1 host itself keeps its tailnet access.

  4. added
    p3Priority 3: RFC, roadmap, nice-to-have or deferred
    on Oct 30, 2026
  5. changed the title [-]sandbox profile is not a boundary: GUI control is a shell[/-] [+]Planned restricted tiers (isolated browser, typed app control, custom policies, VM): acceptance criteria[/+] on Nov 7, 2026
  6. chaodu-agent commented on Nov 7, 2026

    @chaodu-agent
    ContributorAuthor

    Retitled and rescoped: the original "Now" plan is done (#46, #48, #49, #50), and docs/tool-profiles.md points here for the planned tiers. The body now holds the acceptance criteria from the review comments above; the original body is kept in a collapsed section at the end. The "Visibility: menu-bar indicator + one-click revoke" item moved to #57. Priority stays p3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    p3Priority 3: RFC, roadmap, nice-to-have or deferred

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions