Skip to content

Backend parity: capability by surface, with evidence for every cell - #1706

Merged
abrichr merged 5 commits into
mainfrom
claude/parity-audit-2297bd
Nov 3, 2026
Merged

abrichr merged 5 commits into
mainfrom
claude/parity-audit-2297bd

Conversation

@abrichr

@abrichr abrichr commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

What

Findings

  • Real third-party apps have run only in a browser (OpenEMR, Odoo, Frappe Lending, openIMIS, Medplum) and over RDP.
  • The RDP run is benchmark/rdp_avc_real: OpenEMR in Edge inside a real Windows Server 2022 session. It used a scripted Recorder and Replayer.run, not openadapt-flow run.
  • No third-party Windows desktop app has run on any surface.
  • No real Citrix session has been run.
  • Self-repair, rollback and the agent guard refuse or don't exist off the web.

Claims corrected

Where Was Now
CAPABILITIES.md, Surfaces Citrix in_session: "Same evidence as the Windows path" No run inside a Citrix session
CAPABILITIES.md, Surfaces Web: "Real app, every CI build" Real apps run locally; CI runs in-tree apps
CAPABILITIES.md, Surfaces RDP and macOS evidence Names the apps, the scripted harness, and that the macOS trials drove the backend directly
CAPABILITIES.md, Surfaces Page structure first: "Real app" Real app on the web only
CAPABILITIES.md and README Repair and rollback on every surface Repair and rollback rows say web runs
README "record, compile, run and repair work on every surface" Corrected; the Linux and RDP rows in the surfaces table are corrected too
SURFACES.md, LIMITS.md, desktop/LINUX_NATIVE.md A "required" GTK3 AT-SPI CI job That job lives in packages/flow/.github/workflows/ci.yml, which the monorepo never runs
desktop/CITRIX_PIXEL.md win32 client "mock-contract-tested only" It ran in rdp_avc_real
RECORDING_ON_WINDOWS_RDP_CITRIX.md "What has been tested" Updated
rdp_avc_real/README.md "What this proves" Says the harness bypassed the run gate

llms-full.txt is rebuilt.

Checks

  • tests/test_backend_parity.py, test_backend_parity_refusals.py, test_lookup_capabilities.py, test_validate_claims.py, test_ci_workflow_contract.py, test_backend_capability_matrix.py: 100 passed.
  • Root README and copy tests: 22 passed.

Adversarial review

A separate reviewer, given the code and evidence but not the conclusions, checked every cell. Its fixes are in 73f1374:

  • Rollback is split from undo. Rollback is refused off the web; undo propose and undo run are CODE ONLY there.
  • The surface-neutral guarded save is now in the agent-guard row.
  • One rule now covers scripted recordings.
  • Windows record and compile are marked real-VM, from windows_uia. Windows replay now shows the full tally: 4 of 10 runs accepted overall, 3 of 3 on the final candidate.
  • Shadow mode has the same verdict in every column.
  • Web approvals drop to TESTED: the person was scripted, and Cloud was a local stand-in.
  • The self-repair row separates live repair (MISSING) from offline teach (CODE ONLY).
  • Four refusals were added.
  • The Aardwolf RDP batch is now described as a Windows 11 VM.
  • The never-run job is named precisely: flow's linux-atspi-x11, not capture's AT-SPI read.

Rewrites docs/BACKEND_PARITY.md as a capability x surface matrix (web,
Windows, macOS, Linux, RDP, Citrix), each cell QUALIFIED, TESTED,
CODE ONLY or MISSING with its evidence path, and links the twelve
backend-parity issues.

Corrects claims the evidence doesn't support: Citrix in-session has no
run inside a Citrix session, the Linux GTK3 job doesn't run in the
monorepo, the macOS trials drove the backend directly, repair and
rollback run on web apps only, and the RDP numbers name their apps and
scripted harness.
Split rollback (refused off the web) from undo (code only off the web);
add the surface-neutral guarded save; one rule for scripted recordings;
Windows compile and record were run on a real VM, with the full 4 of 10
run tally; shadow mode gets the same verdict on every surface; web
approvals drop to tested (scripted person, local Cloud stand-in); live
repair vs offline teach; more refusals; the Aardwolf RDP batch was a
Windows 11 VM; flow's linux-atspi-x11 job, not capture's AT-SPI read,
is the one that never runs.
Self-repair and repair by default run outside a browser with
repair.session, qualified end to end on macOS AX and not yet on Windows
UI Automation or RDP. Teaching the fix (--hold-on-halt) and rollback
stay web-only.
@abrichr
abrichr merged commit 8d2da19 into main Nov 3, 2026
46 checks passed
@abrichr
abrichr deleted the claude/parity-audit-2297bd branch November 3, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant