Skip to content

The test setup leaves CLAUDE_CONFIG_DIR and the passwd home outside the fixture, and OPENRIG_E2E_REAL_CODEX=1 turns off all home isolation for the whole run #1556

Description

@AdamWhitehurst

What happens

The daemon and cli suites share a setup (test/hermetic-env.setup.ts) that gives each test file a fixture HOME, deletes CODEX_HOME, sets XDG_*, and puts stub codex and claude first on PATH. With it in place I found no test that writes to the real ~/.codex or ~/.claude. Three gaps remain, each of which can reach a real home on some machine or run.

1. CLAUDE_CONFIG_DIR is not scrubbed. The setup never mentions it, so code that honors it uses whatever the runner exports. provisionClaudeState (packages/daemon/src/adapters/claude-code-adapter.ts:841-867) merges onboarding and trust flags into <CLAUDE_CONFIG_DIR>/.claude.json when the managed launch carries the variable, which createDaemon takes from process.env.CLAUDE_CONFIG_DIR (packages/daemon/src/startup.ts:647). It is also read at startup.ts:533 and :1019. prepareHermeticEnv (packages/daemon/test/helpers/hermetic-env.ts:358-376) doesn't scrub it either.

2. OPENRIG_E2E_REAL_CODEX=1 turns the isolation off for every test, not just the two that need it. hermetic-env.setup.ts:117 wraps the whole HOME, CODEX_HOME, XDG_* and stub block in if (process.env["OPENRIG_E2E_REAL_CODEX"] !== "1"). With the variable left set, the whole run uses the real home and binaries: createDaemon tests (about 35 files) copy skills into ~/.claude/skills and ~/.agents/skills (startup.ts:915-935), Codex-node tests write ~/.codex/config.toml (codex-runtime-adapter.ts:193-285), and the real codex and claude run. The two guards that would notice (stub-runtimes-on-path.test.ts:12, fixture-home-cleanup.test.ts:32) are skipIf in exactly that mode.

3. Some code reads the passwd home, not HOME. codex-thread-id.ts:260-268 adds os.userInfo().homedir to its candidate homes and uses it as the default (:178). It lists <real home>/.codex and opens its logs_N.sqlite and state_N.sqlite read-only (:298, :336). The HOME scrub doesn't affect it.

A smaller one: startup.test.ts:431 and :468 replace PATH without the stub dir and call createDaemon, which runs claude --help (startup.ts:2495-2498). A real claude in /usr/bin or /bin would be the one that runs; it only reads help.

Evidence

From source only, on upstream/main df8238f5. The HOME scrub landed 2026-10-05 (#780) and the runtime stubs 2026-10-14 (#1298).

Who it affects

Anyone who runs the suites with CLAUDE_CONFIG_DIR exported (a non-default Claude config dir, or an agent seat whose environment carries it), or who sets OPENRIG_E2E_REAL_CODEX=1 for the two e2e tests and runs the rest of the suite in the same shell. The second is the one that writes to the real home today, and it takes one stray variable.

Possible direction (untested)

  1. Delete CLAUDE_CONFIG_DIR (or point it into the fixture) in the setup and in prepareHermeticEnv.
  2. Make the real-Codex opt-in apply to the two e2e files only, or fail loudly when it's set and any other file loads.
  3. Assert, beside assertFixtureScopedHome, that os.homedir() is the fixture and CODEX_HOME/CLAUDE_CONFIG_DIR are unset or fixture-scoped.
  4. Make CodexThreadIdResolver's default home follow HOME, or require it.
  5. Keep the stub dir in the two startup.test.ts cases that rewrite PATH.

What I did not observe

  • Any run: nothing was executed, so every statement is from reading the source.
  • Whether a read-only SQLite open of a WAL database creates -shm/-wal files; if it does, gap 3 is also a write.
  • packages/tui, packages/ui, packages/test-system, and Docker/staging tests weren't covered.

Activity

  1. Aschenbath commented on Oct 20, 2026

    @Aschenbath
    Contributor

    I am taking this on. I will keep ordinary suites fixture-scoped even when the real-Codex opt-in is exported, scope that exception to the two native e2e files, scrub Claude config overrides in both shared test environments, and prevent default Codex thread discovery from consulting the passwd home behind an explicit HOME. I will add deterministic regressions with synthetic homes and runtime stubs, and run build, all-package typechecks and the Vitest suites before opening one PR.

  2. moved this from Todo to In Progress in OpenRig roadmapon Oct 20, 2026
  3. vedantb7 commented on Oct 20, 2026

    @vedantb7

    I’ve implemented a minimal fix for the identified test-hermeticity gaps and opened a PR.

    The change:

    • prevents CLAUDE_CONFIG_DIR from leaking into hermetic test environments,
    • restricts OPENRIG_E2E_REAL_CODEX=1 to the two intended Codex E2E tests, and
    • ensures the startup tests resolve claude through the existing test stub.

    The changes are limited to test/setup infrastructure; no production code was modified.

  4. mvschwarz commented on Oct 20, 2026

    @mvschwarz
    Owner

    Thanks, @Aschenbath, for taking this on, and @vedantb7, for #1571. An update for you, @Aschenbath: since your comment, @vedantb7 has opened #1571. As it describes itself, it stops CLAUDE_CONFIG_DIR leaking into the test environments, limits OPENRIG_E2E_REAL_CODEX=1 to the two Codex e2e tests, and has the startup tests use the existing claude stub. It changes only test setup, so the part of your plan about Codex thread discovery and the passwd home isn't in it.

    Could you say here whether you still plan to take this on, and what your plan adds beyond #1571? If yours goes further, a PR for the remaining part is welcome. @vedantb7, we're looking at #1571 now. We'll look at each change on its own merits, and we'll leave #1571 unmerged until @Aschenbath has answered here or 48 hours have passed since this comment. After that, #1571 carries on as usual.

  5. Aschenbath commented on Oct 20, 2026

    @Aschenbath
    Contributor

    I am continuing this in #1586 and addressing its daemon CI failure. Beyond #1571, this PR makes default Codex thread discovery honor HOME instead of consulting the passwd home, with synthetic-home SQLite regressions, and keeps scenario children isolated from inherited provider overrides. I will validate the follow-up with the full build, typecheck and test suites.

  6. Aschenbath commented on Oct 20, 2026

    @Aschenbath
    Contributor

    I am continuing the review follow-up in #1586. I will preserve provider roots in an explicitly supplied baseEnv, scrub them only from the inherited default, split the scaffold regression accordingly, document the native tests’ per-file isolation requirement, and include the parent-session integration test plus full build/typecheck/test validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions