[johnl-amd]

The routing report told every reader that skills beyond the room came from the
runner's own config, and to set ANTHROPIC_API_KEY so the run could use an
isolated config dir.

On a run that already had the key set and recorded isolated_config_dir: true,
sixteen extras were still reported, and none of them were present in the runner's
~/.claude/skills. They were the agent CLI's own built-ins (code-review,
dataviz, loop, run, simplify, update-config, workflow-authoring, and
so on), which no config dir excludes.

So the warning named a cause it had not checked and prescribed a fix that was
already in place.

• Isolation is already in the report metadata, so branch on it
• Isolated run: say the extras ship with the agent CLI, that nothing removes
  them, and to read the scores with them in the room
• Non-isolated run: original guidance, unchanged

Worth knowing either way, since the extras change every routing score and there
is currently no way to exclude the built-in ones.

[nowycondro]

🤖 Automated review · review-code · head e7f25aaf

Review — needs work

Full review of the whole change.
No ticket named.

Blocking

• New isolated/non-isolated warning branch has no test coverage — skillscope/routing.py:833-861 (the if meta.get("isolated_config_dir"): branch and its else)
  The change introduces new conditional behavior in render_markdown: which of two different warning messages a reader sees now depends on meta["isolated_config_dir"]. Confirmed by grep against the PR's own tests/test_skillscope.py (2505 lines at this commit) that routing.render_markdown is never invoked by name in the test suite at all — only the unrelated behavior.render_markdown is (line 2441). Neither branch of the new logic, nor the pre-existing warning path it replaces, is exercised by any test. Getting the condition backwards (e.g. an is not/boolean-sense slip) would silently re-introduce the exact bug this PR fixes, with nothing to catch it.
  Confidence 85 · logic · Fix: add a small test that builds a summary with extra_skills_in_session populated and asserts the isolated-branch wording (e.g. "ships with the agent CLI") appears when meta["isolated_config_dir"] is True, and the non-isolated wording ("come from the runner's own config") appears when it is False/absent.

Non-blocking

• docs/usage.md still frames isolation as the fix without mentioning CLI built-ins — docs/usage.md:188-192
  This paragraph tells the reader that if the runner has its own skills installed, "the report says so — set ANTHROPIC_API_KEY so the run can use an isolated config dir." That's accurate for the ~/.claude/skills case it describes, but a reader following this guidance today will still see extras after isolating (the whole premise of this PR) with nothing in the docs preparing them for that. The commit message captures the nuance well; it just didn't make it into the one place a user reads before their first run.
  Confidence 70 · architectural · Fix: add a sentence noting that even with an isolated config dir, the agent CLI's own built-in skills still show up as extras and that's expected, pointing at the runtime warning for detail.

Decisions for the author

• Message-only fix vs. filtering known built-ins out of extras — tradeoff
  The fix explains the extras rather than removing the noise: every isolated run will apparently always show a non-zero, unresolvable "extras" warning now (the CLI's own built-ins), with no way to reach a clean report. That's simple and doesn't require tracking the CLI's bundled skill list across versions, but it also means the warning stops signalling "something's wrong" for isolated runs and starts being background noise the reader learns to skip past — which could mask a genuine contamination later. Worth confirming that's the intended tradeoff versus, e.g., maintaining a known-built-ins allowlist to exclude from extras entirely.

Positive signals

• Branching on meta.get("isolated_config_dir") (already recorded by the existing run metadata) rather than re-deriving isolation state is the right call — it reuses a fact the code already has instead of guessing at it again.

Deployment notes

None — pure report-text change, no schema, config or dependency impact.

What this covered

Full diff 0f734b2d…e7f25aaf (skillscope/routing.py, the only changed file: the render_markdown warning block), read in place against the surrounding function, extras/extra_skills_in_session construction, can_isolate_config/isolate_config and the meta dict built in skillscope/cli.py, the test suite (tests/test_skillscope.py, tests/test_harness.py), and docs/usage.md/README.md for related guidance. Single-pass review (one file, +20/-5); no fan-out was needed for coverage at this size. CI (selftest, 5 checks) is green on the head commit, which is expected since no test exercises the changed lines either way.

Did not run: agent-instruction adherence (no CLAUDE.md/AGENTS.md in this repo); history (checked git log on routing.py, nothing notable beyond normal incremental development); prior-changes (earlier PR review threads on this file were not retrieved).
Not reconciled against any discussion here.

---

Not a human review. This is an automated code review; please verify findings independently before acting on them.

[johnl-amd]

Review — needs work

Full review of the whole change (single commit, single file).
No ticket named; the PR description is self-contained.

Blocking

• The new isolated/non-isolated branch has zero test coverage — skillscope/routing.py:842-854
render_markdown now renders two different user-facing warning strings depending on meta.get("isolated_config_dir"). A full-text search of tests/test_skillscope.py (2505 lines) for routing.render_markdown, routing.summarize, extra_skills_in_session, and isolated_config_dir finds no matches — the only render_markdown/summarize calls in the suite exercise the unrelated behavior module. This was also true of the single warning string the old code produced, but the branch itself is new behavior this change introduces, and as written nothing would fail if either message were swapped, garbled, or regressed back to the old wording.
  Confidence 85 · logic · Fix: add a test that builds a summary dict with extra_skills_in_session non-empty and meta["isolated_config_dir"] set to True and False, and asserts each branch's distinguishing phrase appears in the rendered output.

Non-blocking

• docs/usage.md still only describes the superseded guidance — docs/usage.md:187-192
  The doc says that if the runner has its own skills installed, "the report says so — set ANTHROPIC_API_KEY so the run can use an isolated config dir," with no mention that, once isolated, any leftover extras are the agent CLI's own built-ins that no config change removes — the exact nuance this PR adds to the generated report. A reader who only consults the docs still gets the incomplete story this PR is fixing in the report itself.
  Confidence 80 · architectural · Fix: add a line to this section mirroring the new isolated-run text.

Decisions for the author

• "no config change removes them" is stated as an absolute, in text, about the external claude CLI's behavior — tradeoff
  Tracing stage_workspace() (fresh temp workspace, only the staged routing set) and run_case() (fresh CLAUDE_CONFIG_DIR pointed at an empty temp dir when isolated) confirms the claim is accurate against how isolation works today: nothing in this repo's own skills can survive isolation, so anything left over is necessarily shipped by the CLI. But it's still a claim about another project's future behavior, not something this repo can enforce — if a future CLI build adds its own mechanism to exclude built-ins, this line would go stale silently. Stating it plainly is more useful today and matches present reality; hedging costs clarity for a future that may never arrive. Author's call.

Positive signals

• Keying the branch directly off meta.get("isolated_config_dir") — already present in the report's metadata before this PR, populated from the single routing.summarize() call site in cli.py — adds the new behavior without a new signal or any risk of the flag being absent in practice.
• The isolated-run claim was verified, not just asserted: stage_workspace() builds a fresh temp workspace containing only the routing set, and run_case() points CLAUDE_CONFIG_DIR at an empty temp dir when isolating, so an extra skill surviving both really can only be a CLI built-in.

Deployment notes

None — a text-only change to a Markdown report generator, no schema, config, or API impact.

What this covered

Read in full: skillscope/routing.py (862 lines, PR head), including render_markdown, summarize, RoutingConfig, can_isolate_config, stage_workspace, and run_case. Read the single routing.summarize() call site and its meta construction in skillscope/cli.py. Checked docs/usage.md's description of this warning. Full-text searched tests/test_skillscope.py for any coverage of this code path. Reviewed commit history on this file (3d73bc1..e7f25aa) — one prior addition of this block, one commit in this PR, nothing in between. Fanned out to one independent background worker for a blind second pass over the same diff and surrounding code; its findings (missing test coverage, confirmed non-nullability of isolated_config_dir in practice, no grammar/formatting defect) are consolidated above. CI (selftest workflow, 5 checks) is green on the head commit.
Base 0f734b2d…head e7f25aaf.

Did not run: agent-instruction adherence — no CLAUDE.md/AGENTS.md in this repository; prior-changes — no earlier reviews or prior commits touching this warning block beyond its original addition. Not reconciled against any discussion on this PR (none exists at review time, and none was read as input regardless).

---

🤖 Automated review by agent-hub on AMD AgentHub.
Model claude-sonnet-5[1m] · Not a human review. Reply here if a finding is wrong.