Skip to content

Manager route parsing rejects markdown-formatted Next: line (backtick-wrapped or inline with Task:), causing repeated empty rounds #80

Closed

Description

@liusecret000

Summary
When parse_role_manager_next_step parses the manager's plan output, a Next: cli / Next: done route that is wrapped in backticks (as shown in the manager prompt template examples) or placed on the same line as the following Task: is treated as MANAGER_NEXT_INVALID. The harness then returns "did not contain a valid route" feedback and the manager retries, causing many empty protocol-correction rounds (observed up to 10 rounds, wasting time and tokens on every run).
Environment

lh-harness 0.1.7 (lh-harness doctor all green)
Agent backend: opencode 1.18.29
Model: custom OpenAI-compatible endpoint (LiteLLM → DeepSeek API)
OS: Ubuntu 24.04

Observed behavior
With a moderately complex task, the manager output a valid route but the harness rejected it, looping up to 10 rounds before any executor/auditor round ran:

Harness feedback (same for every empty round):
    "Audit facts: the manager output did not contain a valid route, so no GUI or CLI executor can be assigned. Gap: emit one dominant GUI/CLI subtask or an explicit ask/done/blocked route. Next step: manage again using exactly Next: gui, Next: cli, Next: ask, Next: done, or Next: blocked."

The actual manager output (from provider_stream.jsonl, raw model stream) contained the route, e.g.:
    `Next: cliTask:` Read-only CLI diagnostic ...``

and later for completion:
    `Next: done`

i.e. the route line is backtick-wrapped (exactly matching the style shown in the manager system prompt: "then exactly one route: Next: gui, Next: cli, ...") and in the first case shares the line with the following Task: section.

manager_plan.txt confirmed the same content: the model output the required sections plus a route, yet next_step came back as MANAGER_NEXT_INVALID.

Root cause
parse_role_manager_next_step in role_prompts.py (line ~481) matches each line as an exact, whole-line string after strip().strip("*").replace(" ","").lower():

for line in str(text or "").splitlines():
normalized = line.strip().strip("*").replace(" ", "").replace(" ", "").lower()
normalized = re.split(r"(?:—|–|--|//|#|[((])", normalized, maxsplit=1)[0]
if normalized in {"下一步:gui任务", "下一步:gui任务", "next:gui"}:
return MANAGER_NEXT_GUI
if normalized in {"...", "next:cli"}:
return MANAGER_NEXT_CLI
...
return MANAGER_NEXT_INVALID

Because the manager system prompt itself teaches the route with backticks ("Next: cli"), models commonly emit:
backtick-wrapped routes: next:cli → normalized to next:cli ≠ next:cli
or merge the route with the next section: Next: cliTask:... `` → normalized to ``next:clitask:... ≠ next:cli
Both fail the exact-match set, so a perfectly valid route is dropped.

Reproduction
1、Create any multi-step file task (e.g. "create a config.toml and a build-note.md in output/, then verify").
2、Run lh-harness run --agent opencode --task @task.md.
3、Observe several rounds where manager is done · xx s · next=invalid before the first next=cli appears, and check runs//harness/orchestration/rounds/round_001/manager_plan.txt (backtick-wrapped route) + harness_feedback.txt ("did not contain a valid route").
4、In the completion round, a backtick-wrapped Next: done is also rejected once.

Expected behavior
parse_role_manager_next_step should be tolerant of markdown decoration and trailing commentary. Concretely: strip backticks/asterisks before comparison, and accept the route when the normalized line contains one of the route tokens (with a word boundary), e.g.:

bare = normalized.replace("`", "").replace("*", "")
if "next:gui" in bare or bare in {"下一步:gui任务", "下一步:gui任务"}:
...
This also keeps the existing Chinese-route set working.

Impact
Wasted rounds (up to 10) and tokens on every multi-round task.
Under a strict round budget the task can exhaust max_rounds purely on protocol-correction rounds.
Only affects manager role; executor/auditor pipelines work fine.

Verification of proposed fix
With the tolerant parse locally patched, the same task went from 10 empty rounds → 0 empty rounds (2 rounds / 2.4 min total), including a clean Next: done acceptance.

Activity

  1. Upper9527 commented on Oct 21, 2026

    @Upper9527
    Collaborator

    Fixed by merging #85. Backtick-wrapped routes and routes immediately followed by a Task:/任务: section are now accepted; prefix traps such as Next: clipboard and undelimited prose such as Next: done later remain invalid. The regression reproduces on the old main and passes with the patch. The full combined Python suite passed (457 passed, 1 skipped). Available on main; this does not yet indicate a new PyPI release. Thanks for the detailed reproduction.

  2. Upper9527 commented on Oct 21, 2026

    @Upper9527
    Collaborator

    The route-parsing fix from #85 is now published on PyPI in v0.1.8. Backtick-wrapped Next: routes and routes followed by a Task: / 任务: section on the same line are now accepted.

    Upgrade with pip:

    python -m pip install --upgrade lh-harness==0.1.8

    For a uv tool installation:

    uv tool upgrade lh-harness

    Verify the installed version with lh-harness --version.

    v0.1.8 release notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions