Skip to content

[Bug] DSH adapter: a transient gateway 502 on the host-LLM path is never retried — 25/96 auxiliary calls fail silently (recall filter / capture summarize) #2494

Open

Description

@dfzsxszvs

Pre-submission checklist | 提交前检查

  • I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题,确认此问题尚未被提及
  • I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
  • This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的,而不是一般软件问题

Bug Description | 问题描述

What happens

adapters/deepseek-harness/host-llm.ts → complete() issues a single prepared.stream(request) call and never retries. When the gateway in front of the Messages API transiently answers 502, the call fails immediately as

llm_unavailable: DeepSeek Messages request failed (502)

and the caller silently degrades instead of retrying. Two pipeline stages route through the host LLM, and both are affected:

stage effect of the failure
retrieval.retrieval.filter.v5 recall filtering is skipped — candidates are used unfiltered
capture.summarize the memory write proceeds without its summarization step

memory_add itself is unaffected, so memory is not lost — it is stored without the filter/summarize pass, and nothing is surfaced to the user.

What I expected: a transient 5xx from the gateway should be retried a bounded number of times (as the interactive chat path in the host does), instead of failing the auxiliary call outright.

Measured impact on this installation: 25 of 96 host LLM calls failed (26%).

tool ok fail
system_model_status (host LLM calls) 71 25
system_error (mirror row of each failure) 0 26
memory_add 23 1
memos_search 29 0

These are fast rejections, not timeouts: successful calls take 561–1677 ms, failed ones 51–372 ms (recallTimeoutMs is 3000).

How to Reproduce | 如何重现

From normal use (no special setup):

  1. Install @memtensor/memos-local-plugin 2.0.20 with hostLlmEnabled: true on DSH Desktop and use it normally. Any turn with recall candidates calls the host LLM for retrieval.retrieval.filter.v5; the end of a turn calls it for capture.summarize.

  2. Query the local log table (read-only):

    SELECT tool_name, success, json_extract(output_json, '$.message'), called_at
    FROM api_logs WHERE success = 0 ORDER BY id DESC;
  3. Every failing row is system_model_status / system_error carrying llm_unavailable: DeepSeek Messages request failed (502), in both ops — 25 of 96 host calls over ~2 days here.

Deterministic reproduction: point the host LLM route at a proxy that answers HTTP 502 for the first request of a call and 200 afterwards. The stage fails on the first attempt instead of succeeding on the second, which isolates the missing retry (no mocking beyond the 502 itself).

Timeline showing the interleaving (same op, minutes apart):

  • succeeded 08:35:08, 08:46:18, 08:50:59, 09:09:20
  • failed with the identical 502 at 08:44:15, 08:44:37, 08:49:04, 08:50:20

Environment | 环境信息

  • DSH Desktop: 0.2.0-rc.2
  • Operating System: Windows 11 (10.0.26200), Node 24.21.0
  • MemOS plugin: @memtensor/memos-local-plugin 2.0.20 (npm latest)
  • host LLM: enabled (hostLlmEnabled: true)
  • embedding provider: local (Xenova/all-MiniLM-L6-v2)

Additional Context | 其他信息

  • The message text is produced by the host, not by the plugin: it is the dsh-llm-deepseek error path that renders DeepSeek Messages request failed (<status>) when the response carries an HTTP status.
  • config.maxRetries (default 3) does not apply here: it is read by the direct-provider fetcher, not by the host-bridge path.
  • A larger recallTimeoutMs cannot help — the failure returns in tens of milliseconds.
  • Because the stage is skipped rather than surfaced, the only trace is in api_logs; users cannot tell that filtering or summarization was skipped.
  • Related but distinct: [DSH adapter] Keep slow or parameter-limited host LLM routes from causing false failures #2484 (slow / parameter-limited routes hitting deadlines) and [Bug] memos-local-plugin: host LLM fallback fires after the caller's own signal/deadline cancels the primary, then ignores that signal (runs timeoutMs + 5 s) #2473 / fix(plugin): honor cancellation in host LLM fallback #2478 (cancellation in the host LLM fallback) — same file, different mechanisms.
  • Suggested direction: a bounded retry around the single host call — at most 3 attempts, 250 ms × attempt backoff, bounded by the existing per-call deadline; retry only LLM_UNAVAILABLE with no status or status ≥ 500; never retry 401/402/403, malformed-output errors, or deadline aborts. Verified offline against a mocked stream: [502,502,ok] → 3 attempts, ok; [502,ok] → 2 attempts; [401] → 1 attempt, thrown; [502,502,502] → 3 attempts, thrown; [ok] → 1 attempt.
  • I have the diff and a regression test ready and can split this into a focused PR.

Willingness to Implement | 实现意愿

  • I'm willing to implement this myself | 我愿意自己解决
  • I would like someone else to implement this | 我希望其他人来解决

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Oct 14, 2026
  2. self-assigned this
    on Oct 14, 2026
  3. Memtensor-AI commented on Oct 14, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 6000a6d0b43d3c4f
    Working branch: bugfix/autodev-2494-20261014045802002
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. Memtensor-AI commented on Oct 14, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: 6000a6d0b43d3c4f
    Working branch: bugfix/autodev-2494-20261014045802002
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  5. Memtensor-AI commented on Oct 14, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task 6000a6d0b43d3c4f completed.

    Summary: Fixed the DSH host-LLM adapter so a transient gateway 5xx is retried instead of silently degrading the auxiliary stage (issue #2494).

    Root cause: createDeepSeekHarnessHostLlmBridge().complete() in apps/memos-local-plugin/adapters/deepseek-harness/host-llm.ts dispatched one prepared.stream() call and never retried. A transient 502 from the gateway in front of the host Messages API surfaced as a terminal finish → MemosError(LLM_UNAVAILABLE, status:502), which propagated straight out. The two host-LLM stages (retrieval.retrieval.filter.v5, capture.summarize) then degraded silently — recall filtering fell back to safeCutoff and capture summarization to the heuristic — matching the reporter's 25/96 failed calls. config.maxRetries never applied because it is consumed by the direct-provider fetcher, not the host-bridge path.

    Fix: dispatch now runs through a bounded retry loop (completeAuxiliaryWithRetry): at most 3 attempts, 250 ms × attempt backoff, bounded by the existing per-call deadline(). The classifier (isRetryableAuxiliaryError) retries only LLM_UNAVAILABLE with no status (transport failure) or status ≥ 500, and never 401/402/403, provider aborts (dshCode "ABORTED"), LLM_RATE_LIMITED, LLM_TIMEOUT, malformed output, or deadline/caller aborts. Because a prepared DSH call is single-use, each attempt re-prepares via prepareAuxiliaryCall; the model-capability lookup is cached per route, so re-preparation is a cache hit and adds no round trip. Success-path behavior and the returned HostLlmCompletion shape are unchanged.

    Tests/verification: added 13 regression tests to tests/unit/adapters/deepseek-harness-host-llm.test.ts covering the acceptance matrix ([502,502,ok]→3 ok; [502,ok]→2 ok; [502,502,502]→3 thrown; [ok]→1; [401/402/403]→1 thrown; statusless transport→2 ok; provider abort→1 thrown; RATE_LIMIT→1 thrown; malformed→1 thrown; deadline abort→1 thrown; caller abort during backoff→1). npx tsc -p tsconfig.json --noEmit is clean and npx vitest run tests/unit/adapters/deepseek-harness-host-llm.test.ts tests/unit/llm reports 121/121 passing. Broader suites that fail (better_sqlite3.node missing native binding) were confirmed pre-existing on a clean stash and are unrelated. An independent code review flagged the provider-abort retry gap, which is fixed and covered.

    Deliverables: 2 commits on bugfix/autodev-2494-20261014045802002 (892ab4c initial fix, 7bcf73a abort exclusion), pushed to origin. opsp artifacts written under .ai-tasks/ and openspec/changes/2026-10-14-2494-.../ (proposal/spec/design/verification-report) and archived to the memos-autodev-specs repo (commit 682db96). No PR created — left to the scheduler.

    Base branch: main
    Branch: bugfix/autodev-2494-20261014045802002
    Commit: 7bcf73a
    PR: #2496
    Assigned to: @CarltonXiang
    Reviewers: @whipser030, @hijzy

  6. added
    ai:testingAI agent is running tests | AI 正在运行测试
    and removed
    ai:codingAI agent is coding | AI 正在编码
    on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务ai:testingAI agent is running tests | AI 正在运行测试area:pluginOpenClaw & Hermesstatus:in-progressSomeone or AI is working on it | 人工或 AI 正在处理types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions