Repository navigation
[Bug] DSH adapter: a transient gateway 502 on the host-LLM path is never retried — 25/96 auxiliary calls fail silently (recall filter / capture summarize) #2494
Description
Pre-submission checklist | 提交前检查
- I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题,确认此问题尚未被提及
- I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
- This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的,而不是一般软件问题
Bug Description | 问题描述
What happens
adapters/deepseek-harness/host-llm.ts → complete() issues a single prepared.stream(request) call and never retries. When the gateway in front of the Messages API transiently answers 502, the call fails immediately as
llm_unavailable: DeepSeek Messages request failed (502)
and the caller silently degrades instead of retrying. Two pipeline stages route through the host LLM, and both are affected:
| stage | effect of the failure |
|---|---|
retrieval.retrieval.filter.v5 |
recall filtering is skipped — candidates are used unfiltered |
capture.summarize |
the memory write proceeds without its summarization step |
memory_add itself is unaffected, so memory is not lost — it is stored without the filter/summarize pass, and nothing is surfaced to the user.
What I expected: a transient 5xx from the gateway should be retried a bounded number of times (as the interactive chat path in the host does), instead of failing the auxiliary call outright.
Measured impact on this installation: 25 of 96 host LLM calls failed (26%).
| tool | ok | fail |
|---|---|---|
system_model_status (host LLM calls) |
71 | 25 |
system_error (mirror row of each failure) |
0 | 26 |
memory_add |
23 | 1 |
memos_search |
29 | 0 |
These are fast rejections, not timeouts: successful calls take 561–1677 ms, failed ones 51–372 ms (recallTimeoutMs is 3000).
How to Reproduce | 如何重现
From normal use (no special setup):
-
Install
@memtensor/memos-local-plugin2.0.20 withhostLlmEnabled: trueon DSH Desktop and use it normally. Any turn with recall candidates calls the host LLM forretrieval.retrieval.filter.v5; the end of a turn calls it forcapture.summarize. -
Query the local log table (read-only):
SELECT tool_name, success, json_extract(output_json, '$.message'), called_at FROM api_logs WHERE success = 0 ORDER BY id DESC;
-
Every failing row is
system_model_status/system_errorcarryingllm_unavailable: DeepSeek Messages request failed (502), in both ops — 25 of 96 host calls over ~2 days here.
Deterministic reproduction: point the host LLM route at a proxy that answers HTTP 502 for the first request of a call and 200 afterwards. The stage fails on the first attempt instead of succeeding on the second, which isolates the missing retry (no mocking beyond the 502 itself).
Timeline showing the interleaving (same op, minutes apart):
- succeeded 08:35:08, 08:46:18, 08:50:59, 09:09:20
- failed with the identical 502 at 08:44:15, 08:44:37, 08:49:04, 08:50:20
Environment | 环境信息
- DSH Desktop: 0.2.0-rc.2
- Operating System: Windows 11 (10.0.26200), Node 24.21.0
- MemOS plugin: @memtensor/memos-local-plugin 2.0.20 (npm latest)
- host LLM: enabled (hostLlmEnabled: true)
- embedding provider: local (Xenova/all-MiniLM-L6-v2)
Additional Context | 其他信息
- The message text is produced by the host, not by the plugin: it is the
dsh-llm-deepseekerror path that rendersDeepSeek Messages request failed (<status>)when the response carries an HTTP status. config.maxRetries(default 3) does not apply here: it is read by the direct-provider fetcher, not by the host-bridge path.- A larger
recallTimeoutMscannot help — the failure returns in tens of milliseconds. - Because the stage is skipped rather than surfaced, the only trace is in
api_logs; users cannot tell that filtering or summarization was skipped. - Related but distinct: [DSH adapter] Keep slow or parameter-limited host LLM routes from causing false failures #2484 (slow / parameter-limited routes hitting deadlines) and [Bug] memos-local-plugin: host LLM fallback fires after the caller's own signal/deadline cancels the primary, then ignores that signal (runs timeoutMs + 5 s) #2473 / fix(plugin): honor cancellation in host LLM fallback #2478 (cancellation in the host LLM fallback) — same file, different mechanisms.
- Suggested direction: a bounded retry around the single host call — at most 3 attempts,
250 ms × attemptbackoff, bounded by the existing per-call deadline; retry onlyLLM_UNAVAILABLEwith no status or status ≥ 500; never retry 401/402/403, malformed-output errors, or deadline aborts. Verified offline against a mocked stream:[502,502,ok] → 3 attempts, ok;[502,ok] → 2 attempts;[401] → 1 attempt, thrown;[502,502,502] → 3 attempts, thrown;[ok] → 1 attempt. - I have the diff and a regression test ready and can split this into a focused PR.
Willingness to Implement | 实现意愿
- I'm willing to implement this myself | 我愿意自己解决
- I would like someone else to implement this | 我希望其他人来解决
Activity
Memtensor-AI commentedon Oct 14, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
6000a6d0b43d3c4fWorking branch:
bugfix/autodev-2494-20261014045802002Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
Memtensor-AI commentedon Oct 14, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
6000a6d0b43d3c4fWorking branch:
bugfix/autodev-2494-20261014045802002Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
fix: retry transient gateway 502 on the DSH host-LLM path (MemTensor#…
fix: never retry a provider-side abort on the DSH host-LLM path (MemT…
Memtensor-AI commentedon Oct 14, 2026
✅ AutoDev task
6000a6d0b43d3c4fcompleted.Summary: Fixed the DSH host-LLM adapter so a transient gateway 5xx is retried instead of silently degrading the auxiliary stage (issue #2494).
Root cause:
createDeepSeekHarnessHostLlmBridge().complete()inapps/memos-local-plugin/adapters/deepseek-harness/host-llm.tsdispatched oneprepared.stream()call and never retried. A transient 502 from the gateway in front of the host Messages API surfaced as a terminalfinish→MemosError(LLM_UNAVAILABLE, status:502), which propagated straight out. The two host-LLM stages (retrieval.retrieval.filter.v5,capture.summarize) then degraded silently — recall filtering fell back tosafeCutoffand capture summarization to the heuristic — matching the reporter's 25/96 failed calls.config.maxRetriesnever applied because it is consumed by the direct-provider fetcher, not the host-bridge path.Fix: dispatch now runs through a bounded retry loop (
completeAuxiliaryWithRetry): at most 3 attempts,250 ms × attemptbackoff, bounded by the existing per-calldeadline(). The classifier (isRetryableAuxiliaryError) retries onlyLLM_UNAVAILABLEwith no status (transport failure) or status ≥ 500, and never401/402/403, provider aborts (dshCode "ABORTED"),LLM_RATE_LIMITED,LLM_TIMEOUT, malformed output, or deadline/caller aborts. Because a prepared DSH call is single-use, each attempt re-prepares viaprepareAuxiliaryCall; the model-capability lookup is cached per route, so re-preparation is a cache hit and adds no round trip. Success-path behavior and the returnedHostLlmCompletionshape are unchanged.Tests/verification: added 13 regression tests to
tests/unit/adapters/deepseek-harness-host-llm.test.tscovering the acceptance matrix ([502,502,ok]→3 ok; [502,ok]→2 ok; [502,502,502]→3 thrown; [ok]→1; [401/402/403]→1 thrown; statusless transport→2 ok; provider abort→1 thrown; RATE_LIMIT→1 thrown; malformed→1 thrown; deadline abort→1 thrown; caller abort during backoff→1).npx tsc -p tsconfig.json --noEmitis clean andnpx vitest run tests/unit/adapters/deepseek-harness-host-llm.test.ts tests/unit/llmreports 121/121 passing. Broader suites that fail (better_sqlite3.nodemissing native binding) were confirmed pre-existing on a clean stash and are unrelated. An independent code review flagged the provider-abort retry gap, which is fixed and covered.Deliverables: 2 commits on
bugfix/autodev-2494-20261014045802002(892ab4c initial fix, 7bcf73a abort exclusion), pushed to origin. opsp artifacts written under.ai-tasks/andopenspec/changes/2026-10-14-2494-.../(proposal/spec/design/verification-report) and archived to thememos-autodev-specsrepo (commit 682db96). No PR created — left to the scheduler.Base branch:
mainBranch:
bugfix/autodev-2494-20261014045802002Commit:
7bcf73aPR: #2496
Assigned to: @CarltonXiang
Reviewers: @whipser030, @hijzy
refactor(plugin): reuse shared waitForRetry and guard the retry bound