Repository navigation
Handle reasoning budget - #20297
Handle reasoning budget#20297
Conversation
|
AI disclosure: I used Claude Opus in making most of the changes, auditing and modifying the critical code myself. |
|
Oh, I didn't mention it in the note but this of course entails support for multiple grammars for one server task since the tool grammar is still there. |
|
Some interesting observations from early tests (on Qwen3.5 9B Q8_0):
|
ggerganov
left a comment
There was a problem hiding this comment.
I'll probably need to understand this deeper, but on first look this seems very heavy logic. How important is this functionality?
Specifically the changes in common_sampler seem disproportionately large compared to what this brings to the existing logic. Look for ways to simplify.
A lot of people have been requesting this, especially with the Qwen3.5 models that are seen as too verbose with their reasoning. The changes in the sampler code are basically to the grammar sampler, since the idea is (a) to support more than one grammar simultaneously and (b) to support delayed grammar application (with token counting). Maybe this can be simplified by instead inserting another grammar sampler? Not sure how viable that would be. |
|
In my opinion, we need to think long term. The grammar sampler is incredibly inefficient. We had to revert a change @ggerganov wanted to make that shifts the grammar to the start of the chain to support backend sampling. Merging this will increase the reliance on the grammar sampler and make it more challenging to optimize in the future. I'm of the opinion that a dedicated, simple, reasoning sampler that lives in common would be enough and can be used at the start of the chain--so long as it aligns with the grammar used (if any). |
|
Yes, framing this as a reasoning sampler should definitely be explored. |
1df4d24 to
b201c80
Compare
|
@ggerganov @aldehir aight, reverted all the grammar changes and instead reimplemented it as a clean new reasoning parser. I tested on cli, there is no noticeable overhead on generation (152 t/s both with and without the sampler). |
|
Unless @ggerganov thinks otherwise, I would put it under Other notes:
https://arxiv.org/abs/2508.14444 Overall, I think this is a cleaner approach. It isolates the complexity rather than polluting the already complex grammar sampling logic. |
aldehir
left a comment
There was a problem hiding this comment.
Need some tests around the apply/accept logic. I had some in my example, but feel free to improvise.
|
Funny, wonder what happened here: |
GitHub merge running on Windows? :D |
1dbfbcb to
24eb88d
Compare
|
Aight I got rid of the explosive terminology and fixed the newlines in the process :) |
|
Okay, UTF-8 and tests are done, think this one's ready. |
* v1 * Finished! * Handlie cli * Reasoning sampler * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Less explosive terminology :) * Add utf-8 case and tests * common : migrate reasoning budget sampler to common * cont : clean up * cont : expose state and allow passing as initial state * cont : remove unused imports * cont : update state machine doc string --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Alde Rojas <hello@alde.dev>
This change refactors the reasoning_budget_message parameter from the common params into the sampling parameters specifically. It also removes the reasoning_budget common parameter and standardizes on the existing reasoning_budget_tokens parameter in the sampling configuration. Issue: ggml-org#20429 Original PR: ggml-org#20297
This change refactors the reasoning_budget_message parameter from the common params into the sampling parameters specifically. It also removes the reasoning_budget common parameter and standardizes on the existing reasoning_budget_tokens parameter in the sampling configuration. Issue: ggml-org#20429 Original PR: ggml-org#20297
* v1 * Finished! * Handlie cli * Reasoning sampler * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Less explosive terminology :) * Add utf-8 case and tests * common : migrate reasoning budget sampler to common * cont : clean up * cont : expose state and allow passing as initial state * cont : remove unused imports * cont : update state machine doc string --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Alde Rojas <hello@alde.dev>
* v1 * Finished! * Handlie cli * Reasoning sampler * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Less explosive terminology :) * Add utf-8 case and tests * common : migrate reasoning budget sampler to common * cont : clean up * cont : expose state and allow passing as initial state * cont : remove unused imports * cont : update state machine doc string --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Alde Rojas <hello@alde.dev>
This change refactors the reasoning_budget_message parameter from the common params into the sampling parameters specifically. It also removes the reasoning_budget common parameter and standardizes on the existing reasoning_budget_tokens parameter in the sampling configuration. Issue: ggml-org/llama.cpp#20429 Original PR: ggml-org/llama.cpp#20297
This change refactors the reasoning_budget_message parameter from the common params into the sampling parameters specifically. It also removes the reasoning_budget common parameter and standardizes on the existing reasoning_budget_tokens parameter in the sampling configuration. Issue: ggml-org/llama.cpp#20429 Original PR: ggml-org/llama.cpp#20297
Gemma-4 26B-A4B hat ghost thought channels — das Modell emittiert
{thought}-Tokens in Endlosschleife trotz enable_thinking=False.
--reasoning off (PR ggml-org#20297) ist der zuverlässige Fix, chat_template_kwargs
wurde durch Autoparser-Bug ggml-org#20338 zeitweise ignoriert.
AGENTS.md ergänzt mit vollständiger Thinking-Doku.
* v1 * Finished! * Handlie cli * Reasoning sampler * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Less explosive terminology :) * Add utf-8 case and tests * common : migrate reasoning budget sampler to common * cont : clean up * cont : expose state and allow passing as initial state * cont : remove unused imports * cont : update state machine doc string --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Alde Rojas <hello@alde.dev>
This change refactors the reasoning_budget_message parameter from the common params into the sampling parameters specifically. It also removes the reasoning_budget common parameter and standardizes on the existing reasoning_budget_tokens parameter in the sampling configuration. Issue: ggml-org#20429 Original PR: ggml-org#20297
…#68) * fix(hermes): patch only the top-level model block of config.yaml The writer rewrote the first `default:`, `provider:` and `base_url:` lines anywhere in ~/.hermes/config.yaml. Hermes repeats those keys under auxiliary, delegation, cron, moa, tts, memory and others, so on any config where one of them comes before `model:` (or where `model:` is a scalar or missing) the patch landed in another block, sometimes breaking the YAML, and left the chat route untouched. Hermes then ran on its built-in defaults (empty model, provider auto), fell through to OpenRouter and got "HTTP 400: No models provided" before any request reached Atomic Chat. Edit only direct children of the top-level `model:` key: replace `default`, `provider` and `base_url` in place (dropping a nested value a scalar replaces), add the missing ones at the block's indent, turn a scalar `model: <id>` into a block, and append a model section when there is none. Also never add a second top-level `providers:` or `custom_providers:` key (Hermes' ruamel reader rejects duplicate keys and then runs on defaults), and recognise those keys with a trailing comment. Golden fixture outputs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(server): turn llama.cpp thinking off for Anthropic requests that ask for none The Anthropic API thinks only when a request sets `thinking` to enabled or adaptive. llama.cpp's /v1/messages maps only `enabled` and otherwise leaves the chat template at the server default, which --jinja turns on for every template that can think. Claude Code's session-title request (no `thinking`, max_tokens 32000) made Qwen3.5 4B think for 20k tokens, and on a one-slot server the user's own turn waited behind it (QA B08). For a /messages request to a llama.cpp engine (llamacpp, llamacpp-upstream, atomic-prism) that has no `thinking` or `thinking: {type: "disabled"}` and does not set the switch itself, the core now adds `chat_template_kwargs: {enable_thinking: false}`, which llama.cpp's Anthropic conversion passes to the template. The chat-completions fallback gets the same kwargs. Requests that ask for thinking pass byte for byte; other engines are untouched. The proxy-http contract records the divergence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(server): pin the byte-exact relay of llama.cpp streams to usage-asking clients B11 (Kilo Code showed no reply while the inspector recorded a completed stream): Kilo (AI SDK openai-compatible with includeUsage) and OpenClaude both send stream_options.include_usage themselves, so the proxy neither rewrites their request nor strips anything from the reply. Pin that with a llama.cpp b12019-shaped thinking-model reply written in small pieces cut mid-line and between an event's two newlines: the client gets the upstream bytes unchanged, watched or not, and a client that did not ask loses only the usage trailer the proxy asked for, with the finish chunk and [DONE] intact across split reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(server): keep max_tokens on a llama.cpp /messages fallback A /messages request that llama.cpp's template rejects (Claude Code sends mid-conversation system messages, which Qwen3.5's template refuses) is retried as chat completions. The translation dropped max_tokens, as the Rust proxy did, so on a server started with n_predict -1 the reply ran until the context was full. llama.cpp's own /v1/messages honours the cap; its fallback now keeps it too. Other engines keep the translation as it was. The proxy-http contract records the divergence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(server): merge mid-conversation system messages before llama.cpp sees them Claude Code sends its reminders as role "system" messages between turns to a model it does not know. llama.cpp's /v1/messages passes them to the chat template where they stand, and a strict template (Qwen3.5: "System message must be at the beginning.") answers 500, so every later turn went through the chat-completions fallback. That fallback merges the system messages into the leading one but loses the reply's token usage (Claude Code reported input_tokens 0 and never learned how full its context was) and the turn's thinking blocks. For a /messages request to a llama.cpp engine, the core now merges those messages into the top-level system the same way before forwarding, so the engine answers natively. Other engines and conversations without system messages are sent as they came. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(integrations): hand the served context window to the agent writers Agents told nothing about the window assume 128K-272K and never compact before a 16K-64K local server refuses. ConfigureInput gains an optional contextLength, and `atomic-chat launch` reads the window the session was loaded with from its /props (falling back to the asked size without fit). The writers use it in follow-up commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): Zed's real config dir and the served window for Zed and ZCode - zedConfigDir(): %APPDATA%\Zed on Windows, $XDG_CONFIG_HOME/zed on Linux, ~/.config/zed on macOS, as Zed resolves it (parity with the app's zed_config_dir). - Zed: max_tokens = served window, max_output_tokens = min(8192, ctx/4), images: false (Zed's own default); 32768 when the window is unknown, since Zed requires a value. - ZCode: contextWindow = served window, omitted when unknown so ZCode's default stands; output cap min(8192, ctx/4). - Both keep the user's default model they replace in a <config>.atomic-chat-backup.json sidecar for the app's Disconnect. - Golden zed fixtures: images false, and the sidecar where the seed had another provider as the default model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): find Zed and ZCode Preview off PATH in `launch` Mirrors the app's zed_cli_candidates / zed_app_candidates and the ZCode Preview build of zcode_app_candidates_for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): tell Kilo, OpenCode, MiMo and pi the served window and keep what Connect replaces The OpenCode-family writers now describe the model the way these agents need for a local server: - `limit: { context, output: min(8192, ctx/4) }` when the window is known (omitted when not), so OpenCode and Kilo compact instead of running with `limit.context = 0`, and MiMo stops assuming a 1M window; - `reasoning: true` and `interleaved: { field: "reasoning_content" }`, so a thinking model's answer is shown as thoughts plus text and its reasoning is sent back within a tool loop (B11); - provider options `timeout: false, headerTimeout: false`, verified in the Kilo, OpenCode 1.15 and MiMo schemas: a long prompt on a slow machine no longer hits the 5-minute first-byte timeout. pi gets `name`, `contextWindow`, `maxTokens` and a per-model `compaction.modelOverrides["atomic/<id>"]` (reserve and recent history a quarter of the window), the schema documented in pi's settings reference. OpenCode and MiMo are read as JSON5 like Kilo (comments and trailing commas are dropped on write); an existing `opencode.jsonc` / `mimocode.jsonc` is written instead of the `.json` it overrides, and our leftovers in that `.json` are removed. KILO_CONFIG_DIR, OPENCODE_CONFIG, OPENCODE_CONFIG_DIR, MIMOCODE_HOME, PI_CODING_AGENT_DIR and XDG_CONFIG_HOME are honoured. MiMo's `$schema` is the one MiMo serves. Before `model`, `provider.atomic`, pi's default provider/model or its `providers.atomic` are overwritten, a value that is not ours is kept in `<file>.atomic-chat-backup.json` for the app's Disconnect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): run Goose with its own environment, start Kilo on our model Goose no longer gets OPENAI_API_KEY, OPENAI_HOST and GOOSE_* written user-wide (shell rc block, `setx`), which pointed every OpenAI SDK tool on the machine at the local server. `launch goose` sets them on the child only: GOOSE_PROVIDER, GOOSE_MODEL, OPENAI_BASE_URL with the server's API prefix (instead of the bare host plus a hard-coded OPENAI_BASE_PATH), OPENAI_API_KEY, and GOOSE_CONTEXT_LIMIT / GOOSE_MAX_TOKENS when the window is known (names checked against goose 1.57). An ambient OPENAI_HOST / OPENAI_BASE_PATH is cleared, since Goose prefers it to OPENAI_BASE_URL. Configuring Goose now only removes what earlier releases wrote: our marked block in .zshenv/.bashrc/.bash_profile (our export lines only) and, on Windows, the user variables recognisably set by `setx`. Kilo's saved per-agent pick beats `model` in its config, so `launch kilo` passes `--model atomic/<id>` (the TUI's own flag). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(contract): agent-config fixtures for the new Kilo, OpenCode, MiMo, pi and Goose output Regenerated by the app's Rust writers (`opencode_family.rs` `dump_family_fixtures`) and replayed byte for byte here: the model entry with thinking, no timeouts and the window, pi's window and compaction override, the backup files Connect leaves for Disconnect, the `.jsonc` target with its `.json` cleaned, MiMo's corrected `$schema`, and Goose writing nothing but removing what earlier releases wrote. Cases may carry `context_length` (index note). CHECKSUM recomputed over this tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(models): count sliding-window KV layer by layer, with a window per slot The KV estimate averaged a sliding-window model over every layer, (full + window) / 2. llama.cpp b12019 keeps the whole window only in the global layers and pad256(n_swa x sequences + n_ubatch) cells in each sliding one (src/llama-kv-cache-iswa.cpp:73), so Gemma 3 was counted at two and a half times its real cache at 32K and small Macs were narrowed far more than needed. The estimate now reads the per-layer sliding_window_pattern (inlined by the GGUF reader at any length, like head_count_kv), a period from the GGUF, or the architecture's built-in pattern (gemma2/3/3n, gpt-oss, cohere2, exaone4, afmoe, smallthinker); Gemma 4's own SWA head sizes and shared KV layers count too. An architecture with a window but no known pattern is counted in full, the safe side. A new layout argument gives the sequences sharing a unified cache and the micro-batch size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(llamacpp): agent-friendly defaults: shared KV slots, answer and thinking caps Integrations audit 2026-11-03 (S3, S4, S5, X6, P3), llama.cpp b12019: - Slots: -kvu is always sent, so --parallel N shares one KV pool of the whole window (src/llama-context.cpp:389-390) and costs no window memory beyond each sliding layer's per-slot window. parallel defaults to 2, so a title or classifier request no longer waits behind an agent turn. The fit margin and the small-Mac ceiling no longer multiply the KV cache by the slots (load-plan.ts); the ceiling is the pool itself. - Continuous batching is sent both ways (--cont-batching / --no-cont-batching): llama.cpp has it on unless told otherwise, so the setting could never turn it off. Its default is now on, as llama.cpp's. - n_predict 0 is automatic: 16384 tokens, never above the window core knows. Clients without max_tokens (Codex via Responses, Kilo, OpenCode) no longer generate until the window is full. -1 stays "no limit". - New reasoning_budget setting (default 8192) sent as --reasoning-budget with --reasoning-budget-message from b8287, the first build that takes a token count (ggml-org/llama.cpp#20297); requests' own budgets win. - --cache-ram 1024/2048 MiB on machines with at most 8/16 GiB of RAM (from b6723, ggml-org/llama.cpp#16391) instead of llama.cpp's 8 GiB. - DEFAULT_CTX_LEN is 32768: a fit-off load without a window gets 32K (or the trained context) instead of the whole trained context, and MLX's default window is 32K; on a Mac below 16 GiB both are narrowed toward the previous 16K by the KV estimate, never below it. - isContextOverflowFinishLength takes the engine's own answer cap into account: a length stop at --n-predict is no longer a context overflow. The args fixtures from the Rust builder are compared without -kvu and --no-cont-batching, recorded as deliberate divergences; the engine contract gains the new flags and build gates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(managed): automatic context length and request count for vLLM and SGLang Integrations audit 2026-11-03, S2: vLLM and SGLang defaulted to 8192 tokens and one request, so every coding agent's 13-27K-token prompt was refused with context_length_exceeded and side requests queued behind the agent's turn. context_length and max_num_seqs / max_running_requests now default to 0, automatic: the load picks the widest pair the engine's own memory rule passes on the card it re-probes right before the container — 64K with two requests, 32K with two, 32K alone, 16K, then the old 8K alone, at which the memory check refuses exactly as before. The window never exceeds the checkpoint's own (max_position_embeddings and the other keys vLLM derives it from), leaves room for the output cap, and with a KV size fixed by hand holds one full sequence of it. Both engines keep one paged pool sized by their memory share, so the window only bounds a request. The chosen pair reaches the container through the launch plan and the session limits (/v1/models context) through planContextLength. Values set by hand stay as they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(settings): move stored values off the defaults core no longer ships settings.json stores every provider value, defaults included, so a new schema default reached fresh installs only. On open, the store now moves a stored value that still equals an old default to the new one, once, and records the migration under state.default_migrations; a value the user sets afterwards, the old default included, stays. A moved value bumps the revision so the app mirrors it. Migrations (integrations audit 2026-11-03): llama.cpp parallel 1 -> 2, cont_batching false -> true (llama.cpp always batched continuously), n_predict -1 -> 0 (automatic cap); vLLM/SGLang context_length 8192 -> 0 and request count 1 -> 0 (automatic). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): keep Claude Code and OpenClaude writers in step with the app The `launch` writers for Claude Code and OpenClaude now match the app's claude_config.rs: each keeps a `<file>.atomic-chat-backup.json` sidecar recording what every key held before, so the app's Disconnect can undo a connection the CLI made; a reconnect undoes the previous one first, and values an earlier release wrote are taken out rather than recorded. Claude Code: empty CLAUDE_CODE_USE_BEDROCK/VERTEX/FOUNDRY, the terminal title off, and with a known window CLAUDE_CODE_MAX_CONTEXT_TOKENS plus CLAUDE_CODE_MAX_OUTPUT_TOKENS (min(8192, ctx/4)); $CLAUDE_CONFIG_DIR is honoured. OpenClaude: the profile carries the server's key again (OC-2: the keyless profile got a 401 from a keyed server), maxContextLength and a per-model output cap with a known window, empty values for the shell variables that make OpenClaude skip its active profile, and the startup profile its OPENAI_API_KEY. Our profile is matched by its id only, and $OPENCLAUDE_CONFIG_DIR and the legacy .config.json are honoured. Golden fixtures for both agents regenerated; the in-place replacement case became openclaude__users_own_atomic_chat_profile_kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(server): keep mid-conversation system messages in place for llama.cpp 6f87e30 moved every system message inside an Anthropic conversation into the top-level system. That rewrites the start of the prompt whenever Claude Code sends a reminder, and llama.cpp then processes the whole conversation again (13-27k tokens on hybrid and sliding-window models, whose prompt cache cannot reuse anything past the change). For a llama.cpp engine, only system messages before the first turn now join `system`. A later one stays where it is, as user text wrapped in <system-reminder>, merged into a neighbouring user turn so the roles still alternate: before the text of the next user message, or at the end of its last tool_result when it answers tool calls (llama.cpp puts a user message's text before its tool results, so a text block there would land between the call and its result). With no user message after it, it goes to the end of the one before, or into a user message of its own. A reminder sent with clear_at "next_user_message" is dropped once a user message follows it. The chat-completions fallback gets the same messages. The Responses shim (Codex to llama.cpp) placed developer and system items the same hoisting way; it now keeps later ones in place too, never between a tool call and its output. The shim contract case that recorded the Rust hoisting is corrected as a known divergence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(server): advertise the served window and answer overflows as prompt too long Agents did not know the window the local engine really serves, so none of them compacted in time (audit X1). GET /v1/models now gives each local model whose window is known `context_length` and `meta.n_ctx` (the field llama-server's own list uses), and Muse Code's catalogue advertises the real window instead of its 32k guess. The window comes from the session: a declared session (vLLM, SGLang, TensorRT-LLM) states it; a llama.cpp session is asked once per loaded generation through /props (default_generation_settings.n_ctx, answered from metadata without a slot). The core wires SessionWindows into the server as `contextWindow`; a session target now carries its generation. Other engines advertise none rather than a guess. On /v1/messages, a llama.cpp context overflow (exceed_context_size_error with n_prompt_tokens/n_ctx, or its message) on the native call or on the chat-completions fallback is answered as the Anthropic API's 400 invalid_request_error "prompt is too long: N tokens > M maximum", the wording Claude Code compacts on; the fallback is no longer tried for it. OpenAI routes keep llama.cpp's own error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(server): keep each request's context window and cached prompt tokens The Activity inspector showed the window of whatever session served the model when the row was opened, not the one the request ran with (QA B04). The trace now records the window of the session a request goes to when it is routed (the session's declared window or the one SessionWindows read) and sends it as `ctx_len` in the finish fields, so the log keeps it after a reload or an unload. The finish fields also carry `cached_tokens`, the part of the prompt the engine reused, from llama.cpp's timings.cache_n, OpenAI's prompt_tokens_details.cached_tokens and Anthropic's cache_read_input_tokens. Prompt tokens now count the whole prompt: Anthropic's input_tokens leaves out cache reads and writes, and llama.cpp's timings.prompt_n only what it processed. A whole Anthropic reply is read for usage and stop reason too. The Responses shim carries the cached part into input_tokens_details instead of a hard-coded 0. The inspector-telemetry contract records the new field as a known divergence (null in every recorded case). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(server): give Codex's MCP namespace tools to llama.cpp Codex sends the tools of each MCP server as a Responses namespace ({type: "namespace", name, tools: [...]}) and runs a call to one only with that namespace. The Responses shim kept only top-level `function` tools, so a local model never saw a Codex MCP tool (audit S7). The shim now offers every function in a namespace as a chat function named <ns>__<name> (the plain name in Codex's default `functions` namespace; no second delimiter after a namespace that already ends in one), described by its own description or else the namespace's. A function_call item replayed from history and a namespaced tool_choice use the same name. In the reply, whole or streamed, a call to such a name is given back its `name` and `namespace`, so Codex routes it to the MCP server. Freeform tools inside a namespace are dropped like top-level ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(server): cover the system-reminder wrapper Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): tell Codex the served window in its profile With the session's window known, the Atomic Chat profile carries model_context_window and model_auto_compact_token_limit (three quarters of it), so Codex compacts before a 16K-64K local server refuses instead of assuming 272K. Unknown, neither line is written. Byte-identical to the desktop writer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): Cline, Droid, Copilot and dsh get the window and a backup Ports the desktop writers (app wip/fam-codex), byte for byte: - A known window is written with a reply cap of a quarter of it (8192 at most), and nothing is invented when it is unknown: Cline's contextWindow/maxTokens on the entry `cline auth` saved, Droid's maxContextLimit/maxOutputTokens (instead of a fixed 16384), Copilot's maxPromptTokens/maxOutputTokens/maxContextWindowTokens, dsh's route. - Each writer records what it overwrote in a <config>.atomic-chat-backup.json sidecar, which the desktop's Disconnect restores from. - Droid names its entry (id, index) and selects it through sessionDefaultSettings.model; the legacy top-level selector is removed. - Copilot writes an atomic-chat provider and model into .copilot/providers.json instead of persisting COPILOT_* in the shell rc, removes exactly the rc block an earlier release wrote, and `launch` selects the model with COPILOT_MODEL=atomic-chat/<model>. - dsh 0.2 imports settings.yaml once into the profile, so the route and the default model go into profiles/web/cordis.patch.yml, re-serializing only our rows; a settings.yaml still present is kept in step, never created, and settings.yaml.imported is never touched. The cline/copilot/droid/dsh agent-config fixtures change accordingly (index note); the desktop writers produce the same bytes for every copilot, droid and dsh case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): start npm-shim agents through cmd.exe on Windows `atomic-chat launch` spawned the detected launcher without a shell. On Windows an npm-installed agent is a `.cmd` shim, which Node refuses to spawn directly (EINVAL) and does not find by its bare name (ENOENT), so every npm agent failed to start. Shims and bare names now run through `cmd.exe /d /s /c "<line>"` with each token quoted for cmd; executables and other platforms spawn as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): Hermes, OpenClaw and Atomic Agent writers record what they replace Hermes: one route for the app and the CLI, bare `custom` with model.base_url, model.api_key and model.context_length, which needs no custom_providers entry (an inline list cannot take one) and carries the server key. The served window is written as served, 65536 only when unknown. HERMES_HOME is honoured on every platform (~, $NAME, ${NAME}, %NAME% expanded) and the sticky profile from `hermes profile use` is the one written. No skeleton is seeded. OpenClaw: the model entry carries contextWindow and maxTokens when the window is known; the request budget moves to the provider (timeoutSeconds 600) and the agents.defaults.timeoutSeconds: 240 older releases seeded is dropped; gateway.auth.mode "none" is seeded only when no token or password is set. Atomic Agent: llm.fallback.appendLocal false when there is no fallback block, so a failure of ours is not retried on local-llama :8080; userModels carries the served window. Each writer records the keys it overwrote in <config>.atomic-chat-backup.json (json-key-backup.ts), the same sidecar the app writes, so the app's Disconnect restores a connection made by `atomic-chat launch`. The golden fixtures of the three agents are regenerated and replayed byte for byte against the app's Rust writers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(integrations): OpenHands, Poolside and Muse get their environment at launch only Their writers persist nothing any more: the variables go on the child `launch` starts (and inline in the app's terminal). Connect removes the marked shell-rc block an older release wrote, from any rc file it may have used, and nothing else; the old prefix sweep that deleted the user's own `export META_*` / `POOLSIDE_*` / `LLM_*` lines is gone with the writer. Poolside keeps `/v1` in POOLSIDE_STANDALONE_BASE_URL as its docs give it and is told the served window (POOLSIDE_STANDALONE_CONTEXT_LENGTH) through childEnv, which `launch` now passes the session's window to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(contract): recompute the app fixture checksum Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(integrations): cover the per-key backup and the legacy shell block Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: record the agent-serving defaults and the window the server advertises Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: keep context-growth tests independent of host RAM, /props and path separators - sessions: mock node:os totalmem so the 32K default is not narrowed on small CI runners (7 GiB macOS, Linux arm64). - public-server: load the overflowing model at an explicit 4K window instead of relying on the old 8K default. - api-events: the external engine stub answers the /props window read and counts only chat requests. - launch: ZCode's window is the one /v1/models advertises. - json-backup: `~/` joins with the platform separator on Windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: bring the new agent writers and overflow mapping up to their coverage floors The macOS unit job runs test:coverage, which failed its per-file floors on the code this branch added. Tests now cover: - goose: every legacy local-host range, the Windows reg/PowerShell path and its IO_ERROR, the placeholder vs server key. - hermes: CRLF input, %YAML directives, wrapped scalars, the providers block, the Disconnect backup reader, ~ and %NAME% homes, .env proxy bypass. - opencode-style, openclaude, pi: stale $schema, unreadable backups, the sibling .json beside a .jsonc, earlier startup profiles. - errors: anthropicPromptTooLong from fields and both message wordings, MLX KV-cache wordings, clientMaxTokens. - mlx model-files, fit-margin, catalog: memory facts, shard renames, the 256-token grid, Zed under LOCALAPPDATA. goose.ts drops a `?? ''` fallback on a regex group that always matches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(e2e): give a cold Windows daemon 45 s to print its ready line On windows-11-arm the first daemon of each e2e file spends 10-15 s in the PowerShell hardware probe before it is ready (main included); on a slow runner that crossed the 20 s wait and failed the start-up tests. Other platforms keep 20 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Adds proper handling for
--reasoning-budget.Currently,
--reasoning-budgetis just a stub that handles one case: 0, and the only think that does is set enable_thinking to false.This PR adds the following flags:
--reasoning on(short-rea on) - enable reasoning via kwargs on model--reasoning off(short-rea off) - disable reasoning via kwargs on model--reasoning-budget-message- a message to be appended before the reasoning close marker to inform the model the reasoning was terminated due to budget constraints, i.e. " ... reasoning budget exceeded" or "... okay, now let's answer."Also,
--reasoning-budgetnow adds an extra grammar with a mechanism called delayed launch. Basically what this does is, when the opening trigger for the grammar is triggered, tokens get counted down and we also watch for the disarm trigger. If the disarm trigger doesn't get triggered before the countdown, the grammar gets launched.This allows setting a real token budget limit on reasoning for models. This also allows disabling thinking for models that normally do not allow that by setting the budget to 0, which is now a different behavior than
--disable-reasoning. Note: while possible, this isn't recommended since if a model was trained to only work with reasoning, it might exhibit aberrant behavior (for example, try to open an extra reasoning section).Supersedes #17750