Skip to content

server: refactor child --> router communication - #24821

Merged
ngxson merged 5 commits into
ggml-org:masterfrom
ngxson:xsn/server_refactor_ipc
Jun 19, 2026
Merged

ngxson merged 5 commits into
ggml-org:masterfrom
ngxson:xsn/server_refactor_ipc

Conversation

@ngxson

@ngxson ngxson commented Jun 19, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

Part of #24822

Refactor the stdin/out communication child --> router

We are now using an "unified" message format: cmd_child_to_router:state:{json_string} and the json string contains both the state + payload

Requirements

@ngxson
ngxson requested review from a team as code owners June 19, 2026 20:24
@ngxson ngxson mentioned this pull request Jun 19, 2026
3 of 4 tasks
Comment thread common/common.h

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this change is unrelated to the current PR

because I planned to move common_params_handle_models to be called by server, I already touched this before realizing that's not a good plan. nevertheless, it will be for the third item of #24822

@ngxson

ngxson commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator Author

@ServeurpersoCom I'm merging this after the CI passes; no need for testing on your side.

I think the next item in my list, "Report model loading progress back to router" will worth testing. The plan is to have /models/sse to report the loading percentage

@ngxson
ngxson merged commit 2b686a9 into ggml-org:master Jun 19, 2026
20 of 25 checks passed
@ServeurpersoCom

Copy link
Copy Markdown
Contributor

That's classy, you're building the advanced version of the initial ask, an instant router response on weights loading. Looking forward to it!

jimbothigpen added a commit to jimbothigpen/af-engn that referenced this pull request Jun 20, 2026
Absorbs PRs ggml-org#24821 ggml-org#24796 ggml-org#24779 ggml-org#24817 ggml-org#23168 ggml-org#24787 ggml-org#24801 ggml-org#24794
ggml-webgpu F16 toggles, server router/agent refactor, cpp-httplib 0.48.0,
utf8 windows fix, docker UI build. No conflicts with our custom backends.
adrianhoehne pushed a commit to adrianhoehne/llama.cpp that referenced this pull request Jul 5, 2026
* server: refactor child --> router communication

* fix wakeup case

* add docs

* improve update_status()

* nits
zommiommy pushed a commit to zommiommy/llama.cpp that referenced this pull request Aug 18, 2026
* server: refactor child --> router communication

* fix wakeup case

* add docs

* improve update_status()

* nits
zbrad pushed a commit to zbrad/llama.cpp that referenced this pull request Sep 10, 2026
* server: refactor child --> router communication

* fix wakeup case

* add docs

* improve update_status()

* nits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants