Skip to content

feat: add Handler::concurrent - #131

Open
Frando wants to merge 5 commits into
mainfrom
Frando/feat-handler-concurrent
Open

Frando wants to merge 5 commits into
mainfrom
Frando/feat-handler-concurrent

Conversation

@Frando

@Frando Frando commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

This adds Handler::concurrent(max_concurrent, f). It runs up to max_concurrent requests of a connection at the same time. The requests run as futures in the task of the connection, so a request that blocks delays the other requests. When the connection ends, handle_connection waits for the requests that still run to complete before returning.

@Frando
Frando force-pushed the Frando/feat-handler branch from 08fc52e to d570769 Compare October 5, 2026 12:20
@Frando
Frando force-pushed the Frando/feat-handler-concurrent branch from a6e5e73 to c2711d0 Compare October 5, 2026 12:20
@Frando
Frando force-pushed the Frando/feat-handler branch from d570769 to da2f4a6 Compare October 5, 2026 13:56
@Frando
Frando force-pushed the Frando/feat-handler-concurrent branch 2 times, most recently from da6ac39 to e6ade36 Compare October 6, 2026 08:51
Frando added a commit that referenced this pull request Oct 6, 2026
The `Handler` type is currently a public alias for an `Arc<dyn Fn>` over
the protocol enum and the two streams of a request. I don't like this
too much: With the public alias the signature is fully fixed and we
cannot change it after 1.0 without a major release. It also makes the
handler impls quite verbose, implementors have box the future and create
the message enum themselves. Additionally, it runs requests
sequentially, while - depending on the application - you'd want to run
them concurrently.

After some thinking and back and forth, I came up with this PR, which is
I think a good improvement, better base for 1.0, and doesn't add too
much complexity.

This PR turns `Handler` into a struct with constructors:

- `Handler::from_sender` sends each request to a `LocalSender`, for
example of an actor.
- `Handler::sequential` runs a function on the message enum, one request
at a time per connection. `Handler::concurrent` follows in #131.
- `Handler::raw` runs a function on the protocol enum and the two
streams, i.e. works like the Handler fn does currently.

Handler functions now return `Result<(), HandlerError>`. On
`Err(HandlerError::close_connection(code, reason))`, the server loop
closes the connection with the given code and reason. `HandlerError`
keeps its cases private, so we can add more after 1.0.

Finally `Handler::call` allows to call the handler closure directly,
e.g. to pass a request from a `Handler::raw` to the handler of another
protocol. `handle_connection` becomes a method of `Handler`.

## Breaking changes

* changed: `irpc::rpc::Handler<S>` is a struct, not a type alias for
`Arc<dyn Fn(S, RecvStream, SendStream) -> BoxFuture<Result<(),
SendError>>>`. Create it with `Handler::from_sender(local)`,
`Handler::sequential(f)`, or `Handler::raw(f)`. A closure built with
`Arc::new(..)` becomes `Handler::raw(..)` with the new return type
below.
* changed: handler functions return `Result<(), HandlerError>`. For
`Handler::raw` this replaces `Result<(), SendError>`, so you need to map
manually and decide whether a failed send should end the connection or
not.
* changed: `irpc::rpc::handle_connection(connection, handler)` is now
`handler.handle_connection(connection)`.
* changed: `irpc::noq::listen`, `irpc::iroh::listen`,
`IrohProtocol::new`, and `Iroh0RttProtocol::new` take `impl
Into<Handler<S>>`. A `LocalSender<S>` can be passed directly.
* removed: `RemoteService::remote_handler`. Use
`Handler::from_sender(local)`, or pass `local` where a handler is
expected.
* removed: `IrohProtocol::with_sender` and
`Iroh0RttProtocol::with_sender`. Use `IrohProtocol::new(local)`.
`Handler::sequential` runs the requests of a connection one after the
other. Depending on the application, you want to run them concurrently.

`Handler::concurrent(max_concurrent, f)` runs `f` on each request, at
most `max_concurrent` requests per connection at a time. The requests run
in a `FuturesUnorderedBounded` in the task of the connection, so a
request that blocks delays the others. When the connection ends,
`handle_connection` waits for the requests that still run.
@Frando
Frando force-pushed the Frando/feat-handler-concurrent branch from e6ade36 to d03537e Compare October 6, 2026 10:56
@Frando
Frando changed the base branch from Frando/feat-handler to main October 6, 2026 10:56
@Frando
Frando marked this pull request as ready for review October 9, 2026 08:03
@rklaehn
rklaehn force-pushed the Frando/feat-handler-concurrent branch from bbd143d to 8dfceda Compare November 6, 2026 09:54
@rklaehn

rklaehn commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

The loop looks right to me: the pending read is kept across iterations, finished requests are polled first, and all running requests are completed before handle_connection returns.

What I'd like to change is the structure. Today all constructors go through one loop (from_sender → sequential → raw). This adds a second one via Mode. But sequential is just concurrent with a limit of 1: with one slot, the next request is only read after the running one has completed, which is exactly what run_sequential does. So:

  • Keep Handler::concurrent as the public constructor, but take max_concurrent: NonZeroUsize instead of panicking on 0.
  • sequential(f) becomes Self::concurrent(NonZeroUsize::MIN, f), and its docs say it is the limit of 1.
  • Remove Mode and run_sequential. The handler stores max_concurrent: NonZeroUsize, and handle_connection always runs the concurrent loop.
  • raw and from_sender stay at 1. from_sender only enqueues into the actor's channel, so more slots would not help, and a raw user who needs more can write their own loop with read_request_raw.

Smaller things:

  • The docs should say that a long-running request, for example a server stream that lasts as long as the connection, holds its slot until it ends. With concurrent(16, ..) and 16 such streams, no further request on that connection is read.
  • A test for a handler error in concurrent mode: the connection closes, no more requests are read, and the running requests still complete.
  • The new tests need #[traced_test], like all tests since test: run the tests with traced_test #141.

This will conflict a little with #164, which changes HandlerError::close_connection to take a VarInt and touches the same close code that this moves into close_connection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants