[Enhancement] use stream load async return to optimize the performance under high concurrency #53139

Search before asking

• I had searched in the issues and found no similar issues.

Description

Currently use libevent to handle http request, there are multi http threads for the same port. When request is sent to one thread, on_header、on_chunk_data、handle can only be executed on this thread.
However, handle method need to wait for load finish, and will block all requests on this thread.

How to reproduce?
use group commit sync mode, it will happens.

Solution

To fix it, we make handle async by cherry-pick #36772.
And, we add a logic to ensure free order is correct.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct