Your Handler, At Last

Almost nowhere near the framework. This is the shortest post in the series, and its shortness is the argument: by the time your code is involved, the interesting work is behind you, and what remains is a couple of percent of the journey.

Let us do the last two percent properly anyway, because there are two real lessons hiding in it.

What is actually left to do

Your event loop has just been told that a descriptor is readable. From here to your function is four steps.

Read. One system call copies whatever is in the socket’s receive queue into a buffer your process owns. It returns a count. That count has no relationship to any message boundary, for all the reasons in Post 07.

Accumulate. Those bytes are appended to a buffer belonging to this connection — because they may be a fragment, and the rest is coming.

Parse. Ask whether the buffer now holds a complete request.

Route and call. If it does, match the path against a table and invoke your function.

The parser is a state machine, and it has to be

The single design constraint on an HTTP parser is the one Post 07 established: you never know that you have the whole message until you have parsed enough of it to know. A read may deliver three bytes. It may deliver a request line, six headers, and the first 40 bytes of the next request.

So the parser cannot be a function that takes a message and returns a structure. It has to be resumable — suspendable in the middle of a header name, storing enough state to continue when more bytes arrive, and able to report not yet as an ordinary, frequent, uninteresting outcome.

The good ones are startlingly fast. A well-written parser takes a few hundred nanoseconds for a typical request header block, because it makes one pass over the bytes, allocates nothing, and hands back offsets into your existing buffer rather than freshly minted strings.

Which points at where the cost actually is. Parsing a dozen headers is cheap. Allocating a dozen strings, putting them in a hash map, lowercasing the keys, and building a request object with accessors for all of it is not cheap — it is often an order of magnitude more expensive than the parsing it decorates.

This is the entire secret of frameworks that advertise low overhead. They are not parsing faster. They are being lazy: keeping the raw buffer, exposing headers as views into it, and deferring every conversion until you ask for that specific header. If your handler reads two headers, you pay for two.

Routing is a similar story and matters less. A tree over path segments, or a list of patterns tried in order — for a hundred routes, on a modern CPU, either is a rounding error. Routing is almost never your problem, whatever a micro-benchmark implies.

Where the 797,000 went

Now the accounting, which is the real content of this post.

The benchmark measured one thing: how long the framework takes to turn bytes into a call and a response back into bytes, with a handler that does nothing, over connections that are already open, on a machine doing nothing else.

Do the division, because it is the interesting part. Eight hundred thousand requests per second on a thirty-two core machine is twenty-five thousand per core, which is forty microseconds of CPU per request — and that forty microseconds is the entire framework: the read, the parse, the route match, the response serialisation, the write.

Your service takes 1.8 milliseconds in the handler and 40 milliseconds at the 99th percentile. So the framework is about two percent of the first number, and a rounding error in the second.

So what is the handler doing for 1.8 milliseconds? Almost always: waiting for something else on the network. A database query. A cache lookup. An internal service.

And here is the observation worth the whole post.

That is where the time is. Not in your framework, and not usually in your code. In the several complete round trips through this entire stack that your handler triggers and then waits on — each one subject to every queue, every scheduling delay, and every backpressure mechanism this series has described.

And the 40-millisecond tail is where Post 02 comes back. The p99 is not the sum of the averages; it is what happens when one of these queues is momentarily full, or a core is busy in softirq, or a retransmission timer fires, or your event loop was held up by a previous handler that did something synchronous.

The connection does not end

One last thing your parser has to get right.

After you write the response, the connection stays open. This has been the default since HTTP/1.1, and it is a large part of why the web is not slower than it is — the alternative is paying for a handshake, and on an encrypted connection a much more expensive one, on every single request.

So your buffer may already contain the beginning of the next request, and your parser has to notice that rather than discarding it. If a client sends several requests without waiting for responses, you may have three complete requests sitting in one buffer, and they must be answered in order, because a stream has only one dimension and there is nowhere else to put the second reply.

That last constraint is worth naming, because it is the reason HTTP/2 exists. One slow response on an HTTP/1.1 connection delays every response queued behind it, no matter how fast those are. HTTP/2 fixes it by adding another framing layer inside the stream — interleaved chunks tagged with a stream identifier — so that responses can be multiplexed.

Which solves the problem at the application layer and leaves it untouched at the transport layer, because TCP still delivers one ordered stream, so a single lost segment still stalls every multiplexed request behind it. Post 12 is where that gets resolved, by an approach that gives up on TCP entirely.

See it for yourself

You can split a request’s time into its parts from the outside, which is a fast way to find out whether the problem is anywhere near your code.

time_connect:       0.000312
time_appconnect:    0.000000
time_starttransfer: 0.041188
time_total:         0.041402

Then, inside the process, put a timer around the handler itself and compare. The gap between the two is everything this series has been about, and on most services it is the larger number.

What to carry forward

From a voltage on a wire to a function call, your framework is responsible for the last couple of percent. That is not a criticism of frameworks — it is what a good abstraction looks like from the inside.

But it does establish a prior, and the prior is useful: when a request is slow, it is probably not slow here. It is slow in something your handler is waiting for, or in a queue somewhere below you that briefly filled.

Which is the whole of Act IV. First the journey back out — which is not the journey in, reversed — and then the diagram that locates every failure you have ever seen.