Skip to main content

Building an HTTP Framer on Raw TCP — and How I Caused a Desync

I'm building an HTTP server from scratch on raw TCP in Node.js (no node:http), to learn backend internals and the bugs attackers find in them. This post covers the first piece: finding where one request ends and the next begins. Along the way I wrote a desync bug myself, and it turned out to be the same core idea behind HTTP request smuggling.

1. TCP has no messages

TCP is a pipe of bytes, not a mailbox of messages. When a client sends one HTTP request, the server can receive it in any number of pieces.

My first server just logged each data event:

socket.on("data", (chunk) => {
    console.log("got", chunk.length, "bytes");
});

Then I sent one request in two parts:

(printf 'GET / HT'; sleep 1; printf 'TP/1.1\r\n\r\n') | nc 127.0.0.1 3000
got 8 bytes
got 10 bytes

One request, two events, split in the middle of the word HTTP. So one data event ≠ one request. The server has to collect bytes itself and decide where a request ends.

2. Where does a request end?

The header block ends with a blank line: \r\n\r\n. (Every header line ends with \r\n; the double one marks the end.)

So the plan:

  1. Keep a buffer per connection.
  2. Append every chunk to it.
  3. Search the whole buffer (not the chunk) for \r\n\r\n.

Searching the whole buffer matters. If the marker gets split across chunks (...\r\n\r then \n), neither chunk contains it, and a chunk-only search would wait forever on a valid request.

My first working version:

let buf = Buffer.alloc(0);

socket.on("data", (chunk: Buffer) => {
    buf = Buffer.concat([buf, chunk]);
    let idx = buf.indexOf('\r\n\r\n');
    if (idx !== -1) {
        console.log("headers complete\n", buf.toString());
    }
});

Split requests worked. Then I sent two requests on the same connection.

3. The desync

(printf 'GET /one HTTP/1.1\r\nHost: x\r\n\r\n'; sleep 1; \
 printf 'GET /two HTTP/1.1\r\nHost: x\r\n\r\n') | nc 127.0.0.1 3000

Output:

headers complete
 GET /one HTTP/1.1
Host: x

headers complete
 GET /one HTTP/1.1      <-- /one again?!
Host: x

GET /two HTTP/1.1
Host: x

/one appeared twice. Tracing the buffer explains it:

chunk 1:  buf = "GET /one ...\r\n\r\n"
          indexOf finds the marker → handle request ✓

chunk 2:  buf = "GET /one ...\r\n\r\n" + "GET /two ...\r\n\r\n"
                 ^ still here — never removed
          indexOf finds the FIRST marker → /one's, again

I handled /one but never removed it. So when chunk 2 arrived, the server believed the next request started at byte 0 — which was still GET /one.

That is a desync: the server's idea of where a request starts no longer matches where it really starts.

There was a second desync hiding in the if. If both requests arrive in one chunk, the if runs once, handles /one, and stops. /two sits in the buffer until some unrelated data shows up. If the server sent one response per request, responses would now be paired with the wrong requests (response queue poisoning — one user can receive another user's response).

4. The fix

Two changes:

  • Trim: after handling a request, cut the buffer at idx + 4 (the marker is 4 bytes). Keep only unhandled bytes.
  • Loop: use while instead of if, so every complete request already in the buffer gets handled.
let buf = Buffer.alloc(0);

socket.on("data", (chunk: Buffer) => {
    buf = Buffer.concat([buf, chunk]);
    let idx = buf.indexOf('\r\n\r\n');
    while (idx !== -1) {
        console.log("headers complete\n", buf.subarray(0, idx + 4).toString());
        buf = buf.subarray(idx + 4);      // drop the handled request
        idx = buf.indexOf('\r\n\r\n');    // search again, or loop forever
    }
});

The last line matters: without re-searching, idx never changes and the loop spins at 100% CPU — a DoS on attacker input.

Both cases now work — two chunks, and both requests in a single chunk:

printf 'GET /one HTTP/1.1\r\nHost: x\r\n\r\nGET /two HTTP/1.1\r\nHost: x\r\n\r\n' | nc 127.0.0.1 3000
headers complete
 GET /one HTTP/1.1
Host: x

headers complete
 GET /two HTTP/1.1
Host: x

5. From my bug to request smuggling

My bug was one server miscounting where a request ends.

Request smuggling is two servers — a front-end proxy and a backend — each counting correctly by their own rules, but with different rules. A simplified CL.TE example:

Attacker sends to the proxy:

  POST / HTTP/1.1
  Content-Length: 44
  Transfer-Encoding: chunked

  0

  GET /admin HTTP/1.1
  ...

Proxy uses Content-Length     → "one request" → forwards everything
Backend uses Transfer-Encoding → request ends at "0\r\n\r\n"
                               → "GET /admin ..." is LEFTOVER in its buffer
Next victim's request arrives  → glued after the leftover
                               → backend runs GET /admin with the victim's cookies

That leftover is exactly what buf.subarray(idx + 4) holds in my code: bytes waiting to become the next request. Smuggling is an attacker choosing what those bytes are, by making the proxy and backend disagree on where idx is.

Takeaways

  • TCP delivers bytes in arbitrary pieces; the server must frame requests itself.
  • Search the accumulated buffer, not the current chunk.
  • After handling a request, trim it — otherwise the next request is read from the wrong offset.
  • Use a loop, not an if — one chunk can contain several requests.
  • Hunting tip: in someone else's parser, look for a buffer that is appended to but never sliced, or an if where a loop belongs. For smuggling, always ask: do these two servers find the end of a request at the same byte?

Still open: a request that never sends \r\n\r\n makes the buffer grow forever (header bomb). That's next.