Kestrel: A Bounded HTTP Server, dlopen libcurl, and a Hand-Built WebSocket

Last time we looked at how Kestrel frames its local messages with a length prefix and protects its loopback links. Today we step out to the edges. A tiny HTTP server that refuses to be overwhelmed. A libcurl client that is never linked. And a WebSocket client written by hand, byte by byte.

• HttpServer
• curl_client
• ws_client

Here is a strange design goal. A trading binary that can run backtests on a laptop with no curl installed at all, yet can still talk HTTPS and WebSockets when it is deployed. How do you get both? The answer is a shared library plugged in at runtime, and about a hundred lines of careful bit manipulation.

• Run without curl
• Use curl when present

We will cover four things in order. First, the bounded HTTP server and its limits. Second, loading libcurl with dlopen and what we demand from every request. Third, the WebSocket handshake. And fourth, the frame format, with a worked example in real hexadecimal bytes. A note: message contents come in episode twenty.

• Server limits
• dlopen
• Handshake
• Frame bytes

The HttpServer has a simple shape. One accept thread polls the listening socket every two hundred milliseconds. Behind it sit a fixed number of worker threads. Between them is a queue, and that queue is bounded. The accept thread never does the slow work of reading a request. It only hands connections off.

• One accept thread
• Poll every 200 ms
• N workers

The queue holds at most sixty-four pending connections. When the sixty-fifth arrives, the server does not wait and does not grow memory. It simply closes that connection. Think of a bouncer at a full door. A client that gets turned away can retry; a server that accepts everything eventually falls over.

• Queue capacity: 64
• 65th connection closed

An unbounded queue looks friendly but hides a failure. If requests arrive faster than workers finish them, the queue grows, latency grows, and memory grows, until something breaks far from the cause. A bound turns that slow disaster into an immediate, visible refusal. This server is local tooling, so predictable behaviour beats politeness.

• Unbounded: silent growth
• Bounded: loud refusal

Every connection gets the same treatment. Socket timeouts of two seconds, for reading and writing. Headers capped at sixteen kibibytes; exceed that and the reply is status 431. A body capped at one mebibyte; exceed that and the reply is 413. And exactly one request per connection, with Connection close.

• 2 s timeouts
• Headers: 16 KiB, 431
• Body: 1 MiB, 413
• Connection: close

Why the two second timeout? Imagine a client that opens a connection and sends one byte a minute. With no timeout it would hold a worker forever. A handful of such clients would starve the entire pool. The timeout makes every worker's time budget finite, so a worker always comes back to the queue.

• Slow clients hold workers
• Timeout frees the worker

The header limit is the pitfall to remember. A naive loop reads until it sees a blank line. A hostile or buggy client can simply never send one, and your buffer grows without end. Kestrel stops at sixteen kibibytes and answers four thirty-one, request header fields too large. The read is bounded before it begins.

• Read until blank line? Dangerous
• Stop at 16 KiB

One more detail. If accept fails because the process ran out of file descriptors, the classic EMFILE style error, a naive loop retries instantly and burns a whole core. Kestrel pauses instead, then tries again. And the scope is deliberately narrow: JSON only, a request for slash is a 404, and the C++ serves no static files. The browser gateway lives elsewhere.

• Accept error: pause
• JSON only
• GET / is 404

Now the client side. Normally you link libcurl at build time. That means curl headers on every build machine, and the library on every machine that runs the binary. For a research box that only runs offline backtests, that is a pointless requirement. So Kestrel does not link curl. It loads it while running.

• Link: required to start
• dlopen: optional

The file curl_client dot cpp calls dlopen on libcurl dot so dot four. If that works, it calls dlsym nine times, once for each function it needs, and stores the pointers. If any step fails, the client is marked unavailable and it records a clear reason. The binary keeps running; only HTTPS features are off.

• dlopen libcurl.so.4
• dlsym nine functions
• Missing? Report reason

Without curl headers, you also lack the named constants. So the option numbers are hard-coded. Curl groups options by type, with base offsets such as the object-point base. The URL option is that base plus two. This is a simplified view: the idea is a small table of named constants that mirror curl's stable numbering.

• No headers, no constants
• Base offset plus index

The trade-off is honest. You take on the duty of keeping those numbers right, and a typo is a runtime surprise instead of a compile error. In return, the build has no curl dependency, the research machines stay lean, and a missing library produces one clear message instead of a loader failure at startup.

• Cost: constants are manual
• Gain: no build dependency

Every request is HTTPS only. The protocols option is set to https, and the redirect protocols option as well. That second one matters: a server could answer with a redirect to plain http, and without the restriction the client would follow it quietly. Peer and host verification are on, and HTTP two is optional.

• Protocols: https only
• Redirects: https only
• Verify peer and host

A curious detail about the source. Kestrel's rule is that C++ files contain no comments, and a script enforces it. A literal https colon slash slash would contain two slashes in a row. So the code assembles the URL from pieces: the word https, then the separator, then the host.

• Rule: zero comments
• Assemble: https + :// + host

A response can be any size, so the write callback enforces a maximum body. Each time curl hands over a chunk, the callback checks whether the total would pass the limit. If so, it returns a short count, curl treats that as an error, and the transfer stops. Memory use is capped no matter what the server sends.

• Write callback counts bytes
• Over limit: abort transfer

Shutdown needs care. If kestrel_md is waiting on a slow request, it should not hang. Curl offers a progress callback, the XFERINFO one, which runs regularly during a transfer. Kestrel checks a cancel flag there and returns nonzero to abort. In practice in-flight requests are cancelled within about one second of shutdown.

• XFERINFO callback
• Cancel flag checked
• Abort within about 1 s

Now ws_client dot cpp. It implements RFC 6455, the WebSocket protocol, on top of an interface called IByteStream. In production that stream is an OpenSSL TLS connection. In tests it is a fake stream that replays scripted bytes. Because the client only sees bytes, the same code runs against both.

• RFC 6455 client
• IByteStream interface
• TLS in prod, fake in tests

A WebSocket begins life as an ordinary HTTP request asking to upgrade. The client sends a random sixteen-byte key, encoded in base64, in the Sec-WebSocket-Key header. The key is random each time, so a cached or replayed answer from some proxy cannot be mistaken for a fresh handshake.

• HTTP request with Upgrade
• Random Sec-WebSocket-Key

The server must answer with a header called Sec-WebSocket-Accept. Its value is the key, joined with a fixed GUID from the RFC, hashed with SHA-1, then base64 encoded. The client computes the same value itself and compares. A mismatch means this is not a real WebSocket server, and the connection is dropped.

• key + GUID
• SHA-1, then base64
• Compare, or disconnect

After the upgrade, two rules apply. Client frames are always masked, with four fresh random bytes each. Server frames must not be masked, and a masked server frame is treated as a protocol violation. Fragmented messages are reassembled, a ping is answered with a pong automatically, and closing uses a handshake with a timeout.

• Client frames masked
• Server frames unmasked
• Auto pong, close timeout

Now the frame itself. Byte zero packs two things. The top bit, hexadecimal eighty, is FIN: one means this is the last fragment of a message. The low four bits are the opcode. One is text, two is binary, eight is close, nine is ping, and ten is pong.

• FIN bit: 0x80
• Opcode: low four bits
• 1 text, 2 binary

Byte one starts with the mask bit, also hexadecimal eighty. The other seven bits hold the length. Under one hundred twenty-six, the length sits right there. A value of one twenty-six means a sixteen-bit length follows. A value of one twenty-seven means a sixty-four-bit length follows. Small messages stay small.

• MASK bit 0x80
• Under 126: inline
• 126: 16-bit, 127: 64-bit

Here is the encoder, trimmed. The first byte ORs the FIN bit with the opcode. Then the mask bit is chosen from whether a mask pointer was given. Short payloads write one length byte. Medium ones write one twenty-six and two bytes, high byte first. Large ones write one twenty-seven and eight bytes, shifting from fifty-six down to zero.

• FIN | opcode
• Three length forms
• Big-endian bytes

Notice the shifts. The length is written most significant byte first, called network byte order. Writing it with shifts and masks, rather than copying the integer's memory, means the code is correct on any machine, whatever its own endianness. It is also easy to test, because each byte is explicit.

• Most significant byte first
• Shifts, not memcpy

After the length come four mask bytes, then the payload. The payload is not sent as is. Byte i is XORed with mask byte i and three, which means the four mask bytes cycle: zero, one, two, three, zero, one, and so on. The server applies the same XOR to recover the original, because XOR is its own inverse.

• Payload[i] XOR mask[i and 3]
• Mask cycles every 4 bytes
• XOR twice restores

The code appends the four mask bytes, then loops over the payload in place. For each index it takes the stored byte, XORs it with mask at index and three, and writes it back. Here start is where the payload begins in the output. The bitwise and with three is a cheap modulo four.

• Append 4 mask bytes
• XOR in place
• i & 3 is i mod 4

Let us encode a real payload. The FYERS ping is three bytes: zero zero, zero one, zero B. We send it as a binary frame, so the opcode is two and FIN is set. Byte zero is eighty OR two, which is eighty-two. We pick the mask one two, three four, five six, seven eight.

• Payload: 00 01 0B
• Byte 0: 82
• Mask: 12 34 56 78

The payload length is three, well under one twenty-six, so it goes inline. The mask bit eighty ORed with three gives eighty-three. So the two header bytes are eighty-two and eighty-three. Then the four mask bytes, copied as they are, one two, three four, five six, seven eight.

• Length 3: inline
• Byte 1: 80 | 03 = 83
• Header: 82 83

Now the payload. Zero zero XOR one two is one two. Zero one XOR three four is three five. Zero B XOR five six: zero B is binary zero zero zero zero one zero one one, five six is zero one zero one zero one one zero, and XOR gives zero one zero one one one zero one, which is five D.

• 00 ^ 12 = 12
• 01 ^ 34 = 35
• 0B ^ 56 = 5D

Putting it together, the nine bytes on the wire are: eighty-two, eighty-three, one two, three four, five six, seven eight, one two, three five, five D. The server XORs the last three with the same mask and gets zero zero, zero one, zero B back. For a three hundred byte payload, the length field would be seven E, zero one, two C, with the mask bit making the first byte F E.

• 82 83 12 34 56 78 12 35 5D
• 300 bytes: 7E 01 2C

Pitfall one: forgetting the mask. A compliant server must close the connection on an unmasked client frame, so the bug shows up as a mysterious disconnect right after the first message. Always masking, with fresh random bytes per frame, is not optional ceremony. The protocol requires it.

• Unmasked frame: server closes
• Mask every client frame

Pitfall two: linking curl directly, so every research machine must have it just to run a backtest. Pitfall three: unbounded reads, whether headers on the server or bodies in the client. Each has the same cure. Decide the limit first, enforce it at the edge, and fail with a clear reason.

• Hard-linking curl
• Unbounded reads
• Limit first, fail clearly

An honest note. The FYERS binary websocket decoder has been tested only against a fake server, and its protocol was reverse-engineered from the official SDKs, with some fields marked unverified. The code logs a one-time protocol facts line on the first real connection. Treat this as design, not production proof.

• Tested against a fake server
• Some fields UNVERIFIED
• One-time protocol facts log

To recap. The HttpServer has one acceptor, bounded queue of sixty-four, and strict limits on time, headers and body. The curl client is loaded with dlopen, restricted to HTTPS, size-capped and cancellable. The WebSocket client verifies the handshake, masks every frame, and encodes lengths in three forms. All of it is bounded and explicit.

• Bounded server
• Runtime libcurl
• Hand-built frames

Next episode we open those frames and decode the FYERS binary market-data protocol inside them. This series is educational, not financial advice.

• Next: FYERS binary protocol
• Subscribe