A Serialization Bottleneck Hiding in Netty's Pipeline
Riemann's TCP protocol is deliberately simple: a client sends a Msg, the server responds with a Msg. Each message is a Protocol Buffers object, prefixed by a four-byte length header that tells the receiver how many bytes to read before parsing the next frame.
That simplicity is deceptive. The path from "client calls send" to "server handles message" passes through a stack of queues—Java IO buffers, kernel TCP buffers, network cards, and the wires themselves. Because throughput is often latency-bound, two optimizations matter most: pipelining (sending many requests before waiting for acks) and batching (packing many events into one Msg). Batching lets the server parse 100 events in a single shot, trading higher latency for dramatically better throughput.
The Pipeline Splits at the Execution Handler
Riemann's Netty pipeline runs on two thread pools. The IO worker pool shuttles bytes from the connection buffer through the early pipeline stages. Eventually an ExecutionHandler takes over, handing the decoded message to a second, handler pool that does the real work—processing events and queries.
Where that ExecutionHandler sits matters. The standard advice is to place it near the end of the pipeline, after the fast decode steps, so long-running work never blocks an IO thread. That's what Riemann did.
But the handoff between pools isn't free. The ExecutionHandler doesn't just pass a reference to the decoded Protobuf object; it enqueues that object into a bounded queue on the handler side. The queue isn't limited by item count, since items vary wildly in size—it's limited by estimated memory. To estimate that, Netty uses a DefaultObjectSizeEstimator.
Estimating Size by Reflection
The estimator knows ChannelBuffers and byte arrays. It knows nothing about Riemann's decoded Msg objects. Faced with an unknown type, it falls back to reflection, recursively walking the object's fields to sum up their sizes.
That reflection walk was the bottleneck. Profiling with Yourkit showed DefaultObjectSizeEstimator consuming 38% of Riemann's CPU—time spent not processing events but guessing how much memory a Protobuf object would occupy on the heap.
The fix had two parts. The first option was to supply a custom ObjectSizeEstimator that knows about Riemann's types. The second was simpler: don't enqueue decoded objects at all. Move the ExecutionHandler earlier in the pipeline, before decoding, so it queues ChannelBuffers whose sizes it can compute in constant time.
That single-line move doubled throughput and halved minimum latency. The benchmark ran on a quad-core Q8300 over loopback, pushing roughly 20 MB/sec. Riemann processed 200,000–215,000 events per second, measured in messages of 100 events each. The synchronous client.sendEvents(anEvent) latency dropped correspondingly, with the tail of the time series showing the pipelining client draining its outbound queue.
Why the Obvious Fix Took 15 Hours
The code that caused this wasn't written in one sitting. It evolved. The UDP server, TCP server, and Graphite server share pipeline code but behave differently. Performance improvements were layered on piecemeal, each change trying to minimize API disruption. Function names stayed stable while the pipeline underneath them drifted into several interacting pieces.
Netty's flexibility—hundreds of names, deeply composable pipeline stages—became a liability. By the time the profiler flagged the estimator, understanding what a channel pipeline factory was, let alone how it was constructed, required significant archaeology.
The resolution came from clarity: expressing the pipeline structure in a way that made the data flow explicit. Once the pieces were visible, the interaction was obvious, and the fix took a half-hour. The lesson isn't about Netty or Protobuf specifically—it's that a well-structured pipeline is debuggable, and a tangled one isn't. Forty percent of CPU time can hide behind a name like DefaultObjectSizeEstimator when no one can see exactly where it sits in the flow.
