Netty-Powered Client Pushes 65K Events Per Second
The Netty redesign of riemann-java-client has unlocked a fully asynchronous write path, yielding dramatic gains for workloads that send small, individual messages. With a small queue of pipelined write promises, the riemann-clojure-client can now push 65K events per second over a single TCP socket—sustaining roughly 120 mbps of traffic.
Bulk Throughput Improves Tenfold
Bulk performance has seen an even more striking improvement. Three threads sharing a single socket, sending messages of 100 events each, achieve approximately 185-200K events per second at over 200 mbps. Earlier tests required 10 sockets and hundreds of threads to reach comparable numbers.
Server-Side Implications
While pushing this much data from a single client is unlikely to be a common use case, the impact on Riemann’s server performance is significant. During the bulk test, the server consumed roughly 115% CPU on a 2.5GHz Q8300. This suggests that a million events per second is within reach on production hardware, though CAS contention in the streams could emerge as a bottleneck at that scale. A sufficiently powerful test box and network would be needed to confirm.
This marks the final major improvement planned for Riemann 0.2.0. Remaining work focuses on packaging and documentation.
