Riemann 0.1.3: Consolidation and New Stream Tools

Riemann 0.1.3 is now available as a tarball or Debian package. This release is primarily a consolidation effort—roughly 2,800 insertions and 1,400 deletions across the codebase—bringing in months of community contributions focused on bug fixes, performance, third-party integrations, and structural cleanup. Several organizations, including Boundary, Blue Mountain Capital, Librato, and Netflix, contributed code, design input, and funding.

Fixes and Configuration Cleanup

A number of behavioral and packaging issues were addressed:

  • streams/tagged-all and tagged-any now accept single strings, not just vectors of tags.
  • The bin/riemann scripts now exec java directly rather than spawning a subprocess.
  • Servers bind to 127.0.0.1 by default instead of potentially binding only to IPv6 localhost.
  • The obsolete :metric_f key in default package configs has been corrected.
  • Default configs have been reorganized and commented more thoroughly to reduce user confusion.
  • Deb packages now treat /etc/riemann/riemann.config as a conffile, avoiding unwanted overwrites on upgrade.

New Streams and Window APIs

Several new stream primitives expand the ways events can be filtered and aggregated:

  • where*: functions as where, but accepts a function instead of an expression.
  • smap and sreduce: streaming map and reduce operations.
  • fold-interval: reduces over time periods.
  • fixed-time-window: emits sets of events every n seconds.
  • fixed-event-window: emits sets of n disjoint events.
  • moving-time-window: passes the last n seconds of events.
  • moving-event-window: passes the last n events.

The stream window system has been unified under one API, providing both fixed and moving windows over time or event counts.

Adapters, Scheduling, and Other Enhancements

Notable additions and improvements in this release include:

  • The Librato metrics adapter is now built in.
  • A Graphite server component (riemann.graphite) accepts events in the Graphite protocol format.
  • Scheduled tasks are more accurate and consume fewer threads; Riemann’s clock can switch between wall-clock and virtual modes for faster, more reliable testing.
  • New riemann.time namespace for controllable centralized clocks and task schedulers.
  • New riemann.pool namespace providing a thread-safe, fixed-size, bounded-latency regenerating resource pool.
  • where now supports an (else) clause for streams invoked when an event does not match the expression.
  • TCP and UDP servers see about a 15% throughput gain.
  • A new Match protocol unifies predicate handling (functions, values, regexes) and is used by where and match.
  • Useless multimethods were converted to regular methods; numerous concurrency improvements made.
  • The PagerDuty adapter is now enabled by default in the configuration.
  • The Graphite adapter now uses a connection pool with proper reconnection and bounded latency.
  • Email formatting includes additional state information in the message body.
  • Indexes are now seqable.
  • Protocol buffer parsing paths were unified; Travis-CI support added.
  • Packaging tasks were added for lein2; tests were revised for clarity and speed.

Also included is riemann.deps, an experimental dependency resolver with an API that is still subject to change.

Looking Ahead

The riemann-tools project has accumulated features that should be released soon. The dashboard is mid-transition between its old and new interfaces and needs to reach feature parity before a riemann-dash release. The website is also slated for a rework with a beginner-friendly howto section. In Riemann itself, potential next steps include Netty performance work and formal benchmark tooling for exploring concurrency tuning. Longer-term plans for 0.1.4 and 0.1.5 include persistent indexes and high availability.