Riak-pipe mapreduce stabilizes API latency after node failures
In the wake of a rough week for a riak cluster, the switch to riak-pipe mapreduce has noticeably improved performance. Previously, when a node went down, mapreduce times would spike dramatically, leading to sluggish API responses and occasional timeouts, particularly for mapreduce-heavy requests such as feeds and comments. With riak-pipe in place, those API latencies have dropped from a range of 3–7 seconds to a stable 600ms—still on the higher side, but far more tolerable for normal operation.
Mysterious recoverable errors persist
One lingering issue: the riak-pipe mapreduce implementation throws roughly a thousand apparently random, recoverable errors per day. These appear as unexplained map_reduce_error entries in the logs, or as cryptic messages like {"lineno":466,"message":"SyntaxError: syntax error","source":"()"}, even when the source is definitely not “()”. The root cause remains unidentified, though the errors seem loosely tied to which node is handling the request.
