Storm + Kafka is a very effective log processing solution. A number of users of Storm use this combination, including us at Twitter in a few instances. Kafka gives you a high throughput, reliable way to persist/replay log messages, and Storm gives you the ability to process those messages in arbitrarily complex ways.
http://www.drdobbs.com/open-source/easy-real-time-big-data-analysis-using-s/240143874
