Real-Time Analytics with Storm

Abstract

So far, you’ve seen how to work with batch data processing in Hadoop. Batch processing is used with data at rest. You typically generate a report at the end of the day. MapReduce, Hive, and HBase all help in implementing batch processing tasks. But there is another kind of data, which is in constant motion, called streams. To process such data, you need a real-time processing engine. A constant stream of click data for a campaign, user activity data, server logs, IoT, and sensor data—in all of these scenarios, data is constantly coming in and you need to process them in real time, perhaps within a window of time. Apache Storm is very well suited for real-time stream analytics. Storm is a distributed, fault-tolerant, open source computation system that processes data in real time and works on top of Hadoop.