Transactional Stream Processing
We can broadly define transactional stream processing as processing streaming data with correctness guarantees. These guarantees include not only properties that are intrinsic to stream processing (e.g., order, exactly-once semantics), but also ACID properties of traditional OLTP-oriented databases, which arise in streaming applications in case of shared mutable state or failures.
Stream processing emerged as a research area in the database community circa early 2000s. The initial focus of the community was on enabling relational-style query processing over ordered and unbounded data from push-based data sources such as sensors. New models, algorithms, and systems were developed to achieve low-latency continuous processing over streams arriving at high or unpredictable rates. Storing streaming data for longer term use beyond answering real-time continuous queries was not a primary concern. Thus, storage management was limited to...
- 1.Hwang JH, Balazinska M, Rasin A, Cetintemel U, Stonebraker M, Zdonik S. High-availability algorithms for distributed stream processing. In: Proceedings of the 21st International Conference on Data Engineering; 2005. p. 779–90.Google Scholar
- 2.Toshniwal A, Taneja S, Shukla A, Ramasamy K, Patel JM, Kulkarni S, et al. Storm @Twitter. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2014. p. 147–56.Google Scholar
- 3.Zaharia M, Das T, Li H, Hunter T, Shenker S, Stoica I. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 24th ACM Symposium on Operating System Principles; 2013. p. 423–38.Google Scholar
- 7.Ramnarayan J, Mozafari B, Wale S, Menon S, Kumar N, Bhanawat H, et al. SnappyData: streaming, transactions, and interactive analytics in a unified engine. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; (2016, to appear).Google Scholar
- 9.Golab L, Bijay KG, Ozsu MT. On concurrency control in sliding window queries over data streams. In: Advances in Database Technology, Proceedings of the 10th International Conference on Extending Database Technology; 2006. p. 608–26.Google Scholar
- 13.Botan I, Fischer PM, Kossmann D, Tatbul N. Transactional stream processing. In: Proceedings of the 15th International Conference on Extending Database Technology; 2012. p. 204–15.Google Scholar
- 14.Kreps J, Narkhede N, Rao J. Kafka: a distributed messaging system for log processing. In: Proceedings of the NetDB Workshop; 2011.Google Scholar