Abstract
Many mobile applications are based on cloud services such as location service, messaging service, etc. Currently most cloud services are based on statically prepared information rather than the real-time analytics results of dynamically captured events. A paradigm shift is to take Continuous Stream Analytics (CSA) as a cloud service, which, however, poses several specific challenges in scalability, latency, time-window semantics and transaction control.
In this work we extend the SQL query engine to unify the processing of static relations and dynamic streams for providing the platform support of CSA service. This platform is significantly differentiated from the current generation of stream processing systems which are in general built separately from the database engine thus unable to take advantage of the functionalities already offered by the existing data management technology, and suffer from the overhead of inter-platform data access and movement.
To capture the window semantics in CSA, we introduce the cycle-based query model and support it in terms of the cut-and-rewind query execution mechanism. This mechanism allows a SQL query to run cycle by cycle for processing the unbounded stream data chunk by chunk, but without shutting the query instance down between chunks for continuously maintaining the application state across the execution cycles, as required by sliding-window oriented operations. We also propose the cycle-based transaction model with cycle-based isolation and visibility. To scale-up analytics computation, we introduce the parallel infrastructure with multi-engines cooperated and synchronized based the common data chunking criteria without centralized coordination. To scale-up service provisioning, we investigate how to stage the continuously generated analytics results efficiently through metadata manipulation without physical data moving and copying.
We have prototyped our approach by extending the PostgreSQL, resulting in a new kind of tightly integrated, highly efficient platform for providing CSA service. We tested the throughput and latency of this service using a well-known stream processing benchmark and with WebOS based Palm phones. The test results show that the proposed approach is highly competitive. Providing CSA cloud service using HP Neoview parallel database engine is currently explored.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arasu, A., Babu, S., Widom, J.: The CQL Continuous Query Language: Semantic Foundations and Query Execution. VLDB Journal 15(2) (June 2006)
Chandrasekaran, S., et al.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: CIDR 2003 (2003)
Chen, Q., Hsu, M.: Experience in Extending Query Engine for Continuous Analytics, Tech. Rep. HPL-2010-44 (2010)
Chen, Q., Hsu, M.: Cooperating SQL Dataflow Processes for In-DB Analytics. In: Proc. CoopIS 2009 (2009)
Chen, Q., Therber, A., Hsu, M., Zeller, H., Zhang, B., Wu, R.: Efficiently Support Map-Reduce alike Computation Models Inside Parallel DBMS. In: Proc. Thirteenth International Database Engineering & Applications Symposium (IDEAS 2009) (2009)
Chen, Q., Hsu, M., Liu, R.: Extend UDF Technology for Integrated Analytics. In: Proc. DaWaK 2009 (2009)
Dean, J.: Experiences with MapReduce, an abstraction for large-scale computation. In: Int. Conf. on Parallel Architecture and Compilation Techniques. ACM, New York (2006)
DeWitt, D.J., Paulson, E., Robinson, E., Naughton, J., Royalty, J., Shankar, S., Krioukov, A.: Clustera: An Integrated Computation And Data Management System. In: VLDB 2008 (2008)
Franklin, M.J., et al.: Continuous Analytics: Rethinking Query Processing in a Network-Effect World. In: CIDR 2009 (2009)
Gedik, B., Andrade, H., Wu, K.-L., Yu, P.S., Doo, M.C.: SPADE: The System S Declarative Stream Processing Engine. In: ACM SIGMOD 2008 (2008)
Jain, N., et al.: Design, Implementation, and Evaluation of the Linear Road Benchmark on the Stream Processing Core. In: SIGMOD (2006)
Liarou, E., et al.: Exploiting the Power of Relational Databases for Efficient Stream Processing. In: EDBT 2009 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, Q., Hsu, M. (2010). Data Stream Analytics as Cloud Service for Mobile Applications. In: Meersman, R., Dillon, T., Herrero, P. (eds) On the Move to Meaningful Internet Systems, OTM 2010. OTM 2010. Lecture Notes in Computer Science, vol 6427. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16949-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-16949-6_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16948-9
Online ISBN: 978-3-642-16949-6
eBook Packages: Computer ScienceComputer Science (R0)