Skip to main content
SpringerLink
Log in

Real-Time Databases

  • Chapter
  • First Online:
Real-Time & Stream Data Management

Part of the book series: SpringerBriefs in Computer Science ((BRIEFSCOMPUTER))

Abstract

While traditional database systems are targeted at providing a consistent snapshot of the application domain, real-time databases acknowledge that data may evolve. Both the architectures and client APIs of real-time databases reflect that facts can change over time and that the system may have to enhance or correct issued information.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    While the depicted approach does not apply to set operations, an update that affects several data items can be transformed into a set of single-item updates to make the approach applicable.

  2. 2.

    InvaliDB’s name is derived from its usage within the Quaestor architecture [Ges+17] where it invalidates cached database queries. The Quaestor architecture implements a scheme for global caching of dynamic content, specifically database query results. Within the Quaestor architecture, InvaliDB is used to detect modifications to query results, so that stale caches can be invalidated in a timely manner.

  3. 3.

    Unlike Meteor’s oplog tailing which may receive incomplete write data (cf. p. 27), InvaliDB therefore does not require database access for real-time query maintenance.

  4. 4.

    For a discussion of how support for real-time aggregations and real-time join queries can be implemented with InvaliDB, see [Win18, Sec. 3.3.2].

  5. 5.

    The same applies to CouchDB’s commercial derivative Cloudant [Clo].

References

  1. Sten F. Andler and Jörgen Hansson, eds. Active, Real-Time, and Temporal Database Systems. Lecture Notes in Computer Science 1553. Springer Berlin Heidelberg, 1998.

    Google Scholar 

  2. J. Chris Anderson, Jan Lehnardt, and Noah Slater. CouchDB: The Definitive Guide. 1st. O’Reilly Media, Inc., 2010. isbn: 0596155891, 9780596155896.

    Google Scholar 

  3. Antonio Badia. “Quantifiers in Action: Generalized Quantification in Query, Logical and Natural Languages”. In: ed. by Ahmed K. Elmagarmid and Amit P. Sheth. Boston, MA: Springer US, 2009. Chap. Extensions, pp. 127–147. isbn: 978-0-387-09564-6. doi: 10.1007/978-0-387-09564-6_9. url: https://doi.org/10.1007/978-0-387-09564-6_9.

    Google Scholar 

  4. Jason Baker et al. “Megastore: Providing Scalable, for Interactive Services.” In: CIDR. Vol. 11. 2011, pp. 223–234. (Visited on 12/19/2014).

    Google Scholar 

  5. Shay Banon. “Percolator”. In: Elastic Blog (Feb. 2011). Accessed: 2017-11-17. url: https://www.elastic.co/blog/percolator.

  6. Baqend. Baqend. Accessed: 2018-05-10. 2018. url: https://www.baqend.com/.

  7. Nikolas Burk, Nilan Marktanner, et al. “GraphQL vs. Firebase”. In: Graphcool Docs (2017). Accessed: 2017-07-18. url: https://www.graph.cool/docs/tutorials/graphql-vs-firebasechi6oozus1/.

  8. Nikolas Burk, Nilan Marktanner, and Johannes Schickling. “How to build a Real-Time Chat with GraphQL Subscriptions and Apollo?” In: Graphcool Docs (2017). Accessed: 2017-07-18. url: https://www.graph.cool/docs/tutorials/worldchat-subscriptionsexample-ui0eizishe/.

  9. Pier Bover. “Firebase: the great, the meh, and the ugly”. In: freeCode-Camp Blog (Jan. 2017). Accessed: 2017-05-21. url: https://medium.freecodecamp.com/firebase-the-great-the-meh-and-the-ugly-a07252fbcf15.

  10. Cloudant NoSQL DB Docs: Filter Functions. Accessed: 2018-05-2. IBM. May 2018. url: https://console.bluemix.net/docs/services/Cloudant/api/design_documents.html#filter-functions.

  11. Tom Coleman. “The Oplog Observe Driver”. In: Meteor Documentation (2016). Accessed: 2017-10-16. url: https://github.com/meteor/docs/blob/cc3f8fe99b3db72c21ea2c0e8d7e574bca860ec6/long-form/oplog-observe-driver.md.

  12. Eric Daniels. “MongoDB Stitch - Backend as a Service (commentary)”. In: Hacker News (2017). Accessed 2017-11-17. url: https://news.ycombinator.com/item?id=14595456.

  13. A. Jesse Jiryu Davis. “New Driver Features for MongoDB 3.6 (commentary)”. In: emptysquare Blog (Oct. 2017). Accessed: 2017-11-17. url: https://emptysqua.re/blog/driver-featuresfor-mongodb-3-6/#comment-3574381334.

  14. A. Jesse Jiryu Davis. “New Driver Features for MongoDB 3.6: Notification API”. In: emptysquare Blog (June 2017). Accessed: 2018-04-23. url: https://emptysqua.re/blog/driver-featuresfor-mongodb-3-6/.

  15. Luigi Dell’Aquila. “LiveQuery”. In: OrientDB Blog (Oct. 2015). Accessed: 2017-07-09. url: http://orientdb.com/livequery/.

  16. David Drobik. “Will Google Build Your Product?” In: David Drobik – Blog (Oct. 2017). Accessed: 2017-12-23. url: https://medium.com/@daviddrobik/will-google-build-your-product-56508d19524a.

  17. Alex Dufetel. “Introducing Cloud Firestore: Our New Document Database for Apps”. In: Firebase Blog (Oct. 2017). Accessed: 2017-12-19. url: https://firebase.googleblog.com/2017/10/introducing-cloud-firestore.html.

  18. Elasticsearch. Elasticsearch. Accessed: 2018-05-10. 2018. URL: https://www.elastic.co/products/elasticsearch/.

  19. Joakim Eriksson. “Real-Time and Active Databases: A Survey”. In: Active, Real-Time, and Temporal Database Systems: Second International Workshop, ARTDB-97 Como, Italy, September 8–9, 1997 Proceedings. Ed. by Sten F. Andler and Jörgen Hansson. Berlin, Heidelberg: Springer Berlin Heidelberg, 1998, pp. 1–23. isbn: 978-3-540-49151-4. doi: 10.1007/3-540-49151-1_1. url: http://dx.doi.org/10.1007/3-540-49151-1_1.

  20. Choose a Database: Cloud Firestore or Realtime Database. Accessed: 2017-12-19. Firebase. Dec. 2017. url: https://firebase.google.com/docs/firestore/rtdb-vs-firestore.

  21. Firestore: Quotas and Limits. Accessed: 2017-12-19. Firebase. Dec. 2017. url: https://firebase.google.com/docs/firestore/quotas.

  22. Order and Limit Data with Cloud Firestore. Accessed: 2017-12-19. Firebase. Dec. 2017. url: https://firebase.google.com/docs/firestore/query-data/order-limit-data.

  23. Perform Simple and Compound Queries in Cloud Firestore. Accessed: 2017-12-19. Firebase. Dec. 2017. url: https://firebase.google.com/docs/firestore/query-data/queries.

  24. Realtime Database Limits. Accessed: 2017-11-17. Firebase. Nov. 2017. url: https://firebase.google.com/docs/database/usage/limits.

  25. Firebase. Firebase. Accessed: 2016-09-18. 2016. url: https://firebase.google.com/.

  26. Firebase. “Best practices for data structure: Avoid nesting data”. In: Firebase Docs (2017). Accessed: 2017-05-21. url: https://firebase.google.com/docs/database/web/structure-data#avoid_nesting_data.

  27. Felix Gessert et al. “Quaestor: Query Web Caching for Database-as-a-Service Providers”. In: Proceedings of the 43rd International Conference on Very Large Data Bases (2017).

    Google Scholar 

  28. Felix Gessert. “The AWS and MongoDB Infrastructure of Parse: Lessons Learned”. In: Baqend Tech Blog (Jan. 2017). Accessed: 2017-11-29. url: https://medium.baqend.com/parse-is-gone-afew-secrets-about-their-infrastructure-91b3ab2fcf71.

  29. Google. Google Cloud Datastore. Accessed: 2018-05-10. 2018. url: https://cloud.google.com/datastore/.

  30. Martijn van Groningen. “Elasticsearch Percolator Continues to Evolve”. In: Elastic Blog (June 2016). Accessed: 2017-11-17. url: https://www.elastic.co/blog/elasticsearch-percolatorcontinues-to-evolve.

  31. Baptiste Jamin. “Reasons Not To Use Firebase”. In: Chris Blog (Sept. 2016). Accessed: 2017-05-21. url: https://crisp.im/blog/why-you-should-never-use-firebase-realtime-database/.

  32. Jacob Katzen et al. Oplog tailing too far behind not helping. Accessed: 2017-07-09. 2015. url: https://forums.meteor.com/t/oplog-tailing-too-far-behind-not-helping/2235.

  33. Todd Kerpelman. “Cloud Firestore for Realtime Database Developers”. In: Firebase Blog (Oct. 2017). Accessed: 2017-12-23. url: https://firebase.googleblog.com/2017/10/cloud-firestorefor-rtdb-developers.html.

  34. Kevin Lacker. “Moving On”. In: Parse Blog (Jan. 2016). Accessed: 2017-11-18. url: http://blog.parseplatform.org/announcements/moving-on/.

  35. Michael Lehenbauer. “Firebase: Now with more querying!” In: Firebase Blog (Nov. 2014). Accessed: 2017-12-23. url: https://firebase.googleblog.com/2014/11/firebase-now-with-morequerying.html.

  36. Andrew Mao et al. My experience hitting limits on Meteor performance. Accessed: 2017-07-09. 2014. url: https://groups.google.com/forum/#!topic/meteor-talk/Y547Hh2z39Y.

  37. Watts Martin. “Changefeeds in RethinkDB”. In: RethinkDB Docs (2015). Accessed: 2017-07-09. URL: https://rethinkdb.com/docs/changefeeds/javascript/#changefeeds-with-filteringand-aggregation-queries.

  38. Watts Martin. “Table joins in RethinkDB”. In: RethinkDB Docs (2016). Accessed: 2017-11-17. url: https://www.rethinkdb.com/docs/table-joins/.

  39. Daniel Mewes. “Scaling, sharding and replication: Running a proxy node”. In: RethinkDB Docs (2016). Accessed: 2017-07-09. URL: https://rethinkdb.com/docs/sharding-and-replication/#running-a-proxy-node.

  40. MongoDB CRUD Concepts: Tailable Cursor. Accessed: 2017-11-13. MongoDB Inc. 2017. url: https://docs.mongodb.com/manual/core/tailable-cursors/.

  41. Frank van Puffelen. “Have you met the Realtime Database?” In: Firebase Blog (July 2016). Accessed: 2017-05-20. url: https://firebase.googleblog.com/2016/07/have-you-met-realtimedatabase.html.

  42. B. Purimetla et al. A Study of Distributed Real-Time Active Database Applications. Tech. rep. Amherst, MA, USA, 1993.

    Google Scholar 

  43. Rapid Docs: Collection. Accessed: 2017-10-06. STRV s.r.o. 2017. url: https://www.rapidrealtime.com/docs/api-reference/javascript/collection.

  44. Realm. “Realm Java 3.0: Collection Notifications, Snapshots and Sorting Across Relationships”. In: Realm Blog (Feb. 2017). Accessed: 2017-07-09. url: https://news.realm.io/news/realm-java-3-0-collection-notifications/.

  45. Realm. Realm. Accessed: 2018-05-10. 2018. url: https://realm.io/.

  46. RethinkDB. RethinkDB. Accessed: 2016-09-18. 2016. URL: https://www.rethinkdb.com/.

  47. Kato Richardson. “Queries, Part 1: Common SQL Queries Converted for Firebase”. In: Firebase Blog (Oct. 2013). Accessed: 2017-12-23. url: https://firebase.googleblog.com/2013/10/queries-part-1-common-sql-queries.html.

  48. Kato Richardson. “Queries, Part 2: Advanced Searches with Firebase, made Plug-and-Play Simple”. In: Firebase Blog (Jan. 2014). Accessed: 2017-12-23. url: https://firebase.googleblog.com/2014/10/firebase-is-joining-google.html.

  49. Ian Thomas Rose. “Real-Time Query Systems for Complex Data Sources”. PhD thesis. Harvard University Cambridge, Massachusetts, 2011.

    Google Scholar 

  50. Alex Rose. “Firebase: The Good, Bad, and the Ugly”. In: Raizlabs Developer Blog (Dec. 2016). Accessed: 2017-05-21. url: https://www.raizlabs.com/dev/2016/12/firebase-case-study/.

  51. Nic Raboy, Priya Rajagopal, and Eric Maxwell. “NDP Episode #19: Mobile Development with Realm”. In: Couchbase Blog (June 2017). audio podcast (explanation by Eric Maxwell, starting at 10:57); Accessed: 2017-09-13. url: https://blog.couchbase.com/ndp-episode-19-mobile-development-realm/.

  52. Custom Build. Accessed: 2017-10-16. RxDB. 2017. url: https://pubkey.github.io/rxdb/custom-build.html.

  53. Query Change Detection. Accessed: 2017-10-16. RxDB. 2017. url: https://pubkey.github.io/rxdb/query-change-detection.html.

  54. RxDB. RxDB. Accessed: 2018-05-10. 2018. url: https://github.com/pubkey/rxdb.

  55. Salvatore Sanfilippo. Redis. Accessed: 2018-05-10. 2018. URL: https://redis.io/.

  56. Pramod J. Sadalage and Martin Fowler. NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence. 1st. Addison-Wesley Professional, 2012. ISBN: 0321826620, 9780321826626.

    Google Scholar 

  57. John A. Stankovic. “Misconceptions About Real-Time Computing: A Serious Problem for Next-Generation Systems”. In: Computer 21.10 (Oct. 1988), pp. 10–19. issn: 0018–9162. doi: 10.1109/2.7053. url: http://dx.doi.org/10.1109/2.7053.

    Article  Google Scholar 

  58. William Stein. “RethinkDB versus PostgreSQL: my personal experience”. In: CoCalc Blog (Feb. 2017). Accessed: 2017-07-09. url: https://blog.sagemath.com/2017/02/09/rethinkdb-vspostgres.html.

  59. James Tamplin. “Firebase is Joining Google!” In: Firebase Blog (Oct. 2014). Accessed: 2017-11-17. URL: https://firebase.googleblog.com/2014/10/firebase-is-joining-google.html.

  60. James Tamplin. “Firebase expands to become a unified app platform”. In: Firebase Blog (May 2016). Accessed: 2017-12-23. url: https://firebase.googleblog.com/2016/05/firebase-expands-to-become-unified-app-platform.html.

  61. James Tamplin. “Cloud Firestore: A New Document Database for Apps (commentary)”. In: Hacker News (2017). Accessed 2017-12-19. url: https://news.ycombinator.com/item?id=15393499.

  62. Ankit Toshniwal et al. “Storm@Twitter”. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. SIGMOD’14. Snowbird, Utah, USA: ACM, 2014, pp. 147–156. isbn: 978-1-4503-2376-5. doi: 10.1145/2588555.2595641. url: http://doi.acm.org/10.1145/2588555.2595641.

  63. Lawrence To, Viv Schupmann, et al. Oracle Database High Availability Best Practices 11g Release 1 (11.1). Accessed: 2017-05-13. Oracle. Dec. 2009. url: https://docs.oracle.com/cd/B28359_01/server.111/b28282/glossary.htm#CHDIDADC.

  64. MengyanWang. “Parse LiveQuery Protocol Specification”. In: GitHub: ParsePlatform/parse-server (Mar. 2016). Accessed: 2017-11-18. url: https://github.com/parse-community/parse-server/wiki/Parse-LiveQuery-Protocol-Specification.

  65. Mengyan Wang. “Parse Server Goes Realtime with Live Queries”. In: Parse Blog (Mar. 2016). Accessed: 2017-11-18. url: http://blog.parseplatform.org/announcements/parse-server-goes-realtime-with-live-queries/.

  66. Jacob Wenger. “List chat group in order of most recently posted”. In: Firebase Google Group (June 2015). Accessed: 2017-07-09. url: https://groups.google.com/forum/#!msg/firebase-talk/d-XjaBVL2Ko/TmkIep44lGgJ.

  67. Wolfram Wingerath. “Scalable Push-Based Real-Time Queries on Top of Pull-Based Databases”. PhD thesis. University of Hamburg, 2019. url: https://invalidb.info/thesis.

  68. David Workman et al. “Large number of operations hangs server”. In: Meteor GitHub Issues (2014). Accessed: 2016-10-01. url: https://github.com/meteor/meteor/issues/2668.

  69. Apache Software Foundation. CouchDB. Accessed: 2018-05-10. 2018. url: http://couchdb.apache.org/.

  70. CoCalc by SageMath, Inc. CoCalc. Accessed: 2018-05-10. 2018. url: https://cocalc.com/.

  71. Graphcool, Inc. Graphcool. Accessed: 2018-05-10. 2018. url: https://www.graph.cool/.

  72. Meteor Development Group. “Livequery”. In: Meteor Change Log v1.0.4 (Mar. 2015). Accessed: 2017-07-09. URL: http://docs.meteor.com/changelog.html#livequery-1.

  73. Meteor Development Group. “Tuning Meteor Mongo Livedata for Scalability”. In: Meteor Blog (May 2016). Accessed: 2017-05-12. url: https://blog.meteor.com/tuning-meteor-mongolivedata-for-scalability-13fe9deb8908.

  74. Meteor Development Group. Meteor. Accessed: 2018-05-10. 2018. url: https://www.meteor.com/.

  75. MongoDB Inc. “MongoDB 3.6.0-rc0 is released”. In: MongoDB Blog (Oct. 2017). Accessed: 2017-11-17. url: https://www.mongodb.com/blog/post/mongodb-360-rc0-is-released.

  76. MongoDB Inc. MongoDB. Accessed: 2018-05-10. 2018. url: https://www.mongodb.com.

  77. MongoDB Inc. MongoDB Stitch. Accessed: 2018-05-10. 2018. url: https://mongodb.com/cloud/stitch.

  78. OrientDB Ltd. OrientDB. Accessed: 2018-05-10. 2018. url: https://orientdb.com/.

  79. STRV s.r.o. Rapid – Realtime Database Services. Accessed: 2018-05-10. 2018. url: https://www.rapidrealtime.com/.

Download references

Author information

Authors and Affiliations

  1. Baqend GmbH, Hamburg, Hamburg, Germany

    Wolfram Wingerath & Felix Gessert

  2. University of Hamburg, Hamburg, Hamburg, Germany

    Norbert Ritter

Authors
  1. Wolfram Wingerath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Norbert Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix Gessert
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2019 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wingerath, W., Ritter, N., Gessert, F. (2019). Real-Time Databases. In: Real-Time & Stream Data Management. SpringerBriefs in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-030-10555-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-10555-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-10554-9

  • Online ISBN: 978-3-030-10555-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation