SciDB is a distributed database management system for managing and processing multidimensional arrays in scientific applications. It was first designed and developed by a group of academics led by Michael Stonebraker before being productized by Paradigm4.
The first XLDB workshop (1st Extremely Large Databases Workshop 2007) in 2007 brought together a group of scientists and industry members to discuss the capabilities of database management systems (DBMSs) at managing non-relational data at extreme scales. A number of key shortcoming were identified, which were presented the following year at XLDB-2 (2nd Extremely Large Databases Workshop 2008). The lack of support for managing scientific data was in particular discussed. Consensus emerged that many Big Science projects presented unique challenges that could not be handled through generic DBMSs and would require a complete rewrite approach (Stonebraker et al. 2007).
- 1st Extremely Large Databases Workshop (2007) http://www-conf.slac.stanford.edu/xldb2007/
- 2nd Extremely Large Databases Workshop (2008) http://www-conf.slac.stanford.edu/xldb2008/
- Cudré-Mauroux P, Kimura H, Lim K, Rogers J, Simakov R, Soroush E, Velikhov P, Wang DL, Balazinska M, Becla J, DeWitt DJ, Heath B, Maier D, Madden S, Patel JM, Stonebraker M, Zdonik SB (2009) A demonstration of SciDB: a science-oriented DBMS. PVLDB 2(2):1534–1537. http://www.vldb.org/pvldb/2/vldb09-76.pdfGoogle Scholar
- Cudre-Mauroux P, Kimura H, Lim KT, Rogers J, Madden S, Stonebraker M, Zdonik SB (2010) Ss-db: A standard science DBMS benchmark. http://www-conf.slac.stanford.edu/xldb10/docs/ssdb_benchmark.pdf
- Duggan J, Stonebraker M (2014) Incremental elasticity for array databases. In: International conference on management of data, SIGMOD 2014, Snowbird, pp 409–420. https://doi.org/10.1145/2588555.2588569
- Ge T, Zdonik SB (2010) A*-tree: a structure for storage and modeling of uncertain multidimensional arrays. PVLDB 3(1):964–974. http://www.comp.nus.edu.sg/~vldb2010/proceedings/files/papers/R86.pdfGoogle Scholar
- Leyshock P, Maier D, Tufte K (2013) Agrios: a hybrid approach to big array analytics. In: Proceedings of the 2013 IEEE international conference on big data, Santa Clara, pp 85–93. https://doi.org/10.1109/BigData.2013.6691558
- Paradigm4 (2011) https://www.paradigm4.com/
- Seering A, Cudré-Mauroux P, Madden S, Stonebraker M (2012) Efficient versioning for scientific array databases. In: IEEE 28th international conference on data engineering (ICDE 2012), Washington, DC (Arlington), pp 1013–1024. https://doi.org/10.1109/ICDE.2012.102
- Soroush E, Balazinska M, Wang DL (2011) Arraystore: a storage manager for complex parallel array processing. In: Proceedings of the ACM SIGMOD international conference on management of data, SIGMOD 2011, Athens, pp 253–264. https://doi.org/10.1145/1989323.1989351
- Stonebraker M, Madden S, Abadi DJ, Harizopoulos S, Hachem N, Helland P (2007) The end of an architectural era (it’s time for a complete rewrite). In: Proceedings of the 33rd international conference on very large data bases, University of Vienna, pp 1150–1160. http://www.vldb.org/conf/2007/papers/industrial/p1150-stonebraker.pdf
- Wu E, Madden S, Stonebraker M (2013) Subzero: a fine-grained lineage system for scientific databases. In: 29th IEEE international conference on data engineering, ICDE 2013, Brisbane, pp 865–876. https://doi.org/10.1109/ICDE.2013.6544881