Encyclopedia of Big Data Technologies

Living Edition
| Editors: Sherif Sakr, Albert Zomaya

Virtual Distributed File System: Alluxio

Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-63962-8_321-1

Synonyms

Definition

Alluxio (2018) is the world’s first memory speed virtual distributed storage. Alluxio is an open source project with a community of over 700 contributors from over 150 organizations. In the big data stack, it is a layer between applications such as Apache Spark (2018) and Apache MapReduce (Apache Hadoop 2018a) and data stores such as Amazon S3 (Amazon 2018) (Simple Storage System) and Apache HDFS (Apache Hadoop 2018b) (Hadoop Distributed File System).

Alluxio presents a set of disparate data stores as a single file system, greatly reducing the complexity of storage APIs (Application Programming Interface), locations, and semantics exposed to applications. For example, an object storage in the cloud and an on-premise distributed file system both appear to applications as subtrees in the Alluxio file system namespace.

Alluxio is designed with a memory centric architecture, enabling applications to leverage memory speed I/O by simply using Alluxio. Because...

This is a preview of subscription content, log in to check access.

References

  1. Alluxio (2018) Alluxio. https://alluxio.org
  2. Alluxio, Baidu (2016) Baidu queries data 30 times faster with Alluxio. http://alluxio-com-site-prod.s3.amazonaws.com/resource/media/Baidu-Case-Study.pdf
  3. Alluxio, Qunar (2017) Qunar performs real-time data analytics up to 300x faster with Alluxio. http://alluxio-com-site-prod.s3.amazonaws.com/resource/media/Qunar_Performs_Real-Time_Data_Analytics_up_to_300x_Faster_with_Alluxio.pdf
  4. Amazon (2018) Amazon S3. https://aws.amazon.com/s3
  5. Apache Hadoop (2018a) Apache Hadoop MapReduce. http://hadoop.apache.org
  6. Apache Hadoop (2018b) Apache Hadoop distributed file system. http://hadoop.apache.org
  7. Apache Spark (2018) Apache Spark. https://spark.apache.org
  8. Apache Zookeeper (2018) Apache Zookeeper. https://zookeeper.apache.org
  9. Google (2018) Protocol buffers. https://developers.google.com/protocol-buffers
  10. Liu S, Dawei S (2017) PerceptIn robotics get a performance boost from Alluxio distributed storage. http://thenewstack.io/powering-robotics-clouds-alluxio

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Alluxio Inc.San MateoUSA

Section editors and affiliations

  • Yuanyuan Tian
    • 1
  • Fatma Özcan
    • 2
  1. 1.IBM Almaden Research CenterSAN JOSEUnited States
  2. 2.IBM Research – AlmadenSan JoseUSA