Abstract
Many systems use ad hoc collections of files and directories to store persistent data. For consumers of this data, the process of properly parsing, using, and updating these filestores using conventional APIs is cumbersome and error-prone. Making matters worse, most filestores are too big to fit in memory, so applications must process the data incrementally while managing concurrent accesses by multiple users. This paper presents Transactional Forest (TxForest), which builds on earlier work on Forest to provide a simpler, more powerful API for managing filestores, including a mechanism for managing concurrent accesses using serializable transactions. Under the hood, TxForest implements an optimistic concurrency control scheme using Huet’s zippers to track the data associated with filestores. We formalize TxForest in a core calculus, develop a proof of serializability, and describe our OCaml prototype, which we have used to build several practical applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By integrating with PADS [6], we could go a step further and specify the contents of the file as well—i.e. a single line containing an integer.
References
Buraga, S.C.: An XML-based semantic description of distributed file systems. In: RoEduNet (2003)
DiLorenzo, J., Mancini, K., Fisher, K., Foster, N.: TxForest: A DSL for Concurrent Filestores. Technical report (2019). https://arxiv.org/abs/1908.10273
DiLorenzo, J., Zhang, R., Menzies, E., Fisher, K., Foster, N.: Incremental forest: a DSL for efficiently managing filestores. In: ACM SIGPLAN Notices, OOPSLA 2016, vol. 51, pp. 252–271 (2016)
Escriva, R., Sirer, E.G.: The design and implementation of the warp transactional filesystem. In: Proceedings of the 13th Usenix Conference on Networked Systems Design and Implementation, NSDI 2016, pp. 469–483. USENIX Association, Berkeley (2016)
Fisher, K., Foster, N., Walker, D., Zhu, K.Q.: Forest: a language and toolkit for programming with filestores. In: Proceedings of the 16th ACM SIGPLAN International Conference on Functional Programming, ICFP 2011, pp. 292–306. ACM, New York (2011). https://doi.org/10.1145/2034773.2034814
Fisher, K., Gruber, R.: PADS: a domain-specific language for processing ad hoc data. In: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2005, pp. 295–304. ACM, New York (2005). https://doi.org/10.1145/1065010.1065046
Fisher, K., Walker, D.: The PADS project: an overview. In: Proceedings of the 14th International Conference on Database Theory, ICDT 2011, ACM, New York (2011). https://doi.org/10.1145/1938551.1938556
Foster, J.N., Greenwald, M.B., Moore, J.T., Pierce, B.C., Schmitt, A.: Combinators for bidirectional tree transformations: a linguistic approach to the view update problem. ACM Trans. Program. Lang. Syst. (TOPLAS) 29(3) (2007). Short version in POPL 2005
Garcia, J., Ferreira, P., Guedes, P.: The PerDiS FS: a transactional file system for a distributed persistent store. In: Proceedings of the 8th ACM SIGOPS European Workshop on Support for Composing Distributed Applications, EW 8, pp. 189–194. ACM, New York (1998). https://doi.org/10.1145/319195.319224
Huet, G.: The Zipper. J. Funct. Program. 7(5), 549–554 (1997). https://doi.org/10.1017/S0956796897002864
Kiselyov, O.: Tool demonstration: a zipper based file/operating system. In: Proceedings of the 2005 ACM SIGPLAN Workshop on Haskell, Haskell 2005 (2005). http://okmij.org/ftp/continuations/ZFS/zfs-talk.pdf
LINQ: NET language-integrated query, February 2007. http://msdn.microsoft.com/library/bb308959.aspx
Liskov, B., Rodrigues, R.: Transactional file systems can be fast. In: Proceedings of the 11th Workshop on ACM SIGOPS European Workshop, EW 2004, ACM, New York (2004). https://doi.org/10.1145/1133572.1133592
Moore, K.F., Grossman, D.: High-level small-step operational semantics for transactions. In: Proceedings of the 35th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2008, pp. 51–62. ACM, New York (2008). https://doi.org/10.1145/1328438.1328448
Schmuck, F., Wylie, J.: Experience with transactions in QuickSilver. In: Proceedings of the Thirteenth ACM Symposium on Operating Systems Principles, SOSP 1991, pp. 239–253. ACM, New York (1991). https://doi.org/10.1145/121132.121171
Syme, D.: Looking ahead with F#: taming the data deluge. Presentation at the Workshop on F# in Education, November 2010
Acknowledgments
The authors wish to thank the anonymous APLAS reviewers for helpful comments on an earlier draft of this paper. Our work is supported in part by the National Science Foundation under grant CNS-1413972 and by the Defense Advanced Research Projects Agency.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
DiLorenzo, J., Mancini, K., Fisher, K., Foster, N. (2019). TxForest: A DSL for Concurrent Filestores. In: Lin, A. (eds) Programming Languages and Systems. APLAS 2019. Lecture Notes in Computer Science(), vol 11893. Springer, Cham. https://doi.org/10.1007/978-3-030-34175-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-34175-6_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34174-9
Online ISBN: 978-3-030-34175-6
eBook Packages: Computer ScienceComputer Science (R0)