Skip to main content
SpringerLink
Log in

Interoperable Convergence of Storage, Networking, and Computation

  • Conference paper
  • First Online:
Advances in Information and Communication (FICC 2019)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 70))

Included in the following conference series:

  • 1528 Accesses

Abstract

In every form of digital store-and-forward communication, intermediate forwarding nodes are computers, with attendant memory and processing resources. This has inevitably stimulated efforts to create a wide-area infrastructure that goes beyond simple store-and-forward to create a platform that makes more general and varied use of the potential of this collection of increasingly powerful nodes. Historically, these efforts predate the advent of globally routed packet networking. The desire for a converged infrastructure of this kind has only intensified over the last 30 years, as memory, storage, and processing resources have increased in both density and speed while simultaneously decreasing in cost. Although there is a general consensus that it should be possible to define and deploy such a dramatically more capable wide-area platform, a great deal of investment in research prototypes has yet to produce a credible candidate architecture. Drawing on technical analysis, historical examples, and case studies, we present an argument for the hypothesis that in order to realize a distributed system with the kind of convergent generality and deployment scalability that might qualify as “future-defining,” we must build it from a small set of simple, generic, and limited abstractions of the low level resources (processing, storage and network) of its intermediate nodes.

Dr. Beck is an Associate Professor at University of Tennessee, Knoxville. he is currently on detail to the National Science Foundation in the Office of Advanced Cyberinfrastructure. The work discussed herein was completed prior to his government service and does not reflect the views, conclusions, or opinions of the National Science Foundation or of the U.S. Government.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mynatt, E., Clark, J., Hager, G., Lopresti, D., Morrisett, G., Nahrstedt, K., Pappas, G., Patel, S., Rexford, J., Wright, H., et al.: A national research agenda for intelligent infrastructure. arXiv preprint arXiv:1705.01920 (2017)

  2. Networking, I. T. Research, and D. N. Program.: Smart and Connected Cities Framework. https://www.nitrd.gov/sccc/materials/scccframework.pdf (2015)

  3. Nahrstedt, K., Cassandras, C.G., Catlett, C.: City-scale intelligent systems and platforms. arXiv preprint arXiv:1705.01990 (2017)

  4. Anderson, T., Peterson, L., Shenker, S., Turner, J.: Overcoming the internet impasse through virtualization. Computer 38(4), 34–41 (2005)

    Article  Google Scholar 

  5. Tennenhouse, D.L., Wetherall, D.J.: Towards an active network architecture. Comput. Commun. Rev. 26, 5–18 (1996)

    Article  Google Scholar 

  6. Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure, pp. 47–48. Advanced computing. Computer systems design. Morgan Kaufmann Publishers (1999)

    Google Scholar 

  7. Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., Bowman, M.: Planetlab: an overlay testbed for broad-coverage services. SIGCOMM Comput. Commun. Rev. 33(3), 3–12 (2003)

    Article  Google Scholar 

  8. McGeer, R., Berman, M., Elliott, C., Ricci, R. (eds.): The GENI Book. Springer, Berlin (2016)

    Google Scholar 

  9. Clark, D.D.: Interoperation, open interfaces, and protocol architecture. Unpredictable Certainty White Pap. 2, 133–144 (1995)

    Google Scholar 

  10. Uux(1p) posix programmer’s manual (2013)

    Google Scholar 

  11. Saltzer, J.H., Reed, D.P., Clark, D.D.: End-to-end arguments in system design. ACM Trans. Comput. Syst. 2(4), 277–288 (1984)

    Article  Google Scholar 

  12. Carpenter, B., Brim, S.: Middleboxes: Taxonomy and issues. RFC 3234, Feb 2002. Network Working Group

    Google Scholar 

  13. Dongarra, J., Golub, G.H., Grosse, E., Moler, C., Moore, K.: Netlib and NA-Net: building a scientific computing community. IEEE Ann. Hist. Comput. 30, 30–41 (2008)

    Article  MathSciNet  Google Scholar 

  14. Chankhunthod, A., Danzig, P.B., Neerdaels, C., Schwartz, M.F., Worrell, K.J.: A hierarchical Internet object cache. In: Proceedings of the 1996 USENIX Technical Conference, pp. 153–163 (1995)

    Google Scholar 

  15. Wessels, D., Claffy, K.: ICP and the Squid web cache. IEEE J. Sel. Areas Commun. 16, 345–357 (1998)

    Article  Google Scholar 

  16. Beck, M., Moore, T.: The Internet2 distributed storage infrastructure project: an architecture for internet content channels. In: Computer Networking and ISDN Systems, pp. 2141–2148 (1998)

    Article  Google Scholar 

  17. Kirkpatrick, D.: IBM’s Olympic Fiasco Department of Groundless Optimism. Fortune Magazine, 9 Sept 1996

    Google Scholar 

  18. Buyya, R., Pathan, M., Vakali, A. (eds.): Content Delivery Networks. Springer, Berlin (2008)

    Google Scholar 

  19. Nygren, E., Sitaraman, R.K., Sun, J.: The Akamai network: a platform for high-performance internet applications. In: SIGOPS Operating Systems Review (2010)

    Google Scholar 

  20. Beck, M., Moore, T., Plank, J.S.: An end-to-end approach to globally scalable network storage. In: ACM SIGCOMM 2002 (2002)

    Google Scholar 

  21. Beck, M., Moore, T., Plank, J.S.: An end-to-end approach to globally scalable programmable networking. In: Future Directions in Network Architecture, pp. 328–339. ACM Press (2003)

    Google Scholar 

  22. David, P.A.: Path dependence: a foundational concept for historical social science. Cliometrica 1(2), 91–114 (2007)

    Article  Google Scholar 

  23. Messerschmitt, D.G.: The convergence of telecommunications and computing: what are the implications today? Proc. IEEE 84(8), 1167–1186 (1996)

    Article  Google Scholar 

  24. Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  25. Peterson, L.L., Davie, B.S.: Computer Networks: A Systems Approach, 5th edn. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2011)

    MATH  Google Scholar 

  26. Beck, M.: On the hourglass model, end-to-end arguments, and deployment scalability. Commun, ACM (to appear 2018)

    Google Scholar 

  27. Will the real end-to-end argument please stand up? http://mercury.lcs.mit.edu/jnc/tech/end_end.html

  28. Ritchie, D.M., Thompson, K.: The Unix time-sharing system. Commun. ACM 17, 365–375 (1974)

    Article  Google Scholar 

  29. Satyanarayanan, M.: The emergence of edge computing. Computer 50(1), 30–39 (2017)

    Article  Google Scholar 

  30. Plank, J.S., Bassi, A., Beck, M., Moore, T., Swany, D.M., Wolski, R.: Managing data storage in the network. IEEE Internet Comput. 5(5), 50–58 (2001)

    Article  Google Scholar 

  31. Beck, M., Arnold, D., Bassi, R., Berman, F., Casanova, H., Moore, T., Obertelli, G., Plank, J., Swany, M., Vadhiyar, S., Wolski, R.: Logistical computing and internetworking: middleware for the use of storage in communication. In: In 3rd Annual International Workshop on Active Middleware Services (AMS) (2001)

    Google Scholar 

  32. Plank, J.S., Atchley, S., Ding, Y., Beck, M.: Algorithms for high performance, wide-area, distributed file downloads. Technical report, Letters (2002)

    Google Scholar 

  33. Beck, M., Liu, H., Huang, J., Moore, T.: Scalable distributed execution environment for large data visualization. In: IEEE Explorer, Nov 2007

    Google Scholar 

  34. Liu, H.: Scalable, data-intensive network computation. Ph.D. thesis, University of Tennessee, Knoxville (2008)

    Google Scholar 

  35. Liu, H., Beck, M., Huang, J.: Dynamic co-scheduling of distributed computation and replication. In: IEEE International Symposium on Cluster Computing and the Grid, May 2006

    Google Scholar 

  36. Kitchin, R.: The Data Revolution: Big Data, Open Data, Data Infrastructures and their Consequences. Sage, London (2014)

    Google Scholar 

  37. Reichman, O.J., Jones, M.B., Schildhauer, M.P.: Challenges and opportunities of open data in ecology. Science 331(6018), 703–705 (2011)

    Article  Google Scholar 

  38. Board, S.S., Council, N.R., et al.: Landsat and Beyond: Sustaining and Enhancing the Nation’s Land Imaging Program. National Academies Press (2014)

    Google Scholar 

  39. Bird, I.: Computing for the Large Hadron Collider. Annu. Rev. Nucl. Part. Sci. 61, 99–118 (2011)

    Article  Google Scholar 

  40. Breeding, M.: Building a digital library of television news. Comput. Libr. 23(6), 47–49 (2003)

    Google Scholar 

  41. Wikipedia Contributors. Information logistics (2017) (Online). Accessed 15 Sept 2018

    Google Scholar 

  42. Banerjee, S., Wu, D.O.: Final report from the NSF Workshop on Future Directions in Wireless Networking. National Science Foundation (2013)

    Google Scholar 

  43. Chen, M., Mao, S., Liu, Y.: Big data: a survey. Mobile Netw. Appl. 19(2), 171–209 (2014)

    Article  Google Scholar 

  44. Avron, H., Maymounkov, P., Toledo, S.: Blendenpik: supercharging LAPACK’s least-squares solver. SIAM J. Sci. Comput. 32(3), 1217–1236 (2010)

    Article  MathSciNet  Google Scholar 

  45. Drineas, P., Mahoney, M.W.: RandNLA: randomized numerical linear algebra. Commun. ACM 59(6), 80–90 (2016)

    Article  Google Scholar 

  46. Valiant, L.G.: A theory of the learnable. Commun. ACM 27, 1134–1142 (1984)

    Article  Google Scholar 

Download references

Acknowledgements

The ideas in this paper were influenced by many spirited discussions with Martin Swany on the integration of storage and processing with scalable networking, and by recent conversations with Glenn Ricart on the definition and justification of interoperable convergence. The concept of “exposed buffer protocol/processing” was coined during discussions between Swany and Beck, although its best definition and implementation are still subject to debate. The authors are also indebted to David Rogers for his professional rendering of the artwork in this any many other papers and presentations, and to Chris Brumgard for his helpful comments.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville, TN, 37996, USA

    Micah Beck, Terry Moore, Piotr Luszczek & Anthony Danalis

Authors
  1. Micah Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Terry Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piotr Luszczek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony Danalis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Micah Beck .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, UK

    Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Beck, M., Moore, T., Luszczek, P., Danalis, A. (2020). Interoperable Convergence of Storage, Networking, and Computation. In: Arai, K., Bhatia, R. (eds) Advances in Information and Communication. FICC 2019. Lecture Notes in Networks and Systems, vol 70. Springer, Cham. https://doi.org/10.1007/978-3-030-12385-7_48

Download citation

Publish with us

Policies and ethics

Search

Navigation