Abstract
NCSTRL (The Networked Computer Science Technical Report Library) is a successful digital library for scientific and technical information. It uses the Dienst protocol that was developed by ARPA-funded CS-TR project. We encountered several problems while implementing NCSTRL based largescale libraries: UPS for Los Alamos and JDL for JTASC. The document collection for these libraries can range from several hundred thousands to few millions. The first problem we found that the native Dienst implementation does not scale beyond approximately 30,000 records. Secondly we found that the implementation is tightly coupled to the Unix platform. Finally, for a large number of hits the NCSTRL search interface support is limited in terms of usability. To address these problems, we replaced the Dienst repository service implementation with an Oracle-based implementation using servlet technology. The Oracle database stores the index information (metadata) and is partitioned horizontally to speed searching through different archives. Furthermore, indexes were built in order to speed the search by different key items such as the author name, the title and the abstract. Our implementation significantly reduced the average wait time for a user for searches that resulted in a large number of hits. In addition, we get all the other benefits of using servlet technology such as efficiency and portability. In this paper, we present the performance results of the new implementation and compare it with that of the implementation of the Dienst protocol in NCSTRL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Accomazi, A., Eichhorn, G., Kurtz, M. J., Grant, C. S., Murray, S. S.,: Astronomical information Discovery and Access: Design and Implementation of the ADS Bibliographic Services. Astronomical data Analysis Software and Systems VI, 125, (1997), 357–360
Browne, S., Dongarra, J., Horner, J., McMahan, P., Wells, S.: National HPCC Software Exchange (NHSE): Uniting the High Performance Computing and Communications Community. D-Lib Magazine, May (1998), http://www.dlib.org/dlib/may98/browne/05browne.html
Davis, J.R., Kraft, D. B., Lagoze, C.: Dienst: Building a Production Technical Report Server. Advances in Digital Libraries. Springer-Verlag, (1995), 211–222
Davis, J. R., Lagoze, C.: The Networked Computer Science Technical Report Library. Cornell CS TR96-1595, July, (1996)
Dushay, N., French, J. C., Lagoze, C.: A Characterization Study of NCSTRL Distributed Searching. Cornell CS TR99-1725, January (1999)
Entlich, R., Garson, L., Lesk, M., Normore, L., Olsen, J., Weibel, S.: Making a Digital Library: The Contents of the CORE Project. ACM Transactions on Information Systems, 15(2), (1997), 103–123
Hunter, J, and Crawfor, W.: Java Servlet Programming. O’Reilly and Associates, October1998.
Leiner, B.M.: The NCSTRL Approach to Open Architecture for the Confederated Digital Library. D-Lib Magazine, December (1998)
Maly, K., French, J., Fox, E., Salman, A.: Wide Area Technical Report Service-Technical Reports Online. Communications of the ACM, p. 45, April (1995)
Maly, K., Nelson, M. L, Zubair, M..: Smart Objects, Dumb Archives A User-Centric, Layered Digital Library Framework. D-Lib Magazine, March (1999), Volume 5 Issue 3
Maly, K., Nelson, M. L., Shen, S. N. T, Zubair M..: Buckets: Aggregative, Intelligent Agents for Publishing. WebNet Journal, Vol. 1, No. 1, March (1999), 58–65
Nelson, M. L., Maly, K., Shen, S. N. T., Zubair, M.: NCSTRL+: Adding Multi-Discipline and Multi-Genre Support to the Dienst Protocol Using Clusters and Buckets. Proceeding of Advances in Digital Libraries 98, Santa Barbara, CA, April 22-24 (1998)
Schatz, B., Chen, H.: Building Large-Scale Digital Libraries. IEEE Computer, 29(5), (1996), 22–26
Schatz, B., Mischo, W. H., Cole, T. W., Hardin, J. B., Bishop, A. P., Chen, H.: Federating Diverse Collections of Scientific Literature. IEEE Computer, 29(5), (1996), 28–36
Sompel, H. V., Nelson, M.L., Lyapunov, V.M., Zubair, M., Liu, X., Krichel, T., Hochestenbach, P., Maly, K., Kholief, M., O’Connell, H.:The UPS Prototype project: exploring the obstacles in creating a cross e-print archive end-user service. D-Lib Magazine, February (2000), Volume 6 Issue 2
Sompel, H. V., Lagoze, C.: The Santa Fe Convention of the Open Archives Initiative. DLib Magazine, February (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Maly, K., Zubair, M., Anan, H., Tan, D., Zhang, Y. (2000). Scalable Digital Libraries Based on NCSTRL/Dienst. In: Borbinha, J., Baker, T. (eds) Research and Advanced Technology for Digital Libraries. ECDL 2000. Lecture Notes in Computer Science, vol 1923. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45268-0_16
Download citation
DOI: https://doi.org/10.1007/3-540-45268-0_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41023-2
Online ISBN: 978-3-540-45268-3
eBook Packages: Springer Book Archive