Efficient Web Searching Using Temporal Factors
Web traversal robots are used to gather information periodically from large numbers of documents distributed throughout the Web. In this paper we study the issues involved in the design of algorithms for performing information gathering of this kind more efficiently, by taking advantage of anticipated variations in access times in different regions at different times of the day or week. We report and comment on a number of experiments showing a complex pattern in the access times as a function of the time of the day. We look at the problem theoretically, as a generalisation of single processor sequencing with release times and deadlines, in which performance times (lengths) of the tasks can change in time. The new problem is called Variable Length Sequencing Problem (VLSP). We show that although the decision version of VLSP seems to be intractable in the general case, it can be solved optimally for lengths 1 and 2. This result opens the possibility of practicable algorithms to schedule searches efficiently when expected access times can be categorised as either slow or fast. Some algorithms for more general cases are examined and complexity results derived.
KeywordsBipartite Graph Release Time Travelling Salesman Problem Access Time Execution Sequence
Unable to display preview. Download preview PDF.
- A. V. Aho et al., Theory of Computing: Goals and directions. Special Report of the National Science Foundation of the USA, 1996Google Scholar
- B. Bollobas, Random Graphs, Academic Press, 1985.Google Scholar
- Nicos Christofides, Worst-case analysis of a new heuristic for the traveling salesman problem, TR CS-93-13, Graduate School of Industrial Administration, Carnegie Mellon University, Pittsburgh, 1976.Google Scholar
- Lars Engebretsen, An Explicit Lower Bound for TSP with Distances One and Two ECCC Report TR98-046 also to appear in Proceedings of 16th International Symposium on Theoretical, Aspects in Computer Science, STACS’99.Google Scholar
- David S. Johnson and Christos H. Papadimitriou, Computational Complexity. In Eugene L. Lawler, Jan K. Lenstra, Alexander H.G. Rinnoy Kan, and David B. Shmoys editors, The Travelling Salesman Problem, chapter 3, pages 37–85. John Willey & Sons, New York, 1985.Google Scholar
- Micali, S. and Vazirani, V. V., An O(v 1/2 e) Algorithm for Finding Maximum Matching in General Graphs, Proceedings of the 21st Annual Symposium on Foundations of Computer Science, pp 17–27, 1980.Google Scholar
- R. C. Miller and K. Bharat, SPHINX: a framework for creating personal, site-specific Web crawlers. Computer Networks and ISDN systems 30, 1998 (proceedings of 7th International World Wide Web Conference)Google Scholar