Efficiently following object references for large object collections and small main memory

Abstract

We consider queries over large object-oriented databases in which one class of objects contains references to another class of objects. In order to answer the query efficiently, the database system needs to be able to follow object pointers from a large collection of objects in a way that minimizes the I/O cost. Traditional techniques require significant redundant I/O when both the referencing class and the referenced class are substantially larger than main memory. We propose a new technique for processing a class of object-oriented queries that is an adaptation of the Jive-Join algorithm of Ross and Li. Our algorithm applies as long as the number of disk blocks in the referenced relation is roughly of the order of the square of the number of blocks that fit in main memory. The cost of the algorithm is at most one pass through each input class extension, one pass through an index file if there is an index, and two passes through a temporary file that contains the object-identifier of the referenced object for each output tuple. Almost all of the I/O is sequential, resulting in minimal seek and rotational latencies. We analyze the cost of our algorithm, and compare our algorithm with a naive algorithm, and with an adaptation of an algorithm due to Valduriez. We demonstrate that under a wide range of circumstances, our algorithm performs significantly better than its competitors. The performance improvement is most dramatic when there is a small amount of memory, and when the input class extensions are very large.

This research was supported by a grant from the AT&T Foundation, by a David and Lucile Packard Foundation Fellowship in Science and Engineering, by a Sloan Foundation Fellowship, by NSF grants IRI-9209029, CDA-90-24735, and by an NSF Young Investigator award.