OntoDBench: Novel Benchmarking System for Ontology-Based Databases

Abstract

Due to the explosion of ontologies on the web (Semantic Web, E-commerce, and so on) organizations are faced with the problem of managing mountains of ontological data. Several academic and industrial databases have been extended to cope with these data, which are called Ontology-Based Databases ($\mathcal{O}\mathcal{B}\mathcal{D}\mathcal{B}$). Such databases store both ontologies and data on the same repository. Unlike traditional databases, where their logical models are stored following the relational model and most of properties identified in the conceptual phase are valuated, OBDBs are based on ontologies which describe in a general way a given domain; some concepts and properties may not be used and valuated and they may use different storage models for ontologies and their instances. Therefore, benchmarking $\mathcal{O}\mathcal{B}\mathcal{D}\mathcal{B}$ represents a crucial challenge. Unfortunately, existing $\mathcal{O}\mathcal{B}\mathcal{D}\mathcal{B}$ benchmarks manipulate ontologies and their instances with characteristics far away from real life applications in terms of used concepts, attributes or instances. As a consequence, it is difficult to identify an appropriate physical storage model for the target $\mathcal{O}\mathcal{B}\mathcal{D}\mathcal{B}$, which enables efficient query processing. In this paper, we propose a novel benchmarking system called OntoDBench to evaluate the performance and scalability of available storage models for ontological data. Our benchmark system allows : (1) evaluating relevant characteristics of real data sets, (2) storing the dataset following the existing storage models, (3) expressing workload queries based on these models and (4) evaluating query performance. Our proposed ontology-centric benchmark is validated using the data sets and workload from the Lehigh University Benchmark (LUBM).