Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The wealth of Linked Data published on the open Web [15] offers a wide range of opportunities that are to date still underexploited in practical applications. Integrating Linked Data from different sources into standard web sites, blogs or other web applications would enable web authors and developers to reuse the vast amount of information already available and create additional value by enriching their content or by syndicating different data sources. However, a more wide-spread use of textual and multimedia resources from Linked Data and, even more so, of time-dependent data from the Internet of Things is currently significantly hindered by their complexity. It thus seems important to lower the threshold for users such as web developers or even normal web authors by providing techniques for using Linked Data without requiring complicated technical installations or the knowledge of powerful yet complex query languages such as Sparql.

To alleviate the problems involved in using linked data, we have developed a JavaScript-based environment, that facilitates the integration of Linked Data in web pages. A main component of this environment is the path query language SemwidgQL that is intended to be significantly easier to use than standard Sparql. A first overview of the SemwidgJS environment was presented in [16]. In this current paper, we focus on the path query language developed and provide a description of its novel extensions. We further present a comprehensive empirical user study comparing Sparql and SemwidgQL. The goal of this study is to explore SemwidgQL’s effectiveness, efficiency, learnability for users, and user preference in comparison to Sparql. The study supports our claim that SemwidgQL is easier to learn, more efficient, and preferred by the learners. To investigate how well SemwidgQL covers the range of Sparql queries used in practice, we further analyzed several hundred thousand log entries of public Sparql endpoints. Results indicate that SemwidgQL can cover most of the requests that are currently made with Sparql.

2 Related Work

Several approaches to support querying, exploring and displaying Linked Open Data have been described in literature so far. Many of these approaches are specialized on browsing (e.g. [2, 7]) and visualizing queried data values or sub graphs (e.g. [8]) respectively, which is generating revealing insights but can only be reused on other websites with large effort.

Fsl [11] and LDPath [14] are path query languages for Rdf data inspired by XPath for Xml. LDPath is part of the Apache Marmotta platform for Linked Data. One drawback of these languages is that they can only return single result lists but not lists of results sets, which is necessary when querying a set of different properties at once. Requesting coherent values from different properties require distinct queries that request each property separately. Therefore, it is not provided that these values stay connected, since the respective order can be different, or values can be added to or removed from the data set between requests. Rules for a translation into Sparql do not exist for these languages and therefore they require direct access to the data or a special interface on the server side. Language extensions for Sparql such as C-Sparql [1] and SparqlStream [5] facilitate the usage of queries over streams of Rdf data. Time windows that restrict the queried data to a period of time can be specified in a special From Stream statement.

The performance of users for different query languages has already been evaluated in pre-Sql times [12]. However, since the effort is very high, user studies that compare different query languages are rarely conducted. Participants require an extensive introduction to be able to use a query language at a satisfying level. Mostly this happens in the context of a lecture. We are not aware of any user study that compares the participants’ performance with Sparql and another query language of the Linked Data area.

3 SemwidgQL

SemwidgQL is a path query language that transcompiles to Sparql. In contrast to queries formulated in other Linked Data path query languages, such as LDPath [14] or Fsl [11], SemwidgQL can therefore be used to query any public Sparql endpoint without further special requirements. Unlike these languages, SemwidgQL is also capable of querying sets of different properties at once, by adding all properties of interest to the Select statement of its Sparql translation. Ultimately, SemwidgQL aims to combine the benefits of Sparql (such as its prevalence in the Linked Open Data area or its ability of returning lists of result sets) with the simplicity of path query languages.

In the following section, we give an overview of SemwidgQL’s core features that have been described in more detail in a previous publication [16]. SemwidgQL has been significantly extended since then and we present further features that were added to facilitate among others the querying of time-sequential data, such as sensor data, and give experienced users more control over the generated Sparql queries via filters and pseudo-filters. Specifications of time windows are comparable to the approaches of Sparql streaming extensions. In contrast to these extensions, SemwidgQL is compatible with regular Sparql endpoints and requires no additional execution environment.

Fig. 1.
figure 1

Basic structure of a SemwidgQL query.

Full size image
Table 1. Basic SemwidgQL queries and their meanings.
Full size table

3.1 Core Features

As a path query language, SemwidgQL traverses Rdf graphs. The traversal is indicated by the dot notation, which is reminiscent of the well-known syntax used in object-oriented programming. Figure 1 shows the simplified basic structure of a SemwidgQL query. Usually a query starts with a resource followed by one or more properties. To further filter the result set, properties can be restricted. Filters are enclosed in parentheses and are appended the property they restrict. The left-hand side of a filter expression is typically a property (or a property path) that refers to the property to restrict outside of the parentheses. The right-hand side specifies a filter value that can be a literal, Iri, or even a nested query. Between them stands a relational operator. Several filter expressions can be combined by logical operators. Furthermore, SemwidgQL allows wildcard selectors, inverse property selections, and multiple property selections. Table 1 shows some exemplary SemwidgQL queries.

3.2 Advanced Features

In addition to SemwidgQL’s core features, we have implemented several filter and pseudo-filter keywords that, among others, simplify restricting language of string literals or allow aggregation of results. Also, they facilitate querying of time-sequential data with flexibly specified sampling intervals. Filter and pseudo-filter keyword expressions can be combined with normal SemwidgQL filter expressions and with each other as well. While filter expressions in SemwidgQL result in filter expressions in Sparql, pseudo-filter expressions can have an impact on different parts of the translated query. An overview of these expressions is given below.

Filter Expressions

  • @lang: With this keyword the language of the property can be filtered by the given language code.

  • @self: This keyword refers to the property to restrict itself. Instead of filtering a property that is related to the property to restrict, it can be filtered directly.

  • @timestart/@timeend: These keywords allow the filtering of values after, before, or (when combined) between two points of time. The right-hand side of the expression can be an absolute date or a relative point in time, depending on the time of the query execution. The expression is parsed as an equation, whose first part is a timestamp or the term now followed by the amount of time that has to be added or subtracted. This can be expressed in seconds, minutes, hours, day, weeks, or a combination of these (e.g. now - 1 h 5 min).

  • @type: This keyword is equivalent to the property rdf:type.

Pseudo-Filter Expressions

  • @aggregate: This keyword allows to apply an aggregate function to the variable of the property within the Select statement. Allowed values are Count, Sum, Min, Max, Avg, and Sample.

  • @hide: If set to true, the variable of the property will not be part of the Select statement.

  • @optional: If set to true, the triple pattern, in which the property is created, will be enclosed in an Optional statement.

  • @predicate: Typically, the predicate of a triple pattern is not part of the Select statement. If set to true, the predicate of the triple pattern, in which the property is created, will be added to the Select statement.

  • @timeinterval: This keyword is used to group and aggregate time-sequential values. On the right-hand side of the expression, a sampling interval can be defined. All returned values within this interval will be aggregated. By default, the Sample aggregate function will be applied to all variables, but different functions can be specified by the @aggregate keyword. Similar to @timestart and @timeend, the length of the interval can be expressed in seconds, minutes, hours, day, weeks, or a combination of these.

Fig. 2.
figure 2

A SemwidgQL query and the corresponding Sparql query that requests the average temperature measurements that were made by a specific sensor during the last week, aggregated on an hourly base.

Full size image

A SemwidgQL query and its rather complex translation into Sparql is shown in Fig. 2. The query contains normal SemwidgQL filter expressions combined with previously presented filter and pseudo-filter keyword expressions. It requests the average temperature measurements that were made by a specific sensor (located in our office) during the last week, aggregated on an hourly base.

4 Empirical User Study

We conducted an empirical user study comparing Sparql and SemwidgQL. SemwidgQL was developed to be effective, efficient, and easy to learn by non-expert users. The goal of our study is to explore whether SemwidgQL can fulfill these requirements in comparison to Sparql. In addition we want to investigate the users’ satisfaction. At the beginning of this section, we will describe the design of the study and its procedure. Afterwards, we will present the results. In conclusion, we interpret and discuss these results.

4.1 Method

Design: We conducted an empirical user study with a mixed methods design and repeated measures, combining objective performance measures and a subjective questionnaire. For the performance measure, participants had to complete several query interpretation and formulation tasks. Effectiveness was measured by the number of correct answers of all query tasks.

Efficiency was measured by the participants’ performance measures for the query formulation tasks. We investigated these tasks regarding nine dependent variables, i.e. (a) number of keystrokes (number of keystrokes made by a participant, including deletion and substitution of characters), (b) number of corrections (number of correcting keystrokes made by a participant, such as backspace, delete, replacing several selected characters etc.), (c) number of conjunct corrections (a coherent sequence of correcting keystrokes forms a conjunct correction, e.g. multiple backspaces in succession; typing of a character ends a conjunct correction), (d) number of pauses (number of pauses taken by a participant; a pause starts after two seconds without a keystroke; a pause might be an indicator for that a participant requires some time to think about further actions that are required to solve the task), (e) time of pauses (accumulated time of pauses in seconds taken by a participant during a task; operationalizes thinking time), (f) time on task (processing time of a task in seconds), (g) number of requests (number of requests a participant made to the Sparql endpoint), (h) fraction of erroneous requests (fraction of requests that could not be executed due to parser errors etc.), (i) display time of solutions (time in seconds that a participant inspected the sample solution; a high display time might be an indicator that participants are uncertain about their solutions and therefore compare their own and the model solution more thoroughly).

Learnability was evaluated by comparing the results of the query formulation tasks from the initial and repeated measures regarding the above listed variables.

User preferences were measured through the answers from the questionnaire. The questionnaire asked to rate six characteristics of Sparql and SemwidgQL on the basis of an equidistant five-point numerical rating scale. The minimum value always had a negative and the maximum value always had a positive connotation. These items were related to the subjective assessment of Sparql’s and SemwidgQL’s learnability, intuitiveness, logical structure, comprehensibility, writing effort, and sophistication. Also, the participants were explicitly asked for their personally preferred language and a brief explanation for their decision.

Participants: The study was attended by seven students (one female), all enrolled on master courses in computer science at our University. The age of the participants was between 23 and 28 years (\(M =25.57; SD =2.07\)). Three participants had already gathered previous experience in Linked Data and Semantic Web from different courses, and one student had already worked with Rdf and Sparql as part of his bachelor thesis.

Procedure: The user study took place in the context of the introductory session of a seminar on “Semantic Web Technologies and Applications” for graduate students in the field of computer science. At the beginning of the seminar, the participants were handed a three-page handoutFootnote 1, which contained an overview of relevant Sparql and SemwidgQL commands, as well as a small Rdf graph that was used for all examples and tasks of the presentation and evaluation. The graph contained, among other things, some information about cities in the region, such as label, population, districts, class, but also temperature measurements of sensors. The data were chosen in such a way that the participants could compensate for misunderstandings through their personal context knowledge.

The introductory session consisted of a three-hour lecture which was divided into three one-hour sections. In the first section, the participants were taught the basic ideas, techniques and formats on which Linked Data and the Semantic Web are built. In the second section, the participants were given an introduction to Sparql and in the third section an introduction to SemwidgQL. As far as it was possible, the procedure corresponded to the procedure of the previous section. Care was taken to explain both languages to a similar extent and it was ensured that the participants understood both languages at a comparable level.

Afterwards, the participants had to complete a set of twelve query tasks. In the first three tasks, they had to interpret predefined Sparql and SemwidgQL queries. In the following nine tasks, they had to query predetermined information using Sparql and SemwidgQL. Each task had to be processed with both languages. Namespace definitions were predefined for both languages. The order of the query languages changed at each task. At any time, participants could query the Sparql endpoint and validate their queries and results. They could quit tasks at any time and move on to the next one. No time limit was set for solving a task. After each task, a model solution was presented.

Subsequently, the participants filled out a questionnaire in which they should specify socio-demographic information and previous experiences with Semantic Web and Linked Data techniques. Then they evaluated Sparql and SemwidgQL regarding the above mentioned characteristics. One week after the introductory session, the study was repeated.

Data Collection: The data of the interpretation and formulation of queries were automatically collected via the specially prepared website on which the participants had to solve their tasks. Each keystroke was recorded and stored together with a time stamp in a central database. It was also recorded when the Sparql endpoint was queried and it was recorded whether the query was valid or contained errors. The time stamps, at which the participants started or ended a task, the model solution was displayed, and a task was marked as successfully completed or marked as canceled by the participants, were recorded as well. The questionnaire data were collected via the online survey portal SoSci SurveyFootnote 2.

4.2 Results

Correctness of Answers: Answers were divided into three categories. Correct answers, answers with minor errors, and incorrect answers. Answers with minor errors are syntactically correct and close to the model solutions, but can contain minor inaccuracies, such as queries that contain a triple pattern for requesting a desired property but do not contain the corresponding variable in the Select statement. Incorrect answers are syntactically incorrect, do not fulfill the requirements given in the task description, or the task was aborted by the user.

In total, we evaluated results of 147 tasks per language. From these results 21 belong to the query interpretation tasks and 126 belong to the query formulation tasks. The participants performed slightly better, when interpreting SemwidgQL queries compared to interpreting Sparql tasks. Regarding SemwidgQL, 86% of the tasks were solved correctly, 14% of the solutions had minor errors. Regarding Sparql, 76% of the tasks were solved correctly, and 24% of the solutions had minor errors. There were no incorrect answers in terms of the query interpretation tasks. With regard to the query formulation tasks, the participants achieved almost equally good results with both languages. Regarding Sparql, 90% of the tasks were solved correctly, 6% of the solutions contained minor errors and 4% were incorrect. Regarding SemwidgQL, 89% of the tasks were solved correctly, 7% of the solutions contained minor errors and 4% were incorrect.

Query Formulation Tasks: In the following subsections, we will describe the results of the nine query formulation tasks (tasks 4–12), the participants had to solve during the evaluation. For each of the following statistical tests, we compared the participants’ performance regarding the nine dependent variables listed in the study design subsection. For the subsequent tests, we restrict the examined data to pairs of correct answers or answers with minor errors, since data from incorrect or canceled solution would doubtlessly distort the results.

Table 2. Differences between Sparql and SemwidgQL.
Full size table

Analysis of Mean Performance: We compared the participants’ performance regarding the above-mentioned dependent variables by calculating multiple dependent t-tests for paired samples. The further described results are presented in detail in Table 2. SemwidgQL’s values regarding six of all nine dependent variables were significantly better compared to Sparql. The number of conjunct corrections (c) was descriptively better regarding SemwidgQL compared to Sparql. However, this difference is not statistically significant. The number of requests (g) and the fraction of erroneous requests (h) were better in Sparql compared to SemwidgQL. These differences are also not statistically significant.

Analysis of Learning Effects: We evaluated, how the participants performance changed between the first and second pass of the user study. We also compared the differences between Sparql and SemwidgQL during these two passes. Again, we calculated multiple dependent t-tests for paired samples. Differences between Sparql and SemwidgQL at each pass are presented in Table 3 and Fig. 3. Differences between the first and second pass for each language are shown in Table 4 and Fig. 3, in combination with the results of the previous tests.

In the first pass, results in terms of SemwidgQL were significantly better regarding four of the nine dependent variables compared to Sparql, and descriptively but not significantly better regarding two further dependent variables. The participants never performed significantly better with Sparql. In the second pass, results regarding SemwidgQL became significantly better at all but one dependent variable compared to the first pass. Results regarding Sparql became significantly better regarding four dependent variables. All other results became descriptively but not significantly better. In comparison to Sparql, participants performed significantly better with SemwidgQL regarding five of all nine dependent variables. Again, the participants never performed significantly better with Sparql.

Fig. 3.
figure 3

Differences between Sparql and SemwidgQL per pass, and differences between first and second pass per language.

Full size image

Complexity-Dependent Analysis: We compared the performance of the participants for Sparql and SemwidgQL regarding a task’s complexity. We assume that complexity of a task is a predictor for the measured responses. For this purpose, we conducted several linear regression analyses for the previously mentioned dependent variables and the complexity of a task as predictor variable.

Table 3. Differences between Sparql and SemwidgQL per pass.
Full size table
Table 4. Differences between first and second pass per language.
Full size table

In various works to determine the difficulty of Sparql (e.g. [10]) or other (database) queries (e.g. [3, 9]) Halstead’s complexity measure [6] has been used. This measure is based on the number of distinct operators and operands as well as the total number of operands of a query or piece of source code. Halstead’s complexity measure tends to produce comparatively high values when Sparql queries contain filter expressions because the number of operators increases noticeably. Thus, it seems to overrate the influence of filter expressions on complexity. Because of this limitation, we developed an alternative complexity measure, which is based on the number of nodes of a query in Sparql Syntax Expressions (Sse) notationFootnote 3. Later on, we show that the empirical data are better represented by the alternative Sse based complexity measure.

To calculate the Sse based complexity measure, we summed up the number of nodes of the Sse syntax tree, but combined all nodes which were required for matching the language in a filter expression into one. Since the participants were taught this filter as a fixed expression in both languages, we assumed that writing this expression requires no additional mental effort than a normal filter expression. Also, we did not count the first projection node (i.e. Select), which occurs in all Select queries. Table 5 shows the complexity values of the Sparql sample solutions of each task in comparison, calculated according to Halstead’s D as well as the Sse based complexity . D and values of tasks without filter expressions (4, 5, 6, 8, 10) are very similar, while D values of tasks with filter expressions (7, 9, 11, 12) are noticeably higher than values. We argue that this method is much closer aligned to the mental processes a user has to perform when solving a task than Halstead’s method. We calculated the regression lines for all response variables with each D and as predictors and Sparql as query language. Based on the yielded coefficient of determination \(R^2\), we calculated a Wilcoxon Signed-Rank Test that supports our statement and indicates that the median for , \(Mdn =.86\), was significantly better than the median for D, \(Mdn =.56\) (\(z=-2.35\), \(p=.016\)).

Table 5. Comparison of complexity of Sparql sample solutions.
Full size table
Table 6. Linear regression analyses with Sse based complexity as predictor.
Full size table
Fig. 4.
figure 4

Linear regression analyses with task complexity as predictor.

Full size image

The results of the linear regression analyses with as predictor variable for all response variables with Sparql and SemwidgQL are presented in Table 6 and Fig. 4. Since we did not want to compare the theoretical complexity of Sparql and SemwidgQL but their practical performance at tasks with different complexities, we chose the complexity value of the Sparql sample solution query as complexity value for the corresponding tasks. Results indicate that there is a significant association between complexity and all response variables except for fraction of erroneous requests (h). The corresponding regression equations have very high \(R^2\) values (one third of them \(\ge .90\)) and the effect sizes f are also high, according to Cohen [4].

The regression lines for the values of SemwidgQL in the examined range of are in all cases, except number of requests (g) and fraction of erroneous requests (h), below the regression lines for the values of Sparql. In four of these seven cases, the slopes of the SemwidgQL lines are less steep than the slopes of the Sparql lines, suggesting that SemwidgQL will perform better then Sparql at more complex tasks. In the remaining three cases, the lines intersect at complexity values above the investigated range.

Fig. 5.
figure 5

Subjective evaluation of Sparql and SemwidgQL.

Full size image
Fig. 6.
figure 6

Cumulative numbers of stated advantages of Sparql and SemwidgQL.

Full size image

Subjective Evaluation by the Participants: In the following subsection, we will present the results of the questionnaire, the participants completed after the query tasks. We calculated multiple Wilcoxon Signed-Rank Tests to compare the participants’ subjective ratings for Sparql and SemwidgQL. Writing effort was rated significantly better regarding SemwidgQL, \(Mdn =4\), compared to Sparql, \(Mdn =2\) (\(z=-3.03\), \(p=.001\)). Sophistication was also rated significantly better regarding SemwidgQL, \(Mdn =4\), compared to Sparql, \(Mdn =2\) (\(z=-3.23\), \(p<.001\)). There were no significant differences regarding the subjective ratings of learnability, intuitiveness, logical structure, and comprehensibility (see Fig. 5). When asked for advantages of SemwidgQL over Sparql, the participants named nine unique characteristics with 30 occurrences in total. Particularly the shortness of queries and the similarity to object orientated programming languages were frequently mentioned. The participants only named three unique advantages of Sparql over SemwidgQL with 5 occurrences in total (see Fig. 6). In 79% of the answers SemwidgQL was named as the preferred query language. Accordingly, Sparql was only preferred in 21% of the answers.

4.3 Discussion

The user study showed that the participants performed significantly better with SemwidgQL regarding most of the evaluated dependent variables compared to Sparql. Especially the time on task, the number of corrections, and the number and time of pauses that the participants took to think about the correct solution of the task indicate that SemwidgQL is easier to use than Sparql.

After the introductory session, the participants achieved better results with SemwidgQL than with Sparql regarding most of the evaluated dependent variables. They improved significantly in the second pass in all but one area with SemwidgQL. Most of the improvements with Sparql were not significant. The participants had already performed better in the first pass with SemwidgQL, and had improved even more in the second pass compared to Sparql. The results suggest that SemwidgQL is easier to learn.

The reason for the better results with SemwidgQL was not that some already simple tasks were made even easier. The linear regression analyses indicate that the good results with SemwidgQL were achieved at all evaluated complexity levels. Some regression lines predict that even more complex tasks than that we have evaluated can be solved better with SemwidgQL. However, it should also be noted that some regression lines indicate that users will perform worse with SemwidgQL at tasks with higher complexity levels than evaluated. Since continuously written SemwidgQL queries can become very unwieldy at a certain length, this is to be expected.

The good results of the objective measures are supported by the participants’ subjective evaluation, the number of mentioned advantages and, of course, the explicit personal preference for SemwidgQL of 79%.

5 Evaluation of SemwidgQL’s Expressiveness

To investigate how well SemwidgQL covers the range of Sparql queries used in practice we analyzed to what extent our language is able to express the queries that occur in the Linked Sparql Queries Dataset (Lsq), collected by Saleem et al. [13]. We extracted 636,876 unique Select queries with 1,526,804 executions and then transformed them into a parameterized form. We mapped all Iris, variables, literals and language tags of each query to a generic format (e.g. SELECT ?v2 WHERE {<i1> ?v1 ?v2}), and replaced all wildcards in Select statements with the corresponding list of variables from the Where statement and harmonized language filter expressions. Completely identically parameterized queries were merged automatically. We were able to manually merge further pattern that were not syntactically but semantically identical (e.g. queries with the same triple patterns in their Where clauses, but in different order, or queries with and without the Distinct keyword, where the Distinct keyword is not able to reduce the result set). Finally, we obtained 1619 unique query patterns where the first 120 patterns of the most frequently executed queries represent 99% of the Select queries executed overall in the Lsq dataset.

Based on these 120 query patterns, we evaluated how well SemwidgQL covers the range of Sparql queries used in practice. From these patterns 66, representing 91% of the overall executed queries, can be directly expressed in SemwidgQL without any limitations. In contrast, 15 of these patterns, representing only 2% of the overall executed queries, can not be expressed. These patterns contain Group By expressions or function calls, such as bound or isLiteral, which are not implemented in SemwidgQL. The remaining 39 patterns, representing 6% of the queries executed overall, make use of Union graph patterns. SemwidgQL does not provide an equivalent for these constructs. Nevertheless, some of these patterns can be expressed without Union but with Filter expressions. Additionally, SemwidgQL allows the declaration of multiple queries in a single statement. These queries are translated into separate Sparql queries. Combining their results is up to the processing program.

We calculated the Sse based complexity measure for the 120 most frequently used query patterns. Most of the requests made (89%) have a value below or equal to 8 and thus lay in the evaluated range of our user study. One third of them have a value of 2 or 3. Few query patterns have values above 20 (up to 58). However, these patterns only represent less than 3% of the requests made.

6 Conclusion

We have presented SemwidgQL, a path query language for Rdf data that transcompiles to Sparql. Our empirical user study indicates that SemwidgQL is easier to learn, more efficient, and preferred by the learners compared to Sparql. An additional evaluation of the Lsq dataset indicates that SemwidgQL, despite its limited expressiveness, is capable of querying most of the data that is currently queried with Sparql. Also, the queries we used in the user study have a comparable complexity to queries that are used in practice. SemwidgQL is not intended as a replacement for Sparql but rather as a more light-weight language that lowers the entry barriers to the Semantic Web and Linked Data area. Results indicate that SemwidgQL is suitable for this purpose.