Multimedia Tools and Applications

, Volume 77, Issue 3, pp 3405–3429 | Cite as

Answering why-not questions on semantic multimedia queries

  • Meng Wang
  • Weitong Chen
  • Sen Wang
  • Jun Liu
  • Xue Li
  • Bela Stantic


Linked data is a promising way to publish media data as resources on the Web and interlink them with other resources. While significant amounts of image, audio and video fragments have been tagged and exposed as linked data, searching and explaining the unexpected query results have been rarely studied. To improve the functionality and usability of SPARQL-based multimedia search engines, we focus on explaining missing items in the query results, or the so-called why-not question in this paper. We first formalize why-not questions on multimedia SPARQL queries and then propose a novel explanation model to answer why-not questions. Our model adopts a to generate logical explanations at the basic graph pattern level, the filter constraint level, or the multimedia function level, respectively, which helps users refine their initial queries. Extensive experimental results on two real-world RDF datasets show that the proposed model and algorithms can provide high-quality explanations both in terms of effectiveness and efficiency.


Why-not Multimedia RDF graph SPARQL Ontology 



This work is sponsored by The Fundamental Theory and Applications of Big Data with Knowledge Engineering under the National Key Research and Development Program of China with grant number 2016YFB1000903; National Science Foundation of China under Grant Nos.61672419, 61532004, and 61532015; MOE Research Center for Online Education Funds under Grant No.2016YB165; Ministry of Education Innovation Research Team No.IRT17R86.


  1. 1.
    Bhowmick SS, Sun A, Truong BQ (2013) Why not, wine?: towards answering why-not questions in social image search. In: Proceedings of the 21st ACM international conference on multimedia. ACM, pp 917–926Google Scholar
  2. 2.
    Bidoit N, Herschel M, Tzompanaki K (2014) Query-based why-not provenance with nedexplain. In: Extending database technology (EDBT)Google Scholar
  3. 3.
    Bizer C, Heath T, Berners-Lee T (2009) Linked data-the story so far. Semantic services, interoperability and web applications: emerging concepts, pp 205–227Google Scholar
  4. 4.
    Calvanese D, Ortiz M, Simkus M, Stefanoni G (2013) Reasoning about explanations for negative query answers in dl-lite. J Artif Intell Res 48:635–669MathSciNetMATHGoogle Scholar
  5. 5.
    Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann AG (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197CrossRefGoogle Scholar
  7. 7.
    Chang X, Yang Y (2016) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems.
  8. 8.
    Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632CrossRefGoogle Scholar
  9. 9.
    Chapman A, Jagadish H (2009) Why not?. In: Proceedings of the 2009 ACM SIGMOD international conference on management of data. ACM, pp 523–534Google Scholar
  10. 10.
    Chen L, Lin X, Hu H, Jensen CS, Xu J (2015) Answering why-not questions on spatial keyword top-k queries. In: IEEE 31st international conference on data engineering (ICDE), 2015. IEEE, pp 279– 290Google Scholar
  11. 11.
    Cui Y, Widom J (2003) Lineage tracing for general data warehouse transformations. VLDB J 12(1):41–58CrossRefGoogle Scholar
  12. 12.
    Damásio CV, Analyti A, Antoniou G (2012) Provenance for sparql queries. In: Proc. ISWC. Springer, pp 625–640Google Scholar
  13. 13.
    Dividino R, Sizov S, Staab S, Schueler B (2009) Querying for provenance, trust, uncertainty and other meta knowledge in rdf. Web Semant Sci Serv Agents World Wide Web 7(3):204– 219CrossRefGoogle Scholar
  14. 14.
    Gao Y, Liu Q, Chen G, Zheng B, Zhou L (2015) Answering why-not questions on reverse top-k queries. Proc VLDB Endowment 8(7):738–749CrossRefGoogle Scholar
  15. 15.
    Hausenblas M, Troncy R, Raimond Y, Bürger T (2009) Interlinking multimedia: how to apply linked data principles to multimedia fragments. In: Proceedings of the 2nd international workshop on linked data on the web (LDOW). MadridGoogle Scholar
  16. 16.
    He Z, Lo E (2014) Answering why-not questions on top-k queries. IEEE Trans Knowl Data Eng 26(6):1300–1315CrossRefGoogle Scholar
  17. 17.
    Herschel M, Hernández MA (2010) Explaining missing answers to spjua queries. Proc VLDB Endowment 3(1-2):185–196CrossRefGoogle Scholar
  18. 18.
    Hu C, Xu Z, Liu Y, Mei L, Chen L, Luo X (2014) Semantic link network-based model for organizing multimedia big data. IEEE Trans Emerg Topics Comput 2(3):376–387CrossRefGoogle Scholar
  19. 19.
    Huang J, Chen T, Doan A, Naughton JF (2008) On the provenance of non-answers to queries over extracted data. Proc VLDB Endowment 1(1):736–747CrossRefGoogle Scholar
  20. 20.
    Islam MS, Liu C, Li J (2015) Efficient answering of why-not questions in similar graph matching. IEEE Trans Knowl Data Eng 27(10):2672–2686CrossRefGoogle Scholar
  21. 21.
    Islam MS, Zhou R, Liu C (2013) On answering why-not questions in reverse skyline queries. In: IEEE 29th international conference on data engineering (ICDE), 2013. IEEE, pp 973– 984Google Scholar
  22. 22.
    Kurz T, Schlegel K, Kosch H (2015) Enabling access to linked media with sparql-mm. In: Proceedings of the 24th international conference on world wide web. ACM, pp 721–726Google Scholar
  23. 23.
    Li Y, Wald M, Omitola T, Shadbolt N, Wills G (2012) Synote: weaving media fragments and linked data. In: Proceedings of the 5th workshop on linked data on the web (LDOW 2012). LyonGoogle Scholar
  24. 24.
    Luo M, Chang X, Nie L, Yang Y, Hauptmann A, Zheng Q (2017) An adaptive semi-supervised feature analysis for video semantic recognition. IEEE Transactions on Cybernetics.
  25. 25.
    Luo M, Nie F, Chang X, Yang Y, Hauptmann AG, Zheng Q (2017) Adaptive unsupervised feature selection with structure regularization. IEEE Transactions on Neural Networks and Learning Systems.
  26. 26.
    Ma Z, Chang X, Xu Z, Sebe N, Hauptmann AG (2017) Joint attributes and event analysis for multimedia event detection. IEEE Transactions on Neural Networks and Learning Systems.
  27. 27.
    Pérez J, Arenas M, Gutierrez C (2009) Semantics and complexity of sparql. ACM Trans Database Syst 34:1–45CrossRefGoogle Scholar
  28. 28.
    Shvaiko P, Euzenat J (2005) A survey of schema-based matching approaches. In: Journal on data semantics IV. Springer, pp 146–171Google Scholar
  29. 29.
    Ten Cate B, Civili C, Sherkhonov E, Tan WC (2015) High-level why-not explanations using ontologies. In: Proceedings of the 34th ACM SIGMOD-SIGACT-SIGAI symposium on principles of database systems. ACM, pp 31–43Google Scholar
  30. 30.
    Theoharis Y, Fundulaki I, Karvounarakis G, Christophides V (2011) On provenance of queries on semantic web data. IEEE Internet Comput 15(1):31–39CrossRefGoogle Scholar
  31. 31.
    The W3C SPARQL Working Group (2013) SPARQL 1.1 overview.
  32. 32.
    Tran QT, Chan CY (2010) How to conquer why-not questions. In: Proceedings of the 2010 ACM SIGMOD international conference on management of data. ACM, pp 15–26Google Scholar
  33. 33.
    Zhang D, Han J, Jiang L, Ye S, Chang X (2017) Revealing event saliency in unconstrained video collection. IEEE Trans Image Process 26(4):1746–1758MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.MOEKLINNS LabXi’an Jiaotong UniversityXi’anChina
  2. 2.The University of QueenslandBrisbaneAustralia
  3. 3.Griffith UniverstiySouthportAustralia

Personalised recommendations