Abstract
Image Quality Metrics (IQMs) automatically detect differences between images. For example, they can be used to find aliasing artifact in the computer generated images. An obvious application is to test if the costly anti-aliasing techniques must be applied so that the aliasing is not visible to humans. The performance of IQMs must be tested based on the ground truth data, which is a set of maps that indicate the location of artifacts in the image. These maps are manually created by people during so called marking experiments. In this work, we evaluate two different techniques of marking. In the side-by-side experiment, people mark differences between two images displayed side-by-side on the screen. In the flickering experiment, images are displayed at the same location but are exchanged over time. We assess the performance of each technique and use the generated reference maps to evaluate the performance of the selected IQMs. The results reveal the better accuracy of the flickering technique.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Akenine-Möller, T., Haines, E., Hoffman, N.: Real-Time Rendering, 3rd edn. A K Peters Ltd., Wallesley (2008)
Čadík, M., Herzog, R., Mantiuk, R., Mantiuk, R., Myszkowski, K., Seidel, H.P.: Learning to predict localized distortions in rendered images. Comput. Graph. Forum 32(7), 401–410 (2013)
Čadík, M., Herzog, R., Mantiuk, R., Myszkowski, K., Seidel, H.P.: New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts. ACM Trans. Graph. (TOG) 31(6), 147 (2012)
Corsini, M., Larabi, M.C., Lavoué, G., Petřík, O., Váša, L., Wang, K.: Perceptual metrics for static and dynamic triangle meshes. Comput. Graph. Forum 32(1), 101–125 (2013)
Lavoué, G., Mantiuk, R.: Quality assessment in computer graphics. In: Visual Signal Quality Assessment, pp. 243–286. Springer, Cham (2015)
Lissner, I., Preiss, J., Urban, P., Lichtenauer, M.S., Zolliker, P.: Image-difference prediction: from grayscale to color. IEEE Trans. Image Process. 22(2), 435–446 (2013)
Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30(4), 40:1–40:14 (2011)
Piórkowski, R., Mantiuk, R.: Using full reference image quality metrics to detect game engine artefacts. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception, pp. 83–90. ACM (2015)
Piórkowski, R., Mantiuk, R., Siekawa, A.: Automatic detection of game engine artifacts using full reference image quality metrics. ACM Trans. Appl. Percept. (TAP) 14(3), 14 (2017)
Rushmeier, H.E., Rogowitz, B.E., Piatko, C.: Perceptual issues in substituting texture for geometry. In: Electronic Imaging, pp. 372–383. International Society for Optics and Photonics (2000)
Sergej, T., Mantiuk, R.: Perceptual evaluation of demosaicing artefacts. In: Image Analysis and Recognition. LNCS, vol. 8814, pp. 38–45. Springer, Cham (2014)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wang, Z., Bovik, A.: Modern Image Quality Assessment. Morgan & Claypool Publishers (2006)
Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers 2004, vol. 2, pp. 1398–1402. IEEE (2003)
Wolski, K., Giunchi, D., Ye, N., Didyk, P., Mantiuk, R., Seidel, H.P., Steed, A., Mantiuk, R.K.: Dataset and metrics for predicting local visible differences. ACM Trans. Graph. (2018)
Zhang, X., Wandell, B.A.: A spatial extension of cielab for digital color-image reproduction. Journal of the Society for Information Display 5(1), 61–63 (1997)
Acknowledgments
The project was partially funded by the Polish National Science Centre (decision number DEC-2013/09/B/ST6/02270).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Piórkowski, R., Mantiuk, R. (2019). Reliability of Local Ground Truth Data for Image Quality Metric Assessment. In: Choraś, M., Choraś, R. (eds) Image Processing and Communications Challenges 10. IP&C 2018. Advances in Intelligent Systems and Computing, vol 892. Springer, Cham. https://doi.org/10.1007/978-3-030-03658-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-03658-4_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03657-7
Online ISBN: 978-3-030-03658-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)