Advertisement

Signal, Image and Video Processing

, Volume 12, Issue 7, pp 1293–1299 | Cite as

PCA-based magnification method for revealing small signals in video

  • Xiu Wu
  • Xuezhi Yang
  • Jing Jin
  • Zhao Yang
Original Paper
  • 144 Downloads

Abstract

Video magnification techniques are useful for visualizing small changes in videos. Current methods are mainly applied for two aspects: motion amplification and color amplification. For instance, Eulerian video magnification (EVM) has shown impressive results in the context of color of human face and subtle head motion caused by the influx of blood at each beat. Such visual results have possible applications in non-contact human physiological parameter measurement, such as heart rate estimation. Unfortunately, the linear EVM is sensitive to noise and frequencies of the changes should be customized, which generates a limitation of applications. This paper presents an advanced EVM for magnifying the signal amplitude in the presence of relatively high noise as well as unknown the frequencies of changes in video. Principal component analysis (PCA) is performed to decompose the frames and the component whose spatial variation best matches small changes to be magnified. The advantage of PCA-based method is that it can select the subtle signals with a denoising process like spatial filtering. Experimental results show that the PCA-based EVM can support larger amplification factors for small changes visualization as well as less noise and artifacts.

Keywords

Eulerian video magnification (EVM) Principal component analysis (PCA) Spatiotemporal analysis Motion magnification 

Notes

Acknowledgements

The authors would like to thank Dr. Michael Rubinstein and Dr. Neal Wadhwa for their doctoral dissertation and technical report. Furthermore, we thank the Massachusetts Institute Of Technology (MIT) Computer Science and Artificial Intelligence Lab (CSAIL) for providing and sharing video sources and codes. We acknowledge funding support from: Training Programme Foundation for Application of Scientific and Technological Achievements of Hefei University of Technology (JZ2016YYPY0051).

References

  1. 1.
    Rubinstein, M., Wadhwa, N., Durand, F., Freeman, W.T., Wu, H.Y.: Revealing invisible changes in the world. Science 339(6119), 519 (2013)Google Scholar
  2. 2.
    Davis, A., Rubinstein, M., Mysore, G.J., Mysore, G.J., Freeman, W.T.: The visual microphone: passive recovery of sound from video. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 79:179:10 (2014).  https://doi.org/10.1145/2601097.2601119 CrossRefzbMATHGoogle Scholar
  3. 3.
    Davis, A., Bouman, K.L., Chen, J.G., Rubinstein, M., Buyukozturk, O., Durand, F., Freeman, W.T.: Visual vibrometry: estimating material properties from small motions in video. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 732–745 (2017).  https://doi.org/10.1109/TPAMI.2016.2622271 CrossRefGoogle Scholar
  4. 4.
    Xue, T., Rubinstein, M., Wadhwa, N., Levin, A., Durand, F., Freeman, W.T.: Refraction wiggles for measuring fluid depth and velocity from video. In: European Conference on Computer Vision. Springer, pp. 767–782 (2014).  https://doi.org/10.1007/978-3-319-10578-9_50
  5. 5.
    Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 3430–3437 (2013).  https://doi.org/10.1109/CVPR.2013.440
  6. 6.
    Wu, H., Rubinstein, M., Shih, E., Guttag, J.V., Durand, F., Freeman, W.T.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH 2012) 31(4), 65 (2012).  https://doi.org/10.1145/2185520.2185561 Google Scholar
  7. 7.
    Alinovi, D., Cattani, L., Ferrari, G., Pisani, F.: Spatio-temporal video processing for respiratory rate estimation. In: IEEE International Symposium on Medical Measurements and Applications, pp. 12–17 (2015).  https://doi.org/10.1109/MeMeA.2015.7145164
  8. 8.
    Liu, C., Torralba, A., Freeman, W.T., Durand, F., Adelson, E.H.: Motion magnification. ACM Trans. Graph. 24(3), 519–526 (2005).  https://doi.org/10.1016/10.1145/1073204.1073223 CrossRefGoogle Scholar
  9. 9.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Phase-based video motion processing. ACM Trans. Graph. 32(4), 1–10 (2013).  https://doi.org/10.1145/2461912.2461966 CrossRefzbMATHGoogle Scholar
  10. 10.
    Rubinstein, M.: Analysis and Visualization of Temporal Variations in Video, pp. 51–97. Massachusetts Institute of Technology, Cambridge (2014)Google Scholar
  11. 11.
    Elgharib, M.A., Hefeeda, M., Durand, F., Freeman, W.T.: Video magnification in presence of large motions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4119–4127 (2015).  https://doi.org/10.1109/CVPR.2015.7299039
  12. 12.
    Zhang, Y., Pintea, S.L., van Gemert, J.C.: Video acceleration magnification. In: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, pp. 502–510 (2017).  https://doi.org/10.1109/CVPR.2017.61
  13. 13.
    Zhao, A., Durand, F., Guttag, J.V.: Estimating a small signal in the presence of large noise. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 671–676 (2015).  https://doi.org/10.1109/ICCVW.2015.91
  14. 14.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Riesz pyramids for fast phase-based video magnification. In: International Conference on Computational Photography, pp. 1–10 (2014).  https://doi.org/10.1109/ICCPHOT.2014.6831820
  15. 15.
    Kooij, J.F.P., van Gemert, J.C.: Depth-aware motion magnification. In: European Conference on Computer Vision. Springer, pp. 467–482 (2016).  https://doi.org/10.1007/978-3-319-46484-8_28
  16. 16.
    Parsi, A., Ghanbari Sorkhi, A., Zahedi, M.: Improving the unsupervised LBG clustering algorithm performance in image segmentation using principal component analysis. Signal Image Video Process. (SIViP) 10(2), 301–309 (2016).  https://doi.org/10.1007/s11760-014-0742-4 CrossRefGoogle Scholar
  17. 17.
    Wadhwa, N., Wu, H.Y., Davis, A., Rubinstein, M., Shih, E., Mysore, G.J., Chen, J.G., Buyukozturk, O., Guttag, J.V., Freeman, W.T., Durand, F.: Eulerian video magnification and analysis. Commun. ACM 60(1), 87–95 (2017).  https://doi.org/10.1145/3015573 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer and InformationHefei University of TechnologyHefeiChina
  2. 2.Anhui Province Key Laboratory of Industry Safety and Emergency TechnologyHefeiChina

Personalised recommendations