Skip to main content
SpringerLink
Log in

PCA-based magnification method for revealing small signals in video

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Video magnification techniques are useful for visualizing small changes in videos. Current methods are mainly applied for two aspects: motion amplification and color amplification. For instance, Eulerian video magnification (EVM) has shown impressive results in the context of color of human face and subtle head motion caused by the influx of blood at each beat. Such visual results have possible applications in non-contact human physiological parameter measurement, such as heart rate estimation. Unfortunately, the linear EVM is sensitive to noise and frequencies of the changes should be customized, which generates a limitation of applications. This paper presents an advanced EVM for magnifying the signal amplitude in the presence of relatively high noise as well as unknown the frequencies of changes in video. Principal component analysis (PCA) is performed to decompose the frames and the component whose spatial variation best matches small changes to be magnified. The advantage of PCA-based method is that it can select the subtle signals with a denoising process like spatial filtering. Experimental results show that the PCA-based EVM can support larger amplification factors for small changes visualization as well as less noise and artifacts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Rubinstein, M., Wadhwa, N., Durand, F., Freeman, W.T., Wu, H.Y.: Revealing invisible changes in the world. Science 339(6119), 519 (2013)

    Google Scholar 

  2. Davis, A., Rubinstein, M., Mysore, G.J., Mysore, G.J., Freeman, W.T.: The visual microphone: passive recovery of sound from video. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 79:179:10 (2014). https://doi.org/10.1145/2601097.2601119

    Article  MATH  Google Scholar 

  3. Davis, A., Bouman, K.L., Chen, J.G., Rubinstein, M., Buyukozturk, O., Durand, F., Freeman, W.T.: Visual vibrometry: estimating material properties from small motions in video. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 732–745 (2017). https://doi.org/10.1109/TPAMI.2016.2622271

    Article  Google Scholar 

  4. Xue, T., Rubinstein, M., Wadhwa, N., Levin, A., Durand, F., Freeman, W.T.: Refraction wiggles for measuring fluid depth and velocity from video. In: European Conference on Computer Vision. Springer, pp. 767–782 (2014). https://doi.org/10.1007/978-3-319-10578-9_50

  5. Balakrishnan, G., Durand, F., Guttag, J.: Detecting pulse from head motions in video. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 3430–3437 (2013). https://doi.org/10.1109/CVPR.2013.440

  6. Wu, H., Rubinstein, M., Shih, E., Guttag, J.V., Durand, F., Freeman, W.T.: Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proc. SIGGRAPH 2012) 31(4), 65 (2012). https://doi.org/10.1145/2185520.2185561

    Google Scholar 

  7. Alinovi, D., Cattani, L., Ferrari, G., Pisani, F.: Spatio-temporal video processing for respiratory rate estimation. In: IEEE International Symposium on Medical Measurements and Applications, pp. 12–17 (2015). https://doi.org/10.1109/MeMeA.2015.7145164

  8. Liu, C., Torralba, A., Freeman, W.T., Durand, F., Adelson, E.H.: Motion magnification. ACM Trans. Graph. 24(3), 519–526 (2005). https://doi.org/10.1016/10.1145/1073204.1073223

    Article  Google Scholar 

  9. Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Phase-based video motion processing. ACM Trans. Graph. 32(4), 1–10 (2013). https://doi.org/10.1145/2461912.2461966

    Article  MATH  Google Scholar 

  10. Rubinstein, M.: Analysis and Visualization of Temporal Variations in Video, pp. 51–97. Massachusetts Institute of Technology, Cambridge (2014)

    Google Scholar 

  11. Elgharib, M.A., Hefeeda, M., Durand, F., Freeman, W.T.: Video magnification in presence of large motions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4119–4127 (2015). https://doi.org/10.1109/CVPR.2015.7299039

  12. Zhang, Y., Pintea, S.L., van Gemert, J.C.: Video acceleration magnification. In: IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, pp. 502–510 (2017). https://doi.org/10.1109/CVPR.2017.61

  13. Zhao, A., Durand, F., Guttag, J.V.: Estimating a small signal in the presence of large noise. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 671–676 (2015). https://doi.org/10.1109/ICCVW.2015.91

  14. Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Riesz pyramids for fast phase-based video magnification. In: International Conference on Computational Photography, pp. 1–10 (2014). https://doi.org/10.1109/ICCPHOT.2014.6831820

  15. Kooij, J.F.P., van Gemert, J.C.: Depth-aware motion magnification. In: European Conference on Computer Vision. Springer, pp. 467–482 (2016). https://doi.org/10.1007/978-3-319-46484-8_28

  16. Parsi, A., Ghanbari Sorkhi, A., Zahedi, M.: Improving the unsupervised LBG clustering algorithm performance in image segmentation using principal component analysis. Signal Image Video Process. (SIViP) 10(2), 301–309 (2016). https://doi.org/10.1007/s11760-014-0742-4

    Article  Google Scholar 

  17. Wadhwa, N., Wu, H.Y., Davis, A., Rubinstein, M., Shih, E., Mysore, G.J., Chen, J.G., Buyukozturk, O., Guttag, J.V., Freeman, W.T., Durand, F.: Eulerian video magnification and analysis. Commun. ACM 60(1), 87–95 (2017). https://doi.org/10.1145/3015573

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Michael Rubinstein and Dr. Neal Wadhwa for their doctoral dissertation and technical report. Furthermore, we thank the Massachusetts Institute Of Technology (MIT) Computer Science and Artificial Intelligence Lab (CSAIL) for providing and sharing video sources and codes. We acknowledge funding support from: Training Programme Foundation for Application of Scientific and Technological Achievements of Hefei University of Technology (JZ2016YYPY0051).

Author information

Authors and Affiliations

  1. School of Computer and Information, Hefei University of Technology, Hefei, 230009, China

    Xiu Wu, Xuezhi Yang, Jing Jin & Zhao Yang

  2. Anhui Province Key Laboratory of Industry Safety and Emergency Technology, Hefei, 230009, China

    Xiu Wu, Xuezhi Yang, Jing Jin & Zhao Yang

Authors
  1. Xiu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuezhi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuezhi Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Yang, X., Jin, J. et al. PCA-based magnification method for revealing small signals in video. SIViP 12, 1293–1299 (2018). https://doi.org/10.1007/s11760-018-1282-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1282-0

Keywords

Search

Navigation