Advertisement

Motion Estimation and Compensation with Neural Fuzzy Systems

  • Hyun Mun Kim
  • Bart Kosko
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 42)

Abstract

This chapter shows how neural fuzzy systems can improve motion estimation and compensation for video compression. Motion estimation and compensation are key parts of video compression. They help remove temporal redundancies in images. Most motion estimation algorithms neglect the strong-temporal correlations within the motion field. The search windows stay the same through the image sequences and the estimation needs heavy computation. We use an unsupervised neural vector quantizer system can use the temporal correlation of the motion field to estimate the motion vectors. First-order and second-order statistics of the motion vectors give ellipsoidal search windows. This algorithm reduces the search area and entropy and gives clustered motion fields. Motion-compensated video coding further assumes that each block of pixels moves with uniform translational motion. This often does not hold and can produce block artifacts. We use a supervised neural fuzzy system to compensate for the overlapped block motion. This fuzzy system uses the motion vectors of neighboring blocks to map the prior frame’s pixel values to the current pixel value. The neural fuzzy system used 196 rules that came from the prior decoded frame. The fuzzy system learns and updates its rules as it decodes the image. The fuzzy system also improved the compensation accuracy. The appendix derives both the fuzzy system and the supervised neural-like learning laws that tune its parameters.

Keywords

Fuzzy System Motion Vector Motion Estimation Motion Compensation Search Window 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    “MPEG-4 Video Verification Model Version 11.0,” ISO/IEC JTC1/SC29/WG11/N2172, Tokyo, March, 1998.Google Scholar
  2. 2.
    Cacoullos, T., “Estimation of a Multivariate Density,” Annals of the Institute of Statistical Mathematics (Tokyo), Vol. 18, No. 2, 179–189, 1966.MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Dickerson, J. A., and Kosko, B., “Fuzzy Function Learning with Covariance Ellipsoids,” Proceedings of the 2nd IEEE International Conference on Fuzzy Systems (IEEE FUZZ-93), 1162–1167, March 1993.Google Scholar
  4. 4.
    Dickerson, J. A., and Kosko, B., “Fuzzy Function Approximation with Supervised Ellipsoidal Learning, ” World Congress on Neural Networks (WCNN ’93), Vol. II, 9–13, July 1993.Google Scholar
  5. 5.
    Dickerson, J. A., and Kosko, B., “Fuzzy Function Approximation with Ellipsoidal Rules, ” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 26, No. 4, 542–560, August 1996.CrossRefGoogle Scholar
  6. 6.
    Dufaux, F., Moccagatta, I., Rouchouze, B., Ebrahimi, T., and Kunt, M., “Motion-Compensated Generic Coding of Video Based on a Multiresolution Data Structure, ” Optical Engineering, Vol. 32, No. 7, 1559–1570, July 1993.CrossRefGoogle Scholar
  7. 7.
    Gall, D. L., “MPEG: A Video Compression Standard for Multimedia Applications, ” Communications ACM, Vol. 34, 46–58, April 1991.CrossRefGoogle Scholar
  8. 8.
    Girod, B., Gall, D., Sezen, M. I., Vitterli, M., and Yasuda, H., “Guest Editorial: Introduction to the Special Issue on Image Sequence Compression, ” IEEE Transactions on Image Processing, Vol. 3, No. 5, 465–468, September 1994.Google Scholar
  9. 9.
    Jain, A. K., and Jain, J. R., “Displace Measurement and Its Application in Inter-frame Image Coding, ” IEEE Transactions on Communication, Vol. 29, 1799–1808, December 1981.CrossRefGoogle Scholar
  10. 10.
    Kappagantula, S., and Rao, K. R., “Motion compensated Interframe Image Prediction, ” IEEE Transactions on Communications, Vol. 33, 1011–1015, September 1985.CrossRefGoogle Scholar
  11. 11.
    Katto, J., Ohki, J., Nogoki, S., and Ohta, M., “A Wavelet Codec with Overlapped Motion Compensation for Very Low Bit-Rate Environment, ” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 4, 328–338, June 1994.CrossRefGoogle Scholar
  12. 12.
    Kim, H. M. and Kosko, B., “Fuzzy Prediction And Filtering in Impulsive Noise,” Fuzzy Sets and Systems, Vol. 77, No. 1, 15–33, January 1996.CrossRefGoogle Scholar
  13. 13.
    Kim, S. and Kuo, C.-C. Jay, “A Stochastic Approach for Motion Vector Estimation in Video Coding,” SPIE’s 1994 Annual Meeting, San Diego, California, July 1994.Google Scholar
  14. 14.
    Kong, S. and Kosko, B., “Fuzzy Subimage Classification in Image Sequence Coding,” Proceedings of ICASSP-92, Vol. Ill, 517–520, March 1992.Google Scholar
  15. 15.
    Kosko, B., “Stochastic Competitive Learning,” IEEE Transactions on Neural Networks, Vol. 2, No. 5, 522–529, September 1991.CrossRefGoogle Scholar
  16. 16.
    Kosko, B., Neural Networks and Fuzzy Systems, Prentice Hall, Englewood Cliffs, NJ, 1991.Google Scholar
  17. 17.
    Kosko, B., Neural Networks for Signal Processing, Prentice Hall, Englewood Cliffs, NJ, 1991.Google Scholar
  18. 18.
    Kosko, B., “Fuzzy Systems as Universal Approximators,” IEEE Transactions on Computers, Vol. 43, No. 11, 1329–1333, November 1994; an early version appears in Proceedings of the 1st IEEE International Conference on Fuzzy Systems (IEEE FUZZ-92), 1153–1162, March 1992.Google Scholar
  19. 19.
    Kosko, B., “Optimal Fuzzy Rules Cover Extrema,” International Journal of Intelligent Systems, Vol. 10, No. 2, 249–255, February 1995.CrossRefGoogle Scholar
  20. 20.
    Kosko, B., “Combining Fuzzy Systems,” Proceedings of the IEEE International Conference on Fuzzy Systems (IEEE FUZZ-95), Vol. IV, 1855–1863, March 1995.Google Scholar
  21. 21.
    Kosko, B., Fuzzy Engineering, Prentice Hall, Englewood Cliffs, NJ, 1996.Google Scholar
  22. 22.
    Kosko, B., “Global Stability of Generalized Additive Fuzzy Systems,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 28, No. 3, 441–452, August 1998.CrossRefGoogle Scholar
  23. 23.
    Li, H., Lundmark, A., and Forchheimer, R., “Image Sequence Coding at Very Low Bitrates: A Review,” IEEE Transactions on Image Processing, Vol. 3, No. 5, 589–609, September 1994.CrossRefGoogle Scholar
  24. 24.
    Mitaim, S., and Kosko, B., “What is the Best Shape for a Fuzzy Set in Function Approximation?,” Proceedings of the 5th IEEE International Conference on Fuzzy Systems (FUZZ-96), Vol. 2, 1237–1243, September 1996.Google Scholar
  25. 25.
    Mitaim, S., and Kosko, B., “Adaptive Joint Fuzzy Sets for Function Approximation,” Proceedings of the 1997 IEEE International Conference on Neural Networks (ICNN-97), Vol. 1, 537–542, June 1997.Google Scholar
  26. 26.
    Orchard, M. T., and Sullivan, G. J., “Overlapped Block Motion Compensation: An Estimation-Theoretic Approach,” IEEE Transactions on Image Processing, Vol. 3, No. 5, 693–699, September 1994.CrossRefGoogle Scholar
  27. 27.
    Orchard, M. T., “New Pel-Recursive Motion Estimation Algorithms Based on Novel Interpolation Kernels,” SPIE, Visual Communications and Image Processing, Vol. 1818, 85–96, 1992.Google Scholar
  28. 28.
    Pacini, P. J., and Kosko, B., “Adaptive Fuzzy Frequency Hopper,” IEEE Transactions on Communications, Vol. 43, No. 6, 2111–2117, June 1995.CrossRefGoogle Scholar
  29. 29.
    Papoulis, A., Probability, Random Variables, and Stochastic processes, McGraw-Hill, 2nd-ed., 1984.MATHGoogle Scholar
  30. 30.
    Parzen, E., “On Estimation of a Probability Density Function and Mode,” Annals of Mathematical Statistics, Vol. 33, 1065–1076, 1962.MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Ramamurthi, B., and Gersho A., “Classified Vector Quantization of Images,” IEEE Transactions on Communications, Vol. COM-34, No. 11, 1105–1115, November 1986.CrossRefGoogle Scholar
  32. 32.
    Specht, D. F. , “A General Regression Neural Network,” IEEE Transactions on Neural Networks, Vol. 2, No. 6, 568–576, 1991.CrossRefGoogle Scholar
  33. 33.
    Widrow B, and Sterns, S. D., Adaptive Signal Processing, Prentice Hall, Englewood Cliffs, NJ, 1985.MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Hyun Mun Kim
    • 1
  • Bart Kosko
    • 1
  1. 1.Signal and Image Processing Institute Department of Electrical Engineering — SystemsUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations