We propose a new approach for classification problem based on the maximum a posteriori (MAP) estimation. The necessary and sufficient condition for the cost function to estimate a posteriori probability was obtained. It was clarified by the condition that a posteriori probability cannot be estimated by using linear programming. In this paper, a kernelized function of which result is the same as that of the MAP classifier is estimated. By relieving the problem from to estimate a posteriori probability to such a function, the freedom of cost function becomes wider. We propose a new cost function for such a function that can be solved by using linear programming. We conducted binary classification experiment by using 13 datasets from the UCI repository and compared the results to the well known methods. The proposed method outperforms the other methods for several datasets. We also explain the relation and the similarity between the proposed method and the support vector machine (SVM). Furthermore, the proposed method has other advantages for classification. Besides it can be solved by linear programming which has many excellent solvers, it does not have regularization parameter such as C in the cost function in SVM and its cost function is so simple that we can consider its various extensions for future work.


Maximum a posteriori Kernel Function Linear Programming Cost Function 


  1. 1.
    Xu, Z., Huang, K., Zhu, J., King, I., Lyu, M.R.: A Novel Kernel-Based Maximum a Posteriori Classification Method. J. Neural Networks 22, 121–146 (2009)Google Scholar
  2. 2.
    Gauvain, J.L., Lee, C.H.: Maximum a Posteriori Estimation for Multivariate Gaussian Mixture Observations of Markov Chains. J. IEEE Trans. Speech and Audio Processing 2, 291–298 (1994)CrossRefGoogle Scholar
  3. 3.
    Chen, K., Wang, H.: Eigenspace-Based Maximum a Posteriori Linear Regression for Rapid Speaker Adaptation. In: Proc. Intl. Conf. on Acoustics, Speech, and Signal Processing, Salt Lake City, USA, vol. 1, pp. 917–920 (2001)Google Scholar
  4. 4.
    Igual, J., Camachoa, A., Bernabeua, P., Vergarab, L.: A Maximum a Posteriori Estimate for the Source Separation Problem with Statistical Knowledge about the Mixing Matrix. J. Pattern Recognition Letters 24, 2519–2523 (2003)CrossRefGoogle Scholar
  5. 5.
    Siohan, O., Myrvoll, T.A., Lee, C.H.: Structural Maximum a Posteriori Linear Regression for Fast HMM adaptation. J. Computer Speech & Language 16, 5–24 (2002)CrossRefGoogle Scholar
  6. 6.
    Ng, T.M., Garg, H.K.: A Maximum a Posteriori Identification Criterion for Wavelet Domain Watermarking. International Journal of Wireless and Mobile Computing 3, 265–270 (2009)CrossRefGoogle Scholar
  7. 7.
    Sueiro, J.C., Arribas, J.I., Munoz, S.E., Vidal, A.R.F.: Cost Functions to Estimate a Posteriori Probabilities in Multiclass Problems. J. IEEE Trans. Neural Networks 10, 645–656 (1999)CrossRefGoogle Scholar
  8. 8.
    Arribas, J.I., Sueiro, J.C., Adali, T., Vidal, A.R.F.: Neural Architetures for Parametric Estimation of a Posteriori Probabilities by Constrained Conditional Density Functions. In: Proc. IEEE Workshop on Neural Networks for Signal Processing (NNSP), Madison, Wisconsin, USA, pp. 263–272 (1999)Google Scholar
  9. 9.
    Miller, J.W., Goodman, R., Smyth, P.: Objective Functions for Probability Estimation. In: International Joint Conference on Neural Networks, Seattle, USA, vol. 1, pp. 881–886 (1991)Google Scholar
  10. 10.
    Ruck, D.W., Rogers, S.K., Kabrisky, M., Oxley, M.E., Suter, B.W.: The Multilayer Perceptron as an Approximation to a Bayes Optimal Discriminant Function. J. IEEE Trans. Neural Network 1, 296–298 (1990)CrossRefGoogle Scholar
  11. 11.
    Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müller, K.R.: Fisher Discriminant Analysis with Kernels. In: IEEE Signal Processing Society Workshop In Neural Networks for Signal Processing IX, Madison, Wisconsin, USA, vol. 10, pp. 41–48 (1999)Google Scholar
  12. 12.
    Zou, H., Zhu, J., Hastie, T.: New Multicategory Boosting Algorithms Based on Multicategory Fisher-Consistent Losses. Annals of Applied Statistics 2, 1290–1306 (2008)zbMATHCrossRefGoogle Scholar
  13. 13.
    Lee, Y., Lin, Y., Wahba, G.: Multicategory Support Vector Machines Theory and Application to the Classification of Microarray Data and Satellite Radiance Data. Journal of the American Statistical Association 99, 67–81 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Huy, T., Takeda, K., Itakura, F.: Maximum a Posterior Probability and Cumulative Distribution Function Equalization Methods for Speech Spectral Estimation with Application in Noise Suppression Filtering. In: Faundez-Zanuy, M., Janer, L., Esposito, A., Satue-Villar, A., Roure, J., Espinosa-Duro, V. (eds.) NOLISP 2005. LNCS (LNAI), vol. 3817, pp. 328–337. Springer, Heidelberg (2006)Google Scholar
  15. 15.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (2004)Google Scholar
  16. 16.
    Kolman, B., Beck, R.E.: Elementary Linear Programming with Applications, 2nd edn. Academic Press, Elsevier Science & Technology Books (1995)Google Scholar
  17. 17.
    Arribas, J.I., Sueiro, J.C., Lopez, C.A.: Estimation of Posterior Probabilities with Neural Networks. In: Hanbook of Neural Engineering, IEEE Press, A John Wiley and Sons Inc. (2007)Google Scholar
  18. 18.
    Loewenster, D.M., Berman, H.M., Hirsh, H.: Maximum A Posteriori Classification of DNA Structure from Sequence Information. In: Proceedings of Pacific symposium on Biocomputing, Hawaii, USA, pp. 667–668 (1998)Google Scholar
  19. 19.
    Jaroudi, A.E., Makhoul, J.: A New Error Criterion for Posterior Probability Estimation with Neural Nets. In: International Joint Conference on Neural Networks, San Diego, CA, USA, vol. 90, pp. 185–192 (1990)Google Scholar
  20. 20.
    Chang, C.C., Lin, C.-J.: LIBSVM: a Library for Support Vector Machines (2001),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Nopriadi
    • 1
  • Yukihiko Yamashita
    • 1
  1. 1.Tokyo Institute of TechnologyMeguro-kuJapan

Personalised recommendations