Generalized Multitarget Linear Regression with Output Dependence Estimation

  • Hector Gonzalez
  • Carlos Morell
  • Francesc J. FerriEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)


Multitarget regression has recently received attention in the context of modern, large-scale problems in which finding good enough solutions in a timely manner is crucial. Different proposed alternatives use a combination of regularizers that lead to different ways of solving the problem. In this work, we introduce a general formulation with several regularizers. This leads to a biconvex minimization problem and we use an alternating procedure with accelerated proximal gradient steps to solve it. We show that our formulation is equivalent but more efficient than some previously proposed approaches. Moreover, we introduce two new variants. The experimental validation carried out, suggests that important performance gains can be obtained with the newly proposed approach in several different publicly available multitarget regression problems.


Multitarget regression Accelerated proximal gradient 


  1. 1.
    Bakir, G.H., Hofmann, T., Schölkopf, B., Smola, A.J., Taskar, B., Vishwanathan, S.V.N.: Predicting Structured Data (Neural Information Processing). The MIT Press, Cambridge (2007)CrossRefGoogle Scholar
  2. 2.
    Borchani, H., Varando, G., Bielza, C., Larrañaga, P.: A survey on multi-output regression. Wiley Interdisc. Rev.: Data Min. Knowl. Disc. 5(5), 216–233 (2015)Google Scholar
  3. 3.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)CrossRefGoogle Scholar
  4. 4.
    Breiman, L., Friedman, J.H.: Predicting multivariate responses in multiple linear regression. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 59(1), 3–54 (1997)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Chen, J., Zhou, J., Ye, J.: Integrating low-rank and group-sparse structures for robust multi-task learning. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 42–50. ACM (2011)Google Scholar
  6. 6.
    Izenman, A.J.: Reduced-rank regression for the multivariate linear model. J. Multivar. Anal. 5(2), 248–264 (1975)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Rothman, A.J., Levina, E., Zhu, J.: Sparse multivariate regression with covariance estimation. J. Comput. Graph. Stat. 19(4), 947–962 (2010)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Simon, N., Friedman, J., Hastie, T.: A blockwise descent algorithm for group-penalized multiresponse and multinomial regression (2013)Google Scholar
  10. 10.
    Spyromitros-Xioufis, E., Tsoumakas, G., Groves, W., Vlahavas, I.: Multi-target regression via input space expansion: treating targets as inputs. Mach. Learn. 104(1), 55–98 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Tibshirani, R., Wainwright, M., Hastie, T.: Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC, Boca Raton (2015)zbMATHGoogle Scholar
  12. 12.
    Tsoumakas, G., Spyromitros-Xioufis, E., Vrekou, A., Vlahavas, I.: Multi-target regression via random linear target combinations. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8726, pp. 225–240. Springer, Heidelberg (2014). Scholar
  13. 13.
    Udell, M., Horn, C., Zadeh, R., Boyd, S., et al.: Generalized low rank models. Found. Trends® Mach. Learn. 9(1), 1–118 (2016)Google Scholar
  14. 14.
    Yuan, M., Ekici, A., Zhaosong, L., Monteiro, R.: Dimension reduction and coefficient estimation in multivariate linear regression. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 69(3), 329–346 (2007)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc.: Ser. B 68(1), 49–67 (2006)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Zhen, X., Mengyang, Y., He, X., Li, S.: Multi-target regression via robust low-rank learning. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 497–504 (2018)CrossRefGoogle Scholar
  17. 17.
    Zhen, X., et al.: Multitarget sparse latent regression. IEEE Trans. Neural Netw. Learn. Syst. 29(5), 1575–1586 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Hector Gonzalez
    • 1
  • Carlos Morell
    • 2
  • Francesc J. Ferri
    • 3
    Email author
  1. 1.Universidad de las Ciencias Informaticas (UCI)La HabanaCuba
  2. 2.Universidad Central Marta Abreu (UCLV)Villa ClaraCuba
  3. 3.Dept. InformàticaUniversitat de València BurjassotSpain

Personalised recommendations