Learning with Partially Shared Features for Multi-Task Learning
The objective of Multi-Task Learning (MTL) is to boost learning performance by simultaneously learning multiple relevant tasks. Identifying and modeling the task relationship is essential for multi-task learning. Most previous works assume that related tasks have common shared structure. However, this assumption is too restrictive. In some real-world applications, relevant tasks are partially sharing knowledge at the feature level. In other words, the relevant features of related tasks can partially overlap. In this paper, we propose a new MTL approach to exploit this partial relationship of tasks, which is able to selectively exploit shared information across the tasks while produce a task-specific sparse pattern for each task. Therefore, this increased flexibility is able to model the complex structure among tasks. An efficient alternating optimization has been developed to optimize the model. We perform experimental studies on real world data and the results demonstrate that the proposed method significantly improves learning performance by simultaneously exploiting the partial relationship across tasks at the feature level.
KeywordsMulti-Task Learning Partially task relationship
The work described in this paper was partially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No. CityU 11300715], and a grant from City University of Hong Kong [Project No. 7004674].
- 1.Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Advances in Neural Information Processing Systems, pp. 19–41 (2007)Google Scholar
- 3.Chen, J., Zhou, J., Ye, J.: Integrating low-rank and group-sparse structures for robust multi-task learning. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 42–50 (2011)Google Scholar
- 4.Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117 (2004)Google Scholar
- 5.Gong, P., Ye, J., Zhang, C.: Robust multi-task feature learning. In: Proceedings of the 18th ACM SIGKDD International conference on Knowledge Discovery and Data Mining, pp. 895–903 (2012)Google Scholar
- 6.Jacob, L., Vert, J.P., Bach, F.R.: Clustered multi-task learning: a convex formulation. In: Advances in Neural Information Processing Systems, pp. 745–752 (2009)Google Scholar
- 7.Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 521–528 (2011)Google Scholar
- 9.Kumar, A., Daume III, H.: Learning task grouping and overlap in multi-task learning. arXiv preprint arXiv:1206.6417 (2012)
- 10.Lee, S., Zhu, J., Xing, E.P.: Adaptive multi-task lasso: with application to eQTL detection. In: Advances in Neural Information Processing Systems, pp. 1306–1314 (2010)Google Scholar
- 11.Lee, G., Yang, E., Hwang, S.J.: Asymmetric multi-task learning based on task relatedness and loss. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 230–238 (2016)Google Scholar
- 14.Maurer, A., Pontil, M., Romera-Paredes, B.: Sparse coding for multitask and transfer learning. In: Proceedings of the 30th International Conference on Machine Learning, pp. 343–351 (2013)Google Scholar
- 15.Obozinski, G., Taskar, B., Jordan, M.: Multi-task feature selection. Statistics Department, UC Berkeley, Technical report 17 (2006)Google Scholar
- 16.Parameswaran, S., Weinberger, K.Q.: Large margin multi-task metric learning. In: Advances in Neural Information Processing Systems, pp. 1867–1875 (2010)Google Scholar