Knowledge and Information Systems

, Volume 51, Issue 1, pp 235–257

Supervised item response models for informative prediction

Regular Paper

DOI: 10.1007/s10115-016-0976-2

Cite this article as:
Idé, T. & Dhurandhar, A. Knowl Inf Syst (2017) 51: 235. doi:10.1007/s10115-016-0976-2

Abstract

Supporting human decision-making is a major goal of data mining. The more decision-making is critical, the more interpretability is required in the predictive model. This paper proposes a new framework to build a fully interpretable predictive model for questionnaire data, while maintaining a reasonable prediction accuracy with regard to the final outcome. Such a model has applications in project risk assessment, in healthcare, in social studies, and, presumably, in any real-world application that relies on questionnaire data for informative and accurate prediction. Our framework is inspired by models in item response theory (IRT), which were originally developed in psychometrics with applications to standardized academic tests. We extend these models, which are essentially unsupervised, to the supervised setting. For model estimation, we introduce a new iterative algorithm by combining Gauss–Hermite quadrature with an expectation–maximization algorithm. The learned probabilistic model is linked to the metric learning framework for informative and accurate prediction. The model is validated by three real-world data sets: Two are from information technology project failure prediction and the other is an international social survey about people’s happiness. To the best of our knowledge, this is the first work that leverages the IRT framework to provide informative and accurate prediction on ordinal questionnaire data.

Keywords

Questionnaire data Item response theory Metric learning 

Copyright information

© Springer-Verlag London 2016

Authors and Affiliations

  1. 1.IBM Research, T. J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations