Abstract
An adaptive test is a computer-based testing technique which adjusts the sequence of questions on the basis of the estimated ability level of the test taker. We suggest the use of credal networks, a generalization of Bayesian networks based on sets of probability mass functions, to implement adaptive tests exploiting the knowledge of the test developer instead of training on databases of answers. Compared to Bayesian networks, these models might offer higher expressiveness and hence a more reliable modeling of the qualitative expert knowledge. The counterpart is a less straightforward identification of the information-theoretic measure controlling the question-selection and the test-stopping criteria. We elaborate on these issues and propose a sound and computationally feasible procedure. Validation against a Bayesian-network approach on a benchmark about German language proficiency assessments suggests that credal networks can be reliable in assessing the student level and effective in reducing the number of questions required to do it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Extension to non Boolean answers is trivial as all answers \(Y_i\) are manifest variables, and, thus, \(Y_i\) can be always regarded as a binary variable with the two values denoting the observed answer \(y_i\) and its negation [15].
- 2.
To have entropy levels between zero and one, we define the entropy of the PMF P(X) as \(H(X):=-\sum _x P(x) \log _b P(x)\), with b number of states of X.
- 3.
E.g., if f(x) and g(x) are convex functions of x, \(h(x,y):= y f(x) + (1-y) g(x)\) is not convex even for \(0 \le y \le 1\).
- 4.
- 5.
These data as well as the software used for the simulations are freely available at http://ipg.idsia.ch/software.php?id=138.
References
Pollard, E., Hillage, J.: Exploring e-learning. Inst. for Empl., Studies Brighton (2001)
Burns H., Luckhardt, C.A., Parlett, J.W., Redfield, C.L.: Intelligent Tutoring Systems: Evolutions in Design. Psychology Press (2014)
Koller, D., Friedman, N., Models, P.G.: Principles and Techniques. MIT Press, Cambridge (2009)
Almond, R.G., Mislevy, R.J., Steinberg, L., Yan, D., Williamson, D.: Bayesian Networks in Educational Assessment. Springer, New York (2015)
Badaracco, M., Martínez, L.: A fuzzy linguistic algorithm for adaptive test in intelligent tutoring system based on competences. Expert Syst. Appl. 40(8), 3073–3086 (2013)
Renooij, S., Witteman, C.: Talking probabilities: communicating probabilistic information with words and numbers. Int. J. Approx. Reason. 22(3), 169–194 (1999)
Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis (1991)
Piatti, A., Antonucci, A., Zaffalon, M.: Building knowledge-based systems by credal networks: a tutorial. In: Baswell, A.R. (ed.) Advances in Mathematics Research, vol. 11. Nova Science Publishers, New York (2010)
Troffaes, M.: Decision making under uncertainty using imprecise probabilities. Int. J. Approx. Reason. 45(1), 17–29 (2007)
Klir, G., Wierman, M.: Uncertainty-Based Information: Elements of Generalized Information Theory. STUDFUZZ, vol. 15. Springer, Heidelberg (1999)
Mauá, D., de Campos, C., Benavoli, A., Antonucci, A.: Probabilistic inference in credal networks: new complexity results. J. Artif. Intell. Res. 50, 603–637 (2014)
Hambleton, R.K., Swaminathan, H.: Item Response Theory: Principles and Applications, vol. 7. Springer Science & Business Media, New York (1985)
Vomlel, J.: Building adaptive tests using Bayesian networks. Kybernetika 40(3), 333–348 (2004)
Plajner, M., Vomlel, J.: Bayesian network models for adaptive testing, arXiv preprint arXiv:1511.08488
Antonucci, A., Piatti, A.: Modeling unreliable observations in bayesian networks by credal networks. In: Godo, L., Pugliese, A. (eds.) SUM 2009. LNCS (LNAI), vol. 5785, pp. 28–39. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04388-8_4
Cozman, F.G.: Credal networks. Artif. Intell. 120, 199–233 (2000)
Antonucci, A., de Campos, C., Zaffalon, M., Huber, D.: Approximate credal network updating by linear programming with applications to decision making. Int. J. Approx. Reason. 58, 25–38 (2014)
Zaffalon, M., Corani, G., Mauá, D.: Evaluating credal classifiers by utility-discounted predictive accuracy. Int. J. Approx. Reason. 53(8), 1282–1301 (2012)
Abellan, J., Moral, S.: Maximum of entropy for credal sets. Int. J. Uncertain. Fuzz. 11(05), 587–597 (2003)
Antonucci, A., Corani, G.: The multilabel naive credal classifier. Int. J. Approx. Reason. 83, 320–336 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mangili, F., Bonesana, C., Antonucci, A. (2017). Reliable Knowledge-Based Adaptive Tests by Credal Networks. In: Antonucci, A., Cholvy, L., Papini, O. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2017. Lecture Notes in Computer Science(), vol 10369. Springer, Cham. https://doi.org/10.1007/978-3-319-61581-3_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-61581-3_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61580-6
Online ISBN: 978-3-319-61581-3
eBook Packages: Computer ScienceComputer Science (R0)