Advertisement

Quantifying Uncertainty

  • Walker H. LandJr.
  • J. David Schaffer
Chapter
  • 326 Downloads

Abstract

This chapter introduces what we believe is a novel approach, allowing a trained classifier system to “know what it doesn’t know” and to use this procedure to allow predictions for new cases to be flagged as unreliable if they lie in a region of feature space where the classifier knows its knowledge is likely to be faulty. We show how this approach may also be used to compare alternative classifiers trained on the same data in terms of their uncertainly areas. The better classifier is the one with the smaller uncertainty area. We illustrate three approaches to define these uncertainty areas, and we are quite sure additional research may identify newer and likely better ways. This work is significantly less mature than that of previous chapters; it represents a very recent insight, and the methods described are quite heuristic. We look forward to others taking this approach in more rigorous directions.

Keywords

Uncertainty area metric Flag unreliable predictions t-SNE algorithm Dimension reduction Trouble-makers 

Abbreviations

2D

Two dimensional

GA-SVM

Genetic algorithm-support vector machine hybrid

GRNN

Generalized regression neural network

MMSE

Mini-mental state exam

MOP

Measure of performance

ROC

Receiver operator characteristic

SNE

Stochastic neighborhood embedding

TM

Trouble-Makers: training cases a learning classifier gets wrong

t-SNE

Student’s t-distribution SNE

References

  1. Donaldson J (2016) An R package for t-SNE (t-Distributed Stochastic Neighbor Embedding), GITHUB, last commit 2016. https://github.com/jdonaldson/rtsne/
  2. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86MathSciNetCrossRefGoogle Scholar
  3. Shepard R (1962) The analysis of proximities: multidimensional scaling with an unknown distance function (parts 1 and 2). Psychometrika 27:125–140, 219–249MathSciNetCrossRefGoogle Scholar
  4. van der Maaten L, Hinton GE (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Walker H. LandJr.
    • 1
  • J. David Schaffer
    • 2
  1. 1.Binghamton UniversityBowieUSA
  2. 2.Binghamton UniversityBinghamtonUSA

Personalised recommendations