Advertisement

The Generalized Regression Neural Network Oracle

  • Walker H. LandJr.
  • J. David Schaffer
Chapter

Abstract

In this chapter, we describe what are best characterized as complex adaptive systems and give several mixture of expert systems as examples of these complex systems. This background discussion is followed by three theoretical sections covering the topics of kernel-based probability estimation systems, a generalized neural network example, and a derivation of an ensemble combination and finally, a two-view ensemble combination. A summary of the equations describing the oracle follows these sections for those readers who do not want to work through all that mathematics. The next section introduces Receiver Operator Characteristic (ROC) analysis, a popular method for quantitatively assessing the performance of learning classifier systems. Next is the definition of “trouble-makers”, and how they were discovered, followed by a discussion of the development of two hybrids: an Evolutionary Programming-Adaptive boosting (EP-AB) and a Generalized Regression Neural Network (GRNN) oracle for the purpose of demonstrating the existence of the trouble-makers by using an ROC measure of performance analysis. That discussion is followed by a detailed discussion of how to perform and evaluate an ROC analysis as well as a detailed practice example for those readers not familiar with this measure of performance technology. This chapter concludes with a research study on how to use the oracle to establish if the data sample size is adequate to accurately meet a 95% confidence interval imposed on the variance (or standard deviation) for the oracle. This is an important research study as very little effort is generally put into establishing the correct data set size for accurate, predictable, and repeatable performance results.

Keywords

Generalized regression neural network oracle Ensemble processing Mixture of experts processing Kernel-based probability estimation Receiver operator characteristic (ROC) curve Trouble-makers Evolutionary programming Adaptive boosting Estimating sample size 

Abbreviations

AB

Adaptive boosting

ANN

Artificial neural network

AUC

Area under the curve

CAS

Complex Adaptive System

EP

Evolutionary programming

FN

False negative

FP

False positive

GRNN

Generalized Regression Neural Network

LDA

Linear discriminant analysis

LR

Logistic regression

MLFN

Multi-layered feed forward neural network

MLP

Multi-layer perceptron

MOE

Margin of error

PNN

Probabilistic Neural Network

ROC

Receiver operator characteristic

SLT

Statistical learning theory

SVM

Support vector machine

References

  1. Bagnasco S, Bottigli U, Cerello P, Cheran S, Delogu P, Fantacci ME, Fauci F, Forni G, Lauria A, Torres EL, Magro R, Masala GL, Oliva P, Palmiero R, Ramello L, Raso G, Retico A, Sitta M, Stumbo S, Tangaro S, Zanon E (2005) GPCALMA: a GRID based tool for mammographic screening. Methods Inf Med 44(2):244–248CrossRefGoogle Scholar
  2. Baker JA, Kornguth PJ, Lo JY, Williford ME, Floyd CE Jr (1995) Breast cancer: prediction with artificial neural network based on BI-RADS standardized lexicon. Radiology 196:817–822CrossRefGoogle Scholar
  3. BI-RADS (1993) Breast Imaging—Reporting and Data System (BI-RADS). American College of Radiology, VirginiaGoogle Scholar
  4. Brown G, Wyatt J, Harris R, Yao X (2005) Diversity creation methods: a survey and categorisation. Inf Fusion 6:5–20CrossRefGoogle Scholar
  5. Cacoullos T (1966) Estimation of a multivariate density. Ann Inst Stat Math 18:179–189MathSciNetCrossRefGoogle Scholar
  6. Hand DJ, Till RJ (2001) A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach Learn 45(2):171–186CrossRefGoogle Scholar
  7. Land WH, Masters T, Lo JY (2000a) Performance evaluation using the GRNN Oracle and a new evolutionary programming/adaptive boosting hybrid for breast cancer benign/malignant diagnostic aids, ANNIEGoogle Scholar
  8. Land WH, Masters T, Lo JY (2000b) Application of a new evolutionary programming/adaptive boosting hybrid to breast cancer diagnosis. In: IEEE congress on evolutionary computation proceedings (CEC2000)Google Scholar
  9. Land WH, Masters TD, Lo JY (2000c) Application of a GRNN oracle to the intelligent combination of several breast cancer benign/malignant predictive paradigms. In: Hanson KM (ed) Medical imaging 2000; image processing, proceedings of SPIE, San Diego, CA, pp 77–85Google Scholar
  10. Land WH, Masters T, Lo JY, McKee D, (2000d) Using evolutionary computation to develop neural network breast cancer benign/malignant classification models. In: 4th world conference on systemics, cybernetics and informatics (SCI2000), vol 10, pp 343–347Google Scholar
  11. Land WH, Margolis D, Kallergi M, Heine JJ (2010) A kernel approach for ensemble decision combinations with two-view mammography applications. Int J Funct Inf Pers Med 3(2):157–182Google Scholar
  12. Lo JY, Baker JA, Kornguth PJ, Floyd CE Jr (1995) Computer-aided diagnosis of breast cancer: artificial neural network approach for optimized merging of mammographic features. Acad Radiol 2:841–850CrossRefGoogle Scholar
  13. Lo JY, Baker JA, Kornguth PJ, Iglehart JD, Floyd CE Jr (1999) Effect of patient history data on the prediction of breast cancer from mammographic findings with artificial neural networks. Acad Radiol 6:10–15CrossRefGoogle Scholar
  14. Masters T (1995) Advanced algorithms for neural networks, A C++ source book. Wiley, New York, ISBN 0-471-10588-09 (paper/disk)Google Scholar
  15. Masters T, Land WH (1997) A new training method for the general regression neural network. In: IEEE international SMC conference proceedings, pp 1990–1995Google Scholar
  16. Masters T, Land WH, Maniccam S (1998) An oracle based on the general regression neural network. In: IEEE international conference on systems, man, and cybernetics—SMC, vol 2, pp 1615–1618Google Scholar
  17. Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philos Trans R Soc Lond A 209:415–446CrossRefGoogle Scholar
  18. Nadaraya EA (1964) On estimating regression. Theory Probab Appl 9:141–142CrossRefGoogle Scholar
  19. Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065MathSciNetCrossRefGoogle Scholar
  20. Pierson M (2002) Complex adaptive systems, a definition. http://radio-weblogs.com/0107584/stories/2002/05/13/complexAdaptiveSystemsADefinition.html [last visited 2015-05-19]
  21. Polikar R (2006) Ensemble based systems in decision making. IEEE Circuits Syst Mag 6(3):21–45CrossRefGoogle Scholar
  22. Powers DMW (2011) Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2(1):37–63MathSciNetGoogle Scholar
  23. Raponi M, Zhang Y, Yu J, Chen G, Lee G, Taylor JMG, Macdonald J, Thomas D, Moskaluk C, Wang Y, Beer DG (2006) Gene expression signatures for predicting prognosis of squamous cell and adenocarcinomas of the lung. Cancer Res 66(15):7466–7472. [http://cancerres.aacrjournals.org/cgi/content/abstract/66/15/7466]CrossRefGoogle Scholar
  24. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  25. Shedden K, Taylor JMG, Enkemann SA, Tsao M-S, Yeatman TJ, Gerald WL, Eschrich S, Jurisica I, Giordano TJ, Misek DE, Chang AC, Zhu CQ, Strumpf D, Hanash S, Shepherd FA, Ding K, Seymour L, Naoki K, Pennell N, Weir B, Verhaak R, Ladd-Acosta C, Golub T, Gruidl M, Sharma A, Szoke J, Zakowski M, Rusch V, Kris M, Viale A, Motoi N, Travis W, Conley B, Seshan VE, Meyerson M, Kuick R, Dobbin KK, Lively T, Jacobson JW (2008) Beer, gene expression–based survival prediction in lung adenocarcinoma: a multi-site, blinded validation study. Nat Med 14(8):822–827CrossRefGoogle Scholar
  26. Specht DF (1990) Probabilistic neural networks. Neural Netw 3:109–118CrossRefGoogle Scholar
  27. Specht DF (1991) A general regression neural network. IEEE Trans Neural Netw 2:568–576CrossRefGoogle Scholar
  28. Stern H (2003) Improving on the mixture of experts algorithm. Computational Neuroscience Project, Dalhousie University, HalifaxGoogle Scholar
  29. Tang K, Wang R, Chen T (2011), Towards maximizing the area under the ROC for Multi_class classification problems. In: Proceedings 25th conference on AI, AAAI, pp 483–488Google Scholar
  30. Taylor JR (1982) An introduction to error analysis. University Science Books, Mill ValleyGoogle Scholar
  31. Vapnik VN (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  32. Vapnik VN (2000) The nature of statistical learning theory. Springer, New YorkCrossRefGoogle Scholar
  33. Watson GS (1964) Smooth regression analysis. Sankhya 26:359–372MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Walker H. LandJr.
    • 1
  • J. David Schaffer
    • 2
  1. 1.Binghamton UniversityBowieUSA
  2. 2.Binghamton UniversityBinghamtonUSA

Personalised recommendations