Abstract
We address the problem of improving search efficiency of incremental variable selection. As one application, we focus on generalized linear models that are linear with respect to their parameters, but their objective functions are not restricted to a standard sum of squared error. In this paper, we present a method for incrementally selecting a set of relevant variables together with a newly proposing criterion based on second-order optimality for our models. In our experiments using a synthetic dataset with tens of thousands of variables, we show that the proposed method was able to completely restore the relevant variables. Moreover, the method substantially improved the search efficiency in comparison to a conventional calculation method. Furthermore, it is shown that we obtained promissing initial results using a real dataset in health-checkup.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bishop, C.M.: Neural networks for pattern recognition. Clarendon Press (1995)
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proc. Int. Conf. on Machine Learning, pp. 89–96 (2005)
Chandran, M., Phillips, S.A., Ciaraldi, T., Henry, R.R.: Adiponectin: more than just another fat cell hormone? Diabetes Care 26, 2442–2450 (2003)
Duncan, B.B., Schmidt, M.I., Pankow, J.S., Bang, H., Couper, D., Ballantyne, C.M., Hoogeveen, R.C., Heiss, G.: Adiponectin and the development of type 2 diabetes: the atherosclerosis risk in communities study. Diabetes 53, 2473–2478 (2004)
Englund Ogge, L., Brohall, G., Behre, C.J., Schmidt, C., Fagerberg, B.: Alcohol consumption in relation to metabolic regulation, inflammation, and adiponectin in 64-year-old Caucasian women: a population-based study with a focus on impaired glucose regulation. Diabetes Care 29, 908–913 (2006)
Gottsater, A., Szelag, B., Kangro, M., Wroblewski, M., Sundkvist, G.: Plasma adiponectin and serum advanced glycated end-products increase and plasma lipid concentrations decrease with increasing duration of type 2 diabetes. Eur. J. Endocrinol. 151, 361–366 (2004)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003)
Hassibi, B., Stork, D.G., Wolf, G.: Optimal brain surgeon and general network pruning. In: Proc. IEEE Int. Conf. on Neural Networks, pp. 293–299 (1992)
Jaleel, F., Jaleel, A., Aftab, J., Rahman, M.A.: Relationship between adiponectin, glycemic control and blood lipids in diabetic type 2 postmenopausal women with and without complication of ischemic heart disease. Clin. Chim. Acta. 370, 76–81 (2006)
LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. Advances in Neural Information Processing Systems 2, 598–605 (1990)
Liu, H., Motoda, H.: Computational methods of feature selection. Chapman & Hall/CRC, Boca Raton (2007)
Mannucci, E., Ognibene, A., Cremasco, F., Dicembrini, I., Bardini, G., Brogi, M., Terreni, A., Caldini, A., Messeri, G., Rotella, C.M.: Plasma adiponectin and hyperglycaemia in diabetic patients. Clin. Chem. Lab. Med. 41, 1131–1135 (2003)
Matsubara, M., Namioka, K., Katayose, S.: Relationships between plasma adiponectin and blood cells, hepatopancreatic enzymes in women. Thromb Haemost, 360–366 (2004)
Nishida, M., Moriyama, T., Sugita, Y., Yamauchi-Takihara, K.: Abdominal obesity exhibits distinct effect on inflammatory and anti-inflammatory proteins in apparently healthy Japanese men. Cardiovasc Diabetol. 6, 27 (2007)
Pischon, T., Girman, C.J., Rifai, N., Hotamisligil, G.S., Rimm, E.B.: Association between dietary factors and plasma adiponectin concentrations in men. Am. J. Clin. Nutr. 81, 780–786 (2005)
Qi, L., Meigs, J.B., Liu, S., Manson, J.E., Mantzoros, C., Hu, F.B.: Dietary fibers and glycemic load, obesity, and plasma adiponectin levels in women with type 2 diabetes. Diabetes Care 29, 1501–1505 (2006)
Qi, L., Rimm, E.B., Liu, S., Rifai, N., Hu, F.B.: Dietary glycemic index, glycemic load, cereal fiber, and plasma adiponectin concentration in diabetic men. Diabetes Care 28, 1022–1028 (2005)
Rivals, I., Personnaz, L.: MLPs (Mono-Layer Polynomials and Multi-Layer Perceptrons) for Nonlinear Modeling. Journal of Machine Learning Research 3, 1383–1398 (2003)
Sakuta, H., Suzuki, T., Yasuda, H., Ito, T.: Adiponectin levels and cardiovascular risk factors in Japanese men with type 2 diabetes. Endocr. J. 52, 241–244 (2005)
Scharf, L.L.: Statistical signal processing. Addison-Wesley, Reading (1991)
Seber, G.A.F., Wild, C.J.: Nonlinear Regression. John Wiley & Sons, Chichester (1989)
Shetty, G.K., Economides, P.A., Horton, E.S., Mantzoros, C.S., Veves, A.: Circulating adiponectin and resistin levels in relation to metabolic factors, inflammatory markers, and vascular reactivity in diabetic patients and subjects at risk for diabetes. Diabetes Care 27, 2450–2457 (2004)
Stoppiglia, H., Dreyfus, G., Dubois, R., Oussar, Y.: Ranking a random feature for variable and feature selection. Journal of Machine Learning Research 3, 1399–1414 (2003)
Takefuji, S., Yatsuya, H., Tamakoshi, K., Otsuka, R., Wada, K., Matsushita, K., Sugiura, K., Hotta, Y., Mitsuhashi, H., et al.: Smoking status and adiponectin in healthy Japanese men and women. Prev Med. (2007)
Tsukinoki, R., Morimoto, K., Nakayama, K.: Association between lifestyle factors and plasma adiponectin levels in Japanese men. Lipids Health Dis. 4, 27 (2005)
Vozarova, B., Weyer, C., Hanson, K., Tataranni, P.A., Bogardus, C., Pratley, R.E.: Circulating interleukin-6 in relation to adiposity, insulin action, and insulin secretion. Obes. Res. 9, 414–417 (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Saito, K., Mutoh, N., Ikeda, T., Goda, T., Mochizuki, K. (2008). Improving Search Efficiency of Incremental Variable Selection by Using Second-Order Optimal Criterion. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2008. Lecture Notes in Computer Science(), vol 5179. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85567-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-85567-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85566-8
Online ISBN: 978-3-540-85567-5
eBook Packages: Computer ScienceComputer Science (R0)