Analysis of the Impact of Sample Size, Attribute Variance and Within-Sample Choice Distribution on the Estimation Accuracy of Multinomial Logit Models Using Simulated Data
- 30 Downloads
Literature review indicates that sample size, attribute variance and within-sample choice distribution of alternatives are important considerations in the estimation of multinomial logit (MNL) models, but their impacts on the estimation accuracy have not been systematically studied. Therefore, the objective of this paper is to provide an empirical examination to the above issues through a set of simulated discrete choice preference and rank ordered preference datasets. In this paper, the utility coefficients, alternative specific constants (ASCs), and the mean and standard deviation of the four attributes for a set of seven hypothetical alternatives are specified as a priori. Then, synthetic datasets, with varying sample size, attribute variance and within-sample choice distribution are simulated. Based on these datasets, the utility coefficients and ASCs of the specified MNLs are re-estimated and compared with the original values specified as the priori. It is found that (1) the estimation accuracy of utility parameters increases as the sample size increases; (2) the utility coefficients can be re-estimated with reasonable accuracy, but the estimates of the ASCs are confronted with much larger errors; (3) as the variances of the alternative attributes increase, the estimation accuracy improves significantly; and (4) as the distribution of chosen choices becomes more balanced across alternatives within sample datasets, the hit-ratio decreases. The results indicate that (a) under a similar setting presented in this paper, a large sample consisting of a few thousand observations (3000–4000) may be needed in order to provide reasonable estimates for utility coefficients, particularly for ASCs; (b) a larger, but realistic attribute space is preferred in the stated preference survey design; and (c) choice datasets with unbalanced “chosen” choice frequency distribution is preferred, in order to better capture the elasticity between the “perceived utility” associated with alternative’s attributes.
KeywordsSample size attribute variance within-sample choice distribution simulated data
Unable to display preview. Download preview PDF.
The authors appreciate the anonymous referees and the editor for their help to improve the quality of the paper. The funding from Hubei Provincial Natural Science Foundation (2015CFB599) and the funding for Top 1% ESI Academic Program from Wuhan University of Technology supported by “the Fundamental Research Funds for the Central Universities” (WUT:2014-VII-036) is appreciated. This study is also supported by the Natural Science and Engineering Research Council (NSERC), Canada and a start-up grant from Wuhan University of Technology. This paper is also partially supported by a grant from the National Natural Science Foundation of China (NSFC No.51778510).
- Bliemer, M.C.J. & Rose, J.M. (2008) Construction of experimental designs for mixed logit models allowing for correlation across choice observations. The 87th Annual TRB Meeting, Washington DC, January 2008, USA.Google Scholar
- Brundell-Freij, K. (1997). How good is an estimated logit model? estimation accuracy analyzed by Monte Carlo simulations. Paper presented at the proceedings of seminar F held at European Transport Forum, Brunel University, England, 1-5 September 1997.Google Scholar
- Cramer, J.S. (1999). Predictive performance of the binary logit model in unbalanced samples. Journal of the Royal Statistical Society: Series D (The Statistician), 48 (1): 85–94.Google Scholar
- Guan, H.Z. (2004). Disaggregated Model-Analysis Tools for Traffic Behavior (in Chinese). China Communications Press, Beijing.Google Scholar
- Hunt, J.D., Zhong, M. & Abraham, J.E. (2007). Examining the accuracy of logit modeling with simulated RP and SP data. Presented at the 2007 World Conference of Transportation Research Conference, Berkeley.Google Scholar
- Koppelman, F.S. & Chu C. (1983). Effect of sample size on disaggregate choice model estimation and prediction. Transportation Research Record: Journal of the Transportation Research Board, 944: 60–69.Google Scholar
- Liang, Y.J. & Yuan, Z.Z. (2014). A logit model for selection of passenger facilities at integrated transport hubs. Journal of Trans-port information and safety, 32 (4):36–40.Google Scholar
- McFadden, D. (1978). Modeling the choice of residential location. Transportation Research Record: Journal of the Transportation Research Board, 673: 72–77.Google Scholar
- Zhang, Y.L., Liang, F.M. & Xie, Y.C. (2007). Crash injury severity analysis using a Bayesian ordered probit model. Presented at 86th Annual Meeting of the Transportation Research Board (No. 07-2335), Washington, D.C..Google Scholar
- Zhong, M., & Hunt, J.D. (2006). Sensitivity analysis of logit formulation and estimation. Presented at the 2006 International Conference on Traffic and Transportation Studies, Xi’an, China.Google Scholar
- Zhou, X., Liu, M., Zhang, D. & Ran, B. (2014). Transfer mode choice of comprehensive passenger transportation terminal based on mixed logit in china. Presented at 93rd Annual Meeting of the Transportation Research Board (No. 14-3968), Washington, D.C..Google Scholar