Guidelines for samplers: measuring a change in behaviour from before and after surveys
This paper addresses the issue of using before and after surveys to evaluate behavioural changes in response to transport policies and investments, a procedure that, we argue is done far too rarely in this profession. Further, it demonstrates very clearly that, in almost all conceivable cases, there are considerable economies to be obtained by using a panel (again, under-utilised in our profession) to undertake evaluation, rather than successive independent cross-sectional surveys. The paper also addresses the critical issue of sample size requirements for measuring changes of a relatively small magnitude in travel behaviour; i.e., to state, with 95% confidence, that if there is a ∂ percent change in behaviour for the sample, there is a ∂ percent ± e percent change in the behaviour of the population, where e is the sampling error. In this paper, we present a method for calculating such sample size requirements from first principles and demonstrate the applicability both hypothetically and then empirically using data from the Puget Sound Transportation Panel. The formulation enables designers of before and after surveys to investigate the trade-offs between the statistical accuracy of their predictions and the sample size requirements systematically, without the need to specify ∂ a priori. This latter point is crucial, we argue, because we have limited information on ∂, yet, as we explain here, it drives the sample size requirements using alternative, well-cited approaches for calculating sample sizes to assess behavioural change. The results have important ramifications both for those implementing transport policies intended to produce behavioural change, especially when a cost-benefit evaluation of the policy is desired, and for those interpreting the results reported in previous studies.
KeywordsSample size Behavioural change Panel surveys Sampling error
Unable to display preview. Download preview PDF.
- Cohen, J.: Statistical Power for the Behavioural Sciences, 2nd edn. Lawrence Erlbaum, Associates, Hillsdale NJ (1988)Google Scholar
- Dowling, R., Ireson, R., Skabardonis, A., Gillen, D., Stopher, P.: Predicting Air Quality Effects of Traffic Flow Improvements, NCHRP Report 535, Transportation Research Board, Washington, DC (2005)Google Scholar
- Golob, J.M., Golob, T.F.: Studying road pricing policy with panel data analysis: The San Diego I-15 HOT lanes. In: Hensher D.A. (ed.) Travel Behaviour Research: The Leading Edge, pp. 869–883 (2001)Google Scholar
- Kish, L.: Survey Sampling. John Wiley and Sons (1965)Google Scholar
- Richardson, A.J., Seethaler, R.K., Harbutt, P.L.: Design Issues for Before and After Surveys of Travel Behaviour Change, Proceedings of the 26th Australasian Transport Research Forum, Wellington, New Zealand (2003)Google Scholar
- Richardson, A.J., Ampt, E.S., Meyburg, A.H.: Survey Methods for Transport Planning. Eucalyptus Press (1995)Google Scholar
- Steffey, D.J. Supernak, Kaschade, C.: San Diego’s I-15 value pricing project: Impact on local businesses, Transportation Research Board 2003 Annual Meeting CD-ROM, 21 pages (2003)Google Scholar
- Stopher, P.R.: Experimental design. In: The effects of Added Transportation Capacity, Travel Model Improvement Program, USDOT, pp. 47–53 (1991)Google Scholar
- Stopher, P.R.: Voluntary travel behaviour change. In: Hensher, D., Button, K.(eds.) Handbook of Transport Strategy, Policy and Institutions, pp. 561–579. Elsevier (Handbook No. 6) (2004)Google Scholar
- Stopher, P.R., Greaves, S.P., Xu, M., FitzGerald, C.M., Lauer, N., Perkins A.: A Panel Approach to Evaluating Voluntary Travel Behaviour Change Programs—South Australia Pilot Survey. CD-ROM Proceedings of the 85th Annual Meeting of the Transportation Research Board, Washington DC, January 2006 (2006)Google Scholar
- Walpole, R.E., Myers, R.H.: Probability and Statistics for Engineers and Scientists, 2nd edn. MacMillan Publishing Co., New York (1978)Google Scholar
- Zumkeller, D., Madre, J.-L., Chlond, B., Armoogum J.: Panel surveys. In: Stopher, P.R., Stecher, C.C. (eds.), Travel Survey Methods—Quality and Future Directions, pp. 363–398. Elsevier, Oxford (2006)Google Scholar