Abstract
Does advertising measurably affect sales? New technologies for tracking individuals’ sales and ad exposure facilitate studying a nationwide retailer’s online brand advertising effectiveness. A controlled experiment on 1,577,256 existing customers measures the causal effect of advertising on purchases, overcoming the attribution problem by exogenously varying ad exposure. The advertising produced a statistically and economically significant effect on in-store sales. The experiment permits a demographic breakdown. Surprisingly, the effects are large for the elderly. Customers aged 65+, comprising only 5 % of customers, increased purchases by 20 % due to the advertising. This represented 40 % of the total effect among all ages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The self-reported age and gender data is not perfectly accurate. We know some users lie about their demographics while filling out online forms, but Fig. 2 does not raise any obvious red flags about deceptive reporting. In any case, inaccurately reported age data would likely make it harder to find differences among differently aged consumers.
We were not able to exclude these potentially invalid observations from our dataset, but they appear to represent a very small part of the sample. To the extent that we include treatment-group members who could not actually have perceived the ads, we will have a slight bias towards measuring zero effect of the advertising.
While a 10 % one-sided test is a much lower standard for statistical significance than is usual in the economics literature, this is a more stringent standard than the threshold used for success in the BehaviorScan experiments of Lodish et al. (1995a), who used a standard of 20 % significance, one-sided. Because of the high variance in sales, we have relatively imprecise estimates of the effects of advertising even with sample sizes in the hundreds of thousands. For more detailed power calculations concerning this experiment, see Lewis and Reiley (2013).
References
Abraham, M. (2008). The off-line impact of online ads. Harvard Business Review, 86(4), 28.
Abraham, M., & Lodish, L. M. (1990). Getting the most out of advertising and promotion. Harvard Business Review, 68(3), 50–60.
Ackoff, R. L., & Emshoff, J. R. (1975a). Advertising research at Anheuser-Busch, Inc. (1963–1968). Sloan Management Review, 16(2), 1–15.
Ackoff, R. L., & Emshoff, J. R. (1975b). A reply to the comments of Yvan Allaire. Sloan Management Review, 16(3), 95–98.
Allaire, Y. (1975). A comment on ‘advertising research at Anheuser-Busch, Inc. (1963–1968). Sloan Management Review, 16(3), 91–94.
Bagwell, K. (2008). The economic analysis of advertising. In M. Armstrong & R. Porter (Eds.), Handbook of industrial organization (Vol. 3, pp. 1701–1844). Amsterdam: Elsevier B.V.
Bergemann, D., & Bonatti, A. (2011). Targeting in advertising markets: Implications for online versus offline media. RAND Journal of Economics, 42(3), 417–443.
Berndt, E. R. (1991). The practice of econometrics: Classic and contemporary. Chapter 8 Reading. Addison-Wesley, Massachusetts.
Dorfman, R., & Steiner, P. O. (1954). Optimal advertising and product quality. American Economic Review, 44(5), 826–836.
Eastlack, J. O., & Rao, A. G. (1989). Advertising experiments at the Campbell Soup Company. Marketing Science, 8(1), 57–71.
Hu, Y., Lodish, L. M., & Krieger, A. M. (2007). An analysis of real world TV advertising tests: A 15-year update. Journal of Advertising Research, 47(3), 341–353.
Law, S., Hawkins, S. A., & Craik, F. I. M. (1998). Repetition-induced belief in the elderly: Rehabilitating age-related memory deficits. Journal of Consumer Research, 25, 91–107.
Levitt, S. & List, J. A. (2008). Field experiments in economics: The past, the present, and the future. NBER Working Paper 14356.
Lewis, R., & Reiley, D. (2013). Online advertising and offline sales: measuring the effects of retail advertising via a controlled experiment on Yahoo! Working paper, Yahoo! Research.
Lodish, L. M., Abraham, M., Kalmenson, S., Livelsberger, J., Lubetkin, B., Richardson, B., et al. (1995a). How T.V. advertising works: A meta-analysis of 389 real world split cable T.V. advertising experiments. Journal of Marketing Research, 32, 125–139.
Lodish, L. M., Abraham, M., Kalmenson, S., Livelsberger, J., Lubetkin, B., Richardson, B., et al. (1995b). A summary of fifty-five in-market experiments of the long-term effect of TV advertising. Marketing Science, 14(3), G133–140.
Richard, S. (1972). The economics of advertising. Amsterdam: North-Holland.
Stephens, N. (1982). The effectiveness of time-compressed television advertisements with older adults. Journal of Advertising, 11, 48–55.
Acknowledgments
We thank Meredith Gordon, Sergei Matusevych, and especially Taylor Schreiner for work on the experiment and data. Yahoo! Incorporated provided financial and data assistance and guaranteed academic independence prior to our analysis so that the results could be published no matter how they turned out. We acknowledge the helpful comments of Avi Goldfarb, Glenn Ellison, Jerry Hausman, Stephen Ryan, Duncan Watts, and two anonymous referees.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Lewis, R.A., Reiley, D.H. Advertising Effectively Influences Older Users: How Field Experiments Can Improve Measurement and Targeting. Rev Ind Organ 44, 147–159 (2014). https://doi.org/10.1007/s11151-013-9403-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11151-013-9403-y