Advertisement

Empirical Software Engineering

, Volume 23, Issue 2, pp 771–799 | Cite as

An empirical study of early access games on the Steam platform

  • Dayi Lin
  • Cor-Paul Bezemer
  • Ahmed E. Hassan
Article

Abstract

“Early access” is a release strategy for software that allows consumers to purchase an unfinished version of the software. In turn, consumers can influence the software development process by giving developers early feedback. This early access model has become increasingly popular through digital distribution platforms, such as Steam which is the most popular distribution platform for games. The plethora of options offered by Steam to communicate between developers and game players contribute to the popularity of the early access model. The model is considered a success by the game development community as several games using this approach have gained a large user base (i.e., owners) and high sales. On the other hand, the benefits of the early access model have been questioned as well. In this paper, we conduct an empirical study on 1,182 Early Access Games (EAGs) on the Steam platform to understand the characteristics, advantages and limitations of the early access model. We find that 15% of the games on Steam make use of the early access model, with the most popular EAG having as many as 29 million owners. 88% of the EAGs are classified by their developers as so-called “indie” games, indicating that most EAGs are developed by individual developers or small studios. We study the interaction between players and developers of EAGs and the Steam platform. We observe that on the one hand, developers update their games more frequently in the early access stage. On the other hand, the percentage of players that review a game during its early access stage is lower than the percentage of players that review the game after it leaves the early access stage. However, the average rating of the reviews is much higher during the early access stage, suggesting that players are more tolerant of imperfections in the early access stage. The positive review rate does not correlate with the length or the game update frequency of the early access stage. Based on our findings, we suggest game developers to use the early access model as a method for eliciting early feedback and more positive reviews to attract additional new players. In addition, our findings suggest that developers can determine their release schedule without worrying about the length of the early access stage and the game update frequency during the early access stage.

Keywords

Early access games Computer games Steam 

Notes

Acknowledgements

We are grateful to Sergey Galyonkin, the owner of Steam Spy, who generously gave us access to all the historical data of Steam collected by Steam Spy for this research. We also thank professor Jennifer R. Whitson from the University of Waterloo and Mr. Patrick Walker from EEDAR for the inspiring discussions about the gaming industry.

References

  1. Al-Ani B, Trainer E, Ripley R, Sarma A, Van Der Hoek A, Redmiles D (2008) Continuous coordination within the context of cooperative and human aspects of software engineering. In: Proceedings of the 2008 international workshop on cooperative and human aspects of software engineering. ACM, pp 1–4Google Scholar
  2. Allen P (2016) Is Steam’s Early Access a ticking time bomb? http://www.gamasutra.com/blogs/paulallen/20160211/265598/is_steams_early_access_a_ticking_time_bomb.php, (last visited: Apr 18, 2017)
  3. Bierzerkers (2016) DEVBLOG 5. http://archive.is/XHOjf, (last visited: Apr 18, 2017)
  4. Cards C (2015) Free to play coming soon! http://store.steampowered.com/news/?appids=360730&headlines=1 (last visited: Apr 18, 2017)
  5. Chambers C, Feng Wc, Sahu S, Saha D (2005) Measurement-based characterization of a collection of on-line games. In: Proceedings of the 5th ACM SIGCOMM conference on internet measurement (IMC), USENIX Association, pp 1–14Google Scholar
  6. Chambers JM, Hastie TJ (1991) Statistical models in S. CRC Press, IncGoogle Scholar
  7. Clifford C (2014) Crowdfunding Generates More Than $60,000 an Hour (Infographic). http://www.entrepreneur.com/article/234051?newsletter=true, (last visited: Apr 18, 2017)
  8. CPx (2015) Pool nation FX - lite NOW AVAILABLE. http://store.steampowered.com/news/?appids=314000&headlines=1 , (last visited: Apr 18, 2017)
  9. Damodaran L (1996) User involvement in the systems design process-a practical guide for users. Behav Inform Technol 15(6):363–377CrossRefGoogle Scholar
  10. Dictionary O (2016) Oxford dictionary definition of Crowdfunding. http://www.oxforddictionaries.com/us/definition/american_english/crowdfunding, (last visited: Apr 18, 2017)
  11. EuroGamer (2014) DayZ week-one sales rocket past 400,000. http://www.eurogamer.net/articles/2014-01-02-dayz-week-one-sales-rocket-past-400-000, (last visited: Apr 18, 2017)
  12. Fine D (2016) Double Fine - What is Double Fine? http://www.doublefine.com/about/, (last visited: Apr 18, 2017)
  13. Foley MJ (2015) Microsoft launches new Office Insider test program. http://www.zdnet.com/article/microsoft-launches-new-office-insider-test-program/, (last visited: Apr 18, 2017)
  14. Gallivan MJ, Keil M (2003) The user–developer communication process: a critical case study. Inf Syst J 13(1):37–68CrossRefGoogle Scholar
  15. Galyonkin S (2016) SteamSpy - All the data and stats about Steam games. http://steamspy.com/, (last visited: Apr 18, 2017)
  16. Greg D (2014) Double fine- action forums. https://archive.is/vfdqw, (last visited: Apr 18, 2017)
  17. Jacobs M, Sihvonen T (2011) In perpetual beta? On the participatory design of facebook games. In: Think design play: the fifth international conference of the digital research association (DIGRA)Google Scholar
  18. Johnson J (1999) Turning chaos into success. Softw Mag 19(3):30–34Google Scholar
  19. Johnson J (2000) Chaos in the new millennium: The ghost of christmas future: the standish group. West Yarmouth, MAGoogle Scholar
  20. Kujala S (2003) User involvement: a review of the benefits and challenges. Behav Inform Technol 22(1):1–16CrossRefGoogle Scholar
  21. LadyAijou (2014) About the early access - edge of space general discussions. http://steamcommunity.com/app/238240/discussions/0/45350790838400003/#c45350790880063101, (last visited: Apr 18, 2017)
  22. LadyAijou (2015) ETA On getting out of early access? - edge of space general discussions. http://steamcommunity.com/app/238240/discussions/0/41973820433803052/#c594820656453333967, (last visited: Apr 18, 2017)
  23. LeBreton J (2014) JP Lebreton’s twitter. https://twitter.com/vectorpoem/status/536933544472240128, (last visited: Apr 18, 2017)
  24. Lin D, Bezemer CP, Hassan AE (2016) Studying the urgent updates of popular games on the steam platform. Empirical Software EngineeringGoogle Scholar
  25. Logfeller (2014) Release date? - Underrail general discussions. http://steamcommunity.com/app/250520/discussions/0/35222218811239879/#c626329186828487604, (last visited: Apr 18, 2017)
  26. Long JD, Feng D, Cliff N (2003) Ordinal analysis of behavioral data. WileyGoogle Scholar
  27. Löwgren J, Stolterman E (2004) Thoughtful interaction design: a design perspective on information technology. MIT PressGoogle Scholar
  28. Maalej W, Happel HJ, Rashid A (2009) When users become collaborators: towards continuous and context-aware user input. In: Proceedings of the 24th ACM SIGPLAN conference companion on object oriented programming systems languages and applications. ACM, pp 981–990Google Scholar
  29. Manes S (1984) Taking a gamble with word vision. PC Mag, pp 211Google Scholar
  30. Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2016) A survey of app store analysis for software engineering. RN 16:02Google Scholar
  31. McDougall J (2010) Community heroes: notch, for minecraft. http://www.pcgamer.com/community-heroes-notch-for-minecraft/ http://www.pcgamer.com/community-heroes-notch-for-minecraft/ (last visited: Apr 18, 2017)
  32. Medierra (2015) When is this game coming out? - Grim dawn general discussions. http://steamcommunity.com/app/219990/discussions/0/530649887209076036/#c530649887215003016, (last visited: Apr 18, 2017)
  33. Minecraft (2016) Minecraft official website. https://minecraft.net/, (last visited: Apr 18, 2017)
  34. O’reilly T (2007) What is web 2.0: design patterns and business models for the next generation of software. Commun Strateg (1):17Google Scholar
  35. Orland K (2011) Minecraft draws over $33 Million in revenue from 1.8m paying customers. http://www.gamasutra.com/view/news/33961/minecraft_draws_over_33_million_in_revenue_from_18m_paying_customers.php, (last visited: Apr 18, 2017)
  36. Orland K (2014) Introducing steam gauge: ars reveals steam’s most popular games. http://arstechnica.com/gaming/2014/04/introducing-steam-gauge-ars-reveals-steams-most-popular-games/ (last visited: Apr 18, 2017)
  37. Pavel Djundik MB (2016) Steam DB - steam database. https://steamdb.info/, (last visited: Apr 18, 2017)
  38. PCGamer (2014) Tim schafer explains spacebase DF-9’s unexpected v1.0 release. http://www.pcgamer.com/tim-schafer-explains-spacebase-df-9s-v1-0-release/, (last visited: Apr 18, 2017)
  39. Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the NSSE and other surveys: are the t-test and Cohen’s d indices the most appropriate choices. In: Annual meeting of the Southern association for institutional researchGoogle Scholar
  40. Stern C (2012) What makes a game indie: a universal definition. http://sinisterdesign.net/what-makes-a-game-indie-a-universal-definition/, (last visited: Apr 18, 2017)
  41. Takkunen P (2015) Moving out of early access. http://store.steampowered.com/news/?appids=316080&headlines=1, (last visited: Apr 18, 2017)
  42. Taylor TL (2006) Beyond management: considering participatory design and governance in player culture. First mondayGoogle Scholar
  43. ThunderPeel2001 (2015) The facts about spacebase DF-9. https://steamcommunity.com/app/246090/discussions/0/616198900637753626/, (last visited: Apr 18, 2017)
  44. TimOfLegend (2014) Tim Schafer answers common questions about v1.0. https://steamcommunity.com/app/246090/discussions/0/613936673464943075/, (last visited: Apr 18, 2017)
  45. Ullrich C, Borau K, Luo H, Tan X, Shen L, Shen R (2008) Why web 2.0 is good for learning and for research: principles and prototypes. In: Proceedings of the 17th international conference on world wide web. ACM, pp 705–714Google Scholar
  46. Valve (2016a) Introducing early access. http://store.steampowered.com/earlyaccessfaq/, (last visited: Apr 18, 2017)
  47. Valve (2016b) Spacebase DF-9 on steam. http://store.steampowered.com/app/246090/, (last visited: Apr 18, 2017)
  48. Waclawek T (2015) Ronin, the turn-based action-platformer, is now on steam. http://store.steampowered.com/news/?appids=274230&headlines=1, (last visited: Apr 18, 2017)
  49. Walker P (2014) Early Access popularity growing, but only 25% have released as a full game. http://www.gamesindustry.biz/articles/2014-11-13-early-access-popularity-growing-but-only-25-percent-have-released-as-a-full-game, (last visited: Apr 18, 2017)
  50. Washburn Jr M., Sathiyanarayanan P, Nagappan M, Zimmermann T, Bird C (2016) What went right and what went wrong: an analysis of 155 postmortems from game development. In: Proceedings of the 38th international conference on software engineering companion. ACM, New York, ICSE ’16, pp 280–289Google Scholar
  51. Welch C (2013) Steam Early Access lets gamers buy and play titles still in development. http://www.theverge.com/2013/3/20/4128644/steam-early-access-buy-and-play-games-still-in-development, (last visited: Apr 18, 2017)
  52. Yin-Poole W (2014) Valve tightens Steam Early Access rules for developers. http://www.eurogamer.net/articles/2014-11-21-valve-tightens-steam-early-access-rules-for-developers, (last visited: Apr 18, 2017)

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Queen’s UniversityKingstonCanada

Personalised recommendations