Keywords

1 Introduction

The video game industry is continuously growing with revenue over $90 billion worldwide [1, 2]. Thousands of people worldwide work on creating video games for entertainment, as well as for education, business and art [3]. The video game market is very competitive, with an increasing demand for sophisticated and more realistic games. Such requirements are barrier for small game studios and indie developers, as they need more resources, in terms of cost, time, expertise, and technology, to keep up with the competition. Therefore, small game studios and indie developers struggle to survive, and this has a direct negative consequence on the video game industry and the economy, as these developers bring in creativity and innovation.

One way to overcome this, is to introduce novel cost-effective technologies which are primarily based on image [4,5,6,7] and video [8,9,10,11] reconstruction techniques. The users of such techniques can capture real-life scenes and objects with conventional apparatus (e.g., smartphones, photo cameras). The captured images/videos are then used to build the 3D realistic models of assets. Image-based reconstruction (IBR) seems to produce better results [12].

Motivation and Research Question.

Although it is known that IBR and photogrammetric methods provide a short-cut for creating 3D assets for video-games development [4,5,6, 13,14,15], to the authors’ knowledge, there is no thorough ecological study involving different stakeholders (game developers and game players) and investigating the interplay of a state-of-the-art IBR game development pipeline on game developers’ efficiency, game performance, and gaming experience. In the context of the reported study a fully integrated research-based state-of-the-art pipeline for games development bootstrapped to game developers’ requirements [16] was used.

Hence, the research questions investigated in this paper are: (a) to explore the effects of games created with IBR on game developers’ efficiency, in terms of time required to create video-games by also considering game performance metrics (game size, loading time, and scalability), and (b) to explore the effects of games created with IBR on gaming experience, in terms of gamers’ preference and graphics quality.

2 Method of Study

2.1 Procedure

We conducted two user studies: (a) a comparative evaluation study during which video game prototypes were developed with the traditional vs. IBR methods aiming to gather quantitative and qualitative feedback related to efficiency and performance measures, and (b) an eye-tracking A/B testing study with gamers playing the game prototypes created during the comparative evaluation study.

Comparative Study of Creating Game Assets with and Without IBR.

For this study, two professional game developers participated and developed two different games over a period of six months (May to October 2016). The participants developed their games within their own working conditions (i.e., ecologically valid conditions), without the intervention or help of our research team. They were asked to keep track of their activities and the time needed to perform the necessary tasks, adopting an activity diary-log approach. We provided them with a diary template. Moreover, several semi-structured interviews were conducted during and after the game development phase.

Comparative Study of Playing Games Created with and Without IBR.

For this in-lab study twelve gamers were recruited (5 females, 7 males), aged 20–32, and played different versions of the games created with the traditional and the IBR approach. They were recruited through email invitations, and they were undergraduate and postgraduate university students. All individuals were experienced gamers, playing video games more than twelve hours per week, and they had never played any of the games used in the study before. After the recruitment of the participants, twelve in-lab sessions were scheduled (i.e., one for each participant). In each session, the participant was asked to play the four game versions, wearing the eye-tracking apparatus. To avoid bias effects, six gamers played first the one game, and the other six played the other game first. In both groups, three participants played the traditional version first, and the rest three played the IBR version first. After the play-session, each participant was asked to fill a user experience questionnaire. Finally, each participant was asked to provide feedback and comments on their gaming experience through an unstructured interview. The participants were not informed that the two versions were developed using different technologies until the end of the interview.

2.2 Instruments and Apparatus

IBR-Based Game Development Integrated Pipeline.

The IBR pipeline used in our study was the CR-PLAY environment [17,18,19,20,21]. CR-PLAY is a state-of-the-art mixed pipeline for creating game assets and integrating them in the game development workflow, using IBR techniques [18, 22]. CR-PLAY users follow a three-step approach to build a game asset: (a) capture real-life objects (e.g., a building) by taking a sequence of photos from multiple angles; (b) reconstruct them as 3D point cloud models; (c) and import them into their standard game development workflow.

The Games.

Two video-games were created and played in our studies: Survive the Weekend and Basketball Stars. Survive the Weekend is a hidden object game in which the player is asked to find a number of items in a play-room within a given time. Basketball Stars is a first-person sport shooting game, and the players’ goal is to shoot as many baskets as possible in a predefined amount of time against an opponent. The two games have a different intersection between the game assets and the game play in terms of visual search and visual attention, as Survive the Weekend requires constant visual attention of players through the whole scene, because they are searching for the hidden items; while Basketball Stars requires players’ attention on specific game assets.

Apparatus.

Game developers used their own computer systems to develop the games. Gamers played the games on a desktop computer (with Intel Core i7-4500U at 2.40 GHz, and 8 GB RAM). The screen used was the LG 22MP48D, at a screen resolution of 1920 × 1080 pixels. To investigate the participants’ visual behavior, Tobii Pro Glasses 2 were used to record their eye movement patterns.

Gaming Experience Questionnaire.

To measure gaming experience, we used the Immersive Experience Questionnaire (IEQ) [23]. IEQ is a credible and highly validated tool to evaluate players’ experience [24, 25]. It consists of 31 questions of Likert type, including questions about graphics quality.

3 Results

3.1 Effects of IBR on Game Developers Efficiency in Creating Video-Game Assets

The development of a game prototype with IBR took approximately 12 man-hours, regardless of the complexity of the scene. However, the time needed to develop a game prototype with the traditional approach was highly dependent on the number of assets to be included in the game, their complexity, and whether they have been used in other projects or were available from third-party providers. In our case study, the game developers were free to re-use any game asset and/or use third-party assets, as they would normally do, strengthening the ecological validity of the study. To create a game based on a simple scene with few items (N = 3), approximately 26 man-hours were required, while for a game based on a complex scene with many items (N = 19), 160 man-hours were required. In both cases, the developers required less time to create the IBR version. The interviews revealed that IBR reversed the process the game developers followed to develop a game, as the IBR game scenario was based on the reconstructed scene, while traditionally they created the scenes based on the scenario.

3.2 Effects of IBR on Game Performance (Size, Loading-Time, and Scalability)

Game performance is a multidimensional variable, which can be measured on game file-size, loading-time, and scalability. In terms of file-size, the executable was bigger for the IBR versions (300 MB for a simple scene, and 330 MB for a complex scene) compared to the traditional versions (15 MB for a simple scene, 20 MB for a complex scene). The loading times were in-line with the file-size findings, and they were increased for the IBR versions (10 s for a simple scene, 20 s for a complex scene) compared to the traditional versions (2 s for both a simple and a complex scene). The aforementioned results reflect on the scene complexity and the size of point cloud models, introduced in IBR. Therefore, this has an impact on the scalability of a game, as developers could not build many levels.

3.3 Effects of IBR on Gaming Experience

Effects of IBR on Game Players’ Preference.

The game players were asked to state which version they preferred: IBR or traditional. Ten participants stated that they preferred the traditional version of Survive the Weekend, a significant difference according to binomial statistical test (p = .019). In contrary, eleven participants stated that they preferred the IBR version of Basketball Stars, a significant difference according to the binomial statistical test (p = .003). When playing the IBR version of Survive the Weekend, the participants had to explore the whole scene to find and collect the hidden items, and they stated that most of the game elements were blurry when they were navigating through the scene. They also stated that many objects were not naturally placed and lighted in the scene. On the other hand, the Basketball Stars scene was more static, as the players could not navigate through the scene. Therefore, no blurring objects were noticed in the IBR version, and the participants found it to be more photorealistic, with smooth transitions and nice interaction.

Graphics Quality.

To measure how much the players valued the graphics quality, we conducted an independent-samples t-test for each game, with the technology factor (traditional and IBR) as the independent variable, and the graphics quality score (calculated using IEQ) as the dependent variable. In both tests, the assumptions were met. Regarding the Survive the Weekend game, the participants preferred the graphics of the traditional version (5.11 ± 1.48 vs. 4.01 ± 1.42, p = .046). Regarding the Basketball Stars game, the participants preferred the graphics of the IBR version (5.09 ± 1.34 vs. 4.32 ± 1.17, p = .042). This was also reflected on the overall immersion score, as the participants were more immersed when playing the traditional version of Survive the Weekend (4.72 ± 0.66 vs. 4.64 ± 0.57, p = .061), and the IBR version of Basketball Stars (5.19 ± 0.51 vs. 4.84 ± 0.59, p = .019).

Eye-Tracking Analysis.

The eye-tracking analysis verified the aforementioned finding, as the participants fixated many times and for long time periods on the whole backdrop scene, visually scanning for the hidden objects, in both versions of Survive the Weekend. On the other hand, they fixated intensively in specific areas of interest (i.e., basket, shooter, power bar) when playing Basketball Stars, as their goals was to shoot and score. The heat-maps of both versions of both games are depicted in Fig. 4.

4 Findings and Discussion

The findings of our studies are summarized into three categories: (a) efficient creation of game assets with IBR, (b) the bottleneck in performance when using IBR, and (c) effective use of IBR depends on game type (Table 1).

Table 1. The main findings of our studies on developers‘efficiency, gaming performance and gaming experience
Full size table

Efficient Creation of Game Assets with IBR.

The IBR technology provided a faster method to create photorealistic game assets in comparison to the traditional approach (Fig. 1). The main benefit for the game developers is that IBR time to delivery is independent of the assets complexity, as our study indicated that the asset creation times were similar for creating games of varying game elements complexity.

Fig. 1.
figure 1

Game developers created the IBR game versions in less time than the traditional game versions, in both complexity levels.

Full size image

The Bottleneck in Performance when Using IBR.

In the presented study, the game performance reflects on the executable file-size, the loading time, and the scalability of the game. In both games, the IBR versions were bigger in size, had longer loading periods, and were less scalable than the traditional ones (Fig. 2). Despite the fact that some of the aforementioned issues are technology-related and can be resolved (e.g., reduce file-size by incorporating sophisticated image compression algorithms), the IBR game performance will remain poor since game assets are not based on polygons. Game developers can create gamers quickly when they use IBR, but the game performance (in terms of file-size, loading time, and scalability) is a bottleneck.

Fig. 2.
figure 2

IBR games were bigger in size than the traditional versions for both simple and complex scenes (left). IBR games had also longer loading times than the traditional versions for both simple and complex scenes (right).

Full size image

Game Genre Affects Preference of IBR Video-Games.

Game players preferred the graphics quality of traditional Survive the Weekend (i.e., a game with heavy interaction with assets throughout the scene) than the graphics of the IBR version; and the graphics of IBR Basketball Stars (i.e., a game with light interaction with the scene), than the graphics of the traditional version. This was also reflected on their answers when they were asked explicitly which of the two versions (traditional or IBR) they preferred. Games that require players to interact with the largest part of the scene or with most of the game assets (e.g., hidden objects games) require high visual attention. Therefore, as users visually explore the game environment, they can detect IBR-related flaws (e.g., bad lightning, non-normal position of an object, blurred assets), which influence their gaming experience (Fig. 3). On the other hand, when the players are engaged with a video game with more static scenes (e.g., point-and-click games), visually focusing on specific areas of the scene, they typically do not detect IBR-related flaws, and they enjoy the game more, as it is more realistic and aesthetically pleasant (Fig. 3).

Fig. 3.
figure 3

Gamers liked more the graphics of the traditional version of Survive the Weekend, and the graphics of the IBR version of Basketball Stars (left); Study participants preferred the traditional version of the visual-search games (e.g., Survive the Weekend), and the IBR version of the point-and-shoot games (e.g., Basketball Stars).

Full size image

Our eye-tracking analysis reflects on this finding (Fig. 4), as the players tend to fixate on various spots of the visual scene evenly, when playing a visual search game. On the other hand, their fixations are more intense and focused on specific areas of interest (e.g., the game assets they interact with) when playing games which use the background scenes to provide a more photorealistic perception, but they are not part of the game-play or the players are not required to interact with them. Overall, the effect of IBR on gaming experience depends on the game type and the degree of the visual search activity (e.g., traditional methods will be preferred for games with heavy visual search, while IBR will be preferred for games with lighter visual interaction).

Fig. 4.
figure 4

The participants visually interacted with the whole scene when playing Survive the Weekend; but, they visually focused on specific areas of interest when playing Basketball Stars.

Full size image

4.1 Limitations and Future Work

The main limitation of our work is the relatively small sample sizes used in our studies. However, the statistical tests performed met all the required assumptions, providing credible results. Another limitation is the use of a research, but state-of-the-art IBR game development pipeline (CR-PLAY). As a future work, we will engage more end-users to evaluate the quality of the produced game elements and measure the gaming experience on games of varying types and characteristics.

5 Conclusion

In this paper, we investigated, through two evaluation studies, the impact of image-based rendering and reconstruction (IBR) on game developers efficiency, games performance and game players’ experience. In the first study, we engaged two professional game developers who created, during a six-month period and within ecologically valid conditions, two versions (traditional and IBR) of two different games. In the second study, twelve experienced gamers played the four game versions in the context of an in-lab, A/B testing, eye-tracking approach. The analysis of the results revealed benefits of the IBR approach, such as reduced time and costs for creating photorealistic game assets. Drawbacks were also revealed, such as increased loading times, huge file sizes of deployed games and scalability issues which can be credited to the very nature of the IBR technology. Regarding, the players’ gaming experience, the analysis of the results revealed mixed outcome regarding players’ preference of games created with the traditional vs. IBR approach which should be further investigated.