WebTics: A Web Based Telemetry and Metrics System for Small and Medium Games

Abstract

In the 1980s game developers were able to pride themselves on developing every part of a game, sometimes even to the hardware being used. However, with the increasing size of game development teams, and the increasing demands on smaller game titles, the use of middleware and other third-party tools are becoming increasingly important to the industry. While there are still game developers who are reluctant to employ code not developed in-house, this position is getting harder to justify with the changes in the marketplace and development strategies. An example of this can be seen with the widespread use of game engines from third parties, e.g. the Unreal Engine and Unity 3D.

Appendix: WebTics API

In this appendix the technology and Application Programming Interface (API) is documented in more detail. The documentation is intended to be useful for developers who have decided to consider using the system, or wish to see how we have implemented our system to improve their own metrics systems.

1.1 Selecting Technology

The API discussed in this chapter is a minimal event and message-passing interface using HyperText Transfer Protocol (HTTP) for all messages. A small library implemented in C++ provides the client-side functionality and a back-end for storing the data is implemented using PHP/MySQL and an Apache server (WAMP/LAMP). Using HTTP for system communication allows easier cross platform integration and development. The only assumption is that there exists a network connection and network handling within the operating system.

Advantages:

• A simple API provides most of the functionality needed in most projects.

• It does not require the user to write any network routines.

• Text-based message-passing makes debugging easier.

• The front-end client can be developed in any language with networking.

Disadvantages:

• This system does not maintain a permanent direct connection, therefore the messages and replies may not arrive or may arrive out of order. They are time-stamped and can be reordered after-the-fact.

• Requires some knowledge of SQL and PHP for editing the visualization and data mining system.

• Not GUI-based, so harder for designers to use.

1.2 How to Start

• 1. Create a local Database and web server for testing. Download the latest version of a LAMP/WAMP server. (The Uniform Server is a good one currently)

• 2. Download the metrics library from GitHub. This will give you the latest stable version of the API.

• 3. Run the SQL build script included in the metrics API download either locally or on a remote server. (If you want to make changes to the default logging system you can edit the setup file).

• 4. Place the php and html files either in your local root or on the server you are using.

• 5. Include the header files and the library files in your project. Make sure you follow the instructions for setting up the metrics class, in the README at the top level of the project.

• 6. Start logging events from your game using the LogEvent calls.

• 7. View your data using phpmyAdmin or the realtime visualization tools in the visualization path of the metrics site included in the API.

1.3 Overview of the API

Using this system requires the inclusion of one C++ header, GameMetrics.h. The system uses a second header file, GameMetricsDefaults.h, containing the enumerated types and labels describing the events to be logged. This header is not included by the programmer, but is loaded as part of the GameMetrics.h file. It may need to be included elsewhere to provide the event types to other program files. The functionality is provided by adding a single source file, GameMetrics.cpp. Alternatively the static library GameMetrics.lib may be linked into the project.

What follows is a summary of the usage of the API we discuss. This process is relatively simple as you only need to include a couple of setup methods for identification and tracking of the player and game session. The Game Metrics class has been implemented using the Singleton design pattern. This enables the metrics systems to be accessed from any location in the program, and controls the problem of multiple instantiations.

Creation of the metric system does not establish a connection to the logging system. A session must be opened explicitly. Any LogEvent() calls used to log an event before the session is opened will have no effect.

1.3.1 Session Order

We will explain each part of the API based on how you would use the system. The pseudo-code for the general structure of a program setting up a logging session is:

• 1. Get the instance of the GameMetrics singleton

• 2. Initialize the metric system

• 3. [optional] Check Authorization if required with unique ID

• 4. [optional] Request and set authorization if not already given for unique ID

• 5. Open metric session with unique ID

• 6. [optional] Register version number and event types if required

• 7. [optional] Get game parameters (type 1)

• 8. [optional] Start play session

• 9. [optional] Get game parameters (type 2)

• 10. While (the game is playing)

  …

  Log events

  …

• 11. [optional] Stop play session

• 12. Close metric session

1.3.1.1 Get the Instance of the Singleton

This method must be called to create the GameMetrics system. The system has been implemented as a Singleton so there is no public constructor. After creation further calls to GetInstance() will return a pointer to the GameMetrics system that has already been created. The calls to GetInstance() may be made from any location in the program. If the system has not previously been initialized, then passing autoInitialise as true will instruct the system to use the default server URL and PHP path defined in GameMetricsDefaults.h. If false is used the developer must use the Initialise() method. The default value is true.

GameMetrics* metricsSystem = GameMetrics.GetInstance(autoInitialise);

The addition API calls will be prefixed with metricsSystem to reflect their usage.

1.3.1.2 Initialize the Metric System

If the system has not been auto-initialized, or for any reason needs to be reinitialized this method may be used. The sever URL and the PHP path are passed to the system.

metricsSystem->Initialise(serverURL, serverPHPPath);

authorised = metricsSystem->IsAuthorised(uniqueID);

1.3.1.3 Request and Set Logging Authorization

After the programmer has obtained permission from the user via any mechanism available to them this method is used to update the database. The unique ID for this application is stored with the version number of the application that has been given permission. The version ID string may be empty if authorization per update is not required.

metricsSystem->SetAuthorised(uniqueID,trueFalse, versioned);

1.3.1.4 Open a Metric Session

This method is called near the start of the program. This sets the timer in the Metric system to zero. All messages sent to the logging system for this session will be time stamped with an offset from this point. At this stage you have to decide whether to track the individual user from session to session. For maximum anonymity, either required by law in a given location or because the user-base will be reluctant to participate, the logging system can be run with a session key linked only to the specific game session. This key allows events within the session to be linked but has no information linked to the user. Over a series of collected data events you will still be able to analyze how the player population behaves but will not be able to trace changes in a specific series of play sessions over time.

The alternative is to send some identifying information to the metric server to be linked to the session ID. There are many options for this. A registered game could sent its serial number for example. A player may be logged into a game environment such as Steam or your own game servers. Alternatively the user may be playing the game within Facebook or a similar social networking system. The availability of identifying information to your metrics system, and your obligation regarding storage and recording will be bound by the terms of service of the overall environment.

The benefit of being able to identify a specific game installation is the tracking of a player over time. The play actions of a group of players with a high win ratio, or high scores allows you to identify strategies that permit the player to beat the game, and guide where the rule may need to be tweaked for balance or to provide a ­continual challenge. The metric session ID and play session ID will not be consistently associated with a single player so an installation ID is required.

Further discussion on using the API will assume some persistent identification of a game installation between game sessions.

metricsSystem->OpenMetricSession(uniqueID);

1.3.1.5 Register Version Number and Event Types if Required

This method sends all the current event labels to the database with the current version number. There are two forms, one without parameters, which sends the values defined in GameMetricsDefaults.h and the other which passes two other user defined arrays of strings to the database.

metricsSystem->RegisterEvents();

metricsSystem->RegisterEvents(versionID,

eventsArray, numberOfEvents,

subeventsArray, numberOfSubevents);

1.3.1.6 Getting the Game Parameters (1)

This is an optional action. If you chose to write your game to have parameters that can be reinitialized during game play you may return a new set of parameters to the game. These parameters may control any aspect of the game, from the points scored for each action to decision thresholds for the AI system or new build cost/times to alter the game balance. A game can receive one of a predefined set of parameters, which is recorded in the metric system. The outcome is observed or the user is questioned about how they enjoyed the play experience. This is of use especially to game designers or developers of serious games.

The text parameter is there only as an option, it may be left NULL. It allows a the current game installation or runtime information to be sent to a more complex back-end system. This information can be used by the back-end system to evaluate which set of parameters should be returned to the game. It is the responsibility of the programmer to manage the memory of this string and delete it after the string has been used.

string* parameters = metricsSystem->RequestParameters(dataString);

string* parameters = metricsSystem->RequestParameters();

1.3.1.7 Start a Play Session

If a game has sets, levels or other well defined breaks in timing or difficulty it is useful to have explicit markers between these stages. Starting a play session places a marker in the metric system that may be used as a delimiter when the metrics are later analyzed.

If the game is a continuous experience then this marker will add no additional clarity to the metrics collected. You may choose to issue StartPlaySession() ­immediately after OpenMetricSession() and StopPlaySession() immediately before a CloseMetricSession() for completeness, or ignore the Start/Stop Play Session events.

metricsSystem->StartPlaySession();

1.3.1.8 Getting the Game Parameters (2)

This is a repeat of the game parameter request that can be performed at program start up. It is again optional. Parameters could be updated on a per set/level or chapter basis by requesting updates at the beginning of these sessions, while requests sent as start up would be used for the entire game session. It is possible to use the call multiple times, with major game parameters established for the entire game session at start up, and minor tweaks on either a random basis or determined by player behavior at the start of each chapter of the game.

string* parameters = metricsSystem->RequestParameters(dataString);

string* parameters = metricsSystem->RequestParameters();

1.3.1.9 Logging Game Events

The metrics system uses a single call to log events, where the developer includes the game event data, and the system adds the logging and game IDs, the logging time and sending time timestamps.

The full event method takes seven parameters to offer flexibility in how it is used.

metricsSystem->LogEvent(type, subtype,

 xInt, yInt, zInt,

 valueDouble, dataString);

There is also a debugging variant of this call. The goal of the debugging version is to support logging during development. The debug version will only send messages when the game is compiled with the _DEBUG flag set in the compiler. The variant allows the developer to put event logging in the program for functional testing which can be compiled out on release.⁵

metricsSystem->LogEventDebug( type, subtype,

 xInt, yInt, zInt,

 valueDouble, dataString);

The developer may implement wrapper methods for any required subset of the parameters which uses the method above with placeholder values to indicate a field is unused. A selection of wrapper methods are already provided by the API. The wrapper methods are also available for LogEventDebug() versions. The default value for unused integers and doubles is −999999, the default value for an unused string is “”.

metricsSystem->LogEvent(type, subtype,

xInt, yInt, zInt,

valueDouble, dataString);

metricsSystem->LogEvent(type, subtype, xInt, yInt, zInt);

metricsSystem->LogEvent(type, subtype, valueDouble);

metricsSystem->LogEvent(type, xInt, yInt, zInt);

metricsSystem->LogEvent(type, valueDouble);

Type and subtype are developer defined. They are logged in the database and it is left to the developer or designer to define the meaning of the stored values. For clarity in the source code it can be advantageous to use enumerated types for the event type and subtype, e.g.:

The player was killed by a grenade at location 12,20,30 and they sustained 70 points of damage;

metricsSystem->LogEvent( EventTypes::PlayerDeath,

  EventSubtypes::Grenade,

12, 20, 30, 70.0);

The player picked up 10 grenades at location 12,20,30;

metricsSystem->LogEvent(EventTypes::PlayerWeaponPickup,

EventSubtypes::Grenade,

12, 20, 30, 10.0);

The player respawned at a location;

metricsSystem->LogEvent( EventTypes::PlayerRespawn , 12, 20, 30);

The player fell 42 m;

metricsSystem->LogEvent( EventTypes::PlayerFall , 42.0);

1.3.1.10 Stop Play Session

This should be used to place a marker in the metric system to terminate a defined period of gameplay. After issuing this command the play session is no longer a valid. Closing a play session twice has not additional effect.

metricsSystem->StopPlaySession();

1.3.1.11 Close Metric Session

This method is used to place a marker in the metric system to indicate the game session has terminated normally. The absence of a close event before the presence of a new open event from the same user may indicate an unscheduled program exit such as a forced quit or program crash. Closing a session twice has no additional effect.

metricsSystem->CloseMetricSession();

1.3.2 Data Storage

The data collected by the logging system is primarily in a single table in your testing database. The table is set up to have indexing on event-type and sub-type which will aid the most common queries. Given that this API is for small developers and academic research, the speed of modern hardware is quite capable of accessing the volume of data normally collected. If there are problems with the speed of the database, then there would need to be time spent improving the back-end implementation of this system.

The data has been logged with several hierarchical data types which allows rapid and accurate partitioning of the views of the data-set. Firstly the logging sessions have been initialized with a unique ID, and each play session’s start and end has also been logged. Within the play sessions, event-type and event-sub-type, which are developer defined and arbitrarily extensible, provide as many game event categories as required.

The system expects to have a network connection but a failure to find a network can be accommodated. If a network is not available, or the OpenSession() attempt fails the system can write log events to a file on disc. On a successful connection to the metric server, if a log file exists on disc it is opened and sent to the server with a marker to indicate it was an offline session. The timing of events relative to the start of the logging session is maintained as is the order so the session can still be used for data mining. Finally all events are logged with a time stamp to allow small parts of a game session, e.g. the 30 s prior to a repeatable crash event, or the first 60 s of an emerging dominant strategy to be viewed.

For viewing the data we recommend that repeated queries to the database should be defined as a database view for efficiency. The data is also well suited for access from a web-based Javascript form. This query may also be built up directly in GUI tools such as phpSqlAdmin.