Usability and Experience of the Creative Industries Through Heuristic Evaluation of Flight Software for Mapping and Photogrammetry with Drones

  • Virginia Santamarina-CamposEmail author
  • María-Ángeles Carabal-Montagud
  • María de-Miguel-Molina
  • Blanca de-Miguel-Molina
Open Access


This work presents a heuristic analysis and evaluation of the main programs of mesh or mosaic flight plans for mapping and photogrammetry. The objective of this study was to identify the best designs linked to certain factors and usability elements to avoid errors and identify opportunities for optimization in the design of the Ground Control System (GCS) software. The GCS, through a graphical user interface (GUI), provides an advanced indoor navigation system for the drone, which was developed within the framework of the H2020 European Project AiRT (Arts Indoor RPAS Technology Transfer) (Definition of AIRT, chiefly Scottish: compass point).


Heuristic Evaluation Remotely Piloted Aircraft Systems (RPAS) Ground Control System (GCS) Flight Plan Software Design Thinking Methodology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

SMEs represent 85% of the stakeholders in the creative industry sector in Europe. They face competition from large companies and often encounter the challenge and need to adapt cutting-edge information and communication technologies (ICTs) with limited resources. For this reason, ICT tools, and in particular technological innovation, are essential for increasing the competitiveness of creative industries, because they expand the creative possibilities and improve the efficiency in all sectors (European Commission 2015).

This work is part of the European Project AiRT,1 “Technology transfer of remotely piloted aircraft systems (RPAS) for the creative industry”, the overall objective of which is to provide European cultural and creative industries with a new tool, enabling them to offer new services and grow in the international market. To achieve this goal, we have designed an RPAS especially for indoor professional use (Santamarina et al. 2018). The main innovations associated with this tool have focused on the integration of the following:
  1. 1.

    An indoor positioning system (IPS), based on ultra-wide-band (UWB) technology. This system allows safe navigation of drones indoors, providing movement and positioning information to a few centimetres on all axes. This can be achieved due to an improved update rate of up to 240 Hz, a four-antenna approach, and adapted positioning algorithms.

  2. 2.

    Software that allows the reconstruction of the indoor space (3D model) where the recording will take place. This allows the creative space to be analysed beforehand and the flight plan to be designed and executed in a safe way.

  3. 3.

    GCS software, which provides an advanced navigation system through the GUI. It also includes active safety measures, with the possibility of planning the flight and designing the audiovisual project.

  4. 4.

    A drone with ready-to-use commercial (COST) element integration on an aerial platform that fulfils the latest passive security measures at a reduced price, integrating a professional camera that operates with 360° rotation both at the top and at the bottom of the drone.


In addition, the value proposition covers a wide range of issues relevant to the target segment of the creative industries: price, security, operation, usability, and onboard technologies. All these aim to offer additional services and are designed to provide a positive customer experience.

2 Methodology Developed in the Design of the Ground Control System Software

The aim of the AiRT project is to generate an innovative tool that focuses on effectively understanding and solving the real needs of the creative industry. In the specific case of the design of the GCS software and GUI, ISO 13407:1999 provides a guide to achieving quality in use by integrating iterative tasks into user-centred design (UCD). UCD considers this as multidisciplinary work, which includes human factors and knowledge as well as ergonomic techniques, with the objective of optimizing the effectiveness and efficiency of the work environment and neutralizing the possible adverse effects of its management (ISO 1999).

To achieve significant innovations, it is necessary to know the end-users and care about their lives (Both 2010). To complement the UCD, the design thinking methodology (Fig. 1) was implemented with the aim of generating a feedback process with the cultural and creative industries through collaborative, participatory, and creative work. This method is mainly composed of five non-linear phases, called empathize, define, ideate, prototype, and test. These lead to a solution that meets the objectives of the end-users and is technologically feasible and commercially viable (Both 2009).
Fig. 1

Methodologies used in the design of the GCS software and GUI. Source: Own elaboration, adapted from Both (2010)

2.1 First Phase—Empathize—and Second Phase—Define

In the specific case of the development of the GCS software and the GUI, in the first phase of the project, known as empathize, three focus groups were carried out in Spain, the United Kingdom, and Belgium, from which information was obtained on thirteen different sectors of the creative industries. This technique constituted an effective qualitative tool to discover the desires, motivations, values, and experiences of our users (Hinton 2004). As a result, during the define phase, a need analysis was carried out through qualitative content analysis and social network analysis (SNA). Therefore, manual coding and categorization of qualitative data (Santamarina et al. 2018) were applied, which formed the basis for the analysis of the functionality of the program executed in the GCS, the functionality of which is provided to the human end-user through the GUI.

2.2 Third Phase—Ideate

In the third phase, based on the synthesis of the information obtained in the focus groups, written scripts were prepared and then transferred to storyboards representing the possible functionalities of the GCS software in different creative scenarios. These helped to communicate the main ideas and needs more clearly. Storyboards, also termed “presentation scenarios”, are image sequences that show the relationship between the user’s actions or inputs and the results of the system, making it easier for the design team to understand them (Maguire and Bevan 2002). This technique can help to control the process of creating solutions and to identify different variables to break down large problems into smaller ones that can be evaluated and solved better (Both 2010). In this way, from the storyboards, the requirements were extracted, which allowed us to obtain, on the one hand, the concrete functionalities to be implemented in the GCS software and, on the other hand, the definition of elements related to the usability aspects of the GUI.

At the same time, the requirements of the GCS software were defined based on the needs of the creative industries. A documentary investigation of 29 flight plan software programs was carried out with the objective of identifying the solutions available in the market that are similar to our product to gain a more objective perspective on the usability aspects and final design of the GUI. The main tool used for the development of this analysis was a heuristic evaluation (Molich and Nielsen 1990), allowing the analysis of the main usability components of flight plan software by experts and focusing the study on the variables ease of learning, efficiency, quality of being remembered, effectiveness, and satisfaction (Nielsen 2012).

2.3 Fourth Phase—Prototype

Once the heuristic evaluation of the flight plan software available in the market had been completed and the design elements extracted from the storyboards had been defined (the user interface of the client application), an iterative design process was initiated using the Wizard of Oz Prototyping Technique (Both 2010). In the first phase, the design of paper prototypes was carried out using visual language (Fig. 2). Therefore, it was taken into account that the user interface should be able to be executed on both laptops and mobile platforms, so the models should use the standard conventions of this type of device, for example tactile gestures. The combination of cards (windows) and visual language to design the user interface of the client application represented an important change in the construction of cooperative models. On the one hand, the use of visual language made it easier to make ideas visible, tangible, and sequential and to encourage divergent collaborative thinking. On the other hand, the use of mobile cards facilitated collaborative work and improved the internal usability testing, as the repetitions were more fluid. At the end of this phase, a model of the user interface was made using the online tool NinjaMok© (NinjaMock 2018) (Fig. 3a, b). This application provided interactivity for a more realistic prototype, reproducing an interactive preview of the user interface layout and facilitating feedback from developers with the creative team and end-users. All these activities aimed to improve the application’s functionality and make it more intuitive or easy to use. Once the iterative design of the model had been completed, the graphical design of the user interface (Fig. 4a, b) was developed based on the results obtained in the heuristic analysis and maintaining the iterative process (Fig. 5a, b).
Fig. 2

Paper prototype of the clients’ UI. Source: AiRT project

Fig. 3

(a and b) Graphical user interface layout design. Source: AiRT project

Fig. 4

(a and b) Graphical design proposal for the graphical user interface. Source: AiRT project

Fig. 5

(ad) Example of the iterative process of graphical design of the graphical user interface from storyboard to final graphical design. Source: AiRT project

From the user interface model generated in the online tool NinjaMok© and the graphical design proposal of each of the windows, a software prototype was developed that implemented the functionalities of the AiRT system. It was based on the prioritized requirements, with the aim of visualizing the solutions and identifying possible improvements.

2.4 Fifth Phase—Tests

In the last phase, end-users made use of the prototypes, based on the selection of scenarios relevant to the creative industries, in the three participating countries. The objective of this stage was to identify failures or to provide new improvements through the participation action research tool (PAR). The purpose of this technique is to obtain relevant data from experts that allow the subsequent interpretation and analysis of the facts based on the experiences (Santamarina et al. 2017). The PAR was divided into two phases. In the first phase, a user test was developed, first from the user guide and then from the prototype. In both cases, the dynamics were filmed with the aim of carrying out a subsequent analysis using qualitative data analysis software. This technique is based on the observation of the way in which a group of users carries out a series of tasks mandated by the evaluator, analysing the usability problems that they face. Finally, in the second phase, the heuristic evaluation of the AiRT system was carried out with the aim of identifying potential usability problems, checking for compliance with previously established usable design principles (heuristic principles) (Wilson 2014).

3 Analysis of the Accessibility and Compatibility of the Flight Plan Software

For the analysis of the flight plan software, 29 mesh- or mosaic-type flight plan software programs were selected for mapping and photogrammetry (Fig. 6). Of these, 10 were photogrammetry and 19 were ground station software programs. Each of them was tested for compatibility with MAVLink, APM, and PX4 communication protocols, on which platforms they could run, and to establish whether they were open source.
Fig. 6

Mosaic or mesh flight plan software for mapping and photogrammetry. Source: Own elaboration from Intel Corporation (2015)

From the analysis carried out, it was determined that the MAVLink, APM, and PX4 communication protocols for information exchange between ground control stations and micro UAVs (unmanned aerial vehicles) were compatible with the DroneDeploy, UgCS, QGroundControl and Mission Planner, AndroPilot, APM Planner, and DroidPlanner2 software programs (Fig. 7). Regarding the compatibility with the different platforms, the software that allowed more installation options was QGroundControl, followed by Pix4D and OpenPilot (Fig. 8). On the other hand, the only open-source software programs were Opendronemap, Open MVS, QGroundControl, Mission Planner, Tower, AndroPilot, APM Planner, OpenPilot, and DroidPlanner2. In conclusion, the software that offered the best accessibility and versatility in relation to its compatibility with communication platforms and protocols was QGroundControl (Fig. 9).
Fig. 7

Analysis of the number of compatible communication protocols (values 0–3). Source: Own elaboration

Fig. 8

Analysis of the number of compatible platforms (values 0–6). Source: Own elaboration

Fig. 9

Accessibility and compatibility analysis with communication platforms (values 0–6) and protocols (values 0–3). Source: Own elaboration

4 Heuristic Evaluation of Flight Plan Software

Heuristic evaluation consists of the study and evaluation of an interface by experts, based on a set of previously defined design principles and standards. It is characterized by its fast and economic analysis, since it involves only one or several experts, who provide different answers based on the same set of rules. These standards, which serve as the basis for evaluation, are called usability principles. According to the ISO standard 9241-11, usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” (ISO/IEC 1998).

The objective of this study was to identify the best designs linked to certain factors and elements of usability to avoid errors and identify opportunities for optimization in the design of the Ground Control System software and the graphical user interface of the AiRT system.

The selection of experts was carried out in compliance with the parity requirement, previous experience in the use of flight plan software, and equal participation of experts from the creative industry sector and other sectors (Fig. 10).
Fig. 10

Selection criteria for the experts to participate in the heuristic evaluation. Source: Own elaboration

The experts were provided with a checklist, which contained a set of questions that assessed usability by blocks in relation to accessibility, identity, navigation, content, consistency, shortcuts, and responses to actions (Wilson 2014). Each variable had to be rated between ‘1’, the lowest score, and ‘5’, the highest rating. The analysis was performed through the viewing of video tutorials or the actual use of the applications. Only the eMotion Sensefly software, as it was linked to the purchase of the equipment (thus, it was not accessible), and the DroidPlanner2 software (discontinued) could not be analysed.

After processing the data, the highest average score linked to accessibility aspects was obtained by the Pix4D software with a score of 4.4 out of 5, followed by DJI Go with a score of 4.2 out of 5 (Fig. 11). In both cases, aspects related to easy software localization, downloading, and installation, compatibility with different platforms, a proper contrast between text and background, font size and spacing, and the proper use of ALT tags were highlighted.
Fig. 11

Average score of the software in relation to accessibility. Source: Own elaboration

Regarding the identity variable, the software programs that obtained the highest scores were Pix4D, DJI Go, DJI Ground Station, and GS Pro, with an average score of 4.3 out of 5 (Fig. 12). In this case, these programs stood out for the adequate handling of the acronym, logo, and slogan of the software as well as for the information provided in relation to the developer company.
Fig. 12

Average score of the software in relation to identity. Source: Own elaboration

Concerning the navigational aspects, the software that achieved first place, with an average score of 4.4 out of 5, was DJI Go (Fig. 13). This was because it facilitates the identification, execution, and progress of tasks from the beginning, with clear and concise icons and menus, and provides adequate help support.
Fig. 13

Average score of the software in relation to navigation. Source: Own elaboration

In content handling, the software that scored the highest was Open MVS, with a score of 4.2, followed by DJI Go and AndroPilot, with a score of 4 out of 5 (Fig. 14). In all three cases, the software provided adequate handling of the main titles, with moderate use of menus and icons, and presented the critical content above the secondary content. On the other hand, the use of consistent styles, colours, and appropriate contrasts is also noteworthy.
Fig. 14

Average score of the software in relation to the content. Source: Own elaboration

In terms of consistency, the software with the highest rating was Open MVS, with an average score of 5 out of 5 (Fig. 15). This application presents a coherent sequence of actions in similar situations and a consistent and familiar structure of commands, screens, menus, and terminology.
Fig. 15

Average score of the software in relation to consistency. Source: Own elaboration

With respect to shortcuts, the software with the highest rating was PhotoModeler, with an average score of 5 out of 5 (Fig. 16). This tool allows users to reduce the number of interactions and increase the pace of use. Moreover, it includes an action generator that enables them to customize workflows and schedule them automatically.
Fig. 16

Average score of the software in relation to shortcuts. Source: Own elaboration

Finally, in relation to the response to the actions, the software programs that stood out were PhotoScan, Open MVS, Free Flight pro, and DroneMapper, with an average score of 4.3 out of 5 (Fig. 17). They presented effective time management, with effects of less than 0.1 s, and, for actions that exceeded 6 s, information on the actions is included.
Fig. 17

Average score of the software in relation to the response to the actions. Source: Own elaboration

5 Conclusion

The combined use of the design thinking methodology together with user-centred design facilitated the construction of the Ground Control System (GCS) software and graphical user interface (GUI), taking into account the user experience.

The software analysis of flight plans facilitated the selection of the programs to be analysed and the preparation of the checklist, which was subsequently completed by the experts. The heuristic evaluation provided an approach to the best designs linked to certain factors and elements of usability of the main software available on the market. This provided the creative participants with ideas and solutions for the development of the GUI of the AiRT RPAS.

As a conclusion to the heuristic evaluation, we can report that the first positions, taking into account all the variables analysed (accessibility, identity, navigation, content, consistency, shortcuts, and responses to actions), are occupied by products of the leading brand in the market, DJI (DJI 2016) (Fig. 18). They offer intuitive, tablet-oriented applications for any type of user.
Fig. 18

Average score of the software in relation to accessibility, identity, navigation, content, consistency, shortcuts, and responses to actions. Source: Own elaboration

The software that was ranked in first place is DJI Go, with an average rating of 4.2 out of 5. It offers a minimalist, easy-to-use interface that displays the most basic data for easy and safe flying. The menus are well distributed, offering an excellent user experience. In the second place is the Pix4D software, with an average rating of 4 out of 5, providing a simple and clean graphical user interface that simplifies and facilitates its use. In the third and fourth places are again software from DJI, DJI Ground Station and GS Pro, with an average rating of 3.9 and 3.8 out of 5, respectively. DJI Ground Station stands out for being quite complete and offering multiple options and tools, although its aesthetic and outdated design compared with the DJI Go product leads to a value loss. On the other hand, GS Pro is prominent for being highly intuitive for any type of user, thus reducing the entry barrier for beginners.


  1. 1.

    The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 732433. Reference: H2020-ICT-2016-2017.


  1. Both T (2009) Design thinking bootleg. Institute of Design Thinking at Stanford, Stanford. Retrieved from
  2. Both T (2010) Bootcamp bootleg. Stanford Institute of Design Thinking at Stanford, Stanford. Retrieved from
  3. DJI (2016) DJI – the future of possible. Retrieved April 21, 2018, from
  4. European Commission (2015) Research and innovation. Participant portal. Retrieved March 20, 2018, from
  5. Hinton A (2004) Observing the user experience: a practitioner’s guide to user research. Morgan Kaufmann. Retrieved from
  6. Intel Corporation (2015) Ground control station | benchmark study (vol 1). Retrieved from
  7. ISO (1999) ISO 13407:1999, Human-centred design processes for interactive systems. Europe. Retrieved from
  8. ISO/IEC (1998) ISO 9241-11: ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability. Int Organ Stand 1998(2):28. Scholar
  9. Maguire M, Bevan N (2002) User requirements analysis: a review of supporting methods. Proceedings of IFIP 17th world computer congress (January), pp 231–246. doi: Scholar
  10. Molich R, Nielsen J (1990) Improving a human-computer dialogue. Commun ACM 33(3):338–348. Scholar
  11. Nielsen J (2012) Usability 101: introduction to usability. Nielsen Norman Group, Articles. doi: Scholar
  12. NinjaMock (2018) NinjaMock online wireframe and mockup tool. Retrieved April 21, 2018, from
  13. Santamarina V, Martínez EM, Carabal MA, De Miguel M (2017) Participatory action research (PAR) in contemporary community art. In: Santamarina V, Carabal MÁ, De Miguel M, De Miguel B (eds) Conservation, tourism, and identity of contemporary community art. Apple Academic Press, New Jersey, pp 105–136CrossRefGoogle Scholar
  14. Santamarina V, De Miguel B, Segarra M, De Miguel M (2018) Importance of indoor aerial filming for creative industries (cis): looking towards the future. In: Innovative approaches to tourism and leisure. Springer, Cham, pp 51–66. doi: Scholar
  15. Wilson C (2014) Heuristic evaluation. In: Nielsen J, Mack RL (eds) User interface inspection methods. Wiley, New York, NY, pp 1–32. doi: Scholar

Copyright information

© The Author(s) 2018

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Virginia Santamarina-Campos
    • 1
    Email author
  • María-Ángeles Carabal-Montagud
    • 1
  • María de-Miguel-Molina
    • 2
  • Blanca de-Miguel-Molina
    • 2
  1. 1.Conserv. & Restoration of Cult. Heritage DepartmentUniversitat Politècnica de ValènciaValenciaSpain
  2. 2.Management DepartmentUniversitat Politècnica de ValènciaValenciaSpain

Personalised recommendations