Background

Despite a significant reduction in global neonatal mortality from 1990 through 2018, approximately 2.5 million newborns die annually. These newborn deaths represent nearly half of the 5.3 million children under five who die each year [1]. The proportion of newborns who contribute to under-five mortality has been growing and the gap between reduction of childhood and infant mortality is rising. This is because strategies for newborn mortality reduction are challenging to implement [2]. The top three causes of newborn deaths continue to be prematurity, intrapartum-related events, and infection [2]. Approximately 10% of babies require help after birth in high-resource settings, compared with 15% in low-resource settings [3, 4]. Skilled personnel who can resuscitate babies immediately after delivery save lives [5, 6]. Standardized training through the Helping Babies Breathe (HBB) program has been shown to improve newborn resuscitation skills [7, 8].

The HBB curriculum, currently in its second edition (HBB 2.0), was designed to train frontline health providers in low-resource settings to provide life-saving interventions for newborns [9]. HBB has been taught in over 80 countries and has been shown to reduce neonatal mortality and morbidity [10]. HBB training materials are standardized with a flowchart that illustrates the steps in the HBB algorithm, accompanied by a facilitator flip chart that provides all the teaching prompts for HBB workshop delivery. The training program emphasizes the practice of individual steps, key skills, and integration of skills into the delivery setting through simulation. Illustrations and a green-yellow–red colour scheme are used to visually guide providers through the HBB algorithm. In Uganda, the HBB program has been deployed with the help of international health implementing partners in collaboration with the Ministry of Health due to the relatively high costs of HBB training [11]. One of the most essential skills HBB teaches is how to ventilate a newborn struggling to breathe, and to start doing so within one minute after birth, referred to as the Golden Minute. Skills deterioration after initial training is a significant barrier to sustained impact of programs like HBB [12]. Low dose high frequency (LDHF) practice, use of simulation, refresher training, and supportive supervision are strategies that help reduce the decay of skills over time [13,14,15,16,17]. However, implementing initial training, refresher training, and supportive supervision are all costly and resource intensive, particularly in settings where births occur at smaller facilities in rural communities spread out over a large geographic region [11]. Since 2010, the Ugandan Ministry of Health alongside implementing partners have been scaling up training for healthcare workers who look after newborns across the country using the HBB program. As of 2014, HBB was offered in more than 40% of health facilities [18].

Mobile health solutions have been increasingly popular as smart devices are ubiquitous. Their rate of ownership in low- and middle-income countries has grown exponentially, with over 5 billion unique users worldwide in 2019 [19], making it especially attractive when designing solutions aiming to reach large audiences [20]. In 2018, there were 19.8 million unique mobile phone users in Uganda, with half of these subscribers using the internet on their mobile phones [21]. This represents mobile internet penetration in 23% of the population. Of these users, only 16% had a smart phone compared with the average of 30% of users having smart phones in Sub-Saharan Africa [21]. Mobile applications have been tested in numerous settings within healthcare and medical education with varying success. Thoughtful development to optimize usability and utility for the target population is important [22,23,24]. Design processes with iterative prototyping have been successfully used to develop mobile apps for various purposes, such as supporting lifestyle interventions for cardiovascular health [25], clinical decision-making for osteoporosis treatment [26], and self-management for dementia patients [27]. Apps targeted towards newborn health [28,29,30,31,32,33,34] have been created and evaluated previously, some employing UCD, but none have informed their design by exploring the cognitive processes of learning newborn resuscitation skills. Understanding the underlying preferences, perceptions, facilitators and barriers that drive behaviours can help tailor the development of features and functions of apps [25, 35, 36]. Applying UCD and assessing human factors during the development of a mobile app can enhance user experience and increase the effectiveness of the tool.

In this paper we describe the UCD process of the development a mobile app (HBB Prompt) to improve knowledge and skills retention after initial HBB training.

Methods

Study overview

The study was conducted at Mbarara University of Science and Technology (MUST) in southwestern Uganda. This article describes Phase 1, the development of HBB Prompt. More details regarding the entire study can be found in the protocol paper [37]. Further details including the focus group discussion (FGD) guides for Phase 1 can be found in Additional file 1 and Additional file 2.

Objectives

The overall objectives of the study were to apply a UCD framework to develop a mobile app to enhance HBB knowledge and skills maintenance (Phase 1) and to test its effectiveness in a pilot setting (Phase 2). The objective of Phase 1a of the study was to understand the facilitators and barriers to learning and maintaining HBB skills for providers with varying levels of experience with newborn resuscitation. Information from Phase 1a was incorporated into modifications of HBB Prompt version 0 (v0) to create version 1 (v1), which was used in Phase 1b UCD testing. Subsequently, the objective for Phase 1b was to evaluate how HBB Prompt v1 does or does not meet the needs of HBB users and how it could be feasibly improved prior to pilot testing in Phase 2 of this study. See Fig. 1.

Fig. 1
figure 1

Overview of HBB Prompt study. The HBB Prompt study is divided into Phase 1, where there were iterative UCD cycles to develop HBB Prompt and Phase 2, where the HBB Prompt app will be piloted. During Phase 1, HBB trainers formed one focus group (FGD 1) and HBB providers formed the second focus group (FGD 2). Each group went through Phase 1a and 1b separately to facilitate UCD of HBB Prompt

Full size image

Participant recruitment

Participants for Phase 1 were recruited by letter from a register of previously HBB-trained healthcare workers who reside and care for newborns in southwestern Uganda. Participants who responded and who were able to attend the scheduled FGD sessions were included in the study. Inclusion criteria for the first focus group was any healthcare provider with HBB trainer experience (i.e. received previous training in HBB and also completed the HBB Master Trainer Course) allowing them to teach the HBB curriculum. Inclusion criteria for the second focus group was frontline providers who had previously received HBB training.

Phase 1: mobile app development

HBB Prompt v0 development

Study co-investigators collectively used their knowledge and experience with the HBB program, resuscitation education, and mobile app development to create foundational features of HBB Prompt v0. In particular, we sought to leverage strategies that have previously been shown to improve skills retention, including LDHF practice, use of simulation, refresher training, and supportive supervision [10, 38, 39]. This provided a framework for our two software developers to build features of the app that we anticipated HBB trainers and providers would request. The basic functions that were identified as most important were: training videos and information portal, self-practice mode and group simulation mode, knowledge assessments, and a practice dashboard.

The HBB Prompt app is an Android-based application that can run on smart devices with an Android Operating System, Version 4.0.3 and above. The Android platform was chosen due to lower cost and higher penetration of Android devices in Uganda and other low- and middle-income countries. The app was designed using Android Studio 3.0, an Integrated Development Environment provided by Google for design and development of Android apps. HBB Prompt, once installed, does not require internet connectivity; however, when internet connectivity is detected, it automatically logs usage data to a remote Linode server, which it maps according to device International Mobile Equipment Identity (IMEIs). Usage data tracked does not require runtime or dangerous permissions and therefore users do not need to be concerned about access to potentially sensitive data on their accounts. The app does not have access to audio or video recording, the camera function, or location data. Usage data recorded include user identification, date and time of login, time spent on each page of the app, and scores on quizzes or simulations. The app is designed to solely track in-app use, which is intended for education and practice only and not for direct patient care. The logged data can later be exported in comma-separated values (csv) format for further analysis.

Phase 1a

Two separate focus groups underwent the same sequence of exploratory activities: discussion of personal experiences with newborn resuscitation, facilitators and barriers to learning HBB knowledge and skills, perceptions of the timeline in acquisition of knowledge and skills, and generation of ideas on how to maintain and sustain HBB knowledge and skills. The composition of each focus group was primarily divided by HBB training experience so that input could be elicited separately from trainers (FGD 1) and frontline users (FGD 2) of the HBB curriculum. See Additional file 1 for details.

Phase 1b

The same focus groups were invited back to help co-design and provide feedback to improve the HBB Prompt v1. Participants explored the app, simulated use of the app and provided feedback regarding app content, interface, navigation, functionality, customizability, and usability. Two iterations of UCD feedback sessions took place prior to finalizing the app for Phase 2 pilot testing. See Additional file 2 for details.

Analysis

FGDs were recorded and thematic analysis was conducted, where investigators independently coded components of the transcript to highlight needs and mental models of the participants. These independent codes were aggregated and synthesized in order to uncover main themes consistent across investigators. After each iteration, categories and themes were reviewed together with the software developers to incorporate feasible improvements to HBB Prompt.

Results

Participants

The first focus group comprised six HBB trainers who were all physicians and the second focus group comprised seven midwives. Table 1 describes the demographic characteristics of the participants in each FGD, including level of practical and simulated experience with newborn care, emergency training, and comfort with mobile devices.

Table 1 Focus group participant demographics and characteristics
Full size table

Phase 1a

Exploring personal experience, facilitators and barriers

Themes that emerged from FGDs are summarized in Table 2. Important factors that motivated HBB learning and skills maintenance were supportive supervision and self-esteem. Additionally, continuing medical education opportunities, peer-to-peer learning, refresher courses, and timely feedback were all identified as positive drivers. Barriers identified were lack of equipment and health system issues such as inadequate transport for escalation of care.

Table 2 Themes to address in HBB learning and skills maintenance during Phase 1a
Full size table

Timeline of learning HBB and perceptions of what was difficult to learn and remember

Participants explored the timeline of learning HBB skills and knowledge, including what was perceived as, and what was in reality, the most difficult to learn and remember (Fig. 2). A concern throughout was how to translate skills from a training environment to real-life scenarios. Before HBB training, many concerns surrounded learning about the steps to help a baby breathe and the fear of not retaining skills. During HBB training, there was fear of not achieving the Golden Minute. Three months after training, participants were worried about forgetting their skills and not effectively debriefing to improve after a resuscitation.

Fig. 2
figure 2

Insights about the timeline in HBB learning and skills maintenance during Phase 1a. Themes from FGDs with HBB trainers and providers regarding their thought process in relationship to the timing of HBB training

Full size image

Co-designing solutions

Prior to being exposed to HBB Prompt v0, participants were asked to co-design solutions to help learn and maintain HBB skills. They identified promoting a safe learning environment as an important goal. To achieve this, suggestions emphasized the use of audio and visual prompts and written and video guides to facilitate different ways of learning. To counteract the inertia to practise, methods of motivating users were discussed, including setting reminders, promoting champions, having dedicated group practice sessions, and creating a friendly competitive environment to increase accountability. Participants also expressed the importance of a community of support, for example, through a message board or group chat to review cases and to encourage each other. Suggested mobile app features included information for each HBB skill (introduction, video demonstration, practice prompts, scenarios, and if possible, ability to provide feedback) and the ability to function offline as internet connectivity can often be unreliable.

Implications on app development

Themes that were not feasible to address using a mobile app included health system challenges such as lack of available equipment and transport for escalation of care for unstable babies. Additionally, due to limited funds to facilitate real-time internet connectivity of the app, the reminders, group chat and message board functions were not implemented. Otherwise, many themes identified in the FGDs were anticipated by the authors and already addressed by features in HBB Prompt v0.

Functions and features of the HBB Prompt v0 and modifications based on Phase 1a

Figure 3 shows screen shots of HBB Prompt v1 and Table 3 describes the features of HBB Prompt. Training mode was designed to demonstrate each step of the HBB algorithm through videos, aimed as a resource for refreshing knowledge and skills. In response to concerns about lack of equipment, a video demonstrating how to reprocess and maintain equipment was added. Other feedback from Phase 1a helped inform the video content, such as breaking down skills into digestible steps and troubleshooting when effective ventilation is not being achieved.

Fig. 3
figure 3

HBB Prompt v1.0 screenshots. a different scenarios users can choose from in Simulation Mode. b Performance checklist that appears once a simulation is complete. c Beginning of a simulation. Note the two countdown timers from the time the simulation begins and the time when the simulated infant is born. d Sample of dashboard statistics

Full size image
Table 3 HBB Prompt app features during Phase 1b
Full size table

Simulation mode was designed to facilitate routine practice of HBB skills, running through various scenarios that emphasize different aspects of resuscitation. Simulation mode was divided into alone mode (for individual practice) and rater mode (for group practice or peer-to-peer learning). This feature encourages and supports LDHF training. Consistent practice and increased familiarity and confidence in carrying out the steps of the Golden Minute enable HBB providers to perform all actions within one minute of a baby’s birth. To support this, timer prompts were built into simulation mode to facilitate practice of the Golden Minute. Furthermore, checklists were added at the end of every simulation to help users debrief what went well and what could be improved.

The Quizzes and knowledge check section was designed to help refresh and consolidate knowledge about the HBB algorithm and rationale for steps in resuscitation, providing real-time feedback regarding information retention.

The Scoreboard/Dashboard was created to increase accountability and motivation, encourage all users to practise HBB consistently, and foster confidence by displaying personal performance benchmarks.

Phase 1b

Table 4 summarizes the feedback that was received during Phase 1b and outlines modifications that were made, where possible, in response to the suggestions from the FGDs. Feedback about HBB Prompt was generally positive and navigation was felt to be overall intuitive. There were two iterations of Phase 1b (one with FGD1 and the other with FGD2). After suggestions were incorporated, the team was satisfied that the HBB Prompt app was ready for deployment in Phase 2 for pilot testing.

Table 4 Summary of feedback for HBB Prompt during Phase 1b
Full size table

Discussion

In Phase 1 of this study, we applied a UCD framework and engaged a group of HBB trainers and providers to develop a mobile app, HBB Prompt, to improve knowledge and skills retention after initial HBB training. Many facilitators and barriers identified were feasible to address with HBB Prompt and reflected anticipated features based on prior knowledge about contributors to skills decay. Participants expressed positive perceptions of the app’s ability to help users practise and maintain HBB knowledge and skills.

Strengths

HBB Prompt is a novel mobile app that facilitates simulation of newborn resuscitation to promote deliberate practice of applying the HBB algorithm to real-life scenarios. We employed a cooperative design UCD framework, which prioritized participation from frontline end-users, both trainers and learners of HBB, since failure of adoption of healthcare technology has been attributed to absence of end-user input [40]. There are different methods to involve end-users in the design and evaluation process, with prototyping and simulation being commonly used in design, and focus groups and observations being common evaluative methods [41]. Understanding the usability of a mobile app and how it meets needs and addresses gaps to improve outcomes have been key to the successful adoption of health technologies. UCD and understanding human factors is crucial since certain ideas may not translate well into a mobile app. Understanding the needs of primary users by exploring acquisition and maintenance of skills and knowledge helped map out areas where HBB Prompt can assist and remaining gaps that need to be addressed through other means [40]. Each function of HBB Prompt was vetted during Phase 1b and largely addressed the issues that were uncovered during Phase 1a.

By iterative testing through participatory design and evaluation, we hoped to overcome the common barrier of failed interventions due to lack of user input. Since the main goal of designing HBB Prompt is to improve knowledge and skills retention after initial HBB training, the UCD process specifically focused on HBB trainer and trainee experiences regarding the process of learning HBB skills and the daily challenges of implementing those skills. We used FGDs to better understand when different cognitive processes took place, influencing our strategies for knowledge and skills retention, a method supported by the applied cognitive task analysis [42]. This strategy has been successful in developing apps for various purposes in healthcare [25, 43, 44], including those in the maternal newborn health domain [29, 32,33,34, 42].

Newborn health apps [28, 34] have been developed to support HBB training by provision of HBB reference material and clinical decision support. HBB Prompt builds on that foundation and adds new features including Simulation mode and the Scoreboard/Dashboard, which aim to promote LDHF practice. NeoTree [29] is a mobile application developed in a single neonatal unit in Malawi using similar methodology as HBB Prompt, but was intended for data capture of neonatal admissions and clinical decision support. The authors used focus groups to better understand acceptability and feasibility of mobile health (mHealth) solutions before and after implementation of NeoTree. The positive impact of the NeoTree app was attributed to UCD and coproduction. HBB Prompt, on the other hand, applies UCD to create and fine-tune the features deployed in the app. Finally, HBB Prompt builds on the success of the Safe Delivery App (SDA) [30]. The SDA [30] contains education material and videos based on the United Nations Basic Emergency Obstetric and Neonatal Care program (BEMONC) and was evaluated in a large cluster-randomized trial in Ethiopia, which found improved knowledge and skills retention and a statistically non-significant 24% reduction in perinatal mortality. HBB Prompt shares many features with SDA, including video and text as educational materials. HBB Prompt goes beyond the reference tool function by supporting simulation-based skills practice, an interactive feature developed with UCD based on FGD themes of the importance of simulation and peer-to-peer learning. Furthermore, HBB Prompt contains a knowledge assessment feature, which was not available in the early version of the SDA that was tested in the cluster-randomized trial.

Facilitating simulation

Our app design was informed by the principles of deliberate practice, a well-established methodology for simulation-based medical education [17]. HBB Prompt differs from available apps because it facilitates hands-on practice and simulation, which is a core component of the HBB 2.0 curriculum. While simulated resuscitation will differ from real-life scenarios, consistent practice through simulation is an effective method to maintain competency in managing high-acuity low-occurrence events, such as apnea in a newborn [16, 38]. Much of HBB Prompt’s design focus was to facilitate users’ ability to practise HBB skills in short, directed ways, either alone or with peers. LDHF practice of specific skills such as bag-mask ventilation has been shown to be effective in enhancing skills retention after training [10, 38, 45, 46]. For individuals who are unable practise skills with others, the Training mode videos serve as a guide, and the function of Simulation—alone mode provides a structure and framework whereby users can benchmark their skills and receive automated external prompts and feedback in order to retain their resuscitation skills.

Use of motivation/behaviour change strategies

The Scoreboard/Dashboard feature was designed to track trends in scores on quizzes, and participants felt it would foster confidence in knowledge retention and increase awareness of knowledge gaps. Being able to visualize frequency and practice patterns serves to increase ownership and accountability for practising as it is a visual audit, providing feedback to each user about habits of practice and where skills may be adequate or lacking [47, 48]. Besides having visual statistics that may benefit end-users, this feature was built to also track app function usage to provide insight in Phase 2 of the study to improve the app and assess which parts of the HBB 2.0 curriculum require more reinforcement. Benchmarking to help healthcare providers inform behaviour change is supported by Social Comparison Theory and Reference Group Theory and has been shown to be a successful strategy to motivate behaviour change [48]. The display of longitudinal trends provides ongoing feedback and is supported by the Feedback Intervention Theory [48]. The Scoreboard/Dashboard feature is unique to HBB Prompt compared with other apps in the newborn mHealth space and evaluating its effectiveness during Phase 2 will further enhance understanding of how audit and feedback can increase retention of knowledge and skills after training.

Limitations

Although we recruited a diverse group of participants, we may have missed some perspectives. One limitation is the absence of larger scale beta-testing prior to deployment in Phase 2. For example, Bucher et al. invited a large group of users to download and beta-test their app, and provide feedback through a UCD questionnaire [34]. However, the size of our FGDs allowed in-depth exploration and likely uncovered input to reach theme saturation. HBB Prompt’s user interface and functions may be less intuitive for potential users who are less familiar with use of technology as all but one participant (in the HBB provider group) owned a smart device and described themselves as comfortable with their use. Even though the target audience for HBB Prompt are those who were previously HBB trained, including input from first-time learners of the curriculum could have provided additional insight to the knowledge and skills acquisition and retention process. There was an intention to recruit a third focus group with naïve HBB learners; however, due to logistical constraints, we were unable to run a separate series of focus groups with pre-service trainees (medical students and nursing students). On the other hand, our in-depth FGDs solicited input from two groups of HBB users with various experiences that are representative of HBB providers in Uganda, where there is a national mandate for all frontline providers to be trained in both HBB and ECEB. Participant experiences, perceptions, and input through the UCD process were likely generalizable to HBB providers in Uganda and translatable to similar resource settings. Recent clinical and simulation experience were similarly variable across both HBB facilitators and HBB providers, which allowed us to capture opinions that would reflect those with differential exposure to deliveries requiring HBB skills.

While our goal of soliciting input from different types of end-users uncovered many themes that were feasible to address with a mobile app, there are additional limitations to consider. App features addressed intrinsic factors such as motivation and confidence, but not health system constraints, which were frequently cited as barriers. Another challenge of UCD is balancing implementation of suggestions and prolonging the timeline of app development. For example, although Simulation—rater mode was felt to work well, having the ability to link two devices so that the user and the rater could engage in the simulation on different screens would have further enhanced its function but this was technically not feasible given the time constraint of the study. Although customization was requested during feedback, it was important to consider how to be specific enough without reducing generalizability. Keeping the app content close to the HBB curriculum would allow it to scale to other settings more easily.

Conclusion

With the ubiquity of mobile devices, apps that are carefully designed and evaluated have the potential to scale rapidly and reach a broad audience to effect large scale change. HBB Prompt was developed with UCD input from HBB trainers and providers to tailor its design to support knowledge and skills maintenance after HBB training. HBB Prompt features were built to address themes uncovered, which included motivation to practise, confidence in knowledge or skills, importance of feedback and peer-to-peer learning. The iterative process of incorporating feedback further improved the features and functions of HBB Prompt as a practice tool to facilitate skills and knowledge retention. Pilot-testing of HBB Prompt in Phase 2 of this study will further evaluate the ability of this UCD process to build a mobile tool to address the important issue of increasing sustainability of HBB training.