Background

Analytics is the process of developing insights or recommendations for actions from historical data (Sharda et al. 2013). Analytics represents a combination of management sciences, computer technology and statistical techniques to identify and solve real-life issues. While there are many views to analytics, in the health domain, there are four major components discussed: descriptive, prescriptive, diagnostic and predictive analytics (Banerjee et al. 2013). Descriptive analytics investigates what has happened and explores issues related to past events. Diagnostics analytics explores why an event has occurred and provides possible causes for such an occurrence. Prescriptive analytics provides reasons as to what should be done about what has happened or about an event occurrence. Predictive analytics discusses matters that are likely to happen.

The modern view of these four components of analytics may differ across interest groups, but these appear to be followed in many health organisations. While quantitative data has been used to anchor these views in the past, in recent years, especially in public health agencies, qualitative data may be used to complement quantitative data in decision-making processes. There appear to be two main reasons for the notion that qualitative data is also essential in major decision-making processes. The first reason is the involvement of many stakeholders commenting on various service quality provisions using social media, as well as internal data gathering forums. The second reason is that there is a general notion that the quantitative data collection process is quite restrictive and may not capture all aspects of stakeholder views.

While the involvement of many stakeholders is encouraging, there is also a dangerous possibility that the data could be diluted because of varying views, especially in unstructured forms of data collection. Therefore, assuring reliability and validity is essential. While many techniques, such as the reliability analysis, are available in quantitative data, researchers normally use ‘saturation’ as a reliability surrogate in qualitative data. We used analytics to provide reliability and validity in qualitative data analysis .

Health Text Analytics

Text analytics refers to deriving high-quality information from text, usually using methods that involve structuring of input text. For example, an unstructured data source can be made available to an application for analyses, and a set of arbitrary patterns based on keywords can be developed, resulting in an output. This process is called text analytics and typically involves tasks such as text categorisation, text clustering and concept extraction. While these processes have been followed in many text analyses, the major difference between text analysis and text analytics is the opportunity to unearth insights and extract value out of information. Unearthing insights requires a comprehensive understanding of the context, stakeholders, their emotional aspects, the problem or issue on hand and the ability to articulate these as part of the analysis leading to text analytics.

In this study, we applied text analytics to a set of interviews conducted in a public service agency in Queensland, Australia . The interviews were conducted in many wards with varying functionalities and mainly explored a technology implementation .

The scope of the study is restricted to Queensland Health (QH) and the implementation of a radically new technology called the ‘Patient Journey Board’ (PJB). The PJB is a touch-screen technology where patient data are consolidated so that carers such as doctors, nurses and allied health professionals can see the data in an unambiguous manner. Data entries are updated on a progressive basis following patient assessment to maintain accuracy and currency.

Methodology

As stated earlier, the scope of this chapter is restricted to qualitative data only. Data was collected from staff in QH wards involved in client care and focused on their behavioural patterns of acceptance and usage of technologies, as well as their opinion on the usage of these technologies. The participants were recruited specifically for this purpose. The recruitment and scheduling aspects were managed by QH through their regular processes.

The qualitative method employed in this study included semi-structured in-depth interviews to gain a sufficient understanding on the topic from QH professionals in their work settings. The aim of these interviews was to identify any unknown factors that may affect the adoption of technology.

In order to extract opinions about technology in specific domains such as QH, the selection of sample is crucial. It is important that the opinions expressed by QH professionals should be unbiased and pertaining to technology only—rather than the effects of technology on current workflows, especially within the scope of this study. The samples for this project consisted of four sets of people from QH. The first set comprised the senior managers who have oversight of strategic operations. The second set is ward managers who have oversight of tactical aspects. The third set is ward professionals who have oversight on operational aspects. The last set of samples included others and was comprised of people who did not fall under the first three categories.

While information systems research identifies a range of sampling techniques such as random and clustering, the sampling technique used for this study may be classified as ‘purposive’ sampling. The study was conducted with QH to meet their immediate needs. As the assessment is ‘real’, extreme care is needed to be exercised to preserve the integrity of QH values and standards. This warranted high levels of experience in conducting the assessment and professional knowledge about the assessment aspects. In order to assure completeness in assessments, this approach of ‘purposive sampling’ was followed in this study.

The instruments of this research consisted of two broad categories of questions. The first category related to the adoption and use of technology. The second category consisted of ‘influencing’ variables. The influencing variables are those that have an influence on the technology adoption but beyond the technology itself. For example, these variables can be organisational or procedural issues. Open-ended questions were included in the instrument to obtain unbiased and non-leading information. Prior to administering the questions, a complete peer review and a pilot study were conducted in order to ascertain the validity of the instrument.

Qualitative Data Analysis

The qualitative data analysis consisted of two phases. The first phase involved generating a set of initial analytics to determine the direction of extracting evidence, and the second phase involved providing reliability and validity .

In terms of initial analytics, the data was analysed in three different ways. The first set of analytics involved a word frequency cloud using the interview transcripts. The purpose of this analytic process was to assure that we explored relevant and appropriate themes, and the word cloud provided assurance through a keyword search. The second set of analytics was undertaken to develop a pictorial representation of the interview data, representing major themes explored. The interconnection between key themes and the links between the themes provided visual assurance that relevant and appropriate themes were considered in the interviews. The third set of analytics involved exploring the occurrences of keywords in terms of frequency distribution. This provided a pseudo statistical validity.

Word Frequency Cloud

The word frequency cloud is a technique in NVivo that documents the most frequently appearing words in a transcript. For the purpose of this study, the number of most frequent words was fixed at 100, and the transcripts of all interviews were run en bloc.

The procedure basically analyses words based on their occurrence, their distance and their context and develops a frequency table of occurrence and percentage. Once this was established, words that did not appear more than a certain arbitrarily chosen number in the overall context were assumed to be less significant in the overall context of the study and removed from the word-frequency-run procedure. This procedure was repeated many times to remove any words that did not make sense or add value to the analysis (e.g. words like ‘yes’, ‘I’, etc.). Figure 1.1 is a screenshot of the ‘word cloud’ to provide an initial viewpoint for further in-depth qualitative analysis.

Fig. 1.1
figure 1

Summary of visual aspects

Full size image

Using NVivo, researchers generated a word frequency map to assure that keywords were addressed in the interviews. The word frequency cloud shown in Fig. 1.1 indicates keywords extracted from interview transcripts and provides an initial direction for further study.

The word cloud demonstrates that many key concepts were included. For example, patient, board, ward, good, information, etc., were the words used by interviewees and represented the gamut of topics discussed. The topics were discussed with the interviewer (in this case, the researcher), and it provided another level of reliability to ensure that each participant had the same opportunity to discuss these terms. The proximity and the size of the words represent the importance and connection between keywords (Fig. 1.1).

The word cloud and word frequency table (Fig. 1.1) provided an initial path for further analysis as the cloud indicated that keywords were captured in the interview process, thus indicating ‘reliability’ . Furthermore, the research team conducted over 60 interviews, and the themes were found to be saturated around the twelfth interview, indicating that the qualitative process employed in this study was reliable and appropriate. In fact, after the sixth interview, most of the technology-related issues were saturated in the interview process, thus indicating a very high level of reliability.

Pictorial Representation of Interconnections

While the word cloud provides keywords and their relationships, it does not represent the major themes. In order to generate major themes, the research team used Leximancer, a text analytics application. The interview transcripts were submitted to this application, and the parameters were set for 1000 iterations so that major themes and their relationships could be understood. Figure 1.2 is a pictorial representation of what was accomplished.

Fig. 1.2
figure 2

Summary of the analytics application returned three major themes

Full size image

From Fig. 1.2, the analytics application returned three major themes—patients, nurses and people. The three circles are overlapping with almost the same perimeter thickness, indicating the equal importance of these three themes. Main issues within the themes are shown as ‘text’ labels, for example ‘time’ in the nurse circle indicates an issue among nurses. Similarly, labels like satisfaction , allied, care, etc., indicate the range of subthemes among the main theme discussed. The lines between these subthemes indicate relationships between the themes.

Keyword Occurrence

The keyword occurrence is a technique used in this study to understand how many times a keyword features during the interview as a way of assessing the importance of the keyword. NVivo produced a frequency distribution and allocated a weight to each keyword so that the strength of each keyword could be assessed. The following is a list of keywords extracted (Table 1.1):

Table 1.1 Summary of keywords occurrence
Full size table

EPJB electronic patient journey boardThe above analytics provided a glimpse of the total picture and highlighted trends to be explored in more detail. The purpose of conducting these three initial steps was to ensure that the research was exploring what it intended to explore and that interviews were conducted in a way so as to answer the main objectives established at the beginning of the study. These three analytics provided adequate reliability and confidence to further explore the text data (Fig. 1.3).

Fig. 1.3
figure 3

Summary of post-analysis themes

Full size image

The research team conducted manual analyses based on the word query facility within the software application to explore main themes. At this stage, interview transcripts were split into pre-technology implementation and post-technology implementation . The following two sections provide a summary of interview comments on the main issues identified during the text analysis process (Fig. 1.4).

Fig. 1.4
figure 4

Summary of post-analysis themes

Full size image

The research team also conducted a tree map to verify reliability of the themes in a pictorial manner.

Tree Map

The next stage in the data analysis process was generating a tree map. The purpose of the tree map was to ascertain that the themes that were extracted adequately met the objectives and were represented by the keywords. In generating the tree map, the visual provision indicates the weight of the keywords as shown by the size of the rectangle allocated to the keywords.

In the word frequency table (Table 1.1), prominent words depicting this study appear on the left side of the diagram. As shown, these words, while indicating frequency and relative importance, also indicate the main theme of the study. This was further confirmed with a cluster analysis as shown in Fig. 1.5.

Fig. 1.5
figure 5

Summary of post-analysis themes

Full size image

Results of the Analytics

Ease of Use

Well we used to have the electronic patient journey board (EPJB) as a whiteboard and that just replaced it, and it is much easier. They only have to—they’re probably not putting in so much as it’s pulling other information from other places, they’re not having to put in as much as they did before. They seem to be … I haven’t heard any complaints, there’s nobody whingeing or griping you know. It’s just having the time for them to do it and the EPJB just makes it convenient because it’s done on that main screen. It’s done with six types of a key, the dates in; it’s easy enough when they’re handing over like I said to do it as they’re speaking. I usually sit there in there of a morning and do it. They might say she’s not going home Wednesday now she’s been extended till next Friday so then you just enter that date and there’s just more consistency with it and if that’s what they want us to do as they hounded us for that to put in length of discharges or stays or whatever.

Time Savings

Yes, because what we did to reduce time was basically instead of each nurse handing over, we’ve gone to just handing over the whole ward. The oncoming TL is to have the EPJB up and click on the next name as they’re speaking and then they’re to go out of that handover and then they sight their patients. The TLs liked it from the TL handover sheet it showed them who the physios were already on, stuff like that.

Uptake

I’d probably say 70–80 %. I think they really like it, put it this way they wouldn’t like to go back to the old way of writing it up on the board.

Currency of Information

Before we had the EPJB we never had to enter discharge dates. We used to get in trouble for it and we would try to and you’d go in there and it’s locked by another user and it would be just be good if Trend could pull that information from the EPJB.

Old White Board Versus EPJB

Well how are they going to do it to if we’re to go back to the old white board it’s just so messy and so untidy and a lot of the time wasn’t getting updated but this way they have to update it because it’s a handover sheet do you know what I mean? It’s the patient’s journey basically.

Training Needs

No it doesn’t take long for them to learn because everything’s there. You’ve just got to tick things and you’ve got your drop down’s like if someone’s had a fall if it’s high medium or low and there’s forms to fill out which we teach them what to do. It all goes in the UR notes, everything’s there. It’s not like we have to write too much and that’s a good thing. If we don’t have to write then the nurses will do it. We’ve basically got train the trainer now. Enough of our staff can train the new ones

Positive Impact

The EPJB has made a huge difference to the impact on our length of stays and a huge impact on giving the staff a lot more knowledge as to where we’re at with our patients. They know where we’re up to with our patient care—they know that the physio’s seen them, they don’t have to go to the UR chart and flick through and say ‘Oh, I think a physio’s seen them’ whereas now we can look at the EPJB—‘physio’s seen them’ we know that for sure

Access

Yes that’s right because out on the ward they’ve got their computers and they can do exactly what they want and then it comes to me on here so I know who’s gone home and who’s going home tomorrow. So bed management—so if name says ‘we need some beds’ she can sit on there and look. I don’t have to get out of my chair to go and run and find the nurses and say ‘hey, what’s going on out here?’ It’s good having a comments section down here.

Attitude

I feel they’re coming to like it the more they use it, the more they realise it is a good system so I’d mark it probably another 8.

From the analytics and manual analysis of available data, we extracted the table of key factors (Table 1.2):

Table 1.2 Summary of comfort of the technology adoption
Full size table

Table 1.2 highlights that users confirmed the technology as being easy to use, reduces time and improves communications. When these three factors are viewed in unison, it is possible to assert that the first objective is accomplished.

Similarly, it is clear that users were satisfied with the technology, and the text data indicates a high level of acceptance . Therefore, it is possible to assert that the second objective has been accomplished.

From the interviews, users were not able to assert regarding the level of quality of services. While there were positive references to this effect, the research did not generate sufficient data to assert this objective.

Conclusions

The text analytics process was useful in establishing the reliability of data as well as for understanding the direction to be taken in conducting qualitative analysis of open-ended semi-structured interviews used to gather interview data. Due to the type of data being collected, it was difficult to collect data in a strictly unified way due to the volume and variation in user-functional aspects.

The research team used various techniques such as word cloud, frequency map and intelligence iterative analysis to guide the data analysis process. Further, text analytics helped the research team to verify whether the interviews were conducted properly and whether the interview data would adequately address the range of themes the research sought to explore.

With this initial direction established, NVivo was used to create themes and nodes and create other structures for analysing and categorising the data. These structures helped identifying the strength of the themes and exploring their inter-relationships. The research team did not expand further upon this aspect of analysis as it is adequately addressed with standard qualitative analysis. This chapter discusses and reports on the three analytic processes that were the focus of this particular research in assuring reliability of data collected.