Keywords

1 Introduction –Why Is This Approach Different to Other Customer Experience Research?

Customer Experience (CE) research covers a wide variety of angles, all worthy but often not brought together for maximum impact. For example UI design testing on ecommerce sites is often concerned with optimising ordering sequences, but may not address the user journey when these orders go wrong. Similarly CE research in larger groups, via focus groups may give generic issues retrospectively – not reflecting “here and now” issues. The related field of Customer Relationship Management (CRM) research often involves processes and systems use in call centres [e.g. 1] – but how often is that tied into customer experience? At a macro level, text analysis of social media can provide hot issues in customer experience, and Net-Promoter Score surveys [2] provide quantitative clues to the same but lack specificity of findings. In the back- end, process mining and mapping gives information that describes what has gone wrong and how badly but doesn’t say what the effect on the customer actually is.

Given Duncan et al. (2013) argue that “most companies perform fairly well on touchpoints, but performance on journeys can set a company apart.” [3], there seems a paucity of methods able to connect qualitative process perspectives with quantitative process mining data on customer experience to help improve such customer “journeys”. Indeed Duncan et al. (2013) [3] sensibly recommend that “a company should draw on customer and employee surveys along with operational data across functions at each touchpoint, to assess performance”, though they acknowledge that this is a difficult endeavour as it requires the acquisition and integration of a large range of different types of empirical material, both qualitative and quantitative.

Our work seeks to achieve this by combining process mining with HCI, something discussed by Holziger [4], and Shneidermann [5] who underline the importance of integrating data mining with effective, usable data visualisation. In a business context, this would enable business managers and company strategists to control what data they are seeking so they can make decisions efficiently and adaptively.

Although a detailed analysis of the customer experience case study is detailed elsewhere (forthcoming) we here concentrate on documenting the adopted methodology as a novel way of enabling large service organisations to detect customer distress episodes and respond to them in a timely fashion, thereby minimising potential brand damage and maximising customer advocacy. Our contribution is thus methodological and practice, providing HCI researchers with a means of linking process mining and HCI, and providing practitioners with a means of detecting, addressing, and better understanding customer distress. We demonstrate our novel combination of process mining and qualitative research through the analysis of a case study of a problematic customer experience journey, and thus demonstrate the benefit of this combined approach. We first look at the approach to the case study which inspired our thinking.

2 Background

The authors of this paper were made aware of a particular customer’s journey via family contacts. It involved the customer’s problematic delivery of a super-fast broadband connection from a telecommunications provider. In the vast majority of these types of orders, such connections are provided with no problems whatsoever. However in this one exceptional case, a process event occurred in the organisation’s back-end, effectively cancelling the customer’s order – which the customer could not know about. When he was made aware of the situation, he struggled to get his service delivery back on track. Problems will always occur in large complex organisations but it is the customer’s experience of how the recovery is dealt with which is a key issue. Due to the duration and number of different representatives of the organisation who were involved in dealing with this customer’s journey it was clear that this case study was worthy of investigation, to understand how the interfaces between the customer and the company, the underlying delivery process, and the recovery process were sub-optimal and where they could be improved.Footnote 1

Table 1. The richness and detail of the data corpus for this one case study allowed the researchers to understand (a) the structured, process-driven recovery path (which was seen to fall onto a failure path on numerous occasions) against (b) how the process failures manifested themselves to the customer, and indeed how the various interfaces between the customer and organisation throughout the recovery process affected customer experience over time. It also informed the initial understanding of the data sources that would be needed in the model for detecting and responding to customer distress more generically. With the data sources in place we started to consider some preliminary questions before creating the model.
Full size table

The researchers had access to the company systems and the customer himself to accumulate a significant data corpus to analyse. The blended skills of the researchers (Information Systems and HCI/ethnography) were then fully deployed to analyse the case. Our theoretical approach reflected a process philosophy [6], and draws heavily upon Langley’s [7] strategies for theorising process data. Langley argues that “process research that incorporate narrative, interpretive and qualitative data are more immediately appealing for the richness of process detail they provide”. We drew upon this argument to believe that linking the often hidden process-data within the company’s data-warehouse with such narrative qualitative data would provide additional insight and address the arguably neglected process dimension of Customer Experience research.

3 Data Corpus for the Case Study

The data corpus for the case study comprised quantitative and qualitative sources and analysis techniques as outlined Table 1 below:

4 Using CRM Notes and Customer Feedback to Identify Distress Indicators

While creating the model, one of the questions we considered was: what indicates customer distress? To approach this we combined CRM system analysis with direct customer experiences (expressed through the customer interview and his own notes) to identify system and process-related issues which manifest themselves in customer distress. For example, drawing lessons from our case study, relevant Distress Indicators included:

  • The length of time an order has been delayed exceeding an acceptable threshold (taking the original scheduled delivery date as a start-point).

  • The number of times the customer has had to call the organisation beyond an acceptable range (e.g. between 0 and 1).

  • The number of different reference numbers given for a customer case, (causing distress by provoking confusion) above an acceptable range (e.g. 1 is acceptable, 2 causes confusion etc.,)

  • The number of different phone numbers given to a customer for different departments throughout a customer journey (again causing distress by provoking confusion) above an acceptable range.

  • The subjective record of the emotional state of the customer, as recorded after an interaction with an advisor and sometimes (but not always) captured in the advisor notes.

The advantage of deriving Distress Indicators quantitatively from the CRM system notes alone, is that they are easily accessible for analysis. However, when they are combined with post-interaction survey results and advisor notes, they can give a more powerful indication – from the system, the advisor and the customer himself - that a distressful episode is taking place and a response is needed.

5 Using Process Mining to Understand Causal Factors for Distress

Again, while creating the model we asked: what can we analyse from the process records to help us locate what causes customer distress? For this, we looked at Process Mining, because, according to Aalst [10], it provides “comprehensive sets of tools to provide fact-based insights and to support process improvements”. It is used to reflect actual events, taken from log files – often across disparate systems - so that comparisons can be made with the process model (i.e. the model of the process in its ideal state) [11]. Our case study, which concerns the recovery of an order which has deviated from the “happy path” of a standard delivery process, gave us an example where process mining enabled the researchers to make this comparison, plus compare where subjective customer experience issues occurred in parallel.

We used a process mining tool developed in-house (called “Aperture” [12]) to generate a process “map” from a number of data sources (events from the CRM system, for example) for the case study (see Fig. 1). We referred to this when determining the process-related distress-inducing causal factors. From doing this, we found the process event which represented a “turning point” in the journey, resulting in the order falling from the happy path into a recovery path.

Fig. 1.
figure 1

Process map of customer journey to identify process events - excerpt

Full size image

Figure 1 shows the results of the “turning point”, as derived from the system logs at this early stage in the customer journey. Note the repeated inbound calls from the customer (indicated by stars in Fig. 1.) and re-work which follows. (This is only part of the entire process map, to give an example of the visualisation and usage of the tool).

In further analysis, the process map also showed us where some automated processes were still continuing (automated communications to the customer for example) despite delivery processes being halted or delayed in the back-end. And it was also possible to see on the map where the customer was responding to these communications. Drawing parallels between this map, the CRM system notes and the customer’s subjective experiences of when these process events occurred gave a powerful and holistic picture of customer experience episodes which caused distress – from the customer’s and the organisation’s perspective. Finally, in collaboration with our engineering teams, we could compare this “actual” state with an ideal process state to see where assumptions in the process were incorrect and needed remedial action.

6 Introducing the Model for Detecting and Responding to Customer Distress

The proposed model above summarises the data sources and steps involved in the Detecting Phase and proposes the outline areas for categorising the resulting responses identified (i.e. the recommendations) in the Responding Phase. This categorisation of recommendations into “People”, “Process” and “System” can assist the organisation in prioritising interventions and applying resources to deliver the improvements suggested (whilst being aware that in some scenarios these three areas may need to be addressed together) (Fig. 2).

Fig. 2.
figure 2

Proposed model for detecting and responding to customer distress

Full size image

Note also within the model the Feedback Loop, which can be used for cost-benefit modelling of any interventions. This is especially true if investment is needed to make the improvements suggested. The model could be used to project likely reductions in the incidence of Distress Indicators, e.g. improved automated communications (Response) can lead to a reduction in number of calls a customer makes to chase the order (Distress Indicator). Improvements can be tactical, or strategic, but all can and should be continually measured so that the Feedback Loop of improvement can be sustained.

Equally, post-intervention success measures can be gained by taking measures of individual elements, but aggregated to give an entire picture of improved customer service and reduced customer distress by quantitative and qualitative means, e.g.

  1. 1.

    From the Customer’s perspective: customer satisfaction scores improve for individual customers and overall (using customer survey scores)

  2. 2.

    From the Advisor’s perspective: subjective measures (“how easy is it to do your job?” etc.) and quantitative efficiency measures (throughput, productivity etc.).

  3. 3.

    From the Process perspective: Using process mining, quantifying the reduction in causal factors, and quantifying the number of repeated actions throughout the process.

Fig. 3.
figure 3

Detecting phase - core issue and causal factor identification

Full size image

7 Detecting and Responding to Customer Distress - the Case Study

In this section we demonstrate how the model contributed to a set of recommended improvements resulting from the detection of one particular core issue. (Incidentally, it was evident in this case study that causal factors often acted together to cause overall customer distress. Similarly, the responses identified were inter-related and interventions needed to be considered collectively).

Figures 3 and 4 demonstrate how the sequence of activities and data sources were used to identify a core issue, how the causal factors of the issue were identified and what responses (recommendations for improvements) were identified as a result:

Fig. 4.
figure 4

Responding phase (Recommendations)

Full size image

8 Next Steps

The application of the model to the case study is still in its infancy at the time of writing – some short-term tactical changes (e.g. html content and UI improvements on advisor KM tools) have been implemented, with longer-term interventions in the pipeline. The next immediate step therefore is to test the efficiency of the model against other individual problematic customer journeys. We also consider approaching the concept of detecting and responding to customer distress within different dimensions, namely time and scale.

Time. Our analysis revealed the significance of time as an actor within the case study, and suggests a need for temporality to be considered in greater detail (reflecting a call of [13, 14]): in this case study, the customer became distressed at various points, with the longevity of the case itself being a cause of distress. Further, polychronicity [15] (the doing of multiple things at the same time) within the CRM system was, through our process analysis, revealed as significant in creating distress as the customer received conflicting messages from different systems which were effectively out of sync with the evolving situation. The “time-map” of the customer journey to demonstrate distress episodes is worthy of further investigation.

Scale. In this case study we analysed one distressed customer’s journey among a base of orders that are being case-managed or are going through the complaints procedure. Rather than seeking to capture a quantitative representation of these large numbers of orders, we adopted a single case study approach which allowed much richer analysis and revealed lessons for wider cases. Although this approach was valuable, there may be possibilities in increasing the scale, i.e. to make the model apply across a larger customer base. For this more automated approaches (such as text analytics) will need to be considered (see Discussion).

For our case study, we were fortunate to have a customer who was willing and able to share his experiences with us in great detail. For the current model, we propose the use of surveys and advisor notes where very detailed qualitative customer research is not practical due to the effort involved in collecting and analysing data. But a next step may be to supplement the surveys and notes data via qualitative in-depth interviews with a smaller number of specific customers after problematic cases. In addition, where Distress Indicators are flagged in particular journeys it may be possible (with privacy considerations) to identify customers with problems early on and engage researchers into the system with the same conditions/problems who must then follow the same journey, documenting it as they progress to provide insight into the customer’s experience of that journey.

9 Discussion

Our model is intended to provide a framework for organisations with a means of linking process mining and HCI techniques, to enable better detection and response to customer distress episodes. We accept that this is not without its challenges, however. For instance, use of the model depends on the skills and availabilities of the practitioners to carry it out, leading to questions such as: Is it better to deploy one researcher with passable knowledge of process mining and qualitative interviewing (for example) to carry out as many steps as they can, or deploy many more researchers who are experts in their individual areas? Would time and resources allow for the latter? Would they be able to work together effectively to provide effective output? For the former, would the output be deep enough to produce valuable insight? But would this approach be cheaper and quicker?

Similarly, engagement with the development community, tasked with deploying recommendations from use of the model, should be done at an early stage, to provide governance and balance the recommendations with business objectives, especially if the recommendations are costly.

In the future, technology developments may be deployed to relieve the workload experienced by the researchers in carrying out the manual steps in this particular case study. Text analytics for example can used to summarise core issues from the vast quantities of free-text in customer communications, interview verbatims and advisor notes. Additionally, data visualisation techniques can be deployed such that business decision makers can decide how to respond to customer distress episodes via an intuitive interface which gives them what they need to respond in shorter time-frames. We hope that our model can evolve to incorporate these developments over time, and that other organisations can use it as a baseline for their own business needs.