Skip to main content
Log in

User and design requirements and production of evidence: using incident analysis data to (1) inform user scenarios and bow ties, and (2) generate user and design requirements

  • Original Article
  • Published:
Cognition, Technology & Work Aims and scope Submit manuscript

Abstract

This paper reports on an innovative human–machine interaction methodology adopted to assess the case, role and requirements for a new ground collision awareness technology. Specifically, this paper reports on the analysis of ground collision incident data and the subsequent advancement of user scenarios and bow-ties based on this data analysis, for the purpose of generating preliminary user and design requirements for this technology. In so doing, the requirements elicitation and validation methods used in this research are framed from an epistemological perspective. Accordingly, the particular methods adopted are presented and discussed in terms of concepts of evidence, bearing witness and the distinction between facts and values. As such, this paper promotes thinking about evidence-based design practices. Overall, this evidence-based approach aims to improve the development of scenarios and associated problem solving around technology cases, user requirements and user interface design features. The proposed method is useful in terms of bridging the gap from data analysis to design, and validating design decisions. In this regard, it is argued that the generation of user scenarios based on the analysis of incident data (i.e. data coding and statistical analysis), and the reframing of such scenarios in terms of bow-ties for the purpose of requirements/design envisionment, extends existing scenario-based design approaches. Although the use of bow-ties is not new, the advancement of bow-ties from data-driven scenarios is. Specifically, the bow-tie method was applied in a design context, to support problem solving around design decisions, as opposed to formal risk analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

(Source: https://www.cgerisk.com/knowledgebase/The_bowtie_method)

Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Baranzini, Christou MD (2010) Human factors data traceability and analysis in the European Community’s Major Accident Reporting System. Cognit Technol Work 12(1):1–12. https://doi.org/10.1007/s10111-009-0129-4

    Article  Google Scholar 

  • Bødker S, Burr J (2002) The design collaboratorium. A place for usability design. ACM Trans Comput Hum Interact 9(2):152–169

    Article  Google Scholar 

  • Boeing (2016) http://www.boeing.com/resources/boeingdotcom/company/about_bca/pdf/statsum.pdf. Accessed 7 July 2017

  • Cacciabue CP (2004) Guide to applying human factors methods: human error and accident management in safety critical systems. Springer, London

    Book  Google Scholar 

  • Cahill J (2014) Human factors research in the wing watch project. In: Proceedings of the preventing aircraft damage conference, Dublin

  • Cahill J, Butler W, Redmond P (2008) Identifying the human factors requirements and associated human computer interaction design concept, for a proposed collision avoidance system, for use by commercial pilots operating on the airport ramp and in taxiway areas. In: Proceedings of the IHCI conference, National University of Ireland, Cork

  • Cahill J, Redmond P, Yous S, Butler W (2010) Human–computer action design (HCI) methods supporting the envisionment, design and evaluation of a collision avoidance system. Poster presented at IHCI 2010 conference, Dublin City University (DCU), Dublin

  • Cahill J, Redmond P, Yous S, Lacey G, Butler W (2012) The design of a collision avoidance system for use by pilots operating on the airport ramp and in taxiway areas. Cognit Technol Work 15(2):219–238

    Article  Google Scholar 

  • Carpignano A, Piccini M (1999) Cognitive theories and engineering approaches for safety assessment and design of automated systems: a case study of a power plant. Cognit Technol Work 1(1):47–61. https://doi.org/10.1007/s101110050010

    Article  Google Scholar 

  • Carroll JM (ed) (1995) Scenario-based design: envisioning work and technology in system development. Wiley, New York

    Google Scholar 

  • Carroll JM (2000) Making use: scenario-based design of human–computer interactions. MIT Press, Cambridge

    Book  Google Scholar 

  • Civil Aviation Authority (2017) Where did bowtie come from? Downloaded from: https://www.caa.co.uk/Safety-initiatives-and-resources/Working-with-industry/Bowtie/About-Bowtie/Where-did-bowtie-come-from-/ 25 Oct 2017

  • Constantine LL, Lockwood LAD (1999) Software for use: a practical guide to the models and methods of usage-centered design. Addison-Wesley, Boston

    Google Scholar 

  • Cooper A (2007) The inmates are running the asylum, 7th edn. SAMS Publishing, Indianapolis

    Google Scholar 

  • Cousins JB, Whitmore E, Shulha L (2013) Arguments for a common set of principles for collaborative inquiry in evaluation. Am J Eval 34(1):7–22

    Article  Google Scholar 

  • Dekker S (2002) The field guide to human error investigations. Cranfield University Press, Bedford

    Google Scholar 

  • Dodd J, Stern-Gillet S (1995) The is/ought gap, the fact/value distinction and the naturalistic fallacy. Dialogue 34(4):727–746

    Article  Google Scholar 

  • Dowell J, Long J (1998) Conception of the cognitive engineering design problem. Ergonomics 41(2):126–139

    Article  Google Scholar 

  • Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors J Hum Factors Ergon Soc 37:32–64. https://doi.org/10.1518/001872095779049543

    Article  Google Scholar 

  • Evidence (2017) In OxfordDictionaries.com. Retrieved from https://en.oxforddictionaries.com/definition/evidence. Accessed 7 July 2017

  • Fact (2017) In OxfordDictionaries.com. Retrieved from https://en.oxforddictionaries.com/definition/fact. Accessed 7 July 2017

  • Goodwin K (2011) Developing effective scenarios. Retrieved from https://www.uie.com/brainsparks/2011/08/05/kim-goodwin-developing-effective-scenarios/. Accessed 7 July 2017

  • Hackos JA, Redish JC (1998) User and task analysis for interface design. Wiley, New York

    Google Scholar 

  • Hollnagel E (2004) Barrier analysis and accident prevention. Ashgate, Aldershot

    Google Scholar 

  • Hollnagel E (2012) Functional resonance analysis method: modelling complex sociotechnical systems. Ashgate, Aldershot

    Google Scholar 

  • Hollnagel E (2014) Safety-I and safety-II: the past and future of safety management. Ashgate, Farnham

    Google Scholar 

  • Hollnagel E, Woods DD (1983) Cognitive systems engineering: new wine in new bottles. Int J Man Mach Stud 18:583–600

    Article  Google Scholar 

  • Hollnagel E, Wears RL, Braithwaite J (2015) From safety-I to safety-II: a white paper. The resilient health care net: published simultaneously by the University of Southern Denmark, University of Florida, USA, and Macquarie University, Australia

  • Idoughi D, Seffah A, Kolski C (2012) Adding user experience into the interactive service design loop: a persona-based approach. Behav Inform Technol 31(3):287–303

    Article  Google Scholar 

  • International Civil Aviation Organisation (2010) Annex 13: aircraft accident and incident investigation

  • John BE (2005) Evidence-based practice in human–computer interaction and evidence maps. ACM SIGSOFT Softw Eng Notes 30:1–5

    Google Scholar 

  • Johnston N (2003) Expertise, skills, habits and repertoires. Outline presented at a meeting of the aerospace research psychology group, Dublin

  • Kontogiannis T, Malakis S (2017) Cognitive engineering and safety organization in air traffic management. CRC Press, Boca Raton

    Book  Google Scholar 

  • Mayhew DJ (1999) The usability engineering lifecycle. Morgan Kaufmann Publishers, Burlington

    Book  Google Scholar 

  • McDonald N (2000) SCARF: safety courses for airport ramp functions, Trinity College, Dublin. http://www.tcd.ie/psychology/aprg/scarf.html 14 Sept 2001

  • McDonald N (2002) The development and evaluation of SCARF—and what happened after. In: Proceedings of the managing risk: the flight-ground safety interface, human factors training and tools conference, Melbourne

  • Niwa Y (2009) A proposal for a new accident analysis method and its application to a catastrophic railway accident in Japan. Cognit Technol Work 11(3):187–204. https://doi.org/10.1007/s10111-008-0112-5

    Article  Google Scholar 

  • Niwa Y, Takahashi M, Kitamura M (2001) The design of human–machine interface for accident support in nuclear power plants. Cognit Technol Work 3(3):161–176. https://doi.org/10.1007/PL00011531

    Article  Google Scholar 

  • Norros L, Salo L (2009) Design of joint systems: a theoretical challenge for cognitive systems engineering. Cognit Technol Work 11(1):43–56. https://doi.org/10.1007/s10111-008-0122-3

    Article  Google Scholar 

  • O’Hare D (2006) Cognitive functions and performance shaping factors in aviation accidents and incidents. Int J Aviat Psychol 16(2):145–156

    Article  Google Scholar 

  • O’Sullivan RG (2012) Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches. Eval Program Plan 35:518–522

    Article  Google Scholar 

  • Pawson R (2006) Evidence-based policy: a realist perspective. Sage, Newcastle upon Tyne

    Book  Google Scholar 

  • Preece J, Rogers Y, Sharp H (2007) Interaction design: beyond human–computer interaction, 2nd edn. Wiley, West Sussex

    Google Scholar 

  • Pruitt J, Grudin J (2003) Personas: practice and theory. In: Proceedings of the 2003 conference on designing for user experiences

  • Reason J (1990) Human error. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Reason J (1995) A systems approach to organizational error. Ergonomics 38(8):1708–1721

    Article  Google Scholar 

  • Reason J (2000) Human error: models and management. BMJ 320:768–770

    Article  Google Scholar 

  • Rheinboldt P (2017) The bowtie techniques. http://www.convencionminera.com/perumin31/encuentros/tecnologia/jueves19/0910-Paulo-Rheinbolt.pdf. Accessed 7 July 2017

  • Rosson MB, Carroll JM (2001) Usability engineering: scenario-based development of human-computer interaction (interactive technologies) 1st edition. Morgan Kaufmann, Burlington, Massachusetts, United States

    Google Scholar 

  • Rosson MB, Carroll JM (2002) Scenario based design. In: Jacko J, Sears A (eds) The human–computer interaction handbook: fundamentals, evolving technologies and emerging applications. Lawrence Erlbaum Associates, Mahwah, pp 1032–1050

    Google Scholar 

  • Shappell SA, Wiegmann DA (1997) A human error approach to accident investigation: the taxonomy of unsafe operations. Int J Aviat Psychol 7(4):269–291. https://doi.org/10.1207/s15327108ijap0704_2

    Article  Google Scholar 

  • Shappell S, Wiegmann D (2001) Applying reason: the human factors analysis and classification system. Hum Factors Aerospace Saf 1:59–86

    Google Scholar 

  • SKYbrary Aviation Safety (2017) Retrieved from http://www.skybrary.aero/index.php/Threat_and_Error_Management_(TEM)_in_Flight_Operations. Accessed 7 July 2017

  • Ulrich R, Zimring C, Xuemei Z, DuBose J, Seo H, Choi Y, Quan Xiaobo, Joseph Anjali (2008) A review of the research literature on evidence-based healthcare design. HERD 1(3):61–125

    Article  Google Scholar 

  • Valent P (2013) Bearing witness. Retrieved from http://www.paulvalent.com/wp-content/uploads/2013/02/Bearing-Witness-Enc-Tr-21.pdf. Accessed 7 July 2017

  • Values (2017) In Businessdictionary.com. Retrieved from http://www.businessdictionary.com/definition/values.html

  • Weaver R, Despotou G, Kelly T, McDermid J (2005) Combining software evidence: arguments and assurance. In: Proceedings of the 2005 workshop on realising evidence-based software engineering (REBSE ‘05). ACM, New York, pp 1–7

  • Webster L, Steinke C (2009) Evidence-based design: a new direction for health care. Design Quarterly, Winter 2009

  • Weener E (2007) Ground accident prevention—the foundations answer. In: Proceedings of the EASS conference, Amsterdam

  • Wiegmann DA, Shappell SA (2003) A human error approach to aviation accident analysis: the human factors analysis and classification system. Ashgate Publishing Ltd, Burlington

    Google Scholar 

  • Wilson PF, Dell LD, Anderson GF (1993) Root cause analysis: a tool for total quality management. ASQC Quality Press, Milwaukee

    Google Scholar 

  • Yoon YS, Ham DH, Yoon WC (2017) A new approach to analysing human-related accidents by combined use of HFACS and activity theory-based method. Cognit Technol Work 19(4):759–783. https://doi.org/10.1007/s10111-017-0433-3

    Article  Google Scholar 

Download references

Funding

The study was funded by Boeing.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Joan Cahill.

Appendices

Appendices

1.1 Appendix 1: Conceptualising contributory factors

See Table 6.

Table 6 Conceptualising contributory factors/levels

1.2 Appendix 2: Research questions and categories

See Table 7.

Table 7 Research questions and categories

1.3 Appendix 3: Scenario 2: preventable by SOCAS, aircraft/vehicle

1.3.1 Data analysis findings

Table 8 defines the relevant findings from the data analysis pertaining to this scenario.

Table 8 Aircraft/vehicle

1.3.2 Scenario narrative

It was late afternoon at a busy regional airport in Europe. It was a summer’s day, and it was hot. There was much activity on the ramp and taxiway areas. Visual conditions were good. Pushback was complete and the crew had received a clearance from Ground Control to commence their taxi. The instruction was to taxi to runway X using taxiway Y. The crew completed their pre-taxi checks and proceeded to taxi. As such, the Flight Crew were responsible for collision detection and avoidance.

The turnaround time had been short and the crew felt under pressure.

An engineer working for the airport authority had just performed a routine inspection of the runway. The engineer was sitting in the driver seat of a patrol vehicle and working alone. At the time, he was wearing a headset and listening to ground control instructions. The engineer/vehicle driver had previously been cleared to return to the base/ramp, using taxiway X. It was the end of a long shift, and he was looking forward to finish his days’ work.

At the time of the incident, the aircraft was on the ramp, about to enter the designated taxiway, and the vehicle had just moved off taxiway X and entered the ramp/apron area. Shortly before approaching the ramp (and exiting the taxiway), the vehicle driver felt thirsty. He reached for a water bottle which was located on the passenger seat. The vehicle driver proceeded to take a drink of water. The vehicle driver was momentarily distracted (loss of situation awareness) and failed to see/notice the approaching aircraft. The crew did not see the vehicle. The vehicle passed under the forward fuselage of the aircraft. This resulted in substantial damage to the aircraft fuselage. There were no injuries or fatalities.

1.4 Appendix 4: Scenario 3: preventable by SOCAS, aircraft/obstacle

1.4.1 Data analysis findings

Table 9 defines the relevant findings from the data analysis pertaining to this scenario.

Table 9 Aircraft to obstacle

1.4.2 Scenario narrative

It was early morning at a busy regional airport in Europe. The particular airport is known to have a complex airport layout and taxiway procedure—with many interconnected taxiways, several remote stand areas and two terminal buildings. Also, there is currently ongoing building work at the airport—in relation to the newly opened second terminal building.

The crew had flown many times together, but were not familiar with the airport.

The flight had arrived with a delay due to a large amount of en route traffic requiring a hold prior to arrival.

The crew had been previously cleared to taxi to parking gate 20 via taxiway X. However, there was a delay due to another aircraft exiting parking gate 20, and the crew were required to hold on the taxiway until the parking gate was clear. After a few minutes, the crew were requested to taxi to parking gate 24 via taxiway Y. Shortly before recommencing the taxi, the crew reviewed the airport taxi charts and briefed on the new taxi procedure.

The crew proceeded with to taxi to parking gate 24, in line with the new ATC instruction. However, there was some confusion/disagreement between the crew over the correct taxiway to take—given the earlier changes to the parking gate and taxiway number. The captain/PF asked the First Officer/PM to contact Ground Control to confirm that they were cleared to taxi to parking gate 24 using taxiway Y. At this time, the PF had his head down (double checking the taxiway charts and taxiway names/numbers), while the PM was on the VHF radio talking with Ground Control. The crew were not surveying the situation ahead and were under considerable stress.

At the time of the incident, visual conditions were good (good visual conditions) and there was much activity on the ramp and taxiway area.

Also, there was a large number of objects and vehicle activity around the parking stand owing to the construction work being carried out on the airfield.

The parking procedure involved using an automatic guidance system. On approaching the assigned stand, the captain who was handing the taxi identified the correct stand and that the Automatic Guidance System had the correct aircraft type and started following the AGS’s instructions. The captain became fixated on the AGS and failed to identify that the jet bridge would be at risk of impacting the aircraft in its current position. The captain/PF asked the First Officer/PM to monitor the outside of the aircraft for collision risks. However, the FO had limited view points to the left side. The PF was primarily fixated on the AGS system to confirm that they were cleared to taxi using taxiway X. Both crew members were distracted and did not notice the air bridge to the left of their position. During the parking the jet bridge hit the side of the front fuselage resulting in a tear with the jet bridge. This resulted in substantial damage to the aircraft. There were no injuries or fatalities.

1.5 Appendix 5: Data analysis (collisions and flight phase)

See Table 10.

Table 10 Proportion of collision events occurring in each flight phase

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cahill, J., Geary, U., Douglas, E. et al. User and design requirements and production of evidence: using incident analysis data to (1) inform user scenarios and bow ties, and (2) generate user and design requirements. Cogn Tech Work 20, 23–47 (2018). https://doi.org/10.1007/s10111-017-0457-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10111-017-0457-8

Keywords

Navigation