Decision making
Examining cognitive tasks in the emergency department
| Document
|
Author Nick Woodier, Paul Davis, Laura Pickup, Kathryn Whitehill & Robert Hutton |
| Abstract Applied Cognitive Task Analysis is an appropriate method to investigate challenging cognitive tasks and the role of expertise in healthcare contexts. Healthcare needs to support the accelerated development of decision-making skills in its novices and also create the optimum conditions in which to make decisions. |
“Taxiing down the runway with half a bolt hanging out the bottom”: affective influences on decision making in general aviation maintenance engineers
| Document
|
Author Kate Kingshott & Anjum Naweed |
| Abstract Maintenance engineers of aircraft in General Aviation work in a highly time pressured, complex and dynamic environment where errors in decision making could have far reaching consequences. However, few studies have investigated the role and influence of affect on the decision-making process in this setting. Using interviews and a scenario invention method, this study investigated the affective influences in decision making and corresponding responses in General Aviation maintenance engineering work in the Australian context. Preliminary findings based on inductive analysis identified a number of themes including affective response to task interruption specific to work colleagues and customers or management personnel, impact of negative rumination and the role of pride as a safety factor. Findings are discussed in terms of the impact of different affective states with implications for future research directions on crew resource training and non-technical skills development. |
Towards Using the Critical Decision Method for Studying Visualisation-Based Decision-Making
| Document
|
Author Setia Hermawati, Lena Cibulski and Glyn Lawson |
| Abstract Visualisations provide significant support for effective reasoning and decision-making processes. Its value mainly lies in its ability to turn raw data into actionable insights that lead to a decision. This requires appropriate visual representations that are designed with the decision-maker's way of reasoning in mind. Understanding the cognitive aspects underlying decision-making with visualisations is therefore crucial. Cognitive task analysis methods have been used to elicit expert knowledge in a variety of decision-making scenarios, with the Critical Decision Method (CDM) focusing on the cognitive bases in naturalistic non-routine incidents. In this study, we aim to determine the feasibility of CDM for capturing the expert knowledge, strategies, and cues involved with visualisation-based decision-making processes. Based on an analysis of four semi-structured interviews, we evaluate the method’s potential to inform the role of visualisation for human decision-making. We anticipate that our reflections on methodological insights can serve as a starting point for other human factors and visualisation researchers, who aim at studying strategies for higher-level decision-making and problem-solving tasks. |
Investigation of UK farmer risk perception and Non-Technical Skills
| Document
|
Author Ilinca-Ruxandra Tone & Dr. Amy Irwin |
| Abstract Livestock operations pose a high risk of injury and fatality in agriculture, especially for lone workers. In other high-risk industries, non-technical skills (NTS) are recognised as important for safe and effective task performance. However, dedicated research ought to be conducted to investigate how these findings apply to farmers, who are suggested to be highly risk tolerant. The current study used the vignette method to investigate farmer risk perception and risk management strategies, including NTS, in four types of cattle-handling risks related to self, equipment, environment, and animal characteristics. A preliminary sample of 50 farmers from the UK and Ireland was recruited through farming forums and organisational contacts to take part in an online qualitative study. Participants were presented with eight scenarios, two per category of risk, and asked to report their reasoning for proceeding or not and to detail any risk management strategies used. Thematic analysis was used to identify patterns. Farmers appeared to evaluate risk in the light of animal welfare and duty. Scenarios concerning faulty equipment and animal characteristics were perceived as too dangerous. Farmers reported using NTS such as task management, situation awareness, and decision-making to reduce risk. Farmers also considered facilities important for safe completion of livestock operations. These findings suggest that future interventions should aim to frame risk based on farmer priorities and to formally raise awareness about the importance of NTS. |
Using immersive simulation to understand and develop warfighters’ cognitive edge
| Document
|
Author Diane POMEROY, Justin FIDOCK, Luke THIELE and Laura CARTER |
| Abstract The Australian Army recognises personnel need a “cognitive edge” over any adversary. To better understand cognitive performance of military personnel in current and future land operating environments and inform training requirements, we have created an immersive tactical team simulator representing possible elements of the future operating environment, including novel use of technologies by adversaries. The most recent study analysed behaviours of two military teams, each consisting of a three vehicle platoon. Examination of individual and team strategies identified the decision making approaches adopted by the individual teams in response to novel and unexpected threats in a high tempo situation. |
Developing Foundation Pharmacist decision making skills: Covid-19 spotlights the need
| Document
|
Author David Gibson, Dominic Furniss & Helen Vosper |
| Abstract The role of the pharmacist is changing, moving from a product focus, centred on the medicine, to a model of delivering person-centred care through the safe and effective use of medicines. This requires the development of enhanced clinical skills. It is recognised that there are significant gaps in current educational programmes, leaving novice pharmacists feeling unprepared for their transition to practice. This situation has been exacerbated by the current Covid-19 pandemic. Of the enhanced clinical skills, one of the most difficult to teach is decision making: often complex and high stakes, it is recognised as one of the hallmarks of the expert practitioner. Despite the importance of this skill in underpinning safe and effective practice, relatively little is known about how experts make such decisions, and there is little support for novices. This case study describes the development of a reflective tool, informed by naturalistic decision making and based on the aviation model of Threat and Error Management. This encourages systems thinking to help novice pharmacists cope with the complexities of decisions relating to real life patient-centred care. |
Challenging the Limits of Cognitive Systems Engineering and Ecological Interface Design: Commander’s Cyber Situational Awareness
| Document
|
Author Rob HUTTON, Hannah BLACKFORD, Kevin BENNETT, Nigel JONES, and Ade FISHER |
| Abstract Military commanders are increasingly required to understand more than just the physical terrain. Understanding activities in cyberspace and their impact on operations presents a number of challenges for military personnel, tech-savvy or not. This paper presents a cognitive systems engineering approach to providing visualization solutions to support commander decision making. An Ecological Interface Design (EID) approach was used. Challenges for supporting cyber situational awareness are described. |
Leaps and Shunts: Designing pilot decision aids on the flight deck using Rasmussen’s ladders
| Document
|
Author Victoria A. Banks, Katherine L. Plant and Neville A. Stanton |
| Abstract When designing a new pilot decision aid for the flight deck, it is important to understand ‘how’ pilots make decisions in abnormal operating scenarios so that we can ensure they are provided with appropriate support. This paper provides a decision ladder analysis of an aircraft engine oil leak using data collected from six commercial airline pilot interviews. Traditionally, decision-making models are used reactively as a means to explore why things go wrong. However, we explore whether these models can also be used prospectively. Our analysis yields a number of possible design implications for the design of a pilot decision aid on the flight deck. |
Novice and Experts Strategies for Understanding Complex Big Data
| Document
|
Author Andreas REITER, Xianxu HOU, Genovefa KEFALIDOU, James GOULDING |
| Abstract Personal data is everywhere. Its complexity grows exponentially as more devices generate data. Understanding and making sense of complex data is fundamental as critical decisions may depend on its interpretation. In this lab-based observation study both novices and experts were exposed to complex medical information. The findings suggest that medical professionals employ different strategies from non-medics during sense-making and task completion. We discuss implications for designing new decision-making tools that support sense-making complex big data. |
Intensive care unit referrals: making decisions
| Document
|
Author Marta Weronika Wronikowska, Verity Westgate, Jody Ede, James Malycha, Lauren Morgan & Peter Watkinson |
| Abstract Referral to an Intensive Care Unit (ICU) is a complex medical process. The decision making involved can be cognitively challenging and subjective. We aimed to identify variables used by clinicians to make decisions during the ICU referral process, define the requirements for cognitive decision making and to detect commonly repeated errors. Applied Cognitive Task Analysis (ACTA) interviews were carried out with 17 doctors and nurses of varying specialties and levels of seniority to create a high-level task analysis of the participant’s role in the ICU referral process. Interviews were audio recorded, transcribed and analysed by two researchers in NVivo 11 software. We identified 188 variables used for clinical decision-making during an ICU referral. Removal of duplicates created 30 discrete variables. We found that there was not one key variable or piece of information that was significant to clinicians. Instead a ‘big picture’ approach was described, where all the data about a single patient was assembled and cognitively processed. ‘Often missed’ factors in the referral process were also identified. The most common was failure to consult family to discern patient wishes. The 30 variables used in the ICU referral process will inform the development of an interface for the Hospital Alerting Via Electronic Noticeboard project. This aims to identify patients at risk of deterioration in hospitals. Patient wishes were often neglected during the process and mechanisms to address this will form part of future work. We propose the addition of ‘F’ for ‘functional status/family’ to the ‘ABCDE’ acronym that is commonly used to evaluate a patient’s condition. |
Exploring team sensemaking with an adaptive report generation assistant
| Document
|
Author Robert J. Houghton and Chris Wragg |
| Abstract Collaborative interpretation and understanding of complex and uncertain information is a pervasive and growing challenge across many industries and domains from defence and ‘blue light’ services to commerce and government. We carried out two studies to evaluate the Adaptive Report Generation Assistant (ARGA), a piece of collaborative software designed to aid team sensemaking by supporting coding of information inputs and visualisation of outputs. In the first study, ARGA was contrasted with pen and paper processes in laboratory trials and in a second, and more ecologically valid trial, ARGA was contrasted with the use of generic shared electronic documents by two larger teams of expert analysts. In both cases, in addition to usability analysis and evaluation of final report quality, team activity was also analysed with reference to recordings, post-hoc interviews and examination of the cognitive artefacts produced. It was found that by structuring input and interpretation phases of the activity and offering greater flexibility in the rework of both ontologies for input and visualisations of output, groups using ARGA generally produced better quality analyses through avoiding premature fixedness and confirmation bias. However, a persistent problem across all groups lay in maintaining consistent visibility of relative information quality and credibility. The findings imply that sensemaking quality can be enhanced by interventions that reduce the administrative and clerical demands of information management and representation. |
“How can a horse be in two places at once?”: Group sensemaking using diverse and ambiguous information
| Document
|
Author Genovefa KEFALIDOU, Robert HOUGHTON |
| Abstract Sensemaking describes the activity of seeking to understand complex situations and extracting meaning from diverse, and sometimes conflicting, information. We report results from a study of groups attempting to construct narratives from diverse forms of fabricated incident information (reflective of the ever increasing diversity available on the internet) deliberately designed to be ambiguous. We found that groups demonstrated a rich range of convergent and divergent behaviours and manipulations of paper-based stimuli. We conclude by providing discussion of how these insights can be used in the design of web-based collaborative sensemaking tools. |
A framework for explainable AI
| Document
|
Author Chris Baber, Emily McCormick & Ian Apperley |
| Abstract The issue of ‘explanation’ has become prominent in automated decision aiding, particularly when those aids rely on Artificial Intelligence (AI). In this paper, we propose a formal framework of ‘explanation’ which allows us to define different types of explanation. We provide a use-cases to illustrate how explanation can differ, both in human-human and human-agent interactions. At the heart of our framework is the notion that explanation involves common ground in which two parties are able to align the features to which they attend and the type of relevance that they apply to these features. Managing alignment of features is, for the most part, relatively easy and, in human-human explanation, people might begin an explanation by itemizing the features they are using (and people typically only mention one or two features). However, providing features without an indication of Relevance is unlikely to provide a satisfactory. This implies that explanations that only present features (or Clusters of features) are incomplete. However, most Explainable AI provides output only at the level of Features or Clusters. From this, the user has to infer Relevance by making assumptions as to the beliefs that could have led to that output. But, as the reasoning applied by the human is likely to differ from that of the AI system, such inference is not guaranteed to be an accurate reflection of how the AI system reached its decision. To this end, more work is required to allow interactive explanation to be developed (so that the human is able to define and test the inferences and compare these with the AI system’s reasoning). |
How sensemaking by people and artificial intelligence might involve different frames
| Document
|
Author Hebah Bubakr and Chris Baber |
| Abstract Sensemaking can involve selecting an appropriate frame to explain a given set of data. The selection of the frame (and the definition of its appropriateness) can depend on the prior experience of the sensemaker as much as on the availability of data. Moreover, artificial intelligence and machine learning systems are dependent on knowledge elicited from human experts, yet, if we trained these systems to perform and think in the same way as a human, most of the tools will be unacceptable to be used as criterion because people consider many personal parameters that a machine should not use. In this paper, we consider how an artificial intelligence system that can be used to filter curriculum vitae (or résumés) might apply frames that result in socially unacceptable decisions. |
Learning from Expert Teams
| Document
|
Author Shaun C. Lamb, Mohammad Naiseh, Jediah R. Clark, Sarvapali D. Ramchurn & Timothy J. Norman |
| Abstract We studied the experimental adoption of an image classifying tool as an organisation plans the adoption within its teams of intelligence analysts. We identified that existing models of expert decision-making and function allocation can be employed to inform the design and adoption of these tools. |
Supporting decision making in a simulated air defence activity
| Document
|
Author Chris Baber and Chris Vance |
| Abstract A simulated air defence task is used to explore the impact of decision support on operator performance under different levels of task complexity. In this simulation, the status of autonomous air vehicles (drones) is indicated by their colour and by their threat level. Threat level is indicated by a polygon display which is automatically updated when drones fly into areas of attack or areas of risk. From two experiments, we can draw some tentative conclusions on the strategies that participants employ and the role that decision support might play in supporting or thwarting these. Contrary to instructions, participants did not always respond to the cue from the polygon display to engage (particularly when the number of drones was high, which resulted in more dynamic changes to the polygon display). Discussion with participants after the experiments suggested that some of them tried to ‘read’ the changes in polygon display in terms of possible paths that the drones were taking. From this, they might have attempted to anticipate when to respond and rely on their anticipation rather than the decision support. Situation awareness was rated lower when participants monitored two types of drone, and this was sufficient to lead to them performing at levels not far from chance. This is concerning in that it suggests that the decision support (in the polygon display) was not regarded as part of the situation awareness that the participants were using. It also raises the possibility of a difference between awareness of the situation and the awareness of decision options. |
Investigating Strategies in Rail Signalling: Comparison of Simulation Methods
| Document
|
Author Nora BALFE, Robert J. HOUGHTON, Jockman CHEUNG and Sarah SHARPLES |
| Abstract Two simulation methods used to study signaller decision-making strategies are described and compared through experimentation. The first method was a ‘static scenario’ method in which expert participants were presented with printouts from a simulated scenario and asked to give their strategy for routing the trains in the area. The second was a more traditional dynamic simulation undertaken on the same high fidelity simulator. The first method allowed greater control over the experiment and presents interesting opportunities for collecting additional qualitative data from participants, but the second method was more realistic and featured improved participant performance. |
