Autonomous systems
Space Utilisation and Comfort in Automated Vehicles: A Shift in Interior Car Design?
Human-Centred Evaluation Approaches for Autonomous Agents: From Review to Practice
| Document | Author Benjamin Bowers, Catherine Harvey, Robert Houghton |
| Abstract The present research explores existing approaches and methods used to study and evaluate novel system components with increasingly autonomous capabilities, such as AI agents, in safety-critical domains. We report a large-scale review of the human–autonomy teaming (HAT) literature and extend the input-mediator-output model of team effectiveness to guide human-centred assessment, culminating in the IMO-A framework. We then test existing metrics and understand approaches to evaluation used by Human Factors researchers using a novel method which leverages an AI-generated video generation tool to develop underwater maritime scenarios across levels of autonomy. Our final recommendations serve as a roadmap for progressing HAT evaluation from fragmented, study-specific measurement choices toward a standardised, IMO-A guided, autonomy-appropriate and multi-method evidence base that can be translated into practitioner-ready early-phase HSI protocols for the safe integration of autonomous agents in safety-critical systems. |
Time added on: the impact of multiple AI teammates on referee decision-making
| Document | Author Maia Low, Jolene Cox, Brandon King, Scott McLean, Chris Baber, Paul Salmon |
| Abstract There is a critical need to understand how the increasing deployment of AI technologies within Human-Autonomy Teams (HATs) impacts performance in different contexts. We investigated the effect of HAT composition (Human-AI dyad versus Human-AI-AI triad) on football referee decision-making performance for foul or no foul decisions in a series of English Premier League match excerpts. The findings demonstrated that decisions took longer in the human-AI-AI triad condition but decision accuracy and confidence were not impacted by HAT composition. |
Clinician perspectives around automating the Emergency Department triage process
| Document | Author Katherine L. Plant, Beverley Townsend, & OlTunde Ashaolu |
| Abstract Healthcare has arguably been the sector most impacted by the Covid-19 pandemic, leaving Emergency Department (ED) medical teams overworked and understaffed. An automated system for ED triage has been developed to help alleviate some of these pressures. Eight ED clinicians were interviewed to capture their views of the automated system. Insights were generated around where this system might add value and areas of challenge or concern. These findings will be used to refine the prototype for end-user testing and support the development of training material for clinicians. |
Using immersive simulation to understand and develop warfighters’ cognitive edge
| Document | Author Diane POMEROY, Justin FIDOCK, Luke THIELE and Laura CARTER |
| Abstract The Australian Army recognises personnel need a “cognitive edge” over any adversary. To better understand cognitive performance of military personnel in current and future land operating environments and inform training requirements, we have created an immersive tactical team simulator representing possible elements of the future operating environment, including novel use of technologies by adversaries. The most recent study analysed behaviours of two military teams, each consisting of a three vehicle platoon. Examination of individual and team strategies identified the decision making approaches adopted by the individual teams in response to novel and unexpected threats in a high tempo situation. |
Introducing an Autonomous Crewmember
| Document | Author Helen MUNCIE |
| Abstract |
Factors Shaping the Passenger Experience in Advanced Air Mobility: Insights from Literature
| Document | Author Lamyea Ahmed, Robin Ward, Michael Bromfield, Sang-Hoon Yeo |
| Abstract Advanced Air Mobility (AAM) offers a new form of sustainable air transportation for passengers moving across urban and regional areas, making passenger experience a crucial factor in its societal adoption and commercial success. This paper reviews literature on key human factors influencing AAM acceptance, including motion comfort, ride quality, trust in pilotless operations and the role of in-flight information. Existing studies indicate that acceleration, vibration and attitude influence in-flight comfort, while research on autonomous mobility highlights the importance of system transparency and the communication of system state for trust. However, current literature shows a limited understanding of angular motion comfort, differences in ride quality across AAM vehicle types, and the effectiveness of auditory in-flight information for autonomous journeys. These gaps highlight clear opportunities for future motion-based experimental work to support passenger-centred AAM design. |
Summoning the demon? Identifying risks in a future artificial general intelligence system
| Document | Author Paul M Salmon, Brandon King, Gemma J. M Read, Jason Thompson, Tony Carden, Chris Baber, Neville A Stanton & Scott McLean |
| Abstract There are concerns that Artificial General Intelligence (AGI) could pose an existential threat to humanity; however, as AGI does not yet exist it is difficult to prospectively identify risks and develop controls. In this article we describe the use of a many model systems Human Factors and Ergonomics (HFE) approach in which three methods were applied to identify risks in a future ‘envisioned world’ AGI-based uncrewed combat aerial vehicle (UCAV) system. The findings demonstrate that there are many potential risks, but that the most critical arise not due to poor performance, but when the AGI attempts to achieve goals at the expense of other system values, or when the AGI becomes ‘super-intelligent’, and humans can no longer manage it. |
Using SUS for Current and Future AI
| Document | Author Richard Farry |
| Abstract The System Usability Scale (SUS) was assessed for its relevance and ease of use for assessing an AI capable of human-like interaction. Participants used SUS to assess Outlook, a contemporary consumer-grade AI interaction partners (smartphone digital assistants), and human teammates as a proxy ‘system’ for future human-like AI interaction partners. The results show that participants considered SUS to be relevant and easy to use for contemporary consumer-grade AI interaction partners, but not for human teammates. However, there was no meaningful difference in their ability to apply SUS between contemporary digital assistants, human teammates, and an email client. Thus, SUS can be used effectively for all of these kinds of systems. |
Drone Swarming – Unlocking the Potential of Human Swarm Teams
| Document | Author Siddharth Shyamsundar |
| Abstract Drone swarms represent a paradigm shift in human autonomy teaming while also bringing unprecedented advantages to civilian and military applications. While enabled by high levels of autonomy, human operators will continue to play a vital role, interacting with the swarm as a single entity, overseeing the mission and making key decisions. This paper discusses the human factors considerations associated with human swarm teams and introduces a bespoke human swarm teaming philosophy for a future drone swarming concept. Integrating autonomy and human information processing models, this concept of control allows the dynamic sharing of tasks between the autonomous swarm and the human operator while optimising key human factors considerations like situational awareness, workload, attention and fatigue. The first principles and designs of a novel human machine interface developed to implement this human swarm teaming philosophy while accounting for the real-world challenges associated with imperfect data transmission and beyond line-of-sight communications are also presented and discussed. |
Quantified minds: Predicting human functional state for human-machine teaming
| Document | Author Kate Ewing & Clare Borras |
| Abstract A new dawn of intelligent machines has re-energised the concept of human-machine teaming (HMT) whereby humans, and autonomous systems, collaborate towards a shared operational goal. Across Defence, Human Factors specialists will be challenged to integrate human-autonomy teams into already complex systems for which knowing the functional state of human teammates will be critical to system optimisation. Presently, innovation in machine learning and data collection methods is making human cognition more available to operational settings than ever before. This paper overviews the state of the art in techniques for estimating human functional state from the perspective of designing complex military systems involving artificially intelligent (AI) agents. Considerations are provided for designers seeking to quantify variables such as mental workload, situation awareness (SA) or the level of demand upon particular communication modes, whether for system operation or design and evaluation. Finally, some examples of methods used in HMT research are presented along with a speculative look at future influences upon the specification of human functional state for use with autonomy in Defence. |
Human Factors Guidance for Robotic and Autonomous Systems (RAS)
Manual control versus management-by-consent in managing multiple threats
| Document | Author Chris Baber & Natan S. Morar |
| Abstract As the use of Uninhabited Aerial Systems (UAS) increases, e.g., for commercial delivery, for surveillance or for hostile action, there are challenges of monitoring and appropriately responding to a crowded airspace. Providing automated support could reduce the challenges. However, such support might also have an impact on the strategy that a human operator deploys. In this paper we present a simulation of Air Defence (in the form of a single-player, interactive game) which is used to study human performance under three conditions. Provision of decision support, i.e., through management by consent, produced better performance, even though it provided a limited situation awareness. A hybrid display produces performance that is superior to the manual control condition and similar to the management by consent condition. We note that provision of the air picture alone resulted in a different form of suboptimal performance, in which sensitivity was significantly lower. In this respect, providing the decision support (in the form of the polygon display) helps to limit tendency for false alarms. |








