User centred design
How well can generative AI design and evaluate user interfaces?
| Document
|
Author Zhenyuan Sun and Chris Baber |
| Abstract The inexorable rise of generative artificial intelligence (GenAI) is threatening a range of work domains. In this paper we explore whether user interfaces produced by GenAI compare with those produce by humans, and whether gen-AI can evaluate user interfaces to a human-like standard. We create user interface designs for a burger ordering app using prompts to Midjourney on Discord, DALL-E 3 on ChatGPT4o, and Stable Diffusion 3 on Stable Assistant. All three GenAI apps had problems with legible text and following prompts provided. However, through adjustment of prompting, DALL-E 3 and Stable Diffusion 3 produced viable designs that met the brief. We compared the resulting designs with commercial products and the designs created by 8 competent (human) user interface designers through a survey of 32 participants evaluating the designs using the UEQ-S. We found no difference in pragmatic quality between designs, but the designs from gen-AI were rated significantly higher on hedonic quality than those from the commercial products or human designers (with the commercial apps having lowest ratings on all measures). We then prompted ChatGPT4o and Stable Assistant using the UEQ-S to evaluate the user interface designs. We found little correlation between the ratings of the gen-AI apps and human raters. This suggests that GenAI might have a place (with appropriate prompt engineering) in generating user interface designs but that, at present, it struggles to produce reliable, human-like evaluation of these. |
Improving Referrals to Tertiary Care – Oxford Acute Referral System (OARS)
| Document
|
Author Matthew WOODWARD, Nick DE PENNINGTON, Lauren MORGAN |
| Abstract |
Comparing User Interface Designs for Explainable Artificial Intelligence
| Document
|
Author Ionut Danilescu & Chris Baber |
| Abstract A well-known approach to Explainable Artificial Intelligence (XAI) presents features from a dataset that are important to the AI system’s recommendation. In this paper, we compare LIME (Local Interpretable Model-free Explanation), to display features from a classifier, with a radar plot, to show relations between these features. Comparative evaluation (with N = 20) shows LIME provides more correct answers, has a higher consistency in answers, and higher rating of satisfaction. However, LIME also showed lower sensitivity (using signal detection), a slightly more liberal response bias, and had a higher rating of subjective workload. Evaluating user interface designs for XAI needs to consider a combination of metrics, and it is time to question the benefit of relying only on features for XAI. |
People Oriented Smart Towns
| Document
|
Author Barry Peter Kirby, Amanda Georgina Kirby & Jim Wilson |
| Abstract The growth of interest in Smart Cities and development into Smart Towns and Smart Communities, has been driven by the technology development, however much of the data developed is not being utilised because it is not clear on its value. The People Oriented Smart Towns (POST) methodology has been developed to put user requirements at the front of Smart developments. This paper describes how that process is being applied to Kidwelly in South Wales and describes the outcomes so far. |
Integration of personas in transport policymaking
| Document
|
Author Phuong Anh Nguyen, Robert Houghton, Amanda Crompton & Sarah Sharples |
| Abstract There is increasing interest in the concept of human-centred design (HCD) for policymaking and the application of Human Factors and Ergonomics (HF/E) methods in this area. In this study, we explored perceptions of a specific method, mainly personas, in the transport policy space. This research aim was accomplished through sixteen interviews with transport policymakers and analysts working in both national and local government, employing critical decision methods and thematic analysis to examine the transcripts. |
Introducing contactless technology into the financial self-service environment
| Document
|
Author Elina Jokisuu, Eleanor Forrest, Maggie McKendry, Marshall Munro & Phil Day |
| Abstract The use of contactless technology has grown; this led to a development to integrate it on an automated teller machine (ATM). This case study illustrates the importance of maintaining a holistic view of the entire system (including the physical interface, the onscreen experience, and the person using the technology), particularly when introducing new technology or interaction paradigms. It also demonstrates the benefit of multiple iterative rounds of ideation, concept creation and evaluation; without multiple cycles an acceptable solution would not have been found. Wider lessons are drawn for the design of hardware. |
Usability evaluation: an investigation on combination of analytical and empirical methods
| Document
|
Author Setia Hermawati & Glyn Lawson |
| Abstract This study investigated the effect of combining analytical (heuristics and cognitive walkthrough) and empirical methods in usability evaluation. Data from two usability studies were used to simulate the outcomes of different combinations of usability evaluation methods. The findings show that the combined analytical methods significantly reduce the number of participants required in the empirical method without compromising the results of the usability evaluation. |
Patient-maintained propofol sedation for orthopaedic surgery: patient variability in system use
| Document
|
Author David Hewson, Nigel Bedforth, James Sprinks, Philip Breedon & Frank Worcester |
| Abstract In the UK, operations not requiring general anaesthesia may be carried out under sedation. This is generally provided by a doctor. As the patient is not controlling the sedation, they may often be either under or over-sedated, due to the doctor misjudging patients’ anxiety and sedation requirements. A potential solution is to allow the patient to control their own depth of sedation. We conducted a case series to examine the efficacy of patient-maintained propofol sedation for patients presenting for lower limb orthopaedic surgery under regional anaesthesia. Twenty-six patients undergoing lower limb surgery were given a handheld button to indicate their request for deepening sedation from a baseline propofol concentration of 0.5 g.ml-1 by 0.2 g.ml-1 increments to a maximum of 2.0 g.ml-1. Twelve patients chose not to press their button. The remaining 14 patients pressed the button a median (range) of 6 (1–29) times, obtaining a mean (SD) estimated effect-site blood propofol concentration of 0.91 g.ml-1 (0.34 g.ml-1). Feedback revealed that patients were satisfied with their sedation, were happy to have control over it, and would use the system again. Despite this consensus, sedation level profiles revealed variability in how patients used the system in terms of button press frequency and timing, associated with their pre-op anxiety and reaction to environmental events during the operation. Whilst this technique can be a safe and effective way of controlling sedation during these types of surgery, future research needs to consider the different ways patients interact with the system. |
PEGASO Project – Teenage Perspectives of an Integrated Technology Platform for the Promotion of a Healthy Lifestyle
| Document
|
Author Alexandra LANG, Laura CONDON, Neil COULSON, Sue COBB, and Sarah ATKINSON |
| Abstract This paper presents the findings from a participatory technology development workshop with teenagers. Sixteen teenagers were recruited in the East Midlands region of England to participate in the evaluation and development of the PEGASO multi-technology system. This system is being developed to promote awareness and motivation for healthier behaviours in teenagers. The teenagers critically analysed early prototypes of mobile apps, a serious game and wearable sensors with regards to their usability, acceptability, intuitiveness and desirability for teenage users. The results reported here will be utilized in the next design iteration of the PEGASO platform. |
Deploying Usability Research Within Low-to-Middle-Income Countries
| Document
|
Author Alejandra Anderson Jimenez |
| Abstract This paper explores the human factors (HF) and usability research conducted during the early-stage development of a novel respiratory device designed to prevent the spread of respiratory viruses in a pandemic scenario. It highlights the application of methods within the disciplines of human factors, user-centred design (UCD) and ethnography in low-and middle-income countries (LMIC) to ensure the device design meets the diverse needs of a global population. The paper highlights the importance of an inclusive, adaptable approach to design and iterative testing to ensure effectiveness and accessibility across varied environments. |
Comparison of safety system designs using risk assessment
| Document
|
Author Dr Mike Tainsh |
| Abstract Current work within the team of ergonomics, safety and operability specialists on user risk assessment for naval systems, has enabled the development of comparative techniques. They start from user system architectures, and the identification of user tasks in a form that supports assessment and mitigation of risks. In this paper the aim is to present assessment techniques for comparing safety aspects of design options. An example addresses the introduction of an automated information system. |
Beyond user-centred design. Crowdsourcing with Serious Games for Design.
| Document
|
Author Edward Oates |
| Abstract Designers rely on direct access to ‘users’ (those who will use the product) to assist in their design process. User-centred design strongly emphasises the full involvement of users in the design process; but what happens when they aren’t available? This study investigates the extent to which Serious Games may offer an asynchronous remote alternative to ‘face-to-face’ design processes through Crowdsourcing. A design process completes with summative usability testing of the product. Again, a lack of access to users is a serious limitation and one that may be ameliorated by remote unmoderated usability testing. The extent to which Serious Games may be the vehicle for remote usability testing is also explored in this research. Results from the Crowdsourcing activity show, from contributed design ideas, that a Serious Game may provide a credible tool for Crowdsourced Design. Remote unmoderated testing has known limitations and the use of Serious Games provides some mitigation, with careful implementation being required. This is a mid-study report on UK Ministry of Defence sponsored research under the Royal Navy ‘DARE Innovation’ initiative. |
A Tool to Generate ‘HF Meaningfulness’ in the Design and Development of Armoured Fighting Vehicles
| Document
|
Author David KEANE and Chris AVIS |
| Abstract The design and development of Armoured Fighting Vehicles (AFVs) at Lockheed Martin UK Ampthill (LMUKA) takes a Systems Engineering approach and incorporates many engineering disciplines. One of these is Human Factors (HF), which contributes to a number of other disciplines. To support this contribution and to generate ‘HF meaningfulness’ the LMUKA HF Team developed the Systems Engineering Comparison Technique (SECT). The SECT takes a User Centred Design (UCD) approach and incorporates the pairwise comparison technique developed by Thurstone (1927). The main aim of SECT is to take subjective scores and generate objective scores that can be used by Design Engineers and Engineering Leads for improving the design of AFVs. This is done by highlighting the design strengths and weaknesses, which are then fed into future design iterations. |
A quick history lesson: Lean UX research at Hampton Court Palace
| Document
|
Author Pete UNDERWOOD |
| Abstract This paper describes the Lean UX research approach used to evaluate the Digital Visitors Guide (DVG) developed for use at Hampton Court Palace. The DVG was tested onsite at the palace over several rounds of research. The DVG was liked by participants, who felt that it was easy to use, engaging, modern and improved their learning experience. Taking a Lean UX approach worked well as insights were delivered, and design recommendations prioritised, efficiently in a collaborative environment. |
Evaluating the use of tactile navigation for motorcycle taxis and couriers
| Document
|
Author Crisman Silban and Chris Baber |
| Abstract In this paper we describe the design and evaluation of a simple prototype device that provides tactile cues to support navigation by motorcyclists. A comparative evaluation shows that the device supports equivalent performance to the use of a visual display. The evaluation was performed by licensed motorcycle taxi drivers in Thailand, on a University campus. The evaluation showed that, in terms of journey time and route accuracy, there was little difference between the two technologies. A further evaluation, of the tactile belt, was conducted in a busy town. Participants were able to follow the route and responded positively to the concept and its implementation in our prototype. We propose that tactile navigation aids can help motorcyclists and that, compared to visual displays, these can be used with reduced risk of distraction. |
User centred approach to developing a Concept of Operations
| Document
|
Author Kate Shield & Elaine Thompson |
| Abstract The paper describes a user centred approach to the development of a Concept of Operations and its application to the Synthetic Environment, which is a digital signalling design tool that will be used to develop seamlessly integrated scheme designs for Network Rail. |
Improving ineffective instructions
| Document
|
Author Andy Brazier |
| Abstract Despite the availability of extensive guidance, many instructions, procedures, and other documents intended to control how work is performed fail to fulfil their primary role. Major incidents and day-to-day operational inefficiencies often reveal that written instructions are either misunderstood, ignored, or incorrectly followed. Style guides typically focus on aspects such as terminology, tense, and reading level, aiming to ensure clarity and readability. However, these guides rarely address the more critical question: what content should be included, or excluded, to make instructions truly effective. Effective instructions go beyond simply describing how a task should be performed; they are tools that support users in completing tasks with greater accuracy, reducing errors, and enhancing overall consistency. The distinction between well-crafted instructions and poorly designed ones is not trivial; it can mean the difference between safe, efficient operations and incidents with serious consequences. This paper explores why many instructions fall short and how adopting a user-centred, task-focused approach can lead to better outcomes for individuals and organisations alike |
