Empathic medical conversation is central to patient-centered care within Narrative Medicine. However, difficulties, such as physicians’ limited empathic capabilities and lack of time, impede the practice. Research on real-time, on-site empathic medical exchanges has been limited in exploring technology to assist and enhance physicians’ capabilities. This paper proposed the Empathic Opportunity Perception and Distinction (EOPD) framework for building physician-AI collaboration based on Intelligence Augmentation (IA) for empathic conversations. The EOPD integrates two multi-modal machine learning (ML) models based on facial and verbal cues, presenting a physician-AI interaction framework and three distinctive visualization components: emotional reference, opportunity reminding and keyword collection, and situation understanding. To assess EOPD's effectiveness and gauge physicians’ and patients’ receptiveness, a prototype system named EMVIS ( EM otional VIS ualization ) was designed and developed. Results from the study demonstrated improvements in physicians’ empathy efforts and perceived empathy performance when using EMVIS, particularly for junior physicians. Physicians and patients held positive attitudes towards EMVIS, with patients expressing a high expectation that EMVIS would improve the physician-patient relationship. The research showed the efficacy of the multi-modal ML models in supporting complex affective empathy and EMVIS in facilitating and complementing empathy concerns. It highlighted the tailored support to junior and senior physicians and emphasized physician-AI collaboration to maintain user autonomy and mitigate potential biases. Future research should explore extensive system applications, tailor visual and interactive support for physicians, and implement adaptive and reflective ML models to improve the effectiveness and efficiency of empathy communications.

Patient-physician communication is a crucial aspect of clinical diagnoses and treatments. However, there are barriers to effective empathic practices, including consciousness, busy working rhythms, and difficulties recognising patients’ implicit emotional expressions. While previous research has attempted to support asynchronous medical conversations, this study has explored the use of emotion visualisation techniques for synchronous, face-to-face medical encounters. After interviewing doctors to understand user requirements, an emotion-visualisation prototype, EMVIS, was created. The prototype was evaluated in a study with 31 patients and 37 healthcare providers within different specialist groups using a contextualised Technology Acceptance Model (TAM) and follow-up interviews. The results indicated that patients and physicians were generally accepting of emotion visualisation for medical encounters. Patients were more interested in their physicians’ attitudes and intentions, while physicians accepted the visualisation, but their requirements differed according to their skill levels and specialities. Hence, four supportive factors - emotional empathy, careful attention, human connection, and reflective conversation - elicited information on how EMVIS contributed to medical conversations. Five future opportunities for the emotion visualisation of medical conversations were discussed in respect of the human factors and potential requirements. These include communicating uncertainty, addressing user diversity, providing explanatory information, managing attention, and supporting negotiations.

As Digital Manufacturing transforms traditionally physical work into more system-monitoring tasks, new methods are required for understanding people’s mental workload and prolonged capacity for focused attention. Many physiological measures have shown promise for detecting changes in cognitive state, and recent advances in sensor technology offer minimally-invasive ways to monitor our cognitive activity. Previous research in functional near-infrared spectroscopy, for example, has observed changes in cerebral hemodynamic response during periods of high demand within tasks. This work investigated the relationships among task demand, fatigue, and attention degradation in a sustained attention task, and their effect on heart rate, breathing rate, nose temperature and hemodynamic response in the prefrontal cortex and middle temporal gyrus. Analysis revealed a small but significant effect of fatigue on heart rate relative to baseline, breathing rate and hemodynamic response. Task demand had a small but significant effect on breathing rate and nose temperature, both relative to baseline, but no difference between levels of demand was observed in heart rate or hemodynamic response. Our results provide insight into what physiological data can tell us about cognitive state, ability to focus, and the impact of fatigue over time.

Understanding validity of user behaviour in Virtual Environments (VEs) is critical as they are increasingly being used for serious Health and Safety applications such as predicting human behaviour and training in hazardous situations. This paper presents a comparative study exploring user behaviour in VE-based fire evacuation and investigates whether this is affected by the addition of thermal and olfactory simulation. Participants (N=43) were exposed to a virtual fire in an office building. Quantitative and qualitative analyses of participant attitudes and behaviours found deviations from those we would expect in real life (e.g. pre-evacuation actions), but also valid behaviours like fire avoidance. Potentially important differences were found between multisensory and audiovisual-only conditions (e.g. perceived urgency). We conclude VEs have significant potential in safety-related applications, and that multimodality may afford additional uses in this context, but the identified limitations of behavioural validity must be carefully considered to avoid misapplication of the technology.

This paper presents a multisensory and low-cost virtual training simulator developed in Unity 3D, with the aim of improving the effectiveness of Occupational Safety and Health (OSH) training. The prototype system facilitates heat and smell feedback functions operated by an Arduino microprocessor and triggered based on the proximity of the avatar to receptive 4 within the Virtual Environment (VE). The prototype enables the creation of bespoke virtual representations using the 3D scanning function of the Google Tango device making multisensory VE OSH training a feasible and versatile approach in the short-term future.

Touchscreen Human-Machine Interfaces (HMIs) inherently demand some visual attention. By employing a secondary device, to work in unison with a touchscreen, some of this demand may be alleviated. In a medium-fidelity driving simulator, twenty-four drivers completed four typical in-vehicle tasks, utilising each of four devices -- touchscreen, rotary controller, steering wheel controls and touchpad (counterbalanced). Participants were then able to combine devices during a final 'free-choice' drive. Visual behaviour, driving/task performance and subjective ratings (workload, emotional response, preferences), indicated that in isolation the touchscreen was the most preferred/least demanding to use. In contrast, the touchpad was least preferred/most demanding, whereas the rotary controller and steering wheel controls were largely comparable across most measures. When provided with 'free-choice', the rotary controller and steering wheel controls presented as the most popular candidates, although this was task-dependent. Further work is required to explore these devices in greater depth and during extended periods of testing.

Journal Article Tailored Scenarios: A Low-Cost Online Method to Elicit Perceptions of Home Technologies Using Participant-Specific Contextual Information Get access Michael Brown, Michael Brown * 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK2Human Factors Research Group, University of Nottingham, Nottingham, UK *Corresponding author: [email protected] Search for other works by this author on: Oxford Academic Google Scholar Tim Coughlan, Tim Coughlan 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK3Computer Science, University of Nottingham, Nottingham, Nottinghamshire, UK Search for other works by this author on: Oxford Academic Google Scholar Jesse Blum, Jesse Blum 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK Search for other works by this author on: Oxford Academic Google Scholar Glyn Lawson, Glyn Lawson 2Human Factors Research Group, University of Nottingham, Nottingham, UK Search for other works by this author on: Oxford Academic Google Scholar Robert Houghton, Robert Houghton 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK2Human Factors Research Group, University of Nottingham, Nottingham, UK Search for other works by this author on: Oxford Academic Google Scholar Richard Mortier, Richard Mortier 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK3Computer Science, University of Nottingham, Nottingham, Nottinghamshire, UK Search for other works by this author on: Oxford Academic Google Scholar Murray Goulden, Murray Goulden 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK Search for other works by this author on: Oxford Academic Google Scholar Unna Arunachalam Unna Arunachalam 1Horizon Digital Economy Research, University of Nottingham, Nottingham, Nottinghamshire, UK Search for other works by this author on: Oxford Academic Google Scholar Interacting with Computers, Volume 27, Issue 1, January 2015, Pages 60–71, https://doi.org/10.1093/iwc/iwu028 Published: 25 July 2014 Article history Received: 02 January 2014 Revision received: 06 June 2014 Accepted: 19 June 2014 Published: 25 July 2014

Touch screens are increasingly used within modern vehicles, providing the potential for a range of gestures to facilitate interaction under divided attention conditions. This paper describes a study aiming to understand how drivers naturally make swipe gestures in a vehicle context when compared with a stationary setting. Twenty experienced drivers were requested to undertake a swipe gesture on a touch screen in a manner they felt was appropriate to execute a wide range of activate/deactivate, increase/decrease and next/previous tasks. All participants undertook the tasks when either driving within a right-hand drive, medium-fidelity simulator or whilst sitting stationary. Consensus emerged in the direction of swipes made for a relatively small number of increase/decrease and next/previous tasks, particularly related to playing music. The physical action of a swipe made in different directions was found to affect the length and speed of the gesture. Finally, swipes were typically made more slowly in the driving situation, reflecting the reduced resources available in this context and/or the handedness of the participants. Conclusions are drawn regarding the future design of swipe gestures for interacting with in-vehicle touch screens.

Touchpads in vehicles offer a range of potential benefits over existing input devices, such as touchscreens. This article describes a study aiming to establish where a touchpad should be located within a right-hand drive vehicle. Sixteen participants (50:50 right/left handed) drove three routes in a right-hand drive simulator while following a lead vehicle at a perceived safe distance. At specific points, participants were asked to carry out three tasks of varying complexity using the touchpad. For each of the routes travelled, the touchpad was positioned in one of the three locations: in the centre console; in the door armrest and in the steering wheel. Differences in the performance and preferences of right-handed people vs. left-handed people were found. Right-handed people rated the door armrest location highly and made few glances towards this location while driving. In contrast, left-handed drivers were more positive towards the centre console location. The steering wheel location required a particularly high-visual demand. It is concluded that, for right-hand drive vehicles, a touchpad should be located in both the centre console and the door armrest to suit the diverse needs of the driver population.

The ‘Internet of Things’ (IoT) can be expected to radically increase the amount of potentially sensitive data gathered in our homes. This study explores the social implications of the presentation of data that could be collected within the household. In particular, it focuses on how ambiguities in these data, combined with existing interpersonal relationships, could influence social dynamics. Thirty-five participants were each presented with three separate household scenarios, involving ambiguous data that were collected and presented via near-future IoT technologies. Each participant was asked to respond to a series of open and closed questions about how they would interpret the data, how they would react to it and their general opinions of the technologies presented. Through qualitative and quantitative analysis of their responses, we contribute an understanding of how people interpret information about those around them. We find a common willingness to make inferences based on ambiguities within the data, even when participants are aware of the limitations of their understanding. We also find that sharing data produced via tagging of everyday objects raises a high level of privacy concern, and that, in a somewhat incoherent stance, users are more comfortable in sharing data publicly than in a targeted fashion with commercial organizations. Our findings also suggest that the age of the target user group has a greater effect on ease of use judgements than the nature of the technology, and we find some evidence that user’s interpretations can be biased by an individual’s age.

Designers of in-vehicle computing systems must consider which input devices are most suitable for use in the safety-critical driving situation. This paper describes a study aiming to establish which tasks are best supported by an in-vehicle touchpad system. Eighteen participants (50:50 right/left handed) drove three routes in a right-hand drive simulator while following a lead vehicle at a perceived safe distance. At specific points, participants were asked to carry out seven tasks of varying qualities using a prototype touchpad system, a touchscreen or a rotary controller interface. Results indicated that participants were most negative (in terms of preferences and performance) with the rotary controller interface. Conversely, the results for the touchpad versus the touchscreen interfaces were clearly task dependent. For instance, with the touchpad, subjective opinions and objective performance were most positive for tasks in which simple commands enabled drivers to bypass the need for complex menu interactions (e.g. changing the interior temperature). In contrast, results for the touchscreen were evidently superior for simple menu selection tasks (e.g. selecting a preset radio station). Conclusions are drawn regarding the nature of tasks that are best suited to alternative input devices within vehicles and the potential for a touchpad/touchscreen solution.