Change detection (CD) is critical in everyday tasks. While current algorithmic approaches for CD are improving, they remain imprecise, often requiring human intervention. Cognitive science research focuses on understanding CD mechanisms, especially through change blindness studies. However, these do not address the primary requirement in real-life CD - detecting changes as effectively as possible. Such a requirement is directly relevant to the visual comparison field - studying visualisation techniques to compare data and identify differences or changes effectively. Recent studies have used Virtual Reality (VR) to improve visual comparison by providing an immersive platform where users can interact with 3D data at a real-life scale, enhancing spatial reasoning. We believe VR could also improve CD performance accordingly. Particularly, VR offers stereoscopic depth perception over traditional displays, potentially enhancing the detection of spatial change. In this paper, we develop and analyse three 3D visual comparison techniques for CD in VR: Sliding Window, 3D Slider, and Switch Back. These techniques are evaluated under synthetic but realistic environments and frequently occurring Perceptual Challenges, including different Changed Object Size, Lighting Variation, and Scene Drift conditions. Experimental results reveal significant differences between the techniques in detection time measures and subjective user experience.

We present a virtual reality (VR) teleoperation interface for a ground-based robot, featuring dense 3D environment reconstruction and a low latency video stream, with which operators can immersively explore remote environments. At the UK Atomic Energy Authority's (UKAEA) Remote Applications in Challenging Environments (RACE) facility, we applied the interface in a user study where trained robotics operators completed simulated nuclear monitoring and decommissioning style tasks to compare VR and traditional teleoperation interface designs. We found that operators in the VR condition took longer to complete the experiment, had reduced collisions, and rated the generated 3D map with higher importance when compared to non-VR operators. Additional physiological data suggested that VR operators had a lower objective cognitive workload during the experiment but also experienced increased physical demand. Overall the presented results show that VR interfaces may benefit work patterns in teleoperation tasks within the nuclear industry, but further work is needed to investigate how such interfaces can be integrated into real world decommissioning workflows.

Digitization of pathology has been proposed as an essential mitigation strategy for the severe staffing crisis facing most pathology departments. Despite its benefits, several barriers have prevented widespread adoption of digital workflows, including cost and pathologist reluctance due to subjective image quality concerns. In this work, we quantitatively determine the minimum image quality requirements for binary classification of histopathology images of breast tissue in terms of spatial and sampling resolution. We train an ensemble of deep learning classifier models on publicly available datasets to obtain a baseline accuracy and computationally degrade these images according to our derived theoretical model to identify the minimum resolution necessary for acceptable diagnostic accuracy. Our results show that images can be degraded significantly below the resolution of most commercial whole-slide imaging systems while maintaining reasonable accuracy, demonstrating that macroscopic features are sufficient for binary classification of stained breast tissue. A rapid low-cost imaging system capable of identifying healthy tissue not requiring human assessment could serve as a triage system for reducing caseloads and alleviating the significant strain on the current workforce.

The recent momentum in aerial manipulation has led to an interest in developing virtual reality interfaces for aerial physical interaction tasks with simple, intuitive, and reliable control and perception. However, this requires the use of expensive subsystems and there is still a research gap between interface design, user evaluations and the effect on aerial manipulation tasks. Here, we present a methodology for low-cost available drone systems with a Unity-based interface for immersive FPV teleoperation. We applied our approach in a flight track where a cluttered environment is used to simulate a demanding aerial manipulation task inspired by forestry drones and canopy sampling. Through objective measures of teleoperation performance and subjective questionnaires, we found that operators performed worse using the FPV interface and had higher perceived levels of cognitive load when compared to traditional interface design. Additional analysis of physiological measures highlighted that objective stress levels and cognitive load were also influenced by task duration and perceived performance, providing an insight into what interfaces could target to support teleoperator requirements during aerial manipulation tasks.

Optoelectronic tweezer-driven microrobots (OETdMs) are a versatile micromanipulation technology based on the application of light induced dielectrophoresis to move small dielectric structures (microrobots) across a photoconductive substrate. The microrobots in turn can be used to exert forces on secondary objects and carry out a wide range of micromanipulation operations, including collecting, transporting and depositing microscopic cargos. In contrast to alternative (direct) micromanipulation techniques, OETdMs are relatively gentle, making them particularly well suited to interacting with sensitive objects such as biological cells. However, at present such systems are used exclusively under manual control by a human operator. This limits the capacity for simultaneous control of multiple microrobots, reducing both experimental throughput and the possibility of cooperative multi-robot operations. In this article, we describe an approach to automated targeting and path planning to enable open-loop control of multiple microrobots. We demonstrate the performance of the method in practice, using microrobots to simultaneously collect, transport and deposit silica microspheres. Using computational simulations based on real microscopic image data, we investigate the capacity of microrobots to collect target cells from within a dissociated tissue culture. Our results indicate the feasibility of using OETdMs to autonomously carry out micromanipulation tasks within complex, unstructured environments.

To capture the geometry of an object by an autonomous system, next best view (NBV) planning can be used to determine the path a robot will take. However, current NBV planning algorithms do not distinguish between objects that need to be mapped and everything else in the environment; leading to inefficient search strategies. In this paper we present a novel approach for NBV planning that accounts for the importance of objects in the environment to inform navigation. Using weighted entropy to encode object utilities computed via semantic segmentation, we evaluate our approach over a set of virtual Gazebo environments comparable to construction scales. Our results show that using semantic information reduces the time required to capture a target object by at least 40 percent.

Grounded haptic devices can provide a variety of forces but have limited working volumes. Wearable haptic devices operate over a large volume but are relatively restricted in the types of stimuli they can generate. We propose the concept of docking haptics, in which different types of haptic devices are dynamically docked at run time. This creates a hybrid system, where the potential feedback depends on the user’s location. We show a prototype docking haptic workspace, combining a grounded six degree-of-freedom force feedback arm with a hand exoskeleton. We are able to create the sensation of weight on the hand when it is within reach of the grounded device, but away from the grounded device, hand-referenced force feedback is still available. A user study demonstrates that users can successfully discriminate weight when using docking haptics, but not with the exoskeleton alone. Such hybrid systems would be able to change configuration further, for example docking two grounded devices to a hand in order to deliver twice the force, or extend the working volume. We suggest that the docking haptics concept can thus extend the practical utility of haptics in user interfaces.

Remote inspection of a complex environment is a difficult, time consuming task for human operators to perform. The need to manually avoid obstacles whilst considering other performance factors i.e. time taken, joint effort and information gained represents significant challenges to continuous operation. This paper proposes an autonomous robotic solution for exploration of an unknown, complex environment using a high DoF robot arm with an eye in hand depth sensor. The main contribution of this work is a new strategy to find the next best view by evaluating frontier regions of the map to maximise coverage, in contrast to many current approaches which densely sample joint or workspace configurations of the robot. Multiple utility functions were evaluated that showed different behaviours. Our results indicated that the presented algorithm can explore an arbitrary environment efficiently while optimising various performance criteria based on the utility function chosen, application constraints and the desires of the user.

Mobile multi-robot construction systems offer new ways to optimise the on-site construction process. In this paper we begin to investigate the functionality requirements for controlling a team of robots to build structures much greater than their individual workspace. To achieve these aims, we present a mobile extruder robot called YouWasp. We also begin to explore methods for collision aware printing and construction task decomposition and allocation. These are deployed via YouWasp and enable it to deposit material autonomously. In doing so, we are able to evaluate the potential for parallelization of tasks and printing autonomy in simulation as well as physical team of robots. Altogether, these results provide a foundation for future work that enable fleets of mobile construction systems to cooperate and help us shape our built environment in new ways.

In this paper, we present a Mobile Agile Printer (MAP) construction robot; a highly agile, 4-legged, omnidirectional robot capable of 3D printing large structures. To overcome dynamic challenges when operating within an outdoors construction site, MAP incorporates a high-DoF 3D printing system connected to a mobile platform with novel features designed to enable disturbance rejection and live adaption to the robot's pose. In doing so, we demonstrate the benefits of designing construction robots with a focus on agility, a compact working volume and ability to operate within a potentially unlimited workspace. Performance tests were conducted showing smooth omni-directional motion as a key requirement for maintaining low 3D printing trajectory deviations over a large volume. In doing so, we show that MAP has the ability to construct in new ways more sensitive to its environment, context and concurrent on-site operations.

In a robot mounted camera arrangement, hand-eye calibration estimates the rigid relationship between the robot and camera coordinate frames. Most hand-eye calibration techniques use a calibration object to estimate the relative transformation of the camera in several views of the calibration object and link these to the forward kinematics of the robot to compute the hand-eye transformation. Such approaches achieve good accuracy for general use but for applications such as robotic assisted minimally invasive surgery, acquiring a calibration sequence multiple times during a procedure is not practical. In this paper, we present a new approach to tackle the problem by using the robotic surgical instruments as the calibration object with well known geometry from CAD models used for manufacturing. Our approach removes the requirement of a custom sterile calibration object to be used in the operating room and it simplifies the process of acquiring calibration data when the laparoscope is constrained to move around a remote centre of motion. This is the first demonstration of the feasibility to perform hand-eye calibration using components of the robotic system itself and we show promising validation results on synthetic data as well as data acquired with the da Vinci Research Kit.

Immersive virtual environments (IVEs) allow participants to interact with their 3D surroundings using natural hand gestures. Previous work shows that the addition of haptic feedback cues improves performance on certain 3D tasks. However, we believe this is not true for all situations. Depending on the difficulty of the task, we suggest that we should expect differences in the ballistic movement of our hands when presented with different types of haptic force-feedback conditions. We investigated how hard, soft and no haptic force-feedback responses, experienced when in contact with the surface of an object, affected user performance on a task involving selection of multiple targets. To do this, we implemented a natural egocentric selection interaction technique by integrating a two-handed large-scale force-feedback device in to a CAVETM-like IVE system. With this, we performed a user study where we show that participants perform selection tasks best when interacting with targets that exert soft haptic force-feedback cues. For targets that have hard and no force-feedback properties, we highlight certain associated hand movement that participants make under these conditions, that we hypothesise reduce their performance.

In this paper we profiled the performance of two types of 3D selections tasks: selection of one target and the selection of two targets. We designed an Immersive Virtual Environment (IVE) to evaluate any differences that may exist, and understand the underlying human behaviour by recording the hand movements' participants made when asked to select a series of 3D objects. To do this, we implemented a natural virtual hand-like interaction technique that participants could control using a large-scale force-feedback device placed into a CAVE™-like IVE system. We also investigated the interaction of no, soft and hard haptic force-feedback responses in addition to three target sizes on user performance. From the results obtained, we show distinct differences in the movement time taken when participants used their right hand to select one target in comparison to the selection of two targets.