Feature-based Visual Simultaneous Localization and Mapping (V-SLAM) employs feature descriptors to recognize landmarks in successive frames. For networked (offloaded and/or collaborative) V-SLAM, the feature descriptors typically need to be transmitted over wireless communication networks with limited bitrates. Conventionally, V-SLAM systems have employed general computer vision feature descriptors whose size (in bytes) accounts for most of the data transmission. We develop a probabilistic model of the V-SLAM feature matching and identify three parameters that govern the probability of the correct feature match: the feature set size, the bit error probability between feature descriptors observing the same landmark, and the probability that the correct feature match exists in the feature set. Based on these three parameters, we formulate a research agenda for achieving high V-SLAM performance (i.e., high correct feature match probability) for reduced V-SLAM feature descriptor sizes (and thus reduced network throughput requirements). We address the first item on this research agenda by pursuing the reduction of the feature set size through a novel Orientation Restriction with Frame-to-Frame Projection (ORFFP). We evaluate ORFFP in comparison to the state-of-the-art ORB-SLAM2 approach using three representative V-SLAM test sequences. Our ORFFP approach reduces the size of the feature set to as little as 9.1% of ORB-SLAM2, allowing for the use of smaller feature descriptors. Thereby, ORFFP reduces the overall required throughput for networked V-SLAM down to roughly a third of the current approach, while achieving essentially the same V-SLAM performance. We also find that for learned descriptor-based models (HashSIFT, BAD), the feature descriptor size can be reduced using ORFFP without severely degrading the performance of V-SLAM. We outline future research directions towards comprehensively addressing the formulated research agenda for enabling networked V-SLAM on low-bitrate networks.

Visual Collaborative Simultaneous Localization and Mapping (SLAM) uses multiple mobile devices to capture images of their surroundings and map their environment collectively while simultaneously determining their position. In centralized approaches, a single server merges visual features from these devices to build a shared map. Offloading computationally intensive tasks benefits resource-limited mobile devices. However, due to unpredictable and heterogeneous communication links, maps at the server and on the devices can become asynchronous, potentially leading to real-time localization failures. To address these issues, we present Reliable Agents in Centralized Visual Collaborative SLAM (RAVIC), which employs an optimised server-to-agent communication strategy. This includes a novel keyframe selection method that exploits the server’s global map to improve tracking under high agent mobility and network latency. In addition, adjustments to agent-to-server communication and agent tracking algorithm reduce the computational demands on mobile devices, leading to a reduction in average tracking processing time of up to 27.2 ms. Our evaluation using benchmark datasets and different network conditions shows a drastic reduction in track loss probability from 61.4% to only 1.0% utilising merged map data in collaborative scenarios, demonstrating that collaboration increases the reliability of individual agents.

Visual Simultaneous Localization and Mapping (SLAM) systems have their numerous applications in robotics and autonomous driving. Computational offloading enables the computationally demanding Visual SLAM systems to run on hardware-constrained mobile devices, such as Unmanned Aerial Vehicles (UAVs) and Automated Guided Vehicles (AGVs). The offloading of SLAM modules to a central server also enables cooperative collaboration between multiple mobile devices. In this paper, we investigate the process of map synchronization between the edge and mobile devices in edge assisted SLAM systems. Due to the condition of uninterrupted execution, it is necessary that the map synchronization can be executed in the running process without losing the localization on the mobile device. Using a state-of-the-art edge assisted SLAM system, we investigate the influences of the movement speed as well as the network latency on the map synchronization and the resulting success ratio of the tracking continuation. By introducing a frame buffer on the mobile device, we have managed to compensate for the negative effect of the synchronization delay and thus increase reliability by up to 37%.

New generations of mobile networks are opening up novel possibilities for controlling robots remotely in real-time. With 5G’s requirement to support use cases that demand low latencies at a high reliability from the communication network, wireless control applications become feasible. A remote operator typically uses a handheld device with buttons or joysticks to control a mobile robot. Joysticks are widely used today. The limitations of two-dimensional controlling and displaying of camera data can cause difficulties. Augmented Reality (AR)-based control provides the ability to control in three dimensional space. Therefore, new user-friendly Human Machine Interfaces (HMIs) can improve the interaction with these robots. In this demonstration, we present a human-in-the-loop application with a novel HMI. With our HMI a remote operator controls a four-legged Boston Dynamics Spot robot with gestures while wearing the AR-Headset Microsoft HoloLens 2. The remote operator receives feedback from the robot in the form of live camera streams visualised on holographic screens.

Edge Simultaneous Localization and Mapping (SLAM) retains only the tracking on the mobile device, while offloading the compute-intensive local mapping and loop close to edge computing. Existing Edge SLAM approaches incur relatively high delays for offloading, resulting in high failure probabilities, i.e., low reliability, for commonly used public SLAM datasets. We discovered that two parameters which had not previously been studied in detail, namely the number of features and the number of keyframes that are bundled for a local map update, play a critical role in the offloading delay. Also, previous approaches updated the local map in the mobile device in a serial manner, incurring map update latencies. We study the numbers of features and bundled keyframes in detail and we parallelize the local map update. We find that judicious parameter settings, namely relatively small numbers of features (750 per frame) and bundled keyframes (1, i.e., effectively no bundling), reduce the map update latency to less than half compared to the previously common settings (1000 features per frame and 6 keyframes used for a map update). For a low network latency of 20ms, these judicious parameter settings in conjunction with our parallelized local map updating, reduce the 79% failure rate of the previous Edge SLAM systems down to 2%.

The Simultaneous Localization and Mapping (SLAM) method is becoming more and more established for the localization of mobile robots in indoor environments. Due to the high complexity of SLAM, high computing resources are necessary, which leads to a shorter runtime. By using edge computing and higher bandwidths of new wireless technologies, the computing can be outsourced. In this work, the SLAM process is offloaded from a mobile robot to an edge cloud and the impact of more computing power is investigated. We show that outsourcing has performance advantages in terms of the update rate of the map generation as well as the localization.

We present SoRo-Track, a two-axis soft robotic actuator (SRA) for solar tracking and building-integrated photovoltaic applications. SRAs are gaining increasing popularity compared to traditional actuators, such as dc motors and hydraulic or pneumatic pistons, due to their inherent compliance, low morphological complexity, high power-to-weight ratio, resilience to external shocks and adverse environmental conditions, design flexibility, ease of fabrication, and low costs. We present the design, modelling and experimental characterisation of SoRo-Track. Finally, we demonstrate the suitability of SoRo-Track for solar tracking applications, which makes it a viable component for dynamic building facades.