3D Gaussian Splatting (3DGS) allows flexible adjustments to scene representation, enabling continuous optimization of scene quality during dense visual simultaneous localization and mapping (SLAM) in static environments. However, 3DGS faces challenges in handling environmental disturbances from dynamic objects with irregular movement, leading to degradation in both camera tracking accuracy and map reconstruction quality. To address this challenge, we develop an RGB-D dense SLAM which is called Gaussian Splatting SLAM in Dynamic Environments (Gassidy). This approach calculates Gaussians to generate rendering loss flows for each environmental component based on a designed photometricgeometric loss function. To distinguish and filter environmental disturbances, we iteratively analyze rendering loss flows to detect features characterized by changes in loss values between dynamic objects and static components. This process ensures a clean environment for accurate scene reconstruction. Compared to state-of-the-art SLAM methods, experimental results on open datasets show that Gassidy improves camera tracking precision by up to 97.9 % and enhances map quality by up to 6 %. Video of experiments is available here: https://www.wixsite.com.com/wen-Gassidy.

This paper proposes a novel dual-filter architecture utilizing RGB-D camera data and dynamic control barrier functions (D-CBFs) for real-time obstacle avoidance in unstructured environments. The proposed method efficiently handles static, suddenly appearing, and dynamic obstacles, maintaining consistent computational performance across diverse scenarios. To achieve this, two key challenges must be addressed. First, the substantial volume of pixel and depth map data requires robust, real-time processing for efficient D-CBF construction. Second, constructing D-CBFs for each obstacle in multi-obstacle scenarios increases optimization solver time. To address these challenges, we adapt the concept of salient object detection (SOD), proposing an enhanced FastSOD (E-FastSOD) method for rapid risk area identification. This approach rapidly filters out low-risk areas, while high-risk regions are mathematically represented utilizing the proposed enhanced minimal bounding circle (E-MBC) technique. We differentiate static and dynamic obstacles by comparing current and previous MBC states, employing Kalman filtering for obstacle state prediction. This setup enables efficient online D-CBF construction for each MBC, balancing computational speed with accurate obstacle representation. Subsequently, the second filter establishes buffer zones around established D-CBFs, activating only those corresponding to zones the robot actually enters, rather than all D-CBFs to increase real-time performance. We prove the system's safety and asymptotic stabilization under this architecture. Simulated and real-world experiments validate our method, demonstrating an equipped mobile robot's ability to accomplish tasks while ensuring safety across diverse, unknown scenarios.

The whisker-inspired tactile sensor is advantageous for enhancing robotic perception in proximate range and darkness via non-intrusive contacts. However, localizing contact along the whisker shaft is challenging due to the non-injective mapping between tangential contacts and the resulting bending moments at the whisker base. Previous studies suggest that incorporating axial force measurements can resolve this ambiguity. In this work, we develop a magnetically transduced whisker sensor that integrates axial force sensing as an additional mechanical signal. The sensor features a tapered whisker with a custom slope and a 3-DoF suspension mechanism, enabling axial displacement at the base, which is proportional to the applied axial force. We construct a Penalized Gaussian Process model trained on synthetic data to estimate the whisker’s motion and refine it with real-data constraints. The design is compact, low-cost, and validated through simulations and real-world experiments to differentiate tangential contacts. Furthermore, we propose an optimization-based approach for estimating instantaneous contact locations. Experimental results demonstrate that the proposed method can effectively track contacts in millimeter-level accuracy with a mean error of 7.17 mm, achieving a higher accuracy with only 4.02 mm in large-deflection and close-to-base regions.

This paper proposes a novel adaptive fast variational sparse Gaussian process (AFVSGP) framework to ensure real-time safety for high-order systems under model uncertainties and dynamic obstacle environments. The framework effectively addresses the challenge of maintaining real-time safety guarantees during unknown trajectory transitions in nonstationary environments. To achieve this, the proposed framework incorporates three key innovations. First, a specialized kernel function is embedded within the VSGP algorithm to decouple control inputs from uncertainties while preserving the convexity of posterior-based safety constraints. Second, an adaptive online incremental learning mechanism is introduced, integrating forgetting capabilities with dynamic reconstruction rules for training datasets and inducing sets, thereby accelerating inference convergence and enabling compact uncertainty prediction with reduced computational complexity. Third, a high-order control barrier function (HOCBF)-based safety filter is developed to synthesize safe control inputs by leveraging the proposed learning model, thereby establishing rigorous probabilistic bounds on the satisfaction of safety specifications. The effectiveness of the proposed framework is validated through both simulation and real-world obstacle avoidance experiments on a 7-DOF Franka robot. The video is available at: https://www.youtube.com/watch?v=2tCKYM_79S8.

Autonomous robots operating in diverse scenarios are expected to safely and efficiently adapt to new, unknown, and cluttered environments. In this paper, we introduce a real-time goal-seeking and exploration framework incorporating novel meta-signed distance functions (MetaSDFs) and metabuffer robust control barrier functions (Meta-BRCBFs). To adapt to environmental changes in real time, we employ Bayesian meta-learning to construct MetaSDFs. Deep neural network weights are initially trained offline, followed by efficient online adaptation at the last Bayesian layer, allowing for online updates at linear time complexity. Each MetaSDF is individually trained for its corresponding obstacle class, enhancing online distance estimation accuracy. Subsequently, buffer zones are constructed around the MetaSDFs to establish corresponding Meta-BRCBFs. These Meta-BRCBFs are activated only when the robot enters these zones, substantially reducing the number of CBFs required. Outside these specified buffer zones, the robot remains ingoal-seekingmode, focusing on task completion. After entering a buffer zone, it transitions toexplorationmode, prioritizing safety and exploring safe pathways, effectively balancing task execution with environmental adaptability. We demonstrate that, under this framework, the system achieves both safety and asymptotic stabilization. Extensive simulations and experiments are conducted to demonstrate our framework’s effectiveness in both simulated scenarios and real-world environments. These tests confirm our framework’s real-time capabilities and safety assurances in dynamic settings where state-of-the-art methods fail. The video is available at: https://www.youtube.com/watch?v=C6eshldAMxA.

This paper presents an adaptive online learning framework for systems with uncertain parameters to ensure safety-critical control in non-stationary environments. Our approach consists of two phases. The initial phase is centered on a novel sparse Gaussian process (GP) framework. We first integrate a forgetting factor to refine a variational sparse GP algorithm, thus enhancing its adaptability. Subsequently, the hyperparameters of the Gaussian model are trained with a specially compound kernel, and the Gaussian model’s online inferential capability and computational efficiency are strengthened by updating a solitary inducing point derived from newly samples, in conjunction with the learned hyperparameters. In the second phase, we propose a safety filter based on high order control barrier functions (HOCBFs), synergized with the previously trained learning model. By leveraging the compound kernel from the first phase, we effectively address the inherent limitations of GPs in handling high-dimensional problems for real-time applications. The derived controller ensures a rigorous lower bound on the probability of satisfying the safety specification. Finally, the efficacy of our proposed algorithm is demonstrated through real-time obstacle avoidance experiments executed using both simulation platform and a real-world 7-DOF robot.

This paper proposes a LiDAR-based goal-seeking and exploration framework, addressing the efficiency of online obstacle avoidance in unstructured environments populated with static and moving obstacles. This framework addresses two significant challenges associated with traditional dynamic control barrier functions (D-CBFs): their online construction and the diminished real-time performance caused by utilizing multiple D-CBFs. To tackle the first challenge, the framework’s perception component begins with clustering point clouds via the DBSCAN algorithm, followed by encapsulating these clusters with the minimum bounding ellipses (MBEs) algorithm to create elliptical representations. By comparing the current state of MBEs with those stored from previous moments, the differentiation between static and dynamic obstacles is realized, and the Kalman filter is utilized to predict the movements of the latter. Such analysis facilitates the D-CBF’s online construction for each MBE. To tackle the second challenge, we introduce buffer zones, generating Type-II D-CBFs online for each identified obstacle. Utilizing these buffer zones as activation areas substantially reduces the number of D-CBFs that need to be activated. Upon entering these buffer zones, the system prioritizes safety, autonomously navigating safe paths, and hence referred to as the exploration mode. Exiting these buffer zones triggers the system’s transition to goal-seeking mode. We demonstrate that the system’s states under this framework achieve safety and asymptotic stabilization. Experimental results in simulated and real-world environments have validated our framework’s capability, allowing a LiDAR-equipped mobile robot to efficiently and safely reach the desired location within dynamic environments containing multiple obstacles. Video and code are available: https://zyzhang4.wixsite.com/iros2024.

The automotive industry is transitioning from traditional ECU-based systems to software-defined vehicles. A central role of this revolution is played by containers, lightweight virtualization technologies that enable the flexible consolidation of complex software applications on a common hardware platform. Despite their widespread adoption, the impact of containerization on fundamental real-time metrics such as end-to-end latency, communication jitter, as well as memory and CPU utilization has remained virtually unexplored. This paper presents a microservice architecture for a real-world autonomous driving application where containers isolate each service. Our comprehensive evaluation shows the benefits in terms of end-to-end latency of such a solution even over standard bare-Linux deployments. Specifically, in the case of the presented microservice architecture, the mean end-to-end latency can be improved by 5–8%. Also, the maximum latencies were significantly reduced using container deployment.

To mitigate unforeseen operational interruptions caused by potential malfunctions in robotic systems employed in industrial automation, we propose an innovative strategy for anomaly detection that incorporates a Transformer-based reconstruction network for identifying irregularities in skill-oriented manufacturing. Leveraging a semantic representation of processes, products, and resources, a semantic manufacturing execution system synthesizes an appropriate robot program and carries out the process. Our technique utilizes these descriptions to partition and automatically assign pertinent process data, facilitating the automated configuration of the anomaly detection pipeline. To overcome limited data availability, we employ a sliding window technique for data augmentation and capitalize on the attention mechanism of the Transformer to effectively extract semantic interdependencies from the time series data. By examining the discrepancies between the reconstructed time series data and the original, we can detect anomalies related to the manufacturing process. Through experiments conducted on an actual robot workcell, we demonstrate that our approach surpasses alternative competitive concepts.

This work presents a novel method for point registration in 3D space. The proposed algorithm utilizes transformation-invariant geometry information to estimate the pose of objects based on correspondences between points in two sets. Conventional methods use geometry descriptors to find these correspondences, which can result in a large number of outliers. Most existing algorithms are error-prone when outliers are present. Instead of formulating point registration as a non-convex optimization problem, we propose an intuitive method that filters out spurious correspondences. This is achieved by evaluating three different geometry-based transformation-invariant descriptors for outlier removal. We construct fully connected graphs with the proposed descriptors on correspondences, and convert the outlier removal problem into a subgraph isomorphism problem that is solved using a binary clustering approach. The resulting inlier clustering is used to estimate the transformation between the two point sets. The effectiveness of the proposed approach is evaluated on standard 3D data and the 3DMatch scan matching dataset, and compared against existing state-of-the-art methods. Results show that our method effectively reduces outliers and performs similarly to these methods.

Software-defined vehicles (SDV) play an important role in future electrical and electronic (E&E) architectures. Their increased flexibility compared to traditional architectures is a crucial factor in the rapid development cycles of autonomous driving. Containerization and virtualization are two key technologies that enable rapid software installation and updates under the SDV framework. These two technologies have been widely adopted in cloud computing, but their performance and suitability in intelligent vehicles still has to be evaluated. In this work, we look at generic performance experiments of containerization and virtualization on both embedded and general-purpose computer systems regarding CPU, memory, network, and disk. We further investigate the impact of virtualization and containerization on the Autoware framework to evaluate scenarios that are close to real-world automotive applications. Additionally, we evaluate performance by splitting the Autoware framework into several dependent service parts, which are installed in separate containers. Extensive experimental results show that virtualization and containerization have no significant performance drop with 0-5% loss compared to a bare-metal setup in terms of CPU, memory, and network. However, both technologies suffer dramatic performance degradation on the disk side, losing 5-15% in containers and 35% in virtualization.