3D Gaussian Splatting (3DGS) allows flexible adjustments to scene representation, enabling continuous optimization of scene quality during dense visual simultaneous localization and mapping (SLAM) in static environments. However, 3DGS faces challenges in handling environmental disturbances from dynamic objects with irregular movement, leading to degradation in both camera tracking accuracy and map reconstruction quality. To address this challenge, we develop an RGB-D dense SLAM which is called Gaussian Splatting SLAM in Dynamic Environments (Gassidy). This approach calculates Gaussians to generate rendering loss flows for each environmental component based on a designed photometricgeometric loss function. To distinguish and filter environmental disturbances, we iteratively analyze rendering loss flows to detect features characterized by changes in loss values between dynamic objects and static components. This process ensures a clean environment for accurate scene reconstruction. Compared to state-of-the-art SLAM methods, experimental results on open datasets show that Gassidy improves camera tracking precision by up to 97.9 % and enhances map quality by up to 6 %. Video of experiments is available here: https://www.wixsite.com.com/wen-Gassidy.

To mitigate unforeseen operational interruptions caused by potential malfunctions in robotic systems employed in industrial automation, we propose an innovative strategy for anomaly detection that incorporates a Transformer-based reconstruction network for identifying irregularities in skill-oriented manufacturing. Leveraging a semantic representation of processes, products, and resources, a semantic manufacturing execution system synthesizes an appropriate robot program and carries out the process. Our technique utilizes these descriptions to partition and automatically assign pertinent process data, facilitating the automated configuration of the anomaly detection pipeline. To overcome limited data availability, we employ a sliding window technique for data augmentation and capitalize on the attention mechanism of the Transformer to effectively extract semantic interdependencies from the time series data. By examining the discrepancies between the reconstructed time series data and the original, we can detect anomalies related to the manufacturing process. Through experiments conducted on an actual robot workcell, we demonstrate that our approach surpasses alternative competitive concepts.

This work presents a novel method for point registration in 3D space. The proposed algorithm utilizes transformation-invariant geometry information to estimate the pose of objects based on correspondences between points in two sets. Conventional methods use geometry descriptors to find these correspondences, which can result in a large number of outliers. Most existing algorithms are error-prone when outliers are present. Instead of formulating point registration as a non-convex optimization problem, we propose an intuitive method that filters out spurious correspondences. This is achieved by evaluating three different geometry-based transformation-invariant descriptors for outlier removal. We construct fully connected graphs with the proposed descriptors on correspondences, and convert the outlier removal problem into a subgraph isomorphism problem that is solved using a binary clustering approach. The resulting inlier clustering is used to estimate the transformation between the two point sets. The effectiveness of the proposed approach is evaluated on standard 3D data and the 3DMatch scan matching dataset, and compared against existing state-of-the-art methods. Results show that our method effectively reduces outliers and performs similarly to these methods.

Modern vehicles include an increasing amount of software, e.g., for autonomous driving capabilities, connectivity, and personalized user experience. The capabilities in current vehicles are still mostly provided by multiple separated embedded systems, while the current trend goes toward purely software-defined vehicles (SDV). Traditional distributed electrical/electronic (E/E) architectures have tightly coupled hardware/software, and the computational power is optimized for the included feature set. For SDVs, a centralized E/E architecture utilizing high-performance computers has been proposed. In contrast to individual embedded systems with limited and fixed functionality, combing a large set of individual software components in a single system leads to a high complexity in the proper allocation of resources. Model-based system engineering (MBSE) has been promoted in the automotive industry to handle complex system design. However, existing MBSE approaches focus mainly on traditional E/E architectures. In this work, we propose an automated and model-based approach that can address the resource allocation problem in SDVs. Users can formally describe the vehicle’s resources, safety/non-safety requirements, and optimization objectives based on existing software engineering standards. The proposed method is not restricted to specific system models, requirements, or optimization goals and is, therefore, compatible with other E/E architectures. By introducing a model-independent transformation from the model information to solver-independent optimization formulas, the resource allocation problem can be solved automatically by a wide range of state-of-the-art solvers. We demonstrate the applicability of this approach in a SDV scenario with a high-performance computer and multiple applications.

Software-defined vehicles (SDV) play an important role in future electrical and electronic (E&E) architectures. Their increased flexibility compared to traditional architectures is a crucial factor in the rapid development cycles of autonomous driving. Containerization and virtualization are two key technologies that enable rapid software installation and updates under the SDV framework. These two technologies have been widely adopted in cloud computing, but their performance and suitability in intelligent vehicles still has to be evaluated. In this work, we look at generic performance experiments of containerization and virtualization on both embedded and general-purpose computer systems regarding CPU, memory, network, and disk. We further investigate the impact of virtualization and containerization on the Autoware framework to evaluate scenarios that are close to real-world automotive applications. Additionally, we evaluate performance by splitting the Autoware framework into several dependent service parts, which are installed in separate containers. Extensive experimental results show that virtualization and containerization have no significant performance drop with 0-5% loss compared to a bare-metal setup in terms of CPU, memory, and network. However, both technologies suffer dramatic performance degradation on the disk side, losing 5-15% in containers and 35% in virtualization.

This paper proposes an efficient, fully convolutional neural network to generate robotic grasps by using 300×300 depth images as input. Specifically, a residual squeeze-and-excitation network (RSEN) is introduced for deep feature extraction. Following the RSEN block, a multi-scale spatial pyramid module (MSSPM) is developed to obtain multi-scale contextual information. The outputs of each RSEN block and MSSPM are combined as inputs for hierarchical feature fusion. Then, the fused global features are upsampled to perform pixel-wise learning for grasping pose estimation. The experimental results on Cornell and Jacquard grasping datasets indicate that the proposed method has a fast inference speed of 5ms while achieving high grasp detection accuracy of 96.4% and 94.8% on Cornell and Jacquard, respectively, which strikes a balance between accuracy and running speed. Our method also gets a 90% physical grasp success rate with a UR5 robot arm.

Rotation invariance is a crucial property for 3D object classification, which is still a challenging task. State-of-the-art deep learning-based works require a massive amount of data augmentation to tackle this problem. This is however inefficient and classification accuracy suffers a sharp drop in experiments with arbitrary rotations. We introduce a new descriptor that can globally and locally capture the surface geometry properties and is based on a combination of spherical harmonics energy and point feature representation. The proposed descriptor is proven to fulfill the rotation-invariant property. A limited bandwidth spherical harmonics energy descriptor globally describes a 3D shape and its rotation-invariant property is proven by utilizing the properties of a Wigner D-matrix, while the point feature representation captures the local features with a KNN to build the connection to its neighborhood. We propose a new network structure by extending PointNet++ with several adaptations that can hierarchically and efficiently exploit local rotation-invariant features. Extensive experimental results show that our proposed method dramatically outperforms most state-of-the-art approaches on standard rotation-augmented 3D object classification benchmarks as well as in robustness experiments on point perturbation, point density, and partial point clouds.

This paper presents two different approaches to generate a time local-optimal and jerk-limited trajectory with blends for a robot manipulator under consideration of kinematic constraints. The first approach generates a trajectory with blends based on the trapezoidal acceleration model by formulating the problem as a nonlinear constraint and a non-convex optimization problem. The resultant trajectory is locally optimal and approximates straight-line movement while satisfying the robot manipulator’s constraints. We apply the bridged optimization strategy to reduce the computational complexity, which borrows an idea from model predictive control by dividing all waypoints into consecutive batches with an overlap of multiple waypoints. We successively optimize each batch. The second approach is a combination of a trapezoidal acceleration model with a 7-degree polynomial to form a path with blends. It can be efficiently computed given the specified blending parameters. The same approach is extended to Cartesian space. Furthermore, a quaternion interpolation with a high degree polynomial under consideration of angular kinematics is introduced. Multiple practical scenarios and trajectories are tested and evaluated against other state-of-the-art approaches.

Inferring a complete 3D geometry given an in-complete point cloud is essential in many vision and robotics applications. Previous work mainly relies on a global feature extracted by a Multi-layer Perceptron (MLP) for predicting the shape geometry. This suffers from a loss of structural details, as its point generator fails to capture the detailed topology and structure of point clouds using only the global features. The irregular nature of point clouds makes this task more challenging. This paper presents a novel method for shape completion to address this problem. The Transformer structure is currently a standard approach for natural language processing tasks and its inherent nature of permutation invariance makes it well suited for learning point clouds. Furthermore, the Transformer’s attention mechanism can effectively capture the local context within a point cloud and efficiently exploit its incomplete local structure details. A morphing-atlas-based point generation network further fully utilizes the extracted point Transformer feature to predict the missing region using charts defined on the shape. Shape completion is achieved via the concatenation of all predicting charts on the surface. Extensive experiments on the Completion3D and KITTI data sets demonstrate that the proposed PCTMA-Net outperforms the state-of-the-art shape completion approaches and has a 10% relative improvement over the next best-performing method.

We propose a surface-to-surface (S2S) point registration algorithm by exploiting the Gaussian Process Implicit Surfaces for partially overlapping 3D surfaces to estimate the 6D pose transformation. Unlike traditional approaches, that separate the corresponding search and update steps in the inner loop, we formulate the point registration as a nonlinear non-constraints optimization problem which does not explicitly use any corresponding points between two point sets. According to the implicit function theorem, we form one point set as a Gaussian Process Implicit Surfaces utilizing the signed distance function, which implicitly creates three manifolds. Points on the same manifold share the same function value, indicated as {1, 0, -1}. The problem is thus converted into finding a rigid transformation that minimizes the inherent function value. This can be solved by using a Gauss-Newton (GN) or Levenberg-Marquardt (LM) solver. In the case of a partially overlapping 3D surface, the Fast Point Feature Histogram (FPFH) algorithm is applied to both point sets and a Principal Component Analysis (PCA) is performed on the result. Based on this, the initial transformation can then be computed. We conduct experiments on multiple point sets to evaluate the effectiveness of our proposed approach against existing state-of-the-art methods.

This paper presents an approach to generate the time-optimal trajectory for a robot manipulator under certain kinematic constraints such as joint position, velocity, acceleration, and jerk limits. This problem of generating a trajectory that takes the minimum time to pass through specified waypoints is formulated as a nonlinear constraint optimization problem. Unlike prior approaches that model the motion of consecutive waypoints as a Cubic Spline, we model this motion with a seven-segment acceleration profile, as this trajectory results in a shorter overall motion time while staying within the bounds of the robot manipulator's constraints. The optimization bottleneck lies in the complexity that increases exponentially with the number of waypoints. To make the optimization scale well with the number of waypoints, we propose an approach that has linear complexity. This approach first divides all waypoints to consecutive batches, each with an overlap of two waypoints. The overlapping waypoints then act as a bridge to concatenate the optimization results of two consecutive batches. The whole trajectory is effectively optimized by successively optimizing every batch. We conduct experiments on practical scenarios and trajectories generated by motion planners to evaluate the effectiveness of our proposed approach over existing state-of-the-art approaches.