The ability to estimate joint parameters is essential for various applications in robotics and computer vision. In this paper, we propose CAPT: category-level articulation estimation from a point cloud using Transformer. CAPT uses an end-to-end transformer-based architecture for joint parameter and state estimation of articulated objects from a single point cloud. The proposed CAPT methods accurately estimate joint parameters and states for various articulated objects with high precision and robustness. The paper also introduces a motion loss approach, which improves articulation estimation performance by emphasizing the dynamic features of articulated objects. Additionally, the paper presents a double voting strategy to provide the framework with coarse-to-fine parameter estimation. Experimental results on several category datasets demonstrate that our methods outperform existing alternatives for articulation estimation. Our research provides a promising solution for applying Transformer-based architectures in articulated object analysis.

We propose an extrinsic calibration method for LiDAR-camera fusion systems using variations in intensities projected from camera images to the LiDAR point cloud. As the input, the proposed method uses a sequence of LiDAR data and camera images captured while moving the system. Once the camera motion is calculated, camera images are projected onto the point cloud. The variations in the projected intensities at each point are large in the presence of errors in the estimated motion or calibration parameters. Consequently, the extrinsic parameters are optimized for cost minimization based on the intensity variance. In addition, a suitable geometry is proposed for the calibration and verified using simulations. Our experimental results showed that the proposed method accurately performed calibrations using a camera and a sparse multi-beam LiDAR or one-dimensional LiDAR.

Event cameras are recent sensors that measure intensity changes in each pixel asynchronously. It is being used due to lower latency and higher temporal resolution compared to traditional frame-based camera. We propose a method of 3D model-based object tracking directly from events captured by event camera. To enable reliable and accurate tracking of objects, we use a new event representation and predict brightness increment images with motion interpolation. Results of object tracking show the new methods significantly improves tracking duration and robustness, both for perspective and fisheye cameras. Our implementation succeeds in tracking objects when the camera speed is reaching 2 m/s.

Sensor fusion has become a popular topic in robotics. However, conventional fusion methods encounter many difficulties, such as data representation differences, sensor variations, and extrinsic calibration. For example, the calibration methods used for LiDAR-camera fusion often require manual operation and auxiliary calibration targets. Implicit neural representations (INRs) have been developed for 3D scenes, and the volume density distribution involved in an INR unifies the scene information obtained by different types of sensors. Therefore, we propose implicit neural fusion (INF) for LiDAR and camera. INF first trains a neural density field of the target scene using LiDAR frames. Then, a separate neural color field is trained using camera images and the trained neural density field. Along with the training process, INF both estimates LiDAR poses and optimizes extrinsic parameters. Our experiments demonstrate the high accuracy and stable performance of the proposed method.

Perceptual Aliasing is one of the main problems in simultaneous localization and mapping (SLAM). Wrong associations between different places may lead to failure of the whole map. Research on structure information is rarely investigated among existing solutions to this problem. In cases of visual SLAM without sensors, such as LiDAR or Inertial Measurement Unit (IMU), structure information can rarely be obtained due to the sparsity of 3D points, which also makes structure analysis complex. This study provides a spherical harmonics (SH) based fast structural representation (SH-FS) in visual SLAM using sparse point clouds, which extracts the structure information from sparse points into single vector. SH-FS was applied in conventional feature-based loop closing process. Furthermore, a structure-aware loop closing method in visual SLAM was proposed to improve the robustness of SLAM systems. Moreover, our methods show a favorable performance in extensive experiments on different large-scale real world datasets.

In this paper, we present a hand-motion-based method for simultaneous articulation-model estimation and segmentation of objects in RGB-D images. The hand-motion information is first used to calculate an initial guess of the articulated model (prismatic or revolute joint) of the target object. Subsequently, the hand trajectory is used as a constraint to optimize the articulation parameters during the ICP-based alignment of the sequential point clouds of the object from the RGBD images. Finally, the target regions are selected from the cluster of aligned point clouds that move symmetrically with respect to the detected articulation model. The experimental results demonstrate the robustness of the proposed method for various types of objects.

In this paper, we propose a dynamic calibration between a mobile robot and a device using simultaneous localization and mapping (SLAM) technology, which we termed as the SLAM device, for a robot navigation system. The navigation framework assumes loose mounting of SLAM device for easy use and requires an online adjustment to remove localization errors. The online adjustment method dynamically corrects not only the calibration errors between the SLAM device and the part of the robot to which the device is attached but also the robot encoder errors by calibrating the whole body of the robot. The online adjustment assumes that the information of the external environment and shape information of the robot are consistent. In addition to the online adjustment, we also present an offline calibration between a robot and device. The offline calibration is motion-based and we clarify the most efficient method based on the number of degrees-of-freedom of the robot movement. Our method can be easily used for various types of robots with sufficiently precise localization for navigation. In the experiments, we confirm the parameters obtained via two types of offline calibration based on the degree of freedom of robot movement. We also validate the effectiveness of the online adjustment method by plotting localized position errors during a robots intense movement. Finally, we demonstrate the navigation using a SLAM device.

This paper proposes a targetless and automatic camera-LiDAR calibration method. Our approach extends the hand-eye calibration framework to 2D-3D calibration. The scaled camera motions are accurately calculated using a sensor-fusion odometry method. We also clarify the suitable motions for our calibration method. Whereas other calibrations require the LiDAR reflectance data and an initial extrinsic parameter, the proposed method requires only the three-dimensional point cloud and the camera image. The effectiveness of the method is demonstrated in experiments using several sensor configurations in indoor and outdoor scenes. Our method achieved higher accuracy than comparable state-of-the-art methods.

3D Sensing systems mounted on mobile platform are emerging and have been developed for various applications. In this paper, we propose a profiler scanning system mounted on a rover to scan and reconstruct a bas-relief with high density and accuracy. Our hardware system consists of an omnidirectional camera and a 3D laser scanner. Our method selects good projection points for tracking to estimate motion stably and reject mismatches caused by difference between the positions of laser scanner and camera using an error metric based on the distance from omnidirectional camera to scanned point. We demonstrate that our results has better accuracy than comparable approach. In addition to local motion estimation method, we propose global poses refinement method using multi modal 2D-3D registration and our result shows good consistency between reflectance image and 2D RGB image.

This paper presents a new flying sensor system to capture 3D data aerially. The hardware system, consisting of a omni-directional laser scanner and a panoramic camera, can be mounted under a mobile platform (e.g., a balloon or a crane) to achieve the aerial scanning with high resolution and accuracy. Since the laser scanner often requires several minutes to complete an omni-directional scan, the raw data is distorted seriously due to the unknown and uncontrollable movement during the scanning period. To overcome this problem, 1) we first synchronize the two sensors and spherically calibrate them together, 2) our approach then recovers the sensor motion by utilizing the spacial and temporal features extracted both from the image sequences and point clouds, and 3) finally the distorted scans can be rectified with the estimated motion and aligned together automatically. In experiments, we demonstrate that the method achieves a substantially good performance for indoor/outdoor aerial scanning in the applications such as Angkor Wat 3D preservation and manufacturing 3D survey with respect to other state-of-the-art methods.