For space missions such as deep space exploration and on-orbit operations, a high-precision 3D model of the target is a prerequisite for achieving autonomous navigation and precise manipulation. However, natural uncontrolled orbits impose strong geometry constraints and require long observation periods, while active orbital maneuvering accelerates data acquisition but increases fuel consumption and reduces mission endurance. This trade-off between maneuvering efficiency and observation completeness has become a bottleneck limiting spacecraft operations on-orbit. To address these challenges, this paper proposes a sensing-planning framework that integrates active observation with orbital maneuvering. First, a 3D reconstruction scheme based on 2D Gaussian splatting (2DGS) is designed, taking uncertainty into account. Next, the optimal observation views are estimated using Bayesian theory, followed by orbit selection combined with fuel consumption and observation time derived from orbital mechanics. Simultaneously, discrete point filtering is applied to improve the reconstruction quality of the 3D mesh in the space environment. Finally, the effectiveness of the proposed method is validated through simulations and experimental comparisons with state-of-the-art (SOTA) in a newly constructed multi-orbital observation space environment darkroom. Code and data are available at: https://github.com/YD-96/Active-2DGS and https://bhpan.buaa.edu.cn/link/AAA6508AF1B8714EF0B91A992489F2228F.

Visual 3D reconstruction is a key technology in the field of computer vision, with significant implications for tasks such as robot manipulation, autonomous driving, and virtual reality. In recent years, methods based on neural radiance fields and Gaussian splatting have gained considerable attention due to their outstanding performance. Surface mesh reconstruction based on the 2D Gaussian model achieves high accuracy, providing critical information for subsequent target-centered perception tasks. However, challenges such as self-occlusion caused by the complex structure of spacecraft, limited observation positions due to orbital constraints, and the high cost of orbital transfer restrict data acquisition. These limitations prevent comprehensive target information from being obtained, as is possible in ground-based sampling, resulting in reconstruction failures or reduced quality. To address the above problem, this paper proposes a uncertainty-aware 2D Gaussian splatting method for 3D mesh reconstruction under restricted viewpoint observation. First, the fixed color value of the 2D Gaussian ellipsoid is expanded into a probability distribution to measure uncertainty. Then, a color negative log-likelihood loss function is designed to train the Gaussian elements to estimate the mean and variance of the probability distribution. The proposed method is validated on a spacecraft reconstruction dataset collected from our local darkroom environment, demonstrating its effectiveness through qualitative and quantitative comparisons of 3D mesh reconstruction, uncertainty estimation, and novel view synthesis.

In asteroid exploration and orbital servicing missions with space robots, accurate 3D structural of the target is typically relied upon for planning landing trajectories and controlling movements. Unlike conventional neural radiance fields (NeRF) studies, which rely on full-view random sampling of targets that can be easily achieved on the ground, spacecraft operations present unique challenges due to the kinematic orbit constraint, the high cost of controlled motion, and limited fuel reserves. This results in limited observation of space targets. In order to obtain 3D structure under close-flybys and restricted observation, we proposed Uncertainty Neural Surfaces (UNS) model based on Bayesian uncertainty estimation. UNS enhance the precision of reconstructed target surfaces under constrained-views, providing guidance for subsequent imaging view design. Specifically, UNS introduces Bayesian estimation based surface uncertainty on neural implicit surfaces. The estimation is calculated based on the degree of self-occlusion of the target and the difference between rendered and actual colors. This approach enables uncertain estimation of 3D space and arbitrary view. Finally, extensive systematic evaluations and analyses of spacecraft model sampling in a local darkroom validate the sophistication of UNS in uncertainty estimation and surface reconstruction quality. Code is available athttps://github.com/YD-96/UNS.

Linear quadratic control with unknown value functions and dynamics is extremely challenging, and most of the existing studies have focused on the regulation problem, incapable of dealing with the tracking problem. To solve both linear quadratic regulation and tracking problems for continuous-time systems with unknown value functions, this article develops a discounted inverse reinforcement learning (DIRL) method that inherits the model-independent property of reinforcement learning (RL). More specifically, we first formulate a standard paradigm for solving linear quadratic control using DIRL. To recover the value function and the target control gain, an error metric is elaborately constructed, and a quasi-Newton algorithm is adopted to minimize it. Furthermore, three DIRL algorithms, including model-based, model-free off-policy, and model-free on-policy algorithms, are proposed. The latter two rely on the expert's demonstration data or the online observed data, requiring no prior knowledge of the system dynamics and value function. The stability, convergence, and existence conditions of multiple solutions are thoroughly analyzed. Finally, numerical simulations demonstrate the effectiveness of the theoretical results.

LiDAR technology holds vast potential across various sectors, including robotics, autonomous driving, and urban planning. However, the performance of current LiDAR sensors is hindered by limited field of view (FOV), low resolution, and lack of flexible focusing capability. We introduce αLiDAR, an innovative LiDAR system that employs controllable actuation to provide a panoramic FOV, high resolution, and adaptable scanning focus. The core concept of αLiDAR is to expand the operational freedom of a LiDAR sensor through the incorporation of a controllable, active rotational mechanism. This modification allows the sensor to scan previously inaccessible blind spots and focus on specific areas of interest in an adaptive manner. By modeling uncertainties in LiDAR rotation process and estimating point-wise uncertainty, αLiDAR can correct point cloud distortions resulted from significant rotation. In addition, by optimizing LiDAR's rotation trajectory, αLiDAR can swiftly adapt to dynamic areas of interest. We developed several prototypes of αLiDAR and conducted comprehensive evaluations in various indoor and outdoor real-world scenarios. Our results demonstrate that αLiDAR achieves centimeter-level pose estimation accuracy, with an average latency of only 37 ms. In two typical LiDAR applications, αLiDAR significantly enhances 3D mapping accuracy, coverage, and density by 8.5×, 2×, and 1.6× respectively, compared to conventional LiDAR sensors. Additionally, αLiDAR's adaptive rotation improves the effective sensing distance by 1.8× and increases the number of perceived objects by 1.9×. A video demonstration of αLiDAR's in action in real world is available at https://youtu.be/x4zc_I_xTaw. The code is available at https://github.com/HViktorTsoi/alpha_lidar.

We present αLiDAR, an innovative LiDAR system that incorporates a controllable active rotational mechanism to broaden the field of view (FOV), enhance resolution, and provide adaptable focusing. This system addresses the inherent limitations of traditional LiDAR sensors, such as narrow FOV, low resolution, and lack of flexible focusing capability. By scanning blind spots and dynamically focusing on areas of interest, αLiDAR significantly surpasses conventional LiDAR sensors. Our prototypes, tested under varied real-world conditions, have demonstrated marked improvements in typical LiDAR applications. Specifically, αLiDAR enhances 3D mapping accuracy, coverage, and density by factors of 8.5, 2, and 1.6, respectively. Furthermore, the adaptive rotational mechanism of αLiDAR extends the effective sensing distance by 1.8× and increases object detection by 1.9×. To see αLiDAR in action, visit our video demonstration at https://youtu.be/x4zc_I_xTaw. Both the hardware and software implementations of αLiDAR are open-sourced at https://github.com/HViktorTsoi/alpha_lidar.

Decomposing a scene into its 3D geometry, surface material textures, and illumination is a challenging but important problem in computer vision and graphics. While recent neural implicit representation based works have shown tremendous advantages, existing methods are not applicable to images illuminated by a single dynamic point light. We propose an entirely self-supervised end-to-end neural implicit representation based reflectance decomposition algorithm for objects under a dynamic point light. Our method adopts a staged training framework to estimate the geometry, light source position, and surface material textures through volume rendering, self-shadow inverse rendering, and physical model based surface rendering respectively. This scheme allows accurate recovery of the surface material textures which are coupled to the dynamic light, improving the reflectance decomposition capability. For evaluation, we collect a new dataset of several synthetic and real world objects illuminated by a moving point light. Experiments show that our method achieves superior reflectance decomposition performance compared to state-of-the-art methods, and the recovered elements can be deployed in existing graphics pipelines to perform relighting, material editing, and scene composition.

This paper investigates the use of reinforcement learning for autonomous exploration in an unknown environment. Autonomous exploration is crucial in many situations, such as urban search, security inspection, environmental mapping, etc. Traditional approaches focused on frontiers are unlikely to span a variety of enormously complex scenarios. Convergence is a little more difficult for learning-based approaches, which can adapt to many different environments. Consequently, a hierarchical exploration framework is built using frontier information. We propose a reinforcement learning-based local decision exploration model that uses deep neural networks to learn the optimal strategy from the environment. To prevent falling into local optimization, we also suggest a global rescue module to assist the robot in returning to the proper exploration track. Compared with other hierarchical methods, the framework is more effective and resilient in many contexts, greatly decreasing the total completion time and path length.

Infrastructure-assisted autonomous driving is an emerging paradigm that expects to significantly improve the driving safety of autonomous vehicles. The key enabling technology for this vision is to fuse LiDAR results from the roadside infrastructure and the vehicle to improve the vehicle's perception in real time. In this work, we propose VIPS, a novel lightweight system that can achieve decimeter-level and real-time (up to 100 ms) perception fusion between driving vehicles and roadside infrastructure. The key idea of VIPS is to exploit highly efficient matching of graph structures that encode objects' lean representations as well as their relationships, such as locations, semantics, sizes, and spatial distribution. Moreover, by leveraging the tracked motion trajectories, VIPS can maintain the spatial and temporal consistency of the scene, which effectively mitigates the impact of asynchronous data frames and unpredictable communication/compute delays. We implement VIPS end-to-end based on a campus smart lamppost testbed. To evaluate the performance of VIPS under diverse situations, we also collect two new multi-view point cloud datasets using the smart lamppost testbed and an autonomous driving simulator, respectively. Experiment results show that VIPS can extend the vehicle's perception range by 140% within 58 ms on average, and delivers a 4X improvement in perception fusion accuracy and 47X data transmission saving over existing approaches. A video demo of VIPS based on the lamppost dataset is available at https://youtu.be/zW4oi_EWOu0.

The century-old development of the automotive industry has spawned one of the greatest Cyber-Physical Systems (CPSs) in the future-unmanned vehicles. The vehicle can obtain environmental information through different sensors, map it to the virtual coordinate system of the vehicle body to make decisions, and finally generate control instructions. However, a series of factors, such as complex road scenes, defective and irregular target sparse sampling, and large coding space, pose challenges to accurate, efficient, and stable perception results. To overcome the most challenging problem of dynamic target tracking, this paper designs a two-stage detection model based on non-uniform polar voxelization sampling of irregular 3D point cloud, which is used with local registration-based search to achieve efficient multi-target tracking. Non-uniform voxelization not only balances the spatial sampling and encoding efficiency of the point cloud for the backbone, but also adapts to the feature aggregation of the detection head, thereby achieving double acceleration. Finally, we tested our model on KITTI Tracking data. The comparison results show that the calculation speed of the final model is greatly improved and the tracking accuracy is competitive in all categories.

Object detection is the most critical and foundational sensing module for the autonomous movement platform. However, most of the existing deep learning solutions are based on GPU servers, which limits their actual deployment. We present an efficient multi-sensor fusion based object detection model that can be deployed on the off-the-shelf edge computing device for the vehicle platform. To achieve real-time target detection, the model eliminates a large number of invalid point clouds through ground filtering algorithm, and then adds texture information (fused from camera image) through point cloud coloring to enhance features. The proposed PV-EncoNet efficiently encodes both the spatial and texture features of each colored point through point-wise and voxel-wise encoding, and then predicts the position, heading and class of the objects. The final model can achieve about 17.92 and 24.25 Frame per Second (FPS) on two different edge computing platforms, and the detection accuracy is comparable with the state-of-the-art models on the KITTI public dataset (i.e., 88.54% for cars, 71.94% for pedestrians and 73.04% for cyclists). The robustness and generalization ability of the PV-EncoNet for the 3D colored point cloud detection task is also verified by deploying it on the local vehicle platform and testing it on real road conditions.

Lidar and optical camera are common sensors in the sensor layer of autopilot system. Lidar can use depth data to obtain accurate relative distance and contour information of obstacles, which is not easily affected by external light conditions. Optical camera can obtain rich object/environment semantic information through high-resolution image, which is relatively mature in technology. The two different sensors are highly complementary, and previous studies show that the fusion of laser point cloud and image data can greatly improve the efficiency of object detection in out door environment. In this paper, a deep convolutional neural network detection model based on Lidar and image information features layered fusion is studied. We try different fusion depth at the CNN model to seek the best solution according to the detection performance. The experimental results on the KITTI dataset show that the detection accuracy of the fusion based on YOLOv3 is 1.08% higher than original model. Another small scale experiment with our own self-driving platform on local area also show the final fusion model can achieve better detection accuracy in real road condition.

Due to easy access to smartphones, recent years have witnessed an increasing interest in using the mobile phone as a sensing and computation platform for vehicle steering detection. However, relatively lower accuracy of smartphone sensors than on-board diagnostic (OBD)-based systems often leads to lower accuracy. We propose an ensemble learning-based model combined with the heuristic algorithm for smartphone-based vehicle steering detection in this paper. Ensemble learning has been widely recognized for its powerful generalization capability, high accuracy, and rapid convergence. However, applying the ensemble learning approach to steering detection of the smartphone-based vehicle entails many challenges due to the limitation of smartphone storage, the constraint on power consumption, and the requirement of being real-time. To address these challenges, we propose a series of techniques to reduce the complexity of the model and energy consumption, while at the same time maintaining high detection accuracy. The performance of the proposed system has been demonstrated using a real dataset and can achieve an accuracy of 97.37%. We also conduct two case studies on real road environment in Beijing with different smartphones.

Due to the unavailability of Vehicle-to-Infrastructure (V2I) communication in current transportation systems, Traffic Light Detection (TLD) is still considered an important module in autonomous vehicles and Driver Assistance Systems (DAS). To overcome low flexibility and accuracy of vision-based heuristic algorithms and high power consumption of deep learning-based methods, we propose a lightweight and real-time traffic light detector for the autonomous vehicle platform. Our model consists of a heuristic candidate region selection module to identify all possible traffic lights, and a lightweight Convolution Neural Network (CNN) classifier to classify the results obtained. Offline simulations on the GPU server with the collected dataset and several public datasets show that our model achieves higher average accuracy and less time consumption. By integrating our detector module on NVidia Jetson TX1/TX2, we conduct on-road tests on two full-scale self-driving vehicle platforms (a car and a bus) in normal traffic conditions. Our model can achieve an average detection accuracy of 99.3 percent (mRttld) and 99.7 percent (Rttld) at 10Hz on TX1 and TX2, respectively. The on-road tests also show that our traffic light detection module can achieve <; + 1:5m errors at stop lines when working with other selfdriving modules.

Among various tools invented to help improve people's oral health, water flossers can achieve better performance than traditional and electronic toothbrushes, and are less harmful than dental floss, especially for those with orthodontic teeth or tooth implant surgeries. However, the water flossers available in the market serve no monitoring or recording functions that can help consumers clean their teeth in a more efficient way. To capture users' motions, this study develops a novel smart water flosser, installing an Inertial Measurement Unit (IMU) sensor on the handle of the flosser. We determine the motion cycle using signal processing techniques and extract a set of statistical characteristics from the data set. We then train and compare different machine learning models as classifiers to recognize the motions of the handle. We find that the Random Forest model achieves the best detection accuracy at 97% and 85% of the whole feature set and optimized set, respectively. Finally we implement an Android App that connects the smart water flosser with a Bluetooth module to show the washing area in real-time and record relevant information for further guidance.

Smartphone built-in sensors are essential components of vehicle steering mode recognition. Related driver assistance systems (DAS) and abnormal driving behavior detection systems have been studied for many years. However, the existing solutions and systems simply collect sensor data with a fixed sliding window and fuse data from multiple sensors using simple thresholds to detect different driving behaviors. The weakness of these solutions can have an adverse impact on the energy consumption and computation complexity of power-limited devices such as smartphones, and may provide coarse-grained results. In this paper, we present a new method to reduce both the energy consumption and the computation complexity, and improve the recognition accuracy of vehicle steering patterns using the following three improvements: 1) a MultiWave filter is designed to replace the fixed sliding window, which is used to identify vehicle steering events; 2) a set of eight statistical sensor features reflecting the vehicle steering modes are identified by extracting statistical features from different sensors and different axes; 3) different machine learning methods are compared based on this feature set in order to improve classifier training (Decision Tree and Random Forest). Evaluation results based on real vehicle datasets show that our improved classifiers have high real-time recognition accuracy of the five most common steering modes: left/right turns, left/right lane changes and U-turns. We also took a simple on-road testing, in which both of the two models detected all the steering behaviors under low vehicle speed.

In this paper, we propose a tooth brushing monitoring system based on acoustic inputs through an asymmetrical sound-field detector. This detector consists of a throat microphone and a Bluetooth earphone equipped on the user's neck and ear, respectively. This system can capture unique acoustic signals generated by the movement of the toothbrush on the surfaces of teeth via the detector. The throat microphone captures the brushing sound travelling through gums, bones, and muscles, which forms unique patterns with less attenuation than the sound travelling through the air. The Bluetooth earphone captures the brushing sound through the air. The tooth surface is divided into 16 parts for detection. By adopting machine learning models with the input of acoustic features from both time and frequency domains, we build a high accuracy detector to distinguish the brushing events happened at each of the 16 parts of the tooth surface. We employ Support Vector Machine (SVM), Hidden Markov Model (HMM), K-Means, C4.5 and Random Forest (RF) to evaluate the performance of our detection system. Experiments show that the RF model performs the best and achieves an average accuracy of 85.69\%. Based on the pre- trained model, we develop an Android-based APP to monitor the user's daily tooth brushing time and help the user form a good habit of tooth brushing.

Aggressive driving is the main cause of traffic accidents all over the world. Aggressive steering is the second leading cause of traffic accidents, just behind speeding. To detect aggressive steering and encourage drivers to cultivate better driving behaviors, automatic recognition of different vehicle steering modes is required so a decision can be taken on whether the behavior is aggressive or not. This paper investigates various patterns of turning, changing lanes and U-turns, and then design and implement a system termed MultiWave that utilizes the gyroscope of a smartphone to automatically detect vehicle steering patterns. MultiWave divides driving behaviors into different categories by analyzing the data collected by gyroscope sensors. Since it analyzes the gyroscopic sensor data of turning, changing lanes and U-turns, MultiWave is flexible and robust for most urban situations. The classification accuracy of MultiWave can reach averages of 92%, 76.5%, and 87% for vehicle turning, changing lanes and U-turn, respectively.