Point cloud classification is critical for three-dimensional scene understanding. However, in real-world scenarios, depth cameras often capture partial, single-view point clouds of objects with different poses, making their accurate classification a challenge. In this paper, we propose a novel point cloud classification network that captures the detailed spatial structure of objects by constructing tetrahedra, which is different from point-wise operations. Specifically, we propose a RISpaNet block to extract rotation-invariant features. A rotation-invariant property generation module is designed in RISpaNet for constructing rotation-invariant tetrahedron properties (RITPs). Meanwhile, a multi-scale pooling module and a hybrid encoder are used to process RITPs to generate integrated rotation-invariant features. Further, for single-view point clouds, a complete point cloud auxiliary branch and a part-whole correlation module are jointly employed to obtain complete point cloud features from partial point clouds. Experimental results show that this network performs better than other state-of-the-art methods, evaluated on four public datasets. We achieved an overall accuracy of 94.7% (+2.0%) on ModelNet40, 93.4% (+5.9%) on MVP, 94.7% (+6.3%) on PCN and 94.8% (+1.7%) on ScanObjectNN. Our project website is https://luxurylf.github.io/RISpaNet_project/.

Three-dimensional reconstruction can help robots and vehicles understand their surroundings for subsequent navigation and manipulation tasks. However, in the case of target occlusion, it is difficult for visual sensors to acquire complete information about objects. In this work, we propose a progressive refinement generative adversarial network (PR-GAN) to recover object shapes guided by transition-awareness. This method directly predicts the missing point cloud from the partial point cloud. Our PR-GAN contains a progressive generation module (PGM) and a discriminator. A self-attention-based encoder is proposed in PGM to capture contextual information between local and global features. To guide encoders in generating accurate point clouds, we further propose a progressive fusion module (PFM) that extracts transition information between point clouds of different scales. Moreover, a part-whole correlation module (PWCM) is designed to extract the transition-awareness between the partial and the whole point clouds to further preserve the details. With the above modules, we enhance the spatial logic perception capability of the network so that PR-GAN can fully extract point cloud features and predict the high-fidelity point cloud. Experimental results show that PR-GAN performs better compared to other methods, evaluated on three public datasets. The code is available at https://github.com/luxurylf/PR-GAN.

Digital twin (DT), as an enabling technology and means for implementing advanced concepts such as smart manufacturing, Industry 4.0, and industrial metaverse, have garnered significant attention from the academic and business communities. However, the traditional DT framework becomes overstretched in the growing number of sophisticated technological applications. This paper proposes a six-dimension model and standardized framework of digital triplet (D-Tri) with the construction of a parallel triplet. This novel model incorporates a third system, the parallel triplet, which is characterized by high fidelity, local digitization, improved monitorability, superior scalability, and independent controllability. The proposed D-Tri also standardizes a data and knowledge dual-driven architecture between physical, parallel, and digital triplets during the period of operation, monitoring, and maintenance. A military aircraft horizontal tail control system is illustrated as a D-Tri case to demonstrate the completeness and superiority of D-Tri. With the further development of the six-dimension model, creation tools, and implementation technology, D-Tri can provide merit for application ideas and schemes in different fields.

Tailplane control system (TCS) is a key component to ensure pitch maneuverability and horizontal stability in flight control. However, due to inherent sealing, time-varying, and uncertainty, conventional control methods involve enormous challenges to guarantee optimal operation of the TCS. This article proposes a digital twin-based predictive control method, called twin predictive control (TPC), to explore tailplane optimal control under complex conditions. First, a digital twin of the TCS is established as a predictive model, and twin adaptation is set up to overcome parameter time-varying and uncertainty for supporting accurate prediction. Then, an optimization model is constructed combining tracking error and control adjustment. Based on model transformation, the optimization objective is theoretically proved convex and coupled with the active-set method to enable efficient optimization solving. Finally, the TPC method, integrating the digital twin and the optimization model, is implemented into a physical experimental system to verify the effectiveness of the proposed method. The experimental results and comparisons show that the TPC method can significantly improve tracking and antiinterference performance. Furthermore, it shows merits in excessive adjustment suppression.

A reliable and accurate visual object 6DoF pose tracking system for texture-less objects plays an important role in various fields of modern industry hence it has been a hot research topic for decades. Traditional feature-based pose tracking methods require rich feature points on objects, and it cannot handle texture-less objects. To tackle this problem, the paper proposes a novel edge-based method for continuous 6DOF pose tracking of texture-less objects. The pose of the object is estimated by minimizing the matching error between extracted image edges and re-projected CAD model edges. The matching error is represented using the Directional Chamfer Matching (DCM) Tensor. Compared with previous methods, the proposed system improves the overall performance of pose tracking system in two ways. Firstly, the method proposes an analytical mathematic model in the optimization process; Secondly, the method proposes an adaptive edge point weighting algorithm to tackle the occlusion or edge weakness problem. Both methods help improve the accuracy and robustness of the pose estimation system. With the benefit of GPU acceleration on DCM Tensor calculation the proposed method could run in real time. Extensive experiments are conducted both on public datasets and CG-rendered synthetic datasets to validate the accuracy and robustness of the proposed algorithm against other state-of-the-art methods.

Traditional mapping modules in Visual Simultaneous Localization and Mapping (i.e. Visual SLAM) systems can only estimate 3D information of isolated sparse or semi-dense feature points. But there are lots of object instances in the environments which geometric information can be utilized to enhance the quality of mapping and localization. Hence, it is required for the Visual SLAM system to utilize high-dimensional features like object instances or structural lines in mapping and localization. To meet the gap between the above requirements and the traditional implementation of Visual SLAM systems, we present in this paper a novel Visual SLAM method that can effectively utilize texture-less object instances for mapping and localization. The proposed Visual SLAM method includes newly designed feature extraction, matching, localization and mapping modules, which jointly use object features and point features to estimate camera 6-DOF poses and do richer map construction. A group of organized raster points is used to represent objects during feature matching and pose estimation process in the proposed Visual SLAM pipeline. Owing to the object feature fusion in the co-visibility graph it could conduct scale aware bundle adjustments to reduce accumulated error. The advantages of proposed Visual SLAM method are demonstrated through experiments conducted both on synthetic datasets and real-world datasets.

The parameter monitoring is desirable in mission-critical applications such as satellite primary energy systems (Power Conditioning Unit, PCU). A fourth-order converter called superbuck is often used to connect solar arrays to the rest of the power system because of its excellent properties of providing continuous input current. Fast parameter monitoring is a mandatory step in order to make a suitable response to predict these failures. Here, a model-based dynamic health monitoring method based on parameters monitoring for superbuck is presented. To address the aforementioned issue, a forgetting factor recursive least squares parameter estimation (FFRLS) method is adopted to estimate the component parameters of the superbuck converter. Finally, simulation experiment reveal the validity of the proposed approach. The proposed approach enables a flexible solution for improving fault tolerance and awareness in superbuck and other power electronics systems.

Facial landmark localization is a crucial initial step Driver Inattention Monitoring. The aim of this paper is to localize driver facial landmarks across large rotation, say [-90°, +90°] in yaw rotation, to cope with real driving conditions. The paper proposes a flexible pipe-line for creating automatically labeled face image to supply wanted dataset. The benefits of CG (Computer Graphics) techniques such as 3D face modelling and morphing, photorealistic rendering and ground truth generation are utilized. To the best of our knowledge this is the first time to combine CG rendering and automatic ground truth labelling techniques with face landmark localization algorithms. The effectiveness of the CG rendered data is proved by cross validation with Multi-PIE dataset. Landmark localization across large rotation is obtained by a system simply integrating the off the-shelves algorithms and trained with the CG rendered data. The experiments of the implemented system on Multi-PIE and real persons show that it could localize facial landmarks across large rotation accurately and in real time.

The state of art Visual SLAM is going from sparse feature to semi-dense feature to provide more information for environment perception, whereas the semi-dense methods often suffer from inaccurate depth map estimation and are easy to become instable for some real-world scenarios. The paper proposes to extend the ORB-SLAM2 framework, which is a robust sparse feature SLAM system tracking camera motion with map maintenance and loop closure, by introducing the unified spherical camera model and the semi-dense depth map. The unified spherical camera model fits the omnidirectional camera well, therefore the proposed Visual SLAM system could handle fisheye cameras which are commonly installed on modern vehicles to provide larger perceiving region. In addition to the sparse corners features the proposed system also utilizes high gradient regions as semi-dense features, thereby providing rich environment information. The paper presents in detail how the unified spherical camera model and the semi-dense feature matching are fused with the original SLAM system. Both accuracies of camera tracking and estimated depth map of the proposed SLAM system are evaluated using real-world data and CG rendered data where the ground truth of the depth map is available.

Possibilistic c-means (PCM) based clustering algorithms are widely used in the literature. In this paper, we develop a noise level based PCM (NPCM) clustering algorithm. The advantage of NPCM is that strong prior information of the dataset is not required, and NPCM needs two kinds of information that is intuitive to specify for the clustering task, i.e., information of the cluster number and information of the property of clusters. More specifically, there are two parameters in NPCM: one specifies the possibly over-specified cluster number, and the other characterizes the closeness of clusters in the clustering result. Both parameters are not required to be exactly specified. Furthermore, we find that the update of bandwidth in adaptive PCM (APCM) is a positive feedback process and the adaptive bandwidth-uncertainty mechanism adopted in NPCM makes this positive feedback process more stronger, which leads to a faster convergence rate. Experiments show that the clustering process can be effectively controlled by the parameters.

This paper proposes a method that realizes moving object detection (MOD) and static obstacle detection (SOD) in real time utilizing the fisheye cameras of the around viewing system (AVM). The topview of the AVM is used to calculate the vehicles movement between two frames using homograph estimation. Image features are detected and tracked evenly using cell detection technique. Then the features are projected onto the unit sphere of the fisheye camera model and the epipolar constraint is used to discriminate moving features from static ones. Moving features are clustered into different objects according to their position and orientation. 3D position of the static feature is calculated using triangular principle and potential obstacle is defined as objects above the ground and near the vehicle. The experiment shows the proposed method is robust and accurate for MOD and SOD.

The paper proposes a novel Robust Discriminative Regression (RDR) to handle the partial facial landmarks invisible problem for facial landmark localization. RDR consists of multiple partial feature regressors and a regression tree combination strategy to copes with the partial invisible problem together with the optimal multi output combination problem. The RDR is implemented with SIFT features and Linear Regression to achieve the balance of accuracy and computation efficiency. Experiments on two widely used “face in-the-wild” databases (LFPW and COFW) show that the proposed RDR outperforms other state-of-the-art facial landmark localization methods especially in the cases of partial occlusion and large pose variation.