This paper presents a gradient-based fault detection method for pitch control systems in avionics. On the basis of the dynamic model of the airplane, the proposed method detects both operator and sensor faults by monitoring the online data. By integrating fault-related behaviors over an extended time window, the method effectively amplifies small changes caused by incipient faults, improving detectability. Theoretical analysis reveals that under normal flight conditions, the gradient has a zero expected value and a finite, analytically tractable variance. These characteristics make the method compatible with traditional fault detection approaches. Sufficient tests on real flight data verify its ability to detect hard-to-identify faults.

The series includes two parts, articulating the two novel avenues of research on intelligent fault diagnosis (FD) for nonlinear feedback control systems. In Part I of the series, we design a novel FD paradigm by elaborating an invertible neural network (INN) for feedback control systems. With the aid of a left manifold, the core idea behind the INN-based FD scheme is as follows: 1) formulation of residual generator used for FD as a projection of system data onto the null space that has the same dimension as system outputs; 2) in a topological space, elaboration of a homeomorphism that delivers an invertible relationship between system outputs and residual signals when the system input is given; and 3) skillful introduction of both the master and slave objective functions to achieve system/parameter identification with information loseless property. Comparing with the existing FD approaches, the three superior strengths of the proposed FD scheme deserving mentation are as follows: 1) it specializes in nonlinear feedback control systems; 2) it can effectively avoid the overfitting problem when approximating or learning nonlinear system dynamics; and 3) control theory guides the whole design, ensuring the interpretability of the learning process. Finally, two studies on nonlinear systems demonstrate the feasibility of the invertible left manifold (ILM)-based FD strategy. Part I would contribute to the future development of machine learning (ML)-based system identification and explainable FD approaches, and also benefits the right manifold-based FD designs in Part II.

Hand-held augmented reality (AR) offers accessible, interactive rehabilitation options for patients with upper limb motor deficits. Incorporating hand-involved interactions (e.g., tangible and mid-air interactions) into hand-held AR provides patients with intuitive manners to perform rehabilitation exercises mimicking real-world activities. Previous work has shown the importance of user experience and motor performance in rehabilitation systems, but little was known in the literature regarding the impact of hand-involved interactions in hand-held AR on user experience and motor performance in rehabilitation exercises. Hence, this study aims to evaluate user experience and motor performance when using three types of hand-involved interactions in hand-held AR rehabilitation: (1) tangible cube (i.e., a space-multiplexed tangible interaction with a physical cube acting as a real proxy to manipulate a virtual object in the same form); (2) tangible controller (i.e., a time-multiplexed tangible interaction with a physical controller applied to manipulate a virtual object); and (3) hand motion (i.e., a form of mid-air interaction to move a virtual object with hands). Based on the findings from self-report, electroencephalography (EEG), and performance measures, this study reveals the advantages of the tangible cube over the tangible controller, both superior to the hand motion in hand-held AR rehabilitation regarding user experience and motor performance. This study offers new understanding of the advantages and disadvantages of various interaction techniques in hand-held AR rehabilitation, emphasizing crucial design considerations for these systems, with a focus on user experience and motor performance in upper limb rehabilitation.

In industrial processes, quality variables are typically sampled at a considerably lower frequency than system inputs due to technical or cost constraints. Dynamic soft sensors utilize temporal prediction to bridge these sampling gaps, thus enabling real-time closed-loop control. However, existing approaches primarily focus on one-step prediction accuracy, potentially leading to significant deviations in long-term predictions. In addition, these methods are incapable of evaluating the reliability of prediction results, subsequently increasing the potential risk of closed-loop systems. To tackle these challenges, this study presents a novel regression modeling approach based on the conditional variational autoencoder (CVAE) framework. In contrast to traditional regression approaches, this method focuses on modeling the transition probability distribution of the system, allowing the model to produce a range of credible quality variable predictions via Monte Carlo (MC) sampling. Based on the CVAEs, the sequential MC method is further employed to simulate diverse potential system state trajectories, thereby achieving multistep soft measurement prediction. Compared with traditional soft measurement techniques, the proposed method demonstrates lower prediction biases and the capacity to assess the credibility of prediction results from a probabilistic standpoint. When online quality variables are assessed by the laboratory, this method can update predictions utilizing the resampling scheme. Two case studies are offered to validate the effectiveness of the proposed scheme.

Virtual object manipulation as a key feature has been studied in virtual reality (VR) environments. Previous studies highlighted user experience on three basic types of virtual object manipulation, translation, rotation and scaling. However, prior literature mainly studied task performance in manipulation modes with different degrees of freedom (DoF), and few studies assessed user experience by evaluating the psychological response, such as mental workload on these three basic manipulation types in virtual environments. This paper compared manipulation modes with 1DoF and 3DoF to assess users’ mental workload as a critical indicator of user experience by electroencephalogram (EEG) measurement and questionnaires in manipulation tasks on the webpage with VR effects (also known as WebVR). By applying signal processing and statistical methods to analyze EEG data from ten subjects, the results demonstrated that the participants generally perceive less mental workload by 1DoF manipulation modes than 3DoF on WebVR. Besides, this study also found some different results between objective and subjective data.

Sharing short personalized videos to various social media networks has become quite popular in recent years. This raises the need for digital retouching of portraits in videos. However, applying portrait image editing directly on portrait video frames cannot generate smooth and stable video sequences. To this end, we present a robust and easy-to-use parametric method to reshape the portrait in a video to produce smooth retouched results. Given an input portrait video, our method consists of two main stages: stabilized face reconstruction, and continuous video reshaping. In the first stage, we start by estimating face rigid pose transformations across video frames. Then we jointly optimize multiple frames to reconstruct an accurate face identity, followed by recovering face expressions over the entire video. In the second stage, we first reshape the reconstructed 3D face using a parametric reshaping model reflecting the weight change of the face, and then utilize the reshaped 3D face to guide the warping of video frames. We develop a novel signed distance function based dense mapping method for the warping between face contours before and after reshaping, resulting in stable warped video frames with minimum distortions. In addition, we use the 3D structure of the face to correct the dense mapping to achieve temporal consistency. We generate the final result by minimizing the background distortion through optimizing a content-aware warping mesh. Extensive experiments show that our method is able to create visually pleasing results by adjusting a simple reshaping parameter, which facilitates portrait video editing for social media and visual effects.

The Internet of Vehicles (IoV) is one important application scenarios for the development of the Internet of things. The software-defined network (SDN) and fog computing could effectively improve the IoV network dynamics, which enables the application to achieve better performance by offloading some tasks to fog node or cloud center. Current computation offloading approaches for IoV and fog computing mostly focus on resource utilization. However, the energy-aware offloading has not been adequately addressed, especially for IoV systems with many battery-powered roadside units (RSU) and electric vehicles (EV). In this paper, we study the offloading problem in SDN and fog computing-based IoV systems. An energy-aware dynamic offloading scheme is proposed to prolong the running time of the IoV system by leveraging available battery power to execute more applications. The remaining battery power is defined as a dynamic weight factor in the execution cost model to adjust the optimization objective. Meanwhile, the dependence between applications is also taken into consideration in the cost model. A heuristic optimization algorithm is designed to solve the optimization problem. We conducted comprehensive experiments and results have shown that the offloading scheme could execute more applications with the available battery power under the constraints of application dependence.

We present a CPU-based real-time cloth animation method for dressing virtual humans of various shapes and poses. Our approach formulates the clothing deformation as a high-dimensional function of body shape parameters and pose parameters. In order to accelerate the computation, our formulation factorizes the clothing deformation into two independent components: the deformation introduced by body pose variation (Clothing Pose Model) and the deformation from body shape variation (Clothing Shape Model). Furthermore, we sample and cluster the poses spanning the entire pose space and use those clusters to efficiently calculate the anchoring points. We also introduce a sensitivity-based distance measurement to both find nearby anchoring points and evaluate their contributions to the final animation. Given a query shape and pose of the virtual agent, we synthesize the resulting clothing deformation by blending the Taylor expansion results of nearby anchoring points. Compared to previous methods, our approach is general and able to add the shape dimension to any clothing pose model. Furthermore, we can animate clothing represented with tens of thousands of vertices at 50+ FPS on a CPU. We also conduct a user evaluation and show that our method can improve a user's perception of dressed virtual agents in an immersive virtual environment (IVE) compared to a realtime linear blend skinning method.