Industrial soft sensing is crucial for accurate process monitoring through reliable inference of dominant sensor variables. However, developing effective data-driven soft sensor models presents challenges, such as achieving domain adaptability, addressing incomplete sensor labels, and learning stochastic data variability. To overcome these challenges, we propose a deep variational potential flow (DVPF) framework for cross-domain soft sensor modeling, taking into account the lack of sensor labels in the target domain. Our framework introduces sequential variational Bayes with recurrent neural network (RNN) parameterization to address the maximum likelihood estimation problem that characterizes cross-domain soft sensing. Central to the framework is a potential flow that performs unsupervised Bayesian inference on the RNN-extracted features to obtain an exact representation of the intractable posterior distribution. Together, these DVPF components learn domain-adaptable features that effectively capture complex cross-domain process dynamics and data variability. We validate the proposed DVPF on a real industrial multiphase flow process across varying operating modes. The results show that the DVPF demonstrates superior performance in cross-domain soft sensing compared to existing deep feature-based domain adaptation methods.

Higher-order sensor networks are more accurate in characterizing the nonlinear dynamics of sensory time-series data in modern industrial settings by allowing multi-node connections beyond simple pairwise graph edges. In light of this, we propose a deep spatio-temporal hypergraph convolutional neural network for soft sensing (ST-HCSS). In particular, our proposed framework is able to construct and leverage a higher-order graph (hypergraph) to model the complex multi-interactions between sensor nodes in the absence of prior structural knowledge. To capture rich spatio-temporal relationships underlying sensor data, our proposed ST-HCSS incorporates stacked gated temporal and hypergraph convolution layers to effectively aggregate and update hypergraph information across time and nodes. Our results validate the superiority of ST-HCSS compared to existing state-of-the-art soft sensors, and demonstrates that the learned hypergraph feature representations aligns well with the sensor data correlations. The code is available at https://github.com/htew0001/ST-HCSS.git

Soft robotic systems are highly susceptible to sensor noise arising from hardware imperfections, environmental disturbances, and the intrinsic compliance of soft materials. These noisy measurements can obscure essential state information and degrade performance in both perception and control tasks. In this paper, we introduce a Contrastive Denoising Variational Recurrent Neural Network (CD-VRNN) designed to address this challenge. The proposed model learns to decompose time-series data into clean signal and noise pathways, improving state estimation accuracy for soft robotic platforms. By incorporating a contrastive objective that enforces a separation between signal-related and noise-related latent representations, the proposed CD-VRNN more effectively isolates noise while retaining critical temporal features. In addition, we incorporate conditional flow-based priors for expressive, state-dependent distributions and skip-connected decoders that preserve subtle signal variations. Experimental results on a pneumatic soft robot and multiple public time-series datasets show that CD-VRNN consistently outperforms existing approaches for denoising and downstream task performance, demonstrating the model’s robustness and strong generalization capacity.

Soft robotic systems heavily depend on accurate sensor data for perception and control; however, this data is often corrupted by missing observations, due to partial sensor coverage, communication failures, or occlusions and noisy measurements stemming from hardware imperfections, environmental disturbances, and the intrinsic compliance of soft materials. Such corruption can obscure critical state information, causing unreliable modeling of soft robotics and degrading control accuracy. To address these challenges, we propose a Contrastive Dual-Latent Autoencoder (CDLAE) that jointly handles missing and noisy data in a single end-to-end framework. Our approach leverages an attention based autoencoder architecture with dual latent pathways, where one focuses on capturing the underlying clean signals while the other isolates noise-related components. A contrastive loss encourages strong separation between these pathways, enhancing the model’s ability to filter noise while reconstructing missing values. Additionally, the autoencoder is trained jointly with a downstream predictive network, ensuring that signal imputation is optimized with respect to the ultimate control task. Experimental evaluations on a pneumatic soft robot platform and multiple public time-series datasets demonstrate that CDLAE consistently outperforms existing methods in handling corrupted data, offering robust, high-fidelity reconstructions that significantly improve soft robot perception and control in real-world conditions.

Accurate waste detection using unmanned aerial vehicles (UAVs) remains a significant challenge due to the non-canonical perspective, varying altitudes, and diverse nature of waste materials such as plastic, with its translucency and irregular appearance. Traditional object detection architectures struggle to adapt to these factors, leading to reduced detection accuracy. To address these challenges, we propose the use of additional contextual information by incorporating altitude information of UAV waste images to enhance the multi-scale detection capabilities of feature pyramid networks, dynamically assigning differential importance to feature fusion modules. We propose a novel altitude-informed tiny-object detection architecture (named AltiDet) in three configurations of backbone sizes, consisting of an Asymmetric Deep Aggregation (ADA) and High-Resolution Feature Extraction (HRFE) module as a backbone, an Altitude-Informed Fusion Pyramid Network (A-IFPN) and an altitude-scaled loss function. Our ADA+HRFE backbone aggregates extracted features iteratively using different nodes while capturing additional spatial and contextual features from the high-resolution layers, to account for the fine-grained and irregular features of waste. The A-IFPN fuses and aggregates Altitude-Weighted Frequency Attention (AWFA) modules to further extract meaningful feature maps from the backbone, improving multi-scale detection. Our altitude-scaled loss function ensures the more challenging higher altitude images are greater emphasized during training. Extensive experiments conducted on our collected multi-class aerial waste dataset (named AltiWaste), the Solid Waste Aerial Detection (SWAD), Trash Annotations in Context (TACO) and other datasets demonstrate the effectiveness and advantages of our proposed architecture, yielding a 1.41% improvement over state-of-the art detectors on the AltiWaste dataset.

Unmanned aerial vehicle (UAV) networks, which carry vital information, are prone to various attacks, and hence security issues are a major concern. In this paper, we design and implement a novel covert attack detection and secure control scheme, which operates between the UAV (the physical layer) and ground control station (GCS) (the cyber layer). Covert attacks can alter the UAV states, and yet cause the signals seen by the controller to appear unchanged, resulting in these attacks being more difficult to detect, and hence more dangerous compared to other types of attacks. To unmask the covert attacks, we construct and inject auxiliary signals to both the controller output and the UAV input. The auxiliary signals cause information of the attack to appear in the controller input, which is then fed to a detection observer to detect the attack. Next, we propose an integrated estimation and secure control scheme, comprising a reconstruction observer (which is a sliding mode observer (SMO)) that estimates the system states and attack signal, and an output-feedback controller that utilizes the estimated signals. We perform a series of transformations to the system, such that the design parameters of both reconstruction observer and secure controller are placed in a framework that is solvable using Linear Matrix Inequalities (LMIs). We also prove that the proposed integrated secure controller causes the output tracking errors to satisfy an${{\mathcal {H}}}_{\infty }$performance index. We also rigorously analyze the system performance, and present the necessary conditions for the scheme to be feasible. Finally, simulations are conducted to verify the effectiveness of the proposed scheme. Note to Practitioners—This paper presents a method to detect covert attacks in the UAV network, and to mitigate against those attacks. Covert attacks are more malicious since they are difficult to detect. The proposed method in this paper consists of auxiliary signal injection and a detection observer that will expose and detect the attacks, and a reconstruction observer and secure controller that will estimate the attacks and mitigate its effect on the plant. The controller and observers are designed using Linear Matrix Inequalities to minimize the${{\mathcal {H}}}_{\infty }$gain from the attack on the plant performance. In addition, this paper also investigates the conditions that the plant must satisfy such that the proposed scheme is feasible, and presents them in an easily verifiable form. Finally, the paper also analyses the performance of the proposed scheme in all scenarios - namely before the attack occurs, when the attack occurs but is not yet detected, and after the attack is detected.

With the advancements of sensor hardware, traf-fic infrastructure and deep learning architectures, trajectory prediction of vehicles has established a solid foundation in intelligent transportation systems. However, existing solutions are often tailored to specific traffic networks at particular time periods. Consequently, deep learning models trained on one network may struggle to generalize effectively to unseen networks. To address this, we proposed a novel spatial-temporal trajectory prediction framework that performs cross-domain adaption on the attention representation of a Transformer-based model. A graph convolutional network is also integrated to construct dynamic graph feature embeddings that accurately model the complex spatial-temporal interactions between the multi-agent vehicles across multiple traffic domains. The proposed framework is validated on two case studies involving the cross-city and cross-period settings. Experimental results show that our proposed framework achieves superior trajectory prediction and domain adaptation performances over the state-of-the-art models.

Soft sensing of hard-to-measure variables is often crucial in industrial processes. Current practices rely heavily on conventional modeling techniques that show success in improving accuracy. However, they overlook the non-linear nature, dynamics characteristics, and non-Euclidean dependencies between complex process variables. To tackle these challenges, we present a framework known as a Knowledge discovery graph Attention Network for effective Soft sensing (KANS). Unlike the existing deep learning soft sensor models, KANS can discover the intrinsic correlations and irregular relationships between the multivariate industrial processes without a predefined topology. First, an unsupervised graph structure learning method is introduced, incorporating the cosine similarity between different sensor embedding to capture the correlations between sensors. Next, we present a graph attention-based representation learning that can compute the multivariate data parallelly to enhance the model in learning complex sensor nodes and edges. To fully explore KANS, knowledge discovery analysis has also been conducted to demonstrate the interpretability of the model. Experimental results demonstrate that KANS significantly outperforms all the baselines and state-of-the-art methods in soft sensing performance. Furthermore, the analysis shows that KANS can find sensors closely related to different process variables without domain knowledge, significantly improving soft sensing accuracy.

In this paper, an output feedback-based active fault tolerant control (FTC) scheme is proposed for an unstable three degree-of-freedom (3-DOF) helicopter equipped only with angular position sensors, of which a single sensor can be faulty. Limited by the available outputs, existing fault estimation (FE) and FTC schemes cannot be applied to this helicopter because when the sensor measuring the elevation or travel angle is faulty, it does not satisfy the minimum-phase and matching conditions required by standard observers. To circumvent this problem, an adaptive interval observer is firstly designed as a fault detection and isolation (FDI) unit to indicate the occurrence and location of a fault, in contrast to the existing FDI methods that require a bank of observers and incur a high computational cost. Then a high-gain observer is combined with a sliding mode observer to form an FE unit that does not require the matching and minimum-phase conditions. Based on the estimate of the fault, an FTC scheme is constructed to ensure an$\mathcal{H}_{\infty}$performance of the faulty system. Finally, physical experiments on the 3-DOF helicopter verify the effectiveness of the proposed scheme.Note to Practitioners—The 3-DOF lab helicopter serves as an ideal experimental platform for control strategies. In this paper, an active fault tolerant control (FTC) scheme is developed for a 3-DOF lab helicopter when any single sensor can be faulty. The helicopter is nonlinear and subject to external disturbances. There are only encoders measuring the attitude angles and no sensors for angular velocities. In this paper, we firstly design a fault detection and isolation (FDI) unit based on an adaptive interval observer to detect the fault and identify its location; it incurs a lower computational cost compared to existing methods that use a bank of observers. Then a high-gain observer is combined with a sliding mode observer to estimate the fault. Based on the estimate of the fault, an FTC scheme is constructed and designed using Linear Matrix Inequalities to ensure an$\mathcal{H}_{\infty}$performance of the faulty system.

Most works on joint state and unknown input (UI) estimation require the assumption that the UIs are linear; this is potentially restrictive as it does not hold in many intelligent autonomous systems. To overcome this restriction and circumvent the need to linearize the system, we propose a derivative-free UI sigma-point Kalman filter (SPKF-nUI), where the SPKF is interconnected with a general nonlinear UI estimator that can be implemented via nonlinear optimization and data-driven approaches. The nonlinear UI estimator uses the posterior state estimate, which is less susceptible to state prediction error. In addition, we introduce a joint sigma-point transformation scheme to incorporate both the state and UI uncertainties in the estimation of SPKF-nUI. An in-depth stochastic stability analysis proves that the proposed SPKF-nUI yields exponentially converging estimation error bounds under reasonable assumptions. Finally, two case studies are carried out on a simulation-based rigid robot and a physical soft robot, i.e., the robots made of soft materials with complex dynamics, to validate the effectiveness of the proposed filter on nonlinear dynamic systems. Our results demonstrate that the proposed SPKF-nUI achieves the lowest state and UI estimation errors when compared to the existing nonlinear state-UI filters.

Recently, data-driven models such as deep neural networks have shown to be promising tools for modelling and state inference in soft robots. However, voluminous amounts of data are necessary for deep models to perform effectively, which requires exhaustive and quality data collection, particularly of state labels. Consequently, obtaining labelled state data for soft robotic systems is challenged for various reasons, including difficulty in the sensorization of soft robots and the inconvenience of collecting data in unstructured environments. To address this challenge, in this paper, we propose a semi-supervised sequential variational Bayes (DSVB) framework for transfer learning and state inference in soft robots with missing state labels on certain robot configurations. Considering that soft robots may exhibit distinct dynamics under different robot configurations, a feature space transfer strategy is also incorporated to promote the adaptation of latent features across multiple configurations. Unlike existing transfer learning approaches, our proposed DSVB employs a recurrent neural network to model the nonlinear dynamics and temporal coherence in soft robot data. The proposed framework is validated on multiple setup configurations of a pneumatic-based soft robot finger. Experimental results on four transfer scenarios demonstrate that DSVB performs effective transfer learning and accurate state inference amidst missing state labels.

Soft sensors are widely used in many industrial systems to monitor key variables that are difficult to measure, using measurements from other available physical sensors. Because physical sensors are susceptible to faults, it is crucial for soft sensor models to be robust against them. Recently, deep learning has shown promising results in developing data-driven soft sensors for various applications. However, existing learning-based soft sensors are still vulnerable to sensor faults, which could deteriorate the performance of the models. In this article, we propose a deep learning-based modeling framework for developing soft sensor models that are robust to sensor faults. Due to the difficulty in obtaining datasets that cover all possible sensor fault characteristics, the proposed framework is developed to be zero-shot such that the model can be trained with only fault-free dataset without requiring any sensor fault patterns, thus greatly saving the time and resources needed to collect such data. Instead, adversarial examples are used as a proxy for faulty sensor inputs so that the model can learn to be adaptive through the proposed two-stage, uncertainty-aware recurrent neural network architecture. We demonstrate our approach to the TE benchmark process and a real industrial multiphase flow process and show that robustness is achieved as the accuracy does not degrade significantly when sensor faults are present during the model evaluation.

In industrial robots, a performance issue is backlash, which is the clearance between mating gears of its joints. Over time, backlash grows through wear and tear, causing inaccuracies in robot positioning. Current methods in backlash detection are performed in low-speed and laboratory settings, or require offline diagnostics. These methods are impractical in actual manufacturing environments, where industrial robots operate continuously at high speeds. Other methods require additional sensors unavailable in typical industrial robots. In this article, we present an online method to quantify backlash and predict the remaining useful life (RUL) in an industrial robot performing cyclic production tasks, using only standard available sensors. To achieve the robot's target position, the input torque oscillates; these oscillations grow as the backlash becomes more severe. We modeled the oscillations as an unknown input, and used an unknown input observer to estimate them and detect/quantify the backlash. Then, a health indicator (HI) is plotted over time and a failure threshold is set based on historical data. Finally, an exponential degradation model is used to predict the RUL of the robot joint. The UIO successfully detected and quantified the backlash through the HI. The degradation model gave a good estimate of the RUL with an accuracy of 20 days after 250 days of operation.

Bacterial chemotaxis in unicellular Escherichia coli, the simplest biological creature, enables it to perform effective searching behaviour even with a single sensor, achieved via a sequence of "tumbling" and "swimming" behaviours guided by gradient information. Recent studies show that suitable random walk strategies may guide the behaviour in the absence of gradient information. This article presents a novel and minimalistic biologically inspired search strategy inspired by bacterial chemotaxis and embodied intelligence concept: a concept stating that intelligent behaviour is a result of the interaction among the "brain," body morphology including the sensory sensitivity tuned by the morphology, and the environment. Specifically, we present bacterial chemotaxis inspired searching behaviour with and without gradient information based on biological fluctuation framework: a mathematical framework that explains how biological creatures utilize noises in their behaviour. Via extensive simulation of a single sensor mobile robot that searches for a moving target, we will demonstrate how the effectiveness of the search depends on the sensory sensitivity and the inherent random walk strategies produced by the brain of the robot, comprising Ballistic, Levy, Brownian, and Stationary search. The result demonstrates the importance of embodied intelligence even in a behaviour inspired by the simplest creature.

This article presents a secure communication scheme based on chaotic systems using a sliding-mode observer (SMO) that is robust against disturbances affecting the transmitter and transmission process. The original message is first encrypted using a$N$-cipher with a key signal and then fed into the state equation, whereas the key signal is fed into the output equation. The chaotic system in the transmitter is re-expressed to decouple the disturbances from the signals of interest. An SMO is then designed based on this form and implemented in the receiver to perform synchronization, and recover the broadcast messages to perform secure communication. The conditions required for the SMO to be feasible are investigated in terms of the transmitter system matrices. A set of design procedures for the chaotic secure communication scheme is outlined. Finally, a simulated example is shown to exhibit the efficacy of the proposed scheme.

In this paper, a fault-tolerant control scheme is proposed for the rigid spacecraft attitude control system subject to external disturbances, multiple system uncertainties, and actuator faults. The angular velocity measurement is unavailable, which increases the complexity of the problem. An observer is first designed based on the super-twisting sliding mode method, which can provide accurate estimates of the angular velocity in finite time. Then, an adaptive fault-tolerant controller is proposed based on neural networks using the information from the observer. It is shown that the attitude orientations converge to the desired values exponentially. Finally, a simulation example is utilized to verify the effectiveness of the proposed scheme.

A novel cyber-physical multiagent framework is proposed to control traffic at an intersection. The vehicles, or physical agents, may pass the intersection smoothly utilizing the timings of traffic lights provided in advance. In the cyberspace, the durations of upcoming traffic lights are computed by a group of cyber agents using a stochastic gradient-based method known as broadcast control of multiagent systems. For this computation of the traffic light durations, a function that represents the cost for blocking a vehicle by the red light is included in the objective function, which is minimized by the cyber agents to collectively provide the least-restrictive right-of-way in a receding horizon control approach to all vehicles at the intersection. The proposed scheme is evaluated through microscopic traffic simulation at various penetration rates of the automated vehicles, and performances are compared with existing schemes.

This paper presents a method for real-time detection and tracking of the human face. This is achieved using the Raspberry Pi microcomputer and the Easylab microcontroller as the main hardware with a camera mounted on servomotors for continuous image feed-in. Real-time face detection is performed using Haar-feature classifiers and ScicosLab in the Raspberry Pi. Then, the Easylab is responsible for face tracking, keeping the face in the middle of the frame through a pair of servomotors that control the horizontal and vertical movements of the camera. The servomotors are in turn controlled based on the state-diagrams designed using SMCube in the EasyLab. The methodology is verified via practical experimentation.

Automatic flight control systems have become increasingly vital ever since the emergence of airplanes. Automatic systems are not subject to fatigue and emotions as compared to the human pilot. As such, the possibility of human errors in flight control is minimized. Current automatic flight control systems are designed based on classical control theories using linear controllers that are complex and inefficient. Since an aircraft is naturally nonlinear in its behavior, it is intuitive to design a nonlinear controller that could cover a wide variety of possible 'extreme' flight conditions. A novel controller utilizing the nonlinear energy method was developed by Akmeliawati for the longitudinal dynamics of an aircraft, and has been proven to provide effective control and better performance when compared to an equivalent linear controller. The novel controller was designed using the passivity-based control (PBC) technique. In this paper, a similar nonlinear controller was designed to direct the roll and yaw motion, which is part of the lateral dynamics of the aircraft. Simulations show that this PBC is able to stabilize both roll and yaw motion of the aircraft.

A common requirement that is implicit in current methods for the design of robust state estimators and robust fault detection filters is that the first Markov matrix must be non-zero, and indeed, full rank. We relax both of these restrictions in this paper to allow the applicability to a wider range of systems. The extended results are then applied to an aircraft fault detection for which the restrictive condition is not satisfied

This paper presents an observer based robust sensor fault reconstruction scheme. A popular observer based robust fault detection method is eigenstructure assignment; where the observer has certain eigenvectors, resulting in the fault detection scheme being decoupled from disturbances. This paper firstly investigates the existence conditions for the right eigenvector assignment method. Then an alternative scheme using reduced order observers is presented, with existence conditions that are less restrictive than that of right eigenvector assignment, whilst still being decoupled from disturbances. Following that, it proposes a design method for the observer using linear matrix inequalities. A design example validates the work in this paper; where the fault reconstruction scheme is totally decoupled from any disturbances

This paper presents an application of a sensor fault tolerant control (FTC) scheme on a flexible joint and flexible link. A linear observer is used to reconstruct the faults. Then the reconstruction is subtracted from the faulty sensors to form a 'virtual sensor' and this signal (instead of the normally used faulty output) is then used to generate the control input. This minimizes performance degradation. A design method to make the virtual sensor insensitive to system uncertainties is presented. Two fault conditions are tested; total failure and incipient faults. Then the robustness is tested by implementing the joint's FTC scheme on the link. Excellent results have been obtained for both cases; the FTC scheme caused the system performance is almost identical to the fault-free scenario, even for simultaneous faults