Demand response (DR) plays an essential role in smart grids to reconfigure the load profile to match the energy supply for reliable and economic operation. This article presents an online adaptive DR strategy based on stochastic learning and optimization for smart microgrids with renewable energy to maximize the operation profit. To enable prompt interaction with the stochastic dynamics of usage demands, renewable generations and real-time pricing, a novel event-driven continuous-time Markov control processes model is introduced to formulate the DR optimization. Based on performance sensitivity analysis, a stochastic learning algorithm that combines potentials estimate and policy iteration is developed to find the optimal DR policy online. The presented strategy is environment-adaptive and computation-efficient, which is competent for real-time control and optimization in unknown environments. Simulations have been conducted to evaluate the performance of the presented strategy, and the results demonstrate the effectiveness.

Sample clustering techniques play a crucial role in the data‐driven state evaluation of electromechanical equipment, and selecting an appropriate similarity measurement method for sample sets helps improve the clustering performance. The Jaccard coefficient is a commonly employed indicator of similarity for scalar set‐type samples. In this paper, we propose an incremental clustering algorithm for matrix‐type samples by defining an improved Jaccard coefficient. First, a new binary relation is formulated to derive a relationship matrix between samples. Second, an undirected graph is given by using the relationship matrix, and an improved pruning operation is provided to simplify the graph by eliminating redundant edges. Then, a new relationship matrix is generated according to the modified graph, which enables the calculation of the improved Jaccard coefficient. By using the improved Jaccard coefficient, the improved incremental clustering algorithm updates cluster centers by selecting a particular sample to maximize the sum of similarities between the selected sample and other samples within the same cluster. Finally, the effectiveness of the proposed incremental clustering algorithm is demonstrated in fault diagnosis and remaining useful life estimation application scenarios, respectively. The experimental results indicate that the improved algorithm outperforms traditional clustering methods.

Muscle fatigue detection is of great significance to human physiological activities, but many complex factors increase the difficulty of this task. In this article, we integrate several effective techniques to distinguish muscle states under fatigue and nonfatigue conditions via surface electromyography (sEMG) signals. First, we perform an isometric contraction experiment of biceps brachii to collect sEMG signals. Second, we propose a neural architecture search (NAS) framework based on reinforcement learning to autogenerate neural networks. Finally, we present an effective two-step training strategy to improve the performance by combining CNN with three types of commonly used statistical algorithms. Meanwhile, we propose a data enhancement algorithm based on empirical mode decomposition (EMD) to generate time-series data for expanding the dataset. The results show that this search algorithm can hunt for high-performing networks, and the accuracy of the best-selected model combined with support vector machine (SVM) for the group is 96.5%. With the same architecture, the average accuracy in individual models is 97.8%. The proposed data enhancement technique can effectively improve the fatigue detection performance, which allows further implementations in the human-exoskeleton interaction systems.

The double inverted pendulum (DIP) system is a benchmark underactuated mechanical system. The control problem of the DIP system is challenging since it not only has high nonlinearity, but also has underactuation degree equal to two. In this article, a switched energy-based swing-up and neural network (NN) controller is proposed to solve the swing-up and tracking control problem of the DIP system when the desired position is time varying. The implementation of the proposed controller can be divided into three steps. In step one, the energy-based control method is adopted to drive the first pendulum from the downward position to the neighborhood of the upright position. In step two, the sliding mode control method is adopted to stabilize the first pendulum. Meanwhile, a method combining energy-based control and “equivalent cart” is adopted to swing up the second pendulum. In step three, on the basis of approximate nonlinear output regulation theory, the NN controller is used for achieving satisfactory position tracking performance. Finally, the effectiveness of our design is verified by experimental results.

Multicast is a promising solution to address spectrum scarcity in wireless video streaming, while facing tough challenge in provisioning quality of service (QoS) to heterogeneous users. This article presents an adaptive modulation and coding (AMC) scheme to accommodate both spatial diversity and temporal variation on users’ channel conditions for QoS provisioning in wireless scalable video multicast. The QoS-guaranteed AMC policy optimization is formulated as constrained stochastic optimization based on Markov decision processes. An adaptive policy iteration algorithm is developed to find the optimal AMC policy online without any prior knowledge of users’ channel statistics or intensive calculations. This algorithm is attractive due to adaptability to unknown environments and less run-time computation expenditure. Simulations results demonstrate the effectiveness of the proposed scheme.

Intention recognition based on surface electromyography (sEMG) signals is pivotal in human-machine interaction (HMI), where continuous motion estimation with high accuracy has been the challenge. The convolutional neural network (CNN) possesses excellent feature extraction capability. Still, it is difficult for ordinary CNN to explore the dependencies of time-series data, so most researchers adopt the recurrent neural network or its variants (e.g., LSTM) for motion estimation tasks. This paper proposes a multi-feature temporal convolutional attention-based network (MFTCAN) to recognize joint angles continuously. First, we recruited ten subjects to accomplish the signal acquisition experiments in different motion patterns. Then, we developed a joint training mechanism that integrates MFTCAN with commonly used statistical algorithms, and the integrated architectures were named MFTCAN-KNR, MFTCAN-SVR and MFTCAN-LR. Last, we utilized two performance indicators (RMSE and [Formula: see text]) to evaluate the effect of different methods. Moreover, we further validated the performance of the proposed method on the open dataset (Ninapro DB2). When evaluating on the original dataset, the average RMSE of the estimations obtained by MFTCAN-KNR is 0.14, which is significantly less than the results obtained by LSTM (0.20) and BP (0.21). The average [Formula: see text] of the estimations obtained by MFTCAN-KNR is 0.87, indicating the anti-disturbance ability of the architecture. Moreover, MFTCAN-KNR also achieves high performance when evaluating on the open dataset. The proposed methods can effectively accomplish the task of motion estimation, allowing further implementations in the human-exoskeleton interaction systems.

Energy-efficient streaming of real-time scalable video over wireless fading channels with imperfect channel state information is considered in this paper. A traffic rate adaptation (TRA) scheme is presented to minimize the transmission energy consumption while satisfying the quality of experience (QoE) requirement by dynamically selecting the delivered encoding layers of streaming media depending on the channel condition. A partially observable Markov decision process model is introduced to formulate the energy-efficient QoE-guaranteed TRA problem. Based on the Lagrange method of multipliers, an online policy iteration algorithm is proposed to solve the constrained optimization problem. This algorithm performs policy evaluation and improvement without intensive computations and prior knowledge of channel statistics. The adaptability to unknown environments and high computation efficiency make the proposed scheme attractive for implementation in real-life mobile systems. Simulation results demonstrate the effectiveness of the proposed method.

In future mobile computing systems, cognitive radio (CR) emerges as a promising solution for alleviating spectrum shortage and satisfying the high bandwidth demand of multimedia streaming, while it presents tough challenges in provisioning user experience-perceived quality of service (QoS) and conserving transmission energy. In this paper, an adaptive rate control (ARC) scheme with the aid of the receive buffer is presented for energy-efficient transmission of streaming media over CR with QoS guarantee. The QoS metric, either display smoothness or transmission delay, is quantified by the state of the receive buffer. Cross-layer information is utilized to form a closed-loop feedback optimal control. A novel analytical model called event-driven discrete-time Markov control process is introduced to formulate the QoS-guaranteed energy-efficient ARC problem. Based on potential theory, a policy iteration algorithm that combines potentials estimation and stochastic approximation is proposed for finding the optimal policy online. By exploiting the system dynamics, this algorithm does not depend on any prior knowledge of channel availability or fading statistics, and it can converge to the global optimum with a low computational cost and reasonable speed. Simulation results demonstrate the effectiveness of the proposed method.