Lexicographic multi-objective problems, which consist of multiple conflicting subtasks with explicit priorities, are common in real-world applications. Despite the advantages of Reinforcement Learning (RL) in single tasks, extending conventional RL methods to prioritized multiple objectives remains challenging. In particular, traditional Safe RL and Multi-Objective RL (MORL) methods have difficulty enforcing priority orderings efficiently. Therefore, Lexicographic Multi-Objective RL (LMORL) methods have been developed to address these challenges. However, existing LMORL methods either rely on heuristic threshold tuning with prior knowledge or are restricted to discrete domains. To overcome these limitations, we propose Lexicographically Projected Policy Gradient RL (LPPG-RL), a novel LMORL framework which leverages sequential gradient projections to identify feasible policy update directions, thereby enabling LPPG-RL broadly compatible with all policy gradient algorithms in continuous spaces. LPPG-RL reformulates the projection step as an optimization problem, and utilizes Dykstra's projection rather than generic solvers to deliver great speedups, especially for small- to medium-scale instances. In addition, LPPG-RL introduces Subproblem Exploration (SE) to prevent gradient vanishing, accelerate convergence and enhance stability. We provide theoretical guarantees for convergence and establish a lower bound on policy improvement. Finally, through extensive experiments in a 2D navigation environment, we demonstrate the effectiveness of LPPG-RL, showing that it outperforms existing state-of-the-art continuous LMORL methods.

In real-world industrial processes, there are many lexicographic multiobjective problems, which consist of conflicting subtasks with explicit priorities. In the past, model predictive control (MPC) is combined with lexicographic optimization to form Lexicographic MPC (LMPC), which handles priorities in a hierarchical manner. Although higher priorities like safety in chemical reactions, can be strictly maintained, the recursive optimization incurs great burden for online computation, which makes LMPC impractical. Considering the great success, especially the low online computation cost of reinforcement learning (RL), several studies have been conducted in lexicographic multiobjective RL (LMORL) to train end-to-end policies for prioritized problems. However, existing LMORL methods are either limited to discrete domains with low precision control in processes, or heavily reliant on hyperparameter tuning with prior knowledge. To address these limitations in continuous process problems, lexicographically projected policy gradient RL (LPPG-RL) is proposed. LPPG-RL performs sequential policy gradient projections to identify a feasible policy update direction, ensuring compatibility with general continuous policy gradient RL algorithms. LPPG-RL employs Dykstra's projection instead of generic solvers to accelerate the direction search, particularly for high-dimension gradient vectors. In addition, LPPG-RL introduces subproblem exploration to mitigate gradient vanishing and improve training stability. Furthermore, convergence proof and stepsize analysis are provided. Finally, two CSTR problems serve as the case study, demonstrating the effectiveness of LPPG-RL through comparisons with baseline LMORL and LMPC methods.

In the production process of hot rolled strip steel plate, the Industrial Internet of Things (IIoT) improves the quality and efficiency of production and manufacturing products, but also because of the large amount of data generated by its equipment, resulting in a significant decline in plate crown fault diagnosis and decision-making ability. To improve the yield and quality of hot strip steel plate, in this paper, an intelligent algorithm based on the Hippopotamus optimization algorithm-Kernel Extreme Learning Machine (HOA-KELM) is proposed. Adaptive synthetic sampling technology (ADASYN) resampling technique is used to deal with multi-class unbalance of data. At the same time, InterpretML and SHapley additive stripping (SHAP) methods based on game theory were introduced to analyze the crown diagnosis model of hot strip, and the influencing factors of the strip in hot rolling were studied. To verify the applicability of the model, the model was tested on the hot rolling production data set and UCI data set. In the hot rolling data set, the model performance Kappa Coefficient, F1 Score, and Accuracy were 0.988, 0.993, and 0.992, respectively, indicating a good effect. It is compared with the common mathematical model. The findings indicate that this model significantly outperforms the conventional approach in solving the problem of strip convexity diagnosis in the hot rolling process, and can be proposed as an effective mathematical model for plate convexity diagnosis to realize the intelligent management of industrial production.

At present, working conditions are becoming more complex, and operation policy requirements are more diverse in process system engineering. To control a process problem, a balance must be found between speed and stability, in that operations should sometimes be faster and other times smoother. Traditional controllers, such as PID and model predictive control are applied in various problems, and some parameters in controllers can be used to represent the operation policy. However, there can be difficulties in tuning parameters, and time costs of online calculation. This article proposes parametric deep reinforcement learning (PDRL) to replace traditional controllers. PDRL has two parts. A vanilla DRL framework is adapted to solve the setpoint tracking problem. With a state and a reward function and robust training tricks, trained agents can be applied to more generalized working conditions. Base agents of different operation policies are trained in advance. With target performance from operators, the target policy can be fitted by base agents with a set of weights, which are first optimized by minimizing the squared error between the target and fitted policy in a basic task, and applied to generalized conditions. A shell benchmark problem is chosen as a case study, whose results show that PDRL has feasibility and stability both in basic and generalized tasks, even in a noisy environment.

With the continuous development of autonomous driving, real-time target detection has become increasingly critical in autonomous driving systems. However, traditional target detection algorithms usually require huge computational resources, limiting their application on embedded autonomous driving platforms. To address this challenge, a lightweight single-stage target detection algorithm is proposed YOLOdrive. The inverted residual structure, linear bottleneck layer, and depth-separable convolution in MobileNetv2 are utilized to improve the YOLOv8 backbone network, while the spatial channel reconstructed convolution is used to improve the C2f module of YOLOv8, and the convolution of YOLOv8 neck and detection head is replaced by the depth-separable convolution. Experimental results verify that the average accuracy of YOLOdrive algorithm on MS COCO2017 data set and VOC2007 data set is improved compared with the baseline model YOLOv8-Nano. The amount of model parameters has been reduced by more than 50%, and the amount of computation in the model has been reduced by more than 70%. The algorithm drastically reduces the amount of parameters and computational complexity of the network, improves the operational efficiency, saves the storage space of the network, and maintains a high-detection performance.

Fault-tolerant control (FTC) strategy can be realized without modifying the peripheral hardware circuit when the open-circuit fault (OCF) occurs in the multiphase motor. However, FTC relies on accurately identifying the fault location and switching to a new reconfiguration fault-tolerant algorithm. This can significantly increase the complexity of the system. To overcome the challenge, this article investigates a passive fault-tolerant scheme of a 2 × 3-phase surface-mounted permanent-magnet synchronous motor (SPMSM) driven by a mono-inverter when single-phase OCF occurs. The state equations based on field-oriented control of 2 × 3-phase SPMSM under healthy and single-phase OCF are discussed. The special motor drive mode and the constraint ofid= 0 make the phase currents of each module passively optimized under the two neutral point configurations (i.e., isolated or connected), thus meeting the demand of restraining torque ripple. In the proposed PFTS, when the OCF occurs, the system does not require to attempt to diagnose or correct faults. Hence, a seamless transition from normal to faulty operation is guaranteed. Moreover, it enhances system reliability and stability. Furthermore, taking an existing 2 × 3-phase SPMSM as an example, the experiments are carried out for validation.

Permanent magnet vernier machines (PMVMs) are recognized as the most promising candidates for high-torque direct-drive applications. However, their market penetration is hindered by the low power factor. This paper aims to address the low-power-factor challenge, by employing the concept of low-coupling winding. As compared with two conventional PMVMs, the proposed PMVM with low-coupling winding exhibits a substantially improved power factor. To objectively evaluate the potential of the proposed PMVM, a comprehensive comparison between the proposed PMVM and the conventional permanent magnet synchronous machine (PMSM) is conducted. The no-load back-EMF, output torque, power factor, core losses, and efficiency are compared under various operating conditions, encompassing the entire torque-speed range. The strengths and weaknesses of the PMVM are analyzed, and the most promising application scenarios are suggested. Finally, a prototype of the proposed PMVM is fabricated to validate the analysis and comparison results.

Axial-flux permanent magnet synchronous motor (AFPMSM) is a promising solution for in-wheel direct-drive electric bikes (e-bikes) due to its disc-type topology, structural compactness, and high torque density. However, the large electric loading of conventional AFPMSMs for high torque output can lead to severe thermal rise and potential fault risk, which is unbearable for long-running operation and security requirement in e-bike scenario. To address this issue, a comparative study is conducted on the performance of AFPMSM with conventional surface-mounted PM array (S-AFPMSM) and Halbach PM array (H-AFPMSM). The design and optimization procedure for both AFPMSMs is conducted. The investigation result shows that S-AFPMSM features over 30% higher torque density than that of a radial-flux PMSM, but the thermal raise is too much for air-cooling condition. By utilizing Halbach PM array with self-shielding magnetization effect, H-AFPMSM has higher magnetic loading and much less heat loading when it outputs the same torque density as that of S-AFPMSM. As a result, the copper loss is reduced by over 30%, and accordingly the efficiency increases to 94%. A prototype of the motor is manufactured and tested, with results that agree well with the theoretical analysis.

In the development of targeted drugs, anticancer peptides (ACPs) have attracted great attention because of their high selectivity, low toxicity and minimal non-specificity. In this work, we report a framework of ACPs generation, which combines Wasserstein autoencoder (WAE) generative model and Particle Swarm Optimization (PSO) forward search algorithm guided by attribute predictive model to generate ACPs with desired properties. It is well known that generative models based on Variational AutoEncoder (VAE) and Generative Adversarial Networks (GAN) are difficult to be used for de novo design due to the problems of posterior collapse and difficult convergence of training. Our WAE-based generative model trains more successfully (lower perplexity and reconstruction loss) than both VAE and GAN-based generative models, and the semantic connections in the latent space of WAE accelerate the process of forward controlled generation of PSO, while VAE fails to capture this feature. Finally, we validated our pipeline on breast cancer targets (HIF-1) and lung cancer targets (VEGR, ErbB2), respectively. By peptide-protein docking, we found candidate compounds with the same binding sites as the peptides carried in the crystal structure but with higher binding affinity and novel structures, which may be potent antagonists that interfere with these target-mediated signaling.

Traditional DFA based DPI (Deep Packet Inspection) string matching architectures either suffer from throughput bottleneck or unfeasible memory requirement, or both. Bloom Filter based schemes, on the other hand, only provide indefinite and unprecise match results. In this paper, we propose a novel string matching data structure called Overlapped Substring Classifier(OSC), which tries to compromise between these two ends. Instead of using incoming byte flow directly, we use OSC to extract the characteristic digest of the incoming string, which we demonstrate would be sufficient for locating a very small set of possible match, using DFA techniques. This type of match ambiguity and false-positive inaccuracy can be tuned to be negligible. The scheme is perfectly suitable for efficient and parallel hardware implementation, which makes ultra high performance and low memory usage simultaneously possible. A hardware architecture is also designed supporting singlethreaded scanning rate of 10Gbp, with only moderate memory requirement and clock rate assumption.

As scalable routers being a promising way to scale to higher capacity, scalable switch fabric as its key component has received a great deal of attention. However, the reliability calculation method for general switch fabric does not suit scalable switch fabrics with special features. In this paper, we study the features of scalable switch fabrics, and propose a novel reliability measure called Failure Influence which focuses on the robustness of scalable switch fabric. Using our improved heuristic algorithms, we analyze the Failure Influence properties of some classical topologies and compare them with Plus 2^i (P2i) of our prior work. Experimental results show that P2i has better Failure Influence properties. Since the Peak Point Problem (PPP) of P2i, Add Edge P2i (AE-P2i) and Failure Influence Optimal P2i (FO-P2i) are also proposed to solve this problem.

A packet scheduler is a primary component of the improved quality of service (QoS) model for today's Internet. Although many fair packet schedulers have been proposed through theoretical consideration, practical high-speed packet schedulers remain elementary. The disparity arises because existent schedulers either lack of necessary QoS guarantee or have an unacceptable cost of computation and storage. In this paper, we propose a simple and efficient packet scheduler called hardware optimized bit reversal permutation (HOBRP) based scheduler. Besides some common merits including low time- and space-complexity, bounded end-to-end delay guarantee and constant fairness index that many well-known schedulers have already owned, our HOBRP still possesses two additional features: One is that the end-to-end delay bound of HOBRP is tunable, which makes itself flexible enough to provide different levels of delay bounds for diverse types of application flows. The other is that all the operations and structures used by HOBRP are very simple and easy to be pipelined and paralleled, which benefits an intuitive high-speed hardware design scheme.