Biopharmaceutical processes typically generate abundant high-frequency online sensor data but suffer from sparse and delayed offline quality measurements, creating a “data-rich but label-poor” dilemma that hinders effective process monitoring. Furthermore, these processes are subject to significant distribution shifts due to batch-to-batch variability and time-varying metabolic states. To address these challenges, this article proposes a novel online time-series contrastive learning framework for soft sensing, validated on an industrial erythromycin fermentation process. We introduce a mechanism-informed soft contrastive learning strategy that utilizes information derived from domain knowledge to construct instance-level and temporal similarity matrices, guiding the model to learn physically meaningful representations from unlabeled data. In addition, we develop a dual-timescale online learning architecture comprising a slow branch for robust representation learning and a fast branch for real-time adaptation. Experimental results on a large-scale industrial dataset demonstrate that the proposed framework significantly outperforms state-of-the-art time-series contrastive learning baselines, particularly in predicting complex rheological indicators like broth viscosity under temporal distribution shifts.

Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet they often refuse to answer legitimate queries-a phenomenon known as overrefusal.Overrefusal typically stems from over-conservative safety alignment, causing models to treat many reasonable prompts as potentially risky.To systematically understand this issue, we probe and leverage the models' safety decision boundaries to analyze and mitigate overrefusal.Our findings reveal that overrefusal is closely tied to misalignment at these boundary regions, where models struggle to distinguish subtle differences between benign and harmful content.Building on these insights, we present RASS, an automated framework for prompt generation and selection that strategically targets overrefusal prompts near the safety boundary.By harnessing steering vectors in the representation space, RASS efficiently identifies and curates boundary-aligned prompts, enabling more effective and targeted mitigation of overrefusal.This approach not only provides a more precise and interpretable view of model safety decisions but also seamlessly extends to multilingual scenarios.We have explored the safety decision boundaries of various LLMs and construct the MORBENCH evaluation set to facilitate robust assessment of model safety and helpfulness across multiple languages.

Open source software has evolved into a fundamental element of the contemporary information sector; however, security threats within its supply chain are persistently rising. Within the collaborative development framework of open source, the introduction of malicious code can lead to significant security vulnerabilities. Conventional methods for detecting these vulnerabilities, which rely on machine learning, face challenges such as a lack of sufficient datasets, inadequate deep semantic understanding, and limitations to single-vulnerability detection. To address these challenges, we introduce a novel approach named VulnTrace, which analyzes historical records of submissions in open source projects to construct a high-quality dataset of vulnerabilities with accurate labels. VulnTrace employs Word2Vec alongside Abstract Syntax Tree (AST) technologies to capture both the semantic and structural details of code segments and utilizes a Transformer model for precise vulnerability identification, thereby enhancing accuracy and interpretability in detection. Experimental results indicate that VulnTrace achieves approximately 93% accuracy, 95% precision, 83% recall and an F1 score of 88% in vulnerability detection tasks, significantly reducing false positives and demonstrating remarkable robustness.

Multivariate soft sensors seek to provide accurate estimation of multiple quality variables through the analysis of measurable process variables, representing a significant advance over the traditional focus on single-quality variable sensors within industrial manufacturing. Current progress stays in applying parameter-sharing neural architectures while ignoring two fundamental issues: (1) catastrophic interference, where the indiscriminate sharing of parameters degrades performance due to the discrepancy of objectives; (2) seesaw optimization, where the optimizer overly focuses on one dominant yet simple objective at the expense of others. To address these issues, we reformulate multivariate soft sensors as a multi-objective optimization problem and propose the Task-aware Mixture-of-Experts framework for achieving the Pareto optimum (TMoE-P). Specifically, to handle issue(1), we propose an Objective-aware Mixture-of-Experts (OMoE) module, which consists of objective-specific and objective-shared experts to realize parameter sharing while accommodating the discrepancy between objectives. To handle issue(2), we devise a Pareto Objective Weighting (POW) module, which dynamically balances the weights of learning objectives to approximate the Pareto optimum among competing objectives. Our evaluations on a public soft sensor benchmark showcase TMoE-P’s superior performance, confirming its enhanced accuracy and robustness.Note to Practitioners—Addressing the burgeoning complexity of estimating multiple quality variables in industrial manufacturing processes, this study introduces a novel Task-aware Mixture-of-Experts framework aiming for the Pareto Optimum (TMoE-P). This framework mitigates the issues of catastrophic interference and seesaw optimization by achieving a delicate balance between parameter sharing and maintaining distinctness between objectives, guiding towards the Pareto optimum. The empirical findings affirm the framework’s capability to adeptly handle the complex interplay of variables, potentially enhancing the efficiency and reliability of industrial processes. While initially developed for multivariate data in industrial applications, the TMoE-P framework’s modular and adaptable nature facilitates its extension to a broad spectrum of data structures, optimizers, and neural architectures. Its adaptability offers practitioners a valuable tool to fulfill specific task requirements, making a modest contribution to industrial process control and monitoring.

Multivariate soft sensors are critical in industrial manufacturing for providing precise estimations of multiple quality variables and ensuring data reliability and completeness. Existing methods predominantly focus on parameter-sharing architectures while overlooking two fundamental issues: 1) catastrophic interference, where sharing parameters across all tasks leads to performance degradation; 2) uncontrollable optimization, where the optimizer lacks controllability over task priorities, misleading specific tasks converge to unexpected values. To handle these issues and enhance multivariate modeling, we propose a controllable Mixture-of-Experts (ControlMoE) framework for industrial soft sensors, consisting of a Mixture-of-Sequential-Experts (MoSE) module and a Proportional-Integral-Derivative Calibrating (PIDC) module. The MoSE module integrates sequential expert networks with task-specific gating networks, enabling parameter sharing while preserving task distinctions to mitigate catastrophic interference. The PIDC module employs an innovative PID controller to dynamically calibrate task learning weights, ensuring expected outcomes through feedback control and enhancing optimization controllability. Evaluations on two industrial datasets from Chinese nuclear monitoring stations demonstrate that ControlMoE achieves superior accuracy and controllability in multivariate soft sensor modeling. Note to Practitioners—This paper tackles the challenges in applying multivariate soft sensors within industrial manufacturing processes, particularly in environments such as nuclear monitoring where precision and control over multiple quality variables are critical. Traditional methods often suffer from performance degradation due to parameter sharing across all tasks-known as catastrophic interference–and lack control in optimizing multiple tasks simultaneously, leading to suboptimal outcomes. Our proposed framework, ControlMoE, introduces a novel architecture that combines sequential expert networks with a dynamic weight adjustment mechanism using PID controllers. This enables precise control over task-specific optimizations, ensuring that the desired importance and sequence of tasks are maintained, thereby enhancing both accuracy and operational control. We believe this framework could be widely applicable in various industrial settings where multivariate monitoring and control are necessary, improving not only performance but also the reliability of the data produced for critical decision-making.

Recommendation systems are pivotal in improving user experience on various digital platforms. However, observational training data in recommendation systems introduce selection bias, which leads to a distributional discrepancy between training data and real-world scenarios, resulting in suboptimal performance. Current causal debiasing methods such as inverse propensity score and doubly robust rely on accurately estimated propensity scores, typically optimized through negative log-likelihood (NLL) minimization. However, recent studies have highlighted the limitations of this approach, as perfect NLL minimization may not adequately correct for selection bias. To address this issue, we propose Wasserstein Balancing Metric (WBM), a novel metric that measures and enhances the balancing capacity of propensity scores in causal debiasing methods by minimizing the Wasserstein discrepancy between reweighted populations. On the basis, we introduce IPS-WBM and DR-WBM, incorporating WBM as a regularizer in standard inverse propensity score and doubly robust estimators, which enhances causal balancing capacity without introducing additional bias. Extensive experiments on three real-world recommendation datasets demonstrate that our methods improve the causal balancing capability of learned propensities and enhance debiasing performance.

Missing data imputation is a critical aspect of the Industrial Internet-of-Things (IIoT), which is uniquely challenged by local relationships within data due to different operational contexts and phases. Current imputation methods struggle to accommodate local relationships due to their black-box nature or limited capacity. To bridge this gap, we approach data imputation as a distribution alignment problem and leverage optimal transport to instantiate it for enhanced capacity. Specifically, we first introduce the similarity preserved optimal transport (SPOT) problem, with a conditional gradient solution to compute the transport cost. Subsequently, we propose the SPOT for imputation (SPOT-I) framework. It minimizes the transport cost of SPOT for distribution alignment and uses the gradient to update imputations, which maintains local similarity and refines imputation due to the characteristics of SPOT. Experiments on IIoT datasets showcase the superiority of SPOT-I over state-of-the-art imputation methods.

Accurate estimation of multiple quality variables is critical for building industrial soft sensor models, which have long been confronted with data efficiency and negative transfer issues. Methods sharing backbone parameters among tasks address the data efficiency issue; however, they still fail to mitigate the negative transfer problem. To address this issue, a balanced mixture-of-experts (BMoE) is proposed in this work, which consists of a multigate mixture-of-experts module and a task gradient balancing (TGB) module. The mixture-of-experts module aims to portray task relationships, while the TGB module balances the gradients among tasks dynamically. Both of them cooperate to mitigate the negative transfer problem. Experiments on the typical sulfur recovery unit demonstrate that BMoE models task relationship and balances the training process effectively, and achieves better performance than baseline models significantly.