Post-click conversion rate (CVR) prediction is a central task in recommender systems, yet selection bias creates a severe distributional gap between the clicked training samples and the entire inference space. To address selection bias, propensity-based methods such as inverse propensity scoring (IPS) and doubly robust (DR) have been adopted, which aim to estimate the unbiased learning objective from biased training samples. However, these approaches assume strictly positive propensities, implying every user-item pair has a nonzero probability of interaction. In practice, such positivity assumption maybe violated, for example, in food-delivery platforms, some restaurants located more than 10 kilometers away will be blocked for recommendation. In this study, we theoretically show that when such zero-propensity samples, termed extrapolation samples exist, both IPS and DR estimators become biased. To overcome this limitation, we propose ExtraDebias method, which enables debiased recommendation in both non-extrapolation and extrapolation samples. Specifically, we first train a propensity model to identify extrapolation samples with extremely small propensity estimates, then estimate their pseudo-label intervals, and derive an upper bound of the learning objective for extrapolation samples. By minimizing the derived upper bound, debiased learning on extrapolation samples is ensured, while unbiased learning on non-extrapolation samples is achieved by standard IPS. Experiments on four real-world offline datasets and one online A/B test show that ExtraDebias effectively minimizes prediction errors on extrapolation samples and achieves optimal performance.

Integrating graph neural networks (GNNs) with variational inference (VI) provides a promising direction for blending structured prior knowledge with observational empirical data for data-driven industrial process modeling. However, this task requires inference of the normalized adjacency matrix (NAM), where each row is normalized to be non-negative and to sum to one, matching the support of Dirichlet distribution. This requirement presents two main technical challenges: 1) intractable Kullback-Leibler (KL) divergence optimization between Dirichlet distributions, and 2) constrained optimization for standard- gradient-descent-based neural network parameter optimization. To handle issue 1), we first formulate the inference of the NAM as a differential equation simulation problem and derive an easy-to-implement expression to iteratively improve the KL divergence without explicitly computing it. Based on this, to alleviate issue 2), we involve Riemannian optimization to precondition this simulation procedure, which ensures that the inferred NAM conforms to the row-normalization constraint. After that, we collectively designate these approaches for NAM inference as Preconditioned-Simulation-Induced Variational Inference ($\psi$-VI), and provide theoretical guarantees of convergence. On this foundation, we propose a new graph neural network architecture, the Preconditioned-Simulation-Induced-based Variational Graph Neural Network ($\psi$-VGNN) for industrial process modeling. Finally, we validate the efficacy of$\psi$-VGNN through comprehensive experiments on industrial modeling tasks.

Accurate imputation of missing data is crucial in the Industrial Internet-of-Things (IIoT), where operations are often compromised by noisy samples from harsh environments. Traditional imputation methods struggle with such noise due to their black-box nature or lack of adaptability. To address this issue, we recast data imputation as a distribution alignment challenge, utilizing the flexibility of optimal transport (OT) to handle noisy samples. Specifically, we first introduce the Proximal Optimal Transport (POT) problem, where the transportation cost is obtained by the network simplex approach with a selective matching mechanism, which renders it capable of matching distributions with noisy samples. Subsequently, we propose the POT-I framework, where the objective is to minimize the transport cost of POT. The produced gradient is used to refine the imputation value, which achieves missing data imputation (MDI) while getting robustness to noisy samples. Experiments on real-world IIoT datasets demonstrate the superiority of POT-I over state-of-the-art imputation methods.

Time series modeling presents unique challenges due to autocorrelation in both historical data and future sequences. While current research predominantly addresses autocorrelation within historical data, the correlations among future labels are often overlooked. Specifically, modern forecasting models primarily adhere to the Direct Forecast (DF) paradigm, generating multi-step forecasts independently and disregarding label correlations over time. In this work, we demonstrate that the learning objective of DF is biased in the presence of label correlation. To address this issue, we propose the Frequency-enhanced Direct Forecast (FreDF), which mitigates label correlation by learning to forecast in the frequency domain, thereby reducing estimation bias. Our experiments show that FreDF significantly outperforms existing state-of-the-art methods and is compatible with a variety of forecast models. Code is available at https://github.com/Master-PLC/FreDF.

Missing data imputation through distribution alignment has demonstrated advantages for non-temporal datasets but exhibits suboptimal performance in time-series applications. The primary obstacle is crafting a discrepancy measure that simultaneously (1) captures temporal patterns—accounting for periodicity and temporal dependencies inherent in time-series—and (2) accommodates non-stationarity, ensuring robustness amidst multiple coexisting temporal patterns. In response to these challenges, we introduce the Proximal Spectrum Wasserstein (PSW) discrepancy, a novel discrepancy tailored for comparing two \textit{sets} of time-series based on optimal transport. It incorporates a pairwise spectral distance to encapsulate temporal patterns, and a selective matching regularization to accommodate non-stationarity. Subsequently, we develop the PSW for Imputation (PSW-I) framework, which iteratively refines imputation results by minimizing the PSW discrepancy. Extensive experiments demonstrate that PSW-I effectively accommodates temporal patterns and non-stationarity, outperforming prevailing time-series imputation methods. Code is available at https://github.com/FMLYD/PSW-I.

Implicit feedback recommendation is challenged by the missing negative feedback essential for effective model training. Existing methods often resort to negative sampling, a technique that assumes unlabeled interactions as negative samples. This assumption risks misclassifying potential positive samples within the unlabeled data, thereby undermining model performance. To address this issue, we introduce PURL, a model-agnostic framework that reframes implicit feedback recommendation as a weakly supervised learning task, eliminating the need for negative samples. However, its unbiasedness hinges on the accurate estimation of the class prior. To address this challenge, we propose Progressive Proximal Transport (PPT), which estimates the class prior by minimizing the proximal transport cost between positive and unlabeled samples. Experiments on three real-world datasets validate the efficacy of PURL in terms of improved recommendation quality. Code is available at https://github.com/HowardZJU/weakrec.

Heterogeneous treatment effect (HTE) estimation from observational data poses significant challenges due to treatment selection bias. Existing methods address this bias by minimizing distribution discrepancies between treatment groups in latent space, focusing on global alignment. However, the fruitful aspect of local proximity, where similar units exhibit similar outcomes, is often overlooked. In this study, we propose Proximity-enhanced CounterFactual Regression (CFR-Pro) to exploit proximity for enhancing representation balancing within the HTE estimation context. Specifically, we introduce a pair-wise proximity regularizer based on optimal transport to incorporate the local proximity in discrepancy calculation. However, the curse of dimensionality renders the proximity measure and discrepancy estimation ineffective-exacerbated by limited data availability for HTE estimation. To handle this problem, we further develop an informative subspace projector, which trades off minimal distance precision for improved sample complexity. Extensive experiments demonstrate that CFR-Pro accurately matches units across different treatment groups, effectively mitigates treatment selection bias, and significantly outperforms competitors. Code is available at https://github.com/HowardZJU/CFR-Pro.

Training time-series forecasting models poses unique challenges in loss function design. Most existing approaches adopt temporal mean squared error, but this study reveals two critical limitations: (1) it ignores the presence of label autocorrelation, which biases it from the true label sequence likelihood; (2) it involves excessive number of tasks, which complicates optimization, especially for long-term forecasting. To address these issues, we introduce Time-o1, a transform-enhanced loss function for time-series forecasting. The central idea is to transform the label sequence into decorrelated components with discriminated significance. Models are then trained to align the most significant components, thereby effectively mitigating label autocorrelation and reducing task amount. Experiments demonstrate that Time-o1 achieves state-of-the-art performance and is compatible with various forecast models. Code is available at https://github.com/Master-PLC/Time-o1.

Iterative imputation is a prevalent method for missing data imputation, where each feature is imputed iteratively by treating it as a target variable estimated from all other features. However, iterative imputation method suffers from two principal limitations: (1) it imposes a single parametric model form to impute all features, neglecting the potential for optimal models to vary among features, which risks model misspecification; and (2) it assumes every feature contains missing values, overlooking the potential presence of non-missing features, termed as oracle features, which are informative for imputation. To address these limitations, we propose kernel point imputation (KPI), a bi-level optimization framework for iterative missing data imputation. At the inner level, KPI adaptively learns the optimal model form for each feature within a reproducing kernel Hilbert space, addressing limitation (1). At the outer level, KPI utilizes oracle features as supervisory signals to iteratively refine the imputations, addressing limitation (2). Experiments demonstrate that KPI outperforms competitive imputation methods. Code is available at https://github.com/FMLYD/kpi.git.

Multivariate soft sensors seek to provide accurate estimation of multiple quality variables through the analysis of measurable process variables, representing a significant advance over the traditional focus on single-quality variable sensors within industrial manufacturing. Current progress stays in applying parameter-sharing neural architectures while ignoring two fundamental issues: (1) catastrophic interference, where the indiscriminate sharing of parameters degrades performance due to the discrepancy of objectives; (2) seesaw optimization, where the optimizer overly focuses on one dominant yet simple objective at the expense of others. To address these issues, we reformulate multivariate soft sensors as a multi-objective optimization problem and propose the Task-aware Mixture-of-Experts framework for achieving the Pareto optimum (TMoE-P). Specifically, to handle issue(1), we propose an Objective-aware Mixture-of-Experts (OMoE) module, which consists of objective-specific and objective-shared experts to realize parameter sharing while accommodating the discrepancy between objectives. To handle issue(2), we devise a Pareto Objective Weighting (POW) module, which dynamically balances the weights of learning objectives to approximate the Pareto optimum among competing objectives. Our evaluations on a public soft sensor benchmark showcase TMoE-P’s superior performance, confirming its enhanced accuracy and robustness.Note to Practitioners—Addressing the burgeoning complexity of estimating multiple quality variables in industrial manufacturing processes, this study introduces a novel Task-aware Mixture-of-Experts framework aiming for the Pareto Optimum (TMoE-P). This framework mitigates the issues of catastrophic interference and seesaw optimization by achieving a delicate balance between parameter sharing and maintaining distinctness between objectives, guiding towards the Pareto optimum. The empirical findings affirm the framework’s capability to adeptly handle the complex interplay of variables, potentially enhancing the efficiency and reliability of industrial processes. While initially developed for multivariate data in industrial applications, the TMoE-P framework’s modular and adaptable nature facilitates its extension to a broad spectrum of data structures, optimizers, and neural architectures. Its adaptability offers practitioners a valuable tool to fulfill specific task requirements, making a modest contribution to industrial process control and monitoring.

Multivariate soft sensors are critical in industrial manufacturing for providing precise estimations of multiple quality variables and ensuring data reliability and completeness. Existing methods predominantly focus on parameter-sharing architectures while overlooking two fundamental issues: 1) catastrophic interference, where sharing parameters across all tasks leads to performance degradation; 2) uncontrollable optimization, where the optimizer lacks controllability over task priorities, misleading specific tasks converge to unexpected values. To handle these issues and enhance multivariate modeling, we propose a controllable Mixture-of-Experts (ControlMoE) framework for industrial soft sensors, consisting of a Mixture-of-Sequential-Experts (MoSE) module and a Proportional-Integral-Derivative Calibrating (PIDC) module. The MoSE module integrates sequential expert networks with task-specific gating networks, enabling parameter sharing while preserving task distinctions to mitigate catastrophic interference. The PIDC module employs an innovative PID controller to dynamically calibrate task learning weights, ensuring expected outcomes through feedback control and enhancing optimization controllability. Evaluations on two industrial datasets from Chinese nuclear monitoring stations demonstrate that ControlMoE achieves superior accuracy and controllability in multivariate soft sensor modeling. Note to Practitioners—This paper tackles the challenges in applying multivariate soft sensors within industrial manufacturing processes, particularly in environments such as nuclear monitoring where precision and control over multiple quality variables are critical. Traditional methods often suffer from performance degradation due to parameter sharing across all tasks-known as catastrophic interference–and lack control in optimizing multiple tasks simultaneously, leading to suboptimal outcomes. Our proposed framework, ControlMoE, introduces a novel architecture that combines sequential expert networks with a dynamic weight adjustment mechanism using PID controllers. This enables precise control over task-specific optimizations, ensuring that the desired importance and sequence of tasks are maintained, thereby enhancing both accuracy and operational control. We believe this framework could be widely applicable in various industrial settings where multivariate monitoring and control are necessary, improving not only performance but also the reliability of the data produced for critical decision-making.

Accurate imputation of missing data is pivotal in real-world industrial applications. Traditional direct imputers, which utilize basic statistics to replace missing elements, offer a practical solution but struggle to adapt to the complex patterns in industrial data, leaving a gap in the research landscape. This study explores the untapped potential of direct imputers, enhancing their adaptability and capacity to handle complex patterns in industrial data through optimal transport (OT) theory, with a focus on preserving local sample-wise similarity as an exemplar. To these ends, we construct a Local Similarity Preserved Transport (LSPT) problem, with a solution algorithm based on the Frank-Wolfe technique to compute transport cost. Subsequently, we propose the LSPT-D framework, which employs the transport cost of LSPT for distribution matching, directing the gradient flow to the missing data points to update the imputations directly. This strategy maintains local similarity throughout the imputation process thereby enhancing the overall imputation quality. Our experiments demonstrate that LSPT-D outperform various baselines in industrial missing data imputation. Note to Practitioners—Accurate missing data imputation is essential for enhancing the reliability of data analytics and reducing decision-making risks in industrial automation. This study introduces LSPT-D, a non-parametric imputation technique based on OT technology. Unique to LSPT-D is its ability to preserve local similarity during the imputation process, rendering it particularly advantageous for datasets with varying operational phases and load conditions. In industrial applications, LSPT-D not only significantly improves imputation quality compared to various baseline methods but also maintains modest running costs. Additionally, it serves as an exemplar for developing OT-based imputation strategies that capitalize on the inherent properties of data to improve imputation performance. However, LSPT-D operates under the independent and identically distributed assumption and is thus best applied in scenarios where temporal dependencies, such as trends and seasonality, are minimal or have been previously neutralized.

Post-click conversion rate (CVR) estimation is a fundamental task in developing effective recommender systems, yet it faces challenges from data sparsity and sample selection bias. To handle both challenges, the entire space multitask models are employed to decompose the user behavior track into a sequence of exposure$\rightarrow $click$\rightarrow $conversion, constructing surrogate learning tasks for CVR estimation. However, these methods suffer from two significant defects: (1) intrinsic estimation bias (IEB), where the CVR estimates are higher than the actual values; (2) false independence prior (FIP), where the causal relationship between clicks and subsequent conversions is potentially overlooked. To overcome these limitations, we develop a model-agnostic framework, namely Entire Space Counterfactual Multitask Model (ESCM2), which incorporates a counterfactual risk minimizer within the entire space multitask framework to regularize CVR estimation. Experiments conducted on large-scale industrial recommendation datasets and an online industrial recommendation service demonstrate that ESCM2 effectively mitigates IEB and FIP defects and substantially enhances recommendation performance.

Recommendation systems are pivotal in improving user experience on various digital platforms. However, observational training data in recommendation systems introduce selection bias, which leads to a distributional discrepancy between training data and real-world scenarios, resulting in suboptimal performance. Current causal debiasing methods such as inverse propensity score and doubly robust rely on accurately estimated propensity scores, typically optimized through negative log-likelihood (NLL) minimization. However, recent studies have highlighted the limitations of this approach, as perfect NLL minimization may not adequately correct for selection bias. To address this issue, we propose Wasserstein Balancing Metric (WBM), a novel metric that measures and enhances the balancing capacity of propensity scores in causal debiasing methods by minimizing the Wasserstein discrepancy between reweighted populations. On the basis, we introduce IPS-WBM and DR-WBM, incorporating WBM as a regularizer in standard inverse propensity score and doubly robust estimators, which enhances causal balancing capacity without introducing additional bias. Extensive experiments on three real-world recommendation datasets demonstrate that our methods improve the causal balancing capability of learned propensities and enhance debiasing performance.

Accurate quality variable inference by process variables is the core of industrial inferential sensor modeling, where recent advancements have seen deep learning (DL) models achieving remarkable success. However, integrating knowledge of unit operations is critical for improving inferential sensor performance, yet it has received little attention. The main challenge lies in the incompleteness and correctness of industrial knowledge due to its semi-empirical nature and inevitable engineering errors. Addressing this, this article introduces the gradient knowledge network based on the graph neural network’s message-passing mechanism within the variational Bayesian inference framework, which naturally copes with the abovementioned issues by fusing observational data. Initially, the prior knowledge about the process variables, which mirrors the graph in graph neural network, is parameterized as Dirichlet distribution based on the analysis of message-passing mechanism. However, the divergence computation and normalization constraints are challenging for model implementation. To navigate these challenges, the Bayesian inference problem is transformed into an optimization problem, subsequently recast as a simulation problem induced by the gradient field, ensuring compatibility with DL backends. Furthermore, a theoretical iteration equation is derived to maintain the normalization constraint. The architecture of the proposed model and its learning algorithm are then detailed. Finally, various experiments are conducted on two real industrial processes to demonstrate the model’s efficacy from the perspective of prediction accuracy, sensitivity analysis, and ablation study.

Enhancing the generalization performance of neural networks given limited data availability remains a formidable challenge, due to the model selection trade-off between training error and generalization gap. To handle this challenge, we present a posterior optimization issue, specifically designed to reduce the generalization error of trained neural networks. To operationalize this concept, we propose a Doubly-Robust Boosting machine (DRBoost) which consists of a statistical learner and a zero-order optimizer. The statistical learner reduces the model capacity and thus the generalization gap; the zero-order optimizer minimizes the training error in a gradient-free manner. The two components cooperate to reduce the generalization error of a fully trained neural network in a doubly robust manner. Furthermore, the statistical learner alleviates the multicollinearity in the discriminative layer and enhances the generalization performance. The zero-order optimizer eliminates the reliance on gradient calculation and offers more flexibility in learning objective selection. Experiments demonstrate that DRBoost improves the generalization performance of various prevalent neural network backbones effectively.

Recommender system aims to recommend items or information that may interest users based on their behaviors and preferences. However, there may be sampling selection bias in the data collection process, i.e., the collected data is not a representative of the target population. Many debiasing methods are developed based on pseudo-labelings. Nevertheless, the validity of these methods relies heavily on accurate pseudo-labelings (i.e., the imputed labels), which is difficult to satisfy in practice. In this paper, we theoretically propose several novel doubly robust estimators that are unbiased when either (a) the pseudo-labelings deviate from the true labels with an arbitrary user-specific inductive bias, item-specific inductive bias, or a combination of both, or (b) the learned propensities are accurate. We further propose a propensity reconstruction learning approach that adaptively updates the constraint weights using an attention mechanism and effectively controls the variance. Extensive experiments show that our approach outperforms the state-of-the-art on one semi-synthetic and three real-world datasets.

Ratings of a user to most items in recommender systems are usually missing not at random (MNAR), largely because users are free to choose which items to rate. To achieve unbiased learning of the prediction model under MNAR data, three typical solutions have been proposed, including error-imputation-based (EIB), inverse-propensity-scoring (IPS), and doubly robust (DR) methods. However, these methods ignore an alternative form of bias caused by the inconsistency between the observed ratings and the users' true preferences, also known as noisy feedback or outcome measurement errors (OME), e.g., due to public opinion or low-quality data collection process. In this work, we study intersectional threats to the unbiased learning of the prediction model from data MNAR and OME in the collected data. First, we design OME-EIB, OME-IPS, and OME-DR estimators, which largely extend the existing estimators to combat OME in real-world recommendation scenarios. Next, we theoretically prove the unbiasedness and generalization bound of the proposed estimators. We further propose an alternate denoising training approach to achieve unbiased learning of the prediction model under MNAR data with OME. Extensive experiments are conducted on three real-world datasets and one semi-synthetic dataset to show the effectiveness of our proposed approaches. The code is available at https://github.com/haoxuanli-pku/KDD24-OME-DR.

Diffusion models have demonstrated competitive performance in missing data imputation (MDI) task. However, directly applying diffusion models to MDI produces suboptimal performance due to two primary defects. First, the sample diversity promoted by diffusion models hinders the accurate inference of missing values. Second, data masking reduces observable indices for model training, obstructing imputation performance. To address these challenges, we introduce $\underline{\text{N}}$egative $\underline{\text{E}}$ntropy-regularized $\underline{\text{W}}$asserstein gradient flow for $\underline{\text{Imp}}$utation (NewImp), enhancing diffusion models for MDI from a gradient flow perspective. To handle the first defect, we incorporate a negative entropy regularization term into the cost functional to suppress diversity and improve accuracy. To handle the second defect, we demonstrate that the imputation procedure of NewImp, induced by the conditional distribution-related cost functional, can equivalently be replaced by that induced by the joint distribution, thereby naturally eliminating the need for data masking. Extensive experiments validate the effectiveness of our method. Code is available at [https://github.com/JustusvLiebig/NewImp](https://github.com/JustusvLiebig/NewImp).

Recommendation systems guide users in locating their desired information within extensive content repositories. Usually, a recommendation model is optimized to enhance accuracy metrics from a user utility standpoint, such as click-through rate or matching relevance. However, a responsible industrial recommendation model must address not only user utility (responsibility to users) but also other objectives, including increasing platform revenue (responsibility to platforms), ensuring fairness (responsibility to content creators), and maintaining unbiasedness (responsibility to long-term healthy development). Multi-objective learning is a promising approach for achieving responsible recommendation models. Nevertheless, current methods encounter two challenges: difficulty in scaling to heterogeneous objectives within a unified framework, and inadequate controllability over objective priority during optimization, leading to uncontrollable solutions.

Nonlinear dynamical probabilistic latent variable model (NDPLVM) and its variants, essential in industrial inferential sensors, face challenges in latent space inference and deep learning (DL) backend implementation. The first issue arises from the assumption that covariates directly infer the latent variable, potentially leading to inaccuracies. The second issue involves the discrepancy between the probabilistic distribution function form of NDPLVMs and data sample-based operation of DL backends. Addressing these, this study introduces the optimal control-NDPLVM (OC-NDPLVM), a model designed to enhance performance by analyzing NDPLVMs learning and tackling these issues. For the first problem, NDPLVMs' learning is reinterpreted as an optimization problem, solved by alternating direction method of multipliers, and selecting the inference network's input via studying optimal solution's structure. To address the second issue, OC-NDPLVM adapts mean and covariance equations for compatibility with DL backends. This model's effectiveness is validated through experiments on inferential sensor datasets.

Missing data imputation is a critical aspect of the Industrial Internet-of-Things (IIoT), which is uniquely challenged by local relationships within data due to different operational contexts and phases. Current imputation methods struggle to accommodate local relationships due to their black-box nature or limited capacity. To bridge this gap, we approach data imputation as a distribution alignment problem and leverage optimal transport to instantiate it for enhanced capacity. Specifically, we first introduce the similarity preserved optimal transport (SPOT) problem, with a conditional gradient solution to compute the transport cost. Subsequently, we propose the SPOT for imputation (SPOT-I) framework. It minimizes the transport cost of SPOT for distribution alignment and uses the gradient to update imputations, which maintains local similarity and refines imputation due to the characteristics of SPOT. Experiments on IIoT datasets showcase the superiority of SPOT-I over state-of-the-art imputation methods.

Monitoring of energy conversion process confronts great difficulties due to extreme value jumps or data packet loss under extreme operating conditions, consequently resulting in data missing. To tackle these issues, researchers propose diffusion-based time series imputation approaches. However, there are critical limitations of methods adopted by these models: first, dense reverse inference (DRI), where conventional diffusion methods suffer from time-consuming imputation procedures and thus the loss of effective information over a long transmission process; second, indirect prediction strategy (IPS), which causes inaccurate temporal data imputation results. To address these limitations, we propose denoising diffusion straightforward models (DDSMs) for missing data imputation in energy conversion process monitoring. Specifically, based on the conditional mechanism, we innovatively use the accelerated sampling strategy in temporal models to reduce the lengthy inference time caused by DRI and propose a well-designed straightforward training algorithm for a straightforward estimation to improve the imprecise inference results acquired by IPS. Extensive experiments on real-world dataset demonstrate the effectiveness and superiority of DDSM and our approach consistently outperforms previous temporal generative methods significantly.

Time-Series Forecasting based on Cumulative Data (TSFCD) is a crucial problem in decision-making across various industrial scenarios. However, existing time-series forecasting methods often overlook two important characteristics of cumulative data, namely monotonicity and irregularity, which limit their practical applicability. To address this limitation, we propose a principled approach called Monotonic neural Ordinary Differential Equation (MODE) within the framework of neural ordinary differential equations. By leveraging MODE, we are able to effectively capture and represent the monotonicity and irregularity in practical cumulative data. Through extensive experiments conducted in a bonus allocation scenario, we demonstrate that MODE outperforms state-of-the-art methods, showcasing its ability to handle both monotonicity and irregularity in cumulative data and delivering superior forecasting performance.

Detecting and diagnosing anomalies in observational data plays a crucial role in various real-world applications, such as e-commerce applet maintenance. Unsupervised machine learning techniques are typically employed for anomaly detection and diagnosis due to their convenience and independence from labeled data. Density estimation (DE), as one of the most widely used unsupervised machine learning techniques for anomaly detection, can be categorized into kernel density estimation (KDE)-based methods and normalizing flow (NF)-based methods. While KDE-based methods offer fast computation speed, they often ignore the complex manifold structure present in observational data. On the other hand, NF-based methods address the manifold issue but suffer from longer computation times. In this study, we propose a novel DE-based anomaly detection & diagnosis method using Stein Variational Gradient Descent (SVGD), aiming to leverage the strengths of KDE and NF approaches. Firstly, we rigorously derive the DE capability of SVGD through mathematical analysis. Subsequently, we demonstrate the ability of the SVGD method to perform anomaly diagnosis based on input feature attribution. Finally, to validate the effectiveness of our approach, we conduct experiments using synthetic, benchmark, and industrial datasets. The results demonstrate the superior performance and practical applicability of our proposed method.

In recommender systems, the collected data used for training is always subject to selection bias, which poses a great challenge for unbiased learning. Previous studies proposed various debiasing methods based on observed user and item features, but ignored the effect of hidden confounding. To address this problem, recent works suggest the use of sensitivity analysis for worst-case control of the unknown true propensity, but only valid when the true propensity is near to the nominal propensity within a finite bound. In this paper, we first perform theoretical analysis to reveal the possible failure of previous approaches, including propensity-based, multi-task learning, and bi-level optimization methods, in achieving unbiased learning when hidden confounding is present. Then, we propose a unified multi-task learning approach to remove hidden confounding, which uses a few unbiased ratings to calibrate the learned nominal propensities and nominal error imputations from biased data. We conduct extensive experiments on three publicly available benchmark datasets containing a fully exposed large-scale industrial dataset, validating the effectiveness of the proposed methods in removing hidden confounding.

Estimating individual treatment effects from observational data is challenging due to treatment selection bias. Prevalent methods mainly mitigate this issue by aligning different treatment groups in the latent space, the core of which is the calculation of distribution discrepancy. However, two issues that are often overlooked can render these methods invalid: (1) mini-batch sampling effects (MSE), where the calculated discrepancy is erroneous in non-ideal mini-batches with outcome imbalance and outliers; (2) unobserved confounder effects (UCE), where the unobserved confounders are not considered in the discrepancy calculation. Both of these issues invalidate the calculated discrepancy, mislead the training of estimators, and thus impede the handling of treatment selection bias. To tackle these issues, we propose Entire Space CounterFactual Regression (ESCFR), which is a new take on optimal transport technology in the context of causality. Specifically, based on the canonical optimal transport framework, we propose a relaxed mass-preserving regularizer to address the MSE issue and design a proximal factual outcome regularizer to handle the UCE issue. Extensive experiments demonstrate that ESCFR estimates distribution discrepancy accurately, handles the treatment selection bias effectively, and outperforms prevalent competitors significantly.

Accurate estimation of multiple quality variables is critical for building industrial soft sensor models, which have long been confronted with data efficiency and negative transfer issues. Methods sharing backbone parameters among tasks address the data efficiency issue; however, they still fail to mitigate the negative transfer problem. To address this issue, a balanced mixture-of-experts (BMoE) is proposed in this work, which consists of a multigate mixture-of-experts module and a task gradient balancing (TGB) module. The mixture-of-experts module aims to portray task relationships, while the TGB module balances the gradients among tasks dynamically. Both of them cooperate to mitigate the negative transfer problem. Experiments on the typical sulfur recovery unit demonstrate that BMoE models task relationship and balances the training process effectively, and achieves better performance than baseline models significantly.

Omnidirectional images (ODI), also known as 360 images, have recently attracted extensive attention from both academia and industry. However, due to storage and transmission limitations, ODIs are usually at extremely low resolution. Thus, it is necessary to restore a high-resolution ODI from a low-resolution ODI, i.e., omnidirectional image super-resolution (ODI-SR). Different from traditional two-dimensional (2D) image SR, the challenge of ODI-SR is the nonuniformly distributed pixel density and geometric distortion across latitudes, which makes traditional SR methods difficult to be applied in ODI-SR. Towards ODI-SR, we propose in this paper a novel latitude-aware upscaling network, namely LAU-Net+, which fully considers the above characteristics of ODIs. In our network, different latitude bands can learn to adopt distinct upscaling factors, which significantly saves the computational resources and improves the SR efficiency. Specifically, a Laplacian multilevel pyramid network is introduced in which the upscaling factor is gradually increased with the number of levels. Each level is composed of a feature enhancement module (FEM), a drop-band decision module (DDM) and a high-latitude enhancement module (HEM). The FEM module serves to enhance the high-level features extracted from the input ODI, while the role of DDM is to dynamically drop the unnecessary high latitude bands and send the remained bands to the next level. The HEM is adopted to further enhance high-level features of dropped latitude bands with a lightweight architecture. In DDM, we develop a reinforcement learning scheme with a latitude adaptive reward to determine which band should be dropped. To the best of our knowledge, our method is the first work which considers the latitude characteristics for ODI-SR task. Extensive experimental results demonstrate that our LAU-Net+ achieves state-of-the-art results on ODI-SR both quantitatively and qualitatively on various ODI datasets.

Accurate estimation of post-click conversion rate is critical for building recommender systems, which has long been confronted with sample selection bias and data sparsity issues. Methods in the Entire Space Multi-task Model (ESMM) family leverage the sequential pattern of user actions, \ie $impression\rightarrow click \rightarrow conversion$ to address data sparsity issue. However, they still fail to ensure the unbiasedness of CVR estimates. In this paper, we theoretically demonstrate that ESMM suffers from the following two problems: (1) Inherent Estimation Bias (IEB) for CVR estimation, where the CVR estimate is inherently higher than the ground truth; (2) Potential Independence Priority (PIP) for CTCVR estimation, where ESMM might overlook the causality from click to conversion. To this end, we devise a principled approach named Entire Space Counterfactual Multi-task Modelling (ESCM$^2$), which employs a counterfactual risk miminizer as a regularizer in ESMM to address both IEB and PIP issues simultaneously. Extensive experiments on offline datasets and online environments demonstrate that our proposed ESCM$^2$ can largely mitigate the inherent IEB and PIP issues and achieve better performance than baseline models.

The omnidirectional images (ODIs) are usually at low-resolution, due to the constraints of collection, storage and transmission. The traditional two-dimensional (2D) image super-resolution methods are not effective for spherical ODIs, because ODIs tend to have non-uniformly distributed pixel density and varying texture complexity across latitudes. In this work, we propose a novel latitude adaptive upscaling network (LAU-Net) for ODI super-resolution, which allows pixels at different latitudes to adopt distinct upscaling factors. Specifically, we introduce a Laplacian multi-level separation architecture to split an ODI into different latitude bands, and hierarchically upscale them with different factors. In addition, we propose a deep reinforcement learning scheme with a latitude adaptive reward, in order to automatically select optimal upscaling factors for different latitude bands. To the best of our knowledge, LAU-Net is the first attempt to consider the latitude difference for ODI super-resolution. Extensive results demonstrate that our LAU-Net significantly advances the super-resolution performance for ODIs. Codes are available at https://github.com/wangh-allen/LAU-Net.

Dynamic adaptive video streaming over hypertext transfer protocol (DASH) plays a key role in video transmission over the Internet. The conventional DASH adaptation approaches mainly focus on optimizing the overall quality of experience (QoE) for all client sides, neglecting the QoE diversity of different users. In this paper, we propose a meta-learning framework for multi-user preferences (MLMP) as a new DASH adaptation approach, which is able to optimize the diverse QoE of different users. Specifically, we first design a subjective experiment to analyze the difference of QoE preferences across users, in which QoE refers to the metrics of visual quality, fluctuation, and rebuffering events. Based on our findings, we formulate the QoE optimization of multi-user preferences as a multi-task deep reinforcement learning (DRL) problem. In our formulation, the QoE preference of each user is modeled in the overall QoE calculation via assigning the weights to the three QoE metrics. Then, the MLMP framework is developed to solve the proposed multi-task DRL problem, such that the preferences regarding visual quality, fluctuation, and rebuffering events can be optimized for different users in DASH adaptation. Finally, the simulation results show that the proposed approach outperforms state-of-the-art DASH adaptation approaches in satisfying the different users' QoE preferences regarding visual quality, fluctuation, and rebuffering events.