Accurate modeling of temporal point processes is critical for reliable event forecasting and informed decision-making. While historical event sequences provide a foundation for intensity estimation, existing approaches often neglect external covariates whose lagged effects impact future intensities across multiple temporal granularities. To address this gap, we propose Multi-Granularity Integration of External Covariates for Temporal Point Processes (METP), a framework for incorporating lagged external influences into intensity modeling. METP extracts periodic structures and decomposes external covariate series into multiple temporal granularities. At each granularity, a lag-aware calibration module is introduced to align covariates with event dynamics. Finally, a hierarchical mixture-of-experts strategy is employed to integrate the multi-granular external covariates with historical event embeddings, enabling a representation of the conditional intensity function with enhanced information. Extensive experiments on public and proprietary datasets demonstrate that METP consistently outperforms existing methods in predictive accuracy.

Accurate medical time series (MedTS) classification is essential for effective clinical diagnosis, yet remains challenging due to complex multi-channel temporal dependencies, information redundancy, and label scarcity. While transformer-based models have shown promise in time series analysis, most are designed for forecasting tasks and fail to fully exploit the unique characteristics of MedTS. In this paper, we introduce MedSpaformer, a transformer-based framework tailored for MedTS classification. It incorporates a sparse token-based dual-attention mechanism that enables global context modeling and token sparsification, allowing dynamic feature refinement by focusing on informative tokens while reducing redundancy. This mechanism is integrated into a multi-granularity cross-channel encoding scheme to capture intra- and inter-granularity temporal dependencies and inter-channel correlations, enabling progressive refinement of task-relevant patterns in medical signals. The sparsification design allows our model to flexibly accommodate inputs with variable lengths and channel dimensions. We also introduce an adaptive label encoder to extract label semantics and address cross-dataset label space misalignment. Together, these components enhance the model’s transferability across heterogeneous medical datasets, which helps alleviate the challenge of label scarcity. Our model outperforms 13 baselines across 7 medical datasets under supervised learning. It also excels in few-shot learning and demonstrates zero-shot capability in both in-domain and cross-domain diagnostics. These results highlight MedSpaformer's robustness and its potential as a unified solution for MedTS classification across diverse settings.

Data tables play a central role in scientific papers. However, their meaning is often co-constructed with surrounding text through narrative interplay, making comprehension cognitively demanding for readers. In this work, we explore how interfaces can better support this reading process. We conducted a formative study that revealed key characteristics of text-table narrative interplay, including linking mechanisms, multi-granularity alignments, and mention typologies, as well as a layered framework of readers’ intents. Informed by these insights, we present TableTale, an augmented reading interface that enriches text with data tables at multiple granularities, including paragraphs, sentences, and mentions. TableTale automatically constructs a document-level linking schema within the paper and progressively renders cascade visual cues on text and tables that unfold as readers move through the text. A within-subject study with 24 participants showed that TableTale reduced cognitive workload and improved reading efficiency, demonstrating its potential to enhance paper reading and inform future reading interface design.

Industrial fault diagnosis increasingly benefits from Large Models (LMs), which can handle diverse data sources. Within this context, multimodal approaches, which integrate inputs like vibration, process, and video data, boost accuracy but often assume all modalities are available, which rarely holds in practice due to sensor failures, deployment limits, etc. While unimodal methods are more practical, they often lack sufficient information for complex scenarios. The key challenge is how to effectively fuse multimodal data and transfer that knowledge to enhance unimodal performance when some modalities are missing. This paper proposes an adaptive multimodal fault diagnosis framework that enables bidirectional enhancement between multimodal and unimodal representations. For model construction, a Cross-Fusion Channel Attention (CFCA) module is introduced to align features across modalities, and then a Shared Temporal Attention (STA) module captures sequential dependencies and facilitates representation sharing. A multi-head diagnosis structure is finally used to jointly supervise both multimodal and unimodal branches. For parameter estimation, we develop a similarity-aware and uncertainty-guided gradient modulation strategy to adaptively balance multimodal and unimodal learning, ensuring stable optimization and knowledge transfer. Experiments on the real-world industrial PRONTO dataset demonstrate that our method achieves superior performance and adaptability across various scenarios with varying modality availability.

Distributed process monitoring for large-scale processes typically involves two key steps: process decomposition and decision fusion. Decision fusion entails combining multiple local monitoring indices from each process block into a unified statistic, thereby facilitating rapid online fault detection throughout the entire process. Although existing studies have presented useful decision fusion strategies, they often overlook the number of potentially faulty blocks, thus limiting their performance. To address this limitation, this paper introduces the distributed probabilistic PCA (DisPPCA) model, which probabilistically describes the entire process in terms of multiple blocks. Building upon DisPPCA, two efficient decision fusion strategies, “Sum” and “Max”, are proposed. The “Sum” strategy demonstrates greater efficiency in detecting faults when a moderate or large number of faulty blocks are present, whereas the “Max” strategy excels at fault detection when there is only one or a small number of faulty blocks. Founded on the likelihood ratio test, the proposed decision fusion strategies possess a sound theoretical basis. Furthermore, their efficiency is substantiated through numerical simulations and a comprehensive case study.

Despite the recent advances in the application of diffusion models to the realm of image restoration, their inherent stochastic nature can often lead to inaccuracies in reconstructing spatial structures and fine details. This paper introduces an innovative paradigm that harnesses the rich knowledge encapsulated in existing high-level pre-trained models to guide the diffusion process, offering a flexible and potent approach to restoration tasks. However, there are significant challenges in leveraging a pre-trained model for image restoration, including a data gap between natural clean images and degradation images, and paradigm differences that cause insufficient intermediate knowledge. To tackle these issues, we introduce informative Messenger prompts and a Self-adapter for the pre-trained model, whose appropriate information acts as explicit constraints for diffusion, enabling reliable result generation. Specifically, ourMeSa-IRsuccessfully adapts to feature exploration for degraded samples, via disseminating distinctive information from degraded instances. Furthermore, it bolsters knowledge representation through the bidirectional interchange of hierarchical information facilitated by the innovative use of messenger prompts. Experimental results demonstrate the state-of-the-art performance of our framework on five tasks in terms of perceptual and distortion metrics. We will release codes at https://github.com/Ephemeral182/MeSa-IR.

In mechanical fault diagnosis (MFD), intelligent fault diagnosis (IFD) methods perform excellently regarding diagnosis accuracy. However, those methods are generally constructed with an excessive number of unprincipled parameters, resulting in uninterpretable architecture, ambiguity in the diagnosis process, and unclear decision-making basis. Thus, a traceable algorithm unrolling (TAU) network for interpretable MFD is proposed to overcome the above limitations. First, a mechanism-driven feature extractor (FE) is constructed by unrolling the iterative algorithm of sparse coding, aiming at encoding interpretable features from vibration signals. Second, a theory-based feature clustering (FC) algorithm is executed through the dynamic routing mechanism of the capsule network (CN), where the inner product serves as a measure for the association between input and output features. Finally, a post hoc interpretability strategy based on the coupling matrix is introduced to investigate how the TAU generates diagnostic results from the learned features and to verify whether these features are associated with faults, thereby enhancing the credibility of the diagnosis results. In addition, the simulation and experiment are designed to verify the decision-making mechanism and diagnostic performance of TAU. The results demonstrate that TAU makes diagnostic decisions based on the high-dimensional feature mapping associated with fault characteristic frequencies. Meanwhile, TAU outperforms the compared methods in fault diagnosis performance.

Forecasting the power consumption of the spacecraft is critical for optimizing its lifespan and task allocation. However, the complex electromagnetic environment of outer space introduces unavoidable noise into the collected electrical signals. Moreover, the various subsystems of a multipower spacecraft are affected differently by internal and external noise, making it challenging for the existing methods to effectively capture the features of long-term power consumption sequences. We propose adaptive frequency-pruning-enhanced (AFPE)-iTransformer, a robust time-series forecasting model designed for spacecraft telemetry forecasting under noise and long-range dependency conditions. The model combines three key components: Legendre memory projection for historical compression, adaptive top-kfrequency pruning for per-channel denoising, and an improved inverted transformer for cross-subsystem attention. Evaluated on three years of Mars Express (MEX) data, our method consistently outperforms the state-of-the-art baselines in both within-year and cross-year forecasting. It also achieves competitive efficiency, with fast model load time and moderate parameter size. While focused on power forecasting, the model’s modular design supports broader applications in telemetry and industrial forecasting. Model code and configurations are open-sourced for reproducibility.

As social service robots become commonplace, it is essential for them to effectively interpret human signals, such as verbal, gesture, and eye gaze, when people need to focus on their primary tasks to minimize interruptions and distractions. Toward such a socially acceptable Human-Robot Interaction, we conducted a study ($N=24$) in an AR-simulated context of a coffee chat. Participants elicited social cues to signal intentions to an anthropomorphic, zoomorphic, grounded technical, or aerial technical robot waiter when they were speakers or listeners. Our findings reveal common patterns of social cues over intentions, the effects of robot morphology on social cue position and conversational role on social cue complexity, and users' rationale in choosing social cues. We offer insights into understanding social cues concerning perceptions of robots, cognitive load, and social context. Additionally, we discuss design considerations on approaching, social cue recognition, and response strategies for future service robots.

Event prediction plays a pivotal role in analyzing consumer behavior for inventory and pricing optimization. In dynamic financial markets, customer behavior is often influenced by price commitment policies, where the historical and pre-announced future transaction price dynamics can lead to complex behavior patterns, such as advance consumption or delayed purchasing. Therefore, these phenomena pose significant challenges to traditional event modeling approaches that rely solely on consumer behaviors. To address this problem, we propose SCAlign, a cross-domain and multi-scale framework for market dynamics alignment, designed for event prediction. Our model integrates both heterogeneous historical and limited observable future commitment prices at different scales, aligning customer behavior with market fluctuations across multiple time scales. Finally, through a Mixture-of-Experts (MoE) framework, the model dynamically fuses these aligned features, enabling adaptive selection of the relevant and appropriate representations for prediction tasks. Empirical evaluations across diverse transaction environments demonstrate that our model outperforms state-of-the-art prediction baselines. Furthermore, it achieves optimal performance across varying data scales, showcasing its robustness and generalizability.

Real-world image dehazing is crucial for enhancing visual quality in computer vision applications. However, existing physics-based haze generation paradigms struggle to model the complexities of real-world haze and lack controllability, limiting the performance of existing baselines on real-world images. In this paper, we introduce GenHaze, a pioneering haze generation framework that enables the one-step generation of high-quality, reference-controllable hazy images. GenHaze leverages the pre-trained latent diffusion model (LDM) with a carefully designed clean-to-haze generation protocol to produce realistic hazy images. Additionally, by leveraging its fast, controllable generation of paired highquality hazy images, we illustrate that existing dehazing baselines can be unleashed in a simple and efficient manner. Extensive experiments indicate that GenHaze achieves visually convincing and quantitatively superior hazy images. It also significantly improves multiple existing dehazing models across 7 non-reference metrics with minimal fine-tuning epochs. Our work demonstrates that LDM possesses the potential to generate realistic degradations, providing an effective alternative to prior generation pipelines.

Through the collaboration of multiple LLM-empowered agents possessing diverse expertise and tools, multi-agent systems achieve impressive progress in solving real-world problems. Given the user queries, the meta-agents, serving as the brain within multi-agent systems, are required to decompose the queries into multiple sub-tasks that can be allocated to suitable agents capable of solving them, so-called agent-oriented planning. In this study, we identify three critical design principles of agent-oriented planning, including solvability, completeness, and non-redundancy, to ensure that each sub-task can be effectively resolved, resulting in satisfactory responses to user queries. These principles further inspire us to propose AOP, a novel framework for agent-oriented planning in multi-agent systems, leveraging a fast task decomposition and allocation process followed by an effective and efficient evaluation via a reward model. According to the evaluation results, the meta-agent is also responsible for promptly making necessary adjustments to sub-tasks and scheduling. Besides, we integrate a feedback loop into AOP to further enhance the effectiveness and robustness of such a problem-solving process. Extensive experiments demonstrate the advancement of AOP in solving real-world problems compared to both single-agent systems and existing planning strategies for multi-agent systems. The source code is available at https://github.com/lalaliat/Agent-Oriented-Planning

Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.

The recent rapid advancements in language models (LMs) have garnered attention in medical time series-text multimodal learning. However, existing contrastive learning-based and prompt-based LM approaches tend to be biased, often assigning a primary role to time series modality while treating text modality as secondary. We classify these approaches under a temporal-primary paradigm, which may overlook the unique and critical task-relevant information embedded in text modality like clinical reports, thus failing to fully leverage mutual benefits and complementarity of different modalities. To fill this gap, we propose a novel textual-temporal multimodal learning paradigm that enables either modality to serve as the primary while being enhanced by the other, thereby effectively capturing modality-specific information and fostering cross-modal interaction. In specific, we design MedualTime, a language model composed of dual adapters to implement temporal-primary and textual-primary modeling simultaneously. Within each adapter, lightweight adaptation tokens are injected into the top layers of LM to encourage high-level modality fusion. The shared LM pipeline by dual adapters not only achieves adapter alignment but also enables efficient fine-tuning, reducing computational resources. Empirically, MedualTime demonstrates superior performance on medical data, achieving notable improvements of 8% accuracy and 12% F1 in supervised settings. Furthermore, MedualTime's transferability is validated by few-shot transfer experiments from coarse-grained to fine-grained medical data.

Previous methods for image geo-localization have typically treated the task as either classification or retrieval, often relying on black-box decisions that lack interpretability. The rise of large vision-language models (LVLMs) has enabled a rethinking of geo-localization as a reasoning-driven task grounded in visual cues. However, two major challenges persist. On the data side, existing reasoning-focused datasets are primarily based on street-view imagery, offering limited scene diversity and constrained viewpoints. On the modeling side, current approaches predominantly rely on supervised fine-tuning, which yields only marginal improvements in reasoning capabilities. To address these challenges, we propose a novel pipeline that constructs a reasoning-oriented geo-localization dataset, $\textit{MP16-Reason}$, using diverse social media images. We introduce $\textit{GLOBE}$, $\textbf{G}$roup-relative policy optimization for $\textbf{L}$ocalizability assessment and $\textbf{O}$ptimized visual-cue reasoning, yielding $\textbf{B}$i-objective geo-$\textbf{E}$nhancement for the VLM in recognition and reasoning. $\textit{GLOBE}$ incorporates task-specific rewards that jointly enhance localizability assessment, visual-cue reasoning, and geolocation accuracy. Both qualitative and quantitative results demonstrate that $\textit{GLOBE}$ outperforms state-of-the-art open-source LVLMs on geo-localization tasks, particularly in diverse visual scenes, while also generating more insightful and interpretable reasoning trajectories. The data and code are available at https://github.com/lingli1996/GLOBE.

The ability to coordinate turn taking during spoken dialogue is crucial for an embodied spoken dialogue system (SDS), e.g., in a humanoid robot. The SDS needs to model transitions in the conversational floor, which describes each party’s stance (either speaking or listening). Further, the SDS needs to signal its perception of the floor to the human, so that they can coordinate floor transitions and resolve conflicts. Conventional SDS employ standalone modules to control floor transitions but do not produce timely and appropriate responses. Recent end-to-end audio LLMs generate responses quickly, but do not coordinate floor transitions as accurately. In this work, we propose an SDS architecture that dynamically adjusts its prompts to an end-to-end audio LLM based upon its perception of the conversational floor state. The LLM output determines not only the audio output, but also the perceived floor state. This enables the system to signal its stance to the human, both when listening and when speaking. We conducted an experiment where a humanoid robot administered a semi-structured interview with human subjects. Results show that, compared with baseline systems using static prompts, dynamic prompting enables the LLM to model floor transitions more accurately, to generate more appropriate signalling, and to interrupt less, leading to smoother turn-taking in dialogue.

In this work, we discover that causal inference provides a promising approach to capture heterophilic message-passing in Graph Neural Network (GNN). By leveraging cause-effect analysis, we can discern heterophilic edges based on asymmetric node dependency. The learned causal structure offers more accurate relationships among nodes. To reduce the computational complexity, we introduce intervention-based causal inference in graph learning. We first simplify causal analysis on graphs by formulating it as a structural learning model and define the optimization problem within the Bayesian scheme. We then present an analysis of decomposing the optimization target into a consistency penalty and a structure modification based on cause-effect relations. We then estimate this target by conditional entropy and present insights into how conditional entropy quantifies the heterophily. Accordingly, we propose CausalMP, a causal message-passing discovery network for heterophilic graph learning, that iteratively learns the explicit causal structure of input graphs. We conduct extensive experiments in both heterophilic and homophilic graph settings. The result demonstrates that the our model achieves superior link prediction performance. Training on causal structure can also enhance node representation in classification task across different base models.

Recent research on explainable recommendation generally frames the task as a standard text generation problem, and evaluates models simply based on the textual similarity between the predicted and ground-truth explanations. However, this approach fails to consider one crucial aspect of the systems: whether their outputs accurately reflect the users' (post-purchase) sentiments, i.e., whether and why they would like and/or dislike the recommended items. To shed light on this issue, we introduce new datasets and evaluation methods that focus on the users' sentiments. Specifically, we construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews using an LLM, and propose to evaluate systems based on whether the generated explanations 1) align well with the users' sentiments, and 2) accurately identify both positive and negative opinions of users on the target items. We benchmark several recent models on our datasets and demonstrate that achieving strong performance on existing metrics does not ensure that the generated explanations align well with the users' sentiments. Lastly, we find that existing models can provide more sentiment-aware explanations when the users' (predicted) ratings for the target items are directly fed into the models as input. The datasets and benchmark implementation are available at: https://github.com/jchanxtarov/sent_xrec.

Abstract Comprehending visualizations requires readers to interpret visual encoding and the underlying meanings actively. This poses challenges for visualization novices, particularly when interpreting distributional visualizations that depict statistical uncertainty. Advancements in LLM‐based conversational interfaces show promise in promoting visualization comprehension. However, they fail to provide contextual explanations at fine‐grained granularity, and chart readers are still required to mentally bridge visual information and textual explanations during conversations. Our formative study highlights the expectations for both lexical and visual feedback, as well as the importance of explicitly linking these two modalities throughout the conversation. The findings motivate the design of VizTA, a visualization teaching assistant that leverages the fusion of visual and lexical feedback to help readers better comprehend visualization. VizTA features a semantic‐aware conversational agent capable of explaining contextual information within visualizations and employs a visual‐lexical fusion design to facilitate chart‐centered conversation. A between‐subject study with 24 participants demonstrates the effectiveness of VizTA in supporting the understanding and reasoning tasks of distributional visualization across multiple scenarios.

Narrative visualization facilitates data presentation and communicates insights, while virtual reality can further enhance immersive and engaging experiences. The combination of these two research interests shows the potential to revolutionize the way data is presented and understood. Within the realm of narrative visualization, empirical evidence has particularly highlighted the importance of camera planning. However, existing works primarily rely on user-intensive manipulation of the camera, with little effort put into automating the process. To fill the gap, this paper proposes CineFolio , a semi-automated camera planning method to reduce manual effort and enhance user experience in immersive narrative visualization. CineFolio combines cinematic theories with graphics criteria, considering both information delivery and aesthetic enjoyment to ensure a comfortable and engaging experience. Specifically, we parametrize the considerations into optimizable camera properties and solve it as a constraint satisfaction problem (CSP) to realize common camera types for narrative visualization, namely overview camera for absorbing the scale, focus camera for detailed views, moving camera for animated transitions, and user-controlled camera allowing users to provide inputs to camera planning. We demonstrate the feasibility of our approach with cases of various data and chart types. To further evaluate our approach, we conducted a within-subject user study, comparing our automated method with manual camera control, and the results confirm both effectiveness of the guided navigation and expressiveness of the cinematic design for narrative visualization.

Data imputation is a crucial task due to the widespread occurrence of missing data. Many methods adopt a two-step approach: initially crafting a preliminary imputation (the "draft") and then refining it to produce the final missing data imputation result, commonly referred to as "draft-then-refine". In our study, we examine this prevalent strategy through the lens of graph Dirichlet energy. We observe that a basic "draft"imputation tends to decrease the Dirichlet energy. Therefore, a subsequent "refine"step is necessary to restore the overall energy balance. Existing refinement techniques, such as the Graph Convolutional Network (GCN), often result in further energy reduction. To address this, we introduce a new framework, the Graph Laplacian Pyramid Network (GLPN). GLPN incorporates a U-shaped autoencoder and residual networks to capture both global and local details effectively. Through extensive experiments on multiple real-world datasets, GLPN consistently outperforms state-of-the-art methods across three different missing data mechanisms. The code is available at https://github.com/liguanlue/GLPN.

Graph Neural Networks (GNNs) with equivariant properties have emerged as powerful tools for modeling complex dynamics of multi-object physical systems. However, their generalization ability is limited by the inadequate consideration of physical inductive biases: (1) Existing studies overlook the continuity of transitions among system states, opting to employ several discrete transformation layers to learn the direct mapping between two adjacent states; (2) Most models only account for first-order velocity information, despite the fact that many physical systems are governed by second-order motion laws. To incorporate these inductive biases, we propose the Second-order Equivariant Graph Neural Ordinary Differential Equation (SEGNO). Specifically, we show how the second-order continuity can be incorporated into GNNs while maintaining the equivariant property. Furthermore, we offer theoretical insights into SEGNO, highlighting that it can learn a unique trajectory between adjacent states, which is crucial for model generalization. Additionally, we prove that the discrepancy between this learned trajectory of SEGNO and the true trajectory is bounded. Extensive experiments on complex dynamical systems including molecular dynamics and motion capture demonstrate that our model yields a significant improvement over the state-of-the-art baselines.

"Boundary awareness" has become increasingly prominent in the social sciences, highlighting the critical role of personal space in interpersonal behavior. Globalization and digitalization, however, have intensified human interconnectedness, blurring boundaries between individuals and sparking interpersonal conflicts. The delineation of public and private spaces, as well as the understanding of others’ boundaries, is primarily shaped by individuals’ cognitive models. Based on the theoretical framework of cognitive differences, we take as our starting point a research question: "How can installation design be used to raise public awareness of unconscious boundary violations caused by cognitive differences, and inspire reflection on achieving balance in interpersonal interactions?" This study provides insights into the relationship between cognitive differences and unconscious boundary violations through an on-site experiment in specific contexts and with target groups. By utilizing artistic installations as intervention tools, we aim to find ways to help the public break away from self-centered cognitive perspectives and seek balance in social interactions. This approach explores the conflicts and integration of subjectivity and objectivity between individuals, offering new perspectives for analyzing interpersonal conflicts. © 2024 Copyright held by the owner/author(s).

Graph self-supervised learning (SSL) has been vastly employed to learn representations from unlabeled graphs. Existing methods can be roughly divided into predictive learning and contrastive learning, where the latter one attracts more research attention with better empirical performance. We argue that, however, predictive models weaponed with powerful decoder could achieve comparable or even better representation power than contrastive models. In this work, we propose a Wiener Graph Deconvolutional Network (WGDN), an augmentation-adaptive decoder empowered by graph wiener filter to perform information reconstruction. Theoretical analysis proves the superior reconstruction ability of graph wiener filter. Extensive experimental results on various datasets demonstrate the effectiveness of our approach.

Graph neural networks (GNNs) are popular weapons for modeling relational data. Existing GNNs are not specified for attribute-incomplete graphs, making missing attribute imputation a burning issue. Until recently, many works notice that GNNs are coupled with spectral concentration, which means the spectrum obtained by GNNs concentrates on a local part in spectral domain, e.g., low-frequency due to oversmoothing issue. As a consequence, GNNs may be seriously flawed for reconstructing graph attributes as graph spectral concentration tends to cause a low imputation precision. In this work, we present a regularized graph autoencoder for graph attribute imputation, named MEGAE, which aims at mitigating spectral concentration problem by maximizing the graph spectral entropy. Notably, we first present the method for estimating graph spectral entropy without the eigen-decomposition of Laplacian matrix and provide the theoretical upper error bound. A maximum entropy regularization then acts in the latent space, which directly increases the graph spectral entropy. Extensive experiments show that MEGAE outperforms all the other state-of-the-art imputation methods on a variety of benchmark datasets.

Non-Pharmaceutical Interventions (NPIs), such as social gathering restrictions, have shown effectiveness to slow the transmission of COVID-19 by reducing the contact of people. To support policy-makers, multiple studies have first modelled human mobility via macro indicators (e.g., average daily travel distance) and then study the effectiveness of NPIs. In this work, we focus on mobility modelling and, from a micro perspective, aim to predict locations that will be visited by COVID-19 cases. Since NPIs generally cause economic and societal loss, such a prediction benefits governments when they design and evaluate them. However, in real-world situations, strict privacy data protection regulations result in severe data sparsity problems (i.e., limited case and location information). To address these challenges and jointly model variables including a geometric graph, a set of diffusions and a set of locations, we propose a model named Deep Graph Diffusion Infomax (DGDI). We show the maximization of DGDI can be bounded by two tractable components: a univariate Mutual Information (MI) between geometric graph and diffusion representation, and a univariate MI between diffusion representation and location representation. To facilitate the research of COVID-19 prediction, we present two benchmarks that contain geometric graphs and location histories of COVID-19 cases. Extensive experiments on the two benchmarks show that DGDI significantly outperforms other competing methods.

In this work, we focus on robust time series representation learning. Our assumption is that real-world time series is noisy and complementary information from different views of the same time series plays an important role while analyzing noisy input. Based on this, we create two views for the input time series through two different encoders. We conduct co-training based contrastive learning iteratively to learn the encoders. Our experiments demonstrate that this co-training approach leads to a significant improvement in performance. Especially, by leveraging the complementary information from different views, our proposed TS-CoT method can mitigate the impact of data noise and corruption. Empirical evaluations on four time series benchmarks in unsupervised and semi-supervised settings reveal that TS-CoT outperforms existing methods. Furthermore, the representations learned by TS-CoT can transfer well to downstream tasks through fine-tuning1.

Graph alignment, which aims at identifying corresponding entities across multiple networks, has been widely applied in various domains. As the graphs to be aligned are usually constructed from different sources, the inconsistency issues of structures and features between two graphs are ubiquitous in real-world applications. Most existing methods follow the "embed-then-cross-compare" paradigm which computes node embeddings in each graph and then processes node correspondences based on cross-graph embedding comparison. However, we find these methods are unstable and sub-optimal when structure or feature inconsistency appears. To this end, we propose SLOTAlign, an unsupervised graph alignment framework that jointly performs Structure Learning and Optimal Transport Alignment. We convert graph alignment to an optimal transport problem between two intra-graph matrices without the requirement of cross-graph comparison. We further incorporate multi-view structure learning to enhance graph representation power and reduce the effect of structure and feature inconsistency inherited across graphs. Moreover, an alternating scheme based algorithm has been developed to address the joint optimization problem in SLOTAlign and the provable convergence result are also established. Finally, we conduct extensive experiments on six unsupervised graph alignment datasets and the DBP15K knowledge graph (KG) alignment benchmark dataset. The proposed SLOTAlign shows superior performance and strongest robustness over seven unsupervised graph alignment methods and five specialized KG alignment methods.1

This paper proposes to learn Multi-task, Multi-modal Direct Acyclic Graphs (MM-DAGs), which are commonly observed in complex systems, e.g., traffic, manufacturing, and weather systems, whose variables are multi-modal with scalars, vectors, and functions. This paper takes the traffic congestion analysis as a concrete case, where a traffic intersection is usually regarded as a DAG. In a road network of multiple intersections, different intersections can only have someoverlapping and distinct variables observed. For example, a signalized intersection has traffic light-related variables, whereas unsignalized ones do not. This encourages the multi-task design: with each DAG as a task, the MM-DAG tries to learn the multiple DAGs jointly so that their consensus and consistency are maximized. To this end, we innovatively propose a multi-modal regression for linear causal relationship description of different variables. Then we develop a novel Causality Difference (CD) measure and its differentiable approximator. Compared with existing SOTA measures, CD can penalize the causal structural difference among DAGs with distinct nodes and can better consider the uncertainty of causal orders. We rigidly prove our design's topological interpretation and consistency properties. We conduct thorough simulations and one case study to show the effectiveness of our MM-DAG. The code is available under https://github.com/Lantian72/MM-DAG.

Pseudo labeling (PL) is a wide-applied strategy to enlarge the labeled dataset by self-annotating the potential samples during the training process. Several works have shown that it can improve the graph learning model performance in general. However, we notice that the incorrect labels can be fatal to the graph training process. Inappropriate PL may result in the performance degrading, especially on graph data where the noise can propagate. Surprisingly, the corresponding error is seldom theoretically analyzed in the literature. In this paper, we aim to give deep insights of PL on graph learning models. We first present the error analysis of PL strategy by showing that the error is bounded by the confidence of PL threshold and consistency of multi-view prediction. Then, we theoretically illustrate the effect of PL on convergence property. Based on the analysis, we propose a cautious pseudo labeling methodology in which we pseudo label the samples with highest confidence and multi-view consistency. Finally, extensive experiments demonstrate that the proposed strategy improves graph learning process and outperforms other PL strategies on link prediction and node classification tasks.

System monitoring and anomaly detection is a crucial task in daily operation. With the rapid development of cyber-physical systems and IT systems, multiple sensors get involved to represent the system state from different perspectives, which inspires us to detect anomalies considering feature dependence relationship among sensors instead of focusing on individual sensor's behavior. In this paper, we propose a novel Graph Relational Learning Network (GReLeN) to detect multivariate time series anomalies from the perspective of between-sensor dependence relationship learning. Variational AutoEncoder (VAE) serves as the overall framework for feature extraction and system representation. Graph Neural Network (GNN) and stochastic graph relational learning strategy are also imposed to capture the between-sensor dependence. Then a composite anomaly metric is established with the learned dependence structure explicitly. The experiments on four real-world datasets show our superiority in detection accuracy, anomaly diagnosis, and model interpretation.

Abstract Individual passenger travel patterns have significant value in understanding passenger’s behavior, such as learning the hidden clusters of locations, time, and passengers. The learned clusters further enable commercially beneficial actions such as customized services, promotions, data-driven urban-use planning, peak hour discovery, and so on. However, the individualized passenger modeling is very challenging for the following reasons: 1) The individual passenger travel data are multi-dimensional spatiotemporal big data, including at least the origin, destination, and time dimensions; 2) Moreover, individualized passenger travel patterns usually depend on the external environment, such as the distances and functions of locations, which are ignored in most current works. This work proposes a multi-clustering model to learn the latent clusters along the multiple dimensions of Origin, Destination, Time, and eventually, Passenger (ODT-P). We develop a graph-regularized tensor Latent Dirichlet Allocation (LDA) model by first extending the traditional LDA model into a tensor version and then applies to individual travel data. Then, the external information of stations is formulated as semantic graphs and incorporated as the Laplacian regularizations; Furthermore, to improve the model scalability when dealing with massive data, an online stochastic learning method based on tensorized variational Expectation-Maximization algorithm is developed. Finally, a case study based on passengers in the Hong Kong metro system is conducted and demonstrates that a better clustering performance is achieved compared to state-of-the-arts with the improvement in point-wise mutual information index and algorithm convergence speed by a factor of two.

Many methods for monitoring multivariate processes are built on principal component analysis (PCA), which, however, simply tells whether the process is faulty or not. In fact, there is still room for the improvement of the early detection performance by exploiting fully the information given by fault directions. To this end, this article proposes a novel directional PCA (diPCA) approach. First, by narrowing down faults to a specified direction or composite mutually orthogonal directions, diPCA can speed fault detection and facilitate accurate fault diagnosis. It also has good theoretical properties that guarantee concise control limits. Second, with appropriate fault directions, diPCA provides a unified framework for process monitoring and includes existing monitoring indices, such as Hotelling’s$T^{2}$and the squared prediction error (SPE), as special cases. Third, diPCA also naturally results in a new combined monitoring statistic, which is composed of both$T^{2}$and SPE, and provides an optimal ratio of their combination. The Monte Carlo simulation results have demonstrated the power of the proposed monitoring and diagnostic methods stemming from diPCA. The proposed methods have also been implemented into the Tennessee Eastman process.

Modern metro systems in big cities have accumulated large amounts of passenger transit transaction data via the deployment of automatic fare collection (AFC) devices, which could facilitate a thorough analysis of passenger flow dynamics. To discover the underlying passenger flow patterns over all stations in an entire metro system, this paper proposes a multivariate functional principal component analysis (MFPCA) method. The functional integrity of a single daily passenger flow profile at each station is explicitly utilized, and the complex correlation among a multitude of daily passenger flow profiles from all stations is comprehensively modeled. Moreover, to simultaneously improve the interpretability of eigenvectors and mitigate the sensitivity to outliers, the MFPCA is formulated as a minimization problem with both a sparsity and a robustness penalty term. A computationally efficient algorithm is developed accordingly to obtain the eigenvectors. The superiority of our proposed sparse and robust MFPCA (SRMFPCA) is validated using a Hong Kong Mass Transit Railway (MTR) dataset. The derived sparse and smooth eigenvectors can be well interpreted as empirically meaningful passenger flow patterns. The results of our method can be further used as solid foundations for station clustering, correlation analysis and outlier identification.

Anomaly detection is an essential task for quality management in smart manufacturing. An accurate data-driven detection method usually needs enough data and labels. However, in practice, there commonly exist newly set-up processes in manufacturing, and they only have quite limited data available for analysis. Borrowing the name from the recommender system, we call this process a cold-start process. The sparsity of anomaly, the deviation of the profile, and noise aggravate the detection difficulty. Transfer learning could help to detect anomalies for cold-start processes by transferring the knowledge from more experienced processes to the new processes. However, the existing transfer learning and multi-task learning frameworks are established on task- or domain-level relatedness. We observe instead, within a domain, some components (background and anomaly) share more commonality, others (profile deviation and noise) not. To this end, we propose a more delicate component-level transfer learning scheme, i.e., decomposition-based hybrid transfer learning ( DHTL ): It first decomposes a domain (e.g., a data source containing profiles) into different components (smooth background, profile deviation, anomaly, and noise); then, each component’s transferability is analyzed by expert knowledge; Lastly, different transfer learning techniques could be tailored accordingly. We adopted the Bayesian probabilistic hierarchical model to formulate parameter transfer for the background, and “ L 2,1 + L 1 ”-norm to formulate low dimension feature-representation transfer for the anomaly. An efficient algorithm based on Block Coordinate Descend is proposed to learn the parameters. A case study based on glass coating pressure profiles demonstrates the improved accuracy and completeness of detected anomaly, and a simulation demonstrates the fidelity of the decomposition results.

Modern manufacturing industries are often featured with a data-rich environment. The real-time behaviors of process variables can be completely recorded as multiple various signal signatures, and the geometric quality of finished products can be thoroughly characterized by their 2-D surface data. Learning the relationship between such signal predictors and surface responses, where the input and output are no longer the conventional scalar variables but are in fact both functions in the time domain and spatial domain, respectively, is critical for quality prediction in many applications nowadays. To this end, this article proposes a novel sparse and structured function-on-function regression (SSF2R) model, where a hierarchical variable selection is developed to identify informative signals and further screen significant elements within the selected signals, and a multitask learning is devised to exploit the smoothness nature of surface response and the similarity structure among a series of subregression tasks. Our SSF2R model is concisely formulated as a convex problem with an efficient iterative algorithm derived to obtain the global optimum. Moreover, our quality prediction can be performed dynamically during an ongoing manufacturing process when only partial observations of the signal predictors are available. The superiority of our proposed method is validated by numerical simulations and a real case study in the semiconductor industry.

Low-rank tensor decomposition and completion have attracted significant interest from academia given the ubiquity of tensor data. However, low-rank structure is a global property, which will not be fulfilled when the data presents complex and weak dependencies given specific graph structures. One particular application that motivates this study is the spatiotemporal data analysis. As shown in the preliminary study, weakly dependencies can worsen the low-rank tensor completion performance. In this paper, we propose a novel low-rank CANDECOMP / PARAFAC (CP) tensor decomposition and completion framework by introducing the L1-norm penalty and Graph Laplacian penalty to model the weakly dependency on graph. We further propose an efficient optimization algorithm based on the Block Coordinate Descent for efficient estimation. A case study based on the metro passenger flow data in Hong Kong is conducted to demonstrate an improved performance over the regular tensor completion methods.

Smart customization service is an important element for smart manufacturing. The success of smart customization requires that designers, manufacturers, and customers with differences in context, semantics, and other cognitive aspects be engaged in a collaborative process. With product configurators reported to have positive impacts on product quality to meet customers' needs, this article attempts to explore an approach for smart customization service based on configurators. To better address the semantic gap between customers and designers/manufacturers, a new configuration mechanism is proposed that takes into consideration customer needs using natural language as the input and maps them to product specifications in the design stage. We collected a massive amount of review text from e-commerce websites and used ELMo, a contextualized word representation based on a deep bidirectional language model, to encode the text. A multitask learning-based neural network was adopted to build the mapping from layman customer needs to product specifications. Our experiments show that this approach can achieve a promising performance for the configuration task and, thereby, facilitate smart customization services. Note to Practitioners-Smart customization has been adopted by various industries to tailor companies' business streams and customize solutions for customers. It is a complicated service process involving cross-functional teams for identifying customer needs and establishing product design and manufacturing specifications. However, communication in the collaborative process may be challenging. Customers may express their needs using layman's terms. The expressed needs can even be ambiguous and imprecise. Miscommunication hinders the efficiency of smart customization services. This article uses natural language processing and machine learning techniques to map product review text, which is crawled from e-commerce websites to the corresponding product specifications. Given a new customer's needs in free-form text, the mapping can automatically identify the satisfactory product configurations. This has the potential to improve the efficiency of product customization and shield customers and companies from the back-and-forth communication procedure in the customization service.

This article proposes a novel time-warped sparse non-negative factorization method for functional data analysis. The proposed method on the one hand guarantees the extracted basis functions and their coefficients to be positive and interpretable, and on the other hand is able to handle weakly correlated functions with different features. Furthermore, the method incorporates time warping into factorization and hence allows the extracted basis functions of different samples to have temporal deformations. An efficient framework of estimation algorithms is proposed based on a greedy variable selection approach. Numerical studies together with case studies on real-world data demonstrate the efficacy and applicability of the proposed methodology.

The advance of new sensing technologies, such as the 3-D laser scanning, creates a data-rich environment for quality control in modern industries. The free-form surfaces of complex manufactured parts can be quickly scanned, producing thousands of data points. To monitor these large-scale surface scanning data, three major challenges have to be solved simultaneously: 1) simple parametric models are no longer sufficient to describe free-form surfaces; 2) the massive data points need fast computations; and 3) the presence of outliers calls for robust analytics. To fulfill this task, this paper proposes a novel monitoring scheme where the control chart is designed based on a new robust bilateral kernel smoothing method. A fast approximation algorithm is also developed for efficient online monitoring. This fast and robust nonparametric control chart shows significant superiority for surface monitoring in our numerical simulations. Finally, a real case study demonstrates the effectiveness of our proposed scheme in monitoring the stability of a 3-D printing process.

Additive manufacturing (AM) or 3-D printing refers to a new class of technologies that actively construct products directly from any 3-D digital model. In the future, the broader applications of AM will require a cost reduction of AM machines. Currently, the products fabricated by low-end machines, such as those fabricated using fused deposition modeling (FDM) processes, suffer from the issue of low dimensional accuracy due to multiple error sources. To properly manage error sources for improved prevision, this paper proposes a novel strategy for error compensation in the FDM processes. First, we consecutively attribute the dimensional inaccuracy to two major error sources that affect the geometric shape of the product: 1) positioning error of the extruder and 2) shape deformation induced by processing error, including material phase change and other variations that occur. The extruder positioning error is characterized by a Kriging model, while the modeling of shape deformation due to processing error follows the method developed by Huang et al. Second, using error equivalence concept, we transform the positioning error into the equivalent amount of design input error. Finally, we adjust the design to compensate for the overall shape deviation. To validate this strategy, we conduct a designed experiment for the shape deviation prediction and the compensation. The experimental results successfully demonstrate the effectiveness of the proposed three-step strategy to manage multiple error sources in the FDM processes.

In recent years, there has been an increasing demand for quality control of multivariate dynamic systems. Conventional directionally invariant charts fail to make use of versatile shift patterns in a multivariate process and are sensitive to general failures only. Directionally variant charts, however, are designed for specific and constant shifts and are not suitable for processes with dynamic and unknown failures. This paper proposes an adaptive T2scheme, which can successfully capture the unknown shift patterns of a multivariate system via an exponentially weighted moving average (EWMA) forecasting procedure. The adaptive scheme preserves the optimality of a directionally variant chart, while provides a scalable extension to Hotelling's T2procedure. The smoothing parameter of the proposed scheme can be tuned for desired shift sizes. Significant improvement of sensitivity over an intended range is demonstrated by Monte Carlo simulation.

With advances in information technology (IT), the research on and practice of information sharing is now having a significant impact on many aspects of supply chains. Nevertheless, few investigations focus on the impact of information sharing on product and process quality. Furthermore, it is still not clear how and what information should be shared or used, and how to quantify the benefits of information sharing in terms of quality improvement. In this research, a "matching problem" is used to demonstrate the impact of information sharing on quality. We quantify and compare the impact of different information-sharing strategies on process and product quality, and suggest that real-time information sharing may lead to dramatic quality improvement for an assembly process, the example here being a two-stage supply chain. The proposed approach to evaluate information sharing in terms of quality improvement can be extended to a more complex supply chain.