Tandem mass spectrometry (MS/MS) is a critical tool for identifying molecular structures. By efficiently separating molecular fragments based on their mass-to-charge (m/z) ratios, it facilitates molecular generation and subsequent scientific discoveries. However, de novo molecular generation from MS/MS spectra remains fundamentally constrained by two paramount challenges: the vast chemical space requires effective structural constraints, and the absence of fine-grained substructural generation weakens the correspondences between spectral features and molecular structures. In this work, we propose MSAnchor, a novel two-stage framework for MS/MS-based molecular structure generation. We mitigate the search space challenge through the introduction of Anchor-Extended Molecular Scaffold (AEMS) representation that explicitly encodes side-chain anchoring points, thereby dramatically reducing combinatorial complexity. Leveraging the explicit attachment sites provided by AEMS, we develop anchor-specific priors that establish effective alignments between spectral features and molecular substructures. This fine-grained substructural correspondence is further enhanced by a modified Conditional Information Bottleneck (CIB) module that extracts the most informative spectral components in a structure-aware manner. These innovations enable MSAnchor to generate molecular structures that closely reflect spectral characteristics while constraining combinatorial complexity. Extensive experiments on the CANOPUS and MassSpecGym datasets demonstrate that MSAnchor achieves state-of-the-art performance in molecular structure prediction from MS/MS spectra, with performance improvements that are particularly more pronounced for molecules with higher complexity.

Graph classification is a critical task in analyzing graph data, with applications across various domains. While graph neural networks (GNNs) have achieved remarkable results, their ability to generalize across graphs of varying scales remains a challenge. Conventional models often perform well on large-scale graphs but struggle with distributions that are skewed towards small scales. Conversely, models tailored to address scale imbalances frequently prioritize small-scale graphs, leading to diminished performance in more balanced scenarios. To overcome these limitations, we introduce a Unbalanced-Balanced Representation Converter (U2B), which exhibits no explicit bias toward graph scales. U2B employs a two-step workflow: a distillation phase to extract base features from both node-level and graph-level representations, followed by a refinement phase to generate unbiased representations for improved balance. In the distillation phase, a static constraint guides node-level adjustments, improving the representation of nodes in small graphs. Simultaneously, a dynamic constraint in the graph-level process mitigates biases toward features from large graphs. To ensure harmony between the representations, a consistency alignment loss is introduced, aligning node-level and graph-level features to create more cohesive and balanced graph representations. Extensive experiments on multiple datasets show that U2B achieves competitive performance.

Zhiqing Cui, Binwu Wang, Qingxiang Liu, Yeqiang Wang, Zhengyang Zhou, Yuxuan Liang, Yang Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Bohan Lin, Kuo Yang, Zelin Tan, Yingchuan Lai, Chen Zhang, Guibin Zhang, Xinlei Yu, Miao Yu, Xu Wang, Yudong Zhang, Yang Wang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Accurate forecasting of sequential data streams is a cornerstone of modern Web services, supporting applications such as traffic management, user behavior modeling, and online anomaly prevention. However, in many Web environments, new domains emerge rapidly and labeled history data is scarce, which makes zero-shot forecasting particularly challenging. Existing time-series pre-trained models (TSPMs) show promise but they lack the ability to dynamically incorporate external knowledge, while conventional retrieval-augmented generation (RAG) methods are rarely extended beyond text. In this work, we present QuiZSF, a retrieval-augmented forecasting framework that integrates search and forecasting for time series data. The framework performs search by retrieving structurally similar sequences from a large-scale time-series database, and it performs forecasting by integrating the retrieved knowledge into the target sequence. Specifically, QuiZSF introduces a ChronoRAG Base, a hierarchical tree-structured database that enables scalable and domain-aware retrieval, a Multi-grained Series Interaction Learner that captures fine- and coarse-grained dependencies between target and retrieved sequences, and a Model Cooperation Coherer that adapts retrieved knowledge to TSPMs. This design teaches models to actively perform search, align auxiliary information across modalities, and leverage it for more accurate forecasting. Extensive experiments on five public benchmarks demonstrate that QuiZSF consistently outperforms strong baselines, ranking first in up to 87.5% of zero-shot forecasting settings while maintaining high efficiency.

Spatio-temporal (ST) prediction is crucial in earth sciences, including meteorological forecasting and urban computing, to name just a few. Access to ample high-quality data, combined with deep models adept at inference, is essential for attaining significant outcomes. Yet, data scarcity and the substantial costs of sensor deployment result in notable data imbalances. Overly specialized models that lack causal linkages further undermine the generalizability of inference techniques. To address these challenges, we first introduce a causal framework for ST predictions, named $\mathtt{NuwaDynamics}$NuwaDynamicsNuwaDynamics, aimed at pinpointing causal regions in data and providing models with the capability for causal reasoning in a dual-phase process. Initially, we employ upstream self-supervision to identify causally significant patches, equipping the model with generalizable insights and performing targeted interventions on non-essential patches to approximate potential testing distributions. This stage is known as the discovery phase. Progressing from discovery, we apply the insights to downstream tasks tailored to specific ST goals, enhancing the model's recognition of a wider potential data distribution and augmenting its causal perceptual abilities (referred to as the Update phase). Additionally, we address environmental controllability and high computational complexity by implementing channel multiplication and conditional generation methods. This process, termed $\mathtt{NuwaDynamics+}$NuwaDynamics+NuwaDynamics+, can further be interpreted as the front-door adjustment technique in the causality domain. Through comprehensive experiments across ten real-world or simulated ST benchmarks, we demonstrate that integrating the $\mathtt{NuwaDynamics+}$NuwaDynamics+NuwaDynamics+ concept substantially improves various model performance. $\mathtt{NuwaDynamics+}$NuwaDynamics+NuwaDynamics+ concept also significantly enhances the versatility across various dynamic ST tasks, such as extreme weather forecasting and long-temporal-step super-resolution predictions.

Abstract Evaluating natural language generation systems is challenging due to the diversity of valid outputs. While human evaluation is the gold standard, it suffers from inconsistencies, lack of standardization, and demographic biases, limiting reproducibility. LLM-based evaluators offer a scalable alternative but are highly sensitive to prompt design, where small variations can lead to significant discrepancies. In this work, we propose an inversion learning method that learns effective reverse mappings from model outputs back to their input instructions, enabling the automatic generation of highly effective, model-specific evaluation prompts. Our method requires only a single evaluation sample and eliminates the need for time-consuming manual prompt engineering, thereby improving both efficiency and robustness. Our work contributes toward a new direction for more robust and efficient LLM-based evaluation.

Spatiotemporal forecasting (STF) is pivotal in urban computing, yet data scarcity in developing cities hampers robust model training. Addressing this, recent studies leverage transfer learning to migrate knowledge from data-rich (source) to data-poor (target) cities. This strategy, while effective, faces challenges as pre-trained models risk absorbing noise and harmful information due to data distribution disparities, potentially undermining the accuracy of forecasts for target cities. To address this issue, we propose a one-stage STF framework named Target-Skewed Joint Training (TSJT). Central to TSJT is a novel Target-Skewed Backward training strategy that selectively refines gradients from source city data, preserving only the elements that positively impact the target city. To further enhance the quality of these gradients, we have designed a Node Prompting Module (NPM). TSJT is crafted for seamless integration with existing STF models, endowing them with the capability to efficiently tackle challenges stemming from data scarcity. Experimental results on several real-world datasets from multiple cities substantiate the efficacy of TSJT in the realm of cross-city transfer learning.

Rainy images suffer from quality degradation due to the synergistic effect of rain streaks and accumulation. The rain streaks are anisotropic and show a specific directional arrangement, while the rain accumulation is isotropic and shows a consistent concentration distribution in local regions. This distribution difference makes unified representation learning for rain streaks and accumulation challenging, which may lead to structure distortion and contrast degradation in the deraining results. To address this problem, a central-surrounding mechanism inspired Synergistic Convolution (SC) is proposed to extract rain streaks and accumulation features simultaneously. Specifically, the SC consists of two parallel novel convolutions: Central-Surrounding Difference Convolution (CSD) and Central-Surrounding Addition Convolution (CSA). In CSD, the difference operation between central and surrounding pixels is injected into the feature extraction process of convolution to perceive the direction distribution of rain streaks. In CSA, the addition operation between central and surrounding pixels is injected into the feature extraction process of convolution to facilitate the modeling of rain accumulation properties. The SC can be used as a general unit to substitute Vanilla Convolution (VC) in current de-raining networks to boost performance. To reduce computational costs, CSA and CSD in SC are merged into a single VC kernel by our parameter equivalent transformation before inferencing. Evaluations of twelve de-raining methods on nine public datasets demonstrate that our proposed SC can comprehensively improve the performance of twelve de-raining networks under various rainy conditions without changing the original network structure or introducing extra computational costs. Even for the current SOTA methods, SC can further achieve SOTA++ performance. The source codes will be publicly available.

Self-Knowledge Distillation (SKD) leverages the student's own knowledge to create a virtual teacher for distillation when the pre-trained bulky teacher is not available. Whilst existing SKD approaches demonstrate gorgeous efficiency in single-label learning, to directly apply them to multi-label learning would suffer from dramatic degradation due to the following inherent imbalance: \textit{targets with unified labels but multifarious visual scales are crammed into one image, resulting in biased learning of major targets and disequilibrium of precision-recall}. To address this issue, this paper proposes a novel SKD method for multi-label learning named Multi-label Self-knowledge Distillation (MSKD), incorporating three Spatial Decoupling mechanisms (i.e. Locality-SD (L-SD), Reconstruction-SD (R-SD), and Step-SD (S-SD)). L-SD exploits relational dark knowledge from regional outputs to amplify the model's perception of visual details. R-SD reconstructs global semantics by integrating regional outputs from local patches and leverages it to guide the model. S-SD aligns outputs of the same input at different steps, aiming to find a synthetical optimizing direction and avoid the overconfidence. In addition, MSKD combines our tailored loss named MBD for balanced distillation. Exhaustive experiments demonstrate that MSKD not only outperforms previous approaches but also effectively mitigates biased learning and equips the model with more robustness.

Transformer-based methods have achieved remarkable performance in event-based object detection, owing to the global modeling ability. However, they neglect the influence of non-event and noisy regions and process them uniformly, leading to high computational overhead. To mitigate computation cost, some researchers propose window attention based sparsification strategies to discard unimportant regions, which sacrifices the global modeling ability and results in suboptimal performance. To achieve better trade-off between accuracy and efficiency, we propose Sparse Mamba (SMamba), which performs adaptive sparsification to reduce computational effort while maintaining global modeling capability. Specifically, a Spatio-Temporal Continuity Assessment module is proposed to measure the information content of tokens and discard uninformative ones by leveraging the spatiotemporal distribution differences between activity and noise events. Based on the assessment results, an Information-Prioritized Local Scan strategy is designed to shorten the scan distance between high-information tokens, facilitating interactions among them in the spatial dimension. Furthermore, to extend the global interaction from 2D space to 3D representations, a Global Channel Interaction module is proposed to aggregate channel information from a global spatial perspective. Results on three datasets (Gen1, 1Mpx, and eTram) demonstrate that our model outperforms other methods in both performance and efficiency.

Time series forecasting plays a crucial role in domains such as finance, healthcare, and climate science. However, as modern time series data become increasingly complex, featuring high dimensionality, intricate spatiotemporal dependencies, and multi-scale evolutionary patterns, traditional analytical methods and existing predictive models face significant challenges. Although Large Language Models (LLMs) excel in capturing long-range dependencies, they still struggle with multi-scale dynamics and seasonal patterns. Moreover, while LLMs' semantic representation capabilities are rich, they often lack explicit alignment with the numerical patterns and temporal structures of time series data, leading to limitations in predictive accuracy and interpretability. To address these challenges, this paper proposes a novel framework, STEM-LTS (Semantic-TEmporal Modeling for Large-scale Time Series). STEM-LTS enhances the ability to capture complex spatiotemporal dependencies by integrating time series decomposition techniques with LLM-based modeling. The semantic-temporal alignment mechanism within the framework significantly improves LLMs' ability to interpret and forecast time series data. Additionally, we develop an adaptive multi-task learning strategy to optimize the model's performance across multiple dimensions. Through extensive experiments on various real-world datasets, we demonstrate that STEM-LTS achieves significant improvements in prediction accuracy, robustness to noise, and interpretability. Our work not only advances LLM-based time series analysis but also offers new perspectives on handling complex temporal data.

Large Language Model (LLM)-based Multi-agent Systems (MAS) have demonstrated remarkable capabilities in various complex tasks, ranging from collaborative problem-solving to autonomous decision-making. However, as these systems become increasingly integrated into critical applications, their vulnerability to adversarial attacks, misinformation propagation, and unintended behaviors have raised significant concerns. To address this challenge, we introduce G-Safeguard, a topology-guided security lens and treatment for robust LLM-MAS, which leverages graph neural networks to detect anomalies on the multi-agent utterance graph and employ topological intervention for attack remediation. Extensive experiments demonstrate that G-Safeguard: (I) exhibits significant effectiveness under various attack strategies, recovering over 40% of the performance for prompt injection; (II) is highly adaptable to diverse LLM backbones and large-scale MAS; (III) can seamlessly combine with mainstream MAS with security guarantees.

This paper explores the use of Large Language Models (LLMs) in the generation and evaluation of analytical reports derived from Earnings Calls (ECs). Addressing a current gap in research, we explore the generation of analytical reports with LLMs in a multi-agent framework, designing specialized agents that introduce diverse viewpoints and desirable topics of analysis into the report generation process. Through multiple analyses, we examine the alignment between generated and human-written reports and the impact of both individual and collective agents. Our findings suggest that the introduction of additional agents results in more insightful reports, although reports generated by human experts remain preferred in the majority of cases. Finally, we address the challenging issue of report evaluation, we examine the limitations and strengths of LLMs in assessing the quality of generated reports in different settings, revealing a significant correlation with human experts across multiple dimensions.

Burst super-resolution (BurstSR) aims to reconstruct high-resolution images by fusing subpixel details from multiple low-resolution burst frames. The primary challenge lies in effectively extracting useful information while mitigating the impact of high-frequency noise. Most existing methods rely on frame-by-frame fusion, which often struggles to distinguish informative subpixels from noise, leading to suboptimal performance. To address these limitations, we introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network. Specifically, we observe that sub-pixels have consistent spatial distribution while noise appears randomly. Considering the entire burst sequence during fusion allows for more reliable extraction of consistent subpixels and better suppression of noise outliers. Based on this, a Query State Space Model (QSSM) is proposed for both inter-frame querying and intra-frame scanning, enabling a more efficient fusion of useful subpixels. Additionally, to overcome the limitations of static upsampling methods that often result in over-smoothing, we propose an Adaptive Upsampling (AdaUp) module that dynamically adjusts the upsampling kernel to suit the characteristics of different burst scenes, achieving superior detail reconstruction. Extensive experiments on four benchmark datasets—spanning both synthetic and real-world images—demonstrate that QMambaBSR outperforms existing state-of-the-art methods.

We introduce Drivelology, a unique linguistic phenomenon characterised as "nonsense with depth" -utterances that are syntactically coherent yet pragmatically paradoxical, emotionally loaded, or rhetorically subversive.While such expressions may resemble surface-level nonsense, they encode implicit meaning requiring contextual inference, moral reasoning, or emotional interpretation.We find that current large language models (LLMs), despite excelling at many natural language processing (NLP) tasks, consistently fail to grasp the layered semantics of Drivelological text.To investigate this, we construct a benchmark dataset of over 1,200+ meticulously curated and diverse examples across English, Mandarin, Spanish, French, Japanese, and Korean.Each example underwent careful expert review to verify its Drivelological characteristics, involving multiple rounds of discussion and adjudication to address disagreements.Using this dataset, we evaluate a range of LLMs on classification, generation, and reasoning tasks.Our results reveal clear limitations of LLMs: models often confuse Drivelology with shallow nonsense, produce incoherent justifications, or miss implied rhetorical functions altogether.These findings highlight a deep representational gap in LLMs' pragmatic understanding and challenge the assumption that statistical fluency implies cognitive comprehension.We release our dataset 1 and code 2 to facilitate further research in modelling linguistic depth beyond surface-level coherence.

Due to the profound impact of air pollution on human health, livelihoods, and economic development, air quality forecasting is of paramount significance. Initially, we employ the causal graph method to scrutinize the constraints of existing research in comprehensively modeling the causal relationships between the air quality index (AQI) and meteorological features. In order to enhance prediction accuracy, we introduce a novel air quality forecasting model, AirCade, which incorporates a causal decoupling approach. AirCade leverages a spatiotemporal module in conjunction with knowledge embedding techniques to capture the internal dynamics of AQI. Subsequently, a causal decoupling module is proposed to disentangle synchronous causality from past AQI and meteorological features, followed by the dissemination of acquired knowledge to future time steps to enhance performance. Additionally, we introduce a causal intervention mechanism to explicitly represent the uncertainty of future meteorological features, thereby bolstering the model’s robustness. Our evaluation of AirCade on an open-source air quality dataset demonstrates over 20% relative improvement over state-of-the-art models. Our source code is available at https://github.com/PoorOtterBob/AirCade.

Typical spatiotemporal graph forecasting methods process graph-structured spatiotemporal data respectively from spatial and temporal perspectives with the idea of divide and conquer. Existing works are incapable of capturing long-term transdimensional correlations among different spatial points in different time planes, i.e., time-space-interlaced correlations. To tackle this issue, we propose a two-stage summarized attention network to establish transdimensional direct message passing routes between different data points in different time planes and spaces, thus enabling the extraction of time-space-interlaced long-term correlations. Specifically, a novel spatiotemporal embedding is proposed to implement time-space-interlaced learning by expanding orthogonal spatial and temporal dimensionalities into one-dimensionality, a series of temporal context fusion units are added to address the fluctuation dislocation insensitivity issue which is caused by time-space-dimension expansion, and an ingenious two-stage design can significantly reduce the computation complexity of such time-space-interlaced learning. Extensive experiments illustrate the superior performance of our proposed approach on real-world spatiotemporal datasets.

Spatiotemporal forecasting facilitates many real world intelligent systems. Combining graph learning with temporal models has recently become popular in spatiotemporal forecasting. Although graph convolution enhances the modeling of spatial correlations, it results in unsatisfactory efficiency and poor extensibility in existing models. Consequently, existing graph-learning-based methods struggle to handle emerging nodes and are difficult to deploy on large-scale datasets. In this paper, we argue that graph learning in spatiotemporal forecasting is to discriminate different nodes thus generating node-specific features. To achieve the same effect of graph learning, we propose to use node embeddings to learn node-specific features and introduce a simple yet well-performing Embedding Enhanced MLP (E2MLP) spatiotemporal forecasting model. A hierarchical MLP framework is proposed, where node embeddings are introduced in each procedure of the framework. E2MLP outperforms existing SOTA spatiotemporal forecasting models on five real-world widely-used datasets, and can be easily scaled to newly emerging nodes without any performance influence on existing nodes. The source codes are publicly available from https://github.com/Ziyan2019/E2MLP

Irregular time series analysis is increasingly essential in data management due to the proliferation of complex data irregularly sampled by real-world systems. Traditional time series models, including RNN-based models and transformer variants, face significant challenges in generalizing to continuous-time paradigms, which are essential for capturing the ongoing dynamics of irregular time series. Neural Ordinary Differential Equations (NODEs) assume a continuous latent dynamic and provide an elegant framework for irregular time series analysis, yet they suffer from limitations like fragmented latent processes and the inability to fully exploit interdependencies among observations. To address these challenges, we propose a novel Differentiable hidden state enhanced neural ODE framework, termed DIFFODE, designed to effectively model irregular time series. Concretely, we introduce an attention-based differential hidden state that maps irregular observations into a continuous hidden state space, enabling the extraction of latent dynamics while preserving temporal continuity. Leveraging the theory of generalized inverses, DIFFODE innovatively derives ODEs to describe hidden state dynamics. Furthermore, we incorporate the Hoyer metric into our framework to enhance its capacity to capture subtle yet critical temporal shifts, significantly improving the accuracy of time series modeling. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of DIFFODE across three key tasks, including irregular time series classification, interpolation, and extrapolation.

Generative flow networks (GFlowNets) have been considered as powerful tools for generating candidates with desired properties. Given that evaluating the property of candidates can be complex and time-consuming, existing GFlowNets train proxy models for efficient online evaluation. However, the performance of proxy models is heavily dependent on the amount of data and is of considerable uncertainty. Therefore, it is of great interest that how to develop an offline GFlowNet that does not rely on online evaluation. Under the offline setting, the limited data results in an insufficient exploration of state space. The insufficient exploration means that offline GFlowNets can hardly generate satisfying candidates out of the distribution of training data. Therefore, it is critical to restrict the offline model to act in the distribution of training data. The distinctive training goal of GFlownets poses a unique challenge for making such restrictions. Tackling the challenge, we propose Conservative Offline GFlowNet (COFlowNet) in this paper. We define unsupported flow, edges containing unseen states in training data. Models can learn extremely little knowledge about unsupported flow from training data. By constraining the model from exploring unsupported flows, we restrict COFlowNet to explore as optimal trajectories on the training set as possible, thus generating better candidates. In order to improve the diversity of candidates, we further introduce a quantile version of unsupported flow restriction. Experimental results on several widely-used datasets validate the effectiveness of COFlowNet in generating high-scored and diverse candidates. All implementations are available at https://github.com/yuxuan9982/COflownet.

In this paper, we use the Fire Dynamics Simulator (FDS) combined with the {\fontfamily{lmtt}\selectfont \textit{supercomputer}} support to create a \textbf{C}ombustion \textbf{K}inetics (CK) dataset for machine learning and scientific research. This dataset captures the development of fires in industrial parks with high-precision Computational Fluid Dynamics (CFD) simulations. It includes various physical fields such as temperature and pressure, and covers multiple environmental combinations for exploring \underline{multi-physics} field coupling phenomena. Additionally, we evaluate several advanced machine learning architectures across our {\fontfamily{lmtt}\selectfont {Open-CK}} benchmark using a substantial computational setup of 64 NVIDIA A100 GPUs: \ding{182} vision backbone; \ding{183} spatio-temporal predictive models; \ding{184} operator learning frameworks. These architectures uniquely excel at handling complex physical field data. We also introduce three benchmarks to demonstrate their potential in enhancing the exploration of downstream tasks: (a) capturing continuous changes in combustion kinetics; (b) a neural partial differential equation solver for learning temperature fields and turbulence; (c) reconstruction of sparse physical observations. The Open-CK dataset and benchmarks aim to advance research in combustion kinetics driven by machine learning, providing a reliable baseline for developing and comparing cutting-edge technologies and models. We hope to further promote the application of deep learning in earth sciences. Our project is available at \url{https://github.com/whscience/Open-CK}.

Existing image super-resolution (SR) techniques often fail to generalize effectively in complex real-world settings due to the significant divergence between training data and practical scenarios. To address this challenge, previous efforts have either manually simulated intricate physical-based degradations or utilized learning-based techniques, yet these approaches remain inadequate for producing large-scale, realistic, and diverse data simultaneously. In this paper, we introduce a novel Realistic Decoupled Data Generator (RealDGen), an unsupervised learning data generation framework designed for real-world super-resolution. We meticulously develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model to create realistic low-resolution images from unpaired real LR and HR images. Extensive experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations, significantly advancing the performance of popular SR models on various real-world benchmarks.

Molecular relational learning (MRL) seeks to understand the interaction behaviors between molecules, a pivotal task in domains such as drug discovery and materials science. Recently, extracting core substructures and modeling their interactions have emerged as mainstream approaches within machine learning-assisted methods. However, these methods still exhibit some limitations, such as insufficient consideration of molecular interactions or capturing substructures that include excessive noise, which hampers precise core substructure extraction. To address these challenges, we present an integrated dynamic framework called Iterative Substructure Extraction (ISE). ISE employs the Expectation-Maximization (EM) algorithm for MRL tasks, where the core substructures of interacting molecules are treated as latent variables and model parameters, respectively. Through iterative refinement, ISE gradually narrows the interactions from the entire molecular structures to just the core substructures. Moreover, to ensure the extracted substructures are concise and compact, we propose the Interactive Graph Information Bottleneck (IGIB) theory, which focuses on capturing the most influential yet minimal interactive substructures. In summary, our approach, guided by the IGIB theory, achieves precise substructure extraction within the ISE framework and is encapsulated in the IGIB-ISE} Extensive experiments validate the superiority of our model over state-of-the-art baselines across various tasks in terms of accuracy, generalizability, and interpretability.

Long-tailed distribution datasets are prevalent in many machine learning tasks, yet existing neural network models still face significant challenges when handling such data. This paper proposes a novel adaptive pruning strategy, LTAP (Long-Tailed Adaptive Pruner), aimed at balancing model efficiency and performance to better address the challenges posed by long-tailed data distributions. LTAP introduces multi-dimensional importance scoring criteria and designs a dynamic weight adjustment mechanism to adaptively determine the pruning priority of parameters for different classes. By focusing on protecting parameters critical for tail classes, LTAP significantly enhances computational efficiency while maintaining model performance. This method combines the strengths of long-tailed learning and neural network pruning, overcoming the limitations of existing approaches in handling imbalanced data. Extensive experiments demonstrate that LTAP outperforms existing methods on various long-tailed datasets, achieving a good balance between model compression rate, computational efficiency, and classification accuracy. This research provides new insights into solving model optimization problems in long-tailed learning and is significant for improving the performance of neural networks on imbalanced datasets. The code is available at https://github.com/DataLab-atom/LT-VOTE.

Long-term time series forecasting (LTSF) has traditionally relied on large parameters to capture extended temporal dependencies, resulting in substantial computational costs and inefficiencies in both memory usage and processing time. However, time series data, unlike high-dimensional images or text, often exhibit temporal pattern similarity and low-rank structures, especially in long-term horizons. By leveraging this structure, models can be guided to focus on more essential, concise temporal data, improving both accuracy and computational efficiency. In this paper, we introduce TimeBase, an ultra-lightweight network to harness the power of minimalism in LTSF. TimeBase 1) extracts core basis temporal components and 2) transforms traditional point-level forecasting into efficient segment-level forecasting, achieving optimal utilization of both data and parameters. Extensive experiments on diverse real-world datasets show that TimeBase achieves remarkable efficiency and secures competitive forecasting performance. Additionally, TimeBase can also serve as a very effective plug-and-play complexity reducer for any patch-based forecasting models. Code is available at https://github.com/hqh0728/TimeBase.

Discovering regularities from spatiotemporal systems can benefit various scientific and social planning. Current spatiotemporal learners usually train an independent model from a specific source data that leads to limited transferability among sources, where even correlated tasks requires new design and training. The key towards increasing cross-domain knowledge is to enable collective intelligence and model evolution. In this paper, inspired by neuroscience theories, we theoretically derive the increased information boundary via learning cross-domain collective intelligence and propose a Synaptic EVOlutional spatiotemporal network, SynEVO, where SynEVO breaks the model independence and enables cross-domain knowledge to be shared and aggregated. Specifically, we first re-order the sample groups to imitate the human curriculum learning, and devise two complementary learners, elastic common container and task-independent extractor to allow model growth and task-wise commonality and personality disentanglement. Then an adaptive dynamic coupler with a new difference metric determines whether the new sample group should be incorporated into common container to achieve model evolution under various domains. Experiments show that SynEVO improves the generalization capacity by at most 42\% under cross-domain scenarios and SynEVO provides a paradigm of NeuroAI for knowledge transfer and adaptation. Code available at [https://github.com/Rodger-Lau/SynEVO](https://github.com/Rodger-Lau/SynEVO).

Recently, spatiotemporal graph convolutional networks have achieved dominant performance in spatiotemporal prediction tasks. However, most models relying on node-to-node messaging interaction exhibit sensitivity to spatiotemporal shifts, encountering out-of-distribution (OOD) challenges. To address these issues, we introduce \textbf{\underline{S}}patio-\textbf{\underline{T}}emporal \textbf{\underline{O}}OD \textbf{\underline{P}}rocessor (STOP), which employs a centralized messaging mechanism along with a message perturbation mechanism to facilitate robust spatiotemporal interactions. Specifically, the centralized messaging mechanism integrates Context-Aware Units for coarse-grained spatiotemporal feature interactions with nodes, effectively blocking traditional node-to-node messages. We also implement a message perturbation mechanism to disrupt this messaging process, compelling the model to extract generalizable contextual features from generated variant environments. Finally, we customize a spatiotemporal distributionally robust optimization approach that exposes the model to challenging environments, thereby further enhancing its generalization capabilities. Compared with 14 baselines across six datasets, STOP achieves up to \textbf{17.01\%} improvement in generalization performance and \textbf{18.44\%} improvement in inductive learning performance. The code is available at https://github.com/PoorOtterBob/STOP.

The out-of-distribution (OOD) generalization challenge is a longstanding problem in graph learning. Through studying the fundamental cause of data distribution shift, i.e., the changes of environments, significant progress has been achieved in addressing this issue. However, we observe that existing works still fail to effectively address complex environment shifts. Existing practices place excessive attention on extracting causal subgraphs, inevitably treating spurious subgraphs as environment variables. While spurious subgraphs are controlled by environments, the space of environment changes encompass more than the scale of spurious subgraphs. Therefore, existing efforts have a limited inference space for environments, leading to failure under severe environment changes. To tackle this issue, we propose a negative inference graph OOD framework (NeGo) to broaden the inference space for environment factors. Inspired by the successful practice of prompt learning in capturing underlying semantics and causal associations in large language models, we design a negative prompt environment inference to extract underlying environment information. We further introduce the environment-enhanced invariant subgraph learning to effectively exploit inferred environment embedding, ensuring the robust extraction of causal subgraph in the environment shifts. Lastly, we conduct a comprehensive evaluation of NeGo on real-world datasets and synthetic datasets across domains. NeGo outperforms baselines on nearly all datasets, which verify the effectiveness of our framework.

Visual pattern recognition usually plays important roles in robotics and automation society where the pattern recognition relies on representation learning. Existing representation learning often neglects two important issues, the diversity of intra-class representation and under-exploited label utilization, especially the negative feedback during training process. Fortunately, prototype learning potentially raises label utilization and encourages intra-class diversity. In this paper, we investigate the intra-class diversity and effective updates in prototype learning for enhanced visual pattern recognition. Specifically, we propose a Label-aware multi-Prototype learning, LamPro, by incorporating the label awareness into both prototype formation and update to improve the representation quality. Firstly, we design a supervised contrastive learning to achieve class-discriminative representations. Secondly, we randomly initialize multiple prototypes and update the nearest prototype upon the arrival of instance, to preserve intra-class diversity. Thirdly, we propose a novel Label-guided Adaptive Updating. We separate the prototype updates from the representation optimization and exploit the label indexes to directly implement the prediction feedback. To correct the model optimization directions, we identify the negative feedback, and correct the prototype updates via queries of labels. Finally, we design a memory-based counter to alternately update these deviated prototypes. Experiments verify the effectiveness of our label-aware and joint multi-prototype updating strategies.

Image restoration aims to recover details and enhance contrast in degraded images. With the growing demand for high-quality imaging (e.g., 4K and 8K), achieving a balance between restoration quality and computational efficiency has become increasingly critical. Existing methods, primarily based on CNNs, Transformers, or their hybrid approaches, apply uniform deep representation extraction across the image. However, these methods often struggle to effectively model long-range dependencies and largely overlook the spatial characteristics of image degradation (regions with richer textures tend to suffer more severe damage), making it hard to achieve the best trade-off between restoration quality and efficiency. To address these issues, we propose a novel texture-aware image restoration method, TAMambaIR, which simultaneously perceives image textures and achieves a trade-off between performance and efficiency. Specifically, we introduce a novel Texture-Aware State Space Model, which enhances texture awareness and improves efficiency by modulating the transition matrix of the state-space equation and focusing on regions with complex textures. Additionally, we design a Multi-Directional Perception Block to improve multi-directional receptive fields while maintaining low computational overhead. Extensive experiments on benchmarks for image super-resolution, deraining, and low-light image enhancement demonstrate that TAMambaIR achieves state-of-the-art performance with significantly improved efficiency, establishing it as a robust and efficient framework for image restoration.

Current spatio-temporal modeling techniques largely rely on the abundant data and the design of task-specific models. However, many cities lack well-established digital infrastructures, making data scarcity and the high cost of model development significant barriers to application deployment. Therefore, this work aims to enable spatio-temporal learning to cope with the problems of few-shot data modeling and model generalizability. To this end, we propose a Universal Spatio-Temporal Correlationship pre-training framework (USTC), for spatio-temporal modeling across different cities and tasks. To enhance the spatio-temporal representations during pre-training, we propose to decouple the time-frequency patterns within data, and leverage contrastive learning to maintain the time-frequency consistency. To further improve the adaptability to downstream tasks, we design a prompt generation module to mine personalized spatio-temporal patterns on the target city, which can be integrated with the learned common spatio-temporal representations to collaboratively serve downstream tasks. Extensive experiments conducted on real-world datasets demonstrate that USTC significantly outperforms the advanced baselines in forecasting, imputation, and extrapolation across cities.

Molecular Relational Learning (MRL) aims to understand interactions between molecular pairs, playing a critical role in advancing biochemical research. Recently, Large Language Models (LLMs), with their extensive knowledge bases and advanced reasoning capabilities, have emerged as powerful tools for MRL. However, existing LLMs, which primarily rely on SMILES strings and molecular graphs, face two major challenges. They struggle to capture molecular stereochemistry and dynamics, as molecules possess multiple 3D conformations with varying reactivity and dynamic transformation relationships that are essential for accurately predicting molecular interactions but cannot be effectively represented by 1D SMILES or 2D molecular graphs. Additionally, these models do not consider the autoregressive nature of LLMs, overlooking the impact of input order on model performance. To address these issues, we propose DO-CoLM: a Dynamic relationship capture and self-adaptive Ordering 3D molecular Conformation LM for MRL. By introducing modules to dynamically model intra-molecular and inter-molecular conformational relationships and adaptively adjust the molecular modality input order, DO-CoLM achieves superior performance, as demonstrated by experimental results on 12 cross-domain datasets.

Accurate prediction of ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) properties is crucial in drug development, as these properties directly impact a drug's efficacy and safety. However, existing multi-task learning models often face challenges related to noise interference and task conflicts when dealing with complex molecular structures. To address these issues, we propose a novel multi-task Graph Neural Network (GNN) model, \textbf{MTGIB-UNet}. The model begins by encoding molecular graphs to capture intricate molecular structure information. Subsequently, based on the Graph Information Bottleneck (GIB) principle, the model compresses the information flow by extracting subgraphs, retaining task-relevant features while removing noise for each task. These embeddings are then fused through a gated network that dynamically adjusts the contribution weights of auxiliary tasks to the primary task. Specifically, an uncertainty weighting (UW) strategy is applied, with additional emphasis placed on the primary task, allowing dynamic adjustment of task weights while strengthening the influence of the primary task on model training. Experiments on standard ADMET datasets demonstrate that our model outperforms existing methods. Additionally, the model shows good interpretability by identifying key molecular substructures related to specific ADMET endpoints.

Air quality prediction plays a crucial role in the development of smart cities, garnering significant attention from both academia and industry. Current air quality prediction models encounter two major limitations: their high computational complexity limits scalability to nationwide datasets, and they often regard weather covariates as optional auxiliary information. In reality, weather covariates can have a substantial impact on air quality indices (AQI), exhibiting a significant causal association. In this paper, we first present a nationwide air quality dataset to address the lack of open-source, large-scale datasets in this field. Then we propose a causal learning model, CauAir, for air quality prediction that harnesses the powerful representation capabilities of the Transformer to explicitly model the causal association between weather covariates and AQI. To address the high complexity of traditional Transformers, we design CachLormer, which features two key innovations: a simplified architecture with redundant components removed, and a cache-attention mechanism that employs learnable embeddings for perceiving causal association between AQI and weather covariates in a coarsegrained perspective. We use information theory to illustrate the superiority of the proposed model. Finally, experimental results on three datasets with 28 as the baseline demonstrate that our model achieves competitive performance, while maintaining high training efficiency and low memory consumption. The source code is available at CauAir Official Repository.

Dynamic graphs are ubiquitous in the real world, presenting the temporal evolution of individuals within spatial associations. Recently, dynamic graph learning research is flourishing, striving to more effectively capture evolutionary patterns and spatial correlations. However, existing methods still fail to address the issue of concept shift in dynamic graphs. Concept shift manifests as a distribution shift in the mapping pattern between historical observations and future evolution. The reason is that some environment variables in dynamic graphs exert varying effects on evolution patterns, but these variables are not effectively captured by the models, leading to the intractable concept shift issue. To tackle this issue, we propose a State-driven environment inference framework (Samen) to achieve a dynamic graph learning framework equipped with concept generalization ability. Firstly, we propose a two-stage environment inference and compression strategy. From the perspective of state space, we introduce a prefix-suffix collaborative state learning mechanism to bidirectionally model the spatio-temporal states. A hierarchical state compressor is further designed to refine the state information resulting in concept shift. Secondly, we propose a skip-connection spatio-temporal prediction module, which effectively utilizes the inferred environments to improve the model's generalization capability. Finally, we select seven datasets from different domains to validate the effectiveness of our model. By comparing the performance of different models on samples with concept shift, we verify that our Samen gains generalization capacity that existing methods fail to capture.

Molecular Relational Learning (MRL) aims to understand interactions between molecular pairs, playing a critical role in advancing biochemical research. With the recent development of large language models (LLMs), a growing number of studies have explored the integration of MRL with LLMs and achieved promising results. However, the increasing availability of diverse LLMs and molecular structure encoders has significantly expanded the model space, presenting major challenges for benchmarking. Currently, there is no LLM framework that supports both flexible molecular input formats and dynamic architectural switching. To address these challenges, reduce redundant coding, and ensure fair model comparison, we propose ModuLM, a framework designed to support flexible LLM-based model construction and diverse molecular representations. ModuLM provides a rich suite of modular components, including 8 types of 2D molecular graph encoders, 11 types of 3D molecular conformation encoders, 7 types of interaction layers, and 7 mainstream LLM backbones. Owing to its highly flexible model assembly mechanism, ModuLM enables the dynamic construction of over 50,000 distinct model configurations. In addition, we provide comprehensive benchmark results to demonstrate the effectiveness of ModuLM in supporting LLM-based MRL tasks.

Drug recommendation systems aim to identify optimal drug combinations for patient care, balancing therapeutic efficacy and safety. Advances in large-scale longitudinal EHRs have enabled learning-based approaches that leverage patient histories such as diagnoses, procedures, and previously prescribed drugs, to model complex patient-drug relationships. Yet, many existing solutions overlook standard clinical practices that favor certain drugs for specific conditions and fail to fully integrate the influence of molecular substructures on drug efficacy and safety. In response, we propose \textbf{SubRec}, a unified framework that integrates representation learning across both patient and drug spaces. Specifically, SubRec introduces a conditional information bottleneck to extract core drug substructures most relevant to patient conditions, thereby enhancing interpretability and clinical alignment. Meanwhile, an adaptive vector quantization mechanism is designed to generate patient–drug interaction patterns into a condition-aware codebook which reuses clinically meaningful patterns, reduces training overhead, and provides a controllable latent space for recommendation. Crucially, the synergy between condition-specific substructure learning and discrete patient prototypes allows SubRec to make accurate and personalized drug recommendations. Experimental results on the real-world MIMIC III and IV demonstrate our model's advantages. The source code is available at \href{https://anonymous.4open.science/r/DrugRecommendation-5173}{https://anonymous.4open.science/}.

Multi-frame video enhancement tasks aim to improve the spatial and temporal resolution and quality of video sequences by leveraging temporal information from multiple frames, which are widely used in streaming video processing, surveillance, and generation. Although numerous Transformer-based enhancement methods have achieved impressive performance, their computational and memory demands hinder deployment on edge devices. Quantization offers a practical solution by reducing the bit-width of weights and activations to improve efficiency. However, directly applying existing quantization methods to video enhancement tasks often leads to significant performance degradation and loss of fine details. This stems from two limitations: (a) inability to allocate varying representational capacity across frames, which results in suboptimal dynamic range adaptation; (b) over-reliance on full-precision teachers, which limits the learning of low-bit student models. To tackle these challenges, we propose a novel quantization method for video enhancement: Progressive Multi-Frame Quantization for Video Enhancement (PMQ-VE). This framework features a coarse-to-fine two-stage process: Backtracking-based Multi-Frame Quantization (BMFQ) and Progressive Multi-Teacher Distillation (PMTD). BMFQ utilizes a percentile-based initialization and iterative search with pruning and backtracking for robust clipping bounds. PMTD employs a progressive distillation strategy with both full-precision and multiple high-bit (INT) teachers to enhance low-bit models' capacity and quality. Extensive experiments demonstrate that our method outperforms existing approaches, achieving state-of-the-art performance across multiple tasks and benchmarks. The code will be made publicly available.

Time series forecasting is a critical and complex task, characterized by diverse temporal patterns, varying statistical properties, and different prediction horizons across datasets and domains. Conventional approaches typically rely on a single, unified model architecture to handle all forecasting scenarios. However, such monolithic models struggle to generalize across dynamically evolving time series with shifting patterns. In reality, different types of time series may require distinct modeling strategies. Some benefit from homogeneous multi-scale forecasting awareness, while others rely on more complex and heterogeneous signal perception. Relying on a single model to capture all temporal diversity and structural variations leads to limited performance and poor interpretability. To address this challenge, we propose a Multi-Agent Forecasting System (MAFS) that abandons the one-size-fits-all paradigm. MAFS decomposes the forecasting task into multiple sub-tasks, each handled by a dedicated agent trained on specific temporal perspectives (e.g., different forecasting resolutions or signal characteristics). Furthermore, to achieve holistic forecasting, agents share and refine information through different communication topology, enabling cooperative reasoning across different temporal views. A lightweight voting aggregator then integrates their outputs into consistent final predictions. Extensive experiments across 11 benchmarks demonstrate that MAFS significantly outperforms traditional single-model approaches, yielding more robust and adaptable forecasts.

The stable periodic patterns present in the time series data serve as the foundation for long-term forecasting. However, existing models suffer from limitations such as continuous and chaotic input partitioning, as well as weak inductive biases, which restrict their ability to capture such recurring structures. In this paper, we propose MoFo, which interprets periodicity as both the correlation of period-aligned time steps and the trend of period-offset time steps. We first design period-structured patches—2D tensors generated through discrete sampling—where each row contains only period-aligned time steps, enabling direct modeling of periodic correlations. Period-offset time steps within a period are aligned in columns. To capture trends across these offset time steps, we introduce a period-aware modulator. This modulator introduces an adaptive strong inductive bias through a regulated relaxation function, encouraging the model to generate attention coefficients that align with periodic trends. This function is end-to-end trainable, enabling the model to adaptively capture the distinct periodic patterns across diverse datasets. Extensive empirical results on widely used benchmark datasets demonstrate that MoFo achieves competitive performance while maintaining high memory efficiency and fast training speed.

The effectiveness of Spatiotemporal Graph Neural Networks (STGNNs) critically hinges on the quality of the underlying graph topology. While end-to-end adaptive graph learning methods have demonstrated promising results in capturing latent spatiotemporal dependencies, they often suffer from high computational complexity and limited expressive capacity. In this paper, we propose MAGE for efficient spatiotemporal forecasting. We first conduct a theoretical analysis demonstrating that the ReLU activation function employed in existing methods amplifies edge-level noise during graph topology learning, thereby compromising the fidelity of the learned graph structures. To enhance model expressiveness, we introduce a sparse yet balanced mixture-of-experts strategy, where each expert perceives the unique underlying graph through kernel-based functions and operates with linear complexity relative to the number of nodes. The sparsity mechanism ensures that each node interacts exclusively with compatible experts, while the balancing mechanism promotes uniform activation across all experts, enabling diverse and adaptive graph representations. Furthermore, we theoretically establish that a single graph convolution using the learned graph in MAGE is mathematically equivalent to multiple convolutional steps under conventional graphs. We evaluate MAGE against advanced baselines on multiple real-world spatiotemporal datasets. MAGE achieves competitive performance while maintaining strong computational efficiency.

Artificial intelligence (AI) has played a transformative role in chemical research, greatly facilitating the prediction of small molecule properties, simulation of catalytic processes, and material design. These advances are driven by increases in computing power, open source machine learning frameworks, and extensive chemical datasets. However, a persistent challenge is the limited amount of high-quality real-world data, while models calculated based on large amounts of theoretical data are often costly and difficult to deploy, which hinders the applicability of AI models in real-world scenarios. In this study, we enhance the prediction of solute-solvent properties by proposing a novel sample selection method: the iterative core subset extraction (CSIE) framework. CSIE iteratively updates the core sample subset based on information gain to remove redundant features in theoretical data and optimize the performance of the model on real chemical datasets. Furthermore, we introduce an asymmetric molecular interaction graph neural network (AMGNN) that combines positional information and bidirectional edge connections to simulate real-world chemical reaction scenarios to better capture solute-solvent interactions. Experimental results show that our method can accurately extract the core subset and improve the prediction accuracy.

Masked Graph Autoencoders (MGAEs) have gained significant attention recently. Their proxy tasks typically involve random corruption of input graphs followed by reconstruction. However, in the molecular domain, two main issues arise: the predetermined mask ratio and reconstruction objectives can lead to suboptimal performance or negative transfer due to overly simplified or complex tasks, and these tasks may deviate from chemical priors. To tackle these challenges, we propose Dynamic and Chemical Constraints (DyCC) for MGAEs. This includes a masking strategy called GIBMS, which preserves essential semantic information during graph masking while adaptively adjusting the mask ratio and content for each molecule. Additionally, we introduce a Soft Label Generator (SLG) that reconstructs masked tokens as learnable prototypes (soft labels) rather than hard labels. These components adhere to chemical constraints and allow dynamic variation of proxy tasks during training. We integrate the model-agnostic DyCC into various MGAEs and conduct comprehensive experiments, demonstrating significant performance improvements. Our code is available at \url{https://github.com/forever-ly/DyCC}.

Long-term time series forecasting (LTSF) aims to predict future trends based on historical data. While longer lookback windows theoretically offer more comprehensive insights, Transformer-based models often struggle with them. On one hand, longer windows introduce more noise and redundancy, hindering the model's learning process. On the other hand, Transformers suffer from attention dispersion and are prone to overfitting to noise, especially when processing long sequences. In this paper, we introduce the Maximum Effective Window (MEW) metric to assess a model's ability to effectively utilize the lookback window. We also propose two model-agnostic modules to enhance MEW, enabling models to better leverage historical data for improved performance. Specifically, to reduce redundancy and noise, we introduce the Information Bottleneck Filter (IBF), which employs information bottleneck theory to extract the most essential subsequences from the input. Additionally, we propose the Hybrid-Transformer-Mamba (HTM), which incorporates the Mamba mechanism for selective forgetting of long sequences while harnessing the Transformer's strong modeling capabilities for shorter sequences. We integrate these two modules into various Transformer-based models, and experimental results show that they effectively enhance MEW, leading to improved overall performance. Our code is available at \url{https://github.com/forever-ly/PIH}.

Long-Tailed Recognition (LTR) presents a significant challenge due to extreme class imbalance, where existing methods often struggle to balance performance across head and tail classes. Directly applying multi-objective optimization (MOO) to leverage multiple LTR strategies can be complex and unstable. To address this, we propose TS-MOF (Two-Stage Multi-Objective Fine-tuning), a novel framework that strategically decouples feature learning from classifier adaptation. After standard pre-training, TS-MOF freezes the feature backbone and focuses on an efficient multi-objective fine-tuning of specialized classifier heads. The core of TS-MOF's second stage lies in two innovations: Refined Performance Level Agreement for adaptive task weighting based on real-time per-class performance, and Robust Deterministic Projective Conflict Gradient for stable gradient conflict resolution and constructive fusion. This approach enables effective synergy between diverse LTR strategies, leading to significant and balanced performance improvements. Extensive experiments on CIFAR100-LT, ImageNet-LT, and iNaturalist 2018 demonstrate that TS-MOF achieves state-of-the-art results, particularly enhancing tail class accuracy (e.g., +3.3\% on CIFAR100-LT IR=100 tail) while improving head class performance, all within a remarkably short fine-tuning period of 20 epochs.

Scientific discovery across diverse fields increasingly grapples with datasets exhibiting pathological long-tailed distributions: a few common phenomena overshadow a multitude of rare yet scientifically critical instances. Unlike standard benchmarks, these scientific datasets often feature extreme imbalance coupled with a modest number of classes and limited overall sample volume, rendering existing long-tailed recognition (LTR) techniques ineffective. Such methods, biased by majority classes or prone to overfitting on scarce tail data, frequently fail to identify the very instances—novel materials, rare disease biomarkers, faint astronomical signals—that drive scientific breakthroughs. This paper introduces a novel, end-to-end framework explicitly designed to address pathological long-tailed recognition in scientific contexts. Our approach synergizes a Balanced Supervised Contrastive Learning (B-SCL) mechanism, which enhances the representation of tail classes by dynamically re-weighting their contributions, with a Smooth Objective Regularization (SOR) strategy that manages the inherent tension between tail-class focus and overall classification performance. We introduce and analyze the real-world ZincFluor chemical dataset ($\mathcal{T}=137.54$) and synthetic benchmarks with controllable extreme imbalances (CIFAR-LT variants). Extensive evaluations demonstrate our method's superior ability to decipher these extremes. Notably, on ZincFluor, our approach achieves a Tail Top-2 accuracy of $66.84\%$, significantly outperforming existing techniques. On CIFAR-10-LT with an imbalance ratio of $1000$ ($\mathcal{T}=100$), our method achieves a tail-class accuracy of $38.99\%$, substantially leading the next best. These results underscore our framework's potential to unlock novel insights from complex, imbalanced scientific datasets, thereby accelerating discovery.

The World Wide Web thrives on intelligent services that depend heavily on accurate time series forecasting to navigate dynamic and evolving environments. Due to the partially-observed nature of real world, exclusively focusing on the target of interest, so-called endogenous variables, is insufficient for accurate forecasting, especially in web systems that are susceptible to external influences. Thus, utilizing exogenous variables to harness external information, i.e., forecasting with exogenous variable (FEV), is imperative. Nevertheless, as the external environment is complex and ever-evolving, inadequately capturing external influences can even lead to learning spurious correlations and invalid prediction. Fortunately, recent studies have demonstrated that large language models (LLMs) exhibit exceptional recognition capabilities across open real-world systems, including a deep understanding of exogenous environments. However, it is difficult to directly apply LLMs for FEV due to challenges of task activation, exogenous knowledge extraction, and feature space alignment. In this work, we devise ExoLLM, an LLM-driven method to sufficiently utilize Exogenous variables for time series forecasting. We begin by Meta-task Instruction to activate the knowledge transfer of LLM from natural language processing to FEV. To comprehensively understand the intricate and hierarchical influences of exogenous variables, we propose Multi-grained Prompts, encompassing diverse external influences, including natural attributes, trend correlations, and period relationships between two types of variables. Additionally, a Dual TS-Text Attention is devised to bridge the feature gap between text and numeric data in LLM. Evaluation on real-world datasets demonstrates ExoLLM's superiority in exploiting exogenous information for forecasting with open-world language knowledge.

Spatiotemporal (ST) learning has become a crucial technique to enable smart cities and sustainable urban development. Current ST learning models capture the heterogeneity via various spatial convolution and temporal evolution blocks. However, rapid urbanization leads to fluctuating distributions in urban data and city structures, resulting in existing methods suffering generalization and data adaptation issues. Despite efforts, existing methods fail to deal with newly arrived observations, and the limitation of those methods with generalization capacity lies in the repeated training that leads to inconvenience, inefficiency and resource waste. Motivated by complementary learning in neuroscience, we introduce a prompt-based complementary spatiotemporal learning termed ComS2T, to empower the evolution of models for data adaptation. We first disentangle the neural architecture into two disjoint structures, a stable neocortex for consolidating historical memory, and a dynamic hippocampus for new knowledge update. Then we train the dynamic spatial and temporal prompts by characterizing distribution of main observations to enable prompts adaptive to new data. This data-adaptive prompt mechanism, combined with a two-stage training process, facilitates fine-tuning of the neural architecture conditioned on prompts, thereby enabling efficient adaptation during testing. Extensive experiments validate the efficacy of ComS2T in adapting various spatiotemporal out-of-distribution scenarios while maintaining effective inferences.

While existing spatiotemporal prediction models have shown promising performance, they often rely on the assumption of input-label spatiotemporal consistency, and their high complexity raises concerns about scalability. To enhance both efficiency and performance, we integrate label information into the learning process and propose a spatiotemporal dynamic theory that outlines a bi-directional learning paradigm. Building on this paradigm, we design BiST, a lightweight yet effective Bi -directional S patio -T emporal prediction model. BiST incorporates two key processes: a forward spatiotemporal learning process and a backward correction process. The forward process utilizes MLP layers exclusively to model input correlations and generate base prediction. In the backward process, we implement a spatiotemporal decoupling module, which can learn the residual modeling deviation between input and label representations from a decoupled perspective. After smoothing the residual with a diffusion module, we can obtain the correction term to correct the base predictions. This innovative design enables BiST to achieve competitive performance while remaining lightweight. We evaluate BiST against 26 baselines across 13 datasets, including a large-scale dataset with ten thousand nodes and a longrange dataset spanning 20 years. An impressive experimental result demonstrates that BiST achieves a 8.13% improvement in performance compared to state-of-the-art models while consuming only 1.86% of the training time and 7.36% of the memory usage.

Pre-trained language models (PLMs) have driven substantial progress in natural language processing but remain vulnerable to adversarial attacks, raising concerns about their robustness in real-world applications. Previous studies have sought to mitigate the impact of adversarial attacks by introducing adversarial perturbations into the training process, either implicitly or explicitly. While both strategies enhance robustness, they often incur high computational costs. In this work, we propose a simple yet effective add-on module that enhances the adversarial robustness of PLMs by removing instance-level principal components, without relying on conventional adversarial defenses or perturbing the original training data. Our approach transforms the embedding space to approximate Gaussian properties, thereby reducing its susceptibility to adversarial perturbations while preserving semantic relationships. This transformation aligns embedding distributions in a way that minimizes the impact of adversarial noise on decision boundaries, enhancing robustness without requiring adversarial examples or costly training-time augmentation. Evaluations on eight benchmark datasets show that our approach improves adversarial robustness while maintaining comparable before-attack accuracy to baselines, achieving a balanced trade-off between robustness and generalization.

Protein-protein interaction (PPI) is a fundamental means of function and signaling in biological systems. The significant increase in demand and cost associated with experimental PPI research requires computational tools to automatically predict and understand PPI. However, existing methods either heavily rely on protein sequences for PPI prediction, or focus on protein structure based on the belief that structure is the key determinant of interactions. But in fact, both have a significant impact on the function of proteins. Therefore, we propose an integrated framework TRGH-PPI based on Transformer and GNN, including a protein feature extraction module and a PPI prediction module, which can simultaneously model both types of protein information and predict PPI. In the protein feature extraction module, we repeatedly use Transformer and GCN to iteratively update the sequence representation and structural features of proteins. The combination of Transformer and GCN enables them to leverage their respective advantages, promote model innovation, and improve the efficiency of graph data processing. In the PPI prediction module, we propose dpdGAT, which uses dot product operations more suitable for PPI prediction and has a dynamic attention mechanism. Numerous experiments have shown that TRGH-PPI is superior to current advanced methods and has shown high accuracy and generalization ability in predicting PPI. By integrating the synergistic modeling of sequence and structural features, the model has achieved an average improvement of 2% -5% in key indicators compared to the SOTA on three datasets.

Rapid and accurate identification of drug-drug interactions (DDIs) among multiple medications is crucial for various medical treatments, and clinical therapies. Currently, the increasing significance of molecular substructure interactions in DDI prediction has become a consensus. However, due to the uneven distribution of substructures in molecules and most methods do not consider the differences in substructure distributions, models may rely on spurious substructure relationships, thereby weakening their out-of-distribution (OOD) generalization ability. To address this challenge, we propose an OOD-DDI framework called IIB-DDI. Specifically, information bottleneck theory is initially utilized to extract core subgraphs of drug pairs. Then, considering the diversity and unknown nature of environments, we introduce the vector quantization to design an environment codebook, where the potential environments in the dataset are clustered into a specified number of categories. Subsequently, we position the extracted core subgraphs under various latent environmental factors to attain invariant core substructures (rationales). Additionally, the learned environmental distribution also could be acted as noise injection for optimizing mutual information, achieving a smoother and more stable training curve, thereby leading to lower loss. Extensive experiments conducted on real-world DDI datasets demonstrate the superiority of our model over state-of-the-art baselines.

Container-based task scheduling is critical for ensuring a reliable, flexible and cost-effective cloud computing mode. However, in different business cloud systems, state-of-the-art scheduling models are not as effective as those in the simulated world due to the sparsity issues associated with sample sizes and features. Herein, we propose a novel containerized task scheduling framework (SA2CTS) based on reinforcement learning (RL) that incorporates cross-modal contrastive learning (CL) loss. This framework optimizes the scheduler's understanding of the container-based cloud state in RL by adding a pretraining stage, promoting accurate scheduling action inference. Specifically, we design a two-stage learning pipeline. The initial stage involves pretraining the model on a large collection of aligned image-text pairs to extract fine-grained scheduling affinity features, and the high-level semantic representations of scheduling tasks are learned in the multimodal space. In the second stage, we fine-tune the pretrained model with multisource cluster feedback, i.e., build a mapping from state representations to scheduling actions through the RL paradigm, achieving task-oriented and semantic-aware scheduling. The experimental results obtained on three large-scale production cluster datasets substantiate that the proposed SA2CTS method can provide average convergence efficiency and resource utilization improvements of 17.57% and 10.42%, respectively, over the state-of-the-art RL scheduling methods.

Subgraph learning has dominated most practices of improving the expressive power of Message Passing Neural Networks (MPNNs). Existing subgraph discovery policies can be classified into node-based and partition-based, which both achieve impressive performance in most scenarios. However, both mainstream solutions still face a subgraph degradation trap. Subgraph degradation is reflected in the phenomenon that the subgraph-level methods fail to offer any benefits over node-level MPNNs. In this work, we empirically investigate the existence of the subgraph degradation issue and introduce a unified perspective, perfect reconstruction, to provide insights for improving two lines of methods. We further propose a subgraph learning strategy guided by the principle of perfect reconstruction. To achieve this, two major issues should be well-addressed, i.e.,(i) how to ensure the subgraphs to possess with ‘perfect’ information? (ii) how to guarantee the ‘reconstruction’ power of obtained subgraphs?First, we propose a subgraph partition strategyRayleigh-resistanceto extract non-overlap subgraphs by leveraging the graph spectral theory. Second, we put forward aQuerymechanism to achieve subgraph-level equivariant learning, which guarantees subgraph reconstruction ability. These two parts,perfect subgraph partitionandequivariant subgraph learningare seamlessly unified as a novelRayleigh-resistanceEquivariantSubgraph learningarchitecture (RayE-Sub). Comprehensive experiments on both synthetic and real datasets demonstrate that our approach can consistently outperform previous subgraph learning architectures.

Pre-training GNNs to extract transferable knowledge and apply it to downstream tasks has become the de facto standard of graph representation learning. Recent works focused on designing self-supervised pre-training tasks to extract useful and universal transferable knowledge from large-scale unlabeled data. However, they have to face an inevitable question: traditional pre-training strategies that aim at extracting useful information about pre-training tasks, may not extract all useful information about the downstream task. In this paper, we reexamine the pre-training process within traditional pre-training and fine-tuning frameworks from the perspective of Information Bottleneck (IB) and confirm that the forgetting phenomenon in pre-training phase may cause detrimental effects on downstream tasks. Therefore, we propose a novelDelayedBottleneckingPre-training (DBP) framework which maintains as much as possible mutual information between latent representations and training data during pre-training phase by suppressing the compression operation and delays the compression operation to fine-tuning phase to make sure the compression can be guided with labeled fine-tuning data and downstream tasks. To achieve this, we design two information control objectives that can be directly optimized and further integrate them into the actual model design. Extensive experiments on both chemistry and biology domains demonstrate the effectiveness of DBP.

An address, a textual description of a geographical location, plays an important role in location-based services such as instant delivery. However, abnormal addresses (i.e., an address without detailed or accurate information) have led to significant costs. In real-world settings, abnormal address detection is not trivial because it needs to be completed in real-time to support massive online queries from mobile devices. In this study, we designFastAddr, a fast abnormal address detection framework, which detects abnormal addresses in real time.FastAddrconsists of a novel contrastive address augmentation module and a lightweight multi-head attention model. We further designFastAddr+ to enhanceFastAddrby utilizing large-scale spatial entities. A comprehensive three-phase evaluation is conducted. (i) We evaluateFastAddron a real-world dataset and it yields the average F1 of 85.7% in 0.058 milliseconds, which outperforms the state-of-the-art models by 47.4% with a similar detection time. (ii) An offline A/B test shows thatFastAddroutperforms the previous model significantly. (iii) We also conduct an online A/B test to compareFastAddrwith the deployed model, which shows an improvement of F1 by more than 20%. Moreover, we conduct two case studies on real industry data, demonstrating both the efficiency and effectiveness ofFastAddr.

Spatiotemporal learning plays a crucial role in mobile computing techniques to empower smart cites. While existing research has made great efforts to achieve accurate predictions on the overall dataset, they still neglect the significant performance heterogeneity across samples. In this work, we designate the performance heterogeneity as the reason for unfair spatiotemporal learning, which not only degrades the practical functions of models, but also brings serious potential risks to real-world urban applications. To fix this gap, we propose a model-independent Fairness-aware framework for SpatioTemporal Graph learning (FairSTG), which inherits the idea of exploiting advantages of well-learned samples to challenging ones with collaborative mix-up. Specifically, FairSTG consists of a spatiotemporal feature extractor for model initialization, a collaborative representation enhancement for knowledge transfer between well-learned samples and challenging ones, and fairness objectives for immediately suppressing sample-level performance heterogeneity. Experiments on four spatiotemporal datasets demonstrate that our FairSTG significantly improves the fairness quality while maintaining comparable forecasting accuracy. Case studies show FairSTG can counter both spatial and temporal performance heterogeneity by our sample-level retrieval and compensation, and our work can potentially alleviate the risks on spatiotemporal resource allocation for underrepresented urban regions.

Understanding mobile traffic data and predicting future trends are essential for wireless operators and service providers to allocate resources efficiently and manage energy effectively. Despite the strong performance of existing models, accurately forecasting mobile traffic remains a challenge due to limited spatial and temporal modeling capabilities and high computational complexity. This paper introduces MobiMixer, a lightweight and efficient multi-scale spatiotemporal mixing model. Its core concept is to integrate multi-scale information from both spatial and temporal dimensions to improve performance on mobile traffic data. We develop a hierarchical interaction module that incorporates super nodes to enable global high-level feature interactions among nodes with common patterns. Additionally, we employ a dynamic time warping strategy to decouple mobile traffic sequences into stable and seasonal components, which are then modeled at different scales using a multi-scale temporal mixing module. We conduct extensive experiments on mobile traffic datasets collected from four international cities. Compared with 21 state-of-the-art benchmark models, MobiMixer demonstrates highly competitive performance, achieving a maximum improvement of 48.49% on the Milan mobile dataset. The model achieves an improvement in training efficiency of up to 10.69 times and reduces memory usage by 33.01%. The source code is available athttps://github.com/PoorOtterBob/Submitted_Code.

Lidars and cameras are critical sensors for 3D object detection in autonomous driving. Despite the increasing popularity of sensor fusion in this field, accurate and robust fusion methods are still under exploration due to non-homogenous representations. In this paper, we find that the complementary roles of point clouds and images vary with depth. An important reason is that the point cloud appearance changes significantly with increasing distance from the Lidar, while the image's edge, color, and texture information are not sensitive to depth. To address this, we propose a fusion module based on the Depth Attention Mechanism (DAM), which mainly consists of two operations: gated feature generation and point cloud division. The former adaptively learns the importance of bimodal features without additional annotations, while the latter divides point clouds to achieve differential fusion of multi-modal features at different depths. This fusion module can enhance the representation ability of original features for different point sets and provide more comprehensive features by using the dual splicing strategy of concatenation and index connection. Additionally, considering point density as a feature and its negative correlation with depth, we build an Adaptive Threshold Generation Network (ATGN) to generate the depth threshold by extracting density information, which can divide point clouds more reasonably. Experiments on the KITTI dataset demonstrate the effectiveness and competitiveness of our proposed models.

Automatic lip-reading (ALR) is the task of recognizing words based on visual information obtained from the speaker's lip movements. In this study, we introduce event cameras, a novel type of sensing device, for ALR. Event cameras offer both technical and application advantages over conventional cameras for ALR due to their higher temporal resolution, less redundant visual information, and lower power consumption. To recognize words from the event data, we propose a novel multigrained spatiotemporal features learning framework, which is capable of perceiving fine-grained spatiotemporal features from microsecond time-resolved event data. Specifically, we first convert the event data into event frames of multiple temporal resolutions to avoid losing too much visual information at the event representation stage. Then, they are fed into a multibranch subnetwork where the branch operating on low-rate frames can perceive spatially complete but temporally coarse features, while the branch operating on high frame rate can perceive spatially coarse but temporally fine features. Thus, fine-grained spatial and temporal features can be simultaneously learned by integrating the features perceived by different branches. Furthermore, to model the temporal relationships in the event stream, we design a temporal aggregation subnetwork to aggregate the features perceived by the multibranch subnetwork. In addition, we collect two event-based lip-reading datasets (DVS-Lip and DVS-LRW100) for the study of the event-based lip-reading task. Experimental results demonstrate the superiority of the proposed model over the state-of-the-art event-based action recognition models and video-based lip-reading models.

Multivariate time series (MTS) prediction has been widely adopted in various scenarios. Recently, some methods have employed patching to enhance local semantics and improve model performance. However, length-fixed patch are prone to losing temporal boundary information, such as complete peaks and periods. Moreover, existing methods mainly focus on modeling long-term dependencies across patches, while paying little attention to other dimensions (e.g., short-term dependencies within patches and complex interactions among cross-variavle patches). To address these challenges, we propose a pure MLP-based HDMixer, aiming to acquire patches with richer semantic information and efficiently modeling hierarchical interactions. Specifically, we design a Length-Extendable Patcher (LEP) tailored to MTS, which enriches the boundary information of patches and alleviates semantic incoherence in series. Subsequently, we devise a Hierarchical Dependency Explorer (HDE) based on pure MLPs. This explorer effectively models short-term dependencies within patches, long-term dependencies across patches, and complex interactions among variables. Extensive experiments on 9 real-world datasets demonstrate the superiority of our approach. The code is available at https://github.com/hqh0728/HDMixer.

With the progress of urban transportation systems, a significant amount of high-quality traffic data is continuously collected through streaming manners, which has propelled the prosperity of the field of spatial-temporal graph prediction. In this paper, rather than solely focusing on designing powerful models for static graphs, we shift our focus to spatial-temporal graph prediction in the dynamic scenario, which involves a continuously expanding and evolving underlying graph. To address inherent challenges, a decoupled learning framework (DLF) is proposed in this paper, which consists of a spatial-temporal graph learning network (DSTG) with a specialized decoupling training strategy. Incorporating inductive biases of time-series structures, DSTG can interpret time dependencies into latent trend and seasonal terms. To enable prompt adaptation to the evolving distribution of the dynamic graph, our decoupling training strategy is devised to iteratively update these two types of patterns. Specifically, for learning seasonal patterns, we conduct thorough training for the model using a long time series (e.g., three months of data). To enhance the learning ability of the model, we also introduce the masked auto-encoding mechanism. During this period, we frequently update trend patterns to expand new information from dynamic graphs. Considering both effectiveness and efficiency, we develop a subnet sampling strategy to select a few representative nodes for fine-tuning the weights of the model. These sampled nodes cover unseen patterns and previously learned patterns. Experiments on dynamic spatial-temporal graph datasets further demonstrate the competitive performance, superior efficiency, and strong scalability of the proposed framework.

One of the most important tasks in ride-hailing is order dispatching, i.e., assigning unserved orders to available drivers. Recent order dispatching has achieved a significant improvement due to the advance of reinforcement learning, which has been approved to be able to effectively address sequential decision-making problems like order dispatching. However, most existing reinforcement learning methods require agents to learn the optimal policy by interacting with environments online, which is challenging or impractical for real-world deployment due to high costs or safety concerns. For example, due to the spatiotemporally unbalanced supply and demand, online reinforcement learning-based order dispatching may significantly impact the revenue of the ride-hailing platform and passenger experience during the policy learning period. Hence, in this work, we develop an offline deep reinforcement learning framework called NondBREM for large-scale order dispatching, which learns policy from only the accumulated logged data to avoid costly and unsafe interactions with the environment. In NondBREM, a Nondeterministic Batch-Constrained Q-learning (NondBCQ) module is developed to reduce the algorithm extrapolation error and a Random Ensemble Mixture (REM) module that integrates multiple value networks with multi-head networks is utilized to improve the model generalization and robustness. Extensive experiments on large-scale real-world ride-hailing datasets show the superiority of our design.

Graph neural networks (GNNs) have achieved state-of-the-art results on many graph representation learning tasks by exploiting statistical correlations. However, numerous observations have shown that such correlations may not reflect the true causal mechanisms underlying the data and thus may hamper the ability of the model to generalize beyond the observed distribution. To address this problem, we propose an Information-based Causal Learning (ICL) framework that combines information theory and causality to analyze and improve graph representation learning to transform information relevance to causal dependence. Specifically, we first introduce a multi-objective mutual information optimization objective derived from information-theoretic analysis and causal learning principles to simultaneously extract invariant and interpretable causal information and reduce reliance on non-causal information in correlations. To optimize this multi-objective objective, we enable a causal disentanglement layer that effectively decouples the causal and non-causal information in the graph representations. Moreover, due to the intractability of mutual information estimation, we derive variational bounds that enable us to transform the above objective into a tractable loss function. To balance the multiple information objectives and avoid optimization conflicts, we leverage multi-objective gradient descent to achieve a stable and efficient transformation from informational correlation to causal dependency. Our approach provides important insights into modulating the information flow in GNNs to enhance their reliability and generalization. Extensive experiments demonstrate that our approach significantly improves the robustness and interpretability of GNNs across different distribution shifts. Visual analysis demonstrates how our method converts informative dependencies in representations into causal dependencies.

Predicting molecular properties is crucial across scientific and industrial domains such as drug discovery and material science. Contrastive learning has gained prominence as an effective method. However, routine strategies often overlook substructure information and run the risk of guiding the model to learn incorrect knowledge due to molecular property changes during augmentation. To address these issues, we propose a novel molecular contrastive learning model named Molecular Contrastive Learning with Learnable Weighted Substructures (MolCLW). Our model aims to enhance the learning of molecular substructures by comparing molecular representations generated at the molecular-level and substructure-level. Meanwhile, a transformer encoder module is dedicated to learning the weight scores of substructures, enabling the model to adjust the similarity level between positive pairs based on the varying masked substructures. In comparison to baselines, MolCLW demonstrates an averaged 2.2% improvement in ROC-AUC on 6 classification benchmarks and an averaged 4.3% decrease in error on 5 regression benchmarks. Furthermore, during the finetuning process, the model can attend to the substructure relevant to the downstream task by adjusting the weights of different substructures. The visualization results establish a link between downstream targets and substructure function, providing valuable insights and strong support for drug discovery, chemical reactions, and research.

Data-driven molecular computing has become an increasingly popular topic in AI for molecular and bioinformatic science. Current molecular modeling exploits the Graph Neural Networks (GNNs) to achieve the representation but they mostly fail to generalize to out-of-distribution (OOD) samples. Even though recent advances on OOD-oriented graph learning discovered the invariant rationale on graphs, they still ignore two important issues, i.e., 1) the increasing number and types of molecules expand patterns of environments on graphs, resulting in failures of invariant rationale based models, 2) the associations between discovered molecular subgraphs and corresponding properties are complex where causal substructures cannot fully interpret the labels. To this end, we propose an environment causal learning framework, EMoNet, to tackle the unresolved OOD challenge in molecular science. Specifically, we model the graph environments via bypassing invariant subgraphs. We first incorporate chemistry principle into our graph growth generator to imitate environment growth, and then devise an environment-GIB to squash out environment and finally introduce a cross-attention causal aggregation, allowing dynamic interactions between environments and invariances. We perform experiments on seven datasets and extensive experiments demonstrate strong generalization ability of EMoNet.

In material science, the properties of crystalline materials largely depend on their structures, and space group is a key descriptor of crystal structure. With the rapid advancement of deep learning, the traditional artificial structure analysis method based on X-ray diffraction (XRD) has become cumbersome and is being gradually supplanted by neural networks. However, existing models are too simplistic and lack a comprehensive understanding of material structure. Our approach XRDMamba integrates chemical knowledge and presents a fresh crystal planes perspective on XRD data. We also introduce a knowledge-driven model for space group identification tasks. We have thoroughly analyzed our approach through numerous experiments, observing its SOTA performance and excellent generalization capabilities. The code is available in ~https://github.com/baigeiguai/XRDMamba.

This paper explores the open research problem of understanding the social behaviors of LLM-based agents.Using Avalon as a testbed, we employ system prompts to guide LLM agents in gameplay.While previous studies have touched on gameplay with LLM agents, research on their social behaviors is lacking.We propose a novel framework, tailored for Avalon, features a multi-agent system facilitating efficient communication and interaction.We evaluate its performance based on game success and analyze LLM agents' social behaviors.Results affirm the framework's effectiveness in creating adaptive agents and suggest LLM-based agents' potential in navigating dynamic social interactions.By examining collaboration and confrontation behaviors, we offer insights into this field's research and applications.Our code is publicly available at https://github.com/ 3DAgentWorld/LLM-Game-Agent.

Seeking for generalizable graph representations becomes hot spot in the area of graph learning. Recently, causality theory has been applied for extracting the causal relations between graph data and labels, which are generalizable under distribution shift and result in better OOD generalization. In this paper, for more accurately capturing causal representation of graph data, we propose a gradient reactivation enhanced causal subgraph extraction method. The proposed model utilizes attention mechanism to extract the causal features and attenuates the confounding effect of shortcut features. For ensuring stability of extracted causal features, we propose a novel gradient reactivation method to filter features with greater effect on making prediction. Extensively experimental result proves the effectiveness of the proposed model.

Graph neural networks (GNNs) have achieved great success in graph classification tasks across many domains. However, the varying quality of real-world graph data leads to stability and reliability issues for real-world applications of graph neural networks (GNNs). Improving the robustness of GNNs would help enhance the quality and safety of GNNs in real-world applications. Recently, there have been studies that incorporate insights from information theory, causal theory, etc. into graph classification tasks to improve robustness. However, these strategies rely on extensive task-specific designs that increase model complexity and limit the scope of the methods. In this work, we leverage the interdependence between model stability and robustness by introducing stability constraints to graph neural network models through two different consistency regularization methods. To balance the trade-off between stability constraints and classification performance, we adaptively adjust the strength of the constraints dynamically using multi-objective optimization, making our method applicable to graph classification tasks of varying scales and domains. Extensive experiments on graph datasets from different domains demonstrate the superiority of our proposed method.

Moiré patterns are commonly seen when taking photos of screens. Camera devices usually have limited hardware performance but take high-resolution photos. However, users are sensitive to the photo processing time, which presents a hardly considered challenge of efficiency for demoiréing methods. To balance the network speed and quality of results, we propose a Fully Connected enCoder-deCoder based Demoiréing Network (FC3DNet). FC3DNet utilizes features with multiple scales in each stage of the decoder for comprehensive information, which contains long-range patterns as well as various local moiré styles that both are crucial aspects in demoiréing. Besides, to make full use of multiple features, we design a Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency. These designs enable our network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.

Spatio-temporal (ST) prediction plays a pivotal role in earth sciences, such as meteorological prediction, urban computing. Adequate high-quality data, coupled with deep models capable of inference, are both indispensable and prerequisite for achieving meaningful results. However, the sparsity of data and the high costs associated with deploying sensors lead to significant data imbalances. Models that are overly tailored and lack causal relationships further compromise the generalizabilities of inference methods. Towards this end, we first establish a causal concept for ST predictions, named NuwaDynamics, which targets to identify causal regions in data and endow model with causal reasoning ability in a two-stage process. Concretely, we initially leverage upstream self-supervision to discern causal important patches, imbuing the model with generalized information and conducting informed interventions on complementary trivial patches to extrapolate potential test distributions. This phase is referred to as the discovery step. Advancing beyond discovery step, we transfer the data to downstream tasks for targeted ST objectives, aiding the model in recognizing a broader potential distribution and fostering its causal perceptual capabilities (refer as Update step). Our concept aligns seamlessly with the contemporary backdoor adjustment mechanism in causality theory. Extensive experiments on six real-world ST benchmarks showcase that models can gain outcomes upon the integration of the NuwaDynamics concept. NuwaDynamics also can significantly benefit a wide range of changeable ST tasks like extreme weather and long temporal step super-resolution predictions.

Real-world tasks are universally associated with training samples that exhibit a long-tailed class distribution, and traditional deep learning models are not suitable for fitting this distribution, thus resulting in a biased trained model. To surmount this dilemma, massive deep long-tailed learning studies have been proposed to achieve inter-class fairness models by designing sophisticated sampling strategies or improving existing model structures and loss functions. Habitually, these studies tend to apply data augmentation strategies to improve the generalization performance of their models. However, this augmentation strategy applied to balanced distributions may not be the best option for long-tailed distributions. For a profound understanding of data augmentation, we first theoretically analyze the gains of traditional augmentation strategies in long-tailed learning, and observe that augmentation methods cause the long-tailed distribution to be imbalanced again, resulting in an intertwined imbalance: inherent data-wise imbalance and extrinsic augmentation-wise imbalance, i.e., two 'birds' co-exist in long-tailed learning. Motivated by this observation, we propose an adaptive Dynamic Optional Data Augmentation (DODA) to address this intertwined imbalance, i.e., one 'stone' simultaneously 'kills' two 'birds', which allows each class to choose appropriate augmentation methods by maintaining a corresponding augmentation probability distribution for each class during training. Extensive experiments across mainstream long-tailed recognition benchmarks (e.g., CIFAR-100-LT, ImageNet-LT, and iNaturalist 2018) prove the effectiveness and flexibility of the DODA in overcoming the intertwined imbalance.

Graph Neural Networks (GNNs) have emerged as the leading deep learning models for graph-based representation learning. However, the training and inference of GNNs on large graphs remain resource-intensive, impeding their utility in real-world scenarios and curtailing their applicability in deeper and more sophisticated GNN architectures. To address this issue, the Graph Lottery Ticket (GLT) hypothesis assumes that GNN with random initialization harbors a pair of core subgraph and sparse subnetwork, which can yield comparable performance and higher efficiency to that of the original dense network and complete graph. Despite that GLT offers a new paradigm for GNN training and inference, existing GLT algorithms heavily rely on trial-and-error pruning rate tuning and scheduling, and adhere to an irreversible pruning paradigm that lacks elasticity. Worse still, current methods suffer scalability issues when applied to deep GNNs, as they maintain the same topology structure across all layers. These challenges hinder the integration of GLT into deeper and larger-scale GNN contexts. To bridge this critical gap, this paper introduces an $\textbf{A}$daptive, $\textbf{D}$ynamic, and $\textbf{A}$utomated framework for identifying $\textbf{G}$raph $\textbf{L}$ottery $\textbf{T}$ickets ($\textbf{AdaGLT}$). Our proposed method derives its key advantages and addresses the above limitations through the following three aspects: 1) tailoring layer-adaptive sparse structures for various datasets and GNNs, thus endowing it with the capability to facilitate deeper GNNs; 2) integrating the pruning and training processes, thereby achieving a dynamic workflow encompassing both pruning and restoration; 3) automatically capturing graph lottery tickets across diverse sparsity levels, obviating the necessity for extensive pruning parameter tuning. More importantly, we rigorously provide theoretical proofs to guarantee $\textbf{AdaGLT}$ to mitigate over-smoothing issues and obtain improved sparse structures in deep GNN scenarios. Extensive experiments demonstrate that $\textbf{AdaGLT}$ outperforms state-of-the-art competitors across multiple graph datasets of various scales and types, particularly in scenarios involving deep GNNs.

Real-world data generally follows a long-tailed distribution, which makes traditional high-performance training strategies unable to show their usual effects. Various insights have been proposed to alleviate this challenging distribution. However, some observations indicate that models trained on long-tailed distributions always show a trade-off between the performance of head and tail classes. For a profound understanding of the trade-off, we first theoretically analyze the trade-off problem in long-tailed learning and creatively transform the trade-off problem in long-tailed learning into a multi-objective optimization (MOO) problem. Motivated by these analyses, we propose the idea of strategy fusion for MOO long-tailed learning and point out the potential conflict problem. We further design a Multi-Objective Optimization based Strategy Fusion (MOOSF), which effectively resolves conflicts, and achieves an efficient fusion of heterogeneous strategies. Comprehensive experiments on mainstream datasets show that even the simplest strategy fusion can outperform complex long-tailed strategies. More importantly, it provides a new perspective for generalized long-tailed learning. The code is available in the accompanying supplementary materials.

Motion forecasting plays a pivotal role in autonomous driving systems, enabling vehicles to execute collision warnings and rational local-path planning based on predictions of the surrounding vehicles. However, prevalent methods often assume complete observed trajectories, neglecting the potential impact of missing values induced by object occlusion, scope limitation, and sensor failures. Such oversights inevitably compromise the accuracy of trajectory predictions. To tackle this challenge, we propose an end-to-end framework, termed Multi-scale Transformer (MSTF), meticulously crafted for incomplete trajectory prediction. MSTF integrates a Multiscale Attention Head (MAH) and an Information Increment-based Pattern Adaptive (IIPA) module. Specifically, the MAH component concurrently captures multiscale motion representation of trajectory sequence from various temporal granularities, utilizing a multi-head attention mechanism. This approach facilitates the modeling of global dependencies in motion across different scales, thereby mitigating the adverse effects of missing values. Additionally, the IIPA module adaptively extracts continuity representation of motion across time steps by analyzing missing patterns in the data. The continuity representation delineates motion trend at a higher level, guiding MSTF to generate predictions consistent with motion continuity. We evaluate our proposed MSTF model using two large-scale real-world datasets. Experimental results demonstrate that MSTF surpasses state-of-the-art (SOTA) models in the task of incomplete trajectory prediction, showcasing its efficacy in addressing the challenges posed by missing values in motion forecasting for autonomous driving systems.

Despite Graph Neural Networks (GNNs) demonstrating considerable promise in graph representation learning tasks, GNNs predominantly face significant issues with overfitting and over-smoothing as they go deeper as models of computer vision (CV) realm.The success of artificial intelligence in computer vision and natural language processing largely stems from its ability to train deep models effectively.We have thus conducted a systematic study on deep GNN models.Our findings indicate that the current success of deep GNNs primarily stems from (I) the adoption of innovations from CNNs, such as residual/skip connections, or (II) the tailor-made aggregation algorithms like DropEdge.However, these algorithms often lack intrinsic interpretability and indiscriminately treat all nodes within a given layer in a similar manner, thereby failing to capture the nuanced differences among various nodes.In this paper, we introduce the Snowflake Hypothesis -a novel paradigm underpinning the concept of "one node, one receptive field".The hypothesis draws inspiration from the unique and individualistic patterns of * Contribute equally to this research.

Traffic prediction is a crucial task in the Intelligent Transportation System (ITS), receiving significant attention from both industry and academia. Numerous spatio-temporal graph convolutional networks have emerged for traffic prediction and achieved remarkable success. However, these models have limitations in terms of generalization and scalability when dealing with Out-of-Distribution (OOD) graph data with both structural and temporal shifts. To tackle the challenges of spatio-temporal shift, we propose a framework called STONE by learning invariable node dependencies, which achieve stable performance in variable environments. STONE initially employs gated-transformers to extract spatial and temporal semantic graphs. These two kinds of graphs represent spatial and temporal dependencies, respectively. Then we design three techniques to address spatio-temporal shifts. Firstly, we introduce a Fréchet embedding method that is insensitive to structural shifts, and this embedding space can integrate loose position dependencies of nodes within the graph. Secondly, we propose a graph intervention mechanism to generate multiple variant environments by perturbing two kinds of semantic graphs without any data augmentations, and STONE can explore invariant node representation from environments. Finally, we further introduce an explore-to-extrapolate risk objective to enhance the variety of generated environments. We conduct experiments on multiple traffic datasets, and the results demonstrate that our proposed model exhibits competitive performance in terms of generalization and scalability.

The long-tailed distribution is the underlying nature of real-world data, and it presents unprecedented challenges for training deep learning models. Existing long-tailed learning paradigms based on re-balancing or data augmentation have partially alleviated the long-tailed problem. However, they still have limitations, such as relying on manually designed augmentation strategies, having a limited search space, and using fixed augmentation strategies. To address these limitations, this paper proposes a novel LLM-based long-tailed data augmentation framework called LLM-AutoDA, which leverages large-scale pretrained models to automatically search for the optimal augmentation strategies suitable for long-tailed data distributions. In addition, it applies this strategy to the original imbalanced data to create an augmented dataset and fine-tune the underlying long-tailed learning model. The performance improvement on the validation set serves as a reward signal to update the generation model, enabling the generation of more effective augmentation strategies in the next iteration. We conducted extensive experiments on multiple mainstream long-tailed learning benchmarks. The results show that LLM-AutoDA outperforms state-of-the-art data augmentation methods and other re-balancing methods significantly.

Out-of-distribution (OOD) generalization issue is a well-known challenge within deep learning tasks. In dynamic graphs, the change of temporal environments is regarded as the main cause of data distribution shift. While numerous OOD studies focusing on environment factors have achieved remarkable performance, they still fail to systematically solve the two issue of environment inference and utilization. In this work, we propose a novel dynamic graph learning model named EpoD based on prompt learning and structural causal model to comprehensively enhance both environment inference and utilization. Inspired by the superior performance of prompt learning in understanding underlying semantic and causal associations, we first design a self-prompted learning mechanism to infer unseen environment factors. We then rethink the role of environment variable within spatio-temporal causal structure model, and introduce a novel causal pathway where dynamic subgraphs serve as mediating variables. The extracted dynamic subgraph can effectively capture the data distribution shift by incorporating the inferred environment variables into the node-wise dependencies. Theoretical discussions and intuitive analysis support the generalizability and interpretability of EpoD. Extensive experiments on seven real-world datasets across domains showcase the superiority of EpoD against baselines, and toy example experiments further verify the powerful interpretability and rationality of our EpoD.

Spatiotemporal learning has become a pivotal technique to enable urban intelligence. Traditional spatiotemporal models mostly focus on a specific task by assuming a same distribution between training and testing sets. However, given that urban systems are usually dynamic, multi-sourced with imbalanced data distributions, current specific task-specific models fail to generalize to new urban conditions and adapt to new domains without explicitly modeling interdependencies across various dimensions and types of urban data. To this end, we argue that there is an essential to propose a Continuous Multi-task Spatio-Temporal learning framework (CMuST) to empower collective urban intelligence, which reforms the urban spatiotemporal learning from single-domain to cooperatively multi-dimensional and multi-task learning. Specifically, CMuST proposes a new multi-dimensional spatiotemporal interaction network (MSTI) to allow cross-interactions between context and main observations as well as self-interactions within spatial and temporal aspects to be exposed, which is also the core for capturing task-level commonality and personalization. To ensure continuous task learning, a novel Rolling Adaptation training scheme (RoAda) is devised, which not only preserves task uniqueness by constructing data summarization-driven task prompts, but also harnesses correlated patterns among tasks by iterative model behavior modeling. We further establish a benchmark of three cities for multi-task spatiotemporal learning, and empirically demonstrate the superiority of CMuST via extensive evaluations on these datasets. The impressive improvements on both few-shot streaming data and new domain tasks against existing SOAT methods are achieved. Code is available at https://github.com/DILab-USTCSZ/CMuST.

Traditional long-tailed learning methods often perform poorly when dealing with inconsistencies between training and test data distributions, and they cannot flexibly adapt to different user preferences for trade-offs between head and tail classes. To address this issue, we propose a novel long-tailed learning paradigm that aims to tackle distribution shift in real-world scenarios and accommodate different user preferences for the trade-off between head and tail classes. We generate a set of diverse expert models via hypernetworks to cover all possible distribution scenarios, and optimize the model ensemble to adapt to any test distribution. Crucially, in any distribution scenario, we can flexibly output a dedicated model solution that matches the user's preference. Extensive experiments demonstrate that our method not only achieves higher performance ceilings but also effectively overcomes distribution shift while allowing controllable adjustments according to user preferences. We provide new insights and a paradigm for the long-tailed learning problem, greatly expanding its applicability in practical scenarios. The code can be found here: https://github.com/DataLab-atom/PRL.

Spatiotemporal traffic forecasting plays a critical role in intelligent transportation systems, which empowers diverse urban services. Existing traffic forecasting frameworks usually devise various learning strategies to capture spatiotemporal correlations from the perspective of volume itself. However, we argue that previous traffic predictions are still unreliable due to two aspects. First, the influences of context factor-wise interactions on dynamic region-wise correlations are under exploitation. Second, the dynamics induce the credibility issue of forecasting that has not been well-explored. In this paper, we exploit the informative traffic-related context factors to jointly tackle the dynamic regional heterogeneity and explain the stochasticity, towards a credible uncertainty-aware traffic forecasting. Specifically, to internalize the dynamic contextual influences into learning process, we design a context-cross relational embedding to capture interactions between each context, and generate virtual graph topology to dynamically relate pairwise regions with context embedding. To quantify the prediction credibility, we attribute data-side aleatoric uncertainty to contexts and re-utilize them for aleatoric uncertainty quantification. Then we couple a dual-pipeline learning with the same objective to produce the discrepancy of model outputs and quantify model-side epistemic uncertainty. These two uncertainties are fed through a spatiotemporal network for extracting uncertainty evolution patterns. Finally, comprehensive experiments and model deployments have corroborated the credibility of our framework.

Graph Neural Networks (GNNs) can learn representative graph-level features to achieve efficient graph classification. But GNNs usually assume an environment where both class and structure distribution are balanced. Although previous works have considered the graph classification problem under the scenario of class imbalance or structure imbalance, they habitually ignored the obvious fact that class imbalance and structural imbalance are often intertwined in the real world. In this paper, we propose a carefully designed structure-driven learning framework called ImbGNN to address the potential intertwined class imbalance and structural imbalance in graph classification. Specifically, we find that feature-oriented augmentation (e.g., feature masking) and structure-oriented augmentation (e.g., edge perturbation) will have differential impacts when applied to different graphs. Therefore, we design optional augmentation based on the average degree distribution to alleviate structural imbalance. Furthermore, based on the imbalance of graph size distribution, we utilize a similarity-friendly graph random walk to extract a core subgraph to improve the accuracy of graph kernel similarity calculation, and then construct a more reasonable kernel-based graph of graphs, thereby alleviating the class imbalance and size imbalance. Extensive experiments on multiple benchmark datasets demonstrate that our proposed ImbGNN framework outperforms previous baselines on imbalanced graph classification tasks. The code of ImbGNN is available in~https://github.com/Xiaovy/ImbGNN.

The training and inference of Graph Neural Networks (GNNs) are costly when scaling up to large-scale graphs. Graph Lottery Ticket (GLT) has presented the first attempt to accelerate GNN inference on large-scale graphs by jointly pruning the graph structure and the model weights. Though promising, GLT encounters robustness and generalization issues when deployed in real-world scenarios, which are also long-standing and critical problems in deep learning ideology. In real-world scenarios, the distribution of unseen test data is typically diverse. We attribute the failures on out-of-distribution (OOD) data to the incapability of discerning causal patterns, which remain stable amidst distribution shifts. In traditional spase graph learning, the model performance deteriorates dramatically as the graph/network sparsity exceeds a certain high level. Worse still, the pruned GNNs are hard to generalize to unseen graph data due to limited training set at hand. To tackle these issues, we propose the Resilient Graph Lottery Ticket (RGLT) to find more robust and generalizable GLT in GNNs. Concretely, we reactivate a fraction of weights/edges by instantaneous gradient information at each pruning point. After sufficient pruning, we conduct environmental interventions to extrapolate potential test distribution. Finally, we perform last several rounds of model averages to further improve generalization. We provide multiple examples and theoretical analyses that underpin the universality and reliability of our proposal. Further, RGLT has been experimentally verified across various independent identically distributed (IID) and out-of-distribution (OOD) graph benchmarks.

Although existing face forgery detection methods achieve satisfactory performance under closed within-dataset scenario where training and testing sets are created by the same manipulation technique, they are vulnerable to samples created by unseen manipulation techniques under cross-dataset scenario. To solve this problem, in this work, we propose a novel adaptive texture and spectral clue mining (ATSC) approach for generalizable face forgery detection. It adaptively adjusts the parameters depended on input images to mine specific intrinsic forgery clues on both spatial and frequency domains. Specifically, ATSC customizes a Texture Clue Mining Module and a Spectrum Clue Selecting Module. The former module exploits instance-aware dynamic convolution on spatial domain to dynamically assemble multiple parallel convolutional kernels based on the learned image-dependent attention maps for effectively capturing subtle texture artifacts on spatial domain. A customized attention loss is also applied to the attention maps as supervision to precisely localize forgery artifacts whilst retain useful background information from suspicious and non-suspicious regions, which drives the module to explore all potential crucial clues and learning robust texture-related forgery feature. Moreover, the latter module applies adaptive frequency filtering mechanism on DCT-based frequency signals, which selects frequency information of interest to capture refined spectrum clues on frequency domain in an input-adaptive manner. Equipped with the above two modules, ATSC can learn more generalizable forgery features for face forgery detection. Extensive experimental results demonstrate the superior generalization ability of the proposed ATSC over various state-of-the-art methods on the challenging benchmarks.

Event cameras respond to temporal dynamics, helping to resolve ambiguities in spatio-temporal changes for optical flow estimation. However, the unique spatio-temporal event distribution challenges the feature extraction, and the direct construction of motion representation through the orthogonal view is less than ideal due to the entanglement of appearance and motion. This paper proposes to transform the orthogonal view into a motion-dependent one for enhancing event-based motion representation and presents a Motion View-based Network (MV-Net) for practical optical flow estimation. Specifically, this motion-dependent view transformation is achieved through the Event View Transformation Module, which captures the relationship between the steepest temporal changes and motion direction, incorporating these temporal cues into the view transformation process for feature gathering. This module includes two phases: extracting the temporal evolution clues by central difference operation in the extraction phase and capturing the motion pattern by evolution-guided deformable convolution in the perception phase. Besides, the MV-Net constructs an eccentric downsampling process to avoid response weakening from the sparsity of events in the downsampling stage. The whole network is trained end-to-end in a self-supervised manner, and the evaluations conducted on four challenging datasets reveal the superior performance of the proposed model compared to state-of-the-art (SOTA) methods.

Traffic object detection under variable illumination is challenging due to the information loss caused by the limited dynamic range of conventional frame-based cameras. To address this issue, we introduce bio-inspired event cameras and propose a novel Structure-aware Fusion Network (SFNet) that extracts sharp and complete object structures from the event stream to compensate for the lost information in images through cross-modality fusion, enabling the network to obtain illumination-robust representations for traffic object detection. Specifically, to mitigate the sparsity or blurriness issues arising from diverse motion states of traffic objects in fixed-interval event sampling methods, we propose the Reliable Structure Generation Network (RSGNet) to generate Speed Invariant Frames (SIF), ensuring the integrity and sharpness of object structures. Next, we design a novel Adaptive Feature Complement Module (AFCM) which guides the adaptive fusion of two modality features to compensate for the information loss in the images by perceiving the global lightness distribution of the images, thereby generating illumination-robust representations. Finally, considering the lack of large-scale and high-quality annotations in the existing event-based object detection datasets, we build a DSEC-Det dataset, which consists of 53 sequences with 63,931 images and more than 208,000 labels for 8 classes. Extensive experimental results demonstrate that our proposed SFNet can overcome the perceptual boundaries of conventional cameras and outperform the frame-based methods, e.g., YOLOX by 7.9% in mAP50 and 3.8% in mAP50:95. Our code and dataset will be available athttps://github.com/YN-Yang/SFNet.

Traffic forecasting is a challenging research topic due to the complex spatial and temporal dependencies among different roads. Though great efforts have been made on traffic forecasting, existing works still have the following shortcomings: i) Most methods only directly perform on the original road network topology which cannot accommodate the diverse traffic patterns and multi-granularity traffic forecasting requirements driven by the natural multi-level urban structure and layout, ii) The existing studies based on the spatio-temporal multi-granularity perspective ignore the interactions between the fine-grained information and coarse-grained information, resulting in the spatio-temporal correlation under multi-granularity inaccurately modeled. To solve the problems, we propose an Adaptive and Interactive Multi-level Spatio-Temporal network (AIMST) for traffic forecasting. Specifically, we first devise a learnable adaptive hierarchical clustering method to automatically generate more coarse-grained graphs from the initial road networks and the traffic data. Then, the spatio-temporal graph convolutional networks are executed on the constructed hierarchical traffic graph of each level correspondingly to capture the spatio-temporal patterns. Furthermore, a multi-level bidirectional interaction module is designed to emphasize the multi-grained interaction patterns among different levels. Extensive experiments on two real-world traffic datasets demonstrate that our framework is superior to several state-of-the-art baselines.

In this paper, we address the issue of modeling and estimating changes in the state of the spatio-temporal dynamical systems based on a sequence of observations like video frames. Traditional numerical simulation systems depend largely on the initial settings and correctness of the constructed partial differential equations (PDEs). Despite recent efforts yielding significant success in discovering data-driven PDEs with neural networks, the limitations posed by singular scenarios and the absence of local insights prevent them from performing effectively in a broader real-world context. To this end, this paper propose the universal expert module – that is, optical flow estimation component, to capture the evolution laws of general physical processes in a data-driven fashion. To enhance local insight, we painstakingly design a finer-grained physical pipeline, since local characteristics may be influenced by various internal contextual information, which may contradict the macroscopic properties of the whole system. Further, we harness currently popular neural discrete learning to unveil the underlying important features in its latent space, this process better injects interpretability, which can help us obtain a powerful prior over these discrete random variables. We conduct extensive experiments and ablations to demonstrate that the proposed framework achieves large performance margins, compared with the existing SOTA baselines.

Spatio-temporal mobility modeling plays a pivotal role in the advancement of mobile computing. Nowadays, data is frequently held by various distributed silos, which are isolated from each other and confront limitations on data sharing. Given this, there have been some attempts to introduce federated learning into spatio-temporal mobility modeling. Meanwhile, the distributional heterogeneity inherent in the spatio-temporal data also puts forward requirements for model personalization. However, the existing methods tackle personalization in a model-centric manner and fail to explore the data characteristics in various data silos, thus ignoring the fact that the fundamental cause of insufficient personalization in the model is the heterogeneous distribution of data. In this paper, we propose a novel distribution-oriented personalizedFederated learning framework forCross-siloSpatio-Temporal mobility modeling (namedFedCroST), that leverages learnable spatio-temporal prompts to implicitly represent the local data distribution patterns of data silos and guide the local models to learn the personalized information. Specifically, we focus on the potential characteristics within temporal distribution and devise a conditional diffusion module to generate temporal prompts that serve as guidance for the evolution of the time series. Simultaneously, we emphasize the structure distribution inherent in node neighborhoods and propose adaptive spatial structure partition to construct the spatial prompts, augmenting the spatial information representation. Furthermore, we introduce a denoising autoencoder to effectively harness the learned multi-view spatio-temporal features and obtain personalized representations adapted to local tasks. Our proposal highlights the significance of latent spatio-temporal data distributions in enabling personalized federated spatio-temporal learning, providing new insights into modeling spatio-temporal mobility in data silo scenarios. Extensive experiments conducted on real-world datasets demonstrate that FedCroST outperforms the advanced baselines by a large margin in diverse cross-silo spatio-temporal mobility modeling tasks.

Human mobility forecasting is the key to energizing considerable mobile computing services. However, we find that the collective mobility suffers the spatiotemporal heterogeneity issue and therefore leads to inferior performances of conventional homogeneous aggregations. Given two fundamental factors, i.e., data and objectives in machine learning, we propose to counter such heterogeneity by improving data utilization and optimization objectives. 1) From data utilization perspective, we discover that such heterogeneity is inherently induced by mobility-related context factors and thus these factors can be exploited to learn heterogeneous mobility patterns. 2) From the optimization perspective, the dependencies among output elements, which give another prior to learning, can extract heterogeneous correlations within output sequences. Specifically, we propose a novel Context-Directional SpatioTemporal Graph Network (CD-STGNet), which tackles the above-mentioned heterogeneity, for achieving accurate mobility predictions. Firstly, we improve data utilization by inputting the encoded context-wise interactions to a direction field learner, which realizes directional spatial aggregations. Secondly, regarding series learning and optimization objectives, a context-trend highway is designed to enable context-aware temporal learning while two regularization objectives are proposed to keep the correlations among predicted elements consistent with the ground-truth. Experiments demonstrate that CD-STGNet surpasses competitive baselines by 13% to 22% and boosts the interpretability of context-directional learning.

Event cameras, such as dynamic vision sensors (DVS), are biologically inspired vision sensors that have advanced over conventional cameras in high dynamic range, low latency and low power consumption, showing great application potential in many fields. Event cameras are more sensitive to junction leakage current and photocurrent as they output differential signals, losing the smoothing function of the integral imaging process in the RGB camera. The logarithmic conversion further amplifies noise, especially in low-contrast conditions. Recently, researchers proposed a series of datasets and evaluation metrics but limitations remain: 1) the existing datasets are small in scale and insufficient in noise diversity, which cannot reflect the authentic working environments of event cameras; and 2) the existing denoising evaluation metrics are mostly referenced evaluation metrics, relying on APS information or manual annotation. To address the above issues, we construct a large-scale event denoising dataset (multilevel benchmark for event denoising, E-MLB) for the first time, which consists of 100 scenes, each with four noise levels, that is 12 times larger than the largest existing denoising dataset. We also propose the first nonreference event denoising metric, the event structural ratio (ESR), which measures the structural intensity of given events. ESR is inspired by the contrast metric, but is independent of the number of events and projection direction. Based on the proposed benchmark and ESR, we evaluate the most representative denoising algorithms, including classic and SOTA, and provide denoising baselines under various scenes and noise levels. The corresponding results and codes are available athttps://github.com/KugaMaxx/cuke-emlb.

JPEG is a widely used compression scheme to efficiently reduce the volume of the transmitted images at the expense of visual perception drop. The artifacts appear among blocks due to the information loss in the compression process, which not only affects the quality of images but also harms the subsequent high-level tasks in terms of feature drifting. High-level vision models trained on high-quality images will suffer performance degradation when dealing with compressed images, especially on mobile devices. In recent years, numerous learning-based JPEG artifacts removal methods have been proposed to handle visual artifacts. However, it is not an ideal choice to use these JPEG artifacts removal methods as a pre-processing for compressed image classification for the following reasons: 1) These methods are designed for human vision rather than high-level vision models. 2) These methods are not efficient enough to serve as a pre-processing on resource-constrained devices. To address these issues, this paper proposes a novel lightweight adaptive feature de-drifting module (AFD-Module) to boost the performance of pre-trained image classification models when facing compressed images. First, a Feature Drifting Estimation Network (FDE-Net) is devised to generate the spatial-wise Feature Drifting Map (FDM) in the DCT domain. Next, the estimated FDM is transmitted to the Feature Enhancement Network (FE-Net) to generate the mapping relationship between degraded features and corresponding high-quality features. Specially, a simple but effective RepConv block equipped with structural re-parameterization is utilized in FE-Net, which enriches feature representation in the training phase while keeping efficiency in the deployment phase. After training on limited compressed images, the AFD-Module can serve as a “plug-and-play” module for pre-trained classification models to improve their performance on compressed images. Experiments on images compressed once (i.e.ImageNet-C) and multiple times demonstrate that our proposed AFD-Module can comprehensively improve the accuracy of the pre-trained classification models and significantly outperform the existing methods.

Graph Neural Networks (GNNs), as the dominant approach in Molecular Representation Learning (MRL), have exhibited remarkable efficacy in diverse tasks, including molecular property prediction and drug discovery. Considering the dynamic and diverse nature of test molecules in real-world contexts, the previous independently and identically distributed (i.i.d.) assumption for training and test molecules is not align with the requirements of practical applications in chemistry. Rationalization has been proposed to enhance the Out-of-Distribution (OOD) generalization of GNN, but a critical concern remains unexplored: low efficiency in rationale discovery. We consider this bottleneck is due to large search space and overly flexible modeling. Here, we introduce a framework called Molecule Stratifier and Invariant Rational Improver (MSIRI) to overcome the challenge. Specifically, MSIRI adopt vector quantization to obtain invariant rationales and the search space is narrowed by substructure and two assumptions based on subgraph matching. Experimental results on ten datasets demonstrate that our approach both achieves significant improvements across various GNN backbones and outperforms other six Out-of-Distribution method, clearly demonstrating its effectiveness in addressing the OOD challenges in MRL.

Profile-Guided Optimization (PGO) is a novel compiler optimization leveraging runtime feedback and has been applied successfully to optimize Android kernels gaining significant performance improvements. However, current studies as well as implementations of PGO-based Android kernel optimizations still suffer from three problems: 1) optimization inflexibility due to restricted algorithms for generating profiles and for simulating real-world usage scenarios; 2) considerable optimization efforts due to the extensive manual interventions needed; and 3) optimization failures due to kernel version fragmentation.This paper presents MePof, the first modular and end-to-end PGO framework for Android kernels. The MePof framework consists of three key components: 1) a tool orchestration, that integrates two novel algorithms for generating profiles, and three methods for simulating real-world scenarios that can be flexibly switched according to the usage scenario; 2) a domain-specific language (DSL) that can specify PGO-based optimization strategies and a corresponding compiler translating the DSL programs into configuration files necessary for optimization; and 3) an adapter that automatically triggers and completes the optimization when the Android kernel version changes.We have implemented a prototype for MePof and have conducted extensive experiments to evaluate its effectiveness, performance, and usability. Experimental results demonstrated that: 1) MePof is effective, with performance improvement 9.39% on average; 2) MePof is efficient by saving up to 39.07% of time than manual optimizations; and 3) MePof is highly usable by requiring only one manual intervention instead of more than 30 manual interventions as in the existing optimization framework.

The images captured under improper exposure conditions often suffer from contrast degradation and detail distortion. Contrast degradation will destroy the statistical properties of low-frequency components, while detail distortion will disturb the structural properties of high-frequency components, leading to the low-frequency and high-frequency components being mixed and inseparable. This will limit the statistical and structural modeling capacity for exposure correction. To address this issue, this paper proposes to decouple the contrast enhancement and detail restoration within each convolution process. It is based on the observation that, in the local regions covered by convolution kernels, the feature response of low-/high-frequency can be decoupled by addition/difference operation. To this end, we inject the addition/difference operation into the convolution process and devise a Contrast Aware (CA) unit and a Detail Aware (DA) unit to facilitate the statistical and structural regularities modeling. The proposed CA and DA can be plugged into existing CNN-based exposure correction networks to substitute the Traditional Convolution (TConv) to improve the performance. Furthermore, to maintain the computational costs of the network without changing, we aggregate two units into a single TConv kernel using structural re-parameterization. Evaluations of nine methods and five benchmark datasets demonstrate that our proposed method can comprehensively improve the performance of existing methods without introducing extra computational costs compared with the original networks. The codes will be publicly available.

Object detection is a fundamental task in computer vision, consisting of both classification and localization tasks. Previous works mostly perform classification and localization with shared feature extractor like Convolution Neural Network. However, the tasks of classification and localization exhibit different sensitivities with regard to the same feature, hence the "task spatial misalignment" issue. This issue can result in a hedge issue between the performances of localizer and classifier. To address these issues, we first propose a novel Dynamic Coefficient Loss to simultaneously consider and balance the performances of classification and localization tasks. To well address anchor label misjudgment issue in irregular- shaped object detection, we define a new classification-aware IoU metric to assign anchors intelligently. Finally, we further introduce the localization factor into NMS by proposing a Classification-Localization balanced NMS. Extensive experiments on MS COCO and PASCAL VOC demonstrate that our proposals can improve RetinaNet by around 1.5% AP with various backbones.

Recently, spatiotemporal graph convolutional networks are becoming popular in the field of traffic flow prediction and significantly improve prediction accuracy. However, the majority of existing traffic flow prediction models are tailored to static traffic networks and fail to model the continuous evolution and expansion of traffic networks. In this work, we move to investigate the challenge of traffic flow prediction on an expanding traffic network. And we propose an efficient and effective continual learning framework to achieve continuous traffic flow prediction without the access to historical graph data, namely Pattern Expansion and Consolidation based on Pattern Matching based (PECPM). Specifically, we first design a pattern bank based on pattern matching to store representative patterns of the road network. With the expansion of the road network, the model configured with such a bank module can achieve continuous traffic prediction by effectively managing patterns stored in the bank. The core idea is to continuously update new patterns while consolidating learned ones. Specifically, we design a pattern expansion mechanism that can detect evolved and new patterns from the updated network, then these unknown patterns are expanded into the pattern bank to adapt to the updated road network. Additionally, we propose a pattern consolidation mechanism that includes both a bank preservation mechanism and a pattern traceability mechanism. This can effectively consolidate the learned patterns in the bank without requiring access to detailed historical graph data. We construct experiments on real-world traffic datasets to demonstrate the competitive performance, superior efficiency, and strong generalization ability of PECPM.

Missing values, which are common in multivariate time series, is most important obstacle towards the utilization and interpretation of those data. Great efforts have been employed on how to accurately impute missing values in multivariate time series, and existing works either use deep learning networks to achieve deterministic imputations or aim at generating different plausible imputations by sampling multiple noises from a same distribution and then denoising them. However, these models either fall short of modeling the uncertainties of imputations due to their deterministic nature or perform poorly in terms of interpretability and imputation accuracy due to their ignorance of the correlations between the latent representations of both observed and missing values which are parts of samples from a same distribution. To this end, in this paper, we explicitly take the correlations between observed and missing values into account, and theoretically re-derive the Evidence Lower BOund (ELBO) of conditional diffusion model in the scenario of multivariate time series imputation. Based on the newly derived ELBO, we further propose a novel multivariate imputation diffusion model (MIDM) which is equipped with novel noise sampling, adding and denoising mechanisms for multivariate time series imputation, and the series of newly designed technologies jointly ensure the involving of the consistency between observed and missing values. Extensive experiments on both the tasks of multivariate time series imputation and forecasting witness the superiority of our proposed MIDM model on generating conditional estimations.

Subgraph learning has received considerable attention in its capacity of interpreting important structural information for predictions. Existing subgraph learning usually exploits statistics on predefined structures e.g., node degrees, occurrence frequency, to extract subgraphs, or refine the contents via only capturing label-relevant information with node-level sampling. Given diverse subgraph patterns, and mutual independence with local correlations on graphs, current solutions on subgraph learning still have two limitations in extraction and refinement stages. 1) The universality of extracting substructure patterns across domains is still lacking, 2) node-level sampling in refinement will distort the original local topology and none explicit guidance eliminating redundant information contribute to inefficiency issue. In this paper, we propose a unified subgraph learning scheme, Poly-Pivot Graph Neural Network (P2GNN) where we designate the centric node of each subgraph as the pivot. In the extraction stage, we present a general subgraph extraction principle, i.e., Local; Asymmetry between the centric and affiliated nodes. To this end, we asymmetrically model the similarity between each pair of nodes with random walk and quantify mutual affiliations in Affinity Propagation architecture, to extract subgraph structures. In the refinement, we devise a subgraph-level exclusion regularization to squash the target-independent information by considering mutual relations across subgraphs, cooperatively preserving a support set of subgraphs and facilitating the refinement process for graph representation. Empirical experiments on diverse web and biological graphs reveal 1.1%~7.3% improvements against best baselines, and visualized case studies prove the universality and interpretability of our P2GNN.

Spatiotemporal (ST) learning has become a crucial technique for urban digitalization. Due to expansions and dynamics of cities, current spatiotemporal models are inclined to suffer distribution shifts between training and testing sets, leading to the OOD delimma. However, few studies focus on such OOD problem in temporal regressions, let alone spatiotemporal learning. Spatiotemporal data usually reveals segment-level heterogeneity within periodicity and complex spatial dependencies, posing challenges to invariance extraction. In this paper, we find that ST relations make sense for generalization and devise a Causal ST learning framework, CauSTG, which enables invariant relation transferred to OOD scenarios. Specifically, we take temporal steps as environments, and transform spatial-temporal relations into learnable parameters. To tackle heterogeneity in periodicity, we partition temporal steps into sub-environments by identifying distinctive trend patterns, enabling re-organized samples trained separately. To extract invariance within ST observations, we propose a spatiotemporal consistency learner and a hierarchical invariance explorer to jointly filter out stable relations. Our spatiotemporal learner quantifies bi-directional spatial consistency and extracts disentangled seasonal-trend patterns via trainable parameters. Further, the hierarchical invariance explorer constructs variation-based filter to achieve both local and global invariances. Experiments reveal that CauSTG can increase at most 10.26% performance against best baselines, and visualized invariant relations can well interpret the physical rationales. The appendix and codes can be available in our Github repository.

Recently, multivariate time series (MTS) forecasting techniques have seen rapid development and widespread applications across various fields. Transformer-based and GNN-based methods have shown promising potential due to their strong ability to model interaction of time and variables. However, by conducting a comprehensive analysis of the real-world data, we observe that the temporal fluctuations and heterogeneity between variables are not well handled by existing methods. To address the above issues, we propose CrossGNN, a linear complexity GNN model to refine the cross-scale and cross-variable interaction for MTS. To deal with the unexpected noise in time dimension, an adaptive multi-scale identifier (AMSI) is leveraged to construct multi-scale time series with reduced noise. A Cross-Scale GNN is proposed to extract the scales with clearer trend and weaker noise. Cross-Variable GNN is proposed to utilize the homogeneity and heterogeneity between different variables. By simultaneously focusing on edges with higher saliency scores and constraining those edges with lower scores, the time and space complexity (i.e., $O(L)$) of CrossGNN can be linear with the input sequence length $L$. Extensive experimental results on 8 real-world MTS datasets demonstrate the effectiveness of CrossGNN compared with state-of-the-art methods.

Spatiotemporal forecasting is an imperative topic in data science due to its critical applications in smart cities. Existing works mostly perform consecutive predictions of following steps with observations continuously obtained, where nearest observations can be exploited as the key knowledge for status estimation. However, the practical issues of early activity planning and sensor failures elicit a new task, non-consecutive forecasting. In this paper, we define spatiotemporal learning systems with missing observations as Grey Spatiotemporal Systems (G2S) and propose a Factor-Decoupled learning framework for G2S to hierarchically decouple multi-level factors, and enable flexible aggregations with uncertainty estimations. We especially select representative sequences to capture periodicity and instantaneous variations, and infer the non-consecutive future statuses under expected exogenous factors, compensating the missing observations. Given the inherent incompleteness and critical applications of G2S, a DisEntangled Uncertainty Quantification is put forward, to identify two types of uncertainty for model interpretations and robustness promotions. Experiments demonstrate that our solution can promote the performance by at least 8.50% on early planning and 2.01%-18.00% on sensor failures. The appendix of this paper can be found at https://github.com/zzyy0929/SDM-G2S.

Spatiotemporal data forecasting is a fundamental task in the field of graph data mining. Typical spatiotemporal data prediction methods usually capture spatial dependencies by directly aggregating features of local neighboring vertices in a fixed graph. However, this kind of aggregators can only capture localized correlations between vertices, and while been stacked for larger receptive field, they fall into the dilemma of over-smoothing. Additional, in temporal perspective, traditional methods focus on fixed graphs, while the correlations among vertexes can be dynamic. And time series components integrated strategies in traditional spatiotemporal learning methods can hardly handle frequently and drastically changed sequences. To overcome those limitations of existing works, in this paper, we propose a novel multi-graph based dynamic learning framework. First, a novel Dynamic Neighbor Search (DNS) mechanism is introduced to model global dynamic correlations between vertices by constructing a feature graph (FG), where the adjacency matrix is dynamically determined by DNS. Then we further alleviate the over-smoothing issue with our newly designed Adaptive Heterogeneous Representation (AHR) module. Both FG and origin graph (OG) are fed into the AHR modules and fused in our proposed Multi-graph Fusion block. Additionally, we design a Differential Vertex Representation (DVR) module which takes advantage of differential information to model temporal trends. Extensive experiments illustrate the superior forecasting performances of our proposed multi-graph based dynamic learning framework on six real-world spatiotemporal datasets from different cities and domains, and this corroborates the solid effectiveness of our proposed framework and its superior generalization ability.

The rapid progress of machine learning (ML) in predicting molecular properties enables high-precision predictions being routinely achieved. However, many ML models, such as conventional molecular graph, cannot differentiate stereoisomers of certain types, particularly conformational and chiral ones that share the same bonding connectivity but differ in spatial arrangement. Here, we designed a hybrid molecular graph network, Chemical Feature Fusion Network (CFFN), to address the issue by integrating planar and stereo information of molecules in an interweaved fashion. The three-dimensional (3D, i.e., stereo) modality guarantees precision and completeness by providing unabridged information, while the two-dimensional (2D, i.e., planar) modality brings in chemical intuitions as prior knowledge for guidance. The zipper-like arrangement of 2D and 3D information processing promotes cooperativity between them, and their synergy is the key to our model's success. Experiments on various molecules or conformational datasets including a special newly created chiral molecule dataset comprised of various configurations and conformations demonstrate the superior performance of CFFN. The advantage of CFFN is even more significant in datasets made of small samples. Ablation experiments confirm that fusing 2D and 3D molecular graphs as unambiguous molecular descriptors can not only effectively distinguish molecules and their conformations, but also achieve more accurate and robust prediction of quantum chemical properties.

Accurate traffic prediction plays a vital role in intelligent transport managements and applications. However, in the vast majority of existing works, the focus is mainly on modeling spatiotemporal correlations in static traffic networks. Thus, the continuous expansion and evolution of traffic networks are ignored. In this work, we study the problem of traffic prediction with expanding road network structures under the continual learning paradigm. Considering the model prediction performance, efficiency, and data accessibility, a SpatioTemporal Knowledge Expansion and Consolidation (STKEC) framework is proposed. This framework contains an influence-based knowledge expansion strategy to help the spatiotemporal learning model integrate new spatiotemporal traffic patterns and a memory-augmented knowledge consolidation mechanism to preserve the learned spatiotemporal patterns without accessing the data in previous graphs. Extensive experiments are conducted on a large-scale dataset and verify the superior performance of STKEC in continual traffic prediction.

Urban housing price is widely accepted as an economic indicator which is of both business and research interest in urban computing. However, due to the complex nature of influencing factors and the sparse property of transaction records, to implement such a model is still challenging. To address these challenges, in this work, we study an effective and fine-grained model for urban subregion housing price predictions. Compared to existing works, our proposal improves the forecasting granularity from city-level to mile-level, with only publicly released transaction data. We employ a feature selection mechanism to select more relevant features. Then, we propose an integrated model, JGC MMN (Joint Gated Co-attention Based Multi-modal Network), to learn all-level features and capture spatiotemporal correlations in all-time stages with a modified densely connected convolutional network as well as current ingredients and future expectations. Next, we devise a novel JGC based fusion method to better fuse the heterogeneous data of multi-stage models by considering their interactions in temporal dimension. Finally, extensive empirical studies on real datasets demonstrate the effectiveness of our proposal, and this fine-grained housing price forecasting has the potential to support a broad scope of applications, ranging from urban planning to housing market recommendations.

Modern intelligent transportation system (ITS) has greatly benefitted people's daily life. However, the chanciness and suddenness of urban anomalies may greatly restrict the trouble-free operations of ITS. To be aware of future urban anomalies and their possible influences, great efforts have been achieved on these two aspects, but comprehensive predictions of urban anomalies including the predictions of distributions and durations, are still beingless. And the spatiotemporal cascade self/mutual exciting influences among anomalies have never been considered in previous studies. In this paper, we propose a novel Anomaly Distribution and Duration Joint-Prediction (A2DJP) algorithm to simultaneously filtrate urban subregions and estimate the duration of corresponding potential anomalies in the future. To capture the spatiotemporal correlations between urban traffics and anomalies, we use a modified Graph Convolution Network and Long Short-Term Memory integrated network. To learn the cascade correlations among anomalies themselves, we devise a novel Spatiotemporal neural Hawkes Process model, which contains a Hawkes Process (HP) based GCN and HP-based LSTM to extract the anomaly-wise spatiotemporal cascading correlations. By fusing the spatiotemporal correlations between traffics and anomalies, we then simultaneously predict the distributions and durations of future anomalies. Extensive experiments on real-world datasets demonstrate that our proposed method significantly outperforms state-of-the-art solutions.

Dynamic Vision Sensor (DVS) can asynchronously output the events reflecting apparent motion of objects with microsecond resolution, and shows great application potential in monitoring and other fields. However, the output event stream of existing DVS inevitably contains background activity noise (BA noise) due to dark current and junction leakage current, which will affect the temporal correlation of objects, resulting in deteriorated motion estimation performance. Particularly, the existing filter-based denoising methods cannot be directly applied to suppress the noise in event stream, since there is no spatial correlation. To address this issue, this paper presents a novel progressive framework, in which a Motion Estimation (ME) module and an Event Denoising (ED) module are jointly optimized in a mutually reinforced manner. Specifically, based on the maximum sharpness criterion, ME module divides the input event into several segments by adaptive clustering in a motion compensating warp field, and captures the temporal correlation of event stream according to the clustered motion parameters. Taking temporal correlation as guidance, ED module calculates the confidence that each event belongs to real activity events, and transmits it to ME module to update energy function of motion segmentation for noise suppression. The two steps are iteratively updated until stable motion segmentation results are obtained. Extensive experimental results on both synthetic and real datasets demonstrate the superiority of our proposed approaches against the State-Of-The-Art (SOTA) methods.

Images captured with improper exposures usually bring unsatisfactory visual effects. Previous works mainly focus on either underexposure or overexposure correction, resulting in poor generalization to various exposures. An alternative solution is to mix the multiple exposure data for training a single network. However, the procedures of correcting underexposure and overexposure to normal exposures are much different from each other, leading to large discrepancies for the network in correcting multiple-exposures, thus resulting in poor performance. The key point to address this issue lies in bridging different exposure representations. To achieve this goal, we design a multiple exposure correction framework based on an Exposure Normalization and Compensation (ENC) module. Specifically, the ENC module consists of an exposure normalization part for mapping different exposure features to the exposure-invariant feature space, and a compensation part for integrating the initial features unprocessed by the exposure normalization part to ensure the completeness of information. Besides, to further alleviate the imbalanced performance caused by variations in the optimization process, we introduce a parameter regularization fine-tuning strategy to improve the performance of the worst-performed exposure without degrading other exposures. Our model empowered by ENC outperforms the existing methods by more than 2dB and is robust to multiple image enhancement tasks, demonstrating its effectiveness and generalization capability for real-world applications. Code: https://github.com/KevinJ-Huang/ExposureNorm-Compensation.

Automatic lip-reading (ALR) aims to recognize words using visual information from the speaker's lip movements. In this work, we introduce a novel type of sensing device, event cameras, for the task of ALR. Event cameras have both technical and application advantages over conventional cameras for the ALR task because they have higher temporal resolution, less redundant visual information, and lower power consumption. To recognize words from the event data, we propose a novel Multi-grained Spatio-Temporal Features Perceived Network (MSTP) to perceive fine-grained spatio-temporal features from microsecond time-resolved event data. Specifically, a multi-branch network architecture is designed, in which different grained spatio-temporal features are learned by operating at different frame rates. The branch operating on the low frame rate can perceive spatial complete but temporal coarse features. While the branch operating on the high frame rate can perceive spatial coarse but temporal refinement features. And a message flow module is devised to integrate the features from different branches, leading to perceiving more discriminative spatio-temporal features. In addition, we present the first event-based lip-reading dataset (DVS-Lip) captured by the event camera. Experimental results demonstrated the superiority of the proposed model compared to the state-of-the-art event-based action recognition models and video-based lip-reading models.

Convolutional image deraining networks have achieved great success while suffering from tremendous computational and memory costs. Most model compression methods require original data for iterative fine-tuning, which is limited in real-world applications due to storage, privacy, and transmission constraints. We note that it is overstretched to fine-tune the compressed model using self-collected data, as it exhibits poor generalization over images with different degradation characteristics. To address this problem, we propose a novel data-free compression framework for de-raining networks. It is based on our observation that deep degradation representations can be clustered by degradation characteristics (types of rain) while independent of image content. Therefore, in our framework, we “dream” diverse in-distribution degraded images using a deep inversion paradigm, thus leveraging them to distill the pruned model. Specifically, we preserve the performance of the pruned model in a dual-branch way. In one branch, we invert the pre-trained model (teacher) to reconstruct the degraded inputs that resemble the original distribution and employ the orthogonal regularization for deep features to yield degradation diversity. In the other branch, the pruned model (student) is distilled to fit the teacher's original statistical modeling on these dreamed inputs. Further, an adaptive pruning scheme is proposed to determine the hierarchical sparsity, which alleviates the regression drift of the initial pruned model. Experiments on various deraining datasets demonstrate that our method can reduce about 40% FLOPs of the state-of-the-art models while maintaining comparable performance without original data.

An address, a textual description of a physical location, plays an important role in location-based services such as on-demand delivery and e-commerce. However, abnormal addresses (i.e., an address without detailed information representing a spatial location) have led to significant costs. In real-world settings like e-commerce, abnormal address detection is not trivial because it needs to be completed in real-time to support massive online queries. In this study, we design FastAddr, a fast abnormal address detection framework, which detects abnormal addresses among millions of addresses in a short time. By investigating and modeling the hierarchical structure of address data, we first design a novel contrastive address augmentation approach to generate training data via learning the entity transition probability matrix. We further design a lightweight multi-head attention model for learning compact address representation by modeling the address characteristics. We conduct a comprehensive three-phase evaluation. (i) We evaluate FastAddr on a real-world dataset and it yields the average F1 of 85.7% in 0.058 milliseconds, which outperforms the state-of-the-art models by 47.4% with similar detection time. (ii) An offline A/B test shows that FastAddr outperforms the previous deployed model significantly. (iii) We also conduct an online A/B test to compare FastAddr with the deployed model, which shows an improvement of F1 by more than 20%. Moreover, a real-world case study demonstrates both the efficiency and effectiveness of FastAddr.

Fusing multimodal heterogeneous data plays a vital role in recognition and prediction tasks in various fields, e.g., action recognition and traffic accident forecast. Yet, there remain some key challenges, such as heterogeneous feature interaction and feature redundancies, that significantly affect the performance of multimodal fusion. To tackle these challenges, we first devise a Unified Feature Interaction Module (UFIM) in which a novel orthogonal attention component is designed to obtain fine-grained inter-modal interaction information among heterogeneous features. Then, we propose a novel Self-Correcting Transformer Module (SCTM) which employs a modified transformer to obtain the one-to-many correlation information between the current modal feature and the merged features of other modalities to alleviate the redundancy problem. Extensive experiments on four cross-domain tasks demonstrate the effectiveness and generalization ability of our proposed method.

Wavelength-dependent light absorption and scattering result in the color cast and contrast degradation, which degrades the visibility of underwater images. Deep-learning-based under-water image enhancement has made significant progress in recent years. However, existing algorithms rely heavily on massive data for training, and the difficulty of collecting massive data in real-world environments limits their applicability. To alleviate this problem, we propose a fast adaptive self-supervised underwater image enhancement method, which models the entire water domain in terms of color and contrast distribution with only a few images. Specifically, to learn the mapping of color correction and contrast enhancement, our framework contains two sub-networks: a Color Mapping Net (CLM-Net) and a Contrast Mapping Net (CTM-Net). The CLM-Net learns the color correction mapping under the constraint of gray world assumption in a self-supervised manner. Besides, the CTM-Net estimates the contrast enhancement mapping under the constraint of the dark channel prior. Thanks to adaptive self-supervised training, we can primarily alleviate the need for collecting massive data and achieve effective underwater image enhancement with only ten images. Experiments demonstrate that the proposed algorithm achieves state-of-the-art performance on four benchmark datasets.

Rust is an emerging programming language designed for secure system programming that provides both security guarantees and runtime efficiency and has been increasingly used to build software infrastructures such as OS kernels, web browsers, databases, and blockchains. To support arbitrary low-level programming and to provide more flexibility, Rust introduced the unsafe feature, which may lead to security issues such as memory or concurrency vulnerabilities. Although there have been a significant number of studies on Rust security utilizing diverse techniques such as program analysis, fuzzing, privilege separation, and formal verification, existing studies suffer from three problems: 1) they only partially solve specific security issues but lack comprehensiveness; 2) most of them require manual interventions or annotations thus are not automated; and 3) they only cover a specific phase instead of the full lifecycle.In this perspective paper, we first survey current research progress on Rust security from 5 aspects, namely, empirical studies, vulnerability prevention, vulnerability detection, vulnerability rectification, and formal verification, and note the limitations of current studies. Then, we point out key challenges for Rust security. Finally, we offer our vision of a Rust security infrastructure guided by three principles: Comprehensiveness, Automation, and Lifecycle (CAL). Our work intends to promote the Rust security studies by proposing new research challenges and future research directions.

Rust is a new safe system programming language enforcing safety guarantees by novel language features, a rich type system, and strict compile-time checking rules, and thus has been used extensively to build system software. For multilingual Rust applications containing external C code, memory security vulnerabilities can occur due to the intrinsically unsafe nature of C and the improper interactions between Rust and C. Unfortunately, existing security studies on Rust only focus on pure Rust code but cannot analyze either the native C code or the Rust/C interactions in multilingual Rust applications. As a result, the lack of such studies may defeat the guarantee that Rust is a safe language.This paper presents CRust, a unified program analysis framework across Rust and C, which enables program analyses to understand the semantics of C code by translating Rust and C into a unified specification language. The CRust framework consists of three key components: (1) a unified specification language CRustIR, which is a strong-typed low-level intermediate language suitable for program analysis; (2) a transformation to build models of C code by converting C code into CRustIR; and (3) program analysis algorithms on CRustIR to detect security vulnerabilities. We have implemented a software prototype for CRust, and have conducted extensive experiments to evaluate its effectiveness and performance. Experimental results demonstrated that CRust can effectively detect common memory security vulnerabilities caused by the interaction of Rust and C that are missed by state-of-the-art tools. In addition, CRust is efficient in bringing negligible overhead (0.23 seconds on average).

With increasing population of modern cities, accurate estimation of regional passenger demands is critical to online taxicab services as such platforms aim at a reformation of taxicab scheduling for a more efficient order dispatching. Though great efforts have been made on passenger demand predictions, existing works still have the following shortcomings: i) they mostly performed based on uniform grid partition, which results in the imbalance of demand volumes among regions and even non-vehicle regions in such partition, ii) none of previous demand forecasting efforts have highlighted the important mutual influences between pick-ups and drop-offs, which are of great significance for taxicab scheduling. To this end, we first devise a multi-kernel based clustering to achieve a taxicab-behavior and geographic-aware sub-region partition, hence a more balanced and compact regional division is obtained. Subsequently, we emphasize the essential factors with regard to mutual transition quantification in taxicab predictions, then propose a Transfer-LSTM and an Origin-Destination-based transition matrix to respectively capture the drop-to-pick and pick-to-drop spatiotemporal transition patterns. Hence, a novel mutual-transition-aware co-prediction framework is devised by capturing complex spatiotemporal interactions between pick-ups and drop-offs. Extensive experiments on two real-world taxicab datasets demonstrate our co-prediction framework is superior to state-of-the-art methods, thus providing novel perspectives to urban human mobility understanding and transition-based taxicab scheduling.

Traffic accident has become a significant health and development threat with rapid urbanizations. An accurate urban accident forecasting enables higher-quality police force pre-allocation and safe route planning for both traffic administrations and travelers, maximumly reducing injuries and damages. Off-the-shelf short-term accident forecasting methods, which focus on modeling static region-wise correlations with existing neural networks, mostly performed on hour levels and with single step. However, given the dynamic nature of road networks and expanding urban areas, it is challenging when the spatiotemporal granularity of forecasting improves as the rareness of accident records and complexity of long-term future dependencies. To address these challenges, we propose a unified framework RiskSeq, to foresee sparse urban accidents with finer granularities and multiple steps in spatiotemporal perspective. In particular, we design region-wise proximity measurements and temporal feature differential operations, and embed them into a novel Differential Time-varying Graph Convolution Network to dynamically capture traffic variations. Considering the hierarchical spatial dependencies and obvious context influences, a hierarchical sequence learning structure is devised by introducing contextual factors into a step-wise decoder. The multi-scale spatial risks are learned jointly to boost the risk predictions based on risk-gather and risk-assign networks. Extensive experiments demonstrate our RiskSeq can increase 5 to 15 percent performances on two datasets.

Understanding and predicting real-time vehicle mobility patterns on highways are essential to address traffic congestion and respond to the emergency. However, almost all existing works (e.g., based on cellphones, onboard devices, or traffic cameras) suffer from high costs, low penetration rates, or only aggregate results. To address these drawbacks, we utilize Electric Toll Collection systems (ETC) as a large-scale sensor network and design a system called VeMo to transparently model and predict vehicle mobility at the individual level with a full penetration rate. Our novelty is how we address uncertainty issues (i.e., unknown routes and speeds) due to sparse implicit ETC data based on a key data-driven insight, i.e., individual driving behaviors are strongly correlated with crowds of drivers under certain spatiotemporal contexts and can be predicted by combining both personal habits and context information. We evaluate VeMo with (i) a large-scale ETC system with tracking devices at 773 highway entrances and exits capturing more than 2 million vehicles every day; (ii) a fleet consisting of 114 thousand vehicles with GPS data as ground truth. Compared with state-of-the-art benchmark mobility models, the experimental results show that VeMo outperforms them by 10 percent on average.

Foreseeing detailed vehicle future trajectories collectively enables a large scope of urban applications such as route planning and commercial advertising. Existing methods focused on predicting future trajectories of urban vehicles with their own fine-grained historical trajectories. Unfortunately, in real-world scenarios, fine-grained trajectories provided by GPS are limited to obtain due to privacy issues and business competitions. In this paper, our solution enables the ubiquitous but coarse-grained location-based surveillance information to predict the fine-grained trajectories of all vehicles with limited number of fine-grained trajectories. We first capture the vectorized semantic representation of trajectories by training the spatiotemporal embedding in large coarse trajectory set. Then, we propose a new measurement to calculate the trajectory similarity, which combines the vehicles' historical behavior similarity and short-term trajectory similarity. The obtained trajectory similarity is then seamlessly embedded into the dynamic graph convolution network in the manner of spatial attention. The dynamic graph convolution sequence-to-sequence module and the fully-connected layer are devised to generate final sequential trajectory predictions. The whole process is to train in a multi-task framework. Extensive experiments on real-world datasets show the excellent performance of our method.

The proliferation of advanced mobile terminals opened up a new crowdsourcing avenue, spatial crowdsourcing, to utilize the crowd potential to perform real-world tasks. In this work, we study a new type of spatial crowdsourcing, called time-continuous spatial crowdsourcing (TCSC in short). It supports broad applications for long-term continuous spatial data acquisition, ranging from environmental monitoring to traffic surveillance in citizen science and crowdsourcing projects. However, due to limited budgets and limited availability of workers in practice, the data collected is often incomplete, incurring data deficiency problem. To tackle that, in this work, we first propose an entropy-based quality metric, which captures the joint effects of incompletion in data acquisition and the imprecision in data interpolation. Based on that, we investigate quality-aware task assignment methods for both single- and multi-task scenarios. We show the NP-hardness of the single-task case, and design polynomial-time algorithms with guaranteed approximation ratios. We study novel indexing and pruning techniques for further enhancing the performance in practice. Then, we extend the solution to multi-task scenarios and devise a parallel framework for speeding up the process of optimization. We conduct extensive experiments on both real and synthetic datasets to show the effectiveness of our proposals.

Many essential services for autonomous vehicles, e.g., navigation on high-quality maps, are designed based on the understanding of traffic conditions, e.g., travel time/speed on road segments, traffic flow, etc. However, most existing traffic condition models lack the consideration of the differentiation for vehicles with different types (e.g., personal vehicles or trucks) and thus they cannot satisfy some type-specific services, e.g., traffic-condition-based routing for autonomous vehicles with different types. To address this challenge, we design a novel vehicular mobility based sensing model called mDrive to predict the travel speed on the road segments, which is targeted for different types of vehicles by utilizing the camera data obtained from the traffic cameras equipped in the road intersections only, without any in-vehicle GPS devices. mDrive addresses the type-aware traffic speed prediction problem with sparse sensors based on three correlations: (1) the spatial correlation of travel speed on the connected road segments; (2) the temporal correlation of travel speed on the consecutive time slots; (3) the type correlation of different vehicular types’ speed on the same road segment. We implement mDrive on traffic camera data from the Chinese city Suzhou and evaluate it by using the detailed GPS data from personal vehicles, taxis, and trucks, with road contextual data as ground truth. The experiment show mDrive outperforms state-of-the-art methods by reducing 6.2% mean relative error on average for all types of vehicles.

The high dynamics and heterogeneous interactions in the complicated urban systems have raised the issue of uncertainty quantification in spatiotemporal human mobility, to support critical decision-makings in risk-aware web applications such as urban event prediction where fluctuations are of significant interests. Given the fact that uncertainty quantifies the potential variations around prediction results, traditional learning schemes always lack uncertainty labels, and conventional uncertainty quantification approaches mostly rely upon statistical estimations with Bayesian Neural Networks or ensemble methods. However, they have never involved any spatiotemporal evolution of uncertainties under various contexts, and also have kept suffering from the poor efficiency of statistical uncertainty estimation while training models with multiple times. To provide high-quality uncertainty quantification for spatiotemporal forecasting, we propose an uncertainty learning mechanism to simultaneously estimate internal data quality and quantify external uncertainty regarding various contextual interactions. To address the issue of lacking labels of uncertainty, we propose a hierarchical data turbulence scheme where we can actively inject controllable uncertainty for guidance, and hence provide insights to both uncertainty quantification and weak supervised learning. Finally, we re-calibrate and boost the prediction performance by devising a gated-based bridge to adaptively leverage the learned uncertainty into predictions. Extensive experiments on three real-world spatiotemporal mobility sets have corroborated the superiority of our proposed model in terms of both forecasting and uncertainty quantification.

Underwater imaging often suffers from color cast and contrast degradation due to range-dependent medium absorption and light scattering. Introducing image statistics as prior has been proved to be an effective solution for underwater image enhancement. However, relative to the modal divergence of light propagation and underwater scenery, the existing methods are limited in representing the inherent statistics of underwater images resulting in color artifacts and haze residuals. To address this problem, this article proposes a convolutional neural network (CNN)-based framework to learn hierarchical statistical features related to color cast and contrast degradation and to leverage them for underwater image enhancement. Specifically, a pixel disruption strategy is first proposed to suppress intrinsic colors’ influence and facilitate modeling a unified statistical representation of underwater image. Then, considering the local variation of depth of field, two parallel sub-networks: Color Correction Network (CC-Net) and Contrast Enhancement Network (CE-Net) are presented. The CC-Net and CE-Net can generate pixel-wise color cast and transmission map and achieve spatial-varied color correction and contrast enhancement. Moreover, to address the issue of insufficient training data, an imaging model-based synthesis method that incorporates pixel disruption strategy is presented to generate underwater patches with global degradation consistency. Quantitative and subjective evaluations demonstrate that our proposed method achieves state-of-the-art performance.

Real-time traffic accident forecasting is increasingly important for public safety and urban management (e.g., real-time safe route planning and emergency response deployment). Previous works on accident forecasting are often performed on hour levels, utilizing existed neural networks with static region-wise correlations taken into account. However, it is still challenging when the granularity of forecasting step improves as the highly dynamic nature of road network and inherent rareness of accident records in one training sample, which leads to biased results and zero-inflated issue. In this work, we propose a novel framework RiskOracle, to improve the prediction granularity to minute levels. Specifically, we first transform the zero-risk values in labels to fit the training network. Then, we propose the Differential Time-varying Graph neural network (DTGN) to capture the immediate changes of traffic status and dynamic inter-subregion correlations. Furthermore, we adopt multi-task and region selection schemes to highlight citywide most-likely accident subregions, bridging the gap between biased risk values and sporadic accident distribution. Extensive experiments on two real-world datasets demonstrate the effectiveness and scalability of our RiskOracle framework.

State-of-the-art image classification algorithms building upon convolutional neural networks (CNNs) are commonly trained on large annotated datasets of high-quality images. When applied to low-quality images, they will suffer a significant degradation in performance, since the structural and statistical properties of pixels in the neighborhood are obstructed by image degradation. To address this problem, this paper proposes a novel deep degradation prior for low-quality image classification. It is based on statistical observations that, in the deep representation space, image patches with structural similarity have uniform distribution even if they come from different images, and the distributions of corresponding patches in low- and high-quality images have uniform margins under the same degradation condition. Therefore, we propose a feature de-drifting module (FDM) to learn the mapping relationship between deep representations of low- and high- quality images, and leverage it as a deep degradation prior (DDP) for low-quality image classification. Since the statistical properties are independent to image content, deep degradation prior can be learned on a training set of limited images without supervision of semantic labels and served in a form of “plugging-in” module of the existing classification networks to improve their performance on degraded images. Evaluations on the benchmark dataset ImageNet-C demonstrate that our proposed DDP can improve the accuracy of the pre-trained network model by more than 20% under various degradation conditions. Even under the extreme setting that only 10 images from CUB-C dataset are used for the training of DDP, our method improves the accuracy of VGG16 on ImageNet-C from 37% to 55%.

Due to the ever-growing concerns over air pollution and energy security, more and more cities have started to replace their conventional taxi fleets with electric ones. Even though environmentally friendly, the rapid promotion of electric taxis raises problems to both taxi drivers and governments, e.g., prolonged waiting/charging time, unbalanced utilization of charging infrastructures, and inadequate taxi supply due to the long charging time. In this article, we conduct the first longitudinal measurement study to understand the long-term evolution of mobility and charging patterns by utilizing 5-year data from one of the largest electric taxi networks in the world, i.e., the Shenzhen electric taxi network in China. In particular, (1) we first perform an electric taxi contextualization about their operation and charging activities; (2) then we design a generic charging event extraction algorithm based on GPS data and charging station data, and (3) based on the contextualization and extracted charging activities, we perform a comprehensive measurement study called ePat to explore the evolution of the electric taxi network from the mobility and charging perspectives. Our ePat is based on 4.8 TB taxi GPS data, 240 GB taxi transaction data, and metadata from 117 charging stations, during an evolution process from 427 electric taxis in 2013 to 13,178 in 2018. Moreover, ePat also explores the impacts of various contexts and benefits during the evolution process. Our ePat as a comprehensive measurement of the electric taxi network mobility and charging evolution has the potential to advance the understanding of the evolution patterns of electric taxi networks and pave the way for analyzing future shared autonomous vehicles.

Urban housing price is widely accepted as an economic indicator of both business and research interest in urban computing. In this work, we propose an effective and fine-grained model for urban subregion housing price predictions. Compared to existing works, our proposal improves the forecasting granularity from city-level to mile-level in spite of data sparsity and complex factors. The fine-grained housing price forecasting has the potential to support a broad scope of applications, ranging from urban planning to housing market recommendations. To achieve that, in this paper, we propose a novel integrated framework, FTD_DenseNet, which incorporates more social and economic features and makes full use of all-level spatiotemporal features. Specifically, the Kalman Filter-based future expection is firstly involved as an influence factor in our model. Extensive empirical studies on real data show the effectiveness of our proposals.

Contrast enhancement and noise removal are coupled problems for low-light image enhancement. The existing Retinex based methods do not take the coupling relation into consideration, resulting in under or over-smoothing of the enhanced images. To address this issue, this paper presents a novel progressive Retinex framework, in which illumination and noise of low-light image are perceived in a mutually reinforced manner, leading to noise reduction low-light enhancement results. Specifically, two fully pointwise convolutional neural networks are devised to model the statistical regularities of ambient light and image noise respectively, and to leverage them as constraints to facilitate the mutual learning process. The proposed method not only suppresses the interference caused by the ambiguity between tiny textures and image noises, but also greatly improves the computational efficiency. Moreover, to solve the problem of insufficient training data, we propose an image synthesis strategy based on camera imaging model, which generates color images corrupted by illumination-dependent noises. Experimental results on both synthetic and real low-light images demonstrate the superiority of our proposed approaches against the State-Of-The-Art (SOTA) low-light enhancement methods.

Due to the ever-growing concerns on the air pollution and energy security, many cities have started to update their taxi fleets with electric ones. Although environmentally friendly, the rapid promotion of electric taxis raises problems to both taxi drivers and governments, e.g., prolonged waiting/charging time, unbalanced utilization of charging infrastructures and reduced taxi supply due to the long charging time. In this paper, we make the first effort to understand the long-term evolving patterns through a five-year study on one of the largest electric taxi networks in the world, i.e., the Shenzhen electric taxi network in China. In particular, we perform a comprehensive measurement investigation called ePat to explore the evolving mobility and charging patterns of electric vehicles. Our ePat is based on 4.8 TB taxi GPS data, 240 GB taxi transaction data, and metadata from 117 charging stations, during an evolving process from 427 electric taxis in 2013 to 13,178 in 2018. Moreover, ePat also explores the impacts of various contexts and benefits during the evolving process. Our ePat as a comprehensive investigation of the electric taxi network mobility and charging evolving has the potential to advance the understanding of the evolving patterns of electric taxi networks and pave the way for analyzing future shared autonomous vehicles.

Understanding and predicting real-time vehicle mobility patterns on highways are essential to address traffic congestion and respond to the emergency. However, almost all existing works (e.g., based on cellphones, onboard devices, or traffic cameras) suffer from high costs, low penetration rates, or only aggregate results. To address these drawbacks, we utilize Electric Toll Collection systems (ETC) as a large-scale sensor network and design a system called VeMo to transparently model and predict vehicle mobility at the individual level with a full penetration rate. Our novelty is how we address uncertainty issues (i.e., unknown routes and speeds) due to sparse implicit ETC data based on a key data-driven insight, i.e., individual driving behaviors are strongly correlated with crowds of drivers under certain spatiotemporal contexts and can be predicted by combining both personal habits and context information. More importantly, we evaluate VeMo with (i) a large-scale ETC system with tracking devices at 773 highway entrances and exits capturing more than 2 million vehicles every day; (ii) a fleet consisting of 114 thousand vehicles with GPS data as ground truth. We compared VeMo with state-of-the-art benchmark mobility models, and the experimental results show that VeMo outperforms them by average 10% in terms of accuracy.

Recent advances in video surveillance systems enable a new paradigm for intelligent urban traffic management systems. Since surveillance cameras are usually sparsely located to cover key regions of the road under surveillance, it is a big challenge to perform a complete real-time traffic pattern analysis based on incomplete sparse surveillance information. As a result, existing works mostly focus on predicting traffic volumes with historical records available at a particular location and may not provide a complete picture of real-time traffic patterns. To this end, in this paper, we go beyond existing works and tackle the challenges of traffic flow analysis from three perspectives. First, we train the transition probabilities to capture vehicles' movement patterns. The transition probabilities are trained from third-party vehicle GPS data, and thus can work in the area even if there is no camera. Second, we exploit the Multivariate Normal Distribution model together with the transferred probabilities to estimate the unobserved traffic patterns. Third, we propose an algorithm for real-time traffic inference with surveillance as a complement source of information. Finally, experiments on real-world data show the effectiveness of our approach.

Modeling statistical regularity plays an essential role in ill-posed image processing problems. Recently, deep learning based methods have been presented to implicitly learn statistical representation of pixel distributions in natural images and leverage it as a constraint to facilitate subsequent tasks, such as color constancy and image dehazing. However, the existing CNN architecture is prone to variability and diversity of pixel intensity within and between local regions, which may result in inaccurate statistical representation. To address this problem, this paper presents a novel fully point-wise CNN architecture for modeling statistical regularities in natural images. Specifically, we propose to randomly shuffle the pixels in the origin images and leverage the shuffled image as input to make CNN more concerned with the statistical properties. Moreover, since the pixels in the shuffled image are independent identically distributed, we can replace all the large convolution kernels in CNN with point-wise (1*1) convolution kernels while maintaining the representation ability. Experimental results on two applications: color constancy and image dehazing, demonstrate the superiority of our proposed network over the existing architectures, i.e., using 1/10~1/100 network parameters and computational cost while achieving comparable performance.

Due to the sparse distribution of video surveillance cameras and low installation rate of dash-mounted online telematics systems, tracking precise trajectories of urban private vehicles is a challenging task. Previous studies on vehicle tracking are mostly concerned with recovering trajectories with low- sampling rate GPS coordinates or captured surveillance information by identifying road traffic patterns from this information. Nevertheless, to the best of our knowledge, none of them have considered using the vehicle encounter information to enhance the sampling rate as well as the time-varying and in-group characteristics of vehicle traffic patterns, let alone to achieve vehicle tracking. With this insight, we divide all vehicles into clusters with a Canopy and K- means combined algorithm for different time periods and use an exponential distribution to approximate the time that vehicles from one cluster encounter a fixed Wi-Fi hotspot in the future during a given time period and at a specific location. Based on these preliminary results, we propose a novel approach to select the optimal data packet transmission scheme between encountered vehicles to transfer vehicle encounter information to the server as much and quickly as possible, and then calculate the trajectories of all private vehicles accurately. We evaluate our solution via real-world private vehicles and road surveillance system datasets. Experimental results demonstrate that our approach outperforms other solutions in terms of the accuracy ratio of vehicle tracking.

Due to the sparse distribution of road video surveillance cameras, precise trajectory tracking for vehicles remains a challenging task. To the best of our knowledge, none of the previous research considered using on-road taxicabs as mobile video surveillance cameras and road traffic flow patterns, therefore not suitable for recovering trajectories of vehicles. With this insight, we model the travel time-cost of a road segment during various time periods precisely with LNDs (Logarithmic Normal Distributions), then use LSNDs (Log Skew Normal Distributions) to approximate the time-cost of an urban trip during various time periods. We propose an approach to calculate possible location and time distribution of the vehicle, select the taxicab to verify the distribution by uploading and checking video clips of this taxicab, finally refine the restoring trajectory in a recursive manner. We evaluate our solution on real-world taxicab and road surveillance system datasets. Experimental results demonstrate that our approach outperforms alternative solutions in terms of accuracy ratio of vehicle tracking.

Due to the sparse distribution of road video surveillance cameras, precise trajectory tracking for hit-and-run vehicles remains a challenging task. Previous research on vehicle trajectory recovery mostly focuses on recovering trajectory with low-sampling-rate GPS coordinates by retrieving road traffic flow patterns from collected GPS information. However, to the best of our knowledge, none of them considered using on-road taxicabs as mobile video surveillance cameras as well as the time-varying characteristics of vehicle traveling and road traffic flow patterns, therefore not suitable for recovering trajectories of hit-and-run vehicles. With this insight, we model the travel time-cost of a road segment during various time periods precisely with LNDs (Logarithmic Normal Distributions), then use LSNDs (Log Skew Normal Distributions) to approximate the time-cost of an urban trip during various time periods. We propose a novel approach to calculate possible location and time distribution of the hit-and-run vehicle in parallel, select the optimal taxicab to verify the distribution by uploading and checking video clips of this taxicab, finally refine the restoring trajectory in a parallel and recursive manner. We evaluate our solution on real-world taxicab and road surveillance system datasets. Experimental results demonstrate that our approach outperforms alternative solutions in terms of accuracy ratio of vehicle tracking.

Underwater images often suffer from color distortion and visibility degradation due to the light absorption and scattering. Existing methods utilize various assumptions/constrains to achieve reasonable solutions for underwater image enhancement. However, these methods share the common limitation that the adopted assumptions may not work for some particular scenes. To address this problem, this paper proposes an end to end framework for underwater image enhancement, where a CNN-based network called UIE-Net is presented. The UIE-net is trained with two tasks, color correction and haze removal. This unified training approach enables learning a strong feature representation for both tasks simultaneously. For better extracting the inherent features in local patches, a pixels disrupting strategy is exploited in the proposed learning framework, which significantly improves the convergent speed and accuracy. To handle the training of UIE-net, we synthesize 200000 training images based on the physical underwater imaging model. Experiments on benchmark underwater images for cross-scenes show that UIE-net achieves superior performance over existing methods.

In urban road systems, it is a challenging task to investigate traffic status of all intersections due to the scarce distribution of road surveillance cameras. Previous research mostly focuses on how to use historical data of camera-equipped intersections to infer their future traffic statuses. However, as far as we know, there does not exist an effective algorithm to infer the real-time traffic statuses of those camera- free intersections by using the traffic information from some other road video cameras in urban road system. In this paper, we first study the spatial- temporal variation characteristics of urban traffic flows from a macroscopic view, including turning ratio models and traveling time models of individual road segments. And then we build a novel traffic impact tree model to calculate the real-time traffic volume for specific camera-free intersections. We evaluate our solutions on real-world taxicab and road surveillance system data-set. The experimental results show that our proposed method outperforms alternative solutions in terms of the accuracy of the reckoned future traffic flow.

Due to high mobility and intermittent connections in vehicular networks, reliable and efficient vehicular communication is a challenging task. Previous research on Vehicle-to-Vehicle (V2V) communication mostly focuses on achieving reliable transmissions from a given source to a given destination by mining moving patterns of taxicabs. However, to the best of our knowledge, none of them considered the habit-driven regularities of individual taxicabs as well as the urban-layout-driven time-varying regularities of crowds of taxicabs synthetically. With this insight, we model both individual and holistic driving patterns by Markov Chain models, then devise a new method to predict possible driving routes for every single taxicab. In addition, we design a new method to evaluate the probability that a single taxicab retrieves information of a specific road segment while it drives through another road segment during a given time period, and also to quantify the expected probability that a single taxicab obtains the information of a given road segment in the near future. With such information, our solution enables the selection of the optimal data packet transmission scheme. We evaluate our solution on a real-world taxicab dataset. Experimental results demonstrate that our approach outperforms alternative solutions in terms of diffusion speed and success ratio of data retrieval.

Recent advances in vehicular networks, GPS and smartphone technologies have changed the paradigm of intelligent taxicab systems. Indeed, taxicab trajectories and online calling information have enabled us to provide more efficient and personalized services. However, existing approaches are not sufficient in exploiting cooperative scheduling techniques and utilizing real time calling information. To this end, in this paper, we model the time-varying regularities of traffic flows, activity ratios of passengers, and unoccupied taxicabs of road segments by mining statistical data on taxicab trajectories. Along this line, we propose a novel approach to calculate the expected revenue of possible routes for individual taxicabs while considering the influence of others, and at the same time, advance a dynamic taxicab scheduling mechanism with online taxicab calling information. Finally, we evaluate our algorithm on real-world taxicab data. Experimental results demonstrate that our approach outperforms existing alternative solutions in terms of average revenue of taxi drivers.

Due to high mobility and intermittent connections in vehicular networks, reliable and efficient Vehicle-to-Vehicle (V2V) communication is a challenging task. The Named Data Networking (NDN) paradigm is recently being applied to achieve efficient V2V communication, however, proactive V2V information diffusion conflicts with the receiver-initiated nature of NDN. This paper bridges this gap by exploiting hierarchical data names to achieve efficient and proactive V2V information diffusion. We first identify a popular subgroup of vehicles, then select them as the diffusion seeds with 3G/4G capability, while others are only equipped with short-range V2V communication. We also design a namespace-based method to optimize data transmission when vehicles are close, in order to maximize the information distribution across geographical space. We evaluate our solution via a real-world taxicab dataset. Experimental results demonstrate that our approach significantly outperforms state-of-the-art solutions in terms of diffusion speed and success rate of data retrieval.

Despite the clear benefits of electric vehicles (EVs) in terms of reducing greenhouse gas emissions and traditional energy consumptions, the popularization of EVs remains a challenge in the short run. When considering electric taxis, urban planners must face the additional issue of providing battery swapping services. While previous studies focused on planning battery swapping stations for private EVs, we investigate ways of supporting the upgrade of an entire urban taxi system, with demands differing both in scale and nature. With this insight, we analyze the historical sensing data of taxi routes, and evaluate the battery swapping demand profile, as well as the driving time between positions in the road network. Based on these inputs, we propose a method to calculate an optimized battery swapping station scheme. Our strategies are then evaluated via a real world 366-day, 3,976-taxi dataset. The results show that compared to uniform deployment, our planning scheme reduces the average time-cost by 67.2%.

Due to high mobility and frequent disconnections in a vehicular network, reliable and efficient vehicular communication is very challenging. Previous studies focus on predicting the trajectories of single vehicles. Due to many random factors, however, there is little regularity in the movements of a single vehicle in an urban area, and this motivates us to take a holistic network perspective. With this insight, we model the time varying regularities of road traffic flows in road segments and intersections by mining statistic trajectories of all vehicles in the network. Based on these regularities and local real-time traffic information, we propose a new method to calculate the expected transfer delay from a current position to a given destination. We also propose a method to collect updated destination information. By combining the above two methods, we design a routing algorithm for vehicle-to-vehicle data transmission in vehicular networks, and then prove that it is a linear-time algorithm. Finally, we evaluate our algorithm by using information of real taxi vehicles. The results show that the performance of our algorithm is significantly better than other solutions in terms of packet delay.

Though there exist ready-made DSRC/WiFi/3G/4G cellular systems for roadway communications, there are common defects in these systems for roadway safety oriented applications and the corresponding challenges remain unsolved for years, i.e., WiFi cannot work well in vehicular networks due to the high probability of packet loss caused by burst communications, which is a common phenomenon in roadway networks; 3G/4G cannot well support real-time communications due to the nature of their designs; DSRC lacks the support to roadway safety oriented applications with hard realtime and reliability requirements [1]. To solve the conflict between the capability limitations of existing systems and the ever-growing demands of roadway safety oriented communication applications, we propose a novel system design and implementation for realtime reliable roadway communications, aiming at providing safety messages to users in a realtime and reliable manner. In our extensive experimental study, the latency is well controlled within the hard realtime requirement (100ms) for roadway safety applications given by NHTSA [2], and the reliability is proved to be improved by two orders of magnitude compared with existing experimental results [1]. Our experiments show that the proposed system for roadway safety communications can provide guaranteed highly reliable packet delivery ratio (PDR) of 99% within the hard realtime requirement 100ms under various scenarios, e.g., highways, city areas, rural areas, tunnels, bridges. Our design can be widely applied for roadway communications and facilitate the current research in both hardware and software design and further provide an opportunity to consolidate the existing work on a practical and easy-configurable low-cost roadway communication platform.

As the advancement of sensing and networking technologies, participatory sensing has raised more and more attention as it provides a promising way enabling public and professional users to gather and analyze private data to understand the world. However, in these participatory sensing applications both data at the individuals and analysis results obtained at the users are usually private and sensitive to be disclosed, e.g., locations, salaries, utility usage, consumptions, behaviors, etc. A natural question, also an important but challenging problem is how to keep both participants and users data privacy while still producing the best analysis to explain a phenomenon. In this paper, we have addressed this issue and proposed M-PERM, a mutual privacy preserving regression modeling approach. Particularly, we launch a series of data transformation and aggregation operations at the participatory nodes, the clusters, and the user. During regression model fitting, we provide a new way for model fitting without any need of the original private data or the exact knowledge of the model expression. To evaluate our approach, we conduct both theoretical analysis and simulation study. The evaluation results show that the proposed approach produces exactly the same best model as if the original private data were used without leakage of the fitted model to any participatory nodes, which is a significant advance compared with the existing approaches [1-5]. It is also shown that the data gathering design is able to reach maximum privacy protection under certain conditions and be robust against collusion attack. Furthermore, compared with existing works under the same context (e.g., [1-5]), to our best knowledge it is the first work showing that not only the model coefficients estimation but also a series of regression analysis and model selection methods are reachable in mutual privacy preserving data analysis scenarios such as participatory sensing.

Though there exist ready-made DSRC/WiFi/3G/4G cellular systems for roadway communications, there are common defects in these systems for roadway safety oriented applications and the corresponding challenges remain unsolved for years, i.e., WiFi cannot work well in vehicular networks due to the high probability of packet loss caused by burst communications, which is a common phenomenon in roadway networks; 3G/4G cannot well support real-time communications due to the nature of their designs; DSRC lacks the support to roadway safety oriented applications with hard real-time and reliability requirements in the most recent study [1].

This demonstration presents a mobile surveillance system that is developed at the University of Science and Technology of China and undergoing a technology transition. The goal of this project is to develop a low-cost, promptly-deployable, mobility manageable, location traceable, self-organizing wireless communication and mobile surveillance system, for real-time video surveillance and on-site guard for public security purposes, e.g., public security against terrorism in large-scale gatherings and events.

Wireless sensor and actor networks (WSANs) have attracted much attention in the recent years. In WSANs, sensor nodes are in charge of the sensing task, while the actor nodes take actions in response to the sensed phenomena or the detected event. In order to execute the correct operations, the applications typically have to process the data or messages from the different sensors in the same order. This paper presents a Consistent Message Ordering (CMO) algorithm, which ensures that messages from the different sensors arrive at the different actors in a consistent order. The significance of the algorithm is fully localized, asynchronous and energy-efficient. We prove the correctness of the algorithm, and analyze the theoretical message complexity. And the experimental results show that CMO algorithm improves the performance at least 60% over TMOS algorithm in many situations.

The recent years, achievements in micro-sensor technology and low-power electronics make wireless sensor networks become into realities in applications.In this paper, we basically describe the implementation of the intensive irrigated agriculture monitoring system based on wireless sensor networks.we give some key technologies of wireless sensor networks according as the system performance requirements for this application, longevity and latency.