Gradient perturbation mechanisms, such as differential privacy (DP), aim to defend against gradient inversion attacks (GIA) by injecting noise into the shared gradients. Recent studies have shown that DP-based defenses lack robustness against advanced GIAs. However, existing gradient inversion methods typically rely on iterative refinement and assume static noise, resulting in low efficiency and limited reconstruction fidelity under high-noise conditions. In this paper, we propose Venom, a novel gradient inversion attack method based on a liquid diffusion mechanism. Venom reconstructs private data directly from DP-protected gradients without requiring any prior knowledge of the noise distribution. Specifically, we design a Structural Prior Extraction (SPE) module that analytically extracts deep feature representations from perturbed gradients through energy-based aggregation, enabling stable pre-reconstruction of users' latent data features. We further introduce a Diffusion-driven Liquid Recovery Network (Diff-LRN) for high-fidelity image reconstruction. Unlike traditional diffusion models that rely on iterative sampling with predefined noise schedules, Diff-LRN performs deterministic single-step reconstruction using adaptive liquid neural dynamics to handle spatially heterogeneous noise patterns. Experiments across four benchmarks demonstrate that Venom achieves a speedup of up to 38,343× over state-of-the-art attacks while maintaining high reconstruction fidelity under strong DP settings. These results challenge prevailing assumptions about DP robustness and underscore the need for more resilient privacy-preserving mechanisms in federated learning.

Long-term series forecasting leverages historical observations to predict extended future sequences and plays a crucial role across various domains. However, conventional models relying on fixed-length lookback windows struggle with the inherent dynamic dependencies and multi-scale characteristics of time series data. These fixed windows either introduce noise through excessive length or omit critical patterns when too short, while optimal window sizes vary significantly with tasks and external conditions. To address this, we propose the Adaptive Lookback Window (ALW) framework, a wavelet transform-driven approach for multi-scale adaptive lookback window selection. ALW decomposes time series into distinct frequency components through wavelet transforms, quantifies the contribution of each historical time step via scale-specific attention mechanisms, and dynamically determines optimal window lengths through backward information accumulation and soft truncation techniques. Finally, refined input features are generated for downstream prediction models via a weighted reconstruction process. Extensive evaluations across multiple public benchmarks demonstrate that ALW, as an efficient plug-and-play technique, not only reduces the MSE of backbone models by an average of 3.2% but also alleviates hyperparameter tuning requirements and enables input feature dimensionality reduction, which curtails subsequent model computational costs.

Energy efficiency has become a critical requirement, driving recommendation systems for resource-constrained environments such as edge devices. Model quantization offers an effective way to build low-bitwidth models while preserving accuracy. However, user–item interaction graphs contain numerous nodes and complex topological structures, leading nodes to exhibit unique similarities and differences. Existing quantization methods uniformly process parameters in high-dimensional DNN layers (e.g., linear, convolutional, or attention layers), while inadequately capturing such similarities among node embeddings. This paper proposes GraphQ, a chunk-wise quantization framework for graph collaborative filtering that supports both the training and post-training phases in a unified perspective. Our core idea is to adaptively partition node embeddings into multiple chunks based on the distribution of embedding values, and then apply chunk-wise quantization. Specifically, for quantization-aware training (QAT), we introduce learnable low-precision quantization factors that partition node embeddings into multiple chunks and are dynamically updated following message passing. For post-training quantization (PTQ), we first cluster nodes and then partition their dimensions into chunks for weight clipping. Extensive experiments on four real-world datasets show that GraphQ outperforms state-of-the-art QAT methods by an average of 27.49% in Recall@10 under the 256-dimensional embedding and 2-bit settings, and surpasses PTQ methods by 78.64% on average under 4-bit settings.

Sequential recommendation models, particularly those based on attention, achieve strong accuracy but incur quadratic complexity, making long user histories prohibitively expensive. Sub-quadratic operators such as Hyena provide efficient alternatives in language modeling, but their potential in recommendation remains underexplored. We argue that Hyena faces challenges in recommendation due to limited representation capacity on sparse, long user sequences. To address these challenges, we propose HyenaRec, a novel sequential recommender that integrates polynomial-based kernel parameterization with gated convolutions. Specifically, we design convolutional kernels using Legendre orthogonal polynomials, which provides a smooth and compact basis for modeling long-term temporal dependencies. A complementary gating mechanism captures fine-grained short-term behavioral bursts, yielding a hybrid architecture that balances global temporal evolution with localized user interests under sparse feedback. This construction enhances expressiveness while scaling linearly with sequence length. Extensive experiments on multiple real-world datasets demonstrate that HyenaRec consistently outperforms Attention-, Recurrent-, and other baselines in ranking accuracy. Moreover, it trains significantly faster (up to 6× speedup), with particularly pronounced advantages on long-sequence scenarios where efficiency is maintained without sacrificing accuracy. These results highlight polynomial-based kernel parameterization as a principled and scalable alternative to attention for sequential recommendation.

Short video platforms face significant challenges in efficiently delivering diverse video analytics to a concurrent user base. These analytics impose substantial storage and computation demands on edge devices, while offloading tasks to the cloud incurs high data transmission overhead. Edge-cloud collaborative computing, which distributes workloads between edge devices and the cloud, offers a promising solution. However, applying this paradigm to short video platforms encounters efficiency and scalability hurdles due to the limited memory capacity of commercial smartphones, the large volume of intermediate analytics data, and the highly dynamic workloads driven by extensive user activity. This paper introduces FaaSVA, a Function-as-a-Service (FaaS)-based edge-cloud video analytics system for short video platforms. FaaSVA enables efficient and scalable video analytics by integrating edge-cloud collaboration with serverless computing. On the edge, it implements a re-utilizing memory provisioning mechanism to allocate, release and reclaim memory for video frames, reducing intermediate data and minimizing the memory footprint on smartphones. On the cloud, a serverless framework supports rapid and elastic adaptation to fluctuating workloads. The system abstracts video analytics functions and utilizes a dynamic, workload-aware Directed Acyclic Graph (DAG) partitioning algorithm to intelligently schedule functions to optimal execution nodes, balancing computation and communication. We implement FaaSVA and show that FaaSVA can reduce video analytics end-to-end latency by 28.46%.

Traffic flow forecasting plays a vital role in improving transportation efficiency, safety, and sustainability in cities. Traditional methods focus on capturing the intricate spatiotemporal dynamics within traffic flows. With the development of web and sensing technology, urban knowledge graphs (KG) emerge as an auxiliary knowledge that represents dependencies among traffic entities. Prior works have highlighted the importance of incorporating knowledge from urban KGs to enhance the accuracy of traffic flow forecasting. Unfortunately, these methods inadequately capture the dynamics within the evolving urban KGs, resulting in sub-optimal traffic forecasts.To bridge this gap, we first perform an analytical study to show the impact of short-term and long-term dynamics of urban KGs on the traffic flow forecasting task. Based on our preliminary observation, we propose Dynamic Knowledge Graph-Informed Traffic Flow Forecasting (DKG-TF2), which is a graph neural network-based spatiotemporal forecasting model, enabling to capture of both static and dynamic characteristics from urban KGs, and integrates it with traffic flow forecasting. Evaluated over a real-world dataset, we demonstrate that the DKG-TF2 model significantly outperforms competitive baselines and achieve a root mean squared error of 3.4923. Further ablation studies reveal the complementary roles of static attributes and dynamic attributes in enhancing forecasting accuracy and model robustness.​

Drone-based crowd tracking remains challenging due to low object distinctiveness, high crowd density, frequent occlusions, and the difficulty of maintaining continuous and precise localization from aerial views. To address these challenges, we proposeRegenTrack, a tracking framework that integrates distance-adaptive fusion with Regeneration Pool (RegenPool) matching. RegenTrack dynamically balances appearance and motion cues for trajectory-object association. Appearance cues are extracted from multi-frame fused trajectory and object features, while motion cues are derived by comparing predicted and observed positions via a motion network. Unmatched trajectories are temporarily stored in RegenPool and re-matched in subsequent frames to mitigate object loss caused by occlusion. Meanwhile, RegenPool evaluates unmatched objects through multi-frame observations to determine whether they should be promoted to new trajectories. In addition, a two-stage cascade matching strategy is employed to further enhance trajectory continuity and stability. Experiments on DRONECROWD, DRONEBIRD, and CROHD demonstrate that RegenTrack achieves notable improvements in both tracking accuracy and reliability for drone-based crowd scenarios. The code will be released at https://github.com/Zebrabeast/RegenTrack.

With the development of mobile and edge computing, the demand for low-bit quantized models on edge devices is increasing to achieve efficient deployment. To enhance the performance, it is often necessary to retrain the quantized models using edge data. However, due to privacy concerns, certain sensitive data can only be processed on edge devices. Therefore, employing Quantization-Aware Training (QAT) on edge devices has become an effective solution. Nevertheless, traditional QAT relies on the complete dataset for training, which incurs a huge computational cost. Coreset selection techniques can mitigate this issue by training on the most representative subsets. However, existing methods struggle to eliminate quantization errors in the model when using small-scale datasets (e.g., only 10% of the data), leading to significant performance degradation. To address these issues, we propose QuaRC, a QAT framework on edge devices via Relative entropy coreset selection and Cascaded layer correction. The framework consists of two main phases: In the coreset selection phase, QuaRC introduces the “Relative Entropy Score” to identify the subsets that most effectively capture the model's quantization errors. During the training phase, QuaRC employs the Cascaded Layer Correction strategy to align the intermediate layer outputs of the quantized model with those of the full-precision model, thereby effectively reducing the quantization errors in the intermediate layers. Experimental results demonstrate the effectiveness of our approach. For instance, when quantizing ResNet-18 to 2-bit using a 1% data subset, QuaRC achieves a 5.72% improvement in Top-1 accuracy on the ImageNet-1 K dataset compared to state-of-the-art techniques.

Atomic electron tomography (AET) is essential for characterizing the atomic structure of functional materials. However, raw 3-D tomograms often exhibit severe artifacts caused by geometric constraints and low radiation doses. Although point-attention-based ensemble augmentation methods effectively remove artifacts in simulated datasets with varying structure factors, they struggle with the complexity of real tomograms that demand multidomain feature learning. Moreover, existing models degrade in multidomain scenarios and incur high parameter counts introduced by point-attention mechanisms, which increase hardware demands. To address these challenges, we propose a sparse mixture of Mambas (MoMambas), a novel 3-D augmentation method that enhances domain generalization. MoMambas decouple domain-specific parameters by integrating a sparse mixture-of-experts (MoE) framework with Mamba-based experts, resolve positional ambiguity in sparse input sequences through positional information enhancement, and boost MoE accuracy via a multihead routing algorithm. Our approach achieves a 22% accuracy improvement over state-of-the-art AET augmentation methods in multidomain learning, reduces the parameter count to just 2.9% of the original, and lowers computational cost by 6%. Codes and data are publicly available at https://github.com/yuy38457/MoMambas.

Long-term series forecasting aims to predict future data over long horizons based on historical information. However, existing methods struggle to effectively utilize long lookback windows due to overfitting, computational resource constraints, or information extraction challenges, thereby limiting them to using limited lookback windows for predicting long-term future series. To address these issues, this paper introduces the Input Refinement and Prediction Auxiliary (IRPA) framework, a lightweight model consisting of four linear layers designed to extract key information from ultra-long lookback windows to enhance limited lookback windows and assist prediction processes. IRPA comprises an Input Refinement Module (IRM) and a Prediction Auxiliary Module (PAM), each constructed from two linear layer sub-modules. The IRM performs effective decomposition and patching of ultra-long series, refining seasonal and trend features to increase the information density in limited lookback windows and mitigate overfitting and parameter inflation. The PAM extracts historical similarities and seasonal patterns from ultra-long lookback windows to significantly improve prediction accuracy. IRPA substantially extends the utilization of lookback windows, offering a lightweight and efficient solution with broad applicability. Experimental results on eight datasets show IRPA reduces the Mean Squared Error (MSE) by an average of 16.1% for various models.

Materials science text mining (MSTM), involving tasks like property extraction and synthesis action retrieval, is pivotal for advancing research by deriving critical insights from scientific literature. Descriptors, serving as essential task labels, often vary in meaning depending on researchers' usage purposes across different mining tasks. (e.g., 'Material' can refer to both synthesis components and participants in fuel cell experiment). This meaning difference makes it difficult for existing methods, fine-tuned to specific task, to handle the same descriptors in other tasks. To overcome above limitation, we propose MatDuck, a simple and effective approach for Zero-Shot MSTM by evoking material knowledge within Large Language Models (LLMs). Specifically, inspired by the Duck Typing principles in programming languages, we present a ClassDefinition-Style Descriptor generation method that evokes task-specific characteristics to address usage variation. Subsequently, we introduce code-style in-context learning for zero-shot tasks, reframing them into code to leverage LLMs' proficiency in code understanding. Extensive experiments on eight benchmark datasets demonstrate that MatDuck, as a plug-and-play approach, significantly improves the Zero-Shot MSTM performance of LLMs by an average of 11.3% across seven tasks.

Efficient crack segmentation is crucial for engineering surface inspection, especially on edge devices where both accuracy and computational efficiency are essential. To address the challenges posed by crack directionality and blurred edges while enhancing performance, we propose a lightweight segmentation model, SCCU-Net, based on cross-spatial synergistic learning and coordinate awareness. The model integrates coordinate information and perceptual sets into a hybrid attention mechanism, significantly boosting segmentation accuracy. We introduce a Mapping Attention Gate (MAG), which utilizes gating signals from fine-grained features to guide cross-spatial learning, alongside an Adaptive Skip Fusion (ASF) to ensure smooth feature integration while optimizing computational resources. Extensive experiments on six benchmark datasets demonstrate that SCCU-Net consistently outperforms existing lightweight models, setting a new benchmark for crack detection on edge devices. Code is available at https://github.com/lsyyy20000830/SCCU-Net.

Federated Unlearning (FU) addresses the "right to be forgotten" in federated learning by removing specific client data's contribution without retraining from scratch. Existing FUs are data-dependent, which make the assumption that systems can access original training data or stored historical parameter updates during unlearning. However, the assumption cannot always hold in practice, as users usually request the deletion of client data and historical parameter updates due to privacy concerns or storage limitations. Therefore, it is crucial to develop a zero-shot FU method without such data access. The key challenge is how to distinguish and remove the impact of target clients without data-level information. Motivated by the idea that if we can learn client-specific personalized information from the model instead of data, FU can be model-centric and data-free, we present the first zero-shot FU framework ZeroFU. By embedding client contributions into the model during learning via condition computation, ZeroFU enables the model to possess personalized features for unlearning. The unlearning is achieved using a proposed GAN-based distillation framework that obfuscates the personalized feature of the target client. Evaluations demonstrate its effectiveness in unlearning under non-IID settings.

Molecular Relational Learning (MRL) is widely applied in molecular sciences. Recent studies attempt to retain molecular core information (e.g., substructures) by Graph Information Bottleneck but primarily focus on information compression without considering the causal dependencies of chemical reactions among substructures. This oversight neglects the core factors that determine molecular relationships, making maintaining stable MRL in distribution-shifted data challenging. To bridge this gap, we propose the Causal Subgraph Information Bottleneck (CausalGIB) for stable MRL. CausalGIB leverages causal dependency to guide substructure representation and integrates subgraph information bottleneck to optimize the core substructure representation, generating stable representations. Specifically, we distinguish causal and confounding substructures by noise injection and substructure interaction based on causal analysis. Furthermore, by minimizing the discrepancy between causal and confounding information within subgraph information bottleneck, CausalGIB captures core substructures composed of causal substructures and aggregates them into molecular representations to improve their stability. Experimental results on nine datasets demonstrate that CausalGIB outperforms state-of-the-art models in two tasks and significantly enhances model’s stability in distribution-shifted data.

Ensuring seamless connectivity is crucial in Vehicular Networks within Intelligent Transportation Systems (ITS). These networks facilitate real-time communication between vehicles and infrastructure, such as roadside units (RSUs) and base stations, enabling applications like traffic management, collision avoidance, and infotainment services. However, maintaining stable connections remains challenging due to network complexity, variable vehicle speeds, and frequent topology changes. These factors lead to increased latency, energy consumption, unnecessary handovers (ping-pong effect), and packet loss, particularly in low cellular coverage areas. To address these challenges, a heterogeneous approach integrating cellular networks (5G, LTE) and Dedicated Short-Range Communication (DSRC) is essential for robust connectivity and high-speed data transfer. Intelligent Vertical Handover (VHO) algorithms are necessary to ensure seamless connections to edge servers and optimize network performance in dynamic vehicular environments. This study proposes a novel handover management approach by integrating K-means clustering, a Deep Maxout Network (DMN), and the Dung Beetle Optimizer (DBO). K-means clustering identifies potential handover margin areas, the DMN evaluates handover necessity based on Received Signal Strength (RSS), vehicle speed, and latency, while the DBO assigns vehicles to servers by considering latency and energy consumption. Further analysis of server assignments mitigates the ping-pong effect by comparing current and previous selections. Simulation results in a low-coverage vehicular scenario show that the proposed method improves decision delay, latency, energy consumption, handover failure rate, ping-pong effect, and throughput. The findings demonstrate extended connection durations, lower latency, decreased energy consumption, and minimal packet loss, contributing to enhanced network efficiency and reliability in vehicular management.

Federated learning (FL) facilitates model training collaboration without needing to compromise on data privacy, which is beneficial for Internet of Things (IoT) settings. However, deploying FL on resource-constrained devices poses significant challenges, including high computational and communication costs, slower-than-expected convergence, and reduced accuracy due to noisy encrypted gradient updates. This paper presents FedFlex, a novel framework for FL that remains privacy-preserving and adds homomorphic encryption (HE) along with an adaptive Dual-Factor Thompson Sampling (DFTS) adaptive client selection algorithm. DFTS identifies clients whose local model updates exhibit a strong correlation with the global model update while ensuring compatibility with the clients’ computational capabilities. In addition, we introduce a Noise-Reduced Gradient Flow that reduces the noise, which allows scaling the encrypted gradients to be efficiently transmitted while reducing the amount of noise and ciphertext. Extensive experiments conducted on multiple benchmark and real-world datasets show that FedFlex increases training accuracy by up to 95.9%, improves convergence speed by up to 3.2×, and decreases communication costs by 2×–4× relative to other existing approaches. These results showcase the flexibility and efficiency of FedFlex in improving secure FL for heterogeneous and resource-constrained IoT systems.

Abstract Large Language Models (LLMs) offer a promising alternative to traditional Materials Science Text Mining (MSTM) by reducing the need for extensive data labeling and fine-tuning. However, existing zero-/few-shot methods still face limitations in aligning with personalized needs in scientific discovery. To address this, we propose ClassMATe, an active knowledge structuring approach for MSTM. Specifically, we first propose a class definition stylization method to structure knowledge, enabling explicit clustering of latent material knowledge in LLMs for enhanced inference. To align with the scientists’ needs, we propose an active needs refining strategy that iteratively clarifies needs by learning from uncertainty-aware hard samples of LLMs, further refining the knowledge structuring. Extensive experiments on seven tasks and eight datasets show that ClassMATe, as a plug-and-play method, achieves performance comparable to supervised learning without requiring fine-tuning or extra knowledge base, highlighting the potential to bridge the gap between LLMs’ latent knowledge and real-world scientific applications.1

Happiness prediction based on large-scale online data and machine learning models is an emerging research topic that underpins a range of issues, from personal growth to social stability. Many advanced machine learning (ML) models with explanations are used for happiness online assessment while maintaining high accuracy of results. However, expert feedback and sociological theory may be absent from these models, which limits the association between prediction results and the right reasons for why they occurred. Sociological studies have shown that primary and secondary relations are inherent in happiness factors, which can be used as domain knowledge to guide model training. Inspired by such insights, this article attempts to provide new insights into the explanation consistency from an empirical study perspective. Then this article studies how to represent and introduce domain knowledge constraints to make ML models more trustworthy. We achieve this by 1) proving that multiple prediction models with additive factor attributions will have the desirable property of primary and secondary relations consistency; and 2) showing that factor relations with quantity can be represented as an importance distribution for encoding domain knowledge. Factor explanation difference is penalized by the Wasserstein distance among prediction models. Experimental results using two online datasets show that domain knowledge of stable factor relations exists. Using this knowledge not only improves happiness prediction accuracy but also reveals more significant happiness factors for assisting decisions.

Multi-label image classification, which involves recognizing multiple objects within a single image, is a fundamental task in computer vision. Recently, Visual-Language Models (VLMs) have made remarkable progress in this area. Many approaches combine textual and visual modalities to understand the entire image. In this paper, we find that there is a direct correlation between the accurate localization of objects and the accuracy of multi-label classification. However, previous research methods did not specifically address localization accuracy, resulting in sub-optimal accuracy. Therefore, we propose the AMITA, namely Attribute-guided Masked Image-Text Alignment for multi-label image representation. AMITA improves localization accuracy by segmenting object masks, thereby enhancing the accuracy of multi-label image classification. Additionally, AMITA introduces an AutoFocus method to handle the localization problem of small objects. AutoFocus conducts recognition by resizing and cropping the image respectively, and automatically selects the images useful for the classification target. Moreover, AMITA incorporates Attribute-guided Prompting to strengthen the semantic distinction among different categories. It uses large language models to obtain the attributes of different categories and carefully designs prompts to enhance the attribute differences among different categories. Finally, extensive experiments on three popular datasets, including MS-COCO, Pascal VOC 2007, and NUS-WIDE, demonstrate the superiority of AMITA.

Unsupervised domain adaptation enables the transfer of knowledge from a labeled source domain to an unlabeled target domain, and its application in crowd counting is gaining momentum. Current methods typically align distributions across domains to address inter-domain disparities at a global level. However, these methods often struggle with significant intra-domain gaps caused by domain-agnostic factors such as density, surveillance angles, and scale, leading to inaccurate alignment and unnecessary computational burdens, especially in large-scale training scenarios. To address these challenges, we propose the Multi-Granularity Optimal Transport (MGOT) distribution alignment framework, which aligns domain-agnostic factors across domains at different granularities. The motivation behind multi-granularity is to capture fine-grained domain-agnostic variations within domains. Our method proceeds in three phases: first, clustering coarse-grained features based on intra-domain similarity; second, aligning the granular clusters using an optimal transport framework and constructing a mapping from cluster centers to finer patch levels between domains; and third, re-weighting the aligned distribution for model refinement in domain adaptation. Extensive experiments across twelve cross-domain benchmarks show that our method outperforms existing state-of-the-art methods in adaptive crowd counting. The code will be available at https://github.com/HopooLinZ/MGOT.

This is an erratum for the article "A Dual Perspective Framework of Knowledge-correlation for Cross-domain Recommendation" published in ACM Trans. Knowl. Discov. Data 18(6): 152:1-152:28 (2024).

Satellite computing has emerged as a promising technology for next-generation wireless networks. This innovative technology provides data processing capabilities, which facilitates the widespread implementation of artificial intelligence (AI)-based applications, especially for image processing tasks involving deep neural network (DNN). With the limited computing resources of an individual satellite, independently handling DNN tasks generated by diverse user equipments (UEs) becomes a significant challenge. One viable solution is dividing a DNN task into multiple subtasks and subsequently distributing them across multiple satellites for collaborative computing. However, it is challenging to partition DNN appropriately and allocate subtasks into suitable satellites while ensuring load balancing. To this end, we propose a collaborative satellite computing system designed to improve task processing efficiency in satellite networks. Based on this system, a workload-balanced adaptive task splitting scheme is developed to equitably distribute the workload of DNN slices for collaborative inference, consequently enhancing the utilization of satellite computing resources. Additionally, a self-adaptive task offloading scheme based on a genetic algorithm (GA) is introduced to determine optimal offloading decisions within dynamic network environments. The numerical results illustrate that our proposal can outperform comparable methods in terms of task completion rate, delay, and resource utilization.

Drone-based crowd tracking faces difficulties in accurately identifying and monitoring objects from an aerial perspective, largely due to their small size and close proximity to each other, which complicates both localization and tracking. To address these challenges, we present the Density-aware Tracking (DenseTrack) framework. DenseTrack capitalizes on crowd counting to precisely determine object locations, blending visual and motion cues to improve the tracking of small-scale objects. It specifically addresses the problem of cross-frame motion to enhance tracking accuracy and dependability. DenseTrack employs crowd density estimates as anchors for exact object localization within video frames. These estimates are merged with motion and position information from the tracking network, with motion offsets serving as key tracking cues. Moreover, DenseTrack enhances the ability to distinguish small-scale objects using insights from the visual-language model, integrating appearance with motion cues. The framework utilizes the Hungarian algorithm to ensure the accurate matching of individuals across frames. Demonstrated on DroneCrowd dataset, our approach exhibits superior performance, confirming its effectiveness in scenarios captured by drones. Our code will be available at: https://github.com/Zebrabeast/DenseTrack.

Cross-domain recommendation (CDR) aims to utilize the information from relevant domains to guide the recommendation task in the target domain, and shows great potential in alleviating the data sparsity and cold-start problems of recommender systems. Most existing methods utilize the interaction information (e.g., ratings and clicks) or consider auxiliary information (e.g., tags and comments) to analyze the users' cross-domain preferences, but such kinds of information ignore the intrinsic semantic relationship of different domains. In order to effectively explore the inter-domain correlations, encyclopedic knowledge graphs (KG) involving different domains are highly desired in cross-domain recommendation tasks because they contain general information covering various domains with structured data format. However, there are few datasets containing KG information for CDR tasks, so in order to enrich the available data resource, we build a KG-enhanced cross-domain recommendation dataset, named Amazon-KG, based on the widely used Amazon dataset for CDR and the well-known KG DBpedia. In this work, we analyze the potential of KG applying in cross-domain recommendations, and describe the construction process of our dataset in detail. Finally, we perform quantitative statistical analysis on the dataset. We believe that datasets like Amazon-KG contribute to the development of knowledge-aware cross-domain recommender systems. Our dataset has been released at https://github.com/WangYuhan-0520/Amazon-KG-v2.0-dataset.

Exploring new therapeutic diseases for existing drugs plays an essential role in reducing drug development costs. However, existing methods for predicting drug–disease association (DDA) lack fusion to multi-neighborhood information, which limits their ability to generalize and forces them to rely on prior knowledge. To this end, we propose a novel DDA model called the Neighborhood Contrastive Learning Heterogeneous Networks (NCH-DDA). NCH-DDA uses both single-neighborhood and multi-neighborhood feature extraction modules to extract important features of drugs and diseases in parallel from multiple potential spaces, such as heterogeneous networks and similarity networks. NCH-DDA fuses single-neighborhood and multi-neighborhood features using contrastive learning to enhance information interaction in different neighborhood spaces, ultimately obtaining universal domain features of drugs and diseases. NCH-DDA uses a combination of predictive loss and triplet loss to reduce dependence on prior knowledge. In different partition schemes of multiple datasets, NCH-DDA achieved the best performance in predicting DDA, outperforming several current state-of-the-art methods. Moreover, NCH-DDA demonstrated better performance in experiments on data sparsity and drug repositioning for Alzheimer’s disease, indicating its greater potential in DDA prediction with sparse omics data and drug repositioning applications.

Federated Learning (FL) preserves privacy by training a global model via gradient exchange between the parameter server and local clients rather than raw data sharing. Edge devices with sensors serve as local clients receiving multimodal signals and contributing multimodal data for training. Despite the privacy-centric design of FL, it remains vulnerable to gradient leakage attacks. However, existing studies predominantly focus on single-modality data recovery from gradients, leaving a critical research void in multimodal data scenarios. In this letter, we proposeMGIS: Mutual Gradient Inversion Strategy, the first gradient inversion attack and defense paradigm dealing with multimodal data. Inspired by knowledge distillation, MGIS utilizes common information (e.g., labels) between different modalities to extract multimodal data from gradients. Experimental results demonstrate that MGIS outperforms single-modality gradient attacks in the quality of privacy data recovery and highlight the increased privacy leakage risk associated with multi-modality data compared to single-modality data.

Food computing, a newly emerging topic, is closely linked to human life through computational methodologies. Meal recommendation, a food-related study about human health, aims to provide users a meal with courses constrained from specific categories (e.g., appetizers, main dishes) that can be enjoyed as a service. Historical interaction data, important user information, is often used by existing models to learn user preferences. However, if a user’s preferences favor less healthy meals, the model will follow that preference and make similar recommendations, potentially negatively impacting the user’s long-term health. This emphasizes the necessity for health-oriented and responsible meal recommendation systems. In this article, we propose a healthiness-aware and category-wise meal recommendation model called CateRec, which boosts healthiness exposure by using nutritional standards as knowledge to guide the model training. Two fundamental questions are raised and answered: (1) How can the healthiness of meals be evaluated? Two well-known nutritional standards from the World Health Organization and the United Kingdom Food Standards Agency are used to calculate the healthiness score of the meal. (2) How can the model training be guided in a health-oriented manner? We construct category-wise personalization partial rankings and category-wise healthiness partial rankings, and theoretically analyze that they meet the necessary properties and assumptions required to be trained by the maximum posterior estimator under Bayesian probability. The data analysis confirms the existence of user preferences leaning towards less healthy meals in two public datasets. A comprehensive experiment demonstrates that our CateRec effectively boosts healthiness exposure in terms of mean healthiness score and ranking exposure while being comparable to the state-of-the-art model in terms of recommendation accuracy.

Recommender System provides users with online services in a personalized way. The performance of traditional recommender systems may deteriorate because of problems such as cold-start and data sparsity. Cross-domain Recommendation System utilizes the richer information from auxiliary domains to guide the task in the target domain. However, direct knowledge transfer may lead to a negative impact due to data heterogeneity and feature mismatch between domains. In this article, we innovatively explore the cross-domain correlation from the perspectives of content semanticity and structural connectivity to fully exploit the information of Knowledge Graph. First, we adopt domain adaptation that automatically extracts transferable features to capture cross-domain semantic relations. Second, we devise a knowledge-aware graph neural network to explicitly model the high-order connectivity across domains. Third, we develop feature fusion strategies to combine the advantages of semantic and structural information. By simulating the cold-start scenario on two real-world datasets, the experimental results show that our proposed method has superior performance in accuracy and diversity compared with the SOTA methods. It demonstrates that our method can accurately predict users’ expressed preferences while exploring their potential diverse interests.

The domain shift of crowd scenes significantly hinders the application of crowd counting models in open scenarios. Although domain adaptation methods for crowd counting have bridged this gap to some extent, they ignore one of the significant causes of domain shift, which is the inter-domain data distribution bias. We discover that there exists a connection between the known and unknown distribution, which can be utilized by similarity mining to address the domain shift. However, there are still challenges related to insufficient and inaccurate similarity mining. In this article, a novel Fine-grained Inter-domain Similarity Mining (FSIM) framework is proposed. To comprehensively explore the similar distributions between source and target domains, we propose a Multi-scale Distribution Alignment (MDA) module based on diffusion retrieval. To enhance the reliability of inter- domain similarity mining, we propose a Multi-retrieval Refinement (MR) module based on evidence theory, which serves as an uncertainty measurement method. Eventually, to eliminate the data distribution bias, we perform model retraining using a similar distribution. Extensive experiments conducted on five standard crowd counting benchmarks, SHA, SHB, QNRF, NWPU, and JHU-CROWD++, show that the proposed FSIM has strong generalizability.

Current sentence-level evidence extraction based methods may lose the discourse coherence of legal articles since they tend to make the extracted sentences scattered over the article. To solve the problem, this paper proposes a Cascaded Answer-guided key segment learning framework for long Legal article Question Answering, namely CALQA. The framework consists of three cascaded modules: Sifter, Reader, and Responder. The Sifter transfers a long legal article into several segments and works in an answer-guided way by automatically sifting out key fact segments in a coarse-to-fine approach through multiple iterations. The Reader utilizes a set of attention mechanisms to obtain semantic representations of the question and key fact segments. Finally, considering it a multi-label classification task the Responder predicts final answers in a cascaded manner. CALQA outperforms state-of-the-art methods in CAIL 2021 Law dataset.

As a crucial task in natural language processing, sentiment analysis is widely applied in different fields. However, various implicit expressions interspersed in explicit expressions have become a stumbling block for the further improvement of sentiment analysis. Existing studies usually introduce external knowledge, such as knowledge graph, the context of implicit expressions, or external corpus, to enhance implicit sentiment detection. Different from them, the core idea of this paper is to form explicit-implicit bundles to ensure each batch has the two expressions, which does not rely on external resources. The proposed MB-CL is a Mutual learning model with Bundle-aware Contrastive Learning for the implicit sentiment analysis. Our MB-CL leverages mutual learning between contextualized and local-perceived feature extractors and adopts bundle-aware contrastive learning to align explicit and implicit expressions with the same sentiment in the embedding space while pushing apart expressions with different sentiments. The experiment results on the SMP2019 dataset show that MB-CL outperforms comparative methods in terms of F1 measure.

Propaganda, commonly used in memes disinformation, can influence the thinking of the audience and increase the reach of communication. Usually logical fallacy, as a kind of popular expression of memes, aims to create a logical reasonable illusion where the conclusion cannot be drawn with the use of correct logic rules. However, this characteristic inherent in memes leads to difficulties for classic multi-label classifiers to understand propagation techniques. To this end, we propose a novel propaganda detection model called Antipersuasion Prompt Enhanced Contrastive Learning (abbreviated as APCL). First, our APCL reformulates the multi-label classification task by leveraging the category words of propaganda technique based prompt engineering, which is converted into an image-text matching (ITM). Second, prompt engineering is designed with a persuasion prompt template and an anti-persuasion prompt template. The former is to build matched text-image pairs, and the latter is to form mismatched text-image pairs which fit the logical fallacy style of memes. Finally, the propagation technique is predicted based on the distances between the above two prompt templates enhanced texts and an input image. Experimental results on the memes dataset of SemEval2021 task 6 show that our APCL outperforms the state-of-the-art multimodal classification models in terms of F1 measures.

Bundle recommendation suggests a set of items to users against their complex needs, where user-bundle interaction learning is key. It is observed that Gromov’s δ-hyperbolicity of the interaction graph in bundle recommendation is smaller (lower is more hyperbolic) than those in traditional item recommendation when measuring a graph’s tree likeness. However, state-of-the-art bundle recommendation methods learn to embed the entities (user, bundle, item) of tree-like interaction graph in Euclidean space, which could cause severe distortion problems. We argue hyperbolic space provides a promising way to get accurate entity embeddings, with this paper proposing a novel bundle recommendation model. The model learns user preferences via hyperbolic graph convolution, aiming at decreasing the distortion of bundle graph node embeddings. Extensive empirical experiments conducted on two real-world datasets confirm that our model achieves promising performance compared to baseline methods representing state-of-the-art bundle recommendation methods.

Domain adaptation is commonly employed in crowd counting to bridge the domain gaps between different datasets. However, existing domain adaptation methods tend to focus on inter-dataset differences while overlooking the intra-differences within the same dataset, leading to additional learning ambiguities. These domain-agnostic factors,e.g., density, surveillance perspective, and scale, can cause significant in-domain variations, and the misalignment of these factors across domains can lead to a drop in performance in cross-domain crowd counting. To address this issue, we propose a Domain-agnostically Aligned Optimal Transport (DAOT) strategy that aligns domain-agnostic factors between domains. The DAOT consists of three steps. First, individual-level differences in domain-agnostic factors are measured using structural similarity (SSIM). Second, the optimal transfer (OT) strategy is employed to smooth out these differences and find the optimal domain-to-domain misalignment, with outlier individuals removed via a virtual "dustbin'' column. Third, knowledge is transferred based on the aligned domain-agnostic factors, and the model is retrained for domain adaptation to bridge the gap across domains. We conduct extensive experiments on five standard crowd-counting benchmarks and demonstrate that the proposed method has strong generalizability across diverse datasets. Our code will be available at: https://github.com/HopooLinZ/DAOT/.

Visual information shows to empower accurately named entity recognition in short texts, such as posts from social media. Previous work on multimodal named entity recognition (MNER) often regards an image as a set of visual objects, trying to explicitly align visual objects and entities. However, these methods may suffer the bias introduced by visual objects when they are not identical to entities in quantity and entity type. Different from this kind of explicit alignment, we argue that implicit alignment is effective in optimizing the shared semantic space learning between text and image for improving MNER. To this end, we propose a de-bias contrastive learning based approach for MNER, which studies modality alignment enhanced by cross-modal contrastive learning. Specifically, our contrastive learning adopts a hard sample mining strategy and a debiased contrastive loss to alleviate the bias of quantity and entity type, respectively, which globally learns to align the feature spaces from text and image. Finally, the learned semantic space works with a NER decoder to recognize entities in text. Conducted on two benchmark datasets, experimental results show that our approach outperforms the current state-of-the-art methods.

Task offloading has been widely used to extend the battery life of intelligent mobile devices. Existing task offloading approaches, focusing on perfecting the balance between latency and energy consumption, completely ignore the impacts of the user preference caused by low battery anxiety. The existence of low battery anxiety - mobile users’ common fear of losing battery energy, especially when the battery energy is already low - causes users to trade high latency for prolonged battery life. Taking into account the user preference impacts on task offloading, we propose a novel offloading approach calledUPOAto obtain refined offloading policies between low latency and energy consumption based on user preferences. In UPOA, we start the study by defining a user preference rule that determines users’ offloading preferences according to battery energy status. Then, we build a fine-grained task offloading model to delineate the task distribution characteristics of each node in its offloading link. Guided by this model, we develop a task prediction algorithm based on the long-short-term-memory neural network model to provide task predictions that facilitate offloading policies. Lastly, we implement a particle-swarm-optimization-based online offloading algorithm. The offloading algorithm provides the best long-term offloading policies by incorporating the user preference determined by our user preference rule and the task predictions generated by our task prediction algorithm. To quantitatively evaluate the performance of UPOA, we conduct extensive experiments in a real-world cloud-edge environment. We compare UPOA with three state-of-the-art offloading approaches, DRA, DRL-E2D, and MUDRL under various conditions. Experimental results demonstrate that UPOA can make effective policies based on user preferences compared with the existing approaches. UPOA reduces average latency by 12.49% when battery energy is sufficient and extends battery life by 20.14% when battery energy is low.

Adaptive traffic signal control is an important and challenging real-world problem that fits well with the task framework of deep reinforcement learning. As one of the critical design elements, the environmental state plays a crucial role in traffic signal control decisions. The state definitions of most existing works mostly contain lane-level queue length, intersection phase, and other features. However, these works are heuristically designed in representing states. This results in highly sensitive and unstable performances of next actions. The paper proposes a Cooperative Max-Pressure enhanced State Learning for the traffic signal control (CMP-SL), which is inspired by the advanced pressure definition for an intersection in the transportation field to cope with this problem. First, our CMP-SL explicitly extends the cooperative max-pressure to the state definition of a target intersection, aiming to obtain accurate environment information by including the traffic pressures of surrounding intersections. From then on, a graph attention mechanism (GAT) is used to learn the state representation of the target intersection in our spatial-temporal state module. Second, since the state is coupled with the reward in reinforcement learning, our method takes the cooperative max-pressure of the target intersection into the reward definition. Furthermore, a temporal convolutional network (TCN) based sequence model is used to capture the historical state of traffic flow. And the historical spatial-temporal and the current spatial state features are concatenated into a DQN network to predict the Q value and generate each phase action. Finally, experiments with two real-world traffic datasets demonstrate that our method achieves shorter vehicle average times and higher network throughput than the state-of-the-art models.

In social media analysis, gender prediction is one of the most important tasks of user profiling. Web users often post messages in a timeline manner to record their living moments. These messages containing texts and images, constitute long multi-modal data that potentially represents the living style, preference, or opinion regarding users. Therefore, it is feasible to predict the gender of a user by utilizing such living moments. However, the rich modalities (time, length, text, and image) of living moments with difficult challenges have not been fully exploited by the research communities for practical applications. To this end, we propose a novel gender prediction framework based on user-posted living Moments MomNet). The MomNet mainly consists of a moment memory module and a central executive module inspired by the two characteristics of working memory theory. One is that humans can associate related information to facilitate memory. Our moment memory module aggregates similar uni-modal moments of a user to form different chunks and encode the chunks into moment memory representations. The other is that humans coordinate information from different modalities to make judgments. Our central executive module is designed to coordinate comprehensive attentions of moment memory representations from texts, images, and their combinations. Finally, a softmax classifier is used to predict gender. Extensive experiments conducted on a real-world public dataset show that our framework achieves 86.63% accuracy and outperforms all state-of-the-art methods in terms of accuracy.

With the development of Internet of Things applications, traditional cloud computing cannot meet the new application requirements, and the mode of cloud edge collaborative processing is becoming more and more important. Cloud computing workflow scheduling algorithm seldom considers the heterogeneity of cloud edge devices, so it is difficult to achieve the balance of completion time, energy consumption, and cost. Cloud-edge collaborative workflow scheduling can reasonably allocate tasks according to the characteristics of dependencies and heterogeneous resources, which can solve the imbalance between performance and cost as well as between performance and energy consumption. Therefore, we propose a heterogeneous earliest completion time scheduling algorithm (W2DG-HEFT) based on a greedy rotation strategy. Considering the differences in computing performance, transmission delay, energy consumption, and cost of cloud edge computing nodes, the candidate queues selected by cloud and edge 2D weights are adjusted and balanced. The HEFT algorithm, which adopts energy consumption awareness and greedy rotation strategy, allocates the appropriate cloud or edge computing nodes according to the task priority in the workflow. Through simulation platform experiment, compared with GPR-HEFT algorithm, when the cost threshold is 2-3 USD per hour, the scheduling completion time on the workflow of 100 tasks is reduced by about 5.4% on average. Compared with MOACS, PSO, and GA, the scheduling completion time on the workflow of 1000 tasks is reduced by 8.28% on average, and the average comprehensive cost is reduced by 23% on the condition of satisfying the time constraints.

Understanding the texts in visual scenes is essential for reasoning tasks when texts carry key information. Therefore, it is a common way to learn multi-modal representation from visual scenes to support Scene-Text Visual Question&Answering. Different modalities, such as text and image, are embedded into a joint semantic space where attention mechanism is widely applied. This paper regards Scene-Text VQA as a kind of cross-modal task where the joint embedding between a scene image and its text answer is a semantic bridge to share the strong semantic clues with the two modalities. To this end, our proposed framework firstly divides the scene recognition features into two types: visual and textual features. And then external cross-modal pre-trained features are introduced to simultaneously guide the representation of visual and text features through cross-modal attention net-works. Finally, Transformer is applied as a decoder to output answers. Conducted on a public Scene-Text VQA dataset, experimental results show that our proposed framework out-performs existing approaches in terms of accuracy.

Deep learning improves the performance of crowd counting, but model migration remains a tricky challenge. Due to the reliance on training data and inherent domain shift, model application to unseen scenarios is tough. To facilitate the problem, this paper proposes a cross-domain Fine-Grained Fragment Diffusion model (FGFD) that explores feature-level fine-grained similarities of crowd distributions between different fragments to bridge the cross-domain gap (content-level coarse-grained dissimilarities). Specifically, we obtain features of fragments in both source and target domains, and then perform the alignment of the crowd distribution across different domains. With the assistance of the diffusion of crowd distribution, it is able to label unseen domain fragments and make source domain close to target domain, which is fed back to the model to reduce the domain discrepancy. By monitoring the distribution alignment, the distribution perception model is updated, then the performance of distribution alignment is improved. During the model inference, the gap between different domains is gradually alleviated. Multiple sets of migration experiments show that the proposed method achieves competitive results with other state-of-the-art domain-transfer methods.

There is thereby an urgent need but it is still a significant challenge to solve long Chinese t ext sentiment. BERT-based pre-trained language model (PLM) has been demonstrated to be the state-of-the-art approach for sentiment analysis. However, BERT can only process 510 tokens at a time, limiting the accuracy of sentiment analysis for long texts. Meanwhile, existing long text truncation methods for this BERT deficiency perform still weak in capture core sentiments. Aiming to better solve long text sentiment analysis, we propose a BERT-based fusion model. Firstly, we elaborately devise a new truncation method to gain four types of embeddings and resample the dataset to alter the imbalanced distribution of labels. Secondly, N BERT-based models are leveraged to joint learn the above four embeddings. Thirdly, we adopt a BiGRU network to fuse N BERT-based models and further utilizes attention mechanism to obtain effective core sentiments of the long Chinese texts. In the end, we employ three ensemble algorithms to optimize our model, improving Micro and Macro F1 by 1.68% and 1.53% respectively.

Person search is a significant computer vision task that requires addressing person detection and re-identification simultaneously. Body parts are frequently misaligned due to variation poses, occlusions, and partial missing, leading to the unsatisfied results of person search. Existing methods usually extract local features from the human body by the key point information, that cannot tackle the recognition task between a pair of persons with different body parts due to misalignment. Moreover, these methods overlook background information (e.g. the carries and the background reference object) which can also supplement effective features for representing the person. In this paper, we propose a part-aligned network with background (PANB) to address this misalignment issue. To learn local fine-grained features of different body parts, we fine-tune a parsing network to divide the body region into seven parts. In particular, our proposed method considers extracting the background features as the eighth part features to extract more robust representations, which is more rational and efficient. Furthermore, we design a reconstruction method to align the parts existing in both the query image and the cropped gallery image. Extensive experiments show that our proposed method achieves competitive performance on CUHK-SYSU and PRW datasets.

Image-text retrieval is one of the most common tasks in multimodal retrieval. It suffers from the problem of information imbalance between modalities, which is so-called modality gap. It remains challenging because prior methods cannot bridge the gap reasonably. With the help of scene graph, we start by designing an auxiliary bi-level graph representation (ABGR) pipeline that can fully mine the potential information and reduce the information redundancy. By doing so, each modality will be represented by lexical word graph that carries the main content of the information. Specifically, we design a graph feature enhancement (GFE) module to embed the graph-structured information in a common subspace while exploring the relationship between lexical words. As a result, a better representation for both image and text can be obtained, which helps us to evaluate the similarity between images and texts more reasonably. Experimental results conducted on two benchmark datasets Flickr30K and MS-COCO demonstrate the effectiveness of our proposed model for cross-modal retrieval task.

In real traffic scenarios, the changes of vehicle resolution that the camera captures tend to be relatively obvious considering the distances to the vehicle, different directions, and height of the camera. When the resolution difference exists between the probe and the gallery vehicle, the resolution mismatch will occur, which will seriously influence the performance of the vehicle re-identification (Re-ID). This problem is also known as multi-resolution vehicle Re-ID. An effective strategy is equivalent to utilize image super-resolution to handle the resolution gap. However, existing methods conduct super-resolution on global images instead of local representation of each image, leading to much more noisy information generated from the background and illumination variations. In our work, a local-enhanced multi-resolution representation learning (LMRL) is therefore proposed to address these problems by combining the training of local-enhanced super-resolution (LSR) module and local-guided contrastive learning (LCL) module. Specifically, we use a parsing network to parse a vehicle into four different parts to extract local-enhanced vehicle representation. And then, the LSR module, which consists of two auto-encoders that share parameters, transforms low-resolution images into high-resolution in both global and local branches. LCL module can learn discriminative vehicle representation by contrasting local representation between the high-resolution reconstructed image and the ground truth. We evaluate our approach on two public datasets that contain vehicle images at a wide range of resolutions, in which our approach shows significant superiority to the existing solution.

Massive Online Open Courses (MOOC) have popularized educational opportunities for students all over the world, while immensely high dropout is becoming a central challenge nowadays.Most researches predict course dropout labels through analyzing the student engagement data.However, these models have high structural complexity with high time cost and cannot provide in-depth insights into why a student is likely to drop out.We devise a lightweight pipeline to simplify the MOOC dropout problem, grasp the core features to make student behaviours interpretable at the model and instance level, visualize the changing trend of predicted label probability estimation with feature values for longitudinally interpreting the sample student behaviours.Based on qualitative insights and quantitative analysis, our main findings are that shorter videos and instructors speak fast are more engaging.Most students complete MOOC learning with a rapid speed, while a few students who watch the video slowly have a higher completion rate.When the frequency of fast-forwarding increases while the percentage of videos watching decreases, the likelihood to drop this course raises.In the end, our pipeline achieves 69.52% AUC, 0.744 R-squared and 0.553 R-squared with 0.982s inference time on the 20238 sample student data.

The outbreak of the COVID-19 pandemic arises enormous attention to online education then knowledge tracking is an increasingly crucial task with its vigorous development.However, the surge of student historical interactions and the lack of prior knowledge is engendering a sequence of issues, such as the decrease in prediction accuracy while the increase in training time.Simultaneously, most existing approaches fail to provide in-depth insights into why a student is likely to answer the question incorrectly and what affects the knowledge state of the student.To address those issues, we propose a multi-head self-attention model named MAKT for dynamic knowledge tracing, which makes the prediction results interpretable at the model and instance level.The customized multi-head self-attention layer has high training efficiency owing to its parallelization capability and spends about 6 seconds in each epoch on a single GPU.We further visualize the attention weights of MAKT and student knowledge acquisition tracking, finding that not all historical interactions are equally important but the recent interactions profoundly establish the knowledge state of students.In the end, extensive experiments on three datasets demonstrate the robustness and superiorities of MAKT, improving ACC by 1.14 % and AUC by 1.20 % on average.

Multi-scale approach representing image objects at various levels-of-details has been applied to various computer vision tasks. Existing image classification approaches place more emphasis on multi-scale convolution kernels, and overlook multi-scale feature maps. As such, some shallower information of the network will not be fully utilized. In this paper, we propose the Multi-Scale Residual (MSR) module that integrates multi-scale feature maps of the underlying information to the last layer of Convolutional Neural Network. Our proposed method significantly enhances the characteristics of the information in the final classification. Extensive experiments conducted on CIFAR100, Tiny-ImageNet and large-scale CalTech-256 datasets demonstrate the effectiveness of our method compared with Res-Family.

Backward tracks have been exploited to improve performance of multi-object tracking (MOT). The existing method brings a stable similarity measurement but neglects unreliable detection. Exploiting predictions of forward tracks has emerged as a popular approach to tackle the task of tracking-by-detection. However, it's observed that missing detection has not been solved well enough which would significantly influence the tracking accuracy. Thus, obtaining more proposals from dual-direction tracking and predictions of tracks is concerned to address the problem of missing detection. In this paper, we propose a dual-direction perception and collaboration network (DPCNet) for MOT that exploits forward and backward tracking to collaboratively track objects. It collects candidates from the dual directions so that they can complement each other in different scenarios. Moreover, we propose a near-online tracking model based on DPCNet to improve the efficiency, which batches the tracking and makes forward and backward tracking in parallel. Experiments conducted on MOT challenge benchmarks demonstrate that the proposed method outperforms the state-of-the-arts.

The keypoint based detectors are a relatively new object detection mechanism, avoiding the complicated computation related to anchor box and achieving state-of-the-art accuracy. However, inference speed is a major drawback of these detectors because of the heavy backbone network. In this paper, we design a novel lightweight backbone named DNet for keypoint-based detection and propose a real-time object detection network. In the backbone part, DNet is able to maintain high-resolution feature maps throughout the process and gradually extract and integrate features across scales. In the detection part, we detect a center keypoint and a pair of corners to predict the bounding boxes, and completely avoid the complicated computation related to anchor boxes. Compared with state-of-the-art real-time detectors, our network achieves superior performance with 30.0% AP on COCO benchmark at 21. 5ms. In addition, the experimental results show that our network is capable of running real-time on embedded devices.

Recent advancements in convolutional neural networks based object detection have enabled analyzing the mounting video data with high accuracy. However, inference speed is a major drawback of these video analysis system because of the heavy object detectors. To address the computational and practicability challenges of video analysis, we propose FastQ, a system for efficient querying over video at scale. Given a target video, FastQ can automatically label the category and number of objects for each frame. We introduce a novel lightweight object detector named FDet to improve the efficiency of query system. First, a difference detector filters the frames whose difference is less than the threshold. Second, FDet is employed to efficiently label the remaining frames. To reduce inference time, FDet detects a center keypoint and a pair of corners from the feature map generated by a lightweight backbone to predict the bounding boxes. FDet completely avoid the complicated computation related to anchor boxes. Compared with state-of-the-art real-time detectors, FDet achieves superior performance with 29.1% AP on COCO benchmark at 25.3ms. Experiments show that FastQ achieves 150 times to 300 times speed-ups while maintaining more than 90% accuracy in video queries.

With explosive surveillance data during day and night, visible-infrared person re-identification (VI-ReID) is an emerging challenge due to the apparent cross-modality discrepancy between visible and infrared images. Existing VI-ReID work mainly focuses on learning a robust feature to represent a person in both modalities despite the modality gap cannot be effectively eliminated. Recent research works have proposed various generative adversarial network (GAN) models to transfer the visible modality to another unified modality, aiming to bridge the cross-modality gap. However, they neglect the information loss caused by transferring the domain of visible images which is significant for identification. To effectively address the problems, we observe that key information such as textures and semantics in an infrared image can help to color the image itself and the colored infrared image maintains rich information from infrared image while reducing the discrepancy with the visible image. We therefore propose a colorization-based Siamese generative adversarial network (CoSiGAN) for VI-ReID to bridge the cross-modality gap, by retaining the identity of the colored infrared image. Furthermore, we also propose a feature-level fusion model to supplement the transfer loss of colorization. The experiments conducted on two cross-modality person re-identification datasets demonstrate the superiority of the proposed method compared with the state-of-the-arts.

Panoramic video is a sort of video recorded at the same point of view to record the full scene. With the development of video surveillance and the requirement for 3D converged video surveillance in smart cities, CPU and GPU are required to possess strong processing abilities to make panoramic video. The traditional panoramic products depend on post processing, which results in high power consumption, low stability and unsatisfying performance in real time. In order to solve these problems, we propose a real-time panoramic video stitching framework. The framework we propose mainly consists of three algorithms, L-ORB image feature extraction algorithm, feature point matching algorithm based on LSH and GPU parallel video stitching algorithm based on CUDA. The experiment results show that the algorithm mentioned can improve the performance in the stages of feature extraction of images stitching and matching, the running speed of which is 11.3 times than that of the traditional ORB algorithm and 641 times than that of the traditional SIFT algorithm. Based on analyzing the GPU resources occupancy rate of each resolution image stitching, we further propose a stream parallel strategy to maximize the utilization of GPU resources. Compared with the L-ORB algorithm, the efficiency of this strategy is improved by 1.6–2.5 times, and it can make full use of GPU resources. The performance of the system accomplished in the paper is 29.2 times than that of the former embedded one, while the power dissipation is reduced to 10 W.

Efficient and accurate semantic segmentation is particularly important especially for applications like autonomous driving which requires real-time inference speed and high performance. Many works try to compromise spatial resolution to achieve real-time inference speed, which leads to poor performance. As a result, real-time segmentation task for embedded devices is still an open problem. In this paper, we focus on building a network with better performance possible while still achieve real-time inference speed. We first use a pyramid kernel size to capture more spatial information instead of using just a 3×3 kernel size for DWConvolution in ShuffleNet v2. Meanwhile, an efficient Vortex Pooling module is employed to aggregate the contextual information and generate high-resolution features. Compared with other state-of-the-art real-time semantic segmentation networks, the proposed network achieves similar inference speed and better performance on embedded device. Specifically, we achieve state-of-the-art 73.46% mean IoU on Cityscapes test dataset, for a 768×1024 input, a speed of 46.1 frames per second on NVIDIA Jetson AGX Xavier embedded development board is achieved.

In recent years, how to use renewable energy to reduce the energy cost of internet data center (IDC) has been an urgent problem to be solved. More and more solutions are beginning to consider machine learning, but many of the existing methods need to take advantage of some future information, which is difficult to obtain in the actual operation process. In this paper, we focus on reducing the energy cost of IDC by controlling the energy flow of renewable energy without any future information. we propose an efficient energy dynamic control algorithm based on the theory of reinforcement learning, which approximates the optimal solution by learning the feedback of historical control decisions. For the purpose of avoiding overestimation, improving the convergence ability of the algorithm, we use the double [Formula: see text]-method to further optimize. The extensive experimental results show that our algorithm can on average save the energy cost by 18.3% and reduce the rate of grid intervention by 26.2% compared with other algorithms, and thus has good application prospects.

This paper presents a novel approach for optical character recognition (OCR) on acceleration and to avoid underfitting by text. Previously proposed OCR models typically take much time in the training phase and require large amount of labelled data to avoid underfitting. In contrast, our method does not require such condition. This is a challenging task related to transferring the character sequential relationship from text to OCR. We build a model based on transductive transfer learning to achieve domain adaptation from text to image. We thoroughly evaluate our approach on different datasets, including a general one and a relatively small one. We also compare the performance of our model with the general OCR model on different circumstances. We show that (1) our approach accelerates the training phase 20-30% on time cost; and (2) our approach can avoid underfitting while model is trained on a small dataset.

Energy data, which consists of energy consumption statistics and other related data in green data centers, grows dramatically. The energy data has great value, but many attributes within it are redundant and unnecessary. Thus attribute reduction for the energy data has been conceived as a critical step. However, many existing attribute reduction algorithms are often computationally time-consuming. To address these issues, we extend the methodology of rough sets to construct data center energy consumption knowledge representation system. By taking good advantage of in-memory computing, an attribute reduction algorithm for energy data using Spark is proposed. In this algorithm, we use a heuristic formula for measuring the significance of attribute to reduce search space, and an efficient algorithm for simplifying energy consumption decision table, which further improve the computation efficiency. The experimental results show the speed of our algorithm gains up to 0.28X performance improvement over the traditional attribute reduction algorithm using Spark.

With the advent of cloud computing, renewable energy is integrated into data center power supply systems increasingly. The power statistics collection may not be available due to the instability of renewable energy, which results in incomplete data. The incomplete energy data will significantly disturb the management of data centers. We further propose a filling algorithm based on complete tolerance class. The algorithm expands the traditional tolerance relation, and fills the missing values of the energy data, which ensures the data integrity. By taking good advantage of in-Memory Computing, We further parallelize and optimize our algorithm using Spark. The experiment results demonstrate that our algorithm outperforms other general filling algorithms in terms of filling accuracy. The proposed algorithm also shows good performance as the missing rate rises up.

As big data applications proliferate, datacenters today are increasingly looking to adopt a scale-out model. Nevertheless, power capacity has become an important bottleneck that restricts horizontal scaling of servers, especially in datacenters that oversubscribe power infrastructure. When a datacenter hits its ceiling for power provisioning, conventionally the owner has to either build another facility or upgrade existing infrastructure-both approaches add huge cost, require significant time, and can further increase carbon footprint. This paper proposes Oasis, a novel datacenter expansion strategy that enables power-/carbon- constrained servers to scale out economically and sustainably. The basic structure of Oasis, called Oasis Node, naturally supports incremental capacity expansion with near-zero environmental impact since it leverages modular solar panels and distributed battery systems to power newly added servers. To optimize the operation of newly added nodes, we further propose a management framework called Ozone. It allows Oasis to jointly perform power supply switching and server speed scaling to improve efficiency locally and globally. We implement a prototype of Oasis and use it as a research platform for evaluating the design tradeoffs of green scale-out datacenters. With Oasis, a green datacenter could gradually double its capacity with near-oracle performance, extended battery lifetime, and 26 percent cost savings.

Convolutional Neural Networks (CNNs) have substantially advanced the state-of-the-art accuracies of object recognition, which is the core function of a myriad of modern multimedia processing techniques such as image/video processing, speech recognition, and natural language processing. GPU-based accelerators gained increasing attention because a large amount of highly parallel neurons in CNN naturally matches the GPU computation pattern. In this work, we perform comprehensive experiments to investigate the performance bottlenecks and overheads of current GPU acceleration platform for scale-out CNN-based big data processing.

Recent years have seen an explosion of data volumes from a myriad of distributed sources such as ubiquitous cameras and various sensors. The challenges of analyzing these geographically dispersed datasets are increasing due to the significant data movement overhead, time-consuming data aggregation, and escalating energy needs. Rather than constantly move a tremendous amount of raw data to remote warehouse-scale computing systems for processing, it would be beneficial to leverage in-situ server systems (InS) to pre-process data, i.e., bringing computation to where the data is located.

As cloud applications proliferate and data-processing demands increase, server resources must grow to unleash the performance of emerging workloads that scale well with large number of compute nodes. Nevertheless, power has become a crucial bottleneck that restricts horizontal scaling (scale out) of server systems, especially in datacenters that employ power over-subscription. When a datacenter hits the maximum capacity of its power provisioning equipment, the owner has to either build another facility or upgrade existing utility power infrastructure -- both approaches add huge capital expenditure, require significant construction lead time, and can further increase the owner's carbon footprint.

As microprocessors become more complex, early design space exploration plays an essential role in reducing the time to market and post-silicon surprises. The trend toward multi-/many- core processors will result in sophisticated large-scale architecture substrates (e.g. non-uniformly accessed caches interconnected by network-on-chip) that exhibit increasingly complex and heterogeneous behavior. While conventional analytical modeling techniques can be used to efficiently explore the characteristics (e.g. IPC and power) of monolithic architecture design, existing methods lack the ability to accurately and informatively forecast the complex behavior of large and distributed architecture substrates across the design space. This limitation will only be exacerbated with the rapidly increased integration scale (e.g. number of cores per chip). In this paper, we propose novel, multi-scale 2D predictive models which can efficiently reason the characteristics of large and sophisticated multi-core oriented architectures during the design space exploration stage without using detailed cycle-level simulations. Our proposed techniques employ 2D wavelet multiresolution analysis and neural network regression modeling. We extensively evaluate the efficiency of our predictive models in forecasting the complex and heterogeneous characteristics of large and distributed shared cache interconnected by a network on chip in multi-core designs using both multi-programmed and multithreaded workloads. Experimental results show that the models achieve high accuracy while maintaining low complexity and computation overhead. Through case studies, we demonstrate that the proposed techniques can be used to informatively explore and accurately evaluate global, cooperative multi-core resource allocation and thermal-aware designs that cannot be achieved using conventional design exploration methods.