3D LiDAR scene completion from point clouds is a fundamental component of perception systems in autonomous vehicles. Previous methods have predominantly employed diffusion models for high‑fidelity reconstruction. However, their multi-step iterative sampling incurs significant computational overhead, limiting its real-time applicability. To address this, we propose LiNeXt: a lightweight, non‐diffusion network optimized for rapid and accurate point cloud completion. Specifically, LiNeXt first applies the Noise‑to‑Coarse (N2C) Module to denoise the input noisy point cloud in a single pass, thereby obviating the multi‑step iterative sampling of diffusion‑based methods. The Refine Module then takes the coarse point cloud and its intermediate features from the N2C Module to perform more precise refinement, further enhancing structural completeness. Furthermore, we observe that LiDAR point clouds exhibit a distance-dependent spatial distribution, being densely sampled at proximal ranges and sparsely sampled at distal ranges. Accordingly, we propose the Distance‑aware Selected Repeat strategy to generate a more uniformly distributed noisy point cloud. On the SemanticKITTI dataset, LiNeXt achieves a 199.8 times speedup in inference, reduces Chamfer Distance by 50.7 percent, and uses only 6.1 percent of the parameters compared with LiDiff. These results demonstrate the superior efficiency and effectiveness of LiNeXt for real-time scene completion.

The KV cache in self-attention has emerged as a major bottleneck in long-context and large-batch inference for LLMs. Existing approaches often treat sparsity prediction and compression as separate modules—relying on auxiliary index structures to select relevant tokens, and on complex quantization schemes to reduce memory usage. This fragmented design introduces redundant overhead and limits scalability. In this paper, we propose a novel paradigm: treating the compressed key representation not merely as storage, but as a self-indexing structure that directly enables efficient sparse attention. By designing a sign-based 1-bit vector quantization (VQ) scheme, our method unifies compression and retrieval in a single, hardware-friendly format. This approach eliminates the need for external indices or learning-based predictors, offering a lightweight yet robust solution for memory-constrained inference. All components are designed to be hardware-efficient and easy to implement. By implementing custom CUDA kernels, our method integrates seamlessly with FlashAttention, minimizing additional runtime and memory overhead. Experimental results demonstrate that our approach delivers both effectiveness and efficiency.

Haoliang Huang, Zihuang Cai, Zhuo Tang, Yifan Liu, Chen Tian, Kenli Li, Changjian Chen. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Pruning is essential for the efficient deployment of Large Language Models (LLMs); however, it causes severe performance degradation due to the structural distortion induced by sparsity.Existing recovery strategies, such as LoRA, predominantly employ global finetuning, often overlooking the mechanistic root of this degradation: the layer-wise accumulation and amplification of local errors.To address this limitation, we propose LaCo (Layerwise Compensation), a framework that reorients the recovery paradigm from global adaptation to hierarchical representation alignment.By sequentially optimizing each layer to reconstruct the model's hidden states, LaCo effectively intercepts the error propagation chain at its source.Extensive experiments demonstrate that LaCo surpasses parameter-efficient baselines in both perplexity reduction and zeroshot reasoning.Notably, it reduces recoverytime memory usage to approximately 1/7 of the baseline and requires only 2,048 unlabeled samples to match a LoRA model trained on 50k examples-achieving a ∼ 25× improvement in data efficiency.

Existing tile size selection approaches are tightly coupled with compiler transformation pipelines, often leading to inaccurate modeling of cache behavior and limited effectiveness for non-rectangular tile shapes. This article presents TileMind , a decoupled analytical model that combines compile-time and runtime information for tile size selection in affine programs. It introduces a transformation-aware pre-tiling step that enables the decoupled selector to remain consistent with compiler transformations while extracting compile-time metadata. The extracted metadata is then combined with profiled runtime characteristics to construct a richer yet tractable feasible domain, within which a nonlinear objective for tile size selection is formulated. This objective is subsequently transformed into a binary product linearization problem, with its nonlinear constraints also linearized for efficient optimization. Finally, an intra-tile optimization aligns computation with data layout to enhance data reuse within tiles. Across two multi-core Intel CPUs, TileMind achieves 1.49× (sequential) and 1.33× (parallel) mean speedups on twenty PolyBench kernels, and 2.08–3.54× speedups on three deep learning workloads over the state-of-the-art analytical model Pluto-tss . Compared with TVM’s latest autotuner MetaSchedule, TileMind delivers 1.35–1.46× mean speedups while reducing tuning overhead by 2–4 orders of magnitude. While demonstrating effectiveness on selecting tile sizes for non-rectangular tile shapes and compatibility with PPCG, Pluto, and TVM, we further provide proof-of-concept results on GPUs, illustrating the potential portability of TileMind across architectures.

The expansion of cloud-based shared storage increases data privacy concerns. While data encryption technologies can safeguard data content, they cannot prevent the leakage of data access patterns. By re-encrypting data and changing storage location after each access, Oblivious Random Access Machine (ORAM) can effectively avoid information leakage from memory access patterns. While ORAM was initially designed for single-user applications, most existing multi-user ORAM solutions have drawbacks, such as dependence on a trusted proxy, high client storage overhead, and low throughput. To address the issues of multi-user ORAM systems, this paper explores the design of proxyless ORAM solutions in shared storage scenarios and proposes MC-ORAM, a new multi-user oblivious data storage framework. It ensures data consistency through client collaboration in a proxyless architecture, achieves higher throughput by differentiating request processing based on privacy protection requirements, and uses recursion for optimization. We implemented MC-ORAM and analyzed its performance using a variety of indicators, as well as conducting comparative evaluations against alternative schemes. The results show that, on average, MC-ORAM reduces response latency by 18.1% and improves throughput by 23.8% compared to TaoStore.

As integrated circuit design continues to scale, three-dimensional integrated circuit (3D-IC) technology has emerged as a promising solution to extend Moore’s Law. Among various physical design stages, placement plays a pivotal role, directly influencing key power, performance, and area (PPA) metrics. In research that does not consider specific 3DIC process technologies, the state-of-the-art ePlace-3D introduces 3D electrostatic field density modeling, enabling nonlinear placement algorithms to truly address 3D placement problems. However, its complex flow results in low runtime efficiency. In this work, we propose LightPlace, a lightweight and effective 3D placement framework. This framework incorporates Virtual Macro Insertion approach and Macro Connection Reconstruction method, which streamline the macro placement phase and allocate space for subsequent standard cell placement. These innovations enhance runtime efficiency and optimize wirelength. Experimental results on the large-scale Modern Mixed-Size (MMS) benchmarks demonstrate that, LightPlace reduces total wirelength by an average of 10% and achieves up to 4× faster runtime compared to the runtime reported for ePlace-3D.

In the Internet of Vehicles (IoV) systems, recognizing driver emotions is crucial to alleviate dangerous driving behaviors caused by emotional instability. Current research predominantly utilizes multimodal data generated by various types of sensors in IoV systems as input to analyze driver emotion changes using multimodal models. However, existing methods are not enough to fully exploit the advantages of large language models (LLM) in information extraction and multimodal feature fusion, which limits the inference capability of emotion recognition models. Therefore, this article proposes an LLM-auxiliary supervision module, which assists in the training phase through LLM to enhance the performance of multimodal emotion recognition models. Specifically, we designed a label text feature extraction (LTFE) module that employs LLM for text data augmentation and extraction, converting label text into semantically informative feature representations. Additionally, we proposed the label-auxiliary supervision (LAS) strategy, which effectively integrates the LLM label text features learned from the LTFE module with the multimodal emotion recognition model during the training phase to enhance the model’s inference ability. Notably, the LTFE and LAS modules are used only during the training phase, ensuring that the backbone model requires minimal computational resources during inference, making it compatible with the computational constraints of intelligent vehicular devices. Extensive experiments conducted on the PPB-Emo, RAVDESS, and IEMOCAP datasets demonstrate that the proposed method outperforms existing approaches in driver emotion recognition tasks.

Graph-structured data appears in domains such as molecular analysis, social networks, and program optimization, where graphs often exhibit implicit heterogeneity, as nodes may look homogeneous in type yet differ significantly in semantics or functionality. Graph Neural Networks (GNNs), while powerful on homophilic graphs, tend to degrade in such settings due to polarity confusion, over-smoothing, and inefficiency caused by dense propagation. We propose a polarity-aware framework for graph classification that addresses these challenges through adaptive directional sparse aggregation. The framework introduces a polarity-aware propagation mechanism that adaptively reinforces or inverts neighbor signals, mitigating contamination under heterophily. A polarity-guided sparse aggregation operator further alleviates over-smoothing, improves scalability by constraining redundant connections, and condenses information flow into more effective representations, while maintaining unbiased estimation with controlled variance. We provide theoretical analyses that characterize the computational complexity, stability properties, and expressive behavior of signed directional aggregation, offering theoretical insights into its computational, stability, and expressive properties. Extensive experiments on molecular and social graph benchmarks with implicit heterophily demonstrate consistent improvements in graph classification accuracy and efficiency. Our method achieves a 2.36% improvement when compared with the strongest baseline on each dataset. In addition, it improves accuracy by 4.53% on average on program optimization strategy recognition tasks, reaching 80.12% overall.

Federated learning (FL) enables collaborative training of a global model while preserving participants' local data privacy, making it ideal for data-sensitive fields like Industrial Internet of Things (IIoT), finance, and healthcare. However, Non-IID data among participants and the presence of malicious participants pose significant challenges to the model's performance and convergence. The global model is difficult to achieve consistent performance across all participants. Therefore, this paper proposes personalized and robust federated learning (PRFL) to handle non-independently and identically distributed (Non-IID) data with malicious participants. First, to enhance the robustness and convergence, a model similarity-based division mechanism is employed. It groups participants with similar data and removes both independent and colluding malicious participants. Second, we propose a three-stage knowledge sharing personalized federated learning framework. Each participant undergoes inner-loop knowledge sharing, outer-loop knowledge sharing, and personalized knowledge distillation, incorporating performance-driven dynamic weighted sharing mechanism. Moreover, extensive experiments demonstrate that PRFL outper forms other advanced personalized federated learning methods across various benchmark datasets, particularly in scenarios with Non-IID data and malicious participants.

Camera-based Semantic Scene Completion (SSC) aims to infer the geometric structure and semantic information in the entire 3D scene from limited 2D images. However, due to the lack of geometric information in the image, existing methods tend to generate fuzzy completion and incorrect semantic boundaries. In this paper, we propose cross-modal knowledge distillation to address this issue, namely PI-Net, which guides the camera-based model to learn accurate 3D geometry to compensate for spatial surroundings information during training. Specifically, we propose a point cloud occupancy prediction model as the teacher, leveraging its output for strong depth supervision signals and spatial voxel information to enhance the student model. To facilitate effective distillation, we design depth guidance distillation to improve geometric predictions, and spatial guidance distillation to assist the student model in better capturing the structural information of the surrounding environment. Finally, prediction domain distillation is incorporated to facilitate holistic learning from point cloud to image. Experimental results demonstrate that PI-Net outperforms state-of-the-art camera-based methods on challenging benchmarks—SemanticKITTI and SSCBench-KITTI-360.

Deep neural networks (DNNs) dominate workloads on cloud and edge platforms. Meanwhile, the hardware platform towards the heterogeneous system with various accelerators. By mapping layers to their different preferred accelerators, the computation cost of each layer can be reduced. While mapping these layers on the same accelerator can reduce the inter-accelerator communication cost. These two costs are often competing and difficult to optimize simultaneously. Therefore, the core challenge in achieving efficient execution of DNN workloads on heterogeneous systems is: how to map layers to achieve the best trade-off between computation and communication costs. Existing works group layers into blocks and perform blockwise mapping to reduce inter-layer communication within blocks. However, when grouping layers, they typically rely on modelagnostic rules, which fail to hide critical inter-layer communication within blocks for diverse DNNs. Moreover, after block mapping, the lack of intra-block resource allocation further increases computation cost of block. In this paper, we proposeGHCoM, a novel block-wise mapping framework for exploring the effective cost trade-offs.GHCoMemploys an adaptive grouping strategy to guide layer grouping based on the topology of DNNs and dynamically adjust the grouping according to the trade-off target. Furthermore,GHCoMconsiders the fine-grained allocation of computation (i.e., processing elements) and communication (i.e., on-chip bandwidth) resources within each block to mitigate interlayer resource contention. To jointly optimize layer grouping, block-wise mapping and intra-block resource allocation,GHCoMleverages a two-level genetic algorithm (GA) with tailored encodings and operators that capture the interdependence across the entire design space. Experiments across various workloads and system configurations show thatGHCoMconsistently outperforms state-of-the-art baselines, achieving 1.08× to 4.79× speedup in execution latency and reducing energy consumption by 1.83% to 87.71%.

Hybrid rice breeding crossbreeds different rice lines and cultivates the resulting hybrids in fields to select those with desirable agronomic traits, such as higher yields. Recently, genomic selection has emerged as an efficient way for hybrid rice breeding. It predicts the traits of hybrids based on their genes, which helps exclude many undesired hybrids, largely reducing the workload of field cultivation. However, due to the limited accuracy of genomic prediction models, breeders still need to combine their experience with the models to identify regulatory genes that control traits and select hybrids, which remains a time-consuming process. To ease this process, in this paper, we proposed a visual analysis method to facilitate interactive hybrid rice breeding. Regulatory gene identification and hybrid selection naturally ensemble a dual-analysis task. Therefore, we developed a parametric dual projection method with theoretical guarantees to facilitate interactive dual analysis. Based on this dual projection method, we further developed a gene visualization and a hybrid visualization to verify the identified regulatory genes and hybrids. The effectiveness of our method is demonstrated through the quantitative evaluation of the parametric dual projection method, identified regulatory genes and desired hybrids in the case study, and positive feedback from breeders.

Camera-based 3D semantic scene completion (SSC) provides dense geometric and semantic perception for autonomous driving. However, images provide limited information making the model susceptible to geometric ambiguity caused by occlusion and perspective distortion. Existing methods often lack explicit semantic modeling between objects, limiting their perception of 3D semantic context. To address these challenges, we propose a novel method VLScene: Vision-Language Guidance Distillation for Camera-based 3D Semantic Scene Completion. The key insight is to use the vision-language model to introduce high-level semantic priors to provide the object spatial context required for 3D scene understanding. Specifically, we design a vision-language guidance distillation process to enhance image features, which can effectively capture semantic knowledge from the surrounding environment and improve spatial context reasoning. In addition, we introduce a geometric-semantic sparse awareness mechanism to propagate geometric structures in the neighborhood and enhance semantic information through contextual sparse interactions. Experimental results demonstrate that VLScene achieves rank-1st performance on challenging benchmarks—SemanticKITTI and SSCBench-KITTI-360, yielding remarkably mIoU scores of 17.52 and 19.10, respectively.

Sparse general matrix-matrix multiplication (SpGEMM) is a core primitive for numerous scientific applications. Traditional hash-based approaches fail to strike a balance between reducing hash collisions and efficiently utilizing fast shared memory, which significantly undermines the performance of executing SpGEMM on GPUs. To address this issue, this paper introduces a novel accumulator design that achieves high shared memory utilization on modern GPUs. For the proposed high shared memory utilization algorithm, i.e., HSMU-SpGEMM1, we further optimize different symbolic stages. Our evaluations with four state-of-the-art hash-based SpGEMM libraries (Nsparse, spECK, OpSparse, and NVIDIA’s cuSPARSE) on three NVIDIA GPUs (Ampere, Ada Lovelace, Turing) demonstrate significant performance benefits from HSMU-SpGEMM.1HSMU-SpGEMM is available at https://github.com/wuminqaq/HSMUSpGEMM

The advent of deep learning (DL) has transformed signal interpretation, enabling more efficient solutions to complex signal processing problems. DL workloads in signal processing typically share the computational resources of GPU clusters. However, the unpredictable nature of the duration of the DL job introduces a dynamic environment in these clusters. Existing GPU cluster schedulers rely heavily on prior knowledge, overlooking the system’s evolving dynamism. Balancing individual job quality of service (QoS) with overall cluster efficiency is critical from a fairness perspective. Unfortunately, most current schedulers prioritize minimizing job completion time (JCT) at the expense of fairness. This paper introduces Dike, an information-agnostic framework. We propose discrete mixed-priority feedback queues and a communication-aware job placement strategy to balance fairness and efficiency. Large-scale trace-driven experiments demonstrate that Dike improves cluster efficiency by over 1.65 × compared to state-of-the-art resource schedulers.

Creating high-quality 3D object representations from a single-view image is challenging. Existing methods tend to infer the geometry and texture information simultaneously within a shared network. However, decoding geometry and texture from a unified network often leads to their entanglement, causing geometric structure collapse or floating artifacts. After revisiting this task, we propose a single-view reconstruction framework based on 3D Gaussian Splatting. The key idea is to decouple Gaussian position attribute generation from texture feature generation. Technically, our framework combines a Geometry Generator, a Texture Generator, and a Gaussian Attributes Decoder. Two parallel branches, Geometry Generator and Texture Generator, aim for point cloud prediction and texture optimization, respectively. Then the Gaussian Attributes Decoder integrates the generated position and texture attributes into a coherent Gaussian point cloud, facilitating efficient novel view synthesis. Extensive qualitative and quantitative evaluations of public datasets demonstrate that our method consistently outperforms existing methods in terms of reconstruction quality and inferring efficiency.

3D Gaussian Splatting (3D-GS) has exhibited impressive progress in novel view synthesis. When given the sparse views, its performance degrades severely, causing many problems like novel views collapse and excessive floaters. Many recent methods take into account fitting input views, inferring missing scene information and optimizing the final scene representation, all through a single stage. After revisiting the task, we propose a novel framework, Task-Disentangled 3D Gaussian Splatting, abbreviated to TD-GS. It splits the sparse views synthesis task into two subtasks: (i) Dense Generation. (ii) Enhanced Synthesis. In the subtask of Dense Generation, we estimate dense views from sparse input. Then in the subtask of Enhanced Synthesis, both the dense views and the sparse input participate in the training of Gaussians to obtain the final Gaussian representation of the scene. In the process of completing the first subtask, we carefully design Gaussian Cloud Denoising to directly edit 3D Gaussians. Also, we introduce two regularization methods to guide the geometric optimization towards an optimal solution. The purpose is to estimate more reliable outputs. Many experiments have validated that our TD-GS outperforms other state-of-the-art methods.

Entity alignment has emerged as a powerful technique for integrating knowledge graphs, facilitating the fusion of heterogeneous knowledge into a unified graph. The state-of-the-art methods combine both graph structures and side information for effective entity alignment. However, they neglect low-quality issues in data. Specifically, the emerging knowledge graphs in diverse fields amass a wealth of entities that lack not only adequate descriptions but also annotated alignments. These two limitations lead to the overfitting problem and degrade the alignment performance. To tackle these challenges, we propose DEBT, an innovative approach that systematically enhances entity alignment. It first enriches the descriptions of entities by aggregating their neighbors and attributes. Then, a bootstrap strategy is utilized to expand the training set by incorporating entity pairs with similarity scores exceeding a dynamically decreasing threshold. Experimental results demonstrate that our method achieves the state-of-the-art accuracy while reducing the number of annotated entity alignment pairs.

This paper presents a novel pipeline for estimating room layouts and reconstructing the 3D shapes of indoor objects. This task remains challenging due to occlusions of indoor scenes, which lead to incomplete shape and poor geometric quality manifested as non-smooth meshes. Our key insight is that occlusions of indoor objects inherently lead to insufficient information in images. Pixel-level features alone are inadequate to recover the complete structure of objects; Therefore, additional prior information is required to supplement the missing occluded details. To address this, we propose a two-stage training strategy. First, a VQ-VAE encodes 3D shapes into a latent space, with a decoder leveraging priors to predict occluded regions. Second, pixel-aligned features are used for deformation, ensuring consistency between the reconstructed shape and the image. Quantitative and qualitative evaluations on the 3D-FRONT and SUNRGB-D datasets demonstrate that our approach surpasses state-of-the-art methods in reconstructing more complete geometric topologies and smoother meshes.

Conventional deep reinforcement learning approaches frequently struggle with large-scale parameter complexity and insufficient interpretability, particularly in safety-critical control signal processing tasks that demand transparency. While deep neural networks are highly effective, they typically function as "black-box" models, hindering insight into decision-making processes and requiring substantial computational resources, thus restricting their use in signal processing for resource-constrained environments. To address these challenges, we propose the Kolmogorov-Arnold Fuzzy-guided Q-Network, a novel framework that integrates the interpretability of fuzzy logic with the nonlinear approximation capabilities of Kolmogorov-Arnold Networks within a Deep Double Q-Network. Our approach leverages fuzzy rules during the initial training phase to accelerate convergence, improve stability, and reduce model complexity. Experimental results on dynamic control signal tasks, such as CartPole-v1, demonstrate that KAFQN achieves superior performance, a higher proposed Reward-Efficiency Index, and enhanced interpretability, making it highly appropriate for resource-constrained applications.

Multi-task reinforcement learning (MTRL) presents critical challenges, such as the complexities of task-switching and maintaining knowledge retention across various tasks, especially within control tasks. These challenges frequently result in sub-optimal decision-making and catastrophic forgetting during task transitions. To mitigate these challenges, we introduce a framework that leverages large language models for strategic guidance, optimizing decision-making and action selection. The framework integrates a Kolmogorov-Arnold Network to ensure interpretability via symbolic regression, enabling effective strategy transfer across diverse environments. Adaptive Elastic Weight Consolidation is employed to prevent catastrophic forgetting, preserving task-specific knowledge across training. A caching mechanism reduces computational overhead linked to large language model queries, enhancing overall efficiency. Experimental evaluations indicate competitive performance, robust interpretability, and effective strategy transfer.

With the advancement of deep learning, network communication has become the most critical factor in model training. Especially, the all-reduce operation can comprise over 70% of the cumulative training duration as a pivotal component within data parallelism. However, existing all-reduce algorithms often perform poorly in complex network topologies, and there is currently no universal and straightforward all-reduce algorithm that can effectively adapt to diverse topological structures.In this work, we propose a topology decoupled all-reduce algorithm. We decouple the network into multiple tree substructures, select the trees with the smallest heights, and then split the data to perform aggregate communication within these selected structures. This approach significantly reduces the number of communications and enhances efficiency. Experimental results show that our topology decoupled all-reduce algorithm reduces communication time compared to NCCL’s by 42.9% and enhances end-to-end training efficiency by 11.7%.

Point cloud completion aims to reconstruct complete shapes from partial observations. Although current methods have achieved remarkable performance, they still have some limitations: Supervised methods heavily rely on ground truth, which limits their generalization to real-world datasets due to the synthetic-to-real domain gap. Unsupervised methods require complete point clouds to compose unpaired training data, and weakly-supervised methods need multi-view observations of the object. Existing self-supervised methods frequently produce unsatisfactory predictions due to the limited capabilities of their self-supervised signals. To overcome these challenges, we propose a novel self-supervised point cloud completion method. We design a set of novel self-supervised signals based on multi-view augmentations of the single partial point cloud. Additionally, to enhance the model’s learning ability, we first incorporate Mamba into self-supervised point cloud completion task, encouraging the model to generate point clouds with better quality. Experiments on synthetic and real-world datasets demonstrate that our method achieves state-of-the-art results.

Secure aggregation has recently gained popularity in federated learning to defend against inference attacks by malicious aggregators or eavesdroppers. However, existing methods usually bring additional communication overhead and possibly impede the convergence rate of the global model. The challenge becomes acute in wireless network environments where bandwidth is severely constrained. Hence, the attainment of effective communication compression while ensuring secure aggregation has been a profoundly demanding and valuable quandary.In this work, we propose a novel uplink communication compression method for federated learning, named FedMPQ. It guarantees both high security and efficiency without compromising accuracy. Specifically, we introduce multiple optional codebooks for clients to ensure almost lossless information under a high compression rate. In addition, FedMPQ achieves high security with a combination of secure aggregation paradigm and differential privacy, preventing data leakage both on server and during communication. The experiments conducted on the LEAF dataset demonstrate that FedMPQ reduces the uplink communications by 90-95% with no sacrifice on accuracy.

The performance of computer vision models in certain real-world applications, such as medical diagnosis, is often limited by the scarcity of available images. Expanding datasets using pre-trained generative models is an effective solution. However, due to the uncontrollable generation process and the ambiguity of natural language, noisy images may be generated. Re-weighting is an effective way to address this issue by assigning low weights to such noisy images. We first theoretically analyze three types of supervision for the generated images. Based on the theoretical analysis, we develop TriReWeight, a triplet-connection-based sample re-weighting method to enhance generative data augmentation. Theoretically, TriReWeight can be integrated with any generative data augmentation methods and never downgrade their performance. Moreover, its generalization approaches the optimal in the order O(√d ln (n)/n). Our experiments validate the correctness of the theoretical analysis and demonstrate that our method outperforms the existing SOTA methods by 7.9% on average over six natural image datasets and by 3.4% on average over three medical datasets. We also experimentally validate that our method can enhance the performance of different generative data augmentation methods.

3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception, which is crucial for enabling accurate and reliable decision-making. However, existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features, thereby failing to acquire effective scene context. These approaches ignore critical motion dynamics and struggle to achieve temporal consistency. To address the above challenges, we propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance. By leveraging optical flow, FlowScene can integrate motion, different viewpoints, occlusions, and other contextual cues, thereby significantly improving the accuracy of 3D scene completion. Specifically, our framework introduces two key components: (1) a Flow-Guided Temporal Aggregation module that aligns and aggregates temporal features using optical flow, capturing motion-aware context and deformable structures; and (2) an Occlusion-Guided Voxel Refinement module that injects occlusion masks and temporally aggregated features into 3D voxel space, adaptively refining voxel representations for explicit geometric modeling. Experimental results demonstrate that FlowScene achieves state-of-the-art performance, with mIoU of 17.70 and 20.81 on the SemanticKITTI and SSCBench-KITTI-360 benchmarks.

Abstract Distributed computing frameworks play a crucial role in supporting compute-intensive applications in the era of big data. The growing demand for computing resources has spurred the interconnection of data centers, leading to the formation of supercomputing Internet. MapReduce is a popular distributed computing framework designed for large independent clusters. The original MapReduce framework deployed on supercomputing Internet performs inefficiently due to redundant geo-distributed reduce operations. Nonetheless, its abstraction remains significant potential. This paper proposes an enhanced MapReduce framework for geo-distributed supercomputing Internet to minimize the necessity for data transmission across data centers. Leveraging hierarchical scheduling techniques, the framework optimizes data locality to mitigate network latency and bandwidth consumption during reduce operations, thereby reducing overall job execution times. The paper introduces a mathematical model for task scheduling within supercomputing Internet and formally describes the data transmission process among data centers. In the job scheduling phase, our framework facilitates efficient overlap of transferring and computing through pre-selected data centers. Meanwhile, in the data transmission phase, the framework aggregate data to reduce the frequency of transmission, thus alleviating the adverse effects on transmission of hierarchical network architecture. Comparative analysis with existing methods demonstrates the efficacy of the proposed framework in addressing similar computational challenges. Empirical evaluations underscore the effectiveness of our method in practice.

Abstract Novel view synthesis for talking heads presents significant challenges due to the complex and diverse motion transformations involved. Conventional methods often resort to reliance on structure priors, like facial templates, to warp observed images into a canonical space conducive to rendering. However, the incorporation of such priors introduces a trade‐off‐while aiding in synthesis, they concurrently amplify model complexity, limiting generalizability to other deformable scenes. Departing from this paradigm, we introduce a pioneering solution: the motion‐conditioned neural radiance field, MoNeRF, designed to model talking heads through latent motion navigation. At the core of MoNeRF lies a novel approach utilizing a compact set of latent codes to represent orthogonal motion directions. This innovative strategy empowers MoNeRF to efficiently capture and depict intricate scene motion by linearly combining these latent codes. In an extended capability, MoNeRF facilitates motion control through latent code adjustments, supports view transfer based on reference videos, and seamlessly extends its applicability to model human bodies without necessitating structural modifications. Rigorous quantitative and qualitative experiments unequivocally demonstrate MoNeRF's superior performance compared to state‐of‐the‐art methods in talking head synthesis. We will release the source code upon publication.

In IoT-driven intelligent perception systems, multi-modal crowd counting using visible-thermal (RGB-T) sensor arrays deployed on edge nodes plays a vital role in urban management. Existing methods primarily focus on inter-modal feature fusion but suffer from noise amplification when directly integrating thermal features with degraded visible images under low-light conditions, leading to inaccurate density estimation and poor nighttime performance. To address this, we propose the first domain-adaptive network for RGB-T crowd counting. First, we introduce a dark-light domain adaptation module based on Retinex decomposition. Unlike traditional approaches that extract only bright and thermal domain features, our module employs a dual decomposition-recomposition process to learn illumination-invariant features from RGB images, enhancing discriminability in dark regions while preserving bright-domain semantics. Second, the module is supervised by feature alignment, reflectance consistency, and decomposition invariance losses, strengthening its robustness under extreme illumination degradation. Finally, a multi-domain progressive decision fusion module is designed to leverage shared and domain-specific features through public feature extraction, dual-domain fusion, and weighted decision processes, improving generalization across varying lighting conditions. Trained solely on well-lit data, our method generalizes effectively to dark scenarios. Experiments on DroneRGBT and RGBTCC datasets demonstrate superior performance over state-of-the-art fusion methods, particularly excelling in low-light crowd counting.

The vision-based semantic scene completion task aims to predict dense geometric and semantic 3D scene representations from 2D images. However, the presence of dynamic objects in the scene seriously affects the accuracy of the model inferring 3D structures from 2D images. Existing methods simply stack multiple frames of image input to increase dense scene semantic information, but ignore the fact that dynamic objects and non-texture areas violate multi-view consistency and matching reliability. To address these issues, we propose a novel method, CDScene: Vision-based 3D Semantic Scene Completion via Capturing Dynamic Representations. First, we leverage a large multi-modal model to extract 2D explicit semantics and align them into 3D space. Second, we exploit the characteristics of monocular and stereo depth to decouple scene information into dynamic and static features. The dynamic features contain structural relationships around dynamic objects, and the static features contain dense contextual spatial information. Finally, we design a dynamic-static adaptive fusion module to effectively extract and aggregate complementary features, achieving robust and accurate semantic scene completion in autonomous driving scenarios. Extensive experimental results on the SemanticKITTI, SSCBench-KITTI360, and SemanticKITTI-C datasets demonstrate the superiority and robustness of CDScene over existing state-of-the-art methods.

Ring all-reduce is currently the most commonly used collective communication technique in the fields of data parallel and distributed computing. It consists of three phases: communication establishment, data transmission, and data processing at each step. However, this method may suffer from increased communication latency as the number of computation nodes increases, excessive communication steps and data processing procedures can lead to insufficient bandwidth utilization. To address this issue, this article proposes an Interleaved Bidirectional Ring (IBing) all-reduce method, which uses specially crafted communication operations to improve communication efficiency by reducing the effects of both communication establishment and data processing time. IBing reduces the number of communication steps by half compared to the Ring all-reduce. The results of extensive experiments indicate that the proposed IBing design can reduce total communication consumption by an average of 8.49% and up to 49.73%.

Graph Neural Networks (GNNs) excel in processing graph-structured data, making them attractive and promising for tasks such as recommender systems and traffic forecasting. However, GNNs’ irregular computational patterns limit their ability to achieve low latency and high energy efficiency, particularly in edge computing environments. Current GNN accelerators predominantly focus on value sparsity, underutilizing the potential performance gains from bit-level sparsity. However, applying existing bit-serial accelerators to GNNs presents several challenges. These challenges arise from GNNs’ more complex data flow compared to conventional neural networks, as well as difficulties in data localization and load balancing with irregular graph data.To address these challenges, we propose BEAST-GNN, a bit-serial GNN accelerator that fully exploits bit-level sparsity. BEAST-GNN introduces streamlined sparse-dense bit matrix multiplication for optimized data flow, a column-overlapped graph partitioning method to enhance data locality by reducing memory access inefficiencies, and a sparse bit-counting strategy to ensure balanced workload distribution across processing elements (PEs). Compared to state-of-the-art accelerators, including HyGCN, GCNAX, Laconic, GROW, I-GCN, SGCN, and MEGA, BEAST-GNN achieves speedups of 21.7 ×, 6.4×, 10.5×, 3.7×, 4.0×, 3.3×, and 1.4× respectively, while also reducing DRAM access by 36.3×, 7.9×, 6.6×, 3.9×, 5.38×, 3.37×, and 1.44×. Additionally, BEAST-GNN consumes only 4.8%, 12.4%, 19.6%, 27.7%, 17.0%, 26.5%, and 82.8% of the energy required by these architectures.

Stream processing is integral to edge computing due to its low-latency attributes. Nevertheless, variability in user group sizes and disparate computing capabilities of edge devices necessitate frequent operator migrations within the stream. Moreover, intricate dependencies among stream operators often obscure the detection of potential bottleneck operators until an identified bottleneck is migrated in the stream. To address this, we propose a Cost-Aware Operator Migration (CAOM) scheme. The CAOM scheme incorporates a bottleneck operator detection mechanism that directly identifies all bottleneck operators based on task running metrics. This approach avoids multiple consecutive operator migrations in complex tasks, reducing the number of task interruptions caused by operator migration. Moreover, CAOM takes into account the temporal variance in operator migration costs. By factoring in the fluctuating data generation rate from data sources at different time intervals, CAOM selects the optimal start time for operator migration to minimize the amount of accumulated data during task interruptions. Finally, we implemented CAOM on Apache Flink and evaluated its performance using the WordCount and Nexmark applications. Our experiments show that CAOM effectively reduces the number of necessary operator migrations in tasks with complex topologies and decreases the latency overhead associated with operator migration compared to state-of-the-art schemes.

Vision-based semantic scene completion task aims to predict dense geometric and semantic 3D scene representations from 2D images. However, 3D modeling from a single view is an ill-posed problem, limited by the field of view and occlusion problems caused by image input. Moreover, existing methods tend to produce erroneous scene hallucinations and overly smooth boundary segmentation due to a lack of information. To address this problem, we propose MixSSC, which mixes the sparsity of forward projection with the denseness of depth-prior backward projection. The aim is to use sparse features to fill information-poor regions and dense features to enhance visible regions. Specifically, we develop the forward-backward mixture module, which enables the generation of scene mixture voxel representation by leveraging the benefits of both forward and backward projection. Subsequently, we design the semantic-spatial fusion module, which utilizes a coarse-to-fine approach to process mixture voxel features at the semantic-spatial level. Extensive experimental results on the SemanticKITTI, SSCBench-KITTI-360 and nuScenes datasets demonstrate the superiority of MixSSC. Our code is available on https://github.com/willemeng/MixSSC.

Pedestrian historical trajectory completion significantly bolsters the predictive accuracy of models. However, traditional statistical models such as Hidden Markov Models (HMM), which focus solely on individual pedestrian trajectories, often fall short in terms of generalization. Conversely, data-driven deep learning approaches demand extensive and meticulous data annotation as well as large datasets. Additionally, leveraging sequential historical data and uncovering the correlation between neighboring pedestrians during absences presents a significant challenge. To address these issues, we introduce a novel trajectory completion method that harnesses prior-posterior knowledge and interactive information, termed CMPT. Our approach commences with the design of a Neighbor Pedestrian Selection module (NPS), adept at identifying neighboring pedestrians through a composite scoring system that evaluates feature similarity and proximity. Subsequently, we employ a Top-Graph Attention Network (T-GAT) to extract multiple correlation sets between preceding and succeeding moments within the scenario. These correlations are then fed into the Markov-Inverse Recovery module (MR), which utilizes prior and posterior insights to flesh out the neighbor influence at the unobserved intervals. Culminating in the Trajectory Reconstruction module (TR), we integrate the completed neighbor influence data with the historical trajectory of the missing pedestrian to finalize the missing trajectory reconstruction. Empirical evidence from our experiments indicates that the Final Distance Error (FDE) of the trajectories completed by CMPT is a commendable 0.30. The source code for CMPT is available from https://github.com/ZYueliang/CMPT-Net.

Machine unlearning involves retracting data records and reducing their influence on trained models, aiding user privacy protection, at a significant computational cost potentially. Weight perturbation-based unlearning is common but typically modifies parameters globally. We propose fine-grained Top-K and Random-k parameters perturbed inexact machine unlearning that address the privacy needs while keeping the computational costs tractable. However, commonly used training data are independent and identically distributed, for inexact machine unlearning, current metrics are inadequate in quantifying unlearning degree that occurs after unlearning. To address this quantification issue, we introduce SPD-GAN, which subtly perturbs data distribution targeted for unlearning. Then, we evaluate unlearning degree by measuring the performance difference of the models on the perturbed unlearning data before and after unlearning. Furthermore, to demonstrate efficacy, we tackle the challenge of evaluating machine unlearning by assessing model generalization across unlearning and remaining data. To better assess the unlearning effect and model generalization, we propose novel metrics, namely, the forgetting rate and memory retention rate. By implementing these innovative techniques and metrics, we achieve computationally efficacious privacy protection in machine learning applications without significant sacrifice of model performance. A by-product of our work is a novel method for evaluating and quantifying unlearning degree.

In the context of distributed deep learning, computational clusters place greater emphasis on static all-reduce communication latency while also needing to support large-scale networking. However, the communication efficacy of current all-reduce algorithms within specialized network topologies requires enhancement. Existing all-reduce communication algorithms inadequately exploit cluster bandwidth, leading to considerable bandwidth idleness. The optimization of communication algorithms becomes imperative to fully utilize the available bandwidth. Addressing this concern, we propose an innovative approach: a topology-aware interleaved all-reduce algorithm for Dragonfly networks (TIAD). Leveraging the inherent characteristics of the Dragonfly network, TIAD employs an interleaved communication mechanism for both intra- and inter-group data collection, significantly augmenting communication efficiency. Moreover, we refine the Dragonfly network with minimal adjustments, aligning it with the theoretical structure of interleaved communication. We also proposed an all-reduce communication method to complement the TIAD algorithm, specifically for scenarios where only a subset of nodes in the Dragonfly network participate in the communication task. Our experiments demonstrate that TIAD exhibits the shortest communication time across diverse node sizes and bandwidth conditions. Notably, our algorithm reduces communication time by up to 23.4% during the collection communication phase in comparison to the PAARD algorithm.

Data locality is crucial for distributed computing systems (e.g., Spark and Hadoop), which is the main factor considered in the task scheduling. Simultaneously, the effects of data locality on reduce tasks are determined by the intermediate data partitioning. While suffering from the problem of data skew, the existing intermediate data partitioning methods only achieves load balancing for reduce tasks. To address the problem, this paper optimizes the data locality for reduce tasks by integrating intermediate data partitioning and task scheduling in Spark framework. First, it presents a distribution skew model to divide the key clusters into skewed and non-skewed distribution. Then, a data locality and load balancing-aware intermediate data partitioning method is proposed, where a priority allocation strategy for the key clusters with skewed distribution is presented, and a balanced allocation strategy for the key clusters with non-skewed distribution is presented. Finally, it proposes a data locality-aware reduce task scheduling algorithm, where an online self-adaptive NARX (nonlinear autoregressive with external input) model is developed to predict the idle time of node. It can ensure that the delayed scheduling decision made can complete the data transmission of reduce tasks earlier. We implement our proposals in Spark-3.5.1 and evaluate the performance using several representative benchmarks. Experimental results indicate that the proposed method and algorithm can reduce the job/application running time by approximately 4% to 46% and decrease the total volume of data transmission by approximately 8% to 54%.

In this article, a novel model-free dynamic event-triggered adaptive learning control scheme is developed for continuous-time linear multiagent systems. This control scheme is different from model-based control scheme in the sense that prior knowledge of the system's model is not required. To further reduce transmission data, an event-triggered control policy based on static event-triggered mechanism (SETM) and dynamic event-triggered mechanism (DETM) is proposed. Compared to SETM, DETM may significantly produce larger average event intervals and maintain control performance. In addition, based on off-policy integral reinforcement learning, an adaptive iteration method is proposed with convergence proof. Numerical tests on both linear and nonlinear multiagent systems are conducted to demonstrate that the proposed scheme can guarantee learning performance and larger triggering intervals. Finally, the learning control scheme is tested on the multiarea power system, which can illustrate the reliability and practicality of this method. Specifically, the load frequency control problem of the multiarea power system is studied using three control schemes, revealing that DETM can achieve a better frequency response at the lowest information transmission rate and ensure the overall quality and reliability of the power system.

The performance of computer vision models in certain real-world applications (e.g., rare wildlife observation) is limited by the small number of available images. Expanding datasets using pre-trained generative models is an effective way to address this limitation. However, since the automatic generation process is uncontrollable, the generated images are usually limited in diversity, and some of them are undesired. In this paper, we propose a human-guided image generation method for more controllable dataset expansion. We develop a multi-modal projection method with theoretical guarantees to facilitate the exploration of both the original and generated images. Based on the exploration, users refine the prompts and re-generate images for better performance. Since directly refining the prompts is challenging for novice users, we develop a sample-level prompt refinement method to make it easier. With this method, users only need to provide sample-level feedback (e.g., which samples are undesired) to obtain better prompts. The effectiveness of our method is demonstrated through the quantitative evaluation of the multi-modal projection method, improved model performance in the case study for both classification and object detection tasks, and positive feedback from the experts.

The integration of blockchain technology into the federated learning (FL) process offers effective measures for safeguarding the security and privacy of model data. However, the inherent consensus mechanism of blockchain technology can introduce long latency, which may hinder the overall efficiency of FL. To address this challenge, we propose a blockchain-based FL training strategy that tackles the following issues: (1) Reducing the number of parameter aggregations in the blockchain network by increasing the number of local training epochs, which effectively minimize the frequency of blockchain authentication and packing operations. (2) Mitigating communication overhead between terminals and edge nodes in the blockchain network by leveraging the wait-free backpropagation technique, which reduces the communication overhead. Experimental results demonstrate that our proposed strategy yields improvements in both convergence efficiency and system scalability.

With the rapid development of blockchain technology, its network size has increased by hundreds or thousands of times. For large-scale blockchain networks, consensus algorithms need to be used to achieve consensus among dispersed nodes. Only blocks approved by more than half of the nodes can truly achieve blockchain. Therefore, communication efficiency in blockchain networks is crucial, and the excessive communication delay generated as the number of nodes increases can seriously affect the processing efficiency of blockchain systems. Therefore, we have designed a consensus mechanism based on grouping for the structural characteristics of P2P communication networks in blockchain systems, and a deep learning based grouping algorithm for the decentralized P2P networks. Through experiments, it can be seen that our algorithm can improve the communication efficiency of 74.3%-76.1% compared to the unoptimized decentralized P2P network nodes.

Camera-based Semantic Scene Completion (SSC) is to infer the full geometry of objects and scenes from only 2D images. The task is particularly challenging for those in-visible areas, due to the inherent occlusions and lighting ambiguity. Existing works ignore the information missing or ambiguous in those shaded and occluded areas, resulting in distorted geometric prediction. To address this issue, we propose a novel method, Bi-SSC, bidirectional geomet-ric semantic fusion for camera-based 3D semantic scene completion. The key insight is to use the neighboring structure of objects in the image and the spatial differences from different perspectives to compensate for the lack of information in occluded areas. Specifically, we introduce a spatial sensory fusion module with multiple association attention to improve semantic correlation in geometric distributions. This module works within single view and across stereo views to achieve global spatial consistency. Experimental results demonstrate that Bi-SSC outperforms state-of-the-art camera-based methods on SemanticKITTI, particularly excelling in those invisible and shaded areas.

Multimodal transformer excels in various applications, but faces great challenges such as high memory consumption and limited data reuse that hinder real-time performance. To address these issues, we propose a processing-in-memory (PIM)-GPU collaboration oriented compiler to accelerate the multimodal transformers. The PIM-GPU collaboration adapts well to multimodal transformers and significantly accelerates model inference. In addition, we introduce a tailored PIM allocation algorithm for variable-length inputs to further improve computation efficiency. Experimental results show that our scheme can achieve an average 15x end-to-end speedup.

Federated learning (FL) supports massive edge devices to collaboratively train object detection models in mobile computing scenarios. However, the distributed nature of FL exposes significant security vulnerabilities. Existing attack methods either require considerable costs to compromise the majority of participants, or suffer from poor attack success rates. Inspired by this, we devise an efficient fake node-based perception poisoning attacks strategy (FNPPA) to target such weaknesses. In particular, FNPPA poisons local data and injects multiple fake nodes to participate in aggregation, aiming to make the local poisoning model more likely to overwrite clean updates. Moreover, it can achieve greater malicious influence on target objects at a lower cost without affecting the normal detection of other objects. We demonstrate through exhaustive experiments that FNPPA exhibits superior attack impact than the state-of-the-art in terms of average precision and aggregation effect.

Compared with conventional SRAM, Spin-Transfer Torque Random Access Memory(STT-RAM) is expected to play a crucial role in future memory technologies with the increasing demands for higher storage density and lower power consumption for modern embedded systems. Moreover, Multi-Level Cell (MLC) STT-RAM outperforms Single-Level Cell (SLC) STT-RAM since it has higher bit density. However, MLC STT-RAM suffers from write performance due to the two-step state transitions (TTs) in memory cells' soft domain. State-of-the-art approaches mitigate this issue by reducing TTs with efficient data coding. Unfortunately, none of the existing works can fully eliminate the TTs. In this work, zeroTT, an optimal (3, 4)-based expansion coding method that eliminates TTs for MLC STT-RAM. The design of ZeroTT considers space overhead and coding complexity, and our experimental results demonstrate that zeroTT can completely avoid TTs, leading to a more efficient MLC STT-RAM memory in terms of access latency, energy consumption, and device lifetime.

Long-term occlusion is one of the most formidable challenges in Multi-Object Tracking (MOT). The motion models of existing SORT-based trackers are unreliable in estimating the motion states of long-term occluded targets. This is mainly because as the occlusion period increases, the increases speed of estimation errors in the motion model increases faster. In practical applications, we believe that the estimation error of the tracker during long-term occlusion is mainly concentrated in the estimation error of the motion model on the velocity of the occluded target. In this work, we have demonstrated that in the long-term occlusion period, appropriately correcting the estimated values of the motion model on the target motion velocity and fully utilizing the temporal and attribute information of the target’s historical trajectory as calculation indicators of correlation are beneficial for improving the robustness of the tracker in long-term occlusion. We refer to our proposed motion correction-based framework as MC-SORT, which mainly consists of a Momentum Compensation Module (MCM) and a Backtracking Re-association (BRA) module. The former can correct the estimated value of the target’s motion state during long-term occlusion, the latter uses the temporal and attribute information of the target’s historical trajectory during long-term occlusion as correlation indicators to measure the degree of correlation between the target and trajectory. Our proposed MC-SORT has the characteristics of simplicity, online, real-time, and plug-and-play, particularly improving the robustness of the tracker in long-term occlusion. The extensive experimental results on the MOT17 and MOT20 datasets demonstrate the robustness and superiority of our framework.

Domain adaptation has been extensively explored in object detection. Through the utilization of self-training and the decoupling of adversarial feature learning from the training of the detector, current methods make detectors more transferable and ensure their discriminability. However, the presence of low-quality pseudo labels during self-training introduces noises to the training phase and thus degrades the model performance. To tackle this challenge, we introduce an I-adapt framework, whose IoU Adapter accurately predicts the Intersection over Union (IoU) between predicted boxes and their corresponding ground-truth boxes in both source and target domains. This enables an effective measure for the pseudo-label quality. Based on this measure, we propose a re-weighting strategy, which enforces the detector to focus on learning from high-quality pseudo labels. We achieve state-of-the-art (SOTA) performance in several cross-domain object detection tasks, proving the effectiveness of I-adapt.

New categories may be introduced over time, or existing categories may need to be reclassified. Class incremental learning (CIL) is employed for the gradual acquisition of knowledge about new categories while preserving information about previously learned ones in such dynamic environments. It might also be necessary to also eliminate the influence of related categories on the model to adapt to reclassification. We thus introduce class-level machine unlearning (MU) within CIL. Typically, MU methods tend to be time-consuming and can potentially harm the model’s performance. A continuous stream of unlearning requests could lead to catastrophic forgetting. To address these issues, we propose a non-destructive eCIL-MU framework based on embedding techniques to map data into vectors and then be stored in vector databases. Our approach exploits the overlap between CIL and MU tasks for acceleration. Experiments demonstrate the capability of achieving unlearning effectiveness and orders of magnitude (upto ~ 278×) of acceleration.

Federated learning (FL) enables a great deal of distributed independent participants to collaborate in training without sharing data. Malicious adversary can poison the local model by backdoor poisoning attacks and utilize the characteristic that server cannot trace original data to make the poisoned model directly aggregated. Especially in the AIoT-FL network that generates large amounts of data in real-time, such an attack is more powerful. In this paper, we design a sybil-based backdoor poisoning attacks (SBPA) against the above vulnerability. Malicious participants inject backdoor triggers into distributed Big Data to covertly complete data poisoning. During subsequent iterative aggregation, the joint model activates the backdoor during testing to achieve misclassification for the backdoor images. Besides, malicious participants create some sybil nodes to join the aggregation by taking advantage of the vulnerability of system devices to be easily disconnected. They are committed to making the poisoned local models aggregated with higher probability. Their goal works on making the final global model misclassify backdoor images while keeping high classification accuracy on the other non-backdoor samples. We conduct extensive experiments on multiple datasets and exhibit more robust performance than the state-of-the-art in various metrics under both data distribution including i.i.d. and non-i.i.d. scenarios.

Remarkable breakthroughs but daunting complexities of deep learning have aroused widespread interest in dedicated deep neural network (DNN) acceleration hardware, among which optical accelerators (OAs) are particularly promising thanks to their unprecedentedly high-performance-per-watt. However, the development of OAs is much slower than that of electrical accelerators due to threefold challenges. First, the OA design space is ample and discrete, making it tough for OA optimization; Second, the ecosystem that facilitates OA development is still in its infancy. Techniques to support OA design remain less explored, limiting both the achievable performance and the innovative development of OAs; and Third, OAs are highly sensitive to fabrication-induced process variations and thermal fluctuations (i.e., PTVs), which degrades OAs’ inference robustness and even renders them unusable in practice. In this article, we develop AutOAS, the first-of-its-kind framework for Automated Optical Accelerator Search, in order to jointly boost acceleration efficiency, inference robustness, and development speed. Our AutOAS comprises four enabling components: 1) a holistic OA search space, which takes full consideration of OAs’ micro-architectures (e.g., the type, shape and size of core functional units for data computation and data access), dataflow choices, DNN-to-accelerator mapping methods, memory hierarchy and PTV mitigation techniques; 2) a PTV Regulator, which can emulate the impact of PTVs on OAs’ inference accuracy based on given PTV profiles, and enables energy-efficient PTV mitigation on OAs; 3) an O-Performance Predictor, which enables accurate yet efficient predictions of an OA’s energy, throughput (latency) and chip area according to the DNN model and OA architecture parameters; and 4) two O-Search Engines (i.e., a differentiable search engine and an evolutionary search engine), which can automatically explore the large design space of OAs and identify the optimal accelerators to maximize the acceleration targets. Based on 10 DNN models widely applied in both computer vision and sequence modeling tasks, extensive experiments and ablation studies validate the effectiveness of our PTV Regulator, O-Performance Predictor, and O-Search Engines, as well as the superior performance of AutOAS-generated OAs.

The concept of data locality is crucial for distributed systems (e.g., Spark and Hadoop) to process Big Data. Most of the existing research optimized the data locality from the aspect of task scheduling. However, as the execution container of Spark's tasks, the executor launched on different nodes can directly affect the data locality achieved by the tasks. This paper tries to improve the data locality of tasks by executor allocation in Spark framework. First, because of different communication modes at stages, we separately model the communication cost of tasks for transferring input data to the executors. Then formalize an optimal executor allocation problem to minimize the total communication cost of transferring all input data. This problem is proven to be NP-hard. Finally, we present a greed dropping heuristic algorithm to provide solution to the executor allocation problem. Our proposals are implemented in Spark-3.4.0 and its performance is evaluated through representative micro-benchmarks (i.e.,WordCount,Join,Sort) and macro-benchmarks (i.e.,PageRankandLDA). Extensive experiments show that the proposed executor allocation strategy can decrease the network traffic and data access time by improving the data locality during the task scheduling. Its performance benefits are particularly significant for iterative applications.

With the increasing scale of cloud computing applications of next-generation embedded systems, a major challenge that domain scientists are facing is how to efficiently store and analyze the vast volume of output data. Compression can reduce the amount of data that needs to be transferred and stored. However, most of the large datasets are in floating-point format, which exhibits high entropy. As a result, existing lossless compressors cannot provide enough performance for such applications. To address this problem, we propose a total variation reduction method for improving the compression ratio of lossless compressors (namely, FPC + and FPZIP + ), which employs a median-based hyperplane to precondition the data. In particular, we first try to exploit the space-filling curve (SFC), a well-known technique to preserve data locality for a multi-dimensional dataset. We show and explain why a raw SFC, such as Hilbert and Z-order curves, cannot improve the compression ratio. Then, we explore the opportunity and theoretical feasibility of the proposed total variation reduction-based algorithm. The experiment results show the effectiveness of the proposed method. The compression ratios are improved up to 48.2% (20.6% on average) for FPZIP and 42.4% (18.4% on average) for FPC. Moreover, through observing the time composition of the proposed method, it is found that the median finding holds a high percentage of the execution time. Hence, we further introduce an approximate median finding algorithm, providing a linear-time overhead reduction scheme. The experiment results clearly demonstrate that this algorithm reduces execution time by an average of 56.7% and 40.7% compared to FPC + and FPZIP + , respectively.

Monocular depth estimation has made considerable advances under clear weather conditions. However, how to learn accurate scene depth under rain and fog conditions and alleviate the negative influence of occlusion, light, visibility, etc., is an open problem. To address this problem, in this article, we split the adverse weather depth estimation network into two subbranches: the depth prediction branch and the masked encoding branch. The depth prediction branch is used for depth estimation. The masked encoding branch, inspired by masked image modeling, uses random masks to simulate occlusion or low visibility often seen in rain and fog, forcing this branch to learn to infer the prediction of masked regions from the context. In order to make the masked encoding better enhance the depth prediction, we designed the mask feature fusion module, which can fuse the depth and spatial context features of the two branches to produce a fine-level depth map. The experimental results on the Foggy Cityscapes and RainCityscapes datasets demonstrate that our method achieves state-of-the-art performance, significantly outperforming previous methods across all evaluation metrics.

Federated Learning (FL) combined with Differential Privacy (DP) is widespread in healthcare, finance, and IoT due to its advantages in multi-client data distribution. However, existing FL approaches overlook the differential impact levels among clients and data redundancy issues, resulting in high computational overhead and limited real-time applicability. Additionally, non-independent identical distribution (Non-IID) and imbalanced datasets in multi-clients pose challenges in privacy preservation and model overfitting. Therefore, we propose a balanced weight strategy for multi-stage federated learning against multi-client data skewing, called BM-FL, which involves clients, intermediate trust servers (ITSs), and the central server (CS). Firstly, to protect data privacy, an improved Laplace$\epsilon$-differential privacy method is employed. Secondly, a novel generative adversarial network (GAN) called BC-GAN is introduced. It is used to generate realistic fake samples and maintain a balanced proportion of samples across different categories. Then, to make full use of each client's valuable data, we designe a balanced weight strategy. Moreover, extensive experimental results clearly demonstrate the effectiveness of BM-FL in efficiently handling classification tasks involving Non-IID and imbalanced datasets while maintaining privacy and security. Furthermore, our method attains superior classification accuracy with fewer training epochs compared to relevant classical algorithms. The code is available athttps://github.com/ylxzjy/BMFL.git.

Incoherent optical DNN accelerators (OAs) are booming thanks to unparalleled performance-per-watt and excellent scalability. To boost their innovative development, a recent work revolutionarily proposed automatic OA search. However, the robustness and the acceleration performance of the generated OAs are below expectation, because the impacts of inter-tile data transfer and fabrication process & thermal variations (i.e., PTVs) on OAs were ignored. Both hinder OA design automation from being a reality. To resolve theses challenges, we develop FIONA, a novel framework for Fine-grained Incoherent Optical DNN Accelerator search towards both superior acceleration efficiency and inference robustness. Compared against 5 state-of-the-art incoherent OAs on 9 DNN benchmarks, extensive experiments and ablation studies validate the effectiveness of FIONA, achieving up to 198.01× acceleration efficiency improvement based on guaranteed robustness.

As a fundamental problem in graph data mining, Densest Subgraph Discovery (DSD) aims to find the subgraph with the highest density from a graph. It has been studied for several decades and found a large number of real-world applications, such as network community detection, regulatory motif discovery in DNA, graph index construction, and fake follower detection. Although there are many existing DSD algorithms, they are often not scalable or efficient to process large-scale graphs, since most of them are serial algorithms and can only leverage the computing resource of a single CPU core. To tackle these issues, in this paper we propose efficient parallel algorithms for solving the DSD problems on both undirected and directed graphs at scale. Our main idea is to use the k-cores (a kind of dense subgraph) to approximate the densest subgraph in the undirected graphs, and then propose efficient parallel algorithms for computing the cores by optimizing the iterative process and also reducing the number of iterations. We further extend this idea for directed graphs by introducing a novel concept, named w-induced subgraph, to avoid unnecessary enumerations of x or y when searching [x,y]-cores (a kind of directed dense subgraph to approximate the densest). To verify the scalability and efficiency of the proposed algorithms, we have conducted extensive experiments on 12 large real-world graphs, and four of them are billion-scale. The experimental results show that our proposed algorithms outperform the state-of-the-art algorithms on both undirected and directed graphs, in terms of scalability and efficiency.

U-network is a kind of full convolutional neural network, which is widely used in the field of medical image segmentation. However, it still has the problem of small targets being unable to be segmented and resulting in unsatisfactory segmentation effect. In order to solve this problem, this paper proposes an enhanced U-network by combining pyramid pooling module (PPM) and convolutional block attention module (CBAM). It’s whole network is U-Net architecture, where PPM with bin sizes of 1×1, 2×2 and 3×3 are used in the downsampling part of the network, which can extract input image features of various dimensions. And CBAM is used in the downsampling part of the network, which combines convolution and attention mechanism, and can pay attention to the image from two aspects of space and channel to improve the segmentation ability of the network. Experimental results show that our network outperforms traditional Ushaped segmentation networks by 30% to 40% in metrics IoU, MAE, and Dice, respectively.

In parallel distributed data processing frameworks like Spark and Flink, task scheduling has a great impact on cluster performance. Though task Scheduling has proven to be an NP-complete problem, a large number of researchers have proposed many heuristic rules to obtain approximate optimal solutions. But most of them ignore the fact that the resource requirements of tasks are dynamically changing during its runtime. Considering the overall task entire lives, the CPU utilization is often lower during the data transfer. Especially for most distributed data processing platforms, data transmission is time-consuming, which usually resulting in low overall CPU utilization. Similarly, network throughput during task calculations is also low in some cases. In this article, we propose a network load variation perception based heuristic task scheduling algorithm, and based on this implement a dual-phase pipeline task scheduler (D2PTS) from the perspective of dynamic resource requirements that aims at maximizing cluster resource utilization, as a supplement to existing data-parallel frameworks. D2PTS divides the states of task into two phases: network-intensive and network-free. To improve the overall resource utilities, this article proposes different algorithms to evaluate the execution time of network sensitive and network free phases respectively. When an executing task is in the network-free phase, D2PTS can additionally schedule a new network-intensive task at the right time. Under this scheduling policy, the two tasks sharing the same CPU core can be executed as a coarse-grained pipeline. This execution method can start tasks earlier and improve resource utilization. Finally, we have implemented our model prototype on Spark 2.4.3 and conducted a number of experiments to evaluate the performance of our model. Experimental results show that D2PTS can not only minimize application makespan, but also improve resource utilization.

With the massive amounts of data generated by industrial Internet of Things (IIoT) devices at all moments, federated learning (FL) enables these distributed distrusted devices to collaborate to build machine learning model while maintaining data privacy. However, malicious participants still launch malicious attacks against the security vulnerabilities during model aggregation. This article is the first to propose Sybil-based collusion attacks (SCA) in the IIoT-FL system for the vulnerabilities mentioned above. The malicious participants use label flipping attacks to complete local poisoning training. Meanwhile, they can virtualize multiple Sybil nodes to make the local poisoning models aggregated with the greatest possibility during aggregation. They focus on making the joint model misclassify the selected attack class samples during the testing phase, while other nonattack classes kept the main task accuracy similar to the nonpoisoned state. Exhaustive experimental analysis demonstrates that our SCA has a superior performance on multiple aspects than the state-of-the-art.

Graph convolutional network (GCN) has been extensively applied to recommender systems (RS) and achieved significant performance improvements through iteratively aggregating high-order neighbors to model the relevance between users and items as well as their characteristics. In the aggregation process, GCN models usually give neighbors the same or trainable weights based on implicit features, ignoring explicit ones. In this work, we take the features with explicit meanings or extracted with specific purpose as explicit ones (e.g., temporal features) and the others contained in user-item network as implicit ones (e.g., user preferences). However, some explicit features and knowledge play an essential role in improving the model representation ability and explainability in recommendation systems. To deal with the limitation, we propose a GCN based framework to embed the explicit features or those extracted with explicit intentions in this work. We also provide specific implementations based on two commonly researched features, temporal evolution and popularity bias. Specifically, we first experimentally analyze the popularity bias of the representation learning in RS based on two commonly used GCN models. Secondly, we propose a general framework to weigh neighbors based on explicit features or intentions. Thirdly, we implement a Temporal and Popularity weighted Aggregator (TPA) for GCN. The Interest-Forgetting Curve is utilized to capture temporal evolution as temporal weights and the data-driven Beta distribution is employed to tune the weights based on the node popularity flexibly. At last, we conduct extensive experiments on three real-world datasets to demonstrate the effectiveness of TPA in improving recommendation accuracy and alleviating the popularity bias.

The service resource allocation strategy optimization problem has always been a hot issue in mobile edge computing (MEC). In this paper, the problem is formulated as a long-term quality of service (QoS) improvement problem while satisfying the budget of MEC service provider (MSP). Since it is very unrealistic to accurately obtain the request information of user equipments (UEs) over a long time, we first transform the original problem into a series of real-time linear programing sub-problems by using Lyapunov optimization method, and propose a centralized algorithm to determine the resource allocation strategies. However, since the sub-problems are still NP-hard problems, it is a huge challenge to determine the strategies for all UEs with the centralized algorithm in a large scale MEC environment. Thus, we then formulate the sub-problems as an N players non-cooperative game, prove that there exists a Nash equilibrium, and develop two iterative algorithms to find the Nash equilibrium while determining the strategies. Experimental results show that the algorithms can take into account QoS and budget of MSP at the same time, and perform better compared to five other common schemes.

Graph partitioning is one of the fundamental problems in many graph-based applications and systems. It enables the division of a graph into smaller sub-graphs for subsequent parallel processing, reducing the processing latency of the graph. The critical path of a graph is the logical path with the longest delay from input to output. The processing time of the graph mainly depends on the delay incurred by the critical path, independent of other paths with small delays. Therefore, it can reduce the processing time of the graph by protecting the critical path of the graph from partition. However, existing approaches to graph partitioning only focus on metrics such as minimum cut and partition balance. As a result, the critical paths of graphs may be destroyed in the partitioning procedure. To address this problem, we present a critical path awareness approach, namely path-metis, to protect the critical paths and alleviate the processing latency after graph partitioning. In path-metis, two efficient strategies, including Slack and critical path fix strategies, are introduced. The Slack strategy, which incorporates critical path information into the weights of DAG, is used as pre-processing before traditional multi-level partitioning methods, like Metis. Then, for the generated partitioning scheme, the critical path fix strategy is proposed to further protect critical paths from being cut. We demonstrate the effectiveness of our approach on both real and synthetic datasets. From the experimental results, compared to Metis, our method improves critical path performance by 17.70%.

Canonical Polyadic Decomposition (CPD) of sparse tensors is an effective tool in various machine learning and data analytics applications, in which sparse Matricized Tensor Times Khatri-Rao Product (MTTKRP) is the major performance bottleneck. To overcome this bottleneck and support efficient applications, this paper presents HPSpTM, an efficient sparse MTTKRP framework, to exploit the multi-level parallelism and reusability on heterogeneous HPC systems. HPSpTM incorporates: (1) a multi-level matrix-driven tiling engine that leverages the process- and thread-level parallelism of the underlying platform and data reusability based on the derived factor matrix-driven MTTKRP algorithm; (2) a tensor-driven parallel execution that enables buffering-aware scheduling and pipeline scheduling to optimize the performance in the tile granularity; (3) a partition-aware light weight data storage that exploits better data locality based on the proposed hierarchical and fine-grained execution; and (4) a performance auto-tuning technique that offers large flexibility for tile size auto-adjusting across various input datasets based on a designed runtime model. Our experiments show that HPSpTM on a Nvidia Tesla P100 obtains the average performance improvement of up to 76.46% over the state-of-the-arts, and HPSpTM achieves the speedup of up to 15.39× when scaling from 8 to 128 core groups, corresponding to processes, on the Sunway TaihuLight supercomputer.

Abstract The prevalence of short texts on the Web has made mining the latent topic structures of short texts a critical and fundamental task for many applications. However, due to the lack of word co-occurrence information induced by the content sparsity of short texts, it is challenging for traditional topic models like latent Dirichlet allocation (LDA) to extract coherent topic structures on short texts. Incorporating external semantic knowledge into the topic modeling process is an effective strategy to improve the coherence of inferred topics. In this paper, we develop a novel topic model—called biterm correlation knowledge-based topic model (BCK-TM)—to infer latent topics from short texts. Specifically, the proposed model mines biterm correlation knowledge automatically based on recent progress in word embedding, which can represent semantic information of words in a continuous vector space. To incorporate external knowledge, a knowledge incorporation mechanism is designed over the latent topic layer to regularize the topic assignment of each biterm during the topic sampling process. Experimental results on three public benchmark datasets illustrate the superior performance of the proposed approach over several state-of-the-art baseline models.

For the rapid growth computation requirements in big data and artificial intelligence area, CPU-GPU heterogeneous clusters can provide more powerful computing capacity compared to CPU clusters. The number of GPUs on single computing node is scalable, which greatly improves the computing capacity of the cluster under the condition of limited cluster size. However, there is a lack of the effective load-balancing scheduling model in multi-GPU hardware environment. This paper proposes AEML, an acceleration engine for multi-GPU load-balancing in distributed heterogeneous environment. AEML can effectively integrate GPUs into distributed processing framework and achieve great load-balance among multiple heterogeneous GPUs. We propose a heterogeneous task execution model based on multiple GPUs and multiple streams (MGMS), which can effectively balance the workload of multiple GPUs. MGMS model utilizes four core techniques: a fine-grained task mapping mechanism, a device resource unified management scheme, a novel resource-aware GPU task scheduling strategy and a feedback-based streams adjustment scheme. The implementation of AEML system is based on Spark 2.4.1 and NVIDIA CUDA 10.0. We comprehensively evaluate the performance of AEML with multiple typical benchmarks. Experimental results show that AEML can fully exploit the computing power of GPUs and achieve great load-balance among multiple heterogeneous GPUs.

The execution time of a stage is extended by a few slow running tasks in Spark computing environments. To tackle this so-called straggler problem, Spark adopts speculative execution mechanism under which the scheduler speculatively launch additional backup for the straggler with the hope to complete early. However, due to the characteristics of tasks and the complexity of runtime environments, the Spark original speculative execution strategy and its improved versions cannot deal with this problem effectively. In this paper, we propose a novel strategy called ETWR to improve the efficiency of speculative execution in Spark. We consider the heterogeneous environment when we around to tackle the three key points of speculative execution: straggler identification, backup node selection and effectiveness guarantee. Based on the task type classification, first, we divide the task into sub-phases and use both the process speed and progress rate within a phase to find the straggler promptly. Second, we use the Locally Weighted Regression model to estimate the execution time of the task, which will be used to calculate the task's remaining time and backup time. Third, we present iMCP model to guarantee the effectiveness of speculative tasks, which can additionally keep load balancing for nodes. Finally, the factors of fast node and better location are considered when choosing proper backup nodes. Extensive experiments show that ETWR can reduce the job execution time by 23.8 percent, and improve the cluster throughput by 33.2 percent compared with Spark-2.2.0.

In this article, we design a service mechanism for profits optimization between multiple cloud providers and multiple cloud customers (many-to-many). We explore this problem from the perspective of game theory but take a different approach compared with existing cloud resource pricing game methods. First, we regard the relationships among multiple cloud customers as an evolutionary game, and formulate the competitions among the multiple cloud providers as a noncooperative game. Eventually, we form a hybrid game model in which the strategy of each customer and each cloud provider is affected not only by the other side but also by customers or cloud providers other than themselves. Second, based on the hybrid game model, we simulate the bargaining process between cloud providers and customers by controlling supply and demand allocation, and try to ultimately achieve a balanced supply and demand state, i.e., a win-win situation. For each cloud customer and provider, we design a utility function. A customer’s utility involves net profits and the cloud providers’ bidding strategies, and a cloud provider’s utility involves net profits and the cloud customers’ demand strategies. Both sides attempt to maximize their own profits under the influences of each other. We prove that our proposed strategies enable each of the two games to converge to their own equilibrium. Finally, the strategies of cloud customers and providers can be implemented through an iterative proximal algorithm ($\mathcal {IPA}$) and a distributed iterative algorithm ($\mathcal {DIA}$). The experimental results validate our methods and show that the proposed method can benefit both multiple cloud providers and customers.

The excavated information will become obsolete when the data changes in dynamic graphs. To compute the up-to-date results, the graph algorithm has to re-compute the entire data from scratch, which will consume huge computation time and resources. To reduce the cost of such calculations, this paper proposes a model called IncGraph to support incremental iterative computation over dynamic graphs. Different from the way of traditional iteration, IncGraph executes the graph algorithm through reusing the results of the previous graph and performs computation on the part of the graph that has changed. IncGraph has two critical components: (1) an incremental iterative computation model that consists of two steps: an incremental step to calculate the results on the changed vertices of the graph, and a merge step to calculate the results on the entire graph by using the results of the previous graph and the incremental step; and (2) an incremental update method to accelerate the iterative process within the iterative graph algorithm. We implement IncGraph model on GraphX and evaluate its performance by using several representative iterative graph algorithms: PageRank, Connected components, and Single Source Shortest Path. The results show that compared with the traditional iteration, when adding the 100k of vertices in different size data sets, the performance optimization ratio of IncGraph is 31.79 percent averagely, and 50.2 percent maximum; and when the percentage of added vertices varied from 0.01 to 10 percent in different data sets, the performance optimization ratio of IncGraph varied from 19.9 to 66.1 percent. Moreover, the result errors of IncGraph is small and can be neglected.

Recently, with the large-scale outbreak of the global financial crisis and public safety incidents (such as COVID-19), high-performance computing has been widely applied to risk prediction, vaccine development, and other fields. In scenarios where high-performance computing infrastructure responds to the instantaneous explosion of computing demands, a crucial issue is to provide large-scale flexible allocation and adjustment of computing capability by rapidly constructing computing clusters. Existing large-scale computing cluster deployment solutions usually utilize source code deployment or other deployment tools. The great challenge of existing deployment methods is to reduce excessive image distribution time and refrain from configuration defects. In this article, we design an intelligent distributed registry deployment (IDRD) architecture based on the OpenStack cloud platform, which adaptively places distributed image repositories using the containerized deployment of multiple registries. We propose a server load priority algorithm to solve multiple registries placement problems in IDRD. Furthermore, we devise a clustering algorithm based on demand density that can optimize the global performance of IDRD and improve large-scale cluster load balancing capabilities, which has been implemented in the TianHe Supercomputing environment. Extensive experimental results demonstrate that IDRD can effectively reduce$30\%$-$50\%$of the distribution time of component images and significantly improve the efficiency of large-scale cluster deployment.

Transformer-based deep neural networks have recently swept the field of natural language processing due to their outstanding performance, and are gradually spreading to more applications such as image/video processing. However, compared with general DNNs, training a sizeable transformer-based model is further time-consuming and memory-hungry. The existing distributed training strategies for general DNNs are not appropriate or can not efficiently handle transformer-based networks. In view of this, we propose an intra-layer model parallelization optimization strategy, AccTFM, which introduces a novel fine-grained pipeline execution and hybrid communication compression strategy to overcome the synchronization bottleneck. Specifically, on one hand, it first decouples the inter-layer computation and communication dependencies, and then searches for the optimal partitioning strategy to maximize the overlap of computation and communication. On the other hand, the hybrid communication compression module consists of token-level top-$k$sparsification and piecewise quantization methods aiming at minimizing communication traffic. Experimental results show that AccTFM accelerates transformer-based DNNs training by up to 2.08x compared to state-of-the-art distributed training techniques.

The high concurrency and high throughput characteristics of graphics processing units (GPUs) have made researchers continue to use it to optimize distributed parallel computing architectures. With the upgrading of processor architecture, GPUs allow multiple kernels to execute concurrently through stream queues. However, due to the different hardware characteristics and kernel properties in distributed architectures, existing research lacks careful consideration of optimization schemes for concurrent streams and kernel block sizes. Unreasonable stream concurrency and kernel block size configuration will lead to prolonged execution time and waste of computing resources during application execution. Therefore, we propose a multi-GPU multi-stream co-concurrency mechanism (MGSC) in a distributed heterogeneous environment, dynamically adjusting the number of concurrent streams and exploring the optimal block size in task scheduling. According to the memory resources and startup overhead occupied in concurrent stream scheduling, a resource-aware concurrent stream adaptive adjustment mechanism is proposed, which can dynamically adjust the number of streams. To explore the optimal block size, we abstract it as a multi-armed bandit problem (MAB) and propose a block size adjustment algorithm based on the upper confidence bound (UCB). We implement MGSC in Spark 3.1.1 and NVIDIA CUDA 11.2. We conduct comparative experiments with multiple typical benchmarks to evaluate the performance of MGSC. The experimental results show that the algorithm can make full use of the computing power of the GPU and significantly reduce the execution time of tasks.

Deep Neural Networks (DNNs) are widely investigated due to their striking performance in various applications of artificial intelligence. However, with DNNs becoming larger and deeper, the computing resource of a single hardware accelerator is insufficient to meet the training requirements of popular DNNs. Hence, it is required to train them using multiple accelerators in a distributed setting. For a better utilization of the accelerators and a faster training, it is necessary to partition the whole process into segments that can run in parallel. However, in this context, intra-layer parallelization techniques (i.e., data and model parallelization) often face communication and memory bottlenecks, while the performance and resource utilization of inter-layer parallelization techniques (i.e., using pipelining) depend on the partitioning possibilities of the model. We present EffTra, a synchronous hybrid parallelization strategy, that uses a combination of intra-layer and inter-layer parallelism to realize a distributed training of DNNs. EffTra employs the idea of dynamic programming to try to search for the optimal partitioning of a DNN model and assigns devices to the obtained partitions. Our evaluation shows that EffTra accelerates training by up to 2.0x and 1.78x compared to state-of-the-art inter-layer (i.e., GPipe) and intra-layer (i.e., data parallelism) parallelization techniques respectively.

In the parallel computing framework of Hadoop/Spark, data skew is a common problem resulting in performance degradation, such as prolonging of the entire execution time and idle resources. What lies behind this issue is the partition imbalance, that causes significant differences in the amount of data processed by each reduce task. This paper proposes a key reassigning and splitting partition algorithm (SKRSP) to solve the partition skew from the source codes of Spark-core_2.11 project, which considers both the partition balance of the intermediate data and the partition balance after shuffle operators. First, we propose a step-based algorithm for sampling the input data to estimate the general key distribution of entire intermediate data. According to the types of the specific applications, we design two algorithms: hash based key reassigning algorithm (KRHP) and rang based key splitting algorithm (KSRP), which can generate appropriate strategy and implement the skew mitigation in shuffle phase. KKSRP generates the weighted bounds to split intermediate data for the type of sort-based applications while KRHP records these reassigned keys and the new reducers these keys belong to for other applications. Finally, we implement SKRSP in Spark 2.2.0 and evaluate its performance through four benchmarks exhibiting significant data skew: WordCount, Sort, Join, and PageRank. The experimental results verify that our algorithm not only can achieve a better partition balance but also reduce the execution time of reduce tasks effectively.

Installing a six-dimensional force/torque sensor on an industrial arm for force feedback is a common robotic force control strategy. However, because of the high price of force/torque sensors and the closedness of an industrial robot control system, this method is not convenient for industrial mass production applications. Various types of data generated by industrial robots during the polishing process can be saved, transmitted, and applied, benefiting from the growth of the industrial internet of things (IIoT). Therefore, we propose a constant force control system that combines an industrial robot control system and industrial robot offline programming software for a polishing robot based on IIoT time series data. The system mainly consists of four parts, which can achieve constant force polishing of industrial robots in mass production. (1) Data collection module. Install a six-dimensional force/torque sensor at a manipulator and collect the robot data (current series data, etc.) and sensor data (force/torque series data). (2) Data analysis module. Establish a relationship model based on variant long short-term memory which we propose between current time series data of the polishing manipulator and data of the force sensor. (3) Data prediction module. A large number of sensorless polishing robots of the same type can utilize that model to predict force time series. (4) Trajectory optimization module. The polishing trajectories can be adjusted according to the prediction sequences. The experiments verified that the relational model we proposed has an accurate prediction, small error, and a manipulator taking advantage of this method has a better polishing effect.

Closer to mobile users geographically, mobile edge computing (MEC) can provide some cloud-like capabilities to users more efficiently. This enables it possible for resource-limited mobile users to offload their computation-intensive and latency-sensitive tasks to MEC nodes. For its great benefits, MEC has drawn wide attention and extensive works have been done. However, few of them address task migration problem caused by distributed user mobility, which can't be ignored with quality of service (QoS) consideration. In this article, we study task migration problem and try to minimize the average completion time of tasks under migration energy budget. There are multiple independent users and the movement of each mobile user is memoryless with a sequential decision-making process, thus reinforcement learning algorithm based on Markov chain model is applied with low computation complexity. To further facilitate cooperation among users, we devise a distributed task migration algorithm based on counterfactual multi-agent (COMA) reinforcement learning approach to solve this problem. Extensive experiments are carried out to assess the performance of this distributed task migration algorithm. Compared with no migrating (NM) and single-agent actor-critic (AC) algorithms, the proposed distributed task migration algorithm can achieve up 30-50 percent reduction about average completion time.

With the development of the Internet of Things, the data that needs to be processed is increasing rapidly. Therefore, the collaboration of cloud and edge emerges as the times require. Edge nodes are mainly responsible for collecting data, and decide to process the data locally or offload to cloud data centers. Cloud data centers are suitable for data analysis, model training, and managing edge nodes. In this article, we focus on the task assignment problems in collaborative edge and cloud environments and study it in a distributed, non-cooperative environment. An M/M/1 queueing model is established to characterize the task transmission. Because of the multi-core processors, we set an M/M/C queueing model to characterize the task computation. We consider the problem from the perspective of game theory and formulate it into a non-cooperative game among multi-agents (multiple edge data centers) in which each agent is informed with incomplete information (allocation strategies) of others. For each agent, we define a function of the expected cost of tasks as the disutility function, and minimize it subject to the QoS constraint. We analyze the existence of Nash equilibrium and develop a Greedy Energy-aware Algorithm (GEA) to choose active servers using the Limit Searching Algorithm (LSA) to find the ceiling utilization. Then we propose the Best Response Algorithm (BRA) to optimize the utility function. The convergence of the BRA algorithm has been discussed. Finally, the results demonstrate that the BRA algorithm can get a solution close to Nash equilibrium and reach it quickly.

The parallel computing capabilities of GPUs have a significant impact on computationally intensive iterative tasks. Offloading part or all of the deep learning tasks from the CPU to the GPU for execution is mainstream. However, a large number of redundant iterative calculations exist in the iterative process of computing tasks. Therefore, we propose a GPU-based distributed incremental iterative computing architecture that can make full use of distributed parallel computing and GPU memory structure. The architecture supports deep learning and other computationally intensive iterative applications by optimizing data placement and reducing redundant iterative calculations. To support block-based data partitioning and coalesced memory access on GPUs, we propose GDataSet, an abstract data set. The GPU incremental iteration manager called GTracker is designed to be responsible for GDataSet cache management on the GPU. In order to solve the limitation of on-chip memory size, we propose a variable sliding window mechanism. It improves the hit rate of cache access and the speed of data access by realizing the best block arrangement between on-chip memory and off-chip memory. Besides, a communication channel based on an incremental iterative model is designed to support data transmission and task communication in cluster computing. Finally, we implement the proposed architecture based on Spark 2.4.1 and CUDA 10.0. Comparative experiments with widely used computationally intensive iterative applications (K-means, LSTM, etc.) show that the incremental iterative acceleration architecture can significantly improve the efficiency of iterative computing.

Large-scale data centers have been widely used for cloud services, and the stability of various cloud services has received additional attention from users. Although service disruptions are not as catastrophic as they once were, their impact might be more extensive than before. These outages may trigger the migration of virtual machines (VMs) located in the failure node. However, the access time of each VM is random, unlike the accident time, which can be predicted. This means that traditional migration caused by service interruptions may result in a large number of unwanted migrations, regardless of the user’s downtime experience. Migration is an expensive process in terms of the resources needed as well as the degradation of application performance during migration. A balance between the recovery time of the service (to minimize the migration resulting from a given placement) and the downtime experience of the users (to minimize the impact of access interruptions) is needed. In this paper, we propose HMGOWM, a hybrid decision-making mechanism for automating the migration of VMs. Our proposed mechanism extends the original VM migration performance cost model, greatly reducing the downtime experience of the users. To achieve high performance and a good load balance, a multi-objective monitoring system for both VMs and physical machine nodes and an adaptive VM migration-scheduling scheme for the OpenStack cloud platform are proposed. Extensive experiment results indicate that the downtime experienced by users can be efficiently reduced and that the implementation of HMGOWM outperforms the original scheduling of the OpenStack cloud platform.

Once defined, the access control policies and regulations would never be changed in a running and state transition process. However, it will give attackers the possibility of discovering vulnerabilities in the system, and the control systems lack the ability of dynamic perception of security state and risk, causing the systems to be exposed to risks. In this article, a dynamic Bell-LaPadula (BLP) model is proposed. The conditional random field (CRF) is introduced into the BLP model to optimize the rules. First, the model formalizes the security attributes, states of system, transition rules, and constraint models on the basis of the state transition of CRFs. After the historical system access logs are processed as the original dataset, a feature selection method is proposed to extract the requests and current states as feature vectors. Second, this article presents a rules training algorithm based on L-BFGS to implement the study and training of datasets, and then marks the logs in the test set through Viterbi algorithm automatically. On the base of these, a rule generation algorithm is proposed to dynamically adjust the access control rules based on the current security status and events of the system. Third, the security of CRFs-BLP is proved by theoretical analysis. Finally, the validity and accuracy of the model are verified by estimating the value of the precision, recall, and F1-score. As the system threats are shown to be decreased obviously from these experiments, this dynamic model can decrease the vulnerabilities and risk effectively.

The quality of herbal medicine has an important impact on human health. In this paper, we proposed a blockchain-based herbal quality and safety supervisory model for the current frequent herbal counterfeiting phenomenon. We manage the production, processing, and trading processes of herbal medicines by exploiting the blockchain's immutable and traceable properties. We proposed a segmented encryption method for information to encrypt the private information of enterprises. We use shared cloud storage to reduce waste of local storage space, and we proposed a verifiable random chain cutting mechanism based on the verifiable random function to handle the redundant blocks of the chain. Our article addressed the problem of herbal source falsification. Automated recording of key factors such as soil and temperature that affect the quality of herbal medicines is done from the seedling stage. Our herbal quality and safety supervisory model used blockchain technology to increase control over the production and distribution of herbal products, reduce herbal counterfeiting, and improve the efficiency of the system.

Nowadays, cloud-supported Internet of Things (Cloud-IoT) has been broadly deployed in smart medical systems, where the limitations of Internet of Things (IoT)-associated medical devices in terms of data access, storage, scalability, and computing are solved through the use of cloud computing architectures. However, with the rapid development of medical equipment and the increasing number of medical devices, it will be extremely difficult to program or manage such an expanding and massive medical IoT system in traditional single-cloud platforms. In this article, we design and implement a multicloud framework for building OpenStack-based platform for medical IoT, referred to as the tri-storage failure recovery system (Tri-SFRS). To implement Tri-SFRS, we combine several techniques to achieve this reduction in effort, including a multicloud cascading architecture, a low-overhead native testing framework, a medical data storage-backup mechanism, and snapshot-volume cascaded operations for b-ultrasonic data. Tri-SFRS is also able to simultaneously enable resource management specialization. Tri-SFRS has been designed as a native component in the OpenStack platform, and it demonstrates the degree of native OpenStack multicloud platform management by our proposed cascading framework. Comparing with the traditional single-cloud OpenStack platform, Tri-SFRS can reduce the resource-request processing latency from B ultrasonic machines by up to 20%. Our experiments also demonstrate the broad applicability of Tri-SFRS.

Recent researches try to integrate word information into the character-based Chinese NER by modifying the structure of the standard BiLSTM-CRF model. They follow the paradigm of explicitly modeling forward and backward sequences, adopting an LSTM variant that takes both characters and words as input for each direction. Though enriching the representations, these models cannot fully exploit the interaction between future and past contexts. In this paper, we propose a novel word-character graph convolution network (WC-GCN) which uses a cross GCN block to simultaneously process the word-character directed acyclic graphs (DAGs) of two directions. To improve the capture of long-distance dependency, a global attention GCN block is introduced to learn node representations conditioned on a global context. In both blocks, unlike previous works where each word is attached to its associated character or taken as a shortcut between LSTM cells, words and characters are treated equally as nodes in the graph and have their instance-specific representations. Experiments on four widely used datasets show that our proposed model can work standalone or with the standard BiLSTM. Both forms can outperform previous LSTM-based models without training on extra corpora while only an external lexicon and its corresponding pretrained character and word embeddings are needed.

For frequent disk I/O and large data transmissions among different racks and physical nodes, intermediate data communication has become the most important performance bottle-neck in most running Hadoop systems. This paper proposes a reduce placement algorithm called CORP to schedule related map and reduce tasks on the near nodes of clusters or racks for data locality. Because the number of keys cannot be counted until the input data are processed by map tasks, this paper applies a reservoir algorithm for sampling the input data, which can bring the distribution of keys/values closer to the overall situation of original data. Based on the distribution matrix of the intermediate results in each partition, by calculating the distance and cost matrices among the cross node communication, the related map and reduce tasks can be scheduled to relatively nearby physical nodes for data locality. We implement CORP in Hadoop 2.4.0 and evaluate its performance using three widely used benchmarks: Sort, Grep, and Join. In these experiments, an evaluation model is proposed for selecting the appropriate sample rates, which can comprehensively consider the importance of cost, effect, and variance in sampling. Experimental results show that CORP can not only improve the balance of reduces tasks effectively but also decreases the job execution time for the lower inner data communication. Compared with some other reduce scheduling algorithms, the average data transmission of the entire system on the core switch has been reduced substantially.

In this article, we investigate code-oriented partitioning computation offloading strategy for multiple user equipments (UEs) and multiple mobile edge computing servers with limited resources (i.e., limited computing power and waiting task queues with finite capacity). This article aims to develop an offloading strategy to decide the execution location, CPU frequency, and transmission power for UE while minimizing the execution overhead (i.e., a weighted sum of energy consumption and computational time) of UE's applications, which is an NP-hard problem. To achieve the objective, first, we transform the problem into a convex optimization problem and find the optimal solution. Second, we propose a decentralized computation offloading strategy (DCOS) algorithm for UE, and define a dictionary data structure for recording the strategy of the UE to reduce the algorithm complexity. Finally, the effectiveness of DCOS, and the impact of various key parameters on the strategy and overhead are demonstrated by simulation experiments.

In this article, two improved finite-time convergent complex-valued zeroing neural network (IFTCVZNN) models are presented and investigated for real-time solution of time-varying reciprocal of complex matrices on account of two equivalent processing ways of complex calculations for nonlinear activation functions. Furthermore, a novel nonlinear activation function is explored to modify the comprehensive performance of such two IFTCVZNN models. Compared with existing complex-valued neural networks converging within the limited time, the proposed IFTCVZNN models with the new activation function have better finite-time convergence and less conservative upper bound. Numerical simulations verify that the maximum of convergence time estimated via Lyapunov stability is theoretically much closer to the actual convergence time.

In the distributed computing framework of Spark, cross-node/rack data transfer produced by map tasks and reduce tasks are common problems resulting in performance degradation, such as prolonging of entire execution time and network congestion. To address these problems, this article utilizes the bipartite graph modelling to propose an optimal locality-aware task scheduling algorithm. By considering global optimality, the algorithm can generate the optimal scheduling solution for both the map tasks and the reduce tasks for data locality. Because of the different communication modes, this article uses a unified graph to model the map task scheduling and the reduce task scheduling respectively. Then, by calculating the communication cost matrix of tasks, we formulate an optimal task scheduling scheme to minimize overall communication cost and transform the problem as the well-known graph problem: minimum weighted bipartite matching (MWBM), which can be resolved by Kuhn-Munkres algorithm. In addition, this article proposes a locality-aware executor allocation strategy to improve the data locality further. We implement our algorithm and strategy in Spark-2.4.1 and evaluate its performance using several representative micro-benchmarks, macro-benchmarks, and HiBench benchmark suite. The experimental results verify that by reducing the network traffic and access latency, the proposed algorithm can improve the job performance substantially compared to some other task scheduling algorithms.

Collaborative filtering (CF), one of the most famous methods for building recommendation systems, recommends relevant items to users or predicting ratings of users’ unknown items. Matrix factorization (MF) models are well-known model to deal with predicting the rating problem. However, the recommen dation system based on matrix factorization is hard to keep up with the rapidly changing real-world data. When ratings on new users or new items come, the static model can not fit well on new data. As a consequence, if the current thing does not apply, the prediction accuracy will lose. In addition, it is a significant computation cost to rebuild the model on the whole data. To capture these changes, in this paper, we construct an online-and-offline Collaborative Filtering with a multi-method model to improve the traditional CF method, called Online SGD with Offline Knowledge (OSGDO for short). Besides, we propose a real-time incremental recommendation framework on Apache Flink, which is a scalable stream and batch data processing platform. Meanwhile, we implement our proposed method on our proposed framework. Our method proves to be good at online training when new observations arrive. And the results of experiments show that the dynamic training process we proposed is more efficient than rebuild the model on all the data. At the same time, our algorithm performs well in practice and can achieve impressive accuracy quickly when it is tested with the well-known data sets of MoviesLens and Netflix.

Measuring semantic textual similarity (STS) lies at the core of many applications in natural language processing (NLP). Recently, most models have considered semantic information or syntactic information, but seldom an unified model to make full use of these two kinds of information. Based on the k nowledge from the trained word vectors, this paper proposes a semantic-embedded dependency tree (SEDT) model based on word2vec and glove, which can be treated as a syntactic-semantic representation. In consideration of the words in a sentence for the contribution of the semantic are different, this model extends the semantic-embedded dependency tree model to an enhanced semantic-embedded dependency tree (ESEDT). And a modified partial tree kernel (MPTK) is proposed to automatically extract the syntactic-semantic patterns in this tree. Because the syntactic information, semantic knowledge, and the contribution distribution of the word attention model are all considered in this model, it can measure more comprehensive sentence semantics to improve the accuracy of STS results. Finally, SEDT/E-SEDT is applied to SemEval semantic textual similarity tasks and evaluate its performance through two widely used benchmarks: the Pearson correlation coefficient and the Spearman correlation coefficient. The experimental results show that SEDT/E-SEDT can effectively improve the accuracies of sentence similarity judgments. Compared with the other similar methods to calculate the semantic similarity, such as some neural network models, SEDT/E-SEDT can obtain better performance on most dataset.

As the Energy Wall and the Reliability Wall become unavoidable, it is a demanding and challenging task to reduce energy consumption in large-scale storage systems in modern data centers while retaining acceptable systems reliability. We propose a reliable energy-efficient storage system called DuoFS, which aims at balancing the energy efficiency, the reliability and the performance of parallel storage systems by seamlessly integrating one HDD-based file system and one SSD-based file system. At the heart of the DuoFS is a transformative middleware layer that dispatches files to the one of the two independent parallel file systems based on the files' I/O access popularity. By replicating popular files to the SSD-based file system and pushing the HDD-based file system into the low-power mode under light workload conditions, DuoFS can reduce significant energy consumption, avoid major factors that harm the storage systems reliability, and extract SSDs good I/O performance. Experimental results show that the DuoFS system saves up to 40% of energy, achieves up to 50% better I/O performance while only sacrificing less than 15% of the system's reliability.

In currently popular access control models, the security policies and regulations never change in the running system process once they are identified, which makes it possible for attackers to find the vulnerabilities in a system, resulting in the lack of ability to perceive the system security status and risks in a dynamic manner and exposing the system to such risks. By introducing the maximum entropy (MaxENT) models into the rule optimization for the Bell-LaPadula (BLP) model, this paper proposes an improved BLP model with the self-learning function: MaxENT-BLP. This model first formalizes the security properties, system states, transformational rules, and a constraint model based on the states transition of the MaxENT. After handling the historical system access logs as the original data sets, this model extracts the user requests, current states, and decisions to act as the feature vectors. Second, we use k -fold cross validation to divide all vectors into a training set and a testing set. In this paper, the model training process is based on the Broyden-Fletcher-Goldfarb-Shanno algorithm. And this model contains a strategy update algorithm to adjust the access control rules dynamically according to the access and decision records in a system. Third, we prove that MaxENT-BLP is secure through theoretical analysis. By estimating the precision, recall, and F1-score, the experiments show the availability and accuracy of this model. Finally, this paper provides the process of model training based on deep learning and discussions regarding adversarial samples from the malware classifiers. We demonstrate that MaxENT-BLP is an appropriate choice and has the ability to help running information systems to avoid more risks and losses.

Latent Dirichlet Allocation (LDA), has been recently used to automatically generate text corpora topics, and applied to sentences extraction based multi-document summarization algorithms. In this paper, we propose a novel approach to automatic generation of aspect-oriented summaries from multiple documents. Our approach is to combine the traditional summary generation algorithm and the the abstract generation algorithm based on deep learning.We employ the improved traditional summary generation algorithm to convert multiple documents into a single document, and then using the resulting single document with the deep learning method to extract the final summary. At first, we apply improved LDA model to cluster sentences in all documents. Second, We employ the extended LexRank algorithm to sort the sentences in each cluster. Third, we use extended Hedge Trimmer algorithm for sentence compression. Fourth, We apply Integer Linear Programming for sentence selection, and in this step ,we get the single document. Finally, We employ the textum on TensorFlow to get the final abstract. The experiments showed that the proposed algorithm achieved better performance compared the other state-of-the-art algorithms on DUC2005 and TAC2010 corpus.

Summary When facing massive statistical data, the k‐means algorithm is very difficult to satisfy the need of data processing as it lacks an effective parallel mechanism. This paper proposes an improved k‐means algorithm (IMR‐KCA) to conduct clustering analysis based on medical data employing MapReduce computing framework. Through analyzing the defects of vast redundancy in the traditional k‐means algorithms, a selection model is firstly proposed to simplify the computations with multiple clustering centers. Based on several proposed theorems, we prove the correctness of this selection model. Second, this paper provides a method to calculate the distances from extreme points to central points, and the original Euclidean distance is replaced with Manhattan distance. For this simplification, a group of theorems are proposed to prove the correctness. Next, we provide a group of implementation algorithms to complete the parallelism of the clustering computation employing the MapReduce framework. Finally, the experimental results illustrate that IMR‐KCA is more reliable and efficient than the direct parallelization of the traditional clustering algorithms based on MapReduce. Copyright © 2017 John Wiley & Sons, Ltd.

With the emergence of the big data age, the issue of how to obtain valuable knowledge from a dataset efficiently and accurately has attracted increasingly attention from both academia and industry. This paper presents a Parallel Random Forest (PRF) algorithm for big data on the Apache Spark platform. The PRF algorithm is optimized based on a hybrid approach combining dataparallel and task-parallel optimization. From the perspective of data-parallel optimization, a vertical data-partitioning method is performed to reduce the data communication cost effectively, and a data-multiplexing method is performed is performed to allow the training dataset to be reused and diminish the volume of data. From the perspective of task-parallel optimization, a dual parallel approach is carried out in the training process of RF, and a task Directed Acyclic Graph (DAG) is created according to the parallel training process of PRF and the dependence of the Resilient Distributed Datasets (RDD) objects. Then, different task schedulers are invoked for the tasks in the DAG. Moreover, to improve the algorithm's accuracy for large, high-dimensional, and noisy data, we perform a dimension-reduction approach in the training process and a weighted voting approach in the prediction process prior to parallelization. Extensive experimental results indicate the superiority and notable advantages of the PRF algorithm over the relevant algorithms implemented by Spark MLlib and other studies in terms of the classification accuracy, performance, and scalability. With the expansion of the scale of the random forest model and the Spark cluster, the advantage of the PRF algorithm is more obvious.

The extreme learning machine (ELM) has become one of the most important and popular algorithms of machine learning, because of its extremely fast training speed, good generalization, and universal approximation/classification capability. The proposal of hierarchical ELM (H-ELM) extends ELM from single hidden layer feedforward networks to multilayer perceptron, greatly strengthening the applicability of ELM. Generally speaking, during training H-ELM, large-scale datasets (DSTs) are needed. Therefore, how to make use of H-ELM framework in processing big data is worth further exploration. This paper proposes a parallel H-ELM algorithm based on Flink, which is one of the in-memory cluster computing platforms, and graphics processing units (GPUs). Several optimizations are adopted to improve the performance, such as cache-based scheme, reasonable partitioning strategy, memory mapping scheme for mapping specific Java virtual machine objects to buffers. Most importantly, our proposed framework for utilizing GPUs to accelerate Flink for big data is general. This framework can be utilized to accelerate many other variants of ELM and other machine learning algorithms. To the best of our knowledge, it is the first kind of library, which combines in-memory cluster computing with GPUs to parallelize H-ELM. The experimental results have demonstrated that our proposed GPU-accelerated parallel H-ELM named as GPH-ELM can efficiently process large-scale DSTs with good performance of speedup and scalability, leveraging the computing power of both CPUs and GPUs in the cluster.

The increasing main memory capacity and the explosion of big data has fueled the development of in-memory big data management and processing. By offering efficient in-memory parallel execution model which eliminates disk I/O bottleneck, existing in-memory cluster computing platforms (e.g., Flink and Spark) have already been proven to be outstanding platforms for big data processing. However, these platforms are merely CPU-based systems now. This paper has proposed GFlink, an in-memory computing architecture on heterogeneous CPU-GPU clusters for big data. Our proposed architecture extends the original Flink from CPU clusters to heterogeneous CPU-GPU clusters, greatly improving the computational power of Flink. Furthermore, we proposed an abstract GPU-based model named as GDST, hiding the programming complexity of GPUs behind the simple and familiar high-level interfaces, and automatically managing task partitioning, device memory, and parallelization on MultiCore GPUs. To achieve high performance and good load-balance, an efficient JVM-GPU communication strategy and an adaptive locality-aware scheduling scheme for three-stage pipeline execution are proposed. Extensive experiment results indicate that not only the high computational power of GPUs can be efficiently utilized, but also the implementations on GFlink outperforms that on the original CPU-based Flink.

Summary In this paper, we propose a method about task scheduling and data assignment on heterogeneous hybrid memory multiprocessor systems for real‐time applications. In a heterogeneous hybrid memory multiprocessor system, an important problem is how to schedule real‐time application tasks to processors and assign data to hybrid memories. The hybrid memory consists of dynamic random access memory and solid state drives when considering the performance of solid state drives into the scheduling policy. To solve this problem, we propose two heuristic algorithms called improvement greedy algorithm and the data assignment according to the task scheduling algorithm, which generate a near‐optimal solution for real‐time applications in polynomial time. We evaluate the performance of our algorithms by comparing them with a greedy algorithm, which is commonly used to solve heterogeneous task scheduling problem. Based on our extensive simulation study, we observe that our algorithms exhibit excellent performance and demonstrate that considering data allocation in task scheduling is significant for saving energy. We conduct experiments on two heterogeneous multiprocessor systems. Copyright © 2016 John Wiley & Sons, Ltd.

Summary As a parallel computation framework, Spark can cache repeatedly resilient distribution datasets (RDDs) partitions in different nodes to speed up the process of computation. However, Spark does not have a good mechanism to select reasonable RDDs to cache their partitions in limited memory. In this paper, we propose a novel selection algorithm, by which Spark can automatically select the RDDs to cache their partitions in memory according to the number of use for RDDs. Our selection algorithm speeds up iterative computations. Nevertheless, when many new RDDs are chosen to cache their partitions in memory while limited memory has been full of them, the system will adopt the least recently used (LRU) replacement algorithm. However, the LRU algorithm only considers whether the RDDs partitions are recently used while ignoring other factors such as the computation cost and so on. We also put forward a novel replacement algorithm called weight replacement (WR) algorithm, which takes comprehensive consideration of the partitions computation cost, the number of use for partitions, and the sizes of the partitions. Experiment results show that with our selection algorithm, Spark calculates faster than without the algorithm, and we find that Spark with WR algorithm shows better performance. Copyright © 2015 John Wiley & Sons, Ltd.

MapReduce offers a promising programming model for big data processing. One significant issue in practical applications is data skew, its an important reason for the emergence of stragglers which makes the data assigned to each reducer imbalance. This paper presents CSRA, an efficient resource allocation algorithm in MapReduce considering data skew. CSRA aims at reducing the running time and coefficient of variation by reordering the task list and splitting the big clusters. Through thinking over the actual status of tasks, this method largely squares up the resource utilization. After we implement CSRA in Hadoop, the experiments show that CSRA has negligible overhead and can speed up the execution time of some popular applications obviously.

With the increasing energy consumption of computing systems and the growing advocacy for green computing, energy efficiency has become one of the critical challenges in high-performance heterogeneous computing systems. Energy consumption can be reduced by not only hardware design but also software design. In this paper, we propose an energy-aware scheduling algorithm with equalized frequency, called EASEF, for parallel applications on heterogeneous computing systems. The EASEF approach aims to minimize the finish time and overall energy consumption. First, EASEF extracts the set of paths from an application. Then, it reconstructs the application based on the extracted set of paths to achieve a reasonable schedule. Finally, it adopts a progressive way to equalize the frequency of tasks to reduce the total energy consumption of systems. Randomly generated applications and two real-world applications are examined in our experiments. Experimental results show that the EASEF algorithm outperforms two existing algorithms in terms of makespan and energy consumption.

Processing large volumes of data has presented a challenging issue, particularly in data-redundant systems. As one of the most recognized models, the conditional random fields (CRF) model has been widely applied in biomedical named entity recognition (Bio-NER). Due to the internally sequential feature, performance improvement of the CRF model is nontrivial, which requires new parallelized solutions. By combining and parallelizing the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) and Viterbi algorithms, we propose a parallel CRF algorithm called MapReduce CRF (MRCRF) in this paper, which contains two parallel sub-algorithms to handle two time-consuming steps of the CRF model. The MapReduce L-BFGS (MRLB) algorithm leverages the MapReduce framework to enhance the capability of estimating parameters. Furthermore, the MapReduce Viterbi (MRVtb) algorithm infers the most likely state sequence by extending the Viterbi algorithm with another MapReduce job. Experimental results show that the MRCRF algorithm outperforms other competing methods by exhibiting significant performance improvement in terms of time efficiency as well as preserving a guaranteed level of correctness.

In order to accelerate the convergence and improve the calculation accuracy for parameter optimization of the Muskingum model, we propose a novel, adaptive hybrid particle swarm optimization (AHPSO) algorithm. With the decreasing of inertial weight factor proposed, this method can gradually converge to a global optimal with elite individuals obtained by hybrid PSO. In the paper, we analyzed the feasibility and the advantages of the AHPSO algorithm. Then, we verified its efficiency and superiority by application of the Muskingum model. We intensively evaluated the error fitting degree based on the comparison with four known formulas: the test method (TM), the least residual square method (LRSM), the nonlinear programming method (NPM), and the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. The results show that the AHPSO has a higher precision. In addition, we compared the AHPSO algorithm with the binary-encoded genetic algorithm (BGA), the Gray genetic algorithm (GGA), the Gray-encoded accelerating genetic algorithm (GAGA) and the particle swarm optimization (PSO), and results show that AHPSO has faster convergent speed. Moreover, AHPSO has a competitive advantage compared with the above eight methods in terms of robustness. With the efficiency of this approach it can be extended to estimate parameters of other dynamic models.

As the data collection volumes growing rapidly, some complex computation are beyond the ability of our classical process methods. A framework combine between MapReduce and workflow can present a good contribution to this problem through parallel processing for the largescale systems. Currently there are several researches on the scheduling policy for this combination framework in homogeneous cluster or simple heterogeneous cluster, however the scheduling on MapReduce-level and workflow-level are detached. Thus we firstly propose a MapReduce-enabled scientific workflow integrated with an optimization scheduling algorithm to consider both level simultaneously and to support complex heterogeneous environment. Our new Model comprise two components: The job prioritizing module to compute the priorities of all jobs, and the task assignment module to allocate suitable slots for each block and schedule the tasks with respect to data-local. We prove by experiment that in our combination framework the new scheduler policy (MRWS) outperforms other polices in this area.