Mixture-of-Experts (MoE) is a sparse neural architecture that significantly increases model capacity while maintaining low computational complexity. However, deploying MoE-based large language models (LLMs) on memory-constrained edge devices remains challenging due to their substantial memory requirements. To address this issue, we propose FIRM-MoE, a fine-grained expert offloading framework designed to enable flexible and efficient MoE inference. The core insight of our approach is to reduce the risk of inaccurate expert loading by decomposing each expert into fine-grained sub-experts and then dynamically allocating them through a fine-grained scheduling strategy. To further reduce the error in expert loading, we introduce a multi-layer expert prediction mechanism and a resource-adaptive expert pre-loading algorithm to enable more robust expert allocation. This design allows our model to achieve more efficient expert utilization and improved resilience to prediction errors. We conduct extensive experiments to demonstrate the superiority of FIRM-MoE across diverse memory constraints. The results show that FIRM-MoE achieves up to 1.5× speedup and 2.8× memory savings in decoding, compared to state-of-the-art MoE offloading strategies.

In recent years, RF fingerprinting (RFF) has emerged as a promising technology for wireless device authentication. However, temporal variations in device load and temperature, along with channel effects, lead to inconsistencies in RFF distributions between training and testing phases. As a result, deep learning (DL)-based recognition models often suffer from degraded performance. To address this problem, we propose the first test-time-adaptation (TTA) approach to improve the domain generalization ability of RFF recognition models. We first analyze the causes of time-varying RFF distribution shifts, such as carrier frequency offset (CFO), and develop a physical impairment-based data augmentation strategy. Based on this, we further propose a physically information-aware prototype to guide the model for TTA. Our method requires no model retraining or labeled test samples, and is a lightweight, nonparametric solution. Finally, our approach is extensively evaluated using mobile phones with the IEEE 802.11 orthogonal frequency division multiplexing (OFDM) system, which demonstrates that our scheme can effectively improve RFF average recognition performance by about 7.8%.

Latent Diffusion Models have become a powerful tool for generating high-fidelity unrestricted adversarial examples. However, the existing methods typically perturb only the initial latent or rely on prompt engineering, which is ill-suited to the iterative nature of the diffusion process, plus optimization instability due to external text prompts and cumulative drift that push the adversarial images off the data manifold. In this paper, we propose a hierarchical attack framework that operates in alignment with the model's generative manifold and leverages intermediate denoising states to maximize attack transferability and visual fidelity. Extensive experiments show that the proposed attack improves adversarial transferability by 10-20% against a diverse set of normally-trained models and achieves over 10.5% higher success rate against adversarially-defended models, while simultaneously enhancing visual quality by 1.0-1.2 FID reduction and 16.7% LPIPS improvements.

Text-to-3D Scene Retrieval (T3SR) aims to retrieve 3D scenes that match users' linguistic queries, enabling intuitive access to 3D scene repositories. Existing approaches rely on joint embedding learning with large amounts of paired text–scene data, which is expensive to collect and often fails to generalize under open-vocabulary queries and diverse scene distributions. In this paper, we propose Struct-Align, a foundation-model-driven framework for zero-shot T3SR that eliminates the need for paired training data. Our key insight is to reformulate T3SR as a single-modality structural alignment problem by converting both 3D scenes and textual queries into a shared, schema-aligned textual representation compatible with pretrained text embedding models. To reliably derive such representations from complex 3D environments, we introduce a role-decomposed scene structuring pipeline that mitigates generative instability and produces semantically consistent scene depictions. To address the inherent semantic asymmetry between query and scene representations, we further propose a locality-aware structural matching strategy that explicitly localizes query intent and performs instance- and relation-level alignment within query-relevant substructures. Extensive experiments on multiple benchmarks demonstrate that Struct-Align outperforms both training-based and zero-shot baselines while exhibiting strong robustness to domain shift.

Different from traditional object detectors such as YOLO, Detection Transformers (DETR) have reshaped the landscape of object detection by replacing heuristic-driven components like Non-Maximal Suppression with a fully end-to-end framework based on one-to-one Hungarian matching. While the majority of research has focused on improving the slow training convergence of DETR, this work investigates their security from an adversarial perspective. We unveil a critical vulnerability stemming directly from DETR’s core design: the deterministic one-to-one mapping between object queries and ground-truth objects can be exploited. This allows an adversary to craft perturbations that selectively manipulate specific target objects – causing them to vanish or be misclassified – while preserving the detection integrity of all other objects in the scene. Our initial analysis reveals that conventional gradient-based attacks are ill-suited for this task, as they induce unintended interference on non-target instances, a phenomenon we term as the “spillover effect”. To overcome this, we re-formulate the attack optimization by incorporating a novel penalty term that explicitly decouples the adversarial influence on target and non-target objects. Furthermore, we provide theoretical analysis to derive perturbation bounds under which the optimal matching assignments remain invariant, offering deeper insights into the model’s stability. Extensive experiments on standard benchmarks demonstrate that our proposed attack significantly improves the success rate and convergence speed while inducing far fewer feature-level artifacts, making the attack both more effective and stealthier.

The emergence of abundant non-stationary radio signal (NSRS) data presents significant opportunities for applications in wireless communications, radar systems, remote sensing, and healthcare. While deep learning models have shown promise in capturing sequence dependencies, deriving generic and fine-grained representations of NSRS data remains challenging due to its complex, dynamic nature and the scarcity of labeled data. The NSRS data are often frequency-sensitive and exhibit minuscule inter-class distances, posing significant challenges for precise classification. To address these issues, we propose a novelDualModalityPatchContrastive (DMPC) framework. This framework leverages a stochastic patching paradigm for diverse local pattern extraction and a time-frequency cross-view optimization for frequency-sensitive feature mining. Furthermore, an Attentive Patch Aggregation (APA) mechanism enhances fine-grained inference under few-shot conditions through patch-level feature voting. Extensive experiments demonstrate the effectiveness of our approach in addressing the unique challenges of NSRS data.

We present SNH-SLAM, a novel expandable dense neural simultaneous localization and mapping (SLAM) method that constructs a neural field in real-time based on run-time observation. To reach this challenging goal without any scene prior, we utilize instant depth supervision to drive the extension of planar convex hulls, where a single hash table maintains multi-level feature units embedded in the planar convex hulls. This design facilitates high-fidelity, hole-free, and low-memory map reconstruction while adding only a tiny time burden to the training process. Our approach performs mapping by minimizing both RGBD-based re-rendering loss and Truncated Signed Distance Field (TSDF) loss. In addition, for camera tracking, our optimization strategy allows SNH-SLAM to converge faster on the pose estimation and maintain robustness. We evaluate our method on common benchmarks and compare it with existing dense neural RGB-D SLAM methods. The evaluation results show the competitiveness of the SNH-SLAM in tracking accuracy, reconstruction quality, memory usage, and frame processing speed. Project page:https://xiaoshumiao123.github.io.

A data lake maintains large amounts of heterogeneous data with different data schemas and query interfaces. Efficiently querying and analyzing the heterogeneous data enables users to gain more complete insights. In this article, we study a novel problem of distributed keyword search across heterogeneous data sources. Traditional distributed search algorithms generally require the predefined crossing edges connecting relevant data instances for communication between different sources, which is unpractical for the data lake due to the schema heterogeneity. To effectively perform keyword search over the data lake, we first introduce canonical graphs and then develop a best-first search algorithm called UnifySea, which explores the answers across different sources based on the unified identification of related instances. To further improve the query efficiency, we propose a novel incremental keyword search algorithm called DistSea, which just need to identify the promising relevant data between different sources. DistSea incrementally calculates the optimal answers based on locally partial evaluation. Equipped with several efficient pruning rules, DistSea reduces unpromising tree calculation across different sources. Experimental evaluations on six real-world benchmarks demonstrate the effectiveness, efficiency, and scalability of the proposed algorithms.

As data centers continue to expand in size and complexity, obtaining global traffic insights necessitates aggregating statistical data from numerous individual nodes, a process critical for effective network management. However, in data centers, existing approaches often rely on querying individual endpoint hosts to gather cluster-wide statistics, which introduces substantial latency and reduces efficiency, particularly in large-scale deployments. To address this issue, we propose HiMon, a cost-efficient and high-accurate distributed monitoring system for optimizing traffic aggregation. HiMon enables distributed nodes to perform real-time, flow-level statistical processing and report the data to a master node with minimal bandwidth consumption. The master node aggregates the collected data to construct a comprehensive global traffic view. To enable high-speed and high-precision perpacket processing on child nodes, we introduce MaxSketch. MaxSketch’s data structure and update strategy allow it to accurately estimate child node traffic with minimal memory and computational overhead. For high-speed aggregation on the master node, we present PolySketch, which significantly boosts aggregation efficiency by delegating most computational tasks to the child nodes. Together, the hierarchical sketch structures of MaxSketch and PolySketch form the HiMon monitoring system. Experimental evaluations demonstrate that HiMon surpasses baseline algorithms, achieving a 17-210× improvement in traffic processing efficiency, a 25-42× reduction in master node bandwidth consumption, and a 3.69-8.97× increase in accuracy.

The emerging multi-modal applications exemplified by multi-DNN inference have renewed interests for mobile intelligence. The goal is to utilize heterogeneous processors on-board to maximize throughput and resource utilization. Among a variety of options, building model-parallel pipelines across different processors is a promising way. However, the existing efforts either focus on optimizing homogeneous DNN executions or simply ignore co-execution slowdown on the shared memory bus. Based on extensive empirical studies and insights with various degrees of resource contention, in this work, we introduce Hetero2Pipe, a two-step pipeline planner based on dynamic programming, and contention-mitigated pipeline bubble minimization to make the problem tractable within manageable search space. The extensive evaluation across three commercial SoCs demonstrates 2-8× speedup compared to the state-of-the-art schemes.

Entity matching (EM) aims to identify whether two entities from different data sources refer to the same real-world entity. Most existing cross-modal EM assume that images have simple scenes containing few objects, or do not fully consider the cross-modal knowledge associated with entities. To support more practical application scenarios such as multi-modal knowledge graph integration and visual question answering in data lakes, we introduce our problem of semantic-driven EM across graph and images in this paper. Current semantically matching solutions over cross-modal data face the obstacle of low training efficiency, since their time complexity quadratically grows with the number of entities. To alleviate this issue, we present a novel framework (namely CrossETR) that follows an exploration-then-refinement paradigm. Firstly, a candidate exploration policy is proposed to boost the training efficiency. It explores candidate pairs according to entity correlations and captures structural semantics by adaptive sampling the most informative neighborhood subgraphs. Secondly, the cross-modal entity representations are refined to break modality heterogeneity to support unsupervised matching prediction. Extensive experimental evaluations on three publicly available benchmarks demonstrate the superiority of CrossETR over state-of-the-art approaches in terms of effectiveness and efficiency. Furthermore, a case study highlights that our proposed semantic-driven EM is promising to improve the performance of downstream tasks such as multi-modal knowledge graph integration.

Entity matching (EM) aims to identify equivalent entities across different data sources. Current EM assumes that these data are either homogeneous with aligned schema or heterogeneous but can be transformed into a unified modality. There is an urgent need to consider the entities with different modalities to support practical application scenarios over data lakes such as multi-modal data integration and recommendation system. It is impractical to unify their data modalities. To support EM on heterogeneous entity with different data formats and modalities, we propose cross-modal entity matching in this paper. Inspired by the promising performance achieved by recent pre-trained models, we perform cross-modal entity matching by prompt-tuning pre-trained multi-modal large models (MMLMs) in an unsupervised manner. However, the prompt-tuning faces three challenging issues: (i) objective gap between pre-training and tuning of MMLMs; (ii) data modality gap between the inputs of MMLMs and our matching task; (iii) prompt efficiency on large data. Therefore, we firstly propose a novel EM framework (namely, CrossEM) that addresses cross-modal EM as a matching probability problem with specific prompt-tuning. Secondly, two alternative prompt generation methods are designed to extract structural knowledge from heterogeneous data to overcome the data modality gap with pre-trained models. Thirdly, we present an improved matching framework (namely, CrossEM+) to boost the prompt efficiency on large heterogeneous data. Experimental evaluations verify that our methods significantly outperform the state-of-the-art approaches on three benchmarks. Furthermore, our case study highlights the considerable potential of cross-modal EM in improving the performance of downstream tasks, thereby benefitting a wider range of research areas.

Fabric defect detection is a crucial step in the textile manufacturing process and serves as a key factor in ensuring high-quality fabric production. However, fabric defect images exhibit significant diversity, with defects often complex and irregularly distributed. Existing fabric defect detection algorithms face challenges such as low detection accuracy and slow processing speeds. In this paper, we propose a novel fabric defect detection algorithm based on YOLOv11, named YOLO-FEE. To reduce model parameters and computational overhead, we integrate a FasterNet module into YOLOv11. To enhance the model’s ability to represent discriminative features, we design the EMA module, which leverages a multi-scale attention mechanism to effectively capture detailed information at various scales. Furthermore, to improve the model’s stability and detection accuracy, we adopt the EIoU bounding-box loss function, which refines the IoU calculation method to more precisely evaluate the overlap between predicted and ground-truth bounding boxes. Experimental results demonstrate that even without applying quantization or other acceleration techniques, our approach shows significant potential to enhance the speed of inference and overall performance.

With the rapid growth of 3D content, there is an increasing need for intelligent systems that can search for complex 3D shapes using simple natural language queries. However, existing approaches face significant limitations. They rely heavily on manually labeled datasets and use fixed similarity thresholds to determine matches, which restricts their ability to generalize and accurately retrieve novel or diverse 3D shapes. To bridge these gaps, this paper introduces Open3DSearch, the first attempt to address the challenge of open-domain text-to-shape precise retrieval. Our core idea is to transform 3D shapes into semantically representative 2D views, thereby enabling the task to be handled by mature large vision-language models (LVLMs) and allowing for explicit cross-modal matching judgments. To realize this concept, we design a view rendering strategy to mitigate potential information degradation during 3D-to-2D conversion while capturing the maximal amount of query-relevant information. To evaluate Open3DSearch and advance research in this field, we present the Uni3D-R benchmark dataset, designed to simulate precise associations between user queries and 3D shapes in open-domain contexts. Extensive quantitative and qualitative experiments demonstrate that Open3DSearch achieves state-of-the-art results.

Load balancers (LBs) are crucial in cloud environments, ensuring workload scalability. They route packets destined for a service (identified by a virtual IP address, or VIP) to a group of servers designated to deliver that service, each with its direct IP address (DIP). Consequently, LBs significantly impact the performance of cloud services and the experience of tenants. Many academic studies focus on specific issues such as designing new load balancing algorithms and developing hardware load balancing devices to enhance the LB's performance, reliability, and scalability. However, we believe this approach is not ideal for cloud data centers for the following reasons: (i) the increasing demands of users and the variety of cloud service types turn the LB into a bottleneck; and (ii) continually adding machines or upgrading hardware devices can incur substantial costs. In this paper, we propose the Next Generation Load Balancer (NGLB), designed to bypass the TCP connection datapath from the LB, thereby eliminating latency overheads and scalability bottlenecks of traditional cloud LBs. The LB only participates in the TCP connection establishment phase. The three key features of our design are: (i) the introduction of anactive address learningmodel to redirect traffic and bypass the LB, (ii) amulti-tenant isolationmechanism for deployment within multi-tenant Virtual Private Cloud networks, and (iii) a distributed flow control method, known ashierarchical connection cleaner, designed to ensure the availability of backend resources. The evaluation results demonstrate that NGLB reduces latency by 16% and increases nearly 3× throughput. With the same LB resources, NGLB improves 10× rate of new connection establishment. More importantly, five years of operational experience has proven NGLB's stability for high-bandwidth services.

Neural implicit representations have recently shown promising progress in dense Simultaneous Localization And Mapping (SLAM). However, existing works have shortcomings in terms of reconstruction quality and real-time performance, mainly due to inflexible scene representation strategy without leveraging any prior information. In this paper, we introduce SP-SLAM, a novel neural RGB-D SLAM system that performs tracking and mapping in real-time. SP-SLAM computes depth images and establishes sparse voxel-encoded scene priors near the surface reconstruction. Simultaneously, we employ triplanes to store scene appearance information, striking a balance between achieving high-quality geometric texture mapping and minimizing memory consumption. Furthermore, in SP-SLAM, we introduce an effective optimization strategy for mapping, allowing the system to continuously optimize the poses of all historical input frames during runtime without increasing computational overhead. We conduct extensive evaluations on five benchmark datasets (Replica, ScanNet, TUM RGB-D, Synthetic RGB-D, 7-Scenes). The results demonstrate that, compared to existing methods, we achieve superior tracking accuracy and reconstruction quality, while running at a significantly faster speed.

In object goal navigation tasks, the robot’s understanding of semantic relationships in the environment is a key factor in its ability to localize target objects. Previously, learning-based methods trained robots using 3D scene datasets to learn semantic relationships. However, these approaches perform poorly in new environments with unfamiliar semantic contexts. In this paper, we propose ChatNav which leverages the powerful knowledge summarizing and reasoning capabilities of a Large Language Model (LLM) for zero-shot inference of explicit semantic relationships. These relationships are further integrated into the navigation system for efficient localization of target objects. ChatNav employs a spatial object clustering algorithm to collect semantic clues and designs common-sense-based prompts for interacting with LLM. It then uses a gravity-repulsion model to convert inference results into heuristic factors for robust navigation decision-making. Our approach requires no additional training and can consistently obtain accurate semantic relationships from LLM, making it well-suited for navigating unknown environments. Experimental results demonstrate the outstanding navigation performance of our proposed method on the Gibson and HM3D datasets, surpassing the current state-of-the-art object goal navigation methods.

Underwater acoustic target recognition (UATR) can be significantly empowered by advancements in deep learning (DL). However, the effectiveness of DL-based UATR methods is often constrained by the limited computing resources available on underwater platforms. Most of the existing knowledge distillation (KD) strategies try to build lightweight DL models, but these strategies rarely consider the acoustic properties of underwater environments, making them less efficient for UATR tasks. Thus, fully harnessing the potential of DL techniques while ensuring the model’s practicality, is one of the urgent problems to be solved in UATR research. In this work, we introduce the union-domain KD (UDKD) to establish an accurate and lightweight UATR model. UDKD integrates two KD strategies: dual-frequency band distillation (DBD) and cross-domain masked distillation (CMD). DBD improves the learning process for a simple student model by decoupling the knowledge of spectrograms into the local structural (i.e., line spectra) and global composition (i.e., propagation patterns) aspects. CMD reduces redundant information from the Fourier Transform process, enabling the student model to concentrate on essential signal elements and to learn underlying time–frequency distribution. Extensive experiments on two real-world oceanic datasets confirm the superior performance of UDKD compared to existing KD methods, i.e., achieving an accuracy of 94.81% ($\uparrow ~3.19$% versus 91.62%). Notably, UDKD showcases a 10.5% improvement in the prediction accuracy of the lightweight student model.

In Earth observation, cloud severely affects the interpretation of optical satellites generated high-resolution images. Cloud-free optical images are vital for downstream tasks such as semantic segmentation and object detection. Thus, the elimination of clouds from optical imagery has emerged as a significant topic in remote sensing. Currently, most existing methods are proposed to leverage the texture information from auxiliary synthetic aperture radar (SAR) images to restore cloud-free images via direct channel merging. However, such a unified feature extraction approach often neglects the inherent distribution disparity between SAR and optical images—the result of differing imaging principles-potentially leading to significant feature loss. To this end, we introduce a network by jointing SAR and optical images multimodal and cross-attention network (MMCANet) to effectively extract multiscale contextual features from SAR imagery and integrate them with optical features. Specifically, instead of simple concatenation of the channels of SAR and optical images, we obtain high-dimensional features from them through independent feature extractors. The integration of these features is facilitated by a cross-attention mechanism that provides a more fine-grained amalgamation of information. Meanwhile, an atrous spatial pyramid pooling (ASPP) module is introduced into the integration of high-level features, which captures multiscale contextual information around clouded areas. In addition, we propose four advanced remote sensing image restoration algorithms that approach image restoration as a series of subtasks, gradually eliminating clouds to enhance performance. Comprehensive assessments show that MMCANet performs well on the SEN 12 MS-CR dataset with peak signal-to-noise ratio (PSNR) of 39.8871, structural similarity index (SSIM) of 0.9672, mean absolute error (MAE) of 0.0081, and spectral angle mapper (SAM) of 2.9884.

The rapid growth of the Internet of Things (IoT) has led to the widespread adoption of the IoT networks in numerous digital applications. To counter physical threats in these systems, automatic modulation classification (AMC) has emerged as an effective approach for identifying the modulation format of signals in noisy environments. However, identifying those threats can be particularly challenging due to the scarcity of labeled data, which is a common issue in various IoT applications, such as anomaly detection for unmanned aerial vehicles (UAVs) and intrusion detection in the IoT networks. Few-shot learning (FSL) offers a promising solution by enabling models to grasp the concepts of new classes using only a limited number of labeled samples. However, prevalent FSL techniques are primarily tailored for tasks in the computer vision domain and are not suitable for the wireless signal domain. Instead of designing a new FSL model, this work suggests a novel approach that enhances wireless signals to be more efficiently processed by the existing state-of-the-art (SOTA) FSL models. We present the semantic-consistent signal pretransformation (ScSP), a parameterized transformation architecture that ensures signals with identical semantics exhibit similar representations. ScSP is designed to integrate seamlessly with various SOTA FSL models for signal modulation recognition and supports commonly used deep learning backbones. Our evaluation indicates that ScSP boosts the performance of numerous SOTA FSL models, while preserving flexibility.

Given the rapid advancement of the physical industrial internet and the growing significance of military reconnaissance, a large amount of signal data will be generated, because signal transmission and reception are the basis of these scenarios. However, due to problems such as air noise, inconsistent transceiver technology, and hacker interference, a large amount of data labels will be lost. With the aim of solving this problem, we propose a Window-based Contrastive Learning for Signal Feature Reconstruction (WCL_SFR) method. This method divides the feature map into small windows and uses similarity to establish a contrastive learning mode, while incorporating a reconstruction module to improve the stability of the model’s capacity for extraction. On two commonly used signal datasets, WCL_SFR generates the most beneficial results, with an improvement of up to 28.32% compared to other contrastive learning methods. In order to simulate real scenarios, we also conducted cross-dataset migration experiments, which achieved accuracies of 65.36% and 66.17%, respectively, which greatly outperformed similar methods. Therefore, WCL_SFR is an innovative development in the field of unsupervised signal recognition.

The prevalence of online query services in human life has attracted significant interest from the fields of economics and databases in determining appropriate pricing for such services. Simultaneously, the utilization of graph analytics across various domains has resulted in substantial social and economic benefits in recent years. As the adoption of graph analytics continues to expand, there is a corresponding need to establish fair pricing models for the information contributed by each participant in the data ecosystem. However, current query-based pricing frameworks cannot be applied to price graph statistics, as they fail to consider buyers' affordability and prevent arbitrage trading. To address this gap, in this paper, we propose a novel framework GSHOP for pricing graph statistic queries. Instead of pricing a precise answer for a query, our framework offers the flexibility to price a set of answers injected with noise. Based on the framework, data owners initially create and publish extended local views (ELVs) to represent their graph data. Additionally, it allows buyers to tolerate a certain degree of noise added to the answer to reduce their payments. The framework accurately quantifies the relationship between noise and price to ensure that payment and compensation are reasonable for the buyer and owners, respectively. We also propose algorithms specifically designed for fundamental graph statistics, including node degrees and subgraph counts such as k-stars and k-cliques. Furthermore, we formally prove that the pricing framework is arbitrage-free. Extensive experimental results on real-life graph data validate the good performance of the proposed framework and algorithms.

Cross-source query serves as a proxy for scene understanding to support many web applications such as rec-ommendation systems, e-commerce, and e-learning applications. In this paper, we propose SVQA that semantically combines the knowledge from available images and graphs to answer the complex question. To this end, we design a graph-based method to unify various data sources into one representation. We then develop a complex question parse method that utilizes the structure of languages to transform the query into a query graph. A graph query engine that performs the query graph over the unified data source while optimizing the query process. To evaluate the proposed system, we build a vanilla dataset called MVQA and show that the state-of-the-art (SOTA) VQA models fail to perform our task. The comprehensive evaluations show that the proposed SVQA is able to reason implicit relationships over multiple images and external knowledge to correctly answer a complex query. We hope that our first attempt provides researchers with a fresh taste of multimodal data analysis.

The rapid proliferation of Internet of Things (IoT) devices has led to an increased need for robust and efficient intrusion detection systems capable of identifying and mitigating novel threats. Traditional methods often struggle with the scarcity of labeled anomaly data, which is highly consequential, particularly in the context of IoT. In this study, we propose a novel few-shot learning approach by leveraging a Multi-Stage Attention Siamese Network (MASiNet) for network traffic intrusion detection based on meta-learning framework. Unlike traditional methods, the proposed MASiNet model is capable of detecting intrusions with minimal labeled samples, addressing the challenge of scarce anomaly data. The model is trained using various attack samples and evaluates unknown samples by comparing similarities with a small set of known attack types. A well-structured cost function design, incorporating two specific losses, is introduced to optimize the effectiveness of the training process. Tested on the NSL_KDD and UNSW-NB15 datasets in a simulated few-shot learning environment, the MASiNet model demonstrates superior performance in terms of accuracy, precision, False Alarm Rate (FAR), outperforming existing methods. Furthermore, we have validated our approach through real-world evaluations. The proposed method provides an effective solution for intrusion detection in the context of few-shot learning, offering a proficient solution that aligns with the dynamic nature of IoT networks.

ICPS software and hardware suffer from low update frequency, making it easier for insiders to bypass external defenses and launch concealed destructive attacks. To address these concerns, we design a device fingerprinting method based on multi-physical features, augmenting current intrusion detection techniques in the ICPS environment. In this article, we use the sorting system as an example, demonstrating that the proposed device fingerprinting technology has generality in the intrusion detection of ICPS control flow. Specifically, we first formalize the physical model of the sorting system to analyze the critical device features. Then, we extract these physical features from the sensor data collected in a physical testbed. Utilizing featurized data, we train a classifier that generates fingerprints in real-time in the production environment. Moreover, we develop a differential detection model based on device fingerprints to discover stealthy insider attacks efficiently. We evaluate the proposed method in a real-world testbed. Experiment results show that the detecting performance of classifiers approaches 100% when the the number of component types is small.

Geo-distributed web-applications (GWA) can be deployed across multiple geographically separated datacenters to reduce the latency of access for users. Finding a suitable deployment for a GWA is challenging due to the requirement to consider a number of different parameters, such as host configurations across a federated infrastructure. The ability to evaluate multiple deployment configurations enables an efficient outcome to be determined, balancing resource usage while satisfying user requirements. We proposeGeoDeploy, a framework designed for finding a deployment solution for GWA. We evaluateGeoDeployusing both a formal algorithmic model and a practical cloud-based deployment. We also compare our approach with other existing techniques.

The recent flourish of deep learning on various tasks is largely accredited to the rich and high-quality labeled data. Nonetheless, collecting sufficient labeled samples is not very practical for many real applications. Few-shot Learning (FSL) provides a promising solution that allows a model to learn the concept of novel classes with a few labeled samples. However, many existing FSL methods are only designed for computer vision tasks and are not suitable for radio signal recognition. This paper calls for a radically different approach to FSL: in contrast to developing a new FSL model, we should focus on transforming the radio signal to be better processed by the state-of-the-art (SOTA) FSL model. We propose Modulated Signal Pre-transformation (MSP), a parameterized radio signal transformation framework that encourages the signals having the same semantics to have similar representations. MSP currently adapts to various SOTA FSL models for signal modulation recognition and can support the mainstream deep learning backbone. Evaluation results show that MSP improves the performance gains for many SOTA FSL models while maintaining flexibility.

Mode decomposition methods are the current workhorse for the analysis of non-stationary signals. However, current attempts at these methods mainly focus on improving accuracy, leaving computational efficiency untouched. To this end, we leverage the neural mode decomposition technique and propose an open-source Neural Mode Estimation (NME) to deliver a large speedup (at least 50×) while maintaining accuracy. Specifically, we transform the mode decomposition problem into an extremum problem of a functional in the cosine transform domain and train a neural network to approximate the solution. We demonstrate in extensive empirical results that NME can provide an improved trade-off between speed and accuracy, enabling fast, high-quality, stable mode decomposition of non-stationary signals.

We study the problem of object reconstruction in a multi-agent collaboration scenario. Specifically, we focus on the reconstruction of specific goals through several cooperative agents equipped with vision sensors to achieve higher efficiency than single agents. Our main insight is that a complete 3D object can be split into several local 3D models and assigned to different agents. In addition, we can use the salient characteristics of the collaboration agent itself to help realize the integration of local models. We develop a novel pipeline that first restores local 3D models from the images obtained from different agents, then the relative poses between collaborative agents are estimated by aligning intrinsic features. After that, all local models are integrated using the estimated parameters. Extensive experiments show that our proposed method is capable of accurately reconstructing 3D objects in the real world in a multi-agent collaborative manner. The full reconstruction pipeline is released to the public as an open-source project.

There has been a growing interest in cross-source searching to gain rich knowledge in recent years. A data lake collects massive raw and heterogeneous data with different data schemas and query interfaces. Many real-life applications require query answering over the heterogeneous data lake, such as e-commerce, bioinformatics and healthcare. In this paper, we propose LakeAns that semantically integrates heterogeneous data schemas of the lake to enhance the semantics of query answers. To this end, we propose a novel framework to efficiently and effectively perform the cross-source searching. The framework exploits a reinforcement learning method to semantically integrate the data schemas and further create a global relational schema for the heterogeneous data. It then performs a query answering algorithm based on the global schema to find answers across multiple data sources. We conduct extensive experimental evaluations using real-life data to verify that our approach outperforms existing solutions in terms of effectiveness and efficiency.

In Earth observation activities, cloud severely affects the interpretation of the high-resolution imagery, generated by optical satellites. Therefore, removing clouds from optical imagery becomes a topic of interest in the remote sensing field. Currently, most methods use auxiliary Synthetic Aperture Radar (SAR) images to reconstruct optical images by merging SAR and optical images into a deep learning network. However, the speckle noise of the SAR image is not taken into the consideration during feature fusion processing, leading to the blurry edges in the reconstructed optical images. To get fine-grained optical images, we propose a novel cloud removal framework based on the edge fusion of SAR and optical images. Firstly, the edge feature of SAR images is extracted by the GRHED. As the prior knowledge, it can provide fine-grained edge information for subsequent reconstruction work. Then channels from three modal data are stacked to guide the reconstruction of optical images by exploiting their correlations and interactions. Furthermore, a structural similarity (SSIM) loss function is introduced to optimize the training network and improve the coherence of the image structure. Experimental results confirm its advantages on the SEN12MS-CR dataset.

Automatic modulation classification (AMC) is an important technology for the monitoring, management, and control of communication systems. In recent years, machine learning approaches are becoming popular to improve the effectiveness of AMC for radio signals. However, the automatic modulation open-set recognition (AMOSR) scheme that aims to identify the known modulation types and recognize the unknown modulation signals is not well studied. Therefore, in this paper, we propose a novel multi-modal marginal prototype framework for radio frequency (RF) signals (MMPRF) to improve AMOSR performance. First, MMPRF addresses the problem of simultaneous recognition of closed and open sets by partitioning the feature space in the way of one versus other and marginal restrictions. Second, we exploit the wireless signal domain knowledge to extract a series of signal-related features to enhance the AMOSR capability. In addition, we propose a GAN-based unknown sample generation strategy to allow the model to understand the unknown world. Finally, we conduct extensive experiments on several publicly available radio modulation data, and experimental results show that our proposed MMPRF outperforms the state-of-the-art AMOSR methods.

Edge computing has gained momentum in recent years, and can provide more immediate analysis of streaming video data. However, the edge devices often lack the computing capabilities (processing power, memory) to guarantee reasonable performance (e.g., accuracy, latency, throughput) for complex video analytics tasks. To alleviate this critical problem, the prevalent trend is to offload some video analytics tasks from the edge devices to the cloud. However, existing offloading approaches fail to consider the dynamic nature of the video analytical tasks (e.g., varying encoding format for different video content) and are unable to adapt system dynamics (e.g., varying workload between the edge and the cloud). To overcome the limitation of existing approaches, we develop an edge-cloud offloading performance model based on the concept of hierarchical queues. The resource constraints (e.g., computing capacity and network bandwidth) of each edge nodes and dynamic edge-cloud network conditions are used to parameterize the performance model. Since finding optimal solutions for the performance model is NP-hard, we develop a two-stage gradient-based algorithm and compare it with some state-of-the-art (SOTA) solutions (e.g., FastVA, DeepDecision, Hill Climbing). Experiments have shown our performance model's advantages and the stability of the proposed offloading approach given different systems (edge-cloud) and video analytics application dynamics.

Blockchain, a distributed and shared ledger, provides a credible and transparent solution to increase application auditability by querying the immutable records written in the ledger. Unfortunately, existing query APIs offered by the blockchain are inflexible and unscalable. Some studies propose off-chain solutions to provide more flexible and scalable query services. However, the query service providers (SPs) may deliver fake results without executing the real computation tasks and collude to cheat users. In this article, we propose a novel intelligent blockchain analytics platform termediQuery, in which we design a game theory based smart contract to ensure the trustworthiness of the query results at a reasonable monetary cost. Furthermore, the contract introduces the second opinion game that employs a randomized SP selection approach coupled with non-ordered asynchronous querying primitive to prevent collusion. We achieve a fixed price equilibrium, destroy the economic foundation of collusion, and can incentivize all rational SPs to act diligently with proper financial rewards. In particular,iQuerycan flexibly support semantic and analytical queries for generic consortium or public blockchains, achieving query scalability to massive blockchain data. Extensive experimental evaluations show thatiQueryis significantly faster than state-of-the-art systems. Specifically, in terms of the conditional, analytical, and multi-origin query semantics,iQueryis 2 ×, 7 ×, and 1.5 × faster than advanced blockchain and blockchain databases. Meanwhile, to guarantee 100% trustworthiness, only two copies of query results need to be verified iniQuery, whileiQuery's latency is$2 \sim 134$× smaller than the state-of-the-art systems.

In recent years, a surging number of unmanned aerial vehicles (UAVs) are pervasively utilized in many areas. However, the increasing number of UAVs may cause privacy and security issues such as voyeurism and espionage. It is critical for individuals or organizations to manage their behaviors and proactively prevent the misbehaved invasion of unauthorized UAVs through effective anomaly detection. The UAV anomaly detection framework needs to cope with complex signals in the noisy-prone environments and to function with very limited labeled samples. This paper proposesBisSiam, a novel framework that is capable of identifying UAV presence, types and operation modes.BisSiamconverts UAVs signals to bispectrum as the input and exploits a siamese network based contrastive learning model to learn the vector encoding. A sampling mechanism is proposed for optimizing the sample size involved in the model training whilst ensuring the model accuracy without compromising the training efficiency. Finally, we present a similarity-based fingerprint matching mechanism for detecting unseen UAVs without the need of retraining the whole model. Experiment results show that our approach outperforms other baselines and can reach 92.85% accuracy of UAV type detection in unsupervised learning scenarios. 91.4% accuracy can be achieved whenBisSiamis used for detecting the UAV type of the out-of-sample UAVs.

This article focuses on a simple, yet fundamental question of distributed edge computing: “how to handle IoT traffic with different levels of sensitivity and criticality by satisfying the application-specific latency constraints?” This question arises in the practical deployment of edge computing, where user data can arrive at a much faster rate than that they can be processed by an edge node. Addressing this question is critical for meeting the latency requirement for latency-sensitive applications, but existing approaches are inadequate to the problem. We presentJanus, a multi-level traffic scheduling system for managing multiple data streams with various degrees of latency constraints. At the edge node level,Janususes multi-level queues to manage data streams with different latency constraints. It then allocates the output bandwidth of the edge node according to the requirements of applications in different priority queues, aiming to reduce the queuing and processing delay of latency-sensitive streams while maximizing the edge-node throughput. At the network level,Janusactively redirects incoming data streams to the less-loaded ones to achieve better network-wide load balance and improve the overall throughput. Experiments show thatJanusreduces the latency to only 16.6% of a non-priority based solution and improves the throughput by 1.7x of a state-of-the-art priority-aware data stream scheduling approach.

Data is increasingly being bought and sold online, and data market platforms have emerged to facilitate these activities. However, current mechanisms for pricing data mainly focus on traditional relational data. In this paper, we propose a framework GQP for pricing graph data on the data market platform. Specifically, given a set of graph price points and a graph query, we can efficiently compute the price of the query based on the graph price points. We first identify an important property (called arbitrage-free) GQP should satisfy with, such that GQP can effectively price the graph query. We then study the exact pricing problem (NP-completeness) and develop an efficient approximation algorithm to solve the problem. We also study the approximate pricing when the query cannot be answered by price points exactly. Furthermore, to avoid the expensive computing cost of updating graph price points, we study the dynamic query pricing and propose novel solutions to reuse the computed graph price points to reduce the computational complexity. Finally, we use real-life data and synthetic data to experimentally verify that the proposed algorithms are able to effectively and efficiently price large graph data based on the framework GQP.

A data lake is a repository for massive raw and heterogeneous data, which includes multiple data models with different data schemas and query interfaces. Keyword search can extract valuable information for users without the knowledge of underlying schemas and query languages. However, conventional keyword searches are restricted to a certain data model and cannot easily adapt to a data lake. In this paper, we study a novel keyword search. To achieve high accuracy and efficiency, we introduce canonical graphs and then integrate semantically related vertices based on vertex representations. A matching entity based keyword search algorithm is presented to find answers across multiple data sources. Finally, extensive experimental study shows the effectiveness and efficiency of our solution.

Real-time attitude estimation of on-orbit spacecraft is a core task in various space applications. Most of the existing methods are based on long-term observation by high-resolution sensors, such as space-borne cameras and ground-based radars. However, when the observation period is limited, it is difficult to obtain target instantaneous attitude information by these methods. To achieve instantaneous attitude estimation from a single camera image, a U-Linked network (ULNet) is proposed in this work. The prior structural constraints of key points are used to reflect the relationship between three-dimensional (3D) target attitude parameters and two-dimensional (2D) images. In this way, target attitude estimation can be solved through the feature point regression when the large-perspective image dataset can be built. The simulation results confirm the feasibility of the proposed method. Besides, the estimation performance of the proposed method also is investigated under different imaging observation conditions.

Big data processing systems, such as Hadoop and Spark, usually work in large-scale, highly-concurrent, and multi-tenant environments that can easily cause hardware and software malfunctions or failures, thereby leading to performance degradation. Several systems and methods exist to detect big data processing systems’ performance degradation, perform root-cause analysis, and even overcome the issues causing such degradation. However, these solutions focus on specific problems such as stragglers and inefficient resource utilization. There is a lack of a generic and extensible framework to support the real-time diagnosis of big data systems. In this article, we propose, develop and validate AutoDiagn. This generic and flexible framework provides holistic monitoring of a big data system while detecting performance degradation and enabling root-cause analysis. We present an implementation and evaluation of AutoDiagn that interacts with a Hadoop cluster deployed on a public cloud and tested with real-world benchmark applications. Experimental results show that AutoDiagn can offer a high accuracy root-cause analysis framework, at the same time as offering a small resource footprint, high throughput, and low latency.

Electronic Stability Program (ESP) is widely used in modern vehicles. Its safety and stability largely depend on the strength and reliability of the MEMS gyroscope. However, the tight coupling between this sensor and the environment brings significant safety hazards to the vehicle. In this study, we describe the physical vulnerability of gyroscopes to high-frequency acoustics and introduce methods for finding resonant frequencies. We devised two methods to inject the attack signal into audio files to make the acoustic attack more stealthy. The realized attack is non-intrusive and does not require tampering with the ESP hardware device, making attack detection more difficult. We also consider a neural network-based defense strategy and verify its effectiveness. The construction of the vehicle simulation system and the above experiments are completed in the co-simulation environment of Carsim and Simulink.

Many real-life applications require processing graph data across heterogeneous sources. In this paper, we define the graph federation that indicates that the graph data sources are temporarily federated and offer their data for users. Next, we propose a new framework FedGraph to efficiently and effectively perform subgraph matching, which is a crucial application in graph federation. FedGraph consists of three phases, including query decomposition, distributed matching, and distributed joining. We also develop new efficient approximation algorithms and apply them in each phase to attack the NP-hard problem. The evaluations are conducted in a real test bed using both real-life and synthetic graph datasets. FedGraph outperforms the state-of-the-art methods, reducing the execution time and communication cost by 37.3 × and 61.8 ×, respectively.

Industry 4.0 have automated the entire manufacturing sector (including technologies and processes) by adopting Internet of Things and cloud computing. To handle the workflows from Industrial Cyber-Physical systems, more and more data centers have been built across the globe to serve the growing needs of computing and storage. This has led to an enormous increase in energy usage by cloud data centers, which is not only a financial burden but also increases their carbon footprint. The private software defined wide area network (SDWAN) connects a cloud provider's data centers across the planet. This gives the opportunity to develop new scheduling strategies to manage cloud providers workload in a more energy-efficient manner. In this context, this article addresses the problem of scheduling data-driven industrial workflow applications over a set of private SDWAN connected data centers in an energy-efficient manner while managing tradeoff of a cloud provider' revenue. Our proposed algorithm aims to minimize the cloud provider's revenue and the usage of nonrenewable energy by utilizing the real-world electricity prices with the availability of green energy on different cloud data centers, where the energy consumption consists of the usage of running application over multiple data centers and transferring the data among them through SDWAN. The evaluation shows that our proposed method can increase usage of green energy for the execution of industrial workflow up to 3× times with a slight increase in the cost when compared to cost-based workflow scheduling methods.

We demonstrate AHOM, a system that can Actively Observe Hazards via Monitoring Social Media Streams. AHOM proposes an active way to include the human in the loop of hazard information ac-quisition for social media. Different from state of the art, it supports bi-directional interaction between social media data processing system and social media users, which leads to the establishment of deeper and more accurate situational awareness of hazard events. We demonstrate how AHOM utilizes Twitter streams and bi-directional information exchange with social media users for enhanced hazard observation.

Graph queries have emerged as one of the fundamental techniques to support modern search services, such as PageRank web search, social networking search and knowledge graph search. As such graphs are maintained globally and very huge (e.g., billions of nodes), we need to efficiently process graph queries across multiple geographically distributed datacenters, running geo-distributed graph queries. Existing graph computing frameworks may not work well for geographically distributed datacenters, because they implement a Bulk Synchronous Parallel model that requires excessive inter-datacenter transfers, thereby introducing extremely large latency for query processing. In this paper, we propose GeoGraph --a universal framework to support efficient geo-distributed graph query processing based on clustering datacenters and meta-graph, while reducing the inter-datacenter communication. Our new framework can be applied to many types of graph algorithms without any modification. The framework is developed on the top of Apache Giraph. The experiments were conducted by applying four important graph queries, i.e., shortest path, graph keyword search, subgraph isomorphism and PageRank. The evaluation results show that our proposed framework can achieve up to 82% faster convergence, 42% lower WAN bandwidth usage, and 45% less total monetary cost for the four graph queries, with input graphs stored across ten geo-distributed datacenters.

SDN has provided significant convenience for network providers and operators in cloud computing. Such a great advantage is extending to the Internet of Things network. However, it also increases the risk if the security of an SDN network is compromised. For example, if the network operator's permission is illegally obtained by a hacker, he/she can control the entry of the SDN network. Therefore, an effective authentication scheme is needed to fit various application scenarios with high-security requirements. In this article, we design, implement, and evaluate a new authentication scheme called the hidden pattern (THP), which combines graphics password and digital challenge value to prevent multiple types of authentication attacks at the same time. We examined THP in the perspectives of both security and usability, with a total number of 694 participants in 63 days. Our evaluation shows that THP can provide better performance than the existing schemes in terms of security and usability.

In software-defined networks (SDNs), the deployment of multiple controllers improves the reliability and scalability of the distributed control plane. Recently, edge computing (EC) has become a backbone to networks where computational infrastructures and services are getting closer to the end user. The unique characteristics of SDN can serve as a key enabler to lower the complexity barriers involved in EC, and provide better quality-of-services (QoS) to users. As the demand for IoT keeps growing, gradually a huge number of smart devices will be connected to EC and generate tremendous IoT traffic. Due to a huge volume of control messages, the controller may not have sufficient capacity to respond to them. To handle such a scenario and to achieve better load balancing, dynamic switch migrating is one effective approach. However, a deliberate mechanism is required to accomplish such a task on the control plane, and the migration process results in high network delay. Taking it into consideration, this article has introduced an efficient switch migration-based load balancing (ESMLB) framework, which aims to assign switches to an underutilized controller effectively. Among many alternatives for selecting a target controller, a multicriteria decision-making method, i.e., the technique for order preference by similarity to an ideal solution (TOPSIS), has been used in our framework. This framework enables flexible decision-making processes for selecting controllers having different resource attributes. The emulation results indicate the efficacy of the ESMLB.

Several real-world problems in domain of healthcare, large scale scientific simulations, and manufacturing are organised as workflow applications. Efficiently managing workflow applications on the Cloud computing data-centres is challenging due to the following problems: (i) they need to perform computation over sensitive data (e.g., Healthcare workflows) hence leading to additional security and legal risks especially considering public cloud environments and (ii) the dynamism of the cloud environment can lead to several run-time problems such as data loss and abnormal termination of workflow task due to failures of computing, storage, and network services. To tackle above challenges, this paper proposes a novel workflow management framework call Deploy on Federated Cloud Framework (DoFCF) that can dynamically partition scientific workflows across federated cloud (public/private) data-centres for minimising the financial cost, adhering to security requirements, while gracefully handling run-time failures. The framework is validated in cloud simulation tool (CloudSim) as well as in a realistic workflow-based cloud platform (e-Science Central). The results showed that our approach is practical and is successful in meeting users security requirements and reduces overall cost, and dynamically adapts to the run-time failures.

The Internet of Things (IoT) is becoming a backbone of sensing infrastructure to several mission-critical applications such as smart health, disaster management, and smart cities. Due to resource-constrained sensing devices, IoT infrastructures use Edge datacenters (EDCs) for real-time data processing. EDCs can be either static or mobile in nature, and this article considers both of these scenarios. Generally, EDCs communicate with IoT devices in emergency scenarios to evaluate data in real-time. Protecting data communications from malicious activity becomes a key factor, as all the communication flows through insecure channels. In such infrastructures, it is a challenging task for EDCs to ensure the trustworthiness of the data for emergency evaluations. The current communication security pattern of “communication before authentication” leaves a “black hole” for intruders to become part of communication processes without authentication. To overcome this issue and to develop security infrastructures for IoT and distributed Edge datacenters, this article proposes a user-centric security solution. The proposed security solution shifts from a network-centric approach to a user-centric security approach by authenticating users and devices before communication is established. A trusted controller is initialized to authenticate and establishes the secure channel between the devices before they start communication between themselves. The centralized controller draws a perimeter for secure communications within the boundary. Theoretical analysis and experimental evaluation of the proposed security model show that it not only secures the communication infrastructure but also improves the overall network performance.

The growth of Internet of Things (IoT) technology brings many new opportunities for applications in areas including smart healthcare, smart buildings, and smart agriculture. These applications must normally distribute the computations, required for extracting value from sensor data, over the IoT infrastructure platforms (e.g., sensors, phones, field-gateways, and clouds). This can be very challenging for IoT application developers due to the heterogeneity of the aforementioned platforms, potentially conflicting nonfunctional requirements (e.g., battery power, latency, and cost), and related deployment criteria, which is impossible to resolve manually. To address the above challenges, we have developed the PATH2iot framework that decomposes a complex IoT application into self-contained micro-operations. Based on the deployment criteria, PATH2iot automatically distributes the set of micro-operations across IoT infrastructure platforms, while respecting their run-time data and control flow dependencies. In our previous work, we have shown how to use the PATH2iot to optimize the battery life of a healthcare wearable. In this article, we describe a new research that significantly extends PATH2iot, which introduces a heuristic model capable of making optimal deployment decisions based on multiple conflicting nonfunctional requirements and selection criteria (user preferences). It does so by leveraging a well-known multicriteria decision-making method called the analytic hierarchical processes (AHP). The applicability of the deployment model is validated based on a real-world digital healthcare analytics use case. The results show that our model is able to find the optimal deployment solution for different user preferences.

Recent advances in composing Cloud applications have been driven by deployments of inter-networking heterogeneous microservices across multiple Cloud datacenters. System dependability has been of the upmost importance and criticality to both service vendors and customers. Security, a measurable attribute, is increasingly regarded as the representative example of dependability. Literally, with the increment of microservice types and dynamicity, applications are exposed to aggravated internal security threats and externally environmental uncertainties. Existing work mainly focuses on the QoS-aware composition of native VM-based Cloud application components, while ignoring uncertainties and security risks among interactive and interdependent container-based microservices. Still, orchestrating a set of microservices across datacenters under those constraints remains computationally intractable. This paper describes a new dependable microservice orchestration framework GA-Par to effectively select and deploy microservices whilst reducing the discrepancy between user security requirements and actual service provision. We adopt a hybrid (both whitebox and blackbox based) approach to measure the satisfaction of security requirement and the environmental impact of network QoS on system dependability. Due to the exponential grow of solution space, we develop a parallel Genetic Algorithm framework based on Spark to accelerate the operations for calculating the optimal or near-optimal solution. Large-scale real world datasets are utilized to validate models and orchestration approach. Experiments show that our solution outperforms the greedy-based security aware method with 42.34 percent improvement. GA-Par is roughly 4× faster than a Hadoop-based genetic algorithm solver and the effectiveness can be constantly guaranteed under different application scales.

It is a long-standing challenge to achieve a high degree of resource utilization in cluster scheduling. Resource oversubscription has become a common practice in improving resource utilization and cost reduction. However, current centralized approaches to oversubscription suffer from the issue with resource mismatch and fail to take into account other performance requirements, e.g., tail latency. In this article we present ROSE, a new resource management platform capable of conducting performance-aware resource oversubscription. ROSE allows latency-sensitive long-running applications (LRAs) to co-exist with computation-intensive batch jobs. Instead of waiting for resource allocation to be confirmed by the centralized scheduler, job managers in ROSE can independently request to launch speculative tasks within specific machines according to their suitability for oversubscription. Node agents of those machines can however, avoid any excessive resource oversubscription by means of a mechanism for admission control using multi-resource threshold control and performance-aware resource throttle. Experiments show that in case of mixed co-location of batch jobs and latency-sensitive LRAs, the CPU utilization and the disk utilization can reach 56.34 and 43.49 percent, respectively, but the 95th percentile of read latency in YCSB workloads only increases by 5.4 percent against the case of executing the LRAs alone.

Modern big data processing systems are becoming very complex in terms of large-scale, high-concurrency and multiple talents. Thus, many failures and performance reductions only happen at run-time and are very difficult to capture. Moreover, some issues may only be triggered when some components are executed. To analyze the root cause of these types of issues, we have to capture the dependencies of each component in real-time. In this paper, we propose SmartMonit, a real-time big data monitoring system, which collects infrastructure information such as the process status of each task. At the same time, we develop a real-time stream processing framework to analyze the coordination among the tasks and the infrastructures. This coordination information is essential for troubleshooting the reasons for failures and performance reduction, especially the ones propagated from other causes.

Containerized web-applications have gained popularity recently due to the advantages provided by the containers including light-weight, packaged, fast start up and shut down and easy scalability. As there are more than 267 cloud providers, finding a flexible deployment option for containerized web-applications is very difficult as each cloud offers numerous deployment infrastructure. Benchmarking is one of the eminent options to evaluate the provisioned resources before product-level deployment. However, benchmarking the massive infrastructure resources provisioned by various cloud providers is a time consuming, tedious and costly process and is not practical to accomplish manually.

IoT-enabled devices continue to generate a massive amount of data. Transforming this continuously arriving raw data into timely insights is critical for many modern online services. For such settings, the traditional form of data analytics over the entire dataset would be prohibitively limiting and expensive for supporting real-time stream analytics. In this work, we make a case for approximate computing for data analytics in IoT settings. Approximate computing aims for efficient execution of workflows where an approximate output is sufficient instead of the exact output. The idea behind approximate computing is to compute over a representative sample instead of the entire input dataset. Thus, approximate computing- based on the chosen sample size - can make a systematic tradeoff between the output accuracy and computation efficiency. This motivated the design of APPROXIOT- a data analytics system for approximate computing in IoT. To realize this idea, we designed an online hierarchical stratified reservoir sampling algorithm that uses edge computing resources to produce approximate output with rigorous error bounds. To showcase the effectiveness of our algorithm, we implemented APPROXIOT based on Apache Kafka and evaluated its effectiveness using a set of microbenchmarks and real-world case studies. Our results show that APPROXIOT achieves a speedup 1:3×-9:9× with varying sampling fraction of 80% to 10% compared to simple random sampling.

Software defined networking (SDN) is a promising network architecture, which decouples the control plane and data plane of a network. However, SDN opens some security challenges, such as man-in-the-middle attacks, spoofing attacks, flooding attacks and so on. In this paper, we focus on flooding attacks which consume the switch buffer and controller resource resulting in SDN framework resource overloaded. To prevent SDN framework from flooding attack, we present a defense approach called PBUF (Packet forwarding based on BUFfer sharing), which pools the idle switches to mitigate threat issues. This approach consists of buffer management and packet forwarding modules. The buffer management module gleans the statistics of incoming packets and then analyzes these statistics to estimate the buffer size by network calculus. Considering that a lot of table-miss packets will be generated and stored in buffer when the flooding attack is happening, the packet forwarding module is designed to forward these table-miss packets to idle switches to prevent the switch or controller to be overloaded. These table-miss packets will be buffered in idle switches and then sent to controller in a limited rate by generating packet_in messages. The simulation results show that PBUF is effective and only introduces a little overhead in SDN framework.

The significant growth in cloud computing has led to increasing number of cloud providers, each offering their service under different conditions - one might be more secure whilst another might be less expensive or more reliable. At the same time user applications have become more and more complex. Often, they consist of a diverse collection of software components, and need to handle variable workloads, which poses different requirements on the infrastructure. Therefore, many organisations are considering using a combination of different clouds to satisfy these needs. It raises, however, a non-trivial issue of how to select the best combination of clouds to meet the application requirements. This paper presents a novel algorithm to deploy workflow applications on federated clouds. First, we introduce an entropy-based method to quantify the most reliable workflow deployments. Second, we apply an extension of the Bell-LaPadula Multi-Level security model to address application security requirements. Finally, we optimise deployment in terms of its entropy and also its monetary cost, taking into account the cost of computing power, data storage and inter-cloud communication. We implemented our new approach and compared it against two existing scheduling algorithms: Extended Dynamic Constraint Algorithm (EDCA) and Extended Biobjective dynamic level scheduling (EBDLS). We show that our algorithm can find deployments that are of equivalent reliability but are less expensive and meet security requirements. We have validated our solution through a set of realistic scientific workflows, using well-known cloud simulation tools (WorkflowSim and DynamicCloudSim) and a realistic cloud based data analysis system (e-Science Central).

In the practical crowdsourcing systems, there exist many cooperative tasks, each of which requires a group of users to perform together, such as finding the shortest multi-hop path or obtaining the media resources from a set of hosts. In this paper, we tackle the problem of how to truthfully and fairly schedule or allocate sufficient users who join mobile crowd-sourcing applications with their smartphones. Moreover, the cooperation among users is taken into account. Thus, we present a novel Cooperative Crowdsourcing (C2) auction mechanism for crowdsourcing multiple cooperative tasks. C2 contains two parts: user selection and payment computation. In the first part, we first prove that users selection with the minimum social cost is NP hard problem and design a greedy algorithm to achieve near-optimal solution in polynomial time. The other part is that the server determines the payments of selected users to avoid the bidder's cheating behavior through a pricing algorithm that if and only if users honestly bid their cost, they can obtain the maximum utility. Both theoretical analysis and extensive simulations demonstrate that C2 auction achieves not only truthfulness, individual rationality and high computational efficiency, but also low overpayment ratio.

The federation of clouds can provide benefits for cloud-based applications. Different clouds have different advantages - one might be more reliable whilst another might be more secure or less expensive. However, being able to select the best combination of clouds to meet the application requirements is not trivial. This paper presents a novel algorithm to deploy workflow applications on federated clouds. Firstly, we introduce an entropy-based method to quantify the most reliable workflow deployments. Secondly, we apply an extension of the Bell-LaPadula Multi-Level security model to meet application security requirements. Finally, we optimise deployment in terms of its entropy and also its monetary cost, taking into account the price of computing power, data storage and inter-cloud communication. To evaluate the new algorithm we compared it against two existing scheduling algorithms: Dynamic Constraint Algorithm (DCA) and Biobjective dynamic level scheduling (BDLS). We show that our algorithm can find deployments that are of equivalent reliability, but are less expensive and also meet security requirements. We have validated our solution using workflows implemented in the e-Science Central cloud-based data analysis system.

The significant increase in the use of cloud computing, has led to an interest in partitioning applications over a set of public and private clouds in order to meet a range of non-functional requirements including performance (for example where private cloud resources alone are insufficient), dependability (e.g. To allow the application to continue to operate even if one cloud fails) and security (for example to ensure that sensitive data is restricted to sufficiently secure clouds and networks). This paper describes a novel deployment planning algorithm to partition complex workflow-based applications over federated clouds, while meeting security requirements. The security issues are based on our previous work which extends the Bell-La Padula model to encompass cloud computing. Selecting the cheapest option for partitioning a workflow over a set of resources has been shown to be an NP-hard problem, which can take impractically long for partitioning large workflows over multiple clouds. We therefore introduce a novel adaptive partitioning algorithm to handle these large workflow applications, which significantly reduces the time required to choose a sufficiently good partitioning option. This is based on generating an initial partitioning, and then adapting it to see if a better solution can be found by bringing together on the same node services with significant communication costs. The algorithm has been implemented and evaluated by using both randomly generated and real world scientific workflows. The experiment results show that our algorithm is thousand times quicker than the exhaustive algorithm presented in our previous work. Yet, on average it generates only 25% more costly solutions. We also compared this algorithm with two other methods commonly used to partition workflows over a set of clouds.

The aim of federated cloud computing is to allow applications to utilise a set of clouds in order to provide a better combination of properties, such as cost, security, performance and dependability, than can be achieved on a single cloud. In this paper we focus on security and dependability: introducing a new automatic method for dynamically partitioning applications across the set of clouds in an environment in which clouds can fail during workflow execution. The method deals with exceptions that occur when clouds fail, and selects the best way to repartition the workflow, whilst still meeting security requirements. This avoids the need for developers to have to code ad-hoc solutions to address cloud failure, or the alternative of simply accepting that an application will fail when a cloud fails. This paper's method builds on earlier work [1] on partitioning workflows over federated clouds to minimise cost while meeting security requirements. It extends it by pre-generating the graph of all possible ways to partition the workflow, and adding weights to the paths through the graph so that when a cloud fails, it is possible to quickly determine the cheapest possible way to make progress from that point to the completion of the workflow execution (if any path exists). The method has been implemented and evaluated through a tool which exploits e-Science Central: a portable, high-level cloud platform. The workflow application is created and distributed across a set of e-Science Central instances. By monitoring the state of each executing e-Science Central instance, the system handles exceptions as they occur at run-time. The paper describes the method and an evaluation that utilises a set of examples.