Vision-Language Models (VLMs) have achieved impressive performance across various tasks, but often struggle to apply newly introduced visual concepts during inference. A common failure pattern is what we call Mixing Things Up: VLMs frequently confuse concept names, resulting in vague descriptions and failure to ground the concept correctly. Existing approaches mainly address person-related concepts through text prompts or tokenizer modifications. However, VLMs still miss or misinterpret untrained visual concepts, underscoring the need to learn new concepts directly from visual input, without relying on prior textual injection. To overcome these limitations, we propose BISCUIT (Basis-aligned Inference through Structured Concept Unification and Identification-aware Tuning), a two-step training method. Step I proposes a dual-stream structure-aware vision encoder that fuses RGB and edge-based embeddings within a shared basis space to enhance concept recognition. Step II enhances generation quality through identification-aware tuning, which encourages alignment between the generated text and the newly introduced visual concepts. Existing methods mainly focus on person concepts and lack comprehensive evaluation across diverse visual categories. We further propose a benchmark BiscuitVQA to evaluate VLMs performance on recognizing and applying novel image-introduced concepts across diverse concept types and task types, including real people, cartoons, animals, and symbolic content. We apply BISCUIT to LLaVA-1.5 and Qwen2.5-VL, achieving competitive results among open-source models and narrowing the gap to Gemini-2.5 and GPT-4o. Interestingly, our BISCUIT maintains strong generalization, showing minimal degradation on other downstream tasks.

Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs.However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances.This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model.Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge.To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance.By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates.Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit.Our code is available at: https://github.com/yangfanww/hiedit.

Deep neural networks (DNNs) deployed on edge devices often suffer from severe performance degradation when exposed to dynamic and continually shifting environments. Test-time adaptation (TTA) has emerged as a promising solution by updating models online with incoming test data. However, edge deployment poses unique challenges: limited computational resources, latency caused by adaptation delays, and knowledge isolation across devices. The situation becomes even more complex in open-world scenarios, where the presence of unknown categories further disrupts adaptation. To overcome these limitations, we propose TeamTTA, a cloud-integrated framework designed for efficient multi-device collaboration open-set test-time adaptation. Specifically, TeamTTA aggregates reliable samples from multiple edge devices through crowdsourcing, uploads them to the cloud, and maintains a memory buffer for continual adaptation. A large vision model (LVM) in the cloud leverages its zero-shot generalization ability to filter out open-set samples and acts as a teacher model, distilling its knowledge into a replicated student edge model stored in the cloud. The adapted model parameters, or alternatively global statistics under poor network conditions, are then transmitted back to the edge devices for efficient inference. Extensive experiments on standard public TTA benchmarks, including corrupted and open-set datasets, show that TeamTTA achieves superior adaptation accuracy, robustness to distribution shifts, and communication efficiency, outperforming state-of-the-art TTA baselines. These results validate the effectiveness of integrating cloud-edge collaboration and LVM-driven knowledge distillation for real-world edge intelligence.

Industrial Cyber-Physical Systems (ICPS) are increasingly vulnerable to sophisticated attacks and operational disturbances that induce subtle and hard-to-detect anomalies, particularly in industrial edge environments. Existing anomaly detection methods often rely on sufficient labeled data and involve excessive computational overhead, hindering real-time detection and lightweight deployment. To address these challenges, we propose a Multimodal Feature fusion-enhanced time series Anomaly Detection framework (MFAD) in ICPS. MFAD enhances the representation of subtle anomalies by jointly modeling temporal dynamics and industrial characteristics through a unified multimodal feature fusion mechanism. Moreover, MFAD adopts a three-stage detection strategy with adaptive thresholding, which further improves robustness under varying operating conditions, while its lightweight overall architecture supports edge deployment. In addition, we provide the Industrial Gas Cyber-Physical System (IGCPS) dataset collected from real-world industrial operations. Experiments on ICPS benchmark datasets of varying scales, including IGCPS, PUMP, WADI, and SWaT, demonstrate that MFAD achieves an F1 score exceeding 96.7% with efficient resource utilization, validating its effectiveness for real-time detection and lightweight deployment in resource-constrained industrial edge environments. Note to Practitioners—This paper is motivated by the increasing need for reliable and efficient anomaly detection in Industrial Cyber-Physical Systems (ICPS), particularly deployed in resource-constrained industrial edge environments. Existing approaches often treat temporal and industrial features separately, rely on sufficient labeled data, and require substantial computational resources, which limits their applicability in real-world industrial settings. In contrast, the proposed MFAD provides a lightweight and practical solution that integrates multimodal feature fusion with robust semi-supervised detection mechanisms to effectively capture subtle anomalies in time series industrial data. The framework is designed with deployment feasibility that it offers strong detection accuracy, low latency, and efficient resource consumption suitable for industrial edge devices. The methods presented here can inform practitioners seeking to enhance the reliability and real-time performance of ICPS anomaly detection systems. Future extensions may focus on expanding MFAD for broader online industrial applications, integrating it with more edge platforms, and enabling large-scale distributed deployment.

In industrial scenarios, data augmentation is an effective approach to improve model performance. However, its benefits are not unidirectionally beneficial. There is no theoretical research or established estimation for the optimal sample size (OSS) in augmentation, nor is there an established metric to evaluate the accuracy of OSS or its deviation from the ground truth. To address these issues, we propose an information-theoretic optimal sample size estimation (IT-OSE) to provide reliable OSS estimation for industrial data augmentation. An interval coverage and deviation (ICD) score is proposed to evaluate the estimated OSS intuitively. The relationship between OSS and dominant factors is theoretically analyzed and formulated, thereby enhancing the interpretability. Experiments show that, compared to empirical estimation, the IT-OSE increases accuracy in classification tasks across baseline models by an average of 4.38%, and reduces mean absolute percentage error (MAPE) in regression tasks across baseline models by an average of 18.80% . The improvements in downstream model performance are more stable.$\mathbf {ICD_{\mathbf {\text{dev}}}}$in the ICD score is also reduced by an average of 49.30% . The determinism of OSS is enhanced. Compared to exhaustive search, the IT-OSE achieves the same OSS while reducing computational and data costs by an average of 83.97% and 93.46% . Furthermore, practicality experiments demonstrate that the IT-OSE exhibits generality across representative sensorbased industrial scenarios.

Deep-neural-network-based autonomous driving perception systems in the Industrial Internet of Things remain vulnerable to adversarial attacks despite their critical role in Industry 4.0. While numerous studies have investigated adversarial attacks on individual perception tasks such as monocular depth estimation or object detection in static situations, real-world complex industrial scenarios introduce two understudied challenges: 1) dynamic scenarios with moving objects and changing viewpoints and 2) simultaneous attacks across multiple perception tasks. In this article, we make three key contributions to address these challenges. First, we introduce a systematic study of environment-aware adversarial attacks in dynamic scenarios by introducing three adversarial attack tasks: object away attack, object close attack, and object creation attack. Second, we propose a novel dynamic scene-oriented adversarial road patch generation framework that accounts for real-world environmental variations. Third, we develop a comprehensive technical framework featuring: improved adversarial loss functions, a dynamic optimization architecture, and an integrated approach combining expectation over transformation with advanced physical augmentation optimization. Extensive experimental results demonstrate the robustness of our proposed method in digital, simulation, and real-world physical domains, as well as under different weather conditions.

In remote sensing image pansharpening, the fundamental objective is to generate high-resolution multispectral (HRMS) image that preserves spectral integrity while enhancing spatial resolution. However, contemporary approaches lack the ability to perceive the quality of input images, thereby failing to preserve desired performance in complex real-world scenarios characterized by panchromatic (PAN) image degradation (e.g., noise contamination or sensor limitations). To address this problem, we present a novel degradation-aware Taylor expansion approximation (DTEA) network for pansharpening, where DTEA includes the following key procedures: Firstly, the PAN image is hierarchically decomposed into feature maps to represent the degradation information through a proposed Taylor expansion approximation network (TEANet). Next, a multi-level information fusion network (MIFNet) is employed to integrate these feature maps with LRMS images, yielding fused maps with rich spatial and spectral information. Finally, the fused map from each layer is utilized to synthesize the desired HRMS image through inverse Taylor expansion, thereby overcoming diverse information degradation. To rigorously evaluate the effectiveness of our DTEA network, we conduct a systematic performance analysis across PAN images with diverse qualities. Extensive quantitative and qualitative experiments on three datasets demonstrate that our method outperforms state-of-the-art approaches while exhibiting excellent generalization capability in real-world scenarios. Source code will be made publicly available on https://github.com/MysterYxby/DTEA.

Reasoning segmentation (RS) interprets implicit textual instructions to accurately segment target regions. This reasoning capability transforms ambiguous non-expert queries into precise pixel-level masks, thereby enabling downstream tasks like area measurement and density analysis with a level of precision unattainable by detection methods. However, existing RS models are not tailored for agriculture and lack domain-specific knowledge, which poses challenges in handling similar pest appearances and small target scales. To bridge this gap, we introduce a fine-grained pest RS task with two subtasks: Pest Discriminative Referring Expression Segmentation (PDRES) and Pest Exclusion Reasoning Segmentation (PERS). Based on this, we propose PestScope, which integrates vision, language, and reasoning for fine-grained pest segmentation. To tackle the exclusion of small non-target pests, we introduce a dedicated [NON] token alongside the standard [SEG] token for target pests. This guides the model to prioritize small target pests and suppress non-target background regions. To further address pest similarity, we propose an Exclusivity Suppression Loss, applying differentiated supervision to [SEG] and [NON] tokens to better separate target and non-target pests. Additionally, we develop an automated dataset construction pipeline to address the scarcity of fine-grained, difficulty-controllable pest RS datasets. It produces 45k and 27.6k image-text-mask samples for the PDRES and PERS tasks, respectively, covering 18 pest categories. Experiments show that in small and similar pest scenarios, integrating PestScope into mainstream models improves average gIoU by 4.28% on PDRES and 6.49% on PERS. For unseen pest categories, gIoU increases by 21.72% and 8.66%, respectively, demonstrating strong generalization. Code and datasets will be available at: https://github.com/aluodaydayup/PestScope.

Autonomous farm vehicles (AFVs) encounter significant challenges in large-scale networking and massive data transmission. The rapid development of global low Earth orbit (LEO) satellite networks provides reliable support for AFVs. However, the time-varying characteristics of the satellite-terrestrial channel and large-scale collaborative scheduling among AFVs pose challenges for joint computation offloading between satellites and AFVs. This paper proposes a satellite and autonomous farm vehicle integrated network (SAFVIN) architecture. We formulate the joint satellite and AFVs computation offloading problem as a Markov decision process (MDP). We propose a deep rein forcement computation offloading (DRCO) method that adapts to satellite networks. Unlike traditional computation offloading methods, the proposed DRCO takes into account the time varying satellite network channel states. The DRCO can rapidly converge to high-quality decisions in satellite network with strong randomness, thereby adapting to dynamic environments more quickly and achieving superior performance. We compare the proposed DRCO with the heuristic coordinate descent (CD), and with deep Q-network (DQN) and deep deterministic policy gradient (DDPG) algorithms. The DRCO achieves a 2% lower latency loss while only incurring 21% of the time overhead required by the CD. Furthermore, unlike DQN and DDPG algorithms, which rely on continuous time frame input and output for network updates, the proposed DRCO can directly leverage past experience to adapt to dynamic satellite network. Compared with other deep reinforcement learning algorithms including DQN and DDPG, the DRCO achieves an average energy consumption reduction of approximately 10%.

Deep neural network (DNN) models are increasingly popular in edge video analytic applications. However, the computeintensive nature of DNN models pose challenges for energyefficient inference on resource-constrained edge devices. Most existing solutions focus on optimizing DNN inference latency and accuracy, often overlooking energy efficiency. They also fail to account for the varying complexity of video frames, leading to sub-optimal performance in edge video analytics. In this paper, we propose an EnergyEfficient Early-Exit (E4) framework that enhances DNN inference efficiency for edge video analytics by integrating a novel early-exit mechanism with dynamic voltage and frequency scaling (DVFS) governors. It employs an attentionbased cascade module to analyze video frame diversity and automatically determine optimal DNN exit points. Additionally, E4 features a just-in-time (JIT) profiler that uses coordinate descent search to co-optimize CPU and GPU clock frequencies for each layer before the DNN exit points. Extensive evaluations demonstrate that E4 outperforms current state-of-the-art methods, achieving up to 2.8× speedup and 26% average energy saving while maintaining high accuracy.

Andong Chen, Yuchen Song, Kehai Chen, Xuefeng Bai, Muyun Yang, Liqiang Nie, Jie Liu, Tiejun Zhao, Min Zhang. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Multi-agent systems (MAS) based on large language models (LLMs) have demonstrated significant potential in collaborative problemsolving.However, they still face substantial challenges of low communication efficiency and suboptimal task performance, making the careful design of the agents' communication topologies particularly important.Inspired by the management theory that roles in an efficient team are often dynamically adjusted, we propose AgentDropout, which identifies redundant agents and communication across different communication rounds by optimizing the adjacency matrices of the communication graphs and eliminates them to enhance both token efficiency and task performance.Compared to state-of-the-art methods, AgentDropout achieves an average reduction of 21.6% in prompt token consumption and 18.4% in completion token consumption, along with a performance improvement of 1.14 on the tasks.Furthermore, the extended experiments demonstrate that AgentDropout achieves notable domain transferability and structure robustness, revealing its reliability and effectiveness.We release our code at https://github. com/wangzx1219/AgentDropout.

Bin Xie, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, Jie Liu, Min Zhang, Liqiang Nie. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Multi-hop question answering faces substantial challenges due to data sparsity, which increases the likelihood of language models learning spurious patterns.To address this issue, prior research has focused on diversifying question generation through content planning and varied expression.However, these approaches often emphasize generating simple questions and neglect the integration of essential knowledge, such as relevant sentences within documents.This paper introduces the Knowledge Composition Sampling (KCS), an innovative framework designed to expand the diversity of generated multi-hop questions by sampling varied knowledge compositions within a given context.KCS models the knowledge composition selection as a sentence-level conditional prediction task and utilizes a probabilistic contrastive loss to predict the next most relevant piece of knowledge.During inference, we employ a stochastic decoding strategy to effectively balance accuracy and diversity.Compared to competitive baselines, our KCS improves the overall accuracy of knowledge composition selection by 3.9%, and its application for data augmentation yields improvements on HotpotQA and 2Wiki-MultihopQA datasets.

In recent years, the significance of pre-trained transformer audio models has been increasingly recognized. However, existing pre-trained transformer audio models are based on single-channel audio. They cannot be directly applied to multi-channel audio for Sound Event Localization and Detection (SELD) tasks. To address this issue, in this paper, we propose SELD-SSAST, a novel model based on the single-channel Self-Supervised Audio Spectrogram Transformer (SSAST). Specifically, we first introduce a fusion feature that enables SSAST to learn the unique features in SELD problems effectively. Secondly, we input the multi-channel audio features into a single SSAST module to learn the temporal information across channels through channel-mixing. Finally, to enable SSAST to learn the relationships between multi-channel audio features, we propose a Convolutional Cross Attention (CCA) module to replace the Transformer’s Self-Attention and an intensity vector (IV) enhanced module to learn the differences between channel features. Our experiments show that using SELD-SSAST improved performance by 23.5% and 20.2% over the baseline on two datasets, respectively. Additionally, with the same data scale, SELD-SSAST outperforms the models in state-of-the-art (SOTA) methods on two datasets.

Vision-Language Models (VLMs) effectively align images and text, but they often struggle with fine-grained reasoning tasks. Fine-grained training also typically demand substantial GPU memory and a large number of trainable parameters. While existing Parameter-Efficient Fine-Tuning (PEFT) methods address these challenges, they also introduce new limitations, such as compromising generalization performance with additional parameters and causing overfitting with partial fine-tuning. To overcome these limitations, we propose History Tracker, a novel PEFT method designed to enhance fine-grained reasoning performance in VLMs. History Tracker leverages the stacked architecture of the image encoder, dividing its layers into shallow, middle, deep, and global groups. It then retrieves and compresses embeddings from these groups into compact tokens, reinforcing the distribution of image embeddings and allowing the text decoder to capture fine-grained details. Furthermore, we introduce a fine-grained dataset to fine-tune and evaluate models. This dataset focuses on tasks where highly similar text needs to be ranked based on its alignment with the given image. Experimental results demonstrate that History Tracker outperforms existing fine-tuning methods on both general benchmarks and fine-grained reasoning tasks. By fine-tuning only 7.03% of parameters, our method strikes a balance between fine-grained reasoning, generalization, and efficiency.

Vision-Language Models (VLMs) have achieved significant advances across various downstream tasks. However, as their performance improves, the increasing number of parameters results in slower prefilling speeds and longer inference times. To overcome these limitations, we observe that most VLMs do not require a large number of image tokens for inference, we propose BOLT (Basis-Oriented Lightweight Token-Trimming), a training-free and cross-attention-free token compression method. Unlike existing approaches, BOLT addresses the challenge of insufficient visual cues in textual prompts by leveraging token internal data distributions. We categorize tokens into three types: key tokens, proxy tokens, and remaining tokens. Then, by applying basis space similarity, we merge and filter the remaining tokens with the proxy tokens to retain the most informative ones. To account for the differences in VLM architectures and model sizes, we evaluate BOLT on LLaVA-Next-Llama3 and LLaVA-1.5 (7B and 13B). Our results show that BOLT achieves state-of-the-art performance, with a 90% token compression ratio leading to a 3.3× increase in pre-filling speed and a 1.5× improvement in inference speed, outperforming other methods.

Instruction fine-tuning is crucial in NLP tasks, enhancing pretrained models' instruction-following capabilities and task-specific performance. However, obtaining high-quality fine-tuning data for large models is challenging due to data collection difficulties and high production costs. To address this, we propose MASTER, a novel data augmentation method that enriches original data through interactions among multiple agents with varying cognitive levels. We simulate three pedagogically grounded teaching scenarios, leveraging multi-agent conversations to generate high-quality teacher-student interaction data. Utilizing MASTER, we construct BOOST-QA, a fine-tuning dataset augmented from existing datasets like Orca-Math-200k, ProcQA, and OpenHermes2.5. Experiments show that models fine-tuned with BOOST-QA perform excellently across multiple benchmarks, demonstrating strong multitask generalization. Notably, MASTER significantly improves models' reasoning abilities in complex tasks, providing valuable insights for future research.

Edge intelligence applications frequently generate deep learning inference tasks with varying Service Level Objectives (SLO, such as accuracy and real-time requirements). For such tasks, recent progressive inference modes support early exit from different branches to satisfy inference requirements. However, existing edge-cloud progressive neural architectures cannot simultaneously achieve high accuracy and real-time performance for different data features. Therefore, we utilize explainable AI technique to construct and train a novel progressive neural architecture E3. E3 can progressively extract the most important features for inference, ensuring higher accuracy at early-exit points. While the less important features in the later stage are highly compressible, thereby reducing edge-cloud transmission overhead. Furthermore, E3 cooperates with online execution control to launch tasks and decide the exit point for each task, ensuring resource utilization and real-time performance, and adapting to bandwidths and deadlines. Experimental results on various edge-cloud platforms, datasets, and reference models demonstrate that E3 is more lightweight, efficient, energy-saving, and incurs almost no additional runtime overhead compared to traditional architectures. Under stringent deadlines, the average accuracy of tasks increases by > 50%, and the deadline satisfaction rate approaches 100%.

Existing task deployment frameworks for edge clusters optimize at the model-level, lacking fine-grained resource awareness and concurrency control, where the urgent tasks are frequently blocked and miss their deadlines. Therefore, we propose a multi-level collaborative deployment framework Coconut for real-time deep learning tasks in the typical heterogeneous edge GPU cluster. Coconut collaboratively optimizes model deployment and fine-grained concurrency control. To address the high complexity of multi-level collaborative optimization, we employ an efficient learning-based search algorithm. Based on the operator-level information, we also pre-train an accurate latency predictor for each device, enabling centralized optimization to further accelerate the search. We conduct a preliminary evaluation in an edge cluster and validate the effectiveness of Coconut.

The communication among nodes in the space–air–sea integrated network (SASIN) relies on collaborative multihop transmission. Hence, effective routing techniques should be designed to optimize multiple indicators. Routing optimization for multihop is usually focused on optimizing a single metric. Moreover, designing effective routing strategies for multihop networks with SASIN is challenging as balancing multiple performance metrics can lead to conflicts. In this article, we propose near-Pareto multiobjective routing optimization for SASIN, which adopts multiobjective combinatorial optimization (MOCOP) to strike a tradeoff among multiple objectives. We establish the SASIN system model, including channel models of communication links between satellites, aircraft, and ships. Furthermore, we use multiobjective optimization methods to formulate objective functions of spectral efficiency, energy efficiency, and delay. We employ the multiobjective evolutionary algorithms (MOEAs) for approximating the set of the Pareto optimal solutions. An improved nondominated sorting genetic algorithm II (INSGA II) and an improved strength Pareto evolutionary algorithm II (ISPEA II) are proposed to generate approximations of the Pareto optimal set. We evaluated the MOCOP formulation, and the SASIN network topology was built based on real data and simulated data. The simulation results indicate that a set of beneficial tradeoff solutions can be obtained for providing flexible selection of communication connections by addressing the multiobjective routing problem formulated. The results demonstrate that the MOEAs utilized have the potential to find Pareto-optimal solutions for SASIN.

The widespread deployment of Internet of Things (IoT) devices has made them vulnerable targets for cyber attacks, highlighting the great significance of IoT device identification for network security management. Existing studies have primarily focused on either manual extraction of excessive network traffic features or heavy reliance on labeled data. To address these limitations, we propose a novel IoT device identification approach (named IoT-SCNet) via semi-supervised contrastive learning of network traffic visual representations. Specifically, IoT-SCNet converts two packet-level features and raw traffic into network traffic images of each device, and applies three augmentation strategies designed for network traffic to construct semantically meaningful positive/negative image pairs. By deep neural networks automatically capturing discriminative patterns and a contrastive task, IoT-SCNet achieves effective identification of diverse IoT device types. Comprehensive evaluations across three benchmark datasets demonstrate the superior performance of IoT-SCNet, achieving remarkable identification accuracies of 99.83% on UNSW, 97.50% on Yourthings, and 99.34% on CIC IoT datasets.

Path planning is an important mean to alleviate traffic congestion and reduce travel cost in the internet of vehicles. Existing path planning strategies primarily rely on shortest path or single-agent algorithms. However, they encounter challenges related to global dynamic coordination safety and resource constraints. Therefore, we propose a Multi-Agent dynamic Path Planning strategy based on the Principal component Federation (MA3PF) to address the these challenges. In this strategy, we introduce the Heuristic Multi-Agent Deep Policy Gradient algorithm (H-MADPG), which incorporates future traffic states and leverages shared agent experiences through migration learning. This approach ensures fast convergence and addresses resource limitations through a combination of centralized evaluation and distributed decision-making. Next, we propose the Adaptive Client-based Principal component Federation learning algorithm (ACPFed) for real-time prediction of future traffic flow. This algorithm utilizes principal component analysis to select model parameters and incorporates Bayesian optimization to dynamically determine client weights. These enhancements improve global coordination security and reduce communication overhead. Experimental results on real and simulated datasets demonstrate that our proposed MA3PF method outperforms existing agent algorithms, such as DARP and MAPF. It achieves superior route planning in complex environments, resulting in a reduction of travel distance by 21.90% and time loss by 39.41%. Additionally, MA3PF improves the real-time prediction accuracy of future traffic states while achieving a significant 70% reduction in communication cost. Note to Practitioners—This paper was motivated by the problem of holding multi-vehicle path planning. Existing path planning approaches only consider path planning for a single vehicle, which 1) fail to consider the challenges posed by resource constraints and data security in multi-vehicle planning 2) neglect the influence of future traffic states on path planning. This paper suggests a new dynamic path planning strategy, which incorporates real-time information about future traffic states, ensures data security, and reduces communication overhead using a specially designed ACPFed algorithm, simultaneously leveraging transfer learning and distributed decision-making to address resource constraints. In this paper, we present a mathematical characterization of the multi-vehicle path planning problem, incorporating future traffic states, and analyze the problem to derive an effective heuristic scheme MA3PF. The experimental results on real and simulated datasets demonstrate that the proposed MA3PF achieves significant improvements in terms of route distance and time loss compared to the traditional agent algorithm. In future research, we will explore more complex road environments and incorporate the effects of safety attacks on dynamic agent path planning.

To improve logistics efficiency, smart logistics sorting is an inevitable trend in logistics development. Existing smart logistics sorting systems suffer from high construction costs or limited scalability. To solve these problems, we design a brand-new two-dimensional conveyor system called Omniveyor, which transports and sorts high-density packages within a limited space. It is assembled from multiple repetitive square conveyor modules, achieving the goal of cost-effectiveness and easy maintenance. To realize automatic sorting, we model the planning problem on Omniveyor and propose a scheduling strategy named MMPPO by reinforcement learning. Unlike traditional path planning, MMPPO assigns actions to modules rather than packages, which reduces scheduling overhead in high-throughput scenarios. Furthermore, we develop a simulation environment to inspect the effectiveness of our method, which solves an intractable package-following problem that has plagued simulation implementation in this field. Experimental results show that MMPPO outperforms baselines in terms of throughput and overall consumption at high densities. Besides, we implement a physical prototype of Omniveyor to validate its feasibility. Note to Practitioners—The motivation of the paper is to solve the sorting problems in the logistics system. Existing logistics sorting systems often have problems such as low sorting efficiency and high costs, making them unsuitable for small-sized warehouses. In this paper, we present a modular 2D desktop logistics system that is both scalable and efficient for sorting. We mathematically describe the platform package transportation process and propose an algorithm that addresses scheduling and planning problems, capable of continuous planning under pipeline input. Preliminary simulation tests indicate that our approach is feasible, and we have also built a small-scale prototype. In future research, we will further expand the scale of the platform and conduct research.

It is a long-standing question to discover causal relations from observed variables in many empirical sciences. However, current causal discovery methods are inefficient when dealing with large-scale observed variables due to challenges in conditional independence (CI) tests or complex computations of acyclicity, and may even fail altogether. To address the efficiency issue in causal discovery from large-scale observed variables, we propose a Hierarchical Causal Discovery (HCD) framework with a bilevel policy that handles this issue by boosting existing models. Specifically, the high-level policy first finds a causal cut set to partition observed variables into several causal clusters and releases the clusters to the low-level policy. The low-level policy applies any causal discovery method to process these causal clusters in parallel and obtain intra-cluster structures for subsequently inter-cluster structure merging in the high-level policy. To avoid missing inter-cluster edges, we theoretically demonstrate the feasibility of causal cluster cut and inter-cluster structure merging. We also prove the completeness and correctness of HCD for causal discovery. Experiments on both synthetic and real-world datasets demonstrate that HCD consistently and significantly enhances the efficiency and effectiveness of existing advanced methods.

Object detection, a fundamental task in computer vision, is crucial for various intelligent edge computing applications. However, object detection algorithms are usually heavy in computation, hindering their deployments on resource-constrained edge devices. Traditional edge-cloud collaboration schemes, like deep neural network (DNN) partitioning across edge and cloud, are unfit for object detection due to the significant communication costs incurred by the large size of intermediate results. To this end, we propose a Difficult-Case based Small-Big model (DCSB) framework. It employs a difficult-case discriminator on the edge device to control data transfer between the small model on the edge and the large model in the cloud. We also adopt regional sampling to further reduce the bandwidth consumption and create a discriminator zoo to accommodate the varying networking conditions. Additionally, we extend DCSB to video tasks by developing an adaptive sampling rate update algorithm, aiming to minimize computational demands without sacrificing detection accuracy. Extensive experiments show that DCSB can detect 97.26%-97.96% objects while saving 74.37%-82.23% network bandwidth, compared to cloud-only methods. Furthermore, DCSB significantly outperforms the latest DNN partitioning methods, reducing inference time by 92.60%-95.10% given an 8Mbps transmission bandwidth. In video tasks, DCSB matches the detection accuracy of leading video analysis methods while cutting the computational overhead by 40%.

Satellite-terrestrial integrated networks based on edge computing can provide computation offloading service to terminal devices in remote areas. However, it faces various limitations, including satellite energy consumption, computation delay, and environmental dynamics, etc. In this paper, we propose a satellite-terrestrial integrated cloud and edge computing network (STCECN) architecture, including satellite layer, terrestrial layer and cloud center, where computing resources exist in multi-layer heterogeneous edge computing clusters. Optimization of system delay and energy consumption is defined as a mixed-integer programming problem. Moreover, we present a deep reinforcement learning-based computation offloading decision algorithm that can adapt to the dynamics and variability of satellite networks. A dual network computation offloading decision method is proposed for delay and energy consumption based on deep reinforcement learning offloading (DRLO), including deep convolutional network update method, quantization strategy, and bandwidth resource allocation. Meanwhile, the proposed method is based on previous experience and integrates deviation adjustment strategies for decision making to solve the problem of pseudo-patch loss caused by satellite network switching. The simulation results indicate that the proposed method performs almost consistently with traditional heuristic algorithms, with only 20% of the time consumption of the latter, and the number of pseudo packet loss also decreases to the original 10–20%.

Advancements in Earth observation sensors on low Earth orbit (LEO) satellites have significantly increased the volume of remote sensing images. This growth has led to challenges such as higher storage demands, downlink bandwidth stress, and transmission delays, particularly for real-time remote sensing image scene classification (RSISC). To address this, we propose a novel Satellite-Terrestrial Collaborative Scene Classification (STCSC) framework that integrates transmission and computation. The framework employs an attention-aware policy on the satellite, which adaptively determines the sequence of images and selection of image blocks for transmission, as well as these blocks' sampling rates. This policy is based on image complexity and the real-time data transmission rate, prioritizing blocks crucial for downstream tasks. On the ground, a classification model processes the received image blocks, balancing classification accuracy and transmission delay. Moreover, we have developed a comprehensive simulation system to validate the performance of our framework, including simulations of the satellite, transmission, and ground modules. Simulation results demonstrate that our STCSC framework can reduce transmission delay by 76.6% while enhancing classification accuracy on the ground by 0.6%. Additionally, our attention-aware policy is compatible with any ground classification model.

Radio frequency sensors can penetrate non-metal objects and provide complementary information to vision sensors for person identification (PID) purposes. However, there is a lack of research on millimeter wave (mmWave) radar for PID under occlusions, particularly in addressing the open-set recognition problem. Thus, we propose an open-set occluded PID (OSO-PID) framework that can deal with various obstacle and occlusion scenarios with open-set recognition capability. We first introduce a new dataset, mmWave-ocPID, comprising mmWave radar measurements and RGB-depth images, collected from 23 human subjects. We next design a novel neural network, mm-PIDNet, for occluded person identification using mmWave radar measurements. mm-PIDNet incorporates a transformer encoder, a bidirectional long short-term memory module, and a novel supervised contrastive learning module to improve PID performance. For open-set recognition, we enhance the mmWave radar-based PID method by integrating supervised contrastive learning with the Weibull models, which can identify out-of-distribution samples. We perform extensive indoor experiments with a variety of obstacles and occlusion scenarios. Our experimental results show that mm-PIDNet achieves an F1-score of 0.93 on average, outperforming state-of-the-art methods by up to 13.41% for occluded cases. For open-set PID, the OSO-PID framework achieves an F1-score above 0.8 when the openness is less than 14.36%.

Fine-grained human motion detection has become increasingly important with the growing popularity of human computer interaction (HCI). However, traditional gesture-based HCI systems often require the design of new operation modes rather than conforming to user habits, thus increasing system learning costs. In this paper, we present TapWristband, a novel wearable sensor-based vibration sensing system that detects finger tapping by measuring wrist vibrations. We first perform real-world experiments to collect measurements for modeling the effects of the tapping motion on wearable wristband sensors including piezoelectric transducer (PZT) and inertial measurement unit (IMU). We find that a damped vibration model can be used to represent the relaxing phase of a vibration response due to tapping motion. Thus, we propose a mutual cross-correlation-based event segmentation algorithm to extract the vibration signal during the relaxing phase. After that, we develop feature extraction and classification algorithms to recognize the tapping patterns of five fingers across twelve key locations of a keypad system. Finally, we performed extensive experiments with thirteen participants to evaluate our system. Experimental results show that our low-cost vibration sensing system can achieve an average accuracy of over 93% with a tapping speed of over 100 taps per minute in real-world tapping scenarios.

Accurately detecting objects and their interrelationships for Video Scene Graph Generation (VidSGG) confronts two primary challenges. The first involves the identification of active objects interacting with humans from the numerous background objects, while the second challenge is long-tailed distribution among predicate classes. To tackle these challenges, we propose STABILE, a novel framework with a spatial-temporal saliency-guided contrastive learning scheme. For the first challenge, STABILE features an active object retriever that includes an object saliency fusion block for enhancing object embeddings with motion cues alongside an object temporal encoder to capture temporal dependencies. For the second challenge, STABILE introduces an unbiased relationship representation learning module with an Unbiased Multi-Label (UML) contrastive loss to mitigate the effect of long-tailed distribution. With the enhancements in both aspects, STABILE substantially boosts the accuracy of scene graph generation. Extensive experiments demonstrate the superiority of STABILE, setting new benchmarks in the field by offering enhanced accuracy and unbiased scene graph generation.

Federated learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the nonindependent identically distributed (Non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the distinction between the training distributions of participating clients and the testing distributions of nonparticipating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations are designed to strengthen FL's ability to generalize and thus ensure that trained models perform better on nonparticipating clients by incorporating a more diverse range of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.

The development of the Internet of Things promotes the increasing demand for edge computing, resulting in a large amount of temporal data. Protecting data from tampering has become key to industrial intelligent management. Blockchain technology has become an ideal choice for ensuring data trustworthiness due to its immutability and other characteristics. However, existing technologies do not provide sufficient support for temporal data. There are still issues such as chaotic data organization, low query efficiency, and insufficient lightweight validation. To address these challenges, we combine the Secure Hash Algorithm and Merkle tree to serialize temporal data. We design a temporal Merkle prefix forest on the blockchain cloud main chain and construct an index for intra-block localization. Verification can be completed by monitoring the latest tree, significantly improving query efficiency. For edge-side devices with limited computing resources, we design a temporal Bloom Merkle tree, where lightweight nodes only need to pass the Merkle proof of the root node to verify data integrity. Experimental results demonstrate that our method significantly improves query efficiency and reduces storage requirements, meeting the reliability and lightweight requirements of temporal data management in the Internet of Things.

Current deep learning compilers have made significant strides in optimizing computation graphs for single- and multi-model scenarios. However, they lack specific optimizations for asynchronous multi-task inference systems. In such systems, tasks arrive dynamically, leading to diverse inference progress for each model. This renders traditional optimization strategies based solely on the original computation graph suboptimal or even invalid. Furthermore, existing operator scheduling methods do not account for parallel task pipelines involving the same model. Task pipelines present additional opportunities for optimization. Therefore, we propose Task-based Operator Parallelism (TOP). TOP incorporates an understanding of the impact of task arrival patterns on the inference progress of each model. It leverages the multi-agent reinforcement learning algorithm MADDPG to cooperatively optimize the task launcher and model scheduler, generating an optimal pair of dequeue frequency and computation graph. The objective of TOP is to enhance resource utilization, increase throughput, and allocate resources judiciously to prevent task backlog. To expedite the optimization process in TOP, we introduce a novel stage partition method using the GNN-based Policy Gradient (GPG) algorithm. Through extensive experiments on various devices, we demonstrate the efficacy of TOP. It outperforms the state-of-the-art in operator scheduling for both single- and multi-model task processing scenarios. Benefiting from TOP, we can significantly enhance the throughput of a single model by increasing its concurrency or batch size, thereby achieving self-acceleration.

The Vision Transformer (ViT) excels in accuracy when handling high-resolution images, yet it confronts the challenge of significant spatial redundancy, leading to increased computational and memory requirements. To address this, we present the Localization and Focus Vision Transformer (LF-ViT). This model operates by strategically curtailing computational demands without impinging on performance. In the Localization phase, a reduced-resolution image is processed; if a definitive prediction remains elusive, our pioneering Neighborhood Global Class Attention (NGCA) mechanism is triggered, effectively identifying and spotlighting class-discriminative regions based on initial findings. Subsequently, in the Focus phase, this designated region is used from the original image to enhance recognition. Uniquely, LF-ViT employs consistent parameters across both phases, ensuring seamless end-to-end optimization. Our empirical tests affirm LF-ViT's prowess: it remarkably decreases Deit-S's FLOPs by 63% and concurrently amplifies throughput twofold. Code of this project is at https://github.com/edgeai1/LF-ViT.git.

Edge servers and mobile devices are often assigned a large number of computing tasks. However, the data involved in computing tasks is often sensitive in terms of privacy. Our initial proposal is a federated edge learning strategy based on real-world scenarios, which combines blurred data or pseudo shared data. Federated learning is used to train device models with the aim of protecting privacy while enabling mobile devices to more effectively utilize data for decision-making. In the case of limited energy on mobile devices, we propose a federated edge learning algorithm with blurred data sharing. This algorithm can generate more accurate models by uploading partially blurred data. In order to further improve model accuracy and protect privacy of mobile devices, we propose a federated edge learning algorithm with pseudo data sharing based on dataset distillation and generative adversarial networks (GANs) in scenarios with relatively sufficient energy. The experimental results on several traditional datasets show that our proposed algorithms outperform traditional algorithms in terms of accuracy and energy consumption.

Vehicle Trajectory Prediction (VTP) plays a pivotal role in the Internet of Vehicles (IoV), significantly aiding in motion planning and accident prevention. Nonetheless, the field faces challenges in distributed data collection and trajectory privacy protection. Existing approaches often incur substantial communication overheads and are not suited for contemporary road environments. Furthermore, there is a noticeable absence of a standardized benchmark. In response to these challenges, our paper introduces an innovative Federated Learning (FL) methodology for VTP. We leverage roadside units (RSUs) as the FL clients, rather than directly using vehicles. This strategy minimizes resource consumption and suits for the current scenario where trajectories are collected by RSUs. In addition, we present FedVTP, a comprehensive benchmark for federated spatial-temporal graph solutions in VTP. FedVTP integrates various strategies and eases for future expansions. We conduct extensive experiments to evaluate the effectiveness of our approach, with detailed analyses provided within FedVTP. This benchmark further encourages the development of new FL strategies for VTP, while enabling equitable comparisons among research works in the field. To facilitate further research and collaboration, our source code will be accessible at https://github.com/FedVTP/FedVTP.

The constrained computing and memory resources at the edge pose challenges for satisfying different service-level objectives (SLOs) of deep learning inference requests. In this paper, we propose a novel edge-cloud progressive inference framework Xpi, which integrates explainable AI technique to facilitate early-exit, and learning-based online execution control to satisfy different SLOs and optimize edge resource overheads. We implement Xpi on an edge-cloud platform, and conduct partial experiments on two datasets. Xpi outperforms several advanced edge-cloud progressive inference frameworks in terms of accuracy and deadline satisfaction rate.

Multi-tenant inference, as a prevalent inference paradigm nowadays, requires deploying multiple deep learning models on the hardware platform to concurrently process inference tasks. Modern platforms are typically equipped with various heterogeneous processors, such as CPU-GPU platform. To reduce resource contention and improve Quality of Service (QoS) in the multi-tenant scenario, existing work has studied cross-processor inference at the model- and layer-level. However, coarse-grained scheduling cannot flexibly account for subtle resource fluctuations, which may lead to task blockages and incur significant processor switching overheads. Such work usually requires extensive modification and retraining of the models. Therefore, we propose a finer-grained operator-level cross-processor scheduling framework COS, which can more precisely divide the computational workloads and switching overheads for the tenants, without modifying or retraining. We introduce a novel intermediate representation to abstract and simplify the scheduling problem, and propose an efficient two-phase search algorithm. COS is automated and easy-to-scale, through experiments on various heterogeneous hardware platforms and models, we demonstrate that COS is more flexible and effective than layer-level scheduling, and achieves higher throughput than single-processor processing in the multi-tenant scenario. Furthermore, COS is an offline optimization method, and its overhead is highly acceptable.

In this paper, we jointly optimize compression, partition and distillation for visual Transformer. We analyze the relationship among the three modules and integrate them into a QoS-aware training framework. By coordinating the model compression, edge-cloud partition, and knowledge distillation during training, the model architecture and accuracy are optimized simultaneously. The framework considers the differences in computing power and memory between edge and cloud, and can trade off QoS metrics such as the memory overhead, end-to-end latency, accuracy at multiple granularities.

We propose an efficient lightweight fine-tuning method that simplifies model design and reduces parameters, focusing on optimizing Visual-Language Models (VLMs) for edge deployment. As VLMs evolve, the parameter size becomes increasingly challenging for edge devices. To overcome this limitation, we combine lightweighting and fine-tuning into a single step. We decompose large linear layers in the vision encoder and introduce smaller matrices in parallel, creating a new path.During fine tuning, performance is improved by reducing the matrix size and increasing the depth, gradually phasing out the original path. We deploy the lightened and fine-tuned model on a Jetson TX2 and shows comparable performance compared to VLMs with larger parameters.

The increasing integration of wearable devices in daily activities has elevated the need for robust authentication methods that safeguard user data. Traditional knowledge-based and biometric authentication techniques face challenges in wearable contexts, including privacy risks and hardware limitations. We propose TapID, a novel authentication approach that leverages the unique relaxation vibrations of wrist bone conduction following a tapping gesture. This method bypasses the need for intrusive data collection and expensive hardware. Our method employs an energy window extraction algorithm and cross-correlation to isolate biometric signals, followed by feature extraction and k-NN classification. Tested on a Raspberry Pi, TapID authenticated users with a 93% success rate in a preliminary trial involving ten individuals, demonstrating its potential for secure and user-friendly wearable authentication.

Object detection is crucial in video analytics pipelines, but there is a need to optimize deep neural networks (DNNs)-based object detection for resource-constrained Internet of Things (IoT) devices devices. The computational constraints inherent to the IoT device inevitably curtail its precision and real-time efficacy in the domain of object detection, with pronounced challenges arising, particularly when confronted with high-resolution video streams. To overcome these limitations, we propose UPD (Using Physical Dynamics), a novel on-device system that enables real-time and accurate object detection for high-resolution video streams. UPD employs a lightweight tracking algorithm for the detection of the majority of video frames, concurrently executing the object detector in a parallel fashion only in select instances. UPD addresses tracking errors by eliminating inaccurate feature points and correcting tracking results using physical information about the object. Unlike previous approaches that depend solely on the high-latency object detector to offset errors, our method is unaffected by the video resolution level. Extensive experiments demonstrate that UPD facilitates real-time analysis of high-resolution videos on IoT devices and significantly improves the overall accuracy (mIoU) compared to state-of-the-art DBT (Detection-Based-Tracking) frameworks, achieving a 100% accuracy improvement on three commonly used datasets. A video demo can be found at https://youtu.be/gKRQPHJ6gmY.

Nowadays, Mobile Edge Computing (MEC) appears as a new computing paradigm with its ability to utilize the computing power of both local devices and edge servers. In MEC, edge pricing and resource allocation are two important problems. Edge servers make a profit by selling computing services to users. To maximize their revenue, they need to determine an appropriate price for each user, and decide the amount of resources allocated to each user. However, none of the existing works consider the effect of users’ task assignment strategy on the revenue of the edge. In fact, edge pricing and resource allocation will affect the users’ task offloading decision, as they expect to minimize their total cost. In turn, the users’ decision will also influence the revenue of the edge. Therefore, the interaction between mobile users and edge servers should be considered carefully and the interests of both sides need to be maximized simultaneously. In this paper, we model the interaction between the two sides as a Stackelberg game. First, given a specified edge pricing and resource allocation strategy, we derive a near-optimal task assignment strategy for each user to minimize the total cost based on a greedy algorithm UTA-G. Then, by applying the backward induction method, two pricing and resource allocation schemes with different granularity, i.e., EPRA-U and EPRA-T are proposed to bring higher revenue to the edge. Experimental results demonstrate that all the proposed algorithms can have good performance in task-intensive, resource-deficient and workload-heavy scenarios.

Multi-source sound localization can find applications in many domains including auditory scene analysis, fault detection and diagnosis in manufacturing, augmented reality, etc. In far fields, 3D sound source localization is equivalent to finding the direction of arrival (DOA), namely, the azimuth and elevation angles of sound sources. Recent DOA estimation pipelines take multichannel audio inputs, extract spectral features from each channel and then feed them into a deep neural network. Unfortunately, the spectral features contain only the time-frequency information of the audio signals, while spatial information is only implicitly captured in the signals across different channels, which is highly dependent on the acoustic array geometry. To embed the spatial information of the sound source into the spectral feature representation, we propose a DSB-based spatial mapping method encode sound source location information. It can be combined with different feature extraction methods and machine learning models for DOA estimation. Furthermore, a redundancy removal procedure is proposed to accelerate DSB computation so that the pipeline can run in real-time on embedded GPUs, such as NVidia Jeston Nano. We conduct extensive experiments using two neural network models along with the DSB method on two datasets. The experiments demonstrate that the DOA errors can be effectively reduced using the DSB method. When combining DSB for feature extraction, the DOA errors are reduced by up to 19.24%. In addition, the feature extraction process is accelerated by up to 30.42% after the application of redundancy removal.

Deploying compute-intensive deep neural networks (DNNs) on resource-constrained end devices has become a prominent trend, enabling localized intelligence. However, efficiently deploying these DNNs at scale poses challenges. To address this, extensive research has focused on the early exit architecture based on convolutional neural networks (CNNs), which dynamically adapt network depth to reduce inference computation. Nevertheless, the sequential execution of all internal classifiers (ICs) and subsequent termination based on an exit criterion is inefficient. Motivated by these insights, we introduce a resolution-adaptive prediction network (RAPNet) architecture. RAPNet comprises a lightweight prediction network that captures global image features and an inference network integrated with an early exit architecture. The prediction network accurately determines the optimal IC position conditioned on the input images for efficient image classification. Additionally, we incorporate resolution-adaptive inference and feature fusion mechanisms by computational reuse, to effectively mitigate image spatial redundancy and improve the accuracy of ICs. We conduct extensive experiments across various data sets and architectures to demonstrate that RAPNet achieves a significantly better accuracy versus computational tradeoff than other recently proposed early exit methods. For instance, when using MobileNet as the base network, RAPNet achieves significant accuracy improvements of 12% and 5.7% on the Tiny Imagenet and CIFAR-100 data sets, respectively, surpassing other early exit methods with similar computational constraints.

Sparse code multiple access (SCMA) has excellent application prospects in satellite-terrestrial links because of its high spectral efficiency and access capacity. In the end-to-end SCMA systems, channel modeling is a fundamental task for the communication algorithm design and performance optimization, which however is very challenging as it requires in-depth domain knowledge and technical expertise in radio signal propagations, especially for modeling satellite-terrestrial fading channels. In this article, a convolutional autoencoder-aided SCMA paradigm based on the stochastic channel modeling and autoencoder structure is developed. We are the first to exploit generative adversarial network to represent the satellite-terrestrial fading channel effects for the convolutional autoencoder-aided SCMA. Specifically, convolutional neural networks (CNNs) are employed to jointly construct the encoder and decoder for SCMA to alleviate the curse of dimensionality. Furthermore, we propose a conditional Wasserstein generative adversarial network with the gradient penalty (CWGAN-GP)-based channel modeling approach to achieve approximately accurate conditional channel distribution. Particularly, the received signal corresponding to the pilot symbol is used as a part of the condition information, and the Wasserstein distance is used as a measure of the distance between the distributions. Gradient penalty is adopted to solve the problem of weight pruning forcing Lipschitz constraints, which leads to some data being unable to converge. The numerical results demonstrate the effectiveness of the proposed approach in terms of the bit error rate (BER), block error rate (BLER), and complexity in satellite-terrestrial fading channels.

Federated learning (FL) is an emerging paradigm to train model with distributed data from numerous Internet of Things (IoT) devices. It inherently assumes a uniform capacity among participants. However, due to different conditions such as differing energy budgets or executing parallel unrelated tasks, participants have diverse computational resources in practice. Participants with insufficient computation budgets must plan for the use of restricted computational resources appropriately; otherwise, they would be unable to complete the entire training procedure, resulting in model performance decline. To address this issue, we propose a strategy for estimating local models without computationally intensive iterations. Based on it, we propose computationally customized federated averaging (CC-FedAvg), which allows participants to determine whether to perform traditional local training or model estimation in each round based on their current computational budgets. Both theoretical analysis and exhaustive experiments indicate that CC-FedAvg has the same convergence rate and comparable performance as FedAvg without resource constraints. Furthermore, CC-FedAvg can be viewed as a computation-efficient version of FedAvg that retains model performance while considerably lowering computation overhead.

With the rapid growth of edge intelligence, the deployment of federated learning (FL) over wireless networks has garnered increasing attention, which is called Federated Edge Learning (FEEL). In FEEL, both mobile devices transmitting model parameters over noisy channels and collecting data in diverse environments pose challenges to the generalization of trained models. Moreover, devices can engage in decentralized FL via Device-to-Device communication while the communication topology of connected devices also impacts the generalization of models. Most recent theoretical studies overlook the incorporation of all these effects into FEEL when developing generalization analyses and ignore designing algorithms to enhance the generalization of models based on their analysis. In contrast, our work presents an information-theoretic generalization analysis for topology-aware FEEL with data heterogeneity and noisy channels. Additionally, we propose a novel regularization method called Federated Global Mutual Information Reduction (FedGMIR) to enhance the performance of models based on our analysis. Numerical results validate our theoretical findings and provide evidence for the effectiveness of the proposed method.

The Industrial Internet of Things (IIoT) has found extensive applications in intelligent transportation. However, as the number of vehicles increases, the issue of traffic congestion becomes more prominent, emphasizing the need for accurate congestion propagation prediction to enhance traffic conditions. Current methods for predicting congestion propagation lack consideration for the influence of communication networks and do not incorporate the path characteristics of congestion propagation. Therefore, we propose a path-based congestion propagation model, UAU_SIS_Path, employing multigrain abstraction of traffic congestion and information propagation. Specifically, UAU_SIS_Path effectively captures the propagation dynamics of congestion in IIoT by leveraging the interaction of two-layer networks and the path propagation characteristics of traffic congestion. Subsequently, we validate the effectiveness of the UAU_SIS_Path model through theoretical analysis, establishing tight upper and lower bounds of the propagation threshold and elucidating the relationship between the propagation of congestion information in the information network and the diffusion of congestion in the road network. Finally, based on theoretical analysis, we examine the impact of our model on congestion control strategies, utilizing path replanning, and traffic restriction as congestion control strategies. Experimental results in simulated road network BA and real road networks of varying sizes (GC, TA, and As) demonstrate the stability and scalability of our model. In comparison to the traditional contact-based propagation model, our model achieves a reduction in congestion propagation rates of 58.2%, 66.9%, 32.6%, and 48.6%, respectively.

The increasing deployment of multiple deep neural networks (DNNs) on edge devices is revolutionizing mobile vision applications, spanning autonomous vehicles, augmented reality, and video surveillance. These applications demand adaptation to contextual and environmental drifts, typically through fine-tuning on edge devices without cloud access, due to increasing data privacy concerns and the urgency for timely responses. However, fine-tuning multiple DNNs on edge devices faces significant challenges due to the substantial computational workload. In this paper, we present PatchLine, a novel framework tailored for efficient on-device training in the form of fine-tuning for multi-DNN vision applications. At the core of PatchLine is an innovative lightweight adapter design called patches coupled with a strategic patch updating approach across models. Specifically, PatchLine adopts drift-adaptive incremental patching, correlation-aware warm patching, and entropy-based sample selection, to holistically reduce the number of trainable parameters, training epochs, and training samples. Experiments on four datasets, three vision tasks, four backbones, and two platforms demonstrate that PatchLine reduces the total computational cost by an average of 55% without sacrificing accuracy compared to the state-of-the-art.

Due to limited resources on edge and different characteristics of deep neural network (DNN) models, it is a big challenge to optimize DNN inference performance in terms of energy consumption and end-to-end latency. In addition to dynamic voltage frequency scaling (DVFS) technique, edge-cloud architecture provides a collaborative approach for efficient DNN inference. However, current edge-cloud collaborative inference methods have not optimized various compute resources on edge devices. Thus, we propose DVFO, a novel DVFS-enabled edge-cloud collaborative inference framework, which co-optimizes DVFS and offloading parameters via deep reinforcement learning (DRL). Specifically, DVFO automatically co-optimizes 1) the CPU, GPU and memory frequencies of edge devices, and 2) the offloaded feature map. In addition, it leverages athinking-while-movingconcurrent mechanism to accelerate the DRL learning process, and aspatial-channel attentionmechanism to identify the less important DNN feature map for efficient offloading. This approach improves inference performance for different DNN models under various edge-cloud network conditions. Extensive evaluations using two datasets and six widely-deployed DNN models on five heterogeneous edge devices show that DVFO significantly reduces the energy consumption by 33% on average, compared to state-of-the-art schemes. Moreover, DVFO achieves up to 28.6%∼59.1% end-to-end latency reduction, while maintaining accuracy within 1% loss on average.

As deep neural networks (DNNs) are increasingly used in a broad spectrum of edge intelligent applications, it is often necessary to provide multi-DNN model inference services, and it is nontrivial for edge inference platforms to simultaneously deliver high-throughput and low-latency. Such edge devices with multi-DNN model pose new challenges for scheduler designs. First, edge devices should be capable of efficiently scheduling multiple heterogeneous DNN models in order to optimize system utilization. Second, each inference request may have different service level objectives (SLOs) to improve quality of service (QoS). To address these challenges, this paper proposes BCEdge, a novel learning-based scheduling framework that incorporates adaptive batching and concurrent execution of DNN inference services on edge devices. We first propose a shared memory policy to reduce the memory contention among multiple DNN models. Afterwards, a utility function is defined to evaluate the trade-off between throughput and latency. The scheduler in BCEdge leverages branch-based deep reinforcement learning (DRL) to maximize utility by 1) optimizing batch size, 2) automatically identifying the number of concurrent instances for multiple DNN models, and 3) determining the shared memory configuration among multiple DNN models. Besides, the lightweight DNN-based prediction model in BCEdge can achieve SLO awareness by reducing the performance interference among multiple DNN models. Our prototype implemented on various edge devices illustrates that BCEdge enhances utility by up to 37.6% and reduces memory usage by up to 38% on average, compared to state-of-the-art schemes, while maintaining the SLO violation rate within 5%.

Crop growth prediction can help agricultural workers to make accurate and reasonable decisions on farming activities. Existing crop growth prediction models focus on one crop and train a single model for each crop. In this article, we develop a ubiquitous growth prediction model for multiple crops, aiming at training a single model for multiple crops. A ubiquitous vision and sensor transformer (ViST) model for crop growth prediction with image and sensor data is developed to achieve the goals. In the proposed model, a cross-attention mechanism is proposed to facilitate the fusion of multimodal feature maps to reduce computational costs and balance the interactive effects among features. To train the model, we combine the data from multiple crops to create a single (ViST) model. A sensor network system is established for data collection on the farm where rice, soybean, and maize are cultivated. Experimental results show that the proposed ViST model has an excellent ubiquitous ability for crop growth prediction with multiple crops.

As one of the basic tasks of computer vision, object detection has been widely used in many intelligent applications. However, object detection algorithms are usually heavyweight in computation, hindering their implementations on resource-constrained edge devices. Current edge-cloud collaboration methods, such as CNN partition over edge-cloud devices, are not suitable for object detection since the large data size of the intermediate results will introduce extravagant communication costs. To address this challenge, we propose a difficult-case based small-big model (DCSB) framework that deploys a difficult-case discriminator on the edge device to control the data transfer between the small model (edge) and the big model (cloud). Upon receiving data, the edge device operates a difficult-case discriminator to classify images into easy cases and difficult cases according to the specific semantics of the images. The difficult cases will be uploaded to the cloud. To reduce bandwidth consumption, we propose a regional sampling method that adaptively down-samples some regions of the difficult case to reduce the amount of transferred data based on the primary results of the lightweight model. Experimental results on VOC, COCO, and HELMET datasets using two object detection algorithms demonstrate that DCSB can detect 93.77%-97.05% objects but save 77.19% -80.55% of network bandwidth compared with the cloud-only method, while the edge-only method can only detect 54.90%-68.28% objects in the same condition. In addition, compared with the state-of-the-art model partition method - CAS, DCSB saves 95.19%-95.80% of the inference time when the transmission bandwidth is 8Mbps.

Edge intelligent applications, such as autonomous driving usually deploy multiple inference models on resource-constrained edge devices to execute a diverse range of concurrent tasks, given large amounts of input data. One challenge is that these tasks need to produce reliable inference results simultaneously with millisecond-level latency to achieve real-time performance and high quality of service (QoS). However, most of the existing deep learning frameworks only focus on optimizing a single inference model on an edge device. To accelerate multi-model inference on a resource-constrained edge device, in this paper we propose POS, a novel operator-level scheduling framework that combines four operator scheduling strategies. The key to POS is a maximum entropy reinforcement learning-based operator scheduling algorithm MEOS, which generates an optimal schedule automatically. Extensive experiments show that POS outperforms five state-of-the-art inference frameworks: TensorFlow, PyTorch, TensorRT, TVM, and IOS, by up to 1.2 × ∼ 3.9 × inference speedup consistently, with 40% improvement on GPU utilization. Meanwhile, MEOS reduces the scheduling overhead by 37% on average, compared to five baseline methods including sequential execution, dynamic programming, greedy scheduling, actor-critic, and coordinate descent search algorithms.

Due to resource constraints, it is challenging to optimize the inference performance in terms of energy consumption and latency on edge devices. In this paper, we leverage both the dynamic voltage frequency scaling (DVFS) technique and edge-cloud collaborative inference to minimize the overall energy consumption. We propose a deep reinforcement learning (DRL)-based method called DVFO to jointly optimize 1) CPU, GPU and memory frequencies, and 2) the ratio of offloaded feature maps in edge-cloud collaboration. Preliminary experimental results show that DVFO reduces the average energy consumption by 33% compared to the baselines. Moreover, it reduces the inference latency by more than 54%.

We organized an online fingerprint-based indoor localization competition in 2021. It attracted 1,170 teams worldwide. The teams were provided with a 60 GB dataset including WiFi, BLE, IMU, and geomagnetic field strength data collected from 204 buildings to build their localization algorithms, which were then evaluated against a separate test dataset. The competition received 28,009 submissions. The top team achieved an average accuracy of 1.50m. This paper reports the lessons we learned from analyzing the submissions, as well as our experiences in organizing the competition, through both qualitatively studying the teams' algorithms and quantitatively characterizing the competition results.

The burgeoning field of smart agriculture is increasingly leveraging unmanned aerial vehicles (UAVs) for data collection. However, inadequate visual features and plant occlusion can hamper visual-based simultaneous localization and mapping (SLAM) of UAVs. As a potential solution, RFID can work as an efficient reference infrastructure, enabling a connection between aerial imagery and real-world contexts. Despite this promise, hurdles remain in attaining high-accuracy, high-throughput, and long-range RFID localization, as well as practical deployment of RFID tags and reader implementation on UAVs. Overcoming these challenges holds significant potential, particularly considering their impact on numerous applications, such as large-scale agricultural management and plant stand reduction detection.

Smart agriculture requires an extensive convergence of information technology and agriculture. Attaining intelligence mandates an enormous amount of data to train models. However, it is challenging to acquire a large number of crop image data, limiting the application and growth of computer vision technology in agriculture. To address this problem, we designed a crop image generation system that combines a large language model with visual language multi-modal large models to augment the scale, variety, and resolution of crop image data. First, the system inputs existing real crop images into the visual language multimodal model to extract features and represent crop images in text form. Then, the system passes the crop text representation to the language model for cleaning and processing, which generates prompts to create crop images. The prompts are input into the visual language multi-modal model to generate crop images based on text representation of crops. The resulting crop images undergo image quality evaluation in the visual language multimodal model, and high-quality crop images are saved to the crop image dataset based on the quality evaluation. These steps lead to the formation of the final generated crop image dataset. The experimental results indicate that the crop images generated using the proposed system are similar to but different from the example images. This characteristic enables the expansion of crop data while circumventing redundancy and allowing for resolution control, which is crucial for dense segmentation tasks. Using this method, the existing data can be enlarged up to 7.5 times.

Unmanned aerial vehicles (UAV) with onboard sensors become a cost-effective way of crop remote sensing in largescale farms. However, current vision-based crop aerial sensing methods suffer from occlusion issue and require large amount of annotation data. In this paper, we target one particular crop species, corn, and propose to use 3D modeling and simulation to help resolve the issues. We first develop a corn-field 3D model and a crop aerial sensing (CAS) simulation framework. Then we use the CAS framework to generate synthetic data to train various deep learning models for corn leaf segmentation. In addition, we change the 3D model parameters in CAS, e.g., distances between individual corn plants, to derive leaf area index (LAI) correction coefficients for various corn plant and row spacings. Our experimental results from real-world UAV images show that our leaf segmentation model using synthetic data from the CAS framework outperforms state-of-the-art segmentation models by 1.4-3.3%. Our simulation results show that the plant and row spacings of a corn-field have significant effects on correcting the UAV image-based LAI, which can be underestimated by a factor of 2.6, due to the overlap and occlusion issues.

We propose mmWave-ocPID, a person identification (PID) method with millimeter-wave radar to identify individuals even when they are heavily occluded by obstacles. We collect a multi-modal dataset comprising mmWave radar point clouds and RGB images obtained from 9 human subjects, with over 180,000 frames for each modality. The mmWave-ocPID prototype employs a novel Neural Network integrated with two augmentation strategies for learning. Our initial experimental results show that mmWave-ocPID can achieve high identification accuracy, even when most of the human body of an individual is occluded in a controlled environment.

Deep neural networks (DNNs) are becoming extremely popular in video analytics applications at the edge. However, compute-intensive DNNs pose new challenges to achieve energy-efficient DNN inference on resource-constrained edge devices. In this paper, we propose E4, an energy-efficient DNN inference framework for edge video analytics. First, E4 analyzes video frame complexity by employing an attention-based cascade module that automatically determines DNN exit points. Second, E4's just-in-time (JIT) profiler leverages coordinate descent search to co-optimize the CPU and GPU clock frequencies for each layer before the DNN exit point. Preliminary experimental results show that E4 outperforms exiting methods in terms of power consumption and inference latency.

Many intelligent applications based on deep neural networks (DNNs) are increasingly running on Internet of Things (IoT) devices. Unfortunately, the computing resources of these IoT devices are limited, which will seriously hinder the widespread deployment of various smart applications. A popular solution is to offload part of computation tasks from the IoT device to cloud by way of device–cloud collaboration. However, existing collaboration approaches may suffer from long network transmission delay or degraded accuracy due to the large amount of intermediate results, bring enormous challenges to the tasks, such as object detection, that require massive computing resources. In this article, we propose an efficient device–cloud collaborative inference (DCCI) object detection framework, which dynamically adjusts the amount of transferred data according to the content of input images. Specifically, a content-aware hard-case discriminator is proposed to automatically classify the input images as hard-cases or simple-cases, the hard-cases are uploaded to the cloud to be processed by a deployed heavyweight model, and the simple cases are processed by a lightweight model deployed to the IoT device, where the lightweight model is automatically compressed based on reinforcement learning according to the resource constraints of the IoT device. Furthermore, a collaborative scheduler based on the runtime load and network transmission capability of IoT devices is proposed to optimize the collaborative computation between IoT devices and the cloud. Extensive experimental evaluations show that compared to the Device-only approach, DCCI can reduce the memory footprint and compute resources of IoT devices by more than 90.0% and 30.87%, respectively. Compared to Cloud-centric, DCCI can save$2.0\times $of network bandwidth. In addition, compared with the state-of-the-art DNN partitioning method, DCCI can save$1.2\times $of inference latency, and$1.3\times $of IoT device energy consumption with the same accuracy constraint.

With the rapid growth of the number of devices generating and collecting data, dispersion becomes an important feature of data in Internet of Things. Federated learning (FL) provides a feasible way to mine information in such distributed data. It involves training machine learning models over multiple distributed participants without raw data transmission. However, due to the data heterogeneity among participants, the performance of the FL model degrades dramatically. Currently, improved methods mainly reduce data heterogeneity from the perspective of modifying the process of model training, which usually have problems, such as high-resource consumption or the need for auxiliary data. In this article, we enhance FL model from another perspective, focusing on data rather than model training. We reduce data heterogeneity by enhancing the trained local data to improve FL performance. Specifically, we propose an FL method based on data augmentation (abbreviated as FedM-UNE), implementing the classic data augmentation method MixUp in federated scenarios without transferring raw data. Furthermore, in order to adapt this method to regression tasks, we first modify MixUp by bilateral neighborhood expansion (MixUp-BNE), and then propose a federated data augmentation method named FedM-BNE based on it. Compared with the conventional FL method, both FedM-UNE and FedM-BNE increase negligible overhead. To demonstrate the effectiveness, we conduct exhaustive experiments on six data sets employing a variety of loss functions. The results indicate that FedM-UNE and FedM-BNE consistently improve the performance of the FL model. Moreover, our methods are compatible with existing FL enhancements, which yield further improvements in performance.

Federated learning (FL) training global machine learning models over distributed edge devices has attracted sustained attentions. However, the heterogeneity of client data severely degrades the performance of FL compared with that in centralized training. On the one hand, it slows down or even stalls global updates, leading to inefficient communication. On the other hand, it enlarges the distances between local models, resulting in an aggregated global model with poor performance. Fortunately, these shortcomings can be mitigated by reducing the angle between the directions in which a local model move. Based on this observation, we propose FedCos, which reduces the directional inconsistency of local models by introducing a cosine-similarity penalty. It promotes local model iterations toward an auxiliary global direction. Moreover, our approach is auto-adapted to various non-identically and independently distributed (IID) settings without an elaborate selection of hyperparameters. Experimental results on both vision and language tasks with a variety of models (including CNN, ResNet, LSTM, etc.) show that FedCos outperforms the well-known baselines and can enhance them under a variety of FL scenes, including varying degrees of data heterogeneity, different number of participants, and cross-silo and cross-device settings. Besides, FedCos improves the communication efficiency by 2–5 times. With the help of FedCos, multiple FL methods require significantly fewer communication rounds than before to obtain a comparable model.

To realize the grand vision of automated driving in smart vehicle cyber-physical systems (CPS), one important task is to support the merging of connected automated vehicles (CAVs) from a metered-ramp to highway. Certain safety rules must be guaranteed. However, this demand is complicated by the inherently unreliable wireless communications. In this article, we focus on the well adopted constant-time-headway (CTH) safety rule. We propose a highway and metered-ramp CAV collaborative merging protocol, and formally prove its guarantee of the CTH safety and liveness under arbitrary wireless data packet losses . These theoretical claims are further validated by our simulations. Furthermore, the simulation results also show significant improvements in the merging efficiency over other solution alternatives. Particularly, the merging success rates are more than 99% better in 11 out of 18 comparison pairs, and 0% (i.e., tied) ∼ 71% better in the remaining 7 comparison pairs.

Precipitation nowcasting, the high-resolution forecasting of precipitation in a short term, is essential in various applications in the real world. Previous deep learning methods use huge samples to learn potential laws, and the learning process lacks regularity, making it difficult to model the complex nonlinear precipitation phenomenon. Inspired by traditional numerical weather prediction models, we propose the MultiModal RNN (MM-RNN), which introduces knowledge of elements to guide precipitation prediction. This constraint forces the movement of precipitation to follow the underlying atmospheric motion laws. MM-RNN not only can provide accurate precipitation nowcasting but other meteorological elements predictions. Besides, it has high flexibility and is compatible with multiple RNN models, such as ConvLSTM, PredRNN, MIM, MotionRNN, etc. We conduct experiments on two multimodal datasets (MeteoNet and RAIN-F) and the results indicate that MM-RNN is superior to common RNN (MultiScale RNN, MS-RNN) using a single radar modality. For the MeteoNet, compared to MS-MotionRNN, the CSI (R ⩾ 10) of MM-MotionRNN increases by 23.4%, and the MSE of MM-MotionRNN decreases by 6.7%. For the RAIN-F, compared to MS-MIM, the HSS (R ⩾ 5) of MM-MIM increases by 209.4%, and the B-MSE of MM-MIM decreases by 4.6%.

Graph embeddings have been widely used for many graph analysis tasks. Mainstream factorization-based and graph-sampling-based embedding learning schemes both involve many hyperparameters and design choices. However, existing techniques often adopt some heuristics for these hyperparameters and design choices with little investigation into their impact, making it unclear what is the exact performance gains of these techniques on graph analysis tasks. Against this background, this paper presents a systematic study on the impact of an extensive list of hyperparameters for both factorization-based and graph-sampling-based graph embedding techniques for homogeneous graphs. We design generalized factorization-based and graph-sampling-based techniques involving these hyperparameters, and conduct a comprehensive set of experiments with over 3,000 embedding models trained and evaluated per dataset. We reveal that much of the performance gains are indeed due to optimal hyperparameter settings/design choices rather than the sophistication of embedding models; appropriate hyperparameter settings for typical embedding techniques can outperform a sizeable collection of 18 state-of-the-art graph embedding techniques by 0.30-35.41% across different tasks. Moreover, we find that there is no one-size-fits-all hyperparameter setting across tasks, but we can indeed provide a list of task-specific practical recommendations for these hyperparameter settings/design choices, which we believe can serve as important guidelines for future research on embedding based graph analyses.

The appearance of Mobile Edge Computing (MEC) successfully solves the bottlenecks of traditional Cloud based networks. Since mobile edges, e.g., base stations, and mobile devices have certain data processing capabilities, it is not necessary to offload all the tasks to the cloud for handling. Therefore, it is quite important to decide the optimal task assignment in MEC systems, and a series of algorithms have been proposed. However, the existing algorithms ignored the data distribution during task assignment, so that their applied ranges are quite limit. Considering the data sharing is quite important in a MEC system, this paper studies task assignment algorithms in Data Shared Mobile Edge Computing Systems in detail. Specifically, three algorithms are proposed to deal with the unordered and ordered holistic tasks respectively. Meanwhile, the situation that the tasks are divisible is also considered, and two algorithms for rearranging the divisible tasks are proposed for different optimization goals. The hardness of the problem, the correctness, complexities, and ratio bounds of the proposed algorithms are analyzed theoretically. Finally, extensive experimental results are carried out. Both theoretical analysis and experiment results show that all the proposed algorithms have high performance in terms of latency, satisfied rate, and energy consumption.

Deep neural networks have demonstrated prominent capacities for image classification tasks in a closed set setting, where the test data come from the same distribution as the training data. However, in a more realistic open set scenario, traditional classifiers with incomplete knowledge cannot tackle test data that are not from the training classes. Open set recognition (OSR) aims to address this problem by both identifying unknown classes and distinguishing known classes simultaneously. In this paper, we propose a novel approach to OSR that is based on the vision transformer (ViT) technique. Specifically, our approach employs two separate training stages. First, a ViT model is trained to perform closed set classification. Then, an additional detection head is attached to the embedded features extracted by the ViT, trained to force the representations of known data to class-specific clusters compactly. Test examples are identified as known or unknown based on their distance to the cluster centers. To the best of our knowledge, this is the first time to leverage ViT for the purpose of OSR, and our extensive evaluation against several OSR benchmark datasets reveals that our approach significantly outperforms other baseline methods and obtains new state-of-the-art performance.

Variations of stochastic gradient decedent (SGD) methods are at the core of training deep neural network models. However, in distributed deep learning, where multiple computing devices and data segments are employed in the training process, the performance of SGD can be significantly limited by the overhead of gradient communication. Local SGD methods are designed to overcome this bottleneck by averaging individual gradients trained over parallel workers after multiple local iterations. Currently, both for theoretical analyses and for practical applications, most studies employ periodic synchronization scheme by default, while few of them focus on the aperiodic schemes to obtain better performance models with limited computation and communication overhead. In this paper, we investigate local SGD with an arbitrary synchronization scheme to answer two questions: (1) Is the periodic synchronization scheme best? (2) If not, what is the optimal one? First, for any synchronization scheme, we derive the performance boundary with fixed overhead, and formulate the performance optimization under given computation and communication constraints. Then we find a succinct property of the optimal scheme that the local iteration number decreases as training continues, which indicates the periodic one is suboptimal. Furthermore, with some reasonable approximations, we obtain an explicit form of the optimal scheme and propose Aperiodic Local SGD (ALSGD) as an improved substitute for local SGD without any overhead increment. Our experiments also confirm that with the same computation and communication overhead, ALSGD outperforms local SGD in performance, especially for heterogeneous data.

With the development of embedded systems and Cross-Technology Communication (CTC) techniques, high throughput communication among heterogeneous IoT devices in the same frequency band (ISM band) can be achieved, which provides opportunities to en-hance the coexistence and cooperation for heterogeneous IoT de-vices. However, such improvement on the throughput is limited, since parallel communication has not been considered by most of the existing CTC techniques. There still exists unavoidable distortion in the reliability of the existing CTC techniques because of the heterogeneous properties of the protocols, hardware, and operating systems. Therefore, to enhance the communication throughput among heterogeneous IoT devices as much as possible, we study the parallel physical-layer CTC (PCTC) in this paper. We propose two advanced physical-layer CTCs. The first one improves the communication reliability between two heterogeneous IoT devices by retrieving the candidate emulation frames with high quality, and the other orthogonalizes the above candidate frames to achieve concurrent transmission. PCTC is designed for WiFi to ZigBee communication, and it is also implemented in USRP B210, which can improve the reliability and concurrency of the physical-layer CTC. Both theoretical analysis and experiment results verify its trans-mission reliability and improvement of throughput by comparing our technique with the existing ones.

With advancement of recent network and chip technologies, IoT devices are becoming smarter with increasing compute power, bandwidth, and storage available on the device. This enables intelligent decision making and information transferring on the devices and unleashes the power of AIoT (Artificial Intelligence of Things) that supports applications such as smart city/agriculture/manufacturing/health care and self-driving scenarios.

We present a containerized mobile sensing simulation (CMOS) framework developed for smart agriculture applications. This framework includes 1) 3D environment and object modeling, 2) mobile platform motion planning and control, and 3) optical sensing simulation, all implemented and connected within containers. Specifically, we build a user-friendly interface for 3D modeling, e.g., cornfield modeling using Blender. We use an unmanned aerial vehicle (UAV) as our mobile sensing platform and integrate UAV 3D model, flight path planning and control with robot operating system (ROS) packages and the Gazebo simulator. We also implemented optical sensing, e.g., collecting RGB image data from cameras in our simulation framework. This framework can be used not only in leaf area index correction and other analytical support for agriculture operations, but also as a synthetic data annotation tool for leaf segmentation and other smart agriculture applications. We demonstrate the major components of the CMOS framework, and how to use it to automatically annotate image data for the leaf segmentation application.

Accurate and timely nowcasting precipitation has huge social and economic benefits. However, the changes of clouds including expansion, dissipation, and distortion are extremely complex, which exacerbates the difficulty of forecasting. Fortunately, it still follows certain meteorological laws, which can be explored based on spatiotemporal information but are not fully considered by previous models. In this paper, we design two modules to focus on these messages based on atmospheric characteristics. Specifically, the SLAM (Spatial Local Attention Memory) module combines local attention and memory mechanism to capture the meteorological spatial relationship, while the TDM (Time Difference Memory) module combines differential technology and memory mechanism to capture the meteorological temporal variants. We combine these two modules with PredRNN and propose PrecipLSTM to sufficiently capture the spatiotemporal dependencies of radar data. We do exhaustive experiments with five baselines on four radar datasets. It is verified that PrecipLSTM achieves state-of-the-art results with fewer parameters than the previous state-of-the-art method.

The cybertwin-driven 6G that can obtain static and dynamic data stream of users provide an exciting potential for a novel muscular human cybertwin beyond traditonally used artificial neural networks (ANNs) and musculoskeletal models (MSMs). In this article, we propose the conceptual design of the muscular human cybertwin and construct a baseline model with an improved generalization ability over ANN and an easier adaptation to new data distributions over MSMs. In particular, we for the first time propose to combine ANN and MSM, which benefits from the combination of learning-based approaches and analytical approaches. We then experimentally compare different manners of the combination and demonstrate the better combining manner on our testing case. Finally, we evaluate our method on an open-sourced dataset and on data from wearable sensors from the aspects of joint moment prediction accuracy, data efficiency, generalization ability, and time efficiency of personalization. Our proposed method achieves accuracy similar with ANN and over 30$\%$better than MSM with sufficient training data. Compared with ANN, the improved data efficiency is presented by the better accuracies with a small amount of training data, and the generalization ability to unseen walking conditions and new subjects are demonstrated by the over 70$\%$accuracy improvements. Moreover, when fine-tuning the model, our algorithm is demonstrated by the time 75$\%$shorter than calibrated MSM and the accuracy improvements.

With the widespread of Human-Cyber-Physical Systems (HCPS), the fine-grained human movement detection becomes more and more important. Especially for the slightly motions of human’s hands, they are not only bring abundant information, but also provide a new way for the interaction between users and systems. In this paper, we focus on the problem of how to detect the human’s typing motion, and designed a new system, named as Typing Wristband, to obtain the vibration of wrist using piezoelectric transducer (PZT). Then, a robust denoising, event detection and classification algorithms are proposed to deal with the signal collected by Typing Wristband and detect the typing motions. Typing Wristband can recognized the movements of 3 fingers and 9 keys with high accuracy. Furthermore, it is very cheap and can be embedded into existing smart devices, e.g. a smart watch, so that it supports the wireless sensing very well in practice. Both of the analysis and experimental results verify that our Typing Wristband has the better performance in terms of accuracy and convenience.

Advances in Edge AI make it possible to achieve inference deep learning for emerging applications, e.g., smart transportation and smart city on the edge in real-time. Nowadays, different industry companies have developed several edge AI devices with various architectures. However, it is hard for application users to justify how to choose the appropriate edge-AI, due to the lack of benchmark testing results and testbeds specifically used to evaluate the system performance for those edge-AI systems. In this paper, we attempt to design a benchmark test platform for the edge-AI devices and evaluate six mainstream edge devices that are equipped with different computing powers and AI chip architectures. Throughput, power consumption ratio, and cost-effectiveness are chosen as the performance metrics for the evaluation process. Three classic deep learning workloads: object detection, image classification, and natural language processing are adopted with different batch sizes. The results show that under different batch sizes, compared with traditional edge devices, edge devices equipped with AI chips have out-performance in throughput, power consumption ratio, and cost-effectiveness by 134×, 57×, and 32×, respectively. From system perspective, our work not only demonstrates the effective AI capabilities of those edge AI devices, but also provide suggestions for AI optimization at edge in details.

The surface electromyography (sEMG) signal-based human-machine interface (HMI) has been widely used for various scenarios of physical human-robot interaction. However, current HMIs based on bipolar myoelectric sensors are hindered by the limitations of global sEMG features, which are prone to variability and delay. In this letter, we define a HMI that takes advantage of the underlying neural information of spinal module activations from bipolar sEMG signals, inspired by recent findings of neural codes. Firstly, the spinal module activations are identified by the spiking trains of the muscle synergies extracted from bipolar sEMG signals. Secondly, we extract the information encoded in both firing rates and spike timings of the spinal module activation in a population coding manner, which follows the information encoding principle of neurons. Thirdly, we map the series of spinal module activations into gait phases, locomotion modes, joint moment and human identity in order to experimentally reveal the physiological information contained in the spinal module activations. The contained information and the benefit of our design are demonstrated and experimentally explained by the presented results and comparisons with the traditionally used global sEMG features. The proposed bipolar myoelectric sensor-enabled human-machine interface could contribute to various scenarios of physical human-robot interaction.

Human Activity Recognition (HAR) refers to recognizing a human's ongoing actions through sensor data. At present, one of the main problems faced by Human Activity Recognition is that different subjects, devices and wearing positions can cause inconsistent sensor data distribution. When a classification model trained using some labeled dataset is used to classify a new unlabeled data with different distributions, there will be a significant performance loss. However, it is difficult to annotate manually sensor data for new subjects. Prior works applying unsupervised domain adaptation methods to solve this problem only used a single source domain. However, in practice, it is common to have multiple labeled source domains. Inspired by a work in the field of computer vision, we propose an unsupervised domain adaptation method for human activity recognition using multiple source domains. Experimental results on a commonly used public HAR dataset show that our model can effectively alleviate the performance loss caused by inconsistent distributions. Moreover, compared with the single-source domain adaptation, the multi-source domain adaptation method can improve the accuracy further.

With advancement of recent network and chip technologies, IoT devices are becoming smarter with increasing compute power, bandwidth, and storage available on the device. This enables intelligent decision making and information transferring on the devices and unleashes the power of AIoT (Artificial Intelligence of Things) that supports scenarios such as smart city/agriculture/manufacturing/health care and self-driving scenarios. The AIoT Workshop is a forum for researchers, scientists, engineers, and practitioners to share and learn AI powered IoT solutions. The AIoT is a multi-disciplinary area, which include but not limited to IoT, AI/ML, embedded systems, and networking. The 4th AIoT workshop will be hosted virtually in conjunction with the 27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2021). The workshop program consists of keynote(s), invited talks, accepted technical paper presentations, as well as an indoor location competition panel.

Recent advances in both lightweight models and edge computing make it possible for inference tasks to be executed concurrently on resource-constrained edge devices. However, our preliminary experiments show that the execution of different lightweight models on edge devices may lead to a performance downgrade. In this paper, we propose a Learning-Based Scheduling Framework---ECSRL, to optimize the latency and power consumption for those inference tasks running in heterogeneous Edge-Cloud systems.

Nowadays, Deep Learning-as-a-Service can be de-ployed in the Internet of Things (IoT) to provide smart services and sensor data processing. However, recent research has re-vealed that some Deep Neural Networks (DNN) can be easily misled by adding relatively small but adversarial perturbations to the input (e.g., pixel mutation in input images). One challenge in defending DNN against these attacks is to efficiently identify and filtering out the adversarial pixels. The state-of-the-art defense strategies with good robustness often require additional model training for specific attacks. To reduce the computational cost without loss of generality, we present a defense strategy called a progressive defense against adversarial attacks (PDAAA) for efficiently and effectively filtering out the adversarial pixel mutations, which could mislead the neural network towards erro-neous outputs, without a-priori knowledge about the attack type. We evaluated our progressive defense strategy against various attack methods on two well-known datasets. Experimental result shows it outperforms the state-of-the-art methods(Adversarial-PGD, Adversarial-Network, and Adversarial-Dual-Network) with dramatically reduced computation cost.

Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input pixels. These pixels have to be filtered out for defending DNN that will cause lots of computations. To reduce the computation, in this paper, the pixels that their slight change will cause the neural network to have a wrong result for prediction are found out and filtered out for defense goal. Experiment results show that our defense method has achieved about 85% defense success rate with filtering out 50 mutation pixels.

Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In order to overcome these challenges, this paper proposed a training-free behavior cloning algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC transforms the Model Predictive Control (MPC) controller into a PieceWise Affine (PWA) function with multi-parametric programming, and uses a neural network to express this function. By this way, off-the-shelf deep reinforcement learning algorithms can be used to fine-tune this neural network. The experiments show that our method can help agent learn at the high reward state region, and converge faster and better.

As the scale and density of data centers continue to grow, cost-effective data center management (DCM) is becoming a significant challenge for enterprises hosting large-scale online and cloud services. Machines need to be monitored, and the scale of operations mandates an automated management with high reliability and real-time performance. The limitations of today’s typical DCM network are many-fold. Primarily, it is a fixed wired network, and hence scaling it for a large number of servers increases its cost. In addition, with server densities increasing over recent years, this network also has to be cabled correctly and the management of this network parallels the complexity of managing a data network, since it needs to be networked with multiple switches and routers. In this article, we propose a wireless sensor network as a cost-effective networking solution for DCM while satisfying the reliability and latency performance requirements of DCM. We have developed CapNet, a real-time wireless sensor network for power capping, a time-critical DCM function for power management in a cluster of servers. CapNet employs an efficient event-driven protocol that triggers data collection only on the detection of a potential power capping event. We deploy and evaluate CapNet in a data center. Using server power traces, our experimental results on a cluster of 480 servers inside the data center show that CapNet can meet the real-time requirements of power capping. CapNet demonstrates the feasibility and efficacy of wireless sensor networks for time-critical DCM applications.

We study the problem of representing and recommending products for grocery shopping. We carefully investigate grocery transaction data and observe three important patterns: products within the same basket complement each other in terms of functionality (complementarity); users tend to purchase products that match their preferences (compatibility); and a significant fraction of users repeatedly purchase the same products over time (loyalty). Unlike conventional e-commerce settings, complementarity and loyalty are particularly predominant in the grocery shopping domain. This motivates a new representation learning approach to leverage complementarity and compatibility holistically, as well as a new recommendation approach to explicitly account for users' 'must-buy' purchases in addition to their overall preferences and needs. Doing so not only improves product classification and recommendation performance on both public and proprietary transaction data covering various grocery store types, but also reveals interesting findings about the relationships between preferences, necessity, and loyalty in consumer purchases.

Decades went by since Mark Weiser published his influential work on how a computer of the 21st century would look like. Over the years, some of the UbiComp features presented in that paper have been gradually adopted by industry players in the technology market. While this technological evolution resulted in many benefits to our society, it has also posed, along the way, countless challenges that we have yet to surpass. In this paper, we address major challenges from two areas that most afflict the UbiComp revolution: security and privacy. We examine open problems on software protection, long-term security, cryptography engineering, and privacy implications. We also point out promising directions towards the solutions of those problems. We claim that if we get all this right, we will turn the science fiction of UbiComp into science fact.

Cloud datacenters must ensure high availability for the hosted applications and failures can be the bane of datacenter operators. Understanding the what, when and why of failures can help tremendously to mitigate their occurrence and impact. Failures can, however, depend on numerous spatial and temporal factors spanning hardware, workloads, support facilities, and even the environment. One has to rely on failure data from the field to quantify the influence of these factors on failures. Towards this goal, we collect failures data along with many parameters that might influence failures from two large production datacenters with very diverse characteristics. We show that multiple factors simultaneously affect failures, and these factors may interact in non-trivial ways. This makes conventional approaches that study aggregate characteristics or single parameter influences, rather inaccurate. Instead, we build a multi-factor analysis framework to systematically identify influencing factors, quantify their relative impact, and help in more accurate decision making for failure mitigation. We demonstrate this approach for three important decisions: spare capacity provisioning, comparing the reliability of hardware for vendor selection, and quantifying flexibility in datacenter climate control for cost-reliability trade-offs.

We consider the problem of continuous computer-vision based analysis of video streams from mobile cameras over extended periods. Given high computational demands, general visual processing must currently be offloaded to the cloud. To reduce mobile battery and bandwidth consumption, recent proposals offload only "interesting" video frames, discarding the rest. However, determining what to discard is itself typically a power-hungry computer vision calculation, very often well beyond what most mobile devices can afford on a continuous basis. We present the Glimpse system, a re-design of the conventional mobile video processing pipeline to support such "early discard" flexibly, efficiently and accurately. Glimpse is a novel architecture that gates wearable vision using low-power vision modalities. Our proposed architecture adds novel sensing, processing, algorithmic and programming-system components to the camera pipeline to this end. We present a complete implementation and evaluation of our design. In common settings, Glimpse reduces mobile power and data usage by more than one order of magnitude relative to earlier designs, and moves continuous vision on lightweight wearables to the realm of the practical.

Low-Power Wide-Area Network (LPWAN) heralds a promising class of technology to overcome the range limits and scalability challenges in traditional wireless sensor networks. Recently proposed Sensor Network over White Spaces (SNOW) technology is particularly attractive due to the availability and advantages of TV spectrum in long-range communication. This paper proposes a new design of SNOW that is asynchronous, reliable, and robust. It represents the first highly scalable LPWAN over TV white spaces to support reliable, asynchronous, bi-directional, and concurrent communication between numerous sensors and a base station. This is achieved through a set of novel techniques. This new design of SNOW has an OFDM based physical layer that adopts robust modulation scheme and allows the base station using a single antenna-radio (1) to send different data to different nodes concurrently and (2) to receive concurrent transmissions made by the sensor nodes asynchronously. It has a lightweight MAC protocol that (1) efficiently implements per-transmission acknowledgments of the asynchronous transmissions by exploiting the adopted OFDM design; (2) combines CSMA/CA and location-aware spectrum allocation for mitigating hidden terminal effects, thus enhancing the flexibility of the nodes in transmitting asynchronously. Hardware experiments through deployments in three radio environments - in a large metropolitan city, in a rural area, and in an indoor environment - as well as large-scale simulations demonstrated that the new SNOW design drastically outperforms other LPWAN technologies in terms of scalability, energy, and latency.

It is paramount to ensure secure and trustworthy operations in Cyber-Physical Systems (CPSs), guaranteeing the integrity of sensing data, enabling access control, and safeguarding system-level operations. In this paper, we address trustworthy operations of next generation CPSs. Our idea is inspired by a trustworthy computing framework known as Proof-Carrying Code, in which foreign executables carry a model to prove that they have not been tampered with and they function as expected. In our context, we leverage the physical world--a channel that encapsulates properties impossible to tamper with remotely, such as proximity and causality--to create a challenge-response function. We call it Proof-Carrying Sensing and use it to help authenticate devices, collected data, and locations. A unique advantage of this approach, vis-à-vis traditional multi-factor or out-of-band authentication mechanisms, is that authentication proofs are embedded in sensor data and can be continuously validated over time and space without resorting to complicated cryptographic algorithms. This, in turn, makes it fit particularly well to CPSs where mobility and resource constraints are common.

In order to match shoppers with desired products and provide personalized promotions, whether in online or offline shopping worlds, it is critical to model both consumer preferences and price sensitivities simultaneously. Personalized preferences have been thoroughly studied in the field of recommender systems, though price (and price sensitivity) has received relatively little attention. At the same time, price sensitivity has been richly explored in the area of economics, though typically not in the context of developing scalable, working systems to generate recommendations. In this study, we seek to bridge the gap between large-scale recommender systems and established consumer theories from economics, and propose a nested feature-based matrix factorization framework to model both preferences and price sensitivities. Quantitative and qualitative results indicate the proposed personalized, interpretable and scalable framework is capable of providing satisfying recommendations (on two datasets of grocery transactions) and can be applied to obtain economic insights into consumer behavior.

Fuel cells are a promising power source for future data centers, offering high energy efficiency, low greenhouse gas emissions, and high reliability. However, due to mechanical limitations related to fuel delivery, fuel cells are slow to adjust to sudden increases in data center power demands, which can result in temporary power shortfalls. To mitigate the impact of power shortfalls, prior work has proposed to either perform power capping by throttling the servers, or to leverage energy storage devices (ESDs) that can temporarily provide enough power to make up for the shortfall while the fuel cells ramp up power generation. Both approaches have disadvantages: power capping conservatively limits server performance and can lead to service level agreement (SLA) violations, while ESD-only solutions must significantly overprovision the energy storage device capacity to tolerate the shortfalls caused by the worst-case (i.e., largest) power surges, which greatly increases the total cost of ownership (TCO). We propose SizeCap, the first ESD sizing framework for fuel cell powered data centers, which coordinates ESD sizing with power capping to enable a cost-effective solution to power shortfalls in data centers. SizeCap sizes the ESD just large enough to cover the majority of power surges, but not the worst-case surges that occur infrequently, to greatly reduce TCO. It then uses the smaller capacity ESD in conjunction with power capping to cover the power shortfalls caused by the worst-case power surges. As part of our new flexible framework, we propose multiple power capping policies with different degrees of awareness of fuel cell and workload behavior, and evaluate their impact on workload performance and ESD size. Using traces from Microsoft's production data center systems, we demonstrate that SizeCap significantly reduces the ESD size (by 85%ofor a workload with infrequent yet large power surges, and by 50% for a workload with frequent power surges) without violating any SLAs.

We present new applications for cryptographic secret handshakes between mobile devices on top of Bluetooth Low-Energy (LE). Secret handshakes enable mutual authentication, with the property that the parties learn nothing about each other unless they have been both issued credentials by a group administrator. This property provides strong privacy guarantees that enable interesting applications. One of them is proximity-based discovery for private communities. We introduce MASHaBLE, a mobile application that enables participants to discover and interact with nearby users if and only if they belong to the same secret community. We use direct peer-to-peer communication over Bluetooth LE, rather than relying on a central server. We discuss the specifics of implementing secret handshakes over Bluetooth LE and present our prototype implementation.

Wireless sensor networks (WSNs) face significant scalability challenges due to the proliferation of wide-area wireless monitoring and control systems that require thousands of sensors to be connected over long distances. Due to their short communication range, existing WSN technologies such as those based on IEEE 802.15.4 form many-hop mesh networks complicating the protocol design and network deployment. To address this limitation, we propose a scalable sensor network architecture - called Sensor Network Over White Spaces (SNOW) - by exploiting the TV white spaces. Many WSN applications need low data rate, low power operation, and scalability in terms of geographic areas and the number of nodes. The long communication range of white space radios significantly increases the chances of packet collision at the base station. We achieve scalability and energy efficiency by splitting channels into narrowband orthogonal subcarriers and enabling packet receptions on the subcarriers in parallel with a single radio. The physical layer of SNOW is designed through a distributed implementation of OFDM that enables distinct orthogonal signals from distributed nodes. Its MAC protocol handles subcarrier allocation among the nodes and transmission scheduling. We implement SNOW in GNU radio using USRP devices. Experiments demonstrate that it can correctly decode in less than 0.1ms multiple packets received in parallel at different subcarriers, thus drastically enhancing the scalability of WSN.

Despite the growing popularity of Solid State Disks (SSDs) in the datacenter, little is known about their reliability characteristics in the field. The little knowledge is mainly vendor supplied, which cannot really help understand how SSD failures can manifest and impact production systems, in order to take appropriate actions. Besides failure data, a detailed characterization requires wide spectrum of data about factors influencing SSD failures, right from provisioning (what models' where and when deployed' etc.) to the operational ones (workloads, read-write intensities, write amplification, etc.). We analyze over half a million SSDs that span multiple generations spread across several datacenters which host a wide range of workloads over nearly 3 years. By studying the diverse set of factors on SSD failures, and their symptoms, our work provides the first look at the what, when and why characteristics of SSD failures in production datacenters.

Despite the growing popularity of Solid State Disks (SSDs) in the datacenter, little is known about their reliability characteristics in the field. The little knowledge is mainly vendor supplied, and such information cannot really help understand how SSD failures can manifest and impact the operation of production systems, in order to take appropriate remedial measures. Besides actual failure data and the symptoms exhibited by SSDs before failing, a detailed characterization effort requires wide set of data about factors influencing SSD failures, right from provisioning factors to the operational ones. This paper presents an extensive SSD failure characterization by analyzing a wide spectrum of data from over half a million SSDs that span multiple generations spread across several datacenters which host a wide spectrum of workloads over nearly 3 years. By studying the diverse set of design, provisioning and operational factors on failures, and their symptoms, our work provides the first comprehensive analysis of the what, when and why characteristics of SSD failures in production datacenters.

Location is a fundamental service for mobile computing. Typical GPS receivers, although widely available for navigation purposes, may consume too much energy to be useful for many applications. Observing that in many sensing scenarios, the location information can be post-processed when the data is uploaded to a server, we design a cloud-offloaded GPS (CO-GPS) solution that allows a sensing device to aggressively duty-cycle its GPS receiver and log just enough raw GPS signal for post-processing. Leveraging publicly available information such as GNSS satellite ephemeris and an Earth elevation database, a cloud service can derive good quality GPS locations from a few milliseconds of raw data. Using our design of a portable sensing device platform called CLEON, we evaluate the accuracy and efficiency of the solution. Compared to more than 30 seconds of heavy signal processing on standalone GPS receivers, we can achieve three orders of magnitude lower energy consumption per location tagging.

Supervised machine-learning algorithms are used to solve classification problems across the entire spectrum of computing platforms, from data centers to wearable devices, and place significant demand on their computational capabilities. In this paper, we propose scalable-effort classifiers, a new approach to optimizing the energy efficiency of supervised machine-learning classifiers. We observe that the inherent classification difficulty varies widely across inputs in real-world datasets; only a small fraction of the inputs truly require the full computational effort of the classifier, while the large majority can be classified correctly with very low effort. Yet, state-of-the-art classification algorithms expend equal effort on all inputs, irrespective of their difficulty. To address this inefficiency, we introduce the concept of scalable-effort classifiers, or classifiers that dynamically adjust their computational effort depending on the difficulty of the input data, while maintaining the same level of accuracy. Scalable effort classifiers are constructed by utilizing a chain of classifiers with increasing levels of complexity (and accuracy). Scalable effort execution is achieved by modulating the number of stages used for classifying a given input. Every stage in the chain contains an ensemble of biased classifiers, where each biased classifier is trained to detect a single class more accurately. The degree of consensus between the biased classifiers' outputs is used to decide whether classification can be terminated at the current stage or not. Our methodology thus allows us to transform any given classification algorithm into a scalable-effort chain. We build scalable-effort versions of 8 popular recognition applications using 3 different classification algorithms. Our experiments demonstrate that scalable-effort classifiers yield 2.79x reduction in average operations per input, which translates to 2.3x and 1.5x improvement in energy for hardware and software implementations, respectively.

We present the results, experiences and lessons learned from comparing a diverse set of technical approaches to indoor localization during the 2014 Microsoft Indoor Localization Competition. 22 different solutions to indoor localization from different teams around the world were put to test in the same unfamiliar space over the course of 2 days, allowing us to directly compare the accuracy and overhead of various technologies. In this paper, we provide a detailed analysis of the evaluation study's results, discuss the current state-of-the-art in indoor localization, and highlight the areas that, based on our experience from organizing this event, need to be improved to enable the adoption of indoor location services.

Understanding indoor mobility patterns of people is important in applications such as targeted advertisement, microclimate control, and delivery of anticipatory notifications. In this article, we devise GreenLocs, a nonparametric, profiling-free, yet lightweight and energy-efficient inference framework, to identify recurring and new places that mobile users visit indoor. Combining WiFi scans and accelerometer readings, GreenLocs can accurately decide a new place and a revisited place with just a few radio signal strength (RSS) samples. GreenLocs consists of three major building blocks, namely, missing data handling algorithms, a nonparametric Bayesian inference model, and a stopping rule, which significantly increases the energy efficiency of the system. GreenLocs is shown to be robust to signal variations and missing data through experimental evaluations using traces collected from mobile phones of different brands/models.

While there has been prior work to underprovision the power distribution infrastructure for a datacenter to save costs, the ability to underprovision the backup power infrastructure, which contributes significantly to capital costs, is little explored. There are two main components in the backup infrastructure - Diesel Generators (DGs) and UPS units - which can both be underprovisioned (or even removed) in terms of their power and/or energy capacities. However, embarking on such underprovisioning mandates studying several ramifications - the resulting cost savings, the lower availability, and the performance and state loss consequences on individual applications - concurrently. This paper presents the first such study, considering cost, availability, performance and application consequences of underprovisioning the backup power infrastructure. We present a framework to quantify the cost of backup capacity that is provisioned, and implement techniques leveraging existing software and hardware mechanisms to provide as seamless an operation as possible for an application within the provisioned backup capacity during a power outage. We evaluate the cost-performance-availability trade-offs for different levels of backup underprovisioning for applications with diverse reliance on the backup infrastructure. Our results show that one may be able to completely do away with DGs, compensating for it with additional UPS energy capacities, to significantly cut costs and still be able to handle power outages lasting as high as 40 minutes (which constitute bulk of the outages). Further, we can push the limits of outage duration that can be handled in a cost-effective manner, if applications are willing to tolerate degraded performance during the outage. Our evaluations also show that different applications react differently to the outage handling mechanisms, and that the efficacy of the mechanisms is sensitive to the outage duration. The insights from this paper can spur new opportunities for future work on backup power infrastructure optimization.

Memory devices represent a key component of datacenter total cost of ownership (TCO), and techniques used to reduce errors that occur on these devices increase this cost. Existing approaches to providing reliability for memory devices pessimistically treat all data as equally vulnerable to memory errors. Our key insight is that there exists a diverse spectrum of tolerance to memory errors in new data-intensive applications, and that traditional one-size-fits-all memory reliability techniques are inefficient in terms of cost. For example, we found that while traditional error protection increases memory system cost by 12.5%, some applications can achieve 99.00% availability on a single server with a large number of memory errors without any error protection. This presents an opportunity to greatly reduce server hardware cost by provisioning the right amount of memory reliability for different applications. Toward this end, in this paper, we make three main contributions to enable highly-reliable servers at low datacenter cost. First, we develop a new methodology to quantify the tolerance of applications to memory errors. Second, using our methodology, we perform a case study of three new dataintensive workloads (an interactive web search application, an in-memory key -- value store, and a graph mining framework) to identify new insights into the nature of application memory error vulnerability. Third, based on our insights, we propose several new hardware/software heterogeneous-reliability memory system designs to lower datacenter cost while achieving high reliability and discuss their trade-off. We show that our new techniques can reduce server hardware cost by 4.7% while achieving 99.90% single server availability.

This paper presents a remote gesture input solution for interacting indirectly with user interfaces on mobile and wearable devices. The proposed solution uses a wearable ring platform worn on users index finger. The ring detects and interprets various gestures performed on any available surface, and wirelessly transmits the gestures to the remote device. The ring opportunistically harvests energy from an NFC-enabled phone for perpetual operation without explicit charging. We use a finger-tendon pressure-based solution to detect touch, and a light-weight audio based solution for detecting finger motion on a surface. The two level energy efficient classification algorithms identify 23 unique gestures that include tapping, swipes, scrolling, and strokes for hand written text entry. The classification algorithms have an average accuracy of 73% with no explicit user training. Our implementation supports 10 hours of interactions on a surface at 2 Hz gesture frequency. The prototype was built with off-the-shelf components has a size similar to a large ring.

Due to poor signal strength, multipath effects, and limited on-device computation power, common GPS receivers do not work indoors. This work addresses these challenges by using a steerable, high-gain directional antenna as the front-end of a GPS receiver along with a robust signal processing step and a novel location estimation technique to achieve direct GPS-based indoor localization. By leveraging the computing power of the cloud, we accommodate longer signals for acquisition, and remove the requirement of decoding timestamps or ephemeris data from GPS signals. We have tested our system in 31 randomly chosen spots inside five single-story, indoor environments such as stores, warehouses and shopping centers. Our experiments show that the system is capable of obtaining location fixes from 20 of these spots with a median error of less than 10 m, where all normal GPS receivers fail.

Mobile wallets promise a future where users do not need to carry physical payment cards. However, the slow adoption of contactless point of sales (POS) terminals by merchants limits the potential of Near-Field Communication (NFC) based payment devices. In this paper, we present Wild Card, a secure and backward compatible solution for making mobile payments at conventional magnetic stripe based POS terminals. Our solution resembles a traditional credit card in its physical dimensions and stays in the phone case. It can be programmatically set by an NFC-enabled mobile phone to any card number that the user owns. The key technologies that enable Wild Card are a fully programmable magnetic stripe, an energy harvesting system that allows the card to be charged and programmed by the phone through NFC, and a security mechanism that makes card information resilient to attacks on mobile devices. With a prototype, we evaluate the feasibility of Wild Card in terms of functionality and energy budget.

Data center management (DCM) is increasingly becoming a significant challenge for enterprises hosting large scale online and cloud services. Machines need to be monitored, and the scale of operations mandates an automated management with high reliability and real-time performance. Existing wired networking solutions for DCM come with high cost. In this paper, we propose a wireless sensor network as a cost-effective networking solution for DCM while satisfying the reliability and latency performance requirements of DCM. We have developed Cap Net, a real-time wireless sensor network for power capping, a time-critical DCM function for power management in a cluster of servers. Cap Net employs an efficient event-driven protocol that triggers data collection only upon the detection of a potential power capping event. We deploy and evaluate Cap Net in a data center. Using server power traces, our experimental results on a cluster of 480 servers inside the data center show that Cap Net can meet the real-time requirements of power capping. Cap Net demonstrates the feasibility and efficacy of wireless sensor networks for time-critical DCM applications.

The ever increasing popularity of social networks and the ever easier photo taking and sharing experience have led to unprecedented concerns on privacy infringement. Inspired by the fact that the Robot Exclusion Protocol, which regulates web crawlers' behavior according a per-site deployed robots.txt, and cooperative practices of major search service providers, have contributed to a healthy web search industry, in this paper, we propose Privacy Expressing and Respecting Protocol (PERP) that consists of a Privacy.tag -- a physical tag that enables a user to explicitly and flexibly express their privacy deal, and Privacy Respecting Sharing Protocol (PRSP) -- a protocol that empowers the photo service provider to exert privacy protection following users' policy expressions, to mitigate the public's privacy concern, and ultimately create a healthy photo-sharing ecosystem in the long run. We further design an exemplar Privacy.Tag using customized yet compatible QR-code, and implement the Protocol and study the technical feasibility of our proposal. Our evaluation results confirm that PERP and PRSP are indeed feasible and incur negligible computation overhead.

Network traffic prioritization is gaining attention in the WSN community, as more and more features are being integrated into sensor networks. Real-world deployment experience suggests that WSN brings new challenges to existing problems, such as resource constraints, low data-rate radios, and diverse application scenarios. We present the RushNet framework that prioritizes two common traffic patterns in multi-hop sensor networks: low-priority (LP) traffic that is large-volume but delay-tolerant, and high-priority (HP) traffic that is sporadic but latency-sensitive. RushNet achieves schedule-free and coordination-free delivery differentiations with the following features. First, RushNet works with most data collection protocols to deliver LP traffic. Second, RushNet leverages transmission power difference and radio capture effect to implement on-demand HP packet delivery with low overhead. Third, RushNet proposes a retrodiction technique to help nodes minimize the overhead of recovering LP packet loss due to concurrent HP traffic. We evaluate RushNet performance with micro-benchmarks and a crowdsourced office comfort monitoring deployment. The deployment results suggest RushNet can achieve a throughput close to network capacity, and deliver 98% of the HP packets with a latency of less than four seconds.

Local search users today decide what business to visit solely based on distance information, and business ratings that can be sparse or stale. We believe that when users search for local businesses, such as bars or restaurants, they need to know more about the ambience of each business, such as how crowded it is, how loud and of what type the music it plays is, as well as how loud the human chatter in the business is. Unfortunately, this information doesn't exist today. In this paper, we propose to automatically crowdsource such rich, local business ambience metadata through real user check-in events. Every time a user checks into a business, the phone is in user's hands, and the phone's sensors can sense the business environment. We leverage the phone's microphone during this time to infer the occupancy and human chatter levels, the music type, as well as the music and noise levels in the business. As people check-in to businesses throughout the day, business metadata can be automatically updated over time, enabling a new generation of local search experience. Using approximately 150 audio traces collected from real businesses of various types over a period of 3 months, we show that by properly extracting the temporal and frequency signatures of the audio signal, it is feasible to train models that can simultaneously infer occupancy, human chatter, music, and noise levels in a business, with higher than 79% accuracy.

The global positioning system (GPS) system is a dominant wireless technology that enables reliable location sensing for a diverse range of outdoor mobile applications. Following rising demands for location sensing, low-cost GPS receivers are becoming widely available; but their energy demands are still too high to be useful for many of these applications. For energy efficient GPS sensing, the possibility of offloading a few milliseconds of raw signal samples and leveraging the greater processing power of the cloud for obtaining a position fix is being actively investigated. In an attempt to reduce the energy cost of this data offloading operation, we propose SparseGPS: a lightweight GPS acquisition mechanism based on sparse approximation.

Due to poor signal-strength, multi-path effects, and a lack of adequate visible satellite vehicles (SV), GPS receivers do not work indoors. This work addresses these challenges by using a mechanically steerable, high-gain directional antenna as the front-end of the GPS receiver along with a robust signal processing technique to acquire satellites in indoor environments. Our experiment on a local warehouse shows that, the system is capable of acquiring 5 or more SVs (which is a requirement for instant GPS technique [4]), whereas a Garmin device barely sees 3, and the system is capable of estimating indoor locations with 3--18 m errors when compared to the ground truth.

The major challenge for accurate fingerprint-based indoor localization is the design of robust and discriminative wireless signatures. Even though WiFi received signal strength indicator (RSSI) signatures are widely available indoors, they vary significantly over time and are susceptible to human presence, multipath, and fading due to the high operating frequency. To overcome these limitations, we propose to use FM broadcast radio signals for robust indoor fingerprinting. Because of the lower frequency, FM signals are less susceptible to human presence, multipath, and fading, they exhibit exceptional indoor penetration, and according to our experimental study they vary less over time when compared to WiFi signals. In this paper, we demonstrate through a detailed experimental study in three different buildings across the US, that FM radio signal RSSI values can be used to achieve room-level indoor localization with similar or better accuracy to the one achieved by WiFi signals. Furthermore, we propose to use additional signal quality indicators at the physical layer (i.e., SNR, multipath, etc.) to augment the wireless signature, and show that localization accuracy can be further improved by more than 5 percent. More importantly, we experimentally demonstrate that the localization errors of FM and WiFi signals are independent. When FM and WiFi signals are combined to generate wireless fingerprints, the localization accuracy increases as much as 83 percent (when accounting for wireless signal temporal variations) compared to when WiFi RSSI only is used as a signature.

The availability of multiple sensors on mobile devices offers a significant new capability to enable rich user and context aware applications. Many of these applications run in the background to continuously sense user context. However, running these applications on mobile devices can impose a significant stress on the battery life, and the use of supplementary low-power processors has been proposed on mobile devices for continuous background activities. In this paper, we experimentally and analytically investigate the design considerations that arise in the efficient use of the low power processor and provide a thorough understanding of the problem space. We answer fundamental questions such as which segments of the application are most efficient to be hosted on the low power processor, and how to select an appropriate low power processor. We discuss our measurements, analysis, and results using multiple low power processors and existing phone platforms.

Many indoor sensing applications leverage knowledge of relative proximity among physical objects and humans, such as the notion of "within arm's reach". In this paper, we quantify this notion using "proximity zone", and propose a methodology that empirically and systematically compare the proximity zones created by various wireless technologies. We find that existing technologies such as 802.15.4, Bluetooth Low Energy (BLE), and RFID fall short on metrics such as boundary sharpness, robustness against interference, and obstacle penetration. We then present the design and evaluation of a wireless proximity detection platform based on magnetic induction - LiveSynergy. LiveSynergy provides sweet spot for indoor applications that require reliable and precise proximity detection. Finally, we present the design and evaluation of an end-to-end system, deployed inside a large food court to offer context-aware and personalized advertisements and diet suggestions at a per-counter granularity.

The maturity and availability of network protocols have enabled wireless sensor networks (WSN) designers to build heterogeneous applications by composing different protocols. A common heterogeneous application combines data collection and dissemination for environmental monitoring with node retasking. While these co-located protocols on the same node have different goals, many of them share requirements and characteristics. Examples of commonalities include the use of bi-directional traffic for reliable transmissions and tree for packet routing. This work explores how the MAC layer can reduce the network transmission overhead of heterogeneous applications by taking advantage of protocol commonalities to aggregate outgoing packets. In other words, this aggregation creates a train of packets destined to the same receiver. Finally, we discuss a strawman implementation of packet train and how our data center monitoring deployment leverages it.

The major challenge for accurate fingerprint-based indoor localization is the design of robust and discriminative wireless signatures. Even though WiFi RSSI signatures are widely available indoors, they vary significantly over time and are susceptible to human presence, multipath, and fading due to the high operating frequency. To overcome these limitations, we propose to use FM broadcast radio signals for robust indoor fingerprinting. Because of the lower frequency, FM signals are less susceptible to human presence, multipath and fading, they exhibit exceptional indoor penetration, and according to our experimental study they vary less over time when compared to WiFi signals. In this work, we demonstrate through a detailed experimental study in 3 different buildings across the US, that FM radio signal RSSI values can be used to achieve room-level indoor localization with similar or better accuracy to the one achieved by WiFi signals. Furthermore, we propose to use additional signal quality indicators at the physical layer (i.e., SNR, multipath etc.) to augment the wireless signature, and show that localization accuracy can be further improved by more than 5%. More importantly, we experimentally demonstrate that the localization errors of FM andWiFi signals are independent. When FM and WiFi signals are combined to generate wireless fingerprints, the localization accuracy increases as much as 83% (when accounting for wireless signal temporal variations) compared to when WiFi RSSI only is used as a signature.

A new technology breakthrough which allows any standard Bluetooth mobile phone to access information services using the same selective and deliberate gesture envisioned for NFC. This technology is called Bluetooth Touchpoint, and it consists of reconfigurable coverage that combines NFC-like, close-proximity communications with the long-range, roaming solution of today's Bluetooth devices. Phase shifting technology combined with a unique antenna configuration makes this realization possible. Therefore, as the world moves toward contactless, wireless communication links, it is believed that the near ubiquity of Bluetooth in mobile devices makes this technology an excellent choice for delivering the benefits of NFC today without the wait, effort and cost associated with adopting NFC globally.

As mobile apps become more closely integrated into our everyday lives, mobile app interactions ought to be rapid and responsive. Unfortunately, even the basic primitive of launching a mobile app is sorrowfully sluggish: 20 seconds of delay is not uncommon even for very popular apps.

We demonstrate the CLEO mobile sensing platform, including ultra-portable, low-power sensors, phone-based data upload software, and a location resolution and data management web service in the cloud. The sensing platform leverages cloud-offloaded GPS (CO-GPS) for location sensing. A sensor communicates with mobile devices using an audio-port-based connector. The back-end cloud service supports OData protocol to make data accessible to other systems, such as World-Wide Telescope (WWT).

Location is a fundamental service for mobile computing. Typical GPS receivers, although widely available, consume too much energy to be useful for many applications. Observing that in many sensing scenarios, the location information can be post-processed when the data is uploaded to a server, we design a Cloud-Offloaded GPS (CO-GPS) solution that allows a sensing device to aggressively duty-cycle its GPS receiver and log just enough raw GPS signal for post-processing. Leveraging publicly available information such as GNSS satellite ephemeris and an Earth elevation database, a cloud service can derive good quality GPS locations from a few milliseconds of raw data. Using our design of a portable sensing device platform called CLEO, we evaluate the accuracy and efficiency of the solution. Compared to more than 30 seconds of heavy signal processing on standalone GPS receivers, we can achieve three orders of magnitude lower energy consumption per location tagging.

This paper addresses the problem of electricity cost management for Internet service providers with a collection of spatially distributed data centers. As the demand on Internet services and cloud computing has kept increasing in recent years, the power usage associated with IDC operations has been uprising significantly. The cyber and physical aspects of IDCs interact with each other, and bring unprecedented challenges in power management. While most existing research focuses on reducing power consumptions of IDCs at one specific location, the problem of reducing the total electricity cost has been overlooked. This is an important problem faced by service providers, especially in the present multielectricity-market environment, where the price of electricity may exhibit temporal and spatial diversities. Further, for these service providers, guaranteeing the quality of service (QoS; i.e., service level objectives) such as service delay guarantees to the end users is of critical importance. This paper studies the problem of minimizing the total electricity cost geared to QoS constraint as well as the location diversity and time diversity of electricity price under multiregional electricity markets. We jointly consider both the cyber and physical management capabilities of IDCs, and exploit both the center-level load balancing and the server-level power control in a unified scheme. We model the problem as a constrained mixed integer programming based on generalized benders decomposition (GBD) technique. Extensive evaluations based on real-life electricity price data for multiple IDC locations demonstrate the effectiveness of our scheme.

Cloud services accessed through mobile devices suffer from high network access latencies and are constrained by energy budgets dictated by the devices' batteries. Radio and battery technologies will improve over time, but are still expected to be the bottlenecks in future systems. Non-volatile memories (NVM), however, may continue experiencing significant and steady improvements in density for at least ten more years. In this paper, we propose to leverage the abundance in memory capacity of mobile devices to mitigate latency and energy issues when accessing cloud services.

Users increasingly rely on their mobile devices to search, locate and discover places and activities around them while on the go. Their decision process is driven by the information displayed on their devices and their current context (e.g. traffic, driving or walking etc.). Even though recent research efforts have already examined and demonstrated how different context parameters such as weather, time and personal preferences affect the way mobile users click on local businesses, little has been done to study how the location of the user affects the click behavior. In this paper we follow a data-driven methodology where we analyze approximately 2 million local search queries submitted by users across the US, to visualize and quantify how differently mobile users click across locations. Based on the data analysis, we propose new location-aware features for improving local search click prediction and quantify their performance on real user query traces. Motivated by the results, we implement and evaluate a data-driven technique where local search models at different levels of location granularity (e.g. city, state, and country levels) are combined together at run-time to further improve click prediction accuracy. By applying the location-aware features and the multiple models at different levels of location granularity on real user query streams from a major, commercially available search engine, we achieve anywhere from 5% to 47% higher Precision than a single click prediction model across the US can achieve.

Workload consolidation is very attractive for cloud platforms due to several reasons including reduced infrastructure costs, lower energy consumption, and ease of management. Advances in virtualization hardware and software continue to improve resource isolation among consolidated workloads but a particular form of resource interference is yet to see a commercially widely adopted solution - the interference due to shared processor caches. Existing solutions for handling cache interference require new hardware features, extensive software changes, or reduce the achieved overall throughput. A crucial requirement for effective consolidation is to be able to predict the impact of cache interference among consolidated workloads. In this paper, we present a practical technique for predicting performance interference due to shared processor cache which works on current processor architectures and requires minimal software changes. While performance degradation can be empirically measured for a given placement of consolidated workloads, the number of possible placements grows exponentially with the number of workloads and actual measurement of degradation is thus not practical for every possible placement. Our technique predicts the degradation for any possible placement using only a linear number of measurements, and can be used to select the most efficient consolidation pattern, for required performance and resource constraints. An average prediction error of less than 4% is achieved across a wide variety of benchmark workloads, using Xen VMM on Intel Core 2 Duo and Nehalem quad-core processor platforms. We also illustrate the usefulness of our prediction technique in realizing better workload placement decisions for given performance and resource cost objectives.

Trajectory-based services require continuous user location sensing. GPS is the most common outdoor location sensor on mobile devices. However, the high energy consumption of GPS sensing prohibits it to be used continuously in many applications. In this paper, we propose a Low Energy Assisted Positioning (LEAP) solution that carefully partitions the GPS signal processing pipeline and shifts delay tolerant position calculations to the cloud. The GPS receiver only needs to be on for less than a second to collect the sub-millisecond level propagation delay for each satellites signal. With a reference to a nearby object, such as a cell tower, the LEAP server can infer the rest of the information necessary to perform GPS position calculation. We analyze the accuracy and energy benefit of LEAP and use real user traces to show that LEAP can save up to 80% GPS energy consumption in typical trajectory-based service scenarios.

Efficient thermal management is important in modern data centers as cooling consumes up to 50% of the total energy. Unlike previous work, we consider proactive thermal management, whereby servers can predict potential overheating events due to dynamics in data center configuration and workload, giving operators enough time to react. However, such forecasting is very challenging due to data center scales and complexity. Moreover, such a physical system is influenced by cyber effects, including workload scheduling in servers. We propose ThermoCast, a novel thermal forecasting model to predict the temperatures surrounding the servers in a data center, based on continuous streams of temperature and airflow measurements. Our approach is (a) capable of capturing cyberphysical interactions and automatically learning them from data; (b) computationally and physically scalable to data center scales; (c) able to provide online prediction with real-time sensor measurements. The paper's main contributions are: (i) We provide a systematic approach to integrate physical laws and sensor observations in a data center; (ii) We provide an algorithm that uses sensor data to learn the parameters of a data center's cyber-physical system. In turn, this ability enables us to reduce model complexity compared to full-fledged fluid dynamics models, while maintaining forecast accuracy; (iii) Unlike previous simulation-based studies, we perform experiments in a production data center. Using real data traces, we show that ThermoCast forecasts temperature better than a machine learning approach solely driven by data, and can successfully predict thermal alarms 4.2 minutes ahead of time.

In this demonstration, we present the architecture, implementation, and applications of LiveSynergy --- a system that provides reliable proximity sensing and open interactive abstractions for physical spaces and objects, to enable rich interactions between humans and their environment.

Along with the surging service demands on the cloud, the energy cost of Internet Data Centers (IDCs) is dramatically increasing. Energy management for IDCs is becoming ever more important. A large portion of applications running on data centers are data-intensive applications. MapReduce (and Hadoop) has been one of the mostly deployed frameworks for data-intensive applications. Both academia and industry have been greatly concerned with the problem of how to reduce the energy consumption of IDCs. However the critical power peak problem for MapReduce clusters has been overlooked, which is a new challenge brought by the usage of MapReduce. We elaborate the power peak problem and investigate the cause of the problem in details. Then we design an adaptive approach to regulate power peaks.

Virtualization is often used in cloud computing platforms for its several advantages in efficiently managing resources. However, virtualization raises certain additional challenges, and one of them is lack of power metering for virtual machines (VMs). Power management requirements in modern data centers have led to most new servers providing power usage measurement in hardware and alternate solutions exist for older servers using circuit and outlet level measurements. However, VM power cannot be measured purely in hardware. We present a solution for VM power metering, named Joulemeter. We build power models to infer power consumption from resource usage at runtime and identify the challenges that arise when applying such models for VM power metering. We show how existing instrumentation in server hardware and hypervisors can be used to build the required power models on real platforms with low error. Our approach is designed to operate with extremely low runtime overhead while providing practically useful accuracy. We illustrate the use of the proposed metering capability for VM power capping, a technique to reduce power provisioning costs in data centers. Experiments are performed on server traces from several thousand production servers, hosting Microsoft's real-world applications such as Windows Live Messenger. The results show that not only does VM power metering allows virtualized data centers to achieve the same savings that non-virtualized data centers achieved through physical server power capping, but also that it enables further savings in provisioning costs with virtualization.

Although mobile phones are ideal platforms for continuous human centric sensing, the state of the art phone architectures today have not been designed to support continuous sensing applications. Currently, sampling and processing sensor data on the phone requires the main processor and associated components to be continuously on, creating a large energy overhead that can severely impact the battery lifetime of the phone. We will demonstrate Little Rock, a novel sensing architecture for mobile phones, where sampling and, when possible, processing of sensor data is offloaded to a dedicated low-power processor. This approach enables the phone to perform continuous sensing three orders of magnitude more energy efficiently compared to the normal approaches.

This paper derives energy-optimal batching periodsfor asynchronous multistage data processing on sensor nodes in the sense of minimizing energy consumption while meeting end-to-end deadlines. Batching the processing of (sensor) data maximizes processor sleep periods, hence minimizing the wakeup frequency and the corresponding overhead. The algorithm is evaluated on mPlatform, a next-generation heterogeneous sensor node platform equipped with both a low-end microcontroller(MSP430) and a higher-end embedded systems processor (ARM). Experimental results show that the total energy consumption of mPlatform, when processing data flowsat their optimal batching periods, is up to 35% lower than that for uniform period assignment. Moreover, processing data at the appropriate processor can use as much as 80% less energy than running the same task set on the ARM alone and 25% less energy than running the taskset on the MSP430 alone.

Frequency overlap across wireless networks with different radio technologies can cause severe interference and reduce communication reliability. The circumstances are particularly unfavorable for ZigBee networks that share the 2.4 GHz ISM band with WiFi senders capable of 10 to 100 times higher transmission power. Our work first examines the interference patterns between ZigBee and WiFi networks at the bit-level granularity. Under certain conditions, ZigBee activities can trigger a nearby WiFi transmitter to back off, in which case the header is often the only part of the Zig-Bee packet being corrupted. We call this the symmetric interference regions, in comparison to the asymmetric regions where the ZigBee signal is too weak to be detected by WiFi senders, but WiFi activity can uniformly corrupt any bit in a ZigBee packet. With these observations, we design BuzzBuzz to mitigate WiFi interference through header and payload redundancy. Multi-Headers provides header redundancy giving ZigBee nodes multiple opportunities to detect incoming packets. Then, TinyRS, a full-featured Reed Solomon library for resource-constrained devices, helps decoding polluted packet payload. On a medium-sized testbed, BuzzBuzz improves the ZigBee network delivery rate by 70%. Furthermore, BuzzBuzz reduces ZigBee retransmissions by a factor of three, which increases the WiFi throughput by 10%.

We consider the problem of computing all-pair correlations in a warehouse containing a large number (e.g., tens of thousands) of time-series (or, signals). The problem arises in automatic discovery of patterns and anomalies in data intensive applications such as data center management, environmental monitoring, and scientific experiments. However, with existing techniques, solving the problem for a large stream warehouse is extremely expensive, due to the problem's inherent quadratic I/O and CPU complexities.

In this paper, we present AdaptSens: a reliable data collection and storage system for solar-powered sensor networks. Unlike battery-operated devices, solar-powered systems have a less predictable energy supply and their ability to harvest energy depends on past spending, thereby creating incentives for adaptive matching of energy supply and demand. Our storage system is novel in its layered architecture and its incremental layer activation mechanism. AdaptSens provides a set of functions, in separate layers, such as sensory data collection, replication (to prevent failure-induced data loss), and storage balancing (to prevent depletion-induced data loss). The mechanism utilizes surplus energy when available by activating more layers, and resorts to progressively more energy-efficient (partial hibernation) modes when energy is scarce. Best reliability is achieved when all layers are active but meaningful intermediate modes allow different degrees of energy conservation. The efficacy of AdaptSens in trading off reliability for energy is tested on both an outdoor system and an indoor testbed. Evaluation results show that AdaptSens minimizes the sum of all data losses when combining the energy, storage and node failure factors.

RACNet is a sensor network for monitoring a data center's environmental conditions. The high spatial and temporal fidelity measurements that RACNet provides can be used to improve the data center's safety and energy efficiency. RACNet overcomes the network's large scale and density and the data center's harsh RF environment to achieve data yields of 99% or higher over a wide range of network sizes and sampling frequencies. It does so through a novel Wireless Reliable Acquisition Protocol (WRAP). WRAP decouples topology control from data collection and implements a token passing mechanism to provide network-wide arbitration. This congestion avoidance philosophy is conceptually different from existing congestion control algorithms that retroactively respond to congestion. Furthermore, WRAP adaptively distributes nodes among multiple frequency channels to balance load and lower data latency. Results from two testbeds and an ongoing production data center deployment indicate that RACNet outperforms previous data collection systems, especially as network load increases.

We consider the problem of collectively approximating a set of sensor signals using the least amount of space so that any individual signal can be efficiently reconstructed within a given maximum (L∞) error ε. The problem arises naturally in applications that need to collect large amounts of data from multiple concurrent sources, such as sensors, servers and network routers, and archive them over a long period of time for offline data mining. We present GAMPS, a general framework that addresses this problem by combining several novel techniques. First, it dynamically groups multiple signals together so that signals within each group are correlated and can be maximally compressed jointly. Second, it appropriately scales the amplitudes of different signals within a group and compresses them within the maximum allowed reconstruction error bound. Our schemes are polynomial time O(α, β approximation schemes, meaning that the maximum (L∞) error is at most α ε and it uses at most β times the optimal memory. Finally, GAMPS maintains an index so that various queries can be issued directly on compressed data. Our experiments on several real-world sensor datasets show that GAMPS significantly reduces space without compromising the quality of search and query.

We present Cypress, a novel framework to archive and query massive time series streams such as those generated by sensor networks, data centers, and scientific computing. Cypress applies multi-scale analysis to decompose time series and to obtain sparse representations in various domains (e.g. frequency domain and time domain). Relying on the sparsity, the time series streams can be archived with reduced storage space. We then show that many statistical queries such as trend, histogram and correlations can be answered directly from compressed data rather than from reconstructed raw data. Our evaluation with server utilization data collected from real data centers shows significant benefit of our framework.

Embedded systems with heterogeneous processors extend the energy/timing trade-off flexibility and provide the opportunity to fine tune resource utilization for particular applications. In this paper, we present a resource model that considers the time and energy costs of run-time mode switching, which considerably improves the accuracy of existing models. Given an application, the software partitioning problem then becomes an optimization over energy cost given deadline constraints, which can be formulate as an integer linear programming (ILP) problem. We apply the resource modeling and software partitioning techniques to a multimodule embedded sensing device, the mPlatform, and present a case study of configuring the platform for a real-time sound source localization application on a stack of MSP430 and ARM7 processor based sensing and processing boards.

Discrete-event (DE) models are formal system specifications that have analysable deterministic behaviors. Using a global, consistent notion of time, DE components communicate via time-stamped events. DE models have primarily been used in performance modeling and simulation, where time stamps are a modeling property bearing no relationship to real time during execution of the model. In this paper, we extend DE models with the capability of relating certain events to physical time. We propose a programming model, called PTIDES (programming temporally integrated distributed embedded systems), which has DE semantics, but with carefully chosen relations between model time and real time. Key to making this model effective is to ensure that constraints that guarantee determinacy in the semantics are preserved at runtime. To accomplish this, we give a distributed execution strategy that obeys DE semantics without the penalty of totally ordered executions based on time stamps. Our technique relies on having a distributed common notion of time, known to some precision. Based on causality analysis of DE models, we define relevant dependency and relevant orders to enable out-of-order execution without compromising determinism and without requiring backtracking

This paper is concerned with reducing communication costs when executing distributed user tasks in a sensor network. We take a service-oriented abstraction of sensor networks, where a user task is composed of a set of data processing modules (called services) with dependencies. Communications in sensor networks consume significant energy and introduce uncertainty in data fidelity due to high bit error rate. These constraints are abstracted as costs on the communication graph. The goal is to place the services within the sensor network so that the communication cost in performing the task is minimized. In addition, since the lifetime of a node, the quality of network links, and the composition of the service graph may change over time, the quality of the placement must be maintained in the face of these dynamics. In this paper, we take a fresh look at what is generally considered a simple but poor performance approach for service placement, namely the greedy algorithm. We prove that a modified greedy algorithm is guaranteed to have cost at most 8 times the optimum placement. In fact, the guarantee is even stronger if there is a high degree of data reduction in the service graph. The advantage of the greedy placement strategy is that when there are local changes in the service graph or when a hosting node fails, the repair only affects the placement of services that depend on the changes. Simulations suggest that in practice the greedy algorithm finds a low cost placement. Furthermore, the cost of repairing a greedy placement decreases rapidly as a function of the proximity of the services to be aggregated.

This paper studies task assignment in a network of resource constrained computing platforms (called microservers). A task is an abstraction of a computational agent or data that is hosted by the microservers. For example, in an object tracking scenario, a task represents a mobile tracking agent, such as a vehicle location update computation, that runs on microservers, which can receive sensor data pertaining to the object of interest. Due to object motion, the microservers that can observe a particular object change over time and there is overhead involved in migrating tasks among microservers. Furthermore, communication, processing, or memory constraints, allow a microserver to only serve a limited number of objects at the same time. Our overall goal is to assign tasks to microservers so as to minimize the number of migrations, and thus be kinetically stable, while guaranteeing that as many tasks as possible are monitored at all times. When the task trajectories are known in advance, we show that this problem is NP-complete (even over just two time steps), has an integrality gap of at least 2, and can be solved optimally in polynomial time if we allow tasks to be assigned fractionally. When only probabilistic information about future movement of the tasks is known, we propose two algorithms: a multi-commodity flow based algorithm and a maximum matching algorithm. We use simulations to compare the performance of these algorithms against the optimum task allocation strategy.

We present a spreadsheet approach to simplifying the process of managing, programming, and interacting with sensor networks and visualizing, archiving and retrieving sensor data. An Excel spreadsheet prototype has been built to demonstrate the idea. This environment provides Excel users, who are already familiar with spreadsheet applications, a convenient and powerful tool for programming and data analysis. We discuss the architecture of this prototype and our experience in implementing the tool. We show two different classes of sensor-net applications built using this platform. We also present performance data on the scalability of the tool with respect to data rate and number of data streams.

No abstract available.

We introduce galsC, a language designed for programming event-driven embedded systems such as sensor networks. galsC implements the TinyGALS (globally asynchronous and locally synchronous) programming model. At the local level, software components are linked via synchronous method calls to form actors. At the global level, actors communicate with each other asynchronously via message passing, which separates the flow of control between actors. A complementary model, called TinyGUYS, is a guarded yet synchronous model designed to allow thread-safe sharing of global state between actors via parameters without explicitly passing messages. The galsC compiler extends the nesC compiler, which allows for better type checking and code generation. Having a well-structured concurrency model at the application level greatly reduces the risk of concurrency errors, such as deadlock and race conditions. The galsC language is implemented on the Berkeley motes and is compatible with the TinyOS/nesC component library. We use a multi-hop wireless sensor network as an example to illustrate the effectiveness of the language.

Microservers are networked embedded devices that accept user tasks on demand and execute them on real world information collected by sensors. Sharing intermediate sensing and computing results among these tasks is critical for optimal resource utilization. This paper presents a service-oriented microserver runtime --- Share and its semantics-based task management design. Event semantics checking and conversion are based on a signal type system (STS) that captures both data values and service triggering. Based on the compatibility of event semantics, redundant computations in uncoordinated tasks are removed from the runtime. A prototype of Share has been experimented with a parking garage sensor network executing three uncoordinated user queries.

No abstract available.

No abstract available.

This paper presents a new group communication scheme, roamingcast, for collaborative information processing in wireless sensor networks. Roamingcast enables efficient communication among a subset of mobile terminals in a collaboration group. Unicast and multicast communication can be considered as special cases of roamingcast in which the subset contains one and all group members, respectively. We propose a Roaming Hub Based Architecture (RoamHBA, pronounced as 'rumba') as one solution to support roaming-cast. We present the distributed construction and dynamic update of a multicast tree, referred as the roaming hub. This roaming hub has the property that an average pair of terminals communicate using the hub with only constant degradation in path length compared to the best possible path. We have developed network layer protocols implementing this mechanism and evaluated their performance in comparison with roaming restricted flooding. We simulated our design using NS-2.

This paper investigates the problem of designing decentralized representations to support monitoring and inferences in sensor networks. State-space models of physical phenomena such as those arising from tracking multiple interacting targets, while commonly used in signal processing and control, suffer from the curse of dimensionality as the number of phenomena of interest increases. Furthermore, mapping an inference algorithm onto a distributed sensor network must appropriately allocate scarce sensing and communication resources. We address the state-space explosion problem by developing a distributed state-space model that switches between factored and joint state spaces as appropriate. We develop a collaborative group abstraction as a mechanism to effectively support the information ow within and across subspaces of the state-space model, which can be efficiently supported in a communication-constrained network. The approach has been implemented and demonstrated in a simulation of tracking multiple interacting targets.

This paper proposes a vehicle-to-vehicle communication protocol for cooperative collision warning. Emerging wireless technologies for vehicle-to-vehicle (V2V) and vehicle-to-roadside (V2R) communications such as DSRC are promising to dramatically reduce the number of fatal roadway accidents by providing early warnings. One major technical challenge addressed in this paper is to achieve low-latency in delivering emergency warnings in various road situations. Based on a careful analysis of application requirements, we design an effective protocol, comprising congestion control policies, service differentiation mechanisms and methods for emergency warning dissemination. Simulation results demonstrate that the proposed protocol achieves low latency in delivering emergency warnings and efficient bandwidth usage in stressful road scenarios.

Modern embedded computing systems tend to be heterogeneous in the sense of being composed of subsystems with very different characteristics, which communicate and interact in a variety of ways-synchronous or asynchronous, buffered or unbuffered, etc. Obviously, when designing such systems, a modeling language needs to reflect this heterogeneity. Today's modeling environments usually offer a variant of what we call amorphous heterogeneity to address this problem. This paper argues that modeling systems in this manner leads to unexpected and hard-to-analyze interactions between the communication mechanisms and proposes a more structured approach to heterogeneity, called hierarchical heterogeneity, to solve this problem. It proposes a model structure and semantic framework that support this form of heterogeneity, and discusses the issues arising from heterogeneous component interaction and the desire for component reuse. It introduces the notion of domain polymorphism as a way to address these issues.

This paper describes information-based approaches to processing and organizing spatially distributed, multimodal sensor data in a sensor network. Energy-constrained networked sensing systems must rely on collaborative signal and information processing (CSIP) to dynamically allocate resources, maintain multiple sensing foci, and attend to new stimuli of interest, all based on task requirements and resource constraints. Target tracking is an essential capability for sensor networks and is used as a canonical problem for studying information organization problems in CSIP. After formulating a CSIP tracking problem in a distributed constrained optimization framework, the paper describes information-driven sensor query and other techniques for tracking individual targets as well as combinatorial tracking problems such as counting targets. Results from simulations and experimental implementations have demonstrated that these information-based approaches are scalable and make efficient use of scarce sensing and communication resources.