Multiple clustering aims to discover diverse latent structures from different perspectives, yet existing methods generate exhaustive clusterings without discerning user interest, necessitating laborious manual screening. Current multi-modal solutions suffer from static semantic rigidity: predefined candidate words fail to adapt to dataset-specific concepts, and fixed fusion strategies ignore evolving feature interactions. To overcome these limitations, we propose Multi-DProxy, a novel multi-modal dynamic proxy learning framework that leverages cross-modal alignment through learnable textual proxies. Multi-DProxy introduces 1) gated cross-modal fusion that synthesizes discriminative joint representations by adaptively modeling feature interactions. 2) dual-constraint proxy optimization where user interest constraints enforce semantic consistency with domain concepts while concept constraints employ hard example mining to enhance cluster discrimination. 3) dynamic candidate management that refines textual proxies through iterative clustering feedback. Therefore, Multi-DProxy not only effectively captures a user's interest through proxies but also enables the identification of relevant clusterings with greater precision. Extensive experiments demonstrate state-of-the-art performance with significant improvements over existing methods across a broad set of multi-clustering benchmarks.

Jinze Li, Yang Zhang, Xin Yang, Jiayi QU, Jinfeng Xu, Shuo Yang, Junhua Ding, Edith Cheuk-Han Ngai. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Xuehe Wang, Edith Cheuk-Han Ngai. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

The explosion of multimedia data in information-rich environments has intensified the challenges of personalized content discovery, positioning recommendation systems as an essential form of passive data management. Multimodal sequential recommendation, which leverages diverse item information such as text and images, has shown great promise in enriching item representations and deepening the understanding of user interests. However, most existing models rely on heuristic fusion strategies that fail to capture the dynamic and context-sensitive nature of user-modal interactions. In real-world scenarios, user preferences for modalities vary not only across individuals but also within the same user across different items or categories. Moreover, the synergistic effects between modalities-where combined signals trigger user interest in ways isolated modalities cannot-remain largely underexplored. To this end, we propose CAMMSR, a Category-guided Attentive Mixture of Experts model for Multimodal Sequential Recommendation. At its core, CAMMSR introduces a category-guided attentive mixture of experts (CAMoE) module, which learns specialized item representations from multiple perspectives and explicitly models inter-modal synergies. This component dynamically allocates modality weights guided by an auxiliary category prediction task, enabling adaptive fusion of multimodal signals. Additionally, we design a modality swap contrastive learning task to enhance cross-modal representation alignment through sequence-level augmentation. Extensive experiments on four public datasets demonstrate that CAMMSR consistently outperforms state-of-the-art baselines, validating its effectiveness in achieving adaptive, synergistic, and user-centric multimodal sequential recommendation.

The ''pre-training, prompt-tuning'' has emerged as a pivotal paradigm in advancing the performance of graph representation learning models across a wide range of downstream tasks. This paradigm leverages the power of pre-trained models and task-specific prompts to bridge the gap between general graph representations and task-specific requirements. Early graph prompt tuning approaches relied on task-specific designs for Graph Neural Networks (GNNs), limiting their adaptability across diverse pre-training strategies. In contrast, another promising line of research has investigated universal graph prompt tuning, which operates directly in the input graph's feature space and builds a theoretical foundation that universal graph prompt tuning can theoretically achieve an equivalent effect of any prompting function, eliminating dependence on specific pre-training strategies. Recent works propose selective node-based graph prompt tuning to pursue more ideal prompts. However, we argue that selective node-based graph prompt tuning inevitably compromises the theoretical foundation of universal graph prompt tuning. In this paper, we strengthen the theoretical foundation of universal graph prompt tuning by introducing stricter constraints, demonstrating that adding prompts to all nodes is a necessary condition for achieving the universality of graph prompts. To this end, we propose a novel model and paradigm, Learning and Editing Universal GrAph Prompt Tuning (LEAP), which preserves the theoretical foundation of universal graph prompt tuning while pursuing more ideal prompts. Specifically, we first build the basic universal graph prompts to preserve the theoretical foundation and then employ actor-critic reinforcement learning to select nodes and edit prompts. Extensive experiments on graph- and node-level tasks across various pre-training strategies in both full-shot and few-shot scenarios show that LEAP consistently outperforms fine-tuning and other prompt-based approaches.

Although existing multimodal recommendation models have shown promising performance, their effectiveness continues to be limited by the pervasive data sparsity problem. This problem arises because users typically interact with only a small subset of available items, leading existing models to arbitrarily treat unobserved items as negative samples. To this end, we propose VI-MMRec, a model-agnostic and training cost-free framework that enriches sparse user-item interactions via similarity-aware virtual user-item interactions. These virtual interactions are constructed based on modality-specific feature similarities of user-interacted items. Specifically, VI-MMRec introduces two different strategies: (1) Overlay, which independently aggregates modality-specific similarities to preserve modality-specific user preferences, and (2) Synergistic, which holistically fuses cross-modal similarities to capture complementary user preferences. To ensure high-quality augmentation, we design a statistically informed weight allocation mechanism that adaptively assigns weights to virtual user-item interactions based on dataset-specific modality relevance. As a plug-and-play framework, VI-MMRec seamlessly integrates with existing models to enhance their performance without modifying their core architecture. Its flexibility allows it to be easily incorporated into various existing models, maximizing performance with minimal implementation effort. Moreover, VI-MMRec introduces no additional overhead during training, making it significantly advantageous for practical deployment. Comprehensive experiments conducted on six real-world datasets using seven state-of-the-art multimodal recommendation models validate the effectiveness of our VI-MMRec.

Multimodal large language models (MLLMs) are increasingly integrated into video conferencing, but mostly for speech-centric tasks such as transcription and summarization. Conferencing adaptation remains dominated by signal-driven congestion control that reacts to bandwidth changes without understanding why certain moments or streams will soon become quality-critical. This separation wastes predictive structure in conversation: text often reveals impending role shifts and interaction-mode transitions—for example, taking the floor or initiating screen sharing—seconds before the corresponding media and bandwidth demands materialize.

Recent advancements in multimodal recommendations, which leverage diverse modality information to mitigate data sparsity and improve recommendation accuracy, have gained significant attention. However, existing multimodal recommendations overlook the critical role of user representation initialization. Unlike items, which are naturally associated with rich modality information, users lack such inherent information. Consequently, item representations initialized based on meaningful modality information and user representations initialized randomly exhibit a significant semantic gap.

Growing awareness of wellness has prompted people to consider whether their dietary patterns align with their health and fitness goals. In response, researchers have introduced various wearable dietary monitoring systems and dietary assessment approaches. However, these solutions are either limited to identifying foods with simple ingredients or insufficient in providing an analysis of individual dietary behaviors with domain-specific knowledge. In this article, we present DietGlance , a system that automatically monitors dietary behaviors in daily routines and delivers personalized analysis from knowledge sources. DietGlance first detects ingestive episodes from multimodal inputs using eyeglasses, capturing privacy-preserving meal images of various dishes being consumed. Based on the inferred food items and consumed quantities from these images, DietGlance further provides nutritional analysis and personalized dietary suggestions, empowered by the retrieval-augmented generation module on a reliable nutrition library. A short-term user study (N = 33) and a 4-week longitudinal study (N = 16) demonstrate the usability and effectiveness of DietGlance , offering insights and implications for future AI-assisted dietary monitoring and personalized healthcare intervention systems using eyewear.

Constructing radio maps traditionally requires extensive site surveys with precise location labels, resulting in costly and time-consuming calibration. Conventional approaches derive labels from inertial measurement units (IMUs), but are constrained by device-level access permissions and the need for pre-installed IMU hardware. In this paper, we present a calibration-free radio map construction method that relies solely on Channel State Information (CSI) measurements, thereby obviating the need for location labels. Our key insight is to embed CSI data into a two-dimensional geographic space using a neural network, without any label information. The primary challenge is to preserve the real-world topology in the embedded space. To address this issue, we propose a novelTopology-Guided Manifold Learning (TGML)approach that learns a low-dimensional embedding through self-supervision based on its mapping to a topological map in the geographic space. Specifically, we introduce an embedding network that learns locally smoothed proximity, thus creating a compact low-dimensional representation in the latent space. We further employ a regularized transport method with a differentiable Sinkhorn distance to establish an optimal mapping between the latent and geographic spaces. We use the resulting mapping deviation to supervise the training of the embedding network, enabling continuous refinement through self-supervision. Experiments in an office environment show that TGML achieves an average localization error of 2.37m and a relative beam estimation error of 6.68%, outperforming state-of-the-art methods.

Federated crowdsourcing has emerged as a promising paradigm for collaboratively solving learning tasks over mobile edge devices, enabling decentralized model training without direct data uploading. However, conventional federated crowdsourcing systems largely overlook two fundamental challenges: (i) the lack of effective incentive mechanisms under unknown participant quality, (ii) the risk of privacy leakage from model uploading. In this paper, we propose an incentive mechanism for federated crowdsourcing based on the Stackelberg game considering$\rho$-zero-concentrated differential privacy andCombinatorialMulti-ArmedBandit mechanism, calledFedCMAB, to tackle the client selection problem with unknown quality and incentive design while preserving confidential information. We model the interaction between the server and clients as a Stackelberg game, where the server dynamically selects a subset of clients and designs rewards to minimize the cost while each client strategically determines its privacy budget to maximize its individual utility. We theoretically establish the existence of the Stackelberg Nash equilibrium. Next, we leverage the combinatorial multi-armed bandit (CMAB) method to learn optimal client selection strategies with provable regret guarantees and derive an upper bound on the cumulative regret of the proposed mechanism. Moreover, we conduct a rigorous Price of Anarchy (PoA) analysis to quantify the efficiency gap between the decentralized equilibrium induced by self-interested clients and the socially optimal solution. Our analyses demonstrate that conventional incentive-agnostic strategy can lead to an unbounded PoA, resulting in severe efficiency loss. In contrast, ourFedCMABframework provably bounds the PoA by a finite constant. Extensive experimental results on multiple datasets demonstrate thatFedCMABconsistently outperforms state-of-the-art baseline methods on both independent and identically distributed (IID) and non-IID data.

Causality has been integrated with machine learning in uncovering and understanding the causal relationship between variables and observed outcomes. However, the centralized training setting of causal machine learning is not adaptable to most practical scenarios, where datasets are distributed, stored, and unsharable due to privacy concerns. Federated learning (FL), a distributed learning framework that allows collaborative training across multiple devices without raw data sharing, emerges as a potential solution to this problem. By integrating FL into causal problems, the discovery and inference of causal relationships across dispersed datasets can be achieved. On the other hand, causality can also enhance FL models in various dimensions, including model interpretability and explainability, generalizability, adversarial robustness, and fairness and bias mitigation. In this article, we provide a comprehensive review of the above two directions and summarize the interplays between causality and FL (short for Causal-FL ) by organizing our discussion around two key questions: (1) how FL enable decentralized causal analysis; and (2) how causality tackles FL challenges. The potential applications of these methods are also introduced, including healthcare, recommendation, economics, social equity, and so on. Moreover, we discuss promising future directions and future challenges to be explored.

The prevalence of voice-related interaction and communication has raised concerns about privacy leakage and security. For example, millimeter-wave (mmWave) radio signals have been exploited as a potential attacker for acoustic eavesdropping. However, speaker variability and low-quality input pose significant challenges for the practical deployment of mmWave-based eavesdropping. In this paper, we proposeSPACE, an acoustic eavesdropping system to recover intelligible speech from low-quality mmWave signals, which can adapt to numerous different speakers and unseen ones.SPACEis a two-stage system that first reconstructs the spectrogram using a novelRadio TransUNetand then synthesizes the waveform through a neural vocoder. Specifically, to alleviate the negative effect of speaker variability, we introduce a speaker encoder to capture speaker features and a fusion network to condition the spectrogram reconstruction based on the extracted speaker characteristics. Further, to facilitate intelligible speech recovery from low-quality input, we design a Frequency Transformation Layer to exploit the correlation among all frequency harmonics and incorporate the neural vocoder to synthesize the speech waveform from the reconstructed spectrogram without using the contaminated phase. The experimental results show thatSPACEoutperforms existing mmWave-based approaches in scenarios with numerous different speakers and unseen speakers.

The exponential growth of online information has made it increasingly difficult for users to identify valuable and relevant content. Recommender systems have emerged as a critical solution to this challenge by tailoring content to individual preferences. With the proliferation of diverse multimedia services, human interaction with the digital world has become inherently multimodal. Consequently, recommender systems capable of comprehending and interpreting multimodal information can more effectively align with individual preferences. With its recent surge in research attention, the field of multimodal recommender systems (MRS) still lacks a comprehensive technical survey. Existing surveys suffer from two critical limitations: 1) Insufficient technical depth: Prior works predominantly focus on categorizing and discussing general structure, neglecting rigorous technical analysis of methodologies and architectures. 2) Absence of cutting-edge works: Due to the rapid evolution of AI technologies, current surveys fail to discuss the latest works that adopt the most advanced techniques. To bridge these gaps, this survey conducts a systematic and technical review of advanced MRS works from MRS's inception to the present. We organize existing works into coherent taxonomies based on their structure and provide in-depth analyses of methodological innovations at each component, including Feature Extraction, Encoder, Multimodal Fusion, and Loss Function. Moreover, we further discuss potential future directions for developing and enhancing MRS. This survey serves as technical guidance for researchers and practitioners, offering insights into the developments, techniques, and future directions of MRS. Notably, we maintain a public repository:https://github.com/Jinfeng-Xu/Awesome-Multimodal-Recommender-Systems.

Beyond traditional user recommendation, group recommendation is a new and popular task that provides recommendations for a group of users. Previous works aggregate member preferences in the group to infer group preference, but this often leads to a coarse-grained inference for group preferences limited by users’ individual preferences. To this end, we exploit that user preferences can be inferred and refined by exploring the group preferences that they participated in. These refined preferences offer additional information beyond the original individual preferences, enabling more fine-grained and satisfactory group preference inference. In this work, we propose a novel Dual-Granularity Graph Variational Auto-Encoder framework (DGGVAE) for group recommendation, which jointly reveals group preferences from both coarse granularity and fine granularity to comprehensively learn group preferences. Specifically, we design a Group Preference Extractor module that extracts group preferences from these two granularities: coarse granularity, which is revealed through original member preferences, and fine granularity, which is revealed through refined member preferences. To extract the correlation between groups, a Group Representation Enhancement module is proposed, which enhances group representations by information from the most similar groups. However, the coarse- and fine-grained group preferences contain uncertainty due to the gap between the original and refined member preferences. To better incorporate dual-granularity group preferences, we design granularity-specific graph variational encoders that learn Gaussian variables on the semantic information for each group. Moreover, with the conditional independence assumption, the granularity-specific Gaussian node embeddings are fused according to the generalized product-of-experts (gPoE), where the semantic information in each granularity is weighted based on the estimated uncertainty level. Extensive experiments show the superiority of DGGVAE over various state-of-the-art methods in training efficiency and accuracy on both group and user recommendation tasks.

Learning-based Android malware detectors degrade over time due to natural distribution drift caused by malware variants and new families. This paper systematically investigates the challenges classifiers trained with empirical risk minimization (ERM) face against such distribution shifts and attributes their shortcomings to their inability to learnstablediscriminative features. Invariant learning theory offers a promising solution by encouraging models to generate stable representations across environments that expose the instability of the training set. However, the lack of prior environment labels, the diversity of drift factors, and low-quality representations caused by diverse families make this task challenging. To address these issues, we propose TIF, the first temporal invariant training framework for malware detection, which aims to enhance the ability of detectors to learn stable representations across time. TIF organizes environments based on application observation dates to reveal temporal drift, integrating specialized multi-proxy contrastive learning and invariant gradient alignment to generate and align environments with high-quality, stable representations. TIF can be seamlessly integrated into any learning-based detector. Experiments on a decade-long dataset show that TIF excels, particularly in early deployment stages, addressing real-world needs and outperforming state-of-the-art methods.

As multimedia information proliferates, multimodal recommendation systems have garnered significant attention. These systems leverage multimodal information to alleviate the data sparsity issue inherent in recommendation systems, thereby enhancing the accuracy of recommendations. Due to the natural semantic disparities among multimodal features, recent research has primarily focused on cross-modal alignment using self-supervised learning to bridge these gaps. However, aligning different modal features might result in the loss of valuable interaction information, distancing them from ID embeddings. It is crucial to recognize that the primary goal of multimodal recommendation is to predict user preferences, not merely to understand multimodal content. To this end, we propose a new Multi-level sElf-supervised learNing for mulTimOdal Recommendation (MENTOR) method, which effectively reduces the gap among modalities while retaining interaction information. Specifically, MENTOR begins by extracting representations from each modality using both heterogeneous user-item and homogeneous item-item graphs. It then employs a multilevel cross-modal alignment task, guided by ID embeddings, to align modalities across multiple levels while retaining historical interaction information. To balance effectiveness and efficiency, we further propose an optional general feature enhancement task that bolsters the general features from both structure and feature perspectives, thus enhancing the robustness of our model.

Graph Collaborative Filtering (GCF) has emerged as a dominant paradigm in modern recommendation systems, excelling at modeling complex user-item interactions and capturing high-order collaborative signals. Most existing GCF models predominantly rely on simplified graph architectures like LightGCN, which strategically remove feature transformation and activation functions from vanilla graph convolution networks. Through systematic analysis, we reveal that feature transformation in message propagation can enhance model representation, though at the cost of increased training difficulty. To this end, we propose FourierKAN-GCF, a novel framework that adopts Fourier Kolmogorov-Arnold Networks as efficient transformation modules within graph propagation layers. This design enhances model representation while decreasing training difficulty. Our FourierKAN-GCF can achieve higher recommendation performance than most widely used GCF backbone models and can be integrated into existing advanced self-supervised models as a backbone, replacing their original backbone to achieve enhanced performance. Extensive experiments on three public datasets demonstrate the superiority of FourierKAN-GCF.

During the COVID-19 pandemic, cheating behaviors in exams showed an increasing trend. To protect fairness, this paper presents the development and evaluation of an AI-powered Anti-cheating System designed to detect and monitor cheating behaviors during online examinations. The system utilizes advanced techniques of computer vision and machine learning. For instance, integrates MediaPipe for face and pose tracking, YOLOv5 for object detection, and a custom GUI built with Tkinter to monitor student behavior in real-time. The system identifies potential cheating through metrics such as downward gaze, excessive motion, unauthorized objects, or people. It will automatically log the incidents and capture timestamped screenshots as cheating evidence. The system also provides a risk level algorithm, which is weighted by various factors, such as gaze, motion, people, and object detection. It can provide a probabilistic assessment eventually. Moreover, triggering alerts and logs when the thresholds are reached. The screenshot evidence enhances post-exam reviews for the teachers or professors. It resets logs per session to ensure isolated incident tracking and addresses scalability through session-based file management. This research contributes to educational technology by offering a reliable and evidence-based proctoring solution. It brings the benefits of reducing human workload to monitor the whole exam while maintaining academic integrity. Future work will explore real-world deployment, user feedback integration, and optimization for diverse hardware environments.

With the rise of Web3, Non-Fungible Tokens (NFTs) have become a new class of digital assets, driving demand for large-scale NFT recommendation systems. Each NFT can be associated to a rich set of semantic, stylistic, and thematic labels, forming a highly complex label space. Similar to e-commerce platforms where detailed product labels enable personalized recommendations, such semantic dependencies between labels can potentially enhance NFT recommendation performance. Thus, NFT recommendation can be naturally formulated as an extreme multi-label (XML) classification problem. Many existing probabilistic label tree (PLT)-based approaches address XML problem by recursively partitioning the label space, which greatly alleviates the demands on expensive computer resources. Yet, the highly skewed distribution of labels in datasets in XML makes tail labels more challenging to predict than head labels. In this paper, Our preliminary analysis reveals that inherent label dependencies can be leveraged to improve tail label recommendations for NFTs. We propose ChainTail, a dependency-aware framework that enhances PLT-based NFT label partitioning and prediction re-scoring. It includes: (1) a Dependency-aware partition module that partitions highly dependent NFT labels into subsets. (2) a Dependency-aware ReScore module that re-ranks prediction scores of labels to eliminate the label-priors. Our experimental results show that ChainTail boosts tail label recommendation on widely used item recommendation datasets.

Non-independent and identically distributed (Non-IID) data across edge clients have long posed significant challenges to federated learning (FL) training. Prior works have proposed various methods to mitigate this statistical heterogeneity. While these methods can achieve good theoretical performance, they may lead to the over-correction problem, which degrades model performance and even causes failures in model convergence. In this paper, we provide the first investigation into the hidden over-correction phenomenon brought by the uniform model correction coefficients across clients adopted by the existing methods. To address this problem, we propose TACO, a novel algorithm that addresses the non-IID nature of clients’ data by implementing fine-grained, client-specific gradient correction and model aggregation, steering local models towards a more accurate global optimum. Moreover, we verify that leading FL algorithms generally have better model accuracy in terms of communication rounds rather than wall-clock time, resulting from their extra computation overhead imposed on clients. To enhance the training efficiency, TACO deploys a lightweight model correction and tailored aggregation approach that requires minimum computation overhead and no extra information beyond the synchronized model parameters. To validate TACO’s effectiveness, we present the first FL convergence analysis that reveals the root cause of over-correction. Extensive experiments across various datasets confirm TACO’s superior and stable performance in practice.

Speculative decoding (SPD) aims to accelerate the auto-regressive token generation process of a target Large Language Model (LLM). Some approaches employ a draft model with multiple heads to predict a sequence of future tokens, where each head handles a token in the sequence. The target LLM verifies the predicted sequence and accepts aligned tokens, enabling efficient multi-token generation. However, existing methods assume that all tokens within a sequence are equally important, employing identical head structures and relying on a single-generation paradigm, either serial or parallel. To this end, we theoretically demonstrate that initial tokens in the draft sequence are more important than later ones. Building on this insight, we propose Gumiho, a hybrid model combining serial and parallel heads. Specifically, given the critical importance of early tokens, we employ a sophisticated Transformer architecture for the early draft heads in a serial configuration to improve accuracy. For later tokens, we utilize multiple lightweight MLP heads operating in parallel to enhance efficiency. By allocating more advanced model structures and longer running times to the early heads, Gumiho achieves improved overall performance. The experimental results demonstrate that our method outperforms existing approaches, fully validating its effectiveness. Our code is available at https://github.com/AMD-AIG-AIMA/Gumiho.

Wild salmon are essential to the ecological, economic, and cultural sustainability of the North Pacific Rim. Yet climate variability, habitat loss, and data limitations in remote ecosystems that lack basic infrastructure support pose significant challenges to effective fisheries management. This project explores the integration of multimodal foundation AI and expert-in-the-loop frameworks to enhance wild salmon monitoring and sustainable fisheries management in Indigenous rivers across Pacific Northwest. By leveraging video and sonar-based monitoring, we develop AI-powered tools for automated species identification, counting, and length measurement, reducing manual effort, expediting delivery of results, and improving decision-making accuracy. Expert validation and active learning frameworks ensure ecological relevance while reducing annotation burdens. To address unique technical and societal challenges, we bring together a cross-domain, interdisciplinary team of university researchers, fisheries biologists, Indigenous stewardship practitioners, government agencies, and conservation organizations. Through these collaborations, our research fosters ethical AI co-development, open data sharing, and culturally informed fisheries management.

The data sparsity problem significantly hinders the performance of recommender systems, as traditional models rely on limited historical interactions to learn user preferences and item properties. While incorporating multimodal information can explicitly represent these preferences and properties, existing works often use it only as side information, failing to fully leverage its potential. In this paper, we propose MDVT, a model-agnostic approach that constructs multimodal-driven virtual triplets to provide valuable supervision signals, effectively mitigating the data sparsity problem in multimodal recommendation systems. To ensure high-quality virtual triplets, we introduce three tailored warm-up threshold strategies: static, dynamic, and hybrid. The static warm-up threshold strategy exhaustively searches for the optimal number of warm-up epochs but is time-consuming and computationally intensive. The dynamic warm-up threshold strategy adjusts the warm-up period based on loss trends, improving efficiency but potentially missing optimal performance. The hybrid strategy combines both, using the dynamic strategy to find the approximate optimal number of warm-up epochs and then refining it with the static strategy in a narrow hyper-parameter space. Once the warm-up threshold is satisfied, the virtual triplets are used for joint model optimization by our enhanced pair-wise loss function without causing significant gradient skew. Extensive experiments on multiple real-world datasets demonstrate that integrating MDVT into advanced multimodal recommendation models effectively alleviates the data sparsity problem and improves recommendation performance, particularly in sparse data scenarios.

The efficiency and scalability of graph convolution networks (GCNs) in training recommender systems remain critical challenges, hindering their practical deployment in real-world scenarios. In the multimodal recommendation (MMRec) field, training GCNs requires more expensive time and space costs and exacerbates the gap between different modalities, resulting in sub-optimal recommendation accuracy. This paper critically points out the inherent challenges associated with adopting GCNs during the training phase in MMRec, revealing that GCNs inevitably create unhelpful and even harmful pairs during model optimization and isolate different modalities. To this end, we propose FastMMRec, a highly efficient multimodal recommendation framework that deploys graph convolutions exclusively during the testing phase, bypassing their use in training. We demonstrate that adopting GCNs solely in the testing phase significantly improves the model's efficiency and scalability while alleviating the modality isolation problem often caused by using GCNs during the training phase. We conduct extensive experiments on three public datasets, consistently demonstrating the performance superiority of FastMMRec over competitive baselines while achieving efficiency and scalability.

Large Language Models (LLMs) hold significant potential for advancing fact-checking by leveraging their capabilities in reasoning, evidence retrieval, and explanation generation. However, existing benchmarks fail to comprehensively evaluate LLMs and Multimodal Large Language Models (MLLMs) in realistic misinformation scenarios. To bridge this gap, we introduce RealFactBench, a comprehensive benchmark designed to assess the fact-checking capabilities of LLMs and MLLMs across diverse real-world tasks, including Knowledge Validation, Rumor Detection, and Event Verification. RealFactBench consists of 6K high-quality claims drawn from authoritative sources, encompassing multimodal content and diverse domains. Our evaluation framework further introduces the Unknown Rate (UnR) metric, enabling a more nuanced assessment of models' ability to handle uncertainty and balance between over-conservatism and over-confidence. Extensive experiments on 7 representative LLMs and 4 MLLMs reveal their limitations in real-world fact-checking and offer valuable insights for further research. RealFactBench is publicly available at https://github.com/kalendsyang/RealFactBench.git.

Recent works in multimodal recommendations, which leverage diverse modal information to address data sparsity and enhance recommendation accuracy, have garnered considerable interest. Two key processes in multimodal recommendations are modality fusion and representation learning. Previous approaches in modality fusion often employ simplistic attentive or pre-defined strategies at early or late stages, failing to effectively handle irrelevant information among modalities. In representation learning, prior research has constructed heterogeneous and homogeneous graph structures encapsulating user-item, user-user, and item-item relationships to better capture user interests and item profiles. Modality fusion and representation learning were considered as two independent processes in previous work. This paper reveals that these two processes are complementary and can support each other. Specifically, powerful representation learning enhances modality fusion, while effective fusion improves representation quality. Stemming from these two processes, we introduce a COmposite grapH convolutional nEtwork with dual-stage fuSION for the multimodal recommendation, named COHESION. Specifically, it introduces a dual-stage fusion strategy to reduce the impact of irrelevant information, refining all modalities using behavior modality in the early stage and fusing their representations at the late stage. It also proposes a composite graph convolutional network that utilizes user-item, user-user, and item-item graphs to extract heterogeneous and homogeneous latent relationships within users and items. Besides, it introduces a novel adaptive optimization to ensure balanced and reasonable representations across modalities. Extensive experiments on three public datasets demonstrate the significant superiority of COHESION over various competitive baselines.

Users often take notes for instructional videos to access key knowledge later without revisiting long videos.Automated note generation tools enable users to obtain informative notes efficiently.However, notes generated by existing research or off-the-shelf tools fail to preserve the information conveyed in the original videos comprehensively, nor can they satisfy users' expectations for diverse presentation formats and interactive features when using notes digitally.In this work, we present NoteIt, a system, which automatically converts instructional videos to interactable notes using a novel pipeline that faithfully extracts hierarchical structure and multimodal key information from videos.With NoteIt's interface, users can interact with the system to further customize the content and presentation formats of the notes according to their preferences.We conducted both a technical evaluation and a comparison user study (N=36).The solid performance in objective metrics and the positive user feedback demonstrated the effectiveness of the pipeline and the overall usability of NoteIt.

Steady state visually evoked potential (SSVEP)-based brain-computer interfaces (BCIs), which are widely used in rehabilitation and disability assistance, can benefit from real-time emotion recognition to enhance human-machine interaction. However, the learned discri-minative latent representations in SSVEP-BCIs may generalize in an unintended direction, which can lead to reduced accuracy in detecting emotional states. In this paper, we introduce a Valence-Arousal Disentangled Representation Learning (VADL) method, drawing inspir-ation from the classical two-dimensional emotional model, to enhance the performance and generalization of emotion recognition within SSVEP-BCIs. VADL distinctly disentangles the latent variables of valence and arousal information to improve accuracy. It utilizes the structured state space duality model to thoroughly extract global emotional features. Additionally, we propose a Multisubject Gradient Blending training strategy that individually tailors the learning pace of reconstruction and discrimination tasks within VADL on-the-fly. To verify the feasibility of our method, we have developed a comprehensive database comprising 23 subjects, in which both the emotional states and SSVEPs were effectively elicited. Experimental results indicate that VADL surpasses existing state-of-the-art benchmark algorithms.

Multimodal recommender systems utilize a variety of information types to model user preferences and item properties, aiding in the discovery of items that align with user interests. Rich multimodal information alleviates inherent challenges in recommendation systems, such as data sparsity and cold start problems. However, multimodal information further introduces challenges in terms of robustness and generalization capability. Regarding robustness, multimodal information magnifies the risks associated with information adjustment and inherent noise, posing severe challenges to the stability of recommendation models. For generalization capability, multimodal recommender systems are more complex and difficult to train, making it harder for models to handle data beyond the training set, posing significant challenges to model generalization capability. In this paper, we analyze the shortcomings of existing robustness and generalization capability enhancement strategies in the multimodal recommendation field. We propose a sharpness-aware minimization strategy focused on batch data (BSAM), which effectively enhances the robustness and generalization capability of multimodal recommender systems without requiring extensive hyper-parameter tuning. Furthermore, we introduce a mixed loss variant strategy (BSAM+), which accelerates convergence and achieves remarkable performance improvement. We provide rigorous theoretical proofs and conduct experiments with nine advanced models on five widely used datasets to validate the superiority of our strategies. Moreover, our strategies can be integrated with existing robust training and data augmentation strategies to achieve further improvement, providing a superior training paradigm for multimodal recommendations.

Leveraging blockchain in Federated Learning (FL) emerges as a new paradigm for secure collaborative learning on Massive Edge Networks (MENs). As the scale of MENs increases, it becomes more difficult to implement and manage a blockchain among edge devices due to complex communication topologies, heterogeneous computation capabilities, and limited storage capacities. Moreover, the lack of a standard metric for blockchain security becomes a significant issue. To address these challenges, we propose a lightweight blockchain for verifiable and scalable FL, namely LiteChain, to provide efficient and secure services in MENs. Specifically, we develop a distributed clustering algorithm to reorganize MENs into a two-level structure to improve communication and computing efficiency under security requirements. Moreover, we introduce a Comprehensive Byzantine Fault Tolerance (CBFT) consensus mechanism and a secure update mechanism to ensure the security of model transactions through LiteChain. Our experiments based on Hyperledger Fabric demonstrate that LiteChain presents the lowest end-to-end latency and on-chain storage overheads across various network scales, outperforming the other two benchmarks. In addition, LiteChain exhibits a high level of robustness against replay and data poisoning attacks.

The development of Internet of Things (IoT) technology has led to the emergence of a large number of Intelligent Sensing Devices (ISDs). Since their limited physical sizes constrain the battery capacity, wireless powered IoT networks assisted by Unmanned Aerial Vehicles (UAVs) for energy transfer and data acquisition have attracted great interest. In this paper, we formulate an optimization problem to maximize system energy efficiency while satisfying the constraints of UAV mobility and safety, ISD quality of service and task completion time. The formulated problem is constructed as a Constrained Markov Decision Process (CMDP) model, and a Multi-agent Constrained Deep Reinforcement Learning (MCDRL) algorithm is proposed to learn the optimal UAV movement policy. In addition, an ISD-UAV connection assignment algorithm is designed to manage the connection in the UAV sensing range. Finally, performance evaluations and analysis based on real-world data demonstrate the superiority of our solution.

In this paper, we leverage unmanned aerial vehicles (UAVs) to enhance mobile crowd sensing (MCS) by addressing two critical challenges: uncontrollable data quality and inevitable unsensed points of interest (PoIs). We introduce a UAV-assisted method to deal with these challenges. To ensure the accuracy of sensing data contributed by human participants, the proposed truth discovery method utilizes UAV-collected sensing data as few-shot samples to train the truth discovery model, which is then employed to calibrate sensing data solely collected by human participants. Additionally, to meet the sensing coverage requirement, we present a method that predicts data values for unsensed PoIs by utilizing their historical sensing data and the sensed neighboring PoIs information. The method employs a graph neural network to capture spatio-temporal relationships of the sensing data, facilitating accurate estimation of unsensed PoIs. Through extensive simulations, our approaches demonstrate superior performance compared to existing methods, showcasing the potential of UAV-assisted MCS for overcoming challenges and enhancing data collection efficiency in various domains.

To maximize the utility of lecture videos, in today’s fast-paced society with dwindling attention spans, various e-learning technologies are introduced, e.g., non-linear learning, bite-sized learning, and personalized lecture video fragment recommendation. In this article, we conduct a detailed performance study on a key enabler for aforementioned technologies: Lecture Video Fragmentation by Key Point Identification in lecture videos. We begin with a taxonomy of existing methods, which are classified into two categories: boundary-based methods, where the fragmentation is achieved using specific methods depending on the modality, and representation-based methods, where the fragmentation task is formulated as a boundary prediction task based on representations of smaller video chunks. Various configurations of these methods are also examined in detail. To conduct an extensive, comprehensive, and objective comparison study, we address the limitations of existing datasets by introducing a new lecture video fragmentation dataset, MITFLD, without any synthetic videos. We also propose a unified framework kpi , which includes the implementation of datasets, metrics, and compared methods to facilitate the experiments and future research on lecture video fragmentation. The experiments cover different configurations of existing methods on two large datasets (AVLecture and MITFLD). Further experiments are also conducted for ablation studies, such as the effect of feature combinations and the influence of lecture modes. Through the experiments, the representation-based method BiLSTM with self-supervised learning representations is found to exhibit promising performance. Key insights and potential future directions are also discussed.

The prolonged lifetime of energy-harvesting (EH) LoRa networks requires that all EH LoRa sensors utilize available harvested energy in an energy-neutral manner to avoid power failures. This requirement is challenging to fulfill due to the unpredictability of ambient-energy sources and the spatio-temporal heterogeneity of sensors’ harvesting abilities. We present RACEME, a novel predictive EH-management framework by exploiting the embedded intelligence capability of EH LoRa devices. It empowers energy-neutral operation in EH LoRa networks by leveraging the spatio-temporal correlation between EH LoRa sensors to optimize the harvested-energy utilization. RACEME is an integrated system that consists of embedded machine learning for predictive EH management on the sensors and online cluster-based data reduction and recovery on the server. To maximize the accuracy of EH predictions, RACEME allows each sensor to implement a machine learning pipeline locally. Coupled with the cluster-based data-reduction feedback from the server, RACEME enables the sensors with higher harvested-energy availability and communication quality to transmit critical data more frequently in a probabilistic manner without sacrificing data quality. Compared with the state of the art, the experimental results reveal that RACEME improves EH-prediction accuracy, network lifetime, and transmission overhead by up to 1.7, 2, and 8 times, respectively.

In today’s fast-paced digital landscape, the attention span of users consuming video content is alarmingly brief, often as short as 15 seconds for music or entertainment videos and 6 minutes for lecture videos. This presents a significant challenge for video producers and platform providers as they seek to engage users with longer content. One promising solution involves recommending specific fragments within longer videos that align with individual user profiles. In this article, we address this challenge by introducing a novel framework for video fragment recommendations, guided by three key insights. First, we implement a Self-Attention Block that captures the inter-fragment contextual effect, enhancing the relevance of recommendations. Second, we incorporate video-level preferences to ensure that the fragment recommendations are consistent with users’ overall interests. Third, we propose a Self-Attentive Herding Effect (SAHE) module to model the intra-fragment contextual effect, specifically the herding effect of time-sync comments within a fragment. To evaluate the effectiveness of our proposed method, we conduct extensive experiments comparing our model against the state-of-the-art approaches in terms of NDCG@K and Recall@K. Our results demonstrate that the model effectively leverages inter-fragment and intra-fragment contextual effects along with video-level preferences, outperforming existing methods. Additionally, we carry out empirical experiments to analyze the key components and parameters of the proposed model, providing further insights into its performance. 1

Encrypted traffic classification is essential for network security and management. However, the encrypted nature makes it challenging to extract representative features from raw traffic data. Existing end-to-end methods ignore byte correlations within packets and potential correlations among packets, hindering the learning of real traffic semantics and leading to suboptimal performance. This paper proposes MsETC, a multi-scale contrastive attention representation learning method for encrypted traffic classification. MsETC divides the raw packet byte sequence into multi-scale patches and then extracts dual views for contrastive learning from both the inter-patch and intra-patch perspectives. This allows the model to capture correlations among bytes within a packet as well as the potential interactions between packets. Extensive experiments on real-world datasets demonstrate that the proposed method achieves superior classification performance with lower complexity.

Group activities are important behaviors in human society, providing personalized recommendations for groups is referred to as the group recommendation task. Existing methods can usually be categorized into two strategies to infer group preferences: 1) determining group preferences by aggregating members' personalized preferences, and 2) inferring group consensus by capturing group members' coherent decisions after common compromises. However, the former would suffer from the lack of group-level considerations, and the latter overlooks the fine-grained preferences of individual users. To this end, we propose a novel group recommendation method AlignGroup, which focuses on both group consensus and individual preferences of group members to infer the group decision-making. Specifically, AlignGroup explores group consensus through a well-designed hypergraph neural network that efficiently learns intra- and inter-group relationships. Moreover, AlignGroup innovatively utilizes a self-supervised alignment task to capture fine-grained group decision-making by aligning the group consensus with members' common preferences. Extensive experiments on two real-world datasets validate that our AlignGroup outperforms the state-of-the-art on both the group recommendation task and the user recommendation task, as well as outperforms the efficiency of most baselines.

Federated learning (FL) inevitably confronts the challenge of system heterogeneity in practical scenarios. To enhance the capabilities of most model-homogeneous FL methods in handling system heterogeneity, we propose a training scheme that can extend their capabilities to cope with this challenge. In this paper, we commence our study with a detailed exploration of homogeneous and heterogeneous FL settings and discover three key observations: (1) a positive correlation between client performance and layer similarities, (2) higher similarities in the shallow layers in contrast to the deep layers, and (3) the smoother gradients distributions indicate the higher layer similarities. Building upon these observations, we propose InCo Aggregation that leverages internal cross-layer gradients, a mixture of gradients from shallow and deep layers within a server model, to augment the similarity in the deep layers without requiring additional communication between clients. Furthermore, our methods can be tailored to accommodate model-homogeneous FL methods such as FedAvg, FedProx, FedNova, Scaffold, and MOON, to expand their capabilities to handle the system heterogeneity. Copious experimental results validate the effectiveness of InCo Aggregation, spotlighting internal cross-layer gradients as a promising avenue to enhance the performance in heterogeneous FL.

The rapid expansion of the Internet of Things (IoT) has raised increasing concern about targeted cyber attacks. Previous research primarily focused on static Intrusion Detection Systems (IDSs), which employ offline training to safeguard IoT systems. However, such static IDSs struggle with real-world scenarios where IoT system behaviors and attack strategies can undergo rapid evolution, necessitating dynamic and adaptable IDSs. In response to this challenge, we propose AOC-IDS, a novel online IDS that features an autonomous anomaly detection module (ADM) and a labor-free online framework for continual adaptation. In order to enhance data comprehension, the ADM employs an Autoencoder (AE) with a tailored Cluster Repelling Contrastive (CRC) loss function to generate distinctive representation from limited or incrementally incoming data in the online setting. Moreover, to reduce the burden of manual labeling, our online framework leverages pseudo-labels automatically generated from the decision-making process in the ADM to facilitate periodic updates of the ADM. The elimination of human intervention for labeling and decision-making boosts the system’s compatibility and adaptability in the online setting to remain synchronized with dynamic environments. Experimental validation using the NSL-KDD and UNSW-NB15 datasets demonstrates the superior performance and adaptability of AOC-IDS, surpassing the state-of-the-art solutions. The code is released at https://github.com/xinchen930/AOC-IDS.

The rapid expansion of AI technology, driven by high-speed networks and high-performance mobile devices, enables personalized and cross-content recommendations across diverse applications. Recent methods consider a large number of textual content keywords or topics as labels for recommendations, transforming the problem into Extreme Multi-label Learning (XML). However, addressing the XML problem in AI-driven recommendation systems that handle extensive user-generated data while ensuring Quality of Service (QoS) faces two main challenges: significant computational costs and inferior tail label prediction performance. We propose SAT, a semantic-aware framework with a tree architecture that effectively tackles these challenges and demonstrates improved performance compared to well-established approaches.

The deployment of learning-based models to detect malicious activities in network traffic flows is significantly challenged by concept drift. With evolving attack technology and dynamic attack behaviors, the underlying data distribution of recently arrived traffic flows deviates from historical empirical distributions over time. Existing approaches depend on a significant amount of labeled drifting samples to facilitate the deep model to handle concept drift, which faces labor-intensive manual labeling and the risk of label noise. In this paper, we propose ReCDA, a Concept Drift Adaptation method with Representation enhancement, which consists of a self-supervised representation enhancement stage and a weakly-supervised classifier tuning stage. Specifically, in the initial stage, ReCDA introduces drift-aware perturbation and representation alignment to facilitate the model in acquiring robust representations from drift-aware and drift-invariant perspectives. Moreover, in the subsequent stage, a meticulously crafted instructive sampling strategy and a robust representation constraint encourage the model to learn discriminative knowledge about benign and malicious activities during fine-tuning, thereby enhancing performance further. We conduct comprehensive evaluations on several benchmark datasets under varying degrees of concept drift. The experiment results demonstrate the superior adaptability and robustness of the proposed method.

Epilepsy is a common condition that causes frequent seizures, significantly impacting patients’ daily lives. Non-invasive EEG is an effective tool for detecting seizure onset. Wearable EEG devices enable real-time monitoring and timely intervention but pose new algorithmic challenges on small model weight sizes and limited training data. Brain-inspired hyperdimensional computing (HDC) presents a potential solution for its small weight size and quick learning ability. Combining local binary pattern (LBP) codes with HDC can capture dynamic features in EEG time series. However, traditional LBP features may not offer sufficient robustness for trend modeling due to their high localization on individual samples, particularly on low-amplitude and non-stationary scalp EEG signals. To address the above challenges, this paper proposes a multi-scale LBP-based HDC (MSLBP-HDC) approach for scalp EEG analysis. Unlike traditional LBP-based HDC focusing only on the local change trend, the designed MSLBP-HDC extracts dynamic features at different time resolutions to detect abnormal cortical oscillations. The lengths of multiple temporal scales in MSLBP-HDC are determined based on the duration of spikes. Our results demonstrate that MSLBP-HDC has the highest specificity for all test seizure types and achieves competitive macroaveraging accuracy with the smallest model weight size in detection, compared to advanced deep learning, support vector machine, and HDC methods. Regarding few-shot learning performance, MSLBP-HDC outperforms existing approaches and achieves high accuracy using only 1% of the training data. Moreover, feature interpretability analysis from space and time domains highlights that MSLBP-HDC successfully extracts seizure-relevant features rather than noise or artifacts, ensuring the algorithm’s reliability.

The evolution of Internet-of-Things (IoT) is fostering the use of intelligent controls for energy conservation. Yet, the efficacy of these strategies is largely tied to diverse load forecasting algorithms. Given the significant contribution of heating, ventilation, and air-conditioning (HVAC) systems to global energy consumption, accurate forecasting of HVAC power usage is crucial for improving overall energy efficiency. However, real-world HVAC load forecasting, bolstered by various IoT devices, is complicated by multiple factors: data variability, power load fluctuations, electronic phenomena (e.g., zero drifts), and the increased time complexity and larger model sizes required to manage accumulating historical data. To address these challenges, we first present an in-depth measurement study on the characteristics of HVAC load at a minute scale based on HVAC data collected in six locations. We propose HALO, a transformer-based framework specifically designed for forecasting HVAC load. HALO incorporates an adaptive data pre-processing stage and a local-global-scale transformer-based load forecasting stage, enabling precise forecasting of HVAC load and optimization of energy utilization. Evaluation based on real-world data traces from a prototype application demonstrates that the proposed framework significantly outperforms existing models.

Smartphones are widely used as portable data collectors for wearable and healthcare sensors that can passively collect data streams related to the environment, health status, and behaviors. Recent research shows that the collected data can be used to monitor not only the physical states but also the mental health of individuals. However, extracting the features of digital phenotypes that characterize major depressive disorder (MDD) is technically challenging and may raise significant privacy concerns. Addressing such challenges has become the focus of many researchers. This article provides a comprehensive analysis of several key issues related to ubiquitous sensing to aid in detecting MDD. Specifically, this article analyzes existing methodologies and feature extraction algorithms used to detect possible MDD through digital phenotyping from smartphone data. In particular, five types of features are summarized and explained, namely, location, movement, rhythm, sleep, and social and device usage. Finally, related limitations and challenges are discussed to provide paths for further research and engineering.

Byzantine-robust federated learning aims at mitigating Byzantine failures during the federated training process, where malicious participants (known as Byzantine clients) may upload arbitrary local updates to the central server in order to degrade the performance of the global model. In recent years, several robust aggregation schemes have been proposed to defend against malicious updates from Byzantine clients and improve the robustness of federated learning. These solutions were claimed to be Byzantine-robust, under certain assumptions. Other than that, new attack strategies are emerging, striving to circumvent the defense schemes. However, there is a lack of systematical comparison and empirical study thereof. In this paper, we conduct an experimental study of Byzantine-robust aggregation schemes under different attacks using two popular algorithms in federated learning,FedSGDandFedAvg. We first survey existing Byzantine attack strategies, as well as Byzantine-robust aggregation schemes that aim to defend against Byzantine attacks. We also propose a new scheme,ClippedClustering, to enhance the robustness of a clustering-based scheme by automatically clipping the updates. Then we provide an experimental evaluation of eight aggregation schemes in the scenario of five different Byzantine attacks. Our experimental results show that these aggregation schemes sustain relatively high accuracy in some cases, but they are not effective in all cases. In particular, our proposedClippedClusteringsuccessfully defends against most attacks under independent and identically distributed (IID) local datasets. However, when the local datasets are Non-IID, the performance of all the aggregation schemes significantly decreases. With Non-IID data, some of these aggregation schemes fail even in the complete absence of Byzantine clients. Based on our experimental study, we conclude that the robustness of all the aggregation schemes is limited, highlighting the need for new defense strategies, in particular for Non-IID datasets.

Human gait refers to a daily motion that represents not only mobility but can also be used to identify the walker by either human observers or computers. Recent studies reveal that gait even conveys information about the walker’s emotion. Individuals in different emotion states may show different gait patterns. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. Compared to traditional emotion detection biometrics, such as facial expression, speech, and physiological parameters, gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject. These advantages make gait a promising source for emotion detection. This article reviews current research on gait-based emotion detection, particularly on how gait parameters can be affected by different emotion states and how the emotion states can be recognized through distinct gait patterns. We focus on the detailed methods and techniques applied in the whole process of emotion recognition: data collection, preprocessing, and classification. Finally, we discuss possible future developments of efficient and effective gait-based emotion recognition using state-of-the-art techniques in intelligent computation and big data.

Due to the revolution of communication technology, the rapidly increasing number of mobile devices in edge networks generates various real-time service requests, requiring a considerable volume of heterogeneous resources all the time. However, edge devices with limited resources cannot afford substantial learning cost, while migrating services requires heterogeneous resources, especially for dynamic networks. To address these issues, we first establish a cooperative service migration framework and formulate a bi-objective optimization problem to optimize service performance and cost. By analyzing the optimal migration ratio of service cooperative migration, we propose an offline expert policy based on global states to provide optimal expert demonstrations. To realize real-time service migration based on observable states, we design a lightweight online agent policy to imitate expert demonstrations and leverage meta update to accelerate the model transfer. Experimental results show that our algorithm is exceptional in training cost and accuracy, and has significant superiors in multiple metrics such as the service latency and payment under different workloads, compared to other representative algorithms.

The correlation between children's personal and family characteristics (e.g., demographics and socioeconomic status) and their physical and mental health status has been extensively studied across various research domains, such as public health, medicine, and data science. Such studies can provide insights into the underlying factors affecting children's health and aid in the development of targeted interventions to improve their health outcomes. However, with the availability of multiple data sources, including context data (i.e., the background information of children) and motion data (i.e., sensor data measuring activities of children), new challenges have arisen due to the large-scale, heterogeneous, and multimodal nature of the data. Existing statistical hypothesis-based and learning model-based approaches have been inadequate for comprehensively analyzing the complex correlation between multimodal features and multi-dimensional health outcomes due to the limited information revealed. In this work, we first distill a set of design requirements from multiple levels through conducting a literature review and iteratively interviewing 11 experts from multiple domains (e.g., public health and medicine). Then, we propose HealthPrism, an interactive visual and analytics system for assisting researchers in exploring the importance and influence of various context and motion features on children's health status from multi-levelperspectives. Within HealthPrism, a multimodal learning model with a gate mechanism is proposed for health profiling and cross-modality feature importance comparison. A set of visualization components is designed for experts to explore and understand multimodal data freely. We demonstrate the effectiveness and usability of HealthPrism through quantitative evaluation of the model performance, case studies, and expert interviews in associated domains.

Many countries around the world are facing a shortage of healthcare resources, especially during the post-epidemic era, leading to a dramatic increase in the need for self-detection and self-management of diseases. The popularity of smart wearable devices, such as smartwatches, and the development of machine learning (ML) bring new opportunities for the early detection and management of various prevalent diseases, such as cardiovascular diseases, Parkinson’s disease, and diabetes. In this survey, we comprehensively review the articles related to specific diseases or health issues based on small wearable devices and ML. More specifically, we first present an overview of the articles selected and classify them according to their targeted diseases. Then, we summarize their objectives, wearable device and sensor data, ML techniques, and wearing locations. Based on the literature review, we discuss the challenges and propose future directions from the perspectives of privacy concerns, security concerns, transmission latency and reliability, energy consumption, multimodality, multisensor, multidevices, evaluation metrics, explainability, generalization and personalization, social influence, and human factors, aiming to inspire researchers in this field.

Federated learning (FL) is a promising paradigm to empower on-device intelligence in Industrial Internet of Things (IIoT) due to its capability of training machine learning models across multiple IIoT devices while preserving the privacy of their local data. However, the distributed architecture of FL relies on aggregating the parameter list from the remote devices, which poses potential security risks caused by malicious devices. In this article, we propose a flexible and robust aggregation rule, called auto-weighted geometric median (AutoGM), and analyze the robustness against outliers in the inputs. To obtain the value ofAutoGM, we design an algorithm based on the alternating optimization strategy. UsingAutoGMas aggregation rule, we propose two robust FL solutionsAutoGM_FLandAutoGM_PFL.AutoGM_FLlearns a shared global model using the standard FL paradigm, andAutoGM_PFLlearns a personalized model for each device. We conduct extensive experiments on the FEMNIST and Bosch IIoT datasets. The experimental results show that our solutions are robust against both model poisoning and data poisoning attacks. In particular, our solutions sustain high performance even when 30% of the nodes perform model or 50% of the nodes perform data poisoning attacks.

Traffic congestion is a severe challenge for intelligent transportation system (ITS). So far, if traffic congestion is perceived in a route, a common solution is searching for another congestion-free route. However, it is observed that not all congestions should be tackled with rerouting, since the extra overhead (e.g., extra travel time, extra fuel consumption, and extra CO2 emission) caused by certain congestions might be lower than that of re-routing. Against this backdrop, an intelligent route guidance strategy is proposed, in which vehicles will trade off the extra overhead of re-routing and that of waiting in congestion. Firstly, the perceived congestion is divided into four basic types. Then, the prediction method about the duration of each congestion type is formulated. Finally, the intelligent route planning mechanism is developed. Simulations demonstrate that the proposed strategy can reduce the travel time, fuel consumption, and CO2 emission for vehicles.

Considering the microphone is easily affected by noise and soundproof materials, the radio frequency (RF) signal is a promising candidate to recover audio as it is immune to noise and can traverse many soundproof objects.In this paper, we introduce Radio2Speech, a system that uses RF signals to recover high quality speech from the loudspeaker.Radio2Speech can recover speech comparable to the quality of the microphone, advancing from recovering only single tone music or incomprehensible speech in existing approaches.We use Radio UNet to accurately recover speech in time-frequency domain from RF signals with limited frequency band.Also, we incorporate the neural vocoder to synthesize the speech waveform from the estimated time-frequency representation without using the contaminated phase.Quantitative and qualitative evaluations show that in quiet, noisy and soundproof scenarios, Radio2Speech achieves state-of-the-art performance and is on par with the microphone that works in quiet scenarios.

Knowledge on water-use patterns in residential settings can help policymakers formulate well-targeted water conservation measures and evaluate the efficacy of such measures. Prior studies have applied machine learning techniques to disaggregate household-level water consumption data, collected by smart meters, into specific end-use categories, such as showering, basin use, kitchen use, and use by washing machine. However, the analysis of the effect of the sampling interval on the classification accuracy level remains an underinvestigated issue. This article seeks to fill this knowledge gap by identifying an optimal sampling interval that can achieve a high level of classification accuracy while overcoming constraints of on-device data storage, data transmission, and energy consumption. To understand the benefits of collecting fine-grained data for machine learning, we have built a high-resolution tap-based Internet of Things (IoT) system comprising a set of Wi-Fi-based tap sensors, gateway infrastructure, and a secure data processing pipeline. Based on empirical tap-based data collected over an eight-month period, we concluded that when the sampling interval decreases slightly from 5 to 1 s, the accuracy level of the end-use classification model increases significantly from 66.6% to 76.1 %. This article also highlights the challenges of deploying IoT sensors to collect water consumption data in a domestic setting. In order to collect sufficient ground-truth data for the training and verification of a generalizable water end-use disaggregation model, it is necessary to sophisticate the flow data collection system by adopting low-power wide-area network technologies and reducing the level of energy consumption of the flow sensing components.

The emerging applications for smart cities intend to promote the quality of citizens’ life. Among them, ubiquitous user connectivity and real-time computation offloading are significant for the ever-increasing requirements of delay-sensitive and mission-critical applications. By integrating human social behaviors (such as relationship, similarity, community, and social ties) with physical Internet of Things (IoT) systems, social IoT systems are promising to provide ubiquitous connectivity among users. As the applications of social IoT systems are transferring from information dissemination to user entertainment (such as image identification, online games, and augmented reality), computation offloading is significant to reduce the execution delay of applications.

Federated learning provides a communication-efficient and privacy-preserving training process by enabling learning statistical models with massive participants without accessing their local data. Standard federated learning techniques that naively minimize an average loss function are vulnerable to data corruptions from outliers, systematic mislabeling, or even adversaries. In this article, we address this challenge by proposing Auto-weighted Robust Federated Learning ( ARFL ), a novel approach that jointly learns the global model and the weights of local updates to provide robustness against corrupted data sources. We prove a learning bound on the expected loss with respect to the predictor and the weights of clients, which guides the definition of the objective for robust federated learning. We present an objective that minimizes the weighted sum of empirical risk of clients with a regularization term, where the weights can be allocated by comparing the empirical risk of each client with the average empirical risk of the best \( p \) clients. This method can downweight the clients with significantly higher losses, thereby lowering their contributions to the global model. We show that this approach achieves robustness when the data of corrupted clients is distributed differently from the benign ones. To optimize the objective function, we propose a communication-efficient algorithm based on the blockwise minimization paradigm. We conduct extensive experiments on multiple benchmark datasets, including CIFAR-10, FEMNIST, and Shakespeare, considering different neural network models. The results show that our solution is robust against different scenarios, including label shuffling, label flipping, and noisy features, and outperforms the state-of-the-art methods in most scenarios.

Traffic violations have become one of the major threats to urban transportation systems, undermining road safety and causing economic losses. Although various methods have been proposed by road authorities and researchers to find out the possible causes of traffic violations, existing methods often fail to diagnose traffic violations from drivers’ perspectives and contexts or consider their visual and comprehension loads while driving. In this work, we propose a driver-centered simulation platform to inspect drivers’ loads in traffic violation hotspots. Specifically, we first build a driving simulator based on the 3D point clouds of real-world traffic violation hotspots. We then recruit drivers to simulate driving in designated traffic scenes. Indicators for drivers’ visual and comprehension loads are derived based on drivers’ feedback. Upon this basis, we build an explainable model to automatically indicate drivers’ visual and comprehension loads under various crowd-sensed traffic scenes. Experiments using real-world data from a Chinese City (Xiamen) and case studies show that our approach successfully derives a set of prominent indicators to effectively diagnose drivers’ visual and comprehension loads in real-world traffic violation hotspots.

Wireless sensor networks have enabled smart infrastructures and novel applications. With the recent roll-out of Narrowband IoT (NB-IoT) cellular radio technology, wireless sensors can be widely deployed for data collection in cities around the world. However, empirical evidence regarding the coverage and connectivity of NB-IoT in dense urban areas is limited. This article presents an empirical study that focuses on evaluating the coverage and connectivity of NB-IoT in a dense urban environment. We have designed an NB-IoT sensor node and deployed over 100 of them in high-rise apartment buildings in Hong Kong. These sensor nodes utilize a commercial NB-IoT network to collect high-resolution water flow data for machine learning model training and provide timely feedback to users. We collect and analyze the empirical NB-IoT signal measurements from the sensor nodes deployed in various challenging outdoor and indoor environments for over three months. These empirical measurements reveal correlations between NB-IoT connectivity and sensor installation environments. We also observe that inter-cell interference, as a result of coverage by multiple neighboring NB-IoT cells in a dense urban environment, is a source of connectivity degradation. We discuss potential issues that IoT application designers and system integrators might encounter in practical NB-IoT devices deployment, and we propose a transmission decision algorithm based on signal measurements for mitigating energy wasted due to transmission failures. Finally, we demonstrate the results and the benefits of using high-resolution water flow data collected by our purpose-built NB-IoT sensor nodes for studying the patterns of domestic water consumption in Hong Kong.

Federated learning (FL) is able to manage edge devices to cooperatively train a model while maintaining the training data local and private. One common assumption in FL is that all edge devices share the same machine learning model in training, for example, identical neural network architecture. However, the computation and store capability of different devices may not be the same. Moreover, reducing communication overheads can improve the training efficiency though it is still a challenging problem in FL. In this paper, we propose a novel FL method, called FedHe, inspired by knowledge distillation, which can train heterogeneous models and support asynchronous training processes with significantly reduced communication overheads. Our analysis and experimental results demonstrate that the performance of our proposed method is better than the state-of-the-art algorithms in terms of communication overheads and model accuracy.

The Internet of Medical Things (IoMT) is an extension and specialization of that original Internet of Things (IoT) concept, and applies to the interconnectedness of devices, software applications, and data which are specific to the medical industry. IoMT can add smart technologies to medical devices to monitor the progression of a disease away from the doctor’s office and learn things that could impact future care guidelines and patients. It can also provide a better way to care for our elderly by tracking vitals and heart performance, glucose and other body systems, and activity and sleeping levels. During the outbreak of pandemic (e.g., COVID-19), IoMT can even be used to detect main symptoms ubiquitously using intelligent sensors and trace the origin of the outbreak based on aggregated IoT data (e.g., geographic mobile data and purchase history). Although most of the contemporary IoMT systems can measure risks, make decisions, and take actions automatically, the lack of emotion-aware abilities will be an obstacle to more harmonious human–machine interaction and more efficient medical process. Besides, mental disorders, such as depression, schizophrenia, and anxiety, have become a more noticeable cause of suffering. The integration of emotion-aware abilities into IoMT can also contribute to monitor emotional dysregulation continuously in subjects with mental disorders or undergoing serious pandemic such as COVID-19, and give these patients personalized therapy recommendations. Research on affective computing has defined a framework to recognize, interpret, and process human affects, but more research is needed to investigate its application to biomedical applications, especially “in the wild” and over extended periods of time, and how to integrate emotion-aware abilities into IoMT organically is still an open question. This special issue aims to create a platform for researchers, developers, and practitioners from both academia and industry to disseminate the state-of-the-art results and to advance the Emotion-Aware ubiquitous computing in IoMT.

The papers in this special section focus on pervasive edge computing (PEC)for industrial Internet of Things. With the development of 5G technology and intelligent terminals, computation, communication, and storage capacities of devices are largely improved. Based on that, pervasive edge computing (PEC) becomes possible, where data can be processed on the network edge with the assistant of those intelligent terminals enhanced by 5G technology except the management of any centralized servers, including clouds and remote servers. The papers in this section solicits original research and practical contributions which advance PEC in industrial IoTs (IIoTs), regarding the architecture, technologies, and applications.

In modern distributed cyber-physical systems (CPS), information fusion often plays a key role in automate and self-adaptive decision making process. However, given the heterogeneous and distributed nature of modern CPSs, it is a great challenge to operate CPSs with the compromised data integrity and unreliable communication links. In this paper, we study the distributed state estimation problem under the false data injection attack (FDIA) with probabilistic communication networks. We propose an integrated ”detection + fusion” solution, which is based on the Kullback-Leibler divergences (KLD) between local posteriors and therefore does not require the exchange of raw sensor data. For the FDIA detection step, the KLDs are used to cluster nodes in the probability space and to partition the space into secure and insecure subspaces. By approximating the distribution of the KLDs with a general$\chi ^2$distribution and calculating its tail probability, we provide an analysis of the detection error rate. For the information fusion step, we discuss the potential risk of double counting the shared prior information in the KLD-based consensus formulation method. We show that if the local posteriors are updated from the shared prior, the increased number of neighbouring nodes will lead to the diminished information gain. To overcome this problem, we propose a near-optimal distributed information fusion solution with properly weighted prior and data likelihood. Finally, we present simulation results for the integrated solution. We discuss the impact of network connectivity on the empirical detection error rate and the accuracy of state estimation.

With the expansion of the IoT, it is important to optimize available bandwidth to reliably support edge to device communications. Thus we propose a wireless network where each edge server communicates with its end devices using its wireless band as a primary channel, assisted by a secondary edge server that can relay communications via its own wireless band as a secondary channel. The network can optimize capacity by balancing load between primary and secondary wireless bands, and we analyze the geometry of achievable rate regions, depending on the state of bands modeled as Rayleigh fading channels. The allocation of a connection to the primary or secondary band is formulated as an optimization problem which is then solved, and illustrated with numerical examples.

As the challenge of depression problems increases today, it is important to effective and timely detect for patients' treatments and depression prevention. Current methods of automated depression diagnosis depend almost entirely on audio, video, and Electroencephalogram (EEG) etc. In this paper, we propose a novel method to detect depression using gait data that is collected by Kinect camera. Its key components include a camera rectification method and a rigid-body representation of the human body. The camera rectification method achieves the purpose of improving data processing accuracy by changing the camera angle. The rigid-body representation can not only improve the robustness of detecting depression patients with noisy input, but also can reduce the classification time. We evaluate our method on the depression gait dataset in postgraduate students. The proposed method has an outstanding performance in classic machine learning algorithms, and the best accuracy can achieve 88.89%. Our solution provides a new method for automatic depression detection (ADD) that has exciting implications in clinical theory and practice, and has the advantages of high accuracy, inexpensive, low time cost, and no-contact.

Motivated by complementing the ubiquitous wireless sensor networks (WSNs) and powerful cloud computing (CC), a lot of attention from both industry and academia has been drawn to Sensor-Cloud (SC). However, SC pricing is barely investigated. Towards pricing for SC, this paper 1) introduces five SC Pricing Models (SCPMs) first. Specifically, to charge a SC user, each SCPM considers one of the following factors respectively: i) the lease period of the SC user; ii) the required working time of SC; iii) the SC resources utilized by the SC user; iv) the volume of sensory data obtained by the SC user; v) the SC path that transmits sensory data from the WSN to the SC user. Further, this paper 2) performs analysis to discuss and exhibit the characteristics of the proposed SCPMs. With that, this paper 3) presents the case studies regarding the application of SCPMs. Eventually, this paper 4) conducts a review about the user behavior study. This paper aims to serve as a very favorable guidance for future research about pricing in SC.

The public's purchase incentive increases dramatically during the holiday season and subsequently returns to normal levels. This seasonality is common in various scenarios and highlights the following questions: how does the public's purchase incentive fluctuate over the course of a year? Which factors are conducive to this seasonal behavior and how can they be modeled? In this paper, we propose a model that explicitly integrates temporal point process theory with the construction of a networked community, to describe the dynamics of collective action propagation with seasonal fluctuation. Furthermore, a database is constructed of sales records for 21 video game consoles and 13 237 video games in France, Germany, Japan, the U.K., the USA, and worldwide from 1989 to 2018. Experimental results suggest that peak desire always appears in the holiday season about one week before Christmas and is about four times higher than consumption desire in a normal period in all areas.

Mobile edge computing (MEC) has risen as a promising paradigm to provide high quality of experience via relocating the cloud server in close proximity to smart mobile devices (SMDs). In MEC networks, the MEC server with computation capability and storage resource can jointly execute the latency-sensitive offloading tasks and cache the contents requested by SMDs. In order to minimize the total latency consumption of the computation tasks, we jointly consider computation offloading, content caching, and resource allocation as an integrated model, which is formulated as a mixed integer nonlinear programming (MINLP) problem. We design an asymmetric search tree and improve the branch and bound method to obtain a set of accurate decisions and resource allocation strategies. Furthermore, we introduce the auxiliary variables to reformulate the proposed model and apply the modified generalized benders decomposition method to solve the MINLP problem in polynomial computation complexity time. Simulation results demonstrate the superiority of the proposed schemes.

Unmanned aerial vehicle (UAV) has been witnessed as a promising approach for offering extensive coverage and additional computation capability to smart mobile devices (SMDs), especially in the scenario without available infrastructures. In this paper, a UAV-assisted mobile edge computing system with stochastic computation tasks is investigated. The system aims to minimize the average weighted energy consumption of SMDs and the UAV, subject to the constraints on computation offloading, resource allocation, and flying trajectory scheduling of the UAV. Due to nonconvexity of the problem and the time coupling of variables, a Lyapunov-based approach is applied to analyze the task queue, and the energy consumption minimization problem is decomposed into three manageable subproblems. Furthermore, a joint optimization algorithm is proposed to iteratively solve the problem. Simulation results demonstrate that the system performance obtained by the proposed scheme can outperform the benchmark schemes, and the optimal parameter selections are concluded in the experimental discussion.

Real-time accurate air quality data is very important for pollution exposure monitoring and urban planning. However, there are limited high-quality air quality monitoring stations (AQMS) in cities due to their high equipment costs. To provide real-time and accurate data covering large area, this paper proposes a novel scheme that jointly considers online multi-hop calibration and spatio-temporal coverage in route selection for mobile sensors. A novel sensor carrier selection problem (SCSP) is formulated, which aims to maximize the spatio-temporal coverage ratio and guarantee the accuracy of measurements through sensor calibration. An online Bayesian based collaborative calibration (OBCC) scheme is proposed to relax the multi-hop calibration constraint in the SCSP. Based on the OBCC, a multi-hop calibration judgment algorithm (MCJA) is proposed to decide whether the data accuracy of a given set of routes can be guaranteed through collaborative calibration. Furthermore, a heuristic sensor route selection algorithm (SRSA) is then developed to solve the SCSP.

Blockchain technology becomes increasingly popular. It also attracts scams, for example, Ponzi scheme, a classic fraud, has been found making a notable amount of money on Blockchain, which has a very negative impact. To help dealing with this issue, this paper proposes an approach to detect Ponzi schemes on blockchain by using data mining and machine learning methods. By verifying smart contracts on Ethereum, we first extract features from user accounts and operation codes of the smart contracts and then build a classification model to detect latent Ponzi schemes implemented as smart contracts. The experimental results show that the proposed approach can achieve high accuracy for practical use. More importantly, the approach can be used to detect Ponzi schemes even at the moment of its creation. By using the proposed approach, we estimate that there are more than 400 Ponzi schemes running on Ethereum. Based on these results, we propose to build a uniform platform to evaluate and monitor every created smart contract for early warning of scams.

The Internet of Things (IoT) enables environmental monitoring by collecting data from sensing devices, including cameras and microphones. The popularity of smartphones enables mobile users to communicate and collect data from their surrounding sensing devices. The mobile devices can obtain useful environmental data from nearby sensors through short-range communication such as Bluetooth. Nevertheless, the limited contact time and the wireless capacity constrain the amount of data to be collected. With the increasing amount of multimedia big data such as videos and pictures from cameras, it is crucial for mobile users to collect prioritized data that can maximize their data utility. In this paper, we propose a distributed algorithm to provide information-centric ubiquitous data collection of multimedia big data by mobile users in the IoT. The algorithm can handle transmissions of multimedia big data recorded by the surrounding cameras and sensors, and prioritize the transmissions of the most important and relevant data. The mobile users construct data collection trees adaptively according to their dynamic moving speeds and the value of information carried by the multimedia and sensor data. The distributed algorithm can support smooth data collection and coordination of multiple mobile users. We provide both numerical analysis and extensive simulations to evaluate the information value, energy efficiency and scalability of our solution. The results showed that our distributed algorithm can improve the value of information up to 50% and reduce energy consumption to half compared with existing approach. Our algorithm also scales perfectly well with increasing number of mobile users and dynamic moving speeds.

Mobile edge computing (MEC) brings computation capacity to the edge of mobile networks in close proximity to smart mobile devices (SMDs) and contributes to energy saving compared with local computing, but resulting in increased network load and transmission latency. To investigate the tradeoff between energy consumption and latency, we present an energy-aware offloading scheme, which jointly optimizes communication and computation resource allocation under the limited energy and sensitive latency. In this paper, single and multicell MEC network scenarios are considered at the same time. The residual energy of smart devices' battery is introduced into the definition of the weighting factor of energy consumption and latency. In terms of the mixed integer nonlinear problem for computation offloading and resource allocation, we propose an iterative search algorithm combining interior penalty function with D.C. (the difference of two convex functions/sets) programming to find the optimal solution. Numerical results show that the proposed algorithm can obtain lower total cost (i.e., the weighted sum of energy consumption and execution latency) comparing with the baseline algorithms, and the energy-aware weighting factor is of great significance to maintain the lifetime of SMDs.

Wireless sensor networks (WSN) based on unmanned aerial vehicles (UAV) are ideal platforms for monitoring dynamics over larger service area. On the other hand, aerial sensor networks (ASNs) are often required to be connected with a command center for sending data and receiving control messages in real time. In this paper, we study the problem of path planning for ASNs with connectivity constraints. The primary goal of path planning is driving UAVs to locations where the most informative measurements can be collected. Meanwhile, the worst link's capacity is assured to be greater than a pre-defined requirement. We proposed a solution for the path planning problem and it consists of two modules: network coordinator (NC) and motion controller (MC). The NC manages the topology of relay-assisted wireless communication networks. For the design of MC, we compare two motion strategies: maximum entropy and maximum mutual information. The simulation results show that our proposed solution achieves accurate signal reconstruction while maintaining the connectivity. We conclude that it's important to enable UAV-to-UAV communications for future ASN-based applications.

Batteries of modern mobile devices remain severely limited in capacity, which makes energy consumption a key concern for mobile applications, particularly for the computation-intensive video applications. Mobile devices can save energy by offloading computation tasks to the cloud, yet the energy gain must exceed the additional communication cost for cloud migration to be beneficial. The situation is further complicated by real-time video applications that have stringent delay and bandwidth constraints. In this paper, we closely examine the performance and energy efficiency of representative mobile cloud applications under dynamic wireless network channels and state-of-the-art mobile platforms. We identify the unique challenges of and opportunities for offloading real-time video applications and develop a generic model for energy-efficient computation offloading accordingly in this context. We propose a scheduling algorithm that makes adaptive offloading decisions in fine granularity in dynamic wireless network conditions and verify its effectiveness through trace-driven simulations. We further present case studies with advanced mobile platforms and practical applications to demonstrate the superiority of our solution and the substantial gain of our approach over baseline approaches.

The papers in this special section focus on the Internet of Things for use in smart cities and urban informatics. CITIES around the world are currently under quick transition toward a low-carbon environment, high quality of living, and resource-efficient economy. Smart systems are crucial components for building sustainable cities and aggregating comprehensive urban informatics. Internet-of-Things (IoT) is an emerging technology evolved from the convergence of wireless technologies, embedded systems, and the Internet. Extensive research on IoT is taking place on sensing and automatic control, network infrastructure and communication, and big data analytics. This facilitates a multidisciplinary approach for developing integrated solutions and creating novel applications to build a sustainable society. Combinations of IoT with autonomous systems, machine intelligence, sensor and actuator networks provide smart functions in wide application domains, including environmental monitoring, smart transportation, industrial production, and security surveillance. This special section on Internet-of-Things for smart cities and urban informatics is focused on the development, adoption, and application of IoT technology to improve the efficiency of cities and enhance urban services in society. It covers topics including smart transportation, smart buildings, data transmission and analytics, system designs, and experiments of IoT for smart cities.

This paper proposes a novel power management solution for resource-constrained devices in the context of Internet of Things (IoT). We focus on smartphones in the IoT, as they are getting increasingly popular and equipped with strong sensing capabilities. Smartphones have complex and asynchronous power consumption incurred by heterogeneous components including their on-board sensors. Their interaction with the cloud allows them to offload computation tasks and access remote data storage. In this work, we aim at monitoring the power consumption behaviours of the smartphones, profiling both individual applications and the system as a whole, to make better decisions in power management. We design a cloud orchestration architecture as an epic predictor of behaviours of smart devices by extracting their application characteristics and resource utilization. We design and implement this architecture to perform energy profiling and data analysis on massive data logs. This cloud orchestration architecture coordinates a number of cloud-based services and supports dynamic workflows between service components, which can reduce energy consumption in the energy profiling process itself. Experimental results showed that small portion of applications dominate the energy consumption of smartphones. Heuristic profiling can effectively reduce energy consumption in data logging and communications without scarifying the accuracy of power monitoring.

In this paper, the multidimensional output Gaussian process (GP) is applied to model urban environmental data collected by sensor networks. Measurements from sensors at different locations are correlated. Moreover, we observe that the pollution level in urban area is highly coupled with human activities and shows periodic patterns accordingly. Based on these observations, we discuss the design of mean and kernel functions with two approaches: (1) composed kernel and maximum likelihood estimation of hyper-parameters, (2) Wiener-Khinchin theorem based approximation of sample covariances. To validate the models, the accuracy of interpolations given by different approaches are compared. The experimental results show that, for the application of interpolation, the dependent GP with the approximated sample covariances as kernels can provide better performance than the independent GP model with composed kernels.

Visual codes are used to embed digital data in physical objects, or they are shown in video sequences to transfer data over screen/camera links. Existing codes either carry limited data to make them robust against a range of channel conditions (e.g., low camera quality or long distances), or they support a high data capacity but only work over a narrow range of channel conditions. We present FOCUS, a new code design that does not require this explicit trade-off between code capacity and the reader's channel quality. Instead, FOCUS builds on concepts from OFDM to encode data at different levels of spatial detail. This enables each reader to decode as much data from a code as its channel quality allows. We build a prototype of FOCUS devices and evaluate it experimentally. Our results show that FOCUS gracefully adapts to the reader's channel, and that it provides a significant performance improvement over recently proposed designs, including Strata and PixNet.

The Internet of Things (IoT) is connecting people and smart devices on a scale that was once unimaginable. One major challenge for the IoT is to handle vast amount of sensing data generated from the smart devices that are resource-limited and subject to missing data due to link or node failures. By exploring cloud computing with the IoT, we present a cloud-based solution that takes into account the link quality and spatio-temporal correlation of data to minimize energy consumption by selecting sensors for sampling and relaying data. We propose a multiphase adaptive sensing algorithm with belief propagation (BP) protocol (ASBP), which can provide high data quality and reduce energy consumption by turning on only a small number of nodes in the network. We formulate the sensor selection problem and solve it using both constraint programming (CP) and greedy search. We then use our message passing algorithm (BP) for performing inference to reconstruct the missing sensing data. ASBP is evaluated based on the data collected from real sensors. The results show that while maintaining a satisfactory level of data quality and prediction accuracy, ASBP can provide load balancing among sensors successfully and preserves 80% more energy compared with the case where all sensor nodes are actively involved.

In recent years, there has been a growing interest in the ability of embedded devices, sensors, and actuators to communicate, and create a ubiquitous cyber-physical world. The growth of the notion of the Internet of Things (IoT) and the rapid development of technologies such as short range mobile communication and improved energy-efficiency is expected to create a pervasive connection of “things.” This will inevitably result in the generation of enormous amount of data, which have to be stored, processed, and accessed. Cloud computing has long been recognized as a paradigm for big data storage and analytics. The combination of cloud computing and IoT can enable ubiquitous sensing services and powerful processing of sensing data streams beyond the capability of individual things, thus stimulating innovations in both fields. For example, cloud platforms allow the sensing data to be stored and used intelligently for smart monitoring and actuation with the smart devices. Novel data fusion algorithms, machine learning methods, and artificial intelligence techniques can be implemented and run centralized or distributed on the cloud to achieve automated decision making. These will boost the development of new applications, such as smart cities, grids, and transportation systems. New challenges, however, arise when IoT meets cloud—there is an urgent need for novel network architectures that seamlessly integrate them, and protocols that facilitate big data streaming from IoT to the cloud. QoS and QoE, as well as data security, privacy, and reliability, are critical concerns during the integration.

Abstract Sinks usually broadcast their addresses for data collection in sensor networks. However, this common operation opens up vulnerability for adversary to attack the sinks and obstruct their normal functions. In this paper, we suggest sink anonymity as a novel approach for data collection, which protects the privacy of the sinks and avoids them from becoming the target of attacks. We provide sink anonymity by omitting the address of the sink in routing, so that the identity and location of the sink are kept private. Our proposed Randomized Routing with Hidden Address (RRHA) scheme prevents the attackers from obtaining the receiver address by capturing the destination field of the packets or predicting the location of the sinks by observing the flow of network traffic. We examined the successful delivery rate, packet travel delay, and protection strength of our proposed scheme by both analysis and simulations. Copyright © 2010 John Wiley & Sons, Ltd.

This paper proposes a novel power management solution for resource-constrained devices in the context of Internet of Things (IoT). We focus on smartphones in the IoT, as they are getting increasingly popular and equipped with strong sensing capabilities. Smartphones have complex and asynchronous power consumption incurred by heterogeneous components including their on-board sensors. Their interaction with the cloud allows them to offload computation tasks and access remote data storage. In this work, we aim at monitoring the power consumption behaviours of the smartphones, profiling both individual applications and the system as a whole, to make better decisions in power management. We design a cloud orchestration architecture as an epic predictor of behaviours of smart devices by extracting their application characteristics and resource utilization. We design and implement this architecture to perform energy profiling and data analysis on massive data logs. This cloud orchestration architecture coordinates a number of cloud-based services and supports dynamic workflows between service components, which can reduce energy consumption in the energy profiling process itself. Experimental results showed that small portion of applications dominate the energy consumption of smartphones. Heuristic profiling can effectively reduce energy consumption in data logging and communications without scarifying the accuracy of power monitoring.

Incorporating ubiquitous wireless sensor networks (WSNs) and powerful cloud computing (CC), Sensor-Cloud (SC) is attracting growing attention from both academia and industry. However, pricing for SC is barely explored. In this paper, filling this gap, five SC pricing models (i.e., SCPM1, SCPM2, SCPM3, SCPM4 and SCPM5) are proposed first. Particularly, they charge a SC user, based on 1) the lease period of the user, 2) the required working time of SC, 3) the SC resources utilized by the user, 4) the volume of sensory data obtained by the user, 5) the SC path that transmits sensory data from the WSN to the user, respectively. Further, analysis is also presented to study and demonstrate the performance of the proposed SCPMs. We believe that the pricing designs and analysis performed in this work could be a very valuable guidance for future researches regarding pricing in SC.

Location based services are getting increasingly popular in participatory sensing systems. They make use of location information on the mobile devices to support applications that improve personal health, object search, and entertainment. However, GPS positioning consumes a lot of energy, which can drain a mobile device's battery. Although WiFi localization and cell tower localization have been suggested as alternatives, they have lower localization accuracy and limited coverage. In this paper, we suggest a novel solution for multiple mobile devices to perform collaborative localization to reduce energy consumption and provide accurate localization. We divide the mobile devices into two groups, the aggregator group and the collector group. The aggregator group turns on their GPS periodically, while the collector group uses the locations of the aggregators to estimate their own locations. We formulate the aggregator set selection problem and propose two novel algorithms to minimize the energy consumption in collaborative localization. Simulations with real traces showed that our proposed solution can save up to 88% of the energy of the entire network.

Web users are immersed in their roles as information producers and propagation pushers. They are unaware of being potential threats to privacy-protection towards themselves and their friends. It is necessary to know who they should beware of most in their friend-networks once their privacy information is divulged inadvertently. In this paper, we aim to identify the vulnerable friend who maximizes the dissemination of privacy information. First we develop a Privacy Receiving-Disseminating (PRD) model to simulate the iterative course of privacy information dissemination within social graph. The subgraph constituted of those users who are involved in the dissemination, called Ultimate Circle of Disseminating (UCD), is then detected by an iterative algorithm. The contribution of each direct friend could be evaluated by comparing the disseminating intensities of detected UCDs before and after unfriending himself. The performance of our work has been validated empirically with the comparison of different unfriending strategies.

The increasingly popular smartphones enable participatory sensing systems to collect location-based sensing data for different tasks. However, GPS positioning is very energy consuming, which could drain a mobile device's battery quickly. High energy consumption may threaten the participants and reduce the sustainability of the participatory sensing systems. In this paper, we propose a collaborative localization strategy with load balancing. Simulations with real traces showed that our proposed solution can save more than 80% of the energy consumption for localization in the entire network with load balancing.

Driving is an integral part of our everyday lives, but it is also a time when people are uniquely vulnerable. Previous research has demonstrated that not only does listening to suitable music while driving not impair driving performance, but it could lead to an improved mood and a more relaxed body state, which could improve driving performance and promote safe driving significantly. In this article, we propose SAfeDJ, a smartphone-based situation-aware music recommendation system, which is designed to turn driving into a safe and enjoyable experience. SAfeDJ aims at helping drivers to diminish fatigue and negative emotion. Its design is based on novel interactive methods, which enable in-car smartphones to orchestrate multiple sources of sensing data and the drivers' social context, in collaboration with cloud computing to form a seamless crowdsensing solution. This solution enables different smartphones to collaboratively recommend preferable music to drivers according to each driver's specific situations in an automated and intelligent manner. Practical experiments of SAfeDJ have proved its effectiveness in music-mood analysis, and mood-fatigue detections of drivers with reasonable computation and communication overheads on smartphones. Also, our user studies have demonstrated that SAfeDJ helps to decrease fatigue degree and negative mood degree of drivers by 49.09% and 36.35%, respectively, compared to traditional smartphone-based music player under similar driving situations.

The advancement of smart phones enables mobile users to collect data from their surrounding sensors using short-range wireless communication. However, the limited contact time and the wireless capacity constrain the amount of data to be collected by the mobile users. It is crucial for mobile users to collect sensing data that can maximize their data utility. In this paper, we propose a distributed algorithm to provide information-centric ubiquitous data collection for multiple mobile users. The mobile users construct data collection trees adaptively according to their dynamic moving speeds. They prioritize data collection according to the information value carried by the sensing data. The distributed algorithm can support smooth data collection and coordination among multiple mobile users. We evaluate the data utility, energy efficiency and scalability of our solution with extensive simulations. The results showed that our distributed algorithm can improve information value up to 50% and reduce energy consumption to half compared with the existing approach.

It is common to deploy stationary sensors in large geographical environments for monitoring purposes. In such cases, the monitored data are subject to data loss due to poor link quality or node failures. Fortunately, the sensing data are highly correlated both spatially and temporally. In this paper, we consider such networks in general, and jointly take into account the link quality estimates, and the spatio-temporal correlation of the data to minimise energy consumption by selecting sensors for sampling and relaying data. In particular, we propose a multi-phase adaptive sensing algorithm with belief propagation protocol (ASBP), which can provide high data quality and reduce energy consumption by turning on only a small number of nodes in the network. We explore the correlation of data, formulate the sensor selection problem and solve it using constraint programming (CP) and greedy search. Bayesian inference technique is used to reconstruct the missing sensing data. We show that while maintaining a satisfactory level of data quality and prediction accuracy, ASBP successfully provides load balancing among sensors and preserves 80% more energy compared to the case where all sensor nodes are actively involved.

Incentive allocation is an important research issue in participatory sensing as it determines the willingness of participants in joining the sensing campaign. Existing incentive approaches either decide the payments without consulting the participants, or require burdensome negotiation procedures like bid-price auction. In this paper, we propose a novel incentive allocation mechanism, which encourages participation and allocates incentives dynamically to achieve accurate sensing results. The proposed mechanism consists of two major elements. The first is a lightweight incentive negotiation procedure, which dynamically offers incentives to participants in spatio-temporal subregions and collects their responses. The second is the optimization problem for incentive allocation as well as it's solution, which aims at maximizing data quality by capturing the amount and the distribution of data samples. Simulations with real datasets confirmed that the proposed solution can provide dynamic incentive offers according to the estimated value of participants' data contribution to the overall quality of sensing result.

Driving is an integral part of our everyday lives, but it is also a time when people are uniquely vulnerable. Poor road condition, traffic congestion and long driving time may bring negative emotion to drivers and increase the chance of traffic accidents. We propose SAfeDJ, a situation-aware in-car music delivery application, which turns people's trips into pleasant journeys and driving into a safe and enjoyable activity. SAfeDJ aims at helping drivers to diminish fatigue and negative emotion. It is built on a vehicular healthcare platform that enables communications among drivers and integrates with multiple types of sensors to promote safe driving. Prototype implementation and initial results of SAfeDJ have demonstrated its desired functionality in drivers' daily lives and feasibility for real-world deployment.

Incentive strategy is important in participatory sensing, especially when the budget is limited, to decide how much and where the samples should be collected. Current auction-based incentive strategies purchase sensing data with lowest price requirements to maximize the amount of samples. However, such methods may lead to inaccurate sensing result after data interpolation, particularly for participants that are massing in certain subregions where the low-price sensing data are usually aggregated. In this paper, we introduce weighted entropy as a quantitative metric to evaluate the distribution of samples and find that the distribution of data samples is another important factor to the accuracy of sensing result. We further propose a greedy-based incentive strategy which considers both the amount and distribution of samples in data collection. Simulations with real datasets confirmed the impact of samples distribution to data accuracy and demonstrated the efficacy of our proposed incentive strategy.

Sensor networks that operate in the unlicensed 2.4~GHz frequency band suffer cross-technology radio interference from a variety of devices, e.g., Bluetooth headsets, laptops using WiFi, or microwave ovens. Such interference has been shown to significantly degrade network performance. We present SoNIC, a system that enables resource-limited sensor nodes to detect the type of interference they are exposed to and select an appropriate mitigation strategy. The key insight underlying SoNIC is that different interferers disrupt individual 802.15.4 packets in characteristic ways that can be detected by sensor nodes. In contrast to existing approaches to interference detection, SoNIC does not rely on active spectrum sampling or additional hardware, making it lightweight and energy-efficient.

Wireless sensor networks have become increasingly popular for environmental and activity monitoring, such as temperature, pollution, parking space, traffic, and crowd monitoring. Mobile users can collect and visualise sensing data by communicating with wireless sensors along their walks using Bluetooth or NFC. They can also share the sensing data on the Internet through 3G or WiFi connectivity. Nevertheless, mobile users may not be able to collect all the data from the sensors due to limited contact times and batteries. It is crucial to collect data with a maximum amount of information from the available resources. In this paper, we tackle the problem by prioritising the sensing data to maximise the data utility considering the quality of information of the sensing data and the communication overhead. We formulate the optimisation problem and propose a greedy algorithm for clustering the sensors and scheduling the data collection. Our greedy algorithm coordinates the mobile users in the sensing field in order to avoid the collection of redundant sensing data. We evaluate the data utility and energy consumption of the proposed algorithm using real mobility traces from the North Carolina state fair. The results demonstrate that our algorithm can significantly improve data utility at low communication overhead compared with an existing algorithm.

Existing publish/subscribe architectures for wireless sensors networks only support stationary sensors interconnected with each other. They cannot handle mobile sensors or remote sensors that are sparsely deployed. In this paper, we propose a novel ubiquitous publish/subscribe system that supports data access from both mobile sensors and stationary sensors. Our system utilizes mobile phones as data mules to relay subscriptions and published data between broker and remote sensors. It provides content-based publish/subscribe services from sensors deployed anywhere without depending on any network infrastructure. We implement our publish/subscribe platform on real hardwares and test it in a hiking trail application. The application allows users to subscribe for sensing data from both stationary sensors and mobile sensors along hiking trails. Extensive experiments are conducted in an outdoor testbed to evaluate the system performances such data delay, number of data received and communication overhead.

Many mobile sensing applications consider users reporting and accessing sensing data through the Internet. However, WiFi and 3G connectivities are not always available in remote areas. Existing data dissemination schemes for opportunistic networks are not sufficient for sensing applications as sensing context has not been explored. In this work, we present a novel context-aware sensing data dissemination framework for mobile users in a remote sensing field. It maximizes information utility by considering such sensing context as sensing type, locality, time-to-live, mobility and user interests. Different from existing works, the mobile users not only collect sensing data, but also upload data to sensors for information sharing. We develop a context-aware deployment algorithm and a hybrid data exchange mechanism for generic sensors and mobile users. We evaluate our solution by both analysis and simulations, and show that it can provide high information utility for mobile users at low communication overhead.

With the prevalence of localization techniques in smart phones, we see the blossom of user-trigger location-based applications in the last few years. However, only minorities of applications can be triggered by the predefined locations without human interaction to perform the associated actions in certain scenarios. One reason is that the inevitable power-intensive operation of location detection by GPS sensor causes battery draining issue. While there exists some work on energy efficiency in continuous location tracking, energy-efficient location detection for matching predefined location of interest has not been fully addressed by previous work. This paper proposes a unified framework for designing energy-efficient location-triggered applications. Our system triggers data collection only when the user reaches pre-defined location of interests. We propose an energy-efficient location detection scheme that support duty-cycles for GPS in cooperation with other on-device sensors intelligently. The results of our real-world experiment demonstrate that mobile devices can reduce the usage of GPS and lower energy consumption significantly in location-triggered applications.

Interactive multi-view video enables users to enjoy the video from different viewpoints. Yet multi-view dramatically increases the video data volume and their computation, making realtime transmission and interactions a challenging task. It therefore calls for efficient view synthesis strategies that flexibly generate visual views. In this paper, we present a collaborative view synthesis strategy for online interactive multi-view video streaming based on Depth-Image Based Rendering (DIBR) view synthesis technology, which generates a visual view with the texture and depth information on both sides. Different from the traditional DIBR algorithm for single view synthesis, we explore the collaboration relationship between different viewpoints synthesis for a range of visual views generation, and propose Shift DIBR (S-DIBR). In S-DIBR, only the projected pixels, rather than all the pixels of the reference view, are utilized for next visual view generation. Therefore, the computation complexity of projection transform, which is the most computation intensive process in the traditional DIBR algorithm, is reduced to fulfill the requirement of online interactive streaming. Experiment results validate the efficiency of our collaborative view synthesis strategy, as well as the bandwidth scalability of the streaming system.

We consider a novel application in wireless sensor networks where mobile phones and wireless sensors can collaborate to collect sensing data. Although mobile phones can perform sensing at different locations, it is a challenge to provide stable sensing quality and availability over the entire area. One approach is to deploy stationary sensors at specific locations to maintain the sensing quality and availability. In this paper, we present a mathematical programming model to minimise the deployment cost by placing a minimum number of sensors at optimal locations. The problem is modelled by integer linear programming considering the sensing capabilities of both the mobile phones and wireless sensors. We evaluated the performance of our solution in terms of sensing quality, number of required sensors, and computation time. The results demonstrate that our approach satisfies the required sensing quality with optimal number of sensors in small sensing fields. It achieves near optimal solution with low computation time for large sensing fields.

Wireless sensor networks (WSNs) have been widely deployed for environmental monitoring and urban sensing applications. With the advancement of mobile phones, mobile users have increasing demands for sensing data relevant to their locations and activities. It is crucial to support reliable ubiquitous sensing for mobile users to retrieve sensing data of their interests anytime and anywhere. In this paper, we propose a novel location-based publish/subscribe framework for mobile users to subscribe for sensing data by simply specifying the event types and target locations of their interests. The framework is developed on a location-based Distributed Hash Table (DHT) overlay formed by a network of brokers. Event data can be subscribed by mobile users and then multicast to the subscribers through efficient location-based routing. To support client mobility, we further propose a reliable protocol to handle client registration, location look-up and client relocation. It can provide reliable data delivery without causing any data loss even when the users are disconnected or moved to a new place. Extensive simulation are conducted to evaluate the message delay, communication overheads, and the support of client mobility.

We present a novel distributed localization algorithm, called Swarm-Intelligent Localization (SIL), for computing the physical locations of nodes in wireless sensor networks. The algorithm assumes that only a small fraction of the nodes have a priori knowledge of their positions, and that noisy distance measurements are available between all neighboring nodes. The algorithm has no explicit global state and it can handle nodes that are both mobile and that can arrive in the network at any time. SIL works in two different phases, a coarse phase where nodes compute rough positions for themselves based on information about remote anchors, and a fine phase where nodes iteratively refine their positions from the coarse phase by collaborating with their neighbors. The average computational complexity per node running SIL is very low, namely constant in the network size and linear in the connectivity of the network. We evaluate the algorithm through extensive simulations. The results indicate that SIL is able to compute accurate positions for the majority of nodes in a wide range of network topologies and settings, and that it can handle difficulties such as large distance measurement errors and low network connectivity.

The rising popularity of web services and their applications to sensor networks enables real-time data collection and queries by users. Unlike traditional periodic data collection, the traffic patterns generated from real-time data collection may expose the interests of users or the locations of unusual events to the attackers. To provide privacy in data collection, we propose a novel probabilistic sampling mechanism that can hide user queries and unusual events in the network, while supporting both routine and on-demand data reporting. Our goal is to prevent attackers from locating the unusual events and identifying interests of users by eavesdropping and analyzing the network traffic. In our probabilistic sampling scheme, the data are carefully reported at random times in order to mask the unusual events and user queries. In the meantime, our scheme can provide users with high data accuracy at minimized communication overheads. Extensive simulations have been conducted to evaluate the security strength, data accuracy and communication overheads of the proposed scheme.

With the popularity and advancements of smart phones, mobile users can interact with the sensing facilities and exchange information with other wireless devices in the environment by short range communications. Opportunistic exchange has recently been suggested in similar contexts; yet we show strong evidence that, in our application, opportunistic exchange would lead to insufficient data availability and extremely high communication overheads due to inadequate or excessive human contacts in the environment. In this paper, we present OppSense, a novel design to provide efficient opportunistic information exchange for mobile phone users in sensing field with data repositories that tackles the fundamental availability and overhead issues. Our design differs from conventional opportunistic information exchange in that it can provide mobile phone users guaranteed opportunities for information exchange regardless the number of users and contacts in different environments. Through both analysis and simulations, we show that the deployment of data repositories plays a key role in the overall system optimization. We demonstrate that the placement of data repositories is equivalent to a connected K-coverage problem, and an elegant heuristic solution considering the mobility of users exists. We evaluate our proposed framework and algorithm with real mobile traces. Extensive simulations demonstrate that data repositories can effectively enhance the data availability up to 41% in low contact environment and significantly reduce the communication overheads to only 28% compared to opportunistic information exchange in high contact environment.

Despite the various applications of wireless sensor network (WSN), experiences from real WSN deployments show that protocol implementations in sensor nodes are susceptible to software failures, which may cause network failures or even breakdown. Pre-deployment protocol conformance testing is essential to ensure reliable communications for WSNs. Unfortunately, existing solutions with simulators cannot test the exact hardware and implementation environment as real sensors, whereas testbeds are expensive and limited to small scale networks and topologies. In this paper, we present RealProct, a novel and reliable framework for testing protocol implementations against their specifications in WSNs. RealProct utilizes real sensors for protocol conformance testing to ensure that the results are close to the real deployment. Using different techniques from those in simulations and real deployments, RealProct virtualizes a large network with any topology and generate non-deterministic events using only a small number of sensors to provide flexibility and to reduce the cost. The framework is carefully designed to support efficient testing in resource-limited sensors. Moreover, test execution and verdict are optimized to minimize the number of runs, while guaranteeing satisfactory false posi tive and false negative rates. We implement RealProct and test it with the IIP TCP/IP protocol stack and a routing protocol developed for WSNs in Contiki-2.4. The results demonstrate the effectiveness of RealProct by detecting several new bugs and all previously discovered bugs in various versions of the μIP TCP/IP protocol stack.

Wireless sensor networks have been widely deployed to perform sensing constantly at specific locations, but their energy consumption and deployment cost are of great concern. With the popularity and advanced technologies of mobile phones, participatory urban sensing is a rising and promising field which utilizes mobile phones as mobile sensors to collect data, though it is hard to guarantee the sensing quality and availability under the dynamic behaviors and mobility of human beings. Based on the above observations, we suggest that wireless sensors and mobile phones can complement each other to perform collaborative sensing efficiently with satisfactory quality and availability. In this paper, a novel collaborative sensing paradigm which integrates and supports wireless sensors and mobile phones with different communication standards is designed. We propose a seamless integrated framework which minimizes the number of wireless sensors deployed, while providing high sensing quality and availability to satisfy the application requirements. The dynamic sensing behaviors and mobility of mobile phone participants make it extremely challenging to estimate their sensing quality and availability, so as to deploy the wireless sensors at the optimal locations to guarantee the sensing performance at a minimum cost. We introduce two mathematical models, a sensing quality evaluation model and a mobility prediction model, to predict the sensing quality and mobility of the mobile phone participants. We further propose a cost-effective sensor deployment algorithm to guarantee the required coverage probability and sensing quality for the system. Extensive simulations with real mobile traces demonstrate that the proposed paradigm can integrate wireless sensors and mobile phones seamlessly for satisfactory sensing quality and availability with minimized number of sensors.

We propose “mode switch”, an adaptive load-sensitive solution that supports both an energy-efficient operation mode for transmitting normal sensor data and an QoS-aware low-latency mode for high priority emergency messages. Typically, sensor networks are considered to operate efficiently w.r.t. energy consumption. A good example is the IEEE 802.15.4 protocol, which has mainly been designed for this purpose. The standard IEEE 802.15.4 beacon-enabled cluster-tree topology supports energy-efficient operation for low-rate data transmissions. However, in the case of detected alarms, high-priority emergency messages need to be transmitted with low-latency and guaranteed delivery. In this case, the cluster-tree topology appears to be inefficient and cannot provide the needed support. In this paper, we show how energy-efficiency and delay-sensitivity can be combined by developing an adaptive solution that completely switches between the fundamentally different operation modes beacon-enabled cluster-tree mode and non-beacon-enabled mesh mode. The key challenge is to provide a fast mode switch that also keeps track of necessary topology information. We present a protocol for this mode switch and demonstrate, based on extensive simulations, that the performance demands can be fulfilled with only reasonable overhead even for mid-size networks. Our simulation results clearly indicate that the system achieves the objected QoS and energy-efficiency with “mode switch” in real-time.

Duty cycles have been widely studied recently to reduce the energy consumption for wireless communications. Although duty cycles can effectively reduce energy consumption, they may also increase the packet delay as the packets can only be forwarded when the next hop is awake. The mechanism is hence not suitable for delivering urgent messages. In this paper, we propose an adaptive cross-layer design for delivering prioritized traffic in wireless communications. Our cross-layer design integrates a modified X-MAC protocol with our generalized Random Re-Routing protocol coherently. Duty cycles will be applied if there is only ordinary traffic in the network. When high priority data are present, duty cycles will be disabled and priority routing will be employed. We generalize the QoS-aware Random Re-Routing protocol to support data with multiple priority levels. Different routing policies and transmission power will be selected according to the priority levels of the data packets. Extensive simulations have been conducted in OMNET++ to evaluate the performance of our system. The results demonstrate that our design can achieve both satisfactory packet delay and energy consumption.

Sensor networks become increasingly popular for a wide range of environmental monitoring and event detection applications. Sensors need to report data to the sinks without missing any important information or unusual events, though reporting large amount of data is infeasible for wireless sensor networks as they usually have limited network capability and energy. In this paper, we propose a quality-of-information aware and energy-efficient data delivery service to report data from sensors to the sinks. Our framework provides high quality-of-information to the users with high data accuracy, short packet delay and low message overhead. Data packets are delivered selectively via different paths based on the information that they are carrying. Packets carrying significant information will be routed with higher priority, while packets with less importance might be dropped or delivered along some slower paths. We propose and implement a routing algorithm based on the SPEED to reduce the number of messages without degrading the quality-of-information. Our work is evaluated by both simulations with real data and real experiments. The experimental results show that our approach can provide high data accuracy and low packet delay with the number of packets significantly reduced.

Sensor networks (SNs) consist of spatially distributed sensors which monitor an environment, and which are connected to some sinks or backbone system to which the sensor data is being forwarded. In many cases, the sensor nodes themselves can serve as intermediate nodes for data coming from other nodes, on the way to the sinks. Much of the traffic carried by SNs will originate from routine measurements or observations by sensors that monitor a particular situation, such as the temperature and humidity in a room or the infrared observation of the perimeter of a house, so that the volume of routine traffic resulting from such observations may be quite high. When important and unusual events occur, such as a sudden fire breaking out or the arrival of an intruder, it will be necessary to convey this new information very urgently through the network to a designated set of sink nodes where this information can be processed and dealt with. This paper addresses the important challenge by avoiding the routine background traffic from creating delays or bottlenecks that impede the rapid delivery of high priority traffic resulting from the unusual events. Specifically we propose a novel technique, the ‘Randomized Re-Routing Algorithm (RRR)’, which detects the presence of novel events in a distributed manner, and dynamically disperses the background traffic towards secondary paths in the network, while creating a ‘fast track path’ which provides better delay and better quality of service (QoS) for the high priority traffic which is carrying the new information. When the surge of new information has subsided, this is again detected by the nodes and the nodes progressively revert to best QoS or shortest-path routing for all the ongoing traffic. The proposed technique is evaluated using a mathematical model as well as simulations, and is also compared with a standard node by a node priority scheduling technique.

The sinks usually in sensor networks usually broadcast their addresses for data collection. However, this common operation opens up vulnerability for adversary to attack the sinks and obstruct their normal functions. In this paper, we suggest sink anonymity as a novel approach for data collection, which protects the privacy of the sinks and avoids them to become the target of attacks. We provide sink anonymity by omitting the address of the sinks in routing, so that the identity and location of the sinks are kept private. Our proposed scheme, Randomized Routing with Hidden Address (RRHA), prevents the attackers from obtaining the receiver address by capturing the destination field of the packets or by predicting the location of the sinks by observing the flow of network traffic. We examined the successful delivery rate, packet delay, and protection strength of our proposed scheme by both analysis and simulations.

Environmental monitoring is one of the most popular applications in wireless sensor networks. Although it is important to obtain a continuous record of the environment, users may gain enough information without receiving every routine sensor measurement. Reducing unnecessary traffic allows better utilization of the network resources. However, it is uneasy to decide on when and what kind of traffic to reduce in a dynamically changing environment. In this paper, we study the problem of information-aware traffic reduction for wireless sensor networks. We propose a two-step information-aware traffic reduction algorithm to address this problem. First, we provide a distributed and real-time algorithm for sensors to classify their measurements and report them selectively based on the importance of information. Then, we propose a bandwidth allocation algorithm to assign different forwarding probabilities to packets considering both the information quality and the network load. Our algorithm can be implemented and integrated easily with existing routing protocols to maximize the quality of information to the users, while reducing the amount of network traffic. We evaluate our approach based on the real sensing measurements to demonstrate the quality of information achieved. Simulations are also conducted to evaluate the performance of our proposed scheme in a larger network.

Wireless sensor networks have attracted increasing attentions considering their potentials for being widely adopted in both emerging civil and military applications. A common practice of sensor networks is to collect data from the sensors and report the data to the sinks or to some pre-defined data rendezvous points via multi-hop communications. Attackers may locate a sink easily by reading the destination field in the packet header or predicting the arrival of a sink at the rendezvous points, which opens up vulnerabilities to location privacy of the sinks. In this work, we propose a random data collection scheme to protect the location privacy of mobile sinks in sensor networks. Data are forwarded along some random paths and stored at random nodes in the network. The sinks move around along some random paths to collect data from the local nodes occasionally, which prevents the attackers from predicting their locations and movements. We analyze different attacks threatening the location privacy of the sinks in sensor networks. We also evaluate the delivery rate, data collection delay and protection strength of our scheme by both analysis and simulations. The results show that our scheme can provide location privacy of mobile sinks effectively, while providing satisfactory data collection services.

Wireless sensor networks (WSN) can report large volumes of slowly varying routine data, while important or significant events can be relatively rare. An important challenge is then to offer the significant or unusual data an adequate routing policy that will allow it to rapidly reach the sink nodes, despite the large volume of routine packets in the network. In this paper we introduce randomized re-routing (RRR), to detect the unusual events in a distributed manner, and dynamically transfer routine data packets to secondary paths in the network, while offering a fast track path with better QoS for the packets carrying unusual data. In this paper we describe the RRR algorithm and evaluate it with extensive simulations.

Wireless sensor-actuator networks, or WSANs, refer to a group of sensors and actuators which collect data from the environment and perform application-specific actions in response. To act responsively and accurately, an efficient and reliable data transport protocol is crucial for the sensors to inform the actuators about the environmental events. Unfortunately, the low-power multi-hop communications in WSANs are inherently unreliable; the frequent sensor and link failures as well as the excessive delays due to congestion further aggravate the problem. In this paper, we propose a latency-oriented fault tolerant data transport protocol in WSANs. We argue that reliable data transport in such a real-time system should resist to the transmission failures, and should also consider the importance and freshness of the reported data. We articulate this argument and provide a cross-layer two-step data transport protocol for on- time and fault tolerant data delivery from sensors to actuators. Our protocol adopts smart priority scheduling that differentiates the event data of non-uniform importance. It balances the workload of sensors by checking their queue utilization and copes with node and link failures by an adaptive replication algorithm. We evaluate our protocol through extensive simulations, and the results demonstrate that it achieves the desirable reliability for WSANs.

Wireless sensor-actuator networks (WSANs) have recently been suggested as an enhancement to the conventional sensor networks. The powerful and mobile actuators can patrol along different routes and communicate with the static sensor nodes. This work is motivated by applications in which the objective is to minimize the data collection time in a stochastic and dynamically changing sensing environment. This is a departure from the previous static and deterministic mobile element scheduling problems. In this paper, we propose PROUD, a probabilistic route design algorithm for wireless sensor-actuator networks. PROUD offers delay-minimized routes for actuators and adapts well to network dynamics and sensors with non-uniform weights. This is achieved through a probabilistic visiting scheme along pre-calculated routes. We present a distributed implementation for route calculation in PROUD and extend it to accommodate actuators with variable speeds. We also propose the Multi-Route Improvement and the Task Exchange algorithms for balancing load among actuators. Simulation results demonstrate that our algorithms can effectively reduce the overall data collection time in wireless sensor-actuator networks. It well adapts to network dynamics and evenly distributes the energy consumption of the actuators.

Wireless sensor-actuator networks (WSANs) have recently been suggested as an enhancement to the traditional sensor networks by employing powerful and mobile actuators. Multiple actuators can patrol along different routes and communicate with the static sensors. To minimize the data collection time, an effective route design is crucial for the actuators to travel in the sensing field. In this paper, we present a mathematical formulation of the route design problem, and show that the general problem is computationally intractable. We then develop a practically efficient algorithm to reduce the waiting time for the sensors. Our algorithm adaptively differentiates the actuator visiting frequencies to the sensors according to their relative weights and data generation patterns. Simulation results demonstrate that our algorithm can effectively reduce the overall data collection time.

This paper investigates the data transport problem for reporting delay-sensitive events in wireless sensor-actuator networks (WSANs). We specifically tailor the protocol design according to the features of WSANs and propose POWER-SPEED, a real-time data transport protocol for WSANs to achieve energy-efficient data transport for delay-sensitive event reporting. In POWER-SPEED, sensor nodes select the next-hop neighbor to actuators according to the spatio-temporal historic data of the upstream QoS condition, which completely avoids control packets. With an adaptive transmitter power control scheme, POWER-SPEED conveys packets in an energy-efficient manner while maintaining soft real-time packet transport. It thus reduces the energy consumption of data transport while ensuring the QoS requirement in timeliness domain. We demonstrate the effectiveness of POWER-SPEED through simulations with NS2.

In a wireless sensor network, multiple nodes would send sensor readings to a base station for further processing. It is well-known that such a many-to-one communication is highly vulnerable to the sinkhole attack, where an intruder attracts surrounding nodes with unfaithful routing information, and then performs selective forwarding or alters the data passing through it. A sinkhole attack forms a serious threat to sensor networks, particularly considering that such networks are often deployed in open areas and of weak computation and battery power. In this paper, we present a novel algorithm for detecting the intruder in a sinkhole attack. The algorithm first finds a list of suspected nodes, and then effectively identifies the intruder in the list through a network flow graph. The algorithm is also robust to deal with cooperative malicious nodes that attempt to hide the real intruder. We have evaluated the performance of the proposed algorithm through both numerical analysis and simulations, which confirmed the effectiveness and accuracy of the algorithm. Our results also suggest that its communication and computation overheads are reasonably low for wireless sensor networks.

Abstract — We propose ι, a novel index for evaluation of pointdistribution. ι is the minimum distance between each pair of points normalized by the average distance between each pair of points. We find that a set of points that achieve a maximum value of ι result in a honeycomb structure. We propose that ι can serve as a good index to evaluate the distribution of the points, which can be employed in coverage-related problems in wireless sensor networks (WSNs). To validate this idea, we formulate a general sensor-grouping problem for WSNs and provide a general sensing model. We show that locally maximizing ι at sensor nodes is a good approach to solve this problem with an algorithm called Maximizing-ι Node-Deduction (MIND). Simulation results verify that MIND outperforms a greedy algorithm that exploits sensorredundancy we design. This demonstrates a good application of employing ι in coverage-related problems for WSNs. I.

Wireless sensor-actuator networks, or WSANs, greatly enhance the existing wireless sensor network architecture by introducing powerful and even mobile actuators. The actuators work with the sensor nodes, but can perform much richer application-specific actions. To act responsively and accurately, an efficient and reliable reporting scheme is crucial for the sensors to inform the actuators about the environmental events. Unfortunately, the low-power multi-hop communications in a WSAN are inherently unreliable; the frequent sensor failures and the excessive delays due to congestion or in-network data aggregation further aggravate the problem. In this paper, we propose a general reliability-centric framework for event reporting in WSANs. We argue that the reliability in such a real-time system depends not only on the accuracy, but also the importance and freshness of the reported data. Our design follows this argument and seamlessly integrates three key modules that process the event data, namely, an efficient and fault-tolerant event data aggregation algorithm, a delay-aware data transmission protocol, and an adaptive actuator allocation algorithm for unevenly distributed events. Our transmission protocol also adopts smart priority scheduling that differentiates the event data of non-uniform importance. We evaluate our framework through extensive simulations, and the results demonstrate that it achieves desirable reliability with minimized delay.

We describe XVIP, an XML-based video information processing system, which extracts information from video and stores the information in a multimedia digital video library. XVIP encapsulates a number of extraction techniques, including scene change detection, video optical character detection and recognition, and geometric coding. It also provides a seamless approach to scale up the contents created in and delivered by the target multimedia digital video library. Furthermore, XVIP can handle multilingual contents. XVIP is based on a multi-modal concept, which treats each content extraction component as a modality, and users can easily add new modalities into XVIP. The information extracted from the video is then stored in a flexible, scalable and reusable way based on a generic XML structure, providing a convenient mechanism for data representation on Web browsers. Also, the content in the XML file can be used to perform knowledge enrichment on top of the primary information extracted from the video. This enriched data representation helps users search for multimedia video content more efficiently.