With the growing prevalence of HDR-capable cinema venues such as Cinity LED theaters, there is an increasing demand to convert existing Standard Dynamic Range (SDR) films into High Dynamic Range (HDR) formats for theatrical presentation. However, existing SDR-to-HDR conversion methods are primarily tailored for consumer-grade content such as television and therefore fall short of the stringent requirements of professional cinematic material. To bridge this gap, we present HDRMovie7K, the first large-scale, lossless dataset of cinematic SDR-HDR frame pairs sourced from professional Digital Cinema Distribution Master (DCDM) workflows. Based on this foundation, we introduce HDRMovieformer, a transformer-based framework featuring a Luminance Estimator module for luminance guidance, a Luminance-Guided Multi-Head Self-Attention to focus on critical fine-detail recovery, and a Chroma Refiner for color accuracy, optimized with a novel Wide Color Gamut Loss. To further evaluate our model in online streaming media scenarios, we introduce HDRMovie1K, a dataset curated from publicly available HDR film clips. Extensive experiments on both HDRMovie7K and HDRMovie1K demonstrate that our method achieves state-of-the-art performance.

In open real-world autonomous driving scenarios, challenges such as sensor failure and extreme weather hinder the generalization of current autonomous driving perception models to these unseen domain, due to the domain shifts between the test and training data. As the parameter scale of autonomous driving perception models grows, traditional test-time adaptation (TTA) methods become unstable and often degrade model performance in most scenarios. To address these challenges, this paper proposes two new robust methods to improve the Batch Normalization with TTA for object detection in autonomous driving: (1) We introduce a new LearnableBN layer based on Geometric Confidence Maximization and Entropy Minimization. Specifically, we modify the traditional BN layer by incorporating auxiliary learnable parameters, which enables the BN layer to dynamically update the statistics according to the different input data. (2) We propose a novel semantic-consistency based dual-stage adaptation strategy, which encourages the model to iteratively search for the optimal solution and eliminates unstable samples during the adaptation process. Extensive experiments on the NuScenes-C dataset shows that our method achieves a maximum improvement of about 10\% using BEVFormer as the baseline across six corruption types and three levels of severity.

Image Aesthetics Assessment (IAA) evaluates visual quality through user-centered perceptual analysis and can guide various applications. Recent advances in Multimodal Large Language Models (MLLMs) have sparked interest in adapting them for IAA. However, two critical limitations persist in applying MLLMs to IAA: 1) the tokenization strategy leads to insensitivity to scores, and 2) the classification-based decoding mechanisms introduce score quantization errors. Current MLLM-based IAA methods treat the task as coarse rating classification followed by probability-to-score mapping, which loses fine-grained information. To address these challenges, we propose ROC4MLLM, offering complementary solutions from two perspectives:1) Representation: We separate scores from the word token space to avoid tokenizing scores as text. An independent position token bridges these spaces, improving the sensitivity of the model to score positions in text. 2) Computation: We apply distinct loss functions for text and score predictions to enhance the sensitivity of the model to score gradients. Decoupling scores from text ensures effective supervision while preventing interference between scores and text in the loss computation. Extensive experiments across five datasets demonstrate that ROC4MLLM achieves state-of-the-art performance without requiring additional training data. Additionally, its plug-and-play design ensures seamless integration with existing MLLMs, boosting their IAA performance.

High-scale image super-resolution (SR) has become increasingly important with the rapid growth of mobile devices and high-resolution displays. However, current SR methods primarily focus on lower scales and generalize poorly to high-scale scenarios due to severe information loss and complex real-world degradations. In this paper, we propose a novel Selective Diffusion Distillation (SDD) framework for real-world high-scale SR, which distills reliable knowledge from a low-scale diffusion teacher to a high-scale student. Specifically, considering severe information loss in high-scale inputs, directly distilling from low-scale models may result in feature misalignment. To address this, we introduce a Degradation-aware Metric Learning (DML) approach to align feature distributions across different degradation levels. In addition, since the diffusion-based teacher may hallucinate artifacts in ambiguous regions, blindly imitating these unreliable outputs can degrade the student’s fidelity. To tackle this, we propose a Region-aware Selective Distillation (RSD) strategy to filter out uncertain predictions and adaptively supervise only on reliable areas. To evaluate the effectiveness of our method, we introduce Real-UltraSR, a new real-world benchmark that contains diverse high-scale LR-HR pairs, including x8, x10, x12, and x14. Extensive experiments demonstrate that our SDD framework achieves state-of-the-art performance across multiple benchmarks.

This paper presents TurboInfer, a system that enables power-efficient LLM inference on mobile devices. The core insight behind TurboInfer is that while LLMs are power-intensive due to their heavy computational demands, the model inference experiences unavoidable GPU stalls caused by tensor preparation for subsequent kernel executions at run-time. These GPU stalls arise from the unique host-controlled execution pipeline tailored to mobile phones and the significant DRAM access contention inherent to the shared memory architecture of mobile System-on-chips (SoCs). With LLM inference requiring hundreds to thousands of kernel executions, these short but frequent GPU stalls accumulate, accounting for over 74% of the token generation latency.

Wi-Fi computational imaging has emerged as a promising paradigm for non-intrusive sensing; however, its practical deployment is severely hindered by dense physical multipath clutter, hardware phase quantization errors, and the limited bandwidth of commercial Wi-Fi. Traditional beamforming-based algorithms often completely lose target focus in complex environments, resulting in severe ghosting artifacts. To overcome these fundamental limitations, we propose RISimg, a novel robust Wi-Fi imaging framework empowered by the spatiotemporal coding of Reconfigurable Intelligent Surface. We design a differential coding strategy to guarantee a well-conditioned and noise-robust sensing matrix, and propose a multi-frequency sparse reconstruction algorithm based on the Least Absolute Shrinkage and Selection Operator. By constructing a large-scale overdetermined system, this physics-driven approach effectively suppresses multipath interference and hardware errors, successfully recovering the basic morphological outlines of complex targets. To further enhance the imaging performance, we propose leveraging a Conditional Diffusion Model to refine the imaging results. By utilizing the reconstruction as a structural prior, a carefully designed Conditional U-Net progressively refines the image through a generative reverse sampling process, restoring high-fidelity continuous boundaries. Extensive evaluations using a hardware prototype built with commercial Wi-Fi devices and a low-cost metasurface demonstrate that the final cascaded CDM achieves an unprecedented Structural Similarity Index Measure of 0.9192 and a Spatial Correlation Coefficient of 0.7336, paving a robust new avenue for Wi-Fi imaging.

In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects' physical commonsense based on both video and audio input, with the main challenge being how to imitate the reasoning ability of humans, even in the scenario of missing modalities. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and the lack of causal reasoning ability in models impedes the progress of implicit physical knowledge inference. To address these issues, our proposed RDCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space using the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. To alleviate the incomplete modality data issue, we introduce a robust multimodal learning method to recover the missing data by decomposing the shared features and model-specific features. Our proposed method is a plug-and-play module that can be incorporated into any baseline, including VLMs. In experiments, we show that our proposed method improves the reasoning accuracy and robustness of baseline methods and achieves the state-of-the-art performance.

Given a single labeled examples, in-context segmentation aims to segment corresponding objects. This setting, known as one-shot segmentation in few-shot learning, explores the segmentation model's generalization ability and has been applied to various vision tasks, including scene understanding and image/video editing. While recent Segment Anything Models (SAMs) have achieved state-of-the-art results in interactive segmentation, these approaches are not directly applicable to in-context segmentation. In this work, we propose the Dual Consistency SAM (DC-SAM) method based on prompt-tuning to adapt SAM and SAM2 for in-context segmentation of both images and videos. Our key insights are to enhance the features of the SAM's prompt encoder in segmentation by providing high-quality visual prompts. When generating a mask prior from support images, we fuse the SAM features to better align the prompt encoder rather than relying solely on a pre-trained backbone. Then, we design a cycle-consistent cross-attention on fused features and initial visual prompts. This design leverages coarse masks from the SAM mask decoder to ensure consistency between features and visual prompts. Next, a dual-branch design is provided by using the discriminative positive and negative prompts in the prompt encoder. Furthermore, we design a simple mask-tube training strategy to adopt our proposed dual consistency method into the mask tube. Although the proposed DC-SAM is primarily designed for images, it can be seamlessly extended to the video domain with the support of SAM2. Given the absence of in-context segmentation in the video domain, we manually curate and construct the first benchmark from existing video segmentation datasets, namedIn-Context Video Object Segmentation (IC-VOS), to better assess the in-context capability of the model. Extensive experiments demonstrate that our method achieves 55.5 (+1.4) mIoU on COCO-20$^{i}$, 73.0 (+1.1) mIoU on PASCAL-5$^{i}$, and a$\mathcal {J\&F}$score of 71.52 on the proposed IC-VOS benchmark.

Arbitrary-scale video super-resolution (VSR) aims to enhance video resolution at continuous scales and has attracted increasing attention in recent years. However, existing methods typically rely on fixed degradation modes, such as bicubic downsampling, which often fail to handle the complex degradations of real-world videos. Current real-world datasets only cover limited scales (e.g., ×2, ×4) and are insufficient to capture the diverse degradations required for arbitrary-scale VSR. To address this, we present RealArbVSR, the first real-world VSR dataset with both integer and decimal scale factors, providing a wider range of degradation levels. Moreover, to generate continuous degradations beyond the collected scales, we propose the Continuous Degradation Generation Network (CDGN), which synthesizes realistic LR videos with arbitrary degradations. Specifically, we design a Scale-aware Degradation Module (SDM) to adaptively learn scale-specific degradations and an Implicit Filter Module (IFM) that represents spatial-temporal features as a continuous feature domain for arbitrary-scale LR frame generation. Extensive experiments demonstrate that our CDGN trained on RealArbVSR produces high-fidelity LR videos with arbitrary degradations and significantly enhances the performance of VSR models in real-world scenarios. The RealArbVSR dataset and source code will be publicly released for further research.

Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process-from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., + 16.0% in CIDEr),action classification (+ 2.7% in accuracy) and quality assessment (+ 2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code are available at https://github.com/MICLAB-BUPT/EFA.

Deploying Tiny Deep Learning (TinyDL) on Internet of Things (IoT) devices is gaining popularity. To accommodate the limited memory, recent methods split tensors into fine-grained parts and plan memory offline to minimize its footprint. However, they fail to adapt to dynamic memory, missing the opportunity to utilize temporarily available memory for faster inference. Additionally, existing approaches focus solely on minimizing memory size while neglecting cache usage characteristics, resulting in frequent cache misses and increased latency. In this paper, we propose RAMS, an efficient framework supporting runtime adaptive memory scaling to fully utilize the dynamic memory. We also propose a cache-friendly memory management approach that minimizes cache miss times. RAMS includes an offline planner to minimize the memory footprint essential for inference and an online manager to determine memory sizes and generate layouts for size-controllable tensors based on available memory. RAMS significantly reduces inference latency while maintaining a compact memory footprint. Extensive experiments on commercial devices running RTOS and Android systems demonstrate that, compared to the state-of-the-art methods, RAMS can efficiently reduce latency by up to 1.57× and 1.48× compared to TFLM and TinyTS, respectively using a comparable memory footprint, while reducing power consumption by 67.74% and 15.97%.

Due to the ability to penetrate darkness, smoke, and fog, wireless sensing technologies offer unique advantages for real-world deployments. Recent studies have mounted wireless sensing devices on mobile systems (such as drones and wheeled robots). Despite their promising potential, these approaches are predominantly designed for non-human platforms that exhibit limited mobility, typically involving translational movement with minimal rotation. When deployed on human carriers, however, the frequent body rotations and complex motion patterns significantly degrade their performance, due to the unpredictable changes in target position. In this paper, we propose Mobi2Still, which utilizes mobile human-equipped millimeter wave (mmWave) radars for people detection and tracking. The core idea is to match recurring environment information across different times to estimate radar motion in reverse, and based on this, enable accurate people detection and tracking. Specifically, Mobi2Still extracts geometric structure of scene objects which is radar-motion-independent to match the recurring environmental objects, thereby adapting both rotation and translation of mmWave radars. Furthermore, to improve Mobi2Still's generalization, we design a velocity-aware calibration mechanism to guide it to focus on scene-independent motion features. Experiments demonstrate that Mobi2Still can accurately detect and track people with 98.10% F1-score and 98.32% MOTA, achieving at least 7.54% F1-score (and 11.20% MOTA) improvements compared with state-of-the-art approaches.

Trace heavy metals in water represent a significant health hazard due to their bioaccumulation in the human body. Current detection methods for these contaminants typically require sophisticated laboratory equipment and procedures, limiting their applicability in the field. In this paper, we show that Terahertz (THz), a prospective frequency band for wireless communication technologies, exhibits distinctive pine-like structures (termed “THz-Pine”) in the three-dimensional space when sensing liquids. Moreover, our measurements reveal that this structure possesses superior capabilities for characterizing trace physical information compared to conventional 2D spectrum methods. Leveraging above findings, we introduce Terahertz Trace Sensing,i.e.,TraSen, a contactless method for detecting and quantifying trace heavy metals in water.TraSenoperates by emitting Terahertz signals vertically above a water sample, capturing the reflected signals, and analyzing the resultingTHz-Pineto determine the type and concentration of trace heavy metals present. To quantify trace heavy metals concentrations, we develop an embedded Physics-Informed 3D point cloud deep learning model. Our extensive dataset, comprising 102,500 Terahertz signal samples across 40 trace heavy metals, demonstrates thatTraSencan effectively identify trace heavy metals with 95.4% accuracy in type identification and 94.3% accuracy in concentration quantification. Specifically, our preliminary findings indicate that the detection limit of Terahertz signals for heavy metals in water is approximately 0.1$\mu g/dL$.

Estimating blood pressure (BP) plays a crucial role in healthcare. Traditionally, BP has been measured using a cuff, which is unsuitable for continuous BP monitoring. Recent advancements in cuffless wearable devices, such as graphene electronic tattoos and PPG sensors provide viable alternatives. However, existing BP estimation methods often overlook the importance of temporal modeling, despite the multi-periodicity and temporal dependencies inherent in cuffless BP signals. Moreover, continuous BP monitoring requires personalized modeling, which is further challenged by data scarcity. To address these two challenges, we introduce a novel Physics-Informed Temporal Network (PITN) with adversarial contrastive learning to enable precise BP estimation with very limited data for three different modalities (i.e, bioimpedance, PPG, millimeter wave). Specifically, we first introduce a novel Physics-Informed Temporal Network for investigating BP dynamics' multi-periodicity for cardiovascular cycle modeling and temporal variation. We then apply adversarial training to generate extra physiological time series data, improving PITN's robustness in the face of sparse subject-specific training data. Furthermore, we utilize contrastive learning to capture the discriminative variations of cardiovascular physiologic phenomena. This approach aggregates physiological signals with similar blood pressure values in latent space while separating clusters of samples with dissimilar blood pressure values. Experiments on three widely-adopted datasets with different modailties demonstrate the superiority and effectiveness of the proposed methods over previous state-of-the-art approaches. The code is available athttps://github.com/Zest86/ACL-PITN.

With the explosive increase in wireless devices, enabling sensing between incompatible radios has become critically beneficial. Integrating diverse IoT devices enhances sensing accuracy by providing richer data, while utilizing the diverse characteristics of heterogeneous signals meets sensing needs in complex environments. However, most existing wireless sensing methods primarily focus on homogeneous signals, while research on sensing with heterogeneous signals is still in its infancy. In this paper, we proposeCrossSense, a novel Cross-Technology Sensing (CTS) framework that enables sensing between incompatible WiFi and LoRa device.CrossSenserecovers the fine-grained trajectory of a WiFi transmitter based on its emulated LoRa signals. To decompose the motion feature components of WiFi transmitter, we develop a chirp difference vector model that utilizes the energy peak within each chirp window for sensing. We model the relationship between sampling frequency offsets and oscillation frequency offsets among heterogeneous devices to guide the extraction of motion features from the emulated signal. We also propose a greedy-based peak enhancement method to calculate the optimized LoRa phases, minimizing the impact of phase discontinuity caused by cyclic prefix (CP) errors. We implement a prototype ofCrossSenseon the USRP platform. The extensive experiments demonstrate thatCrossSensecan achieve an efficient Cross-Technology Sensing with$2.92cm$distance accuracy and$0.26cm/s$speed accuracy over a$120m$sensing range.

Multi-modal human activity recognition (HAR) is a key technology for a wide range of applications and has received widespread attention in recent years. However, the difficulty of achieving generalizability in multi-modal sensing models, combined with heterogeneous and unlabeled downstream data, significantly hinders their broader adoption. In this work, we proposePFHAR, a unified framework for practically adopting multi-modal foundation HAR model to target user groups.PFHARuses a novel dynamic masked contrastive learning method to pre-train a foundation model on various heterogeneous public HAR datasets, ensuring strong generalizability across different modal combinations. It then adopts semi-supervised edge-cloud collaborative learning to fine-tune the pre-trained model with heterogeneous and unlabeled local data, adapting it for the target user group. Our evaluations on public and self-collected datasets demonstrate thatPFHARsignificantly outperforms SOTA baselines in both the pre-training and edge-cloud collaborative fine-tuning stages.

Short-video platforms have grown rapidly by allowing users to browse rich media content through seamless swiping. However, the inherently random nature of swipe behavior creates significant challenges for bandwidth efficiency and playback continuity, often resulting in stalls and unnecessary data transfers. We present OffLoad, a new preloading framework that enhances bandwidth efficiency and playback quality using range-based downloading, which generalizes traditional chunk-based preloading to arbitrary-length segments for finer-grained control. At the core of OffLoad is a two-dimensional watch-time estimation model that jointly captures user preferences and video characteristics. Guided by this estimator, OffLoad introduces a hybrid preloading algorithm that integrates heuristic rules with a learning-based module trained directly on large-scale production data, enabling strong generalization in deployment. Following extensive system-level optimization, OffLoad has been deployed on a commercial short-video platform for more than six months. Our A/B testing results show that OffLoad increases overall user watch-time by 1.1‰, while simultaneously reducing 0.13% rebuffering events and 4.92% of bandwidth consumption.

Optimizing user Quality of Experience (QoE) for live video streaming remains a long-standing challenge. The Bitrate Control Algorithm (BCA) plays a crucial role in shaping user QoE. Recent advancements have seen RL-based algorithms overtake traditional rule-based methods, promising enhanced QoE optimization. Nevertheless, our comprehensive study reveals a pressing issue: current RL-based BCAs are limited to the fixed and formulaic reward functions, rendering them ill-equipped to adapt to dynamic network environments and varied viewer preferences. In this work, we present AraLivePro, an automatically adaptive reward learning method that can be seamlessly integrated with any existing learning-based approach in live streaming contexts. To achieve this goal, we have three main designs. First, we construct a dedicated user QoE assessment dataset for live streaming, which includes thousands of videos with millisecond-level metrics. Second, we custom-design an adversarial model that skillfully aligns human feedback with actual network scenarios. Third, we incorporate a QoE-guaranteed reward calibration to deal with the tail-lag effect, which refers to the delayed human feedback caused by network fluctuations near the end of a video segment. We have deployed AraLivePro in practical video streaming systems and conducted massive experiments in comparison to a series of state-of-the-art BCAs. The experimental results demonstrate that AraLivePro not only elevates overall QoE but also exhibits remarkable adaptability to varied network conditions and users.

Millimeter-wave (mmWave) communication technology with high throughput and high reliability attracts much attention in both academic and industrial fields. This technology plays a pivotal role in next-generation communication networks, offering promising solutions for high-speed data transfer. Localization of mobile mmWave communication devices is essential in this context, as it can effectively guide mmWave beam steering, thereby eliminating the need for cumbersome beam alignment processes. However, existing approaches for commercial mmWave communication devices suffer channel state fluctuations and can only work on static devices. To provide accurate localization for mobile mmWave devices, we propose MobiLoc , a neural network-based approach to enhance localization accuracy in mobile scenarios. Our method leverages Channel Frequency Response (CFR) and the angular spectrum for assistance to determine the positions. We first analyze the feasibility of classifying sensing data into different qualities. Then we implement a neural network architecture specifically designed to identify the sensing data with high quality. The effectiveness of our approach is demonstrated through comprehensive experiments conducted on commercial off-the-shelf (COTS) mmWave communication devices. Results show that MobiLoc can increase the localization accuracy significantly and reduce the median angle estimation error of mobile devices to 1.33ˆ with only single items of CFR measurements.

Object re-identification (ReID) is committed to searching for objects of the same identity across cameras, and its real-world deployment is gradually increasing. Current ReID methods assume that the deployed system follows the centralized processing paradigm, i.e., all computations are conducted in the cloud server and edge devices are only used to capture images. As the number of videos experiences a rapid escalation, this paradigm has become impractical due to the finite computational resources in the cloud server. Therefore, the ReID system should be converted to fit in the cloud-edge collaborative processing paradigm, which is crucial to boost its scalability and practicality. However, current works lack relevant research on this important specific issue, making it difficult to adapt them into a cloud-edge framework effectively. In this paper, we propose a cloud-edge collaborative inference framework for ReID systems, aiming to expedite the return of the desired image captured by the camera to the cloud server by learning the spatial-temporal correlations among objects. In the system, a Distribution-aware Correlation Modeling network (DaCM) is particularly proposed to embed the spatial-temporal correlations of the camera network implicitly into a graph structure, and it can be applied 1) in the cloud to regulate the size of the upload window and 2) on the edge device to adjust the sequence of images, respectively. Notably, the proposed DaCM can be seamlessly combined with traditional ReID methods, enabling their application within our proposed edge-cloud collaborative framework. Extensive experiments demonstrate that our method obviously reduces transmission overhead and significantly improves performance.

Video Internet of Things (VIoT) has shown full potential in collecting an unprecedented volume of video data. How to schedule the domain-specific perceiving models and analyze the collected videos uniformly, efficiently, and especially intelligently to accomplish complicated tasks is challenging. To address the challenge, we build VIoTGPT, the framework based on LLMs to correctly interact with humans, query knowledge videos, and invoke vision models to analyze multimedia data collaboratively. To support VIoTGPT and related future works, we meticulously crafted the VIoT-Tool dataset, including the training dataset and the benchmark involving 11 representative vision models across three categories based on semi-automatic annotations. To guide LLM to act as the intelligent agent towards intelligent VIoT, we resort to ReAct instruction tuning method based on VIoT-Tool to learn the tool capability. Quantitative and qualitative experiments and analyses demonstrate the effectiveness of VIoTGPT. We believe VIoTGPT contributes to improving human-centered experiences in VIoT applications.

Large Vision-Language Models (VLMs), such as GPT-4, have achieved remarkable success across various fields. However, there are few studies on 3D indoor scene generation with VLMs. This paper considers this task as a planning problem subject to spatial and layout common sense constraints. To solve the problem with a VLM, we propose a new global-local tree search algorithm. Globally, the method places each object sequentially and explores multiple placements during each placement process, where the problem space is represented as a tree. To reduce the depth of the tree, we decompose the scene structure hierarchically, i.e. room level, region level, floor object level, and supported object level. The algorithm independently generates the floor objects in different regions and supported objects placed on different floor objects. Locally, we also decompose the sub-task, the placement of each object, into multiple steps. The algorithm searches the tree of problem space. To leverage the VLM model to produce positions of objects, we discretize the top-down view space as a dense grid and fill each cell with diverse emojis to make to cells distinct. We prompt the VLM with the emoji grid and the VLM produces a reasonable location for the object by describing the position with the name of emojis. The quantitative and qualitative experimental results illustrate our approach generates more plausible 3D scenes than state-of-the-art approaches. Our source code is available at https://github.com/dw-dengwei/TreeSearchGen.

Understanding the traffic scenes and then generating highdefinition (HD) maps present significant challenges in autonomous driving. In this paper, we defined a novel Traffic Topology Scene Graph (T2SG), a unified scene graph explicitly modeling the lane, controlled and guided by different road signals (e.g., right turn), and topology relationships among them, which is always ignored by previous high-definition (HD) mapping methods. For the generation of T2SG, we propose TopoFormer, a novel one- stage Topology Scene Graph TransFormer with two newly-designed layers. Specifically, TopoFormer incorporates a Lane Aggregation Layer (LAL) that leverages the geometric distance among the centerline of lanes to guide the aggregation of global information. Furthermore, we proposed a Counterfactual Intervention Layer (CIL) to model the reasonable road structure (e.g., intersection, straight) among lanes under counterfactual intervention. Then the generated T2SG can provide a more accurate and explainable description of the topological structure in traffic scenes. Experimental results demonstrate that TopoFormer outperforms existing methods on the T2SG generation task, and the generated T2SG significantly enhances traffic topology reasoning in downstream tasks, achieving a state-of-the-art performance of 46.3 OLS on the OpenLane-V2 benchmark. Our source code is available at https://github.com/MICLAB-BUPT/T2SG.

The Personalized Aesthetics Assessment (PAA) aims to accurately predict an individual’s unique perception of aesthetics. With the surging demand for customization, PAA enables applications to generate personalized outcomes by aligning with individual aesthetic preferences. The prevailing PAA paradigm involves two stages: pre-training and fine-tuning, but it faces three inherent challenges: 1) The model is pre-trained using datasets of the Generic Aesthetics Assessment (GAA), but the collective preferences of GAA lead to conflicts in individualized aesthetic predictions. 2) The scope and stage of personalized surveys are related to both the user and the assessed object; however, the prevailing personalized surveys fail to adequately address assessed objects’ characteristics. 3) During application usage, the cumulative multimodal feedback from an individual holds great value that should be considered for improving the PAA model but unfortunately attracts insufficient attention. To address the aforementioned challenges, we introduce a new PAA paradigm called PAA+, which is structured into three distinct stages: pre-training, fine-tuning, and continual learning. Furthermore, to better reflect individual differences, we employ a familiar and intuitive application, physique aesthetics assessment (PhysiqueAA), to validate the PAA+ paradigm. We propose a dataset called PhysiqueAA50K, consisting of over 50,000 annotated physique images. Furthermore, we develop a PhysiqueAA framework (PhysiqueFrame) and conduct a large-scale benchmark, achieving state-of-the-art (SOTA) performance. Our research is expected to provide an innovative roadmap and application for the PAA community. The code and dataset are available in here.

This study investigates the trade-off between system stability and offloading cost in collaborative edge computing. While collaborative offloading among multiple edge servers enhances resource utilization, existing methods often overlook the role of queue stability in overall system performance. To address this, a multi-hop data transmission model is developed, along with a cost model that captures both energy consumption and delay. A time-varying queue model is then introduced to maintain system stability. Based on Lyapunov optimization, a dynamic offloading algorithm (LDSO) is proposed to minimize offloading cost while ensuring long-term stability. Theoretical analysis and experimental results verify that the proposed LDSO achieves significant improvements in both cost efficiency and system stability compared to the state-of-the-art.

Human-to-human interaction understanding(HIU) plays a significant role in both physical and mental health of individuals in their daily lives. Nowadays, many of HIU tasks are based on visual information, which can compromise individuals’ privacy in daily life. In this paper, we propose mmHIU, a privacy-preserving HIU system based on mmWave sensing. To achieve mmHIU, we need to address two challenges: Extraction of interaction features and processing of raw data in a manner that preserves interaction features. We design multi-level feature extraction network mmHIU-STNet, and clustering-based frame normalization strategy to address these two challenges. We evaluate mmHIU on 10,811 point cloud sequences from 9 pairs of inter-actors, and results show that its accuracy reaches 90.56%, outperforming baseline methods, with a significant improvement in recognizing complex interaction behaviors.

Subsea Internet of Things (IoT) networks have rapidly developed but still suffer the inefficient underwater-toair communication. Due to wireless signals exhibiting different properties in different media, it is difficult to use any single modality of signal for cross-medium communication. In this paper, we propose Exocoetus, a novel passive relay based water-to-air communication system. By taking advantage of the out-of-specification characteristics of the RF switch, Exocoetus can trigger acoustic-to-RF conversion even if the input voltage is below the standard threshold and use a clamp circuit to further maximize the efficiency of the acoustic-to-RF signal conversion. We design a dual-capacitor circuit-based pulse position modulation method to amplify the acoustic signals emitted by powerconstrained underwater nodes, ensuring reliable communication over greater distances. We implement a prototype of Exocoetus and evaluate its performance in the real environment. The results show that Exocoetus can achieve a communication distance of 6 meters above water and$\mathbf{1.25}$meters underwater.

While deep reinforcement learning (DRL)-based congestion control (CC) algorithms outperform traditional TCP methods in dynamic networks, their robustness is challenged by the complex influence caused by state perturbations from competing flows. Existing DRL-based CC models, constrained by their black-box nature, often suffer performance degradation under dynamic network conditions, which may indirectly alter their observations. To address this challenge, we propose Mona, an adversarial attack framework that explores vulnerabilities in existing DRL-based CC schemes while helping improve their robustness. Leveraging a dual-Critic architecture and a global state representation, Mona enables centralized training with an explicit estimation mechanism embedded in the reward function, facilitating targeted attacks without direct access to the victim’s state. Based on this design, Mona can be distributedly deployed to indirectly perturb the victim flow’s observations by injecting a stealthy flow into the bottleneck link, ultimately inducing suboptimal decisions. Experiments demonstrate Mona’s effectiveness across diverse CC models, reducing victim flow throughput by 8.69%–26.67% under various network conditions in both simulated environments and real-world network deployments, and we conduct a preliminary exploration of defense strategies through a minimax-based adversarial training framework that improves model robustness by 20%.

Exposure correction aims to restore underexposed and overexposed images to normal exposed images in a single network. However, conventional methods primarily focus on correcting non-extreme exposure cases and struggle to accurately restore lightness and structure information in extreme exposure scenarios. Through a thorough investigation, we observe that the extreme exposure correction task is limited by the lack of high-quality benchmark datasets. To address the above challenges, in this paper, we construct the first Extreme Exposure Dataset named EED by manually collecting a large number of diverse scenes. By introducing probabilistic blur kernel, EED not only ensures the rich diversity and brightness distribution of scenes but also approaches the degradation of the real world. To achieve exposure correction in extreme conditions, we propose a novel Extreme Exposure Correction Network by leveraging the mask-aware Fourier transform prior, which decouples lightness and structure components precisely. To restore severe abnormal lightness and lost structure information in extreme exposure scenes, we introduce a well-exposed referenced image to guide the coarse restoration and employ a Timestep-guided Frequency Diffusion Module for further refinement. Extensive experiments demonstrate the superiority of our dataset and method. The dataset will be available at https://github.com/juvenoia/EED.

Millimeter-wave (mmWave) radar enables privacy-preserving gesture recognition but suffers from limited training data, particularly for lying postures. Existing mmWave radar data generation methods are ineffective due to insufficient 2D video data. To this end, we design a novel system named Venus to generate realistic radar data for lying postures using few 2D videos, which addresses two key challenges including i) the simulation of diverse reflected signals and ii) few real-world data leading to low data fidelity. Venus consists of two key components: (i) a gesture sequence generation and signal simulation network, which combines several key modules, movement information extractor, spatio-temporal latent diffusion model, and mmWave signal simulator, to generate diverse gesture vertex sequences under certain conditions and simulate signal propagation characteristics to obtain coarse radar data; (ii) a meta-learning domain adaption network generates realistic radar data with few real-world data via ''meta-learning'' strategy. Extensive experiments on both generated and self-collected datasets demonstrate that Venus significantly outperforms state-of-the-art methods in recognizing gestures performed in lying postures.

In this work, we study how vision-language models (VLMs) can be utilized to enhance the safety for the autonomous driving system, including perception, situational understanding, and path planning. However, existing research has largely overlooked the evaluation of these models in traffic safety-critical driving scenarios. To bridge this gap, we create the benchmark (SafeDrive228K) and propose a new baseline based on VLM with knowledge graph-based retrieval-augmented generation (SafeDriveRAG) for visual question answering (VQA). Specifically, we introduce SafeDrive228K, the first large-scale multimodal question-answering benchmark comprising 228K examples across 18 sub-tasks. This benchmark encompasses a diverse range of traffic safety queries, from traffic accidents and corner cases to common safety knowledge, enabling a thorough assessment of the comprehension and reasoning abilities of the models. Furthermore, we propose a plug-and-play multimodal knowledge graph-based retrieval-augmented generation approach that employs a novel multi-scale subgraph retrieval algorithm for efficient information retrieval. By incorporating traffic safety guidelines collected from the Internet, this framework further enhances the model's capacity to handle safety-critical situations. Finally, we conduct comprehensive evaluations on five mainstream VLMs to assess their reliability in safety-sensitive driving tasks. Experimental results demonstrate that integrating RAG significantly improves performance, achieving a +4.73% gain in Traffic Accidents tasks, +8.79% in Corner Cases tasks and +14.57% in Traffic Safety Commonsense across five mainstream VLMs, underscoring the potential of our proposed benchmark and methodology for advancing research in traffic safety. Our source code and data are available at https://github.com/Lumos0507/SafeDriveRAG.

Event-based image reconstruction has achieved remarkable progress, benefiting from the high temporal resolution and high dynamic range of event cameras. However, most event-based methods focus on enhancing sRGB image quality, neglecting the potential of leveraging event data for RAW-to-sRGB conversion. Due to the limitations of camera sensors, images processed through standard ISP pipelines often suffer from motion blur and color distortion in dynamic scenes. In contrast, RAW images preserve uncompressed scene information, integrating event signals at this stage enables finer texture recovery and more accurate color correction. To tackle these challenges, we propose EvRAW, a novel event-assisted RAW-to-sRGB image reconstruction network that integrates event signals to promote high-fidelity sRGB image reconstruction. Specifically, we introduce a Motion-guided Structural Enhancement (MSE) module that extracts motion patterns from event streams and aggregates dynamic features to restore fine textures. Additionally, we propose an Adaptive Color Correction (ACC) module that performs region-wise gamma correction and channel-wise color decoding to enhance color fidelity under complex lighting conditions. To evaluate performance in challenging real-world scenarios, we collect a pixel-aligned RAW-Event dataset specifically for this task. Extensive experiments demonstrate that EvRAW achieves state-of-the-art performance in RAW-to-sRGB reconstruction on both synthetic and real-world datasets.

Taste perception plays an essential role in promoting human health and maintaining nutritional balance. Current taste perception techniques usually require expensive equipment and delicate storage conditions, which restrict their use primarily to laboratory settings. In this paper, we show that terahertz (THz) signals generate unique fingerprint spectra when interacting with different taste molecules in aqueous solutions. Building on this finding, we propose Molecular-level Taste Sensing (MoleSen), a contact-free gustatory sensing method aimed at achieving wireless human-like perception. Specifically, MoleSen emits terahertz signals towards the aqueous solution, captures the reflected signals, and then determines the type and concentration of the tastes by analyzing the unique fingerprint spectra of the reflected signals influenced by the taste molecules. In MoleSen, we design a bio-inspired deep learning model (DTB, Digital Taste Bud) to identify the subtle taste features diluted by water molecules. Additionally, we incorporate domain adaptive learning to address the issue of feature distribution shifts when multiple tastes are mixed. Through extensive experiments involving over 247,000 samples, we demonstrate that MoleSen can accurately differentiate the five basic tastes—sour, bitter, salty, sweet, and umami—with an accuracy of 98.5% for taste type determination and 96.9% for concentration detection. Moreover, MoleSen outperforms the human's taste sensitivity and achieves a highly accurate perception even for mixed tastes.

In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12\% for LLMs and 16\% for MLLMs compared to existing scheduling methods. Our source code is avaiable at https://github.com/MICLAB-BUPT/STP.

Trajectory similarity computation is essential for various downstream applications, such as anomaly route detection, order matching, and digital contact tracing. However, its effectiveness is confined within a single system due to privacy concerns associated with sharing raw trajectories across different systems. In this paper, we propose CrossSim, a novel framework designed to efficiently retrieve similar trajectories across all systems while preserving individual privacy. Our framework comprises three main components: i) a Trajectory Encoding Model that transforms trajectories into high-quality representations, where similarity relationships are reflected by their distances; ii) a two-stage optimization mechanism, including a Contrastive Similarity Learning stage and a Federated Similarity Learning stage, that alleviates the impact of heterogeneous similarity relationships across different systems on model training without aggregating raw trajectories; iii) a Similar Trajectory Retrieval procedure that obtains top-k similar trajectories from all systems without sharing raw trajectories. We conduct comprehensive experiments on three real-world datasets to evaluate the effectiveness of our proposed framework. The evaluation results demonstrate that CrossSim outperforms all existing schemees.

Heart rate variability (HRV) is an essential indicator of cardiovascular and nervous system function, with wide applications in health monitoring and disease management. Traditional contact-based methods like electrocardiograms (ECG) and photoplethysmography (PPG), while effective, face significant limitations in user experience, such as discomfort during prolonged use, and challenges in long-term, continuous monitoring. Meanwhile, contactless wireless sensing based on mmWave radar offers a promising alternative but is hindered by issues of directional sensing and noise interference. In this paper, we propose mmCG (mmWave Cardiac Gram), a contactless HRV monitoring system. Specifically, it integrates a heartbeat spatial localization method for directional sensing, which significantly improves the SNR, and a dynamic peak search algorithm that leverages heartbeat temporal correlations to effectively mitigate the impact of artifacts. Experimental results show that mmCG achieves advanced performance, reducing the IBI error to 9.44ms, a 51.29% improvement over existing methods. With its lightweight design and enhanced accuracy, mmCG offers a practical solution for daily HRV monitoring, with potential applications in stress management, personalized healthcare, and cardiovascular disease monitoring.

As cloud computing revolutionizes various fields, the demand for scalable and flexible computing resources has grown significantly. Applications in large-scale engineering simulations, artificial intelligence, and data analysis require substantial computational power and memory, placing pressure on traditional systems. With its heterogeneous resource pools, cloud computing offers a promising solution by enabling the dynamic allocation of diverse, distributed resources. These resource pools facilitate parallel task execution, accelerate computations, and enhance system flexibility. However, efficiently managing these resources remains a complex, NP-hard challenge due to the vast search space, resource fragmentation, and the need for dynamic adjustments. In this paper, we first develop a novel multi-task flow representation model using a deep graph neural network (GNN) and a resource pool representation model based on a convolutional neural network (CNN). These models describe the dependencies among multiple tasks, resource requirements, and resource distribution in the multi-DAG job arrival scenario. Then, we design a dynamic resource allocation strategy model based on deep reinforcement learning (DRL) to reduce job processing time. Finally, we compare the performance of our proposed method with the other eight baseline algorithms using test datasets of various scales and under different arrival modes consisting of Montage, CyberShake, Broadband, Epigenomics, LIGO, VGG 16-SVD, and Edge Detection datasets.

On-demand video streaming continues to dominate the Internet, posing a formidable challenge in designing efficient adaptive bitrate (ABR) algorithms to enhance user quality-of-experience (QoE), particularly amplified by increasing video resolutions (e.g., from 1080P to 2K, 4K, and even 8K) and dynamic Internet conditions. Through a comprehensive study, we identify a common limitation in both existing throughput-based and hybrid-based ABR algorithms: they rely on coarse-grained network bandwidth estimation, missing detailed and accurate (i.e., millisecond-level) network variations. This often leads to misguided resolution (corresponding to bitrate level) decisions, resulting in unsatisfactory QoE. In this work, we propose SuperABR, a fine-grained throughput-driven ABR solution aimed at achieving the optimal bitrate adaptation. To accomplish this, SuperABR first incorporates a two-stage learning module, generating fine-grained future throughput to provide a near-Oracle network view. SuperABR then uses this fine-grained throughput to accurately calculate the download duration for a video chunk, transforming it into the optimal resolution decision via a custom-designed QoE benefit model. We have implemented SuperABR as a lightweight plug-in interface on a standard DASH framework and evaluate it over extensive real-world network traces. Extensive experiments demonstrate that SuperABR can generate accurate future throughput, resulting in a remarkable$1.21\sim 1.46\times $QoE improvement over classic ABR solutions.

Despite the recent significant progress in palmprint recognition, there are still challenges in scaling up this technology for real-world scenarios. One major challenge in developing practical, highly accurate recognition models is the shortage of comprehensive public datasets that can be used to evaluate performance at extremely low false accept rates (FAR). Furthermore, obtaining high-precision recognition models is greatly hindered by pattern variance, a notable challenge with the palmprint modality given the current technology pipeline. To address the above problems, we first collect a palmprint dataset, WebPalm, that contains the largest number of identities as well as images that have been disclosed so far. To reduce pattern variance, we propose RegPalm, a novel framework that unifies palmprint orientations (UPO) and learns pairwise spatial registration of palmprints (PPR) in an end-to-end manner. UPO harmonizes the pattern variance between left and right orientations, hence enhancing the network’s perceptual capabilities. PPR decreases both inter-class and intra-class pattern variance to improve the model’s ability to recognize hard examples. RegPalm reinforces the model by discriminating subtle palmprint features, thereby improving its performance under extremely low FAR. RegPalm not only surpasses the current state-of-the-art by 9.3 percentage points (pp) and 12.2 pp in TAR@FAR=1e-6 under the 1:1 and 1:3 open-set protocols, respectively, but also consistently achieves a 16 pp improvement in TAR@FAR=1e-9 on the WebPalm benchmark. The experimental results fully reveal the practicability and superiority of RegPalm in the real world.

Predicting air quality using multimodal data is crucial to comprehensively capture the diverse factors influencing atmospheric conditions. Therefore, this study introduces a multimodal learning framework that integrates outdoor images with traditional ground-based observations to improve the accuracy and reliability of air quality predictions. However, aligning and fusing these heterogeneous data sources poses a formidable challenge, further exacerbated by pervasive data incompleteness issues in practice. In this article, we propose a novel incomplete multimodal learning approach (iMMAir) to recovery missing data for robust air quality prediction. Specifically, we first design a shallow feature extractor to capture modal-specific features within the latent representation space. Then we develop a conditional diffusion-driven recovery module to mitigate the distribution gap between the recovered and true data. This module further incorporates two conditional constraints of temporal correlation and semantic consistency for effective modal completion. Finally, we reconstruct incomplete modalities and fuse available data using a multimodal transformer network to predict the air quality. To alleviate the modality imbalance problem, we employ an adaptive gradient modulation strategy to adjust the optimization of each modality. Experimental results demonstrate that iMMAir significantly reduces prediction errors, outperforming baseline models by an average of 5.6% and 2.5% in air quality regression and classification tasks. Our source code and data are available at https://github.com/pestasu/IMMAir .

The adage "Beautiful Outside But Ugly Inside" resonates with the security and explainability challenges encountered in image aesthetics assessment (IAA). Although deep neural networks (DNNs) have demonstrated remarkable performance in various IAA tasks, how to probe, explain, and enhance aesthetics-oriented "black-box" models has not yet been investigated to our knowledge. This lack of investigation has significantly impeded the commercial application of IAA. In this paper, we investigate the susceptibility of current IAA models to adversarial attacks and aim to elucidate the underlying mechanisms that contribute to their vulnerabilities. To address this, we propose a novel diffusion-based framework as an attacker (DA3Attacker), capable of generating adversarial examples (AEs) to deceive diverse black-box IAA models. DA3Attacker employs a dedicated Attack Diffusion Transformer, equipped with modular aesthetics-oriented filters. By undergoing two unsupervised training stages, it constructs a latent space to generate AEs and facilitates two distinct yet controllable attack modes: restricted and unrestricted. Extensive experiments on 26 baseline models demonstrate that our method effectively explores the vulnerabilities of these IAA models, while also providing multi-attribute explanations for their feature dependencies. To facilitate further research, we contribute the evaluation tools and four metrics for measuring adversarial robustness, as well as a dataset of 60,000 re-labeled AEs for fine-tuning IAA models. The resources are available here.

Action Quality Assessment (AQA), which aims at the automatic and fair evaluation of athletic performance, has gained increasing attention in recent years. However, athletes are often in rapid movement and the corresponding visual appearance variances are subtle, making it challenging to capture fine-grained pose differences and leading to poor estimation performance. Furthermore, most common AQA tasks, such as diving in sports, are usually divided into multiple sub-actions, each of which contains different durations. However, existing methods focus on segmenting the video into fixed frames, which disrupts the temporal continuity of sub-actions resulting in unavoidable prediction errors. To address these challenges, we propose a novel action quality assessment method through hierarchically pose-guided multi-stage contrastive regression. Firstly, we introduce a multi-scale dynamic visual-skeleton encoder to capture fine-grained spatio-temporal visual and skeletal features. Compared to mask or auxiliary visual features, skeletal features provide a more accurate representation during athletic movements. Then, a procedure segmentation network is introduced to separate different sub-actions and obtain segmented features. Afterwards, the segmented visual and skeletal features are both fed into a multi-modal fusion module as physics structural priors, to guide the model in learning refined activity similarities and variances. Finally, a multi-stage contrastive learning regression approach is employed to learn discriminative representations and output prediction results. In addition, we introduce a newly-annotated FineDiving-Pose Dataset to improve the current low-quality human pose labels. In experiments, the results on FineDiving and MTL-AQA datasets demonstrate the effectiveness and superiority of our proposed approach. Our source code and dataset are available at https://github.com/Lumos0507/HP-MCoRe.

Low-light video enhancement is a critical task in computer vision with a wide range of applications. However, there is a lack of high-quality benchmark datasets in this field. To address this issue, we collect a high-quality low-light video dataset using a well-designed camera system. The videos in our dataset feature apparent camera motion and strict spatial alignment. In order to achieve general low-light video enhancement, we propose a Retinex-based method called Light Adjustable Network (LAN). LAN iteratively adjusts the brightness and adapts to different lighting conditions in various real-world scenarios, producing visually appealing results. We further develop a new dataset capture method and low-light video enhancement method to address the limitation of our previous dataset in capturing dynamic scenes and previous method. The new camera setup and capture method enable the recording of real continuous videos and generate the new dataset. Our new low-light video enhancement method, LAN++, leverages a new inter-frame relationship, difference images. It utilizes the texture information contained in the difference images of dynamic scenes to supplement the high-frequency details of the original features, which produce sharper and more realistic output images. The extensive experiments demonstrate the superiority of our low-light video dataset and enhancement method. Our dataset can be downloaded at https://pan.baidu.com/s/1d3EljvVduVM0wUOvzjWaqA?pwd=p45g.

Urban land use, intrinsically linked to people’s daily activities, undergoes continuous evolution, presenting a complex interplay that remains partially understood. To bridge this gap, our study leverages fine-grained human mobility data to predict these changes, adopting a novel approach that conceptualizes “community-level” land use shifts as a regression problem and represents citywide changes through dynamic graphs. We harness recent advancements in graph neural networks (GNNs), which, despite their success in various applications, face challenges in directly predicting land use changes due to the temporal mismatch between the slow evolution of urban land and the immediacy of human mobility data. Our research stands out by introducing a temporal skeleton for dynamic GNNs to synchronize human activity graphs with urban land use changes, a dynamic heterogeneous GNN approach for integrating diverse human activity data to capture essential temporal dependencies, and a novel algorithm powered by causal inference to elucidate the primary factors influencing land use predictions at the community level, all of which contribute to a training process informed by the generated causal graph. Empirically validated on three real-world datasets, our model demonstrates a performance leap over state-of-the-art baselines, marking a pivotal step toward understanding and predicting the dynamics of urban land use.

Real-world time series data is inherently complex, noisy, and exhibits abrupt changes, posing various challenges in data modeling. Given the ubiquity and importance of time-series data, accurately forecasting change points, instead of the overall predictive performance, has become increasingly attractive as it assists in risk mitigation and loss prevention. In this task, we argue that the past and future interactions involving the target points determine the comprehensive structure contributing to abrupt changes. However, traditional left-to-right auto-regressive approaches only consider the historical sequence, resulting in a flawed learning process and limited performance. In this paper, we extend the teacher-student learning and propose a novel S elf-optimizing T eacher and A uto-matching S tudent framework (named ST-AS) to predict change points in time series data. Our framework models change point representations specific to the target points by integrating future knowledge while avoiding data leakage. Specifically, we design a Gumbel-enhanced filter for our self-optimizing teacher, which constructs selected and filtered sub-groups to derive discriminative representations using a positive-unlabeled learning strategy. Given this well-trained teacher, we propose an adaptive pattern matcher for our auto-matching student model, which learns missing information by automatically aligning relevant features. After that, a novel two-stage dual-guided learning process is then designed to mimic teacher’s decision-making behavior and enhance student’s excavate capability. Finally, we conduct extensive experiments on four real-world datasets to demonstrate that our proposed ST-AS exhibits significantly better prediction performance compared to existing state-of-the-art alternatives.

Massive urban-scale vehicle trajectories benefit various downstream applications. However, trajectories collected from existing sensing systems are often incomplete, necessitating the recovery of coarse-grained trajectories. Considering that mobility knowledge learned from a single system is less representative of all vehicles or covers only partial road segments, it becomes essential to combine diverse data from multiple systems to support trajectory recovery. Therefore, we learn the impacts of mobility intentions and dynamic traffic conditions on the movement of vehicles from trajectories aggregated across different systems to recover their travel routes on unobservable road intersections. Nonetheless, aggregating raw data across multiple systems raises privacy concerns. This data isolation compounds challenges in acquiring comprehensive mobility intentions and traffic conditions, thereby impairing recovery performance. In this paper, we proposeCrossTrace, a two-stage framework for privacy-aware cross-system trajectory recovery: in theTraffic Condition Inferencestage, a Split Learning pipeline with a multi-view graph neural network is utilized to infer complete traffic conditions for all road segments; in theTrajectory Recoverystage, a Federated Learning pipeline with dedicated modules is utilized to recover missing points by fusing inferred traffic conditions and mobility intentions. Extensive experiments on two large-scale trajectory datasets demonstrate thatCrossTraceoutperforms all alternative schemes.

Millimeter wave radar is gaining traction recently as a promising modality for enabling pervasive and privacy-preserving gesture recognition. However, the lack of rich and fine-grained radar datasets hinders progress in developing generalized deep learning models for gesture recognition across various user postures (e.g., standing, sitting), positions, and scenes. To remedy this, we resort to designing a software pipeline that exploits wealthy 2D videos to generate realistic radar data, but it needs to address the challenge of simulating diversified and fine-grained reflection properties of user gestures. To this end, we designG3Rwith three key components: i) agesture reflection point generatorexpands the arm's skeleton points to form human reflection points; ii) asignal simulation modelsimulates the multipath reflection and attenuation of radar signals to output the human intensity map; iii) anencoder-decoder modelcombines asampling moduleand afitting moduleto address the differences in number and distribution of points between generated and real-world radar data for generating realistic radar data. We implement and evaluateG3Rusing 2D videos from public data sources and self-collected real-world radar data, demonstrating its superiority over other state-of-the-art approaches for gesture recognition.

Cross-Technology Communication (CTC) is an emerging technique that enables direct interconnection among incompatible wireless technologies. However, for the downlink from WiFi to multiple IoT technologies, serially emulating and transmitting the data of each IoT technology has extremely low spectrum efficiency. In this paper, we propose WiCast, a parallel CTC that uses IEEE 802.11ax to emulate a composite signal that can be received by commodity BLE, ZigBee, and LoRa devices. By taking advantage of OFDMA in 802.11ax, WiCast uses a single Resource Unit (RU) for parallel CTC and sets other RUs free for high-rate WiFi users. But such a sophisticated composite signal is very easily distorted by emulation imperfections, dynamic channel noises, cyclic prefix, and center frequency offset. We propose a CTC link model that jointly models the emulation errors and channel distortions. Then we carve the emulated signal with elaborate compensations in both time and frequency domains. Based on the proposed CTC scheme, a unified Media Access Control approach is introduced to discover and synchronize the heterogeneous IoT devices. We implement a prototype of WiCast using USRP N210 platform along with commodity ZigBee, BLE, and LoRa devices. The extensive experiments demonstrate WiCast can achieve an efficient parallel transmission with the aggregated goodput up to 390.24kbps.

Current WiFi imaging approaches focus on monitoring dynamic targets to facilitate easy object distinction and capture rich signal reflections for image construction. In static object imaging, massive antenna array or emulated antenna array is often necessary. We proposeWiCamera, a novel WiFi imaging prototype that utilizes vortex electromagnetic waves (VEMWs) to monitor stationary human postures using commodity WiFi, by generating human silhouettes with only$3 \times 3$MIMO. VEMWs possess a helical wavefront with different phase variations, enabling the imaging of stationary objects through different OAM (Orbital Angular Momentum) modes with time-division multiplexing.WiCameraemits three OAM modes waves from WiFi devices and utilizes their phase variations for imaging. By ray tracing the received signals to a target image plane,WiCameragenerates a wavefront image. A generative adversarial network (GAN)-based model is further utilized to refine the wavefront image and create a high-resolution human silhouette. The system's output images are evaluated using metrics such as structural similarity index measure (SSIM) and Szymkiewicz-Simpson coefficient (SSC), comparing them to ground truth images captured by cameras. The evaluation shows thatWiCameraperforms consistently well in various environments and with different users, with an SSIM reaching up to 0.89 and an SSC reaching up to 0.93.

Thoracoabdominal Asynchrony (TAA) is a key metric in respiration monitoring, which characterizes the non-parallel periodical motion of human's rib cage (RC) and abdomen (AB) during each breath. Long-term measurement of TAA plays a significant role in respiration health tracking. Existing TAA measurement methods including Respiratory Inductive Plethysmography (RIP) and Optoelectronic Plethysmography (OEP) all intrusive to subjects and have certain requirements on operation conditions, which limit their usage to hospital scenario. To address this gap, we proposemmTAA, the first mmWave-based, non-intrusive TAA measurement system ready for ubiquitous usage in daily-life. InmmTAA, we design a Two-stage RC-AB centroid finding module, aiming to identify the most probable location of RC-AB centroid, which can best represent RC and AB in mmWave sensing scenario. Subsequently, we design TAANet, a novel Convolutional Neural Network (CNN)-based architecture with residual modules, tailored for TAA measurement. Meanwhile, in order to address the imbalance of continuous data, we add imbalance information equalizer including feature and label equalizer during network training. We implementmmTAAon a commonly used multi-antenna mmWave radar. We prototype, deploy and evaluatemmTAAon 25 subjects and 25.7h data in total.mmTAAachieves 4.01$^{\circ }$MAE and 1.56$^{\circ }$average error, close to OEP method.

Edge-cloud collaborative learning emerges as a promising paradigm for adapting pre-trained deep neural network (DNN) models to the ever-changing edge data environments and specific downstream tasks. However, the heterogeneity of edge devices and unlabeled local data hinder the effectiveness of existing collaborative learning approaches. To address the above issues, we proposeACL, a novel adaptive edge-cloud collaborative learning paradigm for heterogeneous devices with unlabeled local data. InACL, we first useFedNAS, a neural architecture search algorithm designed for collaborative learning to generate a customized model on each participating device, and then a lightweight semi-supervised collaborative learning frameworkHSSCLis used to fine-tune the pre-trained DNN model. Compared with the SOTA collaborative learning approaches,ACLachieves significant accuracy improvement, averaging 31.5% for image classification and 15.5% for object detection. Furthermore, it reduces time overhead by 3.1-5.1× and memory overhead by 6.3-12.5×. We will release our models and tools.

LoRa is widely deployed for various applications. Though the knowledge of the channel occupancy is the prerequisite of many aspects of network management, acquiring the channel occupancy for LoRa is challenging due to the large number of possible channels. In this paper, we propose${\sf LoRadar}$, a novel LoRa channel occupancy acquirer based on cross-channel scanning. Our in-depth study finds that Channel Activity Detection (CAD) in a narrow band can indicate the channel activities of wide bands because they have the same slope in the time-frequency domain. Based on this finding, we design a cross-channel scanning mechanism that infers the channel occupancy states of all the overlapping channels by the distribution of CAD results. We elaborately select and adjust the CAD settings to enhance the distribution features and design a pattern correction method to cope with distribution distortions. We also design a CAD scheduler to deal with the low duty-cycle LoRa operations. We implement${\sf LoRadar}$on commercial LoRa platforms and evaluate its performance in the indoor testbed and two outdoor deployed networks. The experimental results show that${\sf LoRadar}$can achieve a detection accuracy of 0.99 and reduce the acquisition overhead by up to 90%, compared to the traversal-based methods.

Recently, an increasing number of few-shot image classification methods have been proposed, and they aim at seeking a learning paradigm to train a high-performance classification model with limited labeled samples. However, the neglect of part-level relationships causes few-shot methods to struggle to distinguish between closely similar subcategories, which makes it difficult for them to solve the fine-grained image classification problem. To tackle this challenging task, this paper proposes a fine-grained few-shot image classification method that exploits both intra-part and inter-part relationships among different samples. To establish comprehensive relationships, we first extract multiple discriminative descriptors from the input image, representing its different parts. Then, we propose to define the metric spaces by interpolating intra-part relationships, which can help the model adaptively find clear boundaries for these confusing classes. Finally, since the unlabeled image has high similarities to all classes, we project these similarities into a high-dimension space according to the inter-part relationship and interpolate a parameterized classifier to discover the subtle differences among these similar classes. To evaluate our proposed method, we conduct extensive experiments on various fine-grained datasets. Without any pre-train/fine-tuning process, our approach clearly outperforms previous few-shot learning methods, which demonstrates the effectiveness of our approach.

Mobile Edge Computing (MEC) is a key component of 5G ecosystem, designed to support applications with stringent latency requirements. The fundamental idea is to deploy servers closer to end-users, such as on the network edge, rather than in remote clouds. While conceptually sound, operational 5G networks often lack coordination with MEC, leading to intolerably long response latency. In this work, we propose Sonata, which tightly integrates 5G RAN and MEC at the user space to ensure the performance of latency-critical video analytics. To achieve this, Sonata precisely customizes users’ service demands by fusing application-layer content changes from MEC servers with instantaneous physical-layer dynamics from the 5G Radio Access Network (RAN). It then enforces a deadline-strict resource provision to meet these service demands through real-time interactions between the 5G RAN and MEC servers, in a lightweight and standard-compatible manner. We prototype and evaluate Sonata on a software-defined 5G MEC platform. Our results demonstrate that Sonata achieves an average reduction in response latency of 67.82% compared to conventional 5G edge systems.

Unmanned Aerial Vehicle (UAV) equipped with a gateway shows great potential for data collection in many scenarios, especially for the areas lacking of public network infrastructures. However, our in-field experiments on UAV-assisted LoRa networks show that a large throughput gap exists between the ground-to-air and ground-to-ground transmissions. We find that the misalignment of the radiation direction of transceiver antennas with height difference leads to additional signal strength loss, which is ignored by existing ground-to-ground transmissions. In this paper, we propose a directivity-aware ground-to-air link model called annulus model to quantify the impact of directivity on the ground-to-air link quality. Based on our model, a new ground-to-air channel access scheme for UAV-assisted LoRa networks,PreLoRa, is proposed. By predicting the link quality variations,PreLoRaschedules the transmission periods and adopts optimal transmission configurations for ground nodes to improve the link throughput. We implementPreLoRaon commercial LoRa platforms and extensively evaluate its performance in the wild. Experimental results show thatPreLoRacan significantly improve data collection throughput by up to 65.5% compared to baseline methods.

Existing CPU scaling approaches have limitations that can lead to inefficient resource allocation and increased penalty costs for tasks with soft deadlines running in container clouds. First, quota allocation based approaches overlook the gap between the obtainable CPU time and allocated quota, causing inefficient CPU utilization and unexpected task behaviors. Second, core allocation based approaches ignore workload dynamics within decision intervals, potentially increasing contention for CPU time among tasks on the same core. Third, existing approaches lack strategies to allocate more resources to critical tasks that incur higher penalty costs when the node’s capacity is insufficient. This article proposes a reinforcement learning based hybrid CPU scaling approach that allocates quota and cores jointly, aiming to minimize penalty costs for timeouts. Based on the embedding generated from a fine-grained CPU demand series, we allocate CPU quotas and determine a dynamic workload-aware core sharing scheme using an attention mechanism that combines respective demands and global criticality regarding penalty costs. Additionally, we integrate the resource gap, CPU time contention, and penalty costs into the reward function to update our model online. The experimental results show the proposed approach achieves state-of-the-art performance.

Exposure correction aims to enhance images suffering from improper exposure to achieve satisfactory visual effects. Despite recent progress, existing methods generally mitigate either overexposure or underexposure in input images, and they still struggle to handle images with mixed exposure, i.e., one image incorporates both overexposed and underexposed regions. The mixed exposure distribution is non-uniform and leads to varying representation, which makes it challenging to address in a unified process. In this paper, we introduce an effective Region-aware Exposure Correction Network (RECNet) that can handle mixed exposure by adaptively learning and bridging different regional exposure representations. Specifically, to address the challenge posed by mixed exposure disparities, we develop a region-aware de-exposure module that effectively translates regional features of mixed exposure scenarios into an exposure-invariant feature space. Simultaneously, as de-exposure operation inevitably reduces discriminative information, we introduce a mixed-scale restoration unit that integrates exposure-invariant features and unprocessed features to recover local information. To further achieve a uniform exposure distribution in the global image, we propose an exposure contrastive regularization strategy under the constraints of intra-regional exposure consistency and inter-regional exposure continuity. Extensive experiments are conducted on various datasets, and the experimental results demonstrate the superiority and generalization of our proposed method. The code is released at: https://github.com/kravrolens/RECNet.

In this paper, we propose a novel model called SGFormer, Semantic Graph TransFormer for point cloud-based 3D scene graph generation. The task aims to parse a point cloud-based scene into a semantic structural graph, with the core challenge of modeling the complex global structure. Existing methods based on graph convolutional networks (GCNs) suffer from the over-smoothing dilemma and can only propagate information from limited neighboring nodes. In contrast, SGFormer uses Transformer layers as the base building block to allow global information passing, with two types of newly-designed layers tailored for the 3D scene graph generation task. Specifically, we introduce the graph embedding layer to best utilize the global information in graph edges while maintaining comparable computation costs. Furthermore, we propose the semantic injection layer to leverage linguistic knowledge from large-scale language model (i.e., ChatGPT), to enhance objects' visual features. We benchmark our SGFormer on the established 3DSSG dataset and achieve a 40.94% absolute improvement in relationship prediction's R@50 and an 88.36% boost on the subset with complex scenes over the state-of-the-art. Our analyses further show SGFormer's superiority in the long-tail and zero-shot scenarios. Our source code is available at https://github.com/Andy20178/SGFormer.

Weakly-supervised temporal action localization aims to locate action regions and identify action categories in untrimmed videos simultaneously by taking only video-level labels as the supervision. Pseudo label generation is a promising strategy to solve the challenging problem, but the current methods ignore the natural temporal structure of the video that can provide rich information to assist such a generation process. In this paper, we propose a novel weakly-supervised temporal action localization method by inferring salient snippet-feature. First, we design a saliency inference module that exploits the variation relationship between temporal neighbor snippets to discover salient snippet-features, which can reflect the significant dynamic change in the video. Secondly, we introduce a boundary refinement module that enhances salient snippet-features through the information interaction unit. Then, a discrimination enhancement module is introduced to enhance the discriminative nature of snippet-features. Finally, we adopt the refined snippet-features to produce high-fidelity pseudo labels, which could be used to supervise the training of the action localization network. Extensive experiments on two publicly available datasets, i.e., THUMOS14 and ActivityNet v1.3, demonstrate our proposed method achieves significant improvements compared to the state-of-the-art methods. Our source code is available at https://github.com/wuli55555/ISSF.

Most current arbitrary-scale image super-resolution (SR) methods has commonly relied on simulated data generated by simple synthetic degradation models (e.g., bicubic down-sampling) at continuous various scales, thereby falling short in capturing the complex degradation of real-world images. This limitation hinders the visual quality of these methods when applied to real-world images. To address this issue, we propose the Continuous Optical Zooming dataset (COZ), by constructing an automatic imaging system to collect images at fine-grained various focal lengths within a specific range and providing strict image pair alignment. The COZ dataset serves as a benchmark to provide real-world data for training and testing arbitrary-scale SR models. To enhance the model's robustness against real-world image degradation, we propose a Local Mix Implicit network (LMI) based on the MLP-mixer architecture and meta-learning, which directly learns the local texture information by simultaneously mixing features and coordinates of multiple independent points. The extensive experiments demonstrate the superior performance of the arbitrary-scale SR models trained on the COZ dataset compared to models trained on simulated data. Our LMI model exhibits the superior effectiveness compared to other models. This study is of great significance in developing more efficient algorithms and improving the performance of arbitrary-scale image SR methods in practical applications. Our dataset and codes are available at https://github.com/pf0607/COZ.

Remarkable advancements have been made in the field of object detection, and given its widespread application, it is of paramount importance to investigate the robustness of detection models. However, previous methods have primarily focused on models based on Convolutional Neural Networks (CNNs), seriously neglecting the Transformer-based models that develop rapidly but exhibit obvious differences in terms of information processing. Therefore, this paper aims to address this gap by exploring potential attacks arising from the self-attention mechanism inhered in Transformer. Specifically, we propose a novel adversarial attack scenario targeting Transformer-based object detection models, where only objects of specific class fail to be detected, while irrelevant objects remain undisturbed. Therefore, human perception is hard to find errors even with the detector fail. To achieve this goal, we introduce an adversarial patch generation method, termed Class-Specific Adversarial (CSAdv) patches, which simultaneously leverages class probability to attack specific objects and utilizes the output from Transformer decoder structures, Query Output, to protect irrelevant objects. Due to the long-range interactions of Transformer, the adversarial patch does not need to directly cover or closely surround the specific objects. Instead, it achieves remote targeted attacks simply by being placed in the corner of image, which greatly enhances the concealment of patches. Extensive experiments are conducted on various benchmark datasets and Transformer-based baselines, and the experimental results show that CSAdv can effectively mask certain class while keeping other classes as unaffected as far as possible.

With the advancement of mobile transmission and the computational limitations of vehicle terminals, cloud-based autonomous driving has gained significant traction. In order to ensure the quality of video transmission, Adaptive Bitrate (ABR) algorithms are commonly employed. In this paper, we conduct extensive measurements on current ABR algorithms and typical autonomous driving tasks, and we identify two key limitations: (i) existing ABR algorithms commonly apply uniform transmission strategies across diverse driving tasks, and (ii) there is no clear correlation between Quality-of-Experience (QoE) and the performance of autonomous driving tasks, though we reveal specific relationships between them. To overcome these challenges, we propose ATA, a novel task-oriented ABR streaming transmission framework tailored for autonomous driving. ATA enables vehicles to upload streaming optimized for specific autonomous driving task, to optimize the performance of the autonomous driving task in fluctuating network environments. To achieve ATA, we devise a task-oriented ABR strategy, by selecting transmission approaches based on the task requirements. Additionally, we propose an initial bitrate selector, grounded in the Pareto frontier model, to ensure the selection of optimal initial bitrates aligning with current bandwidth conditions. Trace-driven experiments demonstrate that ATA significantly outperforms existing ABR schemes in autonomous driving scenarios. Notably, it achieves improvements in Average Precision (AP) for object detection tasks by 5.06% to 14.25%, reduces Average Orientation Error (AOE) in direction prediction tasks by 7.69% to 15.21%, and enhances overall task performance by 10.27% to 12.55% in multitask scenarios.

In recent years, there has been growing interest in the video-based action quality assessment (AQA). Most existing methods typically solve AQA problem by considering the entire video yet overlooking the inherent stage-level characteristics of actions. To address this issue, we design a novel Multi-stage Contrastive Regression (MCoRe) framework for the AQA task. This approach allows us to efficiently extract spatial-temporal information, while simultaneously reducing computational costs by segmenting the input video into multiple stages or procedures. Inspired by the graph contrastive learning, we propose a new stage-wise contrastive learning loss function to enhance performance. As a result, MCoRe demonstrates the state-of-the-art result so far on the widely-adopted fine-grained AQA dataset. Our source code is available at https://github.com/Angel-1999/MCoRe.

Unlike existing studies that focus on intra-channel interference, in this paper, we reveal cross-channel interference when collided chirps with different bandwidths have the same slope in the time-frequency domain. Existing methods are inefficient in resolving this type of interference because the demodulation features they use rely on accurate time-domain distributions including the start and end time of all chirps, which is unavailable for uncompleted chirps within the limited receiving bandwidth. We propose SO-LoRa which utilizes the difference in collided chirps' time-domain distributions to identify the target chirp under interference. SO-LoRa adopts self-dechirp operation that maps the chirp's time-domain distribution to recognizable amplitude change of energy peaks that reflect the difference. However, for real received chirps, the amplitude change is unreliable due to the random phase drift and amplified channel noise. So we propose a phase correction method that uses the model between phase difference and the signal energy. We also design time-domain filtering that suppresses noise before self-dechirp. Finally, to avoid extra false energy peaks generated by self-dechirp confusing the demodulation, we separate chirps which cause peak overlapping into different groups and individ-ually perform self-dechirp. The experiments show that SO-LoRa reduces the Symbol Error Rate (SER) by up to 88.6 % compared with state-of-the-art methods.

Real-world datasets often exhibit long-tailed distributions, compromising the generalization and fairness of learning-based models. This issue is particularly pronounced in Image Aesthetics Assessment (IAA) tasks, where such imbalance is difficult to mitigate due to a severe distribution mismatch between features and labels, as well as the great sensitivity of aesthetics to image variations. To address these issues, we propose an Enhancer against Long-Tail for Aesthetics-oriented models (ELTA). ELTA first utilizes a dedicated mixup technique to enhance minority feature representation in high-level space while preserving their intrinsic aesthetic qualities. Next, it aligns features and labels through a similarity consistency approach, effectively alleviating the distribution mismatch. Finally, ELTA adopts a specific strategy to refine the output distribution, thereby enhancing the quality of pseudo-labels. Experiments on four representative datasets (AVA, AADB, TAD66K, and PARA) show that our proposed ELTA achieves state-of-the-art performance by effectively mitigating the long-tailed issue in IAA datasets. Moreover, ELTA is designed with plug-and-play capabilities for seamless integration with existing methods. To our knowledge, this is the first contribution in the IAA community addressing long-tail. All resources are available in here.

Accurate and timely air quality prediction is crucial for cities and individuals to effectively take necessary precautions against potential air pollution. Existing studies typically rely on building prediction models based on large-scale monitoring data, often designed for specific tasks. Recently, pre-trained large language models (LLMs) have achieved significant progress in various time series analysis tasks due to their powerful representation and inference capabilities. However, their application to air quality data with spatio-temporal features remains largely unexplored. In this work, we propose LLMAir, an adaptive reprogramming approach that adapts pre-trained LLMs for air quality prediction. We first construct spatiotemporal tokens based on monitoring stations by integrating value, node, and time embeddings. Next, we design an adaptive semantic-enhanced reprogramming module to compute similarity matching scores between our spatiotemporal tokens and pre-trained word embeddings for alignment. We employ a semantic regulator to generate the optimal length of word prototypes, which serve as prompt prefixes for adaptive reprogramming and guiding the spatiotemporal token embeddings into the frozen LLM. Additionally, we jointly optimize predictive error and alignment loss to train our model. Experimental results demonstrate that LLMAir achieves state-of-the-art performance in air quality prediction and few-shot forecasting across two real-world datasets.

Aortic stenosis (AS) is one of the most common and severe valvular heart diseases, which can cause sudden cardiac death. Early detection and diagnosis are the most effective ways to prevent the irreversible progression of AS. Existing methods mainly rely on large or complex devices such as echocardiography, 12-leads electrocardiogram, which are marred by too many medical resources like experimental experts. There has been recent research showing promising results using cardiomechanical signals, and the need for a daily, robust, convenient, low-cost, and user-friendly AS detection system has become more critical than ever. In this paper, we propose FinP-AS, an innovative AS detection system that uses photoplethysmogram (PPG) sensors to achieve fast and low-cost detection of AS. However, using simple and cost-effective PPG sensors does not necessarily make the data analysis process straightforward. Firstly, due to the inherent limitations of devices, PPG signals we captured are always fraught with complex noise and exhibit unclear periodic features. To overcome that, we utilize window slicing and embedding strategies on raw PPG signals, which enhance the periodic characteristics of the PPG signal. Additionally, the features of aortic valve activity are significantly attenuated by the time they are transmitted to the finger, which makes AS features more difficult to extract. To tackle the problem, we have refined our asymmetric convolutional network architecture by incorporating depthwise separable convolutions and residual connections, which allows the network to detect subtle features of AS symptoms in the PPG signal from various depths and orientations beneath the subject’s skin, while simultaneously reducing the parameter count by $50 \%$ and easing the training process. An empirical evaluation of the FinP-AS model across nearly 80 subjects demonstrates robust AS detection, with the best accuracy of $94 \%$, and sensitivity of $\mathbf{9 8 \%}$.

The current deployment of extremely shallow-shared-buffer switches in data center networks has posed challenges to widely adopted ECN-based congestion control algorithms, leading to the issue of ECN failure. Switches may not allocate sufficient buffer space for each port, leading to the possibility that the ECN marking threshold exceeds the buffer limit per port. This results in excessive packet loss during bursts, even before the ECN markings take effect. To address this problem, we propose BBQ, an automatic ECN tuning system based on reinforcement learning. BBQ ensures that the ECN threshold does not exceed the buffer capacity allocated to the port, thus avoiding the ECN failure issue. Besides, BBQ is designed to adapt to switches with varying buffer sizes ensuring generalization. We validate the effectiveness of BBQ through experiments conducted with shallow buffering and high bursts. The results show that BBQ efficiently controls the packet loss rate of incast flows to within 3%, 1.2 times lower than State-of-the-Arts in shallow-buffered environments.

Exposure correction aims to enhance visual data suffering from improper exposures, which can greatly improve satisfactory visual effects. However, previous methods mainly focus on the image modality, and the video counterpart is less explored in the literature. Directly applying prior image-based methods to videos results in temporal incoherence with low visual quality. Through thorough investigation, we find that the development of relevant communities is limited by the absence of a benchmark dataset. Therefore, in this paper, we construct the first real-world paired video dataset, including both underexposure and overexposure dynamic scenes. To achieve spatial alignment, we utilize two DSLR cameras and a beam splitter to simultaneously capture improper and normal exposure videos. Additionally, we propose an end-to-end video exposure correction network, in which a dual-stream module is designed to deal with both underexposure and overexposure factors, enhancing the illumination based on Retinex theory. The extensive experiments based on various metrics and user studies demonstrate the significance of our dataset and the effectiveness of our method. The code and dataset are available at https://github.com/kravrolens/VECNet.

In the last year, universal monocular metric depth estimation (universal MMDE) has gained considerable attention, serving as the foundation model for various multimedia tasks, such as video and image editing. Nonetheless, current approaches face challenges in maintaining consistent accuracy across diverse scenes without scene-specific parameters and pre-training, hindering the practicality of MMDE. Furthermore, these methods rely on extensive datasets comprising millions, if not tens of millions, of data for training, leading to significant time and hardware expenses. This paper presents SM4Depth, a model that seamlessly works for both indoor and outdoor scenes, without needing extensive training data and GPU clusters. Firstly, to obtain consistent depth across diverse scenes, we propose a novel metric scale modeling, i.e., variation- based unnormalized depth bins. It reduces the ambiguity of the conventional metric bins and enables better adaptation to large depth gaps of scenes during training. Secondly, we propose a ''divide and conquer'' solution to reduce reliance on massive training data. Instead of estimating directly from the vast solution space, the metric bins are estimated from multiple solution sub-spaces to reduce complexity. Additionally, we introduce an uncut depth dataset, BUPT Depth, to evaluate the depth accuracy and consistency across various indoor and outdoor scenes. Trained on a consumer-grade GPU using just 150K RGB-D pairs, SM4Depth achieves outstanding performance on the most never-before-seen datasets, especially maintaining consistent accuracy across indoors and outdoors. The code can be found here.

Due to the limitations of sensor, traditional cameras struggle to capture details within extremely dark areas of videos. The absence of such details can significantly impact the effectiveness of low-light video enhancement. In contrast, event cameras offer a visual representation with higher dynamic range, facilitating the capture of motion information even in exceptionally dark conditions. Motivated by this advantage, we propose the Real-Event Embedded Network for low-light video enhancement. To better utilize events for enhancing extremely dark regions, we propose an Event-Image Fusion module, which can identify these dark regions and enhance them significantly. To ensure temporal stability of the video and restore details within extremely dark areas, we design unsupervised temporal consistency loss and detail contrast loss. Alongside the supervised loss, these loss functions collectively contribute to the semi-supervised training of the network on unpaired real data. Experimental results on synthetic and real data demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

Image aesthetics assessment (IAA) primarily examines image quality from a user-centric perspective and can be applied to guide various applications, including image capture, recommendation, and enhancement. The fundamental issue in IAA revolves around the quantification of image aesthetics. Existing methodologies rely on assigning a scalar (or a distribution) to represent aesthetic value based on conventional practices, which confines this scalar within a specific range and artificially labels it. However, conventional methods rarely incorporate research on interpretability, particularly lacking systematic responses to the following three fundamental questions: 1) Can aesthetic qualities be quantified? 2) What is the nature of quantifying aesthetics? 3) How can aesthetics be accurately quantified? In this paper, we present a law called "Special Relativity" of IAA (SR-IAA) that addresses the aforementioned core questions. We have developed a Multi-Attribute IAA Framework (MAINet), which serves as a preliminary validation for SR-IAA within the existing datasets and achieves state-of-the-art (SOTA) performance. Specifically, our metrics on multi-attribute assessment outperform the second-best performance by 8.06% (AADB), 1.67% (PARA), and 2.44% (SPAQ) in terms of SRCC. We anticipate that our research will offer innovative theoretical guidance to the IAA research community. All resources are available here.

Due to the poor indoor coverage and positioning accuracy, existing indoor GPS positioning systems leverages additional RF infrastructure as relay with known position. However, in practice, learning the relay position requires establishing an additional connection between user and relays, which is user unfriendly and even infeasible. In this paper, we propose GPSWindow, a plug-and-play indoor GPS positioning system without the prior knowledge of the relay position. By attaching optically transparent metasurfaces to windows, GPSWindow focuses the incident signal towards determined direction and provide an indoor continuous GPS signal coverage. We exploit the difference between consecutive satellite measurements and the Doppler shift measurements to recover the satellite-to-user true distance from the measured satellite-metasurface-user distance, and then locate the user using the traditional trilateration positioning method, eliminating the requirement of relay position. We also design an error correction method that leverages IMU on smartphone and the Doppler shift information to enhance GPSWindow in mobile scenarios. Extensive real-world experiments demonstrate that GPSWindow can provide continuous position service and achieve median positioning accuracy of 3.6m in indoor environments.

Despite the prevalence of Crowdsourced Live Video Streaming (CLVS), video viewers still suffer from low QoE particularly under rush hours, as the existing Content Delivery Network (CDN) is not scalable enough to handle the massive concurrent streaming. The rapid emergence of Web 3.0 provides new incentives for revisiting and applying the classical P2P networking in CLVS. However, the highly dynamic joining or leaving behavior of CLVS viewers frequently interrupts the real-time streaming and leads to low QoE, which demands to retrofit P2P. In this work, we bridge the gap by proposing a reliable P2P-assisted CLVS system named Venus, where viewers can share their streaming content smoothly, without video freeze regardless of viewers leaving. To realize Venus, different from the single-flow sharing in previous P2P video streaming, we design a novel multiflow framework with lightweight redundancy encoding, so as to handle the inherently high viewer dynamics. Correspondingly, we introduce a multiflow scheduler to enable QoE adaption concertedly over heterogeneous multiple flows. Real-world evaluation confirms the benefits of decentralized CLVS streaming, with Venus outperforming the state-of-the-art CDN solution by almost totally eliminating the video stall while enhancing the video quality by 10.2%.

The Bluetooth 5.1 specification introduces the Angle of Arrival feature, which significantly enhances its applications in indoor positioning. Given the height of the target, a single BLE (Bluetooth Low Energy) base station can locate the target, thereby reducing deployment costs. However, existing methods often assume that the target's height is known in advance and remains constant, which is not always true in practice. The height can vary significantly due to user posture changes, such as when picking up a phone from a pocket, leading to positioning errors. In this paper, we propose BleHe, a novel indoor positioning system that incorporates height correction to enhance positioning accuracy in scenarios with dynamic height changes. BleHe detects height changes by utilizing on-device IMU sensor and subsequently notifies the base station of any detected height change events. To avoid altering the commodity Bluetooth protocols, rather than directly modifying application data, we create a side channel to delivery the height change information from the device to the base station by adjusting the BLE packet transmission frequency. This method allows the base station to infer the start and end times of height changes and subsequently refine the target's trajectory using a height-aware particle filtering-based positioning correction method that we propose. Experimental results demonstrate that BleHe achieves an average positioning error of 34.4cm, even in scenarios involving posture changes during walking.

In modern network environments, the Multipath QUIC (MPQUIC) protocol significantly enhances data transmission reliability and efficiency by leveraging multiple paths. However, the challenge lies in developing an effective scheduling algorithm that can adapt to dynamic network conditions. Existing heuristic scheduling algorithms are tailored to specific environments, while learning-based algorithms lack the capability for fine-grained scheduling. To address this, we propose SSRL (RL-based Scheduler Switcher), a framework that dynamically switches among heuristic scheduling algorithms based on real-time network condition recognition. SSRL combines the advantages of both heuristic and learning-based algorithms, thereby enhancing MPQUIC's performance by reducing latency and improving bandwidth utilization while consuming less reorder buffer. We also design a scheduler selection model that leverages LSTM and Double DQN, enabling SSRL to understand network conditions better and make more effective scheduling decisions. We implement SSRL using Pytorch and conduct extensive evaluations with the NS3 network emulator. The results show that SSRL increases throughput by 23% and reduces RTT by 10%.

Network feedback, representing the impact of congestion control actions on the network, is crucial for evaluating the advantages of the actions taken. Previous reinforcement learning (RL)-based congestion control algorithms use average performances over fixed periods to measure network feedback, which fails to accurately capture the impact of an action and leads to performance degradation. In this paper, we propose Zygos, which accurately estimates network feedback. This accurate feedback can benefit other RL-based congestion control algorithms. Zygos contains a distribution-based reward correction mechanism that leverages a RL model to generate relevance distributions for the sequence rewards of each state-action pair, and then aggregates the rewards by weighted average. Zygos also adopts metagradient RL to capture network feedback offset patterns, thereby updating the relevance generation model during the training of the congestion control algorithm. Experiments show that the RL congestion control method using Zygos achieves an average 20–30% improvement in throughput and 20% decrease in delay compared to the original method, highlighting substantial enhancements in RL-based congestion control algorithms.

In this paper, we reveal the existence of partially overlapping interference (POI) when multiple devices concurrently transmit in partially overlapping channels. Existing methods proposed for collisions in the same channel cannot achieve online decoding for target packets under POI due to unpredictable in-window distribution of interfering chirp. We instead propose PrLoRa, a novel method to achieve online LoRa decoding under POI. PrLoRa relies on the insight that only the target chirp is complete in the decoding window. Then PrLoRa adopts a novel operation named phase rotation which converts the difference in chirp's integrity to the amplitude change of energy peak after dechirp and Fast Fourier Transform (FFT). For energy peaks generated by the target chirps, their amplitude change is expected. To use phase rotation in decoding the target chirp, we first establish the theoretical model between amplitude changing and phase rotation, which can be used to infer the expected amplitude change of the target peak. In practice, the peak's amplitude suffers from the influence of channel noise, which causes decoding errors. So, we also propose a noise-aware window setting that can adaptively select the suitable window size for phase rotation according to channel noise. Furthermore, we propose the iterative phase rotation to cope with decoding errors caused by interfering chirps with confusing distributions. The Experimental results show that PrLoRa can reduce the SER by up to 0.92 compared with existing state-of-the-art methods.

The past decade has witnessed an increasing demand for enhancing image quality through exposure, and as a crucial prerequisite in this endeavor, Image Exposure Assessment (IEA) is now being accorded serious attention. However, IEA encounters two persistent challenges that remain unresolved over the long term: the accuracy and generalizability of No-reference IEA are inadequate for practical applications; the scope of IEA is confined to qualitative and quantitative analysis of the entire image or subimage, such as providing only a score to evaluate the exposure level, thereby lacking intuitive and precise fine-grained evaluation for complex exposure conditions. The objective of this paper is to address the persistent bottleneck challenges from three perspectives: model, dataset, and benchmark. 1) Model-level: we propose a Pixel-level IEA Network (P-IEANet) that utilizes Haar discrete wavelet transform (DWT) to analyze, decompose, and assess exposure from both lightness and structural perspectives, capable of generating pixel-level assessment results under no-reference scenarios. 2) Dataset-level: we elaborately build an exposure-oriented dataset, IEA40K, containing 40K images, covering 17 typical lighting scenarios, 27 devices, and 50+ scenes, with each image densely annotated by more than 10 experts with pixel-level labels. 3) Benchmark-level: we develop a comprehensive benchmark of 19 methods based on IEA40K. Our P-IEANet not only achieves state-of-the-art (SOTA) performance on all metrics but also seamlessly integrates with existing exposure correction and lighting enhancement methods. To our knowledge, this is the first work that explicitly emphasizes assessing complex image exposure problems at a pixel level, providing a significant boost to the IEA and exposure-related community. The code and dataset are available in \href{https://github.com/mRobotit/Pixel-level-No-reference-Image-Exposure-Assessment}{\textcolor{red} {here}}.

Cuff-less methods, especially pulse wave analysis (PWA) techniques with PPG/mmWave sensing, have shown great potential for non-intrusive blood pressure (BP) monitoring. However, the state-of-the-art solutions are only validated on small-scale healthy subjects, neglecting patients with abnormal BP and thus a more urgent need for BP monitoring. To bridge the gap, we first build the largest mmWave-BP dataset to our knowledge, including 930 real patients with cardiovascular diseases, and perform extensive experiments, which reveals that all existing PWA methods exhibit far less satisfactory performance with standard deviation errors (STD) exceeding 16 mmHg for systolic BP (SBP) and 11mmHg for diastolic BP (DBP). An in-depth investigation shows that physiological factors have complex effect on vascular elasticity and structure, thus people with very different BP values may exhibit extremely similar pulse waveform, which leads to confusion in model learning. In this work, we propose BP3, which fuses physiological factors into sensing-data-driven deep-learning framework, so as to capture the intricate effect of physiological factors during the whole process of learning pulse waveforms. Evaluation results show that BP3 achieves the mean errors of-1.57 mmHg and -0.34 mmHg, STD of 9.77 mmHg and 7.93 mmHg for SBP and DBP, respectively. Moreover importantly, BP3 shows remarkable gain particularly for subjects with abnormal BP, achieving mean errors that are only 0.48% ~ 20.86% of the state-of-the-art solutions.

The increasingly prevalent livecast streaming causes expensive bandwidth costs and delivery capacity challenges for the content delivery network (CDN) service. As an emerging paradigm, edge computing offers new opportunities to address this issue. The existing works are limited to the data volume pricing model. In contrast, we focus on the 95th-percentile pricing model, which is adopted by many large-scale livecast systems. We propose a Cost-aware Edge-assisted Livecast system (CEL) to minimize the bandwidth cost, consisting of two components: 1) the Shapley values are leveraged to model the actual bandwidth costs for the CDN and edge servers in different time slots, together with acceleration technologies for fast Shapley value estimation and 2) a greedy request scheduling algorithm with theoretical guarantees is proposed to solve the online request scheduling problem, which is NP-hard. Based on real-world data from an operational livecast system, our experiments demonstrate thatCELis time-efficient and achieves at least 14.81% bandwidth cost savings compared with four state-of-the-art methods.

The issue of efficient scheduling and deployment of urban public resources has become increasingly important with the development of technological innovations and the mobility of societies. The arbitrary usage behavior of users causes the unbalanced distribution of resources and makes it difficult for users to get adequate resources in some places but redundant resources in others. Therefore, designing an efficient scheduling policy for public resources becomes crucial to promoting resource utilization and customer satisfaction. In this article, we propose a novel scheduling system for public resources that aligns with the actual value-driven scheduling strategy and take the bike-sharing system as an example. Then, we design a deep reinforcement learning algorithm named two action layer proximal policy optimization (TALPPO) to generate an effective sharing-bike scheduling strategy under realistic constraints, which could help enterprises to make better management and operation decisions. Finally, we compare the proposed algorithm with the other ten baseline models and provide extensive experimental results on two data sets called Mobike (dockless) and Citi Bike (docked) to evaluate the performance of our proposed approach.

Image-to-image (I2I) translation often requires establishing cycle consistency between the source and the translated images across different domains. However, cycle consistency requires redundant reconstruction, and is too restrictive to satisfy the bijection assumption between the two domains. In this paper, we propose SwinIT, a hierarchical Swin-transformer I2I Translation framework without using cycle consistency. Specifically, we carefully design symmetrical encoders for content and style flows, then explore newly proposed adaptive denormalization and normalization strategies. This framework can effectively capture and fuse content and style representations in a coarse-to-fine manner, ensuring our method achieves high performance without cycle consistency. Guided by element-wise feature adaptive denormalization, our model focuses on preserving semantic structure information. Due to the semantic mismatch between unpaired source and exemplar images, we introduce cross-attention adaptive instance normalization to help achieve better alignment. However, because the original optimization objective lacks direct supervision to preserve high-frequency information, rich edge details are lost during the translation. We propose a wavelet transformation matching loss to recover the details by converting the image into multi-frequency parts. We validate our proposed method in various I2I translation tasks, including arbitrary style transfer, multi-modal image synthesis, and semantic image synthesis, demonstrating its effectiveness in both qualitative and quantitative evaluations.

Obtaining urban-scale vehicle trajectories is essential to understand urban mobility and benefits various downstream applications. The mobility knowledge obtained from existing vehicle trajectory sensing techniques is typically incomplete. To fill the gap, we propose$F^{3}VeTrac$, an efficient deep-learning-based vehicle trajectory recovery system that utilizes complementary characteristics of the Camera Surveillance System and the Vehicle Tracking System to obtain fine-grained, fully-road-covered, and fully-individual-penetrative ($F^{3}$) trajectories.$F^{3}VeTrac$utilizes five well-designed modules to model the co-occurrence relationships hidden in both coarse-grained and fine-grained trajectories from the two complementary sensing systems and fuse them to recover the coarse-grained trajectories. We implement and evaluate$F^{3}VeTrac$with two real-world datasets from over 100 million regular vehicle trajectories and 16 million commercial vehicle trajectories in two cities of China, together with an on-field case study based on 251 regular vehicle trajectories collected by 17 volunteers, demonstrating its great advantages over six state-of-the-art alternative schemes. Moreover, we present a downstream application of$F^{3}VeTrac$for traffic condition estimation, which obtains obvious performance gains.

Millimeter wave radar is gaining traction recently for enabling privacy-preserving human sensing. However, the lack of large-scale, dynamic radar datasets impedes progress in developing robust and generalized deep learning models for mobile sensing applications. To address this problem, we resort to designing a software pipeline that leverages wealthy dynamic videos to generate synthetic radar data, but it faces two key challenges including i) incorrect camera and human positions leading to erroneous superposition of signal intensity and ii) the signal reflection of the background and humans in mobile scenes. To this end, we designMidas++to utilize rich videos to generate realistic radar data via two components: (i) ahuman mesh fitting and calibrationcomponent calculates the camera ego-motion parameters to calibrate the extracted human positions; (ii) areflection and noise signal estimationcomponent combines several key modules,depth prediction,reflection model, andspatiotemporal noise estimation, to output coarse radar data, followed by aU-Netmodel to generate realistic radar data. We implement and evaluateMidas++with video data from public data sources and real-world radar data, demonstrating thatMidas++outperforms other state-of-the-art approaches for both activity recognition and object detection tasks.

The diversified mobility on demand (MoD) systems integrate both traditional fuel vehicles and green transportation tools (e.g. shared bicycles and shared e-bikes), which can not only reduce the fleet size of traditional fuel vehicles but also address the demand for short-distance travel and alleviate environmental pollution. However, despite having a variety of travel tools, the existing MoD systems neglect the guidance on passengers according to their preferences and travel characteristics and thus lead to the failure of effective cooperation among multiple travel modes and additional waste of resources. This inspired us to design a novel order allocation mechanism for diversified MoD systems. Specifically, we construct a heterogeneous order graph based on the order sets, transform the minimum fleet problem into the minimum trajectory coverage problem on the heterogeneous order graph and propose a learning-based order allocation method LAMD$^{2}$containing three modules. i) The online breadth-first order search framework fully considers the characteristics of different travel modes and the interaction of multiple vehicles, and then leverages the competitive mechanism to well handle the heterogeneity of travel modes and improve the overall efficiency. ii) The multi-semantic travel mode selection module analyzes users' preferences for diversified travel modes based on multi-semantic historical travel data and then determines the service mode based on the similarity of order spatiotemporal characteristics. iii) The Reinforcement Learning (RL)-based order evaluation module evaluates the long-term benefits of expanding existing For-Hire Vehicle (FHV) trajectories with different orders and updates the behavioral strategies through interactive feedback with the environment. We implement and evaluate the proposed method with a real-world trajectory dataset, demonstrating that LAMD$^{2}$outperforms all the baselines and reduces the fleet size and energy consumption by the average of 2.93% and 8.01%, respectively, compared to the real-world systems.

Green Light Optimized Speed Advisory (GLOSA) systems have emerged to allow drivers to pass traffic lights during a green interval. However, various adaptive and intelligent traffic light control approaches have been adopted in many cities, resulting in the development of current GLOSA technologies lagging behind that of traffic light technologies. When taking diverse dynamic traffic lights into account, it is difficult to model the interactions between vehicles and traffic lights, which is further exacerbated by the hybrid control strategies of traffic lights. To this end, we design a new GLOSA systemSpeedAdvto provide optimal speed advisory for addressing diverse traffic lights. We formulate the problem as a Multi-Agent Markov Decision Process (MAMDP) with an implicit common goal and propose a heterogeneous-agent collaborative framework based on reinforcement learning. Three main modules are used in the system: i) a spatio-temporal relation reasoning module based on the phase-aware attention mechanism pays more attention to the traffic rules and traffic flow diversion of adjacent intersections to predict traffic conditions for a few seconds later; ii) a behavior approximating module based on imitation learning is introduced to approximate the phases of diverse traffic lights; iii) a speed advisory module provides the optimal speed advisory based on policy gradient reinforcement learning relying on the above two modules and other information collected by vehicles. We implement and evaluateSpeedAdvwith a real-world trajectory dataset, together with a field test based on a prototype system, demonstrating thatSpeedAdvimproves the overall performance by at least 24.1% in terms of travel time, energy consumption, safety, and comfort compared to the state-of-the-artGreenDrivemethod.

Wireless sensing offers a promising approach to identify the content of liquids without opening the container or directly touching the liquid. Although existing methods aim to achieve fine-grained identification, i.e., distinguishing a 1% v/v difference in alcohol content, they still have limitations in detecting highly deceptive counterfeit liquors that have much smaller content differences, sometimes as low as 0.2% v/v alcohol content. In this paper, we propose mm Taster, a mobile system that combines the mmWave radar with a smartphone to perform fine-grained and robust alcohol sensing. To achieve the desired fine granularity, we introduce a novel feature extraction model that exploits theunique reflection responses across multiple mmWave frequencies, which provide discriminative information about liquid content. Furthermore, we observe the serious interference of target displacement on identification performance, which hinders the various applications in mobile scenarios. To enhance the robustness, mm Taster incorporates a customizedtranslation-invariantneural network,ConvNet, to remove the location interference and extract stable liquid-dependent features regardless of target displacement. Extensive experimental results demonstrate that mm Taster can accurately distinguish the alcohol differences as low as0.2% v/vwith an accuracy of over90.8%even in scenarios involving diverse displacements and rotations.

Volumetric videos offer viewers more immersive experiences, enabling a variety of applications. However, state-of-the-art streaming systems still need hundreds of Mbps bandwidth to transmit volumetric videos, exceeding the common bandwidth capabilities of mobile devices. We find a research gap in reusing inter-frame redundant information to reduce bandwidth consumption, while the existing inter-frame compression methods rely on the so-calledexplicit correlation, i.e., the redundancy from the same/adjacent locations in the previous frame, which does not apply to highly dynamic frames or dynamic viewports. This paper introduces a new concept calledimplicit correlation, i.e., the consistency of topological structures, which stably exists in dynamic frames and is beneficial for reducing bandwidth consumption. We design a mobile volumetric video streaming system Hermes consisting of an implicit correlation encoder to reduce bandwidth consumption and a hybrid streaming method that adapts to dynamic viewports. Experiments on public datasets show that Hermes achieves a frame rate of 30+ FPS over daily networks and on commodity smartphones, with at least 3.64× and 3.34× improvement compared with two state-of-the-art baselines, respectively.

Deploying deep learning models to edge devices for low-latency and privacy-preserving applications has become a trend. To adapt to heterogeneous devices and data, it is significant to generate customized models. However, existing model adaptation approaches require edge devices to make interactions (collecting hardware information or local data) with the cloud, which raises privacy concerns, increases communication costs, and burdens the cloud. By contrast, we proposeCamoNet, a universal on-device model adaptation framework with zero interaction between devices and the cloud. InCamoNet, a lightweight on-device neural architecture search module is utilized to quickly generate a customized model for subsequent on-device training, followed by an on-device contrastive transfer learning module to effectively leverage unlabeled data for fine-tuning the customized model. Extensive experimental results show thatCamoNetcan effectively run on various edge devices. Compared with the SOTA model adaptation approaches,CamoNetachieves significant accuracy improvement by 25.2% on average for image classification, 10.1% on average for object detection, and reduces the training memory by 4.8-11.4×. We will open-source our models and tools for edge AI developers.

Existing WiFi sensing systems transmit dedicated high-rate packets for accurate sensing. These “sensing packets” greatly affect the main data communication function of WiFi and significantly counteract the promised benefit of reusing WiFi communication for sensing. In this work, we propose WiImg2.0, a lightweight system which involves machine learning techniques to enable WiFi sensing under low packet rate, pushing WiFi sensing one step towards real-life adoption. The key idea is to convert the WiFi CSI samples into images and employ the Generative Adversarial Network (GAN) for CSI image inpainting, relaxing the requirement of high sample rate for sensing. We first recover the sensing data from the antenna spatial domain and then from the sample time domain. To avoid the large training overhead of GAN, we design a lightweight GAN that leverages samples of only three rates in a fixed window to recover the CSI traces of arbitrary rates and varying duration. Experiments show that with just 25 packets per second, WiImg2.0 is able to increase the recognition accuracy for hand gesture recognition and daily activity tracking from the state-of-the-art 59.1% and 65.9% to 86.7% and 96.4%, respectively.

With the rapid advancements in deep learning technologies, person re-identification (ReID) has witnessed remarkable performance improvements. However, the majority of prior works have traditionally focused on solving the problem via extracting features solely from a single perspective, such as uniform partitioning, attention mechanisms, or semantic masks. While these approaches have demonstrated efficacy within specific contexts, they fall short in diverse situations. In this paper, we propose a novel approach, Mutual Distillation Learning For Person Re-identification (termed as MDPR), which addresses the challenging problem from multiple perspectives within a single unified model, leveraging the power of mutual distillation to enhance the feature representations collectively. Specifically, our approach encompasses two branches: a hard content branch to extract local features via a uniform horizontal partitioning strategy and a soft content branch to dynamically distinguish between foreground and background and facilitate the extraction of multi-granularity features via a carefully designed attention mechanism. To facilitate knowledge exchange between these two branches, a mutual distillation and fusion process is employed, promoting the capability of the outputs of each branch. Extensive experiments are conducted on widely used person ReID datasets to validate the effectiveness and superiority of our approach. Notably, our method achieves an impressive 88.7%/94.4% in mAP/Rank-1 on the DukeMTMC-reID dataset, surpassing the current state-of-the-art results. Our source code is available athttps://github.com/KuilongCui/MDPR.

The successful application of semantic segmentation technology in the real world has been among the most exciting achievements in the computer vision community over the past decade. Although the long-tailed phenomenon has been investigated in many fields,e.g., classification and object detection, it has not received enough attention in semantic segmentation and has become a nonnegligible obstacle to applying semantic segmentation technology in autonomous driving and virtual reality. Therefore, in this work, we focus on a relatively underexplored task setting,long-tailed semantic segmentation(LTSS). We first establish three representative datasets from different aspects, i.e., scene, object, and human. We further propose a dual-metric evaluation system and construct the LTSS benchmark to demonstrate the performance of semantic segmentation methods and long-tailed solutions. We also propose a transformer-based algorithm to improve LTSS,frequency-based matcher, which solves the oversuppression problem by one-to-many matching and automatically determines the number of matching queries for each class. Given the comprehensiveness of this work and the importance of the issues revealed, this work aims to promote the empirical study of semantic segmentation tasks. Our datasets, codes, and models will be publicly available.

Fine-grained image classification (FGIC) aims to separate different subcategories from one general superclass, which requires the classification model to extract distinctive representations from subtle yet discriminative regions of the objects. Learning multiple part representations can give a detailed description of the object from different perspectives, boosting the classification performance. However, it still remains a challenging problem to effectively locate diverse parts and extract their features without the assistance of part annotations. In this article, we present a novel method to achieve accurate fine-grained image classification by learning a set of diverse and discriminative part representations without requiring additional supervision. Firstly, our method utilizes a simple attention interaction module to lead learned spatial attentions to focus on different parts, resulting in mutually exclusive part representations. Then, to reduce the impairment of channel coupling among part representations, a part-wise channel weighting module is designed to adjust the amplitudes of different representations adaptively, making them to be diverse along the channel dimension. Moreover, to ensure comprehensive and sufficient part representations, our method introduces multi-granularity feature learning. It enables the extraction of part representations from different semantic and content levels, capturing fine-grained details effectively. To evaluate our method, extensive experiments are conducted on various benchmark fine-grained image datasets, and the results show that our method can achieve outstanding performance for FGIC, demonstrating its effectiveness.

The objective of multi-domain image-to-image translation is to learn the mapping from a source domain to a target domain in multiple image domains while preserving the content representation of the source domain. Despite the importance and recent efforts, most previous studies disregard the large style discrepancy between images and instances in various domains, or fail to capture instance details and boundaries properly, resulting in poor translation results for rich scenes. To address these problems, we present an effective architecture for multi-domain image-to-image translation that only requires one generator. Specifically, we provide detailed procedures for capturing the features of instances throughout the learning process, as well as learning the relationship between the style of the global image and that of a local instance in the image by enforcing the cross-granularity consistency. In order to capture local details within the content space, we employ a dual contrastive learning strategy that operates at both the instance and patch levels. Extensive studies on different multi-domain image-to-image translation datasets reveal that our proposed method outperforms state-of-the-art approaches.

The business growth of live streaming causes expensive bandwidth costs from the Content Delivery Network service. It necessitates traffic adaptation, i.e., adapting video bitrates for cost-efficient bandwidth utilization, especially under the 95$^{\rm \textit {th}}$percentile pricing. However, our data-driven investigations indicate the existing methods are hard to achieve bitrate-cost balance in a long month-level billing cycle due to dynamic traffic patterns. We propose TrafAda, a learning-based cost-aware traffic adaptation method consisting of i) an ultra-long-term bandwidth demand forecasting model to learn complex bandwidth usage patterns, and ii) an imitation learning-based bitrate decision mechanism to optimize the ultra-long-term objective. We have implemented and deployed TrafAda on a large-scale live streaming system in China serving over one billion viewers from 388 cities. The results show that TrafAda improves peak-hour bitrate, quality of experience (QoE), and watching time by 34.75%, 44.56%, and 10.68%, respectively, without extra bandwidth cost, which can be converted to a considerable value for a commercial system.

The rising crowdsourced live video streaming (CLVS) poses great challenges to Internet transport scalability, where a broadcaster’s live video is expected to reach thousands and even millions of viewers in real time. To accommodate such huge concurrent video traffic, the de-facto solution is to employ content delivery network (CDN), which distributes the traffic spatially relative to end viewers, using geographically distributed servers. However, our measurement study over a top operational CLVS platform reveals that CDN is not scalable enough, i.e., it loses efficacy, particularly duringbusy timeand leads to tremendous QoE degradation, e.g., 33.3% video bitrate reduction, in comparison to networkidle time. In this work, we propose Spider, which revives the peer-to-peer (P2P) networking principle to extend the scalability of CLVS system. Beyond traditional P2P for elastic data transmission, Spider retrofits P2P to meet the stringent low-latency requirements of CLVS: proposing a “pair-push” streaming mode to tame the excessive signaling latency; designing a QoE-driven peer pairing algorithm to tackle the Internet path variation and CLVS viewer dynamics. We implement, deploy and evaluate Spider in real-world over 20.9 thousand video sessions. Compared to the de-facto CDN solution, Spider achieves remarkable gains, e.g., video stall rate reductions of 52.57%, video quality gains of 8.22%, and even 66% CDN bandwidth saving. The results validate the feasibility and practicability of embracing P2P for low-latency live video communication for the first time.

Maximizing the quality of experience (QoE) for live video streaming is a long-standing challenge. Traditional video transport protocols, represented by a few deterministic rules, can hardly adapt to the highly heterogeneous and dynamic modern Internet environments. Emerging learning-based algorithms have demonstrated the potential to meet above challenge. However, our measurement study reveals an alarming long-tail performance issue: these learning-based algorithms tend to be bottlenecked by occasional catastrophic events due to the built-in exploration mechanisms. In this work, we propose Loki-plus, which improves the robustness of learning-based model by coherently integrating it with a rule-based algorithm. To enable integration at deep feature level, we first reverse-engineer the rule-based algorithm into an equivalent “black-box” neural network, and then devise a transformer-based continual learning model with effective historical feature reservation. Then, we design a network feature-aware fusion mechanism to fuse the two models in a deep manner. We train Loki-plus in a full-fledged live video system through online learning, and evaluate it over massive video sessions, in comparison to state-of-the-art rule-based and learning-based solutions. The results show that Loki-plus improves not only the average but also the tail performance substantially.

Recent work proposes Cross-Technology Communication (CTC) from IEEE 802.11b to LoRa but has a low efficiency due to the extremely asymmetric data rates. In this paper, we propose that emulates LoRa waveform with IEEE 802.11ax. By taking advantage of the OFDMA in 802.11ax, uses only a small Resource Unit (RU) to emulate LoRa chirps and sets other RUs free for high-rate WiFi users. carefully selects the RU and adopts WiFi frame aggregation to emulate the long LoRa frame. We propose a subframe header mapping method to identify and remove invalid symbols caused by irremovable subframe headers in the aggregated frame. We also propose a mode flipping method to solve Cyclic Prefix (CP) errors, based on our finding that different CP modes have different impacts on the LoRa symbol. To cope with channel dynamics, we design an adaptation mechanism to maximize the goodput with a satisfying SER. We further extend to one-to-many transmission scenario by concurrently emulating LoRa chirps in different RUs. We implement a prototype of on the USRP platform and commodity LoRa device. Experiments demonstrate can efficiently transmit complete LoRa frames with the throughput of 40.037kbps and the SER lower than 0.1.

De-authentication attack is one of the major threats to Unmanned Aerial Vehicle (UAV) communication, in which the attacker continuously sends de-authentication frames to disconnect the UAV communication link. Existing defense methods are based on authentication by digital passwords or physical channel features. But they suffer from replay attacks or cannot adapt to the UAV mobility. In this paper, instead of enhancing the in-channel authentication, we leverage the ambient broadcasting signal to establish a low-cost additional channel for authentication. Different from methods using another dedicated secure communication channel to perform an independent authentication, we use the ambient FM radio broadcasting channel and couple the two channels by encoding parasitic bits on the host signals of the broadcasting channel, which is called parasitic coding. To further enhance the security, we propose the FM-based Parasitic Coding Authentication (FPCA) that leverages elaborate host signal processing and vector coding to ensure that the attacker cannot decode our authentication even knowing the FM receiving frequency. We implement FPCA on the embedded UAV platform. The extensive experiments show that FPCA can resist replay attacks and brute force searching, achieving reliable continuous authentication for UAVs.

Personalized federated learning (PFL) is a framework that targets individual models for optimization, providing better privacy and flexibility for clients. However, in challenging intelligent sensing applications, the heterogeneous client’s data distributions make the aggregation of local models in the server unstable or even hard to converge. To deal with the performance degradation caused by the preceding problem, existing PFL methods focus more on how to fine-tune the global model but ignore the impact of the global model fusion algorithm on the results. In this article, we propose a new explainable neural-aware decoupling fusion based PFL framework, p-FedADF , to address the preceding challenges. It contains two carefully designed modules. The local decoupling module, deployed on the client, utilizes the architecture disentangle technique to decouple the feature extractors in the client’s local model into sub-network according to data categories. It obtains the inference process of feature extraction for different categories of data by training. The global aggregation module, deployed on the server, aligns the sub-network positions for multiple clients and implements a fine-grained generic feature extractor aggregation. In addition, we provide a mask encoding scheme to reduce the communication overhead of transmitting the sub-network sets between the server and clients. Our p-FedADF obtains 1.6%, 0.2%, 2.3%, and 4.5% improvement on a real-world dataset and three benchmark datasets, compared to state-of-the-art methods.

With the widespread adoption of deep learning models in wireless sensing, substantial efforts have been made to develop sophisticated models that improve the accuracy and performance of sensing applications. However, the exploration of potential vulnerabilities in deep learning models has been limited, with existing studies primarily focusing on evaluating wireless adversarial performance in communication or sensing alone. Moreover, there is a lack of a comprehensive definition for attack imperceptibility. In this article, we come up with a definition of the wireless attack imperceptibility for both communication and sensing. Our objective is to create an adversarial perturbation capable of degrading WiFi sensing performance while preserving WiFi communication integrity. To achieve this, we propose WiCAM2.0 to reveal the temporal and spatial attention of a deep neural network, capturing the crucial portions of its input. Then, we design a mask to confine adversarial perturbations in the attended parts only, minimizing the impact on WiFi communication. WiCAM2.0 is a general adversarial framework that integrates adversarial methods such as the Fast Gradient Sign Method and Projected Gradient Descent to generate perturbations, capable of initiating both non-targeted and targeted attacks. We carry out experiments on three popular WiFi sensing applications, including human activity recognition, gesture recognition, and user identification. Extensive experiments are conducted on both public datasets and self-collected datasets.

LoRa is widely deploying in aquatic environments to support various Internet of Things applications. However, floating LoRa networks suffer from serious performance degradation due to the polarization loss caused by the swaying antenna. Existing methods that only control the transmission starting from the aligned attitude have limited improvement due to the ignorance of aligned period length. In this article, we propose PolarScheduler , a dynamic transmission control method for floating LoRa networks. PolarScheduler actively controls transmission configurations to match polarization aligned periods. We propose a V-zone model to capture diverse aligned periods under different configurations. We also design a low-cost model establishment method and an efficient optimal configuration searching algorithm to make full use of aligned periods. To deal with packet collisions in a multiple-node environment, we further propose an Attitude-aware Slot-allocation MAC protocol, which avoids both packet collisions and polarization loss. We implement PolarScheduler on commercial LoRa platforms and evaluate its performance in a deployed network. Extensive experiments show that PolarScheduler can improve the packet delivery rate and throughput by up to 20.0% and 15.7%, compared to the state-of-the-art method.

Images captured in low-light conditions often suffer from significant quality degradation. Recent works have built a large variety of deep Retinex-based networks to enhance low-light images. The Retinex-based methods require decomposing the image into reflectance and illumination components, which is a highly ill-posed problem and there is no available ground truth. Previous works addressed this problem by imposing some additional priors or regularizers. However, finding an effective prior or regularizer that can be applied in various scenes is challenging, and the performance of the model suffers from too many additional constraints. We propose a contrastive learning method and a self-knowledge distillation method for Retinex decomposition that allow training our Retinex-based model without elaborate hand-crafted regularization functions. Rather than estimating reflectance and illuminance images and representing the final images as their element-wise products as in previous works, our regularizer-free Retinex decomposition and synthesis network (RFR) extracts reflectance and illuminance features and synthesizes them end-to-end. In addition, we propose a loss function for contrastive learning and a progressive learning strategy for self-knowledge distillation. Extensive experimental results demonstrate that our proposed methods can achieve superior performance compared with state-of-the-art approaches.

Low-light video enhancement is a challenging task with broad applications. However, current research in this area is limited by the lack of high-quality benchmark datasets. To address this issue, we design a camera system and collect a high-quality low-light video dataset with multiple exposures and cameras. Our dataset provides dynamic video pairs with pronounced camera motion and strict spatial alignment. To achieve general low-light video enhancement, we also propose a novel Retinex-based method named Light Adjustable Network (LAN). LAN iteratively refines the illumination and adaptively adjusts it under varying lighting conditions, leading to visually appealing results even in diverse real-world scenarios. The extensive experiments demonstrate the superiority of our low-light video dataset and enhancement method. Our dataset is available at https://github.com/ciki000/DID.

We present a comprehensive study on a new task named image color aesthetics assessment (ICAA), which aims to assess color aesthetics based on human perception. ICAA is important for various applications such as imaging measurement and image analysis. However, due to the highly diverse aesthetic preferences and numerous color combinations, ICAA presents more challenges than conventional image quality assessment tasks. To advance ICAA research, 1) we propose a baseline model called the Delegate Transformer, which not only deploys deformable transformers to adaptively allocate interest points, but also learns human color space segmentation behavior by the dedicated module. 2) We elaborately build a color-oriented dataset, ICAA17K, containing 17K images, covering 30 popular color combinations, 80 devices and 50 scenes, with each image densely annotated by more than 1,500 people. Moreover, we develop a large-scale benchmark of 15 methods, the most comprehensive one thus far based on two datasets, SPAQ and ICAA17K. Our work, not only achieves state-of-the-art performance, but more importantly offers the community a roadmap to explore solutions for ICAA. Code and dataset are available in here.

Predicting attention regions of interest is an important yet challenging task for self-driving systems. Existing methodologies rely on large-scale labeled traffic datasets that are labor-intensive to obtain. Besides, the huge domain gap between natural scenes and traffic scenes in current datasets also limits the potential for model training. To address these challenges, we are the first to introduce an unsupervised way to predict self-driving attention by uncertainty modeling and driving knowledge integration. Our approach’s Uncertainty Mining Branch (UMB) discovers commonalities and differences from multiple generated pseudo-labels achieved from models pre-trained on natural scenes by actively measuring the uncertainty. Meanwhile, our Knowledge Embedding Block (KEB) bridges the domain gap by incorporating driving knowledge to adaptively refine the generated pseudo-labels. Quantitative and qualitative results with equivalent or even more impressive performance compared to fully-supervised state-of-the-art approaches across all three public datasets demonstrate the effectiveness of the proposed method and the potential of this direction. The code is available at https://github.com/zaplm/DriverAttention.

Crowd logistics, as an emerging delivery paradigm, provides a cost-efficient way of leveraging crowdsourcing couriers to help express enterprises to match the surging delivery demands that are hard to be addressed by regular couriers only during online mega sale days. However, it is a challenging problem how to recruit an appropriate number of crowdsourcing couriers and assign an appropriate number of parcels to them and regular couriers, as many practical issues need to be considered, such as the dynamic competitive crowdsourcing market, the turnover of crowdsourcing couriers, and unique workload patterns of regular couriers. We design a crowdsourcing-assisted express system called COME to coordinate crowdsourcing and regular couriers for minimizing the overall cost of labor payment and parcel backlog. In COME, we design an Opponent-Aware Reinforcement Learning model to learn the recruitment difficulty in a competitive crowdsourcing market to make an appropriate recruitment plan, and design a four-staged approach to make an appropriate parcel assignment plan, which can address not only the dynamic recruitment difficulty but also the dynamic number of couriers. We have implemented and deployed COME on a real-world crowdsourcing-assisted express system in China involving 1358 delivery stations over 145 cities, and extensively evaluated it with a four-year real-world dataset, demonstrating its great advantage over other alternative solutions and showing high feasibility and generality.

Long Range-Frequency Hopping Spread Spectrum (LR-FHSS) is a novel wireless communication technology to improve the coverage of Low-Power Wide-Area Network (LP-WAN). But our measurement finds that given the same set of sub-channels, different Frequency Hopping Sequence (FHS) can result in a reliability difference of up to 52.6 % in terms of Packet Reception Rate (PRR). The key observation indicates that the reliability of LR-FHSS is significantly influenced by the FHS besides the link quality. Hence, in this paper, we propose DFH that takes both link quality and FHS into consideration to improve the reliability of LR-FHSS. We first propose using the hop Signal-to-noise Ratio (SNR), a new indicator to reflect the quality of the sub-channels and establish the PRR prediction model according to hop SNR and FHS. Based on the model, we design an interleaving-based search algorithm to decide the optimal FHS. We implement and evaluate DFH on the commercial transceivers and SDR-based gateway. The results of experiments in real environments show that DFH can improve the PRR by up to 2.76x, compared to the standard LR-FHSS.

The hierarchical cooperative resource scheduling architecture provides a promising direction for efficient collaborative processing of edge computing under the dynamic and intricate landscape of the Industrial Internet of Things (IIoT). However, existing scheduling algorithms often struggle to effectively capture the intricate information features inherent in hierarchical and collaborative domains, leading to suboptimal solutions. To tackle this challenge, we introduce a novel hierarchical cooperative resource scheduling framework based on Graph Neural Networks (GNN) and Deep Reinforcement Learning (DRL). We first leverage hierarchical GNN to facilitate seamless information exchange among internal nodes and adjacent nodes between layers in the hierarchical structure and transform it into node embeddings. These meticulously designed embeddings are then input into the policy model of DRL for the iterative learning process to generate higher-quality solutions by leveraging global feature information. Experiment results unequivocally demonstrate the superiority of our approach over baselines in terms of scheduling performance. Furthermore, our model exhibits robust generalization capabilities across various scenarios.

Existing approaches to scale CPU quota in container clouds lack a consideration of the gap between the promised quota and actually obtainable amount of CPU resources. However, this gap is noticeable with Completely Fair Scheduler (CFS), which is used in Linux kernel to allocate time slices to threads. CFS shares cores among containers regardless of their quotas, causing containers with more threads to exhaust their quotas earlier due to parallel execution across multiple cores. This leaves these cores idle but unavailable for containers with no threads on them, resulting in insufficient CPU time for those containers. As a result, the execution of tasks in these containers becomes slower, potentially exceeding their deadlines and incurring penalty costs. In this paper, we propose a joint optimization approach for CPU scaling and core sharing, which uses Particle Swarm Optimization (PSO) algorithm to iteratively make decisions with the goal of minimizing penalty costs for missing deadlines. In each iteration, a core sharing scheme that aims at reducing idle CPU time will be determined based on the candidate scaling plan represented by each particle’s position. By considering the impact of following periods, we evaluate the penalty costs incurred by the interdependent scaling and core sharing decisions based on the obtainable CPU time for each container, which drives the co-optimization of both aspects of the decisions. The experimental results in a real environment show the proposed approach achieves state-of-the-art performance.

Cross-Technology Communication (CTC) is an emerging technique that enables direct interconnection among incompatible wireless technologies. However, for the downlink from WiFi to multiple IoT technologies, serially emulating and transmitting the data of each IoT technology has extremely low spectrum efficiency. Recent parallel CTC uses IEEE 802.11g to send emulated ZigBee signal and let the BLE receiver decodes its data from the emulated ZigBee signal with a dedicated codebook. It still has a low spectrum efficiency because IEEE 802.11g exclusively uses the whole channel. Besides, the codebook design hinders the reception on commodity BLE devices. In this paper, we propose WiCast, a parallel CTC that uses IEEE 802.11ax to emulate a composite signal that can be received by commodity BLE, ZigBee, and LoRa devices. By taking advantage of OFDMA in 802.11ax, WiCast uses a single Resource Unit (RU) for parallel CTC and sets other RUs free for high-rate WiFi users. But such a sophisticated composite signal is very easily distorted by emulation imperfections, dynamic channel noises, cyclic prefix, and center frequency offset. We propose a CTC link model that jointly models the emulation errors and channel distortions. Then we carve the emulated signal with elaborate compensations in both time and frequency domains to solve the above distortion problem. We implement a prototype of WiCast on the USRP platform and commodity devices. The extensive experiments demonstrate WiCast can achieve an efficient parallel transmission with the aggregated goodput up to 390.24kbps.

Gait recognition based on millimeter waves (mmWave) can recognize people's identity information by sensing their walking posture, which has found versatile usages in many fields, such as smart home, intelligent security, and health monitoring. While this technology has gained extensive attention in recent years, its possibility of being misused is also increasing. The snooper who misuses the technology could monitor the victim's identity information, which is imperceptible due to the characteristics of mmWave-based gait recognition. In this paper, we propose an identity protector called WW-IDguard, which disrupts the snooper at the physical level. The key idea is that the protector sends a unique signal to interfere with not only the signal but also the gait feature of the person “seen” by the snooper. Experiments demonstrate that WW-IDguard can significantly reduce the accuracy of the mmWave-based gait recognition used by snoopers. We also perform a measurement analysis on the basic method.

Transformers have shown great potential in various vision tasks, but none of them have surpassed the best CNN model on image aesthetics assessment (IAA) tasks. IAA is a challenging task in multimedia systems that requires attention to both foreground and background, as well as robustness to noisy and redundant labels. The global and dense attention mechanism of Transformers, designed for saliency-oriented tasks, may miss important aesthetic information in the background, increase the computational cost and slow down the convergence on IAA tasks. To address these issues, we propose an Enhancer for Aesthetics-Oriented Transformers (EAT). EAT uses a deformable, sparse and data-dependent attention mechanism that learns where to focus and how to refine attention by offsets. EAT also guides the offsets to balance the attention between foreground and background according to dedicated rules. Our EAT-enhanced Transformers outperform the previous methods on four representative datasets with fewer training epochs. Code is available in https://github.com/woshidandan/Image-Aesthetics-Assessment

Volumetric videos offer viewers more immersive experiences, enabling a variety of applications. However, state-of-the-art streaming systems still need hundreds of Mbps, exceeding the common bandwidth capabilities of mobile devices. We find a research gap in reusing inter-frame redundant information to reduce bandwidth consumption, while the existing inter-frame compression methods rely on the so-called explicit correlation, i.e., the redundancy from the same/adjacent locations in the previous frame, which does not apply to highly dynamic frames or dynamic viewports. This work introduces a new concept called implicit correlation, i.e., the consistency of topological structures, which stably exists in dynamic frames and is beneficial for reducing bandwidth consumption. We design a mobile volumetric video streaming system Hermes consisting of an implicit correlation encoder to reduce bandwidth consumption and a hybrid streaming method that adapts to dynamic viewports. Experiments show that Hermes achieves a frame rate of 30+ FPS over daily networks and on commodity smartphones, with at least 3.37x improvement compared with two baselines.

Fine-grained image classification focuses on distinguishing objects from different similar subcategories, which requires the classification model to extract subtle yet discriminative descriptors. Recent Vision Transformer (ViT) has shown an enormous potential for this challenging task, but previous ViT-based methods have primarily focused on improving the relationship between image patches, neglecting the limited expressive capability caused by the single class token.To address this limitation, we propose to learn a Multi-part Token Transformer (MpT-Trans), which extends the class token to multiple tokens presenting various parts, enhancing the model's capability of extracting discriminative information. Specifically, our MpT-Trans model interpolates the vision transformer framework with two modules: (i) the Part-wise Shift Learning (PwSL) module is proposed to extend the single class token to a set of part tokens with differentiable shifts, enabling the model to extract informative representations from different perspectives; (ii) the Dual Contrastive Learning (DuCL) module is introduced to exploit the inter-class and inter-part relationships to regularize the learning of part tokens, enhancing their diversity and discrimination for accurate classification. Extensive experiments and ablation study demonstrate that the proposed MpT-Trans achieves state-of-the-art performance on various fine-grained image benchmark datasets, demonstrating the effectiveness of our proposed method.

Existing LoRa only supports the concurrency of orthogonal channels but ignores the large number of non-orthogonal channel concurrency opportunities. In this paper, we propose Mc-LoRa that enables LoRa concurrency for non-orthogonal overlapping channels by solving cross-channel collision that happens when chirps with different bandwidths have the same slope in time-frequency domain. Existing single-channel concurrency methods fail to resolve this new collision because the deterministic symbol offset is invalid anymore due to the asymmetric symbol duration. But we find that when wiping a part of collided signals, the amplitude change of target chirp that aligns with the decoding window is predictable, while the collided chirps experience different changes. We accordingly regard the amplitude change ratio before and after wiping as a new decoding feature. We propose a wiper selection method based on our theoretical model to obtain robust features. We also design noise-aware wiper searching and grouping mechanisms to balance the feature accuracy and computing overhead. The experiments show that Mc-LoRa efficiently decodes packets in non-orthogonal overlapping channels and improves the network throughput by up to 3.4× under cross-channel collision, compared with the state-of-the-art single-channel concurrency methods.

In the Industrial Internet of Things (IIoT) environment, time-sensitive tasks require efficient utilization of computing and network resources to ensure timely completion and fast processing. However, using existing scheduling schemes based on software-defined network (SDN), time-sensitive network (TSN), or information technology (IT) can lead to problems such as resource inefficiency, transmission delays, and task timeouts. To overcome these challenges, this paper proposes a collaborative scheduling method that combines SDN for global resource management and TSN for precise resource allocation. Additionally, a control plane algorithm is introduced to adapt task resources. This approach effectively schedules global computing and network resources, enabling timely and efficient task processing. Extensive experiments demonstrate the superiority of this method in terms of task completion rate and processing time compared to other baselines in various network environments.

In this paper, we propose a Disentangled Counterfactual Learning (DCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects’ physics commonsense based on both video and audio input, with the main challenge is how to imitate the reasoning ability of humans. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and lacking causal reasoning ability in models impedes the progress of implicit physical knowledge inferring. To address these issues, our proposed DCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space by the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model’s reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. Our proposed method is a plug-and-play module that can be incorporated into any baseline. In experiments, we show that our proposed method improves baseline methods and achieves state-of-the-art performance. Our source code is available at https://github.com/Andy20178/DCL.

We present nnPerf, a real-time on-device profiler designed to collect and analyze the DNN model run-time inference latency on mobile platforms. nnPerf demystifies the hidden layers and metrics used for pursuing DNN optimizations and adaptations at the granularity of operators and kernels, ensuring every facet contributing to a DNN model's run-time efficiency is easily accessible to mobile developers via well-defined APIs. With nnPerf, the mobile developers can easily identify the bottleneck in model run-time efficiency and optimize the model architecture to meet system-level objectives (SLO). We implement nnPerf on TFLite framework and evaluate its e2e-, operator-, and kernel-latency profiling accuracy across four mobile platforms. The results show that nnPerf achieves consistently high latency profiling accuracy on both CPU (98.12%) and GPU (99.87%). Our benchmark studies demonstrate that running nnPerf on mobile devices introduces the minimum overhead to model inference, with 0.231% and 0.605% extra inference latency and power consumption. We further run a case study to show how we leverage nnPerf to migrate OFA, a SOTA NAS system, to kernel-oriented model optimization on GPUs.

The rapid development of the Industrial Internet of Things (IIoT) provides a significant opportunity to achieve comprehensive awareness and salient event detection in manufacturing factories. However, because of the limited onboard resources of IIoT terminal devices, it remains a challenging task in the face of the processing requirements of compute-intensive and latency-critical applications. To overcome this challenge, we study the hierarchical collaboration dynamic resource scheduling problem for the IIoT cloud-edge computing model. First, we divide the network into different domains for autonomous management and hierarchical collaboration according to the dynamically available computing resources and transmission delay of edge nodes. Second, we establish a computing model of task data size and resource requirement to maximize the processing benefit of tasks and load balancing between domains, formulate the task-domain Pareto optimality matching problem and the task-node optimal matching problem, which is transformed into the 0-1 Multiple Knapsack Problem (MKP). Third, we develop a hierarchical collaboration dynamic resource scheduling algorithm to solve the above optimal matching problems and set each time slot duration according to the processing rate of the algorithm. Extensive experiments show that our method provides an efficient and reliable scheduling strategy for IIoT in various scenarios with good scalability.

Deep-learning-based (DL-based) face recognition has become an important application in the Internet of Things (IoT) environment. However, recent studies demonstrate that elaborate adversarial examples can mislead the results of DL-based face recognition on mobile and edge devices. Such vulnerability threats the robustness of face recognition systems and causes security issues. Generative adversarial defense methods can reform adversarial examples before input into the face recognition model to improve the accuracy under adversarial attacks. Unfortunately, the existing generative adversarial defense methods cannot completely remove the misleading features of adversarial examples due to the lack of robust encoding ability. In this article, we propose a local consistency generative adversarial network (LC-GAN) framework by adding the constraint of local consistency to force the encoder to mine consistent features in each local area, achieving robust encoding ability consequently. The framework includes three main novel designs. First, we present a patch-wise contrastive learning-based refinement stage with local consistency loss to encode robust identity features from nonsalient areas that are undamaged by adversarial attacks. Second, we use a powerful expert network to guide the training of LC-GAN for eliminating adversarial identity features. Third, we design a multilevel identity loss to enhance the identity preservation ability by unifying the local and global identity features. Experimental results on four widely used face data sets show that LC-GAN outperforms other generative adversarial defense methods.

The unprecedented prosperity of the Industrial Internet of Things (IIoT) promotes the traditional industry transforming into intelligent manufacturing so that the whole production process can be comprehensively controlled to achieve flexible production. Intelligent scheduling, as one of the key enabling techniques, is desired to allocate the production of several machines by an efficient solution with minimum makespan. Existing approaches adopt a fixed search paradigm based on expert knowledge to seek satisfactory solutions. However, considering the varying data distribution and large sized of the practical problems, these methods fail to guarantee the quality of the obtained solution under the real-time requirement. To address this challenge, we formulate the production scheduling problem as a Markov decision process (MDP) and specifically design a job scheduling model made up of a job batching module for the hybrid flow-shop scheduling problem on batch processing machines (HFSP-BPM). Our proposed model consists of an actor network that learns the action under different conditions and a critic network that evaluates the action of the actor. We analyze the convergence of the model under different parameter settings to determine the optimal parameter. Extensive numerical experiments on both publicly available data set and real steel plant production data set demonstrate that the proposed deep reinforcement learning (DRL) approach compared with other baselines, more than 6% average improvements can be observed in many instances.

Emotion recognition, particularly contactless recognition via wireless sensing, has shown its promise in diverse applications. However, the previous works only focus on emotions rather than the person, i.e., the premise is already knowing who the subject is, without considering the issue of identifying subjects. We envision that user identification and emotion recognition together will bring more adaptive and personalized Internet of Things applications, e.g., a smart home system can react to specific emotions of a specific user, independently. In this work, we move forward to investigate the problem of simultaneous user identification, using only physiological indicators embedded in wireless signals reflected off from targets. Toward the objective, in this article, we first carry out a comprehensive measurement study, which validates the feasibility of simultaneous user identification and emotion recognition. Moreover, the measurement also discovers that the key challenge lies in the limitation of artificial features and the substantial emotion feature deviation across different days, which hinders accurate and robust sensing. To resolve the challenge, we design two multiscale neural networks, incorporated with a custom-built feature attention mechanism, so as to obtain rich feature expression and, thus, enhance the important features for accurate recognition. We prototype mmEMO using a commercial off-the-shelf millimeter-wave radar and experimental evaluation shows that mmEMO can achieve 87.68% user identification accuracy and 80.59% emotion recognition accuracy, respectively.

Respiratory rate (RR) is of great value in health care, especially when it can be continuously monitored using wearable devices in daily life. Recent works employ photoplethysmography (PPG) on smartwatch for continuous respiration monitoring, based on a certain medical discovery called respiratory sinus arrhythmia (RSA), which describes the relationship between respiratory and heart rate. However, we find that these works fall short of robustness. In particular, the respiratory estimation accuracy drops significantly when people breathe faster (e.g., after sports). We further identify the root reason that the RSA gradually weakens as the RR increases. In this article, we propose BreathAnalyzer, which can estimate RR accurately even at high RRs. To achieve this, BreathAnalyzer boosts the weakened RSA and also handles the motion artifacts, by integrating features from multiple domains, i.e., frequency, time, and nonlinear Poincare domain, instead of using the single spectrum or raw signal in previous studies. Moreover, BreathAnalyzer custom-designs a tree-based learning model, which fits multidomain features, while considering limitations of smartwatch. We implement BreathAnalyzer prototype on COTS smartwatch, and extensive evaluation demonstrates that BreathAnalyzer outperforms the state-of-the-art approaches, with accuracy improvement by 35.37%–80.42% across a variety of practical scenarios including high RRs.

Recent on-manifold adversarial attacks can mislead gait recognition by generating adversarial walking postures (AWP) with image generation techniques. However, existing defense methods only eliminate adversarial perturbations on each frame isolatedly but ignore the temporal correlation of gait sequence, which leads to vulnerability of robust gait recognition. In this paper, we propose GaitReload, a post-processing adversarial defense method to defend against AWP for the gait recognition model with sequenced inputs. First, GaitReload utilizes sequenced entity recognition (SER) module to detect the adversarial frames by the temporal constraints of gait sequence. Then, we apply bayesian uncertainty filtering-based (BUF-based) gait interpolation to reform adversarial gait examples. After that, we reload the reformed gait sequence and rectify the recognition results with the guidance of reloading strategy. Specifically, SER has a bi-directional frame difference attention and a temporal feature aggregation to boost the detection performance. For training SER, we apply hidden posture selective attack (HPSA) to generate training samples. The extensive experimental results on CASIA-A, CASIA-B, and OU-ISIR demonstrate that GaitReload can defend against adversarial gait by large margins in both RGB and silhouette modes.

Blood pressure (BP), an important vital sign to assess human health, is expected to be monitored conveniently. The existing BP monitoring methods, either traditional cuff based or newly emerging wearable based, all require skin contact, which may cause unpleasant user experience and is even injurious to certain users. In this article, we explore contactless BP monitoring and propose airBP, which emits millimeter-wave signals toward a user’s wrist, and captures the reflected signal bounded off from the pulsating artery underlying the wrist. By analyzing the reflected signal strength of the signal, airBP generates the arterial pulse and further estimates BP by exploiting the relationship between the arterial pulse and BP. To realize airBP, we design a new beam-forming method to keep focusing on the tiny and hidden wrist artery, by leveraging the inherent periodicity of the arterial pulse. Moreover, we custom design a pre-training and neural network architecture, to combat the challenges from the arterial pulse sparsity and ambiguity, so as to estimate BP accurately. We prototype airBP using a coin-size commercial off-the-shelf millimeter-wave radar and perform extensive experiments on 41 subjects. The results demonstrate that airBP accurately estimates systolic and diastolic BP, with a mean error of –0.30 mmHg and –0.23 mmHg, as well as a standard deviation error of 4.80 mmHg and 3.79 mmHg (within the acceptable range regulated by the FDA’s AAMI protocol), respectively, at a distance up to 26 cm.

ACM TIOT launched its first special issue on the theme of wireless sensing for IoT. As an important component of the special issue and a novel practice of the journal, an online virtual workshop will be held, with presentations for each of the accepted articles. Welcome to join us for online discussion! Free registration is required for an attendee of the workshop. The zoom link will be shared to registered attendees before the workshop.

Federated learning, as a distributed machine learning framework, enables clients to conduct model training without transmitting their data to the server, which is used to solve the dilemma of data silos and data privacy. It can work well on clients having similar data characteristics and distribution. However, it has some limitations where the dataset of clients may be different in distribution, quantity, and concept in many application scenarios. Personalized federated learning is a new federated learning paradigm that aims to guarantee client personalized models’ effectiveness when collaborating with the cloud server. Intuitively, providing further facilitated collaborations for the clients with similar data characteristics and distribution can benefit personalized model building. However, due to the invisibility of client data, it is challenging to extract client characteristics and define collaborative relationships among them from a fine-grained view. Moreover, a reasonable collaborative training approach needs to be designed for a distributed server–client framework. In this article, we design a Hierarchical Attention-enhanced Meta-learning Network (HAM) to address this issue. The main advantage of HAM is that it utilizes the meta-learning approach of taking model parameters as features and learns to learn an extra model for each client to analyze similarities according to their local dataset automatically. According to its two-layers framework, HAM can reasonably achieve a tradeoff between clients’ personality and commonality and provides a hybrid model with useful information from all clients. Considering there are two networks (HAM and base network) that need to learn for each client during the federated training process, we then provide an alternative learning approach to train them in an end-to-end fashion. To further clarify the approach, we describe the personalized federated learning settings framework as FedHAM where the HAM network is distributed deployed in each client. Extensive experiments based on two datasets prove that our method outperforms state-of-the-art baselines under different evaluation metrics.

Traffic prediction under various conditions is an important but challenging task. Latest studies have achieved promising results but suffer degraded performance without exception under abnormal conditions (e.g., accidents), as the traffic patterns under abnormal conditions often deviate from the normal seriously. To adapt to both normal and abnormal conditions, we propose theMulti-taskMulti-rangeMulti-subgraphAttentionNetwork (M3AN), a novel deep learning model to explicitly model the impacts of abnormal events for condition-aware traffic prediction. It constructs different subgraphs to model node features to address the abrupt traffic patterns with sparse abnormal event data, and uses an attention mechanism to capture dynamic spatial dependencies. Meanwhile, a multi-task fusion module is built upon a road-segment graph and an intersection graph to enhance the ability of capturing complicated dependencies, together with a multi-range attention module for automatically learning the influences of abnormal events with lower computational complexity. Experimental results on two real-world traffic datasets show that our M3AN outperforms state-of-the-art approaches under both normal and abnormal conditions.

For a new city that is committed to promoting Electric Vehicles (EVs), it is significant to plan the public charging infrastructure where charging demands are high. However, it is difficult to predict charging demands before the actual deployment of EV chargers for lack of operational data, resulting in a deadlock. A direct idea is to leverage the urban transfer learning paradigm to learn the knowledge from a source city, then exploit it to predict charging demands, and meanwhile determine locations and amounts of slow/fast chargers for charging stations in the target city. However, the demand prediction and charger planning depend on each other, and it is required to re-train the prediction model to eliminate the negative transfer between cities for each varied charger plan, leading to the unacceptable time complexity. To this end, we design an effective solution of S imultaneous Demand P rediction A nd P lanning ( SPAP ): discriminative features are extracted from multi-source data, and fed into an Attention-based Spatial-Temporal City Domain Adaptation Network ( AST-CDAN ) for cross-city demand prediction; a novel Transfer Iterative Optimization ( TIO ) algorithm is designed for charger planning by iteratively utilizing AST-CDAN and a charger plan fine-tuning algorithm. Extensive experiments on real-world datasets collected from three cities in China validate the effectiveness and efficiency of SPAP . Specially, SPAP improves at most 72.5% revenue compared with the real-world charger deployment.

The ever-increasing heavy traffic congestion potentially impedes the accessibility of emergency vehicles (EVs), resulting in detrimental impacts on critical services and even safety of people's lives. Hence, it is significant to propose an efficient scheduling approach to help EVs arrive faster. Existing vehicle-centric scheduling approaches aim to recommend the optimal paths for EVs based on the current traffic status while the road-centric scheduling approaches aim to improve the traffic condition and assign a higher priority for EVs to pass an intersection. With the intuition that real-time vehicle-road information interaction and strategy coordination can bring more benefits, we proposeLEVID, aLEarning-based cooperativeVehIcle-roaDscheduling approach including a real-time route planning module and a collaborative traffic signal control module, which interact with each other and make decisions iteratively. The real-time route planning module adapts the artificial potential field method to address the real-time changes of traffic signals and avoid falling into a local optimum. The collaborative traffic signal control module leverages a graph attention reinforcement learning framework to extract the latent features of different intersections and abstract their interplay to learn cooperative policies. Extensive experiments based on multiple real-world datasets show that our approach outperforms the state-of-the-art baselines.

Machine learning models, particularly reinforcement learning (RL), have demonstrated great potential in optimizing video streaming applications. However, the state-of-the-art solutions are limited to an“offline learning”paradigm, i.e., the RL models are trained in simulators and then are operated in real networks. As a result, they inevitably suffer from the simulation-to-reality gap, showing far less satisfactory performance under real conditions compared with simulated environment. In this article, we close the gap by proposing Legato, an online RL framework for real-time mobile interactive video systems. Legato puts many individual RL agents directly into the video system, which make video bitrate decisions in real-time and evolve their models over time. Legato then employs a two-level cooperative learning mechanism to enhance video QoE. First, Legato proposes a score-based robust learning algorithm to eliminate risks of quality degradation caused by the RL model's exploration attempts. Then, Legato adaptively aggregates agents following a network condition-aware manner to form its corresponding high-level RL model that can help each individual to react to unseen network conditions. We implement Legato on an interactive real-time video system. Based on the exhaustive evaluations, we find that Legato outperforms the state-of-the-art algorithms significantly across a wide range of QoE metrics.

The prosperity of Internet of Things (IoT) brings forth the deployment of large-scale sensing systems such as smart cities. To enable the collaboration tasks among distributed devices, time synchronization is crucial. However, due to the long-range and device heterogeneity, accurate time synchronization for a large-scale IoT network is challenging. Existing GPS or NTP solutions either require an outdoor environment or only have low and unstable accuracy. In this paper, we propose AirSync, a novel synchronization method that leverages the widely existed aircraft signals, ADS-B, to synchronize large-scale IoT networks with nodes even in indoor environments. But ADS-B messages have no time stamp and cannot provide a reference time. We leverage the continuity of aircraft movements to estimate the aircraft traveling time. Then devices that observe common aircraft moving segments can calculate their time offset. To obtain the time skew, we propose a combined aircraft linear regression method. We also design a transitive synchronization for devices that cannot observe common aircraft. Besides, we also design a duty-cycled ADS-B message collection method for resource-limited IoT devices. We implement a prototype of AirSync and evaluate its performance in various real-world environments. The results show that AirSync can obtain the sub-ms accuracy.

Volumetric video provides a more immersive holographic virtual experience than conventional video services such as 360-degree and virtual reality (VR) videos. However, due to ultra-high bandwidth requirements, existing compression and transmission technology cannot handle the delivery of real-time volumetric video. Unlike traditional compression methods and the approaches that extend 360-degree video streaming, we propose AITransfer, an AI-powered compression and semantic-aware transmission method for point cloud video data (a popular volumetric data format). AITransfer targets the semantic-level communication beyond transmitting raw point cloud video or compressed video with two outstanding contributions: (1) designing an integrated end-to-end architecture with two fundamental contents of feature extraction and reconstruction to reduce the bandwidth consumption and alleviate the computational pressure; and (2) incorporating the dynamic network condition into end-to-end architecture design and employing a deep reinforcement learning-based adaptive control scheme to provide robust transmission. We conduct extensive experiments on the typical datasets and develop a case study to demonstrate the efficiency and effectiveness. The results show that AITransfer can provide extremely efficient point cloud transmission while maintaining considerable user experience with more than 30.72x compression ratio under the existing network environments.

While 5G has rolled out since 2019 and exhibited versatile advantages, its performance under high/extreme mobility scenes (e.g., driving, high-speed railway or HSR) remains mysterious. In this work, we carry out a large-scale field-trial campaign, taking >13,000 Km round-trips on HSR moving at 250–350 Km/h, with operational 5G cellular coverage along the railway. Our empirical study reveals that coupling interaction among high mobility, 5G handover characteristics, and applications’ sluggish reaction to handover, results in catastrophic damage to user experience: low TCP bandwidth utilization of 26.6% and glitchy 4K VoD streaming. To solve the problem, we propose an edge-assisted mobility management framework called Octopus. Different from previous works, Octopus aims at a standard-compatible and easy-to-deploy solution, thus we take a new design paradigm of exploiting the edge intelligence on multi-access edge computing (MEC). We realize Octopus as a universal MEC service ready for benefiting any third-party mobile applications. We prototype, deploy, and evaluate Octopus in operational 5G, which demonstrates the significant performance gain across the full-range mobile scenarios, e.g., HSR, driving, and walking.

WiFi-based Long Distance (WiLD) networks are promising to cover the rural and remote regions. But the explosive short-range WiFi deployments result in the long-short coexistence. Due to CSMA is ignorance of propagation delay, its carrier sensing is too short to detect long links, leading to the temporal hidden terminal problem that causes serious performance degradation and even starvation of long links. Existing methods for traditional hidden terminal problem are inefficient to cope with this problem because of the different causes. In this paper, we propose CSMA with Protective Jamming (CSMA/PJ), a new WiLD MAC protocol that solves the temporal hidden terminal problem with the minimized influence on uncontrollable short links. The key is generating protective jamming at the WiLD receiver that is sensible to the short links. By leveraging the asymmetric propagation delay of the WiLD transmitter and receiver, we make the jamming protective rather than destructive. We precisely control the jamming right before the arrivals of WiLD packets to set aside channel time for short links. We implement and evaluate CSMA/PJ on commercial devices. The experimental results show that CSMA/PJ can improve the throughput of the WiLD link by$6\times $and$5 \times $compared with the CSMA/CA and RTS/CTS methods.

Time series forecasting in the sensing system aims to predict future values based on historical records that sensors have collected. Previous works, however, usually focus on improving model structure or algorithm for better performance but the perspective of learning proper numeric representations is overlooked. The inappropriate and coarse numeric representations are not expressive enough to capture the intrinsic characteristics of numbers, which will obviously degrade the prediction performance. In this article, we propose Num2vec, an algorithmic framework to learn numeric representations. Specifically, Num2vec lists three main logic characteristics of numbers: arithmetic, direction, and periodicity. By representing numbers into a transition space, Num2vec can translates numbers agilely to different Internet of Things tasks through selecting the corresponding characteristics. According to such a design, Num2vec enjoys flexible numeric representations to fit different Internet of Things time series tasks. Extensive experiments on four real-world datasets show that the approach achieves the best performance when compared with state-of-the-art baselines.

The massive connection of LoRa brings serious collision interference. Existing collision decoding methods cannot effectively deal with the adjacent collisions that occur when the collided symbols are adjacent in the frequency spectrum. The decoding features relied on by the existing methods will be corrupted by adjacent collisions. To address these issues, we propose Paralign , which is the first LoRa collision decoder supporting decoding LoRa collisions with confusing symbols via parallel alignment. The key enabling technology behind Paralign is tha there is no spectrum leakage with periodic truncation of a chirp. Paralign leverages the precise spectrum obtained by aligning the de-chirped periodic signals from each packet in parallel for spectrum filtering and power filtering. To aggregate correlation peaks in different windows of the same symbol, Paralign matches the peaks of multiple interfering windows to the interested window based on the time offset between collided packets. Moreover, a periodic truncation method is proposed to address the multiple candidate peak problem caused by side lobes of confusing symbols. We evaluate Paralign using USRP N210 in a 20-node network. Experimental results demonstrate that Paralign can significantly improve network throughput, which is over 1.46× higher than state-of-the-art methods.

Ambient environment information, including reflectors’ geometrical layout, dimension, and reflectivity, is a key input to versatile millimeter-wave networking and sensing applications. It has found versatile applications in optimizing network coverage and robustness, enhancing mobile link performance, and enabling high-accuracy indoor localization and navigation. Recent approaches of deriving mmWave environment information require heavy infrastructure support and rely on costly software-defined radios, which prevent their usage in practice. In this work, we design and implement e-mmRanger, which can efficiently sense the environment without infrastructure support. e-mmRanger equips a pair of low-cost off-the-shelf mmWave radios in a commodity robot, which constantly samples the ambient environment by exchanging a series of mmWave signals while it moves. It then re-engineers the time-domain signal series to derive the spatial-domain environment structure through novel reflection path extraction and clustering algorithms. Moreover, e-mmRanger accelerates the mapping process by incorporating a novel Space-knit algorithm, which strategically plans an optimal movement route consisting of minimum sampling locations for the robot. Our experiments verify that e-mmRanger can accurately and efficiently sense the surrounding environment, and the learned information can bring multi-fold performance gain over empirical approaches in mmWave networks.

Human activity recognition (HAR) shows significant applicable values in health care. However, it has not been widely used due to some constraints, like privacy concerns of vision-based methods and the inconvenience of wearable devices-based methods. Recently, millimeter-wave (mmWave) based HAR has caught attention increasingly as it is able to provide a non-imaging, contactless and continuous approach for HAR. However, the existing mmWave-based HAR methods have limited ability and scalability to discriminate similar activities. To address existing problems, we present MM-HAT, an end-to-end Transformer network for mmWave point cloud based HAR. We design point-cloud-specific adaptations to Transformers that have gained success in natural language processing and vision. In addition, MM-HAT takes both mmWave point cloud data and target-data extracted from point clouds as input to mitigate the adverse effects of mmWave signals' vulnerability. We have collected a 7-activity mmWave dataset and carried out experiments on it. Evaluation results show that MM-HAT outperforms the existing methods by up to 34.76% (RadHAR) and 20.05% (MMPoint-GNN).

Object detection with camera has achieved promising results using deep learning methods, but it suffers degraded performance under adverse conditions (e.g., foggy weather, poor illumination). To remedy this, some recent studies resort to leveraging the complementary mmWave radar, which is less affected by adverse conditions, and designing effective fusion methods. However, the existing early fusion methods are vulnerable to data noise, while the existing late fusion methods ignore the association of object information between feature maps in the early stage. To overcome these shortcomings, we propose a Global-Local Feature Enhancement Network (GLE-Net), a two-stage deep fusion detector, which first generates anchors from two sensors and uses an auxiliary module to locally enhance the single-branch missing proposals, and then fuses the global features from the multimodal sensors to improve final detection results. We collect two datasets under foggy weather and poor illumination conditions with diverse scenes, and conduct extensive experiments, verifying that the proposed GLE-Net surpasses other state-of-the-art methods in terms of Average Precision (AP).

Adversarial training can defend against universal adversarial perturbation (UAP) by injecting corresponding adversarial samples during training. However, adversarial samples used by existing methods, such as UAP, inevitably include excessive perturbations related to other categories due to its inherent goal of universality. Training with them will cause more erroneous predictions with larger local positive curvature. In this paper, we propose a curvature-aware category adversarial training method to avoid excessive perturbations. We introduce the category-oriented adversarial masks that are synthesized with class distinctive momentum. Besides, we split the min-max optimization loops of adversarial training into two parallel processes to reduce the training cost. Experimental results on CIFAR-10 and ImageNet show that our method achieves better defense accuracy under UAP with less training cost than state-of-the-art baselines.

Adversarial training can boost the robustness of the model by aligning discriminative features between natural and generated adversarial samples. However, the generated adversarial samples tend to have more features derived from changed patterns in other categories along with the training process, which prevents better feature alignment between natural and adversarial samples. Unfortunately, existing adversarial training methods ignore such dynamicity of generated adversarial samples. In this paper, we propose Adversarial Contrastive Decoupling (ACD) to filter the features derived from changed patterns. Specificity, we decouple the changed patterns from adversarial samples and then extract robust representations from remaining features. First, we introduce a decoupling module with a dynamic labeling strategy to explore the dynamicity of generated adversarial samples. Then, we propose a siamese network with contrastive learning mechanism to align remaining robust representations between adversarial and natural samples. Extensive experimental results demonstrate the superior performance of ACD over baselines.

Adaptive bitrate (ABR) algorithm is deployed extensively in commercial video delivery platforms, aiming to ensure users' quality of experience(QoE). Among the majority of existing ABR algorithms, throughput prediction plays a critical role. However, these predictors suffer from neglecting the throughput inconsistency across diverse chunk payloads under the network dynamics, e.g., the actual throughput of downloading a 4K or a 720P chunk is usually different, even when starting from the same moment. In this paper, we propose a payload-aware adaptive algorithm called PAR, which predicts multiple throughput estimations for different target payloads, and utilizes them to make better bitrate adaptation decisions. Trace-driven experiments show that PAR outperforms the existing ABR schemes across diverse network conditions, with the average QoE improvement of 2.66% to 79.43%.

Recent studies show that image retrieval systems are vulnerable to adversarial attacks that adds imperceptible noise to the images. But the imperceptibility of noise limits the attack performance. As a commonly acceptable interference, watermark often appears in images, which can be treated as ‘‘imperceptible’’ but is unexploited. In this paper, we propose a Watermark Attack method on Image Retrieval systems (WAIR). Attacking retrieval systems is challenging due to black-box model, the absence of confidence guide, and attack failures and low efficiency caused by the randomness of traditional evolutionary algorithms. To solve these challenges, we propose a new evolutionary algorithm called Gene Joint Selecting Algorithm (GJSA) that jointly optimize the watermark parameters. We also design the Fitness Record Table (FRT), a new data structure that records the historical attack effect to guide the following evolution and avoid local optimal solutions. Thanks to FRT, WAIR can also reduce the duplicate searching caused by algorithm randomness. The extensive experiments show that WAIR can attack the black-box image retrieval system with a successful rate of 0.806, which is 12.4% higher than the traditional evolutionary algorithm. Moreover, for attacking commercial image retrieval system, we achieve 2$\times$ higher attack success rate on Baidu Image Retrieval API than the existing methods.

LoRa is widely deploying in aquatic environments to support various Internet of Things applications. However, floating LoRa networks suffer from serious performance degradation due to the polarization loss caused by the swaying antenna. Existing methods that only control the transmission starting from the aligned attitude have limited improvement due to the ignorance of aligned period length. In this paper, we propose PolarScheduler, a dynamic transmission control method for floating LoRa networks. PolarScheduler actively controls transmission configurations to match polarization aligned periods. We propose a V-zone model to capture diverse aligned periods under different configurations. We also design a low-cost model establishment method and an efficient optimal configuration searching algorithm to make full use of aligned periods. We implement PolarScheduler on commercial LoRa platforms and evaluate its performance in a deployed network. Extensive experiments show that PolarScheduler can improve the packet delivery rate and throughput by up to 20.0% and 15.7%, compared to the state-of-the-art method.

Cross-Technology Communication (CTC) is an emerging technique that enables direct interconnection among incompatible wireless technologies. Recent work proposes CTC from IEEE 802.11b to LoRa but has a low efficiency due to their extremely asymmetric data rates. In this paper, we propose WiRa that emulates LoRa waveforms with IEEE 802.11ax to achieve an efficient CTC from WiFi to LoRa. By taking advantage of the OFDMA in 802.11ax, WiRa can use only a small Resource Unit (RU) to emulate LoRa chirps and set other RUs free for high-rate WiFi users. WiRa carefully selects the RU to avoid emulation failures and adopts WiFi frame aggregation to emulate the long LoRa frame. We propose a subframe header mapping method to identify and remove invalid symbols caused by irremovable subframe headers in the aggregated frame. We also propose a mode flipping method to solve Cyclic Prefix errors, based on our finding that different CP modes have different and even opposite impacts on the emulation of a specific LoRa symbol. We implement a prototype of WiRa on the USRP platform and commodity LoRa device. The extensive experiments demonstrate WiRa can efficiently transmit complete LoRa frames with the throughput of 40.037kbps and the symbol error rate (SER) lower than 0.1.

LoRa is widely deployed for various applications. Though the knowledge of the channel occupancy is the prerequisite of all aspects of network management, acquiring the channel occupancy for LoRa is challenging due to the large number of channels to be detected. In this paper, we propose LoRadar, a novel LoRa channel occupancy acquirer based on cross-channel scanning. Our in-depth study finds that Channel Activity Detection (CAD) in a narrow band can indicate the channel activities of wide bands because they have the same slope in the time-frequency domain. Based on our finding, we design the cross-channel scanning mechanism that infers the channel occupancy states of all the overlapping channels by the distribution of CAD results. We elaborately select and adjust the CAD settings to enhance the distribution features. We also design the pattern correction method to cope with distribution distortions. We implement LoRadar on commodity LoRa platforms and evaluate its performance on the indoor testbed and the outdoor deployed network. The experimental results show that LoRadar can achieve a detection accuracy of 0.99 and reduce the acquisition overhead by up to 0.90, compared to existing traversal-based methods.

Energy and cost are two primary concerns when leveraging drones for urban sensing. With the advances of wireless charging technologies and the inspiration from the sparse crowdsensing paradigm, this paper proposes a novel drone-based collaborative sparse-sensing framework DroneSense, demonstrating its feasibility for sustainable urban-scale sensing. We focus on a typical use case, i.e., leveraging DroneSense to sense open parking spaces. DroneSense selects a minimum number of Points of Interest (POIs) to schedule drones for physical data sensing and then infers the parking occupancy of the remaining POIs to meet the overall quality requirement. However, drone-based sensing is different from human-centric crowdsensing, resulting in a series of new problems, including which POIs are visited first, when and where to charge drones, which drones to charge first, how much to charge, and when to stop the scheduling. To this end, we design a holistic solution, including context-aware matrix factorization for parking occupancy data inference, progressive determination of task quantity, deep reinforcement learning (DRL) based task selection, energy-aware DRL-based task scheduling, and adaptive charger scheduling. Extensive experiments with a real-world on-street parking dataset from Shenzhen, China demonstrate the obvious advantages of DroneSense.

Recently, it is gaining increasingly attention to incorporate self-supervised technologies into few-shot learning. Previous methods have exclusively focused on image-level self-supervision, but they ignore that capturing subtle part features plays an important role in distinguishing fine-grained images. In this paper, we propose an approach named PaCL that embeds part-level contrastive learning into fine-grained few-shot image classification, strengthening the models' capability to extract discriminative features from indistinguishable images. PaCL treats parts as the inputs of contrastive learning, and it uses a transformation module to involve image-specific information into pre-defined meta parts, generating multiple features from each meta part depending on different images. To alleviate the impact of changes in views or occlusions, we propose to adopt part prototypes in contrastive learning. Part prototypes are generated by aggregating the features of each certain type of part, which are more reliable than directly using part features. A few-shot classifier is adopted to predict query images, which calculates the classification loss to optimize the transformation module and meta parts in conjunction with the loss calculated in contrastive learning. The optimization process will enforce the model to learn to extract discriminative and diverse features from different parts of the objects, even for the samples of unseen classes. Extensive studies show that our proposed method improves the performance of fine-grained few-shot image classification across several backbones, datasets, and tasks, achieving superior results compared with state-of-the-art methods.

Mobile edge computing (MEC), as a key ingredient of the 5G ecosystem, is envisioned to support demanding applications with stringent latency requirements. The basic idea is to deploy servers close to end-users, e.g., on the network edge-side instead of the remote cloud. While conceptually reasonable, we find that the operational 5G is not coordinated with MEC and thus suffers from intolerable long response latency. In this work, we propose Tutti, which couples 5G RAN and MEC at the user space to assure the performance of latency-critical video analytics. To enable such capacity, Tutti precisely customizes the application service demand by fusing instantaneous wireless dynamics from the 5G RAN and application-layer content changes from edge servers. Tutti then enforces a deadline-sensitive resource provision for meeting the application service demand by real-time interaction between 5G RAN and edge servers in a lightweight and standard-compatible way. We prototype and evaluate Tutti on a software-defined platform, which shows that Tutti reduces the response latency by an average of 61.69% compared with the existing 5G MEC system, as well as negligible interaction costs.

With the popularization of deep learning models in wireless sensing, researchers have made considerable efforts to construct sophisticated models to improve the accuracy of related applications. But very few studies have addressed the potential vulnerabilities of deep models, and existing works evaluate wireless adversarial performance only in communication or sensing. None of them has a comprehensive definition of attack imperceptibility. In this paper, we come up with a definition of the wireless attack imperceptibility for both communication and sensing. Our goal is to craft an adversarial perturbation, which can degrade the performance of WiFi sensing without compromising WiFi communication. To achieve this goal, we propose WiCAM to reveal the temporal and spatial attention of a DNN, capturing the crucial portions of its input. Then we design a mask to limit adversarial perturbation in the attended parts only, and thus the impact of the attack on WiFi communication is minimized. WiCAM is a general adversarial framework that can integrate existing adversarial methods such as FGSM and PGD to generate perturbations. We carry out experiments on three popular WiFi sensing applications, including human activity recognition, gesture recognition, and user identification. Extensive experiments are conducted on both public datasets and self-collected datasets. The results show that when declining the accuracy of a target model below 50%, WiCAM can reduce the impact on communication in terms of BER by up to 77.78% in QAM-64, compared to the common adversarial methods.

WiFi has achieved great success in data communication in the past two decades and WiFi signals are recently further exploited for sensing purposes. Promising progress has been achieved and diverse WiFi sensing applications have been enabled. However, one critical issue which was not paid much attention to and we believe would greatly hinder the real-life adoption of WiFi sensing is that it actually affects WiFi communication. The fundamental reason is that WiFi sensing requires high-frequency signal samples and WiFi data packets can not meet this requirement. Therefore, existing WiFi sensing systems transmit dedicated high-frequency packets (200-2000 packets per second) for sensing and these “sensing packets” greatly affect the main data communication function of WiFi. In this work, we propose WiImg, a lightweight system which involves machine learning techniques to enable WiFi sensing under low packet rate, pushing WiFi sensing one step towards real-life adoption. The key idea is to convert the CSI samples into images and improve the Generative Adversarial Network (GAN) for CSI image inpainting, relaxing the requirement of high sample rate in sensing. To avoid the large training overhead of GAN, we design a lightweight GAN that leverages samples of only three rates to recover the CSI traces of any arbitrary rates. Experiments show that with just 25 packets per second, WiImg is able to increase the recognition accuracy for hand gesture recognition and daily activity tracking from the state-of-the-art 59.1% and 65.9% to 86.7% and 96.4%, respectively.

Millimeter wave (mmWave) sensing promises to enable contactless and high-precision “in-air” gesture-based human–computer interaction (HCI). While previous works have demonstrated its feasibility, they require tedious gesture collecting for person-independent recognition and they operate in an off-line mode without considering practical issues, such as segmenting gesture and recognition latency. In this work, we proposeM-Gesture, a person-independent real-time mmWave gesture recognition solution. We first build a compact gesture model with a custom-designed neural network to distill the unique features underlying each gesture, while suppressing personalized discrepancy across different users without extra collection and retraining. Furthermore, we design a system status transition (SST) to decide when a gesture begins and ends, which enables automatic gesture segmentation and hence real-time recognition. We prototypeM-Gestureon a commodity mmWave sensor and demonstrate its advantages using two practical applications: 1) a contactless music player and 2) camera. Extensive experiments and user studies show thatM-Gesturehas an accuracy of 99% and a short response latency within 25 ms. Moreover, we also collect and release a comprehensive mmWave gesture data set consisting of 54 620 instances from 144 persons, which may have an independent value of facilitating future research.

Multistep service-level passenger flow forecasting is of great value in bus transit systems. This task is faced with great challenges due to complicated and dynamic spatial–temporal dependencies, such as interstation semantic dependencies, interline spatial dependencies, and interservice temporal dependencies, which are not effectively modeled by existing methods. To address these challenges, we propose a spatiotemporal hashing multigraph convolution network, called ST-HMGCN. ST-HMGCN constructs two types of subgraphs from perspectives of physical adjacency and semantic similarity to explicitly capture spatial–temporal dependencies among bus stations/lines, and integrates the interservice temporal correlations to achieve the service-level bus passenger flow forecasting. Moreover, it utilizes the hashing graph convolution to extract the dynamic spatial correlations among graph nodes. Furthermore, a temporal-attention block with residual connections is used to model the nonlinear temporal correlations between different time intervals of each station, which significantly reduces the error propagation among prediction time steps. Finally, we use a large-scale real bus operation data set to conduct an extensive evaluation of ST-HMGCN and 11 state-of-the-art baselines, and further leverage the passenger prediction results of our model to provide crowdedness-aware route recommendation. The experimental results verify the effectiveness of the proposed modeling method and its application value in intelligent transportation.

Real-time parking availability prediction is of great value to optimize the on-street parking resource utilization and improve traffic conditions, while the expensive costs of the existing parking availability sensing systems have limited their large-scale applications in more cities and areas. This paper presents the MePark system to predict real-time citywide on-street parking availability at fine-grained temporal level based on the readily accessible parking meter transactions data and other context data, together with the parking events data reported from a limited number of specially deployed sensors. We design an iterative mechanism to effectively integrate the aggregated inflow prediction and individual parking duration prediction for adequately exploiting the transactions data. Meanwhile, we extract discriminative features from the multi-source data, and combine the multiple-graph convolutional neural network (MGCN) and the long short-term memory (LSTM) network for capturing complex spatio-temporal correlations. The extensive experimental results based on a four-month real-world on-street parking dataset in Shenzhen, China demonstrate the advantages of our approach over various baselines.

Reliable traffic flow prediction is of great value in the field of transportation, which, for example, contributes to traffic control and public safety. The key of achieving better performance is to well capture the non-linear spatial-temporal dependency. The state-of-the-art works consider both aspects, but they ignore the effect of the global trend on local dynamics and fail to capture long-term dynamic dependencies. In this article, we propose a novel Global-Local Temporal Convolutional Network (GL-TCN) to break through these limitations. Specifically, a novel local temporal convolutional mechanism is proposed to capture the long-term local dynamics effectively. Meanwhile, the global and local flow patterns are integrated to handle the effect of the global flow trend on local dynamics. To the best of our knowledge, this is the first work to utilize the temporal convolutional network for traffic flow prediction. Experiments on two real-world datasets demonstrate the superior performance of our method over several state-of-the-art baselines.

Heterogeneous information network (HIN) embedding aims to learn the low-dimensional representations of nodes while preserving structures and semantics in HINs. Although most existing methods consider heterogeneous relations and achieve promising performance, they usually employ one single model for all relations without distinction, which inevitably restricts the capability of HIN embedding. In this paper, we argue that heterogeneous relations have different structural characteristics, and propose a novel Relation structure-aware HIN Embedding model, called RHINE. By exploring four real-world networks with thorough analysis, we present two structure-related measures which consistently distinguish heterogeneous relations into two categories: Affiliation Relations (ARs) and Interaction Relations (IRs). To respect the distinctive structural characteristics of relations, in RHINE, we propose different models specifically tailored to handle ARs and IRs, which can better capture the structures in HINs. Finally, we combine and optimize these models in a unified manner. Furthermore, considering that nodes connected via heterogeneous relations may have multi-aspect semantics and each relation focuses on one aspect, we introduce relation-specific projection matrices to learn node and relation embeddings in separate spaces rather than a common space, which can better preserve the semantics in HINs, referring to a new model RHINE-M. Experiments on four real-world datasets demonstrate that our models significantly outperform the state-of-the-art methods in four tasks.

Mobile edge computing enables resource-limited edge clouds (ECs) in federation to help each other with resource-hungry yet delay-sensitive service requests. Contrary to common practice, we acknowledge that mobile services are heterogeneous and the limited storage resources of ECs allow only a subset of services to be placed at the same time. This paper presents a jointly optimized design of cooperative placement and scheduling framework, named JCPS, that pursuessocial cost minimizationover time while ensuring diverse user demands. Our main contribution is a novel perspective on cost reduction by exploiting thespatial-temporal diversitiesinworkload and resource costamong federated ECs. To build a practical edge cloud federation system, we have to consider two major challenges:user deadline preferenceandECs’ strategic behaviors. We first formulate and solve the problem of spatially strategic optimization without deadline awareness, which is proved$\mathcal {NP}$-hard. By leveraging user deadline tolerance, we develop a Lyapunov-baseddeadline-drivenjoint cooperative mechanism under the scenario where the workload and resource information of ECs are known for one-shot global cost minimization. Theservice priorityimposed by deadline urgency drives time-critical placement and scheduling, which, combined with cooperative control, enables workloads migrated across different times and ECs. Given selfishness of individual ECs, we further design an auction-based cooperative mechanism to elicittruthful bidson workload and resource cost. Rigorous theoretical analysis and extensive simulations are performed, validating the efficiency of JCPS in realizing cost reduction and user satisfaction.

Millimeter wave (mmWave) wireless networks are envisioned to bring a very high degree of spatial reuse, i.e., multiple links can operate simultaneously without interference. The vision, however, is becoming doubtful, as recent studies found that non-negligible interference exists due to imperfect beam patterns. In this paper, we extensively measure the spatial reuse issue in a dense 60 GHz mmWave network consisting of multiple access points (AP) and users. Our measurement quantifies the impact of interference on network performance and finds that the existing prediction based on interference-resolving approaches are insufficient. Motivated by the findings, we proposeMDSR, which enhances the spatial reuse in 60 GHz mmWave networks. Instead of relying on interference prediction,MDSRtakes a new measurement principle of building a conflict graph that implicitly takes into account the impact of both beam imperfection and reflections. Using the conflict graph,MDSRimproves the spatial reuse from three dimensions: AP association, user scheduling, and beam selection, which can determine the optimal AP-user-beam combination and minimize interference in each scheduling cycle. We prototype and evaluateMDSRon the testbed using commodity mmWave radios. The evaluation results demonstrate thatMDSRimproves network throughput by multi-folds compared with the state-of-the-art one.

Low Power Wide Area Networks (LPWAN) such as Long Range (LoRa) show great potential in emerging aquatic IoT applications. However, our deployment experience shows that the floating LPWAN suffers significant performance degradation, compared to the static terrestrial deployments. Our measurement results reveal the reason behind this is the polarization and directivity of the antenna. The dynamic attitude of a floating node incurs varying signal strength losses, which is ignored by the attitude-oblivious link model adopted in most of the existing methods. When accessing the channel at a misaligned attitude, packet errors can happen. In this paper, we propose an attitude-aware link model that explicitly quantifies the impact of node attitude on link quality. Based on the new model, we proposePolarTracker, a novel channel access method for floating LPWAN.PolarTrackertracks the node attitude alignment state and schedules the transmissions into the aligned periods with better link quality. To support concurrent access of multiple LoRa nodes, an attitude-based slotted-ALOHA protocol is proposed to reduce collision. We implement a prototype ofPolarTrackeron commercial LoRa platforms and extensively evaluate its performance in various real-world environments. The experimental results show thatPolarTrackercan efficiently improve the packet reception ratio by 50.6%, compared with ALOHA in LoRaWAN.

Multi-Gbps 60 GHz millimeter wave (mmWave) networks, are considered as the enabling technology for emerging applications such as untethered VR and 4K/8K Miracast. However, user motion, and even orientation change, can cause mis-alignment between mmWave transceivers’ directional beams and thus severe link outage. Within the practical 3D spaces, the combination of location and orientation dynamics leads to the exponential growth of beam searching complexity, which substantially exacerbates the outage. In this paper, we first measure the impact of 3D motion on 60 GHz link performance in the context of VR and Miracast applications. We find that 3D motion exhibits inherent non-predictability, so conventional beam steering solutions are no longer effective. Therefore, we propose a model-driven 3D beam-steering mechanism called Parallel Scanner (PSCAN), which can maintain high performance for mobile 60 GHz links. To enable PSCAN, we first discover and prove a hidden interaction between 3D beams and the spatial channel profile of 60 GHz radios. Leveraging on which, PSCAN strategically scans the 3D space to reduce the search latency by more than one order of magnitude. Experiment results based on a custom-built 60 GHz platform demonstrate PSCAN’s remarkable throughput gain, up to$5\times $, compared with the state-of-the-art.

Feature selection is an important research field in machine learning since high-dimensionality is a common characteristic of real-world data. It has two main objectives, which are to maximize the classification accuracy while minimizing the number of selected features. As the two objectives are usually in conflict with each other, it makes feature selection a multi-objective problem. However, the large search space and discrete Pareto front makes it not easy for existing evolutionary multi-objective algorithms. In order to deal with the above mentioned difficulties in feature selection, an entropy driven multiobjective particle swarm optimization algorithm is proposed to remove redundant feature and decrease computational complexity. First, its basic idea is to model feature selection as a multiobjective optimization problem by optimizing the number of features and the classification accuracy in supervised condition simultaneously. Second, a particle initialization strategy based on information entropy is designed to improve the quality of initial solutions, and an adaptive velocity update rule is used to swap between local search and global search. Besides, a specified discrete nondominated sorting is designed. These strategies enable the proposed algorithm to gain better performance on both the quality and size of feature subset. The experimental results show that the proposed algorithm can maintain or improve the quality of Pareto fronts evolved by the state-of-the-art algorithms for feature selection.

Natural human activity recognition (HAR) via millimeter wave (mmWave) sensing is a key to the human-computer interaction (HCI), e.g., activity assistance and living state monitoring. Prior work has shown the feasibility of HAR by utilizing mmWave radar, but it falls short of two real-world issues: poor recognition accuracy in the noisy environment and unable to give real-time response due to long latency. In this paper, we propose m-Activity, which can realize HAR while reducing noise caused by environmental multi-path effects, and operate fluently at runtime. m-Activity first distills the human-orientated movements from the noisy background environment and then classify the movements using a custom-designed lightweight neural network called HARnet. To drive the above methods, we propose a simple but efficient response mechanism to enable real-time recognition. We prototype m-Activity on a commodity mmWave radar chip and evaluate its recognition performance over 5 pre-defined human activities within the detection range of 3m, which results in off-line accuracy of 93.25%, and real-time accuracy of 91.52%. Furthermore, we validate m-Activity’s ability under a complex real-world scenario, i.e., fitness center, which is full of severe multi-path effects caused by various strong metal reflectors.

In edge computing environment where network connections are often unstable and workload intensity changes frequently, the proper scaling mechanism and service placement strategy based on microservices are needed to ensure the edge services can be provided consistently. However, the common elastic scaling mechanism nowadays is threshold-based responsive scaling and has reaction time in the order of minutes, which is not suitable for delay-sensitive applications in the edge computing environment. Moreover, auto-scaling strategy and service replica placement are considered separately. If the scaled service replicas are misplaced on the edge nodes with limited resources or significant communication latency between upstream and downstream neighbours, the Quality of Service (QoS) cannot be guaranteed even with the auto-scaling mechanism. In this paper, we study the joint optimization of dynamic auto-scaling and adaptive service placement, and define it as a task delay minimization problem while satisfying resource and bandwidth constraints. Firstly, we design a multi-stage auto-scaling model based on workload prediction and performance evaluation of edge nodes to dynamically create an appropriate number of service replicas. Secondly, we propose a Dynamic Adaptive Service Placement (DASP) approach to iteratively place each service replica by using Adaptive Discrete Binary Particle Swarm Optimization (ADBPSO) algorithm. DASP can determine the current optimal placement strategy according to dynamic service replica scaling decision in a short time. The placement results of the current round will guide the optimization of the next cycle iteratively. The experimental evaluation shows that our approach significantly outperforms the existing methods in reducing the average task response time.

With the rapid development of high-speed Internet access and popularization of high-performance smart devices, past decade has witnessed the great development of crowdsourced live streaming (CLS) service. Transcoding and video distribution are essential in CLS service to guarantee viewer engagement. Large CLS systems gradually migrate their services to multi-cloud platforms. However, highly dynamic viewers' requests influence transcoding and CDN distribution decisions, eventually lead to fluctuation in QoE and increase in operational cost. It is challenging for the CLS system to serve viewer's requests in multi-cloud platforms with fluctuation in cloud transcoding and distribution performance. In this paper, we propose a collaborative framework of cloud transcoding and distribution supporting CLS service. First, we define cost model and QoE model in multi-cloud platforms, comprehensively considering cloud transcoding and distribution. Second, we propose a collaborative cost-efficient approach based on multi-agent decision model. We use a G-Greedy exploration approach to learn what actions to take by exploration and exploitation based on the state of current environment. The trace-driven experiments demonstrate that our proposed approach is cost-efficient and QoE-maintained and can reduce operational cost compared with alternatives (5.37%-21.21%) while maintaining QoE of viewers.

Low Power Wide Area Networks (LPWAN) such as Long Range (LoRa) show great potential in emerging aquatic IoT applications. However, our deployment experience shows that the floating LPWAN suffer significant performance degradation, compared to the static terrestrial deployments. Our measurement results reveal the reason behind this is due to the polarization and directivity of the antenna. The dynamic attitude of a floating node incurs varying signal strength losses, which is ignored by the attitude-oblivious link model adopted in most of the existing methods. When accessing the channel at a misaligned attitude, packet errors can happen. In this paper, we propose an attitude-aware link model that explicitly quantifies the impact of node attitude on link quality. Based on the new model, we propose PolarTracker, a novel channel access method for floating LPWAN. PolarTracker tracks the node attitude alignment state and schedules the transmissions into the aligned periods with better link quality. We implement a prototype of PolarTracker on commercial LoRa platforms and extensively evaluate its performance in various real-world environments. The experimental results show that PolarTracker can efficiently improve the packet reception ratio by 48.8%, compared with ALOHA in LoRaWAN.

Deep learning-based person re-identification (Re-ID) methods have achieved the significant performance of matching person images across camera views, which plays an important role in the construction of the smart city. Recent works of adversarial attacks have explored the serious vulnerability of deep Re-ID systems. However, existing attacks are performed with idealized conditions and ignore the real-world environments, such as occlusion caused by walking habits. In this paper, we propose a two-stage method to perform an occlusion resilient adversarial attack for better evaluation of deep Re-ID systems. Specifically, we construct the occlusion template from the observation and statics of pedestrian walking habits. Then, we design a partition training strategy for a better combination of occluded and exposed adversarial patches. During the training, we introduce contextual loss to penalize the semantic distance of attacked images with the same identity. The extensive experiments on Market1501 demonstrate the performance of our method.

Deep neural networks are easily affected by adversarial patches, causing prediction errors. However, existing adversarial patches are trained with specific model-dataset pairs and only effective for the images with the predetermined size in the dataset. The semantic information of the patch will be distorted when scaling the image, which is a common preprocessing process in practical applications. In this paper, we propose SRA Patch (Scaling-Resilient Adversarial Patch), a new adversarial patch resilient to image scaling. Specifically, we generate the patch in a block-wise way and utilize the superpixel method to resist the loss of semantic information during scaling. Further, we introduce the ensemble model as a Black-Box indicator to address the noise space shrinking issue, which is caused by the small size and the block operation of SRA patch. Finally, we leverage Class Activation Mapping to extract the region with salient features as the final patch to improve the ratio of effective semantic features on the patch to remain during scaling. Extensive experiments have demonstrated that our SRA patch has much stronger attack capability and scaling robustness than existing methods.

Image de-distortion is very important because distortions will degrade the image quality significantly. It can benefit many computational visual media applications that are primarily designed for high-quality images. In order to address this challenging issue, we propose a stacked semantically-guided network, which is the first try on this task. It can capture and restore the distortions around the humans and the adjacent background effectively with the stacked network architecture and the semantically-guided scheme. In addition, a discriminative restoration loss function is proposed to recover different distorted regions in the images discriminatively. As another important effort, we construct a large-scale dataset for image de-distortion. Extensive qualitative and quantitative experiments show that our proposed method achieves a superior performance compared with the state-of-the-art approaches.

Maximizing the quality of experience (QoE) for real-time video is a long-standing challenge. Traditional video transport protocols, represented by a few deterministic rules, can hardly adapt to the heterogeneous and highly dynamic modern Internet. Emerging learning-based algorithms have demonstrated potential to meet the challenge. However, our measurement study reveals an alarming long tail performance issue: these algorithms tend to be bottle-necked by occasional catastrophic events due to the built-in exploration mechanisms. In this work, we propose Loki, which improves the robustness of learning-based model by coherently integrating it with a rule-based algorithm. To enable integration at feature level, we first reverse-engineer the rule-based algorithm into an equivalent "black-box" neural network. Then, we design a dual-attention feature fusion mechanism to fuse it with a reinforcement learning model. We train Loki in a commercial real-time video system through online learning, and evaluate it over 101 million video sessions, in comparison to state-of-the-art rule-based and learning-based solutions. The results show that Loki improves not only the average but also the tail performance substantially (26.30% to 44.24% reduction of stall rate and 1.76% to 2.17% increase in video throughput at 95-percentile).

The quality of photos is highly susceptible to severe weather such as heavy rain; it can also degrade the performance of various visual tasks like object detection. Rain removal is a challenging problem because rain streaks have different appearances even in one image. Regions where rain accumulates appear foggy or misty, while rain streaks can be clearly seen in areas where rain is less heavy. We propose removing various rain effects in pictures using a hybrid multiscale loss guided multiple feature fusion de-raining network (MSGMFFNet). Specially, to deal with rain streaks, our method generates a rain streak attention map, while preprocessing uses gamma correction and contrast enhancement to enhanced images to address the problem of rain accumulation. Using these tools, the model can restore a result with abundant details. Furthermore, a hybrid multiscale loss combining L 1 loss and edge loss is used to guide the training process to pay attention to edge and content information. Comprehensive experiments conducted on both synthetic and real-world datasets demonstrate the effectiveness of our method.

Deep convolutional neural networks (DCNNs) have been widely deployed in real-world scenarios. However, DCNNs are easily tricked by adversarial examples, which present challenges for critical applications, such as vehicle classification. To address this problem, we propose a novel end-to-end convolutional network for joint detection and removal of adversarial perturbations by denoising (DDAP). It gets rid of adversarial perturbations using the DDAP denoiser based on adversarial examples discovered by the DDAP detector. The proposed method can be regarded as a pre-processing step—it does not require modifying the structure of the vehicle classification model and hardly affects the classification results on clean images. We consider four kinds of adversarial attack (FGSM, BIM, DeepFool, PGD) to verify DDAP's capabilities when trained on BIT-Vehicle and other public datasets. It provides better defense than other state-of-the-art defensive methods.

Mobile crowdsensing (MCS) and unmanned vehicle sensing (UVS) provide two complementary paradigms for large-scale urban sensing. Generally, MCS has a lower cost but often confronts sensing imbalance and even blind areas due to the limitation of human mobility, whereas UVS is often capable of completing more demanding tasks at the expense of limited energy supply and hardware cost. Thus, it is significant to investigate whether we could integrate the two paradigms for high-quality urban sensing in a cost-effective collaborative way. However, it is nontrivial due to complex and long-term optimization objectives, uncontrolled dynamics, and a large number of heterogeneous agents. To address the collaborative sensing problem, we propose an actor-critic-based heterogeneous collaborative reinforcement learning (HCRL) algorithm, which leverages several key ideas: local observation to handle expanded state space and extract the states of neighbor nodes, generalized model to avoid environment nonstationarity and ensure the scalability and stability of network, and proximal policy optimization to prevent the destructively large policy updates. Extensive simulations based on a mobility model and a realistic trace data set are conducted to confirm that HCRL outperforms the state-of-the-art baselines.

The advancements of AI and the exponential growth of sensory data are unlocking a wave of intelligent sensing applications. To overcome the shortcoming of centralized learning and local training, Google proposes federated learning that allows users to collectively reap the benefits of shared models trained from decentralized data. However, directly applying federated learning to intelligent sensing applications faces two deficiencies: 1) omitting personalities of local models and 2) high latency. Aiming at these limitations, in this article, we propose a new framework, Federated Sensing, to enable edge-cloud elastic collaborative learning from decentralized sensory data. We design an elastic local update algorithm that can train the personalized models by setting specific updating weights for each node based on the difference between the global and local model. Our algorithm takes both the global consistency and the personalities of the local models. We further propose an n-softsync model aggregation method that significantly reduces training time by combining the synchronous and asynchronous aggregations. Extensive experiments are conducted on two real-world data sets of air quality from Beijing and Los Angeles. Compared with existing federated learning techniques, our framework improves the model performance at least by 2.61% and 18.8% in two data sets, respectively. Besides, it reduces the cloud idle time to 25.5% of the total time, which verifies the advantages of our method in terms of both model performance and training overhead.

Monitoring the status of urban environmental phenomenon, which provides fundamental sensory information, is of great significance for various field of urban research. In this article, we propose a new framework, environmental signal reconstruction generative adversarial network, for reconstructing high-quality environmental signal via sensory data from sparsely distributed monitoring sites. Our framework is based on the generative adversarial network (GAN), in which a three-layer convolutional neural network (CNN)-based generative model is proposed to learn an end-to-end mapping between low- and high-quality signals and a discriminative model is introduced for quantizing the reconstruction accuracy. Considering the scattered distribution of sensory data, we further propose a metric called impact map for building loss function and guiding the adversarial training. Experiments with real-world air quality data of Beijing demonstrate that our method outperforms the state-of-the-art data inference techniques in terms of signal recovery accuracy.

The Internet of Things (IoT) aims to interconnect and converge the physical world and cyberspace. It represents the future of networking, and leads the wave of the third industrial revolution. As IoT applications evolve, more and more regions lacking infrastructures are also experiencing an urgent need to deploy and connect IoT devices into the network. Low earth orbit satellites, due to its wide coverage and relatively low latency, can provide network access for IoT system. In this article, we propose a low-latency heterogeneous network MAC approach for space-earth integrated IoT network, which can significantly reduce network latency, improve system scalability and balance total system load. Specifically, we design a geographic priority time slot allocation strategy which ensures optimal load and latency, besides, we also design a continuous forward time slot allocation strategy, which guarantees data timeliness while reducing the utilization of satellite resources. Then we design three algorithms based on above strategies, which can effectively reduce network latency and balance the satellite resources. The simulation experimental results are consistent with the theoretical results, which embodies the effectiveness of the algorithm.

Mobile CrowdSensing (MCS) is a new paradigm that leverages pervasive mobile devices to efficiently collect the big sensory data, enabling various large-scale applications. However, people's concerns about the loss of individual privacy seriously hinder the prevalence of MCS applications. Differential privacy is widely focused owing to its rigorous definition and strong privacy guarantee, but the state-of-the-art studies still demonstrate its weakness on correlated data, resulting in compromising individual privacy. In this paper, we investigate the influence of sensing data correlation on differential privacy protection for MCS systems, and explore the perturbation mechanisms from two different perspectives. From a protector's perspective, based on the Bayesian Network to model the probabilistic relationship among sensing data, we use the classical definition of differential privacy to deduce the scale parameter, and present one perturbation mechanism. From an adversary's perspective, based on the Gaussian correlation model to describe the data correlation, we analyze the importance of the maximum correlated group to compute the Bayesian differential privacy leakage, and then provide another perturbation mechanism. Compared with the existing solutions, our mechanisms are applicable to arbitrary aggregate query function, and can avoid introducing too much noise. Moreover, we demonstrate the effectiveness of our mechanisms through extensive simulations.

Deep neural networks have achieved remarkable success for video-based action recognition. However, most of existing approaches cannot be deployed in practice due to the high computational cost. To address this challenge, we propose a new real-time convolutional architecture, called Temporal Convolutional 3D Network (T-C3D), for action representation. T-C3D learns video action representations in a hierarchical multi-granularity manner while obtaining a high process speed. Specifically, we propose a residual 3D Convolutional Neural Network (CNN) to capture complementary information on the appearance of a single frame and the motion between consecutive frames. Based on this CNN, we develop a new temporal encoding method to explore the temporal dynamics of the whole video. Furthermore, we integrate deep compression techniques with T-C3D to further accelerate the deployment of models via reducing the size of the model. By these means, heavy calculations can be avoided when doing the inference, which enables the method to deal with videos beyond real-time speed while keeping promising performance. We validate our approach by studying its action representation performance on four benchmarks over three different tasks. Our method achieves clear improvements on UCF101 action recognition benchmark against the state-of-the-art real-time methods by 5.4% in terms of accuracy and 2 times faster in terms of inference speed with a less than 5MB storage model. The source code and the pre-trained models are publicly available at https://github.com/tc3d.

Mobility models, which reproduce traces with basic crowd mobility patterns, are crucial for realistic mobile network simulation and performance evaluation of the planning strategies used for urban networks (such as transit network, communication network, and crowdsensing network). However, trajectories generated by traditional models are often perceived as not realistic for urban context or lack of scalability and universality. This article presents a data-driven trajectory generating technique, named as TraG, that produces synthetic trajectories with the help of some real-world trajectories. Our technique can automatically extract the context features and statistical mobility features, which characterize the mobility of a specific urban crowd from the input empirical traces and, then, regenerate more trajectories based on demand. Moreover, we also summarize the power-law scale correlation of crowd mobility based on four real-world open datasets, including public bicycle traces in New York City and Washington, D.C., taxicab traces in San Francisco and Shenzhen. Finally, we validate the proposed TraG model via the continuous San Francisco taxicab traces, and the result demonstrates that the trajectories simulated by TraG not only inherit the fundamental statistical features of crowd mobility from real traces, but also reflect the features of urban context.

Bus services play a crucial role in urban transit. It is significant to achieve the fine-grained service-level passenger flow prediction (SPFP), namely to predict the total number of passengers for each service of each bus line passing through each station during the next short-term interval. However, it faces great challenges due to complex factors including inter-station and inter-line spatial dependencies, intra-station and inter-service temporal dependencies, and internal/external influences. To address these challenges, we propose a multitask deep-learning (MDL) approach, calledMDL-SPFP, to jointly predict the arriving bus service flow, line-level on-board passenger flow and line-level boarding/alighting passenger flow by leveraging well-designed deep neural networks calledARM. The MDL framework can mutually reinforce the prediction of each type of flow, and finally integrate the outputs to achieve the fine-grained service-level prediction. The ARM network combines three modules, Attention mechanism, Residual block and Multi-scale convolution, to well capture various complex non-linear spatio-temporal dependencies and influence factors. Extensive experiments based on a large-scale realistic bus operation dataset are conducted to confirm that our MDL-SPFP approach outperforms 10 state-of-the-art baselines, and improves 22.39% accuracy than the best baseline.

Incentive mechanism design has aroused extensive attention for crowdsourcing applications in recent years. Most research assumes that participants are already in the system and aware of the existence of crowdsourcing tasks. Whereas in real-life scenarios without this assumption, it is more effective to leverage incentive tree mechanisms that incentivize both users' direct contributions and solicitations to other users. Although such mechanisms have been investigated, we are the first to propose budget-balanced incentive tree mechanisms, called generalized lottrees, which require the total payout to be equal to the announced budget, while guaranteeing several desirable properties including continuing contribution incentive, continuing solicitation incentive, value proportional to contribution, unprofitable solicitor bypassing, and unprofitable Sybil attack. Moreover, three types of generalized lottree mechanisms, 1-Pachira, K-Pachira and Sharing-Pachira, are presented for supporting diversified requirements. A solid theoretical guideline on the mechanism selection is provided based on the Cumulative Prospect Theory. Both extensive simulations and realistic experiments with 82 users are conducted to confirm our theoretical analysis.

Cross-Technology Communication (CTC) is an emerging technique that enables direct interconnection among incompatible wireless technologies. However, CTC channels are inherently asymmetric because of either the one-way nature of emulation or the asymmetric communication range caused by the asymmetric transmission power. In this paper, we focus on establishing symmetric CTC over asymmetric CTC channels. The bottleneck is the short communication range from the low-power and narrow-band technology to the high-power and wide-band technology. To compensate the inevitable distortions, we take advantage of the channel asymmetry and construct chirps in WiFi Channel State Information (CSI) to extend the communication range from ZigBee to WiFi. We build the theoretical model of CSI chirp based CTC and design c-Chirp, a novel CTC from ZigBee to WiFi. Due to channel asymmetry and discreteness, the WiFi receiver can only observe partial and distorted CSI chirps. To cope with this issue, we design a matching based chirp decoding method as well as an adaptation algorithm to reliably decode the symbols. We further extend c-Chirp to one-to-multiple concurrent transmission scenario. The evaluation results show that c-Chirp can achieve a communication range of 60m, which is 6× longer than ZigFi, an existing representative CTC from ZigBee to WiFi.

Cross-Technology Interference (CTI) badly harms the transmission reliability for low-power networks such as ZigBee at 2.4-GHz band. Though promising, channel hopping still faces challenges because the increasingly dense deployment of CTI leaves very few available channels. Selecting a good channel with the least overhead is crucial but challenging. Most of the existing works are heuristic methods that choose a channel far from the current one to avoid adjacent channels that may be correlatively interfered by CTI with a wider bandwidth such as WiFi. However, we observe that the correlated channels influenced by the same CTI source do not necessarily have the same channel qualities and even the opposite state, due to the uneven spectrum power density of CTI. Such channel opportunities are unexplored and wasted. In this article, we propose CoHop, a quantitative correlation-based channel hopping method for low-power wireless networks. We establish a quantitative model that describes the correlation of channel qualities to capture channel opportunities and calculate channel quality without probing, to reduce probing overhead. The probing sequence is optimized based on the Pearson Correlation Coefficient and the prediction-based probing algorithm. We implement CoHop on TinyOS and evaluate its performance in various environments. The experimental results show that CoHop can increase the Packet Reception Ratio by 80%, compared with existing methods.

In the past few years, with the rapid development of CPU-GPU heterogeneous computing, the issue of task scheduling in the heterogeneous cluster has attracted a great deal of attention. This problem becomes more challenging with the need for efficient co-execution of tasks on the GPUs. However, the uncertainty of heterogeneous cluster and the interference caused by resource contention among co-executing tasks can lead to the unbalanced use of computing resource and further cause the degradation in performance of computing platform. In this article, we propose a two-stage task scheduling approach for streaming applications based on deep reinforcement learning and neural collaborative filtering, which considers fine-grained task division and task interference on the GPU. Specifically, the Learning-Driven Workload Parallelization (LDWP) method selects an appropriate execution node for the mutually independent tasks. By using the deep Q-network, the cluster-level scheduling model is online learned to perform the current optimal scheduling actions according to the runtime status of cluster environments and characteristics of tasks. The Interference-Aware Workload Parallelization (IAWP) method assigns subtasks with dependencies to the appropriate computing units, taking into account the interference of subtasks on the GPU by using neural collaborative filtering. For making the learning of neural network more efficient, we use pre-training in the two-stage scheduler. Besides, we use transfer learning technology to efficiently rebuild task scheduling model referring to the existing model. We evaluate our learning-driven and interference-aware task scheduling approach on a prototype platform with other widely used methods. The experimental results show that the proposed strategy can averagely improve the throughout for distributed computing system by 26.9 percent and improve the GPU resource utilization by around 14.7 percent.

Gait recognition, i.e., recognizing persons from their walking postures, has found versatile applications in security check, health monitoring, and novel human-computer interaction. The millimeter-wave (mmWave) based gait recognition represents the most recent advance. Compared with traditional camera-based solutions, mmWave based gait recognition bears unique advantages of being still effective under non-line-of-sight scenarios, such as in black, weak light, or blockage conditions. Moreover, they are able to accomplish person identification while preserving privacy. Currently, there are only few works in mmWave gait recognition, since no public data set is available. In this paper, we build a first-of-its-kind mmWave gait data set, in which we collect gait of 95 volunteers 'seen' from two mmWave radars in two different scenarios, which together lasts about 30 hours. Using the data set, we propose a novel deep-learning driven mmWave gait recognition method called mmGaitNet, and compare it with five state-of-the-art algorithms. We find that mmGaitNet is able to achieve 90% accuracy for single-person scenarios, 88% accuracy for five co-existing persons, while the existing methods achieve less than 66% accuracy for both scenarios.

Recent studies on automatic neural architecture search techniques have demonstrated significant performance, competitive to or even better than hand-crafted neural architectures. However, most of the existing search approaches tend to use residual structures and a concatenation connection between shallow and deep features. A resulted neural network model, therefore, is non-trivial for resource-constraint devices to execute since such a model requires large memory to store network parameters and intermediate feature maps along with excessive computing complexity. To address this challenge, we propose MemNAS, a novel growing and trimming based neural architecture search framework that optimizes not only performance but also memory requirement of an inference network. Specifically, in the search process, we consider running memory use, including network parameters and the essential intermediate feature maps memory requirement, as an optimization objective along with performance. Besides, to improve the accuracy of the search, we extract the correlation information among multiple candidate architectures to rank them and then choose the candidates with desired performance and memory efficiency. On the ImageNet classification task, our MemNAS achieves 75.4% accuracy, 0.7% higher than MobileNetV2 with 42.1% less memory requirement. Additional experiments confirm that the proposed MemNAS can perform well across the different targets of the trade-off between accuracy and memory consumption.

Monitoring the status of urban environmental phenomena is of great significance for urban research and management. While The monitoring sites are often insufficient and unevenly, interpolation values vary in urban spaces non-linearly. It is difficult to find a method that fulfills the requirements of accuracy, robustness, and flexibility for various types of phenomena. In this paper, we present a new kind of deep learning driven spatial interpolation method which works on the grid data that can be applied on the unevenly distributed sites. To generate better accurate spatial continuous data, we design the S2 attention structure and incorporate it with the GAN to turn it into SI-AGAN which can model spatial dependencies across different regions via sparsely and unevenly distributed sampling. It can directly learn an end-to-end mapping between low- and high-quality environmental signals without in-depth knowledge of the phenomenon. Experiments on two real-world air-pollution datasets demonstrate that our training strategy effectively makes the GAN work for the interpolation of uneven data and our proposed SI-AGAN significantly outperforms previous state-of-the-art spatial interpolation methods.

Cross-Technology Communication (CTC) is an emerging technique that enables the direct communication across incompatible wireless technologies. Without modifying any hardware, CTC establishes mutually sensible side channel by manipulating the packet transmissions and encodes information by constructing transmission patterns in terms of signal strength, packet interval, and etc. However, the transmission patterns are prone to the coexisting interference, leading to the unreliability of CTC. Most of the existing methods deal with the reliability problem by reactive retransmission of the corrupted packets, which incurs large delay. In this paper, we propose RCTC, a rateless-coding based CTC that proactively copes with the unreliability. Since the computation ability of low-power ZigBee nodes is limited, we carefully design the coding combination with proper degree distribution to balance the trade-off between reliability and decoding latency. We also propose a coding adaptation algorithm to adapt to the channel dynamics. We implement a prototype on commercial WiFi and ZigBee platforms. The experiment results show that RCTC can reduce the BER by up to 92.6%, compared to existing CTC methods.

Fine-grained vehicle recognition is a challenging problem due to the subtle intra-category appearance variation, which requires the recognition model can capture discriminative features from distinguishing regions. The structure is an important characteristic of vehicles which can help to find substantial parts and learn distinguishing representations. In this paper, we propose an approach that introduces the structure graph into consideration to learn distinguishing representations for vehicle recognition. Our proposed method first constructs a global structure graph from the features generated by the convolutional network and then it applies the graph as the guidance to produce effective representations of vehicles. The results of extensive experiments demonstrate that our proposed method can produce more promising results than other state-of-the-art methods. The results of the visualization illustrate that our approach can construct a suitable structure graph and the global structure information facilitates learning discriminative representations at crucial parts of vehicles.

Knowledge distillation was pioneered to transfer the generalization ability of a large teacher deep network to a light-weight student network. The student network can retain the high quality of the teacher network, yet exhibiting low computational complexity and storage requirement, which is attractive for deploying a deep convolution neural network on a resource-constrained mobile device. However, most of the existing methods focus on transferring the probability distribution of a softmax layer in a teacher network and neglect the intermediate representations. However, we find that the intermediate representation is critical for a student network to better understand the transferred generalization as compared to the probability distribution only. In this paper, therefore, we propose such a knowledge transfer adversarial network method which holistically considers both intermediate representations and probability distributions of a teacher network. To transfer the knowledge of intermediate representations, we set high-level teacher feature maps as a target, toward which the method trains student feature maps. Furthermore, to support various structures of a student network, we arrange a novel teacher-to-student layer. Finally, the proposed method employs an adversarial learning process. Specifically, it includes a discriminator network to fully exploit the spatial correlation of feature maps during the training process of a student network. The experimental results demonstrate that the proposed method can significantly improve the performance of a student network on two important vision tasks, image classification and object detection.

Image translation across diverse domains has attracted more and more attention. Existing multi-domain image-to-image translation algorithms only learn the features of the complete image without considering specific features of local instances. To ensure the important instance to be more realistically translated, we propose a cross-granularity learning model for multi-domain image-to-image translation. We provide detailed procedures to capture the features of instances during the learning process, and specifically learn the relationship between style of the global image and the style of an instance on the image through the enforcing of the cross-granularity consistency. In our design, we only need one generator to perform the instance-aware multi-domain image translation. Our extensive experiments on several multi-domain image-to-image translation datasets show that our proposed method can achieve superior performance compared with the state-of-the-art approaches.

Anomaly detection in surveillance videos, as a special case of video-based action recognition, has been of increasing interest in multimedia community and public security. Action recognition in videos faces some challenges, such as cluttered background, illumination conditions. Besides these above difficulties, detecting anomaly in surveillance videos has several unique problems to be solved. For example, the lack of sufficient training samples is one of the main challenges for detecting anomalies in surveillance videos. In this paper, we propose to utilize transfer learning to leverage the good results from action recognition for anomaly detection in surveillance videos. More specially, we explore some techniques based on action recognition models from the following aspects: training samples, temporal modules for action recognition, network backbones. We draw some conclusions. First, more training samples from surveillance videos lead to higher classification accuracy. Second, stronger temporal modules designed for recognizing action and deeper networks do not achieve better results. This conclusion is reasonable since deeper networks tend to over-fitting, especially for the small-scale training set. Besides, to distinguish the hard examples from normal activities, we separately train a neural network to classify the hard category and normal events. Then we fuse the binary network and previous network to generate the final prediction for general anomaly detection. On the benchmarks of CitySCENE, our framework achieves promising performance and obtains the first prize for general anomaly detection and the second prize for specific anomaly detection.

Machine learning models, particularly reinforcement learning (RL), have demonstrated great potential in optimizing video streaming applications. However, the state-of-the-art solutions are limited to an "offline learning" paradigm, i.e., the RL models are trained in simulators and then are operated in real networks. As a result, they inevitably suffer from the simulation-to-reality gap, showing far less satisfactory performance under real conditions compared with simulated environment. In this work, we close the gap by proposing OnRL, an online RL framework for real-time mobile video telephony. OnRL puts many individual RL agents directly into the video telephony system, which make video bitrate decisions in real-time and evolve their models over time. OnRL then aggregates these agents to form a high-level RL model that can help each individual to react to unseen network conditions. Moreover, OnRL incorporates novel mechanisms to handle the adverse impacts of inherent video traffic dynamics, and to eliminate risks of quality degradation caused by the RL model's exploration attempts. We implement OnRL on a mainstream operational video telephony system, Alibaba Taobao-live. In a month-long evaluation with 543 hours of video sessions from 151 real-world mobile users, OnRL outperforms the prior algorithms significantly, reducing video stalling rate by 14.22% while maintaining similar video quality.

Opportunistic forwarding seizes early forwarding opportunities in duty-cycled networks to reduce delay and energy consumption. But increasingly serious Cross-Technology Interference (CTI) significantly counteracts the benefits of opportunistic forwarding. Existing solutions try to reserve the channel for low-power networks by implicit avoidance or explicit coordination but ignore the potential of high-power CTI's superior capability. In this paper, we propose a new paradigm for low-power opportunistic forwarding in CTI environments. Instead of keeping high-power CTI devices silent, we directly involve them into the forwarding, as cross-technology forwarders. We design Portal to solve the challenges of realizing cross-technology opportunistic forwarding. To be transparent to the low-power networks, Portal adopts cross-technology rebroadcasting to enable the fast overhearing and forwarding of cross-technology data. To maximize the performance gain of using heterogeneous forwarders while minimizing the influence on legacy high-power traffic, we propose a post-forwarding forwarder selection and a traffic scheduling method. We also propose a feature-based ACK recognition method and a jamming-based ACK replying mechanism to forward the unreliable ACKs from asymmetric regions. Extensive experiments demonstrate that Portal not only avoids the CTI but also breaks through the existing performance limit.

As a new type of highly collaborative and shared intelligent network between producers and production environments, Industrial Internet of Things (IIOT) has been taken an important part of the fourth industrial revolution. IIOT generates large amounts of sensory data which need to be processed rapidly. However, the cloud-based data processing method consumes a long time and huge network overhead, which further affects the quality of service. On the other hand, the emerging edge computing also cannot process data efficiently because of limited compute and network resource. In this paper, we propose a four-layer network architecture based on SDN for the industrial internet of things scenario. Through effective transmission and computation coupling, the processing response efficiency is improved. We present a three-level computation offloading method to realize the optimization of network delay and power consumption. Theory and experiments show that the method proposed in this paper can effectively reduce the computation power consumption and response time.

Thanks to Industrial Internet of Things (IIoT), traditional industry is transforming to the fine and flexible production. To comprehensively control the dynamic industrial processes that includes marketing and production, accurate productivity is a vital factor that can reduce the idle operation and excessive pressure of the equipment. Due to increasing requirements of flexible control desired by IIoT, the productivity forecast also demands finer granularity. However, due to the neglect of multiple related factors and the ignorance of the multi-frequency characteristics of productivity, existing methods fail to provide accurate fine-grained productivity forecasting service for IIoT. To fill this gap, we propose a multivariate and multi-frequency Long Short-Term Memory model (mmLSTM) to predict the productivity in the granularity of day. mmLSTM takes equipment status and order as new supporting factors and leverages a multivariate LSTM to model their relationship to productivity. mmLSTM also integrate a multi-level wavelet decomposition network to thoroughly capture the multi-frequency features of productivity. We apply the proposed method in a real-world steel factory and conduct a comprehensive evaluation of performance with the productivity data in nearly two years. The result shows that our method can effectively improve the prediction accuracy and granularity of industrial productivity.

Cross-Technology Interference (CTI) badly harms the transmission reliability for low-power networks such as ZigBee at 2.4GHz band. Though promising, channel hopping still faces challenges because the increasingly dense deployment of CTI leaves very few available channels. Selecting a good channel with the least overhead is crucial but challenging. Most of the existing works are heuristic methods that choose a channel far from the current one to avoid adjacent channels that may be correlatively interfered by CTI with a wider bandwidth such as WiFi. However, we observe that the correlated channels influenced by the same CTI source do not necessarily have the same channel qualities and even the opposite state, due to the uneven spectrum power density of CTI. Such channel opportunities are unexplored and wasted. In this paper, we propose CoHop, a quantitative correlation based channel hopping method for low-power wireless networks. We establish a quantitative model that describes the correlation of channel qualities to capture channel opportunities and calculate channel quality without probing, to reduce probing overhead. We implement CoHop on TinyOS and evaluate its performance in various environments. The experimental results show that CoHop can increase the Packet Reception Ratio (PRR) by 80%, compared with existing methods.

Cross-Technology Communication (CTC) is an emerging technique that enables direct interconnection among incompatible wireless technologies. However, CTC channels established by existing methods are inherently asymmetric because of either the one-way nature of emulation in physical-level CTC or the asymmetric communication range caused by the asymmetric transmission power. In this paper, we focus on establishing symmetric CTC over asymmetric CTC channels. The bottleneck is the short communication range from the low-power and narrow-band technology to the high-power and wide-band technology because the asymmetric bandwidth and transmission power lead to serious symbol distortions. To compensate the inevitable distortions, we take advantage of the channel asymmetry and construct chirps in WiFi Channel State Information (CSI) to enhance the patterns used for conveying data. In this way, we can extend the communication range from ZigBee to WiFi. We theoretically build the model of CSI chirp based CTC and design c-Chirp, a novel CTC from ZigBee to WiFi. Due to channel asymmetry and discreteness, the WiFi receiver can only observe partial and distorted CSI chirps. To cope with this issue, we design a matching based chirp decoding method as well as an adaptation algorithm to reliably decode the symbols. We conduct extensive experiments to evaluate c-Chirp. The results show that c-Chirp can achieve a 60m communication range from ZigBee to WiFi, which is 6× longer than ZigFi, an existing representative CTC from ZigBee to WiFi.

Millimeter wave (mmWave) wireless networks are envisioned to bring a very high degree of spatial reuse, i.e., multiple links can operate concurrently without interference. The vision, however, is becoming doubtful, as recent studies found that non-negligible interference exists due to imperfect beam patterns generated by commodity mmWave radios and strong reflections. In this paper, we conduct an extensive measurement on the spatial reuse issue in a dense 60 GHz mmWave network consisting of multiple access points (AP) and users. Our measurement quantifies the impact of interference on network performance and finds that the existing prediction-based interference-resolving approaches are insufficient. Motivated by the findings, we propose mmMuxing, which enhances the spatial reuse in 60 GHz mmWave networks. Instead of relying on interference prediction, mmMuxing takes a new measurement principle of building a conflict graph that implicitly takes into account the impact of both beam imperfectness and reflections. Using the conflict graph, mmMuxing designs a joint user-beam selection algorithm, which can determine the optimal user-beam combination and lead to the minimum interference in each schedule. We prototype and evaluate mmMuxing over testbed using commodity mmWave radios. The evaluation results demonstrate that mmMuxing improves network throughput by multi-folds compared with the state-of-the-art.

The prosperity of Internet of Things (IoT) brings forth the deployment of large-scale sensing systems such as smart cities. The distributed devices upload their local sensing data to the cloud and collaborate to fulfill the large-area tasks such as pollutant diffusion analysis and target tracking. To accomplish the collaboration, time synchronization is crucial. However, due to the long range and device heterogeneity, accurate time synchronization for a large-scale IoT network is challenging. Existing GPS or NTP solutions either require an outdoor environment or only have low and unstable accuracy. In this paper, we propose AirSync, a novel synchronization method that leverages the widely existed aircraft signals, ADS-B, to synchronize large-scale IoT networks with nodes even in indoor environments. But ADS-B messages have no time stamp and cannot provide a reference time. We leverage the continuity of aircraft movements to estimate the aircraft traveling time. Then devices that observe common aircraft moving segments can calculate their time offset. To obtain the time skew, we propose a combined aircraft linear regression method. We also design a transitive synchronization for devices that cannot observe common aircraft. We implement a prototype of AirSync and evaluate its performance in various real-world environments. The results show that AirSync can obtain the sub-ms accuracy.

5G, as a monumental shift in cellular communication technology, holds tremendous potential for spurring innovations across many vertical industries, with its promised multi-Gbps speed, sub-10 ms low latency, and massive connectivity. On the other hand, as 5G has been deployed for only a few months, it is unclear how well and whether 5G can eventually meet its prospects. In this paper, we demystify operational 5G networks through a first-of-its-kind cross-layer measurement study. Our measurement focuses on four major perspectives: (i) Physical layer signal quality, coverage and hand-off performance; (ii) End-to-end throughput and latency; (iii) Quality of experience of 5G's niche applications (e.g., 4K/5.7K panoramic video telephony); (iv) Energy consumption on smartphones. The results reveal that the 5G link itself can approach Gbps throughput, but legacy TCP leads to surprisingly low capacity utilization (< 32%), latency remains too high to support tactile applications and power consumption escalates to 2 - 3x over 4G. Our analysis suggests that the wireline paths, upper-layer protocols, computing and radio hardware architecture need to co-evolve with 5G to form an ecosystem, in order to fully unleash its potential.

Information-centric networking (ICN) is regarded as a promising architecture for Internet of Things (ICN-IoT) and access control is one of the critical problems to enable secure ICN-IoT. This article proposes high efficient access control (HAC), a high efficient access control system for ICN-IoT. Specifically, HAC enables access control via an elaborate designed hierarchical key tree (HKT) mechanism based on the hierarchical naming scheme of ICN. The proposed HKT contains the hierarchical authority information and allow users to locally derive the key according to their needs, thus greatly decreasing the overhead in IoT's many-to-many communication scenario. To ensure the security and efficiency of HKT, we further propose a level-oriented ciphertext policy attribute-based encryption (LOCP-ABE) algorithm such that users can only obtain the authority level according to their attributes, and also utilize the ICN's receiver-driven model and the in-network caching mechanism to speed up the distribution efficiency. Moreover, an attribute-based command verification mechanism is used to improve the efficiency of command verification for resource-constrained and isolated IoT edge. We evaluate the proposed HAC by theoretical security analysis and real-world experiments. The theoretical analysis proves that the proposed HAC is secure and experiment results show that HAC can greatly improve the access control efficiency in ICN-IoT compared with the state of the art.

Different from online promotion, the outdoor billboard advertising industry suffers from a lack of audience-targeted delivery and quantitative dissemination evaluation, which undermine its impact in practice and hinder it from fast development. To bridge this gap, in this paper, we leverage crowdsensing vehicle trajectory data to empower audience-targeted billboard advertising. More specifically, by integrating the information of mobility transition, traffic conditions (traffic volume and average speed), and advertisement semantic topics, we propose a quantitative model to quantify advertisement influence spread, with a special consideration on influence overlapping among mobile users. Based on it, an influence maximization-targeted billboard advertising problem is formulated to find k advertising units over spatiotemporal dimensions, with the goal of maximizing the total expected advertisement influence spread. To tackle the efficiency issue for solving large combinatorial optimization problem, we employ a divide-and-conquer mechanism, and propose a utility evaluation-based optimal searching approach. Extensive experiments on real-world taxicab trajectories clearly validate the effectiveness and efficiency of our proposed approach.

The emerging millimeter-wave (mmWave) networking technology promises to unleash a new wave of multi-Gbps wireless applications. However, due to high directionality of the mmWave radios, maintaining stable link connection remains an open problem. Users' slight orientation change, coupled with motion and blockage, can easily disconnect the link. In this paper, we propose RoMil, a robotic mmWave relay that optimizes network coverage through wireless sensing and autonomous motion/rotation planning. The robot relay automatically constructs the geometry/reflectivity of the environment, by estimating the geometries of all signal paths. It then navigates itself along an optimal moving trajectory, and ensures continuous connectivity for the client despite environment/human dynamics. We have prototyped RoMil on a programmable robot carrying a commodity 60 GHz radio. Our field trials demonstrate that RoMil can achieve nearly full coverage in dynamic environment, even with constrained speed and mobility region.

Building an open global sensing layer is critical for the Internet of Things (IoT). In this article, we present a Sensory Data-Centric Networking (SDCN) architecture for inter-networking two main networked sensing systems in IoT—wireless sensor networks and mobile sensing networks. Specifically, the proposed SDCN is a systematic solution including NDNs for sensor nodes in the Zigbee network, NDNm for mobilephones in the Wi-Fi network, and NDNg for gateways. Considering the sensing requirement of IoT, we first design a novel Spatio-Temporal 16 Tree (ST16T) naming scheme associated with the scope-matching method. Based on the naming scheme, we further propose the related discovery methods, network switching mechanism, forwarding, and routing strategies according to the features of large-scale sensing and resource-constrained environment. A proof-of-concept prototype is implemented and further is deployed on our campus (BUPT) and the Great Wall (Shaanxi, China) for Environment Monitoring Project. Several experiments are conducted on the deployed platform. The experimental results show that SDCN outperforms the state-of-the-arts and gains a great performance improvement in terms of energy consumption, data collection efficiency, memory footprint, and time delay.

Vital nodes identification, that is, finding a set of nodes whose absence would cause a collapse of the network, is a significant project in network science. Despite there are a plenty of methods to identify the vital nodes, two major problems still need to be solved, that is how to select these nodes and how to determine the number of them. In this study, we focus on dealing with these two problems via proposing a multiobjective memetic algorithm for vital nodes identification task. First, vital nodes identification task is modeled as a biobjective optimization problem by analyzing the characteristic of vital nodes. Then, a memetic strategy and specific evolutionary operators inspired by multiple centralities are designed to execute local and global search. In addition, a long-tail property is found from the Pareto front of this bi-objective optimization problem, and the simulation results always show an obvious knee region. Hence, an adaptive learning method to determine the size of vital nodes is designed by searching for the knee point. At last, the proposed framework is tested on the scale free networks and real-life networks, and the simulation results validate its effectivity in contrast with the state-of-art greedy methods.

Google congestion control (GCC) is the de-facto standard for web real-time communications (WebRTC) applications and has been implemented in mainstream browsers including Chrome and Firefox. While GCC is designed to achieve high video bit-rate and low latency simultaneously, we find that GCC's performance is far from satisfactory particularly under good network conditions. In particular, we collect a GCC trace dataset with over 1.18 million sessions from a major crowd-sourced live video streaming service provider. We perform in-depth analytics using the dataset, which shows that the sending video bit-rate unnecessarily experiences frequent rollbacks caused by minor fluctuation of transmission delay. To address the issue, we propose a mechanism called GCC-β, which can distinguish normal network fluctuation from real network congestion, and then adaptively sends appropriate bitrates. We implement GCC-β in the WebRTC framework and evaluate its performance using test-bed experiments. The results show that GCC-β is able to avoid up to 90% unnecessary bitrate rollbacks.

In this paper, we focus on the coverage and capacity analysis for LEO satellite network supporting Internet of Things. We propose a metric of coverage degree to measure average cover level during a return circle of a satellite constellation in different latitudes, and furter consider the spot beam communication process with TDMA as a discrete time queueing process to caculate the number of devices in a unit area, which can be accessed during a period. Finally, we derive the relationship among device density, maximum tolerant delay, and coverage degree of the satellite constellation. The obtained theoretical results are verified by extensive simulation experiments.

Weakly Supervised Object Detection (WSOD) is a challenging visual understanding task due to the absence of expensive human annotations like bounding boxes and segmentation. Recent WSOD methods usually generate a bounding box for the most contrastive part of the object rather than the entire object. Some methods alleviate the incompleteness problem of detection object with segmentation supplement or classifier refinement. However, segmentation requires high model cost and classifier refinement highly relies on the quality of initial candidate boxes at the beginning. In this paper, we propose a WSOD classifier refinement approach to overcome candidate boxes initialization and high segmentation model cost problems. Our approach can get high-quality class-specific activation maps for the objects and generate nail boxes at the maximum response point of the activation map to suppress incorrect refinement direction. Compared with previous methods, our WSOD classifier refinement approach can achieve 42.1% mAP on PASCAL VOC 2007 benchmarks without high segmentation model cost.

Discovering social relations in images can make machines better interpret the behavior of human beings. However, automatically recognizing social relations in images is a challenging task due to the significant gap between the domains of visual content and social relation. Existing studies separately process various features such as faces expressions, body appearance, and contextual objects, thus they cannot comprehensively capture the multi-granularity semantics, such as scenes, regional cues of persons, and interactions among persons and objects. To bridge the domain gap, we propose a Multi-Granularity Reasoning framework for social relation recognition from images. The global knowledge and mid-level details are learned from the whole scene and the regions of persons and objects, respectively. Most importantly, we explore the fine-granularity pose keypoints of persons to discover the interactions among persons and objects. Specifically, the pose-guided Person-Object Graph and Person-Pose Graph are proposed to model the actions from persons to object and the interactions between paired persons, respectively. Based on the graphs, social relation reasoning is performed by graph convolutional networks. Finally, the global features and reasoned knowledge are integrated as a comprehensive representation for social relation recognition. Extensive experiments on two public datasets show the effectiveness of the proposed framework.

Affective computing has attracted researchers' attention in recent years. Emotion recognition is part of affective computing, which aims to recognize how the person feels, such as happy, sad, anger, disgust, fear, surprise. Traditional works about emotion recognition mainly focus on the characteristic of the person itself, such as audio, text, facial expression, body posture. However, the feelings of people can easily be affected by the context information. In this paper, we utilize the context to construct an affective graph to reason the emotional states. In detail, we detect the context using Region Proposal Network (RPN) to extract nodes as the input of the Graph Convolution Network (GCN), which transfers the convolution operation from Euclidean data structure to nonEuclidean data structure. The GCN learns the affective relationship during the back-propagation process. Moreover, the body feature is extracted by Convolution Neural Network (CNN). The output of GCN and CNN are combined finally to infer the discrete emotion categories and the continuous dimensions for VAD (Valence, Arouse, Dominance) measurement. Our method achieves higher performance than the baseline based on the EXOTIC dataset.

IoT device are usually built as edge networks in smaller areas, but with the development of technology, more and more areas without infrastructure need to be covered. LEO satellites, due to its full coverage of the Earth and relatively low latency, can provide network access for IoT system. In this paper, we propose a hybrid TDMA MAC protocol LST-MAC for IoT network based on LEO satellite, which can reduce network delay and improve the system scalability. Specifically, we design a geographic priority time slot allocation strategy which can ensure optimal load and delay, besides, we also design a continuous forward time slot allocation strategy, which guarantees real-time data while reducing the utilization of satellite resources. Then we design two algorithms based on both strategies, which can effectively reduce network delay and balance the satellite resources. The simulation experimental results are consistent with the theoretical results, which embodies the effectiveness of the algorithm.

Low Earth Orbit (LEO) satellite networks play an indispensable role in global communications. To cope with the dynamic nature of the LEO networks, a flexible management architecture that can provide low-latency configuration and routing services is desired. Recent advances of Software Defined Networks (SDN) inspire researches using geostationary satellites, GEO, as the controllers to build the SDN solutions for LEO satellite networks. However, GEO-based solutions inevitably face the bottleneck problem because all the routing requests have to be processed and forwarded by a limited number of GEO satellites. In this paper, we propose HTCA, a Hierarchical Terrestrial Controllers Architecture based on SDN that reuses the ground stations instead of dedicated GEO satellites to establish a more scalable control plane. But the limited coverage of a ground station brings about design challenges for consistent network management and seamless routing service. HTCA adopts an online network view integration method to support flexible and consistent management. A load-aware routing method is also designed for HTCA to provide seamless and low-latency routing service. The experiment results demonstrate that HTCA can achieve agile configuration and reduce the instruction update time by 86.87%, compared with the GEO-based solution.

Serverless computing has emerged as a new compelling paradigm for the deployment of applications and services, which enables developers to focus more on business logic rather than on infrastructure. Serverless computing platform enables the function container scales to zero, which results in a serious problem called cold start. Cold start severely affects the responsiveness of serverless computing platform and limits the use and adoption of serverless computing by a broader range of applications. The traditional strategies reduce the cold start latency at the expense of resources. How to simultaneously minimize the cold start latency and reduce the resources consumption of strategy implementation is a challenging problem. In this paper, we firstly propose an Adaptive Warm-Up Strategy (AWUS) to predict the function invoking time and warm up the functions, thus reducing the cold start latency. We use the function chain model to improve the AWUS. We adopt a fine-grained regression method to predict non-first functions in the function chain more accurately. Secondly, we propose an Adaptive Container Pool Scaling Strategy (ACPSS) to reduce the function launching time. We dynamically adjust the capacity of the container pool to reduce the resources waste. The AWUS and ACPSS work together to reduce the cold start latency and the resources waste. Finally, we implement a serverless computing platform and conduct extensive experiments to evaluate our strategy. The evaluation results demonstrate the effectiveness of our strategies.

Nowadays we are witnessing an explosive growth of mobile data traffic, but users still often experience insufficient or unstable network bandwidth in many realistic scenarios. Unmanned aerial vehicle mounted base stations (UAV-BSs) provide a novel and promising solution for serving regions with bandwidth shortfall. It is significant to investigate how to deploy UAVs for maximizing the sum throughput of a set of clients scattered in various locations in an effective and efficient way. A basic idea is to use RF ray tracing simulations as a hint to narrow down the search space of UAVs for conducting measurements. Furthermore, we study two key sub-problems, chunk selection, which finds an optimal subset of chunks in the region as the search space of UAVs, and chunk search, which plans the scanning path to cover the search space with the min-max time consumption required for all UAVs. They are both proved to be NP-hard, and heuristic algorithms are proposed to solve them efficiently. A prototype system, UAV-Net, is implemented to conduct measurements by a UAV mounted WiFi AP communicating with 7 clients scattered in a campus, and extensive simulations are combined, reporting an obvious throughput gain with a small measurement overhead and time consumption.

Ambient environment information, including reflectors' location, dimension and reflectivity, is a key input to many millimeter-wave (mmWave) networking and sensing applications. It has found versatile applications in optimizing network coverage and robustness, enhancing mobile link performance, and enabling high-accuracy indoor localization and navigation. Recent approaches of deriving mmWave environment information require heavy infrastructure support or non-trivial human labor, and rely on costly software defined radios, which prevent their usage in practice. In this work, we design and implement mmRanger, a system can automatically sense environment without any infrastructure support. mmRanger equips a pair of low-cost off-the-shelf mmWave radios in a commodity robot, which constantly samples the ambient environment by exchanging a series of mmWave signals while it moves and rotates. mmRanger then re-engineers the time-domain signal series to derive the spatial-domain environment structure, through novel reflection path extraction and reflector mapping algorithms. Our experiments verify that mmRanger can accurately sense a given environment with minimal overhead, and the learned information can bring 1.6× and 2.1× performance gain, in terms of network coverage and mobile link throughput, respectively, over empirical approaches in mmWave networks.

Anomaly detection in surveillance videos, as a special case of video-based action recognition, is an important topic in multimedia community and public security. Currently, most of the state-of-the-art methods utilize deep learning to recognize the patterns of anomaly or action. However, whether deep neural networks really learn the essence of the anomaly or just remember the background is an important but often neglected problem. In this paper, we develop a series of experiments to validate the existence of background-bias phenomenon, which makes deep networks tend to learn the background information rather than the pattern of anomalies to recognize abnormal behavior. To solve it, we first re-annotate the largest anomaly detection dataset and design a new evaluation metric to measure whether the models really learn the essence of anomalies. Then, we propose an end-to-end trainable, anomaly-area guided framework, where we design a novel region loss to explicitly drive the network to learn where is anomalous region. Besides, given very deep networks and scarce training data for anomaly, our architecture is trained with a meta learning module to prevent severe overfitting. Extensive experiments on the benchmark show that our approach outperforms other methods on both the previous and our proposed evaluation metrics through reducing the influence of the background information.

Despite the pervasive use of real-time video telephony services, the users' quality of experience (QoE) remains unsatisfactory, especially over the mobile Internet. Previous work studied the problem via controlled experiments, while a systematic and in-depth investigation in the wild is still missing. To bridge the gap, we conduct a large-scale measurement campaign on \appname, an operational mobile video telephony service. Our measurement logs fine-grained performance metrics over 1 million video call sessions. Our analysis shows that the application-layer video codec and transport-layer protocols remain highly uncoordinated, which represents one major reason for the low QoE. We thus propose \name, a machine learning based framework to resolve the issue. Instead of blindly following the transport layer's estimation of network capacity, \name reviews historical logs of both layers, and extracts high-level features of codec/network dynamics, based on which it determines the highest bitrates for forthcoming video frames without incurring congestion. To attain the ability, we train \name with the aforementioned massive data traces using a custom-designed imitation learning algorithm, which enables \name to learn from past experience. We have implemented and incorporated \name into \appname. Our experiments show that \name outperforms state-of-the-art solutions, improving video quality while reducing stalling time by multi-folds under various practical scenarios.

Despite the pervasive use of real-time video telephony services, their quality of experience (QoE) remains unsatisfactory, especially over the mobile Internet. We conduct a large-scale measurement campaign on \appname, an operational mobile video telephony service. Our analysis shows that the application-layer video codec and transport-layer protocols remain highly uncoordinated, which represents one major reason for the low QoE. We thus propose \name, a machine learning based framework to resolve the issue. We train \name with the massive data traces from the measurement campaign using a custom-designed imitation learning algorithm, which enables \name to learn from past experience following an expert's iterative demonstration/supervision. We have implemented and incorporated \name into the \appname. Our experiments show that \name outperforms state-of-the-art solutions, improving video quality while reducing stalling time by multi-folds under various practical scenarios.

The emerging millimeter-wave (mmWave) networking technology promises to unleash a new wave of multi-Gbps wireless applications. However, due to high directionality of the mmWave radios, maintaining stable link connection remains an open problem. Users' slight orientation change, coupled with motion and blockage, can easily disconnect the link. In this paper, we propose miDroid, a robotic mmWave relay that optimizes network coverage through wireless sensing and autonomous motion/rotation planning. The robot relay automatically constructs the geometry/reflectivity of the environment, by estimating the geometries of all signal paths. It then navigates itself along an optimal moving trajectory, and ensures continuous connectivity for the client despite environment/human dynamics. We have prototyped miDroid on a programmable robot carrying a commodity 60 GHz radio. Our field trials demonstrate that miDroid can achieve nearly full coverage in dynamic environment, even with constrained speed and mobility region.

Unmanned Aerial Vehicle (UAV) provides an effective way to create an Aerial Panoramic Map (APM). Generally, it consists of three steps: 1) select a set of locations from a map, 2) take photos from different angles at each selected location one by one, and 3) make panoramic images by the Panoramic Mosaic technology, and then create an APM with these images. However, it is always labor-intensive and time-consuming to complete these steps in a large region such as a campus and a park, due to multiple reasons: 1) inexperience for location selection, 2) low-efficiency for manually operating UAV to fly among different locations one by one and make photos from different angles, and 3) limited energy supply and low-efficiency for a single UAV. DJI GO[1] has been developed to simplify a part of operations in steps 2) and 3), by which we only need one button to take photos automatically from different angles at a selected location. However, it is still required to manually select locations and control UAV to fly among different locations. Moreover, the defects of using a single UAV still exist. By contrast, we aim to design an Auto nomously C ooperative U AV system platform for P anoramic map generation, AutoCUP, which leverages multiple UAVs to full-automatically and high-efficiently complete all steps of creating an APM.

Sweep coverage uses mobile nodes for monitoring Points of Interests (PoIs) in a sensing field. With sweep coverage, information can be effectively gathered without using a lot of static sensors. In this paper, we study the sweep coverage problem with sensing and transmission delay constraints, which is regarded as the Timely Sweep Coverage problem. Specifically, we investigate how to use the minimum number of mobile nodes to cover all PoIs under the two delay constraints. We propose two heuristic algorithms, MR-MinExpand and CoTSweep, to provide timely sweep coverage under different scenarios. MR-MinExpand leverages the difference between sensing and transmission delay constraints to schedule the route for each mobile node. CoTSweep considers the scenario where some PoIs cannot be covered by a single mobile node, and leverages the collaboration between mobile nodes to enable the sink node to collect data from remote PoIs. Extensive simulations and comparisons with previous works are conducted to validate the advantages of our algorithms.

This article proposes IoTCache, a popularity-based caching solution for Internet of Things (IoT). We first build a large popularity dataset that reflects peoples interest and request pattern for IoT data. By analyzing the popularity features of the dataset, we propose the popularity evolving model (PEM) for answering the fundamental problem that what is the popularity pattern of IoT data? Then, we design a data-driven popularity prediction method, which consists of two parts: 1) deep neural network (DNN)-based PEM for generating predicted popularity and 2) statistic-based PEM for dealing with the cold boot problem. Furthermore, we present a popularity-based evicting and prefetching algorithm to address what to cache and when to cache problems. We evaluate IoTCache on two IoT platforms which are on the basis of content delivery network (CDN) and information-centric network (ICN), respectively. The experimental results show that IoTCache can significantly increase the cache hit ratio, and decrease the IoT edge traffic and data latency.

Efficient typing or text-input on mobile devices, such as smartphones and wearables is a long-standing problem, due to the miniature touchscreen on the devices. Recently, touchscreen-free solutions leveraging on acoustic sensing have been proposed, with the advantage of low cost and ubiquitous availability. However, existing solutions usually require people to write in print-style, and more importantly, they are highly vulnerable to environmental change, i.e., they need repetitive training upon slight deviation of writing places or device locations. Therefore, they are far from practical usage. In this paper, we propose a novel acoustic-based text-input system called UbiWriter, which can recognize freestyle handwriting with high ubiquity, i.e., one-time training and writing elsewhere. UbiWriter is built on a new letter recognition principle, which treats the acoustic signal from writing a letter as a complete trajectory, and then distills the recognition feature that is resilient to environmental change. For the actual realization of the principle, we adopt and incorporate a series of techniques, including a feature-preserved fast letter alignment, ${K}$ -nearest neighbor letter classification, and language structure-driven word recognition. We also design and implement an APP with cloud-computing support, in order to facilitate real-time text input. Extensive experimental results demonstrate that UbiWriter outperforms the state-of-the-art under various practical settings.

Mobile augmented reality (Mobile AR) is gaining increasing attention from both academia and industry. Hardware-based Mobile AR and App-based Mobile AR are the two dominant platforms for Mobile AR applications. However, hardware-based Mobile AR implementation is known to be costly and lacks flexibility, while the App-based one requires additional downloading and installation in advance and is inconvenient for cross-platform deployment. In comparison, Web-based AR (Web AR) implementation can provide a pervasive Mobile AR experience to users thanks to the many successful deployments of the Web as a lightweight and cross-platform service provisioning platform. Furthermore, the emergence of 5G mobile communication networks has the potential to enhance the communication efficiency of Mobile AR dense computing in the Web-based approach. We conjecture that Web AR will deliver an innovative technology to enrich our ways of interacting with the physical (and cyber) world around us. This paper reviews the state-of-the-art technology and existing implementations of Mobile AR, as well as enabling technologies and challenges when AR meets the Web. Furthermore, we elaborate on the different potential Web AR provisioning approaches, especially the adaptive and scalable collaborative distributed solution which adopts the osmotic computing paradigm to provide Web AR services. We conclude this paper with the discussions of open challenges and research directions under current 3G/4G networks and the future 5G networks. We hope that this paper will help researchers and developers to gain a better understanding of the state of the research and development in Web AR and at the same time stimulate more research interest and effort on delivering life-enriching Web AR experiences to the fast-growing mobile and wireless business and consumer industry of the 21st century.

This paper builds a Food Delivery Network (FooDNet in short) using spatial crowdsourcing (SC). It investigates the participation of urban taxis to support on demand take-out food delivery. Unlike existing SC-enabled service sharing systems (e.g., ridesharing), the delivery of food in FooDNet is more time-sensitive and the optimization problem is more complex regarding high-efficiency, huge-number of delivery needs. In particular, two on demand food delivery problems under different situations are studied in our work: (1) for O-OTOD, the food is opportunistically delivered by taxis when carrying passengers, and the optimization goal is to minimize the number of selected taxis to maintain a relatively high incentive to the participated drivers; (2) for D-OTOD, taxis dedicatedly deliver food without taking passengers, and the aim is to minimize the number of selected taxis (i.e., to raise the reward for each participant) and the total traveling distance to reduce the cost. A two-stage approach, including the construction algorithm and the Adaptive Large Neighborhood Search (ALNS) algorithm based on simulated annealing, is proposed to solve the problem. We have conducted extensive experiments based on the real-world datasets, including city-wide restaurant data, cell tower data, and the large-scale taxi trajectory data with 10,000 taxis in the city of Chengdu, China. Experimental results demonstrate that our proposed algorithms are more effective and efficient than baselines, fulfilling the food delivery service using a smaller number of taxis within the given time.

With the increasing popularity of video delivery among mobile users, the problem of explosive traffic growth becomes more and more serious for the mobile wireless networks. To deal with this problem, in this paper we propose to exploit opportunistic transmission and the idea of crowdsourcing to offload mobile video traffic. Based on this idea, a Quality of VIDEO (QoV) oriented pricing incentive scheme, namely Vbargain, is proposed to stimulate mobile users to deliver video data collaboratively. More specifically, in our scheme, the network is regarded as a Virtual Market, and the video packets are treated as commodities, which are dynamically priced according to their expected marginal gains on the quality of reconstructed video; the process of video delivery is regarded as a sequence of packet transactions which are modeled as two-person cooperative games. Driven by the profit-hunting nature of mobile users, the video data is delivered from the source to its destination. In order to gain deep insight into our scheme, we also analyze the overhead and the influence of start-up capital on the performance of video delivered theoretically, and give its lower bound and upper bound. Our simulation results, based on both the synthetic and real-life traces of mobile users, verify the efficiency of our scheme and analysis.

Multi-user MIMO (MU-MIMO) can serve multiple users concurrently, and is the key technology to enable ultra-high-speed wireless access. However, in practice MU-MIMO networks are far from their full potential due to the poor scalability problem, including high computational complexity at PHY layer and large-overhead channel contention at MAC layer. Moreover, cross-cell interference among multiple MU-MIMO cells also counteracts network performance. In this paper, we perform a systematic study on MU-MIMO and propose a fully scalable MU-MIMO user selection protocol called Guidepost. In contrast with previous works, Guidepost builds on a novel principle of indirection channel orthogonality evaluation, so as to decouple and simplify the complicated computational/contention interaction among users. Based on the principle, Guidepost first achieves scalable MU-MIMO user selection with only linear computational complexity. Second, Guidepost realizes distributed user selection through a two-dimensional prioritized contention mechanism, which can single out the best concurrent users efficiently by utilizing both the time and frequency domain resources. Third, Guidepost incorporates a lightweight AP-assisted contention mechanism to handle cross-cell interference in distributed MU-MIMO (netMIMO) where users are widely distributed and cannot sense each other. Software-radio based implementation and experimentation show that Guidepost significantly outperforms state-of-the-art methods under various traffic patterns and node mobility.

Gait recognition is an attractive human recognition technology. However, existing gait recognition methods mainly focus on the regular gait cycles, which ignore the irregular situation. In real-world surveillance, human gait is almost irregular, which contains arbitrary dynamic characteristics (e.g., duration, speed, and phase) and varied viewpoints. In this paper, we propose the attentive spatial-temporal summary networks to learn salient spatial-temporal and view-independence features for irregular gait recognition. First of all, we design the gate mechanism with attentive spatial-temporal summary to extract the discriminative sequence-level features for representing the periodic motion cues of irregular gait sequences. The designed general attention and residual attention components can concentrate on the discriminative identity-related semantic regions from the spatial feature maps. The proposed attentive temporal summary component can automatically assign adaptive attention to enhance the discriminative gait timesteps and suppress the redundant ones. Furthermore, to improve the accuracy of cross-view gait recognition, we combine the Siamese structure and Null Foley-Sammon transform to obtain the view-invariant gait features from irregular gait sequences. Finally, we quantitatively evaluate the impact of the irregular gait and viewpoint interval between matching pairs on gait recognition accuracy. Experimental results show that our method achieves state-of-the-art performance in irregular gait recognition on the OULP and CASIA-B datasets.

With the remarkable proliferation of smart mobile devices, mobile crowdsensing has emerged as a compelling paradigm to collect and share sensor data from surrounding environment. In many application scenarios, due to unavailable wireless network or expensive data transfer cost, it is desirable to offload crowdsensing data traffic on opportunistic device-to-device (D2D) networks. However, coupling between mobile crowdsensing and D2D networks, it raises new technical challenges caused by intermittent routing and indeterminate settings. Considering the operations of data sensing, relaying, aggregating, and uploading simultaneously, in this article, we study collaborative mobile crowdsensing in opportunistic D2D networks. Toward the concerns of sensing data quality, network performance and incentive budget, Minimum-Delay-Maximum-Coverage (MDMC) problem and Minimum-Overhead-Maximum-Coverage (MOMC) problem are formalized to optimally search a complete set of crowdsensing task execution schemes over user, temporal, and spatial three dimensions. By exploiting mobility traces of users, we propose an unified graph-based problem representation framework and transform MDMC and MOMC problems to a connection routing searching problem on weighted directed graphs. Greedy-based recursive optimization approaches are proposed to address the two problems with a divide-and-conquer mode. Empirical evaluation on both real-world and synthetic datasets validates the effectiveness and efficiency of our proposed approaches.

Video-based action recognition with deep neural networks has shown remarkable progress. However, most of the existing approaches are too computationally expensive due to the complex network architecture. To address these problems, we propose a new real-time action recognition architecture, called Temporal Convolutional 3D Network (T-C3D), which learns video action representations in a hierarchical multi-granularity manner. Specifically, we combine a residual 3D convolutional neural network which captures complementary information on the appearance of a single frame and the motion between consecutive frames with a new temporal encoding method to explore the temporal dynamics of the whole video. Thus heavy calculations are avoided when doing the inference, which enables the method to be capable of real-time processing. On two challenging benchmark datasets, UCF101 and HMDB51, our method is significantly better than state-of-the-art real-time methods by over 5.4% in terms of accuracy and 2 times faster in terms of inference speed (969 frames per second), demonstrating comparable recognition performance to the state-of-the-art methods. The source code for the complete system as well as the pre-trained models are publicly available at https://github.com/tc3d.

Sparse sensory data caused by insufficient monitoring sites and their incomplete records becomes the main challenge of fine-grained environment sensing. In this paper, we develop a novel inference framework, named Federated Region- Learning (FRL), for urban environment sensing. The proposed framework inherits the basic idea of federated learning, and also considers the regional characteristics during the distribution of training samples so as to improve the inference accuracy. Moreover, we exploit an edge computing architecture to implement the FRL for improving the computational efficiency. We also apply FRL to PM2.5 monitoring in Beijing. The evaluation shows that our FRL improves computational efficiency nearly 3 times than centralized training mode and increases accuracy by more than 5% compared with normal distributed training.

With the explosive growth of mobile devices such as smartphones, it is convenient for participants to perform mobile crowd sensing (MCS) tasks. It is a useful way to recruit participants to perform location-dependent tasks. We first propose Min-Max Task (MMT) planning problem in MCS systems, considering time-sensitivity and heterogeneity of sensing tasks. In other words, how to design a cooperation scheme, in which the participants spend as little time as possible. Then, to address MMT problem, we propose a Memetic based Bidirectional General Variable Neighborhood (MBGVN) algorithm, in which all tasks are separated into groups and traveling path is designed for each participant. Finally, extensive experiments are conducted to demonstrate the benefits of our scheme, outperforming other similar state-of-the-art algorithms.

License plate recognition (LPR) plays an important role in intelligent transport systems. The existed LPR systems are mostly based on hand-crafted methods for detection, segmentation, and recognition, which cannot accurately recognize the license plate in unconstrained surveillance environments. In this paper, we propose a Multi-Task Generative Adversarial Network (MTGAN) based LPR system, which combines the license plate super-resolution and recognition in one end-to-end framework. In the proposed MTGAN, we design a Fully Connected Network (FCN) as generative network (GN), which can combine knowledge from data distribution and domain prior knowledge of license plate to generate the spatial corresponding and high-resolution plate images in the synthesis pipeline. More important, a multi-task discriminative network is designed in MTGAN to combine the super-resolution and recognition in an adversarial manner to enhance each other. The experiments on the built real-world license plate dataset show that the proposed LPR system can generate high-resolution license plates as well as recognize them with higher accuracy than state-of-the-art LPR systems.

Crowdsourcing provides a novel and effective way of constructing a location image database for mobile visual location recognition. Compared with traditional location image databases, a crowdsourced database has richer information for location images, with various angles, times, distances and weathers, providing great potential for high recognition accuracy. However, it is inevitable to have various disturbances on these location images, hindering the potential. To address this challenge, we first propose a Common Crucial Feature (CCF) detection algorithm to exclude unimportant visual features from crucial features. To achieve a good balance between the efficiency and accuracy, we further propose a CCF based Visual Hash Bits (VHB) scheme to encode CCF features into hash bits to vote for most matching images. Extensive experiments are conducted on a crowdsourced dataset with 9,064 location images, demonstrating that our scheme outperforms other state-of-the-art schemes.

Cross-view gait recognition is a challenging problem when view-interval and pose variation are relatively large. In this paper, we propose Cycle-consistent Attentive Generative Adversarial Networks (CA-GAN) to map different views' gait images to view-consistent and photorealistic gait images for cross-view gait recognition. In CA-GAN, the generative network is composed of two branches, which simultaneously perceives human's global contexts and local body parts information respectively. Moreover, we design a novel Attentive Adversarial Network (AAN) to adaptively learn different weights for the discriminator's receptive fields with attention mechanism. Furthermore, as it is hard to collect the pose-aligned gait image pairs from different views for training CA-GAN’ we combine forward cycle-consistency loss and adver-sarial loss to learn the transformation relationship from source views to target view. The combined loss function can also preserve the discriminative gait structures of different identities at the training stage. Finally, we directly exploit the synthesized view-consistent gait images for cross-view gait recognition task. Experimental results on CASIA-B demonstrate that our method not only outperforms the state-of-the-art methods in cross-view gait recognition, but also presents compelling perceptual results even across the large view-interval.

Unmanned Aerial Vehicles (UAVs) have been widely used in various applications such as inspection, security surveillance, and aerial photography, in which the cooperation of multiple UAVs is significantly important for better accomplishing complex tasks due to the limited capability for individual UAV s. Task planning is the primary issue for the cooperation of multiple UAV s, and has attracted extensive research interests. However, most research fails to account adequately for limited energy on each UAV, which involves in many factors such as different operations for performing a task and various movement patterns besides the distance and turns that have been commonly considered. By contrast, we conduct a series of experiments to obtain the energy model of UAV s. Furthermore, we focus on the energy-efficient min-max task planning (E2M2TP) problem by considering the heterogeneity of tasks and integrating various energy factors, which is beneficial for balancing the workload and energy consumption among UAV s and thus reducing the number of required UAVs. We show that E2M2TP is NP-hard, and propose an energy-aware variable neighbor search (EVNS) algorithm to iteratively optimize both task allocation and path planning. Extensive simulations are conducted to validate that EVNS outperforms the other state-of-the-art algorithms.

Named Data Networking (NDN), a revolution of the IP architecture, provides an information-centric paradigm to transmit video traffic which occupies most of the Internet traffic. However, the control schemes in the field of video NDN research can easily result in full bottleneck bandwidth at the cost of high video delay and frequent packet loss. This is because the loss-based or transient-queue-delay-based congestion control schemes respond to the signal after the congestion is formed. And NDN multi-source feature can't make the loss and delay-based signals accurately reflect the network situation. In addition, the current video applications need to choose the suitable video rate to stream across the network. The bandwidth estimation scheme of video rate control ignoring multi-source reduces accuracy in NDN. In this paper, we firstly build a max-plus algebra based model to characterize the interest-data transmission process for video consumer and to estimate the NDN bandwidth delay product. The hops (lived time)that data pass through from the producer to the consumer represent the multi-source feature in the model. Then we propose a Time-Lived based congestion and rate control scheme for the NDN video consumer. Extensive experiments have been conducted to evaluate our scheme from the perspective of throughput, delay, and queue length. The results show that our scheme can improve the throughput of the video consumer while reducing the round trip time by shortening the queue length.

Hybrid CPU-GPU cluster has become a promising computing paradigm for large-scale video analytics. However, the uncertainty and variability of workloads and heterogeneous resources in the cluster can lead to the unbalanced use of the hybrid computing resources and further cause the performance degradation of the computing platform. This problem becomes more challenging with the computation complexity and dependencies of video tasks in the hybrid cluster. In this paper, we focus on the video workload parallelization problem with fine-grained task division and feature description in the hybrid CPU-GPU cluster. Firstly, for achieving high resource utilization and task throughput, we propose a two-stage video task scheduling approach based on deep reinforcement learning. In our approach, a task execution node is selected by the cluster-level scheduler for the mutually independent video tasks, and then the node-level scheduler assigns the interrelated video subtasks to the appropriate computing units. By using the deep Q-network, the two-stage scheduling model is online learned to perform the current optimal scheduling actions according to the runtime status of cluster environments, the characteristics of video tasks, and the dependencies between video tasks. Secondly, based on the transfer learning technology, a scheduling strategy generalization method is proposed to efficiently rebuild the task scheduling model referring to the existing model. Finally, we conduct the extensive experiments to analyze the impact of the model parameters on the scheduling actions, and then the experimental results also validate that our learning based task scheduling approach outperforms the other widely used methods.

Currently, license plate recognition plays an important role in numerous applications and a number of technologies have been proposed. However, most of them can only work with single-line license plates. In the practical application scenarios, there are also existing many multi-line license plates. The traditional approaches need to segment the original input images for double-line license plates. This is a very difficult problem in the complex scenes. In order to solve this problem, we propose an end-to-end neural network for both single-line and double-line license plate recognition. It is segmentation-free for the original input license plate images. We view each of these whole images as a unit on feature maps after deep convolution neural network directly. A large number of experiments show that our method is effective. It is better than the state-of-the-art algorithms in SYSU-ITS license plate library data.

Microservice based cloud architecture becomes a promising solution to deal with the challenges of large-scale intelligent video applications. However, the current service selection methods usually do not consider both the fine-grained online service capability and the features of video tasks, and this will result in the degradation of the overall efficiency of the service composition. In this paper, we propose a novel Performance-aware Service PAth Selection (PSPAS) approach for the microservice based video cloud computing platform. Firstly, we establish a fine-grained time estimation model which synthetically considers the processing capability of microservice instances, the characteristics of video processing tasks, and the data transfer conditions between microservice instances. Then, based on the proposed performance model, we search and update the optimal microservice path by using the shortest path algorithm. Finally, the experiment evaluation results demonstrate the effectiveness of our method.

60 GHz networks, with multi-Gbps bitrate, are considered as the enabling technology for emerging applications such as wireless Virtual Reality (VR) and 4K/8K real-time Miracast. However, user motion, and even orientation change, can cause mis-alignment between 60 GHz transceivers' directional beams, thus causing severe link outage. Within the practical 3D spaces, the combination of location and orientation dynamics leads to exponential growth of beam searching complexity, which substantially exacerbates the outage and hinders fast recovery. In this paper, we first conduct an extensive measurement to analyze the impact of 3D motion on 60 GHz link performance, in the context of VR and Miracast applications. We find that 3D motion exhibits inherent non-predictability, so conventional beam steering solutions, which targets 2D scenarios with lower search space and short-term motion coherence, fail in practical 3D setup. Motivated by these observations, we propose a model-driven 3D beam-steering mechanism called Orthogonal Scanner (OScan), which can maintain high performance for mobile 60 GHz links in 3D space. OScan discovers and leverages a hidden interaction between 3D beams and the spatial channel profile of 60 GHz radios, and strategically scans the 3D space so as to reduce the search latency by more than one order of magnitude. Experiment results based on a custom-built 60 GHz platform along with a trace-driven emulator demonstrate OScan's remarkable throughput gain, up to 5×, compared with the state-of-the-art.

LEO Satellites, which can provide Internet access points for IoT devices deployed in areas without infrastructure, will play an important role in the IoT area. In this paper, we propose a joint TDMA MAC protocol SL-MAC for LEO satellite supported IoT, which can reduce network delay and improve the energy efficiency. Specifically, we design a continuous forward time slot allocation strategy, which guarantees real-time data while reducing the utilization of satellite resources. Then we design two algorithms based on the forward time slot allocation strategy, which can effectively reduce network delay. The simulation experimental results are consistent with the theoretical results, which embodies the effectiveness of the algorithm.

Multimedia big data is one of the cornerstones of Internet of Things (IoT). IoT research subjects naturally connect with multimedia and big data. At the same time, multimedia data and application occupies a large proportion of the landscape of IoT. Moreover, there are many exciting new research directions in multimedia big data based IoT, such as directional sensor networks, video opportunistic transmission, physical object location, image or voice based physical object searching, and so on. Due to the dramatic development of IoT, we have bigger and bigger data sets, and we are stepping into the world of multimedia big data.

The technology of Internet of Things (IoT) has been gaining great popularity in recent years, as it provides an effective and immediate bridge between the physical world and the virtual objects in the cyber space, which can lead to innovative applications and services with high efficiency and productivity. However, IoT is just at the beginning stage of a longer journey. In-depth research and development efforts on systems, networks and architectures of IoT for efficient large-scale deployments are still required to fill the gaps between the current performance and service requirements, particularly with the predicted importance of IoT in the upcoming years, improved connectivity and communication among numerous devices will become necessary and critical.

Providing high trustworthy service is the most fundamental task for any cloud computing platform. Users are willing to deliver their computing tasks and the most sensitive data to cloud data centers, which is based on the trust relationship established between users and cloud service providers. However, with the development of collaboration cloud computing, how to provider fast response for a large number of users' service requests becomes a challenging problem. In order to quickly provide highly trustworthy services, the service platform must efficiently and quickly reply tens of millions of service requests, and automatically match-make tens of thousands of service resources. In this context, lightweight and fast (high-speed, low-overhead) trust computing schemes become the fundamental demand for implementing a trustworthy and collaborative cloud service. In this paper, we propose an innovative and parallel trust computing scheme based on big data analysis for the trustworthy cloud service environment. First, a distributed and modular perceiving architecture for large-scale virtual machines' service behavior is proposed relying on distributed monitoring agents. Then, an adaptive, lightweight, and parallel trust computing scheme is proposed for big monitored data. To the best of our knowledge, this paper is the first to use a blocked and parallel computing mechanism, the speed of trust calculation is greatly accelerated, which makes this trust computing scheme very suitable for a large-scale cloud computing environment. Performance analysis and experimental results verify feasibility and effectiveness of the proposed scheme.

Compared with person reidentification, which has attracted concentrated attention, vehicle reidentification is an important yet frontier problem in video surveillance and has been neglected by the multimedia and vision communities. Since most existing approaches mainly consider the general vehicle appearance for reidentification while overlooking the distinct vehicle identifier, such as the license plate number, they attain suboptimal performance. In this paper, we propose PROVID, a PROgressive Vehicle re-IDentification framework based on deep neural networks. In particular, our framework not only utilizes the multimodality data in large-scale video surveillance, such as visual features, license plates, camera locations, and contextual information, but also considers vehicle reidentification in two progressive procedures: coarse-to-fine search in the feature domain, and near-to-distant search in the physical space. Furthermore, to evaluate our progressive search framework and facilitate related research, we construct the VeRi dataset, which is the most comprehensive dataset from real-world surveillance videos. It not only provides large numbers of vehicles with varied labels and sufficient cross-camera recurrences but also contains license plate numbers and contextual information. Extensive experiments on the VeRi dataset demonstrate both the accuracy and efficiency of our progressive vehicle reidentification framework.

Multi-cloud collaborative environment consists of multiple data centers, which is a typical processing platform for big data. This paper focuses on the trust computing requirement of multi-cloud collaborative services and develops a Data-driven and Feedback-Enhanced Trust (DFET) computing pattern across multiple data centers with several innovative mechanisms. First, a trust-aware service monitoring architecture is proposed based on distributed soft agents to serve as middleware for multi-cloud trust computing and task scheduling. A data-driven trust computation scheme based on multi-indicator monitoring data is then proposed. The integration of several key service indicators into trust computing makes this scheme suitable for service-oriented cloud applications. More importantly, according to the intrinsic relationship among users, monitors, and service providers, we propose an enhanced and hierarchical feedback mechanism that can effectively reduce networking risk while improving system dependability. Theoretical analysis shows that DFET pattern is highly dependable against garnished and bad-mouthing attacks. We also build a prototype system to verify the feasibility of DFET pattern and the experiments yield meaningful observations that can facilitate the effective utilization of DFET in the large-scale multi-cloud collaborative environment.

Distributed computing framework such asHadoop is now a prominent solution for the efficient big dataprocessing. However, the current data placement strategiesin these frameworks lack in consideration of the types ofstorage devices and the features of surveillance video tasks.So using the current distributed computing framework forlarge-scale video processing in the cloud computing clusterwith hybrid storage architecture will lead to the low utilizationof the high-performance devices and the skew of the videotask completion time. In this paper, we propose a novel Two-Stage Data Distribution Approach (TSDDA) for optimizingthe data placement in the cloud computing platform. Ourapproach adopts an accurate Processing Time Prediction Model(PTPM) to estimate the processing time of the video task byincorporating the several important video task features, theheterogeneous I/O capabilities and processing abilities in thehybrid cluster. In our approach, an initial data placementalgorithm is used to place the data needed by the videoprocessing jobs on the appropriate processing nodes in theprocess of the initial data loading, and then a dynamic datamigration algorithm is further used to migrate the video datafrom HDD to SSD for further improving the SSD’s utilizationand the processing efficiency during the process of the jobexecution. Finally, we conduct the extensive experiments, andthe experimental results verify the accuracy of PTPM and showthat the proposed TSDDA outperforms the current method

Information-centric networking (ICN), as a major branch of the future network architecture, shifts from address-based communication to named content retrieval. The special characteristics of ICN, such as named content, in-network cache and receiver-driven, make the content diffusion process in ICN be different from traditional point-to-point packet transmission. Thus existing transmission analysis theory cannot illustrate the content diffusion process of ICN accurately. Inspired by the epidemic-like spread model, whose infected-recovered process is similar to ICN's cached-evicted process, we propose a content diffusion model (CDM) to illustrate the content diffusion process in ICN. We also evaluate the accuracy of CDM by a lightweight ICN simulator developed. The results show that our proposed CDM can illustrate the content diffusion process in ICN accurately.

The performance of mobile networks is significantly influenced by the mobility patterns of wireless device holders. Human mobility models, which yields synthetic trajectories with essential mobility patterns of crowd, are important for the research and development of mobile networks. However, traditional models are often perceived as not realistic for depicting urban context (such as urban hotspots and direction of crowd flow). This paper proposes a temporal correlation based model that produce large-scale of synthetic trajectories on the basis of a short period of time real human traces. The trajectories generated by our model not only maintain the inherent statistical features of human mobility, but also “learn” the urban context from real traces. For simulating crowd mobility in a city, our model first extract the statistical features and urban context features from the the real traces, then regenerate synthetic trajectories on the basis of temporal correlation of human mobility. We validate our model and findings through three open datasets: taxicabs' traces in San Francisco, public bicycles' traces in Washington D.C. and New York City.

Named Data Networking (NDN), a revolution of the IP architecture, provides an information-centric paradigm to transmit video traffic which occupies most of the Internet traffic. However, the data caching schemes exploited by the current NDN can easily result in the unfair distribution of cache space, which is not suitable for performance isolation when there are multiple network users and the cache space is of high utilization for video applications. In this paper, we propose a max-min utility fairness scheme for the built-in cache allocation of video applications. We propose a computing method which calculates the benefit of all video data cached in the NDN node. Based on the benefit of individual user and the relevance of video data, NDN node evicts the video data which belongs to users with the most benefits. Extensive simulation experiments have been performed to evaluate our scheme in terms of user fairness, average delay and hit ratio. The results show that our scheme can guarantee the fairness for different video Users, meanwhile, reducing the packet round trip times by improving the cache hit ratio.

Accurate human body orientation estimation (HBOE) can significantly promote the analysis of human behavior. However, conventional methods cannot holistically exploit the complementary nature of spatial and temporal information for H-BOE. Different from existing methods, we propose an end-to-end temporal-spatial deep learning framework to accurately estimate the human body orientation. In this framework, we firstly utilize the convolutional neural network to capture the spatial information for human orientation. Furthermore, the spatial-temporal information are fused in the recurrent neural networks (RNNs), which can automatically memorize a long-term temporal information of human orientation transformation. More important, to effectively adapt different moving speeds and diversity actions of people, we design a weighted sequence loss function, which can capture the significant orientation conversion to guide the RNN training. According to the comprehensive evaluations, the proposed method greatly outperforms the states-of-the-art methods. Although only utilizing the 2D information, it can perform better than the 3-D/RGB-D based approaches.

Low robustness under mobility is the Achilles' heel of the emerging 60 GHz networking technology. Instead of using omni-directional antennas as in existing Wi-Fi/cellular networks, 60 GHz radios communicate via highly-directional links formed by phased-array beam-forming, so as to upgrade wireless link throughput to multi-Gbps. However, user motion causes misalignment between the Tx's and Rx's beam directions, and often leads to link outage. Legacy 60 GHz protocols realign the beams by scanning alternative Tx/Rx beams. But unfortunately this tedious process can easily overwhelm the useful channel time, leaving the Tx/Rx in misalignment most of the time during mobility. In this paper, we propose Beam-forecast, a novel model-driven beam steering approach that can sustain high performance for mobile 60 GHz links. Beam-forecast is built on the observation that 60 GHz channel profiles at nearby locations are highly-correlated. By exploiting this correlation, Beam-forecast can reconstruct the channel profile as the Tx/Rx moves, without explicit channel scanning. In this way, it can predict new optimal beams and realign links for mobile users with minimal overhead. We evaluate Beam-forecast using a reconfigurable 60 GHz testbed along with a trace-driven simulator. Our experiments demonstrate multi-fold throughput gain compared with state-of-the-art under various practical scenarios.

In this paper, we address the challenging problem of vehicle license plate image super-resolution. Different from existing image super-resolution approaches only resorted to one single image, we propose to leverage complementary information from multiple images to recover the license plate numbers. To achieve this goal, we design a principled license plate images super-resolution framework which is composed of two components: progressive vehicle search and Domain Priori GAN (DP-GAN). Particularly, we design a null space based progressive vehicle search approach to retrieve the relevant images captured by different cameras given one vehicle with a low-resolution license plate. To handle the extremely varied license plate images caused by different sensors, times, depths, and viewpoints, we also propose a DP-GAN framework to generate multiple spatial correspondences and high-resolution plate images. In the generator network of DP-GAN, a license plate synthesis pipeline is exploited to generate the nearly canonical license plates. In the discriminator network, a spatial split layer is designed to simultaneously preserve the global and local manufacture standards of the license plate. Finally, a multiple images super-resolution GAN is exploited to combine all the synthetic license plates into one high-resolution image. Different from previous super-resolution criteria mainly focus on pixel-level detail recovery condition, we leverage the downstream tasks, i.e. license plate recognition and vehicle search as criteria. The results on a new collected real-world dataset demonstrate that the proposed method achieves the beyond human-level license plate super-resolution performance for automatic license plate recognition and vehicle search.

This paper presents the method that underlies our submission to the popularity prediction task of Social Media Prediction Challenge 2017. The task is designed to predict the impact of sharing different posts for a publisher on social media. There are many factors that influence image popularity; these include not only the visual features of the image, but also the social features, such as user characteristics of its poster and even the upload time. In this project, we propose a fast and effective framework for popularity prediction. First, we investigate and extract visual and social features of images. For the visual feature, we introduce 1) global feature descriptors, such as Local Binary Pattern and Color Names, 2) local feature descriptors, such as Local Maximal Occurrence, and 3) deep features. For the social feature, we adopt users features (average views, group count, and member count), post features (title length, description length, and tag count), and time features (month, weekday, day, and hour). Furthermore, we fed a fusion of multi-feature to Linear Regression, Matrix Factorization based on Time and feature Cluster, and Support Vector Regression models respectively, and present comparative analysis of the prediction results. Finally, we choose the best model to predict the popularity scores of the test images. Experimental results demonstrate that our method can achieve 0.8581, 1.4062 and 0.8625 in terms of Spearman Ranking Correlation, Mean Absolute Error, and Mean Squared Error, respectively.

Recognizing object location by taking a photo with smartphones is useful for many location-based services. However, start-of-the-art technologies for both localization and location recognition have difficulty in achieving satisfactory performance. Moreover, it is a challenging issue to construct and maintain a large-scale image database for existing visual-based location recognition systems. To cope with these issues, we introduce CrowdOLR, a crowdsourcing based object location recognition system, which collects one location image together with various rich sensory data (GPS coordinates, azimuth angle, tilt angle, etc.) as a fingerprint of a location query and matches it to a fingerprint database crowdsourced from users' smartphones. We designed a simple and efficient user action mode and proposed a series of fingerprint extracting, searching, and matching methods, so that CrowdOLR satisfies five desirable properties: high recognition accuracy, user friendliness, quick response, no/little site survey, and timely update. We implemented CrowdOLR and collected 8100 location fingerprints of 162 objects for performance evaluation. Extensive experiments demonstrate that CrowdOLR achieves promising results in various complicated and realistic scenarios.

Many current video surveillance cloud platforms are built based on Virtual Machine (VM) technology that usually induces the loss of service performance and brings some challenges in the service management agility. In addition, the efficient resource provisioning for the heterogeneous video services is also a challenging issue in such a dynamic and shared cloud environment. In this paper, we firstly design a novel video surveillance cloud platform that employs the lightweight container technology and is defined according to the ITU standards. Our platform can provide a flexible and reconfigurable video microservice management environment with the high service capacity. Secondly, we propose a predictive fine-grained resource provisioning approach that can periodically predict the future workload and perform the proactive resource supply for the video microservices in the cloud. Our approach utilizes the service similarity matching and the time-series nearest neighbor regression to efficiently predict the future resource requirements, and dynamically optimizes the usage of resources based on predictive results while ensuring quality of service. Finally, we implement the proposed platform, and conduct the extensive experiments. The experimental results indicate that the proposed solution provides the higher service deployment density, accurately predicts the resource demands and significantly improves the resource utilization.

Large-scale surveillance video analytic is a kind of typical big data application. The Spark framework combined with Hadoop Distributed File System (HDFS) is a promising solution for the efficient surveillance video processing. However, the current HDFS distributes data to multiple nodes according to the disk space availability, and this data placement mode will lead to the serious skew of the video task completion time and the performance degradation of the distributed video processing in the heterogeneous Spark cluster. In this paper, we firstly design a distributed surveillance video processing platform architecture which supports the seamless integration with the standard video surveillance system. Our platform uses the Spark computing framework over the data stored in HDFS. Then, we propose a novel proactive video data placement strategy to schedule the input video data into the appropriate cluster node adaptively. Our strategy adopts a novel Computing Time Prediction Model (CTPM) which can accurately estimate the execution time of the video processing task by incorporating the several important video task features. In our strategy, an Initial Data Placement Algorithm (IDPA) is used to place the data needed by the video processing jobs on the appropriate cluster node in the process of the initial data loading, and then a Data Rebalance Algorithm (DRA) is further used to schedule the data for the workload balancing during the process of the job execution. Finally, we build a distributed surveillance video processing system according to the proposed platform architecture and conduct the extensive experiments. The experimental results verify the accuracy of CTPM and show that our system can reduce the overhead of the data transferring and improve the job execution efficiency compared with the current widely used methods.

As the remarkable characteristics of remote accessed, robust and security, gait recognition has gained significant attention in the biometrics based human identification task. However, the existed methods mainly employ the handcrafted gait features, which cannot well handle the indistinctive inter-class differences and large intra-class variations of human gait in real-world situation. In this paper, we have developed a Siamese neural network based gait recognition framework to automatically extract robust and discriminative gait features for human identification. Different from conventional deep neural network, the Siamese network can employ distance metric learning to drive the similarity metric to be small for pairs of gait from the same person, and large for pairs from different persons. In particular, to further learn effective model with limited training data, we composite the gait energy images instead of raw sequence of gaits. Consequently, the experiments on the world's largest gait database show our framework impressively outperforms state-of-the-arts.

Urban environmental monitoring, which provides crucial environmental information, is of great importance in various fields of urban research. The rising popularity of smart-phones and vehicles equipped with onboard sensors points out an ubiquitous computing mode for city-scale environmental monitoring, namely “crowdsensing”. This paper studies the problem that how to promote the ability of crowdsensing for fine-grained environmental monitoring. Based on a new metric for measuring the sensing ability of crowdsensing, the resolution of sensing image, we propose a method of generating sensing images via crowdsensing networks and enhance the ability of crowdsensing from a data perspective. To achieve a higher resolution, we not only utilize temporal correlation of mobile sensing nodes but also fuse the sensory data with correlated environmental information. Finally, we conduct numerical simulations to verify the promotion of resolution beyond traditional spatial interpolation-based approach.

Vehicle, as a significant object class in urban surveillance, attracts massive focuses in computer vision field, such as detection, tracking, and classification. Among them, vehicle re-identification (Re-Id) is an important yet frontier topic, which not only faces the challenges of enormous intra-class and subtle inter-class differences of vehicles in multicameras, but also suffers from the complicated environments in urban surveillance scenarios. Besides, the existing vehicle related datasets all neglect the requirements of vehicle Re-Id: 1) massive vehicles captured in real-world traffic environment; and 2) applicable recurrence rate to give cross-camera vehicle search for vehicle Re-Id. To facilitate vehicle Re-Id research, we propose a large-scale benchmark dataset for vehicle Re-Id in the real-world urban surveillance scenario, named “VeRi”. It contains over 40,000 bounding boxes of 619 vehicles captured by 20 cameras in unconstrained traffic scene. Moreover, each vehicle is captured by 2~18 cameras in different viewpoints, illuminations, and resolutions to provide high recurrence rate for vehicle Re-Id. Finally, we evaluate six competitive vehicle Re-Id methods on VeRi and propose a baseline which combines the color, texture, and highlevel semantic information extracted by deep neural network.

With the increasing popularity of video delivery among mobile users, the explosive traffic growth problem becomes more and more serious for the mobile wireless networks. We propose to exploit opportunistic transmission and the idea of crowdsourcing to offload mobile video traffic. Specifically, we propose a QoV (Quality of Video) oriented pricing incentive scheme, namely Vbargain, to stimulate mobile users to deliver video data collaboratively. In our scheme, the video packets are treated as commodities, which are dynamically priced according to their expected marginal gains on the quality of reconstructed video; the process of video delivery is regarded as a sequence of packet transactions which are modeled as two-person cooperative games. Our simulation results, based on both the synthetic and real-life traces of mobile users, verify the efficiency of our scheme.

Wireless Sensor Networks (WSNs), the major technique in the sensing layer of Internet of Things. Because the WSN is extensively regarded as a data-centric network, more and more researchers start to exploit information-centric networking (ICN), a major branch of future network architecture, to design WSNs. In this paper, we propose a collaborative caching strategy for information-centric wireless sensor network (ICN-WSN). The proposed strategy consists of three parts: the node betweenness based cache size adjustment, the data replacement frequency based cache decision, and the content value based cache replacement algorithm. Based on our developed prototype of ICN-WSN, we compared our caching strategy with traditional strategies. Extensive experiments have been performed in terms of energy consumption, hit ratio, average delay and average hops. The results show that our strategy can improve the hit ratio of content caching, and in the meanwhile reduce the average delay, average hops, and energy consumption.

Internet of Things (IoT), as the trend of future networks, begins to be used in many aspects of daily life. It is of great significance to recognize the networking problem behind developing IoT. In this paper, we first analyze and point out the key problem of IoT from the perspective of networking: how to interconnect large-scale heterogeneous network elements and exchange data efficiently. Combining our on-going works, we present some research progresses on three main aspects: 1) the basic model of IoT architecture; 2) the internetworking model; and 3) the sensor-networking mode. Finally, we discuss two remaining challenges in this area.

Automatically identifying characters in movies has attracted researchers' interest and led to several significant and interesting applications. However, due to the vast variation in character appearance as well as the weakness and ambiguity of available annotation, it is still a challenging problem. In this paper, we investigate this problem with the supervision of actor-character name correspondence provided by the movie cast. Our proposed framework, namely, Cast2Face, is featured by: 1) we restrict the assigned names within the set of character names in the cast; 2) for each character, by using the corresponding actor and movie name as keywords, we retrieve from the Google image search and get a group of face images to form the gallery set; 3) the probe face tracks in the movie are then identified as one of the actors by a robust kernel multitask joint sparse representation and classification method; and 4) the conditional random field model with consideration of the constraints between face tracks is introduced to enhance the final labeling. Finally, the assigned actor name of a face track is then mapped to the character name based on the cast again. Besides face naming, we further apply the proposed method to spotlight the summarization of a particular actor in his/her movies. We conduct extensive experiments and empirical evaluations on several feature-length movies to demonstrate the satisfying performance of our method.

Mobile crowd sensing (MCS) is a new paradigm that takes advantage of pervasive mobile devices to efficiently collect data, enabling numerous novel applications. To achieve good service quality for an MCS application, incentive mechanisms are necessary to attract more user participation. Most existing mechanisms apply only for the offline scenario where all users report their strategic types in advance. On the contrary, we focus on a more realistic scenario where users arrive one by one online in a random order. Based on the online auction model, we investigate the problem that users submit their private types to the crowdsourcer when arriving, and the crowdsourcer aims at selecting a subset of users before a specified deadline for maximizing the value of services (assumed to be a nonnegative monotone submodular function) provided by selected users under a budget constraint. We design two online mechanisms, OMZ and OMG, satisfying the computational efficiency, individual rationality, budget feasibility, truthfulness, consumer sovereignty, and constant competitiveness under the zero arrival-departure interval case and a more general case, respectively. Through extensive simulations, we evaluate the performance and validate the theoretical properties of our online mechanisms.

Hadoop is widely deployed distributed computing framework and makes creating distributed applications much easier. However, unlike text data, there is no existing video r/w interface for Hadoop, and many existing video analytic applications implemented in C/C++ are not compatible with Hadoop framework. In this paper, we propose an open source Hadoop video processing interface HVPI to extend Hadoop to support video analytic applications. It provides easy-to-use video r/w interface for developers to quickly build large-scale video analytic applications based on Hadoop, and native processing interface to help users easily port existing video analytic applications written in C/C++ into Hadoop platform. We also present two typical use cases of HVPI and do experiments based on them. Experimental results demonstrate that the applications built based on HVPI are both scalable and efficient.

Video surveillance service has become one of the most popular services of video data center. Different from traditional VoD and IPTV services, video surveillance service is based on video data center which receives video stream from IP camera and forward the video stream to end user. However, current works only consider the forwarding video stream scheduling from media server to end user and focus on minimizing media server usage during video delivery. It is lack of a full consideration on network cost optimization of both receiving and forwarding streams as well as capacity evaluation of the media server. In this paper, we present a video surveillance service based on data center and propose an efficient resource scheduling approach for online multi-camera video delivery. We not only provide a fine-grained resource usage model for media servers, but also optimize network resources on both receiving and forwarding streams in video data center. We formulate the resource scheduling problem as a constrained integer optimization problem to minimize the total resource cost. We first propose an optimal solution for linear resource cost function using drift-plus-penalty optimization method. For non-linear resource cost functions, we present a heuristic scheduling approach to reduce both media server cost and network cost. The experimental results show that our approaches obviously decrease the total resource cost of the video data center on both media servers and networks.

Named Data Networking (NDN), a revolution of the IP architecture, provides an information-centric routing for designing the unified Internet of Things (IoT) network protocol. However, the data forwarding scheme exploited by the current NDN generates lots of communication overhead and invalid caching hits, which is not suitable for IoT. This is because there exist large amounts of weak network devices with constrained resources in IoT compared to the Internet. In this paper, we first build a caching time model to characterize the caching process for individual packet and estimate the packet's caching time on the next hop node. Based on this model, we propose a cache-aware named-data forwarding scheme for the typical weak devices in IoT-wireless sensor nodes. Extensive simulation experiments have been performed to evaluate our scheme in terms of forwarding number, average delay and valid hit ratio. The results show that our scheme can reduce the packet number during multicast forwarding of router nodes, meanwhile guarantee valid hit ratio of content caching.

Multi-class weather classification from single images is a fundamental operation in many outdoor computer vision applications. However, it remains difficult and the limited work is carried out for addressing the difficulty. Moreover, existing method is based on the fixed scene. In this paper we present a method for any scenario multi-class weather classification based on multiple weather features and multiple kernel learning. Our approach extracts multiple weather features and takes properly processing. By combining these features into high dimensional vectors, we utilize multiple kernel learning to learn an adaptive classifier. We collect an outdoor image set that contains 20K images called MWI (Multi-class Weather Image) set. Experimental results show that the proposed method can efficiently recognize weather on MWI dataset.

Mobile crowd-sensing applications produce useful knowledge of the surrounding environment, which makes our life more predictable. However, these applications often require people to contribute, consciously or unconsciously, location-related data for analysis, and this gravely encroaches users' location privacy. Aggregate processing is a feasible way for preserving user privacy to some extent, and based on the mode, some privacy-preserving schemes have been proposed. However, existing schemes still cannot guarantee users' location privacy in the scenarios with low density participants. Meanwhile, user accountability also needs to be considered comprehensively to protect the system from malicious users. In this paper, we propose a participant-density-aware privacy-preserving aggregate statistics scheme for mobile crowd-sensing applications. In our scheme, we make use of multi-pseudonym mechanism to overcome the vulnerability due to low participant density. To further handle sybil attacks, based on the Paillier cryptosystem and non-interactive zero-knowledge verification, we advance and improve our solution framework, which also covers the problem of user accountability. Finally, the theoretical analysis indicates that our scheme achieves the desired properties, and the performance experiments demonstrate that our scheme can achieve a balance among accuracy, privacy-protection and computational overhead.

With the rapid advancements of mobile computing, mobile location recognition is becoming an important and useful service, which recognizes the logical locations of places/scenes that users are interested in, instead of physical coordinates. Most of the existing mobile location recognition systems utilize the image as visual fingerprint of a place, and need to construct a large-scale visual fingerprint database in advance. However, collecting visual fingerprints is a labor-intensive and time-consuming procedure. In order to address this problem, we propose a novel crowdsourcing-based framework, and leverage a variety of sensors embedded in smartphones to collect richer location fingerprints for exploring their positive effects. To achieve higher recognition accuracy, we propose an object-centric fingerprint searching which can sufficiently take advantage of smartphone sensors and determine more accurate searching space than the traditional user-centric method. We build a crowdsourcing-based database with richer fingerprints and implement a location recognition system, called CrowdLR. Extensive experiments verify that our object-centric method can achieve promising results maintaining around 10% precision higher than the user-centric method.

This paper proposes a novel approach for automatically controlling stereoscopic camera parameters that specifically addresses challenges in stereo 3D animation production process.Our proposed camera control method produces stereo contents with preferable depth perception and guarantees visual comfort by optimization of camera parameters. We introduce an attention tracking method to calculate convergence plane, avoiding window violation and minimizing visual conflict. Moreover, we derive an smoothing function on convergence plane that reduces depth jump over time. Then, we calculate the inter-axial separation using a perceived depth mapping. We describe how to implement our method on the Maya plug-in and test the stereo effect using professional stereo 3D animation scenes. The experimental results, including a user study, show that our method enhances the stereo effect. Our controller provides automatic camera control that can be helpful in creating comfortable and faster stereo 3D animations.

As one of the future networking architecture, Content Centric Networking (CCN) regards content as a first-class network entity and fits Internet of Things (IoT) applications well. CCN retrieves contents by their unique hierarchical names. However, in IoT, different autonomous networks have their own naming rules. When a mobile node enters a new network environment and interacts with local networks, it can hardly know the exact name of the content that it needs. In this paper, we propose Tag-assisted Content Centric Networking (TCCN) for IoT, which is a more flexible and efficient scheme for the content sharing between mobile node and local IoT network. We design the tables and structures of three basic components of CCN. Based on counting bloom filter, we further design Tag Filter (TF) for interest-content matching and forwarding information lookup. By using TF, Content Store (CS) can quickly check whether coming Interests can be satisfied locally. Moreover, in our designed Tag-based Forwarding Information Base (T-FIB), we change the traditional prefix-faces structure into face-tags structure, which can greatly reduce the size of FIB and the lookup time consumption. We design the Tag-based Caching Strategy (TCS), by which the node can decide whether to cache the coming content according to the TF and a counter threshold. By using TCS, contents with similar set of tags are cached intensively, which benefits the forwarding information.We conduct simulations to evaluate the validity and performance of our proposed methods.

Using insights from biological processes could help to design new optimization techniques for long-standing computational problems. This paper exploits a cellular computing model in the slime mold physarum polycephalum to solve the Steiner tree problem which is an important NP-hard problem in various applications, especially in network design. Inspired by the path-finding and network formation capability of physarum, we develop a new optimization algorithm, named as the physarum optimization, with low complexity and high parallelism. To validate and evaluate our proposed models and algorithm, we further apply the physarum optimization to the minimal exposure problem which is a fundamental problem corresponding to the worst-case coverage in wireless sensor networks. Complexity analysis and simulation results show that our proposed algorithm could achieve good performance with low complexity. Moreover, the core mechanism of our physarum optimization also may provide a useful starting point to develop some practical distributed algorithms for network design.

Oriented by requirement of trust management in multiple cloud environment, this paper presents T-broker, a trust-aware service brokering scheme for efficient matching cloud services (or resources) to satisfy various user requests. First, a trusted third party-based service brokering architecture is proposed for multiple cloud environment, in which the T-broker acts as a middleware for cloud trust management and service matching. Then, T-broker uses a hybrid and adaptive trust model to compute the overall trust degree of service resources, in which trust is defined as a fusion evaluation result from adaptively combining the direct monitored evidence with the social feedback of the service resources. More importantly, T-broker uses the maximizing deviation method to compute the direct experience based on multiple key trusted attributes of service resources, which can overcome the limitations of traditional trust schemes, in which the trusted attributes are weighted manually or subjectively. Finally, T-broker uses a lightweight feedback mechanism, which can effectively reduce networking risk and improve system efficiency. The experimental results show that, compared with the existing approaches, our T-broker yields very good results in many typical cases, and the proposed system is robust to deal with various numbers of dynamic service behavior from multiple cloud sites.

The rising popularity of smartphones and vehicles equipped with onboard sensors sheds lights on building a city-scale sensing system for urban surveillance. This paper proposes a novel metric, urban resolution, to measure the quality of urban sensing. Urban resolution describes how sensitivity the urban sensing system could achieve for environment monitoring applications. Then, we study the relationship between resolution r and number of sensing nodes s, and reveal the linear growth relationship between √r and √s . Furthermore, by employing a commonly used human/vehicle mobility model, SLAW, we find that the distribution model of urban sensing nodes is able to be described by a truncated Pareto distribution, and derive the complementary cumulative distribution function (CCDF) of urban resolution. The CCDF reveals the radio of the sub-regions which satisfy the required sensing quality to the whole region. Our findings provide valuable insights to infer the urban sensing quality according to the scale of urban sensing system or determine how many smartphone/vehicles needed for participating in urban sensing applications. Finally, based on five real datasets-three human/vehicle trajectory datasets and two environment monitoring datasets, we examine the metric of urban resolution and evaluate the main results in this paper.

This paper proposes a service operator-aware trust scheme (SOTS) for resource matchmaking across multiple clouds. Through analyzing the built-in relationship between the users, the broker, and the service resources, this paper proposes a middleware framework of trust management that can effectively reduces user burden and improve system dependability. Based on multidimensional resource service operators, we model the problem of trust evaluation as a process of multi-attribute decision-making, and develop an adaptive trust evaluation approach based on information entropy theory. This adaptive approach can overcome the limitations of traditional trust schemes, whereby the trusted operators are weighted manually or subjectively. As a result, using SOTS, the broker can efficiently and accurately prepare the most trusted resources in advance, and thus provide more dependable resources to users. Our experiments yield interesting and meaningful observations that can facilitate the effective utilization of SOTS in a large-scale multi-cloud environment.

Human-carried or vehicle-mounted sensors can be exploited to collect data ubiquitously for building various sensing maps. Most of existing mobile sensing applications consider users reporting and accessing sensing data through the Internet. However, this approach cannot be applied in the scenarios with poor network coverage or expensive network access. Existing data forwarding schemes for mobile opportunistic networks are not sufficient for sensing applications as spatial-temporal correlation among sensory data has not been explored. In order to build sensing maps satisfying specific sensing quality with low delay and energy consumption, we design COUPON, a novel cooperative sensing and data forwarding framework. We first notice that cooperative sensing scheme can eliminate sampling redundancy and hence save energy. Then we design two cooperative forwarding schemes by leveraging data fusion: Epidemic Routing with Fusion (ERF) and Binary Spray-and-Wait with Fusion (BSWF). Different from previous work assuming that all packets are propagated independently, we consider that packets are spatial-temporal correlated in the forwarding process, and derive the dissemination law of correlated packets. Both the theoretic analysis and simulation results show that our cooperative forwarding schemes can achieve better tradeoff between delivery delay and transmission overhead. We also evaluate our proposed framework and schemes with real mobile traces. Extensive simulations demonstrate that the cooperative sensing scheme can reduce the number of samplings by 93 percent compared with the non-cooperative scheme; ERF can reduce the transmission overhead by 78 percent compared with Epidemic Routing (ER); BSWF can increase the delivery ratio by 16 percent, and reduce the delivery delay and transmission overhead by 5 and 32 percent respectively, compared with Binary Spray-and-Wait (BSW).

Along with a large amount of data being outsourced to the cloud, it is imperative to enforce a secure, efficient and privacy-aware access control scheme on the cloud. Decentralized Attribute-based Encryption (ABE) is a variant of multi-authority ABE scheme which is regarded as being more suited to access control in a large-scale cloud. Constructing a decentralized ABE scheme should not need a central Attribute Authority (AA) and any cooperative computing, where most schemes are not efficient enough. Moreover, they introduced a Global Identifier (GID) to resist the collusion attack from users, but corrupt AAs can trace a user by his GID, resulting in the leakage of the user's identity privacy. In this paper, we design a privacy-preserving decentralized access control framework for cloud storage systems, and propose a decentralized CP-ABE access control scheme with the privacy preserving secret key extraction. Our scheme does not require any central AA and coordination among multi-authorities. We adopt Pedersen commitment scheme and oblivious commitment based envelope protocols as the main cryptographic primitives to address the privacy problem, thus the users receive secret keys only for valid identity attributes while the AAs learn nothing about the attributes. Our theoretical analysis and extensive experiment demonstrate the presented scheme's security strength and effectiveness in terms of scalability, computation and storage.

Metadata is widely used to facilitate user defined queries and high-level event recognition applications in traffic surveillance videos. Current metadata extraction approaches rely on some computer vision algorithms, which are not accurate enough in the real world traffic scenes, and do not deal with big surveillance data efficiently. In this paper, we design a novel metadata extraction and metadata correction system. Firstly, we define the structure of metadata to determine which attribute (e.g., vehicle enter time, license plate number, vehicle type) we need to extract. Based on this structure, we employ a three-phase method to extract metadata. Secondly, we propose a graph-based metadata correction approach for compensating the accuracy of metadata extraction method. It fuses the big metadata of whole camera network, automatically detects suspicious metadata and corrects them based on the metadata spatial-temporal relationship and the image similarity. As the centralized framework may not be able to cope with the huge amount of data generated by traffic surveillance system, our system is implemented in a distributed fashion using Hadoop and HBase. Finally, the experimental results on real world traffic surveillance videos demonstrate the efficiency of our system, and also demonstrate that the metadata quality is significantly improved after metadata correction.

Participatory sensing applications can learn the aggregate statistics over personal data to produce useful knowledge about the world. Since personal data may be privacy-sensitive, the aggregator should only gain desired statistics without learning anything about the personal data. To guarantee differential privacy of personal data under an untrusted aggregator, existing approaches encrypt the noisy personal data, and allow the aggregator to get a noisy sum. However, these approaches suffer from either high computation overhead, or lack of efficient group management to support dynamic joins and leaves, or node failures. In this paper, we propose a novel privacy-preserving aggregation scheme to address these issues in participatory sensing applications. In our scheme, we first design an efficient group management protocol to deal with participants' dynamic joins and leaves. Specifically, when a participant joins or leaves, only three participants need to update their encryption keys. Moreover, we leverage the future ciphertext buffering mechanism to deal with node failures, which is combined with the group management protocol making low communication overhead. The analysis indicates that our scheme achieves desired properties, and the performance evaluation demonstrates the scheme's efficiency in terms of communication and computation overhead.

Cloud storage access control is very important for the security of outsourced data, where Attribute-based Encryption (ABE) is regarded as one of the most promising technologies. Current researches mainly focus on decentralized ABE, a variant of multi-authority ABE scheme, because conventional ABE schemes depend on a single authority to issue secret keys for all of users, which is very impractical in a large-scale cloud. A decentralized ABE scheme should not rely on a central authority and can eliminate the need for collaborative computation. However, constructing such an efficient and practical decentralized ABE scheme remains a challenging research problem. In this study, we design a new decentralized ciphertext-policy attribute-based encryption access control scheme for cloud storage systems. Firstly, our scheme dose not require any central authority and global coordination among multiple authorities. Then, it supports any LSSS access structure and thus can encrypt data in terms of any boolean formula. In addition, we also utilize Proxy Re-encryption technique to overcome the user revocation problem in decentralized ABE schemes, thus making our scheme more practical. Our security and performance analysis demonstrate the presented scheme's security strength and efficiency in terms of flexibility and computation.

Recently numerous sensing applications from the Internet of Things(IoT), such as traffic monitoring and noise map making, require the expensive installation cost. To handle this challenge, crowd sensing, as a new paradigm, has received extensive concerns. To achieve good service quality, incentive mechanisms are necessary to attract more users to participate in crowd sensing. However, Traditional incentive mechanisms such as the Vickrey-Clarke-Groves (VCG) mechanism and its variants, are not applicable for practical continuous crowd sensing applications, where bids cannot be solicited and only posted pricing mechanisms can be implemented. To tackle the issue, we propose a novel crowd sensing framework by applying all-pay auctions, and then design a collection-behavior based multi-parameter posted pricing mechanism by applying crowd aversion and posted pricing mechanisms. Simulation results indicate that incentive mechanisms in our proposed framework outperform the existing solution with respect to user participation and submission quality.

Crowd sensing is a new paradigm which leverages the ubiquity of sensor-equipped mobile devices to collect data. To achieve good quality for crowd sensing, incentive mechanisms are indispensable to attract more participants. Most of existing mechanisms focus on the expected utility prior to sensing, ignoring the risk of low quality solution and privacy leakage. Traditional incentive mechanisms such as the Vickrey-Clarke-Groves (VCG) mechanism and its variants are not applicable here. In this paper, to address these challenges, we propose a behavior based incentive mechanism for crowd sensing applications with budget constraints by applying sequential all-pay auctions in mobile social networks (MSNs), not only to consider the effects of extensive user participation, but also to maximize high quality of sensing data submission for the platform (crowd sensing organizer) under the budget constraints, where users arrive in a sequential order. Through an extensive simulation, results indicate that incentive mechanisms in our proposed framework outperform existing solutions.

Mobile crowdsourcing is a new paradigm which leverages pervasive smartphones to efficiently collect and upload data, enabling numerous novel applications. Recently, a class of new mechanisms have been proposed to determine near-optimal prices of sensing tasks for online crowdsourcing markets, where users arrive online and the crowdsourcer has budget constraints. In particular, the mechanisms can motivate extensive users to participate in online crowdsourcing markets. Although it is so promising in real-life environments, there still exist many security and privacy challenges. In this paper, we present a heterogeneous-user based privacy-preserving verifiable incentive mechanism for online crowdsourcing markets with the budget constraint, not only to explore how to protect the privacy of the bids, selection preferences, and identity from participants, but also to make the verifiable payment between the crowdsourcer (the crowdsourcing organizer) and online sequential arrival users. Results indicate that our privacy-preserving verifiable mechanisms achieve the same results as the generic one without privacy preservation.

Mobile crowdsourced sensing (MCS) is a new paradigm which takes advantage of pervasive smartphones to efficiently collect data, enabling numerous novel applications. To achieve good service quality for a MCS application, incentive mechanisms are necessary to attract more user participation. Most of existing mechanisms apply only for the offline scenario where all users' information are known a priori. On the contrary, we focus on a more realistic scenario where users arrive one by one online in a random order. Based on the online auction model, we investigate the problem that users submit their private types to the crowdsourcer when arrive, and the crowdsourcer aims at selecting a subset of users before a specified deadline for maximizing the value of services (assumed to be a non-negative monotone submodular function) provided by selected users under a budget constraint. We design two online mechanisms, OMZ and OMG, satisfying the computational efficiency, individual rationality, budget feasibility, truthfulness, consumer sovereignty and constant competitiveness under the zero arrival-departure interval case and a more general case, respectively. Through extensive simulations, we evaluate the performance and validate the theoretical properties of our online mechanisms.

Reliable and real-time crowd detection is one of the most important tasks in intelligent video surveillance system. Previous works focus on counting the number of pedestrians in the crowd directly or use holistic features of crowd scenes for crowd detection. However, the former methods will be invalid in complex crowded scenes, and the latter methods will be confused for feature selection. In this paper, we propose a simple but effective model - Gradient Magnitude Entropy (GME) model for crowd detection. Our model is based on a key observation - the value of GME in a region which will increase as the number of pedestrians grows. Thus, we can estimate the degree of crowd when the value of GME is larger than some threshold, without counting the number of pedestrians. Extensive experiments show that our GME model outperforms state-of-the-art techniques on several challenging datasets. Furthermore, our method can process in real time for practical surveillance applications.

Crowd counting with heavy occlusions is highly desired for public security. However, few works have been studied towards this goal. Most previous systems only count passing people robustly without heavy occlusions, else they have to estimate the crowds to a certain extent. To solve this difficult problem, this paper proposes an effective algorithm by combining head detection and motion flow estimation together. First, we detect each head in the crowd by using our proposed scene-adaptive scheme on depth data. Then, we estimate the motion flow based on the interest points in each head region on color data. We can ultimately achieve multi-direction crowd counting results with the loop of above steps. Based on this approach, we have built a practical system for reliable crowd counting. Extensive experimental results show that our system is effective.

Due to the ability to provide more precise and user friendly information, video streaming delivery over vehicular ad hoc networks (VANETs) has become a hot topic in recent years. In spite of many challenges, several routing schemes have been proposed. However, these schemes excessively focused on the minimization of delivery delay, and did not realize that immoderate utilization of wireless fading channels could incur high distortion due to high probabilities of video package loss and damage. Moreover, in these schemes, the interference from neighbors is not considered during the relay selection, which could decrease the delivery quality greatly. Therefore, in this paper, we take interference into account and formulate rate distortion model of live video streaming in VANETs. Based on the model, we propose a novel routing protocol, which could maximize the end-to-end delivery quality of live video streaming by seeking a balance between distortion and delay. The performance of our protocol is validated based on the real-life traces.

In recent years, more and more people begin to focus on data delivery through opportunistic contacts among mobile users, and in most case, the data that we expect to share with others are videos. Due to that the delivery of video data will consume numerous resources of devices, few people will provide such service for others without any benefit. Thus, this paper centers on the incentive mechanism design for video delivery in opportunistic networks. Up to now, several incentive schemes for opportunistic networks have been proposed, however, they all mainly focus on the encouragement of general data forwarding, and almost none of them aims to stimulate nodes to realize the performance objective of networks. In this paper, we propose a novel quality-oriented incentive mechanism for video delivery, which takes the characteristics of video data into account and formulates the video delivery among nodes as a game. This mechanism could not only encourage nodes to participate in the video delivery but also stimulate them to follow the defined rules to optimize the video delivery quality of whole networks. The performance of this mechanism is validated through extensive simulations.

In this paper, we propose a novel semi-supervised learning strategy to address the problem of celebrity identification. The video context information is explored to facilitate the learning process based on the assumption that faces in the same video track share the same identity. Once a frame within a track is recognized confidently, the label can be propagated through the whole track, referred to as the confident track. More specifically, given a few static images and vast face videos, an initial weak classifier is trained and gradually evolves by iteratively promoting the confident tracks into the “labeled” set. The iterative selection process enriches the diversity of the “labeled” set such that the performance of the classifier is gradually improved. This learning theme may suffer from semantic drifting caused by errors in selecting the confident tracks. To address this issue, we propose to treat the selected frames as related samples-an intermediate state between labeled and unlabeled instead of labeled as in the traditional approach. To evaluate the performance, we construct a new dataset, which includes 3000 static images and 2700 face tracks of 30 celebrities. Comprehensive evaluations on this dataset and a public video dataset indicate significant improvement of our approach over established baseline methods.

The Minimal Exposure Problem (MEP), which corresponds to the quality of coverage, is a fundamental problem in wireless sensor networks. This paper exploits a biological model of physarum to design a novel biology-inspired optimization algorithm for MEP. We first formulate MEP and the related models, and then convert MEP into the Steiner problem by discretizing the monitoring field to a large-scale weighted grid. Inspired by the path-finding capability of physarum, we develop a biological optimization solution to find the minimal exposure road-network among multiple points of interest, and present a Physarum Optimization Algorithm (POA). Furthermore, POA can be used for solving the general Steiner problem. Extensive simulations demonstrate that our proposed models and algorithm are effective for finding the road-network with minimal exposure and feasible for the Steiner problem.

As a fundamental problem of wireless sensor networks, the minimal exposure path problem corresponding to the sensor network's worst-case coverage plays an important role in the applications for detecting intrusions. However, most existing works about minimal exposure path are based on omnidirectional sensors. In contrast, this paper studies the minimal exposure path problem for two different types of directional sensing models: the binary sector model and directional sensitivity model. For the binary sector model, we construct a special Voronoi diagram, called sector centroids-based Voronoi diagram, to transform the minimal exposure path problem from a continuous geometric problem into a discrete geometric problem. By using the sector centroids-based Voronoi diagram, we develop an approximate algorithm to find the minimal exposure path in the sensors deployment field. For the directional sensitive model, we formulate the minimal exposure path problem by using two sensing intensity functions: all-sensor intensity function and maximum-sensor intensity function and then generate two weighted grids to convert the minimal exposure path problem into two discrete geometric problems. On the basis of the aforementioned weighted grids, we also develop two approximation algorithms to find the minimal exposure path for the directional sensitive model. We conduct extensive experiment simulations to validate and evaluate our proposed models and algorithms. Copyright © 2012 John Wiley & Sons, Ltd.

Recent years, more and more enterprises migrate their applications to the cloud for cost saving. During the application migration, an application usually needs to be re-designed from a workflow to an execution plan on a cloud platform. Based on the execution plan, many scheduling optimization approaches have been proposed to improve the performance of the system. In this paper, we argue that the concurrency optimization of each task in the workflow can significantly improve the system efficiency under delay or resource constraint. To generate an efficient execution plan, we optimize the concurrency of the task based on the critical path of the workflow. Furthermore, we deal with the scheduling problem in two different scenarios respectively. To save the costs under a certain deadline, we propose a delay constrained task scheduling approach. To shorten the completion time under certain resources, we propose a resource constrained task scheduling approach. Finally, by deploying a video analysis application on a cloud platform testbed, we validate the proposed two scheduling approaches.

Content-Centric Networking (CCN) is a newly proposed networking architecture for the future Internet. It has superiorities over today's IP-based architecture in some aspects. On the other hand, the Internet of Things (IoT) is a direction of the future Internet. Due to the existence of large amounts of weak network devices (NDs) with constrained resources, current CCN models that are designed mainly oriented to Super Routers (SRs) in the core networks are not appropriate for the edge networks in IoT. In this paper, we propose an internetworking scheme for weak NDs based on task mapping. In this scheme, weak NDs with constrained resources map the overcapacity tasks (in terms of storing, publishing, and retrieving) to SRs. Then, based on this scheme, we present two specific strategies for NDs as producers and consumers, respectively. Finally, we take cameras as typical NDs to illustrate the process of the communication by using our proposed scheme.

In recent years, more and more people have begun to focus on data delivery among mobile users through opportunistic networks, and in most cases, such as emergency, traffic accident and disaster, we need to transmit video data to other users by the means of opportunistic contacts between mobile users. Meanwhile, the buffers of mobile wireless devices are usually limited, then some messages which have different importance on video recovery, will be dropped during transmission. Therefore, it's imperative to design efficient buffer management strategy to improve the video delivery quality. Several policies have been presented, but they are just designed for general data and not suitable for video transmission. In this paper, we comprehensively take the temporal correlation of video data and the diffusivity of messages into account, and propose DSVM, a novel buffer management strategy. Extensive simulations validate its performance, and up to about 3dB Peak Signal-to-Noise Ratio (PSNR) gain can be achieved over the state-of-the-art buffer management policies.

Routing in opportunistic networks is difficult due to the intermittently connected environment and lack of global view on network topology. In such scenarios, nodes attempt to transmit packets in a store-carry-and forward manner. The main issue is which forwarding mechanism achieves the best trade off between successful packet delivery ratio and cost. We address this challenge by proposing the Fuzzy Forwarding (FF) approach, in which packets are preferred jointly considering the knowledge of performance metrics (e.g., the mean delivery delay and average number of hops per packet) and that of node contacts (e.g., durations, times and locations). FF gives higher priority to packets, as compared to the heuristic knowledge, if they have shorter TTL (time to live) and smaller number of hops, and nodes desiring to relay them have higher delivery probability to destination. Furthermore, to deal with the incompleteness and uncertainty of such knowledge, a fuzzy logic engine is developed for the computation of packet preference. The trace-driven simulation results demonstrate that FF achieves significantly better cost compared to the state-of-the-art works, while keeping better delivery ratios and mean delivery delays under different TTL requirements.

How to deliver live video streaming in vehicular ad hoc networks (VANETs) has been a hot topic in recent years. However, due to high vehicular speed, frequent disconnection and dynamic topology, it's difficult to guarantee the low-delay delivery of real-time data, and up to now, there exists no scheme that can deliver such kind of data with satisfied quality. In this paper, we take fully advantage of the existing wireless enabled traffic cameras, and propose CAVD, a novel live video streaming delivery strategy. As the core part of this strategy, we design an algorithm to adaptively select proper camera as Optimal Buffering Point (OBP) for each segment of live video streaming, and through the cooperations of these OBPs, the live video streaming could be delivered to a mobile target vehicle with the minimal start-up delay while satisfying the required visual quality and playback performance. Extensive simulations have been done on real-life taxi trace of Beijing, and the results show that our proposed strategy outperforms existing solutions greatly.

Opportunistic sensing is a new paradigm which exploits human-carried or vehicle-mounted sensors to collect data ubiquitously for large-scale urban sensing. Existing work lacks an in-depth investigation on the sensing quality of such sensing systems, which faces two basic problems: 1) how to measure the sensing quality? and 2) how many humans or vehicles are necessary to satisfy the sensing quality requirement of the whole urban area? To solve the first problem, we propose a metric called Inter-Cover Time (ICT) to characterize the opportunity with which a sub region is covered, which reflects the sensing quality directly. According to the empirical measurement studies on real mobility traces of thousands of taxis collected in Beijing and Shanghai, we find that the aggregated ICT Distribution (ICTD) closely resembles a truncated power-law distribution regardless of the size of sub regions and the number of vehicles. We also analyze the reasons behind this particular pattern by the evaluation on four known mobility models. To solve the second problem, we further propose a metric called opportunistic coverage ratio based on the ICTD to characterize the relationship between the sensing quality of an urban area and vehicle number. Our results provide fundamental guidelines on the measurement of sensing quality and network planning for opportunistic urban sensing applications.

With the popularity and advancements of smartphones, mobile users can sense the city using variety of sensors opportunistically, and forward the sensory data to the monitoring center for building sensing maps through intermittent connections with short-range communications. In order to build sensing maps satisfying specific sensing quality with low delay and energy consumption, we design COUPON, a novel cooperative sensing and data forwarding framework. We first notice that cooperative sensing scheme can eliminate sampling redundancy and hence save energy. Then we design two cooperative forwarding schemes by leveraging data fusion: Epidemic Routing with Fusion (ERF) and Binary Spray-and-Wait with Fusion (BSWF). Different from previous work assuming that all packets are propagated individually, we consider that packets are spatial-temporal correlated in the forwarding process, and derive the dissemination law of correlated packets. We theoretically prove that our cooperative forwarding schemes can achieve better tradeoff between delivery delay and transmission overhead. We evaluate our proposed framework and schemes with real mobile traces. Extensive simulations demonstrate that the cooperative sensing scheme can reduce the number of samplings by 93% compared with the non-cooperative scheme, ERF can reduce the transmission overhead by 78% compared with Epidemic Routing (ER), BSWF can increase the delivery ratio by 16%, and reduce the delivery delay and transmission overhead by 5% and 32% respectively, compared with Binary Spray-and-Wait (BSW).

Performance of data forwarding in opportunistic networks benefits considerably if one can make use of human mobility in terms of social structures. However, it is difficult and time-consuming to calculate the centrality and similarity of nodes by using solutions for traditional social networks, this is mainly because of the transient node contact and the intermittently connected environment. In this paper, we are interested in the following question: Can we explore some other stable social attributes to quantify the centrality and similarity of nodes? Taking GPS traces of human walks from the real world, we find that there exist two known phenomena. One is public hotspot, the other is personal hotspot. Motivated by this observation, we present Hoten (HOTspot ENtropy), a novel forwarding metric to improve the performance of opportunistic routing. First, we use the relative entropy between the public hotspots and the personal hotspots to compute the centrality of nodes. Then we utilize the inverse symmetrized entropy of the personal hotspots between two nodes to compute the similarity between them. Third, we exploit the entropy of personal hotspots of a node to characterize its personality. Besides, we propose a method to ascertain the optimized size of hotspot. Finally, we compare our routing strategy with state-of-the-art works through extensive trace-driven simulations, the results show that Hoten largely outperforms other solutions, especially in terms of packet delivery ratio and the average number of hops per message.

The use of wireless sensor networks (WSNs) for closing the loops between the cyberspace and the physical processes is more attractive and promising for future control systems. For some real-time control applications, controllers need to accurately estimate the process state within rigid delay constraints. In this paper, we propose a novel in-network estimation approach for state estimation with delay constraints in multihop WSNs. For accurately estimating a process state as well as satisfying rigid delay constraints, we address the problem through jointly designing in-network estimation operations and an aggregation scheduling algorithm. Our in-network estimation operation performed at relays not only optimally fuses the estimates obtained from the different sensors but also predicts the upper stream sensors' estimates which cannot be aggregated to the sink before deadlines. Our estimate aggregation scheduling algorithm, which is interference free, is able to aggregate as much estimate information as possible from the network to the sink within delay constraints. We proved the unbiasedness of in-network estimation, and theoretically analyzed the optimality of our approach. Our simulation results corroborate our theoretical results and show that our in-network estimation approach can obtain significant estimation accuracy gain under different network settings.

Diagnosis for wireless sensor networks is difficult, due to the limited resources and the ad hoc manner of networks. The existing approaches mainly focus on collecting diagnosis metrics, which bring heavy communication overhead to the network. We present a new model called DSD for network diagnosis which deduce the root causes for failures using the sensing data traces. We discover that the characteristics of the sensing data reflect the network status in some way, according to considerable experiments in the GreenOrbs project. We mine the relationships between the sensing data and the failures in the sensor networks, and record them in a failure knowledge library. Through this diagnosis mechanism, we deduce the root cause of the failures without adding any additional network burden. Moreover, the failure knowledge library can be used to improve the efficiency of diagnosis. We analyze the three months sensing data from the GreenOrbs project, and experimental results show that the proposed scheme can improve the diagnosis performance with low energy cost.

In Wireless Multimedia Sensor Networks (WMSNs), data fusion and collaborative in-network processing operations often require effective multimedia synchronization control. Extensive researches have been done in the traditional networks. Most of these works assume that there exists a powerful synchronization controller in the network. However, for WMSNs, the in-network processing of the multimedia content is usually performed based on the resource-constrained sensors. Traditional synchronization control algorithms fail to run on the low-end hardware platform. In this paper, we propose an adaptive synchronization control scheme with multi-level buffer to address this problem. Main contributions of this paper are as follows. (i) To overcome the resource constraint of WMSNs, we perform the synchronization control in a distributed way by combining the sensors in the stream transmission path. (ii) By evaluating the current network state, we adjust the level number of the buffer adaptively to balance the delay and delay jitter. (iii) Through analyzing the buffer state of each level, we propose an efficient packet scheduling scheme to reduce the delay jitter actively. (iv) We implement the proposed scheme and verify its effectiveness in our practical WMSNs platform. The experiment results show that the proposed scheme can maintain the streams synchronization effectively with the low-end sensors.

Semantic understanding from video is one of the most challenging tasks in video analysis. However, it has not been taken enough attention. In this paper, we focus on understanding the semantics of video in the driving scene. We present a coarse-to-fine method to parse the driving scene, and obtain the high-level semantic information of the scene. In the coarse phase, we divide the captured frame into four separate parts based on edge density entropy and scene context. In the fine phase, we join multi-class object segmentation and detection algorithms together in a unified Conditional Random Filed (CRF) model for each part understanding. Moreover, the object probabilistic location prior knowledge based on training and previous edge density entropy result is also integrated into our approach for better object localization. Experimental results show that our proposed method is effective comparing to current state-of-the-art approaches.

Real-time accurate crowd counting is one of important tasks in intelligent visual surveillance systems. Most previous works can only count passing people robustly without heavy occlusions which are very common in the practical surveillance scenes. To solve this difficult problem, we propose a new method for crowd counting for RGB-D (RGB plus depth) data using a commodity depth camera. In our method, we first detect each head-shoulder of the passing or still person in the surveillance region with fast template matching based on depth information including pedestrian filling with convex hull segmentation. Then, we track and count each detected head-shoulder based on RGB information bidirectionally. By using this approach, we have built a practical system for robust and fast crowd counting. Extensive experimental results show that our method achieves significant improvement comparing to states-of-the-art approach, and the built system is not only robust to heavy occlusions, but also can be deployed in the real time crowd counting application scenes.

With rapid improvements and miniaturization in hardware, sensor nodes equipped with acoustic and visual information collection modules promise an unprecedented opportunity for target surveillance applications. This paper investigates a critical task of target surveillance, multi-class classification, in distributed multimedia sensor networks. We first analyze the procedure of target classification utilizing the acoustic and visual information. Then, we propose a binary classification tree based framework for distributed target classification in multimedia sensor networks. The proposed framework includes three main components: Generation of binary classification tree, Division of binary classification tree, and Selection of multimedia sensor nodes. Finally, we conduct an experimental application of target classification and extensive simulations to validate and evaluate our proposed framework and related schemes.

Using insights from biological processes could help to design new optimization techniques for long-standing computational problems. This paper exploits a cellular computing model in the slime mold physarum polycephalum to solve the minimal exposure path problem which is a fundamental problem corresponding to the worst-case coverage in wireless sensor networks. We first formulate the minimal exposure path problem, and then convert it into the shortest path problem by discretizing the monitoring field to a large-scale weighted grid. Inspired by the path-finding capability of physarum, we develop a new optimization algorithm, named as the physarum optimization, for solving the shortest path problem. Our proposed algorithm is with low-complexity and high-parallelism. Moreover, the core mechanism of our physarum optimization is also helpful for designing new graph algorithms and improving routing protocols and topology control in self-organized networks.

Most existing works on sensor coverage mainly concentrate on the full coverage models which ensure that all points in the deployment region are covered at the expense of high complexity and cost. In contrast, the exposure-path prevention does not require full coverage sensor deployment, and instead it only needs the partial coverage, because the exposure paths are prevented as long as no moving objects or phenomena can go through a deployment region without being detected. Towards this end, we focus on the partial coverage by applying the percolation theory to solve the exposure path problem for wireless sensor networks. Specifically, we propose a bond-percolation based scheme by mapping the exposure path problem into a bond percolation model. Using this model, we derive the analytical expressions of critical densities for wireless sensor networks under random sensor deployment.

This work presents BurTrap, a networking system which integrates wireless modules (such as TelosB nodes) with networked surveillance cameras to automatically, accurately, timely track and identify burglar who stole the property. First, we design an energy-efficient wakeup scheduling protocol that guarantees a successful target tracking while reducing the communication energy consumption of the portable wireless module. Then, we identify burglar among all the objects appeared in the obtained video information by performing trajectory fitting between the estimated geometric trajectory and the estimated local visual trajectory. Through extensive experiments, we show that BurTrap can pinpoint the burglar with extremely high accuracy.

Nature is an invaluable source of inspiration for long-standing computational problems. Genetic algorithms and ant colony optimization are the well-known successes. Drawing the inspirations from these successful experience, we exploit the computing model in a single-celled organism -- physarum polycephalum, and further design a new heuristic algorithm, Physarum Optimization, to the minimal exposure problem of wireless sensor networks. More importantly, our proposed physarum optimization algorithm can be also applied to the general Steiner tree problem.

Exposure problem, which corresponds to the quality of coverage, is a fundamental problem in wireless sensor networks. In this paper, we exploit a cellular computing model in the physarum for solving the minimal exposure problem. We first use the road-network among all points of interesting (PoIs) in the monitoring filed to formulate the minimal exposure problem, and then convert it into the Steiner tree problem by discretizing the monitoring field to a large-scale weighted grid. Inspired by the path-finding capability of physarum, we develop a new heuristic algorithm, named as the physarum optimization, to solve the Steiner tree problem with low complexity and high parallelism. Extensive simulations demonstrate that our proposed models and algorithm are effective for finding the road-network with minimal exposure.

Performance modeling of epidemic routing is challenging because of the unguaranteed end-to-end connectivity and lack of global information in delay-tolerant scenarios. Existing works analyze the scaling law of epidemic protocol based on the assumption that each node has the same infectivity. Whereas, the most recent work indicates that the distribution of infected nodes has spatial-temporal correlation rather than homogeneity, i.e., nodes in different locations have different infectivities, which leads to defectiveness of the existing solutions. In this paper, by exploring the reason behind this difference, we try to relax the assumption and rebuild the model for epidemic routing. We first introduce the concept of infection rate to reflect the infectivity of infected nodes. Second, we propose an effective method to compute the average infection rate and use it to derive a generic scaling law. Third, we give an explicit expression for the generic scaling law, which provides us with upper bound. We finally compare our model with the existing works through theoretical analysis and simulations. The results show that our model has a closer match than those of the existing works and gets some insights into the spatial distribution of infection process.

In this paper we study multi-copy routing schemes in opportunistic networks. Compared to single-copy protocols, the multi-copy schemes expedite the mean delivery delay while consuming more resources and exposing lower packet delivery ratio in resource-constrained system. To enhance the system capacity, most recent works explore the impact of social structure on network performance. Their results indicate that integrating the social relationship into opportunistic routing can greatly improve the performance metrics especially in term of packet delivery ratio. Considering this fact, we propose Hug, a “HUman Gathering point” assisted routing for delay-tolerant environments and evaluate it with the other two earlier presented Spray-wait and Epidemic routing protocols through simulations and theoretical analysis. Our results show that Hug achieves higher packet delivery ratio and lower communication overhead than Spray-wait and Epidemic routing while keeping a reasonable delay.

Mobile sensors are a viable choice for providing monitoring service on a set of Points of Interest (PoIs) in a large sensing field. In some applications, each PoI should be covered periodically (sweep coverage), and the collected data should be delivered to the sink node timely (timely transmission), namely both the sensing and transmission delay constraints for each PoI should be satisfied, which we call as timely sweep coverage. We investigate how to optimize the movement path of one mobile sensor to satisfy the two delay constraints for each PoI, so that the required movement velocity for the mobile sensor is minimized. We consider two cases: 1) all PoIs are placed along a straight line (linear case), and 2) all PoIs are arbitrarily placed on a plane (general 2-D case). Under the linear case, the optimal algorithm is presented. Under the general 2-D case, we prove the problem is NP-hard, and two algorithms, STSP and ITSP, are presented. We prove that the approximation ratio of STSP depends on the ratio between the sensing and transmission delay constraints. The ITSP can improve the solution much especially when the two delay constraints differ greatly for each PoI. Extensive simulation results are provided to evaluate our algorithms.

Probabilistic forwarding methods have been exploited in opportunistic networks to reduce the overhead of epidemic routing. However, most existing methods make all the nodes forward messages with the same probability (i.e., equal scheme), which causes the energy unbalance of nodes. To guarantee the energy balance of nodes and prolong the network lifetime, we design a differentiated scheme, i.e., different nodes are assigned with different forwarding probabilities based on their respective energies. We model the message dissemination based on the differentiated scheme, and formulate two optimization problems: maximize the message deliver probability under the constraint on the total energy consumption, and based on this, maximize the network lifetime under the constraint on the energy consumption of each node. By solving these two optimization problems, we derive the optimal differentiated forwarding probabilities by theoretical analysis. Our simulation results show that our designed differentiated scheme can guarantee the message deliver probability and extend the network lifetime, compared with the equal scheme.

Reliable and real-time people counting is crucial in many applications. Most previous works can only count moving people from a single camera, which cannot count still people or can fail badly when there is a crowd (i.e., heavy occlusion occurs). In this article, we build a system for robust and fast people counting under occlusion through multiple cameras. To improve the reliability of human detection from a single camera, we use a dimensionality reduction method on the multilevel edge and texture features to handle the large variations in human appearance and poses. To accelerate the detection speed, we propose a novel two-stage cascade-of-rejectors method. To handle the heavy occlusion in crowded scenes, we present a fusion method with error tolerance to combine human detection from multiple cameras. To improve the speed and accuracy of moving people counting, we combine our multiview fusion detection method with particle tracking to count the number of people moving in/out the camera view (“border control”). Extensive experiments and analyses show that our method outperforms state-of-the-art techniques in single- and multicamera datasets for both speed and reliability. We also design a deployed system for fast and reliable people (still or moving) counting by using multiple cameras.

Deriving the proper density to achieve the region coverage for random sensors deployment is a fundamentally important problem in the area of wireless sensor networks. Most existing works on sensor coverage mainly concentrate on the two-dimensional (2D) plane coverage which assume that all the sensors are deployed on an ideal plane. In contrast, sensors are also deployed on the three-dimensional (3D) rolling surfaces in many real applications. Toward this end, we study the coverage problem of wireless sensor networks for the rolling terrains, and derive the expected coverage ratios under the stochastic sensors deployment. According to the different terrain features, we investigate two kinds of terrain coverage problems: the regular terrain coverage problem and the irregular terrain coverage problem. Specifically, we derive the general expression of the expected coverage ratio for an arbitrary surface z=f(x, y) and build two models, cone model and Cos-revolution model, to estimate the expected coverage ratios for regular terrains. For irregular terrains, we propose a digital elevation model (DEM) based method to calculate the expected coverage ratio and design an algorithm to estimate the expected coverage ratio of an interested region by using only the contour map of this region. We also conduct extensive simulations to validate and evaluate our proposed models and schemes.

Data collection is a fundamental task in wireless sensor networks. In many applications of wireless sensor networks, approximate data collection is a wise choice due to the constraints in communication bandwidth and energy budget. In this paper, we focus on efficient approximate data collection with prespecified error bounds in wireless sensor networks. The key idea of our data collection approach ADC (Approximate Data Collection) is to divide a sensor network into clusters, discover local data correlations on each cluster head, and perform global approximate data collection on the sink node according to model parameters uploaded by cluster heads. Specifically, we propose a local estimation model to approximate the readings of sensor nodes in subsets, and prove rated error-bounds of data collection using this model. In the process of model-based data collection, we formulate the problem of selecting the minimum subset of sensor nodes into a minimum dominating set problem which is known to be NP-hard, and propose a greedy heuristic algorithm to find an approximate solution. We further propose a monitoring algorithm to adaptively adjust the composition of node subsets according to changes of sensor readings. Our trace-driven simulations demonstrate that ADC remarkably reduces communication cost of data collection with guaranteed error bounds.

ABSTRACT Camera sensor networks have recently emerged as a critical research topic. In this paper, we investigate the coverage problem for camera sensor networks. Specially, compared to the coverage problem for target detection which has been intensively studied, this paper studies the coverage problem from the perspective of target localization. We first propose a novel localization‐oriented sensing model based on the perspective projection of the camera sensors. Then, under the random uniform deployment strategy, we analyze how the probability of the localization‐oriented coverage (L‐coverage for short) changes with the sensors number and the parameters of the proposed sensing model. Finally, we conduct extensive simulations to validate our model and theoretical analysis about L‐coverage probability. The obtained results show that our scheme can be effectively applied for practical scenarios. Copyright © 2011 John Wiley & Sons, Ltd.

Multimedia sensor networks (MSNs), which allow capturing acoustic and visual information, provide an unprecedented opportunity for variety of applications. This paper investigates the heterogeneous coverage problem in MSNs, i.e., how many multimedia sensors should be deployed to guarantee that each point in the monitored region is covered by multiple types of heterogeneous sensors. It is different from the coverage problem in conventional homogeneous sensor networks, mainly because that it is based on a heterogeneous sensing model, which is a hybrid of the omni-sensing model and the directional-sensing model. We propose a mathematical model to describe the relationships among the number of multimedia sensors, the sensing radius of the acoustic collection module, the sensing radius and the sensing angle of the visual collection module and the heterogeneous coverage rate. Our simulation results show that deploying our model in practical scenarios is effective.

Recently, the view-based 3D model retrieval methods have received great research attentions. However, these methods are difficult to preserve the spatial structure of 3D models. In this paper, we propose a novel view-based 3D model retrieval method to solve this problem. Our method is based on the two-level (3D model-level and 2D image-level) spatial structure. Firstly, we extract the spatial structure circular descriptor (SSCD) images from 3D models. The SSCD images can preserve the spatial structure on the 3D model-level. Then, we modify the bag-of-features (BOF) method to extract view-based features from these SSCD images. The modified BOF method can preserve the spatial structure on the 2D image-level. Finally, we calculate the similarity between the query model and the models in the databases by adapting the earth mover distance method. Experimental results show that our method can achieve satisfactory retrieval performance for both the articulated models and the rigid models.

This paper presents a fast and accurate pedestrian detection method. To find a balance between speed and accuracy, we propose a Multi-Pose Learning Boosted Integrable Features Pool (MPL-Boosted IFP) approach. Our method achieves high recall-rate while taking the speed-advantage of cascade-of-rejectors approach. We build different types of feature sets, in which features are extremely fast to compute by using integral image. These features are used for building a large number of candidate weak classifiers by using linear SVM. Finally, MPL-Boost method selects the best weak classifiers suited for detection and construct the rejector-based cascade detector. The experiment results show our method achieve better detection precision than HOG and HOG-LBP classifier, meanwhile, speed up these methods near 30 times.

Human detection has attracted much attention in recent years due to its widespread applications. Most existing multi-camera systems focus on locating moving people in each camera and thus resolve the occlusion, which cannot detect still people in images. To overcome this problem, we extend previous 3D search method from single camera to multi-camera. We first use the method of multiple view geometry to construct the 3D search grid. Each grid point on the 3D ground plane is represented by a cylinder. Then, we re-project these cylinders to each view and classify the re-projected sub-images. Finally, by fusing the detected results from each camera, we can detect still people accurately and handle occlusion effectively. Experiments show that our method is comparable to state-of-the-art techniques on challenging datasets, without assuming that people are moving.

Moving object detection is one of the most important tasks in intelligent visual surveillance systems. Gaussian Mixture Model (GMM) has been most widely used for moving object detection, because of its robustness to variable scenes. However, to the best of our knowledge, existing GMM based methods can not detect moving objects which gradually stop and keep still state for a while. In this paper, we present an Enhanced Gaussian MixtureModel, called EGMM, to handle this problem. We integrate an Initial Gaussian Background Model (IGBM) and an extended Kalman filter based tracker with GMM, to enhance its performance. Experimental results show that our EGMM based method has a lower miss rate at the same false positives per image comparing to GMM based method for moving pedestrian detection, and it also has a higher detection rate for abandoned object detection comparing to GMM based method.

Shot Boundary Detection (SBD) is the fundamental process in video processing area. However, according to the most existing SBD methods [1], [2], researchers have paid much attention to detect the shot boundaries as accurately as possible with expensive computation cost. In this paper, we propose an approach to accelerate the SBD process by reducing both spatial pixels and temporal frames while keeping satisfactory performance. Our method accelerates the SBD process mainly from two points of view. In spatial domain, the proposed approach only uses the pixels falling into the defined focus region rather than all pixels in a frame; in temporal domain, a step-skip method is employed to reduce the processed frames. Through reducing both detection region and scope, using the corner distribution of frames to remove most of false boundaries, the speed of our SBD approach is improved obviously. We conduct extensive experiments to evaluate the proposed approach, and the results show that our approach can obviously accelerate the SBD process by reducing the computation complexity while keeping satisfactory performance.

With the rapidly increasing of 3D models, the view-based 3D model retrieval methods have received significant research attention. The previous view-based methods can achieve ideal retrieval result for the rigid models, but they only obtain poor retrieval result for the deformable models because they can not preserve the topological structure well. In this paper, we propose a view-based 3D model retrieval algorithm using topological structure. We extract the view-based features from the images rendered at the salient topological points. To preserve the topological structure of the 3D model, a multiresolutional reeb graph (MRG) is constructed according to the salient topological points. We take the view-based features as the attribute information of the corresponding MRG nodes. The comparison between two 3D models is transformed to compute the similarity of the corresponding MRGs. Experimental results on two standard benchmarks show that our algorithm can achieve satisfactory retrieval performance for both the deformable models and the rigid models.

We address the problem of pedestrian detection in still images. Current pedestrian detection systems are hard to improve both speed and accuracy simultaneously. In order to achieve a balance between speed and accuracy, we propose a novel MPL-Boosted cascade of weak FIK-SVM classifiers. Our method achieves high recall while taking the speed-advantage of cascade-of-rejectors approach. Each feature in our algorithm corresponds to a 66-D HOG-LBP feature vector that describe a block. The weak classifiers we use are the separating hyper-plane computed by using a FIK-SVM. We use MPL-Boost to select features from a large set of possible blocks. The integral image and convoluted trilinear interpolation are used for rapid calculation of block feature. For a 320×240 image, the system can process 16 frames per second with sparse scan, while defeat the accuracy level of existing methods.

Data collection is a fundamental and important issue in wireless sensor networks. Recent research has shown that using mobile nodes to collect and carry data in WSNs has many advantages over static multi-hop routing. In this paper, we focus on the problem of reducing the number of mobile nodes require by data collection in WSNs. Each mobile node visits the sensor nodes, that assigned to it, every t seconds to pick up the cached data. The data gathered by mobile nodes far away from the base station are transferred to the base station through mobile nodes closer to the base station. We formally prove that the problem of data collection in WSNs with the minimum mobile nodes is NP hard. In order to reduce the number of mobile nodes, We propose two algorithms. One tries to reduce the number of mobile nodes and optimize their travel paths, and the other one carefully schedules the movement of all mobile nodes to make sure that they can cooperate with each other to transfer all gathered data to the base station. Our simulation results show that our approach can notably reduce the number of required mobile nodes as much as 46%.

Target tracking is a main application of wireless sensor networks (WSNs), and has been studied widely [4], [10]. In this work, we study indoor passive tracking problem using WSNs, in which we assume no equipment is carried by the target and the tracking procedure is passive. We propose to use light to track a moving target in WSNs. To our best knowledge, this is the first work which tracks a moving object by using light sensors and general light sources. We design a novel probabilistic protocol (system) iLight to track a moving target and several efficient methods to compute the target's moving patterns (like height, etc.) at the same time. We implement and evaluate our tracking system iLight in a testbed consisting of 40 sensor nodes, 10 general light sources and one base station. Through extensive experiments, we show that iLight can track a moving target efficiently and accurately.

A critical function of wireless sensor networks (WSNs) is data gathering. While, one is often only interested in collecting a relevant function of the sensor measurements at a sink node, rather than downloading all the data from all the sensors. This paper studies the capacity of computing and transporting the specific functions of sensor measurements to the sink node, called aggregation capacity, for WSNs. It focuses on random WSNs that can be classified into two types: random extended WSN and random dense WSN. All existing results about aggregation capacity are studied for dense WSNs, including random cases and arbitrary cases, under the protocol model (ProM) or physical model (PhyM). In this paper, we propose the first aggregation capacity scaling laws for random extended WSNs. We point out that unlike random dense WSNs, for random extended WSNs, the assumption made in ProM and PhyM that each successful transmission can sustain a constant rate is over-optimistic and unpractical due to transmit power limitation.We derive the first result on aggregation capacity for random extended WSNs under the generalized physical model. Particularly, we prove that, for the type-sensitive perfectly compressible functions and type-threshold perfectly compressible functions, the aggregation capacities for random extended WSNs with n nodes are of order Θ ((log n)-β/2-1) and Θ (((log n)-β/2)/(log log n)), respectively, where β >; 2 denotes the power attenuation exponent in the generalized physical model.

We evaluate the performance of epidemic protocol in opportunistic networks. Early works follow an independent model which assumes that different messages in the network disseminate independently. Whereas, most recent work has pointed out that the same event can be detected by multiple nodes at different locations or different moments. Hence, the messages sensed by different nodes, indicating the same event, have spatial-temporal correlations. It is necessary to build an accurate mathematical model for reflecting the message dissemination process of epidemic protocol for further designing or optimizing the family of flooding protocols. However, the independent model does not provide good performance estimates in this situation. In this paper, we try to solve the problem by using a correlated model which takes into account the correlations among messages and permits the different messages denoting the same event to be aggregated in their propagation processes, and the numerical results show a close match with our theoretical analysis. Compared to the previous work, our correlated model, on the one hand, allows a network engineer to implement such a system with optimized performance confidently in an intermittently connected environment, on the other hand, our work offers a fresh insight into the spatial-temporal correlations of the messages and achieves good performance metrics in scalability.