Inter-agent communication serves as an effective mechanism for enhancing performance in collaborative multi-agent reinforcement learning (MARL) systems. However, the inherent communication latency in practical systems induces both action decision delays and outdated information sharing, impeding MARL performance gains, particularly in time-critical applications like autonomous driving. In this work, we propose a Value-of-Information aware Low-latency Communication (VIL2C) scheme that proactively adjusts the latency distribution to mitigate its effects in MARL systems. Specifically, we define a Value of Information (VoI) metric to quantify the importance of delayed messages on the recipient agent's decision. We then design a VoI aware resource allocation method that dynamically prioritizes message transmission based on each delayed message's importance. Moreover, we propose a progressive message reception mechanism to adaptively adjust the reception duration based on received messages. We derive the optimized VoI aware resource allocation and theoretically prove the performance advantage of the proposed VIL2C scheme. Extensive experiments demonstrate that VIL2C outperforms existing approaches under various communication conditions. These gains are attributed to the low-latency transmission of high-VoI messages via resource allocation and the elimination of unnecessary waiting periods via adaptive reception duration.

Ke Ma, Jiaqi Tang, Bin Guo, Xueting Han, Ruonan Xu, Qingfeng He, Ziheng Wang, Xu Wang, Qifeng Chen, Zhiwen Yu, Yunhao Liu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Timing-driven global placement plays a decisive role in the final performance of very large-scale integration (VLSI) circuits, but is consistently challenged by the trade-off between design accuracy and efficiency. Most existing methods rely on coarse-grained net-weighting strategies. While these approaches are straightforward to implement, they cannot precisely identify and optimize complex timing paths, such as paths with sharing effects or large slack deviations. To overcome this bottleneck, we propose a high-precision slack-aware global placement engine called HPPlacer, which includes the following three key techniques: 1) a local clock buffer-to-flip-flop connection optimization method, 2) a path-level differentiable timing optimization model, and 3) a dynamic adjustment mechanism-based pin-pair weighting strategy. With the proposed method, efficient chip placement with excellent timing behaviors can be generated automatically within a short period of time. The experimental results on multiple benchmark circuits confirm that HPPlacer leads to significant improvements in both timing performance and wirelength compared to state-of-the-art placement tools.

The rise of vehicle-infrastructure (V2I) collaboration enables safer and broader perception. To process large-scale V2I video streams, vision-language models (VLMs) are promising as they unify multi-view vision into end-to-end task grounding, reducing handcrafted design. We use Vision Mixture-of-Experts (V-MoE) as the distributed visual backbone of VLMs, leveraging sparse expert routing to enable conditional computation across diverse viewpoints under resource constraints. Yet, V-MoEs face a critical challenge: large-scale data shifts over minutes to hours in V2I systems, amplified by agnostic participants and biased features propagating through experts. To maintain accuracy efficiently, we find it beneficial to co-adapt multiple V-MoEs on edge servers, avoiding the latency and privacy risks of cloud offloading and the accuracy sacrifices of on-device methods. However, the resource-constrained edge poses challenges for efficient co-adaptation: i) DRAM fragmentation and imbalance limit expert parallelism, ii) memory-I/O bottlenecks restrict computation reuse, and iii) asynchronous adaptation increases task-switch overhead. Also, prior work rarely explores the upper bound of concurrent tasks under limited edge resources, a critical factor for practical V2I deployment. To address these, we present AdaSprite. By combining cooperative elastic scaling with multi-level multiplexing, AdaSprite optimizes expert lifespans to reduce DRAM fragmentation, exploits predictable activation patterns for efficient I/O reuse, and employs twin-buffer scheduling to leverage sparsity. On a weak edge, AdaSprite supports up to 17 concurrent V2I tasks (vs. up to 6 for baselines), improving SLO attainment by 1.6x and throughput by 2.1x. Also, it allows users to trade accuracy and concurrency for second-level adaptation.

Modern smart manufacturing pipelines have pervasively collaborated human workers, mobile robots, and industrial Internet of Things (IIoT) in shared workspaces for versatile production tasks. Despite the promising capacity of individual entities, the performance of these IIoT systems largely relies on pipeline coordination, i.e., task dispatching between humans and robots, which is particularly challenging under heterogeneous physical constraints and complex environmental uncertainties. Nonetheless, existing works either rely on traditional operation frameworks that lack scalability for large-scale complex production, or propose customized solutions for fixed agent models, overlooking the evolving nature of IIoT environments. To address these limitations, this paper proposes COACH, a human-robot collaborative manufacturing system that enables robust constraint-aware coordination across humans, robots, and IIoT. Specifically, COACH designs a scalable contextual encoder to represent the evolving relationships among human and robot agents in dynamic heterogeneous graphs. With that, a novel experience-driven task dispatcher is developed, enabling both high-performance and computation-efficient policy generation concerning the status of IIoT. To accommodate changing human fatigue and pipeline scales, COACH further develops a curriculum-enhanced reinforcement learning module for efficient dispatcher adaptation. Extensive evaluations using both synthetic testbeds and real-world manufacturing datasets demonstrate that COACH improves the feasible ratio of manufacturing pipelines by up to 27.4% and achieves up to 13.9% improvement in time efficiency compared to competing baselines across diverse job scales and environmental settings.

LLM-based empathetic dialogue systems enhance agents' emotional support capabilities. Previous approaches primarily relied on Chain-of-Thought (CoT) prompting to extract key dialogue cues and further strengthened the agent's sensitivity to these signals through supervised fine-tuning. However, such methods overly depend on the information extraction capability of LLMs, leading to unstable reasoning and limited interpretability. To simultaneously improve an agent's ability to proactively explore solutions through rational reasoning while attending to users' sensitive emotions via empathetic understanding, we propose Neuro-Sym Supporter, a hybrid decision-making emotional support agent that integrates symbolic reasoning with deep learning. This model combines rational inference with emotional empathy, enabling the agent to generate supportive responses that balance logic and emotion. Specifically, we introduce Sym-Mind, a differentiable logic-based reasoning framework for emotional support strategy selection, which unifies interpretability with stable performance. Experimental results on public datasets demonstrate that our approach consistently outperforms multiple competitive baselines in both automatic and human evaluations, validating its effectiveness.

The proliferation of Internet of Things (IoT) devices is enabling transformative applications across domains such as smart cities, healthcare, and environmental monitoring. However, the large-scale deployment of these devices also raises sustainability concerns. The integration of intelligence at the edge through the emergence of the Artificial Intelligence of Things (AIoT) amplifies computational requirements and energy consumption. This increase is especially pronounced in resource-constrained settings, consequently exacerbating their ecological impacts. To address these energy and environmental challenges, recent research has increasingly focused on energy harvesting (EH) techniques, energy-efficient sensing methods, and ultra-low-power computing architectures. This survey provides a comprehensive review of the current state of ecological and sustainable AIoT systems powered by energy harvesting. It systematically categorizes existing approaches based on energy sources, hardware-software co-optimization strategies, and communication protocols. In addition, it critically analyzes key challenges, particularly energy intermittency, system scalability, and the balance between performance and energy efficiency. This survey also highlights promising research directions, such as bio-inspired energy harvesting, neuromorphic computing, and integrated sensing-communication frameworks. By summarizing current advances and highlighting open research challenges, this survey aims to provide a structured roadmap for the development of next-generation sustainable AIoT systems.

Ubiquitous wireless sensing has facilitated a variety of intelligent applications, yet its widespread deployment is constrained by manual calibration of the node locations, whereas existing localization methods generally depend on multiple fixed anchors for device-free topology estimation or require dedicated transceivers for device-based schemes, which limit accuracy and practical scalability. In this paper, we proposeNodeLoc, a universal two-stage node localization framework that integrates local topology estimation with global calibration through robot-assisted alignment, achieving multi-node localization with only one fixed anchor and without requiring any receivers on the robot. Specifically, the framework first establishes coarse-grained relative topology through inter-node signal measurements, and then employs the robot’s trajectories to refine global positions. Furthermore, to address inherent challenges of node localization (i.e., the non-uniqueness of local topology and the ambiguity in global position mapping), we design an optimized localization algorithm by exploring the geometric constraints and trajectory calibration of multiple receivers. A prototype system based on the proposed framework is implemented and evaluated in real-world indoor environments. Experimental results demonstrate that the system achieves 80% localization errors within 0.61mand orientation errors within 18.7°, while providing higher accuracy and significantly improved robustness and scalability compared to state-of-the-art approaches.

Rapid advances in microfluidics technologies have facilitated the emergence of highly integrated lab-on-a-chip (LoC) biochip systems. With such a coin-sized biochip, complicated bioassay procedures can be executed efficiently without any human intervention. To ensure the correctness of assay outcomes, however, cross-contamination among different fluid samples and reagents needs to be dealt with separately during assay execution. As a consequence, washing operations have to be introduced and a washing path network needs to be established on the chip to remove the residues left behind in flow channels/devices. Also, chip drying after washing operations is crucial for maintaining some properties (e.g., pH values) of the subsequent reagents, so that precision degradation caused by residual buffer fluids can be avoided for those concentration-sensitive assays. To realize optimized assay procedures, we consider both washing operations and chip drying for the first time and propose an integer linear programming (ILP)-based path-driven washing and drying cooptimization method called PathDriver-WD for continuous-flow LoC biochip systems. The proposed method includes the following four key techniques: 1) The necessity of contamination removals and channel drying is analyzed systemically to avoid unnecessary washing and drying operations, 2) washing and drying operations are integrated with the regular removal of excess fluids, so that extra channel occupation can be minimized, 3) practical computation models are adopted to evaluate the durations of different washing and drying operations, and 4) optimized washing/drying paths and time windows are computed and assigned so that the completion time of assays can be minimized. Simulation results on multiple benchmarks demonstrate that the proposed method leads to highly efficient washing and drying procedures as well as minimized assay completion time.

With the popularity of mobile devices, a variety of motion sensors are integrated to enhance the user experience. Although existing studies demonstrated that non-acoustic motion sensors can be attacked by adversaries, they overlook the limited sampling frequencies of motion sensors (e.g., < 500 Hz) in mobile devices and are evaluated in the controlled laboratory settings. In this article, we explore a new attack model on non-acoustic motion sensors based on the off-the-shelf mobile devices. We propose a general framework named VoiceFormer to synthesize high-fidelity speeches based on the vibrations of accelerometers and gyroscopes with a low sampling frequency. Specifically, in VoiceFormer , we introduce a signal alignment approach to remove the time offsets between two nonsynchronous signals, and leverage Time Interleaved Analog-Digital-Conversion (TI-ADC) to generate a high-frequency synthetic signal (e.g., > 8 KHz) based on the vibration signals of accelerometers and gyroscopes on the same motherboard. To synthesize the high-fidelity acoustic waveforms, we propose a wavelet-based generative adversarial network to learn the spatiotemporal latent mapping between vibrations and original speech signals. Extensive experimental results demonstrate the feasibility of voice synthesis by spying the low-frequency non-acoustic motion sensors in off-the-shelf mobile devices. VoiceFormer shows impressive performance in the synthesized acoustical signals with a Mean Opinion Score of 3.38. Although there are significant differences of mobile devices in hardware settings, VoiceFormer shows robust performance in synthesizing intelligible voice signals. Our results suggest that eavesdropping an off-the-shelf mobile device remotely by fusing non-acoustic sensors is feasible.

The proliferation of fake reviews, often produced by organized groups, undermines consumer trust and fair competition on online platforms. These groups employ sophisticated strategies that evade traditional detection methods, particularly in cold-start scenarios involving newly launched products with sparse data. To address this, we propose theDiversity- andSimilarity-awareDynamicGraphAttention-enhancedGraphConvolutionalNetwork (DS-DGA-GCN), a new graph learning model for detecting fake reviewer groups. DS-DGA-GCN achieves robust detection since it focuses on the joint relationships among products, reviews, and reviewers by modeling product-review-reviewer networks. DS-DGA-GCN also achieves adaptive detection by integrating a Network Feature Scoring (NFS) system and a new dynamic graph attention mechanism. The NFS system quantifies network attributes, including neighbor diversity, network self-similarity, as a unified feature score. The dynamic graph attention mechanism improves the adaptability and computational efficiency by captures features related to temporal information, node importance, and global network structure. Extensive experiments conducted on two real-world datasets derived from Amazon and Xiaohongshu demonstrate that DS-DGA-GCN significantly outperforms state-of-the-art baselines, achieving accuracies of up to89.8% and 88.3%, respectively.

Adapting the on-device deep learning (DL) models to continual and unpredictable domain shifts is crucial for mobile applications like autonomous driving and augmented reality, which require seamless user experiences in ever-changing environments. Test-time adaptation (TTA) offers a promising solution that fine-tunes models withunlabeledreal-time data just before making predictions. However, TTA introduces a significant challenge, its forward-backward-reforward pipeline increases latency, compromising responsiveness in time-sensitive mobile scenarios. In this paper, we present AdaShadow, a responsive test-time adaptation framework designed for non-stationary mobile data distributions and resource dynamics. AdaShadow focuses on selectively updating only adaptation-critical layers, minimizing the latency impact. While this idea has been explored in general on-device training, the unsupervised and real-time nature of TTA presents unique challenges in assessing layer importance, predicting latency, and planning layer updates efficiently. AdaShadow addresses these challenges through abackpropagation-free importance assessorfor rapid identification of critical layers, a unit-basedruntime latency predictorthat accounts for resource constraints, and anonline layer update schedulerto ensure prompt retraining. Furthermore, a memory I/O-aware computation reuse scheme optimizes the reforward pass to further reduce delays. Our evaluations show that AdaShadow reduces adaptation latency by up to 3.7× (achieving ms-level response time) and improves accuracy by up to 25.4%, all while maintaining low memory and energy costs across CNN, and ViT models, outperforming state-of-the-art methods.

Instant delivery has become an essential service in daily life, requiring strict delivery timelines. However, traditional delivery methods that employ human couriers struggle to meet the soaring delivery demands due to labor shortages. While researchers have explored alternative solutions using ground vehicles (e.g., crowdsourced taxis) and Unmanned Aerial Vehicles (UAVs), their inherent limitations, such as constrained delivery detour for crowdsourced taxis and limited battery capacity of UAVs, greatly constrain their effectiveness. To address these challenges, this paper proposes a novel air-ground delivery paradigm that cooperatively integrates UAVs and crowdsourced taxis. First, UAV stations are strategically deployed based on delivery gaps between the delivery demands and taxis' delivery capacity, instead of delivery demands only; Then, a predictive UAV repositioning strategy is designed to bridge instantaneously dynamic delivery gaps. Thereafter, a transfer learning-based (TL-based) algorithm that mines the delivery knowledge of human couriers is designed to optimize the cooperative performance. This algorithm extracts behavioral insights from human couriers and transfers them to enhance the delivery capabilities of UAVs and taxis. Finally, parcel assignment is formulated as optimization problems aimed at maximizing total preferences of UAVs and taxis, and maximizing delivery number while minimizing cost, respectively. Evaluations on real-world datasets demonstrate that the proposed method delivers 27.4% more parcels, saves 19.2% delivery cost, and preserves 36.3% more of the travel experience of taxi passengers than the state-of-the-art (SOTA) air-ground cooperative approach for instant delivery.

UAV-enabled edge storage systems provide data storage services to users by deploying UAVs in areas lacking infrastructure coverage, overcoming delay limitations and improving Quality of Service (QoS). Most existing studies focus on storing replicas on UAVs to ensure low-latency data access. Nonetheless, replica-based strategies incur high storage cost, posing significant challenges for UAVs with limited storage resources. In this paper, we introduce erasure coding into the UAV-enabled edge storage system, aiming to reduce user data access latency while minimizing storage cost. However, the mobility of users and the non-fully-connected nature of the UAV network pose new challenges for the coupled decisions of data encoding, block placement, and access. In this paper, we propose a Mobility-Enhanced Hierarchical Deep Reinforcement Learning algorithm (ME-HDRL). Specifically, we design a trajectory prediction algorithm combining CNN and ConvLSTM to account for user mobility in decision-making. We further decompose the original problem into two subproblems: data encoding and placement, as well as block access. A hierarchical deep reinforcement learning algorithm involving multiple UAV agents and an edge agent is proposed to collaboratively learn optimal decisions. To improve the convergence of the algorithm, we design an invalid action filter to reduce the action space. Experimental results show that our approach outperforms existing rule-based and reinforcement learning-based algorithms in various scenarios, exhibiting significant convergence improvements and a substantial reduction in both storage cost and user data access latency.

With increasing computational demands and limited network resources in marine environments, efficient service caching and task offloading have become critical. In such environments, Autonomous Underwater Vehicles (AUVs) rely on Unmanned Surface Vehicles (USVs) as relays, forming a cross-domain network comprising underwater acoustic and above-water RF links. However, the heterogeneity in bandwidth, latency, and bit error rates introduces challenges for reachability analysis and delay estimation. This paper addresses the joint optimization of caching, task offloading, and resource allocation in a cross-domain marine network composed of offshore base stations, USVs, and AUVs. To tackle the inherent heterogeneity in network links and decision timescales, we formulate the problem as a two-time-scale Hierarchical Markov Decision Process (H-MDP) and propose a Two Time-Scale Deep Reinforcement Learning (T2S-DRL) approach that integrates a hybrid policy network and a lightweight structure-aware action masking mechanism. The large time-scale agent optimizes caching decisions, while the short time-scale agent focuses on offloading and resource allocation. Extensive simulations show that our approach significantly reduces task execution delay and energy consumption, validating its effectiveness.

Federated learning (FL) enables mobile devices to collaboratively train deep learning models while maintaining data privacy and minimizing communication overhead. However, traditional FL methods typically assume access to pre-collected datasets, an impractical assumption for real-world mobile devices that continuously collect sensor data streams without retaining them while operating, due to limited memory constraints. Streaming Federated Learning (SFL) partially addresses this limitation by supporting online learning and asynchronous model aggregation directly from live data streams. Yet, a critical challenge in mobile data streams is thetemporal class imbalance, which may biase the streaming FL process toward early-arriving data classes during the critical learning period (CLP), the initial training phase when the model exhibits the highest plasticity. To address this, we propose${\sf AdaCLP}$, a training scheduler that tracks global model plasticity using a staleness-decayed mechanism specifically designed for the dynamics of real-world mobile environments.${\sf AdaCLP}$dynamically adjusts local training hyperparameters in SFL to prolong the CLP, thereby preserving model plasticity in the face of time-varying, imbalanced data distributions. Furthermore, a CLP-aware dynamic voltage and frequency scaling (DVFS) strategy is integrated to reduce the energy cost of prolonged training, aligning with the tight energy budgets of mobile devices. Experimental results demonstrate that${\sf AdaCLP}$can effectively handle either temporally imbalanced or periodic data, improving accuracy by up to 11% while reducing energy consumption by 69.5% compared to state-of-the-art methods, without modifying the standard FL training pipeline. These demonstrate that${\sf AdaCLP}$is a practical and efficient solution for real-world mobile FL deployment, enabling robust on-device adaptation to evolving data streams.

Online medical consultation is one of important mobile services worldwide, in which patients can make consultation more conveniently through phones anytime and anywhere. However, expert-level online consultations are expensive due to the shortage of medical professionals, while AI models are unreliable because they have unpredictable risks. Therefore, we introduce human-machine collaboration to medical online consultation and focus on symptom inquiry, as the basis for disease diagnosis. There are two key issues: 1) how to design an intelligent assignment strategy that can determine which doctors or models participate in each turn? 2) how to design an effective execution strategy that can improve the machine's inquiry ability among considerable symptoms? To address the above issues, we propose the Human-AI Diagnostic Team (HADT) framework based on Hierarchical Reinforcement Learning, which aims to achieve high accuracy with low manpower. Specifically, HADT has two layers. The upper one is responsible for assignment, in which we propose a module called master that enables intelligent human-machine assignments through the masked reinforcement learning with reward shaping. The lower one is responsible for execution, consisting of a doctor and a proposed module called machine. This module can effectively ask about symptoms through the masked hierarchical reinforcement learning with bottom-up training. Experiments on the public datasets show that HADT can achieve up to 89.4% accuracy with only 10.9% human effort, as confirmed by real clinical doctors using the designed interface with mobile devices.

Multidevice cooperative wireless sensing offers a promising solution for human motion recognition, owing to its superior privacy preservation and robustness. In the sensing process, devices continuously extract features from channel echoes and transmit them to a fusion center for motion recognition over successive time slots. The intertwined sub-processes of sensing and communication jointly determine recognition accuracy, yet simultaneously compete for limited radio resources. Moreover, the dynamic nature of practical environments further complicates this interplay due to the presence of moving interference sources and time-varying number of cellular users sharing the available bandwidth. Therefore, it is of paramount importance to jointly optimize sensing and communication resource allocation among devices and across time slots, while meticulously accounting for the impacts of dynamic environment to maximize recognition accuracy. In this paper, we propose a task-oriented integrated sensing and communication (ISAC) system for multidevice cooperative wireless motion recognition in dynamic environments. Specifically, we formulate a joint sensing and communication resource allocation problem to maximize recognition accuracy, represented by a discriminant gain metric that explicitly accounts for both sensing quality and communication constraints. Since this problem is a fractional program, we transform the original sum-of-ratios objective function into an equivalently subtractive form that facilities the development of a two-step iterative offline optimization (TSIO) algorithm to achieve the benchmark performance. Furthermore, to effectively cope with dynamic environmental influences, we further design a multi-agent reinforcement learning (MARL)-based online optimization (MRLO) scheme, which predicts environmental conditions at the subsequent time slot and adaptively optimizes resource allocation. Extensive numerical results illustrate that the proposed algorithm significantly enhances the recognition accuracy with dynamic environment influences, compared to existing benchmark algorithms. It is also observed from results that the sensing performance primarily drives recognition accuracy when energy is limited, whereas communication performance becomes the dominant factor under bandwidth constraints.

Collaborative agents, consisting of at least one human and one mobile robot agent working toward a common objective, are increasingly prevalent and effective in both social and industrial spheres, such as manufacturing. The inherent heterogeneity of these agents requires efficient and scalable Task Scheduling and Allocation (TSA) schemes that match individuals to tasks based on their abilities and meet specific temporal constraints, maximizing performance in less time. Existing works face challenges as exact methods rely on assumptions and deterministic models, which struggle to scale and infer time-varying, stochastic human task performance. While offline reinforcement learning shows promise, it is time-consuming and heavily dependent on training data that is often scarce in practical factory settings. To address these challenges, we formulate the TSA problem in mobile multi-agent teams as a temporal-constrained contextual decision-making process and propose the Bayesian Optimization-augmented Team coordination among Heterogeneous agents (BOTH), a novel scalable and training-free scheduling approach. The core idea is to use Gaussian Processes (GP) to iteratively infer agent dynamics in real-time, enabling the automatic derivation of a robust TSA solution that requires no prior data and adapts to varying problem sizes. We start by employing a heterogeneous graph-based encoder to extract representative context from the individual differences among team agents and tasks, considering strict temporal constraints. Following this, we propose a GP-driven Bayesian optimizer to intelligently explore and exploit optimal task assignments for each context, without making assumptions about the system. Experiments on synthetic and real datasets demonstrate that BOTH boosts accuracy and time efficiency compared to competing baselines, even within a few iterations.

Flow-based microfluidic biochips have attracted much attention over the past two decades. By integrating diverse micro-components, e.g., mixers and filters, on a miniaturized planar substrate, complicated bioassays such as protein crystallization and drug screening can be executed automatically without requiring human invention, thus becoming a promising alternative to traditional cumbersome laboratory equipment. As manufacturing technology advances, it has become possible to implement hundreds of thousands of microvalves within a single chip. This breakthrough has given rise to fully programmable valve array (FPVA) biochips, representing a next-generation platform in flow-based microfluidics that offers enhanced reconfigurability and operational flexibility. Nevertheless, the exponential increase in valve density has introduced significant design complexity when implementing sophisticated assay protocols. As a result, the design automation of FPVAs has emerged as a critical research frontier, attracting considerable attention from both academia and industry. This review article systematically examines recent advances in FPVA design automation, involving computer-aided design methods for architectural synthesis, volume management, sample preparation, automated testing, fault localization, error recovery, and washing optimization. These techniques enable FPVA users to concentrate on assay protocol development while delegating implementation-specific design and optimization tasks to design automation tools. Furthermore, we analyze emerging security implications in FPVAs, particularly focusing on bioassay accuracy and reliability that ensure experimental reproducibility. Finally, potential trajectories for future research are discussed in detail to further promote the integration level and widespread application of FPVAs.

Persuasion, a vital social skill, influences beliefs, attitudes, and behaviors through conversation. Yet, current dialogue agents either rely on scenario-specific strategies, restricting their cross-context adaptability, or neglect persuasion’s logical structure. They focus on isolated strategy classification, overlooking the significance of fine-grained sequential planning for real-world scenarios. To address these limitations, inspired by basic human mental activities, we present PersuHSG, an adaptive persuasion strategy planning framework. The core idea is to conceptualize persuasion as a tripartite framework comprising cognition, affection, and volition, with each stage represented as a graph layer and principle-based strategies for efficient multi-stage persuasion. Specifically, we first develop PersuInstruct, a fine-tuning dataset to improve dialogue agents’ strategic planning and response generation. Then, we propose a graph-aware planning algorithm for stage-strategy-response reasoning to generate persuasive responses for diverse scenarios. Extensive experiments confirm that PersuHSG significantly enhances the persuasiveness of Large Language Models (LLMs), allows smaller models (e.g., 9B, 13B) to achieve competitive performance, and demonstrates the efficacy of structured strategy planning in improving model efficiency and adaptability.

With the rapid growth of smart cities, low-altitude airspace opening, and crowd and mobile computing, robust air–ground integrated perception systems are essential for urban safety. Traditional multi-target tracking and capture methods suffer from perceptual blind spots limiting observability, constrained maneuverability hindering proactive target restriction, and unstable coordination under partial observability in multi-agent systems. To address these challenges, we propose Cog- Coor, a cognition-driven collaborative heterogeneous coordination framework that unifies perception, reasoning, and action through a closed-loop architecture, incorporating heterogeneous crowd computing to extend observability and enable proactive coordination. Specifically, we introduce: (1) an Active–Passive Heterogeneous Collaboration (APHC) model with a Collaborative Voting-based Block Mechanism (CVBM), enabling Unmanned Aerial Vehicles (UAVs) to trigger ground roadblocks to mitigate perceptual blind spots and constrain target motion; (2) a Cognition-driven Bayesian Collaborative Reward Mechanism (CBCR) that stabilizes multi-agent coordination under partial and heterogeneous observations by adjusting collaboration benefits based on agent reliability; and (3) a Cognition-driven Perception– Scheduling Integration Mechanism (CPSI) that integrates UAV motion control and blocking decisions into a single multi-agent reinforcement learning policy, allowing perception-driven actions to actively constrain target maneuverability. Experiments on real urban road data simulations show that CogCoor outperforms baseline methods, increasing target captures by 8.06% and reducing energy consumption by 7.62% on average in resource-constrained environments. In high coordination demand scenarios, target captures improve by 5.98% on average while energy consumption decreases by 5.91%, confirming its effectiveness in complex urban settings.

Satellite-assisted Mobile Edge Computing (MEC) is a promising paradigm for enabling low-latency and high-efficiency computing in deep-sea and far-offshore marine environments. However, the inherent heterogeneity of three-layer marine networks—comprising satellites, Unmanned Surface Vehicles (USVs), and Autonomous Underwater Vehicles (AUVs)—introduces unique challenges. These include the coupling of underwater acoustic and above-water Radio Frequency (RF) communication links, the highly constrained computing and caching resources of edge devices, and the strong interdependence between service caching, vertical task offloading, and horizontal migration. Existing solutions often overlook these cross-layer dynamics and the spatio-temporal interactions among network nodes, leading to suboptimal task scheduling and degraded system utility. To address these challenges, we formulate a joint optimization problem that maximizes the Quality of Experience (QoE), with decision variables spanning caching placement, task migration, and offloading under resource constraints. Through rigorous theoretical analysis, we prove that the formulated problem is NP-hard, highlighting its inherent computational intractability. To overcome this, we propose an Attention-Enhanced Multi-Agent Reinforcement Learning algorithm (AE-MARL), which adopts a hybrid policy network to learn discrete decisions and continuous resource allocation. Furthermore, a lightweight attention module is integrated to infer the importance of partial observations and guide collaborative decision-making across agents. Extensive experiments and analysis under diverse system configurations demonstrate that AE-MARL consistently outperforms state-of-the-art baselines.

Continuous crowdsensing in Mobile CrowdSensing (MCS) involves ongoing monitoring to gather real-time data over extended periods. A key challenge is determining appropriate intervals between consecutive data samplings to capture temporal variations, especially in unknown dynamic environments. Traditional Age-of-Information (AoI) driven methods maintain data freshness but can be costly and result in data redundancy. To address this, we integrate the AoI metric with information entropy difference to create a novel indicator, Composite Data Value (CDV), balancing data freshness and redundancy. Based on it, we investigate the online adaptive sampling problem for continuous crowdsensing in unknown dynamic environments. This problem is challenging due to the vast space of sensing strategies, difficulty in estimating rewards with unknown distributions and varying rates of change, and the degradation of optimal strategies as the environment evolves. Using a multi-armed bandit framework, we propose AdaScs, an online adaptive sampling optimization approach to maximize long-term CDV performance. First, AdaScs develops compact sensing strategies through limited trials. Then, it adaptively performs online sampling based on evolving reward estimations, identifying optimal strategies, detecting environmental drifts, updating strategies, and adjusting cycle lengths. Our results show that AdaScs outperforms all baselines, with accuracy increasing by 22.6% and reaction time at last improving by 54.6%.

Mobile Edge Computing (MEC) significantly enhances service response speeds and improves the Quality of Service (QoS) by facilitating the offloading of computation-intensive tasks from Mobile Users (MUs) to nearby Edge Servers (ESs). However, due to the inherent selfishness of involved entities, MUs may be unwilling to offload tasks without reasonable resource pricing, and ESs may lack motivation to provide computation resources without appropriate compensation. Furthermore, improving the utilization of ESs' computation resources and achieving efficient task scheduling remains a major challenge. To ad dress these issues, we introduce a profitable broker between ESs and MUs, and propose TORP, a Two-stage Stackelberg Game based Computation Offloading and Resource Pricing mechanism, to maximize the utility of each entity. Specifically, we model the interactions among three entities (i.e., the broker, MUs, and ESs) as a two-stage Stackelberg game, where the interactions between the broker and ESs is defined as Stage I, while the interactions between the broker and MUs is defined as Stage II. By using the backward induction method, we theoretically prove the Stackelberg Equilibrium (SE) for each stage of the two-stage Stackelberg, and the SE of the whole game. Then, recognizing that the optimization problem is a Mixed-Integer Nonlinear Programming (MINLP) problem, an Alternating Iteration-Based Resource Pricing and Task Offloading Algorithm (AIPOA) is proposed to obtain the optimal solution. Finally, we perform extensive simulations comparing TORP against multiple base lines. Experimental results show that TORP achieves substantial improvements, enhancing the utilities of three entities by about 2.00%-52.69% under different scenarios.

To enhance the processing of complex multi-modal documents (e.g. e-books, long web pages, etc.), it is an efficient way for users to take digital screenshots of key parts and reorganize them into a new collage E-Note. Existing methods for assisting collage layout design primarily employ a semantic relevance-first strategy, with arranging related contents together. Though capable, it can not ensure the visual readability of screenshots and may conflict with human natural reading patterns. In this paper, we introduce CollageNoter for real-time collage layout design that adapts to various devices (e.g. laptop, tablet, phone, etc.), offering users with visually and cognitively well-organized screenshot-based E-Notes. Specifically, we construct a novel two-stage pipeline for collage design, including 1) readability-first layout generation and 2) cognitive-driven layout adjustment. In addition, to achieve real-time response and adaptive model training, we propose a cascade transformer-based layout generator named CollageFormer and a size-aware collage layout builder for automatic dataset construction. Extensive experimental results have confirmed the effectiveness of our CollageNoter.

Despite the growing integration of deep models into mobile terminals, the accuracy of these models declines significantly due to various deployment interferences. Test-time adaptation (TTA) has emerged to improve the performance of deep models by adapting them to unlabeled target data online. Yet, the significant memory cost, particularly in resource-constrained terminals, impedes the effective deployment of most backward-propagation-based TTA methods. To tackle memory constraints, we introduce Surgeon, a method that substantially reduces memory cost while preserving comparable accuracy improvements during fully test-time adaptation (FTTA) without relying on specific network architectures or modifications to the original training procedure. Specifically, we propose a novel dynamic activation sparsity strategy that directly prunes activations at layer-specific dynamic ratios during adaptation, allowing for flexible control of learning ability and memory cost in a data-sensitive manner. Among this, two metrics, Gradient Importance and Layer Activation Memory, are considered to determine the layer-wise pruning ratios, reflecting accuracy contribution and memory efficiency, respectively. Experimentally, our method surpasses the baselines by not only reducing memory usage but also achieving superior accuracy, delivering SOTA performance across diverse datasets, architectures, and tasks.

Agent-assisted memory recall is one critical research problem in the field of human-computer interaction. In conventional methods, the agent can retrieve information from its equipped memory module to help the person recall incomplete or vague memories. The limited size of memory module hinders the acquisition of complete memories and impacts the memory recall performance in practice. Memory theories suggest that the person’s relevant memory can be proactively activated through some effective cues. Inspired by this, we propose a novel strategy-guided agent-assisted memory recall method, allowing the agent to transform an original query into a cue-rich one via the judiciously designed strategy to help the person recall memories. To this end, there are two key challenges. (1) How to choose the appropriate recall strategy for diverse forgetting scenarios with distinct memory-recall characteristics? (2) How to obtain the high-quality responses leveraging recall strategies, given only abstract and sparsely annotated strategy patterns? To address the challenges, we propose a Recall Router framework. Specifically, we design a 5W Recall Map to classify memory queries into five typical scenarios and define fifteen recall strategy patterns across the corresponding scenarios. We then propose a hierarchical recall tree combined with the Monte Carlo Tree Search algorithm to optimize the selection of strategy and the generation of strategy responses. We construct an instruction tuning dataset and fine-tune multiple open-source large language models (LLMs) to develop MemoCue, an agent that excels in providing memory-inspired responses. Experiments on three representative datasets show that MemoCue surpasses LLM-based methods by 17.74% in recall inspiration. Further human evaluation highlights its advantages in memory-recall applications.

It is challenging to simultaneously address multiple related tasks using a unified multi-task model and consistently balance conflicts across these tasks. The conflicts arise because each task competes to update the shared module in a manner that can better align with its own requirements. To address the conflicts, existing multi-task learning methods primarily balance task losses or gradients of the shared module, but frequently overlook the differences in layer-wise conflicts, which can enable a more fine-grained conflict-averse. For details, each task establishes its competitive influence over a particular layer by assigning varying degrees of importance to it. Attenuating the gradients of tasks with relatively lower importance during updates in specific layers may not only balance gradient conflicts but also facilitate the progression of other tasks. Based on this, we proposed Layer-wise Competitive Multi-task Learning, wherein multiple tasks compete for gradient update weights within the shared modules of a multi-task model. This approach aims to achieve layer-specific gradient balance by considering each task’s relative importance at different layers within the shared module. Tasks of relatively lower importance for a specific layer will receive smaller gradient updates, thereby facilitating faster convergence for more important tasks.

Depth estimation and scene segmentation are two important tasks in intelligent transportation systems. A joint modeling of these two tasks will reduce the requirement for both the storage and training efforts. This work explores how the multi-task distillation could be used to improve such unified modeling. While existing solutions transfer multiple teachers’ knowledge in a static way, we propose a self-adaptive distillation method that can dynamically adjust the knowledge amount from each teacher according to the student’s current learning ability. Furthermore, as multiple teachers exist, the student’s gradient update direction in the distillation is more prone to be erroneous where knowledge forgetting may occur. To avoid this, we propose a knowledge trajectory to record the most essential information that a model has learnt in the past, based on which a trajectory-based distillation loss is designed to guide the student to follow the learning curve similarly in a cost-effective way. We evaluate our method on multiple benchmarking datasets including Cityscapes and NYU-v2. Compared to the state-of-the-art solutions, our method achieves a clearly improvement. The code is provided in the supplementary materials.

Artificial Intelligence (AI) systems rapidly advance in online medical consultations, where doctors diagnose through online dialogue. Recent AI models have made significant progress in symptom inquiry; however, the disease diagnosis accuracy remains low and unreliable, failing to replace doctors’ role fully. Although some studies attempt to assist doctors by providing AI-generated advice, this advice often has high error rates and lacks complementarity, needing further improvement. Therefore, we aim to introduce a reliable and effective human-AI collaboration system. There are two key challenges. 1) How to design an advice strategy that improves the accuracy of the advice? 2) How to develop an optimal AI teammate for the human-AI team to enhance the overall team utility? To address these challenges, we propose the Human-AI collaboration diagnosis framework with Combination advice (HAComb). Specifically, to ensure the accuracy of advice, we introduce a human-AI combination advice that uses Bayesian methods to integrate doctors’ predicted labels with AI model outputs. To enhance team utility, we design a loss function that incorporates both AI loss and team utility loss. Experiments on four real-world datasets show that HAComb outperforms single human and other human-AI collaboration methods in diagnosis accuracy and team utility.

Video advertising has become a popular marketing strategy on e-commerce platforms, requiring high-level semantic reasoning like selling point discovery, narrative organization. Previous rule-based methods struggle with these complex tasks, and learning-based approaches demand large datasets and high training costs. Recently, Large Language Models have opened incredible opportunities for advancing intelligent video advertisement editing. However, Input-output (IO) prompting and Chain-of-Thought (CoT) struggle to adapt to the nonlinear thinking hierarchy of video editing, where editors iteratively select shots or revert them to explore potential editing solutions. While Tree-of-Thought (ToT) offers a conceptual structure that mirrors this hierarchy, it falls short in aligning with effective video advertising strategies and lacks robust fact-checking mechanisms. To address these, we propose a novel framework, Tree-of-AdEditor (ToAE), which constructs a reasoning tree to mimic human editors, and incorporates domain-specific theories and heuristic fact-checking to identify optimal editing solutions. Specifically, motivated by effective advertisement principles, we develop a "local-global" mechanism to guide LLM in both the shot level and sequence level decision-making. We introduce a visual incoherence pruning module to provide external heuristic fact-checking, ensuring visual attractiveness and reducing computation costs. Quantitative experiments and expert evaluation demonstrate the superiority of our method compared to baselines.

Recent studies indicate that human-AI collaboration performs better than either alone, particularly in medical diagnosis. Beyond collaboration methods that focus on assigning tasks to humans or AI, like deferral, combining human and AI decisions with their confidence scores is emerging as a promising strategy. Due to high cognitive load, doctors often struggle to provide confidence assessments, necessitating explicit human uncertainty evaluation through a limited number of additional expert predictions. There are two challenges. (1) how to actively collect limited yet representative expert predictions? (2) how to accurately evaluate human uncertainty with limited expert predictions? To address the challenges, we propose ActiveHAI, an active human-AI diagnosis method that reduces expert costs through a median-window sampling strategy that actively selects representative samples near the estimated median; and evaluate expert confidence through an evaluator module that integrates sample features and expert predictions, converting them into probability distributions. Experiments on three real-world datasets show that ActiveHAI surpasses doctor and other human-AI methods by 16.3% and 3.6% in accuracy, respectively. Furthermore, ActiveHAI reaches 97.2% relative accuracy, even with just eight expert predictions per class.

Deep Neural Networks (DNNs) play a crucial role in the smart Internet of Things (IoT), with widespread applications in inference tasks like interactive games, intelligent driving, and augmented reality. Along with these promising applications, various task-offloading methods were proposed to improve the utilization of system resources, given that DNN model inference typically requires substantial computational power. However, existing offloading methods focus primarily on a specific model, and research addressing heterogeneous DNN models (with different structures and layers) remains limited in practical IoT environments. Directly integrating these methods would require frequent re-initialization to adapt to changes in the search space during the offloading of mixed heterogeneous DNN inference tasks, resulting in insufficient flexibility and the waste of computational resources. Thus, we propose AutoCut, a global heterogeneous model offloading service based on a customized multi-objective differential evolution algorithm, to find low-latency and energyefficient offloading partitions. AutoCut utilizes a group-layer granularity partitioning that avoids frequent changes in the search space when continuously offloading heterogeneous DNN inference tasks, thereby improving search efficiency. Experiments with six popular models show that AutoCut significantly improves inference performance regarding latency and energy efficiency.

The emerging paradigm Social Internet of Things (SIoT) integrates social networking elements into the Internet of Things, enabling smart devices to establish and manage interactions autonomously. This enhances collaboration and adaptability but also increases complexity and vulnerability, particularly from malicious devices exploiting these relationships. To address this, trust evaluation of devices becomes crucial. Traditional approaches, like weighted sums and Bayesian inference, struggle with the dynamic nature of SIoT environments. Recent advancements in Graph Neural Networks (GNNs) show promise, yet existing models often fail to capture the complexities of SIoT's dynamic and heterogeneous nature. In this paper, we propose TrustLive, a GNN-based framework for real-time trust evaluation in dynamic SIoT settings. TrustLive first employs a heterogeneous graph to represent smart devices and their interactions, which are then encoded via a customized graph embedding technique for trust feature extraction. It further incorporates Graph Convolutional Networks for trust aggregation and Temporal Convolutional Networks to capture trust evolution. Moreover, a Memory-Augmented Incremental Update mechanism is added to ensure low-latency updates by processing only the latest data while preserving accuracy with historical results. Experimental results demonstrate that TrustLive outperforms current methods in both accuracy and efficiency, offering a robust solution for trust evaluation in SIoT.

Efficient order dispatching is crucial for online ride-hailing systems, directly influencing user experience and platform revenue. Traditional methods often focus on maximizing immediate revenue through local observations of individual vehicles, ignoring the long-term potential benefits , the dynamic nature of dispatching systems, and the importance of collaboration among distributed vehicles. This typically results in suboptimal performance. To address these issues, we propose FedMARL4OD, a novel Federated Multi-Agent Deep Reinforcement Learning framework designed to optimize order dispatching. This framework integrates local learning via Multi-Agent Reinforcement Learning (MARL) for individual vehicles and global learning via Federated Multi-Agent Reinforcement Learning (FedMARL) across all vehicles. Specifically, we introduce an innovative reward mechanism in local learning that considers both the current revenue of each order and the supply–demand dynamics of the system related to potential future revenue, thereby improving dispatching performance. Moreover, we introduce a scalable model aggregation method in global learning that explicitly models interactions among distributed vehicles to facilitate collaborative learning . By progressively integrating local and global insights through average parameter aggregation, this method not only reduces communication overhead and enhances the learning efficiency of agents, but also ensures system scalability and maintains data privacy. Extensive real-world simulations demonstrate that FedMARL4OD outperforms baseline methods , achieving a 9.17% increase in Accumulated Driver Income (ADI) and a 7.75% improvement in Order Response Rate (ORR). The ADI improvements demonstrate the framework’s effectiveness in boosting revenue, while the enhanced ORR indicates a quicker fulfillment of users’ requests, improving user experience.

Abstract Persuasion, as one of the crucial abilities in human communication, has garnered extensive attention from researchers within the field of intelligent dialogue systems. Developing dialogue agents that can persuade others to accept certain standpoints is essential to achieving truly intelligent and anthropomorphic dialogue systems. Benefiting from the substantial progress of Large Language Models (LLMs), dialogue agents have acquired an exceptional capability in context understanding and response generation. However, as a typical and complicated cognitive psychological system, persuasive dialogue agents also require knowledge from the domain of cognitive psychology to attain a level of human-like persuasion. Consequently, the cognitive strategy-enhanced persuasive dialogue agent (defined as CogAgent ), which incorporates cognitive strategies to achieve persuasive targets through conversation, has become a predominant research paradigm. To depict the research trends of CogAgent, in this paper, we first present several fundamental cognitive psychology theories and give the formalized definition of three typical cognitive strategies, including the persuasion strategy, the topic path planning strategy, and the argument structure prediction strategy. Then we propose a new system architecture by incorporating the formalized definition to lay the foundation of CogAgent. Representative works are detailed and investigated according to the combined cognitive strategy, followed by the summary of authoritative benchmarks and evaluation metrics. Finally, we summarize our insights on open issues and future directions of CogAgent for upcoming researchers.

Autonomous aerial vehicle (AAV) crowdsensing, as a complement to mobile crowdsensing, can provide ubiquitous sensing in extreme environments and has gathered significant attention in recent years. In this article, we investigate the issue of sensing data storage in AAV crowdsensing without edge assistance, where sensing data is stored locally in the AAVs. In this scenario, replication scheme is usually adopted to ensure data availability, and our objective is to find an optimal replica distribution scheme to maximize data availability while minimizing system energy consumption. Given the NP-hard nature of the optimization problem, traditional methods cannot achieve optimal solutions within limited timeframes. Therefore, we propose a centralized training and decentralized execution deep reinforcement learning (DRL) algorithm based on actor-critic, named “MUCRS-DRL.” Specifically, this method derives the optimal replica placement scheme based on AAV state information and data file information. Simulation results show that compared to the baseline methods, the proposed algorithm reduces data loss rate, time consumption, and energy consumption by up to 88%, 11%, and 11%, respectively.

With the rapid development of artificial intelligence (AI), integrating deep neural networks (DNNs) into mobile and embedded devices has become an important trend. This integration significantly enhances the ability of these devices to collect and analyze perceptual data. Traditionally, the integration paradigm relies on cloud based training and deployment on mobile devices. However, the dynamic characteristics and privacy issues related to real-world perceptual data require training on the device. Despite its advantages, the limited computing resources of mobile devices constitute a key bottleneck that hinders the efficiency of model training. To address this issue, parallel distributed training across mobile device clusters has become a feasible paradigm. However, the inherent mobility of these devices not only increases the possibility of training interruptions, but also exacerbates the inefficiency caused by data imbalance. These challenges make traditional cloud based model parallelization methods unsuitable for mobile environments. To overcome these limitations, this paper proposes a novel model parallel system CrowdMesh designed for mobile device clusters. CrowdMesh consists of three key modules: (1) alliance-game-based dynamic cluster startup and construction,(2) computation-cost-based DL model parallel, (3) device-mobility-aware parameter propagation module. These modules work together to address training interruptions and restarts caused by device mobility, as well as efficiency issues caused by data imbalance.The experimental results show that CrowdMesh performs better than existing cloud and device based model parallelization baselines in various training tasks and deep learning models, and can reduce training latency by more than 18%.

Deep learning is reshaping mobile applications, with a growing trend of deploying deep neural networks (DNNs) directly to mobile and embedded devices to address real-time performance and privacy. To accommodate local resource limitations, techniques like weight compression, convolution decomposition, and specialized layer architectures have been developed. However, the dynamic and diverse deployment contexts of mobile devices pose significant challenges. Adapting deep models to meet varied device-specific requirements for latency, accuracy, memory, and energy is labor-intensive. Additionally, changing processor states, fluctuating memory availability, and competing processes frequently necessitate model recompression to preserve user experience. To address these issues, we introduce AdaScale, an elastic inference framework that automates the adaptation of deep models to dynamic contexts. AdaScale leverages a self-evolutionary model to streamline network creation, employs diverse compression operator combinations to reduce the search space and improve outcomes, and integrates a resource availability awareness block and performance profilers to establish an automated adaptation loop. Our experiments demonstrate that AdaScale significantly enhances accuracy by 5.09%, reduces training overhead by 66.89%, speeds up inference latency by 1.51 to$6.2\times $, and lowers energy costs by$4.69\times $.

Training deep neural networks (DNNs) on large-scale datasets is often inefficient with large computational needs and significant energy consumption. Although great efforts have been taken to optimize DNNs, few studies focused on the inefficiency caused by the data samples with less value for model training. In this article, we empirically demonstrate that sample complexity is important for model efficiency and selecting representative samples is constructive to the model efficiency. In particular, we propose hardness-aware dataset pruning method ($\gamma$-Razor) to select representative samples from large-scale datasets to remove the less valuable data samples for model training.$\gamma$-Razor is a two-stage framework that includes interclass sampling and intraclass sampling. First, we introduce the inverse self-paced learning strategy to learn hard samples and adjust their weights adaptively according to the inverse frequency of effective samples of each class. For intraclass sampling, hardness-aware cluster sampling algorithm is proposed to downsample easy samples within each class. To evaluate the performance of$\gamma$-Razor, we conducted extensive experiments on three large-scale datasets for image classification tasks. The experimental results show that models trained with the pruned datasets show competitive performances against their counterparts trained with the original large-scale datasets in terms of robustness and efficiency. Furthermore, models trained with the pruned datasets converge faster with lower energy consumption.

While artificial intelligence (AI) technology has advanced in real-world applications, there is a strong motivation to develop hybrid systems where AI algorithms and humans collaborate, promoting more human-centered approaches in AI system design. This has led to the emergence of a novel human-machine computing (HMC) paradigm, which combines human cognitive abilities with machine computational power to create a collaborative computing framework that meets the demands of large-scale, complex tasks and enables human-machine symbiosis. Human processing units (HPUs) are crucial computing resources in HMC-oriented systems, and efficient HPU resource provisioning is key to boosting system performance. However, existing schemes often fail to assign tasks to the most suitable HPUs and optimize HPU utility, as they either cannot quantitatively measure skills or overlook utility concerns during task assignment and scheduling. To address these challenges, this article proposes a fine-grained HPU virtualization (FingHV) approach, which leverages virtualization techniques to improve flexibility, fairness, and utility in the provisioning process. The core idea is to use a tree-based skill model to precisely measure the levels and correlations of multiple skills within individual HPUs, and to apply a mixed time/event-based scheduling policy to maximize HPU utility. Specifically, we begin by proposing a hierarchical multiskill tree to model HPU skills and their correlations. Next, we formulate the HPU virtualization problem and present a fine-grained virtualization method, which includes a quality-driven HPU assignment process and a mixed time/event-based scheduling policy to improve resource-sharing efficiency. Finally, we evaluate FingHV on a synthetic dataset with varying task sizes and a real-world case. The results demonstrate that FingHV improves global matching quality by up to 39.7% and increases HPU utility by 11.2% compared to the baselines.

Nowadays, Mobile Crowd Sensing (MCS) has become a popular paradigm for large-scale data collection using ubiquitous mobile sensing devices. However, most existing works do not consider that requester's payments are unknown prior, and assume that workers are honest, which may not be true in practice. To address these problems, we propose a novel Truthful and Dual-direction Combinatorial Multi-Armed Bandit (TD-CMAB) scheme, which maximizes the total profit of the dual-direction platform for both the worker side and the requester side. Specifically, for the worker side, to overcome the problem that the platform is not clear whether sensed data are true, we propose a worker recruitment strategy that identifies and recruits honest workers at low cost through the Upper Confidence Bound (UCB) algorithm based on truth data discovery. For the requester side, where requesters’ payments are unknown prior, we model requester selection as a CMAB problem and solve it by the proposed adaptive UCB algorithm. Furthermore, we theoretically prove the worst regret bound of the TD-CMAB. Finally, we evaluate the effectiveness of the TD-CMAB scheme through extensive experiments using the Beijing taxi dataset.

Centralized trading platforms have long been the preferred choice for users, despite growing concerns regarding data privacy. Users have to place their trust in these platforms and provide sensitive personal information, like identities and financial accounts. However, these centralized platforms often lack transparency, making it challenging to ensure fairness, privacy, and security against both external and internal risks. In contrast, a decentralized fair trading paradigm, harnessing the potential of blockchain technology, is rapidly emerging. It empowers individuals to engage in the exchange of digital assets with others while guaranteeing fairness, efficiency, and privacy. In this paper, we conduct a comprehensive survey of decentralized fair trading. We commence by providing fundamental definitions of fair trading and tracing its evolution over time. We then delve into the essential framework of on-chain and off-chain trading and highlight key improvements that enhance the efficiency of decentralized fair trading within various application scenarios. Furthermore, we undertake a thorough analysis of privacy and security enhancements within the scope, summarizing defenses against known attacks. Finally, we outline the challenges and offer insights into the future prospects of decentralized fair trading, with the aim of inspiring the development of more innovative and promising designs in this evolving trend.

Recently, blockchain-based decentralized mobile crowdsensing systems have emerged to eliminate traditional centralized trust and to achieve transparent task assignments via smart contracts. It allows workers to select tasks freely, thereby maximizing their benefits. However, prior designs rarely considered the globally optimal task assignment that significantly impacts the efficiency and quality of task performance, like maximizing the task completion ratio and minimizing the total travel distance of workers. So in this paper, we propose DCrowd, a new blockchain-based mobile crowdsensing system, to realize the decentralized, transparent, and globally optimal task assignment. In brief, we first introduce the Proof of Task Assignment consensus mechanism. This allows miners to conduct globally optimal task assignments off-chain, leverages smart contracts to perform lightweight verification for task assignment results on-chain, and stores the globally optimal task assignment in a customized block. Then, we devise the Weight-Prioritized Task Selection strategy and Threshold-based Adaptive Minimum Cost Flow algorithm, to further optimize the system performance and guide miners in competing for minting rights. A thorough theoretical analysis is provided. Extensive experiments on real-world datasets indicate that DCrowd can reduce the broadcast and consensus latency by over 50% and improve the throughput by over 87% compared with existing systems.

Graphic designs are pervasive in our daily lives and widely used to communicate information hierarchically to humans. To achieve this, the layout plays an essential role in guiding readers to understand the importance of different elements and comprehend the content. To deal with the rapidly increasing demands of graphic designs, recent studies attempt to automatically generate layouts based on category information and spatial relations, often resulting in layouts with poor communication quality. In this article, we make the first attempt to explore element importance-aware layout generation under the guidance of a novel design graph, which attracts readers’ attention to a layout by formulating aesthetic relations implicitly involved in graphic designs between element pairs. The core of our approach is a learning-based framework with a new multimodality cascade transformer (MCT) in a coarse-to-fine manner. A hierarchical multimodality fusion (HMF) mechanism and two new losses are introduced to guide the training process progressively. We further collect a new fine-grained advertisement poster layout dataset containing more than 30 K layouts labeled with 91 element labels. Both qualitative and quantitative experiments demonstrate the effectiveness of our approach against existing works. We also conduct user studies and cognitive experiments to evaluate the direct adaptability and attractiveness of generated layouts.

The how-to vlog has gained popularity as a medium for learning practical skills, such as cooking and handcrafting. The sequencing process of these videos should meticulously integrate storytelling and cinematographic elements to ensure audience comprehension and engagement, posing a significant challenge for novice creators. Pioneer efforts have adhered to predefined editing rules or assembled clips based on textual logic, limiting their applicability across diverse educational scenarios and causing visual discontinuities. In contrast, we aim to extract rich cinematographic patterns from well-edited professional how-to vlogs to enhance shot assembly. To this end, we identify two crucial aspects influencing educational value: narrative continuity and scale transition. We model the shot assembly process as a task of selecting the next shot and devise a cinematographic-aware contrastive model to learn representations that distinguish between the good next shots against the bad ones. This method incorporates a two-stream cinematographic-aware encoding module for explicit factor encoding and a situation-adaptive attention-based integration module to accommodate varying assembly scenarios. Quantitative results from our novel professional user-generated vlogs dataset (proVlog-HowTo) clearly demonstrate the proposed method’s effectiveness. User study results further indicate its superiority in generating videos with narrative continuity and smooth transitions.

With the development of intelligent technology in coal mine industrial Internet of Things (IoT), the demand for salient object detection (SOD) in underground coal mine space has been increasing. The complex scenes and variable backgrounds in coal mine bring challenges for SOD, such as blurred edges, high computational complexity, and long processing times, making it difficult to meet the accuracy and real-time requirements of coal mine industrial IoT applications. To address these issues, we propose the dynamic weighting hybrid attention (DWHA)-partial convolution multiscale strip pooling (PCMSP) network for SOD in the coal mine industrial IoT. First, we introduce the DWHA module, which dynamically fuses self-attention for global context and SBAM for refining channel and spatial information, improving saliency detection accuracy. Second, we propose the PCMSP lightweight module, which the multiscale strip pooling introduces multiscale dilations, enhancing the ability to capture multiscale information and improve feature representation. By using partial convolution, which reduces the consumption of computing resources and running time while ensuring boundary quality. The experimental results indicate that, using self-built dataset for underground coal mine SOD, the DWHA-PCMSP network outperforms the four SOTA: BASNet, U2Net, SUCA, and EDN by achieving an increase of 2.82% in F1-score, a decrease of 23.70% in MAE, a reduction of 72.3G in FLOPs, and an improvement of 6.6 FPS in speed, compared to the worst-performing model.

Spatial Crowdsourcing (SC), which outsources location-dependent tasks to workers for physical completion, is gaining popularity. Recently, more complex tasks have emerged that require a group of workers collaborating in a coalition. Several pioneering studies have examined this issue using the server assigned tasks mode from an overall perspective, such as maximizing the total benefits of all workers. Unfortunately, maximizing the overall benefit does not necessarily align with maximizing individual benefits. In practice, crowd workers are often self-interested and autonomous, making decisions based on their personal perspectives. In this article, under the worker selected tasks mode, we investigate an important problem: Selfish Workers Coalition Formation (SWCF) problem in SC. Here, selfish workers autonomously form coalitions to accomplish tasks to maximize their individual benefits. Achieving a stable coalition formation for SWCF problem requires balancing cooperation and competition. First, we transform the SWCF problem into a hedonic coalition formation game using a devised exploited skills-based reward distribution model. Subsequently, we propose a distributed algorithm HCFTA and prove its Nash stability and performance bounds. Additionally, to enhance coalition formation efficiency, we propose a Markov blanket coloring parallel optimization algorithm MCPHCF . Extensive experiments demonstrate the superiority of the proposed methods on both synthetic and real-world datasets.

Motor imagery, as a paradigm of brain-computer interface, holds vast potential in the field of medical rehabilitation. Addressing the challenges posed by the non-stationarity and low signal-to-noise ratio of EEG signals, the effective extraction of features from motor imagery signals for accurate recognition stands as a key focus in motor imagery brain-computer interface technology. This paper proposes a motor imagery EEG signal classification model that combines functional brain networks with graph convolutional networks. First, functional brain networks are constructed using different brain functional connectivity metrics, and graph theory features are calculated to deeply analyze the characteristics of brain networks under different motor tasks. Then, the constructed functional brain networks are combined with graph convolutional networks for the classification and recognition of motor imagery tasks. The analysis based on brain functional connectivity reveals that the functional connectivity strength during the both fists task is significantly higher than that of other motor imagery tasks, and the functional connectivity strength during actual movement is generally superior to that of motor imagery tasks. In experiments conducted on the Physionet public dataset, the proposed model achieved a classification accuracy of 88.39% under multi-subject conditions, significantly outperforming traditional methods. Under single-subject conditions, the model effectively addressed the issue of individual variability, achieving an average classification accuracy of 99.31%. These results indicate that the proposed model not only exhibits excellent performance in the classification of motor imagery tasks but also provides new insights into the functional connectivity characteristics of different motor tasks and their corresponding brain regions.

Traditional liquid detection approaches are often time-intensive and invasive, typically requiring the opening of containers for examination. While recent initiatives have proposed several innovative solutions, including camera-based and vibration sensor-based techniques, these approaches still face limitations in terms of convenience. The development of radio frequency (RF) technology, particularly millimeter-wave (mmWave) radar, offers a promising solution for non-invasive and contactless liquid detection. In particular, during the past few years, a number of radar-based sensing systems have been developed to detect or identify liquids. However, little work has been done on the simultaneous detection of multiple liquids. To fill this gap, we design a novel composite reflection model, which overcomes the detection challenges due to composite interference and environmental reflections, by utilizing the consistency and uniqueness of the reflection signals from multiple liquid targets. Based on the proposed model, we develop a system namedMultiScanner, which is able to detect different types of liquids in multi-target scenarios, exhibiting high location independence without the need for extensive data training. Extensive experiments validate the effectiveness ofMultiScanner, achieving up to 95.91% accuracy in detecting 10 hazardous-normal liquid combinations in 2-target scenarios. Moreover, even in more complex 5-target scenarios, an detection accuracy of 86.49% can be obtained. To the best of our knowledge, this is the first study that uses RF signals for multi-liquid detection.

The rise of mobile devices with abundant sensor data and computing power has driven the trend of federated learning (FL) on them. Personalized FL (PFL) aims to train tailored models for each device, addressing data heterogeneity from diverse user behaviors and preferences. However, due to dynamic mobile environments, PFL faces challenges intest-time data shifts, i.e., variations between training and testing. While this issue is well studied in generic deep learning through model generalization or adaptation, this issue remains less explored in PFL, where models often overfit local data. To address this, we introduce${\sf ClassTer}$, a shift-robust PFL framework. We observe that class-wise clustering of clients in cluster-based PFL (CFL) can avoid class-specific biases by decoupling the training of classes. Thus, we propose a paradigm shift from traditional client-wise clustering toclass-wise clustering, which allowseffective aggregationof cluster models into a generalized one via knowledge distillation. Additionally, we extend ClassTer toasynchronousmobile clients to optimize wall clock time by leveraging critical learning periods and both intra- and inter-device scheduling. Experiments show that compared to status quo approaches,${\sf ClassTer}$achieves a reduction of up to 91% in convergence time, and an improvement of up to 50.45% in accuracy.

There are many deep learning (DL) powered mobile and wearable applications today continuously and unobtrusively sensing the ambient surroundings to enhance all aspects of human lives. To enable robust and private mobile sensing, DL models are often deployed locally on resource-constrained mobile devices using techniques such as model compression or offloading. However, existing methods, either front-end algorithm level (i.e. DL model compression/partitioning) or back-end scheduling level (i.e. operator/resource scheduling), cannot be locally online because they require offline retraining to ensure accuracy or rely on manually pre-defined strategies, struggle withdynamic adaptability. The primary challenge lies in feeding back runtime performance from theback-endlevel to thefront-endlevel optimization decision. Moreover, the adaptive mobile DL model porting middleware withcross-level co-adaptationis less explored, particularly in mobile environments withdiversityanddynamics. In response, we introduce CrowdHMTware, a dynamic context-adaptive DL model deployment middleware for heterogeneous mobile devices. It establishes anautomated adaptation loopbetween cross-level functional components, i.e. elastic inference, scalable offloading, and model-adaptive engine, enhancing scalability and adaptability. Experiments with four typical tasks across 15 platforms and a real-world case study demonstrate that${\sf CrowdHMTware}$can effectively scale DL model, offloading, and engine actions across diverse platforms and tasks. It hides run-time system issues from developers, reducing the required developer expertise.

The integration of deep neural network (DNN) intelligence into embedded mobile devices is expanding rapidly, supporting a wide range of applications. DNN compression techniques, which adapt models to resource-constrained mobile environments, often force a trade-off between efficiency and accuracy. Distributed DNN inference, leveraging multiple mobile devices, emerges as a promising alternative to enhance inference efficiency without compromising accuracy. However, effectively decoupling DNN models into fine-grained components for optimal parallel acceleration presents significant challenges. Current partitioning methods, including layer-level and operator or channel-level partitioning, provide only partial solutions and struggle with the heterogeneous nature of DNN compilation frameworks, complicating direct model offloading. In response, we introduce AdaKnife, an adaptive framework for accelerated inference across heterogeneous mobile devices. AdaKnife enables on-demand mixed-granularity DNN partitioning via computational graph analysis, facilitates efficient cross-framework model transitions with operator optimization for offloading, and improves the feasibility of parallel partitioning using a greedy operator parallelism algorithm. Our empirical studies show that AdaKnife achieves a 66.5% reduction in latency compared to baselines.

The rise of mobile devices equipped with numerous sensors, such as LiDAR and cameras, has driven the adoption of multi-modal deep intelligence for distributed sensing tasks, such as smart cabins and driving assistance. However, the arrival time of mobile sensory data vary due to modality size and network dynamics, which can lead to delays (if waiting for slow data) or accuracy decline (if inference proceeds without waiting). Moreover, the diversity and dynamic nature of mobile systems exacerbate this challenge. In response, we present a shift toopportunisticinference for asynchronous distributed multi-modal data, enabling inference as soon as partial data arrives. While existing methods focus on optimizing modality consistency and complementarity, known as modal affinity, they lack acomputationalapproach to control this affinity in open-world mobile environments.${\sf AdaFlowLite}$pioneers the formulation of structured cross-modality affinity in mobile contexts using a hierarchical analysis-based normalized matrix. This approach accommodates the diversity and dynamics of modalities, generalizing across different types and numbers of inputs. Employing an multi-modal lightweight Swin Transformer (MMLST),${\sf AdaFlowLite}$facilitates real-time and flexible data imputation, adapting to various modalities and downstream tasks without retraining. Experiments show that${\sf AdaFlowLite}$significantly reduces inference latency by up to 80.4% and enhances accuracy by up to 62.1%, while achieving nearly a 50% reduction in energy consumption, outperforming status quo approaches. Also, this method can enhance LLM performance to preprocess asynchronous data.

As computational hardware advance, integrating deep learning (DL) models into mobile devices has become ubiquitous for visual tasks. However, “data distribution shift” in live sensory data can lead to a degradation in the accuracy of mobile DL models. Conventional domain adaptation methods, constrained by their dependence on pre-compiled static datasets for offline adaptation, exhibit fundamental limitations in real-time practicality. While modern online adaptation methodologies enable incremental model evolution, they remain plagued by two critical shortcomings: computational latency from excessive resource demands on mobile devices that compromise temporal responsiveness, and accuracy collapse stemming from error accumulation through unreliable pseudo-labeling processes. To address these challenges, we introduce AdaShift, an innovative cloud-assisted framework enabling real-time online model adaptation for vision-based mobile systems operating under non-stationary data distributions. Specifically, to ensure real-time performance, the adaptation trigger and plug-and-play adaptation mechanisms are proposed to minimize redundant adaptation requests and reduce per-request costs. To prevent accuracy collapse, AdaShift introduces a novel anti-collapse parameter restoration mechanism that explicitly recovers knowledge, ensuring stable accuracy improvements during model evolution. Through extensive experiments across various vision tasks and model architectures, AdaShift demonstrates superior accuracy and 100ms-level adaptation latency, achieving an optimal balance between accuracy and real-time performance compared to baselines.

This study introduces a novel approach for preventing overexertion in home fitness through fine-grained detection of respiratory parameters. To overcome the robustness limitation associated with using a composite signal for wireless sensing, we introduce an optimization-based signal separation model. This model effectively disentangles composite signals into static and dynamic components, while preserving the intricate details of target movements or activities. Specifically, by constructing a reference signal derived from the dominant static component, we eliminate time-varying phase shifts and leverage the invariant property of the dynamic component’s amplitude for precise separation. A system calledFinerSenseis developed, which is able to accurately and robustly detect fine-grained respiratory parameters such as respiration rate, depth, and inhalation-to-exhalation ratio with accuracy rates exceeding 97%, 95%, and 91%, respectively. Extensive experiments show that the developed system outperforms state-of-the-art baselines significantly, empowering users to optimize exercise intensity and duration while mitigating the risk of overexertion. We believe that this work is able to facilitate the seamless transition of wireless sensing systems from laboratory prototypes to practical and user-friendly applications.

Integrating Unmanned Aerial Vehicles (UAVs) and autonomous vehicles within the crowdsensing paradigm offers a promising approach to collecting environment-relevant data over large spatial areas, particularly in disaster-stricken or high-risk regions. However, deploying crowdsensing systems in emergency response scenarios presents substantial challenges. The lack of prior environmental knowledge complicates the selection of optimal sensing locations and strategy optimization, often relying on costly trial-and-error methods. Additionally, realtime decision-making is critical in such scenarios, requiring the rapid identification of optimal deployment strategies. Yet, the absence of prior knowledge further complicates the assessment of the optimality of these strategies. This gap remains inadequately addressed in existing research. To address this, we present the first framework that frames these challenges as a rapid online strategy optimization problem for mobile agent-based crowdsensing systems operating in unknown environments during emergency response scenarios. We propose DGap-UCB, a novel approach within the multi-armed bandit (MAB) framework, which efficiently identifies the optimal sensing strategy with highconfidence guarantees. Leveraging the Upper-Confidence Bound (UCB) technique, DGap-UCB iteratively refines strategy selection based on reward feedback. To accelerate learning, we introduce a gap-confidence pair (Δt, δt)-based Quick Stopping Criterion, enabling rapid and high-confidence identification of the optimal strategy. Extensive experiments on both synthetic and real-world datasets demonstrate the superiority of DGap-UCB over stateof-the-art techniques

Unlike traditional edge caching paradigms, similarity edge caching enables the retrieval of similar content from local caches to fulfill user requests, reducing reliance on remote data centers and improving system performance. Although several pioneering works have contributed to similarity edge caching, most focus on single-edge nodes and/or static environment settings, which are impractical for real-world applications. To address this gap, we investigate the similarity caching problem in dynamic cooperative edge networks, where a set of edge nodes cooperatively serve requests generated from arbitrary distributions with similar content over fluctuating transmission links. This presents a significant challenge, as it requires balancing content similarity with delivery latency over the transmission network and learning the environment in real-time to optimize caching policies. We frame this problem within an adversarial Multi-Armed Bandit framework to accommodate the continuously changing operational environment. To solve this, we propose an online learning-based approach named MABSCP, which dynamically updates caching policies based on real-time feedback to minimize the service cost of edge caching networks. To enhance implementation efficiency, we devise both an offline compact strategy construction method and an online Gibbs sampling method. Finally, trace-driven simulation results demonstrate that our proposed approach outperforms several existing methods in terms of system performance.

Mobile video applications today have attracted significant attention. Deep learning model (e.g., deep neural network, DNN) compression is widely used to enable on-device inference for facilitating robust and private mobile video applications. The compressed DNN, however, is vulnerable to the agnostic data drift of the live video captured from the dynamically changing mobile scenarios. To combat the data drift, mobile ends rely on edge servers to continuously evolve and re-compress the DNN with freshly collected data. We design a framework, AdaEvo, that efficiently supports the resource-limited edge server handling mobile DNN evolution tasks from multiple mobile ends. The key goal of AdaEvo is to maximize the average quality of experience (QoE), i.e., the proportion of high-quality DNN service time to the entire life cycle, for all mobile ends. Specifically, it estimates the DNN accuracy drops at the mobile end without labels and performs a dedicated video frame sampling strategy to control the size of retraining data. In addition, it balances the limited computing and memory resources on the edge server and the competition between asynchronous tasks initiated by different mobile users. With an extensive evaluation of real-world videos from mobile scenarios and across four diverse mobile tasks, experimental results show that AdaEvo enables up to 34% accuracy improvement and 32% average QoE improvement.

Visual Simultaneous Localization and Mapping (vSLAM) is a prevailing technology for many emerging robotic applications. Achieving real-time SLAM on mobile robotic systems with limited computational resources is challenging because the complexity of SLAM algorithms increases over time. This restriction can be lifted by offloading computations to edge servers, forming the emerging paradigm ofedge-assisted SLAM. Nevertheless, the exogenous and stochastic input processes affect the dynamics of the edge-assisted SLAM system. Moreover, the requirements of clients on SLAM metrics change over time, exerting implicit and time-varying effects on the system. In this paper, we aim to push the limit beyond existing edge-assist SLAM by proposing a new architecture that can handle the input-driven processes and also satisfy clients’ implicit and time-varying requirements. The key innovations of our work involve a regional feature prediction method for importance-aware local data processing, a configuration adaptation policy that integrates data compression/decompression and task offloading, and an input-dependent learning framework for task scheduling with constraint satisfaction. Extensive experiments prove that our architecture improves pose estimation accuracy and saves up to 47% of communication costs compared with a popular edge-assisted SLAM system, as well as effectively satisfies the clients’ requirements.

Unmanned Aerial Vehicle (UAV)-assisted Mobile Edge Computing (MEC) has received extensive attention because it provides resilient computation services for multiple Mobile Users (MUs). However, due to the increasing scale of offloaded tasks, the uncertain mobility of MUs, and the limited energy budget of UAV and MUs, it is extremely challenging to achieve satisfactory Quality-of-Service (QoS). Non-Orthogonal Multiple Access (NOMA), a promising technology to serve multiple MUs with limited communication resources, has great potential to be integrated with MEC. To this end, this paper proposes a QoS-oriented NOMA-enhanced UAV-MEC system, which aims to capture the potential gains of uplink NOMA and enable more MUs to benefit from edge computing servers in resource-constrained UAV-assisted MEC environments. This synergy reduces MUs' uplink energy consumption but poses new challenges in resource allocation and UAV trajectory design. To address these challenges, we define a new metric called System Overhead Ratio (SOR) to reflect the system's QoS, and then consider a joint optimization problem of resource allocation, transmission power control, and UAV trajectory design, with the goal of minimizing the SOR. Given the NP-hard nature of the optimization problem, we propose a Lyapunov and convex optimization-based Low-complexity Online Resource allocation and Trajectory optimization method (LORT) to solve it, and further analyze the convergence and complexity of LORT. Finally, extensive simulations show that the proposed method surpasses other benchmarks, reducing the SOR by approximately$10\%$-$25\%$under various scenarios.

With the integration of IoT and 5G technologies, UAV crowd sensing has emerged as a promising solution to overcome the limitations of traditional Mobile Crowd Sensing (MCS) in terms of sensing coverage. As a result, UAV crowd sensing has been widely adopted across various domains. However, existing UAV crowd sensing methods often overlook the semantic information within sensing data, leading to low transmission efficiency. To address the challenges of semantic extraction and transmission optimization in UAV crowd sensing, this paper decomposes the problem into two sub-problems: semantic feature extraction and task-oriented sensing data transmission optimization. To tackle the semantic feature extraction problem, we propose a semantic communication module based on Multi-Scale Dilated Fusion Attention (MDFA), which aims to balance data compression, classification accuracy, and feature reconstruction under noisy channel conditions. For transmission optimization, we develop a reinforcement learning-based joint optimization strategy that effectively manages UAV mobility, bandwidth allocation, and semantic compression, thereby enhancing transmission efficiency and task performance. Extensive experiments conducted on real-world datasets and simulated environments demonstrate the effectiveness of the proposed method, showing significant improvements in communication efficiency and sensing performance under various conditions.

The intelligent dialogue system, aiming at communicating with humans harmoniously with natural language, is brilliant for promoting the advancement of human-machine interaction in the era of artificial intelligence. With the gradually complex human-computer interaction requirements, it is difficult for traditional text-based dialogue system to meet the demands for more vivid and convenient interaction. Consequently, Visual-Context Augmented Dialogue (VAD) System, which has the potential to communicate with humans by perceiving and understanding multimodal information (i.e., visual context in images or videos, textual dialogue history), has become a predominant research paradigm. Benefiting from the consistency and complementarity between visual and textual context, VAD possesses the potential to generate engaging and context-aware responses. To depict the development of VAD, we first characterize the concept model of VAD and then present its generic system architecture to illustrate the system workflow, followed by a summary of multimodal fusion techniques. Subsequently, several research challenges and representative works are investigated, followed by the summary of authoritative benchmarks and real-world application of VAD. We conclude this article by putting forward some open issues and promising research trends for VAD, e.g., the cognitive mechanisms of human-machine dialogue under cross-modal dialogue context, mobile and lightweight deployment of VAD.

Edge Federated Learning (EFL) has demonstrated significant potential in the field of Artificial Intelligence of Things (AIoT) by protecting data privacy and reducing communication costs. However, in real-world scenarios, multiple independent edge networks seldom collaborate due to factors such as data heterogeneity and the absence of a central server. Mobile devices, acting as bridges across different environments, offer an opportunity to enable dynamic collaboration among multiple edge networks. Nevertheless, as mobile devices transition between edge networks, they may encounter new classes with only a few samples, leading to catastrophic forgetting of previous knowledge and overfitting in new environments. To address this challenge, we propose Cross-F$^{2}$SCIL, a Federated Few-Shot Class Incremental Learning method that enables on-demand dynamic collaboration in mobile edge network environments. Cross-F$^{2}$SCIL allows mobile devices to efficiently learn new class knowledge from few-shot samples upon entering new edge networks while consolidating prior knowledge to prevent forgetting. Specifically, to mitigate the forgetting caused by new class overwriting on devices and parameter dilution at the server, we design a two-phase training framework. In the first phase, we learn a local model using Prototype Augmentation to enhance the retention of prior knowledge. In the second phase, we obtain the global model via Hierarchical Personalized Parameter Aggregation to effectively integrate learned knowledge across devices. To effectively learn new class information while reducing overfitting, we incorporate Self-Supervised Knowledge Aggregation and Prototype Knowledge Fusion to enhance model generalization and seamlessly integrate new classes into the existing model. Compared to the best-performing baseline on each dataset, Cross-F$^{2}$SCIL achieves an average improvement of 5.52% in Average Accuracy across five datasets, with the maximum improvement reaching 7.97%.

Mobile Edge Computing (MEC) leverages embedded devices to deliver low-latency and cost-effective intelligent services, such as autonomous driving. Federated learning allows MEC devices to collaboratively learn a global model without exposing private data. However, the dynamic real-world environments that MEC devices operate in can lead to catastrophic forgetting of previously learned knowledge. While continual learning has been applied in federated learning to retain knowledge about past data, there are significant barriers when extending it for evolving MEC environments due to unbearable communication and computation costs for resource-constrained devices. We address this by proposing Sparse-FCL, a federated continual learning framework that uses sparse training to reduce training overhead and improve model performance. Specifically, to retain the generalization knowledge of specific tasks, we introduce a progressive neuron selection via multi-device collaboration module, which gradually selects the key neurons that show importance on multiple devices in each continual learning task. In addition, we propose a task-adaptive topology exploration module, aiming to provide more pliable dynamic sparse training configurations for the scenarios of federated continual learning. Experiments on FCL benchmarks demonstrate Sparse-FCL's superior accuracy under high sparsity levels, and it also achieves reductions of 80.4%, 89.5%, and 19.7% in communication, computation, and storage overhead, respectively, compared to existing federated continual learning methods.

UAV-enabled multi-access edge computing (MEC) is expanding possibilities for integrated space-air-ground networks, especially in the 5G era and beyond. In this scenario, tasks from mobile users (MUs) are offloaded to nearby UAVs for execution, with results returned upon completion. However, the unpredictable mobility of MUs, coupled with dynamic network conditions and fluctuating resource availability, can degrade the reliability of communication links, leading to increased delivery latency, particularly for tasks involving large computational results. To meet stringent QoS requirements, adaptive task migration across UAVs is essential to minimize latency. To address this issue, in this paper, we first investigateComputationTaskMiGration (CTMiG) problem in UAV-enabled dynamic MEC networks, focusing on joint optimization of task-serving (offloading and migration) decisions to reduce latency for all MUs. We propose the ILCTS algorithm, an imitation learning-based joint optimization method that adaptively adjusts scheduling strategies in response to environmental changes. An improved PPO algorithm is first proposed to train a policy and generate expert data, followed by generative adversarial imitation learning to imitate the data and continuously explore new ones through online learning to enhance the policy. Experimental results demonstrate that our algorithm achieves superior performance in training accuracy and average latency compared to other representative methods.

Rapid advances in microfluidics technologies have facilitated the emergence of highly integrated lab-on-a-chip (LoC) biochip systems. With such coin-sized biochips, complicated bioassay procedures can be executed efficiently without any human intervention. To ensure the correctness and precision of assay outcomes, however, cross-contamination among different fluid samples/reagents needs to be dealt with separately during assay execution. As a consequence, wash operations have to be introduced and a wash path network needs to be established on the chip to remove the residues left in flow channels. To realize optimized assay procedures with efficient wash operations, we propose PathDriver-Wash in this paper, a path-driven wash optimization method for continuous-flow LoC biochip systems. The proposed method includes the following three key techniques: 1) The necessity of contamination removals is analyzed systemically to avoid unnecessary wash operations, 2) wash operations are integrated with the regular removal of excess fluids, so that extra path occupations caused by wash can be minimized, and 3) optimized wash paths and time windows are computed and assigned to wash operations, so that the completion time of assays can be minimized. Experimental results demonstrate that the proposed method leads to highly efficient wash procedures as well as minimized assay completion times.

Multi-Camera Active Object Tracking is an attractive technique in the area of intelligent surveillance, where cameras share their observations via the wireless communication to collaboratively track the target. Due to the variability in wireless channel, the dynamic transmission delay between cameras significantly affects the collaboration performance, especially when the tracking is time-sensitive. In this paper, we propose a channel-aware multi-camera active object tracking (CAMAOT) system, to achieve the stable and improved tracking performance. Specifically, a communication decision module is designed in CAMAOT, where the cameras’ communication graph and communication resource allocation adapt to the channels. Our experiments demonstrate that for time-varying channels, CAMAOT has a stable performance improvement over other systems, particularly when the communication resources are limited.

Online disease diagnosis, gathering the patients’ symptoms and making diagnoses through online dialogue, grows rapidly worldwide. Manual-based approach, e.g., Haodaifu, employs real-world doctors, providing high-quality but high-cost medical services. In contrast, machine-based approach, e.g., 01bot, that utilizes machine learning models can make automatic diagnosis but lacks reliable accuracy. While some work has enabled human-AI collaboration in disease diagnosis, their collaboration pattern is simple and needs to be further improved. Therefore, we aim to introduce a doctor-enhanced and low-cost human-AI collaboration pattern. There are two key challenges. 1) How to utilize expert knowledge in doctor feedback to enhance AI’s capability? 2) How to design a collaboration workflow to achieve a low-cost doctor workload while ensuring accuracy? To address the above challenges, we propose the Human-AI collaboration framework for disease diagnosis via doctor-enhanced transformer, called HAIformer. Specifically, to enhance AI’s capability, we propose a machine module that leverages doctors’ medical knowledge through doctor-enhanced attention, using a graph attention-based matrix; to reduce doctor workload, we propose an activation module that uses two units in a cascading manner for human-AI allocation. Experiments on four real-world datasets show that HAIformer can achieve up to 91.2% accuracy with only 18.9% human effort and one-third of dialogue turns. Further real-world clinic study highlights its advantages in practical applications.

Federated learning (FL) learns an optimal global model by aggregating local models trained on distributed data from different devices. Due to heterogeneous data distributions across devices, local models will be divergent, resulting in the global model’s performance degradation. Recent studies attempt to balance local models to obtain a global model that can adapt to each device. But they ignore a more challenging problem that redundant local models from devices will break the balance, resulting in the global model overfitting redundant local models. Therefore, we propose FedSmooth, a novel global aggregation algorithm. FedSmooth first identifies the redundant local models without sensitive local information (e.g., label distribution), then designs a smooth global aggregation to strengthen the effect of local models that can accelerate finding the optimal global model. Experimental results show that our method outperforms 4 SOTA baseline methods even if there is more redundancy.

With the growing reliance on digital systems in today's mobile Internet era, robust authentication methods are crucial for safeguarding personal data and controlling access to resources. Conventional methods, such as knowledge-based and biometric-based authentication, are widely used but still have some usage limitations and potential security concerns, like wearing protective suits/masks or being imitated by attackers with ulterior motives. In this paper, we propose another earphone-based authentication system, namely EarPass, that leverages users' unique head motion patterns in response to a very short period of music segment. Here, we employ a Convolutional Neural Network (CNN)-based feature extractor to capture and map distinct head motions into a well-separated latent space, achieving high-dimensional data extraction. We demonstrate the consistency, uniqueness, and robustness of head motion patterns through extensive experiments and reach a 98.2% F1-score, indicating superior performance compared to conventional authentication methods. Additionally, EarPass is user-friendly, secure, and adaptable to various environments, including noisy and movement-oriented scenarios. By integrating the authentication system into Android devices, we showcase its real-world applicability and low energy consumption with minimal latency. The source code of EarPass will be open-source to further research and collaboration within the community.

In autonomous robotic systems, the parallel processing of multiple tasks often competes for limited resources, affecting system performance and the robot’s responsiveness to environmental changes. Traditional computational task scheduling methods often overlook the dynamic nature of task priorities in autonomous robotic systems, where task importance can shift based on interactions with the external environment. Therefore, there’s a crucial need for a mechanism capable of adaptively adjusting task scheduling in response to environmental changes, ensuring timely access to resources for critical tasks. To address this challenge, this study presents Priorest, a neural network model that incorporates multimodal data processing and multitask learning. Priorest integrates sensor data with logs monitoring computational device performance to predict events influencing task priority, enabling task adjustments while preserving essential resource allocations. When deployed in autonomous robotic systems, Priorest’s event-prediction-based adjustment strategy reduced critical task completion times by 18.7%, which demonstrates the effectiveness of Priorest in enhancing parallel task scheduling.

During the past few years, with the advent of large-scale pre-trained language models (PLMs), there has been a significant advancement in cross-domain text classification with limited labeled samples. However, most existing approaches still face the problem of excessive computation overhead. While some non-pretrained language models can reduce the computation overhead, the performance could sharply drop off. To resolve few-shot learning problems on resource-limited devices with satisfactory performance, we propose a prototype rectification framework, ProtoRectifier, based on pre-trained model distillation and episodic meta-learning strategy. Specifically, a representation refactor based on DistilBERT is developed to mine text semantics. Meanwhile, a novel prototype rectification approach (i.e., Mean Shift Rectification) is put forward by making full use of the pseudo labeled query samples, so that the prototype of each category can be updated during the meta-training phase without introducing additional time overhead. Experiments on multiple real-world datasets demonstrate that ProtoRectifier outperforms the state-of-the-art baselines, not only achieving high cross-domain classification accuracy but also reducing the computation overhead significantly.

As AI-enabled machines become increasingly prevalent, there is a strong impetus to harness the complementary strengths of humans and machines to enhance productivity and reduce costs in collaborative workspaces such as manufacturing and warehouses [1]. However, efficient team coordination remains challenging due to the heterogeneity of team agents and the dynamic nature of human agents. Existing exact methods often rely on assumptions and mathematical models, which struggle to scale and accurately predict time-varying human performance [2]. While offline Reinforcement Learning (RL) demonstrates potential, it is time-consuming and heavily reliant on training data, often limited in practical factory settings [3]. Therefore, a scalable and data-efficient team coordination method that considers the varying capabilities of heterogeneous agents in collaborative systems is urgently needed to facilitate effective human-machine partnerships.

Visual Crowdsensing (VCS) has gradually become an emerging research field as the built-in cameras of smart mobile devices have become a common recording tool in daily life. To meet the task requirements of VCS applications in the sensing process, sensing platforms usually use distributed data acquisition to collect image data from different sources. However, this leads to a large amount of redundant data in the final collected data set, which seriously affects the data quality. To solve the above problems, this paper proposes a dynamic redundancy detection method (VCSRD) for visual crowdsensing data based on contrastive learning. The method fuses the multimodal information of metadata and visual content through comparative clustering to realize the redundancy detection of data. Not only improves the accuracy of redundant data detection but also has some flexibility for data under different tasks. The effectiveness and flexibility of the proposed method under different data sizes are verified through experiments on real image datasets. Compared to the baseline method, VCSRD shows superior performance on all three clustering metrics.

Sparse Mobile Crowdsensing (Sparse MCS) is an emerging paradigm for data collection that significantly reduces the cost of smart city data acquisition while ensuring data quality by collecting data only from a subset of sensing cells and inferring data from the remaining unsensed cells using spatial and temporal correlations. Existing research has primarily focused on single sensing task scenarios, yet practical applications often involve multiple types of sensor data. These data types exhibit correlations that can be leveraged to optimize task allocation, thereby reducing sensing costs. This paper addresses the task assignment problem in multi-task scenarios, proposing the MDIA method based on information entropy theory and similarity matrices to finely assess the importance of data points. Building upon this, we further propose the MTA method, which employs a hybrid sensing mode (combining participatory sensing and opportunistic sensing) to select appropriate participants for sensing critical areas. Extensive experiments conducted on two real-world sensing datasets containing multiple data types validate the effectiveness of the proposed methods.

Video Anomaly Detection (VAD) systems can autonomously monitor and identify disturbances, reducing the need for manual labor and associated costs. However, current VAD systems are often limited by their superficial semantic understanding of scenes and minimal user interaction. Additionally, the prevalent data scarcity in existing datasets restricts their applicability in open-world scenarios. In this paper, we introduce HAWK, a novel framework that leverages interactive large Visual Language Models (VLM) to interpret video anomalies precisely. Recognizing the difference in motion information between abnormal and normal videos, HAWK explicitly integrates motion modality to enhance anomaly identification. To reinforce motion attention, we construct an auxiliary consistency loss within the motion and video space, guiding the video branch to focus on the motion modality. Moreover, to improve the interpretation of motion-to-language, we establish a clear supervisory relationship between motion and its linguistic representation. Furthermore, we have annotated over 8,000 anomaly videos with language descriptions, enabling effective training across diverse open-world scenarios, and also created 8,000 question-answering pairs for users' open-world questions. The final results demonstrate that HAWK achieves SOTA performance, surpassing existing baselines in both video description generation and question-answering. Our codes/dataset/demo will be released at https://github.com/jqtangust/hawk.

Medical online consultation is important to healthcare worldwide, with hundreds of millions of participants each year. However, expert-level online consultations are expensive due to the shortage of medical professionals, while AI models are unreliable because they have unpredictable risks. Therefore, we introduce human-machine collaboration to medical online consultation and focus on symptom inquiry, as the basis for disease diagnosis. There are two key issues: 1) how to design an intelligent assignment strategy that can determine whether doctors or models participate in each turn? 2) how to design an effective execution strategy that can improve the machine's inquiry ability among considerable symptoms? To address the above issues, we propose the Human-AI Diagnostic Team (HADT) framework based on Hierarchical Reinforcement Learning (HRL), which aims to achieve high accuracy with low manpower. Specifically, HADT has two layers. The upper one is responsible for assignment, in which we propose a module called master that enables intelligent human-machine assignments through the masked RL with reward shaping. The lower one is responsible for execution, consisting of a doctor and a proposed module called machine. This module can effectively ask about symptoms through the masked HRL with bottom-up training. Experiments on the public datasets show that HADT can achieve up to 89.4% accuracy with only 10.9% human effort, as confirmed by real clinical doctors using our online interface.

On-device adapting to continual, unpredictable domain shifts is essential for mobile applications like autonomous driving and augmented reality to deliver seamless user experiences in evolving environments. Test-time adaptation (TTA) emerges as a promising solution by tuning model parameters with unlabeled live data immediately before prediction. However, TTA's unique forward-backward-reforward pipeline notably increases the latency over standard inference, undermining the responsiveness in time-sensitive mobile applications. This paper presents AdaShadow, a responsive test-time adaptation framework for non-stationary mobile data distribution and resource dynamics via selective updates of adaptation-critical layers. Although the tactic is recognized in generic on-device training, TTA's unsupervised and online context presents unique challenges in estimating layer importance and latency, as well as scheduling the optimal layer update plan. AdaShadow addresses these challenges with a backpropagation-free assessor to rapidly identify critical layers, a unit-based runtime predictor to account for resource dynamics in latency estimation, and an online scheduler for prompt layer update planning. Also, AdaShadow incorporates a memory I/O-aware computation reuse scheme to further reduce latency in the reforwardpass. Results show that AdaShadow achieves the best accuracy-latency balance under continual shifts. At low memory and energy costs, Adashadow provides a 2x to 3.5x speedup (ms-level) over state-of-the-art TTA methods with comparable accuracy and a 14.8% to 25.4% accuracy boost over efficient supervised methods with similar latency.

The rise of mobile devices equipped with numerous sensors, such as LiDAR and cameras, has spurred the adoption of multi-modal deep intelligence for distributed sensing tasks, such as smart cabins and driving assistance. However, the arrival times of mobile sensory data vary due to modality size and network dynamics, which can lead to delays (if waiting for slower data) or accuracy decline (if inference proceeds without waiting). Moreover, the diversity and dynamic nature of mobile systems exacerbate this challenge. In response, we present a shift to opportunistic inference for asynchronous distributed multi-modal data, enabling inference as soon as partial data arrives. While existing methods focus on optimizing modality consistency and complementarity, known as modal affinity, they lack a computational approach to control this affinity in open-world mobile environments. AdaFlow pioneers the formulation of structured cross-modality affinity in mobile contexts using a hierarchical analysis-based normalized matrix. This approach accommodates the diversity and dynamics of modalities, generalizing across different types and numbers of inputs. Employing an affinity attention-based conditional GAN (ACGAN), AdaFlow facilitates flexible data imputation, adapting to various modalities and downstream tasks without retraining. Experiments show that AdaFlow significantly reduces inference latency by up to 79.9% and enhances accuracy by up to 61.9%, outperforming status quo approaches. Also, this method can enhance LLM performance to preprocess asynchronous data.

Cross-Domain Recommendation~(CDR) seeks to enable effective knowledge transfer across domains. Most existing works rely on either representation alignment or transformation bridges, but they come with shortcomings regarding identifiability of domain-shared and domain-specific latent factors. Specifically, while CDR describes user representations as a joint distribution over two domains, these methods fail to account for its joint identifiability as they primarily fixate on the marginal distribution within a particular domain. Such a failure may overlook the conditionality between two domains and how it contributes to latent factor disentanglement, leading to negative transfer when domains are weakly correlated. In this study, we explore what should and should not be transferred in cross-domain user representations from a causality perspective. We propose a Hierarchical causal subspace disentanglement approach to explore the Joint IDentifiability of cross-domain joint distribution, termed HJID, to preserve domain-specific behaviors from domain-shared factors. HJID abides by the feature hierarchy and divides user representations into generic shallow subspace and domain-oriented deep subspaces. We first encode the generic pattern in the shallow subspace by minimizing the Maximum Mean Discrepancy of initial layer activation. Then, to dissect how domain-oriented latent factors are encoded in deeper layers activation, we construct a cross-domain causality-based data generation graph, which identifies cross-domain consistent and domain-specific components, adhering to the Minimal Change principle. This allows HJID to maintain stability whilst discovering unique factors for different domains, all within a generative framework of invertible transformations that guarantee the joint identifiability. With experiments on real-world datasets, we show that HJID outperforms SOTA methods on both strong- and weak-correlation CDR tasks.

In the realm of the Internet of Things, a multitude of interconnected and intelligent devices enable profound interaction with the physical world. Within this context, a combination of aerial and ground robots for active target tracking aims to leverage their complementary capabilities to improve task efficiency in practice. However, most previous works rely on homogeneous and centralized methods among heterogeneous robots, which often leads to serious communication challenges that may undermine multirobot collaboration. At the same time, in nature, crows and wolves effectively collaborate in predation by utilizing their complementary abilities and communication mechanisms. Inspired by this phenomenon, we propose aerial-ground collaborative tracking (AGCoTrack), an independent reinforcement learning framework integrated with an efficient communication module for AGCoTrack. Empowered by this module, aerial and ground robots can dynamically determine the timing and content of communication. We conduct experiments in various environments, and the results demonstrate the ability of our method to reduce both communication frequency and bandwidth in comparison with baselines.

Gas disaster is one of the major disasters faced by coal mines, and more than 50% of gas disasters are concentrated in the working face. However, due to the small number of gas monitoring sensors and the heavy workload of artificial inspection of the working face, it is very difficult to monitor the high coverage of the working face. Aiming at the problems of low coverage of monitoring data and high acquisition cost, this article uses sparse mobile crowdsensing (MCS) to monitor the gas concentration, which specifically involves the cell selection of sensing area and the data inference of gas concentration in unaware area. First, this article combines gas source distribution and working face air flow characteristics to efficiently divide the gas concentration sensing cells. We propose cell selection based on distributed weighted self-attention mechanism deep reinforcement learning (DWS-DQN). The cell selection algorithm utilizes an attention mechanism to capture the key states of reinforcement learning to assist in optimization and decision making. Second, we propose gas concentration inference based on diffusion coefficient weighting (DCW). Based on the gas concentration diffusion coefficient of coal mine working face, we weighted the quantitative results of different characteristics to construct the gas concentration inference model. Finally, experiments on two real coal mines sensing data sets verify the effectiveness of our proposed algorithms. Compared to the baseline method, the DWS-DQN model and DCW model both exhibit good performance. The method based on the combination of DWS-DQN and DCW reduces the average MAPE result by 6.87%.

Relying on the development of crowdsourcing ideas and mobile crowd sensing (MCS) technology, many tasks that originally required a lot of manpower and material resources have been solved efficiently. However, with the development of urbanization, the traditional MCS systems have gradually been unable to cope with the demands of massive sensing tasks and high spatio-temporal sensing coverage. The challenges are as follows: 1) The scarcity of participants and the limitation of human motion rules lead to the existence of spatio-temporal blind spots in the process of collecting sensing data; 2) The single type of participants limits the sensing ability of the system and the types of data that can be collected, which affects sensing precision and quality. With the emergence of various intelligent sensing terminals in the city, the heterogeneous crowd sensing that integrates human, machine and things participants has become a new generation of sensing mode. In the spatio-temporal related sensing scenarioes, this article designs a new human-machine-things cooperative scheduling (HMT-CS) algorithm framework by comprehensively considering the diverse sensing skills, spatio-temporal trajectories, sensing costs of heterogeneous participants and system total budget constraints. The algorithm can match suitable heterogeneous participants for each task, which greatly improves the sensing quality, sensing fairness and overall utility of heterogeneous MCS systems. We combined multiple public real urban datasets to conduct an in-depth comparative analysis and comprehensive evaluation of the algorithm, and the results show that our method is superior to other baselines in all indicators.

With rapid advancements in artificial intelligence (AI) technologies, AI-powered machines are increasingly capable of collaborating with humans to enhance decision-making in various human–machine collaboration scenarios, e.g., medical diagnosis, criminal justice, and autonomous driving. As a result, human–machine computing (HMC) has emerged as a promising computing paradigm that integrates the expertise of humans with the reliable data processing capabilities of machines. Using HMC to facilitate the processing of domain-specific tasks has a lot of potential, but is limited in system-level scalability, i.e., there is no one common easy-to-use interface. In this article, we present human-machine operating system(hmOS), an open extensible platform for researchers to experiment with HMC for investigating system-centric human–machine collaboration problems.hmOSsupports flexible human–machine collaboration on the strength of the quality-aware task decomposition and allocation. To achieve that, the underlying system architecture and runtime environment are first developed to build a foundational abstraction for the kernel ofhmOS. Second,hmOSfacilitates flexible human–machine collaboration through a suitability-based task allocation mechanism, quality estimation guided by fuzzy rules, and iterative feedback on result tuning. We implement the newly proposedhmOSin a prototype featuring interactive interfaces. Finally, we conduct extensive and realistic experiments to validate the effectiveness of our platform across diverse tasks, showcasing the broad feasibility ofhmOS.

As one of the fundamental tasks of autonomous driving, depth perception aims to perceive physical objects in three dimensions and to judge their distances away from the ego vehicle. Although great efforts have been made for depth perception, LiDAR-based and camera-based solutions have limitations with low accuracy and poor robustness for noise input. With the integration of monocular cameras and LiDAR sensors in autonomous vehicles, in this article, we introduce a two-stream architecture to learn the modality interaction representation under the guidance of an image reconstruction task to compensate for the deficiencies of each modality in a parallel manner. Specifically, in the two-stream architecture, the multi-scale cross-modality interactions are preserved via a cascading interaction network under the guidance of the reconstruction task. Next, the shared representation of modality interaction is integrated to infer the dense depth map due to the complementarity and heterogeneity of the two modalities. We evaluated the proposed solution on the KITTI dataset and CALAR synthetic dataset. Our experimental results show that learning the coupled interaction of modalities under the guidance of an auxiliary task can lead to significant performance improvements. Furthermore, our approach is competitive against the state-of-the-art models and robust against the noisy input. The source code is available at https://github.com/tonyFengye/Code/tree/master .

Traffic prediction is one of the fundamental spatio-temporal prediction tasks in urban computing, which is of great significance to a wide range of applications, e.g., traffic controlling, vehicle scheduling, etc. Recently, with the expansion of the city and the development of public transportation, long-range and long-term spatio-temporal correlations play a more important role in traffic prediction. However, it is challenging to model long-range spatial dependencies and long-term temporal dependencies simultaneously in two aspects: 1) complex influential factors, including spatial, temporal and external factors. 2) multiple spatio-temporal correlations, including long-range and short-range spatial correlations, as well as long-term and short-term temporal correlations. To solve these issues, we propose a spatio-temporal memory augmented multi-level attention network for fine-grained traffic prediction, entitled ST-MAN. Specifically, we design a spatio-temporal memory network to encode and memorize fine-grained spatial information and representative temporal patterns. Then, we propose a multi-level attention network to explicitly model both short-term local spatio-temporal dependencies and long-term global spatio-temporal dependencies at different spatial scales (i.e., grid and region levels) and temporal scales (i.e., daily and weekly levels). In addition, we design an external component that takes external factors and spatial embeddings as inputs to generate location-aware influence of the external factors much more efficiently. Finally, we design an end-to-end framework optimized with the contrastive objective and supervised objective to boost model performance. Empirical experiments over coarse-grained and fine-grained real-world datasets demonstrate the superiority of the ST-MAN model compared to several state-of-the-art baselines.

Cell selection and data offloading are the keys to obtaining MCS services with low sensing cost and low data processing delay. Due to the spatiotemporal correlation between data and the local-area coverage of edge servers, cell selection and data offloading will affect each other and require co-optimization. To achieve the co-optimization, we design the method OptInter based on the hierarchical reinforcement learning. OptInter can realize the interactive training between cell selection model and data offloading model. Finally, we evaluate our proposed method based on four datasets, each of which composited by real-world (e.g., NO$_{2}$concentration, AQI value, Didi order, and Didi trajectory) data and simulated data. Compared with the four baseline methods (e.g., OptMOEA/D, OptStageCD, OptStageDC, and OptWeight), the comprehensive performance of our proposed method can be improved by 11.83%, 20.48%, 10.14%, and 42.27% on average, respectively.

During the last decade, there have been lots of efforts on wireless sensing using Wi-Fi signals, which can be divided into two categories, i.e., the pattern-based approach and the model-based approach. Recently, more and more attention has been paid on the model-based approach, mainly due to its superiority of no need for collecting a large dataset or retraining the model for new environments. However, existing models are mainly designed for Line-of-Sight (LoS) scenarios, which are not applicable to Non-Line-of-Sight (NLoS) scenarios, such as through-wall sensing. To bridge this gap, we put forward a through-wall wireless sensing model to reveal the sensing mechanism of Wi-Fi signals in NLoS scenarios. In particular, arefraction-awareFresnel zone model is developed by taking into account both the reflection propagation and the refraction propagation of Wi-Fi signals. For the first time, we discover that the geometric distribution of Fresnel zones becomes uneven, due to the difference in dielectric constants between the air and the wall. Specifically, some areas become denser and other areas become sparser, leading to thesqueeze effectandstretch effectof Fresnel zones. Inspired by the insight, we further put forward a new metric namedcompression-ratioto quantify the through-wall sensing capability of Wi-Fi signals. Meanwhile, a set of algorithms are developed to guide the deployment of Wi-Fi sensing systems. To validate the proposed model, we implement a through-wall respiration sensing prototype system. Experiments show that the respiration detection performance varies significantly when the user locates in different areas. Specifically, for two sensing locations (one in the compression area and the other in the expansion area) symmetrically distributed on both sides of the transceivers’ connection line, the difference in mean absolute errors (MAE) can exceed 3 times.

As a non-intrusive sensing approach, the gait-based human identification technique attracts extensive attention. For the gait-based human identification technique, the unique gait feature is captured and extracted. Owing to the strong environment robustness and good privacy protection, the radar, especially the single-input multiple-output (SIMO) Doppler radar, is proposed as a promising way to capture the gait feature. However, the existing SIMO Doppler radar-based methods require the person to walk along a straight-line trajectory, which hinders their practical application. In this paper, we propose a gait-based human identification system for the weakly constrained trajectory, called CovertEye. In CovertEye, the person can be identified, when he/she walks along variable directions. To this end, we propose a trajectory segmentation algorithm to divide the trajectory into many straight-line trajectory segments. Based on the trajectory segments, we design the gait-based human identification method. In particular, we propose a normalization method to eliminate the differences in the direction of movement and the length among trajectory segments. The normalized signal spectrogram is exploited for the deep learning based feature extraction and human identification. We develop a prototype of the CovertEye system. The extensive experimental results demonstrate that our proposed system can achieve the identification accuracy of 82:4%.

Mobile edge computing (MEC) brings computational resources to the edge of network that triggers the paradigm shift of centralized machine learning towards federated learning. Federated learning enables edge nodes to collaboratively train a shared prediction model without sharing data. In MEC, heterogeneous edge nodes may join or leave the training phase during the federated learning process, resulting in slow convergence of dynamic communities and federated learning. In this paper, we propose a fine-grained training strategy for federated learning to accelerate its convergence rate in MEC with dynamic community. Based on multi-agent reinforcement learning, the proposed scheme enables each edge node to adaptively adjust its training strategy (aggregation timing and frequency) according to the network dynamics, while compromising with each other to improve the convergence of federated learning. To further adapt to the dynamic community in MEC, we propose a meta-learning-based scheme where new nodes can learn from other nodes and quickly perform scene migration to further accelerate the convergence of federated learning. Numerical results show that the proposed framework outperforms the benchmarks in terms of convergence speed, learning accuracy, and resource consumption.

With the rise of Artificial Intelligence of Things (AIoT), integrating deep neural networks (DNNs) into mobile and embedded devices has become a significant trend, enhancing the data collection and analysis capabilities of IoT devices. Traditional integration paradigms rely on cloud-based training and terminal deployment, but they often suffer from delayed model updates, decreased accuracy, and increased communication overhead in dynamic real-world environments. Consequently, on-device training methods have garnered research focus. However, the limited local perception data and computational resources pose bottlenecks to training efficiency. To address these challenges, Federated Learning emerged but faces issues such as slow model convergence and reduced accuracy due to data privacy concerns that restrict sharing data or model details. In contrast, we propose the concept of trusted clusters in the real world (such as personal devices in smart spaces, trusted devices from the same organization/company, etc.), where devices in trusted clusters focus more on computational efficiency and can also share privacy. We propose CrowdLearning, a decentralized distributed training framework based on trusted AIoT device collectives. This framework comprises two collaborative modules: A heterogeneous resource-aware task offloading module aimed at alleviating training latency bottlenecks, and an efficient communication data reallocation module responsible for determining the timing, manner, and recipients of data transmission, thereby enhancing DNN training efficiency and effectiveness. Experimental results demonstrate that in various scenarios, CrowdLearning outperforms existing federated learning and distributed training baselines on devices, reducing training latency by 55.8% and lowering communication costs by 67.1%.

Mobile Crowdsensing (MCS) has become a popular sensing paradigm, where a number of participants use their mobile devices to collectively share and extract information related to a certain common interest. In this trend, many typical applications, such as environmental monitoring, intelligent transportation, and public safety, are emerging in our daily lives, and the need to quickly develop various new applications is becoming more urgent. However, existing programming frameworks for MCS applications either target specific scenarios that lack extensibility or require considerable development effort and expertise, hindering innovation in this direction. In order to reduce the burden of developing new MCS applications, we devise a developer-oriented generic programming framework, namely CrowdKit. It abstracts the common and fundamental data models and functions of MCS applications and makes them reusable. Meanwhile, it follows the principles of modular design, visual development, and automatic code generation to further bring extensibility and drastically reduce the difficulty and time cost of developers. Moreover, its algorithm modules can accommodate various advanced MCS algorithms, thus narrowing the gap between theory and practice. We implement and release a full-fledged version of CrowdKit, and conduct comprehensive case study and user study to demonstrate its simplicity, generality, extensibility and high efficiency.

Federated Learning (FL) in mobile edge computing (MEC) systems has recently been studied extensively. In ubiquitous environments, there are usually cross-edge devices that learn a series of tasks across multipleindependentedge FL systems. Due to the differences in the scenarios and tasks of different FL systems, cross-edge devices will forget past tasks after learning new tasks, which is unacceptable for devices that pay system costs to participate in FL. Continual learning (CL) is a viable solution to this problem, which aims to train a model to learn a series of tasks without forgetting old knowledge. Currently, there is no work to investigate the problem of CL in a cross-edge FL scenario. In this paper, we proposeCross-FCL, aCross-edgeFederatedContinualLearning framework. Specifically, it enables devices to retain the knowledge learned in the past when participating in new task training through a parameter decomposition based FCL model. Then various cross-edge strategies are introduced, including biased global aggregation and local optimization, to trade off memory and adaptation. We conducted experiments on a real-world dataset and other public datasets. Extensive experiments demonstrate that Cross-FCL achieves best accuracy on IID and highly non-IID tasks with a low storage cost compared to other baselines.

This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution. Nevertheless, existing MARL algorithms ignore effective information aggregation which is fundamental for improving the learning capacity of decentralized agents. In this paper, we design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop atopology-aware information aggregationstrategy to extract correlation-related information from unstructured data gathered in the road network. Particularly, we transfer the road network topology into a graph shift operator by forming a diffusion process on the topology, which subsequently facilitates the construction of graph signals. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning. Extensive experiments based on both synthetic and real-world datasets verify that our proposal outperforms existing decentralized algorithms.

Federated learning (FL) is a distributed machine learning methodology that can achieve collaborative model training among clients without collecting their private training data. Despite the great benefits in privacy protection, FL still faces challenges like limited computation capabilities of clients (e.g., end devices) and significant communication overheads when applied to mobile edge-cloud networks. To address these issues, this paper proposes a novel three-layer FL framework with Parameter Selection and Pre-synchronization (PSPFL) to achieve fast and accurate model training in mobile edge-cloud networks. The basic idea of PSPFL is that clients can select partial model parameters for transmission. Then, base stations aggregate these model parameters cooperatively (i.e., pre-synchronization) and send the aggregated results to the server for global model update periodically. However, there is an intrinsic trade-off between parameter transmission overhead and model training loss. To strike a desirable balance between them, we investigate the optimal parameter pre-synchronization round and local training round under PSPFL. Specifically, we propose an Alternating Minimization (AM) algorithm to obtain the initial local training round and parameter pre-synchronization round. Moreover, we integrate Deep Q-network with AM (namely DQNAM) to explore and update the optimal solution. Finally, extensive simulations are conducted to evaluate the performance of the proposed method on commonly used datasets. The results show that the proposed method can reduce the sum of FL completion time and training loss by an average of 20.72% - 69.25% compared to benchmark methods.

Efficiently obtaining the up-to-date information in the disaster-stricken area is the key to successful disaster response. Unmanned aerial vehicles (UAVs), workers and cars can collaborate to accomplish sensing tasks, such as life detection task in disaster-stricken areas. In this paper, we explicitly address the route planning for a group of agents, including UAVs, workers, and cars, with the goal of maximizing the sensing task completion rate. we propose a MARL-based heterogeneous multi-agent route planning algorithm called MANF-RL-RP. The algorithm has made targeted designs in terms of global-local dual information processing and model structure for heterogeneous multi-agent, making it effectively considers the collaboration among heterogeneous agents and the long-term impact of current decisions. Finally, we conducted detailed experiments based on the rich simulation data. In comparison to the baseline algorithms, namely Greedy-SC-RP and MANF-DNN-RP, MANF-RL-RP has exhibited a significant performance improvement. Compared to MANF-DNN-RP and Greedy-SC-RP, the task completion rate based on MANF-RL-RP increased by an average of 8.82% and 56.8%, respectively.

Incentive mechanisms are essential to incentive workers carrying mobile handheld devices to participate in mobile crowd sensing and finally achieve good truth discovery performance. However, malicious workers may report false or malicious data to defraud rewards, resulting in service quality degradation. Moreover, the existing incentive mechanism is challenging to identify malicious workers when recruiting workers in reality, which results in low accuracy of truth discovery and waste of cost. In this paper, we propose an Incentive-based Truth Discovery (ITD) scheme to incentive credible workers to submit high-quality data, thereby enhancing the accuracy of truth discovery. In the ITD, an unmanned aerial vehicle (UAV)-assisted split-aggregation truth discovery mechanism is proposed firstly to infer the truth. The addition of the UAV can improve the accuracy of truth discovery and assist in evaluating workers’ trust. Then, we evaluate the quality of participants’ data and propose a data quality-based trust meter to update each worker’s trust to guide future recruitment efforts. Finally, a Quality-aware Trustworthy Incentive (QTI) mechanism is proposed to select credible workers for data collection and provide them with reasonable payments. The experimental results show that ITD improves the accuracy of truth discovery by 98.47%, over the state-of-the-art, at a sensing cost reduced by as high as 43.89%.

With the rapid development of deep learning, recent research on intelligent and interactive mobile applications (e.g., health monitoring, speech recognition) has attracted extensive attention. And these applications necessitate the mobile edge computing scheme, i.e., offloading partial computation from mobile devices to edge devices for inference acceleration and transmission load reduction. The current practices have relied on collaborative DNN partition and offloading to satisfy the predefined latency requirements, which is intractable to adapt to the dynamic deployment context at runtime. AdaMEC, a context-adaptive and dynamically combinable DNN deployment framework, is proposed to meet these requirements for mobile edge computing, which consists of three novel techniques. First, once-for-all DNN pre-partition divides DNN at the primitive operator level and stores partitioned modules into executable files, defined as pre-partitioned DNN atoms. Second, context-adaptive DNN atom combination and offloading introduces a graph-based decision algorithm to quickly search the suitable combination of atoms and adaptively make the offloading plan under dynamic deployment contexts. Third, runtime latency predictor provides timely latency feedback for DNN deployment considering both DNN configurations and dynamic contexts. Extensive experiments demonstrate that AdaMEC outperforms state-of-the-art baselines in terms of latency reduction by up to 62.14% and average memory saving by 55.21%.

Recently, in response to the low efficiency and high transmission latency of traditional centralized content delivery networks, especially in congested scenarios, edge caching has emerged as a promising method to bring content caching closer to the edge of the network. However, traditional content delivery methods might still lead to low utilization of cache resources. To tackle this challenge, this paper investigates a content recommendation-based edge caching method in multi-tier edge-cloud networks while considering content delivery and cache replacement decisions as well as bandwidth allocation strategies. First, we consider a multi-tier edge caching-enabled content delivery network architecture combined with a content recommendation system and formulate the optimization problem with the objective of minimizing long-term content delivery delay and maximizing cache hit rate. Second, considering time-varying system environments and uncertain content demands, we approximate the optimization process of content delivery and cache replacement for each agent as a Partially Observable Markov Decision Process (POMDP) and propose a single-agent Deep Deterministic Policy Gradient (DDPG)-based method. Subsequently, we extend the POMDP to a multi-agent scenario. To address the issue of agents converging to local optima and establish more personalized models, we propose a Federated Distributed DDPG-based method (FD3PG) to solve the corresponding problem in a multi-agent system. Finally, simulation results demonstrate that the proposed FD3PG achieves lower delivery delay and higher cache hit rate compared with other baselines in various scenarios. Specifically, compared with FADE, MADRL, and DDPG, FD3PG achieves a significant decrease in average delivery delay, approximately 10%, 11%, and 35% on the Synthetic dataset, and 12%, 14%, and 48% on the MovieLens Latest Small dataset, respectively.

Code translation, i.e., translating one kind of code language to another, plays an important role in scenarios such as application modernization and multi-language versions of applications on different platforms. Even the most advanced machine-based code translation methods can not guarantee an error-free result. Therefore, the participance of software engineer is necessary. Considering both accuracy and efficiency, it is suggested to work in a human-machine collaborative way. However, in many realistic scenarios, human and machine collaborate ineffectively - model translates first and then human makes further editing, without any interaction. To solve this problem, we propose hmCodeTrans, a novel method that achieves code translation in aninteractive human-machine collaborative way. It can (1) save the human effort by introducing two novel human-machine collaboration patterns: prefix-based and segment-based ones, which feed the software engineer's sequential or scattered editing back to model and thus enabling the model to make a better retranslation; (2) reduce the response time based on two proposed modules: attention cache module that avoids duplicate prefix inference with cached attention information, and suffix splicing module that reduces invalid suffix inference by splicing a predefined suffix. The experiments are conducted on two real datasets. Results show that compared with the baselines, our approach can effectively save the human effort and reduce the response time. Last but not least, a user study involving five real software engineers is given, which validates that the proposed approach owns the lowest human effort and shows the users’ satisfaction towards the approach.

Automatically detecting fake news has become increasingly necessary. Conventional approaches to fake news detection (FND) require a large number of training instances, which are not available in the scenario of new event FND (NEFND). More advanced methods address this problem through domain adaption (DA) to improve the overall performance of all events, or by transferring knowledge from source events. However, these methods either lack a target-oriented design or fail to perform effective transfer due to data scarcity in new events. This work focuses on the NEFND problem and proposes a multi-source selective transfer learning approach. Specifically, an integrated learner is built to make decisions, and an event-level transferability generator is designed to select more transfer-worthy source events, so as to achieve event-level selective transfer. Additionally, a two-stage training algorithm with a re-weighting optimization mechanism is also designed to highlight more transferable source instances, so as to achieve instance-level selective transfer and improve the performance on the target event. Experiments on the real-world multi-event fake news dataset that simulates the NEFND scenario are conducted to evaluate the effectiveness and superiority of the proposed approach.

Explainability is crucial for the successful use of AI for fake news detection (FND). Researchers aim to improve the explainability of FND by highlighting important descriptions in crowd-contributed comments as clues. From the perspective of law and sociology, there are distinct clues that are easy to discover and understand, and faint clues that require careful observation and analysis. For example, in fake news related to COVID-Omicron showing increased pathogenicity and transmissibility, distinct clues might involve virologists’ opinions regarding the inverse correlation between pathogenicity and transmissibility. Meanwhile, faint clues might be reflected in an infected person’s claim that the symptoms are milder than a cold (indirectly indicating reduced pathogenicity). Occasionally, the statements of some ordinary eyewitnesses can decisively reveal the truth of the news, leading to the judgment of fake news. Existing methods generally use static networks to model the entire news life-cycle, which makes it fail to capture the subtle dynamic interactions between individual clues and news. Thereby faint clues, whose relations to the truth of news are challenging to be characterized and extracted directly, are more likely to be overshadowed by distinct clues. To address this issue, we propose an explainable FND method, dubbed as PiercingEye, which leverages dynamic interaction information to progressively mine valuable clues. PiercingEye models the news propagation topology as a dynamic graph, with interactive comments serving as nodes, and employs the time-semantic encoding mechanism to refine the modeling of temporal interaction information between comments and news to preserve faint clues. Subsequently, it utilizes the self-attention mechanism to aggregate distinct and faint clues for FND. Experimental results demonstrate that PiercingEye outperforms state-of-the-art methods and is capable of identifying both faint and distinct clues for humans to debunk fake news.

It is pivotal and challenging for crowdsensing systems to guarantee the reliable uploading of sensory data from source devices (workers) to a centralized platform, in order to process sensing tasks accurately and fast. On one hand, with limited communication resources, uploading a massive amount of sensory data is not cost-effective. On the other hand, the disruption of uploading is inevitable because of stochastic network environments and worker dropout, resulting in extra wasting of resources. To address that, we focus on a collaborative uploading scenario and propose to reduce the uploading latency of sensory data by adaptive data allocation while retaining data integrity at the destination. A key technical challenge is to identify proper collaborative paths such that corresponding data allocation and uploading are reliable enough. As such, we formulate a joint optimization problem with the minimization goal of uploading latency by considering both path selection and data allocation. To mine helpful information from unstructured topology-aware data, we propose a new diffusion graph convolution module by forming information aggregation based on the diffusion process that characterizes the stochastic correlation of devices. After transforming the original problem into a primal-dual problem, an algorithm is then developed by adapting Advantage Actor-Critic (A2C) framework embedded with the diffusion graph convolution module. With extensive experiments, it is validated that the newly developed algorithm improves collaborative uploading by reducing uploading latency and also stabilizing the queue state of intermediate devices, compared to existing heuristic and learning-based methods.

An increasing number of energy-recycling consensus mechanisms are being employed to address the drawback of proof of work (PoW) wasting computation and energy. For instance, the computing power wasted in solving difficult but meaningless PoW puzzles is used to conduct practical federated learning tasks and train deep learning models. However, there remains a neglected issue of task difficulty adjustment. To address this problem, we propose a method for measuring task difficulty and an algorithm for adjustment to achieve controlled minting and stable transaction processing capacity for cryptocurrency based on energy-recycling consensus mechanisms. Our research evaluates the effectiveness of this algorithm and highlights the potential benefits of this approach.

The rapid evolution of public knowledge is the trend of the present era; rendering previously collected data susceptible to obsolescence. The continuously generated new knowledge could further affect the performance of the model trained with previous data, such a phenomenon is called temporal misalignment. A vanilla mitigation approach is to periodically update the model in a centralized learning scheme. However, in a decentralized learning framework like Federated Learning (FL), such a patch requires clients to upload the data, which contradicts FL's intention to protect clients' privacy. Furthermore, considering the stationary defenses in FL, new knowledge could be misjudged and rejected as malicious attacks, which hinders the further update of the model. Yet dynamically adapting defenses requires meticulous fine-tuning and harms the scalability. Thus in this poster, we raise such practical concern and discuss it in the context of FL. We then build a prototype of a GPT2-based FL framework and conduct experiments to demonstrate our perspective. The performance in new knowledge drops by 33.47% compared with the previous data, which justify the FL with defenses strategy can misjudge the new knowledge.

Learning accurate cross-domain preference mappings in the absence of overlapped users/items has presented a persistent challenge in Non-overlapping Cross-domain Recommendation (NOCDR). Despite the efforts made in previous studies to address NOCDR, several limitations still exist. Specifically, 1) while some approaches substitute overlapping users/items with overlapping behaviors, they cannot handle NOCDR scenarios where such auxiliary information is unavailable; 2) often, cross-domain preference mapping is modeled by learning deterministic explicit representation matchings between sampled users in two domains. However, this can be biased due to individual preferences and thus fails to incorporate preference continuity and universality of the general population. In light of this, we assume that despite the scattered nature of user behaviors, there exists a consistent latent preference distribution shared among common people. Modeling such distributions further allows us to capture the continuity in user behaviors within each domain and discover preference invariance across domains. To this end, we propose a Distributional domain-invariant Preference Matching method for non-overlapping Cross-Domain Recommendation (DPMCDR). For each domain, we hierarchically approximate a posterior of domain-level preference distribution with empirical evidence derived from user-item interactions. Next, we aim to build distributional implicit matchings between the domain-level preferences of two domains. This process involves mapping them to a shared latent space and seeking a consensus on domain-invariant preference by minimizing the distance between their distributional representations therein. In this way, we can identify the alignment of two non-overlapping domains if they exhibit similar patterns of domain-invariant preference. Experiments on real-world datasets demonstrate that DPMCDR outperforms the state-of-the-art approaches with a range of evaluation metrics.

The rising popularity of ear-wear devices equipped with motion sensors has brought concerns regarding privacy issues due to their powerful sensing capabilities. Previous studies have shown the potential for speech eavesdropping using earphone motion sensors with a sampling frequency of 1000 Hz. However, as the risks of such attacks continue to escalate, mobile operating systems like Android have imposed limitations on the sampling frequency, typically no more than 200 Hz, to avoid such attacks. The lower sampling frequency reduces the amount of collected information within the same timeframe, potentially leading to decreased accuracy. In this paper, we further investigate the effectiveness of utilizing earphone motion sensors for inferring sensitive information at a sampling frequency of 200 Hz while directly using raw data without any data transformation to prevent information loss. We employ a channel attention mechanism to dynamically adjust axis weights to address the varying energy levels across different sensor axes. Meanwhile, we analyze the impact of sampling frequency, environment, and volume on speech recognition performance. Additionally, we explore the extraction of other information from speech signals, such as speaker identity and gender. Our experiments on two datasets demonstrate high recognition accuracy for all three tasks at the 200Hz sampling frequency. We expect our work to raise awareness among manufacturers regarding the privacy issues associated with earphone motion sensors.

The advantages of modular robot systems stem from their ability to change between different configurations, enabling them to adapt to complex and dynamic real-world environments. Then, how to perform the accurate and efficient change of the modular robot system, i.e., the self-reconfiguration problem, is essential. Existing reconfiguration algorithms are based on discrete motion primitives and are suitable for lattice-type modular robots. The modules of freeform modular robots are connected without alignment, and the motion space is continuous. It renders existing reconfiguration methods infeasible. In this paper, we design a parallel distributed self-reconfiguration algorithm for freeform modular robots based on multi-agent reinforcement learning to realize the automatic design of conflict-free reconfiguration controllers in continuous action spaces. To avoid conflicts, we incorporate a collaborative mechanism into reinforcement learning. Furthermore, we design the distributed termination criteria to achieve timely termination in the presence of limited communication and local observability. When compared to the baselines, simulations show that the proposed method improves efficiency and congruence, and module movement demonstrates altruism.

Recent deep-learning-based face recognition systems have achieved significant success. However, most existing face recognition systems are vulnerable to spoofing attacks where a copy of the face image is used to deceive the authentication. A number of solutions are developed to overcome this problem by building a separate face anti-spoofing model, which however brings in additional storage and computation requirements. Since both recognition and face anti-spoofing tasks stem from the analysis of the same face image, this paper explores a unified approach to reduce the original dual-model redundancy. To this end, we introduce a compressed multi-task model to simultaneously perform both tasks in a lightweight manner, which has the potential to benefit lightweight IoT applications. Concretely, we regard the original two single-task deep models as teacher networks and propose a novel multi-teacher-assisted knowledge distillation method to guide our lightweight multi-task model to achieve satisfying performance on both tasks. Additionally, to reduce the large gap between the deep teachers and the light student, a comprehensive feature alignment is further integrated by distilling multi-layer features. Extensive experiments are carried out on two benchmark datasets, where we achieve the task accuracy of 93% meanwhile reducing the model size by 97% and reducing the inference time by 56% compared to the original dual-model.

Face verification is a trending way to verify someone’s identity in broad applications. But such systems are vulnerable to face spoofing attacks via, for example, a fraudulent copy of a photo, making it necessary to include face liveness detection as an additional safeguard. Among most existing studies, the face liveness detection is realized in a separate machine learning model in addition to the model for face verification. Such a two-model configuration may face challenges when deployed onto platforms with limited computation power and storage (e.g. mobile phone, IoT devices), especially considering each model may have millions of parameters. Inspired by the fact that humans can verify a person’s identity and liveness at a single glance from a face, we develop a novel system, named FaceLivePlus, to learn a single and universal face descriptor for the two tasks (face verification and liveness detection) so that the computational workload and storage space can be halved. To achieve this, we formulate the underlying relationship between the two tasks, and seamlessly embed this relationship in a distance ranking deep model. The model directly works on features rather than classification labels, which makes the system well generalized on unseen data. Extensive experiments show that our average half total error rate (HTER) has at least 15% and 8% improvement from the state-of-the-arts on two benchmark datasets. We anticipate this approach could become a new direction for face authentication.

Layout generation is important in the field of graphic design and has attracted intensive research attention recently. To further prompt human-computer interactions, we construct the ALDA to assist users throughout the design process, which achieves adaptive and diverse content expansions upon only one element for beginners and generates high-quality posters subsequently. Specifically, to obtain diverse contents, we propose aesthetic-aware design graphs (AGs) for effective poster representation and propose a self-constrained blending strategy upon related examples. In addition, we build a novel layout generator to better arrange elements conditioned on our AGs. Finally, we implement ALDA as an online tool with a set of controllable factors to enhance its practicality.

Recently, a novel Mobile Agent-based Mobile Crowdsensing (MA-MCS) paradigm has emerged, which utilizes unmanned aerial vehicles, pilotless automobiles, etc., to implement data sensing from surroundings. Practically, in time-sensitive applications the freshness of collected data is of critical importance. However, restricted by the intermittent bandwidth of wireless channels, the generated data might not be successfully transferred to the application server for further processing. Fortunately, benefit from the hosted edge computing capability, it is feasible to locally process the sensing data and then transfer the concise result. In this paper, we study a novel and practical problem in the Age-of-Information (AoI) driven MA-MCS systems in wireless edge computing. Specifically, restricted by the issues of battery recharging, etc., we strive to schedule the mobile agents and make transmission/computing decisions in a collaborative manner, aiming at minimizing the total AoI threshold violation. To this end, we propose a two-phase deep reinforcement learning-based solution, namely KCDDQN, including spatiotemporal clustering and decision-making learning. Finally, extensive simulations are carried out to demonstrate the effectiveness of our proposed KCDDQN approach.

Cloud-edge collaboration, as an emerging computing paradigm, aims to solve the shortcomings of remote transmission of conventional cloud computing. More precisely, it combines the powerful resource service capability of cloud computing with the advantages of low latency and relatively low energy consumption of edge computing to achieve the goal of optimization of various applications. However, with the rapid growth of computation-intensive industrial tasks, the overload problem of edge networks is becoming increasingly serious. Prior studies usually assume that the real-time state of edge resources has been known when selecting the offloading strategy so as to classify and execute tasks, but do not consider the fragmentation and heterogeneity features of edge computing resources. In light of these, we first generalize and model the computing resources of the edge nodes uniformly and then propose new heterogeneous task classification and recognition methods empowered by edge intelligence. We conduct intensive experiments to justify that our proposed design can minimize the data transmission delay caused by repeated computational tasks while saving energy consumption.

Multi-camera Active Object Tracking (AOT) has become a promising technique to continuously track the moving target in the intelligent surveillance system. The communication among cameras provides an essential way to share the information and improve the tracking performance. To achieve the efficient collaboration among cameras, we propose a novel communication-efficient multi-camera AOT system in this paper, where one camera cooperates with other cameras to realize the active object tracking by sharing its visual observations when necessary. In the proposed system, we design a binary classified neural network based switch to determine whether to communicate. Moreover, we propose an attention based communication group construction method, which allows the camera to decide who to communicate. Through the control of whether to communicate and who to communicate, many communication connections of less importance are pruned. We construct a high-fidelity environment to mimic the real-world scenario. Experimental results demonstrate that compared to the existing systems, the proposed system can significantly reduce the occupied bandwidth and achieve an improved tracking performance, especially for the high-speed moving target.

Recent sequential recommendation models rely increasingly on consecutive short-term user-item interaction sequences to model user interests. These approaches have, however, raised concerns about both short- and long-term interests. (1) short-term: interaction sequences may not result from a monolithic interest, but rather from several intertwined interests, even within a short period of time, resulting in their failures to model skip behaviors; (2) long-term: interaction sequences are primarily observed sparsely at discrete intervals, other than consecutively over the long run. This renders difficulty in inferring long-term interests, since only discrete interest representations can be derived, without taking into account interest dynamics across sequences. In this study, we address these concerns by learning (1) multi-scale representations of short-term interests; and (2) dynamics-aware representations of long-term interests. To this end, we present an Interest Dynamics modeling framework using generative Neural Processes, coined IDNP, to model user interests from a functional perspective. IDNP learns a global interest function family to define each user's long-term interest as a function instantiation, manifesting interest dynamics through function continuity. Specifically, IDNP first encodes each user's short-term interactions into multi-scale representations, which are then summarized as user context. By combining latent global interest with user context, IDNP then reconstructs long-term user interest functions and predicts interactions at upcoming query timestep. Moreover, IDNP can model such interest functions even when interaction sequences are limited and non-consecutive. Extensive experiments on four real-world datasets demonstrate that our model outperforms the state-of-the-art on various evaluation metrics.

Abstract Program translation aims to translate one kind of programming language to another, e.g., from Python to Java. Due to the inefficiency of translation rules construction with pure human effort (software engineer) and the low quality of machine translation results with pure machine effort, it is suggested to implement program translation in a human–machine cooperative way. However, existing human–machine program translation methods fail to utilize the human’s ability effectively, which require human to post-edit the results (i.e., statically modified directly on the model generated code). To solve this problem, we propose HMPT (Human-Machine Program Translation), a novel method that achieves program translation based on human–machine cooperation. It can (1) reduce the human effort by introducing a prefix-based interactive protocol that feeds the human’s edit into the model as the prefix and regenerates better output code, and (2) reduce the interactive response time resulted by excessive program length in the regeneration process from two aspects: avoiding duplicate prefix generation with cache attention information, as well as reducing invalid suffix generation by splicing the suffix of the results. The experiments are conducted on two real datasets. Results show compared to the baselines, our method reduces the human effort up to 73.5% at the token level and reduces the response time up to 76.1%.

As important indicators of urban public safety, public safety and environmental security (PSES) is related to residents’ living security and greatly affects their quality of life and happiness index. Due to PSES characteristics, such as diverse forms, wide distribution, and unpredictable occurrence times, traditional solutions consume huge manpower, and time in the implementation process. Although some professional software and hardware systems have emerged to assist in solving the problems, there are still challenges, such as limited sensing coverage and monotonous sensing modes, lack of interaction and understanding between systems and tasks, and scarcity of effective system architecture and functional modules. To meet these challenges, we design a PSES multiterminal fusion system (SafeCity) based on the idea and technology of heterogeneous mobile crowd sensing. With collaboration among humans, machines, and things (H-M–T), the proposed system makes full use of the idle mobility, sensing, and computing resources in the city, and systematically provides a solution to the various PSES issues. Apart from the system architecture, functions, core mechanism, and algorithm libraries, the task execution flow is explained in depth through the description of several cases. We implement a prototype to verify the rationality and effectiveness of SafeCity. And, comprehensive comparison and evaluation show that SafeCity is far superior to other solutions in terms of function, performance, and stability.

With the popularity of embedded devices such as LIDAR sensors and depth cameras, the resulting point clouds become the main data format for representing the 3D world and spawn various smart mobile applications. A key technology for enabling these applications to furnish high-quality services is real-time point cloud registration on mobile devices, which synthesizes a complete model or a large-scale scene from multiple partial scans. It aims to deliver increasing sensing range, faster 3D reconstruction and more robust robot navigation. Unfortunately, the performance of these applications is limited by the scale and partial loss of raw point cloud frame. The existing solutions for point cloud registration are difficult to deploy on mobile devices due to their complex models and assumptions about point cloud pairs with large overlap, which cause significant delay and inaccuracy. This paper proposes VPRNet - the first voxel-based registration solution that can achieve real-time partial-to-partial registration with competitive registration quality while being more advantageous for large-scale point clouds on mobile devices. We conduct real-world experiments and extensive simulations cross various datasets and platforms to validate the efficacy of VPRNet and further compare the performance with state-of-the-art approaches.

It is commonly agreed that a recommender system should use not only explicit information (i.e., historical user-item interactions) but also implicit information (i.e., incidental information) to deal with the problem of data sparsity and cold start. The knowledge graph (KG), due to its expressive structural and semantic representation capabilities, has been increasingly used for capturing auxiliary information for recommender systems, such as the recent development of graph neural network (GNN) based models for KG-aware recommendation. Nevertheless, these models have the shortcoming of insufficient node interactions or improper node weights during information propagation, which limits the performance of recommender systems. To address this issue, we propose a Multi-dimension Interaction based attentional Knowledge Graph Neural Network (MI-KGNN) for enhanced KG-aware recommendation. MI-KGNN characterizes similarities between users and items through information propagation and aggregation in knowledge graphs. As such, it can optimize the updating direction of node representation by fully exploring multi-dimension interactions among nodes during information propagation. In addition, MI-KGNN introduces a dual attention mechanism, which allows users and items to jointly determine the weight of neighbor nodes. As a result, MI-KGNN can effectively capture and represent both structural (i.e., the topology of interactions) and semantic information (i.e., the weight of interactions) in the knowledge graph. Experimental results show that the proposed model significantly outperforms baseline methods for top-K recommendation. Specifically, the recall rate is increased by 5.78%, 6.66%, and 3.22% on three public datasets, compared with the best performance of existing methods.

With the global prevalence of mobile devices, concerns about mobile devices regarding privacy breaches and data leakage are rising. Although sensor permissions are required for mobile applications to access outputs of built-in sensors, motion sensors (e.g., accelerometer and gyroscope) can be visited directly without permission requirement. Extant studies have shown that motion sensors may cause breaches of confidential information, such as passwords, digits, and voice-based commands, but whether it is possible to synthesize intelligible speech waveforms from low-resolution motion sensors has been understudied. In this article, we present an escalated side-channel attack of built-in speakers by synthesizing intelligible speech waveforms from low-resolution vibration signals. Opposite to traditional classification problems, we formulate this task as a generative problem and introduce an end-to-end synthesis framework dubbed asAccMyrinxto eavesdrop on the speaker via the low-resolution vibration signals. InAccMyrinx, we introduce the data alignment solution to provide the pair-wise voice-vibration sequences and present wavelet-based MelGAN (WMelGAN) with multi-scale time-frequency domain discriminators to generate intelligible acoustic waveforms. We conducted intensive experiments and demonstrated the feasibility of synthesizing the intelligible acoustic signals from low-resolution solid-borne vibration signals. Compared with existing synthesis solutions, our proposed solution outperforms the baselines in both subject and object metrics with the smoothed word error rate of 42.67% and the Mel-Cepstral distortion of 0.298. In addition, the quality of synthetic speeches could be impacted by several factors, including gender, speech rate, volume, and sampling frequency.

The lack of rich visual information affects the shopping experience of the visually impaired (VI), including identifying and selecting commodities. Recent studies on VI assistance have focused on commodity identification but neglected to provide fine-grained and intuitive pick-up guidance, which is not user-friendly enough. Therefore, we propose a user-driven shopping assistance system to improve the shopping experience for VI users. We first conduct an in-depth interview with VI, then implement a prototype shopping assistance system—MobiEye, with real-time video analysis. Further, we evaluate the prototype system and identify two directions to optimizing the existing system: (1) Improving the pick-up accuracy in dense placement; and (2) reducing the latency and communication overhead. To address these two problems, we design a new guidance strategy for dense placements and propose a mobile-edge cocomputing strategy with a motion predictor and a communication gate to filter the transmitted images. Finally, we invited VI participants to evaluate the effectiveness and efficiency of MobiEye. The experimental results show that MobiEye achieved a 13% improvement in pick-up success rate and a 12 s reduction in average pick-up time compared with other shopping assistance systems.

Differential mymargin privacy has been deployed to machine learning platforms to preserve the privacy of data in use. A long neglected but important fact is that data privacy is a non-replenishable resource and should be carefully scheduled to maximize its utility gain. In this work, we propose a new privacy budgeting system—DPlanner, which estimates data blocks’ importance to queries and assigns fractional privacy budget to those data blocks contributing most to a query. The scheduler is novelly designed to include two-fold randomness, which satisfies differential privacy with tight budgets, at the same time guarantees the expected utility in the worst-case query sequence when queries arrive in an online fashion. Experiments in a variety of machine learning settings have shown that our DPlanner outperforms the state-of-the-art schedulers by serving at least 25% more queries, or reducing the total privacy consumption by over 50%.

Within-basket recommendation is to recommend suitable items for the current basket with some already known items. The within-basket auxiliary item recommendation ( WBAIR ) is to recommend auxiliary items based on the primary items in the basket. Such a task exists in many real-life scenarios. Unlike the associations between items that can be transmitted in both directions, primary and auxiliary relationships are unidirectional. Then, the suitable matching patterns between primary and auxiliary items cannot be explored by traditional directionless methods. Therefore, we design the Matc4Rec algorithm to integrate the primary and auxiliary factors, and finally recommend items that not only match the interests of users but also satisfy the primary and auxiliary relationships between items. Specifically, we capture the pattern from three aspects: matchability within-basket , matchability between baskets , and ubiquity . By exploiting this pattern, the designed algorithm not only achieves good results on real-world datasets but also improves the interpretability of recommendations. As a result, we can know which commodities are suitable as auxiliary items. The experiment results demonstrate that our algorithm can also alleviate the cold start problem.

Recently, dynamic graph representation learning has attracted more and more attention from both academic and industrial communities due to its capabilities of capturing different real-world phenomena. For a dynamic graph represented as a sequence of timestamped events, there are two kinds of evolutionary essences: temporal and repetitive information. At present, the temporal information of interactions (e.g., timestamps) have been deeply explored. However, as another vital nature of dynamic graphs, the repetitive information of interactions between two nodes is neglected, which may lead to inaccurate node representation. To address this issue, we propose a novel continuous-time dynamic graph representation learning model, which consists of a node-level-memory based module, a historical high-order neighborhood based vertical aggregation module and a repetitive-topological information based horizontal aggregation module. In particular, to characterize the evolving pattern of the repetitive information of interactions between a pair of nodes, we put forward a repetitive-interaction based attention mechanism to integrate the two key attributes (i.e., the content and the number of interactions) of repetitive interactions at different moments, based on the insight that the repetitive behaviors of nodes are widespread and essential. We conduct extensive experiments including future link prediction tasks (for transductive and inductive learning) and dynamic node classification task, and results on three real-life dynamic graph datasets demonstrate that the proposed method significantly outperforms state-of-the-art baselines, for both observed nodes and new ones.

Air pollution seriously affects public health, while effective air quality prediction remains a challenging problem since the complex spatial-temporal couplings exist in multi-area monitoring data of the city. Current approaches rarely consider relative geographical locations when capturing spatial-temporal relations, instead the latent inter-dependencies (i.e., implicit spatial relations) of data as a replacement. However, such relations cannot necessarily reflect the diffusion of air pollutants in the real world, and genuine location-related information could be lost during the implicit relation learning process. In this article, we introduce a new concept, geospatial-temporal data, and propose a novel deep neural network architecture, CoupledGT, to learn the geospatial-temporal couplings within data for air quality prediction. Specifically, the asymmetric diffusion relation of air quality data between two areas is first explicitly represented by the newly developed planar Gaussian diffusion (PGD) equation. And then, a geospatial couplings diffuser (GCD) is designed to parameterize the PGD equation and learn multi-areas diffusion mutually affected geospatial couplings. Besides, the RNN is employed to capture temporal couplings of each area, and incorporated with GCD to learn both shared and unique characteristics of the geospatial-temporal data simultaneously, which empowers the generalization and efficiency of the model. Extensive experiments on two real-world datasets demonstrate our method is robust and outperforms existing baseline methods in air quality prediction tasks.

Knowledge-enhanced dialogue systems aim at generating factually correct and coherent responses by reasoning over knowledge sources, which is a promising research trend. The truly harmonious human-agent dialogue systems need to conduct engaging conversations from three aspects as humans, namely (1) stating factual contents (e.g., records in Wikipedia), (2) conveying subjective and informative opinions about objects (e.g., user discussions on Twitter), and (3) impressing interlocutors with diverse expression styles (e.g., personalized expression habits). The existing knowledge base is a standardized and unified coding for factual knowledge, which could not portray the other two kinds of knowledge to make responses more informative and expressive diverse. To address this, we present CrowdDialog , a crowd intelligence knowledge-enhanced dialogue system, which takes advantage of “crowd intelligence knowledge” extracted from social media (with rich subjective descriptions and diversified expression styles) to promote the performance of dialogue systems. Firstly, to thoroughly mine and organize the crowd intelligence knowledge underlying large-scale and unstructured online contents, we elaborately design the C rowd I ntelligence K nowledge G raph ( CIKG ) structure, including the domain commonsense subgraph, descriptive subgraph, and expressive subgraph. Secondly, to reasonably integrate heterogeneous crowd intelligence knowledge into responses while ensuring logicality and fluency, we propose the G ated F usion with D ynamic Knowledge- D ependent ( GFDD ) model, which generates responses from the semantic and syntactic perspective with the context-aware knowledge gate and dynamic knowledge decoding. Finally, extensive experiments over both Chinese and English dialogue datasets demonstrate that our approach GFDD outperforms competitive baselines in terms of both automatic evaluation and human judgments. Besides, ablation studies indicate that the proposed CIKG has the potential to promote dialogue systems to generate fluent, informative, and diverse dialogue responses.

To meet the growing bike-sharing demands and make people’s travel convenient, the companies need to add new stations at locations where demands exceed supply. Before making reliable decisions on adding new stations, it is required to understand the spatial effects of new stations on the station network. In this paper, we study the deployment of the new station by estimating its varied causal effects on the demands of nearby stations, e.g., how does adding a new station (treatment) causally influence the demands (outcome) of nearby stations? When working with observational data, we should control hidden confounders, which cause spurious relations between treatments and outcomes. However, previous studies use historical data of the individual unit (e.g., the station’s historical demands) to approximate its hidden confounders, which cannot deal with the lack of historical data for new stations. And the conventional methods overlook the differences between units, which cannot be applied to our problem. To overcome the challenges, we propose a novel model (CausalSE) to estimate the varied effects of new stations on nearby stations, which uses the shared knowledge (i.e., similar traveling patterns among stations) to approximate hidden confounders. Experimental results on real-world datasets show that CausalSE outperforms 6 state-of-the-art methods.

Research on human mobility drives the development of economy and society. How to predict when and where one will go accurately is one of the core research questions. Existing work is mainly concerned with performance of mobility prediction models. Since accuracy of predict models does not indicate whether or not one’s mobility is inherently easy to predict, there has not been a definite conclusion about that to what extent can our predictions of human mobility be accurate. To help solve this problem, we describe the formalized definition of predictability of human mobility, propose a model based on additive Markov chain to measure the probability of exploration, and further develop an information theory based method for quantifying the predictability considering exploration of human mobility. Then, we extend our method by using mutual information in order to measure the predictability considering external influencing factors, which has not been studied before. Experiments on simulation data and three real-world datasets show that our method yields a tighter upper bound on predictability of human mobility than previous work, and that predictability increased slightly when considering external factors such as weather and temperature.

Online gathering large-scale heterogeneous tasks and multi-skilled participant can make the tasks and participants to be shared in real time. However, their online gathering will bring many intractable objective requirements, which makes task-participant matching become extremely complex. To cope well with the gathering, we design a hierarchy tree and time-series queue to organize tasks and participants. The data structures we designed can effectively meet all requirements that are brought due to tasks and participants gathering online. In addition, based on the designed data structures, we study online large-scale heterogeneous task allocation problem from three aspects: the computing pattern, the tree creation method, and the extension of matching strategy. Our best method (TsPY) is based on parallel computing in the computing pattern, adopts time first and then space in the tree creation method, and increases the short-distance first strategy in the matching strategy. Finally, we conducted detailed experiments under the conditions of different participant geographical distributions (i.e., uniform distribution, Gaussian distribution, and check-in empirical distribution), different sensing methods (i.e., participatory sensing and opportunistic sensing), and different recommendation methods (i.e., point recommendation and trajectory recommendation). The experimental results show that TsPY has a good performance in multiple indicators such as algorithm running time, task-participant matching rate, participant travel distance, and redundant tasks removed. Compared with serial computing, parallel computing can reduce the algorithm running time by more than 66% on average in our experimental environment. Compared with space first and then time, creating a tree based on time first and then space can increase task-participant matching rate by more than 13% on average. Increasing the short-distance first strategy can reduce the participant travel distance by more than 4% on average.

Although crowdsensing has emerged as a popular information collection paradigm, its security and privacy vulnerabilities have come to the forefront in recent years. However, one big limitation of previous research is that the security domain and the privacy domain are typically considered separately. Therefore, it is unclear whether the defense methods in the privacy domain will have unexpected impact on the security domain. To bridge this gap, in this paper, we propose a novel Disguise-based Data Poisoning Attack (DDPA) against the differentially private crowdsensing systems empowered with the truth discovery method. Specifically, we propose a novel stealth strategy, i.e., disguising the malicious behavior as privacy behavior, to avoid being detected by truth discovery methods. With this stealth strategy, the shortcoming of failing to maximize the attack effectiveness is avoided naturally through structuring a bi-level optimization problem, which can be solved with the alternating optimization algorithm. Moreover, we show that the differentially private crowdsensing systems are vulnerable to data poisoning attacks, and enhancing the level of privacy will bring more serious security threats. Finally, the evaluation results on the real-world dataset Emotion and the synthetic dataset SynData demonstrate that DDPA can not only achieve maximum utility damage but also remain undetected.

With the enrichment of types and numbers of sensing terminals, more and more devices such as mobile phones, smart wearables, mobile robots, drones have appeared in our life, enabling the development of Mobile Crowd Sensing (MCS) technology. MCS systems has gradually changed from isomorphic sensing to heterogeneous collaborative sensing, and finally evolved into a heterogeneous multi-source sensing mode of the fusion of humans, machines and objects (things). However, state-of-the-art systems/frameworks do not well support efficient interactions and Heterogeneous Crowd Agents (HCA) management in Heterogeneous MCS (H-MCS) systems. With this in mind, this article aims at two major gaps: Efficient interaction and collaboration of HCA, automated modeling and flexible management of HCA. To deal with the challenges, we design an ontology-based interaction and management middleware (CrowdManager). Three core modules that constitute the middleware: HCA Information Extraction and Representation (HER), Ontology-based HCA Construction & Management (OCM), and Communication and Interaction Module (CIM) are well demonstrated. Extensive comparative evalution suggests that our approach not only brings rich and efficient HCA management and interactive functions to H-MCS systems, but also reduces communication time and various resource occupancy rates by more than 50%.

Next mobile app recommendation aims to recommend the next app that a user is most likely to use based on the user’s app usage behaviors, which is beneficial for improving user experience, app pre-loading, and system optimization. However, existing works ignore the complex correlations between apps in the app usage sessions. In addition, they do not consider the dynamics of user interests over time. To address these concerns, we propose a novel model named dynamic usage graph network (DUGN) to recommend the next app that a user is most likely to use. To model the complex correlations among apps explicitly, we adopt the dynamic graph structure to learn the dynamics of user interests. Firstly, we extract user interests in each app usage graph by using the hierarchical graph attention mechanism. Secondly, we capture user interests evolving over time, and generate the dynamic user embeddings by modeling the temporal dependencies among multiple app usage graphs. Finally, we obtain the current user interests in the current app usage graph, fuse multiple user interests and generate comprehensive user embeddings for next mobile app recommendation. We conduct experiments on real-world datasets. The results show that our model outperforms the state-of-art recommendation methods.

Mobile Crowdsensing (MCS), which assigns outsourced sensing tasks to volunteer workers, has become an appealing paradigm to collaboratively collect data from surrounding environments. However, during actual task implementation, various unpredictable disruptions are usually inevitable, which might cause a task execution failure and thus impair the benefit of MCS systems. Practically, via reactively shifting the pre-determined assignment scheme in real time, it is usually impossible to develop reassignment schemes without a sacrifice of the system performance. Against this background, we turn to an alternative solution, i.e., proactively creating a robust task assignment scheme offline. In this work, we provide the first attempt to investigate an important and realisticRoBustTaskAssignment (RBTA) problem in MCS systems, and try to strengthen the assignment scheme's robustness while minimizing the workers’ traveling detour cost simultaneously. By leveraging the workers’ spatiotemporal mobility, we propose an assignment-graph-based approach. First, an assignment graph is constructed to locally model the assignment relationship between the released MCS tasks and available workers. And then, under the framework of evolutionary multi-tasking, we devise a population-based optimization algorithm, namelyEMTRA, to effectively achieve adequate Pareto-optimal schemes. Comprehensive experiments on two real-world datasets clearly validate the effectiveness and applicability of our proposed approach.

With the increasing prominence of smart mobile devices, an innovative distributed computing paradigm, namely Mobile Crowdsourcing (MCS), has emerged. By directly recruiting skilled workers, MCS exploits the power of the crowd to complete location-dependent tasks. Currently, based on online social networks, a new and complementary worker recruitment mode, i.e., socially aware MCS, has been proposed to effectively enlarge worker pool and enhance task execution quality, by harnessing underlying social relationships. In this paper, we propose and develop a novel worker recruitment game in socially aware MCS, i.e.,Acceptance-awareWorkerRecruitment (AWR). To accommodate MCS task invitation diffusion over social networks, we design a Random Diffusion model, where workers randomly propagate task invitations to social neighbors, and receivers independently make a decision whether to accept or not. Based on the diffusion model, we formulate the AWR game as a combinatorial optimization problem, which strives to search a subset of seed workers to maximize overall task acceptance under a pre-given incentive budget. We prove its NP hardness, and devise a meta-heuristic-based evolutionary approach namedMA-RAWRto balance exploration and exploitation during the search process. Comprehensive experiments using two real-world data sets clearly validate the effectiveness and efficiency of our proposed approach.

Until recently, a novel spatial crowdsourcing paradigm, namely Three-Dimensional (3D) spatial crowdsourcing, has emerged, in which the task requestors and the workers need travel to their designated third-party workplaces, e.g., shared offices, to deliver certain services, such as DiDi station ride-sharing service, Quyundong sport training service in the Online-To-Offline (O2O) applications. In 3D spatial crowdsourcing applications, a core issue is to develop an efficient global tasklist plan, based on the tripartite matching among the three parties, i.e., task requestors, workers and workplaces, which is different from the conventional spatial crowdsourcing. In this context, one key challenge is how to suitably schedule the available workers with the consideration of the interests of all the parties, under the constraint of worker resource. To answer the questions, in this paper, we propose and study a new problem, namely Social-Welfare-driven Task Scheduling (SWTS) problem, which strives to schedule the workers' continuous routines, i.e., successively implementing tasks for different requestors at different workplaces, to promote the social welfare for all the involved parties. We prove our studied problem is NP-hard, and devise two heuristic optimization algorithms to solve it. Finally, we conduct extensive experiments which verify the efficiency and effectiveness of the proposed algorithms on both real and synthetic data sets.

The wide deployment of machine learning algorithms has become a severe threat to user data privacy. As the learning data is of high dimensionality and high orders, preserving its privacy is intrinsically hard. Conventional differential privacy mechanisms often incur significant utility decline as they are designed for scalar values from the start. We recognize that it is because conventional approaches do not take the data structural information into account, and fail to provide sufficient privacy or utility. As the main novelty of this work, we proposeMatrix Gaussian Mechanism(MGM), a new$ (\epsilon,\delta)$-differential privacy mechanism for preserving learning data privacy. By imposing the unimodal distributions on the noise, we introduce two mechanisms based on MGM with an improved utility. We further show that with the utility space available, the proposed mechanisms can be instantiated with optimized utility, and has a closed-form solution scalable to large-scale problems. We experimentally show that our mechanisms, applied to privacy-preserving federated learning, are superior than the state-of-the-art differential privacy mechanisms in utility.

Automatically detecting fake news as early as possible becomes increasingly necessary. Conventional approaches of fake news early detection (FNED) verify news' veracity with a predefined and indiscriminate detection position, which depends on domain experience and leads to unstable performance. More advanced methods address this problem with a proposed concept of adaptive detection position (ADP), i.e. the position where the veracity of the news record can be concluded. Yet these methods either lack theoretical reliability or weaken complex dependencies among multi-aspect clues, thus failing to provide practical and reasonable detection. This work focuses on the adaptive FNED problem and proposes a novel efficient and reliable deep state space model, namely AdaDebunk, which models the complex probabilistic dependencies. Specifically, a Bayes' theorem-based dynamic inference algorithm is designed to infer the ADPs and veracity, supporting the accumulation of multi-aspect clues. Besides, a training mechanism with hybrid loss is also designed to solve the over-/under-fitting problems, which further trades off the performance and generalization ability. Experiments on two real-world fake news datasets are conducted to evaluate the effectiveness and superiority of AdaDebunk. Compared with the state-of-the-art baselines, AdaDebunk achieves a 10% increase in F1 performance. Meanwhile, a case study is provided to demonstrate the reliability of AdaDebunk as well as our research motivation.

Intelligent bots are evolving with the development of artificial intelligence, especially the deep learning method. Many skills like semantic judgment, speech recognition, and text generation have been added, making bots more like real persons. The latest ones, such as Microsoft XiaoIce, Amazon Alexa, and Apple Siri, focus on enhancing general functionalities but still overlook the personality of the bot itself nevertheless, e.g., unchanging name and its virtual appearance. To further personalize the user experience, we desire to make the appearance of intelligent bots more diverse, i.e., appearing capable of autonomously changing its characteristic appearance according to users’ contexts like the changing geolocation. In this paper, we designe a personalized appearance transformation framework for the next generation intelligent bots. Specifically, Multi-modal crowd-intelligence technology is used for differential analysis of various regions, and generative adversarial network (GAN) is customized to render the bot appearance target domain. We also collecte new region-specific data sets from social media platforms, implement a fully-fledged prototype, and demonstratedthe effectiveness of our proposed framework.

In the era of mobile computing and sharing econ-omy, spatial crowdsourcing has become an emerging paradigm where participants who meet the spatial requirements are actively joined in various tasks. However, previous work did not fully consider the heterogeneous nature of tasks with both spatio-temporal and sensing requirements. And for the participants, they contribute different amounts and types of sensing data due to the fact that their devices usually have various sensing capabilities. In light of this, it is desired to study the task allocation problem in such a heterogeneous spatial crowdsourcing scenario. Specifically, to accommodate the dynamic and complex features of this scenario, we design a Multi-Agent Soft Actor-Critic algorithm (TA-DSAC) that relies on spatio-temporal con-straint. Firstly, we construct available task allocation regions according to the spatio-temporal characteristics of the tasks and participants, as well as the matching degree and aggregate sensing quality, and then set an agent for each region. Next, the agents are trained based on discretized Soft Actor-Critic (SAC) and Centralized Training with Decentralized Execution (CTDE), making the agents self-adaptive to changes in this crowdsourcing environment. Extensive evaluations with real datasets justify the effectiveness of our proposed algorithm.

As an enabler of sixth-generation communication technology (6G), Federated Learning (FL) triggers a paradigm shift from "connected things" to "connected intelligence". FL implements on-device learning, where massive end devices jointly and locally train a model without private data leakage. However, FL suffers from problems of low accuracy and convergence rate when no data is shared to the central server and the data distribution is non-IID. In recent years, attempts have been made on hybrid FL, where very small amounts of data (e.g., less than 1%) is shared from the participants. With the opportunities brought by shared data, we notice that the server is capable of receiving the data in order to assist the FL process and mitigate the challenge of non-IID. Notably, existing hybrid FL only applies the model-level technologies belonging to the traditional FL and does not make full use of the characteristics of shared data to make targeted improvements. In this paper, we propose FedAux, a novel hybrid FL method at knowledge-level, which utilizes shared data to construct an auxiliary model and then transfer general knowledge to traditional aggregated model or client model for enhancing the accuracy of global model and speeding up the convergence of global model. We also propose two specific knowledge transfer strategies named c-transfer and i-transfer. We conduct extensive analysis and evaluation of our methods against the well-known FL methods, FedAvg and Hybrid-FL protocol. The results indicate that FedAux shows higher accuracy (10.89%) and faster convergence rate compared with other methods.

Surface inspection of industrial equipment defection plays a vital role in real production. Traditional inspection routines require a large number of inspection workers, which not only affects production efficiency but also leads to unreliable results. Computer vision-based detection approaches, e.g., using the deep learning method, have shown great potential in this trend. Specifically, the semantic segmentation algorithm based on Convolutional Neural Network (CNN) can extract relatively complete feature information. And the Transformer, which emerged from the field of Natural Language Processing (NLP), also performs well in maintaining and transmitting semantic information. In light of these, we propose to design a segmentation model called eSwin-UNet, i.e., enhanced Swin-UNet, that leverages the advantages of the CNN and Transformer. It uses multi-scale information fusion to better integrate the feature information in the CNN and Transformer branches. Moreover, it also utilizes deep supervision and makes two branches for collaborative training to further improve accuracy. By testing with the MVTec ITODD dataset, Fl-Score and Jaccard achieve results of 0.7891 and 0.6516 respectively, which outperform most current models.

The huge amount of video data produced by ubiqui tous cameras imposes significant challenges for users to efficiently obtain useful video information. Multi-view video summarization (MVS) aggregates multi-view videos into information-rich video summaries by considering content correlations within each view and between multiple views. Existing MVS methods fail to concentrate on performance across scenarios and usually achieve satisfactory performance on specific training datasets. However, when faced with unseen video scenarios, the quality of the summaries generated by existing methods may degrade. Moreover, they usually only use cameras for data acquisition, which require a large amount of network bandwidth to transfer the data to the server for processing. To bridge this gap, we propose a context-adaptive online reinforcement learning multi-view video summarization framework (COORS) that meets the low response latency performance requirements of context adaptation while ensuring camera hardware compatibility. Specifically, COORS enables retraining in new contexts by extracting contextindependent rewards, while improving model convergence speed based on representation learning and replica playback. Extensive experiments show that COORS has better performance compared to the state-of-the-art baselines.

Driven by state-of-the-art AI technologies, human-AI collaboration has become an important area in computer supported teamwork research. Principles for Human-Machine Computing (HMC) have been discussed to accomplish complex goals by outsourcing some computational steps to humans and collaboratively achieving more accurate results. In HMC systems, however, the human participant brings great challenges to efficient provisioning of resources. Virtualization provides ideas for increasing the agility, flexibility and scalability of resources and has been applied in traditional computer systems, such as cloud computing. Unfortunately, existing hardware virtualization scheme is not ready to address utilization, and performance limitations associated with Human Processing Unit (HPU) resources. To tackle this problem, in this paper, we propose an efficient HPU resource virtualization framework for HMC systems. In particular, we firstly describe the modeling details of the HPU resource. And on this basis, we present the Time Division Multiplexing (TDM)-based virtualization scheme which aims to establish the mapping between each real HPU (rHPU) and its virtual HPUs (vHPUs). Secondly, we apply our minds to address the vHPU reconfiguration problem by managing the vHPU waiting queue, and propose DvR-PSO algorithm. Finally, the performance of our proposed HPU virtualization framework is evaluated through simulation experiments, and the results show that our solution can make remarkable effectiveness, which can serve as guidelines for future research on HMC systems.

Social recommendation with Graph Neural Networks(GNNs) learns to represent cold users by fusing user-user social relations with user-item interactions, thereby alleviating the cold-start problem associated with recommender systems. Despite being well adapted to social relations and user-item interactions, these supervised models are still susceptible to popularity bias. Contrastive learning helps resolve this dilemma by identifying the properties that distinguish positive from negative samples. In its previous combinations with recommender systems, social relationships and cold-start cases in this context are not considered. Also, they primarily focus on collaborative features between users and items, leaving the similarity between items under-utilized. In this work, we propose socially-aware dual contrastive learning for cold-start recommendation, where cold users can be modeled in the same way as warm users. To take full advantage of social relations, we create dynamic node embeddings for each user by aggregating information from different neighbors according to each different query item, in the form of user-item pairs. We further design a dual-branch self-supervised contrastive objective to account for user-item collaborative features and item-item mutual information, respectively. On one hand, our framework eliminates popularity bias with proper negative sampling in contrastive learning, without extra ground-truth supervision. On the other hand, we extend previous contrastive learning methods to provide a solution to cold-start problem with social relations included. Extensive experiments on two real-world social recommendation datasets demonstrate its effectiveness.

Neural Multi-task Learning is gaining popularity as a way to learn multiple tasks jointly within a single model. While related research continues to break new ground, two major limitations still remain, including (i) poor generalization to scenarios where tasks are loosely correlated; and (ii) under-investigation on global commonality and local characteristics of tasks. Our aim is to bridge these gaps by presenting a neural multi-task learning model coined Hierarchical Task-aware Multi-headed Attention Network (HTMN). HTMN explicitly distinguishes task-specific features from task-shared features to reduce the impact caused by weak correlation between tasks. The proposed method highlights two parts: Multi-level Task-aware Experts Network that identifies task-shared global features and task-specific local features, and Hierarchical Multi-Head Attention Network that hybridizes global and local features to profile more robust and adaptive representations for each task. Afterwards, each task tower receives its hybrid task-adaptive representation to perform task-specific predictions. Extensive experiments on two real datasets show that HTMN consistently outperforms the compared methods on a variety of prediction tasks.

Online medical diagnosis refers to diagnosing diseases and providing treatment suggestions on the websites. It develops rapidly and has become a new choice for patients to seek medical treatment. Although manual online medical diagnosis is reliable, it has problems such as low efficiency, heavy burden on doctors, and long waiting time for patients. Relying on machines for automatic disease diagnosis is highly efficient, which, however, has low accuracy and reliability. In general, online medical diagnosis usually has two stages: inquiry and diagnosis. Inquiry stage refers to asking about the patient’s physiological, where the questions are usually streamlined, and thus can be handled by the machine. Diagnosis stage is to diagnose the disease and provide medical recommendations, which has strict requirements for accuracy and safety, and thus should be handled by the human. Inspired by this, in the paper we propose a human-machine collaboration based online medical diagnosis system, i.e., HM-MDS. In inquiry stage, the system employs the machine. It uses the BERT+CRF to identify symptoms in the patient’s dialogue and uses a DQN-based method to ask about symptoms. In diagnosis stage, the system employs both the machine and the human. The machine generates a pre-diagnosis result by calculating disease probability. Then the human doctor gives the final diagnosis result by checking the pre-diagnosis result and revising it if necessary. Obviously, HM-MDS can effectively save human doctor’s time as well as patient’s time, while ensure the accuracy of the diagnosis result. We conduct experiments on a real-world dataset. The results show our approach improves the online medical diagnosis’s reliability as well as patient satisfaction, and ensures diagnosis accuracy. The time cost for a reliable medical diagnosis is reduced to 36% compared with pure manual work.

Mobile crowd sensing and computing (MCSC) has become a hot research area in recent years. This article presents our vision of the next generation of MCSC, crowd intelligence with the deep fusion of human, machine, and Internet of Things (IoT), namely, CrowdHMT. It aims to build a self-organizing, self-learning, self-adaptive, and continuous-evolving smart space with the deep fusion of Crowdsourced human, machine, and IoT intelligence. This article first characterizes the concept of CrowdHMT. We further investigate its challenges and techniques, and present its main application areas. Finally, we make discussions about the open issues and future research directions of CrowdHMT.

Crowd-inspired intelligent manufacturing space (CrowdIM) aims to leverage the aggregated power of heterogeneous human–machine–things (HMT) agents for improving the efficiency of intelligent manufacturing. A significant scientific problem in CrowdIM is how to improve individual skills and crowd intelligence through cooperation, complementation, competition, and confrontation among HMT agents. The emergence mechanism of biological crowd intelligence provides an inspiration to address this challenge. This article explores the mapping mechanisms between natural crowd intelligence and CrowdIM, from the aspects, such as collective dynamics, self-adaptive mechanism, crowd intelligence optimization, graph structure mapping model, evolutionary game dynamics, multiagent learning, and so on. We further propose a general model of CrowdIM and expound it through a typical case study.

The proliferation of vehicle-to-vehicle (V2V) communication techniques has resulted in collaborative vehicle localization (CVL) approaches that localize a target vehicle by leveraging the state information of nearby vehicles. However, CVL approaches typically require a large search space to locate the real position of a target vehicle and assume small measurement errors of nearby vehicle information, which limit the efficiency and robustness of the existing methods. In this article, we propose an efficient and error-tolerant CVL approach (referred to asE2T-CVL) that increases localization efficiency and accuracy even in the case of large measurement errors of nearby vehicle information. Unlike the existing CVL approaches, our approach prunes the search space for the position of the target vehicle through a pruning-based strategy that considers the relative positions of nearby vehicles. To determine the position of the target vehicle from the search space, we propose a displacement-based selection method to reduce the influence of the measurement errors of nearby vehicle information. The localization accuracy and efficiency of the proposed approach are then evaluated using simulated global positioning system trajectories in a large road network in New York City. The experimental results show that the proposed approach achieves higher localization efficiency and greater accuracy even with large measurement errors compared to state-of-the-art CVL approaches.

Identity recognition plays an important role in ensuring security in our daily life. Biometric-based (especially activity-based) approaches are favored due to their fidelity, universality, and resilience. However, most existing machine learning-based approaches rely on a traditional workflow where models are usually trained once for all, with limited involvement from end users in the process and neglecting the dynamic nature of the learning process. This makes the models static and cannot be updated in time, which usually leads to high false positive or false negative. Thus, in practice, an expert is desired to assist with providing high-quality observations and interpretation of model outputs. It is expedient to combine both advantages of human experts and the computational capability of computers to create a tight-coupling incremental learning process for better performance. In this study, we develop RLTIR, an interactive identity recognition approach based on reinforcement learning, to adjust the identification model by human guidance. We first build a base tree-structured identity recognition model, and an expert is introduced in the model for giving feedback upon model outputs. Then, the model is updated according to strategies that are automatically learned under a designated reinforcement learning framework. To the best of our knowledge, it is the very first attempt to combine human expert knowledge with model learning in the area of identity recognition. The experimental results show that the reinforced interactive identity recognition framework outperforms baseline methods with regard to recognition accuracy and robustness.

Artificial Intelligence of Things (AIoT) has recently accepted significant interests. Remarkably, embedded artificial intelligence (e.g., deep learning) on-device transforms IoT devices into intelligent systems that robustly and privately process data. Quantization technique is widely used to compress deep models for narrowing the resource gap between computation demands and platform supply. However, existing quantization schemes induce unsatisfaction for IoT scenarios since they are oblivious to dynamic changes of application context (e.g., battery and hierarchical memory availability) during the long-term operation. Subsequently, they will mismatch the user-desired resource efficiency and application lifetime. Also, to adapt to the dynamic context, we can neither accept the latency for model retraining with existing hand-crafted quantization nor the overhead for quantization bit width researching with prior on-demand quantization. This article presents a context-aware and self-adaptive deep model quantization (CAQ) system for IoT application scenarios. CAQ integrates a novel switchable multigate quantization framework, optimizing the quantized model accuracy and energy efficiency in diverse contexts. Based on the learned model, CAQ can switch among different gating networks in a context-aware manner and then adopt it to automatically capture the representation importance of various layers for optimal quantization bit-width selection. The experimental results show that CAQ achieves up to 50% storage savings with even 2.61% higher accuracy than the state-of-the-art baselines.

Ubiquitous Internet of Things (IoT) sensors in the smart city generate various urban utility sequential data, such as electricity and water usage records, which are defined as multivariate utility time series (MUTS). Due to the complex behavior of human beings, MUTS contains more complicated relationships, which go beyond general multivariate time series (TS). Specifically, multifaceted temporal couplings exist in MUTS, including intra-/inter-TS, short-to-long term, evolving, and polarized (positive/negative) relationships. Existing multisequence predictors including the latest deep-learning methods either weaken short-to-long term representation or omit evolving and polarized couplings. This work focuses on MUTS sensory data representation and prediction and proposes a novel approach—CoupledMUTS for multifaceted temporal coupling relational learning. MUTS representation module generates multidimensional representations to reveal short-to-long temporal couplings in MUTS. Gated coupling units (GCUs) module learns evolving couplings by filtering weak positive/negative relations. And dual-stage fusion module integrates multifaceted temporal couplings in both intra-TS and inter-TS for prediction. Extensive experiments on two real-world utility data sets demonstrate that our method outperforms existing shallow and deep models in utility demand prediction.

With the growth of Internet of Things (IoT) devices, smart travel methods, such as sharing-bike and ride-hailing become popular commuting methods. With people’s growing needs and the rapid dynamics in a city environment, simply using a single travel demand for prediction may be insufficient. Alternatively, modeling multiple travel demands simultaneously can deepen our understanding toward the status of these potentially correlated demands and deploy the transportation in the city better. An important observation in this work is that multiple travel demands in a city often show common patterns, referred to as the shared mobility-aware knowledge. In addition, there are also unique patterns that characterize individual travel demand resulting in unique knowledge. To better leverage the shared and unique knowledge, we propose a novel framework (MultiST) to predict multiple spatial–temporal sequences (multiple travel demands) via two components that extract the shared and unique spatial–temporal dependencies, respectively. For the unique component, we use convolutional neural networks and gated recurrent units to embed unique knowledge. For the shared component, we design a recurrent Gaussian cell to extract temporal dependencies. Empirical results show that MultiST outperforms six state-of-the-art baseline methods and three variants of MultiST. We further visualize the temporal dependencies of the shared knowledge and discuss the practical implications.

Task allocation is a key issue in mobile crowdsensing (MCS), which affects the sensing efficiency and quality. Previous studies focus on the allocation of tasks that have already been published to the platform, but there are some very urgent tasks that need to be executed once they were detected. Existing studies for either delay-tolerant or time-sensitive tasks have a certain time delay from task publishing to execution, so it is impossible to achieve task detection then execution seamlessly. Thus, we first define the instant sensing and then instant actuation (ISIA) problem in MCS and propose a new model to solve it. We aim to allocate POIs where ISIA tasks are most likely to be detected to workers with similar sensing types so that these tasks can be executed once they are detected. This article presents a two-phase task allocation framework called ISIATasker. In the sensing locations clustering and sensor selection phase, we cluster independent sensing locations into several POIs and then select the optimal cooperative sensor set for each POI to assist workers in completing sensing. In the POIs allocation phase, we propose a method called PA-DDQN based on deep reinforcement learning to plan an optimal path for each worker, thus maximizing the overall sensing type matching degree and POI coverage to enable ISIA. Finally, extensive experiments are conducted based on real-world data sets to demonstrate that the matching degree and POI coverage of ISIATasker outperform other baselines.

Most of current urban logistic systems fail to strike a nice trade-off between speed and cost. An express logistic service often implies a high delivery cost. Crowdsourced logistics is a promising solution to alleviating such contradiction. In this article, we propose a new form of crowdsourced logistics that organizes passengers and packages in a shared room, i.e., using taxis that are already transporting passengers as package hitchhikers to achieve on-time deliveries. It is well-recognized that taxi drivers are good at delivering passengers to their destinations efficiently. As a result, the proposed new urban logistics system has potentials to lower the cost and accelerate package deliveries simultaneously. Specifically, we propose a probabilistic framework containing two phases called CrowdExpress for the on-time package express service. In the first phase, we mine the historical taxi GPS trajectory dataofflineto build the package transport network. In the second phase, we develop anonlinetaxi scheduling algorithm to adaptively discover the path with the maximum arriving-on-time probability “on-the-fly” upon real-time passenger-sending requests, and direct the package routing accordingly. Finally, we evaluate the system using the real-world taxi data generated by over 19,000 taxis in a month in the city of New York, US. Results show that around 9,500 packages can be successfully delivered daily on time with the success rate over 94 percent.

In recent years, various intelligent activity recognition systems have been developed based on radio frequency signals such as radar, Wi-Fi, and radio frequency identification (RFID). When only one target is present, these systems can often provide high accuracy in recognizing different activities. However, such activity identification systems often fail to work due to signal interference when multiple targets coexist. To address this problem, we propose a multitarget gesture recognition system, named SoDar, based on a commercial single-input multi-output (SIMO) dual-channel Doppler radar. First, we employ endpoint detection, low-pass filtering, and discrete wavelet transform for data preprocessing. Then, we design a multitarget signal separation algorithm by maximizing the signal-to-noise ratio, and further refine the obtained signal based on principle component analysis. Afterward, we put forward a two-stage feature extraction method to extract both static and dynamic features from each separated signal. Finally, a classification model is trained to recognize the gestures of multiple targets. To verify the performance of SoDar, we selected nine different combinations of six gestures for two targets and collected more than 8000 data samples. Experimental results showed that the accuracy of two-target gesture recognition is above 90%.

The blooming of fake news on social networks has devastating impacts on society, the economy, and public security. Although numerous studies are conducted for the automatic detection of fake news, the majority tend to utilize deep neural networks to learn event-specific features for superior detection performance on specific datasets. However, the trained models heavily rely on the training datasets and are infeasible to apply to upcoming events due to the discrepancy between event distributions. Inspired by domain adaptation theories, we propose an end-to-end adversarial adaptation network, dubbed as MetaDetector , to transfer meta knowledge (event-shared features) between different events. Specifically, MetaDetector pushes the feature extractor and event discriminator to eliminate event-specific features and preserve required meta knowledge by adversarial training. Furthermore, the pseudo-event discriminator is utilized to evaluate the importance of news records in historical events to obtain partial knowledge that are discriminative for detecting fake news. Under the coordinated optimization among all the submodules, MetaDetector accurately transfers the meta knowledge of historical events to the upcoming event for fact checking. We conduct extensive experiments on two real-world datasets collected from Sina Weibo and Twitter. The experimental results demonstrate that MetaDetector outperforms the state-of-the-art methods, especially when the distribution discrepancy between events is significant.

Recently,fake newshas been readily spread by massive amounts of users in social media, and automatic fake news detection has become necessary. The existing works need to prepare the overall data to perform detection, losing important information about the dynamic evolution of crowd opinions, and usually neglect the issue of uneven arrival of data in the real world. To address these issues, in this article, we focus on a kind of approach for fake news detection, namelyprogressive detection, which can be achieved by thedynamic Probabilistic Graphical Model. Based on the observation on real-world datasets, we adaptively improve the Kalman Filter to theLabeled Variable Dimension Kalman Filter(LVDKF) that learns two universal patterns from true and fake news, respectively, which can capture the temporal information of time-series data that arrive unevenly. It can take sequential data as input, distill the dynamic evolution knowledge regarding a post, and utilize crowd wisdom from users’ responses to achieve progressive detection. Then we derive the formulas using the Forward, Backward, and EM Algorithm, and we design a dynamic detection algorithm using Bayes’ theorem. Finally, we design experimental scenarios simulating progressive detection and evaluate LVDKF on two public datasets. It outperforms the baseline methods in these experimental scenarios, which indicates that it is adequate for progressive detection.

The prediction of express delivery sequence, i.e., modeling and estimating the volumes of daily incoming and outgoing parcels for delivery, is critical for online business, logistics, and positive customer experience, and specifically for resource allocation optimization and promotional activity arrangement. A precise estimate of consumer delivery requests has to involve sequential factors such as shopping behaviors, weather conditions, events, business campaigns, and their couplings. Despite that various methods have integrated external features to enhance the effects, extant works fail to address complex feature-sequence couplings in the following aspects: weaken the inter-dependencies when processing heterogeneous data and ignore the cumulative and evolving situation of coupling relationships. To address these issues, we propose DeepExpress—a deep-learning-based express delivery sequence prediction model, which extends the classic seq2seq framework to learn feature-sequence couplings. DeepExpress leverages an express delivery seq2seq learning, a carefully designed heterogeneous feature representation, and a novel joint training attention mechanism to adaptively handle heterogeneity issues and capture feature-sequence couplings for accurate prediction. Experimental results on real-world data demonstrate that the proposed method outperforms both shallow and deep baseline models.

In this article, we propose and study a novel data-driven framework for Targeted Outdoor Advertising Recommendation (TOAR) with a special consideration of user profiles and advertisement topics. Given an advertisement query and a set of outdoor billboards with different spatial locations and rental prices, our goal is to find a subset of billboards, such that the total targeted influence is maximum under a limited budget constraint. To achieve this goal, we are facing two challenges: (1) it is difficult to estimate targeted advertising influence in physical world; (2) due to NP hardness, many common search techniques fail to provide a satisfied solution with an acceptable time, especially for large-scale problem settings. Taking into account the exposure strength, advertisement matching degree, and advertising repetition effect, we first build a targeted influence model that can characterize that the advertising influence spreads along with users mobility. Subsequently, based on a divide-and-conquer strategy, we develop two effective approaches, i.e., a master–slave-based sequential optimization method, TOAR-MSS, and a cooperative co-evolution-based optimization method, TOAR-CC, to solve our studied problem. Extensive experiments on two real-world datasets clearly validate the effectiveness and efficiency of our proposed approaches.

For academic research and business intelligence, trajectory data has been widely collected and analyzed. Releasing trajectory data to a third party may lead to serious privacy leakage, which has spawned considerable researches on trajectory privacy protection technology. However, existing work suffers from several shortcomings. They either focus on point-based location privacy, ignoring the spatio-temporal correlations among locations within a trajectory, or they protect the privacy of each user separately without considering privacy leakage of the social relationship between trajectories of different users. Besides, they fail to balance privacy protection and data utility. Motivated by these limitations, in this article, we propose S 3 T -Trajectory, which is a utility-aware and privacy-preserving trajectory synthesis model that Resists social relationship privacy attacks. Specifically, we first develop a time-dependent Markov chain based on an adaptive spatio-temporal discrete grid to efficiently and accurately capture human mobility behavior. Then, we propose three mobility feature metrics from spatio-temporal, semantic, and social dimensions. On the basis of the metrics, we construct a bi-level optimization problem to accomplish the utility-aware and privacy-preserving trajectory synthesizing. The upper-level objective guarantees data utility and the lower-level optimization problems (or upper-level constraints) provides two-layer privacy protection for S 3 T -Trajectory, i.e., resisting location inference attacks and social relationship privacy attacks. We conduct extensive experiments on large-scale real-world datasets loc-Gowalla and loc-Brightkite. The experimental results demonstrate the effectiveness and robustness of S 3 T Trajectory. Compared with the baseline models, S 3 T Trajectory achieves between 7.8% and 23.8% performance improvement in resisting social relationship privacy attacks and achieves at least 5.19% improvement regarding data utility.

For safety planning in crowd evacuation, it is important to predict the evacuation decisions made by different individuals and understand the reasons behind these decisions. To this end, this paper proposes an automated approach that can learn prioritized fuzzy decision rules from crowd data to predict and understand the evacuation decisions of a real human. A coevolutionary fuzzy rule miner based on genetic fuzzy-system is designed to select necessary decision features from available ones and learn both rule structure and associated rule parameters from training data. The learned fuzzy rule contains multiple sub-rules, each of which can represent evacuation strategies of different individuals in a given scenario and the features in the fuzzy condition of the sub-rule are organized and evaluated in a sequential order to reflect the priorities of different features. Based on training and testing on four evacuations scenarios of two real-world datasets, it is shown that our proposed approach can learn decision rules that are competitive to the existing evacuation decision models in terms of prediction accuracy. More importantly, it is also demonstrated that our learned rules complying with the proposed prioritized fuzzy rule representation can facilitate the interpretation of evacuation behaviors, such as “herding under zero visibility of exit” and “diminished importance on the distance to exit”, which are aligned to the field observations from real crowd evacuation.

With the rise of crowdsourcing and mobile crowdsensing techniques, a large number of crowdsourcing applications or platforms ($\mathbb {CAP}$) have appeared. In the mean time,$\mathbb {CAP}$-related models and frameworks based on different research hypotheses are rapidly emerging, and they usually address specific issues from a certain perspective. Due to different settings and conditions, different models are not compatible with each other. However,$\mathbb {CAP}$urgently needs to combine these techniques to form a unified framework. In addition, these models needs to be learned and updated online with the extension of crowdsourced data and task types; thus, requiring a unified architecture that integrates lifelong learning concepts and breaks down the barriers between different modules. This paper draws on the idea of ubiquitous operating systems and proposes a novel OS (CrowdOS), which is an abstract software layer running between native OS and application layer. In particular, based on an in-depth analysis of the complex crowd environment and diverse characteristics of heterogeneous tasks, we construct the OS kernel and three core frameworks including task resolution and assignment framework (TRAF), integrated resource management (IRM), and task result quality optimization (TRO). In addition, we validate the usability of CrowdOS, module correctness and development efficiency. Our evaluation further revealsTRObrings enormous improvement in efficiency and a reduction in energy consumption.

Indoor localization serves as the basis of sensing and understanding human behaviors and further providing personalized services in many scenarios, such as retail stores, warehouses and libraries. However, existing indoor localization technologies cannot fulfill the requirement of such scenarios due to incapable of identifying different persons, severe object occlusion when there are multiple persons, or privacy concerns. On the basis of wide deployment of RFID tags in such scenarios, in this paper we develop a RFID-based localization system, i.e., ShopSense, which is not only able to accurately localize multiple people simultaneously but also differentiate them even when there are a lot of obstacles in the environment. Extensive experiments demonstrate that ShopSense can locate the shopping cart at a median tracking error of 20 cm and can locate the customer’s location with a median tracking error of 25 cm.

In this paper, a general location selection strategy is proposed based on active learning, which involves iterations of a selector and an estimator. We implement four instances of this general strategy to embody it: KAL (Active Learning based on Kriging), TAL (Active Learning based on Regression Tree), KMAL (Active Learning based on Kriging and MPGR) and TMAL (Active Learning based on Regression Tree and MPGR). The estimator of KAL or TAL can estimate the air quality at remaining locations from air quality samples at monitoring locations leveraging spatial or cross-domain correlation of air quality. The selecting indicators of their selectors are designed to measure the uncertainty of unlabeled samples according to their estimators. KMAL and TMAL are upgrades of the former two respectively, by introducing MPGR (Manifold Preserving Graph Reduction) to also take the representativeness of unlabeled samples into account. The experimental results show that the proposed strategy can achieve low estimation error with few monitoring locations. Particularly, given the same budget (i.e., the number of monitoring locations), the estimation error is reduced from about 20% of baselines to 15% by KAL and to 5% by KMAL, and TAML likewise.

App survival prediction is a significant task in mobile service development. It differs from existing prediction tasks in two aspects. First, rather than the traditional survival prediction in bioinformatics where all the patients’ survival probabilities decay in a similar way, apps’ survival pattern varies from each other. Second, affected by multiple factors, an app's popularity is time-varying and sequence-dependent, which makes existing short-term prediction methods not applicable due to error accumulation. These characteristics bring great difficulties in app survival prediction. In this paper, we propose AppLife, a framework that fuses multi-source influence factors and utilizes Multi-Task Learning (MTL) to combine the state information of mobile app for survival prediction. First, we analyze how the app survival is affected by multi-source factors, including download history, ratings, and reviews. Second, to overcome error accumulation in long-term prediction, we propose a novel MTL based approach. The approach estimates whether an app is surviving at each time interval during the life cycle of apps and leverages relatedness among tasks to improve the prediction performance. Last, we collect a large-scale dataset with more than 35,000 apps, based on which we evaluate our proposed framework and results show that it outperforms the seven state-of-the-art methods.

App popularity prediction is a significant task in mobile service development, which predicts an app's future popularity based on its current behaviors. It provides benefits from app development to targeted investment. Popularity is affected by two factors, i.e., internal ones like reviews and external ones like interaction among apps. However, most related studies only explore internal factors but neglect external ones. In fact, external factor plays an important role in popularity prediction modelling since it is the promoting and/or inhibiting influence resulted by app interaction. The app interaction has two major characteristics, i.e., interactivity and dynamicity, which brings challenges to app popularity prediction due to two reasons: 1) interactivity—it is hard to evaluate the existence and influence intensity of interactions; 2) dynamicity—the nature of interaction influence, e.g., promoting or inhibiting, and its intensity on popularity change with time. In this paper, we propose DeePOP, a popularity prediction model that innovatively leverages time-varying hierarchical interactions. First, we propose Hierarchical Interaction Graph, which is first studied in this work, to organically characterize the relationship and influence among apps. Second, DeePOP integrates internal factors and time-varying hierarchical interactions as inputs to build the prediction model. It develops multi-level modules based on Recurrent Neural Network with attention mechanism and generates multi-step time series predictions by fusing the outputs of modules. Experiments on a real-world dataset show that DeePOP outperforms state-of-the-art methods in prediction accuracy, effectively reducing the Root Mean Square Error (RMSE) to 0.088.

The cross-domain recommender systems aim to alleviate the data sparsity problem in a target domain by transferring knowledge from a source domain. However, existing works ignore the latent information underlying the user-item interactions. In addition, they don’t explicitly model the intra-domain and cross-domain interactions. To address these concerns, we propose a novel model named cross-graph knowledge transfer network to improve the recommendation performance. To explicitly model intra-domain and cross-domain interactions, we utilize the graph structure to transfer knowledge across domains. Firstly, we design a neighbor sampling method to extract useful intra-domain and cross-domain interactions. Secondly, we aggregate multiple interactive information in each domain and generate intra-domain embeddings by using intra-domain attention mechanism. Thirdly, we fuse the information from two domains to generate effective user and item embeddings by using cross-domain attention mechanism. Finally, we feed user and item embeddings into the domain-specific prediction layers for personalized recommendation. We conduct experiments on real-world datasets. The results show that our model outperforms five state-of-art methods.

Deep neural networks (DNNs) play an important role in a variety of intelligent applications (e.g. image classification and target recognition), yet at the cost of heavy computation burden, that makes DNNs difficult to deploy on resource-constrained IoT devices. To solve this problem, there are two categories of model computation adjustment methods: model compression and model segmentation. However, model compression mainly reduces resource consumption at the cost of accuracy while model segmentation reduces resource consumption according to the cost of communication latency. In this paper, we propose Joint Search for Model Compression and Segmentation (JointCS) that highlights the following aspects: 1) we integrate both model compression and model segmentation under an automatic and progressive framework, it simplifies model to fit the different IoT resource requirements. JointCS achieves a series slim models that outperform better both in accuracy and latency. 2) we train a network architecture-aware latency predictor to fast measure the latency of the slimed model on heterogeneous IoT devices. 3) we introduce a search algorithm to select the optimal state in progressively joint search. Finally, we evaluate the performance of our proposed method for image classification on CIFAR datasets comparing with the state-of-the-art approach, the inference time of the proposed method has inference speedup of 12.2 % −30.9 % under the same accuracy.

Keystroke dynamics-based authentication (KDA) is one of human behavioral-based authentication methods based on the unique typing rhythm of an individual. Nevertheless, the typing characteristics gradually change over time. Various solutions have been suggested to remedy the concept drift problem, including multimodal and unimodal adaptive methods. However, these solutions don't consider that temporal concept drift has a negative impact on performance and update frequency increases computation cost. The paper proposes weighted EDDM to detect concept drift and capture permanent concept drift (behavioral natural aging). Experimental results show that our method can accurately capture behavioral natural aging and filter temporal concept drift. Our proposed method has better performance and less computation.

Automated Guided Vehicles (AGVs) have been widely used for material handling in flexible shop floors. Each product requires various raw materials to complete the assembly in production process. AGVs are used to realize the automatic handling of raw materials in different locations. Efficient AGVs task allocation strategy can reduce transportation costs and improve distribution efficiency. However, the traditional centralized approaches make high demands on the control center’s computing power and real-time capability. In this paper, we present decentralized solutions to achieve flexible and self-organized AGVs task allocation. In particular, we propose two improved multi-agent reinforcement learning algorithms, MAD-DPG-IPF (Information Potential Field) and BiCNet-IPF, to realize the coordination among AGVs adapting to different scenarios. To address the reward-sparsity issue, we propose a reward shaping strategy based on information potential field, which provides stepwise rewards and implicitly guides the AGVs to different material targets. We conduct experiments under different settings (3 AGVs and 6 AGVs), and the experiment results indicate that, compared with baseline methods, our work obtains up to 47% task response improvement and 22% training iterations reduction.

Thanks to the growing popularity and functionality, smartphone has become rapidly valuable potential tool for human behavior research, e.g., friendship relationship recognition, friend ship prediction, etc. Until recently, there have been many research efforts to study this issue using the sensed data collected from smartphones. However, almost previous works in finding friendship strength are based on several physical features or a few dimensions, such as using Bluetooth scanning and demographic data to explain friendship. Actually, friendship is complicated and coupled with many factors, such as physical propinquity, social, physical and psychological homophily. So, it is necessary and beneficial to examine it comprehensively, by taking into account all the involved factors. Aiming at closing part of this research gap, in this paper, from cross-space perspectives, we launch a friendship relationship study with smartphone-based sensing paradigm from cyber space, physical mobility, and personality trait homophily. By integrating the involved heterogeneous interactions, we propose a Deep AutoEncoder-based unified framework to predict the strength of friendship connections between users, where the friendship strength is categorized and asymmetrical. We conduct extensive experiments on a practically collected sensing data set, and show the efficiency and effectiveness of our proposed approaches.

With increasing expansion and urbanization of cities, the gap is constantly widening between the current provision of urban public toilets and the fast-growing toileting demand. Building new ones becomes a promising way to alleviate such issue. Nevertheless, where to build them in a city is challenging. Different from other location selection (e.g., commercial sites) problems, the selection of public toilet locations is hard to quantify and evaluate. On one hand, the toileting demand that determines whether the new public toilet is needed cannot be measured accurately. On the other hand, the modeling of the toileting demand is also complicated, being influenced by multiple factors, e.g., human mobility, human activity, and geographical characteristics. In this article, we propose a novel data-driven framework named ToiletBuilder to address it, which consists of three components, i.e., region identification, region representation, and region classification. Specifically, region identification obtains many reachable regions with the reasonable size. Region representation extracts city-specific features from multiple urban data to characterize location selection influencing factors for each region. A deep embedding model is further applied to learn a high-order and concise semantic representation. By labeling some regions with the true positive label (i.e., having public toilets served in these regions) in advance, region classification trains a positive-unlabeled (PU) learning model from these samples to identify unlabeled positive ones. Finally, we conduct extensive experiments based on four real-world data sets including road network, river network, taxi trajectory, and POI data, in the city of Chongqing, China. Results demonstrate the effectiveness of our proposed approach.

Sparse mobile crowdsensing (MCS) requires participants to collect data from partial cells and then intelligently infer the data of the rest cells. Since collecting data from different cells will probably result in different data inference quality, cell selection (i.e., which cells need to be selected to collect data) is a critical issue in Sparse MCS. Currently, state-of-the-art cell selection algorithms are implemented based on reinforcement learning. These algorithms ignore the problem that the urban environment is usually time dependent, and the cell selection model needs to be kept up-to-date to adapt to the time-dependent environment. However, Sparse MCS applications require participants to collect data only in a few cells, which makes it hard to obtain suitable training data for continuous cell selection model learning. To solve this problem, we model the spatiotemporal correlations in the collected sparse data, and then design various methods to update training data based on it. Particularly, these methods make full use of the gradual changes of data in time and space, and reasonably transform and splice sparse data at different moments. Finally, updated training data is fed to the cell selection model to keep it up-to-date. We conduct experimental evaluations by performing several sensing tasks in air quality monitoring. The results show that our proposed methods can effectively update training data as well as the cell selection model. Compared with several baselines, our best method can reduce inference error by more than 10% on average.

With the rapid development of smart devices and high-quality wireless technologies, mobile crowdsourcing (MCS) has been drawing increasing attention with its great potential in collaboratively completing complicated tasks on a large scale. A key issue toward successful MCS is participant recruitment, where a MCS platform directly recruits suitable crowd participants to execute outsourced tasks by physically traveling to specified locations. Recently, a novel recruitment strategy, namely Word-of-Mouth(WoM)-based MCS, has emerged to effectively improve recruitment effectiveness, by fully exploring users' mobility traces and social relationships on geo-social networks. Against this background, we study in this paper a novel problem, namely Expected Task Execution Quality Maximization (ETEQM) for MCS in geo-social networks, which strives to search a subset of seed users to maximize the expected task execution quality of all recruited participants, under a given incentive budget. To characterize the MCS task propagation process over geo-social networks, we first adopt a propagation tree structure to model the autonomous recruitment between the referrers and the referrals. Based on the model, we then formalize the task execution quality and devise a novel incentive mechanism by harnessing the business strategy of multi-level marketing. We formulate our ETEQM problem as a combinatorial optimization problem, and analyze its NP hardness and high-dimensional characteristics. Based on a cooperative co-evolution framework, we proposed a divide-and-conquer problem-solving approach named ETEQM-CC. We conduct extensive simulation experiments and a case study, verifying the effectiveness of our proposed approach.

Gesture recognition is key to enabling natural human-computer interactions. Existing approaches based on wireless sensing focus on accurate identification of arm gesture types. It remains a challenge to recognize and profile the details of arm gestures for precise interaction control. In addition, current approaches have strict positioning requirements between radars and users, making them difficult for real-world deployment. In this article, we adopt the multisensor approach and present gesture-radar-a dual Doppler radar-based gesture recognition and profiling system, which can capture subtle arm gestures with less positioning or environmental dependence. Gesture-radar uses two vertically placed Doppler radars to collect complementary sensing data of gestures, based on which cross-analysis can be performed for gesture recognition and profiling. Specifically, we first propose a two-stage classification model and enhance the signal proximity matching method by applying constraint functions to the DTW algorithm, aiming to improve the accuracy of gesture type recognition. Afterward, we establish and analyze unique features from the time-frequency spectrogram, which can be used to characterize in-depth gesture details, e.g., the angle or range of an arm movement. Experimental results show that gesture-radar achieves up to 93.5% average accuracy for gesture type recognition, and over 80% precision for profiling gesture details. This proves that the proposed approach is viable and can work in real-world environments.

In recent years, with the development of deep learning, text-generation technology has undergone great changes and provided many kinds of services for human beings, such as restaurant reservation and daily communication. The automatically generated text is becoming more and more fluent so researchers begin to consider more anthropomorphic text-generation technology, that is, the conditional text generation, including emotional text generation, personalized text generation, and so on. Conditional Text Generation (CTG) has thus become a research hotspot. As a promising research field, we find that much attention has been paid to exploring it. Therefore, we aim to give a comprehensive review of the new research trends of CTG. We first summarize several key techniques and illustrate the technical evolution route in the field of neural text generation, based on the concept model of CTG. We further make an investigation of existing CTG fields and propose several general learning models for CTG. Finally, we discuss the open issues and promising research directions of CTG.

Optimal store placement aims to identify the optimal location for a new brick-and-mortar store that can maximize its sale by analyzing and mining users’ preferences from large-scale urban data. In recent years, the expansion of chain enterprises in new cities brings some challenges because of two aspects: (1) data scarcity in new cities, so most existing models tend to not work (i.e., overfitting), because the superior performance of these works is conditioned on large-scale training samples; (2) data distribution discrepancy among different cities, so knowledge learned from other cities cannot be utilized directly in new cities. In this article, we propose a task-adaptative model-agnostic meta-learning framework, namely, MetaStore, to tackle these two challenges and improve the prediction performance in new cities with insufficient data for optimal store placement, by transferring prior knowledge learned from multiple data-rich cities. Specifically, we develop a task-adaptative meta-learning algorithm to learn city-specific prior initializations from multiple cities, which is capable of handling the multimodal data distribution and accelerating the adaptation in new cities compared to other methods. In addition, we design an effective learning strategy for MetaStore to promote faster convergence and optimization by sampling high-quality data for each training batch in view of noisy data in practical applications. The extensive experimental results demonstrate that our proposed method leads to state-of-the-art performance compared with various baselines.

In e-commerce platforms, the online descriptive information of products shows significant impacts on the purchase behaviors. To attract potential buyers for product promotion, numerous workers are employed to write the impressive product descriptions. The hand-crafted product descriptions are less-efficient with great labor costs and huge time consumption. Meanwhile, the generated product descriptions do not take consideration into the customization and the diversity to meet users’ interests. To address these problems, we propose one generic framework, namely DeepDepict, to automatically generate the information-rich and personalized product descriptive information. Specifically, DeepDepict leverages the graph attention to retrieve the product-related knowledge from external knowledge base to enrich the diversity of products, constructs the personalized lexicon to capture the linguistic traits of individuals for the personalization of product descriptions, and utilizes multiple pointer-generator network to fuse heterogeneous data from multi-sources to generate informative and personalized product descriptions. We conduct intensive experiments on one public dataset. The experimental results show that DeepDepict outperforms existing solutions in terms of description diversity, BLEU, and personalized degree with significant margin gain, and is able to generate product descriptions with comprehensive knowledge and personalized linguistic traits.

Store site recommendation aims to predict the value of the store at candidate locations and then recommend the optimal location to the company for placing a new brick-and-mortar store. Most existing studies focus on learning machine learning or deep learning models based on large-scale training data of existing chain stores in the same city. However, the expansion of chain enterprises in new cities suffers from data scarcity issues, and these models do not work in the new city where no chain store has been placed (i.e., cold-start problem). In this article, we propose a unified approach for cold-start store site recommendation, Weighted Adversarial Network with Transferability weighting scheme (WANT), to transfer knowledge learned from a data-rich source city to a target city with no labeled data. In particular, to promote positive transfer, we develop a discriminator to diminish distribution discrepancy between source city and target city with different data distributions, which plays the minimax game with the feature extractor to learn transferable representations across cities by adversarial learning. In addition, to further reduce the risk of negative transfer, we design a transferability weighting scheme to quantify the transferability of examples in source city and reweight the contribution of relevant source examples to transfer useful knowledge. We validate WANT using a real-world dataset, and experimental results demonstrate the effectiveness of our proposed model over several state-of-the-art baseline models.

With the rapid development of mobile app ecosystem, mobile apps have grown greatly popular. The explosive growth of apps makes it difficult for users to find apps that meet their interests. Therefore, it is necessary to recommend user with a personalized set of apps. However, one of the challenges is data sparsity, as users’ historical behavior data are usually insufficient. In fact, user’s behaviors from different domains in app store regarding the same apps are usually relevant. Therefore, we can alleviate the sparsity using complementary information from correlated domains. It is intuitive to model users’ behaviors using graph, and graph neural networks have shown the great power for representation learning. In this article, we propose a novel model, Deep Multi-Graph Embedding (DMGE), to learn cross-domain app embedding. Specifically, we first construct a multi-graph based on users’ behaviors from different domains, and then propose a multi-graph neural network to learn cross-domain app embedding. Particularly, we present an adaptive method to balance the weight of each domain and efficiently train the model. Finally, we achieve cross-domain app recommendation based on the learned app embedding. Extensive experiments on real-world datasets show that DMGE outperforms other state-of-art embedding methods.

In modern online advertising systems, the click-through rate (CTR) is an important index to measure the popularity of an item. It refers to the ratio of users who click on a specific advertisement to the number of total users who view it. Predicting the CTR of an item in advance can improve the accuracy of the advertisement recommendation. And it is commonly calculated based on users’ interests. Thus, extracting users’ interests is of great importance in CTR prediction tasks. In the literature, a lot of studies treat the interaction between users and items as sequential data and apply the recurrent neural network (RNN) model to extract users’ interests. However, these solutions cannot handle the case when the sequence length is relatively long, e.g., over 100. This is because of the vanishing gradient problem of RNN, i.e., the model cannot learn a users’ previous behaviors that are too far away from the current moment. To address this problem, we propose a new Core Interest Network (CIN) model to mitigate the problem of a long sequence in the CTR prediction task with sequential data. In brief, we first extract the core interests of users and then use the refined data as the input of subsequent learning tasks. Extensive evaluations on real dataset show that our CIN model can outperform the state-of-the-art solutions in terms of prediction accuracy.

With the proliferation of Internet of Things (IoT) devices in the consumer market, the unprecedented sensing capability of IoT devices makes it possible to develop advanced sensing and complex inference tasks by leveraging heterogeneous sensors embedded in IoT devices. However, the limited power supply and the restricted computation capability make it challenging to conduct seamless sensing and continuous inference tasks on resource-constrained devices. How to conduct energy-efficient sensing and perform rich-sensor inference tasks on IoT devices is crucial for the success of IoT applications. Therefore, we propose a novel energy-efficient collaborative sensing framework to optimize the energy consumption of IoT devices. Specifically, we explore the latent correlations among heterogeneous sensors via an attention mechanism in temporal convolutional network to quantify the dependency among sensors, and characterize the heterogeneous sensors in terms of energy consumption to categorize them into low-power sensors and energy-intensive sensors . Finally, to decrease the sampling frequency of energy-intensive sensors , we propose a multi-task learning strategy to predict the statuses of energy-intensive sensors based on the low-power sensors . To evaluate the performance of the proposed collaborative sensing framework, we develop a mobile application to collect concurrent heterogeneous data streams from all sensors embedded in Huawei Mate 8. The experimental results show that latent correlation learning is greatly helpful to understand the latent correlations among heterogeneous streams, and it is feasible to predict the statuses of energy-intensive sensors by low-power sensors with high accuracy and fast convergence. In terms of energy consumption, the proposed collaborative sensing framework is able to preserve the energy consumption of IoT devices by nearly 50% for continuous data acquisition tasks.

Recently, the modeling and representation of multivariate time series has attracted much attention in the field of machine learning and data mining, due to its wide application potentials in biomedicine, finance, industry and so on. During the last decade, deep learning has achieved great success in many tasks. However, a large number of labeled data samples are needed to train a satisfactory model which has a huge amount of parameters, especially in cases that the inputs are multivariate time series (i.e., multi-dimension) and have complex relationships with the outputs. We propose a Multi-level attention-based prototype Network (MapNet) to model multivariate time series. Specifically, we first encode the time series based on deep learning and calculate the prototype for each class. Afterwards, we propose a multi-level attention mechanism to further optimize the prototype, including a short-term encoder as well as a long-term encoder. Experiments based on two public datasets demonstrate that MapNet outperforms state-of-the-art baseline models and is more applicable for few-shot dataset.

With the recent surge of smart wearable devices, it is possible to obtain the physiological and behavioral data of human beings in a more convenient and non-invasive manner. Based on such data, researchers have developed a variety of systems or applications to recognize and understand human behaviors, including both physical activities (e.g., gestures) and mental states (e.g., emotions). Specifically, it has been proved that different emotions can cause different changes in physiological parameters. However, other factors, such as activities, may also impact one’s physiological parameters. To accurately recognize emotions, we need not only explore the physiological data but also the behavioral data. To this end, we propose an adaptive emotion recognition system by exploring a sensor-enriched wearable smart watch. First, an activity identification method is developed to distinguish different activity scenes (e.g., sitting, walking, and running) by using the accelerometer sensor. Based on the identified activity scenes, an adaptive emotion recognition method is proposed by leveraging multi-mode sensory data (including blood volume pulse, electrodermal activity, and skin temperature). Specifically, we extract fine-grained features to characterize different emotions. Finally, the adaptive user emotion recognition model is constructed and verified by experiments. An accuracy of 74.3% for 30 participants demonstrates that the proposed system can recognize human emotions effectively.

In the Internet-of-Things (IoT) era, user authentication is essential to ensure the security of connected devices and the customization of passive services. However, conventional knowledge-based and physiological biometric-based authentication systems (e.g., password, face recognition, and fingerprints) are susceptible to shoulder surfing attacks, smudge attacks, and heat attacks. The powerful sensing capabilities of IoT devices, including smartphones, wearables, robots, and autonomous vehicles enable continuous authentication (CA) based on behavioral biometrics. The artificial intelligence (AI) approaches hold significant promise in sifting through large volumes of heterogeneous biometrics data to offer unprecedented user authentication and user identification capabilities. In this survey article, we outline the nature of CA in IoT applications, highlight the key behavioral signals, and summarize the extant solutions from an AI perspective. Based on our systematic and comprehensive analysis, we discuss the challenges and promising future directions to guide the next generation of AI-based CA research.

It is still a challenge to detect anomalous events in video sequences in the field of computer vision due to heavy object occlusions, varying crowded densities and complex situations. To address this, we propose a novel human-machine cooperative approach which uses human feedback on anomaly confirmation to inform and enhance video anomaly detection. Specifically, we analyze the spatio-temporal characteristics of sequential frames of a video from the appearance and motion perspective from which spatial and temporal features are identified and extracted. We then develop a convolutional autoencoder neural network to compute an abnormal score based on reconstruction errors. In this process, a group of experts will provide human feedback to a certain proportion of classified frames to be incorporated into the model, and also the final judgment for the event anomalies for training and classification. The proposed approach is evaluated on 3 publicly available surveillance datasets, showing improved accuracy and competitive performance (93.7% AUC) with respect to the best performance (90.6% AUC) of the state-of-the-art approaches. The approach has not been previously seen to the best of our knowledge.

A Point-of-Interest (POI) refers to a specific location that people may find useful or interesting. In modern cities, a large number of POIs emerge, grow, stabilize for a period, then finally disappear. The stages (e.g., emerge and grow) in this process are called lifetime statuses of a POI. While a large body of research has been devoted to identifying and recommending POIs, there are few studies on inferring the lifetime status of POIs. Indeed, the predictive analytics of POI lifetime status can be valuable for various tasks, such as urban planning, business site selection, and real estate appraisal. In this article, we propose a multitask learning approach, named inferring POI lifetime status, to inferring the POI lifetime status with multifaceted data sources. Specifically, we first define three types of POI lifetime status, i.e., booming, decaying, and stable. Then, we formulate a serial classification problem to predict the sequential/successive lifetime statuses of POIs over time. Leveraging geographical data and human mobility data, we examine and integrate three aspects of features related to the prosperity of POIs, i.e., region popularity, region demands, and peer competitiveness. Next, as the booming/decaying POIs are relatively rare in our data, we perform stable class decomposition to alleviate the imbalance between stable POIs and booming/decaying POIs. Finally, we develop a POI lifetime status classifier by exploiting the multitask learning framework as well as the multiclass kernel-based vector machines. We perform extensive experiments using large-scale and real-world datasets of New York City. The experimental results validate the effectiveness of our approach to automatically inferring POI lifetime status.

Task allocation is a major challenge in Mobile Crowd Sensing (MCS). While previous task allocation approaches follow either the opportunistic or participatory mode, this paper proposes to integrate these two complementary modes in a two-phased hybrid framework called HyTasker. In the offline phase, a group of workers (called opportunistic workers) are selected, and they complete MCS tasks during their daily routines (i.e., opportunistic mode). In the online phase, we assign another set of workers (called participatory workers) and require them to move specifically to perform tasks that are not completed by the opportunistic workers (i.e., participatory mode). Instead of considering these two phases separately, HyTasker jointly optimizes them with a total incentive budget constraint. In particular, when selecting opportunistic workers in the offline phase of HyTasker, we propose a novel algorithm that simultaneously considers the predicted task assignment for the participatory workers, in which the density and mobility of participatory workers are taken into account. Experiments on two real-world mobility datasets demonstrate that HyTasker outperforms other methods with more completed tasks under the same budget constraint.

Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.

Crime prediction has always been a crucial issue for public safety, and recent works have shown the effectiveness of taking spatial correlation, such as region similarity or interaction, for fine-grained crime modeling. In our work, we seek to reveal the relationship across regions for crime prediction using Continuous Conditional Random Field (CCRF). However, conventional CCRF would become impractical when facing a dense graph considering all relationship between regions. To deal with it, in this paper, we propose a Neural Network based CCRF (NN-CCRF) model that formulates CCRF into an end-to-end neural network framework, which could reduce the complexity in model training and improve the overall performance. We integrate CCRF with NN by introducing a Long Short-Term Memory (LSTM) component to learn the non-linear mapping from inputs to outputs of each region, and a modified Stacked Denoising AutoEncoder (SDAE) component for pairwise interactions modeling between regions. Experiments conducted on two different real-world datasets demonstrate the superiority of our proposed model over the state-of-the-art methods.

Cyber crimes including computer virus/malwares, spam, illegal sales, and phishing websites are proliferated aggressively via the disguised Uniform Resource Locators (URL). Although numerous studies were conducted for the URL classification task, the traditional URL classification solutions retreated due to the hand-crafted feature engineering and the boom of newly generated URLs. In this paper, we study the representation learning of URLs, and explore the URL classification using deep learning. Specifically, we propose URL2vec to extract both the structural and lexical features of URLs, and apply temporal convolutional network (TCN) for the URL classification task. The experimental results show that URL2vec outperforms both word2vec and character-level embedding for URL representation, and TCN achieves the best performance than baselines with the precision up to 95.97%.

Traditionally, vehicle tracking is accomplished using predeployed video camera networks, which relies on stationary cameras and searches for the target vehicle from videos. In this paper, we develop CrowdTracking, i.e., a crowd tracking system that people can collaboratively keep track of the moving vehicle by taking photographs, especially in areas where video cameras are deficient. In other words, the underlying support of CrowdTracking is mobile crowdsensing. Several novel ideas underpin CrowdTracking. First, the vehicle can be rapidly localized by using both photographing contexts (including the location and the shooting direction) of the photographer and the road network. Second, the moving speed of the vehicle can be estimated according to two localization results and the trajectory will be predicted. As a result, through continuously collecting photographs of the moving vehicle on different roads, the vehicle can be tracked and localized almost in real time. Through precisely localizing the specified vehicle, two optimization objectives are met: 1) maximizing the tracking coverage to the vehicle's actual trajectory and 2) minimizing the number of participants who are assigned vehicle-tracking tasks. We evaluate the localization method with a real dataset and report about 6 m error. We also evaluate the vehicle-tracking method of CrowdTracking using a synthetic data set and experimental results validate its effectiveness and efficiency.

In recent years, takeout ordering and delivery (TOD) has become an emerging service due to its convenience and efficiency. However, current online ordering platforms still suffer from some issues, such as limited delivery coverage and delayed delivery. To address these issues, we propose a spatial crowdsourcing (SC)-based system called crowd delivery network (CrowDNet) to have packages take hitchhiking rides with existing taxis. We first tackle passenger riding queries based on an evolutionary algorithm and then insert appropriate food delivery requests into a partial schedule with an improved insertion approach. Finally, we propose a ranking module based on the modern portfolio theory to recommend the delivery path, which can achieve a balance between the delivery cost and timely services. Evaluations based on three real-world datasets demonstrate that our proposed algorithms outperform baseline methods.

Store site recommendation is one of the essential business services in smart cities for brick-and-mortar enterprises. In recent years, the proliferation of multisource data in cities has fostered unprecedented opportunities to the data-driven store site recommendation, which aims at leveraging large-scale user-generated data to analyze and mine users’ preferences for identifying the optimal location for a new store. However, most works in store site recommendation pay more attention to a single data source which lacks some significant data (e.g., consumption data and user profile data). In this paper, we aim to study the store site recommendation in a fine-grained manner. Specifically, we predict the consumption level of different users at the store based on multisource data, which can not only help the store placement but also benefit analyzing customer behavior in the store at different time periods. To solve this problem, we design a novel model based on the deep neural network, named DeepStore, which learns low- and high-order feature interactions explicitly and implicitly from dense and sparse features simultaneously. In particular, DeepStore incorporates three modules: 1) the cross network; 2) the deep network; and 3) the linear component. In addition, to learn the latent feature representation from multisource data, we propose two embedding methods for different types of data: 1) the filed embedding and 2) attention-based spatial embedding. Extensive experiments are conducted on a real-world dataset including store data, user data, and point-of-interest data, the results demonstrate that DeepStore outperforms the state-of-the-art models.

User profile can be used to characterize a person and help us better understand him/her, which further can be utilized to provide enhanced personalized services. When using mobile phone, some of one's information are unavoidably and unobtrusively passed or stored, which makes it possible to draw the user profile. In this paper, we propose to infer user profile, including age, gender, and personality traits based on mobile phone sensory data. Specifically, we capture data when unlocking screen, playing games as well as some basic mobile phone information, app usage, and screen status by using common available sensors in commodity mobile phones. By analyzing the differences in users' phone usage, we extracted features for user profile inference. Random Forest regression and random forest classification models are separately used to estimate age and gender of the user while support vector regression algorithm is applied to identify personality traits. In addition, we evaluate the model through real-life experiments conducted with a total of 84 phone users. Experimental results show that our approach effective, achieving an RSME of 4.3696 in age estimation and precision of 91.70% in gender detection. As for personality traits identification, the root mean square errors of openness, conscientiousness, extraversion, agreeableness, and neuroticism are 0.29, 0.3506, 0.465, 0.3022, and 0.452, respectively.

Housing demand estimation is an important topic in the field of economic research. It is beneficial and helpful for various applications including real estate market regulation and urban planning, and therefore is crucial for both real estate investors and government administrators. Meanwhile, given the rapid development of the express industry, abundant useful information is embedded in express delivery records, which is helpful for researchers in profiling urban life patterns. The express delivery behaviors of the residents in a residential community can reflect the housing demand to some extent. Although housing demand has been analyzed in previous studies, its estimation has not been very good, and the subject remains under explored. To this end, in this article, we propose a systematic housing demand estimation method based on express delivery data. First, the express delivery records are aggregated on the community scale with the use of clustering methods, and the missing values in the records are completed. Then, various features are extracted from a less sparse dataset considering both the probability of residential mobility and the attractiveness of residential communities. In addition, given that the correlations between different districts can influence the performances of the inference model, the commonalities and differences of different districts are considered. After obtaining the features and correlations between different districts being obtained, the housing demand is estimated by using a multi-task learning method based on neural networks. The experimental results for real-world data show that the proposed model is effective at estimating the housing demand at the residential community level.

Mobile crowdsensing (MCS) is a new paradigm to collect sensing data and infer useful knowledge over a vast area for numerous monitoring applications. In urban environments, as more and more applications need to utilize multi-source sensing information, it is almost indispensable to develop a generic mechanism supporting multiple concurrent MCS task assignment. However, most existing multi-task assignment methods focus on homogeneous tasks. Due to the diverse spatiotemporal task requirements and sensing contexts, MCS tasks often differ from each other in many aspects (e.g., spatial coverage, temporal interval). To this end, in the paper, we present and formalize an important Heterogeneous Multi-Task Assignment (HMTA) problem in mobile crowdsensing systems, and try to maximize data quality and minimize total incentive budget. By leveraging the implicit spatiotemporal correlations among heterogeneous tasks, we propose a two-stage HMTA problem-solving approach to effectively handle multiple concurrent tasks in a shared resource pool. Finally, in order to improve the assignment search efficiency, a decomposition-and-combination framework is devised to accommodate large-scale problem scenario. We evaluate our approach extensively using two large-scale real-world data sets. The experimental results validate the effectiveness and efficiency of our proposed approach.

This paper builds a Food Delivery Network (FooDNet in short) using spatial crowdsourcing (SC). It investigates the participation of urban taxis to support on demand take-out food delivery. Unlike existing SC-enabled service sharing systems (e.g., ridesharing), the delivery of food in FooDNet is more time-sensitive and the optimization problem is more complex regarding high-efficiency, huge-number of delivery needs. In particular, two on demand food delivery problems under different situations are studied in our work: (1) for O-OTOD, the food is opportunistically delivered by taxis when carrying passengers, and the optimization goal is to minimize the number of selected taxis to maintain a relatively high incentive to the participated drivers; (2) for D-OTOD, taxis dedicatedly deliver food without taking passengers, and the aim is to minimize the number of selected taxis (i.e., to raise the reward for each participant) and the total traveling distance to reduce the cost. A two-stage approach, including the construction algorithm and the Adaptive Large Neighborhood Search (ALNS) algorithm based on simulated annealing, is proposed to solve the problem. We have conducted extensive experiments based on the real-world datasets, including city-wide restaurant data, cell tower data, and the large-scale taxi trajectory data with 10,000 taxis in the city of Chengdu, China. Experimental results demonstrate that our proposed algorithms are more effective and efficient than baselines, fulfilling the food delivery service using a smaller number of taxis within the given time.

In recent years, bike-sharing systems have been widely deployed in many big cities, which provide an economical and healthy lifestyle. With the prevalence of bike-sharing systems, a lot of companies join the bike-sharing market, leading to increasingly fierce competition. To be competitive, bike-sharing companies and app developers need to make strategic decisions and predict the popularity of bike-sharing apps. However, existing works mostly focus on predicting the popularity of a single app, the popularity contest among different apps has not been explored yet. In this paper, we aim to forecast the popularity contest between Mobike and Ofo, two most popular bike-sharing apps in China. We develop CompetitiveBike, a system to predict the popularity contest among bike-sharing apps leveraging multi-source data. We extract two novel types of features: coarse-grained and fine-grained competitive features, and utilize Random Forest model to forecast the future competitiveness. In addition, we view mobile apps competition as a long-term event and generate the event storyline to enrich our competitive analysis. We collect data about two bike-sharing apps and two food ordering & delivery apps from 11 app stores and Sina Weibo, implement extensive experimental studies, and the results demonstrate the effectiveness and generality of our approach.

User behaviors in online social networks convey not only literal information but also one’s emotional attitudes towards the information. To compute this attitude, we define the concept of emotion role as the concentrated reflection of a user’s online emotional characteristics. Emotion role detection aims to better understand the structure and sentiments of online social networks and support further analysis, e.g., revealing public opinions, providing personalized recommendations, and detecting influential users. In this article, we first introduce the definition of a fine-grained emotion role, which consists of two dimensions: emotion orientation (i.e., positive, negative, and neutral) and emotion influence (i.e., leader and follower). We then propose a Multi-dimensional Emotion Role Mining model (MERM) to determine a user’s emotion role in online social networks. Specifically, we tend to identify emotion roles by combining a set of features that reflect a user’s online emotional status, including degree of emotional characteristics, accumulated emotion preference, structural factor, temporal factor, and emotion change factor. Experiment results on a real-life micro-blog reposting dataset show that the classification accuracy of the proposed model can achieve up to 90.1%.

With the remarkable proliferation of smart mobile devices, mobile crowdsensing has emerged as a compelling paradigm to collect and share sensor data from surrounding environment. In many application scenarios, due to unavailable wireless network or expensive data transfer cost, it is desirable to offload crowdsensing data traffic on opportunistic device-to-device (D2D) networks. However, coupling between mobile crowdsensing and D2D networks, it raises new technical challenges caused by intermittent routing and indeterminate settings. Considering the operations of data sensing, relaying, aggregating, and uploading simultaneously, in this article, we study collaborative mobile crowdsensing in opportunistic D2D networks. Toward the concerns of sensing data quality, network performance and incentive budget, Minimum-Delay-Maximum-Coverage (MDMC) problem and Minimum-Overhead-Maximum-Coverage (MOMC) problem are formalized to optimally search a complete set of crowdsensing task execution schemes over user, temporal, and spatial three dimensions. By exploiting mobility traces of users, we propose an unified graph-based problem representation framework and transform MDMC and MOMC problems to a connection routing searching problem on weighted directed graphs. Greedy-based recursive optimization approaches are proposed to address the two problems with a divide-and-conquer mode. Empirical evaluation on both real-world and synthetic datasets validates the effectiveness and efficiency of our proposed approaches.

Estimation of housing requirement is beneficial for many applications such as guidance of house trading and real estate market regulation. Although there have been studies focusing on the demand analysis of urban resources, estimation of housing requirement is still under explored. To this end, in this paper we propose a systematic housing demand inference method, named Housing Demand Inference Model (HDIM), to estimate housing demand by exploiting the residential mobility of communities based on express delivery data. In this work, we first aggregate the express delivery records at community scale with clustering methods. Then, we propose a useful method to infer residential mobility by extracting express delivery related features and community related features. Since the features extracted are sparse for some residents, we utilize Regularized Singular Value Decomposition Model (RSVD) to construct missing values of features. After that, we infer residential mobility probability of each community by taking advantage of the less sparse features. We also consider community attractiveness as one of the factors influencing housing demand with the help of community profiles and geographical data. With the residential mobility probability and community attractiveness being obtained, we estimate housing demand with a regression model. Finally, experimental results on real-world data show that our model is effective to infer housing demand for communities in urban areas.

A job title usually implies the responsibility and the rank of a job position. While traditional job title analysis has been focused on studying the responsibilities of job titles, this paper attempts to reveal the rank of job titles. Specifically, we propose to extract job title hierarchy from employees' career trajectories. Along this line, we first quantify the Difficulty of Promotion (DOP) from one job title to another by a monotonic transformation of the length of tenure based on the assumption that a longer tenure usually implies a greater difficulty to be promoted. Then, the difference of two job title ranks is defined as a mapping of the DOP observed from job transitions. A Gaussian Bayesian Network (GBN) is adopted to model the joint distribution of the job title ranks and the DOPs in a career trajectory. Furthermore, a stochastic algorithm is developed for inferring the posterior job title rank by a given collection of DOPs in the GBN. Finally, experiments on more than 20 million job trajectories show that the job title hierarchy can be extracted precisely by the proposed method.

The popularity prediction of mobile apps provides substantial value to a broad range of applications, ranging from app development to targeted advertising. However, most previous studies do this work by establishing regression models for impact factors, or using clustering and classification algorithms. It does not fully investigate the process of popularity evolution and the reasons behind it. In this paper, we discuss and analyze the potential predictors, especially the impact of early evolutionary patterns on future popularity. To this end, we first explore six basic evolutionary patterns and six impact factors that are closely related to app popularity. After detailed analysis, we present CrowdPop, a popularity prediction model based on the Random Forest algorithm, to quantify patterns and factors as predictors of CrowdPop. The experiment results with a real-world dataset of 126 apps indicate that, compared with baseline methods, our CrowdPop performs better in mobile app popularity prediction.

Human behavior recognition is an effective way to realize natural human-computer interactions. Existing wireless sensing enabled gesture recognition technologies require a single person environment or an absolutely fixed position between the device and the user, which is not practical for daily use. In this paper, we present a non-contact radar-based gesture recognition system, named Gesture-Radar, which is able to capture arm gestures with low environmental dependence using a single Doppler radar. Our prototype design of Gesture-Radar is based on the dual channel Doppler information which contains specific Doppler shift and some other information reflected from the user while performing a certain gesture, and concretely we propose a two-stage classification method to identify arm gestures. Experimental result shows while in a single user environment, Gesture-Radar achieves up to 96.4% average classification accuracy for recognizing 5 different kinds of gestures and can work effectively while the distance between the user and the radar is within 3 meters. We also demonstrate that Gesture-Radar can be well adapted to multi-person environments.

Spatial crowdsourcing (SC) is an emerging paradigm of crowdsourcing, which commits workers to move to some particular locations to perform spatio-temporal-relevant tasks (e.g., sensing and activity organization). Task allocation or worker selection is a significant problem that may impact the quality of completion of SC tasks. Based on a conceptual model and generic framework of SC task allocation, this paper first gives a review of the current state of research in this field, including single task allocation, multiple task allocation, low-cost task allocation, and quality-enhanced task allocation. We further investigate the future trends and open issues of SC task allocation, including skill-based task allocation, group recommendation and collaboration, task composition and decomposition, and privacypreserving task allocation. Finally, we discuss the practical issues on real-world deployment as well as the challenges for large-scale user study in SC task allocation.

This paper proposes CrowdTracker, a novel object tracking system based on mobile crowd sensing (MCS). Different from traditional video-based object tracking approaches, CrowdTracker recruits people to collaboratively take photographs of the object to achieve object movement prediction and tracking. The optimization objective of CrowdTracker is to effectively track the moving object in real time and minimize the cost on user incentives. Specifically, the incentive is determined by the number of workers assigned and the total distance that workers move to complete the task. In order to achieve the objective, we propose the movement prediction (MPRE) model for object movement prediction and two other algorithms for task allocation, namely, T-centric and P-centric. T-centric selects workers in a task-centric way, while P-centric allocates tasks in a peoplecentric manner. By analyzing a large number of historical vehicle trajectories, MPRE builds a model to predict the object's next position. In the predicted regions, CrowdTracker selects workers by utilizing T-centric or P-centric. We evaluate the algorithms over a large-scale real-world dataset. Experimental results indicate that CrowdTracker can effectively track the object with a low incentive cost.

Human computer operations such as writing documents and playing games have become popular in our daily lives. These activities (especially if identified in a non-intrusive manner) can be used to facilitate context-aware services. In this paper, we propose to recognize human computer operations through keystroke sensing with a smartphone. Specifically, we first utilize the microphone embedded in a smartphone to sense the input audio from a computer keyboard. We then identify keystrokes using fingerprint identification techniques. The determined keystrokes are then corrected with a word recognition procedure, which utilizes the relations of adjacent letters in a word. Finally, by fusing both semantic and acoustic features, a classification model is constructed to recognize four typical human computer operations: 1) chatting; 2) coding; 3) writing documents; and 4) playing games. We recruited 15 volunteers to complete these operations, and evaluated the proposed approach from multiple aspects in realistic environments. Experimental results validated the effectiveness of our approach.

Navigation services using digital maps make people's travel much easier. However, these services often fail to provide specific routes to those destinations that lack micro data in digital maps, such as a small laundry store in a shopping area. In this paper, we propose CrowdNavi, a last mile navigation service in outdoor environments using crowdsourcing based on the guider-follower model. First, we collect trajectories of guiders and images of reference objects along trajectories. To guide followers by reference objects along the route, we design a Semantic Crowd Navigation model to generate fine-grained maps by integrating guiders' data. Second, we design two score functions to fulfill two main requirements and plan hints. Last, we provide context-aware navigation for followers based on the fine-grained map and detect deviation in real-time. Real world experiments conducted in three different areas show that our proposed system in combination with images of reference objects is efficient.

Monitoring group mobility and structure is crucial for understanding group activities and social relations. In this paper, we develop algorithms for fine-grained mobility classification and structure recognition of social groups utilizing mobile devices. First, we present a method that recognizes four levels of group mobility, including stationary, strolling, walking, and running. Second, using multiple types of mobile sensors, a novel relative position relationship estimation algorithm is developed to understand different moving group structures. We have conducted real-life experiments in which 12 volunteers moved in different small groups either in an office building or a shopping mall with various speeds and structures. Experimental results show that our approach achieves an accuracy of 99.5 percent in group mobility level classification and about 80 percent in group structure recognition.

With the rapid development of mobile networks and the proliferation of mobile devices, spatial crowdsourcing, which refers to recruiting mobile workers to perform location-based tasks, has gained emerging interest from both research communities and industries. In this paper, we consider a spatial crowdsourcing scenario: in addition to specific spatial constraints, each task has a valid duration, operation complexity, budget limitation, and the number of required workers. Each volunteer worker completes assigned tasks while conducting his/her routine tasks. The system has a desired task probability coverage and budget constraint. Under this scenario, we investigate an important problem, namely heterogeneous spatial crowdsourcing task allocation (HSC-TA), which strives to search a set of representative Pareto-optimal allocation solutions for the multi-objective optimization problem, such that the assigned task coverage is maximized and incentive cost is minimized simultaneously. To accommodate the multi-constraints in heterogeneous spatial crowdsourcing, we build a worker mobility behavior prediction model to align with allocation process. We prove that the HSC-TA problem is NP-hard. We propose effective heuristic methods, including multi-round linear weight optimization and enhanced multi-objective particle swarm optimization algorithms to achieve adequate Pareto-optimal allocation. Comprehensive experiments on both real-world and synthetic data sets clearly validate the effectiveness and efficiency of our proposed approaches.

The wide spread use of social network services, especially location based services, has transformed social networks into an important information source of real-world events. Many event detection systems using geo-tagged posts from social networks have been developed in recent years. Besides detecting real-world events, it is also desirable for government officials, news media, and police, etc., to identify on-site users of an event, from whom we could gather valuable information regarding the process of events and investigate suspects when an event is associated with crime or terrorist. However, due to the high uncertainty of human mobility patterns and the low probability of users sharing their location information, it is difficult to identify on-site users while a social event unfolds, and research work in this area is still in its infancy. In this paper, we propose a Fused fEature Gaussian prOcess Regression (FEGOR) model, which exploits three influential factors in social networks for on-site user identification: mobility influence, content similarity, and social relationship. By integrating these factors, we are able to estimate the distance between a user and a social event even when the user's location profile is unknown, thus identify on-site users. Experiments on a real-world Twitter dataset demonstrate the effectiveness of our model, achieving a minimum mean absolute error of 1.7km and outperforming state-of-the-art methods.

Volatile Point-of-Interests (vPOIs) refer to those small businesses which appear and disappear quickly in cities. How to maintain and incubate small business in the urban area is a big concern for both business owners and government administrators. Therefore, the prediction task for the rise and fall of vPOIs is valuable for both shopkeepers and administrators by supporting a variety of applications in urban economics. In this paper, we propose a framework, named FRFP, to predict the prosperity of vPOIs over time. Specifically, due to the data sparsity and skewness of the individual vPOIs, we first aggregate vPOIs prosperities at focal areas w.r.t. each vPOI category. Then we develop the dynamic-continuous CRF (DC-CRF) model to integrate the association between input and output as well as the correlations between outputs from temporal, spatial and contextual perspectives. Finally, we conduct empirical experiments on real-world data from Google Maps and NYC OpenData. The evaluation results demonstrate that our proposed approach outperforms baseline algorithms with considerable margins. In addition, we explore the predictability of different explanatory variables and provide actionable insights for both shopkeepers and urban planners.

Respiration rate plays an important role in human health monitoring. Traditional respiration rate monitoring techniques usually require users to wear some special equipment, which is not convenient for the elderly and the baby. Recently, Wi-Fi based respiration detection technique has attracted much attention due to its device-free and low-deployment-cost. However, most existing studies focus on respiration detection in experimental environments, without considering the impact of people around (it often occurs in our daily life), therefore, if there are several people in the system, their detection will fail. To address this open issue, we propose TinySense, a novel approach that can detect multiple persons' respiration at a time. In particular, we use multiple TX-RX antenna pairs to capture the Wi-Fi Channel State Information (CSI), filter out the data whose time-of-arrival (TOA) is bigger than a truncation threshold and remove subcarriers that are greatly affected by the multi-path effect. As a result, we can obtain the respiration data of each person from the mixed received signal. Experiments demonstrate the effectiveness of our approach on two-user respiration detection.

This paper builds a Food Delivery Network (FooDNet) that investigates the usage of urban taxis to support on demand take-out food delivery by leveraging spatial crowdsourcing. Unlike existing service sharing systems (e.g., ridesharing), the delivery of food in FooDNet is more time-sensitive and the optimization problem is more complex regarding high-efficiency, huge-number of delivery needs. In particular, we study the food delivery problem in association with the Opportunistic Online Takeout Ordering & Delivery service (O-OTOD). Specifically, the food is delivered incidentally by taxis when carrying passengers in the O-OTOD problem, and the optimization goal is to minimize the number of selected taxis to maintain a relative high incentive to the participated drivers. The two-stage method is proposed to solve the problem, consisting of the construction algorithm and the Large Neighborhood Search (LNS) algorithm. Preliminary experiments based on real-world taxi trajectory datasets verify that our proposed algorithms are effective and efficient.

Greenhouse gas emission by the increasing number of vehicles have become a significant problem in modern cities. To save energy and protect environment, recommending fuel-efficient routes to drivers becomes a promising way to alleviate this issue. To this end, in this paper, we present a novel fuel-efficient path-planning framework called GreenPlanner, which contains two phases. In the first phase, we build a personalized fuel consumption model (PFCM) for each driver, based on the individual driving behaviors and the physical features (e.g., traffic lights, stop signs, road network topology) along the routes. In the second phase, with the real-time traffic information collected via the mobile crowdsensing manner, we are able to estimate and compare the cost fuel among different routes for a given driver, and recommend him/her with the most fuel-efficient one. We evaluate the two-phase framework using the real-world datasets, consisting of road network, POI, the GPS trajectory data and the OBD-II data generated by 559 taxis in one day in the city of Beijing, China. Experimental results demonstrate that, compared to the baseline models, the proposed model achieves the best accuracy, with a mean fuel consumption error of less 7% for paths longer than 10 km. Moreover, users could save about 20% fuel consumption on average if driving along our suggested routes in our case studies.

The rapid development of social networks has woven themselves into people's daily life and become indispensable platforms with superior commercial and scientific values. Both companies and governments have discovered the potential effectiveness of employing social network users for information diffusion. Rather than only selecting a group of users who are interested in target topic, it is more beneficial to choose users with desired emotion preference to help diffuse information under certain emotional expectation. In this paper, we propose an emotional participant selection system that not only considers user's topic preference, but also more importantly takes user's emotional influence into account. Specifically, a dynamic forgetting mechanism is applied to learn user's topic preference, and independent cascade model is leveraged to construct emotional influence. Combining these two features, we develop an algorithm that can accomplish the task for emotional participant selection. Experimental results on a real-world data set validate the effectiveness of our proposed method.

Mobile crowd photographing (MCP) is an emerging area of interest for researchers as the built-in cameras of mobile devices are becoming one of the commonly used visual logging approaches in our daily lives. In order to meet diverse MCP application requirements and constraints of sensing targets, a multifacet task model should be defined for a generic MCP data collection framework. Furthermore, MCP collects pictures in a distributed way in which a large number of contributors upload pictures whenever and wherever it is suitable. This inevitably leads to evolving picture streams. This paper investigates the multiconstraint-driven data selection problem in MCP picture aggregation and proposes a pyramid-tree (PTree) model which can efficiently select an optimal subset from the evolving picture streams based on varied coverage needs of MCP tasks. By utilizing the PTree model in a generic MCP data collection framework, which is called CrowdPic, we test and evaluate the effectiveness, efficiency, and flexibility of the proposed framework through crowdsourcing-based and simulation-based experiments. Both the theoretical analysis and simulation results indicate that the PTree-based framework can effectively select a subset with high utility coverage and low redundancy ratio from the streaming data. The overall framework is also proved flexible and applicable to a wide range of MCP task scenarios.

Pedestrians distracted by smartphones are easy to meet with various dangers when crossing or walking on the street, such as the obstacles on the sidewalk (e.g., temporary parking and road repairing). Existing works about pedestrian safety are mostly based on the sensing capabilities from a single device. The surrounding information that can be learned, however, is quite limited or incomplete. Therefore, in many cases the dangers cannot be detected and the pedestrians cannot be alerted. In this paper, a novel system called CrowdWatch is proposed, which leverages mobile crowd sensing and crowd intelligence aggregation to detect temporary obstacles and make effective alerts for distracted walkers. To detect obstacles, we first study the regular rules of pedestrians' avoidance behaviors from the aspects of turn-making and visual contexts. The Dempster-Shafer evidence theory is then used to fuse the behavior and visual contexts, and further calculate the confidence of obstacle existence. Afterwards, we leverage the features of pedestrians' traces to characterize an appropriate dangerous area, which is used to alert distracted walkers. The conducted experiments with 36 participants and different obstacle settings indicate that the crowd-intelligencebased obstacle detection method is effective and the accuracy of reminding attains 83.3%.

The papers in this special section focus on cyber-physical man-machine systems with particular emphasis on situation, activity, and goal awareness deployed in these systems. Recent advances in sensing technologies, the Internet of Things, pervasive computing, smart environments have transformed traditional embedded ICT systems into an ecosystem of interconnected and collaborating smart objects, devices, embedded systems, and most importantly humans. Such systems, often referred to as cyber-physical systems (CPS), are usually human user-driven or user-centered, and are aimed at providing people and businesses with a wide range of innovative applications and services. For example, a “smart home” can monitor and analyze the daily activities of its inhabitants, usually the elderly or individuals with disabilities, so that personalized context-aware assistance can be provided. A “smart city” can monitor, manage, and potentially control all basic city functionalities such as transport, energy supply, and waste collection, at a higher level of automation by collecting and harnessing sensor data across a large geographic expanse. In order to respond in real-time to an individual user’s specific needs in dynamic and complex situations, and to support ergonomics and user-friendliness through consideration of human factors such as privacy, dignity, and behavior characteristics, cyber-physical human–machine systems need to be aware of the physical environment and human participant behavior. This awareness enables effective and fast feedback loops between sensing and actuation, possibly with cognitive and learning capabilities adapting to participant preferences, capabilities, and the modality of human–machine interaction as well as dynamic situations.

Worker selection is a key issue in mobile crowd sensing (MCS). While the previous worker selection approaches mainly focus on selecting a proper subset of workers for a single MCS task, a multitask-oriented worker selection is essential and useful for the efficiency of large-scale MCS platforms. This paper proposes ActiveCrowd, a worker selection framework for multitask MCS environments. We study the problem of multitask worker selection under two situations: worker selection based on workers' intentional movement for time-sensitive tasks and unintentional movement for delay-tolerant tasks. For time-sensitive tasks, workers are required to move to the task venue intentionally and the goal is to minimize the total distance moved. For delay-tolerant tasks, we select workers whose route is predicted to pass by the task venues and the goal is to minimize the total number of workers. Two greedy-enhanced genetic algorithms are proposed to solve them. Experiments verify that the proposed algorithms outperform baseline methods under different experiment settings (scale of task sets, available workers, varied task distributions, etc.).

Despite the great demand on and attempts at package express shipping services, online retailers have not yet had a practical solution to make such services profitable. In this paper, we propose an economical approach to express package delivery, i.e., exploiting relays of taxis with passengers to help transport package collectively, without degrading the quality of passenger services. Specifically, we propose a two-phase framework called crowddeliver for the package delivery path planning. In the first phase, we mine the historical taxi trajectory data offline to identify the shortest package delivery paths with estimated travel time given any Origin-Destination pairs. Using the paths and travel time as the reference, in the second phase we develop an online adaptive taxi scheduling algorithm to find the near-optimal delivery paths iteratively upon real-time requests and direct the package routing accordingly. Finally, we evaluate the two-phase framework using the real-world data sets, which consist of a point of interest, a road network, and the large-scale trajectory data, respectively, that are generated by 7614 taxis in a month in the city of Hangzhou, China. Results show that over 85% of packages can be delivered within 8 hours, with around 4.2 relays of taxis on average.

Moving destination prediction offers an important category of location-based applications and provides essential intelligence to business and governments. In existing studies, a common approach to destination prediction is to match the given query trajectory with massive recorded trajectories by similarity calculation. Unfortunately, due to privacy concerns, budget constraints, and many other factors, in most circumstances, we can only obtain a sparse trajectory dataset. In sparse dataset, the available moving trajectories are far from enough to cover all possible query trajectories; thus the predictability of the matching-based approach will decrease remarkably. Toward destination prediction with sparse dataset, instead of searching similar trajectories over the sparse records, we alternatively examine the changes of distances from sampling locations to final destination on query trajectory. The underlying idea is intuitive: It is directly motivated by travel purpose, people always get closer to the final destination during the movement. By borrowing the conception of gradient descent in optimization theory, we propose a novel moving destination prediction approach, namely MGDPre. Building upon the mobility gradient descent, MGDPre only investigates the behavior characteristics of query trajectory itself without matching historical trajectories, and thus is applicable for sparse dataset. We evaluate our approach based on extensive experiments, using GPS trajectories generated by a sample of taxis over a 10-day period in Shenzhen city, China. The results demonstrate that the effectiveness, efficiency, and scalability of our approach outperform state-of-the-art baseline methods.

Visual crowdsensing is successfully applied in numerous application areas, yet little work has been done on measuring and improving the quality of worker contributed visual data. Rather than evaluating the visual quality based on traditional metrics such as resolution, we focus on data diversity, which is crucial for a broad stream of visual crowdsensing tasks. Two representative diversity-oriented task types are studied, namely static object imagery and evolving event photography. The former aims to collect multi-facet/ aspect yet low redundant data about a stationary object, while the latter wants to detect and collect details of key scenes throughout an event. We link these quality needs with data utility and propose a unified visual crowdsensing framework called UtiPay. Data utility is characterized by the macro and micro diversity needs: at the macro level, the pyramid-tree approach is proposed for multi-attribute-based data grouping; at the micro level, we use several strategies for intra-group data selection and worker contribution measurement. To study the impact of our proposed utility measurement approaches, we propose two utility-enhanced payment schemes as incentive mechanisms: Uti and Uti-Bid. Experiments over several user studies with a total of 43 subjects validate the performance of UtiPay for measuring and enhancing the data quality of visual crowdsensing tasks.

Guest Editorial.

Human identification plays an important role in human-computer interaction. There have been numerous methods proposed for human identification (e.g., face recognition, gait recognition, fingerprint identification, etc.). While these methods could be very useful under different conditions, they also suffer from certain shortcomings (e.g., user privacy, sensing coverage range). In this paper, we propose a novel approach for human identification, which leverages Wi-Fi signals to enable non-intrusive human identification in domestic environments. It is based on the observation that each person has specific influence patterns to the surrounding Wi-Fi signal while moving indoors, regarding their body shape characteristics and motion patterns. The influence can be captured by the Channel State Information (CSI) time series of Wi-Fi. Specifically, a combination of Principal Component Analysis (PCA), Discrete Wavelet Transform (DWT) and Dynamic Time Warping (DTW) techniques is used for CSI waveform- based human identification. We implemented the system in a 6m*5m smart home environment and recruited 9 users for data collection and evaluation. Experimental results indicate that the identification accuracy is about 88.9% to 94.5% when the candidate user set changes from 6 to 2, showing that the proposed human identification method is effective in domestic environments.

Task allocation or participant selection is a key issue in Mobile Crowd Sensing (MCS). While previous participant selection approaches mainly focus on selecting a proper subset of users for a single MCS task, multi-task-oriented participant selection is essential and useful for the efficiency of large-scale MCS platforms. This paper proposes TaskMe, a participant selection framework for multi-task MCS environments. In particular, two typical multi-task allocation situations with bi-objective optimization goals are studied: (1) For FPMT (few participants, more tasks), each participant is required to complete multiple tasks and the optimization goal is to maximize the total number of accomplished tasks while minimizing the total movement distance. (2) For MPFT (more participants, few tasks), each participant is selected to perform one task based on pre-registered working areas in view of privacy, and the optimization objective is to minimize total incentive payments while minimizing the total traveling distance. Two optimal algorithms based on the Minimum Cost Maximum Flow theory are proposed for FPMT, and two algorithms based on the multi-objective optimization theory are proposed for MPFT. Experiments verify that the proposed algorithms outperform baselines based on a large-scale real-word dataset under different experiment settings (the number of tasks, various task distributions, etc.).

Mobile social media enables people to record ongoing physical events they witness and share them instantaneously online. However, since these event pictures are often individually provided, they are typically fragmented and possess high redundancy. Though there have been studies about visual event summarization, they pay little attention to collaborative sensing, subevent detection, and event summary. In this paper, we present several building blocks for a cooperative visual sensing and sharing system. We create a virtual opportunistic community associated with an event, where members collaborate to cover different aspects of the event. More specifically, a crowd-powered approach is first used to localize the event. We then propose three subevent segmentation methods based on crowd-event interaction patterns. Based on the segmentation results, we summarize the event at two levels: multi-facet subevent summary and crowd-behavior-based highlights. Experiments over 21 online datasets and two real world datasets demonstrate the effectiveness of our approaches.

A vehicle's fuel consumption is strongly related to both its loading and the driver's driving behavior, such as aggressive/tender acceleration, improper/proper gear change or running/stopping the engine while waiting. This paper introduces EasyRoute, an economical route recommendation system for modern vehicles, implemented in smartphones, to improve fuel efficiency. EasyRoute senses the vehicle's fuel consumption through the On-Board Diagnostics (OBD) adapter and then models the driver's personal fuel consumption according to OBD data from two aspects: i) when the vehicle is moving and ii) when the vehicle is idling or waiting. Based on the crowdsourced traffic information, EasyRoute can near-correctly predict total fuel consumptions of different routes and recommend drivers with the greenest route. We describe the EasyRoute framework and evaluate it by collecting OBD and GPS data from 559 taxis in Beijing. Comparing with some commonly used baselines with error metrics, the experimental results show that using a small 10-minute dataset for training, the total fuel consumption estimated by EasyRoute has a relative error of at least 30% less than the baselines.

Monitoring group mobility and structure is crucial for public safety management and emergency evacuation. In this paper, we propose a fine-grained mobility classification and structure recognition approach for social groups based on hybrid sensing using mobile devices. First, we present a method which classifies group mobility into four levels, including stationary, strolling, walking and running. Second, by combining mobile sensing and Wi-Fi signals, a novel relative position relationship estimation algorithm is developed to understand moving group structures of different shapes. We have conducted real-life experiments in which eight volunteers form two to three small groups moving in a teaching building with different speed and structures. Experimental results show that our approach achieves an accuracy of 99.5% in mobility classification and about 80% in group structure recognition.

The study of geo‐social behaviors has long been a scientific problem. In contrast to traditional social science, which suffers from the problems such as high data collection cost and imported user subjectivity, a new approach is presented to study social behaviors based on mobile phone sensing data. Different from other similar studies on mobile social sensing, three different types of geo‐social behaviors, including online interaction, offline interaction, and mobility patterns, are characterized based on a newly released Nokia mobile phone data set. We further discuss the impact factors to these behaviors as well as the correlation among them. The findings in this article are crucial for many different fields, ranging from urban planning, location‐based services, to social recommendation.

This paper presents a group-aware mobile crowd sensing system called MobiGroup, which supports group activity organization in real-world settings. Acknowledging the complexity and diversity of group activities, this paper introduces a formal concept model to characterize group activities and classifies them into four organizational stages. We then present an intelligent approach to support group activity preparation, including a heuristic rule-based mechanism for advertising public activity and a context-based method for private group formation. In addition, we leverage features extracted from both online and offline communities to recommend ongoing events to attendees with different needs. Compared with the baseline method, people preferred public activities suggested by our heuristic rule-based method. Using a dataset collected from 45 participants, we found that the context-based approach for private group formation can attain a precision and recall of over 80%, and the usage of spatial-temporal contexts and group computing can have more than a 30% performance improvement over considering the interaction frequency between a user and related groups. A case study revealed that, by extracting the features such as dynamic intimacy and static intimacy, our cross-community approach for ongoing event recommendation can meet different user needs.

Location-based social networks (LBSNs) provide people with an interface to share their locations and write reviews about interesting places of attraction. The shared locations form the crowdsourced digital footprints, in which each user has many connections to many locations, indicating user preference to locations. In this paper, we propose an approach for personalized travel package recommendation to help users make travel plans. The approach utilizes data collected from LBSNs to model users and locations, and it determines users' preferred destinations using collaborative filtering approaches. Recommendations are generated by jointly considering user preference and spatiotemporal constraints. A heuristic search-based travel route planning algorithm was designed to generate travel packages. We developed a prototype system, which obtains users' travel demands from mobile client and generates travel packages containing multiple points of interest and their visiting sequence. Experimental results suggest that the proposed approach shows promise with respect to improving recommendation accuracy and diversity.

It is an important yet challenging task for investors to determine the most suitable type of shop (e.g., restaurant, fashion) for a newly opened store. Traditional ways are predominantly field surveys and empirical estimation, which are not effective as they lack shop-related data. As social media and location-based services (LBS) are becoming more and more pervasive, user-generated data from these platforms are providing rich information not only about individual consumption experiences, but also about shop attributes. In this paper, we investigate the recommendation of shop types for a given location, by leveraging heterogeneous data that are mainly historical user preferences and location context from social media and LBS. Our goal is to select the most suitable shop type, seeking to maximize the number of customers served from a candidate set of types. We propose a novel bias learning matrix factorization method with feature fusion for shop popularity prediction. Features are defined and extracted from two perspectives: location, where features are closely related to location characteristics, and commercial, where features are about the relationships between shops in the neighborhood. Experimental results show that the proposed method outperforms state-of-the-art solutions.

Micro-blog has been increasingly used for the public to express their opinions, and for organizations to detect public sentiment about social events or public policies. In this article, we examine and identify the key problems of this field, focusing particularly on the characteristics of innovative words, multi-media elements, and hierarchical structure of Chinese “Weibo.” Based on the analysis, we propose a novel approach and develop associated theoretical and technological methods to address these problems. These include a new sentiment word mining method based on three wording metrics and point-wise information, a rule set model for analyzing sentiment features of different linguistic components, and the corresponding methodology for calculating sentiment on multi-granularity considering emoticon elements as auxiliary affective factors. We evaluate our new word discovery and sentiment detection methods on a real-life Chinese micro-blog dataset. Initial results show that our new diction can improve sentiment detection, and they demonstrate that our multi-level rule set method is more effective, with the average accuracy being 10.2% and 1.5% higher than two existing methods for Chinese micro-blog sentiment analysis. In addition, we exploit visualization techniques to study the relationships between online sentiment and real life. The visualization of detected sentiment can help depict temporal patterns and spatial discrepancy.

The newly emerging event-based social networks (EBSNs) extend social interaction from online to offline, providing an appealing platform for people to organize and participate realworld social events. In this paper, we investigate how to select potential participants in EBSNs from an event host's point of view. We formulate the problem as mining influential and preferable invitee set, considering from two complementary aspects. The first aspect concerns users' preference with respect to the event. The second aspect is influence maximization, which aims to influence the largest number of users to participate the event. In particular, we propose a novel Credit Distribution-User Influence Preference (CD-UIP) algorithm to find the most influential and preferable followers as the invitees. We collect a real-world dataset from a popular EBSNs called "Douban Events", and the experimental results on the dataset demonstrate the proposed algorithm outperforms the state-of-the-art prediction methods.

Discovering the determinants of job change and predicting the individual job change occasion are essential approaches for understanding the professional careers of human. However, with the evolution of labor division and globalization, modern careers become more self-directed and dynamic, which makes job change occasion difficult to predict. Fortunately, the emerging online professional networks and location-based social networks provide a large amount of work experience and daily activity records of individuals around the world, which open a venue for the accurate job change analysis. Discovering the determinants of job change and predicting the individual job change occasion are essential approaches for understanding the professional careers of human. However, with the evolution of labor division and globalization, modern careers become more self-directed and dynamic, which makes job change occasion difficult to predict. Fortunately, the emerging online professional networks and location-based social networks provide a large amount of work experience and daily activity records of individuals around the world, which open a venue for the accurate job change analysis. In this paper, we aggregate the work experiences and check-in records of individuals to model the job change motivations and correlations between professional and daily life. Specifically, we attempt to reveal to what extent the job change occasion can be predicted based on the career mobility and daily activity patterns at the individual level. Following the classical theory of job mobility determinants, we extract and quantify the environmental conditions and personal preference of careers from the perspective of industrial/regional constraints and personal interests/demands. Besides, we investigate the factors of activity patterns which may be correlated with job change as cause and effect results. First, we quantify the consumption diversity, sentiment fluctuation and geographic movement from the check-in records as indicators. Then, we leverage the center-bias level assignment and multi-point snapshot mechanism to capture historical and parallel migration. Finally, experimental results based on a large real-world dataset show that the job change occasions can be accurately predicted with the aggregated factors.

The papers in this special section provide the opportunity for researchers, practitioners, and application developers to review and discuss the state-of-the-art and trends of MCS (mobile crowd sensing)techniques and applications or propose new solutions. The ubiquitous sensor-rich mobile devices (e.g., smartphones, wearable devices, and smart vehicles) have been playing an increasing important role in the evolution of the Internet of Things (IoTs), which bridges the digital space and physical space at a societal scale. Their powerful computing/communication capacities, huge population, and inherent mobility make mobile-device networks a much more flexible and cost-effective IoT solution than static sensor networks.

Leveraging the regularities of people's trajectories, mobility prediction can help forecast social interaction opportunities. In this paper, in order to facilitate real-world social interaction, we aim to predict “serendipitous” social interactions, which are defined as unplanned encounters and interaction opportunities and regarded as emerging social interactions. We collected GPS trajectory data from people' daily life on campus and use it as empirical mobility traces to generate decision trees and model trees to predict next venues, arrival times, and user encounter. Mobility regularities are mainly considered in these prediction models, and mobility contexts (e.g., time, location, and speed) act as decision nodes in the classification trees. Experimental results using collected GPS data showed that our system achieves 90% accuracy for predicting a user's next venue using a decision tree algorithm, with minute-level (around 5 min) prediction error for arrival time using the model tree algorithm. Two prototype applications were developed to support serendipitous social interaction on campus, and the feedback from a user study with 25 users demonstrated the usability of these two applications.

Participatory sensing [Burke 2006] is an emerging computing paradigm that tasks everyday mobile devices to form participatory sensor networks.It allows the increasing number of mobile phone users to share local knowledge acquired by their sensorenhanced devices, such as monitoring of pollution or noise levels and traffic conditions.The sensing data from volunteer contributors can be further analyzed and processed to form crowd intelligence [Zhang et al. 2011], which can be elaborated into three dimensions: personal awareness, social awareness, and urban awareness.Layered on these concepts, we have raised the new term mobile crowd sensing and computing (MCSC) to characterize crowd intelligence extraction from large-scale and heterogeneous usercontributed data [Guo et al. 2014].A formal definition of MCSC is as follows: a new sensing paradigm that empowers ordinary citizens to contribute data sensed or generated from their mobile devices, then aggregates and fuses the data in the cloud for crowd intelligence extraction and human-centric service delivery.It has the following three features compared to participatory sensing:-MCSC leverages both sensed data from mobile devices (from the physical space) and user-contributed data from mobile social network services (from the cyber space).In other words, MCSC counts both explicit and implicit user participation for data collection.-Having both online and offline user-contributed data, MCSC highlights the usage of heterogeneous crowdsourced data for crowd intelligence extraction.

Planning an itinerary before traveling to a city is one of the most important travel preparation activities. In this paper, we propose a novel framework called TripPlanner, leveraging a combination of location-based social network (i.e., LBSN) and taxi GPS digital footprints to achieve personalized, interactive, and traffic-aware trip planning. First, we construct a dynamic point-of-interest network model by extracting relevant information from crowdsourced LBSN and taxi GPS traces. Then, we propose a two-phase approach for personalized trip planning. In the route search phase, TripPlanner works interactively with users to generate candidate routes with specified venues. In the route augmentation phase, TripPlanner applies heuristic algorithms to add user's preferred venues iteratively to the candidate routes, with the objective of maximizing the route score while satisfying both the venue visiting time and total travel time constraints. To validate the efficiency and effectiveness of the proposed approach, extensive empirical studies were performed on two real-world data sets from the city of San Francisco, which contain more than 391 900 passenger delivery trips generated by 536 taxis in a month and 110 214 check-ins left by 15 680 Foursquare users in six months.

During the last decade, microblog has become an important social networking service with billions of users all over the world, acting as a novel and efficient platform for the creation and dissemination of real-time information. Modeling and revealing the information propagation patterns in microblogging services cannot only lead to more accurate understanding of user behaviors and provide insights into the underlying sociology, but also enable useful applications such as trending prediction, recommendation and filtering, spam detection and viral marketing. In this article, we aim to reveal the information propagation patterns in Sina Weibo, the biggest microblogging service in China. First, the cascade of each message is represented as a tree based on its retweeting process. Afterwards, we divide the information propagation pattern into two levels, that is, the macro level and the micro level. On one hand, the macro propagation patterns refer to general propagation modes that are extracted by grouping propagation trees based on hierarchical clustering. On the other hand, the micro propagation patterns are frequent information flow patterns that are discovered using tree-based mining techniques. Experimental results show that several interesting patterns are extracted, such as popular message propagation, artificial propagation, and typical information flows between different types of users.

Community bulletin boards serve an important function for public information sharing in modern society. Posted fliers advertise services, events, and other announcements. However, fliers posted offline suffer from problems such as limited spatial-temporal coverage and inefficient search support. In recent years, with the development of sensor-enhanced mobile devices, mobile crowd sensing (MCS) has been used in a variety of application areas. This paper presents FlierMeet, a crowd-powered sensing system for cross-space public information reposting, tagging, and sharing. The tags learned are useful for flier sharing and preferred information retrieval and suggestion. Specifically, we utilize various contexts (e.g., spatio-temporal info, flier publishing/reposting behaviors, etc.) and textual features to group similar reposts and classify them into categories. We further identify a novel set of crowd-object interaction hints to predict the semantic tags of reposts. To evaluate our system, 38 participants were recruited and 2,035 reposts were captured during an eight-week period. Experiments on this dataset showed that our approach to flier grouping is effective and the proposed features are useful for flier category/semantic tagging.

Micro-blog has been increasingly used for the public to express their opinions, and for organisations to detect public sentiment about social events. In contrast to the effort and progress made in English-based micro-blog analysis, research on Chinese micro-blog received relatively little attention. In this paper we examine and identify the key problems of this field, focusing particularly on the characteristics of innovative words, emoticon elements and hierarchical structure of Chinese “Weibo”. Based on the analysis we propose and develop associated theoretical and technological methods to address these problems. These include the development of new sentiment word mining method based on three wording standards and point-wise metrics, a rule set model for analyzing sentiment features of different linguistic components, and the corresponding methodology for calculating sentiment on multi-granularity considering emoticon elements. We use original Chinese tweets from a dataset of Sina Weibo to test and evaluate our new word discovery and sentiment detection methods. Initial results show that our new diction can improve sentiment detection, and demonstrate that our multi-level rule set method is more effective by giving 10.2% and 1.5% higher average accuracy than two existing methods for Chinese micro-blog sentiment analysis. In addition, we exploit visualisation techniques to study the relationships between online sentiment and real life, which can help depict the correlation between public emotions and events.

The newly emerging event-based social networks (EBSNs) connect online and offline social interactions, offering a great opportunity to understand behaviors in the cyber-physical space. While existing efforts have mainly focused on investigating user behaviors in traditional social network services (SNS), this paper aims to exploit individual behaviors in EBSNs, which remains an unsolved problem. In particular, our method predicts activity attendance by discovering a set of factors that connect the physical and cyber spaces and influence individual's attendance of activities in EBSNs. These factors, including content preference, context (spatial and temporal) and social influence, are extracted using different models and techniques. We further propose a novel Singular Value Decomposition with Multi-Factor Neighborhood (SVD-MFN) algorithm to predict activity attendance by integrating the discovered heterogeneous factors into a single framework, in which these factors are fused through a neighborhood set. Experiments based on real-world data from Douban Events demonstrate that the proposed SVD-MFN algorithm outperforms the state-of-the-art prediction methods.

Internet of Things (IoT) invasions a future in which digital and physical entities (e.g., mobile devices, wearable devices) can be linked, by means of appropriate information and communication technologies, to enable a whole new class of applications and services. In this paper, we study the dissemination of content over a mobile social network, which has become an attractive proxy for investigating human behaviors due to the rapid development of mobile phones. One of the most interesting and challenging problems about content dissemination is that how much attention of a specific post from a user can ultimately gain? Hence, in other words, can we forecast the crowds׳ concern in the mobile social networking environment, and how? We try to tackle this issue by exploring approaches to predict the amount of reposts any given post will obtain in Sina Weibo, a well-known mobile social networking service in China. We examine several novel implicit factors impacting the popularity of content, such as Modality, MaxMediaWeight and Activeness. Furthermore, we propose a RepostsTree based method to model the reposting process in a temporal dynamic manner. Experimental results over the collected data from Sina Weibo indicate that our method is effective on content diffusion prediction in mobile social networks.

Human memory often fails. People are frequently beset with questions like “Who is that person? I think I met him in Tokyo last year.” Existing memory aid tools cannot well support the recall of names effectively. This paper explores the memory recall enhancement issue from the perspective of memory cue extraction and associative search, and proposes a generic methodology to extract memory cues from heterogeneous, multimodal, physical/virtual data sources. Specifically, we use the contact name recall in the academic community as the target application to showcase our proposed methodology. We further develop an intelligent social contact manager that supports 1) autocollection of rich contact data from a combination of pervasive sensors and Web data sources, and 2) associative search of contacts when human memory fails. The system is validated by testing the performance of contact data collection techniques. An empirical user study on contact memory recall is also conducted, through which several findings about contact memorizing and recall are presented. Classic cognitive psychology theories are used to interpret these findings.

The Internet of Things (IoT) is a technical revolution that represents the future of computing and communications. Under its vision, the next-generation Internet will promote the harmonious interaction between human, society, and smart things. The current research in IoT is mainly from the perspective of connecting and managing things. The humanized, social side of IoT, however, is still not explored. In this article, we intend to present the IoT from the human-centric perspective. By analyzing the tight-coupled relationship between human and opportunistic connection of smart things (e.g., mobile phones, vehicles), we propose Opportunistic IoT. It enables information sharing and dissemination within/among opportunistic communities that are formed with the movement and opportunistic contact nature of human. We characterize the bi-directional effect between human and opportunistic IoT, present the innovative application areas, and discuss the challenges raised by this new computing paradigm.

Over the last few years, many efforts have been devoted to revealing human mobility patterns. However, the regularity and variability of human mobility from a microscopic view, i.e., what factors affect human mobility patterns, has yet not been investigated. In this paper, we aim to study the impact factors that may affect the regularity and variability of human mobility patterns using social network analysis. Specifically, we introduce the spatial interaction matrix to represent the interaction strength and interaction semantics among spatial regions. Based on the spatial interaction matrix, we investigate the factors that impact the mobility patterns, including temporal factors, occupational factors and age factors. Our experimental results demonstrate that lots of factors such as environmental, temporal and age factors contribute to the shape of human mobility patterns.

Social and Community Intelligence (SCI) represents an emerging area that aims at revealing individual/group behaviors, social interactions as well as community dynamics by mining the digital traces left by people while interacting with cyber-physical spaces. The digital traces are generated mainly from three information sources: Internet and Web applications, static infrastructure, mobile devices and wearable sensors. In this workshop we hope to get people from different disciplines together to share their visions and insights on how to tackle the challenges faced by SCI, such as participatory sensing, heterogeneous data fusion, intelligence extraction, privacy issues, and so on.

The ability to leverage the power of a network of social contacts is important to get things done. However, as the number of contacts increases, people often find it difficult to maintain their contact network by using merely memory, and are frequently encompassed with questions like "who is that person, I met him in Tokyo last year". Existing contact tools make up for the shortage of unreliable human memory by storing contact information in the digital format, but laying much burden on users on manually inputting contact data. This paper, however, presents a social contact management system called SCM, which supports the auto-collection of rich contact data by exploring the aggregated power of pervasive sensing and Web intelligence techniques. Regarding that people often need to leverage several associated things (e.g., meeting location) to fetch other information about a contact (e.g., his name), we also develop an associative contact retrieval method. The effectiveness and runtime performance of our system is validated through a set of experiments.