Cross-modal semantic communication plays a crucial role in emergency response systems. However, there are still two major challenges in practical applications: insufficient semantic compression due to the computing constraints of the source device, and poor signal generation at the sink device caused by harsh communication environment. To this end, this work fully leverages multi-modal interactive large AI model (LAM) distillation to enhance sparse feature representation for dynamic residual compression, while utilizing LAM adversarial feature mapping to repair the feature impairment for accurate signal generation, thereby achieving low latency and high reliable communication. Specifically, we first propose a scalable cross-modal semantic communication framework, which constructs a multi-modal semantic knowledge base (MSKB) by semantic similarity to support cross-modal semantic codec. On this basis, residual-guided cross-modal semantic encoding (RCSE) is designed, which employs video-infrared bidirectional knowledge distillation to lightweight LAM for feature extraction, and initially compresses features by inter-modal correlations. Further, according to the link state, the similar features in MSKB are dynamically stripped and the residual features are compressed for low-latency transmission. Additionally, an adversarial mapping-based cross-modal semantic decoding (AMCSD) is developed, which leverages adversarial mutual information (MI) to map features in LAM to impaired features, and discriminate semantic fidelity to enhance the mapping robustness, ensuring accurate signal generation. Finally, an emergency simulation platform is constructed. Experiments demonstrate that our scheme reduces the transmission latency by 27.64% and improves the signal generation precision by 14.06%.

With the popularity of unmanned inspection and telemedicine applications, edge source devices need to transmit multimodal data such as video, text, infrared, etc. in real-time in a computing power and bandwidth-constrained environment, whereas traditional single-modal compression and steganography methods struggle to balance efficiency and security. To address the current lack of end-to-end efficient methods that jointly consider multimodal coding and steganography, this paper is the first to propose a cross-modal private and covert communication scheme with constraints of computation and bandwidth, unifying cross-modal fusion, invertible steganography, and hyperprior image compression into a single model, and introducing a multi-stage joint training strategy. First, a lightweight composition-steganography-compression pipeline is designed at the edge source device, which composes visible and infrared signals into a semantically enhanced image, generates a natural stego image through invertible steganography, and produces a compact bit-stream through deep compression, achieving a 51.5% reduction in bitrate and a 48.3% reduction in encoding latency. Second, a multi-stage decoding-cross-modal reconstruction pipeline is built at the sink device to sequentially complete bitstream decompression, inverse steganography, and cross-modal reconstruction, ultimately outputting visible images (PSNR 30.37 dB) and infrared images (PSNR 31.58 dB), with a 40.9% reduction in decoding latency. Finally, a three-stage joint training further enhances PSNR and saves an additional 6% bitrate. Experimental results validate the efficiency, security, and robustness of the proposed method in resource-constrained environments.

Multi-modal traffic is becoming dominant across vertical domains. Rather than reconstructing raw data with content-agnostic fidelity, many emerging applications are inherentlytask-orientedand exhibit exploitable inter-modal correlations. In this paper we study such scenarios within the Shannon framework, modeling them as a CEO-type multi-terminal source coding problem under logarithmic loss, where asemantic priorSis available at both the encoders and the decoder. We refer to this formulation astask-oriented distributed cross-modal source coding(TD-CMSC) to emphasize its multimodal sensing application, while the underlying mathematical problem remains a classical log-loss CEO / multi-terminal source coding model augmented with the semantic prior S. On the theoretical side, we characterize the corresponding rate–distortion region under log-loss with semantic prior S in a general multi-encoder setting, and, under a total-rate constraint, we derive the associated distortion– rate function together with an explicit single-letter expression for the optimal per-modality rate allocation at the extreme points of the region. On the technical side, leveraging these results, we design a coding system for vector Gaussian sources that combines task-oriented conditional quantization with successiveWyner–Ziv coding. Numerical experiments confirm the feasibility and rate efficiency of the proposed design.

Federated learning emerges as a key enabler toward pervasive intelligence across IoT ecosystems with provable privacy guarantees. While recent efforts on client selection have been made for optimizing its communication efficiency in iterative model aggregation over resource-constrained networks, their scalability fundamentally breaks down in dense deployments. This limitation stems from the NP-hard complexity of congestion-aware scheduling, where co-channel interference creates exponentially growing solution spaces. In this context, we present a novel client selection framework with asymptotic scalability in massive IoT, which leverages the intrinsic graph topology with insights from statistical physics. First, this work formulate a universal client selection problem, capturing both positive network externalities derived from collaborative knowledge exchange and congestion effects induced by co-channel interference. This formulation is then transformed into a node classification task via Ising spin Hamiltonian mapping, establishing an explicit connection between federated learning, statistical physics, and graph optimization. Building on this foundation, we develop a lightweight graph neural solver that adaptively selects clients via recognizing node state with iterative neighbor aggregation of learnable embeddings. Comprehensive experiments validate that our approach maintains state-of-the-art scheduling performance, while scaling to network sizes orders of magnitude beyond what conventional methods can handle.

Immersive communication, as a key usage scenario in 6 G, aims to provide interactive experiences by delivering real-time, high-fidelity sensory feedback (e.g., vision and touch). However, simultaneously achieving low latency, high data rate, and high reliability often poses a conflicting challenge from a transmission perspective. Unlike the optimization of a physical transmission environment (e.g., RIS-THz), in this work, we propose a cross-sensory transmission strategy that involves both encoding and networking, leveraging the potential correlations among various sensory modalities to support both data compression and enhancement. On the encoding side, we explore explainable surface semantics (e.g., texture, compliance) as intermediaries to associate visual and tactile sensory modalities for the design of a cross-sensory visual coding method. This method compresses the massive volume of visual data based on semantic correlations, significantly reducing bitrates to enable low-latency transmission. On the networking side, a cross-sensory masked pre-training approach is incorporated under a wide range of simulated packet loss. This approach facilitates fast and precise reconstruction of lost data using minimal observed data packets from both modalities, compensating for transmission reliability degradation under random and significant packet loss rates. Experimental results from a constructed VR education platform demonstrate that the proposed transmission strategy improves the data compression rate by more than 33% while maintaining a tolerance for packet loss rates of at least 50%.

With the rise of immersive communications, live tactile streaming has become essential for delivering active tactile feedback. However, mainstream bitrate-scalable streaming methods—designed for traditional audio/video applications—overlook the latency-sensitive nature of tactile streams, significantly degrading the user's Quality of Experience (QoE) in multi-modal scenarios. To address this challenge, we propose a novel live tactile streaming strategy that integrates both coding and transmission optimizations. For coding, we design the Dual-Scalable Tactile Coding (DSTC) framework, which provides scalable options for both latency and bitrate. For transmission, we develop the Tactile Adaptive Bitrate (TABR) framework, which dynamically selects the optimal configuration based on fluctuating network conditions and time-varying transmission demands. Technically, DSTC offers multiple latency options by configuring variable window lengths during buffering, achieved through fixed-length single-window encoding followed by multi-window temporal fusion. Inspired by the base-enhancement mechanism, DSTC quantizes residuals by adjusting quantization bit-width, providing multiple bitrate options while preserving perceptual quality. Finally, by reducing the combined selection space into two subspaces, TABR jointly optimizes latency and bitrate options to maximize QoE. Experimental results demonstrate that our strategy achieves over 94.4% compression, enables flexible scalability for both latency and bitrate, and significantly enhances QoE under dynamic network conditions.

Task-oriented communications for multi-modal applications emerge with the intelligence-oriented evolution of Internet of Things, where massive computation offloading with heterogeneous streaming requirements greatly challenges the existing mobile networks. Compared with semantic coding which reduces intra-modality redundancy by feature extraction, cross-modal coding further exploits inter-modality association and thus acts as a promising solution. However, unresolved information-theoretic issues hinder its full promise: 1) how to characterize the achievable region of cross-modal coding for task-oriented communications, and 2) to what extent can task-oriented communications benefit from exploiting inter-modality association. Therefore, this work first establishes a cross-modal rate-distortion function for task-oriented communications, and proves the feasibility of optimizing its information-bottleneck inspired transformation for guiding the design of learnable codec. In particular, the optimal feature representation is specified by a converging iterative solver under perfect statistical knowledge. Second, we prove a new bound on compression gains of cross-modal coding in task-oriented communications, based on a sufficient condition for cross-modal representation to be effective. Furthermore, a typical learnable codec is designed, whose loss function can be theoretically interpreted by our derived results. Finally, experimental evaluations verify the positive correlation between cross-modal coding gains and inter-modality association levels.

Multi-modal visual signals are prevalent in emergency communications. To ensure high reliability of signal transmission under bandwidth constraints, it is crucial to compress redundant information both within and between modalities as much as possible, and ensure the fidelity of the reconstructed signals. Most existing studies depend exclusively on single-modal coding schemes and fail to effectively leverage the semantic correlations between modalities. In this paper, we introduce an end-to-end general cross-modal visual coding scheme, namely CMVC, which aims to jointly compress multi-modal visual signals (such as visible and infrared signals). First, we propose a cross-modal asynchronous entropy module that extracts common features using a cross-attention mechanism. Additionally, we enhance the accuracy of common features extraction by maximizing mutual information loss. This module further compresses multi-modal visual signals by compressing only the residual features between modalities. Second, we propose a cascaded enhancement module based on cross-modal Mamba that fuses complementary information to enhance the reconstruction quality of multi-modal visual signals. Finally, extensive experimental results demonstrate that our scheme significantly outperforms other advanced methods on visible-infrared datasets. Even at low bitrates, multi-modal visual signals can still achieve excellent reconstruction quality. Additionally, our scheme exhibits outstanding compression and reconstruction performance when applied to visible-depth signals, effectively demonstrating its robustness and generalizability.

Extensive previous studies demonstrate that visual saliency prediction significantly reduces bandwidth consumption in immersive communications, which serve as one key application in 6G by providing multi-sensory interactive experiences. Unlike traditional visual saliency prediction models designed for specific scenarios (e.g., visual-only or visual-audio contexts), this paper proposes a general cross-modal saliency prediction method. The proposed method fully exploits inter-modal correlations, and importantly, explores whether incorporating increasing modalities always benefits visual saliency prediction. Specifically, we construct an open-source multi-modal saliency dataset containing audio, video, and haptic modalities. Next, we propose a cross-modal saliency prediction (CMSP) model, which is capable of leveraging various modal combinations to generate saliency maps. In particular, CMSP extracts correlations among multi-modal features across spatial-temporal and channel dimensions, enabling adaptive fusion among various numbers of modalities. The experimental results show that for visual saliency prediction, integrating additional modalities can improve performance, but it comes at the cost of increased computational complexity and inference time. Therefore, the selection of modalities should be based on the accuracy requirements and the available computational resources.

With the rise of the Tactile Internet, delivering real-time and high-fidelity tactile feedback is crucial for enhancing immersion in multimedia services. However, existing tactile coding methods fail to simultaneously adapt to the diverse delay requirements of multimedia services and the time-varying network bandwidth. To address these challenges, this paper proposes a Scalable Tactile Coding (STC) method, which provides flexible coding delays and bitrates across multiple levels while ensuring human perceptual quality. Specifically, we first propose a tactile coding framework based on a non-stationary autoencoder for efficient compression, which features both delay-scalable and rate-scalable advantages. Second, by trading the balance between delay and computation, we design a sliding window approach that utilizes overlapping coding to reduce buffer delay. This approach provides multiple delay options to accommodate diverse delay requirements, thus realizing the desired delay-scalable effect. Third, inspired by the base-enhancement strategy of scalable video coding, we design a non-uniform quantization method to compress residual signals, which can be leveraged to dynamically enhance tactile signal fidelity. By adjusting the quantization bit-width, this method provides multiple levels of bitrates, thereby achieving a rate-scalable effect. Extensive experimental results demonstrate that STC achieves a bitrate reduction of over 92.3 % while maintaining satisfactory perceptual quality. Additionally, STC further supports flexible bitrate adjustment and satisfy various delay requirements.

Traditional communication faces significant challenges in multimodal scenarios, including surging network capacity demands and the neglect of semantic value. Although semantic communication achieves data compression through semantic feature extraction and refinement, existing methods have drawbacks such as inflexible compression, high computational complexity, and the separation of feature extraction and refinement from transmission scheduling, making it difficult to trade-off semantic integrity and transmission efficiency. To this end, this paper proposes a task-oriented cross-modal semantic transmission scheme, which is based on the task requirements, dynamically adjusts the feature fusion strength and utilizes semantic correlations to enhance the task-related features, and then evaluates the feature priority and selects the task-critical features for transmission, realizing the best of both worlds. Specifically, we design a task feedback-based cross-modal feature fusion method, which establishes a mapping between computing state and feature fusion level, and dynamically optimizes the fusion weight decomposed by the fusion level through task loss. On this basis, the cross-modal features are aligned and complemented using semantic correlation to refine the task-relevant features. Further, we propose a feature priority-based multi-mode semantic transmission method, which determines the feature priority by a task-response-based feature importance assessment model. Accordingly, a reinforcement learning (RL)-based dual-modal feature selection strategy is designed to select task-critical features for reliable transmission by coupling transmission performance with task requirements. Additionally, simulation results show that compared with the baseline, our method improves task accuracy by 10.6% and reduces transmission latency by 7.2% on average.

Incorporating tactile feedback into traditional audio-visual multimedia services has gradually become the killer application in the Beyond 5G era. However, current state-of-the-art tactile coding approaches struggle to achieve extreme compression, thereby significantly impacting the transmission quality of visual streams due to resource competition. To address this challenge, this paper proposes an efficient cross-modal coding method that fully leverages semantic correlations between visual and tactile modalities. Specifically, we first construct a crossmodal coding architecture based on the predictive coding, enabling flexible and scalable bitrates to meet diverse compression requirements and adapt to dynamic network conditions. By introducing explainable surface semantics (e.g., friction, texture) as intermediates, we then associate visual modality with tactile modality to extract their potential correlations from the perspectives of multiple physical properties. Finally, we design a cross-modal feature fusion module through exploiting semantic correlations to further improve reconstruction quality of tactile signals, thereby reducing the bitrates required for tactile residuals and facilitating more efficient transmission. Experimental results demonstrate that the proposed tactile coding method achieves high bitrate compression, with almost no impact on visual stream quality.

Various distributed learning approaches emerge for enabling ubiquitous intelligence in Internet of Things (IoT) without sacrificing data privacy. To improve communication efficiency in frequent knowledge exchange over resource-constrained IoT, different techniques for client selection have been proposed. However, the intractable scalability issues remain to be addressed in massive IoT, since highly-coupled co-channel interference adds exponential complexity to combinatorial client selection. In this work, we develop a client selection framework highly-scalable to large-scale networks with thousands of devices, which exploits the inherent graph structure derived from knowledge exchange and co-channel interference. Specifically, we first model a client selection problem for jointly optimizing learning performance and system cost under volatile network conditions. The formulated problem is encoded into a node classification problem by a directed graph. Subsequently, a general yet simple solver is designed based on graph neural networks, which selects clients by classifying node status with recursive neighborhood aggregation of node representations. Finally, extensive experimental results demonstrate that the proposed approach can perform on par with state-of-the-art methods, while scaling to networks whose size is orders of magnitude larger than they can handle.

Robots equipped with multimodal sensing capabilities play an important role in the Internet of Robotic Things (IoRT), especially in emergency rescue. However, existing rescue robots pose challenges for human operators in achieving precise manipulation due to the absence of haptic signals. Additionally, limited and fluctuating bandwidth in emergency rescue renders current cross-modal haptic generation schemes ineffective. To overcome this dilemma, we propose a novel cross-modal haptic generation scheme that enhances scalability across diverse network conditions by leveraging correlations among audio-visual–haptic modalities. Specifically, we first propose an edge-device collaboration-based architecture that dynamically extracts semantics from audio-visual signals, tailored to the current network conditions, for haptic generation. This is achieved by predicting the network state at the edge and performing multimodal encoding and fusion at the device. Next, we design a scalable cross-modal haptic generation scheme that implements the optimal generation strategy based on received audio-visual semantics of different granularities, ensuring real-time acquisition of coarse-grained or fine-grained haptic feedback. Finally, numerical experimental results conducted on a multimodal dataset and a simulated emergency environment indicate that the proposed scheme reliably generates haptic signals in emergency rescue.

The expansion of the Industrial Internet of Things (IIoT), driven by informatization and intelligence, has made real-time monitoring for ensuring system safety and efficient production more critical than ever. However, in extreme industrial environments, harsh communication conditions and limited computing power impede reliable data transmission and processing, thus posing significant challenges to precise and continuous monitoring for massive IIoT. To this end, this paper proposes a digital twin (DT)-enabled hierarchical monitoring scheme by fully considering communication, computing, and control (3C) collaboration. Specifically, we first propose a DT-enabled 3C collaboration architecture, which builds an accurate digital representation of physical entities, and then completes the missing state based on shared-specific features to evolve continuously across the device lifecycle. Next, we design a multi-agent reinforcement learning (MARL)-based collaborative monitoring method, which formulates the joint optimization problem of compression rate and region assignment for monitoring nodes according to 3C performance. Then, we propose a mask state-assisted MARL scheduling method to refine the mask state space, take advantage of MARL’s distributed decision-making and centralized evaluation, and ensure timely monitoring of massive IIoT. Finally, we build a mine industry simulation platform and verify the effectiveness of the proposed method. The numerical results demonstrate a marked improvement in monitoring utility over traditional methods.

The metaverse services are promising to embrace multi-sensory experiences of human beings, which mainly include audio-visual and tactile senses. From the perspective of wireless transmission, tactile transmission requires ultra-reliable low-latency communications, while audio-visual transmission requires enhanced mobile broadband communications. Besides, the audio-visual segment can be divided into several correlated data packets, any loss of packets would result in failed decoding at users, thus degrading users’ immersive experiences. In multi-user wireless metaverse systems, the heterogeneous transmission characteristics of multi-modal streams and integrity requirements of audio-visual stream transmission pose a great challenge to the limited wireless resource scheduling. To this end, we design a multi-user resource schedule scheme for multi-modal stream transmission by jointly considering the integrity of audio-visual stream transmission and the puncturing-based tactile stream transmission. We model the multi-modal perception utility function based on the multi-attribute utility theory and wireless transmission performance of multi-modal streams. Then, we formulate the average multi-modal perception utility maximization problem, and we adopt the Lyapunov theory to decompose the original maximization problem. Furthermore, we integrate the matching-based two-timescale spectrum resource allocation algorithm and alternating direction method of multipliers-based power allocation algorithm to obtain the optimal spectrum and power allocation strategies. Simulation results show that, compared with the resource allocation scheme without considering the transmission integrity, the average multi-modal perception utility of the proposed scheme is maximumly improved by 25%.

To fulfill the demands of emerging multi-modal services, the cross-modal semantic communication paradigm comes into being. It fully utilizes potential semantic correlations among modalities to address polysemy and ambiguity issues, enhancing transmission reliability. However, applying cross-modal semantic communication in resource-constrained mobile scenarios introduces new challenges, including radio spectrum bandwidth limitations and fluctuations for the transmitter, and computing resource constraints for the receiver, which leads to potential transmission failures. To bridge this gap, this paper proposes a cross-modal semantic transmission strategy for mobile scenarios (MobileCMST). We first construct the framework for MobileCMST. Within this framework, a semantic encoder is designed to achieve redundancy elimination for visual and haptic signals. Then, a semantic delivery approach is developed to cope with bandwidth fluctuations and multipath fading channels. Finally, an efficient semantic decoder based on a visual-haptic semantic-integrated diffusion model is proposed. It employs the Mamba backbone to reconstruct high-quality signals with lightweight computational complexity. Extensive experiments demonstrate the excellent performance of the proposed MobileCMST strategy in resource-constrained mobile scenarios.

Multi-modal services that integrate signals such as audio, video, and haptic are poised to dominate the 5G and beyond era. Due to the presence of inter-modal redundancy and interference, it is highly challenging to simultaneously meet the demands for real-time, reliability, and high capacity in multi-modal services. Cross-modal communication, taking full use of inter-modal correlations, has been viewed as a promising solution. This paper aims to provide theoretical support for this approach by addressing two key issues: i) the benefits of cross-modal source coding (CMSC) in eliminating inter-modal redundancy, and ii) the benefits of cross-modal channel coding (CMCC) in alleviating inter-modal interference. Specifically, we first construct a cross-modal communication system model to illustrate how inter-modal correlations can be leveraged by incorporating common semantic information in both source coding and channel coding. Next, we analyze the reduction in error probability for a target modality’s symbols at the same coding rate and coding delay, both with and without assistance from other modalities and common semantics, to quantify the benefits of CMSC. Finally, we analyze the differences in achievable channel degrees of freedom and sum capacities under the same channel conditions, both with and without the utilization of common semantics, to quantify the advantages of CMCC. In the three case studies, numerical results validate the theoretical soundness of CMSC and CMCC for binary sources over a Gaussian channel, as well as their feasibility on a practical audio, video and haptic teleoperation platform.

With the rise of the Internet of Everything, machine-type wireless video applications like intelligent surveillance are increasingly becoming mainstream network services. However, the massive transmission of video streams heavily burdens wireless networks. Existing methods typically establish a correlation between the transmitted content and the quality of experience or service, making it challenging to balance trade-offs among decision quality, bandwidth utilization, and latency requirements. To address this issue fundamentally, this paper proposes a novel perspective by designing wireless video transmission strategies from the angle of decision quality, where only the content that impacts the decision outcomes is transmitted. The highlight of this paper is designing a lightweight yet efficient binary classifier that predicts whether the current content will change decision outcomes based on content discrepancies, measured only through pixel-level differences and macroblock-level similarity. Furthermore, these discrepancies can be used to distinguish the background, thereby further saving bandwidth by reusing the background information. Additionally, a fine-tuning module is incorporated to flexibly update the classifier model, ensuring adaptability across various scenarios. Extensive results demonstrate that the proposed strategy achieves over 81% redundant bandwidth savings while maintaining decision quality and catering to strict latency requirements on computation-constrained devices.

Wireless emergency networks play a crucial role in natural disasters, enabling seamless communication between explorers (e.g., rescue robots) and remote command centers. However, unstable communication links and limited computational resources hinder explorers from directly transmitting massive real-time content for remote human observation or locally computing the latest detection results. To address this challenge, this paper introduces a cross-modal semantic communications paradigm, where our highlights are characterized by precise semantic extraction, low-complexity implementation, and progressive semantic transmission. Specifically, by simplifying the desired task from human-oriented signal recovery to the machine-oriented decision, we firstly deploy a lightweight cross-modal semantic encoder on the explorer, which precisely extracts compact semantics relevant to the decision-making based on inter-modal correlations. Then, to mitigate the impact of intermittent network connections, we develop a scalable semantic transmission strategy that encodes extracted semantics as base and enhanced semantics, progressively delivering them once the network becomes active. Numerical results indicate remarkable benefits of cross-modal semantic communications in terms of decision accuracy, model size, and transmission latency.

Integrating visual-haptic remote control is an important application direction in the Industrial Internet of Things (IIoT). Cross-modal communications are considered to be an effective technology to support this application. However, due to limited bandwidth and competition between modalities, the quality of visual transmission and the end user’s immersive experience cannot be guaranteed in practical scenarios. To overcome this dilemma, this paper proposes a super-resolution reconstruction strategy for cross-modal communications. Specifically, the sender only transmits low-resolution images and haptic signals, while a haptic-aided super-resolution reconstruction (HaSR) approach is designed at the receiver. This approach involves semantic correlation-based modal fusion and generative adversarial principle-based visual generation, which enable the reconstruction of high-resolution images using the received low-resolution images and haptic signals. Experimental results from a standard dataset and a practical remote industrial control platform validate the effectiveness of the proposed strategy.

Multi-modal services, integrating various modalities such as audio, visual, and haptic, have emerged as leading multimedia applications in the 5G era and beyond. To fulfill the demands for low latency, high reliability, and large capacity, cross-modal transmission schemes have been proposed. Typically, these schemes emphasize on either audio-visual or haptic modality, and prioritize flawless transmission of one modality to assist the other modality streaming. However, these prerequisite and assumption do not hold for generic multi-modal services and communication environments, where determining the priority of modality and guaranteeing flawless transmission becomes challenging. To address this fundamental problem, in this paper, we introduce a strategy toward generic cross-modal transmission, enabling visual and haptic modalities to assist each other as needed. The strategy includes a visual-haptic mutual stream delivery mechanism at the sender and a visual-haptic mutual signal reconstruction approach at the receiver. The former aims to eliminate redundancy in visual and haptic streams through mutual assistance, while the latter adaptively handles impaired, missing, or delayed visual or haptic signals by leveraging modality-aware knowledge transfer and semantic-aware signal generation techniques. The proposed strategy demonstrates excellent performance through experiments conducted on a standard multi-modal dataset and a practical visual-haptic communication platform.

To ensure the users’ immersive experience in cross-modal communication, overcoming the end-to-end (E2E) latency through prediction has attracted attention and shown its superiority. However, existing prediction schemes encounter formidable challenges in the presence of multi-modal signals, primarily to adapt and satisfy the prediction requirements of diverse multi-modal services, as well as to fully exploit and effectively utilize the correlation features of multi-modal signals for precise prediction. To this end, this work presents a flexible prediction scheme for low-latency cross-modal communication. Specifically, we first propose an adaptive prediction-aware cross-modal communication framework, which reduces the delay by predicting and transmitting the future multi-modal signals in advance, and flexibly adjusts the prediction horizon to satisfy the prediction accuracy of different multi-modal services. Next, we design an information gain-assisted graph attention (IGGA) method for cross-modal signal prediction, which leverages the graph attention block to extract the intra-modal, inter-modal spatial and temporal correlation features, and effectively optimize and utilize these features with the information gain (IG), thereby facilitating precise cross-modal signal prediction. Finally, numerical experiments conducted on a self-built dataset, a public dataset, and a multi-modal acupuncture platform demonstrate the superiority of the proposed scheme in low-latency cross-modal communication.

The multi-modal robotic teleoperation, as an important application in human-computer interaction (HCI), is playing a significant role in various domains such as industry, healthcare, and education. However, existing robotic teleoperation systems face significant challenges with multi-modal signals, primarily in designing a cross-modal communication architecture that caters to diverse modal requirements and ensuring high-quality cross-modal signal reconstruction even in poor network conditions. To this end, this work proposes a general cross-modal signal reconstruction scheme by taking full advantage of the correlation among different modality signals. Specifically, we first propose a scalable cross-modal communication architecture that meets the diverse needs of various modality signals using multi-modal encoding and multi-directional decoding, eliminating the need for a specialized feature extraction model. Next, we design a masked auto-encoder with discriminator assistance (MAE-D) cross-modal signal reconstruction method, which leverages the idea of generative confrontation by combining the codec for signal reconstruction with the discriminator responsible for assessing the authenticity of the reconstructed signal to achieve accurate and efficient cross-modal signal reconstruction. Finally, numerical experiments conducted on our self-built multi-modal dataset, a public dataset, and a teleoperation simulation platform demonstrate that the proposed scheme offers significant advantages in cross-modal signal reconstruction.

With the explosive growth of online multi-modal applications that typically include audio, video, and haptic signals, immersive experience (IE) improvement has been broadly regarded as one of the most important tasks. Compared with traditional quality of experience (QoE) improvement for online audio/video applications, it highlights two sequential technical challenges to be resolved: i) much more stringent demand of real-time improvement due to the incorporation of delay-sensitive haptic signals, and ii) high-dimensional instead of existing one-dimensional (i.e.,network-level) paradigm for better online improvement. To get over this dilemma, this work systematically addresses the following three fundamental problems: i) which factors influence IE, ii) how to online improve IE, and iii) to what extent of the corresponding IE improvement can be achieved. To this end, we first comprehensively explore and categorize the influence factors on IE from various dimensions. Then, by combing network resource scheduling with the multi-domain collaboration of user profile, device specification, and application type, an online IE improvement strategy is proposed based on the efficient linear contextual bandit with the$L_{1}$-norm estimation. Finally, we derive the theoretical bound of IE improvement, scaling at a poly-logarithmical function of data dimension. Numerical results on the practical system also demonstrate the remarkable improvement on IE.

Multi-modal applications are expected to dominate in the 5G and B5G era. However, traditional source coding methods are not efficient or reliable due to neglecting semantic redundancy and mutual influences between different modalities' sources. To address this, cross-modal source coding (CMSC) has been proposed as a promising solution. However, there are still two main challenges: determining the optimum rate of CMSC considering delay and reliability constraints, and designing a practical CMSC near the optimum rate. To tackle these challenges, this paper focuses on studying the optimum source coding rate of CMSC and its practical implementation. On the theoretical side, an$(n,\epsilon)$-achievable rate region is derived, representing the source coding rates subject to a fixed blocklength$n$and the target error probability$\epsilon$. Additionally, the optimum source coding rate can be approximated by calculating the infimum of the$(n,\epsilon)$-achievable rate region with a rate dispersion function. On the technical side, a general implementation for CMSC is proposed, which fully leveraging channel coding and artificial intelligence (AI) semantic analysis to achieve the optimum rate. Numerical results demonstrate that CMSC can obtain 50% improvement in theory and 37.5% enhancement in practice against the baseline model abstracted from traditional schemes when multi-modal sources are semantically correlated.

In fine-grained visual categorization (FGVC), most part-based frameworks do not work effectively in some extremely challenging scenarios such as partial occlusion. This limitation is due to the heavy disorder of local features extracted from such occluded targets. To address this issue, we propose a global information-assisted network (GIAN), where auxiliary global information can search the useful elements of local information and integrate with them for an efficient unified feature representation. In particular, in order to acquire the global information, we design a global attention-concentrated convolutional neural network (GAC-CNN) by extending a convolutional neural network with a nonlocal GCN module. Then, the unified feature representation is produced by two strategies. On the one hand, a global–local aggregation strategy is developed to selectively integrate global features with local features through consistency evaluation and reweighting method. On the other hand, an alternative knowledge distillation strategy is developed to help generate more powerful global and local features. Two strategies collaboratively make the unified features more robust and more discriminative than traditional part-based features. Experimental results show that the proposed GIAN can achieve accuracies of 92.8%, 93.8%, and 95.7% on CUB-200-2011, FGVC Aircraft, and Stanford Cars, respectively.

Digital twin (DT) has emerged as a key enabler for the intelligent-oriented evolution of mobile networks. With the rise of privacy concerns for enabling intelligent applications in DT-empowered mobile networks (DTMNs), federated learning has garnered wide attention due to its potential on breaking down data silos. However, the data privacy of federated learning is greatly threatened by emerging gradient leakage attacks, and the need for frequent knowledge exchange limits its training efficiency over resource-constrained DTMNs. To circumvent such dilemmas, this work first proposes a privacy-enhanced federated learning framework based on cloud-edge-client collaborations. Particularly, model splitting between clients and edge servers makes gradient leakage attacks computationally prohibitive, and cloud-side partial model aggregation provides hierarchical data utility. To improve the training efficiency of the proposed learning framework, we further establish its communication and computation cost models, and develop a DT-assisted multi-agent deep reinforcement learning-based resource scheduler for joint client association and channel assignment. Finally, as a case study of intelligent applications in DTMNs, a human-robot collaborative nursing task is designed to evaluate the practical performance of our proposed scheduler. Experimental results show its superiority in saving training costs and preserving learning accuracy.

With the widespread prevalence of smart devices, mobile crowdsensing (MCS) becomes a new trend to encourage mobile nodes to participate in cooperative data collection in various Internet of Things (IoT) applications. In location-dependent MCS, location information of mobile nodes are collected and analyzed by service provider to assist in task allocation. If the service provider is not fully trusted, mobile node's privacy is leaked and accessed by unauthorized parties. How to preserve privacy while maintaining task allocation accuracy and efficiency becomes challenging. To this end, we propose a learning-based mechanism that involves two parts: 1) privacy-preserving task release and task allocation; 2) accurate and efficient task allocation. In the first part, we design a location-based symmetric key generator, which enables two parties to self-generate a symmetric key without depending on fully trusted authorities. By utilizing this key generator and Proxy Re-encryption, we propose a privacy preserving protocol to protect location information in task release and task allocation. In the second part, we design a reinforcement learning based task allocation algorithm to optimize the winners selection, which obtains high accuracy and efficiency. The performance analysis reveals that our proposed mechanism achieves accurate and efficient task allocation while preserving privacy in location-dependent MCS.

Cross-modal communications, devoting to collaboratively delivering and processing audio, visual, and haptic signals, have gradually become the supporting technology for the emerging multi-modal services. However, the inevitable resource competitions among different modality signals as well as the unexpected packet loss and latency during transmission seriously affect quality of the received signals and end user's immersive experience (especially visual experience). To overcome these dilemmas, this paper proposes a cross-modal signal reconstruction strategy from the perspective of human's perceptual facts. It tries to guarantee visual signal quality by considering potential correlations among modalities when processing audio and haptic signals. On the one hand, a time-frequency masking-based audio-haptic redundancy elimination mechanism is designed by resorting to the similarity of audio-haptic characteristics and human's masking effects. On the other hand, based on the fact that non-visual perception can assist to form and enhance visual perception, an audio-haptic fused visual signal restoration (AHFVR) approach for handling the impaired and delayed visual signals is proposed. Experiments on a standard multi-modal database and a constructed practical platform evaluate the performance of the proposed perception-aware cross-modal signal reconstruction strategy.

With the haptic technology continuously enlarging the eHealth Industry Internet of Things (IIoT) ecosystem, haptic perception service which requires effective haptic signal reconstruction for immersive experience has become an indispensable function. However, the majority of existing haptic signal reconstruction methods are generally inefficient because of undergoing extremely complex operations or inefficient feature representations. To resolve this dilemma, this article proposes a long short-term memory-based force reconstruction network (LSTM-FRN) by designing a novel sparse attention module for low-latency reconstruction and a novel metric learning-based constraint for high-precision reconstruction, yielding an excellent tradeoff between the computational complexity and feature representation. To train our network, we construct a large-scale data set of synchronous needle motion signals and haptic signals in acupuncture needle insertion. Finally, we build an interactive needle insertion training system (HapAR-NITS) by integrating augmented reality (AR), the LSTM-FRN-based haptic reconstruction as well as a skill assessment subsystem. Comprehensive experiments demonstrate that the proposed multiple technologies enable our HapAR-NITS to achieve satisfying immersive experience and manipulation effects.

The Internet of Things (IoT) is evolving into the Internet of Video Things (IoVT) that supports massive smart devices and multiple video applications. However, how to effectively control massive devices and transmit large-volume video data have become challenges in the current IoVT. Inspired by device-to-device (D2D) communications and coalitional game, this article constructs a self-organized D2D collaborative video content sharing framework for the IoVT. Specifically, we first propose a collaboration mechanism by introducing the social attributes of IoVT devices and their owners. In this mechanism, D2D collaborative coalitions are automatically formed among IoVT devices and video content is shared in the coalitions through D2D links. In this way, the burden of controlling massive IoVT devices and video data traffic are offloaded. Then, by integrating the scalability of the scalable-high-efficiency-video-coding (SHVC) streams and the flexibility of D2D networking, a collaborative video streaming strategy is developed. It takes advantage of provider set arrangement and transmission scheduling to reduce the impact of network instability on video services. Simulation results verify the effectiveness of the proposed mechanism and strategy.

Personalized content sharing will inevitably become one of the core applications of mobile Internet of Things. However, the existing strategies for content sharing are far from effective content personalization, since they either fail to protect the diversity of shared content or harm the enthusiasm of users to participate in cooperation. How to optimize the tradeoff between content personalization and sharing efficiency thus becomes an extremely challenging problem. To circumvent this dilemma, we propose a social-aware personalized content-sharing strategy based on mobile crowdsensing (MCS), which specially introduces positive network externalities derived from MCS and the social network. Specifically, we design a two-stage pricing-participation game to model the interactions between mobile users and a profit-making service provider. By solving the subgame-perfect Nash equilibrium (NE) of the proposed game, an efficient participation mechanism and an optimal-pricing strategy are developed. First, users’ decision selection of whether to join MCS is modeled as a social-aware MCS participation game (SA-MPG), and two algorithms for solving the Pareto-optimal NE of SA-MPG are designed. Subsequently, the pricing issue for network operators is investigated by exploiting the supermodularity of SA-MPG. Stochastic network model and real-world data set-based simulations corroborate the significant gain of our proposed strategy.

Machine-type communication (MTC) has been considered as one of the key enablers for intelligent Internet of Things (IoT) applications. However, existing evaluating metrics, no matter the Quality of Service (QoS) or Quality of Experience (QoE), cannot truly and accurately reflect the quality of MTC when it serves machine decision making. To address this problem, this work proposes a new performance index, Quality of Decision (QoD), to elegantly capture the essence of MTC and precisely depict the corresponding functionality. First, the physical components and logical processes that affect the quality of MTC in analytics-oriented scenarios are carefully studied. Second, through jointly considering the factors of the commonness and individuality, we propose a layered QoD framework capable of independently evaluating and monitoring the quality of data to be acquired, delivered, and processed for enabling machine decision making. Third, we design a typical QoD-driven transmission scheme for video analytics by avoiding over provisioning of sensing and communication capabilities, which shows super efficiency compared to traditional human-perception-oriented approaches. We believe that the utilization of QoD will significantly promote the application of MTC in building various intelligent IoT systems.

Medical robots with audio-video-haptic streams, as indispensable devices for remote healthcare, are playing ever-increasing roles in mitigating the spread of infectious diseases. However, existing medical robots are far from precise and efficient because of the following two technical challenges, including i) how to ensure the haptic fidelity for precise manipulation, and ii) how to alleviate the impact of haptic streams on the quality of visual navigation for efficient operation. To this end, this work explores the benefits of edge-based cross-modal communications (CMCs), which take full advantage of potential correlations among different modalities’ streams, to realize high reliability and throughput. Specifically, to compensate for the reliability loss caused by wireless transmission, a semantic-aided cross-modal reconstruction framework is firstly designed at edge nodes for high haptic fidelity. Then, a user experience-driven stream scheduling strategy is developed to enhance the visual quality by fully leveraging edge computing and network slicing. In particular, different from traditionally interrupting audio/video stream transmission to prioritize haptic streams, we jointly schedule resources to different modalities’ streams via estimating haptic arrival time. Finally, as a classical case study, we independently construct a remote throat swab sampling platform based on CMCs to evaluate practical performance, and numerical results indicate the significant improvements in terms of various metrics.

Multi-modal services, typically integrated by visual, audio, and haptic signals, have been considered as promising killer services in 5G and beyond 5G era. However, due to essential difference among these signals, how to guarantee quality of multi-modal services is a significant technical challenge. Existing transmission schemes, which deliver and process each modality signal separately, cannot meet such requirements as low latency, high reliability, and high throughput. To get over the dilemma, this paper proposes a general cross-modal transmission strategy by taking advantage of the potential correlation among modalities, which consists of a delivery mechanism at the sender and a signal restoration procedure at the receiver. On the one hand, a visual-aided haptic content compression method is designed for the delivery mechanism. By utilizing the category correlation among modalities, haptic signals with similar visual content can be effectively compressed, reducing transmission burden. On the other hand, a fine-grained haptic to image synthesis (FHIS) approach is proposed for realizing signal restoration. Through exploring strong matching properties among modalities, the FHIS can restore the impaired, missing, delayed visual images from the received haptic signals. Experiments on a standard visual-haptic database and a practical platform evaluate the performance of the constructed cross-modal transmission strategy.

Multi-modal services, typically integrating such signals as audio, video, and haptic, will become an inevitable application trend of the 5G and beyond. However, due to the essential differences among the haptic and audio/video signals, the existing coding schemes usually fail to satisfy the critical requirements in terms of the rate distortion performance. Inspired by the phenomenon that hearing, sight and touch are highly correlated, we provide an affirmative answer by proposing the framework of cross-modal coding, which compresses multi-modal signals aided by their semantic correlation. In particular, the highlights of this work lie in addressing three fundamental technical problems: i) how to exploit the semantic correlation among different modalities, ii) to what extent of benefit we can get from cross-modal coding, and iii) how to design a general cross-modal codec. On the theoretical end, we determine the minimum number of bits required to compress haptic signals under the rate conditions of video streams through investigating their semantic correlation. On the technical end, we design a general cross-modal codec to approach the optimal compression limit by using the AI-enabled cross-modal prediction and channel coding. Numerical results demonstrate that the proposed cross-modal coding can achieve significant benefits relative to the existing schemes, especially when multi-modal signals have strong semantic correlation.

Distributed learning is proposed as a promising technique to reduce heavy data transmissions in centralized machine learning. By allowing the participants training the model locally, raw data is unnecessarily uploaded to the centralized cloud server, reducing the risks of privacy leakage as well. However, the existing studies have shown that an adversary is able to derive the raw data by analyzing the obtained machine learning models. To tackle this challenge, the state-of-the-art solutions mainly depend on differential privacy and encryption techniques (e.g., homomorphic encryption). Whereas, differential privacy degrades data utility and leads to inaccurate learning, while encryption based approaches are not effective to all machine learning algorithms due to the limited operations and excessive computation cost. In this work, we propose a novel scheme to resolve the privacy issues from the anonymous authentication approach. Different from the two types of existing solutions, this approach is generalized to all machine learning algorithms without reducing data utility, while guaranteeing privacy preservation. In addition, it can be integrated with detection schemes against data poisoning attacks and free-rider attacks, being more practical for distributed learning. To this end, we first design a pairing-based certificateless signature scheme. Based on the signature scheme, we further propose an anonymous and efficient authentication protocol which supports dynamic batch verification. The proposed protocol guarantees the desired security properties while being computationally efficient. Formal security proof and analysis have been provided to demonstrate the achieved security properties, including confidentiality, anonymity, mutual authentication, unlinkability, unforgeability, forward security, backward security, and non-repudiation. In addition, the performance analysis reveals that our proposed protocol significantly reduces the time consumption in batch verification, achieving high computational efficiency.

The emerging multi-modal services, characterized as the integration of audio, visual, and haptic signals, will become the killer applications in 5 G and beyond 5 G era. In order to support multi-modal services, cross-modal communications come into being. However, when adopting cross-modal communications to haptic-dominant multi-modal services, there still face several technical challenges. On the one hand, haptic signals are very sensitive to interference and easy to be damaged or even missing during transmission. On the other hand, it needs to generate virtual haptic signals when real touch sensory information is hard to be gathered. To get over the dilemma, this paper proposes a haptic signal reconstruction strategy for cross-modal communications. First, a cloud-edge collaboration-based cross-modal communication architecture is constructed. Then, an audio-visual-aided haptic signal reconstruction (AVHR) approach under this architecture is designed by leveraging the potential correlation among modalities. It can be further divided into three components: feature extraction by cloud-edge transfer, shared semantic learning by multi-modal fusion, and haptic signal generation by semantic constraints. Finally, experiments on a standard audio-visual-haptic dataset and a practical cross-modal communication platform show that the proposed AVHR approach has better reconstruction performance when compared with the competing schemes.

Integrated sensing, communication, and computation (ISCC) will become a key enabler for automation applications. However, since the performance region of ISCC has a higher dimension than those of traditional wireless networks, existing schedulers typically fail to simultaneously meet the heterogeneous requests in ISCC. In this work, we propose a novel wireless scheduling architecture to explore the coordination gains of sensing, communication, and computation from a perspective of joint optimization. Specifically, we first construct an implementation framework of ISCC by combining the mobile edge computing paradigm with the integrated sensing and communication technology, where the inherent tradeoff between sensing, communication, and computation performance is characterized. Next, a joint device association and subchannel assignment problem is formulated to capture the network externalities induced by resource competition among mobile devices with multi-functional requirements. Due to its intractability, we then reformulate it in the matching theoretical manner. To obtain a mutually satisfactory solution under externalities, an iterative matching algorithm is developed by introducing pairwise stability and proved to be convergent and stable. The extensive simulations elucidate the significant superiority of our proposed scheme over those externality-unaware wireless schedulers.

With the advancement of Internet of Things (IoT) and computing paradigms, massive data are collected and processed to enhance intelligent applications. However, by deliberately sending some queries, an attacker may be able to derive the sensitive information of IoT data owners. To prevent privacy leakage during IoT data query, differential privacy (DP) hides private information by introducing noise to the query results. As DP introduces randomized noise that will affect query accuracy (data utility), the tradeoff between privacy preservation and data utility is a challenge. In this article, we first propose a novel optimization framework for single query to minimize the privacy cost, while satisfying both personalized DP and customized data utility. We design a reinforcement learning-based algorithm for single query optimization framework (SQOF_RL) to solve the optimization problem efficiently. Then, we propose a SQOF_RL and SVT-based batch query optimization mechanism (S2BQOM) to answer more queries privately. The performance evaluation shows that SQOF_RL and S2BQOM can effectively optimize single query and batch queries in terms of privacy cost, data utility, personalized privacy, and query satisfaction. Finally, the performance analysis reveals that our work can be applied to multiple linear/nonlinear query functions instead of one particular query function.

Win-win cooperation has been broadly treated as one of the most promising goals for device-to-device (D2D) content sharing, especially for the ultrareliable low-latency communications (URLLC) scenario. Unfortunately, the exiting solutions are built on several seemly unpractical conditions: 1) abundant prior information on network and user; 2) optimization variables limited to 1-D space; and 3) a preset order on users' decision making. In this work, we propose a win-win-driven D2D content-sharing scheme by exploring the blind matching theory. Specifically, we first derive the closed-form expressions of latency and reliability performance for D2D content-sharing scenarios. Accordingly, the URLLC-oriented joint optimization problem for provider-demander pairing and power control of potential providers is formulated as a two-sided one-to-one context-free matching game, which involves a collection of agreement functions of potential providers and demanders' aspiration levels and exploits a modified notion of pairwise stability as the solution concept. Then, we design a distributed algorithm by utilizing the market and information decentralization characteristics of the blind matching algorithm. Both theoretical analysis and numerical results validate the performance properties, including convergence, optimality, and complexity.

Cross-modal applications that elaborately integrate audio, video, and haptic streams will become the mainstream of the eHealth systems. However, existing stream schedulers usually fail to simultaneously meet the cross-modal transmission requests in terms of low latency, high reliability, high throughput, and low complexity. To circumvent this dilemma, this article proposes a general cross-modal stream scheduling scheme by fully taking advantage of the characteristics of different modal streams and their underlying temporal, spatial, and semantic relevance. Specifically, we first propose a hierarchical stream category framework, in which the transmission priority of the modal stream instead of the data flow can be flexibly settled. Next, we design a series of modal-aware stream scheduling schemes by jointly making use of the network slice and mobile edge computing to achieve the tradeoff among the various metrics. Importantly, the transmission strategy can be adjusted adaptively to realize the optimal resource allocation. Subsequently, we analyze the relationship among the user experience, multi-modal impact, and stream scheduling through investigating the interacted impacts among the different modal streams, then develop a user experience based scheduling switch strategy to improve the application generality and reduce the performance fluctuation. Numerical objective and subjective results demonstrate the efficiency of the proposed cross-modal scheduling scheme.

Cross-modal communication is playing an increasingly important role in improving receivers' immersive experience. The main challenge lies in ensuring the heterogeneous requirements of the cross-modal stream. Especially, the discontinuity of the received haptic signals caused by the delay should be eliminated. Unfortunately, existing schemes study the cross-modal stream separately, which leads to that the haptic signal is distorted, or the audio-visual quality is reduced. To solve this problem fundamentally, we propose a joint transmission framework combining prediction and device-to-device (D2D) communication by taking advantage of the correlation of the haptic signals and the proximity feature of the receivers. Specifically, on the theoretical end, to completely eliminate the discontinuity, we propose a prediction mechanism by predicting and sending the future signals in advance. To compensate for the reliability loss brought by prediction, D2D links are efficiently established on the receivers' side. On the technical end, we first design a minimum resource (e.g., power) consumption search algorithm based on the binary search to obtain the optimal prediction horizon. Moreover, we develop a simple but efficient transmission mode selection algorithm based on the Hungarian algorithm. Experimental results demonstrate the advantages of our proposed scheme in saving the power consumption.

Currently, video streaming services put more emphasis on user feeling or satisfaction than before. How to design suitable model and algorithm to effectively and efficiently evaluate user quality of experience (QoE) has become a significant technical challenge. To get over this dilemma, this paper proposes the broad forest, a non-neural network style broad model for streaming video QoE evaluation. The design target of broad forest is to take advantage of representation potential of forest and low complexity characteristic of broad learning system. Specifically, we first give the construction of broad forest. Then, the associated incremental learning algorithm for efficiently supporting the added structure and inputs is designed. Finally, we apply the broad forest to streaming video QoE evaluation. Experimental results show that the broad forest can not only guarantee accuracy, but also decrease training time. When subjective features are considered, it can further promote performance of QoE evaluation.

The cloud-based Internet-of-Things (IoT) has been applied to support ubiquitous data collection and centralized data processing among various applications. Equipped with powerful resources, a semi-trusted cloud is able to deduce private information by launching inference attack. Homomorphic Encryption (HE) has been proposed as an effective way to preserve privacy from inference attack while allowing certain computation over ciphertext. However, HE leads to longer latency due to additional communication and computation overheads. In this paper, we propose an optimization framework in privacy-preserving access control under cloud-fog computing systems. The optimization goal is to maximize the average user satisfaction in the system, where cost and latency serve as key metrics measuring user satisfaction. Due to the NP-hardness of the formulated problem, we propose a low-complexity suboptimal algorithm to solve it, where the access offloading decision making, user cooperation, and resource allocation are considered. Simulation results are presented to show the advantages of our proposed algorithm in terms of the average USI (User Satisfaction Index) and the number of users with zero USI.

The emergence of a massive Internet-of-Things (IoT) ecosystem is changing the human lifestyle. In several practical scenarios, IoT still faces significant challenges with reliance on human assistance and unacceptable response time for the treatment of big data. Therefore, it is very urgent to establish a new framework and algorithm to solve problems specific to this kind of fast autonomous IoT. Traditional reinforcement learning and deep reinforcement learning (DRL) approaches have abilities of autonomous decision making, but time-consuming modeling and training procedures limit their applications. To get over this dilemma, this article proposes the broad reinforcement learning (BRL) approach that fits fast autonomous IoT as it combines the broad learning system (BLS) with a reinforcement learning paradigm to improve the agent's efficiency and accuracy of modeling and decision making. Specifically, a BRL framework is first constructed. Then, the associated learning algorithm, containing training pool introduction, training sample preparation, and incremental learning for BLS, is carefully designed. Finally, as a case study of fast autonomous IoT, the proposed BRL approach is applied to traffic light control, aiming to alleviate traffic congestion in the intersections of smart cities. The experimental results show that the proposed BRL approach can learn better action policy at a shorter execution time when compared with competing approaches.

Personalized networking video service, as an inevitable trend recently, has become the core part for users' quality of experience (QoE) improvement. Unfortunately, existing schemes of QoE improvement are far from personalized since most of them only focus on network-level or user-level optimization. How to realize personalized QoE improvement for networking video service has been widely considered as a fundamental technical challenge. To get over this dilemma, this work proposes a personalized QoE improvement scheme by fully taking advantage of the time-varying influences on users' QoE, including user-awareness, device-awareness and contextawareness. The highlights of this work lie in that, the proposed scheme realizes the personalization comprehensively considering all these three dimensions, meanwhile, it is so robust that can be applied to the application scenario where the observable users' data is not sufficient. Specifically, we firstly design a comprehensive data collection strategy and accurately classify these collected data. Then, an efficient deep learning (DL)-based model for personalized characteristics extraction is proposed to precisely characterize personalization with temporal, spatial and periodic correlations. Subsequently, to resolve the data sparsity issue, a federated learning (FL)-based architecture with privacy-protection is designed by securely exchanging encrypted parameters with other users. Importantly, we design an optimization scheme based on comprehensive MOS formula for personalized QoE improvement. Experimental results demonstrate that the proposed scheme has a significantly better performance on the personalized QoE improvement.

The cooperative D2D content sharing mode is that multiple users can share content with each in a cooperative way. In this mode, we can improve the transmission efficiency and transmission stability of communication. Although there are many literatures on cooperative D2D, the mobility problem in this transmission mode has been ignored by many people. Moreover, traditional D2D communication is a one-to-one transmission mode, but cooperative D2D is a many-to-one transmission mode. There is a big difference of link establishment and content transmission between these two modes. And the impact of mobility on network topology is also very different. Therefore, the existing caching scheme and retransmission mechanism for D2D can not be applied to cooperative D2D. Meanwhile, whether in D2D or cooperative D2D, the success of the user requesting content is closely related to the social relationship between users, due to the social selfishness of D2D users. Therefore, in order to improve the performance of cooperative D2D content sharing, we exploit user interest similarity and mobility and propose a social-and-mobility-aware caching strategy for collaborative D2D scenarios. Not only that, we model the retransmission problem as a Knapsack problem and design the retransmission mechanism when the transmission link is interrupted to ensure the reliability of content sharing with the greedy algorithm. Finally, our simulation results show that our proposed caching placement scheme and retransmission scheme can improve the performance of cooperative D2D content sharing. In addition, we achieve a valuable caching guideline in cooperative D2D scenarios.

More and more network media users concern about their privacy issues since they know that their network behaviors are being observed, and thus the observable users' data are not reliable and sufficient in this case. How to evaluate the true quality of experience (QoE) of the privacy-aware users has become a significant technical challenge because of the most majority of existing data-driven QoE evaluation schemes based on the premise of the true and adequate users' observations. To get over this dilemma, this paper proposes a systematic and robust QoE evaluation scheme with unreliable and insufficient observation data. Specifically, we first translate the subjective privacy-aware QoE evaluation problem into an objective rational user analysis procedure. Then, a semantics-based similarity measurement for multidimensional correlation analysis is constructed to classify the observable data. Subsequently, the highlight of this paper lies in proposing a class-level joint user classification and data cleaning strategy by frequently updating the training processes. Through elaborately designing an iterative framework, it can effectively resolve the data sparsity and inconsistency problems due to the user privacy-aware preferences. Importantly, we also introduce an efficient QoE model construction method for online implementation, and numerical results validate its efficiency for different kinds of privacy-aware users.

With the increasing popularity of IPTV industry, QoE has been regarded as one of the most promising evaluation indicators for IPTV service. However, due to the increasing amount of TV programs and users' mixed preferences, how to recommend interesting programs for users is still a challenging and urgent problem. Existing related researches ignore the personalized recommendation and the prediction of prospective interests for different users from large amounts of TV programs. To solve this problem, this work proposes an enhanced latent Dirichlet allocation (LDA) model to analyze user behaviors and recommend personalized programs of the users' mixed interests. Specifically, we put forward a new attribute called viewing ratio to calculate the proportion of program's time viewed by the user, which could measure users' subjective viewing experience from objective indicators. Based on the proposed model, we improve the accuracy of user behaviors modeling and prediction of prospective interests. Experimental results show that our model has better performances of programs recommendation and TV viewing experience than other models.

Data service has been considered as one the most prominent characteristics for Industrial Internet of Things (IIoT). This paper studies how to design an optimal computing manner for a general IIoT system. On the theory end, we analyze the relationship between the data processing and the energy consumption through investigating the content correlation of the captured data. Importantly, we derive an exact expression for the performance of IIoT by combining computation with intelligence. On the application end, we design an efficient way to obtain a threshold by approximating the performances of different computing manners, and show how to apply it to practical IIoT applications. We believe that the proposed computation rules hold great significance for the IIoT designer, that is, it is better to use distributed computing manner when the content correlation is high, otherwise, centralized computing manner is better.

Massive multimedia services have been considered as one the most prominent characteristics for smart cities. In this paper, we propose an energy-efficient content delivery system via the device-to-device communications, which realizes the large-scale content delivery among mobile devices with constrained energy, unpredictable demand, limited storage, random mobility, and opportunistic transmission. The highlights of this paper lie in two parts. On the theoretical end, through exploring the relationship among the coding, storage, and transmission, a systematic energy-saving content delivery fashion is investigated. On the technical end, a totally distributed content delivery system is designed in a simple and efficient manner, in which each device only utilizes local information to make decisions and implements its own scheme individually. Importantly, the proposed scheme is realized in a practical smart city system, and numerical results demonstrate that it is flexible to various users' needs and communication environments.

This paper proposes a novel quality assessment scheme for IPTV operation and maintenance. It is an integrated system to make the IPTV network fault location diagnosis more efficiently and accurately. Specifically, the potential user complaint and potential warning facility are integrated for constructing the IPTV service quality assessment system. When handling the determination of the potential warning facility, objective Quality of Experience (QoE) indicators reflecting users' viewing behaviors are considered and integrated with traditional Quality of Service (QoS) indicators. Based on this, a novel feature selection algorithm is proposed for replacing existing ones in decision tree generation and pruning, efficiently and feasibly realizing faulted equipment prediction. Experimental results show that the prediction accuracy for faulted equipments can be further enhanced when compared with existing algorithms.

This work explores the true user QoE according to the users' preferences and behaviors when the users know that they are being observed and concern about their privacy. We propose a systematic privacy-aware QoE evaluation scheme based on the observable user data. Firstly, we translate the subjective privacy- aware QoE evaluation problem into the objective rational user analysis procedure. Then, a novel class-level joint user classification and data cleaning strategy is proposed by frequently updating the training processes. In particular, an efficient correlation analysis and QoE model framework is constructed for online implementation. Our results reveal that the true user QoE can be precisely captured if only some conditions are satisfied even for the privacy-aware users.

As a useful tool for improving the user's quality of experience (QoE), delay announcement has received substantial attention recently. However, how to make a simple and efficient delay announcement in the cloud mobile media environment is still an open and challenging problem. Unfortunately, traditional convex and stochastic optimization-based methods cannot address this issue due to the subjective user response with respect to the announced delay. To resolve this problem, this paper analytically studies the characteristics of delay announcement by analyzing the components of the user response and designs a QoE-driven delay announcement scheme by establishing an objective user response function. On the methodology end, the user response associated with the announced delay is approximated in the framework of fluid model, where the interaction between the system performance and delay announcement is well described by a series of mathematical functions. On the technology end, this paper develops a novel state-dependent announcement scheme that is more reliable than the other competing ones and can improve the user's QoE dramatically. Extensive simulation results validate the efficiency of the proposed delay announcement scheme.

Device-to-device (D2D) content sharing has become a promising solution to support the growing popularity of multimedia contents for local services. Considering the randomness of content location, the limited storage and transmission capability of devices, and the coexistence of altruistic and selfish user behaviors, how to optimally match the demanders to the providers of contents and how to stimulate an efficient cooperation are of importance for achieving the full benefits of D2D content sharing. Especially when the base-station-to-device (B2D), D2D, and novel multi-D2D sharing modes coexist, the issue of content sharing mode selection plays the predominant role in such matching. In this paper, we introduce a notion of social-aware rate, which combines the social selfishness from the social knowledge with the link rate to ensure the physical link quality and the effective cooperation together. Then, the social-aware rate-based content sharing mode selection problem is modeled as a maximum weighted mixed matching problem, which can be computationally reduced to a submodular welfare problem subject to a matroid constraint. Subsequently, we develop a best-effort distributed algorithm framework, which displays alternatives of various computation complexities and approximation ratios to satisfy the diverse practical needs.

It is well known that delay announcement is an economical and efficient way to improve the user satisfaction since the waiting time (delay) is an important performance metric for media cloud. However, how to accurately estimate the delay in an online-implementation manner is still an open and challenging problem. In this study, we study the data-driven delay estimation in a practical cloud media with heavy traffic, and propose an accurate estimation strategy only with a small amount of dataset. Importantly, we explicitly model the subjective announcement-dependent user response via an objective response function through the elaborate data analysis and model. On the theoretical end, the user response in terms of the estimated delay is characterized by the time window data-cleaning, where an appropriate dataset is set up through the window function analysis. On the technical end, we analyze the conditions for data-driven delay estimation, and prove that the proposed method is able to obtain a near-optimal solution within a finite time period. Extensive simulation results demonstrate the efficiency of the proposed delay estimation method.

As video traffic has dominated the data flow of smartphones, traditional cellular communications face substantial transmission challenges. In this work, we study mobile device-to-device (D2D) video distribution that leverages the storage and communication capacities of smartphones. In such a mobile distributed framework, D2D communication represents an opportunistic process to selectively store and transmit local videos to meet the future demand of others. The performance is measured by the service time, which denotes the elapsed period for fulfilling the demand, and the corresponding implementation of each device depends on the video’s demand, availability, and size. The main contributions of this work lie in (1) considering the impact of video size in a practical mobile D2D video distribution scenario and proposing a general global estimation of the video distribution based on limited and local observations; (2) designing a purely distributed D2D video distribution scheme without the monitoring of any central controller; and (3) providing a practical implementation of the scheme, which does not need to know the video availability, user demand, and device mobility. Numerical results have demonstrated the efficiency and robustness of the proposed scheme.

Abstracdt Content delivery via device‐to‐device (D2D) communications is a promising technology for offloading the heavy traffic for future mobile communication networks. As security is a critical concern for the users, we focus on improving the secrecy capacity for content dissemination in D2D communications. In this work, we explore the inherent characteristics of wireless channels to prevent eavesdropping. Firstly, we propose a power control scheme to obtain the optimal transmission powers for the D2D links without violating secrecy requirement of cellular users. Then, we formulate the problem as a stochastic optimization problem, aiming at maximizing the secrecy capacity gain of D2D communications. By solving the expected value model for the stochastic optimization problem, the optimal D2D links are selected to realize maximal ergodic secrecy capacity gain. Specifically, a weighted conflict graph is formulated according to the protocol model. Thus, the optimization problem has been transformed to the maximum weighted independent set problem, which is solved by a greedy weighted minimum degree algorithm. Simulation results demonstrate that the content dissemination scheme with power control can bring high secrecy capacity gain to the network. Copyright © 2016 John Wiley & Sons, Ltd.

Abstract Device‐to‐device (D2D) communications have been viewed as a promising data offloading solution in cellular networks because of the explosive growth of multimedia applications. Because of the nature of distributed device location, distributed caching becomes an important function of D2D communications. By taking advantage of the caching capacity of the device, in this work, we explore the device storage and file frequent reuse to realize distributed content dissemination, that is, storing contents in mobile devices (namedhelpers). Specifically, we first investigate the average and lower bound of helper amount by dividing the network into small areas where the nodes are within each other's communication radius. Then, optimal helper amount is derived based on average helper amount and network topology. Subsequently, a location‐based distributed helper selection scheme for distributed caching is proposed based on the given optimal helper amount. In particular, nodes are selected as helpers according to their locations and degrees, and contents are placed in the manner for maximizing total user utility. Extensive simulation results demonstrate the factors that affect the optimal helper amount and the total user utility. Copyright © 2015 John Wiley & Sons, Ltd.

Recently, attracted by the abundant computation and networking resources in the cloud, an increased number of mobile media services have been hosted by the cloud computing platforms. These mobile media clouds (MMCs) run hundreds of thousands of servers and consume megawatts of power with massive carbon emission. Virtualization is adopted to allocate resources elastically from a shared resource pool. Therefore, effective virtual machine (VM) consolidation is of paramount importance to reduce energy consumption. We especially focus on applications with different resource (e.g., CPU, memory, network) demands. In this paper, we formulate this problem as a mixed integer linear programming (MILP) problem. We prove that the optimal energy efficiency can be obtained for a homogeneous cloud. Based on this analytical result, we develop an approximation algorithm for VM consolidation and placement which jointly considers CPU and network constraints. Through extensive simulations, we validate the effectiveness of the proposed algorithm.

In this paper, energy trading for the distributed smart grid architecture is projected as an incomplete information game —a viewpoint that contrasts from all the existing pieces of literature available on the broader issue of energy management in smart grid. The incomplete information is considered as the real-time demand and price to grid and customers, respectively, due to the packet loss in the communication network. Therefore, the paper addresses a realistic scenario, in which real-time information to the destination may not be guaranteed to be received adequately, due to the packet loss. In the proposed scheme, we introduce two types of intelligent agents— customer-agents and grid-agent . The customer-agents are deployed at the customers׳ end, and are capable of estimating adequately the real-time price decided by the grid. On the contrary, the grid-agent is deployed at the service provider׳s end, and are also capable of estimating adequate real-time energy demand from the customers. Therefore, one of the key advantage of the proposed agent-based scheme is that the customers and the grid are not involved in complex calculations in order to take real-time decisions for cost-effective energy management, while there is information loss in the communication networks. In the proposed game model, the grid-agent and the customers agents are the players, and estimate real-time demand and price based on the probability of belief to each other. We show the existence of Bayesian Nash Equilibrium in the proposed model, where the utility of the players is maximized. We compare the real-time price with and without packet loss as the price with incomplete and complete information, respectively. We observe that the proposed model is beneficial for the grid, as its utility is maximized. The simulation results show that the utility of the grid increases approximately 40% over that of the existing ones under the scenario of information incompleteness.

Specific computing (SC) and diverse computing (DC), as two main visual computation manners, have been widely utilized in Media Cloud. However, how to choose SC or DC in a practical scenario is still an open and challenging problem. Unfortunately, the traditional fluid-based analysis method cannot address this issue due to the uncertain relationship between the computing manner and service dynamics. In this paper, we analytically study the characteristics of SC and DC by designing a so-called collapsing approximation (CA) method to precisely approximate the distribution of the service dynamics. On the qualitative end, we derive an exact expression for the dynamics of CA, thus enabling the cloud designer to choose different computing manners according to the application requests and analyze its impact on the degrees of DC. On the technical end, we show that the evolution of the service dynamics process can be approximated by the unique solution to a collapsing model over a finite time period. The highlight of this paper lies in demonstrating that the optimal computing configuration should largely depend on SC, and a little on DC.

Device-to-Device (D2D) communication has been proposed to be a promising data offloading solution in the coming big data age, with multimedia dominating the digital contents. As quality of experience (QoE) is the major determining factor in the success of new multimedia applications, we novelly propose a QoE-driven cooperative content dissemination (QeCS) scheme in the paper. Specifically, all the users predict the QoE of the potential connections characterized by mean opinion score (MOS) and send the results to the content provider (CP). Then CP formulates a weighted oriented graph based on the network topology and MOS of each potential connection. By factorizing the graph, the content dissemination fashion is established through seeking 1-factor with the maximum weight thus achieving maximum total user MOS.

In this work, we study the problem of green uplink resource sharing over mobile device-to-device (D2D) communications underlaying cellular network. We first construct a analysis model of energy efficiency, which takes into account different sharing modes, as well as QoS requirement and spectrum utilization of each user. Then, we formulate the sharing problem as a non-transferable coalition formation game, with the characteristic function which accounts for the gains in terms of energy efficiency and the costs in terms of mutual interference. Then, the resulting coalition structure shows the energy-efficient sharing strategy on the joint mode selection, uplink link reusing allocation, and power management. Moreover, we develop a distributed coalition formation algorithm based on the merge-and-split rule and the Pare to order. The distributed solution is characterized through stability notions and is adapted to user mobility. Simulation results are provided to demonstrate the effectiveness of our proposed game model and algorithm.

Adaptive wireless video scheduling has been widely studied to improve network performance. However, the majority of existing scheduling algorithms assume that they are able to converge instantaneously to adapt to a dynamic network state, that is, the execution time of the scheduling can be ignored. Nevertheless, due to the limited computation capacity of wireless nodes, this assumption is very difficult, sometimes even impossible, to satisfy in practice. This motivates us to address in this paper the following challenging question: what is the effect of the execution time on the scheduling performance? To this end, we first characterize the scheduling as a stochastic optimization problem that enables us to open up a new degree of performance to exploit in a tractable manner. Next, we build a connection between the execution time and video quality, and rigorously prove that the execution time is disadvantageous to the stability region, but advantageous to the flow balance. Therefore, these results are helpful to shed insights on fundamental scheduling guidelines on designing an efficient video transmission system.

Traditional distributed wireless video scheduling is based on perfect control channels in which instantaneous control information from the neighbors is available. However, it is difficult, sometimes even impossible, to obtain this information in practice, especially for dynamic wireless networks. Thus, neither the distortion-minimum scheduling approaches aiming to meet the longterm video quality demands nor the solutions that focus on minimum delay can be applied directly. This motivates us to investigate the distributed wireless video scheduling with delayed control information (DCI). First, to exploit in a tractable framework, we translate this scheduling problem into a stochastic optimization rather than a convex optimization problem. Next, we consider two classes of DCI distributions: 1) the class with finite mean and variance and 2) a general class that does not employ any parametric representation. In each case, we study the relationship between the DCI and scheduling performance, and provide a general performance property bound for any distributed scheduling. Subsequently, a class of distributed scheduling scheme is proposed to achieve the performance bound by making use of the correlation among the time-scale control information. Finally, we provide simulation results to demonstrate the correctness of the theoretical analysis and the efficiency of the proposed scheme.

Traditional distributed flow scheduling for wireless body sensor networks (WBSNs) is designed based on perfect control channels where the instantaneous control information from the neighbors is available. However, in practice it is very difficult, sometimes even impossible, to obtain this information especially for dynamic WBSNs. This motivates us in this paper to study the distributed flow scheduling with heterogeneous delayed control information (DCI). First, we translate this scheduling problem into a stochastic optimization problem that opens up a new methodology to exploit in a tractable framework. Subsequently, we investigate the relationship between the DCI and scheduling performance, and derive a general performance property bound for any distributed scheduling. Importantly, a class of asynchronous flow scheduling scheme is proposed to achieve the performance bound by making use of the correlation among the time-scale control information.

Data of the internet of things (IoT) typically consists of packets with small payloads since the protocol header is a significant and energy-expensive overhead. A variety of applications in IoT make it more complicated than that in wireless sensor networks in which one application generally exists. In this work, we present a scheme that replaces conventional frame design with a multiple-frame solution which aims at maximizing the network bandwidth utility for prolonging the network lifetime of energy-constrained IoT. Specifically, it is designed for inheriting all advantages of the fixed size packet MAC protocols for wireless sensor networks. Basically, the proposed scheme can be viewed as a proper extension of fixed-length frame scheme whether in wireline network or wireless network in IoT. The effectiveness of scheme is demonstrated by comparing the energy efficiency with the fixed-size frame for WSN and the variable-size frame for WLANs. This would be of great use in energy-constrained IoT applications.

Mobile Health (m-Health) systems include the use of mobile devices and applications that interact with patients and caretakers. However, mobile devices have several constraints (such as, processor, energy, and storage resource limitations), affecting the quality of service and user experience. This paper proposes a novel cooperation strategy for m-Health services and applications. This contribution addresses two related limitations to m-Health applications with service-oriented architectures, namely the network infrastructure and Internet connectivity dependency. It follows a reputation-based approach as an incentive method for cooperation, which includes a Web service to manage all the network cooperation. It is responsible for verifying the cooperation status of neighbor nodes and to provide relay nodes the required data in order to perform a full data request. A performance evaluation study in a real scenario is presented, using an available m-Health application, called SapoFit. For performance evaluation purposes, an analytical model is also considered in order to compare the obtained experiment results. It is clearly shown that referred dependencies are relevantly decreased, providing mobile nodes without Internet connectivity a free of charge and suitable alternative to access its remotely stored health information. It also improves the service delivery probability while increasing the overall network throughput.

In this work, we investigate the properties of energy-efficiency (EE) and spectrum-efficiency (SE) for video streaming over mobile ad hoc networks by developing an energy-spectrum-aware scheduling (ESAS) scheme. To describe a practical mobile scenario, we use a random walk mobility model, in which each node can choose its mobility direction and velocity randomly and independently. Through rigorous analysis and extensive simulations, we demonstrate that the node mobility is beneficial to EE but not to SE. The contributions of this work are twofold: 1) We propose an ESAS scheme with a dynamic transmission range, which significantly outperforms the previous minimum-distortion video scheduling in terms of joint EE and SE performance; 2) We derive an achievable EE-SE tradeoff range and a tight upper/lower bound with respect to energy-spectrum efficiency index for various node velocities. We believe that this work helps to shed insights on the fundamental design guidelines on building an energy and spectrum efficient mobile video transmission system.

Traditional α -fairness resource allocation in wireless multimedia communications assumes that the quality of experience (QoE) model (or utility function) of each user is available to the base station (BS), which may not be valid in many practical cases. In this paper, we consider a blind scenario where the BS has no knowledge of the underlying QoE model. Generally, this consideration raises two fundamental questions. Is it possible to set the fairness parameter α in a precisely mathematical specific α -fairness resource allocation schememanner? If so, is it possible to implement a specific α -fairness resource allocation scheme online? In this work, we will give positive answers to both questions. First, we characterize the tradeoff between the performance and fairness by providing an upper bound of the performance loss resulting from employing α -fairness scheme. Then, we decompose the α-fairness problem into two subproblems that describe the behaviors of the users and BS and design a bidding game for the reconciliation between the two subproblems. We demonstrate that, although all users behave selfishly, the equilibrium point of the game can realize the α-fairness efficiently, and the convergence time is reasonably short. Furthermore, we present numerical simulation results that confirm the validity of the analytical results.

Most existing Quality of Experience (QoE)-driven multimedia resource allocation methods assume that the QoE model of each user is known to the controller before the start of the multimedia playout. However, this assumption may be invalid in many practical scenarios. In this paper, we address the resource allocation problem with incomplete information where the realized mean opinion score (MOS) can only be observed over time, but the underlying QoE model and playout time are unknown. We consider two variants of this problem: 1) the form of the QoE model is known but the parameters are unknown; 2) both the form and the parameters of the QoE model are unknown. For both cases, we develop dynamic resource allocation schemes based on online test-optimization strategy. Simply speaking, one first spends appropriate time on testing the QoE model, then optimizes the sum of the MOS in the remaining playout time. The highlight of this paper lies in resolving the inherent tension between the test and optimization by jointly considering the uncertainties of QoE model and playout time. Furthermore, we derive tight bounds on the MOS loss incurred by the proposed schemes in comparison with the optimal scheme that knows the QoE model a priori and prove that the performance gap, as the playout time tends to infinity, asymptotically shrinks to zero.

Vehicular delay-tolerant networks (VDTN) assumes the use of the delay-tolerant network (DTN) concept for vehicular communications in order to cope several issues, such as highly dynamic network topology, short contact durations, disruption, intermittent connectivity, variable node density, and frequent network fragmentation. These challenging characteristics of vehicular networks affect the design and construction of a network management solution for VDTNs. The standard simple network management protocol (SNMP) is widely used on conventional networks and it is not directly deployable on VDTNs. Then, this paper proposes an SNMP-based solution for VDTNs supporting load-related information collection from VDTN nodes using SNMP. It presents the design and the demonstration of this network management application in a laboratory-based testbed.

In this work, with respect to the uncertainty about the individual information, we investigate the relay power allocation problem from the energy-efficient, Pareto optimal, and competitive fairness perspective. At first, we design an easy-implementation energy efficiency metric, which aims at striking a balance between the QoS provisioning and the energy consumption. Then, an auction mechanism is proposed for relay power allocation. By transferring the auction mechanism into a game, we prove the existence, uniqueness, and Pareto optimality of the Nash equilibrium (NE) for our auction game, and show that the allocation strategies from the NE can achieve the energy efficiency in terms of the proposed metric. Next, we develop a distributed relay power allocation algorithm based on our best-response functions to reach the Pareto optimal NE. Importantly, we not only certify the convergence of the proposed algorithm, but also provide quantitative analysis on it. Extensive simulations results are conducted to confirm the validity of the analytical results.

In this work, we propose a correlated equilibrium (CE)-based energy-efficient resource allocation scheme for uplink OFDMA systems. At first, we construct an energy-efficient resource allocation game, where each subcarrier is viewed as a player to choose the most satisfying user, and the objective is to balance the tradeoff between the total energy efficiency and the fairness. Since the CE can achieve better performance by helping the non-cooperative players coordinate their strategies, we employ the CE to analyze the proposed game. Next, we derive the condition under which the CE is Pareto optimal and employ linear programming duality to show its closed-form expressions. Furthermore, we present a linear programming method and a distributed algorithm based on the regret matching procedure to implement the CE, respectively. Simulation results demonstrate that our scheme is able to achieve good convergence, Pareto optimality, and fairness.

In this paper, we consider a power-aware multimedia communications over internet of things (IoT). Specifically, we consider a generic IoT scenario where a multimedia server provides heterogeneous applications without knowing the application's quality of experience (QoE) model and playout period. Our objective aims at dynamically adjusting the power allocation for each application over a uncertain period to maximize the system overall mean opinion score (MOS). Note that the practical QoE model can be observed over time, but the underlying functional relationship between the power and MOS is unknown. The highlight of this paper is to develop a dynamic powering algorithm, in which one learns the satisfaction function and optimizes power-aware user satisfaction with on-line operation. More precisely, the proposed algorithm performance is measured in terms of loss which denotes the MOS loss compared to the optimal one. Numerical simulation results validate the efficiency of the proposed algorithm.

In this work, we study the quality-delay tradeoff for video streaming over mobile ad hoc network by utilizing a class of scheduling schemes. We show that node spatial mobility indeed impacts on the performance of wireless video transmission under the assumption that all the nodes can identically and uniformly visit the entire network. To describe a practical mobile scenario, we consider a random walk mobility model in which each node can randomly and independently choose its mobility direction at each time-slot. The contributions of this work are twofold: 1) It investigates the optimal node velocity for the mobile video network which helps to identify the impact of mobility on the video performance; 2) It derives the achievable quality-delay tradeoff range for any node mobility velocity, and thus it is helpful to design appropriate quality and delay requirements. These results provide insights on network design and fundamental guidelines on establishing an efficient mobile wireless video transmission system.

Wireless sensor networks can be used to improve both safety critical and unsafety critical aircrafts systems. Using wireless sensor networks can help to increase the number of sensors as well the system redundancy and also helps to reduce the aircraft system weight and complexity, improving the fuel efficiency and maintenance costs. Supporting standard protocols in all wireless sensor nodes simplifies the application development, configuration and maintenance. The wireless sensor network devices can also be used to monitor the physiological pilot's parameters. This paper presents a complete and innovator solution, mainly based on standard protocols, to monitor light aircraft and gliders pilot's physiologic parameters. The proposed system does not interfere with pilot's agility, is simple to install, configure and operate. To evaluate the system, a real testbed was deployed.

Owing to the evolution of green communications, energy efficiency is treated as an important performance metric of a uplink orthogonal frequency division multiple access (OFDMA) system. In this study, the authors propose an energy-efficient resource allocation scheme by using the correlated equilibrium (CE). At first, the authors construct an energy-efficient resource allocation game, where each subcarrier is viewed as a player to choose the most satisfying user, and the objective is to balance the tradeoff between the total energy efficiency and the fairness. Since the CE can achieve better performance by helping the non-cooperative players coordinate their strategies, the authors employ the CE to analyse the proposed game. Next, the authors derive the condition under which the CE is Pareto optimal and employ linear programming duality to show its closed-form expressions. Furthermore, the authors present a linear programming method and a distributed algorithm based on the regret matching procedure to implement the CE, respectively, which can help us determine the desired resource allocation. Simulation results demonstrate that our scheme is able to achieve good convergence, Pareto optimality and fairness.

In this work, we investigate the impact of mobility on video streaming over multi-hop wireless networks by utilizing a class of scheduling schemes. We show that node spatial mobility has the ability to improve video quality and reduce the transmission delay without the help of advanced video coding techniques. To describe a practical mobile scenario, we consider a random walk mobility model in which each node can randomly and independently choose its mobility direction, and all the nodes can identically and uniformly visit the entire network. The contributions of this work are twofold: 1) It studies the optimal node velocity for the mobile video system. In this case, it is possible to achieve almost constant transmission delay and video quality as the number of nodes increases; 2) It derives an achievable quality-delay tradeoff range for different node velocities. Therefore, it is helpful to shed insights on network design and fundamental guidelines on establishing an efficient mobile video transmission system.

In this work, we study the problem of how to strike a balance between the QoS provisioning and the energy efficiency when a cooperative communication scheme is applied to a clustered wireless sensor network. Specifically, we first characterize the tradeoff by a multi-variable optimization problem, with the goal of balancing the outage performance and the network lifetime. Then, we horizontally decompose the problem into the concatenation of two subproblems: i) the long-haul transmit power per sensor node, and ii) the set of assisting cluster nodes. For the former one, an optimal long-haul transmit power solution is proposed based on the Lambert W function. The latter one is modeled as a coalition formation game, where the characteristic function is designed based on the combination of the former subproblem's results. Furthermore, an optimal algorithm is proposed by using a dynamic coalition formation process based on the best-reply process with trial opportunity. Extensive simulation results are presented to demonstrate the effectiveness of our proposed scheme.

Cross-layer design has been widely used in wireless sensor networks, especially to improve the network lifetime, as can be seen in the literature. In this paper, a cross-layer solution is combined to a transmission advertisement scheme to improve a slotted ALOHA-based wireless sensor network throughput and lifetime. This medium access method scheme has been chosen because it does not add protocol information to be transmitted with data bits, reducing transmission overhead when compared to other medium access methods. Finally, the combination of a cross-layer design and the advertisement scheme has proven to increase the network throughput by more than 10% and to double the network lifetime.

Distributed scheduling for hybrid media flows over wireless body sensor network faces three technical challenges: constrained communication channel, random node placement and strict transmission latency. In this work, we study this problem by jointly considering the above three challenges to achieve the minimum media distortion and optimal network resource utilization. At first, we construct a general flow transmission model according to network's transmission mechanism, as well as media-aware flow's characteristics. Then, a distributed scheduling scheme is proposed based on dynamic network resource update. It is proved that the proposed scheme can achieve the optimal scheduling solution with an exponential convergence rate, and an explicit form of the asymptotic convergence rate is provided. Furthermore, the realization of the distributed scheduling scheme through the collaboration between the network and the sources is the highlight of this paper. Extensive simulation results are provided to demonstrate the effectiveness of our proposed scheme.

The emergence of Web 2.0 has transformed the web into a more dynamic and interactive environment, offering a set of tools that enhance contact and collaboration between users. Several applications, such as online social networks, wikis and blogs, support such Web vision. This study elaborates on the tremendous potential of online social networks to enhance e-learning experience, creating an atmosphere of cooperation and easy interaction among users (teachers and students). A traditional learning content management system is rigid in nature, limiting the student learning process. As a result, the concepts of community, relationship and interaction among users are needed to overcome its limitations. This contribution addresses the evolution from the traditional content learning management systems to a new conceptual learning approach, using personal learning environments. Improving the learning experience using an e-learning platform is the main motivation for this work. Then, several online social networks-related modules were proposed for an e-learning platform, called personal learning environment box, creating a space for sharing and collaboration turning students more active in shaping their learning process. The proposal was evaluated through a utility survey to users, after four months term of system usage, and the results are extremely promising.

High quality multimedia forensics service is increasingly critical for delay-sensitive applications over heterogeneous networks. Up to now, it is a challenging problem, where the demand for less forensics overhead, higher authentication level and smaller transmission delay needs to be reconciled with the limited and often dynamic network resources. Traditional multimedia forensics mechanisms, however, either overlook the available network resource or neglect the interaction between multimedia forensics and network scheduling. This work presents a novel framework for delay-sensitive multimedia applications over resource-limited heterogeneous networks by jointly considering multimedia forensics, network adaptation, and deadline-driven scheduling. In particular, we develop a joint forensics-scheduling scheme, which allocates the available network resources based on the affordable forensics overhead and expected quality of service, adaptively adjusts the scalable media-aware forensics, and schedules the transmissions to meet the application's delay constraints. Through analysis and simulation, we demonstrate that the proposed scheme not only can provide a satisfying multimedia forensics service with nearly full utilization of the network resource, but also can achieve substantial performance improvements compared to other reference approaches.

Existing scheduling schemes for real-time wireless video services generally focus on system throughput or delay, and do not take into account the relationship between the transmission delay and video distortion. In this paper, we develop and evaluate a delay-distortion-aware wireless video scheduling scheme in the framework of cross-layer information adaptation. At first, we construct a general video distortion model according to the observed wireless network parameters, as well as each video sequence's rate-distortion characteristic. Then, we exploit a distortion-aware wireless video scheduling scheme and derive a bound on the asymptotic decay rate of the video distortion. Furthermore, the relationship between the delay and distortion is studied by taking into account video application delay and distortion requirements for specific wireless network environments. The proposed scheme is applied to heuristically find optimal tradeoff between the delay and distortion. Extensive simulation results are provided to demonstrate the effectiveness and feasibility of the proposed scheme.

Security is increasingly becoming an important issue in the design of multimedia applications. However, existing scheduling schemes for real-time multimedia services over heterogeneous networks generally do not take into account security requirements when making control decisions. In this paper, we develop and evaluate a security-aware multimedia scheduling scheme in heterogeneous environment. Firstly, we exploit a multimedia transmission model in the context of heterogeneous wireless networks. Then, a security-aware scheduling scheme is proposed by taking into account applications' timing and security requirements in addition to precedence constraints. Furthermore, the proposed scheme is applied to heuristically find resource allocations, which maximize the quality of security and the probability of meeting deadlines for all the multimedia applications. Additionally, extensive experiments are provided to demonstrate the effectiveness of the proposed scheme.

An important issue of supporting multi-user video streaming over wireless networks is how to optimize the systematic scheduling by intelligently utilizing the available network resources while, at the same time, to meet each video's Quality of Service (QoS) requirement. In this work, we study the problem of video streaming over multi-channel multi-radio multihop wireless networks, and develop fully distributed scheduling schemes with the goals of minimizing the video distortion and achieving certain fairness. We first construct a general distortion model according to the network¿s transmission mechanism, as well as the rate distortion characteristics of the video. Then, we formulate the scheduling as a convex optimization problem, and propose a distributed solution by jointly considering channel assignment, rate allocation, and routing. Specifically, each stream strikes a balance between the selfish motivation of minimizing video distortion and the global performance of minimizing network congestions. Furthermore, we extend the proposed scheduling scheme by addressing the fairness problem. Unlike prior works that target at users' bandwidth or demand fairness, we propose a media-aware distortion-fairness strategy which is aware of the characteristics of video frames and ensures max-min distortion-fairness sharing among multiple video streams. We provide extensive simulation results which demonstrate the effectiveness of our proposed schemes.

An important issue of supporting multi-user video streaming over wireless networks is how to optimize the systematic scheduling by intelligently utilizing the available network resources while, at the same time, to meet each video's QoS (quality of service) requirement. In this work, we study the problem of video scheduling over multi-channel multi-radio multi-hop networks with the goals of minimizing the video distortion. At first, we construct a general distortion model according to the network's transmission mechanism, as well as video's rate-distortion characteristics. Then, by joint considering the channel assignment, rate allocation and routing, we develop a fully distributed scheduling scheme to get an optimal QoS performance. Furthermore, the realization of the distributed scheduling scheme through cooperation among the channel, link and source is the highlight of this paper. Extensive simulation results are provided which demonstrate the effectiveness of our proposed scheme.

The support for multiple high-definition video streams in wireless home networks requires appropriate routing and rate control measures, ascertaining the reasonable links for transmitting each stream and the rate of the video to be delivered over the chosen links. In this paper, we invest the combination of the routing and rate control in a united convex optimization formulation and propose a distributed joint solution based on cross-layer design. We first develop a distortion model which captures both the impact of encoder quantization and packet loss due to network congestion on the overall video quality. Then, the optimal joint rate control and routing scheme is realized by adapting its rate to the time-varying traffic and minimizing the overall network congestion. Furthermore, simulation results are provided, which demonstrate the effectiveness of our proposed joint routing and rate control scheme in the context of wireless home networks.

An important issue for supporting multimedia applications in multiple heterogeneous networks, a typical pervasive computing environment, is how to optimize the rate allocation by intelligently utilizing the available network resources while, at the same time, to meet each application's QoS (Quality of Service) requirement. In this work, we develop and evaluate a rate allocation scheme in terms of audio and video applications based on a cross-layer design framework. At first, we construct a general distortion model according to the observed parameters in each network, as well as each application's rate-distortion characteristic. Then, the rate allocation is formulated as a convex optimization problem that minimizes the sum of the expected distortion of all applications. Furthermore, the realization of the distributed rate allocation algorithm for achieving an optimal or close-to-optimal end-to-end QoS under the overall limited resource budget is the highlight of this paper. Simulation results are provided which demonstrate the effectiveness of our proposed distributed rate allocation scheme.