Task-oriented video compression aims to eliminate redundancy while preserving task-critical information. However, existing spatial domain methods incur high computational overhead, whereas temporal domain approaches often rely on black-box confidence scores that may discard taskrelevant frames. These methods follow an implicit informationpreservation paradigm that entangles redundancy and relevance, yielding opaque decision-making. In this paper, we therefore reformulate it as an explicit and explainable frame filtering method. Instead of relying on black-box confidence scores, task relevance is quantified by an explainable saliency score grounded in explicit model attribution and statistical aggregation, providing a transparent criterion for frame filtering. This offline-defined saliency metric supervises a lightweight online regressor, so that each online decision inherits the same explainable semantic meaning while avoiding heavy computation at the edge. Specifically, frame filtering is decomposed into two sequential and explainable steps: a similarity detector first removes structurally redundant frames, and a saliency-guided filter then discards frames with limited contribution to the downstream task. Experimental results demonstrate the effectiveness of the proposed method.

Robots equipped with multimodal sensing capabilities play an important role in the Internet of Robotic Things (IoRT), especially in emergency rescue. However, existing rescue robots pose challenges for human operators in achieving precise manipulation due to the absence of haptic signals. Additionally, limited and fluctuating bandwidth in emergency rescue renders current cross-modal haptic generation schemes ineffective. To overcome this dilemma, we propose a novel cross-modal haptic generation scheme that enhances scalability across diverse network conditions by leveraging correlations among audio-visual–haptic modalities. Specifically, we first propose an edge-device collaboration-based architecture that dynamically extracts semantics from audio-visual signals, tailored to the current network conditions, for haptic generation. This is achieved by predicting the network state at the edge and performing multimodal encoding and fusion at the device. Next, we design a scalable cross-modal haptic generation scheme that implements the optimal generation strategy based on received audio-visual semantics of different granularities, ensuring real-time acquisition of coarse-grained or fine-grained haptic feedback. Finally, numerical experimental results conducted on a multimodal dataset and a simulated emergency environment indicate that the proposed scheme reliably generates haptic signals in emergency rescue.

To fulfill the demands of emerging multi-modal services, the cross-modal semantic communication paradigm comes into being. It fully utilizes potential semantic correlations among modalities to address polysemy and ambiguity issues, enhancing transmission reliability. However, applying cross-modal semantic communication in resource-constrained mobile scenarios introduces new challenges, including radio spectrum bandwidth limitations and fluctuations for the transmitter, and computing resource constraints for the receiver, which leads to potential transmission failures. To bridge this gap, this paper proposes a cross-modal semantic transmission strategy for mobile scenarios (MobileCMST). We first construct the framework for MobileCMST. Within this framework, a semantic encoder is designed to achieve redundancy elimination for visual and haptic signals. Then, a semantic delivery approach is developed to cope with bandwidth fluctuations and multipath fading channels. Finally, an efficient semantic decoder based on a visual-haptic semantic-integrated diffusion model is proposed. It employs the Mamba backbone to reconstruct high-quality signals with lightweight computational complexity. Extensive experiments demonstrate the excellent performance of the proposed MobileCMST strategy in resource-constrained mobile scenarios.

Federated learning (FL) has been regarded as a promising paradigm for enabling distributed model training over resource-limited edge devices. Although FL maintains data locality and enhances model generalization, it faces challenges such as model leakage and pressure from frequent model updates. Some existing schemes, such as differential privacy and model encryption, can partially alleviate these issues while sacrificing the accuracy of the modeling training or increasing the computational overheads in training. To address this issue, we design a low-complexity linear collaborative FL (LCFL) framework to enhance the privacy and accuracy of FL. Specifically, we propose the collaborative secrecy transmission (CST) algorithm by integrating a variant of Shamir's secret-sharing with the model segmentation, which can compresses and encrypts the local models for FL. The decoding complexity of the CST algorithm is only$O(N^{3})$under the compression ratio of$N$, which reduces the communication overhead and computational complexity. We conduct a quantitative analysis of the model error induced by the CST algorithm and derive its closed-form upper bound. Within LCFL, we formulate an optimization problem to maximize the global model accuracy in wireless FL by optimizing compression ratios, bandwidth allocation, and transmit-powers. Subsequently, we propose a low-complexity algorithm to solve this problem effectively. Numerical simulations demonstrate the efficacy of LCFL in improving FL's accuracy and security, and the results validate the efficiency of the proposed optimization scheme for wireless FL. The source code can be downloaded from the Github:https://github.com/MinITerence/LCFL.

Haptic data compression has gradually become a key issue for emerging real-time haptic communications in Tactile Internet (TI). However, it is challenging to achieve a trade-off between high perceptual quality and compression ratio in haptic data compression scheme. Inspired by the perspective of embodied AI, we propose a cross-modal haptic compression scheme for haptic communications to improve the perception quality on TI devices in this paper. Since multimodal fusion is routinely employed to improve the ability of system in cognition, we assume that haptic codec is guided by visual semantics to optimize parameter settings in the coding process. We first design a multi-dimensional tactile feature fusion network (MTFFN) relying on multi-head attention mechanism. The MTFFN extracts the multi-dimensional features from the material surface and maps them to infer the coding parameters. Secondly, we provide second-order difference and linear interpolation to establish an criterion for the determination of optimal codec parameters, which are customized by the material categories so as to give high robustness. Finally, the simulation results reveal that our compression scheme can efficiently make a personalized codec procedure for different materials, obtaining more than 17% improvement in terms of compression ratio with high perceptual quality at the same time.

Integrating visual-haptic remote control is an important application direction in the Industrial Internet of Things (IIoT). Cross-modal communications are considered to be an effective technology to support this application. However, due to limited bandwidth and competition between modalities, the quality of visual transmission and the end user’s immersive experience cannot be guaranteed in practical scenarios. To overcome this dilemma, this paper proposes a super-resolution reconstruction strategy for cross-modal communications. Specifically, the sender only transmits low-resolution images and haptic signals, while a haptic-aided super-resolution reconstruction (HaSR) approach is designed at the receiver. This approach involves semantic correlation-based modal fusion and generative adversarial principle-based visual generation, which enable the reconstruction of high-resolution images using the received low-resolution images and haptic signals. Experimental results from a standard dataset and a practical remote industrial control platform validate the effectiveness of the proposed strategy.

With the rapid advancement of deep learning and multimedia technologies, image inpainting has made significant progress in generating desirable content for damaged images. To fill in the corrupted part of the image, existing methods either utilize intrinsic information within the visual modality for single-modal image inpainting or consider semantic correlations from non-visual modalities for cross-modal image inpainting. However, when the corrupted part is extensive, intrinsic information alone within visual modality is insufficient to inpaint the entire image. Cross-modal image inpainting methods cannot guarantee image quality, as they do not explore the relationship among the corrupted part of the image, the remaining part of the image, and the non-visual modality. In order to handle this issue, a haptic to image inpainting scheme is proposed. Specifically, feature extraction and cross-attention-based feature index are firstly constructed to find the correspondence between the corrupted part of the image and the haptic modality. Then, cycle-consistency-based feature translation is implemented to explore the intrinsic correlations among the above two modalities, facilitating the formation of the corrupted part. Finally, the formed corrupted part is combined with the remaining part to realize image inpainting. Experimental results show the effectiveness of the proposed scheme.

Multi-modal services, integrating haptic modality into audio-visual applications, will further improve user's interactive and immersive experience. However, the quality of haptic signals in multi-modal services is very sensitive to transmission interference. Even in certain virtual scenarios, it is necessary to generate “virtual” haptic signals. Although several cross-modal reconstruction schemes are used to guarantee the quality of haptic signals, the weakly-supervised and weakly-paired issues still exist. To resolve this problem, a fine-grained audio-visual aided haptic signal reconstruction (FGHR) is proposed. Firstly, feature extraction with subcategory label exploration is designed by using the cross-modal transfer-based deep clustering network (CT-DCN). Then, the fusion features with shared semantic learning is performed under the triplet constraint. Finally, haptic signal reconstruction is realized by imposing the fine-grained semantic constraints. Experimental results show the effectiveness of the proposed scheme.

Multi-modal services, integrating various modalities such as audio, visual, and haptic, have emerged as leading multimedia applications in the 5G era and beyond. To fulfill the demands for low latency, high reliability, and large capacity, cross-modal transmission schemes have been proposed. Typically, these schemes emphasize on either audio-visual or haptic modality, and prioritize flawless transmission of one modality to assist the other modality streaming. However, these prerequisite and assumption do not hold for generic multi-modal services and communication environments, where determining the priority of modality and guaranteeing flawless transmission becomes challenging. To address this fundamental problem, in this paper, we introduce a strategy toward generic cross-modal transmission, enabling visual and haptic modalities to assist each other as needed. The strategy includes a visual-haptic mutual stream delivery mechanism at the sender and a visual-haptic mutual signal reconstruction approach at the receiver. The former aims to eliminate redundancy in visual and haptic streams through mutual assistance, while the latter adaptively handles impaired, missing, or delayed visual or haptic signals by leveraging modality-aware knowledge transfer and semantic-aware signal generation techniques. The proposed strategy demonstrates excellent performance through experiments conducted on a standard multi-modal dataset and a practical visual-haptic communication platform.

With the continuous evolution of networking technologies, multi-modal services that involve video, audio, and haptic contents are expected to become the dominant multimedia service in the near future. Edge caching is a key technology that can significantly reduce network load and content transmission latency, which is critical for the delivery of multi-modal contents. However, existing caching approaches only rely on a limited number of factors, e.g., popularity, to evaluate their importance for caching, which is inefficient for caching multi-modal contents, especially in dynamic network environments. To overcome this issue, we propose a content importance-based caching scheme which consists of a content importance evaluation model and a caching model. By leveraging dueling double deep Q networks (D3QN) model, the content importance evaluation model can adaptively evaluate contents' importance in dynamic networks. Based on the evaluated contents' importance, the caching model can easily cache and evict proper contents to improve caching efficiency. The simulation results show that the proposed content importance-based caching scheme outperforms existing caching schemes in terms of caching hit ratio (at least 15% higher), reduced network load (up to 22% reduction), average number of hops (up to 27% lower), and unsatisfied requests ratio (more than 47% reduction).

Haptic technology enables robots to touch and understand the interactions between objects in the reality. Advanced haptic sensing systems can not only collect pressure, temperature and stiffness of touched objects, but also avoid destructive operations, and assist in navigation and posture control for robots. In order to smoothly interact with different types of objects, in the haptic system, it is necessary to develop haptic object recognition methods for effective haptic perception capability. However, compared to RGB images, haptic images collected by opticallybased haptic sensors are similar in appearance, which makes traditional convolutional neural networks (e.g.,ResNet, VGG, etc.) ineffective. Therefore, in this paper, we are inspired by popular attention mechanism and multi-scale strategies, and propose a cross-scale attention based haptic object recognition network for object-robot interaction. In particular, On the one hand, we design a cross-scale attention module in convolutional neural networks to acquire spatial contextual feature. On the other hand, we design a learnable bilinear fusion strategy to integrate above spatial contextual feature with original haptic feature, so as to effectively discriminate haptic images. Experimental results on ViTac dataset have shown the effectiveness of our approach.

With the development of human–machine interactions, users are increasingly evolving toward an immersion experience with multidimensional stimuli. Facing this trend, cross-modal collaborative communication is considered an effective technology in the Industrial Internet of Things (IIoT). In this article, we focus on open issues about resource reuse, pair interactivity, and user assurance in cross-modal collaborative communication to improve Quality of Service (QoS) and users’ satisfaction. Therefore, we propose a novel architecture of modal-aware resource allocation to solve these contradictions. First, taking all the characteristics of multimodal into account, we introduce network slices to visualize resource allocation, which is modeled as a Markov decision process (MDP). Second, we decompose the problem by the transformation of probabilistic constraint and Lyapunov Optimization. Third, we propose a deep reinforcement learning (DRL) decentralized method in the dynamic environment. Meanwhile, a federated DRL framework is provided to overcome the training limitations of local DRL models. Finally, numerical results demonstrate that our proposed method performs better than other decentralized methods and achieves superiority in cross-modal collaborative communications.

The Internet of Things (IoT) has already emerged as one of the most popular directions in today’s information and communication technology (ICT) domain. With its advancement over different application areas, such as smart home, smart healthcare, industry 4.0, etc., a huge amount of data has been generated by billions of IoT devices, which aggravates the shortcomings of the network layer (IP)-based networks, such as limited expressiveness of IP addressing, inefficient support for mobility, and in-network caching. Building IoT on top of information-centric networking (ICN) is believed to be a promising solution to tackle the above challenge, especially the in-network caching of ICN can significantly benefit IoT in terms of reducing data and saving IoT devices’ energy. However, caching IoT data is more challenging than caching traditional Internet content, e.g., video, because IoT data are usually valid within a certain period of time, and IoT devices are typically constrained with battery. Hence, in this survey, we first review the current implementation proposals of ICN-based IoT (ICN-IoT). Next, we present the conventional caching decision policies and replacement policies which could be adopted to mitigate the aforementioned challenges, e.g., reducing IoT traffic, saving energy, and reducing data retrieval latency. Further, since leveraging machine learning (ML) techniques have the potential to further improve the caching efficiency by dealing with uncertainties, e.g., predicting unknown information, adaptively interacting with the environment, we also demonstrate the recently proposed ML-based caching schemes for ICN-IoT. In addition, we outline the open research issues and point out the future opportunities of caching in ICN-IoT.

Advanced network technologies and ubiquitous connected devices are boosting the development of the Internet of Things (IoT) at an unprecedented pace. However, as most of the connected IoT devices are battery powered, the energy consumption issue has become the bottleneck of the IoT’s development. Caching is a promising approach to reducing the energy consumption of the battery-powered devices since the requested data packets can be retrieved from intermediate nodes in the network, e.g., routers, instead of from the remote battery-powered IoT devices, which allows the IoT devices to spend more time in the sleep mode. To realize in-network caching and overcome the IP-based networks’ inefficiency support for IoT, building IoT over information-centric networking (ICN) is a promising approach advocated by researchers. However, existing works in this area assume the network environments are static, which hinders the development of existing approaches in the real dynamic network environments. In this article, we leverage the deep$Q$-networks (DQNs) to propose an intelligent caching scheme (named as iCache) that can automatically adjust the caching nodes’ caching parameters to make caching decisions for the dynamic network environments. Extensive evaluations were conducted and the results show that the proposed iCache outperforms the existing approaches in terms of the total energy consumption (e.g., more than 29% reduction compared to the caching transient data (CTD) caching scheme) and the average number of hops (e.g., more than 20% reduction compared to the CTD caching scheme).

Cross-modal communications, devoting to collaboratively delivering and processing audio, visual, and haptic signals, have gradually become the supporting technology for the emerging multi-modal services. However, the inevitable resource competitions among different modality signals as well as the unexpected packet loss and latency during transmission seriously affect quality of the received signals and end user's immersive experience (especially visual experience). To overcome these dilemmas, this paper proposes a cross-modal signal reconstruction strategy from the perspective of human's perceptual facts. It tries to guarantee visual signal quality by considering potential correlations among modalities when processing audio and haptic signals. On the one hand, a time-frequency masking-based audio-haptic redundancy elimination mechanism is designed by resorting to the similarity of audio-haptic characteristics and human's masking effects. On the other hand, based on the fact that non-visual perception can assist to form and enhance visual perception, an audio-haptic fused visual signal restoration (AHFVR) approach for handling the impaired and delayed visual signals is proposed. Experiments on a standard multi-modal database and a constructed practical platform evaluate the performance of the proposed perception-aware cross-modal signal reconstruction strategy.

The Internet of Things (IoT) is evolving into the Internet of Video Things (IoVT) that supports massive smart devices and multiple video applications. However, how to effectively control massive devices and transmit large-volume video data have become challenges in the current IoVT. Inspired by device-to-device (D2D) communications and coalitional game, this article constructs a self-organized D2D collaborative video content sharing framework for the IoVT. Specifically, we first propose a collaboration mechanism by introducing the social attributes of IoVT devices and their owners. In this mechanism, D2D collaborative coalitions are automatically formed among IoVT devices and video content is shared in the coalitions through D2D links. In this way, the burden of controlling massive IoVT devices and video data traffic are offloaded. Then, by integrating the scalability of the scalable-high-efficiency-video-coding (SHVC) streams and the flexibility of D2D networking, a collaborative video streaming strategy is developed. It takes advantage of provider set arrangement and transmission scheduling to reduce the impact of network instability on video services. Simulation results verify the effectiveness of the proposed mechanism and strategy.

Personalized content sharing will inevitably become one of the core applications of mobile Internet of Things. However, the existing strategies for content sharing are far from effective content personalization, since they either fail to protect the diversity of shared content or harm the enthusiasm of users to participate in cooperation. How to optimize the tradeoff between content personalization and sharing efficiency thus becomes an extremely challenging problem. To circumvent this dilemma, we propose a social-aware personalized content-sharing strategy based on mobile crowdsensing (MCS), which specially introduces positive network externalities derived from MCS and the social network. Specifically, we design a two-stage pricing-participation game to model the interactions between mobile users and a profit-making service provider. By solving the subgame-perfect Nash equilibrium (NE) of the proposed game, an efficient participation mechanism and an optimal-pricing strategy are developed. First, users’ decision selection of whether to join MCS is modeled as a social-aware MCS participation game (SA-MPG), and two algorithms for solving the Pareto-optimal NE of SA-MPG are designed. Subsequently, the pricing issue for network operators is investigated by exploiting the supermodularity of SA-MPG. Stochastic network model and real-world data set-based simulations corroborate the significant gain of our proposed strategy.

Multi-modal services, typically integrated by visual, audio, and haptic signals, have been considered as promising killer services in 5G and beyond 5G era. However, due to essential difference among these signals, how to guarantee quality of multi-modal services is a significant technical challenge. Existing transmission schemes, which deliver and process each modality signal separately, cannot meet such requirements as low latency, high reliability, and high throughput. To get over the dilemma, this paper proposes a general cross-modal transmission strategy by taking advantage of the potential correlation among modalities, which consists of a delivery mechanism at the sender and a signal restoration procedure at the receiver. On the one hand, a visual-aided haptic content compression method is designed for the delivery mechanism. By utilizing the category correlation among modalities, haptic signals with similar visual content can be effectively compressed, reducing transmission burden. On the other hand, a fine-grained haptic to image synthesis (FHIS) approach is proposed for realizing signal restoration. Through exploring strong matching properties among modalities, the FHIS can restore the impaired, missing, delayed visual images from the received haptic signals. Experiments on a standard visual-haptic database and a practical platform evaluate the performance of the constructed cross-modal transmission strategy.

Multi-modal services, typically integrating such signals as audio, video, and haptic, will become an inevitable application trend of the 5G and beyond. However, due to the essential differences among the haptic and audio/video signals, the existing coding schemes usually fail to satisfy the critical requirements in terms of the rate distortion performance. Inspired by the phenomenon that hearing, sight and touch are highly correlated, we provide an affirmative answer by proposing the framework of cross-modal coding, which compresses multi-modal signals aided by their semantic correlation. In particular, the highlights of this work lie in addressing three fundamental technical problems: i) how to exploit the semantic correlation among different modalities, ii) to what extent of benefit we can get from cross-modal coding, and iii) how to design a general cross-modal codec. On the theoretical end, we determine the minimum number of bits required to compress haptic signals under the rate conditions of video streams through investigating their semantic correlation. On the technical end, we design a general cross-modal codec to approach the optimal compression limit by using the AI-enabled cross-modal prediction and channel coding. Numerical results demonstrate that the proposed cross-modal coding can achieve significant benefits relative to the existing schemes, especially when multi-modal signals have strong semantic correlation.

The emerging multi-modal services, characterized as the integration of audio, visual, and haptic signals, will become the killer applications in 5 G and beyond 5 G era. In order to support multi-modal services, cross-modal communications come into being. However, when adopting cross-modal communications to haptic-dominant multi-modal services, there still face several technical challenges. On the one hand, haptic signals are very sensitive to interference and easy to be damaged or even missing during transmission. On the other hand, it needs to generate virtual haptic signals when real touch sensory information is hard to be gathered. To get over the dilemma, this paper proposes a haptic signal reconstruction strategy for cross-modal communications. First, a cloud-edge collaboration-based cross-modal communication architecture is constructed. Then, an audio-visual-aided haptic signal reconstruction (AVHR) approach under this architecture is designed by leveraging the potential correlation among modalities. It can be further divided into three components: feature extraction by cloud-edge transfer, shared semantic learning by multi-modal fusion, and haptic signal generation by semantic constraints. Finally, experiments on a standard audio-visual-haptic dataset and a practical cross-modal communication platform show that the proposed AVHR approach has better reconstruction performance when compared with the competing schemes.

Cross-modal applications that elaborately integrate audio, video, and haptic streams will become the mainstream of the eHealth systems. However, existing stream schedulers usually fail to simultaneously meet the cross-modal transmission requests in terms of low latency, high reliability, high throughput, and low complexity. To circumvent this dilemma, this article proposes a general cross-modal stream scheduling scheme by fully taking advantage of the characteristics of different modal streams and their underlying temporal, spatial, and semantic relevance. Specifically, we first propose a hierarchical stream category framework, in which the transmission priority of the modal stream instead of the data flow can be flexibly settled. Next, we design a series of modal-aware stream scheduling schemes by jointly making use of the network slice and mobile edge computing to achieve the tradeoff among the various metrics. Importantly, the transmission strategy can be adjusted adaptively to realize the optimal resource allocation. Subsequently, we analyze the relationship among the user experience, multi-modal impact, and stream scheduling through investigating the interacted impacts among the different modal streams, then develop a user experience based scheduling switch strategy to improve the application generality and reduce the performance fluctuation. Numerical objective and subjective results demonstrate the efficiency of the proposed cross-modal scheduling scheme.

Currently, video streaming services put more emphasis on user feeling or satisfaction than before. How to design suitable model and algorithm to effectively and efficiently evaluate user quality of experience (QoE) has become a significant technical challenge. To get over this dilemma, this paper proposes the broad forest, a non-neural network style broad model for streaming video QoE evaluation. The design target of broad forest is to take advantage of representation potential of forest and low complexity characteristic of broad learning system. Specifically, we first give the construction of broad forest. Then, the associated incremental learning algorithm for efficiently supporting the added structure and inputs is designed. Finally, we apply the broad forest to streaming video QoE evaluation. Experimental results show that the broad forest can not only guarantee accuracy, but also decrease training time. When subjective features are considered, it can further promote performance of QoE evaluation.

Nowadays, accurate traffic flow prediction toward 5G cellular wireless network has become an indispensable part for future artificial intelligence (AI)-assisted network. Meanwhile, low delay communication is also the essential part in the upcoming 5G era. However, traditional deep learning models applied in traffic flow prediction have many drawbacks, such as too much running time and computational resources. To tackle these issues, especially jointly considering effectiveness and efficiency, we design a deep-broad learning system (DBLS) for traffic flow prediction. Specifically, based on broad learning system (BLS), we firstly adopt deep representative learning to extract meaningful information from raw data in mapped feature nodes. Then, to further improve the performance of prediction, we add some other nodes i.e., enhancement nodes generated from mapped features as extra inputs to enhance the representative capability. Finally, taking mapped features nodes and enhancement nodes as inputs of the last-layer neural network, we apply ridge regression to compute the final weights quickly. Experimental results demonstrate that our proposed DBLS can make full use of advantages of both deep neural network and traditional BLS to increase the accuracy of traffic flow prediction, meanwhile, maintaining low complexity and running time.

In order to solve the issue of isolated data islands and data security and personal privacy in the development of artificial intelligence, federated machine learning effectively solves the problem of sharing knowledge while protecting user privacy and data security. In a wireless sensor network, a secure learning framework is particularly needed, so that each sensor node can jointly learn knowledge without leaking local node data. Compared to traditional regression analysis algorithms, quantile regression can more fully describe the relationship between response values and its covariates by estimating conditional quantile sequences rather than a single value (such as the mean). In this paper, we propose a quantile regression federated learning framework that applies quantile regression with federated learning frameworks to wireless sensor networks, studies the performance of the algorithms, and the effectiveness of the algorithms is verified by simulations.

With the deployment of compute-intensive applications on mobile devices, some work aims to support these applications by enhancing mobile edge computing (MEC) using device-to-device (D2D) communication technology. Different from previous work, we investigate the energy consumption optimization of MEC assisted by idle user equipment (UE) in a mobile environment when combining the two technologies, and propose an optimization method for continuous time. In this paper, we build a D2D-MEC model with user mobility. To optimize the processing decision to save the energy consumption in continuous time under this model, we define long-term costs and formulate the problem of minimizing long-term costs as a markov decision process (MDP) problem. The complex MDP problem is decomposed into two sub-problems, where the explicit cost is minimized first and then the long-term cost. Due to the uncertainty of the environment caused by the mobility of the UE and the high dimensionality of the environmental information, we use the reinforcement learning method based on the neural network approximation to minimize the long-term cost.

Content sharing via device to device (D2D) communication is considered as a promising method to improve the performance of cellular network. However, in D2D networks, the placement of multimedia content is still an complex and urgent issue. Hence, it is significant to figure out a fine-grained content collaboration placement and delivery strategy in D2D networks. For the efficient utilization of the storage and downloading capacity of user devices, we introduce a distributed content segment sharing strategy. In order to improve QoE with limited network resource, such strategy provides users multimedia service with differentiated quality. Then, we formulate QoE-driven D2D content segment placement with service differentiation as a submodular maximization problem, and a polynomial time algorithm is proposed. Simulation results shows that proposed algorithm outperforms the algorithms without distributed content segments sharing mechanism in terms of QoE. It also demonstrates that proposed algorithm has better on the performance of cache content diversity and fairness.

The emergence of a massive Internet-of-Things (IoT) ecosystem is changing the human lifestyle. In several practical scenarios, IoT still faces significant challenges with reliance on human assistance and unacceptable response time for the treatment of big data. Therefore, it is very urgent to establish a new framework and algorithm to solve problems specific to this kind of fast autonomous IoT. Traditional reinforcement learning and deep reinforcement learning (DRL) approaches have abilities of autonomous decision making, but time-consuming modeling and training procedures limit their applications. To get over this dilemma, this article proposes the broad reinforcement learning (BRL) approach that fits fast autonomous IoT as it combines the broad learning system (BLS) with a reinforcement learning paradigm to improve the agent's efficiency and accuracy of modeling and decision making. Specifically, a BRL framework is first constructed. Then, the associated learning algorithm, containing training pool introduction, training sample preparation, and incremental learning for BLS, is carefully designed. Finally, as a case study of fast autonomous IoT, the proposed BRL approach is applied to traffic light control, aiming to alleviate traffic congestion in the intersections of smart cities. The experimental results show that the proposed BRL approach can learn better action policy at a shorter execution time when compared with competing approaches.

Personalized networking video service, as an inevitable trend recently, has become the core part for users' quality of experience (QoE) improvement. Unfortunately, existing schemes of QoE improvement are far from personalized since most of them only focus on network-level or user-level optimization. How to realize personalized QoE improvement for networking video service has been widely considered as a fundamental technical challenge. To get over this dilemma, this work proposes a personalized QoE improvement scheme by fully taking advantage of the time-varying influences on users' QoE, including user-awareness, device-awareness and contextawareness. The highlights of this work lie in that, the proposed scheme realizes the personalization comprehensively considering all these three dimensions, meanwhile, it is so robust that can be applied to the application scenario where the observable users' data is not sufficient. Specifically, we firstly design a comprehensive data collection strategy and accurately classify these collected data. Then, an efficient deep learning (DL)-based model for personalized characteristics extraction is proposed to precisely characterize personalization with temporal, spatial and periodic correlations. Subsequently, to resolve the data sparsity issue, a federated learning (FL)-based architecture with privacy-protection is designed by securely exchanging encrypted parameters with other users. Importantly, we design an optimization scheme based on comprehensive MOS formula for personalized QoE improvement. Experimental results demonstrate that the proposed scheme has a significantly better performance on the personalized QoE improvement.

With the continuous development of smart city, the task of base station traffic flow prediction has become an urgent problem to be solved. In the traffic flow prediction, historical traffic data can provide valuable and important information. However, traditional algorithms cannot allocate reasonable attention to historical traffic during model fitting and prediction. In other words, the attention capability of models is not considered. In order to handle this issue, this paper proposes a base station traffic flow prediction scheme based on Long Short-Term Memory with attention mechanism (A-LSTM). The designed A-LSTM scheme contains three steps. Firstly, the collected base station data should be cleaned up. Subsequently, feature engineering is performed. Finally, the traffic flow prediction algorithm based on the A-LSTM algorithm is realized. It is noted that missing values padding can also be performed under the framework of the A-LSTM. Experiments on real dataset show that the adopted A-LSTM can more effectively model the long term sequential relationship in the data, comparing with the LSTM and the other related algorithms, and achieve the best performance in base station traffic flow prediction for the dataset.

In a P2P-based video on demand (P2P-VoD) streaming system, the load of nodes is one of the most important influencing factors for system performance. On one hand, some nodes may receive a lot of requests which may result in overload. On the other hand, the other some nodes may receive too few requests, leading to low utilization. Therefore, designing an effective load balancing strategy is crucial for system performance promotion. Existing researches handle this issue by considering node status in the past period of time, not concerning the potential load of nodes in the future. In this paper, we propose a load balancing strategy based on request queue (RQLB) for P2P-VoD system. Specifically, we firstly define the priority of the request by considering urgency, scarcity and the playing smoothness factors, making the high priority requests be handled in time. Subsequently, by considering the upload bandwidth, reliability and load degree, we define the utilization function of the node, so that the relatively low priority requests will be transferred to nodes with low utilization and reliability. Simulation results show that our proposed strategy can effectively solve existing load imbalancing problem in the P2P-VoD system, enhancing user watching quality.

The cooperative D2D content sharing mode is that multiple users can share content with each in a cooperative way. In this mode, we can improve the transmission efficiency and transmission stability of communication. Although there are many literatures on cooperative D2D, the mobility problem in this transmission mode has been ignored by many people. Moreover, traditional D2D communication is a one-to-one transmission mode, but cooperative D2D is a many-to-one transmission mode. There is a big difference of link establishment and content transmission between these two modes. And the impact of mobility on network topology is also very different. Therefore, the existing caching scheme and retransmission mechanism for D2D can not be applied to cooperative D2D. Meanwhile, whether in D2D or cooperative D2D, the success of the user requesting content is closely related to the social relationship between users, due to the social selfishness of D2D users. Therefore, in order to improve the performance of cooperative D2D content sharing, we exploit user interest similarity and mobility and propose a social-and-mobility-aware caching strategy for collaborative D2D scenarios. Not only that, we model the retransmission problem as a Knapsack problem and design the retransmission mechanism when the transmission link is interrupted to ensure the reliability of content sharing with the greedy algorithm. Finally, our simulation results show that our proposed caching placement scheme and retransmission scheme can improve the performance of cooperative D2D content sharing. In addition, we achieve a valuable caching guideline in cooperative D2D scenarios.

More and more network media users concern about their privacy issues since they know that their network behaviors are being observed, and thus the observable users' data are not reliable and sufficient in this case. How to evaluate the true quality of experience (QoE) of the privacy-aware users has become a significant technical challenge because of the most majority of existing data-driven QoE evaluation schemes based on the premise of the true and adequate users' observations. To get over this dilemma, this paper proposes a systematic and robust QoE evaluation scheme with unreliable and insufficient observation data. Specifically, we first translate the subjective privacy-aware QoE evaluation problem into an objective rational user analysis procedure. Then, a semantics-based similarity measurement for multidimensional correlation analysis is constructed to classify the observable data. Subsequently, the highlight of this paper lies in proposing a class-level joint user classification and data cleaning strategy by frequently updating the training processes. Through elaborately designing an iterative framework, it can effectively resolve the data sparsity and inconsistency problems due to the user privacy-aware preferences. Importantly, we also introduce an efficient QoE model construction method for online implementation, and numerical results validate its efficiency for different kinds of privacy-aware users.

Traffic flow prediction has been regarded as one of the urgent problems for wireless communication applications. This paper presents an integrated model to predict the traffic flow of base station. Firstly, we predict the traffic flow with basic Echo State Network (ESN) and Autoregressive Integrated Moving Average (ARIMA). Algorithms show poor prediction performance due to the existence of mutation values. Therefore, we add wavelet transform into our model to decompose traffic flow, which could make traffic signal smooth and easy to predict. Besides, through the wavelet decomposition and single reconstruction of the original traffic flow data, we obtain data sequences with a better regularity. Then we can get the final predicted traffic flow data with a linear sum of the single prediction. Experimental result shows that the integrated model can achieve a better prediction accuracy by comparing to ESN and ARIMA. The integrated model can decrease the Mean Absolute Percentage Error (MAPE) by 6% and reduce the Normalized Root Mean Square Error (NRMSE) to some extent.

Realizing accurate prediction of base station traffic and effectively controlling the entire network has become a major problem that needs to be solved urgently in the rapidly developing mobile communications environment. We propose a base station traffic prediction method based on STL-LSTM model, and introduce a Seasonal and Trend decomposition using Loess (STL) method based on robust local weighted regression to achieve smoothness. By this method, the trend, period, and noise of the base station data are separately decomposed to achieve efficient use of data. Then the paper introduces a long-term short-term memory network (LSTM), uses its back-propagation time training and overcomes the characteristics of the disappearance gradient to predict the processed data to achieve the prediction closest to the true value. The experimental results show that using this algorithm to predict the base station traffic has better performance comparing with the other algorithms. And the high-accuracy prediction can be realized effectively according to the dynamic transformation of the real state of the base station traffic.

With the increasing popularity of IPTV industry, QoE has been regarded as one of the most promising evaluation indicators for IPTV service. However, due to the increasing amount of TV programs and users' mixed preferences, how to recommend interesting programs for users is still a challenging and urgent problem. Existing related researches ignore the personalized recommendation and the prediction of prospective interests for different users from large amounts of TV programs. To solve this problem, this work proposes an enhanced latent Dirichlet allocation (LDA) model to analyze user behaviors and recommend personalized programs of the users' mixed interests. Specifically, we put forward a new attribute called viewing ratio to calculate the proportion of program's time viewed by the user, which could measure users' subjective viewing experience from objective indicators. Based on the proposed model, we improve the accuracy of user behaviors modeling and prediction of prospective interests. Experimental results show that our model has better performances of programs recommendation and TV viewing experience than other models.

User Quality of Experience (QoE) has been brought to service providers' attention with the boom of multimedia services, such as Internet Protocol Television (IPTV). In this paper, we propose a QoE prediction model based on improved probabilistic neural network (PNN) to study the mapping relationship between IPTV viewing records and the user QoE. Specifically, we combine the particle swarm optimization (PSO) with PNN, utilizing PSO to search the spread parameter in PNN, thus this parameter can be automatically obtained, saving much time and effort. Experimental results show that the PNN-PSO can achieve the highest G-mean in comparison with other models. Moreover, it finds the best spread automatically and consequently increases prediction accuracy by 13% compared with the PNN.

With the popularity of IPTV in television industry, IPTV operators pay more attention on user's experience so that they could gain more revenue under fierce competition. Quality of Experience (QoE) is used to describe the user's degree of satisfaction and the prediction of QoE is gradually to be the interest for IPTV operators. To solve this problem, we propose a QoE prediction algorithm. Specifically, we preprocess the raw data at first and select the important attributes influencing QoE. This work introduces a new attribute called viewing ratio. It can effectively reflect users' subjective feelings. Subsequently, we combine the BP neural network with Adaboosting, proposing the BP_Ada algorithm to predict the QoE. Experimental results show that the proposed algorithm has better performance comparing with the competing algorithms.

IPTV is a new multimedia service over the Internet. The rapid development of IPTV makes the assessment of quality of experience in IPTV a hot topic to the service providers. In this paper, we study the relationship between the record of some viewing parameters from the IPTV set-top box and the users' Quality of Experience (QoE). Firstly, we analyze the data and choose some important attributions and then map the trouble tickets table to QoE representing acceptable or unacceptable. According to the imbalanced feature of the dataset, we proposed the multi-layer neural network using BP algorithm based on SGD for prediction of QoE. To avoid overfitting, we apply dropout method to the model when training the dataset. At last, we compare the proposed model to SVM and Decision Tree. Experimental results show that the proposed methods can indeed improve the accuracy of QoE prediction.

This paper proposes a novel quality assessment scheme for IPTV operation and maintenance. It is an integrated system to make the IPTV network fault location diagnosis more efficiently and accurately. Specifically, the potential user complaint and potential warning facility are integrated for constructing the IPTV service quality assessment system. When handling the determination of the potential warning facility, objective Quality of Experience (QoE) indicators reflecting users' viewing behaviors are considered and integrated with traditional Quality of Service (QoS) indicators. Based on this, a novel feature selection algorithm is proposed for replacing existing ones in decision tree generation and pruning, efficiently and feasibly realizing faulted equipment prediction. Experimental results show that the prediction accuracy for faulted equipments can be further enhanced when compared with existing algorithms.

This work explores the true user QoE according to the users' preferences and behaviors when the users know that they are being observed and concern about their privacy. We propose a systematic privacy-aware QoE evaluation scheme based on the observable user data. Firstly, we translate the subjective privacy- aware QoE evaluation problem into the objective rational user analysis procedure. Then, a novel class-level joint user classification and data cleaning strategy is proposed by frequently updating the training processes. In particular, an efficient correlation analysis and QoE model framework is constructed for online implementation. Our results reveal that the true user QoE can be precisely captured if only some conditions are satisfied even for the privacy-aware users.

Device-to-device (D2D) communication has been widely studied to improve network performance and considered as a potential technological component for the next generation communication. Considering the diverse users’ demand, Quality of Experience (QoE) is recognized as a new degree of user’s satisfaction for media service transmissions in the wireless communication. Furthermore, we aim at promoting user’s Mean of Score (MOS) value to quantify and analyze user’s QoE in the dynamic cellular networks. In this paper, we explore the heterogeneous media service distribution in D2D communications underlaying cellular networks to improve the total users’ QoE. We propose a novel media service scheme based on different QoE models that jointly solve the massive media content dissemination issue for cellular networks. Moreover, we also investigate the so-called Media Service Adaptive Update Scheme (MSAUS) framework to maximize users’ QoE satisfaction and we derive the popularity and priority function of different media service QoE expression. Then, we further design Media Service Resource Allocation (MSRA) algorithm to schedule limited cellular networks resource, which is based on the popularity function to optimize the total users’ QoE satisfaction and avoid D2D interference. In addition, numerical simulation results indicate that the proposed scheme is more effective in cellular network content delivery, which makes it suitable for various media service propagation.

Good Quality of Experience is critical to the success of IPTV business development and promotion. To this end, the paper combines status data from the set-top box with the data of user's complaints and then selects the appropriate model to predict user's QoE. Firstly, we clean and conduct some statistical analysis for the dataset. Then, random under-sampling and synthetic over-sampling are applied to the dataset after these procedures. In order to get better performance, this paper improves the Synthetic Minority Over-sampling Technique (SMOTE) algorithm. In addition, we compare the decision tree model and k-Nearest Neighbors (k-NN) model in user's complaint dataset. Through rigorous modeling and prediction, extensive experimental results show that k-NN model performs better than the decision tree in terms of predicting the user's complaint.

In this letter, we propose a Student's t-hidden Markov model with truncated stick-breaking priors (TSB-SHMM). In the TSB-SHMM, the priors for elements in the initial state vector and the state transition matrix are constructed by stick-breaking procedure with a truncation level, and the observation emission distributions are the Student's t-mixtures. Then we derive an inference algorithm for estimating the parameters of the proposed TSB-SHMM. Experimental results on the synthetic data and text-dependent speaker identification illustrate that the TSB-SHMM can automatically determine the number of states and are robust to untypical observed data.