Federated Learning (FL) is a promising learning paradigm that allows for training a shared model by coordinating multiple distributed devices, namely, clients, without exposing their raw data. To mitigate excessive communication overhead and enhance practicality, a variant known as Hierarchical FL (HFL) has been introduced, which integrates edge servers between the cloud server and clients. In HFL, the number of potential clients is typically large, making full client participation impractical due to various resource constraints. As a result, it is essential to develop a sampling strategy that effectively selects suitable clients for federated optimization. While several methods have been proposed to protect the privacy of communicated models, we argue that the outcomes of client sampling are closely tied to the local data of clients, thereby raising privacy concerns, like the risk of differential attacks. Motivated by this, we propose a Two-step Privacy-Preserving client Sampling framework (TPPS) designed to protect against both attacks on communicated models and potential vulnerabilities in client sampling outcomes. Initially, we consider the diverse privacy requirements of clients by presenting a double-layer noise mechanism. We then conduct a thorough analysis of the impact of this noise mechanism, proposing a novel client sampling strategy that seeks to balance the trade-off between privacy and training performance. The insight lies in maintaining a real-time sampling probability for each client, which can be acutely tuned based on personalized privacy needs and previous training feedback. We provably show that TPPS achieves local differential privacy, a bounded sampling regret, and a privacy-related convergence rate. Furthermore, we conduct extensive simulations based on open datasets, showing the robustness and applicability of TPPS in enhancing privacy while optimizing HFL performance.

With the advancement of graph representation learning, self-supervised graph contrastive learning (GCL) has emerged as a key technique in the field. In GCL, positive and negative samples are generated through data augmentation. While recent works have introduced model-based methods to enhance positive graph augmentations, they often overlook the importance of negative samples, relying instead on rule-based methods that can fail to capture meaningful graph patterns. To address this issue, we propose a novel model-based adversarial contrastive graph augmentation (ACGA) method that automatically generates both positive graph samples with minimal sufficient information and hard negative graph samples. Additionally, we provide a theoretical framework to analyze the process of positive and negative graph augmentation in self-supervised GCL. We evaluate our ACGA method through extensive experiments on representative benchmark datasets, and the results demonstrate that ACGA outperforms state-of-the-art baselines.

Federated learning (FL) enables the training of a global machine learning model among multiple local clients in a collaborative fashion without directly sharing the details of their data. Due to this advantage, it has been utilized in a wide range of applications where privacy is a critical concern and has attracted great attention for graph representation learning (GRL). Despite the offered advances, there still exist two major challenges in the FL for GRL across distributed graph data, including heterogeneity and complementarity. In order to tackle these challenges, a novel personalized federated graph augmentation (PFGA) framework is proposed in this work. Unlike existing techniques, it utilizes generative models as bridges to enable information sharing among clients, thereby facilitating the collaborative training of GRL models. Instead of directly using the generative model trained on each client individually, we aggregate them into the globally generative model to gain a global view of the entire graph, which effectively alleviates the heterogeneity and complementarity issues simultaneously. We formulate the training of the generative and GRL models as a min-max adversarial learning problem and theoretically prove the convergence. Furthermore, the effectiveness of the method is demonstrated using experimental results on six real-world datasets.

Multi-party real-time video conferencing has become an indispensable service in industrial production and daily life. However, the current dynamic and limited network resources can no longer meet the growing service demands of users, resulting lagging and low visual quality. The emerging semantic transmission, together with the network-wide redundant computation capacity, provides new opportunities towards a new paradigm of semantic video conferencing. The key challenge of such fusion lies in the interplay of traditional streaming adaptation and the new semantic processing, calling for a holistic mechanism to optimize the service provision with compatibility and efficiency. In this paper, we for the first time address this challenge, and propose SemConf, a novel framework that integrate the semantic transmission into the video conferencing towards optimal user QoE. Our extensive evaluations, against state-of-the-art baselines, reveal that SemConf achieves a substantial improvement in QoE, with up to 33.6% enhancement in bandwidth-constrained environment. Overall, this work highlights the critical role of the coordination algorithm in balancing computational load and network throughput, showcasing SemConf as a transformative approach in the realm of semantic video conferencing.

Live streaming that supports real-time interaction has become increasingly popular. To support the ensuing requirements on low end-to-end latency, RTM, the state-of-the-art live streaming system at Douyin, replaces the HTTP-FLV streaming protocol with WebRTC. To tailor the WebRTC stack to the live streaming scenario, we focus on optimizing first-frame delay, startup video rebuffering, audio-to-video drift, and per-session CPU usage. Those are the top-priority metrics identified from an importance analysis with respect to two user engagement metrics, i.e., viewer penetration and viewing time. To date, WebRTC-based streaming in RTM has been in operation for 4 years, and serves billions of viewer sessions every day. It dramatically optimizes QoE metrics (e.g., end-to-end latency reduced by 54.5%), and delivers statistically significant user engagement gains (e.g., number of paid orders increased by 0.8%). In this paper, we report our deployment experiences comprehensively.

Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.

Viewport prediction is the crucial task for adaptive 360-degree video streaming, as the bitrate control algorithms usually require the knowledge of the user's viewing portions of the frames. Various methods are studied and adopted for viewport prediction from less accurate statistic tools to highly calibrated deep neural networks. Conventionally, it is difficult to implement sophisticated deep learning methods on mobile devices, which have limited computation capability. In this work, we propose an advanced learning-based viewport prediction approach and carefully design it to minimize transmission and computation overhead for mobile terminals. To improve viewport prediction accuracy, we utilize both spatial information through a saliency prediction model and temporal information through a modified LSTM model. Different computations introduced by the neural network models are distributed across the network to keep the computation light on mobile devices. To better adapt to the content dynamics in live streaming, we employ the model-agnostic meta-learning (MAML) method for video saliency prediction. The learned saliency prediction model with optimized initialization via offline meta-training can be fast fine-tuned online using a few samples. We further discuss how to integrate this mobile-friendly viewport prediction (MFVP) approach into a typical 360-degree video live streaming system by formulating and solving the bitrate adaptation problem. Extensive experiment results demonstrate that our approach achieves real-time prediction for live video streaming and surpasses existing methods in prediction accuracy on mobile terminals, which, together with our bitrate adaptation algorithm, significantly improves the streaming QoE from various aspects. Compared to baseline methods, MFVP achieves a 4.7–28.7% improvement in accuracy and demonstrates faster adaptability to dynamic content changes, enabling rapid fine-tuning and adjustment. When integrated into a streaming system and paired with our adaptive bitrate allocation algorithm, MFVP enhances overall video quality by 5.6–12.9% and reduces quality fluctuations by 33.3–50.9%.

Incentive mechanisms in Spatial Crowdsourcing (SC) have been widely studied as they provide an effective way to motivate mobile workers to perform spatial tasks. Yet, most existing mechanisms only involve single tasks, neglecting the presence of complementarity and substitutability among tasks. This limits their effectiveness in practice cases. Motivated by this, we consider task bundles for incentive mechanism design and closely analyze the mutual exclusion effect that arises with task bundles. We then develop a combinatorial incentive mechanism, including three key policies: In the offline case, we propose a combinatorial assignment policy to address the conflict between mutual exclusion and assignment efficiency. We next study the conflict between mutual exclusion and truthfulness, and build a combinatorial pricing policy to pay winners that yields both incentive compatibility and individual rationality. In the online case with unknown workers’ utilities, we present an online combinatorial assignment policy that balances the exploration-exploitation trade-off under the mutual exclusion constraints. Through theoretical analysis and numerical simulations using real-world mobile networking datasets, we demonstrate the effectiveness of the proposed mechanism.

Streaming 360° videos demands excessive bandwidth. Tile-based streaming and super-resolution are two widely studied approaches to alleviate bandwidth shortage and enhance user experience in such real-time video streaming systems. The former prioritizes the transmission of a fraction of the 360° video according to the user viewport, while the latter enhances the streamed video in higher resolutions through computations. However, these two approaches bring substantial complexity and computation overhead and thus suffer from resource bottlenecks due to the constrained mobile hardware. This paper proposes TBSR, a practical mobile 360° video streaming system that incorporates in-time super-resolution with tile-based streaming on commodity mobile devices. We present the designs of three key mechanisms, including a rate adaptation method with macro tile grouping to reduce decoding computations, a decoding and SR scheduler for different types of tasks to achieve the best cost efficiency, and the workload adjustment method to control the amount of tasks given the available capabilities. We further implement the TBSR prototype. Our performance evaluation shows that TBSR outperforms the existing methods, improving QoE quality by up to 32% and bandwidth savings by 26%.

Information-centric Networking in edge computing environments exhibits the potential to significantly enhance the efficiency, reliability, and security of data transmission, making it a promising technology for future network deployments. However, the differences from traditional networks require applications to actively redevelop and redeploy onto edge devices, incurring additional costs for the proliferation of ICN applications. In this paper, we propose a system to adapt QUIC protocol over ICN networks in multi-access edge networks. The system aims to expand the application repertoire for ICN by providing a smooth transition of applications using QUIC to run on ICN networks. We design an ICN-QUIC conversion layer to manage the transmission of data from QUIC-based applications. We implement and evaluate the designed system. The experiment results show that, compared to existing networks, our system can enhance the transmission efficiency, i.e., up to 20 times better goodput in multicast situations, and achieves comparable results in unicast scenarios. We test the scalability of our system in real edge and wireless environments. We also deploy the real applications over the proposed system to demonstrate its compatibility in ICN and MEC environments.

With the popularity of mobile devices and the continuous advancement of mobile network technology, running online augmented reality (AR) on lightweight mobile devices is much more desirable than on heavy and expensive head-mounted devices that are difficult to satisfy users. Mobile edge computing can assist in supporting AR applications running on mobile devices, which copes with compute-intensive and delay-sensitive requirements. However, subject to the limited and heterogeneous edge resources, offloading tasks to edge devices is not easy, especially if the application requires multi-party interaction. It is challenging to develop a credible task placement scheme that satisfies user experience with flexible use of edge resources. This article focus on the task offloading placement problem for AR overlay rendering in multi-party mobile augmented reality system. We first present our observations about performance bottlenecks of edge devices and explain the necessity of splitting the AR overlay rendering pipeline. We then formulate a joint optimization problem of task placement decisions, aiming to maximize the user experience of quality and minimize the service cost. We develop a novel decision approach based on deep reinforcement learning (DRL) to address this complex problem. Finally, we verify the effectiveness and superiority of the proposed method through extensive evaluation experiments.

The outbreak of COVID-19 has dramatically promoted the explosive proliferation of multi-party realtime video streaming (MRVS) services, represented by Zoom and Microsoft Teams. Different from Video-on-Demand (VoD) or live streaming, MRVS enables all-to-all realtime video communication, bringing significant challenges to service providing. First, unreliable network transmission can cause network loss, resulting in delay increase and visual quality degradation. Second, the transformation from two-party to multi-party communication makes resource scheduling much more difficult. Moreover, optimizing the overall QoE requires a global coordination, which is quite challenging given the various impact factors such as bitrate and loss.In this paper, we propose the SJA framework, which is, to our best knowledge, the first server-driven joint loss and bitrate adaptation framework in multi-party realtime video streaming services towards maximized QoE. We comprehensively design an appropriate QoE model for MRVS services to capture the interplay among perceptual quality, variations, bitrate mismatch, loss damage, and streaming delay. We mathematically formulate the QoE maximization problem in MRVS services. A Lyapunov-based relaxation and the SJA algorithm are further designed to address the optimization problem with close-to-optimal performance. Evaluations show that our framework can outperform the SOTA solutions by 18.4% ∼ 46.5%.

Tile-based streaming and super resolution are two representative technologies adopted to improve bandwidth efficiency of immersive video steaming. The former allows selective download of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to reconstruct the received video into higher quality using advanced neural network models. In this work, we propose CASE, a collaborated adaptive streaming and enhancement framework for mobile immersive videos, which integrates super resolution with tile-based streaming to optimize user experience with dynamic bandwidth and limited computing capability. To coordinate the video transmission and reconstruction in CASE, we identify and address several key design issues including unified video quality assessment, computation complexity model for super resolution, and buffer analysis considering the interplay between transmission and reconstruction. We further formulate the quality-of-experience (QoE) maximization problem for mobile immersive video streaming and propose a rate adaptation algorithm to make the best decisions for download and for reconstruction based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which presents stable performance with considerable QoE improvement, while enabling trade-off between playback smoothness and video quality.

Given the rapid growth of user-generated videos, internet traffic has been heavily dominated by online video streaming. Caching videos on edge servers in close proximity to users has been an effective approach to reduce the backbone traffic and the request response time, as well as to improve the video quality on the user side. Video popularity, however, can be highly dynamic over time. The cost of cache replacement at edge servers, particularly that related to service interruption during replacement, is not yet well understood. This paper presents a novel lightweight video caching algorithm for edge servers, seeking to optimize the hit rate with real-time decisions and minimized cost. Inspired by recent advances in deep Q-learning, our DQN-based online video caching (DQN-OVC) makes effective use of the rich and readily available information from users and networks. We decompose the Q-value function as a product of the video value function and the action function, which significantly reduces the state space. We instantiate the action function for cost-aware caching decisions with low complexity so that the cached videos can be updated continuously and instantly with dynamic video popularity. We used video traces from Tencent, one of the largest online video providers in China, to evaluate the performance of our DQN-OVC and to compare it with state-of-the-art solutions. The results demonstrate that DQN-OVC significantly outperforms the baseline algorithms in the edge caching context.

Viewport prediction is the crucial task for viewport-adaptive 360-degree video streaming. Various viewport prediction methods are studied and adopted from less accurate statistic tools to highly calibrated deep neural networks. Conventionally, it is difficult to implement sophisticated deep learning methods on mobile devices, which have limited computation capability. In this work, we propose an advanced learning-based viewport prediction approach and carefully design it to introduce minimal transmission and computation overhead for mobile terminals. We further discuss how to integrate this mobile-friendly viewport prediction (MFVP) approach into the adaptive 360-degree video live streaming by formulating and solving the bitrate adaptation problem. Extensive experiment results show that our prediction approach can work in real-time for live streaming and can achieve higher accuracies compared to other existing prediction methods on mobile clients, which, together with our proposed bitrate adaptation algorithm, significantly improves the streaming Quality-of-Experience (QoE) from various aspects.

Video streaming plays a critical role in the video analytics pipeline and thus its adaptation scheme has been a focus of optimization. As machine learning algorithms have become main consumers of video contents, the streaming adaptation decision should be made to optimize their inference performance. Existing video streaming adaptation schemes for video analytics are usually designed to adapt to bandwidth and content variations separately, which fail to consider the coordination between transmission and computation. Given the nature of batch transmission in video streaming and batch processing in deep learning-based inference, we observe that the choices of the batch sizes directly affects the bandwidth efficiency, the response delay and the accuracy of the deep learning inference in video analytics. In this work, we investigate the effect of the batch size in transmission and processing, formulate the optimal batch size adaptation problem, and further develop the deep reinforcement learning-based solution. Practical issues are further addressed for Implementation. Extensive simulations are conducted for performance evaluation, whose results demonstrate the superiority of our proposed batch adaptive streaming approach over the baseline streaming approaches.

Nowadays, although the rapidly growing demand for short video sharing has brought enormous commercial value, considerable bandwidth usage becomes a problem for service providers. To save costs of service providers, the short video applications face a critical conflict between maximizing the user quality of experience (QoE) and minimizing the bandwidth usage. Most of existing bitrate adaptation methods are designed for the livecast and video-on-demand instead of short video applications. In this paper, we propose a QoE-aware adaptive download control mechanism to ensure the user QoE and save the bandwidth, which can download the appropriate video according to user retention probabilities and network conditions, and pause the download when the buffered data is enough. The extensive simulation results demonstrate the superiority of our proposed mechanism over the other baseline methods.

Tile-based approach is widely adopted in adaptive 360\textdegree~video streaming systems. Existing QoE-driven streaming approaches usually obtain the tile selection and adjust the bitrate based on the viewport prediction with a fixed tiling, which fail to consider the unstable prediction performance. However, varying the tiling of the video can produce different number of tiles with different sizes, and thus can have distinct impacts on error tolerance for viewport prediction and on decoding complexity for resource-constrained mobile client. In this work, we introduce adaptive tiling into the conventional bitrate adaptation for mobile 360degree~video streaming. We first analyze the impacts of tilings on tile selection and decoding time, which verify the benefit of tiling adaptation in various practical aspects. We then formulate the QoE optimization problem for adaptive tiling and bitrate streaming and discuss the design details of our adaptation algorithm, which can adapt to the performance of viewport prediction and the decoding capabilities of mobile clients in addition to the conventional influencing factors. Finally, the superiority of our proposed approach compared with the state-of-the-art methods is evaluated through extensive trace-driven simulations.

Emerging multimedia applications like VR, AR, etc., exhibit unique transmission features, such as block-based transmission, dynamic prioritization for different contents, and deadline-aware delivery, which should be carefully managed but fail to be considered in the design of existing transmission control algorithms. In this work, we propose a delay-sensitive congestion control algorithm with a hybrid of coarse-grained and fine-grained control to improve the QoE scores. The coarse-grained control scheme maintains a low queuing delay and avoids missing the deadline in the steady state. The fine-grained control scheme rapidly reacts to the network dynamics based on our bandwidth estimation model. For the block scheduling, we heuristically model the realistic priority of each block by examining the trade-off among the remaining time, the remaining size, and the priority score of each block. Extensive experiments are conducted to evaluate the performance of our solution, which show that our solution significantly outperforms other baseline algorithms.

Today’s multimedia applications usually organize the contents into data blocks with different deadlines and priorities. Meeting/missing the deadline for different data blocks may contribute/hurt the user experience to different degrees. With the goal of optimizing real-time multimedia communications, the transmission control scheme needs to make two challenging decisions: the proper sending rate and the best data block to send under dynamic network conditions. In this paper, we propose a delay-sensitive and priority-aware transmission control scheme with two modules, namely, rate control and block selection. The rate control module constantly monitors the network condition and adjusts the sending rate accordingly. The block selection module classifies the blocks based on whether they are estimated to be delivered before deadline and then ranks them according to their effective priority scores. The extensive simulation results demonstrate the superiority of our proposed scheme over the other representative baseline approaches.

Nowadays, surveillance cameras have been pervasively equipped with vehicles in public transport systems. For the sake of public security, it is crucial to upload recorded surveillance videos to remote servers timely for backup and necessary video analytics. However, continuously uploading video content generated by tens of thousands of vehicles can be extremely bandwidth consuming. In this work, we investigate the video uploading problem for moving buses by proposing to deploy dedicated access points (AP) at bus stops to facilitate video uploading. We define the harmonic objective for our problem, which includes minimizing the video uploading delay and minimizing the AP deployment cost. This problem is with two fundamental challenges. Firstly, it is difficult to balance the bandwidth capacity allocated to many buses because a bus obtains bandwidth resource from a series of APs deployed at stops along its route. Secondly, due to the randomness of bus movement and the complexity of bus routes, it is hard to predict the workload of an AP. Hence, it is challenging to estimate the delay of uploading video content through an AP. To cope with these challenges, we propose a water filling placement (WFP) algorithm, aiming to balance the aggregated bandwidth allocated to each bus. A queuing model is established to analyze the uploading delay of video content. We further resort to machine learning models to factor the influence of bus routes into our queuing model. Finally, a convex problem is formulated to optimize the harmonic objective, which can be optimally solved with the gradient descent (GD) based algorithm. We validate the correctness of our theoretical analysis and demonstrate the effectiveness of our method by carrying out extensive experiments using bus traces collected in Shenzhen city of China. In comparison with benchmark algorithms, our solution can always achieve the best performance.

The pervasive penetration of mobile smart devices has significantly enriched Internet applications and undoubtedly reshaped the way that users access Internet services. Different from traditional desktop applications, mobile Internet applications require users to input via touch screens and view outputs on the displays with considerably limited size. The significant conflict between the limited-size of touch screens and the richness of online media contents widely exists in dynamic-viewport mobile applications, a class of mobile Internet applications that download contents beyond the user's viewing region (referred to as viewport). As dynamic-viewport mobile applications usually use HTTP for content downloading, to improve their quality of experience (QoE) and cost efficiency, in this paper, we present a Mobile-Friendly HTTP middleware (MF-HTTP), which can interpret user touch screen inputs and optimize the HTTP downloading of media objects for such applications. We first demystify screen scrolling in mobile operating systems and precisely break down the viewport moving process. We identify the key influential factors for media object downloading and develop an optimal download scheme. Towards building a practical middleware, we further discuss and address the implementation issues in detail. We implement a MF-HTTP prototype based on Android platforms and evaluate the performance of MF-HTTP by conducting concrete case studies on two representative dynamic-viewport mobile applications, namely, web browsing and 360-degree video streaming.

Recently, Federated Learning (FL) has drawn tremendous attentions due to its ability to protect client's privacy. In FL, clients collaboratively train machine learning models by merely sharing intermediate computations, i.e., gradients of model parameters. However, training a complicated model involves multiple rounds of interactions between clients and the server via the Internet. Consequently, communication is a primary bottleneck of FL attributed to the poor network conditions and the large amount of interchanged computations. To overcome the communication bottleneck, we propose the ClusterGrad algorithm to compress gradients which can considerably reduce the volume of communicated computations. Our design is based on the fact that there is only a small fraction of gradients whose values are far away from the origin in each round of interaction in FL. We first identify these essential gradients that are far away from 0 using the K-means algorithm. These gradient values are approximated by a novel clustering based quantization algorithm. Then, the rest gradients lying close to 0 are approximated with a single value. We can prove that ClusterGrad outperforms the latest FL gradient compression algorithms: Probability Quantization (PQ) and Deep Gradient Compression (DGC). We conduct extensive experiments with the CIFAR-10 datasets which further demonstrate that ClusterGrad can achieve compression ratio (used interchangeably with compression rate) 123 on average in comparison with PQ and DGC with compression ratios 16 and 60 respectively.

Mobile augmented reality (MAR) augments a real-world environment (probably surrounding or close to the mobile user) by computer-generated perceptual information. Utilizing the emerging edge computing paradigm in MAR systems can reduce the power consumption and computation load for the mobile devices and improve responsiveness of the MAR service. Different from existing studies that mainly explored how to better enable the MAR services utilizing edge computing resources, our focus is to optimize the video generation stage of the edge-based MAR services-efficiently using the available edge computing resources to render and encode the augmented reality as video streams to the mobile clients. Specifically, for multi-party AR applications, we identify the advantages and disadvantages of two encoding schemes, namely colocated encoding and spilt encoding, and examine the trade-off between performance and scalability when the rendering and encoding tasks are colocated or split. Towards optimally placing AR video rendering and encoding in the edge, we formulate and solve the rendering and encoding task assignment problem for multi-party edge-based MAR services to maximize the QoS for the users and the edge computing efficiency. The proposed task assignment scheme is proved to be superior through extensive trace-driven simulations and experiments on our prototype system.

RFID-based human activity identification has become a key component in today's Internet-of-Things applications. State-of-the-art solutions mostly focus on the simple scenario with a single person in the open space. Extension to the more realistic realworld scenarios with multiple persons however is non-trivial. Given the much richer interactions among them, the backscattered signals will inevitably mixed, obscuring the information of individual activities. This is further complicated with multi-path in a common indoor environment. In this paper, we however argue that, though often considered harmful, the rich interactions combined with multi-path indeed offer more observable data. After careful processing the raw signals, critical information about the activities can be unveiled through modern learning tools. We present M2AI, which for the first time accommodates both multi-path and multi-object for activity identification. M2AI incorporates a phase calibration mechanism to automatically eliminate the frequency hopping offsets, and a novel decoupling mechanism for the periodogram and pseduospectrum in the raw signal mixture. The refined data are then fed into an advanced deep-learning engine that integrates a Convolutional Neural Network and a Long Short Term Memory network, which examines both spatial and temporal information in realtime for activity identification. Our M2AI is readily deployable using off-the-shelf RFID readers. We have implemented an M2AI prototype with Impinj UHF passive tags and a Speedway R420 reader. Experiments with multiple objects in a multipath-rich indoor environments report an activity identification accuracy of 97%, a significant gain (27%) over state-of-art solutions.

The pervasive penetration of mobile smart devices has significantly enriched Internet applications and undoubtedly reshaped the way that users access Internet services. Different from traditional desktop applications, mobile Internet applications require users to input via touch screens and view outputs on the displays with considerably limited size. The significant conflict between the limited-size of touch screens and the richness of online media contents requires the mobile Internet applications to download contents way beyond the user's viewing region (referred as viewport). In this paper, we present a Mobile-Friendly HTTP middleware (MF-HTTP), which interprets user touch screen inputs and optimize the HTTP downloading of media objects to improve quality of experience (QoE) and cost efficiency. We first demystify screen scrolling in mobile operating systems and precisely break down the viewport moving process. We identify the key influential factors for media object downloading and develop an optimal download scheme. Towards building a practical middleware, we further discuss and address the implementation issues in detail. We implement a MF-HTTP prototype based on Android platforms and evaluate the performance of MF-HTTP by conducting concrete case studies on two representative applications, namely, web browsing and 360-degree video streaming.

Today's multimedia content generation and sharing have been dramatically boosted by the deep penetration of broadband wireless accesses and the much improved processing power of smart mobile terminals. Mobile users can now instantly capture and share short video clips (usually of several seconds) anywhere and anytime, and consume them with convenient touch screen operations. Theinstant video clip sharinghas emerged as a mainstream application; such pioneers as Twitter's Vine, Miaopai, Instagram, and Snapchat have seen great acceptance, particularly by the youth community. In this paper, we present an initial study on instant video clip sharing. Taking Twitter's Vine as a representative, we systematically investigate its distinct mobile interface, service framework, and user watching behaviors, revealing how this mainstream multimedia service type differentiates from its traditional counterparts. Our trace measurement and analysis demonstrate that instant mobile video clips have a much shorter lifespan and highly skewed popularity that quickly decays over time. This is further aggravated by the unique screen scrolling operation for video browsing. As such, the download-and-watch scheduling used by existing platforms can hardly achieve quality user experience and cost efficiency. We closely investigate and model the input user gestures for scrolling, including drag and fling, and analyze the scheduling policy, partitioning it into prefetching scheduling and watch-time download scheduling. We develop effective solutions toward both subproblems as well as their integration with screen scrolling. The superiority of our enhancement is demonstrated by extensive trace-driven evaluation.

Mobile personal livecast (MPL) services are emerging and have received great attention recently. In MPL, numerous and geo-distributed ordinary people broadcast their video contents to worldwide viewers. Different from conventional social networking services like Twitter and Facebook, which have a tolerance for interaction delay, the interactions (e.g., chat messages) in a personal livecast must be in real-time with low feedback latency. These unique characteristics inspire us to: (1) investigate how the relationships (e.g., social links and geo-locations) between viewers and broadcasters influence the user behaviors, which has yet to be explored in depth; and (2) explore insights to benefit the improvement of system performance. In this article, we carry out extensive measurements of a representative MPL system, with a large-scale dataset containing 11M users. In the current costly and limited cloud-based MPL system, which is faced with scalability problem, we find: (1) the long content uploading distances between broadcasters and cloud ingesting servers result in an impaired system QoS, including a high broadcast latency and a frequently buffering events; and (2) most of the broadcasters in MPL are geographically locally popular (the majority of the views come from the same region of the broadcaster), which consume vast computation and bandwidth resources of the clouds and Content Delivery Networks. Fortunately, the emergence of edge computing, which provides cloud-computing capabilities at the edge of the mobile network, naturally sheds new light on the MPL system; i.e., localized ingesting, transcoding, and delivering locally popular live content is possible. Based on these critical observations, we propose an edge-assisted MPL system that collaboratively utilizes the core-cloud and abundant edge computing resources to improve the system efficiency and scalability. In our framework, we consider a dynamic broadcaster assignment to minimize the broadcast latency while keeping the resource lease cost low. We formulate the broadcaster scheduling as a stable matching with migration problem to solve it effectively. Compared with the current pure cloud-based system, our edge-assisted delivery approach reduces the broadcast latency by about 35%.

Crowdsourced content sharing has become a fast-growing activity in today's online social networks, where contents of interest are created by diverse source users and conveyed over the network as friends view and reshare. The rapid and boundless propagation in a mobile crowd however often creates severe bottlenecks on the server side and incurs significant energy and monetary costs on the mobile side, particularly with the still expensive 3G/4G cellular connections. This paper presents SoCrowd, a novel framework for large-scale content sharing in a mobile crowd by exploiting contacts, i.e., users happen to move close with such short range low power communications as WiFi and bluetooth being enabled. We formulate the scheduling problem for social content propagation in a mobile crowd with contacts, and present optimal dynamic programming solution, which further motivates a series of practical heuristics. The effectiveness of SoCrowd has been demonstrated by extensive simulations driven by realworld traces and datasets.

During the past decade, we have witnessed the pervasive penetration of mobile smart devices such as smartphones, tablets, and wearable devices, which significantly enrich Internet applications and improve user experience. In the foreseeable future, mobile smart devices are predicted to take up over 50% of global devices/connections and surpass 4/5 of mobile data traffic by 2021 [1]. Such mobile smart devices as smartphones, phablets, and tablets, undoubtedly reshape the way that users access Internet services, e.g., web browsing. Different from traditional desktop applications, in which users interact via interfaces like large displays, keyboards, and mouses, mobile applications require users to enter the inputs through touch screens and allow them to view the outputs on limited size of displays. This distinct feature introduced by mobile hardware interfaces brings both challenges and opportunities to mobile-based Internet applications. On one hand, mobile service providers should prepare multiple copies of media contents with different resolutions and even multiple versions of application UI layouts to fit various sizes of screens on heterogeneous devices. On the other hand, as media contents are usually organized in certain order in mobile-based Internet applications, it is possible to predict the viewing region (referred as viewport hereafter) given the user inputs and the fixed size of display.

There has been explosive growth in Mobile Personal Livestreaming (MPL) market since 2016. MPL services are booming not only because they introduce the popular live content by spontaneous and personalized broadcasters, but also because they are deliberately designed to be the innovative social networking service (SNS) platforms. The latter is a very important aspect that distinguishes MPL from the traditional livestreaming services. In this paper, we study the social networking of a large scale MPL service “Inke” (with more than 200 million registered users, 15 million daily active users) in China. By analyzing the dataset we crawl and the features of Inke app, we show that the social media stickiness of Inke comes from three aspects: the follower-followee model, the virtual-gift-based incentive mechanism, and the multi-perspective interactivity between broadcasters and viewers. First, Inke introduces the follower-followee model rather than the traditional broadcaster-viewer model, and every user in Inke can be a broadcaster. This makes MPL have some different patterns from both the traditional livestreaming services and SNS platforms. Second, Inke use virtual gift giving and user ranking as its incentive mechanism. Our measurement results show that this mechanism can indeed enhance user stickiness. Furthermore, Inke incorporates a variety of features during broadcasting to strengthen interactivity. The insight we gain in this paper has important implications for both existing and future designs.

Mobile personal livecast (MPL) services are emerging and have received great attention recently. Unlike traditional livecast services with commercial content providers (e.g., live TV), the live contents in MPL are crowdsourced from and consumed among geo-distributed individuals. Although there exist typical social relationships in MPL (i.e., follower-followee), different from conventional social networking services like Twitter and Facebook, which have much of a tolerance for interaction delay, the interactions in MPL must be in real-time. These unique characteristics intrigue us to investigate how the relationships (e.g., social links and geo-locations) between viewers and broadcasters influence the user behaviors, which has yet to be explored in depth. In this paper, we carry out extensive measurements of Inke, one of the most popular MPL providers, with a large-scale dataset containing 11M users. Our key findings are as follows. First, compared with traditional livecast services, the user interests shift much more frequently and the average viewing duration is considerably shorter in MPL. Second, the existence of social relationships significantly strengthens viewer stickiness---followers dedicating longer viewing time (contributing 81% of the total viewing time) and being 2x more patient when suffering poor network connectivity than non-followers. Third, most of the broadcasts in MPL are geographically local-popular (the majority of the views come from the same region of the broadcaster). Based on these critical observations, we provide insights that can enhance the MPL system design from the perspectives of efficient resource allocation and envision a future MPL framework that collaboratively utilizes the cloud and edge computing resources to improve efficiency and scalability for Inke-like services.

Batteries of modern mobile devices remain severely limited in capacity, which makes energy consumption a key concern for mobile applications, particularly for the computation-intensive video applications. Mobile devices can save energy by offloading computation tasks to the cloud, yet the energy gain must exceed the additional communication cost for cloud migration to be beneficial. The situation is further complicated by real-time video applications that have stringent delay and bandwidth constraints. In this paper, we closely examine the performance and energy efficiency of representative mobile cloud applications under dynamic wireless network channels and state-of-the-art mobile platforms. We identify the unique challenges of and opportunities for offloading real-time video applications and develop a generic model for energy-efficient computation offloading accordingly in this context. We propose a scheduling algorithm that makes adaptive offloading decisions in fine granularity in dynamic wireless network conditions and verify its effectiveness through trace-driven simulations. We further present case studies with advanced mobile platforms and practical applications to demonstrate the superiority of our solution and the substantial gain of our approach over baseline approaches.

In today's mobile devices, the battery reservoir remains severely limited in capacity, making power consumption a key concern in the design and implementation of mobile applications. In this paper, we closely examine one widely adopted approach to improve the energy efficiency of mobile applications-adaptively offloading the computation to the remote cloud. In particular, we measure the power consumption of computation offloading for two representative real-world mobile cloud applications under various wireless network conditions and identify the unique features of data transmission for computation offloading. We then formulate the power-aware scheduling problem for computation offloading and present a scheduling algorithm that makes adaptive offloading decisions according to the dynamic network conditions. Simulation results show that our proposed method can achieve better battery performance, which also reveal that computation-intensive and delay-tolerant tasks are more likely to benefit from offloading.

Nowadays technology advances of wireless networking and mobile devices have made anytime anywhere data access become readily available. This also enables crowdsourced content capturing and sharing, especially for such multimedia data as video. One example is Twitter's Vine, which mainly target mobile devices, allowing users to create ultra-short video clips and instantly share with their followers. In this paper, we take an initial study on this new generation of mobile instant video clip sharing service and explore the potentials towards its further enhancement. We closely investigate its unique mobile interface, featured user behaviors with screen scrolling, revealing the key differences between Vine-enabled anytime anywhere data access patterns and that of traditional counterparts. We then examine the scheduling policy to maximize the user watching experience as well as the cost efficiency. We show that the generic scheduling problem involves two subproblems, namely, pre-fetching scheduling and watch-time download scheduling, and develop effective solutions towards both of them. The superiority of our solution is demonstrated by extensive trace-driven simulations. To the best of our knowledge, this is the first work on modeling and optimizing the view experience of the instant video clip sharing service on mobile devices.

With the rapid development of mobile networking and end-terminals, anytime and anywhere data access becomes readily available nowadays. Given the crowd sourced content capturing and sharing, the preferred length becomes shorter and shorter, even for such multimedia content as video. A representative is Twitter's Vine service, which, mainly targeting mobile users, enables them to create ultra-short video clips, and instantly post and share them with their followers. In this paper, we present an initial study on this new generation of instant video clip sharing service enabled by mobile platforms and explore the potentials for its further enhancement. Taking Vine as a case study, we closely investigate its unique user behaviors, revealing how such Vine-enabled anytime anywhere data access patterns differentiate mobile instant video clip sharing from its traditional counterparts. We then formulate a generic scheduling problem to maximize the user watching experience as well as the efficiency on the monetary and energy costs. To better solve it, we divide the problem into two sub problems, specifically, the pre-fetching scheduling problem and the watch-time download scheduling problem, and conquer them separately. We further demonstrate the preliminary evaluation result to show the superiority of our solution. To the best of our knowledge, this is the first work on modeling and optimizing the instant video clip sharing on mobile devices.

YouTube is arguably the most popular online videos sharing site nowadays. To further augment its service with better revenue, it has started working with content owners (known as YouTube partners) whose copyrighted videos and channels have pulled massive audience. By uploading high-quality premium videos, the partners have essentially changed the user-generated content feature of YouTube and further increased YouTube's popularity. Understanding the latest YouTube access pattern is thus crucial to both YouTube and its partners, as well as to other providers of relevant services. In this paper, we for the first time analyze a large-scale YouTube dataset from a partner's view. We make effective use of Insight, a new analytics service of YouTube that offers inside statistics for partners about their content accesses and audience behaviours. From the raw Insight data that are confined to simple scalars and charts, we reveal the inherent relationship among the various metrics that affect the popularity of the videos. Our findings facilitate YouTube partners to adapt their content deployment and user engagement strategies, having great potentials for them to collaborate with YouTube to generate more views and subsequently increasing their revenues.

With the rapidly development of mobile networking and end-terminals, anytime and anywhere data access become readily available nowadays. Given the crowdsourced content capturing and sharing, the preferred length becomes shorter and shorter, even for such multimedia content as video. A representative is Twitter's Vine service, which, available exclusively to mobile users, enables them to create ultra-short video clips, and instantly post and share them with their followers. In this paper, we present an initial study on this new generation of instant video clip sharing service over mobile platforms, taking Vine as a case. We closely investigate the architecture of Vine, and reveal how its service is empowered with a combination of advanced mobile and cloud computing platforms. Through a dataset of over 50, 000 video clips and over 1, 000, 000 user profiles, which is available online for academic use, we examine the unique viewing behaviors of Vine uses, particularly batch viewing and passive viewing. We further analyze the video lifetime and propagation patterns in this new service, as well as the distinct social relations therein. Our study lead to critical observations that would help with improving the energy-efficiency and scalability of Vine-like services.

MAP is a connectivity-based routing protocol aimed at improving the load balance performance of traditional geographical routing methods. It attempts to find parallel routing paths by taking advantage of the concept of skeleton in the continuous domain. However, MAP suffers seriously from overloading the sensor nodes that are close to the skeleton. In this paper, we propose a contour-based routing protocol, CAR, that does not require geographical information, produces short routing paths, and achieves outstanding load balancing. Our experimental results show that CAR outperforms MAP in terms of both load balancing and routing path length.

Video streaming has become one of the most popular networked applications and, with the increased bandwidth and computation power of mobile devices, anywhere and anytime streaming has become a reality. Unfortunately, it remains a challenging task to compress high-quality video in real-time in such devices given the excessive computation and energy demands of compression. On the other hand, transmitting the raw video is simply unaffordable from both energy and bandwidth perspective.