In this work, we explore a new communication paradigm for immersive 3D video conferencing, termed parameterized communication, which dramatically reduces bandwidth usage by eliminating the need to exchange excessive volumetric data. Instead, this approach extracts a compact set of informative parameters representing key elements in the 3D space, transmits only these parameters, and reconstructs the scene on the receiving end. Translating this concept into practice, we present Empilo, a mobile 3D conferencing system composed of a face parameter extractor and a neural rendering-based scene generator. However, while neural rendering excels at synthesizing arbitrary views of objects without explicit 3D models, its heavy computational demands present a major obstacle for mobile deployment. To overcome this challenge, we propose a novel technique called truncated ray marching, which significantly reduces computational overhead by replacing iterative MLP inferences with a single-pass of a shallow neural network. Furthermore, to ensure a consistently immersive experience, we structure the neural-free lightweight renderer as a decoupled component, dedicated to delivering rapid responsiveness to dynamic viewpoint changes. These breakthroughs on computation together enable Empilo to rely entirely on mobile resources, achieving real-time performance with a frame generation time of 30.3 ms and a re-rendering latency of just 6.6 ms—all while operating at an exceptionally low bitrate of 24 kbps. Our approach provides valuable guidance for the practical deployment of 3D conferencing, envisioning accessibility on par with platforms like FaceTime and Zoom.

In this work, we explore the possibility of a new delivery method for lossless data, namely compressive transmission. It aims at minimizing the transmission data volume at runtime by exploiting the tailored information shared between the sender and the receiver. There are two approaches to leverage shared information for compression: 1) using a DNN-based codec as a proxy for shared information and 2) applying redundancy elimination using deduplication. However, these approaches have not been studied in depth to utilize the trade-off between the compression rate and the amount of shared information. Compared to these approaches, compressive transmission is unique as it fully leverages the abundance of information available on both sides, which is chosen and placed purposely. To bring the concept to reality, we propose nCTX, a neural network-powered Compressive Transmission System that adaptively exploits a generative model and matching blocks. nCTX extracts the optimal semantic data from the input data, exploiting shared information to closely imitate the original and compensate it with the offset (i.e., difference). Extensive evaluations in mobile platforms confirm that nCTX reduces the transmission volume significantly by 25.8% and 23.3% compared to FLIF and RC, the state-of-the-art image codecs, respectively, in comparable or shorter computation times.

Offloading neural network inferences from resource-constrained mobile devices to an edge server over wireless networks is becoming more crucial as the neural networks get heavier. To this end, recent studies have tried to make this offloading process more efficient. However, the most fundamental question on extracting and offloading the minimal amount of necessary information that does not degrade the inference accuracy has remained unanswered. We call such an ideal offloading semantic offloading and propose N-epitomizer, a new offloading framework that enables semantic offloading, thus achieving more reliable and timely inferences in highly-fluctuated or even low-bandwidth wireless networks. To realize N-epitomizer, we design an autoencoder-based scalable encoder trained to extract the most informative data and scale its output size to meet the latency and accuracy requirements of inferences over a network. We also accelerate N-epitomizer by exploiting light-weight knowledge distillation for the encoder design and decoder slimming for the decoder design, reducing its overall computation time significantly. Moreover, we extend our N-epitomizer to support multiple DNNs by extracting and offloading the union of the essential information required for each DNN. Our evaluation shows that N-epitomizer achieves exceptionally high compression for images without compromising inference accuracy, which is 21$\times$, 77$\times$, and 192$\times$higher than JPEG compression, and 20$\times$, 55$\times$, and 86$\times$higher than the state-of-the-art DNN-aware image compression GRACE for semantic segmentation, depth estimation, and classification, respectively. Our results show N-epitomizer’s strong potential as the first semantic offloading system to guarantee end-to-end latency even under highly varying cellular networks.

In a variety of network applications, there exists a significant amount of shared data between two end hosts. Examples include data synchronization services that replicate data from one node to another. Given that shared data may have a high correlation with new data to transmit, we question how such shared data can be best utilized to improve the efficiency of data transmission. To answer this, we develop an inter-data encoding technique, SyncCoding, that effectively replaces bit sequences of the data to be transmitted with the pointers to their matching bit sequences in the shared data so called references. By doing so, SyncCoding can reduce data traffic, speed up data transmission, and save energy consumption for transmission. Our evaluations of SyncCoding implemented in Linux show that it outperforms existing popular encoding techniques, Brotli, LZMA, Deflate, and Deduplication. The gains of SyncCoding over those techniques in the perspective of data size after compression in a cloud storage scenario are about 12.5, 20.8, 30.1, and 66.1 percent, and are about 78.4, 80.3, 84.3, and 94.3 percent in a web browsing scenario, respectively.

In this work, we raise a question on why the abundant information previously shared between a server and its client is not effectively utilized in the exchange of a new data which may be highly correlated with the shared data. We formulate this question as an encoding problem that is applicable to general data synchronization services including a wide range of Internet services such as cloud data synchronization, web browsing, messaging, and even data streaming. To this problem, we propose a new encoding technique, SyncCoding that maximally replaces subsets of the data to be transmitted with the coordinates pointing to the matching subsets included in the set of relevant shared data, called references. SyncCoding can be easily integrated into a transport layer protocol such as HTTP and enables significant reduction of network traffic. Our experimental evaluations of SyncCoding implemented in Linux shows that it outperforms existing popular encoding techniques, Brotli, LZMA, Deflate, and Deduplication in two practical use networking applications: cloud data sharing and web browsing. The gains of SyncCoding over Brotli, LZMA, Deflate, and Deduplication in the encoded size to be transmitted are shown to be about 12.4%, 20.1%, 29.9%, and 61.2% in the cloud data sharing and about 78.3%, 79.6%, 86.1%, and 92.9% in the web browsing, respectively. The gains of SyncCoding over Brotli, LZMA, and Deflate when Deduplication is applied in advance are about 7.4%, 10.6%, and 17.4% in the cloud data sharing and about 79.4%, 82.0%, and 83.2% in the web browsing, respectively.