Recent years have seen a tremendous growth in both the capability and popularity of automatic machine analysis of media, especially images and video. As a result, a growing need for efficient compression methods optimised for machine vision, rather than human vision, has emerged. To meet this growing demand, significant developments have been made in image and video coding for machines. Unfortunately, while there is a substantial body of knowledge regarding rate-distortion theory for human vision, the same cannot be said of machine analysis. In this paper, we greatly extend the current rate-distortion theory for machines, providing insight into important design considerations of machine-vision codecs. We then utilise this newfound understanding to improve several methods for learned image coding for machines. Our proposed methods achieve state-of-the-art rate-distortion performance on several computer vision tasks - classification, instance and semantic segmentation, and object detection.

Thanks to advances in computer vision and AI, there has been a large growth in the demand for cloud-based visual analytics in which images captured by a low-powered edge device are transmitted to the cloud for analytics. Use of conventional codecs (JPEG, MPEG, HEVC, etc.) for compressing such data introduces artifacts that can seriously degrade the performance of the downstream analytic tasks. Split-DNN computing has emerged as a paradigm to address such usages, in which a DNN is partitioned into a client-side portion and a server side portion. Low-complexity neural networks called ‘bottleneck units' are introduced at the split point to transform the intermediate layer features into a lower-dimensional representation better suited for compression and transmission. Optimizing the pipeline for both compression and task-performance requires high-quality estimates of the information-theoretic rate of the intermediate features. Most works on compression for image analytics use heuristic approaches to estimate the rate, leading to suboptimal performance. We propose a high-quality ‘neural rateestimator’ to address this gap. We interpret the lower-dimensional bottleneck output as a latent representation of the intermediate feature and cast the rate-distortion optimization problem as one of training an equivalent variational auto-encoder with an appropriate loss function. We show that this leads to improved rate-distortion outcomes. We further show that replacing supervised loss terms (such as cross-entropy loss) by distillation-based losses in a teacher-student framework allows for unsupervised training of bottleneck units without the need for explicit training labels. This makes our method very attractive for real world deployments where access to labeled training data is difficult or expensive. We demonstrate that our method outperforms several state-of-the-art methods by obtaining improved task accuracy at lower bi-trates on image classification and semantic segmentation tasks.

Split computing has emerged as a recent paradigm for implementation of DNN-based AI workloads, wherein a DNN model is split into two parts, one of which is executed on a mobile/client device and the other on an edge-server (or cloud). Data compression is applied to the intermediate tensor from the DNN that needs to be transmitted, addressing the challenge of optimizing the rate-accuracy-complexity trade-off. Existing split-computing approaches adopt ML-based data compression, but require that the parameters of either the entire DNN model, or a significant portion of it, be retrained for different compression levels. This incurs a high computational and storage burden: training a full DNN model from scratch is computationally demanding, maintaining multiple copies of the DNN parameters increases storage requirements, and switching the full set of weights during inference increases memory bandwidth. In this paper, we present an approach that addresses all these challenges. It involves the systematic design and training of bottleneck units - simple, low-cost neural networks - that can be inserted at the point of split. Our approach is remarkably lightweight, both during training and inference, highly effective and achieves excellent rate-distortion performance at a small fraction of the compute and storage overhead compared to existing methods.

Video traffic comprises a large majority of the total traffic on the internet today. Uncompressed visual data requires a very large data rate; lossy compression techniques are employed in order to keep the data-rate manageable. Increasingly, a significant amount of visual data being generated is consumed by analytics (such as classification, detection, etc.) residing in the cloud. Image and video compression can produce visual artifacts, especially at lower data-rates, which can result in a significant drop in performance on such analytic tasks. Moreover, standard image and video compression techniques aim to optimize perceptual quality for human consumption by allocating more bits to perceptually significant features of the scene. However, these features may not necessarily be the most suitable ones for semantic tasks. We present here an approach to compress visual data in order to maximize performance on a given analytic task. We train a deep auto-encoder using a multi-task loss to learn the relevant embeddings. An approximate differentiable model of the quantizer is used during training which helps boost the accuracy during inference. We apply our approach on an image classification problem and show that for a given level of compression, it achieves higher classification accuracy than that obtained by performing classification on images compressed using JPEG. Our approach also outperforms the relevant state-of-the-art approach by a significant margin.

Efficient radio resource allocation is a fundamental optimization problem for wireless networks, and has been widely studied in the past. However, wireless systems are evolving to have a much larger parameter space along with richer set of applications and user requirements leading to significant increase in complexity. This paper draws from recent breakthroughs in applying deep reinforcement learning (RL) for control problems containing large state space dimensionality. In particular, a deep RL approach is explored for the problem of allocating time and frequency resources in OFDMA wireless systems to optimize different objective functions using per-station channel quality and traffic information as inputs. Such approaches hold the potential for agents to learn resource allocation and scheduling policies directly from experience rather than using carefully crafted heuristic algorithms based upon models of the environment and stations. Learning directly from experience also means that the policies that result from the online learning should be more robust to imperfect inputs such as noisy, delayed or missing information compared to model-based heuristic approaches. The results in this paper show promise for a deep RL agent using a policy gradient algorithm to learn policies that approach or exceed the performance of well-known model-based approaches such as max-weight and proportional fair policies. The online adaptation algorithms used for the deep RL agent also demonstrate reasonable adaptability and robustness to varying traffic conditions.

There are four major components in application systems with internet-of-things (IoT): sensors, communications, computation and service, where large amount of data are acquired for ultra-big data analysis to discover the context information and knowledge behind signals. To support such large-scale data size and computation tasks, it is not feasible to employ centralized solutions on cloud servers. Thanks for the advances of silicon technology, the cost of computation become lower, and it is possible to distribute computation on every node in IoT. In this paper, we take video sensing network as an example to show the idea of distributed computing in IoT. Existing related works are reviewed and the architecture of a system-on-a-chip solution for distributed smart cameras is proposed with coarse-grained reconfigurable image stream processing architecture. It can accelerate various computer vision algorithms for distributed smart cameras in IoT.

Techniques of video summarization have attracted significant research interests in the past decade due to the rapid progress in video recording, computation, and communication technologies. However, most of the existing methods analyze the video in an off-line manner, which greatly reduces the flexibility of the system. On-line summarization, which can progressively process video during video recording, is then proposed for a wide range of applications. In this paper, an on-line summarization method using Gaussian mixture model is proposed. As shown in the experiments, the proposed method outperforms other on-line methods in both summarization quality and computational efficiency. It can generate summarization with a shorter latency and much lower computation resource requirements.

The tremendous growth of video content over the Internet and evolution towards more personalized video applications has led to a need for a network and service infrastructure better suited to today's content and mobility needs. Quality of Experience (QoE) management is one of the major challenges for video delivery given the limited bandwidth for wireless access network causing video packet drops and the core network congestion impacting video service delay. In this context, the current TCP-based model used for HTTP streaming presents challenges due to throughput variation and high latency. We present a context-adaptive cross-layer optimization approach to enhance video streaming over TCP that introduces network and content awareness in TCP operation to provide adaptive reliability and smoother throughput. Our proposed solution requires implementation changes only on the receiver side without impacting the TCP sender implementation. We validated the proposed solution through OPNET simulations and the results show some significant performance gains in both TCP throughput and video QoE metrics.

Distributed video coding (DVC) is a potential solution for distributed video sensors in wireless visual sensor and machine-to-machine (M2M) networks. However, the error resilience schemes have not been fully investigated in literatures. In this paper, we propose several error resilience schemes for key frame transmission of DVC, including resending, refining, and hybrid modes. The resending mode asks the transmitters to resend the lost packets as automatic repeat-request, the refining mode conceals the corrupted key frames by temporal error concealment techniques with considering the characteristics of DVC, and the hybrid mode can adaptively select the better mode packet-by-packet with the proposed rate-distortion optimized mode decision. Experimental results show that the proposed schemes can achieve around 6-dB gain in PSNR compared with a baseline approach with intra error concealment. Furthermore, the hybrid mode can achieve 0.5-dB gain in PSNR for some sequences compared with the better one among the resending and refining modes.

Video sensors are widely used in many applications such as security monitoring and home care. However, the growth of the number of sensors makes it impractical to stream all videos back to a central server for further processing, due to communication bandwidth and server storage constraints. Multi-view video summarization allows us to discard redundant data in the video streams taken by a group of sensors. All prior multi-view summarization methods, however, process video data in an off-line and centralized manner, which means that all videos are still required to be streamed back to the server before conducting the summarization. This paper proposes an on-line, distributed multi-view summarization system, which integrates the ideas of Maximal Marginal Relevance (MMR) and MS-Wave, a bandwidth-efficient distributed algorithm for finding k-nearest-neighbors and k-farthest-neighbors. Empirical studies show that our proposed system can discard redundant videos and keep important keyframes as effectively as centralized approaches, while transmitting only 1/6 to 1/3 as much data.

Low-power wireless video sensor nodes play important roles for applications in machine-to-machine (M2M) network. Several design issues to optimize the power consumption of a video sensor node are addressed in this paper. For the video coding engine selection, the comparison between conventional video coding system and distributed video coding (DVC) system shows that although the rate-distortion performance of existing DVC codec still has room to improve, it can provide lower power consumption with a noisy transmission channel. Furthermore, it also demonstrated that video analysis unit can help to filter out video contents without event-of-interest to reduce transmission power. Finally, several future research directions are addressed, and the trade-off between the video analysis unit, video coding unit, and data transmission should be further studied to design wireless video sensors with optimized power consumption.

Distributed video coding (DVC), a new video coding paradigm based on Slepian-Wolf and Wyner-Ziv theories, is a promising solution for implementing low-power and low-cost distributed wireless video sensors since most of the computation load is moved from the encoder to the decoder. It has been showed that there is still room to improve the coding efficiency of the current DVC codec. In this paper, we propose a hybrid coding structure with frame-level coding mode selection (CMS) to allow the DVC encoder to flexibly choose channel coding or entropy coding to code each band. The experimental results show the significant improvement on the rate-distortion (R-D) performance with only slight increase in the encoding complexity compared to the DISCOVER codec. The proposed DVC system performs comparably to H.264 No Motion with much lower encoding complexity.

We formulate a PHY/APP cross-layer resource allocation problem for efficient transmission of video streams in a multiple-user LTE downlink system. To achieve a predetermined video PSNR, the number of bits needed for different users is calculated using a parameterized Rate-Distortion (RD) model proposed by [4]. We design an admission policy which assigns frequency-time OFDMA channel resource to video users based on a cross layer metric. For the admitted users, we use a tracking algorithm to serve the users as the video source content and channel conditions evolve. The goal of the resource allocation is to maximize the number of admitted users from a fixed user population and guarantee that the video quality of admitted users will achieve the predetermined PSNR threshold. The performance of the proposed algorithms is compared to four baseline algorithms which exploit different levels of cross layer knowledge about the channel and source characteristics, and the results show benefits from cross-layer optimization.

Low-power and low-cost distributed wireless video sensors play important roles for applications in machine-to-machine (M2M) and wireless sensor networks. Distributed video coding (DVC), an emerging coding technology based on Wyner-Ziv theory, seems to be a possible solution for implementing low-power video sensors since most of the computational complexity is moved from the encoder to the decoder. In this paper, existing works on DVC are discussed with rate-distortion and power consumption analyses compared with H.264/AVC-based approaches. We show that, since more transmission power is required for compensating the lower rate-distortion performance, the power consumption of sensor nodes using DVC is just similar to that of using H.264/AVC with zero motion vectors. Therefore, there is still a room for improvement to make DVC applicable for distributed wireless video sensors. Based on our analysis results, several possible research directions, such as studies on the trade- off between hardware cost and system power consumption, are also addressed in this paper under a unified DVC framework.

The paper provides a tutorial overview of ultrawideband (UWB) radio technology for high-speed wireless connectivity. Subsequent to establishing a historical and technological context, it describes the new impetus for UWB systems development and standardization resulting from the FCC's recent decision to permit unlicensed operation in the 3.1-10.6 GHz band subject to modified Part 15 rules and indicates the potential new applications that may result. Thereafter, the paper provides a system architect's perspectives on the various issues and challenges involved in the design of link layer subsystems. Specifically, we outline current developments in UWB system design concepts that are oriented to high-speed applications and describe some of the design tradeoffs involved.

The performance of a RAKE receiver for a pulse based ultra-wideband (UWB) communications system is studied in a realistic channel model that is based on an extensive set of indoor channel measurements. The RAKE receiver is shown to contribute to a mitigation of the ISI. In particular, at low input SNR values and small number of RAKE taps, it is shown that employing additional RAKE taps for energy capture is more important to overall system performance than employing equalization to combat the ISI. The performance of the RAKE with some realistic channel estimation errors is also studied.

Systems using ultra-wideband (UWB) technology have been shown to achieve very high data rates (100 Mbps and above) for short-range, indoor applications. UWB signals exhibit a number of performance advantages in terms of multipath resolution and fading reduction. However, UWB systems can also exhibit significant implementation complexity in timing, synchronization, and signal correlation. This paper presents a differential detector that rakes in some of the multipath energy, while relaxing some of the stringent implementation requirements. We analyze the performance and the implementation of the differential detector, and compare it to a RAKE receiver with a correlator detector in the presence of AWGN and multipath.

Ultra-wideband radio systems operate using extremely short duration signaling pulses. Spread spectrum techniques for multiple access and interference suppression are commonly considered for such systems. We compare time-hopping spread spectrum with pulse position modulation (TH-PPM) and time-hopping/direct-sequence spreading with antipodal signaling (TH/DS-BPSK) in terms of their multiple access performance, and present some arguments for the use of one versus the other in a given system.

The architecture and implementation of a novel VLSI bit-level systolic Vector Quantization processor is described. This architecture offers very high data throughput for real-time processing applications. Given a codebook of sizeNand dimensionk, an input vector can be quantized everyNclock cycles, compared to O(kN) cycles for a Single-Instruction, Single-Data (SISD) machine. Any distortion measure which can be expressed as a Euclidean vector inner product can be computed with this array.