The generic matrix multiply (GEMM) routine comprises the compute and memory-intensive parts of many information retrieval, machine learning, and object recognition systems that process integer inputs. Therefore, it is of paramount importance to ensure that integer GEMM computations remain robust to silent data corruptions (SDCs), which stem from accidental voltage or frequency overscaling, or other hardware nonidealities. In this paper, we introduce a new method for SDC mitigation based on the concept of numerical packing. The key difference between our approach and all existing methods is the production of redundant resultswithinthe numerical representation of the outputs, rather than as a separate set of checksums. Importantly, unlike well-known algorithm-based fault tolerance approaches for GEMM, the proposed approach can reliably detect the locations of the vast majority of all possible SDCs in the results of GEMM computations. An experimental investigation of voltage-scaled integer GEMM computations for visual descriptor matching within state-of-the-art image and video retrieval algorithms running on an Intel i7-4578U 3 GHz processor shows that SDC mitigation based on numerical packing leads to comparable or lower execution and energy-consumption overhead in comparison with all other alternatives.

Delay-sensitive applications, such as live and interactive video, are mainstream in today's Internet, and set to increase with the emergence of web-based video conference. For such applications quality cannot be captured solely by a flow's throughput: interactive video needs to avoid fluctuations in both visual quality and delay. This is far more valuable than achieving a fleeting increase in throughput. Emerging real-time protocols are being designed with these goals in mind, but it is important to evaluate these methods when sharing the network with real-world traffic. Much of today's Internet traffic uses TCP CUBIC, we therefore quantify its impact on interactive video experience (visual quality and delay): we measure the degradation imposed to interactive video when sharing a network bottleneck with TCP CUBIC traffic. To understand this impact, we compare performance when two delay-based congestion control schemes are used, one for interactive video and another for file transfer and show that these algorithms can assure good video experience and appropriate download time to their respective users. In contrast, we suggest that achieving such a “low-delay coexistence” with TCP CUBIC would require use of Active Queue Management (AQM) techniques. The paper therefore provides quantitative evidence that AQM can force loss-based TCP and delay-sensitive flows to reach a stable equilibrium point that is similar to the one naturally achieved when TCP flows are governed by delay-based mechanisms.

With multimedia and Internet enabled devices being ubiquitous, mechanisms that ensure multimedia flows do not congest the Internet are crucial components of multimedia systems that are embraced rather than opposed by network service providers. The Real-time Transport Protocol (RTP) Circuit Breaker is designed to terminate RTP/UDP flows that cause excessive congestion in the network. Multimedia users congesting the network have their flows terminated, as dictated by the RTP circuit breaker congestion rule. Users who obtain little quality from a multimedia session, and consume network resources to no avail, should also cease transmission. This is the mandate of the RTP circuit breaker media usability rule. We propose an algorithm for this rule, and show that it avoids wasting network resources on flows that deliver no quality to the user.

Real-time media applications often ignore ongoing congestion if there is no option to reduce the rate. These applications pose a threat to themselves and other traffic. Reducing the transmission rate requires reducing the amount of packets rather than spreading the transmission over a longer interval. Loss-based congestion control mechanisms are unsuitable for this requirement. Also this rate reduction with popular video codecs, e.g. MPEG4, is often problematic. This paper investigates the problems associated with real-time video transmission over the Internet. We investigate a rate control method minimizing delay and losses and report preliminary but promising results.

The generic matrix multiply (GEMM) routine comprises the compute and memory-intensive part of many information retrieval, relevance ranking and object recognition systems. Because of the prevalence of GEMM in these applications, ensuring its robustness to transient hardware faults is of paramount importance for highly-efficientlhighly-reliable systems. This is currently accomplished via error control coding (ECC) or via dual modular redundancy (DMR) approaches that produce a separate set of “parity” results to allow for fault detection in GEMM. We introduce a third family of methods for fault detection in integer matrix products based on the concept of numerical packing. The key difference of the new approach against ECC and DMR approaches is the production of redundant results within the numerical representation of the inputs rather than as a separate set of parity results. In this way, high reliability is ensured within integer matrix products while allowing for: (i) in-place storage; (ii) usage of any off-the-shelf 64-bit floating-point GEMM routine; (iii) computational overhead that is independent of the GEMM inner dimension. The only detriment against a conventional (i.e. fault-intolerant) integer matrix multiplication based on 32-bit floating-point GEMM is the sacrifice of approximately 30.6% of the bitwidth of the numerical representation. However, unlike ECC methods that can reliably detect only up to a few faults per GEMM computation (typically two), the proposed method attains more than “12 nines” reliability, i.e. it will only fail to detect 1 fault out of more than 1 trillion arbitrary faults in the GEMM operations. As such, it achieves reliability that approaches that of DMR, at a very small fraction of its cost. Specifically, a single-threaded software realization of our proposal on an Intel i7-3632QM 2.2GHz processor (Ivy Bridge architecture with AVX support) incurs, on average, only 19% increase of execution time against an optimized, fault-intolerant, 32-bit GEMM routine over a range of matrix sizes and it remains more than 80% more efficient than a DMR-based GEMM.

In video communication systems, due to quantization and transmission errors, mismatches between the transmitter- and receiver-side information may occur, severely impacting the reconstructed video. Theoretical understanding of the quality degradation ensuing from such mismatches is essential when targeting quality-of-service for video communications. In this paper, by viewing the mismatches in the transform coefficients and the adaptive parameters of the temporal analysis of a video coding system as perturbations in the synthesis system, we derive analytic approximations for the expected reconstruction distortion. Our theoretical results are experimentally assessed using adaptive temporal decompositions within a video coding system based on motion-adaptive temporal lifting decomposition. Since we focus on the generic case of adaptive lifting transforms, our results can provide useful insights for estimation-theoretic resiliency mechanisms to be considered within standardized transform-based codecs.

Video source, carrier and client diversification have led the video coding community to develop scalable video codecs supporting efficient decoding at varying resolution, frame rate and quality. Scalable video has several advantages over a nonscalable approach, but a large scale deployment is far from trivial and a lot of open questions remain. To resolve these, we developed a complete video delivery chain for scalable wavelet-based video. This includes a video server, a negotiation framework, a video scaling infrastructure and two scalable video clients, one pure software client and one real-time, hardware accelerated client. This paper describes the complete chain and identifies and quantifies the impact of using scalable video in every link of this chain.

Real-time delivery of video over best-effort error-prone packet networks requires scalable erasure-resilient compression systems in order to 1) meet the users' requirements in terms of quality, resolution, and frame-rate; 2) dynamically adapt the rate to the available channel capacity; and 3) provide robustness to data losses, as retransmission is often impractical. Furthermore, the employed erasure-resilience mechanisms should be scalable in order to adapt the degree of resiliency against transmission errors to the varying channel conditions. Driven by these constraints, we propose in this paper a novel design for scalable erasure-resilient video coding that couples the compression efficiency of the open-loop architecture with the robustness provided by multiple description coding. In our approach, scalability and packet-erasure resilience are jointly provided via embedded multiple description scalar quantization. Furthermore, a novel channel-aware rate-allocation technique is proposed that allows for shaping on-the-fly the output bit rate and the degree of resiliency without resorting to channel coding. As a result, robustness to data losses is traded for better visual quality when transmission occurs over reliable channels, while erasure resilience is introduced when noisy links are involved. Numerical results clearly demonstrate the advantages of the proposed approach over equivalent codec instantiations employing 1) no erasure-resilience mechanisms, 2) erasure-resilience with nonscalable redundancy, or 3) data-partitioning principles.

Real time delivery of video over best-effort networks requires compression systems that dynamically adapt the rate to the available channel capacity and exhibit robustness to data loss as retransmission is often impractical. Error-resilience, however, significantly lowers the coding performance when rigid design is performed based on a worst-case scenario. This paper presents an original scalable video coding scheme that couples the compression efficiency of the open-loop architecture with the robustness of multiple description source coding. The use of embedded multiple description quantization and a novel channel-aware rate-allocation allow for shaping on-the-fly the output bit-rate and the degree of resilience. As a result, robustness to data losses is traded for better visual quality when transmission occurs over reliable channels, while error-resilience is introduced when noisy links are involved. The advantage of our proposal is demonstrated in the context of packet-lossy networks.

Recently proposed scalable wavelet-based video codecs using spatial-domain motion compensated temporal filtering (SDMCTF) offer competitive compression performance when compared to H.264 and generate embedded bit-streams supporting quality, resolution and temporal scalability. To be able to support a large range of bit-rates with optimal compression efficiency, these codecs require a quality-scalable motion vector coding technique. Such an algorithm based on the integer wavelet transform followed by embedded coding of the wavelet coefficients was proposed in the recent past. In this paper, we present a quality-scalable motion vector coding algorithm using median-based motion vector prediction. The compression performance of the proposed algorithm is compared to that of the wavelet-based technique and is found to be superior. Additionally, the proposed motion vector codec is incorporated into an SDMCTF-based video codec and the benefits of using quality-scalable motion vector representations are experimentally demonstrated.

An enhanced, off-line, segment-based rate control approach is proposed for controlling the distortion variation across successive segments of a video sequence when encoding with single-layer (MPEG-4 Baseline, MPEG-4 AVC) and wavelet video codecs. Consistent quality is achieved by a time-efficient, predictive rate-distortion modeling per segment. The individual segments are either shots or subshots, that are defined based on shot segmentation and activity analysis techniques. The rate control method solves the quality stability problem of state-of-the-art codecs, especially for less mature rate control modules (such as present in MPEG-4 AVC). The processing overhead compared to classical two-pass VBR encoding is limited while the distortion variation is significantly reduced.

Video coding techniques yielding state-of-the-art compression performance require large amount of computational resources, hence practical implementations, which target a broad market, often tend to trade-off coding efficiency and flexibility for reduced complexity. Scalable video coding instead, not only provides seamless adaptation to bit-rate variation, but also allows the end user to trim down the resources he needs to perform real-time decoding by limiting the process to a subset of the original content. Hence, by choosing the quality, frame-rate and/or resolution of the reconstructed sequence, each decoder can meet its hardware limitations without affecting the encoding process of the media provider. This paper proposes a preliminary analysis of the memory-access behavior of a fully scalable video decoder and investigates the capability of selecting the operational settings in order to adapt to the available hardware resources on the target device.