Complex application specific SoC are being developed for hardware support artificial intelligence (AI) applications. Such a complex SoCs are integrating a large number of on-chip and off-chip memories, numerous cores and interfaces including in our case a hierarchy of embedded TAPs, as well as security measures and Design-For-Test (DFT) structures. In this case-study paper, we demonstrate using IEEE 1687-2014 (IJTAG) to integrate all these different components into a single, unifying methodology. From this, we derive the benefits of workflow efficiency and fast silicon bring-up. For example, we can report that silicon bring-up of the DFT of the entire SoC was completed in about 4 days, and other bring-up aspects of the Soc were also completed in very little time.

In a previous work [1] we applied the idea of HEVC intra coding to compression of 32 b/p seismic images. Results are significantly better than those from a licensed commercial wavelet-based codec that is currently used at Shell for seismic image compression, which performs on par with JPEGXR. Building upon [1], this paper exploits HEVC-based inter predictive coding for 3D 32 b/p seismic data. We propose a new model for the Lagrange multiplier in R-D optimization to accommodate 32 b/p bit-depth and extended quantization parameter range. We also focus on reducing the complexity of motion estimation to meet application needs. Experiments with our new codec show a 95% complexity reduction of encoding time at 10:1 compression ratio with only 1 dB loss on average PSNR. The compression performance and subjective quality of our codec received high evaluation marks from Shell geologists.

Inspired by the high performance of High Efficiency Video Coding (HEVC), this paper reports our work on applying the ideas of HEVC intra coding to compression of high-depth images such as 32 bits per pixel (b/p) seismic data. Compared to a licensed commercial wavelet-based codec that is currently used for seismic image compression, which performs on par with JPEG-XR, our new image codec significantly improves the PSNR vs. compression ratio performance. The codec's subject performance is rated by geologist as highly satisfactory.

Identifying “interesting” features, such as faults, unconformities, and other events in subsurface images is a challenging task in seismic data processing. Existing state-of-the-art methods usually involve manual intervention in the form of a visual inspection by an expert, but this is time-consuming, expensive, and error-prone. In this paper, we propose an efficient, automatic approach for seismic feature extraction. The core idea of our approach involves interpreting a given 2D seismic image as a function defined over the vertices of a specially chosen underlying graph. This enables us to formulate the feature extraction task as an instance of the Prize-Collecting Steiner Tree problem encountered in combinatorial optimization. We develop an efficient algorithm to solve this problem, and demonstrate the utility of our method on a number of synthetic and real examples.

Localizing fault lines and surfaces in seismic subsurface images is a daunting challenge. Existing state-of-the-art approaches usually involve visual interpretation by an expert, but this is time-consuming, expensive and error-prone. In this paper, we propose some initial steps towards a new algorithmic framework for automatic fault localization. The core of our approach is a deterministic model for 2D images that we call the Constrained Generalized Earth Mover's Distance (CGEMD) model. We propose an algorithm that returns the best approximation in the model for any given input 2D image X; the output of this algorithm is then post-processed to reveal the locations of the faults in the image. We demonstrate the validity of this approach on a number of synthetic and real-world examples.

Blue Gene/Q (BG/Q) is an early representative of increasing scale and thread count that will characterize future HPC systems: large counts of nodes, cores, and threads; and a rich programming environment with many degrees of freedom in parallel computing optimization. So it is both a challenge and an opportunity to it to accelerate the seismic imaging applications to the unprecedented levels that will significantly advance the technologies for the oil and gas industry. In this work we aim to address two important questions: how HPC systems with high levels of scale and thread count will perform in real applications; and how systems with many degrees of freedom in parallel programming can be calibrated to achieve optimal performance. Based on BG/Q's architecture features and RTM workload characteristics, we developed massive domain partition, MPI , and SIMD Our detailed deep analyses in various aspects of optimization also provide valuable experience and insights into how can be utilized to facilitate the advance of seismic imaging technologies. Our BG/Q RTM solution achieved a 14.93x speedup over the BG/P implementation. Our multi-level parallelism strategies for Reverse Time Migration (RTM) seismic imaging computing on BG/Q provides an example of how HPC systems like BG/Q can accelerate applications to a new level.

We present an optimized Blue Gene/P implementation of Reverse Time Migration, a seismic imaging algorithm widely used in the petroleum industry today. Our implementation is novel in that it uses large communication bandwidth and low latency to convert an embarrassingly parallel problem into one that can be efficiently solved using massive domain partitioning. The success of this seemingly counterintuitive approach is the result of several key aspects of the imaging problem, including very regular and local communication patterns, balanced compute and communication requirements, scratch data handling, multiple-pass approaches, and most importantly, the fact that partitioning the problem allows each sub-problem to fit in cache, dramatically increasing locality and bandwidth and reducing latency. This approach can be easily extended to next-generation imaging algorithms currently being developed. In this paper we present details of our implementation, including application-scaling results on Blue Gene/P.

In this paper we present our recent research and development work for multicore computing of Line of Sight (LoS) on the Cell Broadband Engine (CBE) processors. LoS can be found in many applications where real-time high performance computing is required. We will describe an efficient LoS multi-core parallel computing algorithm, including the data partition and computation load allocation strategies to fully utilize the CBE's computational resources for efficient LoS viewshed parallel computing. In addition, we will also illustrate a successive fast transpose algorithm to prepare the input data for efficient Single-Instruction-Multiple-Data (SIMD) operations. Furthermore, we describe the data input and output (I/O) management scheme to reduce the (I/O) latency in Direct-Memory-Access (DMA) data fetching and storing operations. The performance evaluation of our LoS viewshed computing scheme over an area of interest (AOI) with more than 4.19 million points has shown that our parallel computing algorithm on CBE takes less than 25.5 ms, which is several orders of magnitude faster than the available commercial systems.

Video surveillance has become one of the most important tools for public safety and security. In this paper, we present our new work on developing efficient parallel computing algorithms and schemes for implementing H.264 video encoder engine on Cell Broadband Engine (CBE) blade for high performance large scale video surveillance applications. Extending our previous work on H.264 video encoding on CBE, we have developed new parallel computing schemes in computational load partition, dynamic task scheduling, motion estimation, and mode selection. We partition the intensive H.264 video encoding computation load into four major functional modules, namely, the pre-processing module, the motion estimation module, the mode selection and transform/quantization module, and the Context Adaptive Binary Arithmetic Coding (CABAC) module. The task scheduler dynamically assigns a waiting computing task to a SPE as soon as it becomes available. Our new implementation has achieved more than 5X performance improvement to encode 32 standard-definition (SD 720x480 pixel resolution) H.264 video streams simultaneously at 30 frames per second with one Cell Blade that consists of 16 Synergistic Processor Elements (SPEs) and two control Power Processor Elements (PPEs) or 448 SD channels of H.264 video streams on a single chassis Cell Blade Center with 14 Cell Blades.

In this paper we present a novel algorithm to speed up the inter mode decision process for the H.264/AVC encoding. The proposed inter mode decision scheme determines the best coding mode between P16×16 and P8×8 based on learning theoretic classification algorithms to discern between mode classes based on the evaluation of a simple set of features extracted from a motion compensated macroblock. We show that the proposed method can reduce the number of macroblocks for P8×8 mode testing by 80% on average, at the cost of only a small loss of 0.1 dB in compression performance.

This letter tackles the problem of reducing the complexity of H.264 decoding. Since deblocking accounts for a significant percentage of H.264 decoding time, our focus is on the H.264 in-loop deblocking filter. Observing that branch operations are costly and that in the deblocking process there are events with significantly high probability of occurrence, we regroup and simplify the branch operations. We apply the idea of Huffman tree optimization to speed up the boundary strength derivation and the true-edge detection by taking advantage of the biased statistical distribution. Our analyses and experiments show that the proposed techniques can reduce the deblocking computation time typically by a factor of more than seven times, while maintaining the bit-exact output.

The problem of secure collaboration between two agents, A and B, with limited mutual trust is considered. Specifically, we consider a formulation wherein agent A would like to share information with agent B but only if B has correlated information. Our solution is based on the principles of source coding with decoder-only side information. The key idea is that agent A can encode its information by using a Slepian-Wolf code at a rate which enables agent B to correctly decode only if the information which B has satisfies a conditional entropy constraint. Furthermore, our solution allows the two agents to interact and negotiate the rate on the fly. It is shown that such interaction not only reduces the transmission rate, but also allows secure collaboration to be established even when neither agent knows the joint statistics of their information. Finally, we demonstrate the utility of our solution in a simple medical imaging application.

Aiming at improving compression performance, we consider mode selection for Wyner-Ziv video compression where a block of pixels in a video frame, after discrete cosine transform(DCT), can be either encoded by using H.264 Intra mode or Wyner-Ziv(WZ) mode with side information processed at the decoder. Under the constraint of encoding complexity, an iterative algorithm is proposed to find the best partition of a video frame into these two modes in the sense of minimizing the overall compression rate. It is shown that the algorithm always converges. Experimental results on standard video test sequences show that by using the proposed algorithm for mode selection, one can achieve about 0.4dB gain for WZ-encoded frames over a WZ video compression system without intra mode at rate 0.2 bits per pixel. Furthermore, in all the experiments our algorithm converges in 3 iterations.

Side information (SI) generation is one of the key components of a Wyner-Ziv coder. In this paper we present a novel multi-frame SI generation approach which uses adaptive temporal filtering to estimate the pixel values for SI and motion vector filtering for refinement. For temporal filtering, we derive the optimal mean squared error temporal filter when the noise can be evaluated, and propose a similarity weighted temporal filter when the knowledge of the noise is not available. The temporal filter adapts on the quality of the motion estimation. The quality of SI generation is further improved by using motion vector filtering to reduce the noise effect from motion estimation. Experimental results indicate that the proposed SI generation approach yields good performance in terms of SI quality and conditional entropy.

In-loop deblocking filter is identified as the most time consuming part for H.264 high profile decoders. This paper proposes an improved platform and encoder independent deblocking scheme for H.264 high profile codec speedup. Two key techniques are introduced in the proposed algorithm: a statistical analysis based hybrid boundary strength derivation scheme and a more efficient logic expression for the B-slice boundary strength derivation. Compared to previously proposed algorithms, significant computation can be saved, while maintaining the bit-exact output. The proposed techniques can be used in both standard conforming encoders and decoders.

In this paper we present a new motion estimation scheme that minimizes the objective distance function with a constraint of similarity measure to exploit the motion correlation among adjacent pixel blocks with similar statistics features. We formulate this correlation as a similarity measure on the motion vectors between the current pixel block and its neighboring blocks weighted by the corresponding statistical similarity. We then use this similarity measure as a constraint in the objective distance function to reduce the noise effects and improve performance by effectively trading off the difference in the pixel values with the smoothness in the adjacent motion vectors. Thus, in motion estimation, our new scheme not only minimizes the pixel differences but tries to preserve the motion smoothness among statistically similar neighbors. We applied this new motion estimation scheme to a distributed video coding system for side information generation for Wyner-Ziv decoding and compared its performance to the scheme without similarity constraint. The results have shown that our motion estimation scheme can achieve significant gains in the fidelity of the side information and the decoded Wyner-Ziv frames over the scheme without the similarity constraint.

This paper proposes a novel scheme to achieve deblocking speedup for H.264 high profile decoders. The proposed approach is to use statistics dependent decoding which takes advantage of the biased statistical distribution in video streams. Specifically, in the proposed scheme, Huffman tree structures are introduced for boundary strength derivation, and hierarchical true edge detection is applied in the boundary filtering process to reduce the computation. As a result, significant computation can be saved in the deblocking process, while bit-exact output is maintained. This platform and encoder independent scheme can be incorporated into both standard conforming encoders and decoders. Since deblocking accounts for a significant percentage of decoding time, the scheme is especially important for decoder implementations. The analyses and experiments show that the proposed scheme could reduce the deblocking computational load by a factor of more than three times.

Compared with either heuristic or RD optimization based approaches, frame-level bit allocation strategies based on constant quality models are inadequately studied. They are, however, inherently more capable of achieving consistent video quality over the whole sequence for real-time single-pass rate control tasks. Moreover, no existing model is error-resilient. In this paper, we propose a novel error resilient source-channel constant distortion (SCCD) model, whose uniquely selected frame complexity measure effectively accomodates the joint impact from both the nonstationary source video signal and the lossy channel. Compared with its existent counterparts, our model has a higher modelling accuracy, and can be generally applied in frame-level bit allocation to enhance error robustness of rate control. As an example, an error resilient frame-level bit allocation solution is proposed based on the model. Simulation results demonstrate that for lossy video transmission, the proposed method can achieve more consistent and higher overall visual qualities than that of the other nonerror-resilient schemes, which proves the remarkable potential of the new model.

Power dissipation, data rate, and processing time are crucial constraints for embedded multimedia devices. We present a video coding solution given the constraints of bit rate and decoding power dissipation. Coupled with a small dedicated video internal memory at a decoder, the coding scheme is designed for the best operational rate-distortion and decoding power cost trade off. It can substantially decrease the data traffic to the external memory at decoder. A decrease in data traffic to the external memory at decoder will result in faster real-time processing and power savings. The encoder, given the prior knowledge of the decoder's dedicated video internal memory management scheme, optimally regulates its choice of motion compensated predictors to reduce the decoder's external memory accesses. This video coding scheme can be used in any standard or proprietary encoder to generate a compliant output stream decodable by a standard general purpose processor-based or dedicated hardware-based decoder with power restriction. Simulation results show that with a relatively small dedicated internal memory, our scheme may reduce the power dissipation significantly while keeping the comparable picture quality.

Low power dissipation and fast processing time are crucial requirements for embedded multimedia devices. This paper presents a technique in video coding to decrease the power consumption at a standard video decoder. Coupled with a small dedicated video internal memory cache on a decoder, the technique can substantially decrease the amount of data traffic to the external memory at the decoder. A decrease in data traffic to the external memory at decoder will result in multiple benefits: faster real-time processing and power savings. The encoder, given prior knowledge of the decoder’s dedicated video internal memory cache management scheme, regulates its choice of motion compensated predictors to reduce the decoder’s external memory accesses. This technique can be used in any standard or proprietary encoder scheme to generate a compliant output bit stream decodable by standard CPU-based and dedicated hardware-based decoders for power savings with the best quality-power cost trade off. Our simulation results show that with a relatively small amount of dedicated video internal memory cache, the technique may decrease the traffic between CPU and external memory over 50%.

There have been increasing market demands to develop more effective schemes for applying MPEG video coding standard to meet the challenges from the emerging applications. We present a real-time MPEG video coding system with information look-ahead for constant bit rate (CBR) applications, such as Video-on-Demand (VoD) over ADSL. This scheme employs two MPEG encoders. Both encoders operate at the same CBR. The second encoder has a buffer to delay the input by an amount of time relative to the first encoder to create a look-ahead window. In encoding, the first encoder collects the information of statistics and rate-quality characteristics. An on-line information processor then uses the collected information to derive the best coding strategy for the second encoder to encode the incoming frames in the look-ahead window. The second encoder uses the encoding parameters from the processor as the coding guide to execute the coding strategy and generate the final bitstream. Results on an IBM MPEG-2 encoder testing system with a 15 frame look-ahead window have shown that the scheme can obtain a 0.6 to 1.6 dB improvement in PSNR over the conventional single encoder on MPEG test video sources. Moreover, measured by the Tektronix Inc.'s Picture Quality Analysis (PQA) System, this scheme achieved 0.87 /spl sim/ 1.35 reduction in PQA measurement. The visual quality improvement is also very evident.

There have been increasing market demands to develop more effective schemes for applying MPEG video coding standard to meet the challenges from the emerging applications. We present a real-time MPEG video coding system with information look-ahead for constant bit rate (CBR) applications, such as video-on-demand (VoD) over ADSL. This scheme employs two MPEG encoders. Both encoders operate at the same CBR. The second encoder has a buffer to delay the input by an amount of time relative to the first encoder to create a look-ahead window. In encoding, the first encoder collects the information of statistics and rate-quality characteristics. An on-line information processor then uses the collected information to derive the best coding strategy for the second encoder to encode the incoming frames in the look-ahead window. The second encoder uses the encoding parameters from the processor as the coding guide to execute the coding strategy and generate the final bitstream. Results on an IBM MPEG-2 encoder testing system with a 15 frame look-ahead window have shown that the scheme can obtain a 0.6 to 1.6 dB improvement in PSNR over the conventional single encoder on MPEG test video sources. Moreover, measured by the Tektronix Inc.'s picture quality analysis (PQA) system, this scheme achieved 0.87 /spl sim/ 1.35 reduction in PQA measurement. The visual quality improvement is also very evident.

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.

Image quality assessment plays an important role in various image processing applications. A great deal of effort has been made in recent years to develop objective image quality metrics that correlate with perceived quality measurement. Unfortunately, only limited success has been achieved. In this paper, we provide some insights on why image quality assessment is so difficult by pointing out the weaknesses of the error sensitivity based framework, which has been used by most image quality assessment approaches in the literature. Furthermore, we propose a new philosophy in designing image quality metrics: The main function of the human eyes is to extract structural information from the viewing field, and the human visual system is highly adapted for this purpose. Therefore, a measurement of structural distortion should be a good approximation of perceived image distortion. Based on the new philosophy, we implemented a simple but effective image quality indexing algorithm, which is very promising as shown by our current results.

Objective image/video quality measures play important roles in various image/video processing applications, such as compression, communication, printing, analysis, registration, restoration and enhancement. Most proposed quality assessment approaches in the literature are error sensitivity-based methods. We follow a new philosophy in designing image/video quality metrics, which uses structural distortion as an estimation of perceived visual distortion. We develop a new approach for video quality assessment. Experiments on the video quality experts group (VQEG) test data set shows that the new quality measure has higher correlation with subjective quality measurement than the proposed methods in VQEG's Phase I tests for full-reference video quality assessment.

There has been an increasing need recently to develop objective quality measurement techniques that can predict perceived video quality automatically. This paper introduces two video quality assessment models. The first one requires the original video as a reference and is a structural distortion measurement based approach, which is different from traditional error sensitivity based methods. Experiments on the video quality experts group (VQEG) test data set show that the new quality measure has higher correlation with subjective quality evaluation than the proposed methods in VQEG's Phase I tests for full-reference video quality assessment. The second model is designed for quality estimation of compressed MPEG video stream without referring to the original video sequence. Preliminary experimental results show that it correlates well with our full-reference quality assessment model.

Recently, there have been two interesting trends in image and video coding research. One is to use human visual system (HVS) models to improve the current state-of-the-art coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate-scalable video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. We follow these two trends and propose a foveation scalable video coding (FSVC) algorithm, which supplies good quality-compression performance as well as effective rate scalability to support simple and precise bit rate control. A foveation-based HVS model plays a key role in the algorithm. The algorithm is amenable to the inclusion of various HVS models and adaptable to different video communication applications.

Region of interest (ROI) image and video compression techniques have been widely used in visual communication applications in an effort to deliver good quality images and videos at limited bandwidths. Most image quality metrics have been developed for uniform resolution images. These metrics are not appropriate for the assessment of ROI coded images, where space-variant resolution is necessary. The spatial resolution of the human visual system (HVS) is highest around the point of fixation and decreases rapidly with increasing eccentricity. Since the ROIs are usually the regions "fixated" by human eyes, the foveation property of the HVS supplies a natural approach for guiding the design of ROI image quality measurement algorithms. We have developed an objective quality metric for ROI coded images in the wavelet transform domain. This metric can serve to mediate the compression and enhancement of ROI coded images and videos. We show its effectiveness by applying it to an embedded foveated image coding system.

Embedded rate scalable video coding allows for the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. This is a very attractive feature for many multimedia communication applications. Motion estimation (ME) /motion compensation (MC) techniques are widely employed in various video coding systems to reduce temporal information redundancy. One of the major challenging problems in ME/MC based rate scalable video coding is how to generate the prediction frame from the previous frame to match the current frame. This problem is more difficult in rate scalable coding than in fixed rate coding because the decoding data rate is unavailable to the encoder. We propose an adaptive frame prediction scheme for foveation scalable video coding (FSVC), which is a new video coding algorithm that combines a foveation-based human visual system (HVS) model with a wavelet-based rate scalable coding algorithm. The new frame prediction algorithm provides an adaptive mechanism to control the prediction errors while reduce error propagation.

We propose an elegant yet efficient adaptive subband coding scheme in which the rate allocation and structures of the quantizers are jointly optimized based on the time-varying rate-distortion characteristics of the input signals. By employing the tree-structured vector quantizers and the generalized BFOS pruning algorithm, the operations of the rate allocation and quantization are elegantly combined, whereby the adaptation is both convenient and optimal. The results on video coding have shown that the adaptive scheme can achieve substantial performance improvement over the non-adaptive scheme.

In general, growth algorithms for optimal tree-structured vector quantizers do not exist. In this paper we show that if the source satisfies certain conditions; namely, that of diminishing marginal returns; optimal growth algorithms do exist. We present such an algorithm and compare its performance with that of other tree growth algorithms. Even for sources that do not meet the necessary conditions for the growth algorithm to be optimal, such as for speech with unknown statistics, it is seen by simulation that the algorithm outperforms other known growth algorithms, For sources that do not satisfy the required conditions, the algorithm presented here can also be used to grow the initial tree for the pruning process. The performance of such pruned trees is superior to that of trees pruned from full trees of the same rate.>

We study the performance of conditional entropy-constrained vector quantizers when used to quantize subbands of the displaced frame differences derived from video sequences. Chou and Lookabaugh (1990) originally suggested a locally optimal design of this new kind of vector quantizer which can be accomplished through a generalization of the well known entropy-constrained vector quantizer (ECVQ) algorithm. This generalization of the ECVQ algorithm to a conditional entropy-constrained is called CECVQ, i.e., conditional ECVQ. The non-memoryless quantization performed by the conditional entropy-constrained VQ is based on the current vector to be encoded and the previous encoded vector. A new algorithm for conditional entropy-constrained vector quantizer design is derived and it is based on the pairwise nearest neighbour technique presented by Equitz (1989).>