In the new video coding standard H.264/AVC, motion estimation (ME) is allowed to search multiple reference frames. Therefore, the required computation is highly increased, and it is in proportion to the number of searched reference frames. However, the reduction in prediction residues is mostly dependent on the nature of sequences, not on the number of searched frames. Sometimes the prediction residues can be greatly reduced, but frequently a lot of computation is wasted without achieving any better coding performance. In this paper, we propose a context-based adaptive method to speed up the multiple reference frames ME. Statistical analysis is first applied to the available information for each macroblock (MB) after intra-prediction and inter-prediction from the previous frame. Context-based adaptive criteria are then derived to determine whether it is necessary to search more reference frames. The reference frame selection criteria are related to selected MB modes, inter-prediction residues, intra-prediction residues, motion vectors of subpartitioned blocks, and quantization parameters. Many available standard video sequences are tested as examples. The simulation results show that the proposed algorithm can maintain competitively the same video quality as exhaustive search of multiple reference frames. Meanwhile, 76 %-96 % of computation for searching unnecessary reference frames can be avoided. Moreover, our fast reference frame selection is orthogonal to conventional fast block matching algorithms, and they can be easily combined to achieve further efficient implementations.

Mathematical morphology operations are applied in many real-time applications, such as video segmentation. For real-time requirement, efficient hardware implementation is necessary. This paper proposes a new architecture named Partial-Result-Reuse (PRR) architecture for mathematical morphological operations with flat structuring elements. Partial results generated during calculation process are kept and reused in this architecture to reduce hardware cost. With PRR concept and self-affinity property of structuring elements, the proposed architecture is more cost-effective and more general than existing morphology architectures. Moreover, it can be combined with systolic array to give consideration to both flexibility and hardware cost. We also propose a methodology to generate PRR architecture. With graphic method, the PRR architecture can be easily generated, and it can deal with structuring elements of any shape. The very large scale integration implementation of the PRR architecture shows that the area of processing element is small and can be fully piplelined without large overhead. The maximum frequency of the chip is 200 MHz in simulation, while processing speed of 550 morphological operations/s on a 720/spl times/480 frame can be achieved.

Automatic video segmentation plays an important role in real-time MPEG-4 encoding systems. Several video segmentation algorithms have been proposed; however, most of them are not suitable for real-time applications because of high computation load and many parameters needed to be set in advance. This paper presents a fast video segmentation algorithm for MPEG-4 camera systems. With change detection and background registration techniques, this algorithm can give satisfying segmentation results with low computation load. The processing speed of 40 QCIF frames per second can be achieved on a personal computer with an 800 MHz Pentium-III processor. Besides, it has shadow cancellation mode, which can deal with light changing effect and shadow effect. A fast global motion compensation algorithm is also included in this algorithm to make it applicable in slight moving camera situations. Furthermore, the required parameters can be decided automatically, which can enhance the proposed algorithm to have adaptive threshold ability. It can be integrated into MPEG-4 videophone systems and digital cameras.

In the new video coding standard, MPEG-4 AVC/JVT/H.264, motion estimation is allowed to use multiple reference frames. The reference software adopts a full search scheme, and the increased computation is in proportion to the number of searched reference frames. However, the reduction of prediction residues is highly dependent on the nature of the sequences, not on the number of searched frames. We present a method to speed up the matching process for multiple reference frames. For each macroblock, we analyze the available information after intra prediction and motion estimation from the previous frame to determine whether it is necessary to search more frames. The information we use includes selected mode, inter prediction residues, intra prediction residues, and motion vectors. Simulation results show that the proposed algorithm can save up to 90% of unnecessary frames while keeping the average miss rate of optimal frames less than 4%.

In the new video coding standard, MPEG-4 AVC/JVT/H.264, motion estimation is allowed to use multiple reference frames. The reference software adopts full search scheme, and the increased computation is in proportion to the number of searched reference frames. However, the reduction of prediction residues is highly dependent on the nature of sequences, not on the number of searched frames. In this paper, we present a method to speed up the matching process for multiple reference frames. For each macroblock, we analyze the available information after intra prediction and motion estimation from previous one frame to determine whether it is necessary to search more frames. The information we use includes selected mode, inter prediction residues, intra prediction residues, and motion vectors. Simulation results show that the proposed algorithm can save up to 90% of unnecessary frames while keeping the average miss rate of optimal frames less than 4%.

This paper describes an efficient motion estimation algorithm, the predictive line search (PLS), for real-time implementations of MPEG-4 encoder on multimedia processors. The motion-vector predictor is used as the starting point in the search process because the correlation between neighboring motion vectors is strong. The line search pattern is used in the proposed algorithm to reduce the memory access as well as to exploit the special multimedia processor instructions for sum of absolute difference calculations. Experimental results show that the performance of the PLS is very close to that of the full-search (FS) algorithm. Compared with the well-known diamond search and one-dimensional FS, the PLS shows better performance and robustness, especially for high motion sequences. A prototype MPEG-4 encoding system is implemented on a 216-MHz multimedia processor with very long instruction word architecture to verify the effectiveness of the PLS. Real-time encoding of MPEG-4 Simple Profile Level 3 (CIF, 30 fps) can be achieved with only 57% of the processor load.

Watershed transform is a key operator in video segmentation algorithms. In this paper, a new fast watershed algorithm named P-Watershed for image sequences segmentation is proposed. By utilizing the temporal coherence property of video signal, this algorithm updates watersheds instead of searching watersheds in whole image. The watershed process can be accelerated and the segmentation results are almost the same as those of conventional algorithms. Moreover, an intra-inter watershed scheme (IP-Watershed) is also proposed to further improve the results. Experimental results show this algorithm can save 50% computation without degrading the segmentation results.

An efficient moving object segmentation algorithm suitable for real-time content-based multimedia communication systems is proposed in this paper. First, a background registration technique is used to construct a reliable background image from the accumulated frame difference information. The moving object region is then separated from the background region by comparing the current frame with the constructed background image. Finally, a post-processing step is applied on the obtained object mask to remove noise regions and to smooth the object boundary. In situations where object shadows appear in the background region, a pre-processing gradient filter is applied on the input image to reduce the shadow effect. In order to meet the real-time requirement, no computationally intensive operation is included in this method. Moreover, the implementation is optimized using parallel processing and a processing speed of 25 QCIF fps can be achieved on a personal computer with a 450-MHz Pentium III processor. Good segmentation performance is demonstrated by the simulation results.

This paper proposes a new cost-effective architecture for mathematical morphology named partial-result-reuse (PRR) architecture. For many real-time applications of mathematical morphology, the hardware implementation is necessary; however, the hardware cost of most existing morphology architectures is too high when dealing with large structuring elements. With the partial-result-reuse concept and self-affinity property of general structuring elements, the proposed architecture is more cost-effective and more general than the existing morphology architectures. It can deal with morphological operations with arbitrary structuring elements and can be used for other semi-group operations, and only 2[log/sub 2/n] comparators are needed for n/spl times/n structuring elements. Simulation shows that this architecture can dramatically reduce the hardware cost of morphological operations with all kinds of structuring elements.

A fast watershed based video segmentation algorithm is proposed in this paper. Watershed based video segmentation algorithms are the mainstream because of high subjective quality; however, they are too computational intensive for real-time applications. Combining prior fast segmentation algorithm, which is based on change detection and background registration, and predictive watershed, a novel fast watershed algorithm for image sequences segmentation, this algorithm can give precise segmentation results and achieve real-time requirement. Simulation shows processing speed of 9.4 QCIF fps can be achieved on a PC with a Pentium-III 800 MHz processor.