Multimedia distribution over the Internet is becoming increasingly popular. A novel framework for Internet video streaming is proposed. For video compression, our previously developed three-dimensional significance-linked connected component analysis (3D-SLCCA) codec is applied. 3D-SLCCA provides high coding efficiency, multiresolution video representation, transmission error resilience, and low computational complexity. For audio coding, the GSM standard is used. For error control, retransmission and error concealment are jointly applied. Multiresolution-multicast transmission is implemented by assigning different multicast group addresses to different video layers. Thus each receiver subscribes to the maximum number of layers that both its hardware resource and network capability can handle. By using a hierarchically structured multicast tree, each node is responsible for caching packets, collecting NACK packets, and sending repair packets. This not only significantly reduces the latency, but also efficiently solves the "ACK implosion" problem. As opposed to data transmission, reliable multicast is not required by the network infrastructure. Based on timing constraint and importance of lost packets, each receiver decides whether to request retransmission or apply error concealment. Finally, synchronization is accomplished by using the timestamp mechanism of RTP.

A novel mobile communications system is proposed, which provides not only effective but also efficient video access for mobile users when communicating over low-bandwidth error-prone wireless links. The middleware implemented by a mobile proxy server at the mobile support station is designed for the seamless integration of mobile users with video servers, so the specific details of the underlying protocols and source/channel coding techniques are hidden to both the video server and mobile client. Based on the concept of application-level framing, the application (video codec in our case) plays a significant role in network communications that most of functionalities of the system are implemented as part of the application. As such, at the application layer, adaptive source- and channel-coding techniques are developed to jointly provide the user with the highest possible video quality. For efficient source coding, our high-performance low-complexity video-coding algorithm called a 3-D significance-linked connected component analysis (3D-SLCCA) is chosen. Due to its high robustness against channel-error propagation, 3D-SLCCA is well suited for wireless environments. For error-resilient channel coding, a multilayer transmission error-control mechanism is developed. Since there is no additional requirement imposed on either the mobile client or the video server, mobile users interact with the server in exactly the same way as stationary users. Extensive computer experiments demonstrate the effectiveness of the proposed system.

In the paper, three error resilient coding schemes are proposed and integrated with our previously developed high performance significance-linked connected component analysis (SLCCA) wavelet image codec. First, the cross-scale significance-links that exploit cross-scale dependency are reduced thereby error resilience of the source codec is increased. Second, hierarchical resynchronization is applied, i.e., resynchronization markers are inserted in the bitstream after each bit-plane and possibly per each scale and/or subband. Third, bit-plane-wise unequal error protection is applied, with the parity bits being exponentially distributed among bit-planes. Extensive performance evaluation demonstrates that the proposed joint source-channel coding algorithm for image transmission is not only highly efficient but also highly robust.

Error resilient coding strategies are proposed and integrated with the high performance significance-linked connected component analysis (SLCCA) wavelet image codec. First, the source bitstream is packetized so that the error propagation within the bitstream is minimized. Second, hierarchical resynchronization is applied, i.e., resynchronization markers are inserted in the bitstream after each bit-plane and possibly per each scale or subband. Third, the significance-links that exploit the cross-scale dependency of wavelet coefficients are reduced. Finally, bit-plane-wise unequal error protection is applied, with the parity bits being exponentially distributed among bit-planes. Extensive performance evaluation over a binary symmetric channel demonstrates the high error resilience and efficiency of the integrated joint source-channel coding strategies with the SLCCA source codec.

In the paper, three error resilient coding schemes are proposed and integrated with our previously developed high performance significance-linked connected component analysis (SLCCA) wavelet image codec. First, the cross-scale significance-links that exploit cross-scale dependency are reduced thereby error resilience of the source codec is increased. Second, hierarchical resynchronization is applied, i.e., resynchronization markers are inserted in the bitstream after each bit-plane and possibly per each scale and/or subband. Third, bit-plane-wise unequal error protection is applied, with the parity bits being exponentially distributed among bit-planes, Extensive performance evaluation demonstrates that the proposed joint source-channel coding algorithm for image transmission is not only highly efficient but also highly robust.

Multimedia distribution over the Internet is becoming increasingly popular. Since the Internet was designed for computer data communication, satisfying the different characteristics and requirements of multimedia streams poses significant challenges. For effective and efficient Internet video streaming, many issues (e.g., multiresolution representation, multicast transmission, error control, synchronization, etc.) must be addressed. In the paper, a novel framework for Internet video streaming is proposed. For video compression, our previously developed three-dimensional significance-linked connected component analysis (3D-SLCCA) codec is applied. 3D-SLCCA provides high coding efficiency, multiresolution video representation, transmission error resilience, and low computational complexity. For audio coding, the GSM standard is used. For error control, retransmission and error concealment are jointly applied. Multiresolution-multicast transmission is implemented by assigning different multicast group addresses to different video layers. Thus each receiver subscribes to the maximum number of layers that both its hardware resource and network capability can handle. By using hierarchically structured multicast tree, each node is responsible for caching packets, collecting NACK packets, and sending repair packets. This not only significantly reduces the latency, but also efficiently solves the "ACK implosion" problem. As opposed to data transmission, reliable multicast is not required by the network infrastructure. Based on timing constraint and the importance of lost packets, each receiver decides whether to request retransmission or apply error concealment. Finally, synchronization is accomplished by using the timestamp mechanism of RTP. When the network does not support multicasting, proxy servers implemented on workstations can be used to perform similar functionalities.

A highly robust channel coding scheme to be seamlessly integrated into our previously developed high performance wavelet based significance-linked connected component analysis (SLCCA) image coding technique is proposed. The SLCCA source coding algorithm is slightly modified to enable synchronization after each bit-plane. Thus even after uncorrectable errors, the decoding can be continued at the next refinement level. An unequal error protection scheme is also developed by using Reed-Solomon codes. The proposed scheme outperforms P.G. Sherwood and K. Zeger's technique (1997; 1998) by 0.66 dB on average for the "Lena" image in peak signal-to-noise ratio. Furthermore, the proposed codec has very low computational complexity, well applicable for the power constrained mobile scenario.

Based on our previously developed significance-linked connected component analysis (SLCCA) data organization and representation strategy, two three-dimensional wavelet video coding techniques are proposed covering a wide range of applications areas. In three-dimensional SLCCA (3D-SLCCA), both the wavelet decomposition and SLCCA data organization and representation strategy are extended to 3-D case. In Haar wavelet SLCCA (N-SLCCA), the SLCCA data organization and representation strategy is separately applied on each of the two temporal subbands obtained by Haar wavelet-based temporal decomposition. The developed codecs not only provide comparable or superior performance in comparison to state-of-the-art video coding standards, but also have several desirable properties such as scalability, low computational complexity, and error resilience, well suited for networked video communications.

A novel system providing effective and efficient access to video archives for mobile users communicating over low-bandwidth error-prone wireless links is proposed. The developed system has three components. First, the middleware, implemented as mobile proxy server located at the mobile support station, is chosen for the seamless integration of both mobile users and video servers by hiding the specific details of the underlying system. Second, for efficient video transmission, our 3D significant-linked connected-component analysis video codec, which is well-suited for wireless environments, is used. Finally, multilayer transmission error control mechanism is applied to cope with channel errors. As a result, no additional requirements are imposed on the mobile client and video server, so that mobile users interact with the server in exactly the same way as stationary users, and all they experience are better picture quality, less jerky video and shorter delays.

A novel middleware-based system providing effective and efficient video access for mobile users communicating over low bandwidth error-prone wireless links is proposed. The middleware, implemented as mobile proxy server located at the mobile support station, is chosen for the seamless integration of mobile users and video servers by hiding the specific details of the underlying system. For efficient video transmission, our three-dimensional significant-linked connected component analysis video codec is well suited for wireless environments, is used. To cope with wireless transmission errors, we propose an multilayer error control mechanism, which incorporates synchronization, unequal forward error protection, and retransmission. As a result, mobile users experience better picture quality, less jerky video, and shorter delays when accessing video files.

A novel hybrid wavelet video coding algorithm termed video significance-linked connected component analysis (VSLCCA) is developed for very low bit-rate applications. In the proposed VSLCCA codec, first, fine-tuned motion estimation based on the H.263 Recommendation is developed to reduce temporal redundancy, and exhaustive overlapped block motion compensation is utilized to ensure coherency in motion compensated error frames. Second, the wavelet transform is applied to each coherent motion compensated error frame to attain global energy compaction. Third, significant fields of wavelet-transformed error frames are organized and represented as significance-linked connected components so that both the within-subband clustering and the cross-scale dependency are exploited. Last, the horizontal and vertical components of motion vectors are encoded separately using adaptive arithmetic coding while significant wavelet coefficients are encoded in bit-plane order by using high order Markov source modeling and adaptive arithmetic coding. Experimental results on eight standard MPEG-4 test sequences show that for intraframe coding, on average the proposed codec exceeds H.263 and ZTE (zero-tree entropy) in peak signal-to-noise ratio by as much as 2.07 and 1.38 dB at 28 kbit/s, respectively. For entire sequence coding, VSLCCA is superior to H.263 and ZTE by 0.35 and 0.71 dB on average, respectively.

Recent success in wavelet image coding is mainly attributed to a recognition of the importance of data organization and representation. There have been several very competitive wavelet coders developed, namely, Shapiro's (1993) embedded zerotree wavelets (EZW), Servetto et al.'s (1995) morphological representation of wavelet data (MRWD), and Said and Pearlman's (see IEEE Trans. Circuits Syst. Video Technol., vol.6, p.245-50, 1996) set partitioning in hierarchical trees (SPIHT). We develop a novel wavelet image coder called significance-linked connected component analysis (SLCCA) of wavelet coefficients that extends MRWD by exploiting both within-subband clustering of significant coefficients and cross-subband dependency in significant fields. Extensive computer experiments on both natural and texture images show convincingly that the proposed SLCCA outperforms EZW, MRWD, and SPIHT. For example, for the Barbara image, at 0.25 b/pixel, SLCCA outperforms EZW, MRWD, and SPIHT by 1.41 dB, 0.32 dB, and 0.60 dB in PSNR, respectively. It is also observed that SLCCA works extremely well for images with a large portion of texture. For eight typical 256x256 grayscale texture images compressed at 0.40 b/pixel, SLCCA outperforms SPIHT by 0.16 dB-0.63 dB in PSNR. This performance is achieved without using any optimal bit allocation procedure. Thus both the encoding and decoding procedures are fast.

Perhaps, Sarnoff Corporation's zerotree entropy (ZTE) coder is the most successful wavelet video coder published so far which exploits the statistical properties of wavelet-transformed images by utilizing novel data representation and organization strategies. A high performance hybrid video coding algorithm termed video significance-linked connected component analysis (VSLCCA) is developed. It is quite encouraging that, at least empirically convinced, the wavelet transform with aids of those published innovative data representation and organization methods can be an invaluable asset in video coding if motion-compensated error frames are coherent. In VSLCCA, time domain motion estimation followed by exhaustive overlapped block motion compensation is utilized to ensure coherency, and then wavelet transform is applied to each error frame with significant wavelet coefficients being encoded by highly efficient SLCCA technique. Experimental results on standard MPEG-4 test sequences show that VSLCCA is superior to H.263 and ZTE by 0.48 dB and 0.77 dB on average, respectively.

An automatic spatio-temporal video sequence segmentation algorithm is proposed. To address this very difficult computer vision problem, several novel algorithms have been developed which use both spatial and temporal information. First, a novel temporal segmentation algorithm is developed based on our previous work in motion estimation. Second, an iterative split-and-merge spatial segmentation scheme is proposed with an initial segmentation being provided by a recursive conditioned dilation operation merging pixels into homogeneous regions, followed by an iterative refinement algorithm to obtain the final spatial segmentation. Third, temporal and spatial segmentation are linked to form spatio-temporal segmentation with the shape of each object being simplified by a morphological close-opening operation to obtain the final segmentation. Performance evaluation shows that the developed algorithm can successfully segment moving objects without any human intervention.

In the paper, our high performance significance-linked connected component analysis (SLCCA) image compression algorithm is extended to three-dimensional wavelet coding resulting in a highly efficient and scalable video codec termed 3DSLCCA. Advantageous features of 3DSLCCA include low computational complexity, rate, spatial, temporal, and SNR scalability, and prevention of error propagation at the price of acceptable degradation in both objective and subjective quality when compared to the state-of-the-art H.263 standard.

An efficient image database system is developed. The most important features of the proposed system include compressed domain indexing, searching by using scalable features, and progressive image transmission. User interaction is involved both at the search refinement stage and in the display of the query results. The most important query types include query by color layout and query by wavelet-coefficient clustering information. The indexing and searching algorithms are tightly coupled with the underlying image compression algorithm by which means the images are stored in the database, reducing both the complexity and the storage requirements of the database management system. In this research, we utilize our previously developed (B.-B. Chai et al., 1997, 1998) high-performance wavelet image coding algorithm, termed "significance-linked connected component analysis", which not only renders a very high compression performance when compared to other top-ranked wavelet image coding algorithms and the JPEG standard, but also inherently supports scalable features and progressive transmission. Computer experiments demonstrate the efficiency of the developed system.

Recent success in wavelet image coding is mainly attributed to recognition of the importance of data organization and representation. There have been several very competitive wavelet coders developed, namely, Shapiro's embedded zerotree wavelets (EZW), Servetto et al.'s morphological representation of wavelet data (MRWD), and Said and Pearlman's set partitioning in hierarchical trees (SPIHT). In this paper, we develop a novel wavelet image coder called significance-linked connected component analysis (SLCCA) of wavelet coefficients that extends MRWD by exploiting both within-subband clustering of significant coefficients and cross-subband dependency in significant fields. Extensive computer experiments on both natural and texture images show convincingly that the proposed SLCCA outperforms EZW, MRWD, and SPIHT. For example, for the "Barbara" image, at 0.5 bpp SLCCA outperforms EZW and SPIHT by 1.75 dB and 0.89 dB in PSNR, respectively. This outstanding performance is achieved without using any optimal bit allocation procedure, thus both the encoding and decoding procedures are fast.

Recent success in wavelet image coding is mainly attributed to the recognition of importance of data organization and representation, There have been several very competitive wavelet coders developed, namely, embedded zerotree wavelets (EZW), morphological representation of wavelet data (MRWD), and set partitioning in hierarchical trees (SPIHT). In this paper, we develop a novel wavelet image coder called significance-linked connected component analysis (SLCCA) of wavelet coefficients that extends MRWD by exploiting both within-subband clustering of significant coefficients and cross-subband dependency in significant fields. Computer experiments on both natural and texture images show convincingly that the proposed SLCCA outperforms EZW, MRWD, and SPIHT as well. For example, for the "Barbara" image, at 0.50 bpp SLCCA outperforms EZW and SPIHT by 1.71 dB and 0.85 dB in PSNR, respectively. It is also observed that SLCCA works extremely well for images with a large portion of texture. This outstanding performance is achieved without using any optimal bit allocation procedure. Thus both the encoding and decoding procedures are fast.

A highly efficient algorithm termed adaptive forward-backward vector quantization (AFBVQ) is developed for variable bit rate quantization of linear predictive coding (LPC) coefficients and integrated with the FS1016 Federal Standard Code Excited Linear Predictive (CELP) coder. This results in a high performance low bit rate speech coder called as AFBVQ-CELP which brings in two-fold bit rate reduction by backward LPC indexing and by forward LPC VQ. In AFBVQ, a previously decoded and temporally close speech signal is re-segmented into overlapping blocks. As the LPC coefficients calculated from one of those synthetic blocks are spectrally close to the current unquantized LPC coefficients, the backward LPC indexing is used to encode the current speech block; otherwise, the forward linear prediction is practised with the split vector quantization supported by a very efficient codebook initialization termed Mixture Gaussian Clustering (MGC). When compared to FS1016 CELP coder, AFBVQ-CELP reduces the LPC bit rate by 18 bit-per-frame (bpf) at the same spectral distortion. It means the overall bit rate is reduced from 4.8 kbps (FS1016 CELP) to 4.2 kbps. Furthermore, the proposed AFBVQ consistently outperforms the traditional forward LPC VQ by 3 bpf with the same spectral distortion. Subjective listening tests show that with AFBVQ-CELP the LPC bit rate can be further reduced to 8.4 bpf, resulting in 3.94 kbps overall bit rate without compromising the decoded speech quality.

A novel variable-rate linear predictive coding (LPC) parameter quantization scheme is proposed, in which linear prediction is done by using either the current (forward LPC) or previously decoded (backward LPC) speech blocks. The proposed LPC quantization scheme was integrated into the FS1016 Federal Standard CELP coder. Significant LPC bit rate reduction is achieved without compromising the decoded speech quality.