Aerial navigation of drones utilizes First-Person View (FPV) images for autonomous navigation to fly safely and stably in complex environments. The end-to-end monocular aerial navigation method is concise on hardware and dataset yet vulnerable in generalization capability. In this paper, we propose the Stable Variational Encoder (SVE), an end-to-end framework for monocular UAV navigation that addresses critical challenges in Imitation Learning — balancing accuracy, stability, and generalization. Stable Variation replaces the unstable variation of conventional VAEs with a trainable Deviation Embedding that remains invariant during testing, thus enhancing output reliability while preserving generalization capacity. The SVE omits the encoder and directly applies variation to the original images, which can preserve more details and achieve better navigation results. Extensive experiments on the AirSim platform demonstrate that the SVE outperforms competing methods, particularly in high-difficulty scenarios with large gate displacements and diverse weather conditions, and the experiments demonstrate the effectiveness of Stable Variation. Furthermore, real-world experiments confirm the effective sim-to-real transfer, highlighting the practicality of the SVE for autonomous drone applications.

Most existing methods for predicting drug-drug interactions (DDI) predominantly concentrate on capturing the explicit relationships among drugs, overlooking the valuable implicit correlations present between drug pairs (DPs), which leads to weak predictions. To address this issue, this paper introduces a hierarchical multi-relational graph representation learning (HMGRL) approach. Within the framework of HMGRL, we leverage a wealth of drug-related heterogeneous data sources to construct heterogeneous graphs, where nodes represent drugs and edges denote clear and various associations. The relational graph convolutional network (RGCN) is employed to capture diverse explicit relationships between drugs from these heterogeneous graphs. Additionally, a multi-view differentiable spectral clustering (MVDSC) module is developed to capture multiple valuable implicit correlations between DPs. Within the MVDSC, we utilize multiple DP features to construct graphs, where nodes represent DPs and edges denote different implicit correlations. Subsequently, multiple DP representations are generated through graph cutting, each emphasizing distinct implicit correlations. The graph-cutting strategy enables our HMGRL to identify strongly connected communities of graphs, thereby reducing the fusion of irrelevant features. By combining every representation view of a DP, we create high-level DP representations for predicting DDIs. Two genuine datasets spanning three distinct tasks are adopted to gauge the efficacy of our HMGRL. Experimental outcomes unequivocally indicate that HMGRL surpasses several leading-edge methods in performance.

Many real-life tasks can be abstracted as sparse reward visual scenes, which can make it difficult for an agent to accomplish tasks accepting only images and sparse reward. To address this problem, we split it into two parts: visual representation and sparse reward, and propose our novel framework, called Image Augmentation based Momentum Memory Intrinsic Reward (IAMMIR), which combines self-supervised representation learning with intrinsic motivation. For visual representation, we acquire a representation driven by a combination of image-augmented forward dynamics and reward. To handle sparse reward, we design a new type of intrinsic reward called Momentum Memory Intrinsic Reward (MMIR), which uses the difference between the outputs from the current model (online network) and the historical model (target network) to indicate the agent's state familiarity. We evaluate our method on a visual navigation task with sparse reward in Vizdoom and demonstrate that it achieves state-of-the-art performance in terms of sample efficiency. Our method is at least 2 times faster than existing methods and reaches a 100% success rate.

Transformer holds significance in deep learning (DL) research. Node embedding (NE) and positional encoding (PE) are usually two indispensable components in a Transformer. The former can excavate hidden correlations from the data, while the latter can store locational relationships between nodes. Recently, the Transformer has been applied for hyperspectral image (HSI) classification because the model can capture long-range dependencies to aggregate global features for representation learning. In an HSI, adjacent pixels tend to be homogeneous, while the NE does not identify the positional information of pixels. Therefore, PE is crucial for Transformers to understand locational relationships between pixels. However, in this area, most Transformer-based methods randomly generate PEs without considering their physical meaning, which leads to weak representations. This article proposes a new graph generative structure-aware Transformer (GraphGST) to solve the above-mentioned PE problem when implementing HSI classification. In our GraphGST, a new absolute PE (APE) is established to acquire pixels’ absolute positional sequences (APSs) and is integrated into the Transformer architecture. Moreover, a generative mechanism with self-supervised learning is developed to achieve cross-view contrastive learning (CL), aiming to enhance the representation learning of the Transformer. The proposed GraphGST model can capture local-to-global correlations, and the extracted APSs can complement the spectral features of pixels to assist in NE. Several experiments with real HSIs are conducted to evaluate the effectiveness of our GraphGST. The proposed method demonstrates very competitive performance compared with other state-of-the-art (SOTA) approaches. Our source codes will be provided in the following linkhttps://github.com/yuanchaosu/TGRS-graphGST.

Recognition of small moving targets in space has become one of the frontier scientific researches in recent decade. Most of them focus on detection and recognition in star image with sidereal stare mode. However, in this research field, few researches are about detection and recognition in star image with track rate mode. In this paper, a novel approach is proposed to recognize the moving target in single frame by machine learning method based on elliptical characteristic extraction of star points. The technical path about recognition of moving target in space is redesigned instead of traditional processing approaches. Elliptical characteristics of each star point can be successfully extracted from single image. Machine learning can achieve the classification goal in order to make sure that all moving targets can be extracted. The experiments show that our proposed approach can have better performance in star images with different qualities.

The augmented reality (AR) applications have been widely used in the field of Internet of Things (IoT) because of good immersion experience for users, but their ultralow delay demand and high energy consumption bring a huge challenge to the current communication system and terminal power. The emergence of mobile-edge computing (MEC) provides a good thinking to solve this challenge. In this article, we study an energy-efficient task offloading and resource allocation scheme for AR in both the single-MEC and multi-MEC systems. First, a more specific and detailed AR application model is established as a directed acyclic graph according to its internal functionality. Second, based on this AR model, a joint optimization problem of task offloading and resource allocation is formulated to minimize the energy consumption of each user subject to the latency requirement and the limited resources. The problem is a mixed multiuser competition and cooperation problem, which involves the task offloading decision, uplink/downlink transmission resources allocation, and computing resources allocation of users and MEC server. Since it is an NP-hard problem and the communication environment is dynamic, it is difficult for genetic algorithms or heuristic algorithms to solve. Therefore, we propose an intelligent and efficient resource allocation and task offloading algorithm based on the deep reinforcement learning framework of multiagent deep deterministic policy gradient (MADDPG) in a dynamic communication environment. Finally, simulation results show that the proposed algorithm can greatly reduce the energy consumption of each user terminal.

The deep convolutional neural network (CNN) based tracking-by-detection framework recently has become one of the most popular trackers. However, these methods are either time consuming or have greatly reduced performance. This article aims to achieve nearly identical tracking accuracy with the state-of-the-art CNN tracking-by-detection algorithm with relatively faster speed. We study the existing excellent trackers under the CNN tracking-by-detection framework and introduce the following: a multiscale feature pyramid fusion neural network based on dilated convolutions is constructed to learn a scale-invariant discriminative representation for tracking small objects, a hard-threshold weighted cross-entropy loss function is proposed to decrease the gap between object classification and tracking, and a mutual learning-based training policy is used to fuse the information from the network trained by image patches with different contextual regions to further improve the tracking performance. We conduct comprehensive experiments on visual object tracking benchmarks that validate the achievement of competitive performance of the proposed tracker with relatively faster speed both on qualitative and quantitative criteria. Additionally, the experimental results reveal that the proposed mutual learning-based training policy can accelerate the convergence speed and achieve better generalization performance.

Instead of only focusing on the hit ratio of the videos cached in Mobile Edge Computing (MEC) server, we propose a more refined video segment content and client statusbased MEC cache update strategy, to improve clients' Quality of Experience (QoE). First, based on both the segment popularity and importance, we divide MEC cache into three parts which can be flexibly transformed into each other by combing the requested times of segments, transmission capability and clients' playback status together. Furthermore, we present the client's cache priority utility function and formulate a problem to maximize the utility function subject to the constraints of MEC cache size and transmission capacity. The brand and branch method is employed to obtain the optimal solution. Simulation results show that our algorithm can improve system throughput, hit ratio of video segment, playback frozen time as well as backhaul traffic.

For many real applications, it's equally important to detect objects accurately and quickly. In this paper, we propose an accurate and efficient single shot object detector with feature aggregation and enhancement (FAENet). Our motivation is to enhance and exploit the shallow and deep feature maps of the whole network simultaneously. To achieve it we introduce a pair of novel feature aggregation modules and two feature enhancement blocks, and integrate them into the original structure of SSD. Extensive experiments on both the PASCAL VOC and MS COCO datasets demonstrate that the proposed method achieves much higher accuracy than SSD. In addition, our method performs better than the state-of-the-art one-stage detector RefineDet on small objects and can run at a faster speed.

More and more scholars focus on mobile edge computing (MEC) technology, because the strong storage and computing capabilities of MEC servers can reduce the long transmission delay, bandwidth waste, energy consumption, and privacy leaks in the data transmission process. In this paper, we study the cache placement problem to determine how to cache videos and which videos to be cached in a mobile edge computing system. First, we derive the video request probability by taking into account video popularity, user preference and the characteristic of video representations. Second, based on the acquired request probability, we formulate a cache placement problem with the objective to maximize the cache hit ratio subject to the storage capacity constraints. Finally, in order to solve the formulated problem, we transform it into a grouping knapsack problem and develop a dynamic programming algorithm to obtain the optimal caching strategy. Simulation results show that the proposed algorithm can greatly improve the cache hit ratio.

The star images obtained through the CCD camera can visually display the star structure. In order to get the wide starry image, we need to extract the characteristics of star images to achieve the star image stitching. In the star images, star points, whose characteristics are limited, are easily influenced by noise and are also difficult to extract. The number of stars is too large to stitch accurately. Thus, we propose a stitching algorithm based on blocking star images. First, we establish the maximum intensity projection model based on time sequence to locate the star points accurately. Then, according to the relative positions of star points, the block model is introduced to realize the establishment of the characteristics. Finally, the star image stitching is achieved from the perspective of the characteristic similarity. The experiments illustrate that CM (combination measure) reaches 0.87, and the proposed algorithm has better anti-noise performance and robustness.

Discriminative correlation filters (DCFs) have drawn increasing interest in visual tracking. In particular, a few recent works treat DCFs as a special layer and adding it into a Siamese network for visual tracking. However, they adopt shallow networks to learn target representations, which lack robust semantic information in deeper layers and make these works fail to handle significant appearance changes. In this paper, we design a novel network to fuse multi-level convolutional features, each level of which characterize target from different perspectives. Then we integrate our network with the DCF layer to construct an end-to-end deep architecture for visual tracking. The overall architecture is trained end-to-end offline to adaptively learn target representations, which are not only enabled to encode high-level semantic features and low-level spatial detail features, but also closely related to correlation filters. Experiments show that our proposed tracker achieves superior performance against state-of the-art trackers.

In this study, the authors focus on segment characteristic analysis and joint optimisation of modulation and coding scheme (MCS) selection, resource block (RB) assignment and the block error ratio (BLER) determination and segment adaptation scheme to satisfy dynamic adaptive streaming over HTTP (DASH) clients over wireless networks. First, the authors define a utility function as the continuous playback time that the packets scheduled with the allocated resource can support. Then, the authors formulate the MCS selection, RB assignment and BLER determination into a mathematical model. The relationship among the above three factors can be explored instead of performing MCS selection and RB assignment with the fixed BLER. By decomposing the original problem into some sub‐problems, the authors can get a solution to the original problem with low complexity. At the client level, the authors develop an adaptive segment request strategy based on the playback information, the segments' characteristics and the estimated transmission rate. To decrease the influence of the inaccurate estimations, an adaptive guard time interval based on the real playback information and transmission information of the previous segments is introduced. Simulation results show that the proposed algorithm can efficiently improve playback continuity and video quality over the existing algorithms.

Currently, the structural similarity index metric (SSIM) is recognised generally and applied widely in image quality assessment (IQA). However, using SSIM to evaluate contrast‐distorted images from TID2013 and CSIQ databases is low effective. In this study, the authors improve SSIM for contrast‐distorted images by combining it with the contrast sensitivity characteristics of human visual system (HVS). In the improved method, first, they combine the visual characteristics to propose a model that HVS perceives the real image. Then, this model is used to eliminate the visual redundancy of real images. Afterwards, the perceived images are evaluated using SSIM. Furthermore, 241 contrast‐distorted images from TID2013 and CSIQ databases were used in experiments. The results have shown that in comparison with SSIM scores, the scores obtained by the improved SSIM are more consistent with the subjective assessment scores. Moreover, the Pearson linear correlation coefficient and Spearman rank order correlation coefficient between subjective and objective scores are averagely improved by 12.83 and 22.78%, respectively. In addition, the assessment accuracy of the improved SSIM is better than that of five commonly used IQA metrics. Also, it has an excellent generalisation performance. These results show that the assessment performance of the improved SSIM is effectively enhanced.

HyperText Transfer Protocol (HTTP) adaptive streaming (HAS) has undoubtedly become the most cost-effective solution in delivering video contents over the IP networks. While many works have been proposed to improve the performance of HAS, the HAS in the cellular networks is still a challenging problem due to the limited radio resource and the dramatically increasing need for the mobile video service. Specifically, the main problem comes from the incongruous behaviors between rate adaptation and resource allocation. The centralized scheme is a good choice to solve this problem, which determines rate adaptation of all the clients at the base station. However, the existing schemes still have three main weaknesses: 1) poor performance in playout continuity; 2) synchrony in requesting; and 3) low efficiency of radio resource. In this paper, a playout continuity driven framework (PCDF) is proposed to tackle all these drawbacks. First of all, an optimization problem is formulated to optimize the multiclient HAS over long-term evolution network. Then, a PCDF is proposed to solve the problem, which consists of three main features. First, a requesting set is defined to make clients request in a partially synchronous manner, which can not only eliminate the drawback of synchrony but also take advantage of the centralized manner. Second, we propose a two-step resource allocation scheme, of which the two steps work in different temporal scales and cooperate to improve the efficiency of radio resource. Third, with the requesting set and resource allocation, the original optimization problem can be simplified, while it is still NP hard. To solve the problem, we propose a dynamic programming approach, with which the optimal solution can be obtained in polynomial time. Moreover, a greedy algorithm is proposed to decrease the complexity. The experimental results of PCDF demonstrate that the proposed framework can both effectively shorten the playout interruption duration and improve the frame quality in various circumstances.

In this study, the authors consider the power optimisation based on the MOS (mean of score) models of different services over multi‐user femtocell systems. They first formulate a mathematic model to minimise the total power consumption, subject to the MOS requirements and the network resource constraint. To solve the formulated problem, they first propose an optimal power allocation (OPA) scheme, in which the optimal power allocation and RB (resource block) assignment can be obtained by employing the Lagrange dual decomposition method. However, iterative update of the Lagrange variables leads to high complexity. To deal with this, they further propose a utility‐based suboptimal power optimisation algorithm (SPA). The utility of assigning each RB to each user is defined as the power consumption difference caused by adding the RB to the set of the RBs already assigned to the user. To calculate the utility, they develop a low‐complexity optimal power allocation scheme. Based on the calculated utility, each RB is assigned to the user with the maximum utility. Experimental results demonstrate that OPA can indeed acquire an upper bound of the system performance. SPA can improve the MOS of the users with acceptable complexity compared with other existing algorithms.

Robust scale and rotation estimation is an important and challenging problem in visual object tracking. There have been proposed many sophisticated trackers to track the location of a target accurately, but most of them do not take much attention to the scale and rotation estimation. Inspired by the success of the correlation filters in visual tracking, we proposed a novel scale-and-rotation correlation filter (SRCF) in the Fourier domain to realize the scale and rotation estimation. We thus constructed a tracker with this scale-and-rotation correlation filter and a corrected kernel correlation filter. Our tracker was tested on a full benchmark dataset consisting of 50 video in comparison with fifteen state-of-art trackers. Both the success plots and the precision plots show that our tracker achieved superior performance in real time.

This paper describes an image quality assessment (IQA) metric based on the visual perception of image contents (VPIC)). In the metric, VPIC is firstly modelled by simulating the nonlinearity of luminance perception, masking properties and contrast sensitivity characteristics of human visual system (HVS). Then the source and distorted images are processed by this model respectively, and their intensity differences are calculated. Finally, based on the intensity differences, an IQA model is built. And 47 reference images and 1549 distorted images in the LIVE, TID2008 and CSIQ databases are tested with the IQA metric. The results show that it is helpful to improve the consistency between the objective IQA scores and the subjective mean opinion scores (MOSs) combining the visual perception and complexity of image contents.

In this study, a playout buffer and discontinuous reception (DRX) aware scheduling scheme (PBDAS) is proposed to improve the video streaming transmission over the long term evolution systems by considering the characteristics of both the DRX mechanism and the HTTP streaming. First, to reconcile the three mechanisms, the metric called remaining playout time (RPT) is proposed, which can be used to estimate the playout status of the clients. With the RPT, the scheduler can accurately differentiate the urgency among the clients. Then, a two level resource allocation schemes is proposed. In the scheduler, the upper level is responsible for determining the scheduling set, while the lower level will be in charge of allocating the resource to the user equipments (UEs) in the set. With the scheduling set, the priority of the UEs can be accurately distinguished. For different scheduling set, different forms of metrics are designed, with which both the efficiency of the radio resource and the playout continuity can be significantly improved. In addition, a playout buffer level estimation method is given to improve the feasibility of the implementation of PBDAS. By analysing the behaviour of the playout buffer, the buffer level can be estimated with the cumulative serving rate and the HTTP streaming information, which can be obtained in the eNodeB and from the quality of experience reports, respectively. The simulation results show that PBDAS can shorten the total interruption duration of the existing schemes by about 50% as well as keep the power consumption at a low level.

In this paper, an accurate model to predict the slice-level transmission distortion for H.264/AVC video streams is presented. Our model gives full consideration to many new features of the H.264/AVC standard, especially intra prediction as well as inter prediction, and can be used for content-aware scheduling and performance optimization. In order to reduce complexity, the extracted information of the compressed domain from the video streams is exploited, to estimate the transmission distortion for each 4×4 block. By adding these distortions of all the 4×4 blocks in a slice up, the transmission distortion of the slice is obtained. Since the total average concealment distortion of each slice is independent on packet loss, it is introduced in the model as a known value to modify the compressed domain based transmission distortion. Simulation results show that our model outperforms the existing ones with respect to the accuracy for H.264/AVC video streams.

Considering the images captured under hazy weather conditions are blurred, a new dehazing algorithm based on overlapped sub-block homomorphic filtering in HSV color space is proposed. Firstly, the hazy image is transformed from RGB to HSV color space. Secondly, the luminance component V is dealt with the overlapped sub-block homomorphic filtering. Finally, the processed image is converted from HSV to RGB color space once again. Then the dehazing images will be obtained. According to the established algorithm model, the dehazing images could be evaluated by six objective evaluation parameters including average value, standard deviation, entropy, average gradient, edge intensity and contrast. The experimental results show that this algorithm has good dehazing effect. It can not only improve degradation of the image, but also amplify the image details and enhance the contrast of the image effectively.

Although many frequency estimation methods are available, few are designed for high-quality speech and audio processing systems, which typically use the modified discrete cosine transform (MDCT) as their analysis filter bank. In this letter, we propose a low complexity frequency estimator that is suitable for MDCT-based systems and that operates in the odd-DFT domain. Taking a complex exponential in noise as the input and deriving the analytical expression of its odd-DFT coefficient, we obtain an interpolated odd-DFT-based frequency estimator. Experiments show that the proposed estimator outperforms all other reported odd-DFT/MDCT domain estimators and has precision that is similar to that of representative DFT domain frequency estimators. The overhead for incorporating this estimator into a speech and audio processing system is small due to the simple odd-DFT to MDCT conversion. The corresponding magnitude and phase estimators are also proposed in this letter.

In order to improve the performance of singer identification, we propose a system to separate singing voice from music accompaniment for monaural recordings. Our system consists of two key stages. The first stage exploits the nonnegative matrix partial co-factorization (NMPCF), which is a joint matrix decomposition integrating prior knowledge of singing voice and pure accompaniment to separate the mixture signal into singing voice portion and accompaniment portion. In the second stage, based on the separated singing voice obtained by the first stage, the pitches of singing voice are first estimated and then the harmonic components of singing voice can be distinguished. For a frame, the distinguished harmonic components are regarded as reliable while other frequency components unreliable, thus the spectrum is incomplete. With those harmonic components, the complete spectrums of singing voice can be reconstructed by a missing feature method, spectrum reconstruction, obtaining a refined signal with more clean singing voice. Experimental results demonstrate that, from the point view of source separation, the singing voice refinement can further improve ΔSNR in contrast with the singing voice separation using NMPCF, while for the point view of singer identification, the singing voice separated by NMPCF is more appropriate than the refined singing voice.

Content-based video copy detection is undoubtedly one of the most effective solutions to video content tracing and copyright protection. It extracts features from videos and determines whether a copy occurs by comparing the extracted features. While a lot of work has been reported to address this problem with good performance, very few considered it from the perspective of a dynamic searching process. In this paper, we treat the copy detection in video streams as a sequential matching problem and take into consideration the connections between temporary results and forthcoming input. Specifically, we propose a video copy detection system that involves a novel frame fusion scheme and an adaptive search strategy. The proposed frame fusion scheme relies on path merging in a graph model, which is able to work in an online manner and provide informative temporary fusion results. Based on these temporary results, query content predictions can be generated, which will be fed back to the frame search engine to instruct it to adaptively adjust the search strategy. The experimental results show that the proposed frame fusion scheme achieves competitive detection and localization accuracies compared with the state-of-the-art methods. Meanwhile, with the assistance of the adaptive search strategy, the computational complexity of frame similarity search is dramatically reduced at a cost of a slight decrease in accuracy.

In this paper we first build an accurate and general transmission distortion model for H.264/AVC video streams to estimate the importance of each packet. Then, a utility-based optimized cross-layer resource allocation, MCS selection, and packet scheduling algorithm for real-time video transmission over HSDPA is studied. The utility here refers to the packet utility which is defined as a function of the packet urgency and the packet importance . Transmitting the packets with greater utility values firstly in each TTI can provide a higher level of video quality for the users. Thus, we formulate an optimization problem with the objective of maximizing the aggregate packet utility over all the users under the HSDPA physical resource constraints . In order to obtain an optimal transmission strategy, a dynamic programming -based cross-layer algorithm (DPCLA) is proposed. In view of its high complexity, we further develop a greedy-based cross-layer algorithm (GCLA) to find a suboptimal solution. Simulation results show that the proposed utility-based optimized cross-layer scheme, including the proposed transmission distortion model and the algorithms of DPCLA and GCLA, can provide higher received video quality than the existing ones for video transmission over HSDPA.

IEEE 802.16 network introduces a multimedia data scheduling service with different quality of service QoS requirements. The scheduling service manages transmission resources according to data types, satisfying the requirements of different connections or users. On the basis of the data types defined in the service, we discuss a normalized QoS metric for the multimedia connections in the paper. The QoS value of a connection can be determined just by three components: the data type of the connection, its desired resources, and its allocated resources. Then, we propose an optimum bandwidth allocation solution, which can maximize the utility of base station. Next, we propose a call admission control scheme utilizing the bandwidth allocation solution. In the scheme, the occupied resource of ongoing connections will be regulated for the entry admission of a new connection, without degrading the network performance and the QoS of ongoing connections. Finally, the simulation results confirm that the proposed scheme with the normalized QoS can achieve better trade-off between ongoing connections and new connections.Copyright ©2012 John Wiley & Sons, Ltd.

In this paper we study a utility based optimized cross-layer multiuser scheduling for real-time video transmission over HSDPA. The utility here refers to the packet utility which is defined as a function of the packet urgency and the packet importance. Transmitting the packets with greater utility values firstly in each TTI can provide a higher level of satisfaction (video quality) for the users. Thus, we formulate an optimization problem with the objective of maximizing the aggregate packet utility over all the users under the HSDPA physical layer constraints. By employing the dynamic programming method, an optimal packet scheduling strategy can be obtained. In view of its high complexity, we further develop a method of suboptimal solution, the greedy algorithm. Simulation results show that the proposed scheduling algorithms can provide higher received video quality than the existing ones for video transmission over HSDPA.

In this paper, we propose a joint content and channel aware (JCCA) resource allocation algorithm for video transmission in OFDMA relay system. It can maximize accumulation of allocated subcarriers' contribution for video transmission so as to improve overall quality of users' received video under resource constrained conditions. As is explained in this work, the algorithm is implemented by packet scheduling, path selection and subcarrier allocation, in which time-varying data content and channel state are jointly considered at the same time. In the simulation results, we make comparison between the JCCA resource allocation algorithm and others for video transmission in OFDMA relay system. It is demonstrated that the proposed algorithm can perform better in improving overall quality of users' received video.

In this paper, a novel adaptive Lagrange multiplier selection model in rate-distortion optimization (RDO) is proposed to determine the motion estimation mode during motion compensated temporal filter (MCTF) decomposition in the context of 3D wavelet-based scalable video codec (SVC). First, the motion activity of temporal subbands is investigated. Then, the model parameters for different MCTF levels are estimated. Finally, an optimization model for each temporal subband is obtained from the adaptive Lagrange multiplier selection. We demonstrate the accuracy and performance of our proposed model through extensive numerical simulations. Experimental results illustrate that the proposed model is adaptive with the characteristics of the temporal subbands, suggesting that our model can effectively improve the video quality in terms of both the PSNR and the mean structural similarity index (mean SSIM).

In this paper a Media Aware Radio Link Adaptation (MARLA) scheme for H.264/AVC video transmission is proposed to improve the video delivery quality over HSDPA networks. First of all, by taking both the packet importance and urgency into consideration, we design a transmission utility function to measure the packet priority. Secondly, an optimization model for radio link adaptation is built to maximize the video transmission utility for each mobile station. Finally, through analysis on the characteristics of the radio link adaptation mechanism in HSDPA, we obtain a low complexity optimal solution to the optimization model. Simulation results show that our proposed scheme can achieve higher video quality than the existing algorithms.

In this paper, a playback continuity driven cross-layer design is proposed for HTTP streaming in LTE systems in which the stringent delay deadline of the application layer is translated into the transmission capacity requirement at the MAC layer of eNodeB. First the client buffer information such as the fullness of the buffer and the playback information is easily obtained at the client. Based on the buffer information, the continuous playback time interval that the completely received segments in the client buffer can support is calculated. To keep continuous playback, the remaining packets of the un-completely received segment should arrive at the client during the time interval. Considering the video information in MPD from the HTTP server and the buffer information about the un-completely received segment, the sum size of the remaining packets in MAC queue can be obtained. From the time interval and the sum size, the transmission rate requirement that the MAC layer should satisfy can be acquired. For a scheduling period with fixed duration called TTI, the transmission capacity at the MAC layer can be easily acquired. Then a new mathematical model is proposed to maximize the system throughput subject to the transmission capacity requirements of the clients and the total RB constraint of the physical layer. Then a resource allocation scheme with low complexity is developed to solve the proposed problem. Simulation results show that the proposed algorithm can efficiently improve the playback continuity compared with other existing algorithms.

In this study, a network acknowledgement‐based and error‐propagation‐aware (NEPA) importance model for H.264/AVC video packet is proposed. Unlike those existing models, the network acknowledgement information is firstly studied to predict the video importance in the author's NEPA model. As a result, the authors can dynamically predict the importance of the current video packet, when its referenced frames are lost. Moreover, by means of analysing the reference relationships among the macro‐blocks (MBs), an error propagation/conceal map is drawn to accurately estimate the error propagation effect. The simulation results demonstrate that their NEPA model could achieve an acceptable accuracy for any video sequence as long as its group of pictures (GoP) size is not too large. Furthermore, an application example which adopts the NEPA model in High Speed Downlink Packet Access scheduling system demonstrates the effectiveness of the NEPA model.

Our objective is to train support vector machines (SVM)-based localized multiple kernel learning (LMKL), using the alternating optimization between the standard SVM solvers with the local combination of base kernels and the sample-specific kernel weights. The advantage of alternating optimization developed from the state-of-the-art MKL is the SVM-tied overall complexity and the simultaneous optimization on both the kernel weights and the classifier. Unfortunately, in LMKL, the sample-specific character makes the updating of kernel weights a difficult quadratic nonconvex problem. In this paper, starting from a new primal-dual equivalence, the canonical objective on which state-of-the-art methods are based is first decomposed into an ensemble of objectives corresponding to each sample, namely, sample-wise objectives. Then, the associated sample-wise alternating optimization method is conducted, in which the localized kernel weights can be independently obtained by solving their exclusive sample-wise objectives, either linear programming (for l1-norm) or with closed-form solutions (for lp-norm). At test time, the learnt kernel weights for the training data are deployed based on the nearest-neighbor rule. Hence, to guarantee their generality among the test part, we introduce the neighborhood information and incorporate it into the empirical loss when deriving the sample-wise objectives. Extensive experiments on four benchmark machine learning datasets and two real-world computer vision datasets demonstrate the effectiveness and efficiency of the proposed algorithm.

Video applications over wireless network have attracted more and more attention in recent years. However, due to the unique characteristics of video, limited available resources and the time-varying wireless channel state, video transmission over wireless network is still a challenging task. In this paper, a novel cross-layer design is proposed to maximize the overall received video quality with the limited network resource and the stringent deadline constraints over OFDMA network. Considering the information extracted from the Application layer, Media Access Control layer and Physical layer, we formulate the packet scheduling, subcarrier assignment and power allocation into a mathematical model. By employing Lagrange dual decomposition, asymptotically optimal packet scheduling and resource allocation strategy can be obtained. To reduce the computation complexity, we further develop a suboptimal solution which performs packet scheduling and subcarrier assignment jointly but power optimization independently. Finally, we validate our proposed algorithms in a multi-user scenario where all the video sequences are pre-encoded by H.264/AVC encoder. Experimental results demonstrate that our proposed algorithms can indeed improve the received video quality significantly compared with other existing algorithms.

This study for singer identification of mono popular music is in two stages. In the first stage, computational auditory scene analysis (CASA) is exploited to segregate singing voice units. For each frame, the estimated binary T-F mask indicates the time-frequency (T-F) units dominated by singing voice which are considered reliable, and other units are unreliable or missing. Thus the spectrum is incomplete. In the second stage, two missing feature methods, reconstruction and marginalization are used to identify the singer by dealing with the incomplete spectrum data. In the reconstruction module, the complete spectrum is first reconstructed and then converted to obtain the Gammatone frequency cepstral coefficients (GFCCs), which are further used to identify the singer. In the marginalization module, the probabilities of the singer's voice are computed on the basis of only the reliable components. We find that the reconstruction module outperforms the marginalization module, while both modules have significantly good performances, especially at signal-to-accompaniment ratios (SARs) of 0 dB and -3 dB, in contrast to other system.

With the growing popularity of video applications, the provision of high-quality video transmission for multi-users over High Speed Downlink Packet Access ( HSDPA ) networks gradually becomes a research focus. In HSDPA, the downlink shared channel is shared among the users to transmit packets. Hence, the packet scheduling algorithms play an important role in the HSDPA system performance. In this paper, we propose a Delay Prediction based Content and Channel Aware (DPCCA) low-complexity scheduling algorithm for real-time video transmission over HSDPA. The basic idea is that a video user with lower packet delays should relinquish an extra scheduling opportunity to the video user with larger packet delays as compensation and a video user predicted to have inevitable packet loss due to timeout should transmit the important packets first to reduce the degradation of video quality. Simulation results show that the proposed scheduling algorithm outperforms the existing ones with respect to the received video quality and fairness for video transmission over HSDPA.

In this study, the authors propose a fine granularity resource allocation algorithm for enhancing overall quality of users’ received video in the orthogonal frequency division multiple access system. Subcarrier's contribution for video transmission is used as the guideline of resource allocation in this algorithm. The subcarrier with the largest contribution is given priority to be allocated on the basis of two‐level paths search in the whole resource allocation tree. Moreover, the authors design the full search implementation scheme and the simplified implementation scheme for the resource allocation algorithm, respectively. The full search implementation scheme can obtain superior performance gain with high complexity. For reducing the complexity, the authors further propose a simplified implementation scheme subject to some minor quality degradation. At last, the authors compare the fine granularity resource allocation algorithm with other existing resource allocation algorithms for video transmission. In the simulation results, it is shown that the proposed algorithm acquires much more superior performance in overall quality of users’ received video.

The authors propose an effective algorithm for multiple pedestrians tracking, which is constructed in the framework of particle filtering, and it is based on the combination of online boosting tracker and the histogram of oriented gradient (HOG) descriptor for human detection. The combination for the detector and tracker lies on following aspects. First, each detection result is associated to a tracker implemented by the online boosting, which gives the authors scheme robustness for multiple similar objects and then, the output of support vector machine classifier based on HOG is dynamically fused as a component in the observation metric in particle filtering, which makes the tracker more accurate in some difficult conditions. Finally, the states of some particles are replaced by the state given by the detector, so that the tracker can recover from failure quickly. Experiments show the effectiveness of their scheme.

Video transmission over OFDMA (orthogonal frequency-division multiple-access) system is a challenging problem which involves better received video quality and stringent transmission delays. In this paper, we present a cross-layer design based on the joint optimization of resource allocation and packet scheduling in which the importance of each video packet is taken into consideration. The objective is to maximize the received video quality of all the users subject to the network resource constraint. By employing the Lagrange dual decomposition method, we can obtain the global optimal solution to the optimization problem. We also propose a method of suboptimal solution to reduce the high complexity. Simulation results show that the proposed algorithms have superior performance.

The state-of-the-art image retrieval approaches usually quantize SIFT features into visual words. Researchers proposed the notion of bundled feature which simply employs MSER to bundle SIFT features into groups. In this paper, we propose a large scale partial-duplicate image retrieval scheme using invariance weight of SIFT and SROA geometric consistency based on the bundled feature. We calculate the invariance weight of a SIFT feature by voting the number of the SIFT features in the transformed image space. Considering each bundled feature in polar coordinate system, we propose two consistency parameters: the Scale and Radius consistency parameter, and the Orientation and Angle consistency parameter (SROA geometric consistency). Experiments demonstrate that the invariance weight of SIFT feature can improve the performance of image retrieval, and the SROA geometric constraint is more effective and powerful than the existing geometric constraints for the bundled feature.

Our objective is to train SVM based Localized Multiple Kernel Learning with arbitrary$l_{p}$-norm constraint using the alternating optimization between the standard SVM solvers with the localized combination of base kernels and associated sample-specific kernel weights. Unfortunately, the latter forms a difficult$l_{p}$-norm constraint quadratic optimization. In this letter, by approximating the$l_{p}$-norm using Taylor expansion, the problem of updating the localized kernel weights is reformulated as a non-convex quadratically constraint quadratic programming, and then solved via associated convex Semi-Definite Programming relaxation. Experiments on ten benchmark machine learning datasets demonstrate the advantages of our approach.

This paper proposes an effective scale- and rotation-invariant local binary pattern (LBP) feature for texture classification. A circular neighboring set of an image pixel is defined as a scale-adaptive texton by taking into account the fundamental local structure property of the pixel. The scale space of a texture image is derived by the Laplacian of the Gaussian and then employed to determine the optimal scale of each pixel reflecting the characteristic length of the corresponding structure and determining the radius of the scale-adaptive texton. Different pixels have different optimal scales, resulting in the scale invariance. Contrary to the traditional LBP features that usually ignore global spatial information, the proposed method also defines subuniform patterns of each uniform pattern to improve the discrimination. For each uniform pattern, the subuniform pattern with the maximum statistical value is defined as the dominant orientation subuniform pattern. It is moved to the first column, and the others are circularly shifted. Experimental results demonstrate a good discrimination capability of the proposed scale- and rotation-invariant LBP in texture classification. Particularly, the LBP based on the scale-adaptive texton is promising to be powerful for texture description and scale-invariant texture classification, and the circular shift subuniform LBP can further improve the performance in the rotation-invariant texture classification.

Localized multiple kernel learning (LMKL) is an attractive strategy for combining multiple heterogeneous features in terms of their discriminative power for each individual sample. However, models excessively fitting to a specific sample would obstacle the extension to unseen data, while a more general form is often insufficient for diverse locality characterization. Hence, both learning sample-specific local models for each training datum and extending the learned models to unseen test data should be equally addressed in designing LMKL algorithm. In this paper, for an integrative solution, we propose a probability confidence kernel (PCK), which measures per-sample similarity with respect to probabilistic-prediction-based class attribute: The class attribute similarity complements the spatial-similarity-based base kernels for more reasonable locality characterization, and the predefined form of involved class probability density function facilitates the extension to the whole input space and ensures its statistical meaning. Incorporating PCK into support-vectormachine-based LMKL framework, we propose a new PCK-LMKL with arbitrary l(p)-norm constraint implied in the definition of PCKs, where both the parameters in PCK and the final classifier can be efficiently optimized in a joint manner. Evaluations of PCK-LMKL on both benchmark machine learning data sets (ten University of California Irvine (UCI) data sets) and challenging computer vision data sets (15-scene data set and Caltech-101 data set) have shown to achieve state-of-the-art performances.

For the video packets delivery over 802.11e WLAN networks, an improved cross-layer mapping (ICLM) scheme is presented in this paper. There are four access categories (ACs) in the 802.11e MAC layer. The ICLM scheme maps the video packets to the ACs according to both the significance of the video packets and the current network conditions. The simulation results show that this scheme can effectively enhance the end-to-end video quality over WLAN networks.

This paper investigates model based hand tracking in gaze directed video which contains everyday manipulation activity of human in kitchen environment. The video is recorded by a gaze-directed camera, which can actively directs at the visual attention area from the person who wears the camera. Here we present a method based on the combination of 2D and 3D hand model, which can estimate the position of hand in image accurately and the pose of hand in 3D roughly. The method uses 2D model tracking result to initialize and predict 3D tracking, which saves the number of particles and makes it possible for local configuration adapting. To evaluate our re sult, we try our algorithm on several pieces of video both from normal camera and gaze-directed camera. The error ratio of the distance between the ground truth and tracking result is used as an objective measurement for evaluating our method. Trajectory of hand movement and results of projected model for every frame show that our method is effective and makes a good foundation for future recognition and analysis.

With the increasing popularity of broadband wireless networks, video transmission over WiMAX networks has attracted more and more attention in both industrial and academic fields. In this paper, a Priority-based EDF (PEDF) scheduling algorithm, which combines EDF (Earliest Deadline First) with the characteristics of the multimedia, is proposed for H.264 video delivery. To meet the QoS requirements of different video frames, an adaptive deadline is assigned to change the miss rate of video packet dynamically in PEDF. In this way, we can better protect the more important video frames against loss. Simulation results show that the PEDF can achieve higher PSNR than the legacy RR, WFQ and EDF algorithms, and the quality of reconstructed video is improved significantly.

A novel Priority-Aware Two Rate Three Color Marker (PATRTCM) is presented in this paper. PATRTCM retains the token buckets scheme of Two Rate Three Color Marker (TRTCM) and its associated parameters, but introduces three marking probabilities, which are calculated in terms of the relative importance of the video packet and the traffic conditions in the network. According to the probabilities, a video packet is marked red, yellow or green. Simulation results show that PATRTCM can improve the end-to-end video quality over DiffServ Networks.

In this study, the authors study a game-theoretic framework for the problem of multiuser rate allocation in multimedia communications. The authors consider the multimedia users to be autonomous, that is, they are selfish and behave strategically. The authors propose a rate allocation framework based on a pricing mechanism to prevent the selfish users from manipulating the network bandwidth by untruthfully representing their demands. The pricing mechanism is used for message exchange between the users and the network controller. The messages represent network-aware rate demands and corresponding prices. The authors show that a Nash equilibrium can be obtained, according to which the controller generates allocations that are efficient, budget balanced and satisfy voluntary participation. Simulation results demonstrate the validity of the proposed framework.

Learning the sample-specific discriminative features based on numerous local learning may not scale well to real world scene classification tasks and suffer from the risk of overfitting. Hence we cast it in SVM based localized multiple kernel learning framework, and design a new strategy to alternately optimize the standard SVM solver and the sample-specific kernel weights, by either a linear programming (forl1-norm) or with closed-form solutions (forlp-norm). Experiments on both natural scene dataset and cluttered indoor scene dataset demonstrate the effectiveness and efficiency of our approach.

This paper provides a novel method for visual object tracking based on the combination of local scale-invariant feature transform (SIFT) description and global incremental principal component analysis (PCA) representation in loosely constrained conditions. The state of object is defined by the position and shape of a parallelogram, which means that tracking results are given by locating the object in every frame using parallelograms. The whole method is constructed in the framework of particle filter which includes two models: the dynamic model and the observation model. In the dynamic model, particle states are predicted with the help of local SIFT descriptors. Local key point matching between successive frames based on SIFT descriptors provides us an important cue for the prediction of particle states; thus, we can efficiently spread particles in the neighborhood of the predicted position. In the observation model, every particle is evaluated by local key point-weighted incremental PCA representation, which can describe the object more accurately by giving large weights to the pixels in the influence area of key points. Moreover, by incorporating the dynamic forgetting factor, we can update the PCA eigenvectors online according to the object states, which makes our method more adaptable under different situations. Experimental results show that compared to other state-of-the-art methods, the proposed method is robust especially under some difficult conditions, such as strong motion of both object and background, large pose change, and illumination change.

In H.264/AVC, the encoders have to try all the block types (16×16, 16×8, 8×16, 8×8, 8×4, 4×8 and 4×4) for each macro-block motion estimation and inter mode decision. All motion vectors and residuals are passed into Lagrange rate distortion (RD) cost model to calculate all RD cost values exhaustively, which introduce considerable computation complexity and resource demanding. In this paper, we introduce a JND (Just Noticeable Difference) profile and analyze how this profile characterizes rate and distortion. Based on JND profile, we devise a perceptual rate distortion (PRD) model with low-complexity to replace the Lagrange RD cost model. We scheme a novel inter macro-block partitioning method based on this PRD model. Our model and method first adopt perceptual filters to find the best mode of every macro-block and then start the motion estimation. Simulation results demonstrated that the inter mode decision based on PRD model won about 80% encoding time reduction and better quality than the inter mode decision based on the RD cost value decision.

We propose a hierarchical GIST model embedding multiple biological feasibilities for scene classification. In the perceptual layer, spatial layout of Gabor features are extracted in a bio-vision guided way: introducing diagnostic color information, tuning the orientations and scales of Gabor filters, as well as the spacial pooling size to a biological feasible value. In the conceptual layer, for the first time, we attempt to build a computational model for the biological conceptual GIST by kernel PCA based prototype representation, which is specific task orientated as biological GIST, and also in accordance with the unsupervised learning assumption in the primary visual cortex and prototype similarity based categorization in human cognition. Using around 200 dimensions, our model is shown to outperform existing GIST models, and to achieve state-of-the-art performances on four scene datasets.

Scale and rotation invariant texture classification is a challenging and significant topic in texture analysis. This paper presents a scale and rotation invariant Gabor Texture Descriptor (GTD) for texture classification. Firstly, Gabor filters with different orientations and scales are carried out on an image. Secondly, to obtain a scale invariant GTD, different scale energies of these Gabor filtered images are calculated, and the optimal matching scales of the GTD are chosen by the sliding window filter with the scale energies. Thirdly, to extract a rotation invariant GTD, a new method using Discrete Fourier Transform is employed on the scale invariant GTD. At last, the distances between the GTDs of a query image and the target images are calculated for texture classification. Five different GTDs are employed for texture classification to demonstrate the effectiveness of the proposed scale and rotation invariant GTD.

The three-dimensional subband/wavelet codecs use 5/3 filters rather than Haar filters for the motion compensation temporal filtering (MCTF) to improve the coding gain. In order to curb the increased motion vector rate, an adaptive motion mode driven spatiotemporal motion vector prediction (AMDST-MVP) scheme is proposed. First, by making use of the direction histograms of four motion vector fields resulting from the initial spatial motion vector prediction (SMVP), the motion mode of the current GOP is determined according to whether the fast or complex motion exists in the current GOP. Then the GOP-level MVP scheme is thereby determined by either the S-MVP or the AMDST-MVP, namely, AMDST-MVP is the combination of S-MVP and temporal-MVP (T-MVP). If the latter is adopted, the motion vector difference (MVD) between the neighboring MV fields and the S-MVP resulting MV of the current block is employed to decide whether or not the MV of co-located block in the previous frame is used for prediction the current block. Experimental results show that AMDST-MVP not only can improve the coding efficiency but also reduce the number of computation complexity.

In this letter, we develop a general delivery system for streaming using various coding standards over DiffServ (DS) networks. In the delivery system, the coder at the application layer pre-marks the compressed data according to their importance in terms of the current configuration and characteristics of the coder. Then the system maps these pre-marks to a fixed number of source marks at the real-time transport protocol layer. In terms of the current network congestion state, the edge router of a DS network uses the improved two rate three color maker to map these source marks to differentiated services code points. At last, the core routers transmit these packets according to the per-hop forwarding behaviors. Experimental results show that this system can greatly improve the end-to-end delivery quality of a streaming application.

Even though the fixed HH's (Fraunhofer Heinrich-Hertz-Institute) scheme for multi-view video coding can get very good performance by fully utilizing the predictions in both the temporal and view directions, the complexity of this inter-prediction is very high. This paper presents some techniques to reduce the complexity while maintaining the coding performance.

Objects like field lines and players are important for semantic analysis in soccer video. In this paper, we propose effective methods for field lines and players detection and recognition. Regions of field lines and players are first segmented from shot of wide angle view. Gray value top-hat transform is then performed on the segmented region to detect field lines. Mid-lines, end-lines and penalty-lines are then recognized based on their positions. Template matching method is adopted to detect and recognize players. Four types of templates for players are matched on the segmented region. Accurate position of player can be obtained after template matching, and the type of player is also identified. Experiment results on various data-sets with different production styles show effectiveness of our method.

In this paper we introduce a biorthogonal frequency-varying modulated lapped transform (BFV-MLT) with flexible tiling behavior of time-frequency plane and attenuation of ringing artifacts. Biorthogonality allows accessorial freedom in designing the analysis and synthesis filter-banks and FVMLT provides nonuniform decomposition of signals with ideal time or frequency resolutions. By constructing a new approach to FV-MLT, we incorporate biorthogonality into FV-MLT easily and achieve flexible representations of signals plus freedom in filter-bank design. We also prove that perfect reconstruction holds in BFV-MLT which confirms its application in compression schemes.

Information of text in videos and images plays an important role in semantic analysis. In this paper, we propose an effective method for text detection and localization in noisy background. The algorithm is based on corner response. Compared to non-text regions, there often exist dense edges and corners in text regions. So we can get relatively strong responses from text regions and low responses from non-text regions. These responses provide us useful cues for text detection and localization in images. Then using a simple block based threshold scheme, we get candidate regions for text. These regions are further verified by combining other features such as color and size range of connected component. Finally, text line is located accurately by the projection of corner response. The experimental results show the effectiveness of our methods.

The performance of the recent JPEG2000 Part 10 standard, known as JP3D, is evaluated for the lossy and lossless compression of hyperspectral imagery. Experimental results using a Karhunen-Loève transform (KLT) for spectral decorrelation and a 2D wavelet transform for spatial decorrelation compare the performance of JP3D against 2D JPEG2000 as specified by Part 2 of the standard. JP3D is used with both the 2D arithmetic-coding contexts as specified in the JP3D standard as well as non-standard experimental 3D contexts. Results reveal that, while for lossless coding, JP3D very slightly surpasses the performance of JPEG2000 Part 2, for lossy coding, JP3D fails to match the rate-distortion performance of the 2D Part-2 coder.

Object categorization plays an important role in computer vision, semantic based image content understanding, and image retrieval. Wavelet packet transform provides a very good observation for the images by sub-band filtering. Different objects have distinctive characteristics in the sub-bands of wavelet packets, which should be discriminative for objects classification. In this paper, an object categorization method using hierarchical wavelet packet texture descriptors is proposed. Comparisons between Gabor texture descriptor, pyramid of histograms of orientation gradients (PHOG) and the proposed hierarchical wavelet packet texture descriptors on the widely used OT, Scene-13 and Sport event datasets are also given. Experimental results show that object categorization performances of the proposed texture descriptors are better than that of Gabor texture descriptor and as good as that of PHOG shape descriptor. Object categorization performances of the texture descriptors under various decomposition levels and wavelet bases are discussed. Performances of texture descriptors of global and local images with different partition patterns are also analyzed.

An application-driven cross-layer design of multiuser H.264/AVC video transmission over wireless networks is proposed in this paper. An objective function for cross-layer optimization is developed based on the parameters abstracted from the application layer, the media access control layer and the physical layer. Our objective is to maximize the video perceptual quality after delivery with constraint of the limited wireless resources. Simulation results show that the proposed scheme performs significantly than the conventional scheduling schemes for video transmission.

Transmission distortion analysis for video streams is a considerably challenging task. In this letter, a compressed-domain-based (CDB) transmission distortion model for precoded H.264/advanced video coding video streams is developed. Unlike the earlier schemes, which were based on pixel domain and required a complete decoding of the compressed video streams, the CDB model only requires some information on the video features, which can be directly extracted from the compressed video streams. Therefore, the complexity of the calculations is substantially reduced, which is well suited for real-time applications. More specifically, the model is applicable to the real-time transmission for precoded video streams, such as video on demand and mobile video. The experimental results demonstrate high accuracy of the model. Furthermore, an application example using the CDB model in resource allocation in real-time multiuser video communication reveals the applicability and effectiveness of the model.

Parts of compressed video streams may be lost or corrupted when being transmitted over bandwidth limited networks and wireless communication networks with error-prone channels. Error concealment (EC) techniques are often adopted at the decoder side to improve the quality of the reconstructed video. Under the conditions of a high rate of data packets that arrives at the decoder corrupted, it is likely that the incorrectly decoded macro-blocks (MBs) are concentrated in a connected region, where important spatial reference information is lost. The conventional EC methods usually carry out the block concealment following a lexicographic scan (from top to bottom and from left to right of the image), which would make the methods ineffective for the case that the corrupted blocks are grouped in a connected region. In this paper, a temporal error concealment method, adaptive error concealment order determination (AECOD), is proposed to recover connected corrupted regions. The processing order of an MB in a connected corrupted region is adaptively determined by analyzing the external boundary patterns of the MBs in its neighborhood. The performances, on several video sequences, of the proposed EC scheme have been compared with those obtained by using other error concealment methods reported in the literature. Experimental results show that the AECOD algorithm can improve the recovery performance with respect to the other considered EC methods.

Event detection is essential for sports video summarization, indexing and retrieval [1]. In this paper, based on three generally defined shot types, goal events are detected by combining heuristic rules with unsupervised fuzzy c-means algorithm. First heuristic rule based primary selection/filtering for potential goals is carried out in the shot layer which composed of three generally defined shot types, together with the number of frames within each shot is recorded as representative feature. Then to further classify goal events from other potential goals unsupervised fuzzy c-means (FCM) algorithm is adopted. The main contribution of this work is the combination of heuristic rule which is based on three generally defined shot types with unsupervised fuzzy c-means algorithm. And when defuzzified with prior knowledge of the number of goals within each match, accurate and robust results can be achieved over five half matches from different series produced by different broadcast stations.

This paper proposes a novel scene change detection algorithm based on the 3D wavelet transform. In scene change including cut, dissolve and fade, the frames have particular temporal and spatial layouts. The dissolve and the fade have strong temporal and spatial correlation; on the contract the correlation of the cut is weak. The 3D wavelet transform can effectively express the correlation of the several successive frames since the low-frequency and high-frequency component coefficients have the proper statistics regularities which can effectively identify the shot transitions. Three features are computed to describe the correlation of the shot transitions, which are input to support vector machines for scene change detection. Experimental results show that the method is effective for the gradual shot transition detection.

In this paper, we propose several improvements to the 3D-tarp coder for the lossy compression of hyperspectral imagery. Specific ameliorations include use of principal component analysis instead of a wavelet transform for spectral decorrelation, use of the quincunx wavelet transform instead of the traditional dyadic decomposition in the spatial direction, and spectral partitioning with skipping of insignificant zeros. Experimental results reveal that the enhanced coder achieves improved rate-distortion performance.

The starting and ending (S & E) frames of each text that embeds in a scene or is graphically added to video, provide not only important clues for highlight events detection in semantic-based video analysis, indexing and retrieval but also hint for decoding the videos structure and classification. In this paper, a fast text tracking method of determining S & E frames is proposed, which consists of two modules: text matching, where a mean absolute difference (MAD) based line against line matching (LALM) criterion is proposed to determine whether the detected texts in two frames are similar or not; and S & E frames determination, where an MAD based line matching algorithm is introduced in the direct current (DC) domain to determine the accurate positions of the S & E frame. Experiments conducted with a variety of video sources show that the proposed text tracking method is fast and robust.

An embedded lossy-to-lossless coder for hyperspectral images is presented. The proposed coder couples a reversible integer-valued Karhunen-Loeve transform with an extension into 3-D of the tarp-based coding with classification for embedding (TCE) algorithm that was originally developed for lossy coding of 2-D images. The resulting coder obtains lossy-to-lossless operation while closely matching the lossy performance of JPEG2000. Additionally, for lossless compression, it consistently outperforms not only JPEG2000 but, often, several prominent purely lossless methods.

In this paper, we propose an efficient compression algorithm for hyperspectral images. It is based on the correlation coefficients adaptive AT-3DSPIHT coding in the domain of WPT (wavelet packet transform). According to the characteristics of correlation coefficients between spectral bands, a binary tree spectral band grouping algorithm is carried out to divide the adjacent spectral bands into different mode groups. Along with this, WPT with the corresponding decomposition levels and a proper AT-3D zerotree are determined adaptively. Several AVIRIS images are used to evaluate the proposed algorithm. Compared with the existing 3D-based algorithms, the proposed adaptive AT-3DSPIHT achieves the best compression performance at lower rates. Moreover, at the low correlated adjacent bands, our proposed algorithm also beats the 2DSPIHT and JPEG2000-MC algorithm respectively.

Packet marking plays a key role in video transmission over DiffServ Networks. Legacy marker systems mark video packets mainly based on the coding types of corresponding frames in order to achieve priority transmission. In this paper, we propose a novel packet marker system for realtime H.264 video delivery. Our system marks packets depending on the corresponding frames' contribution to the video quality. The frame relative importance index (FRII) is defined to estimate each frame's contribution, using the degradation of video quality caused by the loss of information in that frame. Consequently, packets of each frame are differently protected according to the FRII in delivery and the video perceptual quality is improved. Simulation results show that our system outperforms the legacy marker systems in terms of the quality of the delivered H.264 video streams.

Error concealment (EC) techniques are often adopted at decoder side to improve the reconstructed video qualities. In this paper, a texture based selective boundary matching algorithm (TSBMA) is proposed to conceal erroneous macro-blocks (EMBs). Different from the conventional boundary matching algorithms which are carried out progressively in the raster scan mode, the processing order of the TSBMA for an EMB in a connected EMBs region is determined by the texture intensity information of its neighboring blocks, which are correct or recovered. The EMB with maximum texture intensity in its neighbors are firstly recovered. It is likely that the EMB which has more corrected MBs in its neighbors with stronger texture intensity has higher possibility to be recovered accurately, which will provide valuable information to recover the left EMBs in its neighbors. Thus, the proposed TSBMA can avoid the dependent recovery problem in the conventional BMAs. Moreover, the influence of boundary width and confidences of boundaries on the recovery results are discussed. Comparison results with different error concealment methods on several video sequences show the effectiveness of the proposed TSBMA.

Based on motion mode adaptive lifting motion compensation, an embedded high performance asymmetric 3DSPIHT wavelet color video codec (MA-MC-AT3DSPIHT) is proposed in this paper. First, by making use of the direction histograms of motion vectors, the motion mode is determined according to whether or not the non-static motion vectors consistency exists in the GOP, and different temporal filters are imposed thereby for motion compensation. Then, the chrominance components at spatial high frequency subbands are quantified conforming to the JPEG2000 quantification scheme to further improve the coding performance. Experimental results show that the new MA-MC-AT3DSPIHT wavelet color video codec guarantees at least 6.0 db PSNR improvement at all rates over MPEG2. Moreover, with less than one seventh motion vectors coding cost, it yields a PSNR performance competitive with some of the other results reported in the literature.

In this paper, we present a 3D hyperspectral images lossless compression algorithm, which is based on the asymmetric 3D quincunx structured wavelet transform (A3D-QWT) and the adaptive classification coding. A3D-QWT is a simpler 3D transform compared to the classical asymmetric 3D pyramidal wavelet structure (A3D-DWT) with nearly equal entropy; furthermore the spectral correlation after this quincunx structured transform is higher, which is useful to the adaptive classification coding. The adaptive classification coding can make full use of not only the spatial correlation but also the spectral correlation characteristics of hyperspectral images. Experiments show that our method is capable of providing a higher compression performance for hyperspectral images.

In this letter, we propose an efficient lossless compression algorithm for hyperspectral images; it is based on an adaptive spectral band reordering algorithm and an adaptive backward previous closest neighbor (PCN) prediction with error feedback. The adaptive spectral band reordering algorithm has some strong points. It can adaptively determine the range of spectral bands needed to be reordered, and it can efficiently find the optimum branches. Hyperspectral images have a large number of spectral bands, which express the same land cover structure and have high correlation. The adaptive backward PCN prediction with error feedback can sufficiently make use of this correlation. Experiments show that implementing both the reordering of the spectral bands before prediction and the prediction with error feedback improve compression performance

A novel hyperspectral-image lossless compression scheme in the wavelet domain is proposed in this letter. This scheme is based on the context-based adaptive classified arithmetic-coding technique. The adaptive classified scheme divides each of the residual images between the two adjacent wavelet images into different classes, resulting in not only skipping the coding of a lot of insignificant zeros but also making the similar coefficients cluster together. Through experiments, we found that, when similar coefficients are clustered together, the arithmetic coding can achieve a higher performance than no clustering. Therefore, we can say that the adaptive classified scheme makes a better use of the characteristics of hyperspectral images and the characteristics of the arithmetic-coding technique. Experiments show that our proposed scheme is capable of providing high compression performance.

In this paper, we propose a generic framework to human perception analysis in video understanding based on multiple visual cues. Video features that prominently influence human perception, such as motion, contrast, special scenes, and statistical rhythm, are first extracted and modeled. A perception curve that corresponds to human perception change is then constructed from these individual models using linear or priority based fusion approach. As an important application of the perceptive analysis framework, a feasible scheme for video summarization is implemented in order to demonstrate the validity, robustness, and generality of the proposed framework. The frames that correspond to the peak points in these individual models and the fusion curve are extracted as multilevel summarizations that include video keywords, keyframes, and dynamic segments. The subjective evaluations from a supplementary volunteer study on video summarizations indicate that the analysis framework is effective and offer a promising approach to semantic video management, access, and understanding

Video text information plays an important role in semantic-based video analysis, indexing and retrieval. Video texts are closely related to the content of a video. Text-based video analysis, browsing and retrieval are usually carried out in the following for steps: video text detection, localization, segmentation and recognition. Videos are commonly stored in compressed formats where MPEG coding techniques are adopted. In this paper, a DCT coefficient based multilingual video text detection and localization scheme for compressed videos is proposed. Candidate text blocks are detected in terms of block texture constraint. An adaptive method for the horizontal and vertical aligned text lines determination is then designed according to the run length of the horizontal and vertical block numbers. The remaining block regions are further verified by local block texture constraints. And the text block region can be localized by virtue of the horizontal and vertical block texture projections. Finally, a foreground and background integrated (FBI) video text segmentation approach is adopted in this paper to eliminate the complex background in text regions. The final experimental results show the effectiveness of our methods

In H.264/AVC, a novel criterion named the rate distortion optimization (RDO) is employed to select the optimal coding modes for each macroblock (MB) within the intra prediction coding pictures, which can achieve a high compression ratio while leading to a great increase in the complexity and computational load unfortunately. In this paper, a fast mode decision algorithm based on integer transform and adaptive threshold is proposed. Before the intra prediction, integer transform operations on the original image are executed to find the directions of local textures. According to this direction, only a small part of the possible intra prediction modes are tested at the first step. If the summation of differences (SAD) of the reconstructed block corresponding to the best mode is smaller than an adaptive threshold, calculation is terminated. Otherwise, more possible modes are needed. Simulations show that the fast mode decision algorithm proposed in this paper can accelerate the speed of intra picture coding significantly only with a negligible PSNR loss or a bit rate increment.

In this paper, we propose a lossless compression algorithm for hyperspectral images; it is based on the adaptive classified arithmetic coding in wavelet domain and the adaptive spectral band reordering algorithm. The adaptive classified scheme divides each of the residual images after wavelet transform into different classes, and then the adaptive arithmetic coding is performed for each of the classes. This classified coding scheme saves a lot of coding bits. The adaptive spectral band reordering algorithm finds out the nearly best reference band for each of the bands, so the spectral correlation is better used. Combining these two algorithms makes full use of the characteristics of hyperspectral images. Experiments show that our method is capable of providing a high compression performance.

Scene change detection is a fundamental step in automatic video indexing, browsing and retrieval. Fade in and fade out are two kinds of gradually changing scenes which are difficult to be detected in comparison with the abruptly changing scenes. The salient character of flashlight effect is the luminance change, which is caused by abrupt appearance or disappearance of the illumination source. Performance of shot boundary detection is not satisfactory for the video sequences containing flashlights, if no flashlight discrimination strategy is adopted. In this paper, an effective fades and flashlight detection method is proposed for both the compressed and uncompressed videos, based on the accumulating histogram difference (AHD). This fades detection method is proposed in terms of their mathematical models. AHDs of all the two consecutive frames during fades transitions can be classified into six cases. The flashlight detection method is proposed based on the AHD and the energy variation characters. AHD and energy variation characters for the starting and ending frames of a flashlight have certain regularities, which can also be expressed by cases. Thus the fades and flashlight detection problems are converted into cases matching ones. Experimental results on several test video sequences with different bit rates show the effectiveness of the proposed AHD based fades and flashlight detection method

In this paper, a new method for generating different texture images is presented. This method involves a simple transform from a certain one-dimensional (1-D) signal to an expected two-dimensional (2-D) image. Unlike traditional methods, the input signal is generated by a simple 1-D function in our work instead of a sample texture. We first transform the 1-D input signal into frequency domain using fast Fourier transform. Based on the sufficient analysis in 2-D discrete cosine transform (DCT) domain, where each of the coefficients expresses a texture feature in a certain direction, the 2-D pseudo-DCT coefficients are then constructed by appropriately rearranging the Fourier coefficients in terms of their frequency components. Finally, the corresponding texture image can be produced by 2-D inverse DCT algorithm. We applied the proposed method to generate several stochastic textures (i.e., cloud, illumination, and sand), and several structural texture images. Experimental results indicate the good performance of the proposed method.

This paper presents a novel rate-distortion (R-D) model for rate control of video coding. First, we investigate the well-known heat conduction equation (HCE) in the heat conduction process for a thin bar. HCE describes the dynamic distribution of temperature in a thin bar using a partial differential equation (PDE). Motivated by the heat conduction process in a thin bar, a new rate transmission equation (RTE) is proposed to describe the dynamic behaviors of the bit-rate fluctuation during video coding. Second, a particular solution of RTE is employed as our proposed R-D model, which consists of two variables, i.e., the distortion D and the source statistical character M. Based on our model, a corresponding rate-control scheme is proposed for MPEG coding. Finally, extensive experimental results are reported to show that, compared with the well-known MPEG-4 Q2 rate-control scheme, our proposed work achieves better buffer status, higher control accuracy, and stabler and more consistent picture quality.

In this paper, a novel lifting integer wavelet transform based on difference correlation structure (DCCS-LIWT) is proposed. First, we establish a relationship between the performance of a linear predictor and the difference correlations of an image. The obtained results provide a theoretical foundation for the following construction of the optimal lifting filters. Then, the optimal prediction lifting coefficients in the sense of least-square prediction error are derived. DCCS-LIWT puts heavy emphasis on image inherent dependence. A distinct feature of this method is the use of the variance-normalized autocorrelation function of the difference image to construct a linear predictor and adapt the predictor to varying image sources. The proposed scheme also allows respective calculations of the lifting filters for the horizontal and vertical orientations. Experimental evaluation shows that the proposed method produces better results than the other well-known integer transforms for the lossless image compression.

This paper presents two error concealment (EC) techniques for image transmission in JPEG2000, one for the lowest frequency coefficients (the low-frequency EC) and the other for high frequency coefficients (the high-frequency EC). The low-frequency EC algorithm uses a data hiding technique and the packet structure of JPEG2000. The low-frequency coefficients, which are taken as the hidden data, are extracted from the compressed bitstream, and embedded back into the same bitstream. The restored hidden data is used to conceal errors. The high-frequency reconstruction is performed on a bitplane basis. The damaged bitplanes are recovered according to the correlation in the wavelet subbands structure, in which the edge information is detected at first. Experiments show the effectiveness of these algorithms.

A sampling theorem for frames in translation-invariant subspaces of L/sup 2/(R) is established, which is a generalization of a result of W. Chen and S. Itoh for Riesz bases (see IEEE Trans. Sig. Processing, vol.46, p.2822-4, 1998). Then, some necessary conditions for sampling are derived for Riesz bases. Some relationships and properties are also derived about the relevant functions.

A wavelet packet image coding algorithm for synthetic aperture radar (SAR) image data compression is proposed in this paper. High rate-distortion (R-D) performance of this algorithm [wavelet packet quantization universal trellis-coded quantization (WPQUTCQ)] is achieved by incorporating the wavelet packet transform for representing the rich texture information of SAR image data, the quadtree technique for classifying the wavelet packet coefficients, and the universal trellis-coded quantization (UTCQ). For a typical SAR image, WPQUTCQ outperforms set partitioning in hiearchical trees and JPEG2000 by about 1.21 and 0.81 dB in peak signal-to-noise ratio (PSNR) at 2 bpp, respectively. The effect of best basis selection on the R-D performance of coding algorithm for SAR image data compression is also evaluated through comparing the effects of different cost functions (e.g., entropy, the texture energy, which is designed for SAR image data compression particularly, and the cost function, based on the actual coding strategy proposed by us) on the R-D performance of WPQUTCQ. The experimental results show the suitability of the cost function based on the actual coding strategy for SAR image data compression when compared with other cost functions. For a typical SAR image, the coding results by WPQUTCQ corresponding to the cost function based on the actual coding strategy achieve gains 0.59 and 0.49 dB on average in PSNR when compared with the coding results corresponding to entropy and the texture energy, respectively.

Several scene-detection algorithms, which are only based on bit rate fluctuations, have been proposed. All of them are presented on the fixed thresholds, which are obtained by the empirical records of the video characteristics. Due to the sensitivity of these methods to the accuracy of the records, which are generally obtained by testing several values repeatedly, bad performance evaluation might be observed for the actual scene detection, especially for real-time video traffic. In this paper, we review the previous works in this area, and study the correlation between the scene duration and the scene change at the frame level, and simultaneously investigate the local statistical characteristics of scenes such as variance and peak bit rate etc. Based on this analysis, an effective decision function is first constructed for the scene segmentation. Then, we propose a scene-detection algorithm using the defined dynamic threshold model, which can capture the statistical properties of the scene changes. Experimental results using 15 variable bit rate MPEG video traces indicate good performances of the proposed algorithm with significantly improved scene-detection accuracy.

The success in discrete cosine transform(DCT) image coding is mainly attributed to recognition of the importance of data organization and representation. In this paper, we proposed an embedded image coder based on quadtree set partition in DCT domain (EZDCT) which is suitable for many kinds of DCT coefficients reorganization schemes. The experimental results show that it is among the state-of-the-art DCT-based image coders when compared with the famous DCT-based image coders, such as EZDCT and MRDCT. For example, for the Barbara image, EQDCT outperforms JPEG EZDCT and MRDCT by 3.3,1.71,1.70 dB in peak-signal-to-noise ratio at 0.25 bpp, respectively.

An improved parallel interference cancellation (PIC) detector is presented based on the conditional likelihood function. The improved PIC benefits from an additional correcting module after the last stage, where the maximum likelihood algorithm is employed using only the conditional likelihood function, which is calculated from tentative decision information. Analysis and simulation show that the new algorithm can improve the bit error rate (BER) performance of PIC with much lower computational complexity than the optimum detector.

In this paper, we present a wavelet packet image coding algorithm based on quadtree classification and UTCQ. It is composed of four parts: (1) wavelet packet decomposition and best basis selection based on a new cost function, (2) a quadtree classification procedure, used to classify the wavelet packet coefficients into two sets: a significant one and an insignificant one, (3) the universal trellis coded quantization, used to code the significant coefficients sets, (4) the entropy coder, used to code the indices of the universal trellis coded quantizer output. The image coding results, calculated from actual file sizes and images reconstructed by the decoding algorithm are either comparable to or surpass previous results for texture-rich images.

Burstiness property plays an important role in the variable bit rate (VBR) video traffic. One of the most significant issues in providing quality-of-service (QoS) guarantees for the real-time multimedia communications in a high-speed network is to estimate the burstiness accurately. In this paper, an efficient burstiness estimation model (fractal exponent model) for VBR video stream is proposed. In order to improve the accuracy of the burstiness measure, a new operator is firstly introduced, which is similar to the convolution operator. Then, using burstiness index derived from the fractal exponent function, we can estimate the burstiness characteristics of VBR video correctly. Simulation results using 20 VBR MPEG video traces indicate good performance of the proposed algorithm with significantly improved burstiness estimation accuracy.

Abstract – In this paper, we propose a new wavelet packet image coding technique for synthetic aperture radar(SAR) remote-sensing image data compression. The image compressibility of this algorithm(WPQTCQ)is improved by incorporating wavelet packet transform for rich texture component of SAR image data, the quadtree set partition for sorting the wavelet packet transform coefficients and universal trellis coded quantization. The experimental results show that WPQTCQ is among the state-of-theart image coders for SAR image data. To a typical SAR image at 2bpp, WPQTCQ out performs SPIHT and JPEG2000 by about 2.45dB and 1.34dB in peak signal-to-noise ratio, respectively.

In this paper, we presented a novel wavelet packet image coding approach which provides the functionality of fine granular bitstream scalability. The proposed progressive wavelet packet image coding scheme consists of three parts, wavelet packet decomposition, a quadtree sorting procedure for classifying wavelet coefficients and universal trellis-coded quantization for quantizing the sorted coefficients. The image coding results, calculated in PSNR and images reconstructed by the decoding algorithm, are either comparable to or surpass previous results, due to the flexible representation ability of wavelet packet, the effective quadtree classifier and the improved granular fidelity of the UTCQ over scalar quantization.

In order to guarantee quality of service (QoS) over Internet, traffic analysis, traffic management have been active research areas. A lot of facts show the Internet traffic and variable bit rate videos streaming all are characterized by self-similar property. Hurst parameter as an important factor that reflects the self-similar property is a key to traffic management and QoS. In this paper existing wavelet methods for the estimation of the Hurst parameter of self-similar traffic is systematically analyzed and examined. The effects of wavelet functions, vanishing moments and wavelet decomposition levels to the results of wavelet methods for acquiring the Hurst parameter are investigated via numerical experiments. Some useful conclusions are drawn on the relationship between the accuracy of the methods and the selection of the order of vanishing moments and the selection of wavelet functions.

When compressed video is transmitted over practical communication channels, there are error bits or loss packets. Especially, even if there is a few errors in an intra-frame, it can propagate both in spatial and temporal largely. In order to improve the image quality of decoding, the error concealment in the decoding side is necessary. In this paper, a damaged macroblock of the luminance component is separated into four sub-blocks. The motion vector of each sub-block is assumed to be the motion vector of the corneous region which nearest the sub-block. Finally, the overlapped motion compensation is employed to further improve the error concealment. The scheme is also used to conceal the error of the inter-frame. The simulation results show that it is effective. Compared with the conventional schemes, the concealment average peak signal-to-noise ratio is increased about 2dB at 5% and 10% loss rate of macro block.

Numerous techniques have been developed over the last several decades to efficiently transmit video across noisy channel. In this paper, we propose an unequal error protection scheme for the progressive video compression through cascading a bit-sensitive-based error control channel coding with an embedded multiresolutional 3D-SPIHT scalable video coder. It provides an efficient manner for the joint source-channel coding system to trade off the rate-distortion (RD) performance and the error-resilient performance for the unknown uncontrollable noisy channels at a given available transmission rate. This coding system is easy to implement and has acceptable complexity. The performance of the joint source-channel coding approach under various channel conditions is evaluated.

In the field of content-based visual information analysis, the detection of visual features is a significant topic. In order to process video data efficiently, visual features extraction is required. Many advanced video applications require direct manipulation of compressed video data. An effective approach that detects edges in MPEG compressed images is proposed. First, DCT coefficients of 8/spl times/8 subblocks are analyzed on their meaning in determining boundaries. Then, we consider two types of ideal linear edges cutting through a block of size 8/spl times/8. Based on these edge models, we derive an edge detection approach from ten normalized DCT coefficients, obtaining the general rules of edge classification. Finally, we test the proposed algorithm on different images, and compare our method with other edge classification approaches. Simulations show that our approach can be used to estimate the edge information of images from their DCT coefficients more effectively than those proposed previously.

A new method of nonseparable nonlinear wavelet decomposition is proposed, which is suited for the task of image compression, especially for lossless coding applications. It is based on a certain statistical operator that is defined here according to the Markov random field theory. In contrast to the previous nonlinear predictors such as the median or morphological operators, this statistical operator can sufficiently take advantage of the statistical correlation between neighboring pixels. It can be used to realize integer-valued wavelet transforms, which can avoid quantization with the image detail signals being zero (or almost zero) in the smooth gray-level variation areas at a big probability. Numerical results show that the entropy of the coefficients in the transform domain obtained with this new method is smaller than that obtained with the other nonlinear transform methods.

A high performance full scalable video codec called embedded multiresolution motion-compensated three-dimensional SPIHT (EM-MC-3DSPIHT) is presented. By combining the bit allocation of motion vector and the scalable spatio-temporal orientation wavelet tree with the EM-MC-3DSPIHT algorithm, it not only provides a full scalable decoding, but also significantly improves the rate-distortion (RD) performance compared with that of Kim et al's codec (see IEEE Trans. on Circuit and Syst. Video Tech., vol.10, p.1374-87, Dec. 2000), particularly at low bit rates. Simulations show that the proposed video codec provides performance not only better than that of MPEG-2 at medium to high bit rates but also comparable to H.263 at low bit rates.

With the development of the Internet, variable bit rate (VBR) video will be the major component of future multimedia services. In order to guarantee quality of service (QoS) in real-time transmission, on-line prediction of VBR video traffic integrated with a mechanism for dynamic resource allocation in RCBR (renegotiate constant bit rate) networks has been an active research area. We exploit the short-range dependence (SRD) and long-range dependence (LRD) characteristics of MPEG video and develop a novel structure-based LMS algorithm to predict the bandwidth required by the future frame and group of pictures (GOP). Compared to the LMS algorithm, the modified LMS algorithm can predict the bit rate of frames quickly and accurately without any delay. Through analysis of the queuing of video traffic in the network buffer, dynamic bandwidth allocation using prediction in RCBR networks exhibits better performance than fixed bandwidth allocation.

To guarantee quality of service (QoS) in future integrated service networks, traffic sources must be characterized to capture the traffic characteristics relevant to network performance. Recent studies reveal that multimedia traffic shows burstiness over multiple time scales and long-range dependence (LRD). Its burstiness and self-similarity make it hard to control and guarantee QoS, so analysis of its queuing behavior attracts much interest and attention. A new scene based M/G/1 queuing model is put forward to analyze the queuing behavior of video streams in the network buffer. Integrated with the scene length distribution, the variance of scene size and the average bit rate of GOPs, the proposed model can accurately describe the queuing behavior of video traffic overall.

To the problem of no overall optimal merger for one-way merger in the segmentation algorithm proposed by Wang et al., (1999), we propose a method of overall optimal search and merger. At the same time, for the problem of merging a segment which has non-value (value-segment) and a segment whose values are zeros entirely (zeros-segment) to a large segment in Wang's method, we also propose a corresponding method to solve the problem. The main techniques use the local cosine transform (LCT) algorithm for a single small segment, rather than folding processing using its original neighboring data, instead of making zero extension, and then fold the each zero-extension segment. A great deal of numerical simulations validate that this new improved technique solves several problems of the binary-based segment algorithm and Wang's segment algorithm; it not only obtains adapted effective segmentation results, but also there are not many redundancy segmentations.

For time series prediction by neural networks, the neural network should take advantage of the relationship between time series, and one time series can then be predicted based on the information provided not only by itself, but also by other related ones. We use neural networks to predict the multivariate time series combined from the open, high, low and close Shanghai Stock Exchange index series.

It is well known that the stock market, viewed as a complex, open, and nonlinear dynamical system, is affected simultaneously by many factors, such as international environment, government policies, political situation, economic situation, the public psychology over some events, some rumors, and so on, which intrinsically influence each other and make the relationships very complicated. Some of these have long influences on the market, while others have short influences on it. The arguments of synergetics, cooperation and competition among the state variables led to the case in which the system is governed by only a few slow variables. But we have no way to exactly know which and how the states govern the evolution of the system. All that we have available is the observable generated by the states, time series (index price series) from the system, which carries the information on the system of interest. How can we understand the dynamics of the system from the observable, say, the evolution of the system? We reconstruct the attractor of stock market from its stock index series with respect to delay embedding theorem (F. Takens, 1981). The attractor can then be fully unfolded in our reconstructed phase space without trajectory intersections, getting a diffeomorphic copy of the original attractor. It is suffice to say that the evolution in reconstructed phase space faithfully images, on the whole, the evolution in the original phase space, consequently laying a theoretical foundation for predicting stock index series.