Video Frame Interpolation (VFI) aims to synthesize realistic intermediate frames from preceding and following video frames. Although many VFI methods perform well on specific motion types, their versatility in handling both large and small motions remain limited. In this work, we propose ATM-VFI, which is a novel hybrid CNN-Transformer architecture that effectively combines the strengths of the CNN (efficiency and considering the detail information) and the transformer (well adopting the global information). It utilizes an Attention-to-Motion (ATM) module and adopts a dual-branch (local and global branches) mechanism to intuitively formulate motion estimation and estimate global and local motion adaptively. Furthermore, we introduce a four-phase training procedure leveraging small-to-medium and large motion datasets to enhance versatility and training stability. Extensive experiments demonstrate that the proposed ATM-VFI algorithm outperforms state-of-the-art methods. It can well interpolate the video frames with a variety of motion types while maintaining high efficiency.

Variational mode decomposition (VMD) is a widely used method for analyzing and denoising temporal and non-stationary signals. Several extensions of VMD, such as the wavelet transform with VMD (VMD-DWT), non-local means with VMD (VMD-NLM), and their combination (VMD-DWT-NLM), have demonstrated satisfactory performance. However, these VMD-based methods often require substantial online computation due to the non-linear decomposition process, especially when processing large datasets. To address this challenge, this study proposes a novel approach called neural VMD (NVMD), which integrates VMD’s decomposition capabilities with the powerful feature extraction of neural networks (NN), while adaptively selecting the optimal number of intrinsic mode functions (IMFs) for temporal signal analysis and denoising. Two systems, NVMD(A) and NVMD(P), were developed, incorporating autoencoder-based NN and progressive NN, respectively. We evaluated the proposed NVMD framework on the task of ECG signal denoising using the MIT-BIH dataset, contaminated with various noise types and different signal-to-noise ratio (SNR) levels. Experimental results show that the proposed method significantly improves the SNR, reduces computational complexity, and adaptively selects the optimal number of IMFs for effective ECG denoising.

Anomaly segmentation for urban-driving scenes has gained increasing attention in recent years due to safety concerns in autonomous driving. The ability to accurately detect and segment anomalous objects is critical to ensure the safety of autonomous driving systems in real-world environments. In this paper, we propose two key innovative techniques to address this challenge: (i) introducing a novel inlier training loss derived from the optimal transport, which enhances the ability to identify pixels from inlier classes; and (ii) applying the softplus loss instead of the hinge loss, as the former provides a faster convergence rate and better anomaly segmentation performance. Our approach demonstrates state-of-the-art results, evaluated by the area under precision-recall curve (AuPRC), on the road anomaly and SMIYC (obstacle tracking) datasets, thereby pushing the boundaries of anomaly detection in urban-driving scenarios.

Interpolation aims to retrieve missing values. Resulting from the inaccuracy of sensors, channel distortion, and a variety of measurement errors, signals are contaminated with noise. To well interpolate these noisy signals, an adaptive interpolation algorithm is proposed in this work. First, a novel atom selection mechanism, which incorporates the ridge regression problem and the iterative hard-thresholding algorithm, is applied to identify dominant frequencies. The essential time-frequency information is thus precisely extracted. Prolate spheroidal wave functions, which are highly concentrated in the time-frequency plane, are then appropriately modulated. Finally, atoms are refined to form a proper basis dictionary for signal expansion and interpolation. Comprehensive experiments are conducted to validate that the proposed RidIHT algorithm has outstanding performance in interpolating from noisy samples.

Age estimation has long been a crucial topic in image processing, with applications across various scenarios. Since the aging process and the aging rate for men and women are different, in this work, we propose AGMixer, which leverages the gender information to reduce the error of age estimation. We use facial representation learning (FaRL) pretrained by the vision transformer (ViT) as a feature extractor and employ a mixer layer for effective feature fusion, achieving lower error rates in age estimation. To well utilize gender features and ordinal information in age, we annotated gender labels for CACD2000, CLAP2016, and FG-NET and improved the ordinal distance encoded regularization (ORDER) loss. We compared our method with others on UTKFace, AFAD, AgeDB, CACD2000, CLAP2016, and FG-NET datasets. Experiments show that the proposed algorithm achieves the lowest mean absolute error (MAE) across ALL of the 6 datasets. We make the source code public on GitHub.

This paper presents new theoretical and application aspects on inferring the parameters in integral transforms. An integral transform, e.g., the linear canonical transform (LCT), usually has many parameters. It is a difficult issue to find the optimal parameters of the integer transform. In this work, we propose a generalized inference method to determine the optimal parameters of the LCT and the fractional Fourier transform (FrFT). With proper design of object functions, optimal parameter estimation can be performed effectively and automatically. The LCT with the optimal parameter set can well represent a complex signal and minimize the energy outside of the low-frequency band in terms of the ℒ1or the ℒ2norm. It is applicable in signal processing, filter design, system modeling, and machine learning.

Pedestrian recognition is crucial for computer vision and self-driving system design. In this work, the Swin Transformer (SwinT), which can capture global contextual information and handle long-range dependencies, is adopted to perform pedestrian detection in the complex scene, including the heavy occlusion scenario. The SwinT is capable to capture multi-scale features and spatial relationships in images, making it well-suited for the challenging task of occluded pedestrian detection. We also apply a two-stage detector based on the faster R-CNN framework, which consists of a cascade region proposal network (RPN) and a region of interest (ROI) head, and use anchors and the focal loss during the RPN training process. The experiments conducted on Euro City Persons and CityPersons datasets demonstrate the outstanding performance of the proposed architecture in detecting heavily occluded pedestrians, highlighting its ability to handle challenging scenarios that traditional methods may struggle with.

Interpolation and extrapolation are to fill and predict missing values. To perform these works well, the bases that are highly concentrated in the time-frequency plane and match the characteristics of signals should be adopted. In this work, we propose several algorithms to deal with these tasks. Their concepts are based on generalized chirp-modulated and time-frequency division forms of prolate spheroidal wave functions, boasting attracting mathematical properties. Time-frequency divided prolate spheroidal wave functions can well approximate a signal whose energy distributes sparsely in the time-frequency plane. Moreover, generalized chirp modulation can well model a signal whose instantaneous frequency varies with time. Several experiments are conducted and the effectiveness of our proposed algorithms is manifested by the experimental results and quantitative evaluation.

Image feature point extraction is critical in several computer vision tasks like object detection, image stitching, visual re-localization, 3D reconstruction, and simultaneous localization and mapping (SLAM). In recent years, learning-based feature extractor approaches have become increasingly prevalent in computer vision and can achieve even better matching performance. Nevertheless, how to label and describe the discriminative feature is problematic. In this paper, we proposed a weakly-supervised learning approach that integrates a transformer and U-net-like convolution networks to better consider global and local contexts. Moreover, we concatenate multiple-size dilation convolutions to achieve a wider receptive field. Experiments show that the proposed approach has a performance and is robust to viewpoint change and illumination variation.

Salient object detection (SOD) is a preprocessing step for several computer vision techniques, including visual tracking, image captioning, image segmentation, and so on. In this work, several techniques are adopted to improve the accuracy of SOD. Instead of directly using edge maps as guidance, we improve the adaptive two-stream encoder by employing a clever technique to generate body maps and detail maps, which can provide much information for the final predictions. Regarding body maps and detail maps, different parameter contrasts are provided for users to choose the desired results. Compared to state-of-the-art SOD algorithms, our method outperforms almost all other methods on twelve datasets under two evaluation metrics.

Photographs taken through a glass window are susceptible to disturbances due to reflection. Therefore, single image reflection removal is crucial to image quality enhancement. In this paper, a novel learning architecture that can address this ill-posed problem is proposed. First, a novel reflection removal pipeline was designed to reconstruct the missing information caused by the camera imaging process using the proposed missing recovery network. Second, to address the issues in existing reflection removal strategies, we revisit several auxiliary priors and integrate them by defining an energy function. To solve the energy function, a convolutional neural network-based optimization scheme was proposed. Finally, we investigated the dark channel responses of reflection and clean images and found an interesting way to distinguish between these two types of images. We prove this property mathematically and propose a novel loss function called dark channel loss to improve performance. Experiments show that the proposed method outperforms state-of-the-art reflection removal methods both quantitatively and qualitatively.

Vehicle re-identification (ReID) has attracted considerable attention in computer vision. Although several methods have been proposed to achieve state-of-the-art performance on this topic, re-identifying vehicle in foggy scenes remains a great challenge due to the degradation of visibility. To our knowledge, this problem is still not well-addressed so far. In this paper, to address this problem, we propose a novel training framework called Semi-supervised Joint Defogging Learning (SJDL) framework. First, the fog removal branch and the re-identification branch are integrated to perform simultaneous training. With the collaborative training scheme, defogged features generated by the defogging branch from input images can be shared to learn better representation for the re-identification branch. However, since the fog-free image of real-world data is intractable, this architecture can only be trained on the synthetic data, which may cause the domain gap problem between real-world and synthetic scenarios. To solve this problem, we design a semi-supervised defogging training scheme that can train two kinds of data alternatively in each iteration. Due to the lack of a dataset specialized for vehicle ReID in the foggy weather, we construct a dataset called FVRID which consists of real-world and synthetic foggy images to train and evaluate the performance. Experimental results show that the proposed method is effective and outperforms other existing vehicle ReID methods in the foggy weather. The code and dataset are available in https://github.com/Cihsaing/SJDL-Foggy-Vehicle-Re-Identification--AAAI2022.

In this paper, an ill-posed problem of multiple adverse weather removal is investigated. Our goal is to train a model with a ‘unified’ architecture and only one set of pretrained weights that can tackle multiple types of adverse weathers such as haze, snow, and rain simultaneously. To this end, a two-stage knowledge learning mechanism including knowledge collation (KC) and knowledge examination (KE) based on a multi-teacher and student architecture is proposed. At the KC, the student network aims to learn the comprehensive bad weather removal problem from multiple well-trained teacher networks where each of them is specialized in a specific bad weather removal problem. To accomplish this process, a novel collaborative knowledge transfer is proposed. At the KE, the student model is trained without the teacher networks and examined by challenging pixel loss derived by the ground truth. Moreover, to improve the performance of our training framework, a novel loss function called multi-contrastive knowledge regularization (MCR) loss is proposed. Experiments on several datasets show that our student model can achieve promising results on different bad weather removal tasks simultaneously. The code is available in our project page.

Single image reflection removal is a crucial technique which can improve the performance of object detection, semantic segmentation, and various computer vision applications. In this paper, we present a novel reflection removal algorithm using bi-channel priors (i.e., the dark channel prior and the bright channel prior). We observe that the values of dark channel pixels are not near 0, and those of bright channel pixels are not closer to 1 under the reflection scenario. We first demonstrate these phenomena statistically and mathematically. Then, we apply these properties as the constraints in optimizing the proposed reflection removal process. Extensive experiments on several well-known benchmarks demonstrate that our approach achieves desirable reflection suppression results compared with other methods.

In image processing, smoke may degrade visibility and deteriorate the performance of high-level vision applications. Therefore, single image smoke removal is crucial for computer vision. Currently, existing smoke removal algorithms mainly leverage handcrafted priors. Moreover, these methods usually apply haze removal methods to perform smoke removal due to the similarity between smoke and haze. However, these methods cannot sufficiently address the degradation of thick smoke and may suffer from residual smoke and color distortion problems due to the non-global and non-homogeneous distribution of smoke. In this paper, to solve the aforementioned problems, an end-to-end deep neural network called DesmokeNet is proposed. We construct a two-stage recovered pipeline to remove the smoke in different thicknesses. The light and thick smoke is first removed locally by the smoke removal network (SRN). The missing pixels in the thick smoke are then recovered by the pixel compensation network (PCN). Moreover, we proposed the thickness-aware pixel loss and the dark channel loss to suppress the residual smoke. To further increase the discriminative ability of the DesmokeNet, we proposed self-attentive feature consensus loss and multi-level contrastive regularization loss to improve the performance of smoke removal. Finally, to train the proposed method, we construct the first large-scale dataset containing synthetic and real-world data. Extensive experiments show that the proposed method outperforms favorably against other state-of-the-art methods quantitatively and qualitatively.

Music information retrieval is of great interest in audio signal processing. However, relatively little attention has been paid to the playing techniques of musical instruments. This work proposes an automatic system for classifying guitar playing techniques (GPTs). Automatic classification for GPTs is challenging because some playing techniques differ only slightly from others. This work presents a new framework for GPT classification: it uses a new feature extraction method based on spectral-temporal receptive fields (STRFs) to extract features from guitar sounds. This work applies a supervised deep learning approach to classify GPTs. Specifically, a new deep learning model, called the hierarchical cascade deep belief network (HCDBN), is proposed to perform automatic GPT classification. Several simulations were performed and the datasets of: 1) data on onsets of signals; 2) complete audio signals; and 3) audio signals in a real-world environment are adopted to compare the performance. The proposed system improves upon the F-score by approximately 11.47% in setup 1) and yields an F-score of 96.82% in setup 2). The results in setup 3) demonstrate that the proposed system also works well in a real-world environment. These results show that the proposed system is robust and has very high accuracy in automatic GPT classification.

Snow is a highly complicated atmospheric phenomenon that usually contains snowflake, snow streak, and veiling effect (similar to the haze or the mist). In this literature, we propose a single image desnowing algorithm to address the diversity of snow particles in shape and size. First, to better represent the complex snow shape, we apply the dual-tree wavelet transform and propose a complex wavelet loss in the network. Second, we propose a hierarchical decomposition paradigm in our network for better under-standing the different sizes of snow particles. Last, we propose a novel feature called the contradict channel (CC) for the snow scenes. We find that the regions containing the snow particles tend to have higher intensity in the CC than that in the snow-free regions. We leverage this discriminative feature to construct the contradict channel loss for improving the performance of snow removal. Moreover, due to the limitation of existing snow datasets, to simulate the snow scenarios comprehensively, we propose a large-scale dataset called Comprehensive Snow Dataset (CSD). Experimental results show that the proposed method can favorably outperform existing methods in three synthetic datasets and real-world datasets. The code and dataset are released in https://github.com/weitingchen83/ICCV2021-Single-Image-Desnowing-HDCWNet.

Single image haze removal is crucial in computer vision. In open literatures, two kinds of dehazing strategies (prior-based and learning-based methods) have been developed. However, they have a trade-off between detail preservation and the image quality. Prior-based methods reconstruct the detail well but have lower image quality while learning-based methods achieve better recovered quality but lose the detail. In this paper, to mitigate this dilemma, a hierarchical architecture using the discrete wavelet transform (DWT) is proposed. It divides the dehazing problem into two parts: detail and background reconstruction. Based on investigating how haze affects the image in the wavelet domain, two networks for detail and background reconstruction are proposed. To avoid color distortion and the detail loss, the anti-vanish wavelet loss and the bound penalty are proposed. The multi-level wavelet component discriminator is proposed for further improvement. Experiments show that the proposed network can achieve superior performance in all metrics.

A novel vehicle tracking algorithm robust to multi- viewpoint pattern and occlusion is proposed. To improve accuracy, after object detection, the overlapping parts of bounding boxes are removed before block matching. It is helpful for reducing the interference from nearby vehicles or objects. To perform block matching well, in addition to partial similarity and position correlation, many advanced features, including saliency features and several new global features, are adopted. These features can retrieve important information from vehicles and are helpful for tracking. Experiments show that the proposed algorithm achieves favorable performance against state-of-the-art vehicle tracking methods, including rule-based and learning-based methods.

Image deblurring is important to improve the quality of the image captured from a camera. To well deblur an image, it is an important issue to estimate the distributions of noise and edges, especially for the methods based on maximum likelihood estimation (MLE). In this paper, first, we propose a method for noise estimation based on the smooth prior. That is, for most of natural images, there should be some smooth regions. Then, after predicting the smooth regions and determining their gradient distributions, the noise distribution of the image can be well estimated. Then, based on the estimated noise and the additivity property of the Gaussian distributions, the local edge distributions can be determined. Then, the parameters of the MLE model can be assigned adaptively by the ratio of the standard deviations of edge and noise distributions. Experiments show that the proposed image deblurring algorithm based on the local Gaussian model and smooth prior has better performance than state-of-the-art image deblurring algorithms, including conventional and learning based ones.

The Lempel-Ziv-Welch (LZW) algorithm achieves outstanding performance and is widely used in text encoding. However, when the bit-width for each code is limited and the amount of data to be encoded is huge, the dictionary will be prematurely full. As a result, the new string cannot be added to the dictionary in the later encoding process and thus the performance is compromised. In this paper, an improved LZW algorithm is proposed to address this problem. In the proposed algorithm, the string is added to the dictionary only if the frequency of the string reaches the threshold. In this way, the dictionary is left only for the common strings. The experiments on the test patterns consisting of 300,000 characters show that the proposed algorithm further enhances the compression rates and is efficient for data compression. For example, the compression rate of the proposed algorithm is 6.0% larger than that of the LZW algorithm when each codeword contains 12 bits.

Onsets are criterion points to separate an audio signal into several notes. In this paper, we combine the advantages of conventional rule-based onset detection methods and convolutional neural network (CNN) based methods and propose an advanced onset detection algorithm. Different from rule-based methods, we apply the CNN to avoid tuning thresholds empirically. Different from existing CNN-based methods, which apply the original signal as the input directly, we construct a data with 204 feature layers and use it as the CNN input. Simulations show that the proposed algorithm has much better performance than both rule-based and existing CNN-based onset detection methods.

In this paper, a linear transform that is a further generalization of the linear canonical transform (LCT) and the fractional Fourier transform is proposed. Instead of the chirp terms adopted by the LCT, we apply exponent functions with higher order phases before and after applying the scaled Fourier transform. With the proposed higher order phase linear canonical transform (HOPLCT), one can convert the time-frequency distribution with a bent shape, which is usually the case for vocal signals, into a rectangular shape. It can much reduce the bandwidth of a signal and is very helpful for signal sampling and filter design.

Fundamental frequency determination is critical for music and radar signal analysis. In practice, the fundamental frequency is hard to be determined precisely especially when the signal-to-noise ratio (SNR) is low. In this paper, we propose an algorithm using both feature extraction and machine learning to determine fundamental frequency precisely. First, several features, including the correlation in the time-frequency domain and the differences to the previous/next local minima, are extracted. Then, a learning-based classifier is applied. The proposed algorithm can estimate the fundamental frequency accurately even when the SNR is about -9dB and the signal length is only 4 seconds.

In this work, an efficient and robust learning-based JPEG2000 architecture is proposed. It uses machine learning techniques for predicting and encoding the decision bit in the embedded block coding with optimized truncation (EBCOT) process. First, we apply non-locally weighted ridge regression to predict the quantized wavelet coefficients in the LL subband. Then, during the EBCOT process, we perform inter/intra subband prediction and inter/intra bit plane symbol prediction to estimate the activity of the decision bit using the deep learning architecture. Then, the binary prediction result is treated as an additional context and the decision bit is eventually coded using an advanced context-based adaptive binary arithmetic coder. Simulations show that the proposed framework provides the same visual quality as conventional codecs with as much as 30% bitrate savings.

Images captured in a hazy environment usually suffer from bad visibility and missing information. Over many years, learning-based and handcrafted prior-based dehazing algorithms have been rigorously developed. However, both algorithms exhibit some weaknesses in terms of haze removal performance. Therefore, in this work, we have proposed the patch-map-based hybrid learning DehazeNet, which integrates these two strategies by using a hybrid learning technique involving the patch map and a bi-attentive generative adversarial network. In this method, the reasons limiting the performance of the dark channel prior (DCP) have been analyzed. A new feature called the patch map has been defined for selecting the patch size adaptively. Using this map, the limitations of the DCP (e.g., color distortion and failure to recover images involving white scenes) can be addressed efficiently. In addition, to further enhance the performance of the method for haze removal, a patch-map-based DCP has been embedded into the network, and this module has been trained with the atmospheric light generator, patch map selection module, and refined module simultaneously. A combination of traditional and learning-based methods can efficiently improve the haze removal performance of the network. Experimental results show that the proposed method can achieve better reconstruction results compared to other state-of-the-art haze removal algorithms.

In this paper, we proposed a novel haze removal algorithm based on a new feature called the patch map. Conventional patch-based haze removal algorithms (e.g. the Dark Channel prior) usually performs dehazing with a fixed patch size. However, it may produce several problems in recovered results such as oversaturation and color distortion. Therefore, in this paper, we designed an adaptive and automatic patch size selection model called the Patch Map Selection Network (PMS-Net) to select the patch size corresponding to each pixel. This network is designed based on the convolutional neural network (CNN), which can generate the patch map from the image to image. Experimental results on both synthesized and real-world hazy images show that, with the combination of the proposed PMS-Net, the performance in haze removal is much better than that of other state-of-the-art algorithms and we can address the problems caused by the fixed patch size.

Face detection is one of the most popular topics in computer vision. There are several well-known techniques for face detection, such as the Viola-Jones detector. However, the performance of the Viola-Jones detector is limited since it mainly applies the simple Haar-based features. Many advanced methods, especially the convolutional neural network (CNN) based method, have very good performance in face detection. However, they require huge amount of training data. Moreover, most of existing algorithms are not robust to rotation, head-up, and head-down cases. In this paper, we find that, with some modifications, the Viola-Jones detector can also have very good performance in face detection. In addition to the Haar features, we also apply the prominent features and the color information. With the contour information, the edge-aware filter, the background smoother, the fuzzy classifier, and the relative locations, the prominent features, such as eyes, mouths, noses, and ears, can be extracted accurately. With these features, the accuracy of face detection can be much improved. Simulations show that, even if huge amount of training data is not applied, the proposed algorithm has better performance than state-of-the-art face detection methods, including the CNN-based method.

Context-based adaptive arithmetic coding (CAAC) has high coding efficiency and is adopted by the majority of advanced compression algorithms. In this paper, five new techniques are proposed to further improve the performance of CAAC. They make the frequency table (the table used to estimate the probability distribution of data according to the past input) of CAAC converge to the true probability distribution rapidly and hence improve the coding efficiency. Instead of varying only one entry of the frequency table, the proposed range-adjusting scheme adjusts the entries near to the current input value together. With the proposed mutual-learning scheme, the frequency tables of the contexts highly correlated to the current context are also adjusted. The proposed increasingly adjusting step scheme applies a greater adjusting step for recent data. The proposed adaptive initialization scheme uses a proper model to initialize the frequency table. Moreover, a local frequency table is generated according to local information. We perform several simulations on edge-directed prediction-based lossless image compression, coefficient encoding in JPEG, bit plane coding in JPEG 2000, and motion vector residue coding in video compression. All simulations confirm that the proposed techniques can reduce the bit rate and are beneficial for data compression.

This paper presents an image representation approach which is based on matrix factorization in the complex domain and called exemplar-embed complex matrix factorization (EE-CMF). The proposed EE-CMF approach can very effectively improve the performance of facial expression recognition. Moreover, Wirtinger's calculus was employed to determine derivatives. The gradient descent method was utilized to solve the complex optimization problem. Experiments on facial expression recognition verified the effectiveness of the proposed EE-CMF. It provides consistently better recognition results than standard NMFs.

This paper proposes a dynamic tracking attention model (DTAM), which mainly comprises a motion attention mechanism, a convolutional neural network (CNN) and long short-term memory (LSTM), to recognize human action in a video sequence. In the motion attention mechanism, the local dynamic tracking is used to track moving objects in feature domain and global dynamic tracking corrects the motion in the spectral domain. The CNN is utilized to perform feature extraction, while the LSTM is applied to handle sequential information about actions that is extracted from videos. It effectively fetches information between consecutive frames in a video sequence and has an even higher recognition rate than does the CNN-LSTM. Combining the DTAM with the visual attention model, the proposed algorithm has a recognition rate that is 3.6% and 4.5% higher than that of the CNN-LSTMs with and without the visual attention model, respectively.

Much research has been conducted in computer vision about feature extraction. In recent years, binary descriptors have been proved to be extremely fast and yet highly discriminative. So-BRIEF aims at bringing the benefits of this kind of local descriptors to the recognition of rectangular objects, such as books, CD covers, boxes, paints, boards and cell phones. It takes advantage of the special geometry of these objects to recognize very efficiently an image, even rotated or distorted. Our main contribution is this new descriptor along with the search for the optimal values of parameters leading to an extremely fast image matching process. We compare it to 2D-DCT based description techniques. This paper also encompasses a discussion about a new method for efficient detection of rectangular structures.

Face detection is very important for video surveillance, human-computer interaction, and face recognition. In this paper, a very robust face detection algorithm that can well detect rotated, in-plane, and out-of-plane faces without large amount of training data is proposed. First, several techniques, including the entropy rate superpixel (ERS) and the skin filter, are applied to obtain face candidate regions. Then, angle compensation and non-maximum suppression are applied to improve the accuracy of face detection. Moreover, to find out-of-plane faces, one can apply the flipping-and-blending technique, i.e., blending the face candidate with its flipping version to create a face that is similar to the frontal one. With it, even if there are no training data for out-of-plane faces, one can successfully detect the faces in the out-of-plane case. Simulations on the FEI dataset and the BaoFace dataset show that the proposed algorithm is efficient and outperforms state-of-the-art face detection approaches.

Robust sparse representation has been applied to tackle some challenging problems in face recognition. In this paper, we propose a new method called occlusion pattern based sparse representation classification (OPSRC). First we find the contiguous occlusion area in the query image to create an occlusion pattern. Then, we add the occlusion pattern to all face images in the face image dictionary, resulting in an occlusion dictionary. The original dictionary and occlusion dictionary are solved together. By assuming some frequently occluded parts of the face and using additive form of half-quadratic optimization that has good performance of error detection, we create occlusion patterns and construct a dictionary robust to occlusion, especially in the case of per sample of each person. Our proposed method OPSRC outperforms other benchmark methods, revealing its potential on surveillance and security application.

This paper proposes a single-image blur kernel estimation algorithm that utilizes the normalized color-line prior to restore sharp edges without altering edge structures or enhancing noise. The proposed prior is derived from the color-line model, which has been successfully applied to non-blind deconvolution and many computer vision problems. In this paper, we show that the original color-line prior is not effective for blur kernel estimation and propose a normalized color-line prior which can better enhance edge contrasts. By optimizing the proposed prior, our method gradually enhances the sharpness of the intermediate patches without using heuristic filters or external patch priors. The intermediate patches can then guide the estimation of the blur kernel. A comprehensive evaluation on a large image deblurring dataset shows that our algorithm achieves the state-of-the-art results.

In this paper, a very efficient image denoising scheme, which is called nonlocal means based on bidirectional principal component analysis, is proposed. Unlike conventional principal component analysis (PCA) based methods, which stretch a 2D matrix into a 1D vector and ignores the relations between different rows or columns, we adopt the technique of bidirectional PCA (BDPCA), which preserves the spatial structure and extract features by reducing the dimensionality in both column and row directions. Moreover, we also adopt the coarse-to-fine procedure without performing nonlocal means iteratively. Simulations demonstrated that, with the proposed scheme, the denoised image can well preserve the edges and texture of the original image and the peak signal-to-noise-ratio is higher than that of other methods in almost all the cases.

Freezing of gait (FOG) is a symptom of Parkinson's disease (PD). In this paper, we develop a real-time FOG detection algorithm. To achieve the goal of real time, we adopt the generalized spectrogram with asymmetric windows and the asymmetric smooth filter. Furthermore, instead of using only the frequency domain information, we apply the features in both the time domain and the frequency domain. Simulations show that the proposed FOG detection algorithm has higher accuracy than other methods. Moreover, the time delay of the proposed algorithm is only 1 second, which is 2.25-3.75 times less than that of existing methods.

Due to a variety of accidents, banknotes may be broken into several fragments. These fragments are usually stained, burned, partially lost, and twisted, which makes banknote reconstruction a hard problem. Since the fragments are always not intact, the traditional edge and texture based fragment assembling methods cannot be applied here. In this paper, we develop a framework for banknote reconstruction. We applied the techniques of SIFT point matching, RANSAC, and feature-based alignment. Moreover, convex quadratic optimization based on maximizing the reconstructed area and avoiding overlapping is adopted. Several simulations are given to demonstrate the effectiveness of our framework.

In this paper, a new adaptive scanning scheme, which is called local prediction based adaptive scanning (LPBAS), is proposed for discrete cosine transform (DCT) based image compression techniques including JPEG and H.264/AVC intra coding. The conventional zigzag scan order is widely used in image and video coding standards, but it ignores the statistical properties of DCT blocks and has limited performance. In this paper, the LPBAS scheme is proposed to achieve the entropy coding gain, where the scan order patterns are adaptively generated and updated based on the statistics of local neighboring DCT blocks. The proposed scheme improves the efficiency of the two image coding systems, JPEG and the H.264/AVC intra coding system. Simulation results showed that the proposed scheme indeed outperforms the zigzag scanning method and other existing adaptive scanning methods.

In this paper, a new algorithm which adopts the techniques of double prediction and the Pareto probability model was applied to encode the DC term in the JPEG compression process. Conventionally, the DC term was encoded by differential coding, i.e., the difference of the DC values between the current block and the previous block. In this paper, we first use the DC terms of four adjacent blocks to predict the current DC value. We then further use the prediction error of the four adjacent blocks to estimate the variance of the prediction error of the current block. We call it the double prediction algorithm. Next, the Pareto distribution is applied to model the probability distribution of the prediction error. Simulation results show that, with the proposed algorithms, the data size required for DC terms is significantly reduced by 25% ~ 60% and a much higher compression rate can be achieved.

In recent years, the SSIM was proposed for image and vocal signal assessments to match human perception. The existing SSIMs for vocal signals are similar to those for images. However, the human perceptions for voices and images are different. If two vocal signals differ only by phase, delay, or logistic frequency shift, they are heard similarly. In this paper, we propose the non-uniform sampling frequency mean SSIM (NUS-FMSSIM) to highly match the human perception for voices. Simulations show that it is more robust to phase change, time shift, and logistic frequency shift than the existing SSIMs for vocal signals.

Properly designed context models can increase the compression gain. In this paper, we propose a new lossless image coding scheme with two proposed algorithms: nonlocal context modeling and adaptive prediction (NCMAP). Since structural self-similarity often exists in natural images, we use the probability to measure the similarity between the powers of prediction errors for the pixels to be coded. Furthermore, the spatial distance and the intensity range are also considered for context generation. Moreover, a prediction scheme that adaptively combines the weighted edge-directed prediction (WEDP) and the nonlocal predictor (NLP) is also proposed. With the proposed context generating and prediction strategies, better compression performances can be achieved. Simulations show that the proposed scheme outperforms existing methods for lossless image compression.

Image interpolation is important for computer vision. Most of the existing image interpolation methods are based on the optimization in the mean square error (MSE) sense. In this paper, we incorporate the structural similarity (SSIM) based metric into the framework of the nonlocal edge-directed image interpolation (NLEDI) method. In the proposed algorithm, a missing pixel is interpolated using the weighted average of neighboring patches where the weights are determined by the SSIM-based metric instead of the MSE measurement. Simulations show that our proposed structural similarity-based NLEDI (SSNLEDI) scheme outperforms existing image interpolation methods and has higher PSNR values and better visual qualities.

This paper proposes joint-probability-based adaptive Golomb coding (JPBAGC) to improve the performances of the Golomb family of codes, including Golomb coding (GC), Golomb-Rice coding (GRC), exp-Golomb coding (EGC), and hybrid Golomb coding (HGC), for image compression. The Golomb family of codes is ideally suited to the processing of data with geometric distribution. Since it does not require a coding table, it has higher coding efficiency than Huffman coding. In this paper, we find that there are many situations in which the probability distribution of data is not only geometric, but also depends on the probability distribution of the other data. Accordingly, we used the joint probability of generalizing the Golomb family of codes and exploiting the dependence between neighboring image data. The proposed JPBAGC improves the efficiency of many image and video compression standards, such as the joint photographic experts group (JPEG) compression scheme and the H.264-intra JPEG-based image coding system. Simulation results demonstrate the superior coding efficiency of the proposed scheme over those of Huffman coding, GC, GRC, EGC, and HGC.

In the conventional JPEG algorithm, an image is divided into eight by eight blocks and then the 2-D DCT is applied to encode each block. In this paper, we find that, in addition to rectangular blocks, the 2-D DCT is also orthogonal in the trapezoid and triangular blocks. Therefore, instead of eight by eight blocks, we can generalize the JPEG algorithm and divide an image into trapezoid and triangular blocks according to the shapes of objects and achieve higher compression ratio. Compared with the existing shape adaptive compression algorithms, as we do not try to match the shape of each object exactly, the number of bytes used for encoding the edges can be less and the error caused from the high frequency component at the boundary can be avoided. The simulations show that, when the bit rate is fixed, our proposed algorithm can achieve higher PSNR than the JPEG algorithm and other shape adaptive algorithms. Furthermore, in addition to the 2-D DCT, we can also use our proposed method to generate the 2-D complete and orthogonal sine basis, Hartley basis, Walsh basis, and discrete polynomial basis in a trapezoid or a triangular block.

Salient region detection is useful for several image-processing applications, such as adaptive compression, object recognition, image retrieval, filter design, and image retargeting. A novel method to determine the salient regions of images is proposed in this paper. The L₀ smoothing filter and principle component analysis (PCA) play important roles in our framework. The L₀ filter is extremely helpful in characterizing fundamental image constituents, i.e., salient edges, and can simultaneously diminish insignificant details, thus producing more accurate boundary information for background merging and boundary scoring. PCA can reduce computational complexity as well as attenuate noise and translation errors. A local-global contrast is then used to calculate the distinction. Finally, image segmentation is used to achieve full-resolution saliency maps. The proposed method is compared with other state-of-the-art saliency detection methods and shown to yield higher precision-recall rates and F-measures.

In this paper, a new age estimation framework considering the intrinsic properties of human ages is proposed, which improves the dimensionality reduction techniques to learn the connections between facial features and aging labels. To enhance the performance of dimensionality reduction, a distance metric adjustment step is introduced in advance to achieve a suitable metric in the feature space. In addition, to further exploit the ordinal relationship of human ages, the “label-sensitive” concept is proposed, which regards the label similarity during the learning phase of distance metric and dimensionality reduction. Finally, an age-specific local regression algorithm is proposed to capture the complicated aging process for age determination. From the simulation results, the proposed framework achieves the lowest mean absolute error against the existing methods.

A method of natural image classification by an effective color quadtree segmentation together with a more effective codebook with the color local thresholding classifier for content-based image retrieval (CBIR) is proposed. The vector quantization (VQ) based image retrieval schemes have good performance, but the importance of color edge intensive blocks is neglected. Our proposed method has two main improvements. First, quadtree segmentation based on both hue and gray-level information is applied to classify the blocks into the homogeneous and high-detail ones. Second, a color local thresholding classifier is proposed to further classify the high-detail blocks based on edge information. Simulation results show that our proposed scheme outperforms the existing methods, including the Quadtree CVQ-based scheme, the VQ-based scheme, and other methods.

All of the existing TV-point discrete fractional signal transforms require O(N2) computation complexity. In this paper, we propose a new discrete fractional signal transform whose computation complexity can be reduced to O(N1.5). This new transform is a fractional version of a DFT-based signal transform called as the Hirschman optimal transform (HOT) in the literature. Eigenvalues and eigenvectors properties of the HOT are also developed. Moreover, the proposed discrete fractional HOT transform is extended to further reduce the required computation complexity to linear order O(N). As an application example, we apply this new computationally efficient discrete fractional signal transform to encrypt digital images.

With the fast development of multimedia, it is crucial to find the way to search image database effectively. The vector quantization (VQ) based image retrieval method is popular in recent years. In this paper, we propose the quadtree classified vector quantization (QCVQ) scheme to improve the VQ method by exploiting the visual importance of image blocks and using the edge information to describe the content of each block efficiently. Moreover, we also apply the adaptive block size. The simulation results show that, compared with the previous image retrieval algorithms using VQ and chromaticity moments (CM), our proposed scheme has obviously better average retrieval rate and higher average precision.

In this paper, a new video retargeting method based on temporal information and seam carving is presented. Two video energy functions, motion weight prediction and pixel-based optimization, are proposed to take the temporal information into account and make dynamic programming available during the process of retargeting. The motion weight prediction exploits both the block-based motion estimation and Gaussian masks to predict the coarse location of seams in the current frame and reduce the search range of dynamic programming. The pixel-based optimization then utilizes the concept of pixel-based optical flow to explore better temporal relations between the current frame and previous frames in the reduced search range. The experimental results show that combining these two video energy functions as well as dynamic programming, the proposed method could achieve content-aware and temporal smoothing retargeting results with less computational complexity.

As the Walsh (Hadamard) transform can be generalized into the Jacket transform, in this paper, we generalize the Haar transform into the Jacket-Haar transform. The entries of the Jacket-Haar transform are 0 and ±2k. Compared with the original Haar transform, the basis of the Jacket-Haar transform is general and more suitable for signal processing. Furthermore, with the proposed generalization algorithm, it is possible to define the N-point Jacket-Haar transform, where N is not a power of 2. From our simulations, the proposed Jacket Haar transform has better performance in ECG signal analysis.

Coefficient scanning plays an important role in block-based image and video coding standards, such as JPEG, MPEG, and the latest H.264/AVC. In these coding standards, the zigzag scanning method is used for image / frame coding and the field scanning method is used for field coding. Although these scanning methods can achieve acceptable coding efficiency, they do not take the statistical properties of the quantized coefficients of each block into consideration. Therefore, coding redundancy still exists. In this paper, we propose a novel adaptive zigzag scanning scheme, which is called the context-based adaptive coefficient scanning method, and apply it in the lossy JPEG baseline algorithm. Simulation results show that our proposed method achieves better coding efficiency than both the conventional zigzag scanning method and the two-stage zigzag scanning (TSZS) method adopted in JPEG.

Clinical examination of Congenital Muscular Torticollis (CMT) is often carried out by ultrasound equipments. However, a variety of subjective factors during diagnosis may result in wrong decision. Thus, we propose an image processing algorithm to derive the objective judgment on the healthiness of muscle in this paper. We first apply image segmentation technique, such as the fast scanning algorithm, for ultrasonic muscle image segmentation. Then, the proposed algorithms are applied to determine the healthiness of muscle fibers. We furthermore propose a score criterion to evaluate the degree of injury. The experimental results show that the injury score measured by the proposed methods can successfully determine whether the muscle is hurt and infer the extent of fibrosis.

The Fourier descriptor is an efficient and effective way to describe a closed boundary. However, for a non-closed segment, since the two non-adjacent end points result in signal discontinuity, after eliminating the high-frequency part, the reconstructed segment has large error near the two ends. In this paper, we propose a warping method to connect the two ends and perform odd-symmetric extension to smooth the warped segment around them. With these modifications, the high-frequency components near the two ends can be much reduced and we can obtain the reconstructed segment with accurate end-point locations even when only the low frequency coefficients are preserved. This method could also be used for a closed boundary with a pre-segmentation process, and the experimental result shows that with the same boundary compression rate, our method has better reconstruction quality than directly extracting Fourier descriptors on the closed boundary.

It is known that the 2-D DCT basis is complete and orthogonal in a rectangular region. In this paper, we introduce the way to generate the complete and orthogonal 2-D DCT basis in a trapezoid region or a triangular region without using the complicated Gram-Schmidt method. Moreover, since a polygon can be decomposed several triangular regions, the proposed method is also suitable for the polygonal region. Our algorithm can much generalize the JPEG algorithm. Instead of dividing an image into 8 by 8 blocks, we can divide an image into trapezoid or triangular regions and then transform and code each of them. In addition to the DCT basis, our method can also be used for generating the 2-D complete and orthogonal DFT basis, KLT basis, Legendre basis, Hadamard (Walsh) basis, and polynomial basis in the trapezoid and triangular regions.

The integer color transform is a reversible operation that can transform one color coordinate into another one and both the inputs and the outputs are of integer forms. In this paper, we improve the integer color transforms derived in previous works. First, we relax the constraint that the scaling for each row should be the same. From this, the method of deriving the integer color transform becomes more flexible and we can derive the integer color transform with less implementation time and higher accuracy. Moreover, we use the new criterion, bit extension, to measure the performance of the integer color transform and propose a new way for accuracy analysis. With the proposed method, we derive the reversible integer RGB-to-YCbCr, KLA, XYZ, UVW, RcGcBc, and YUV transforms with even higher accuracy successfully.

Different images have different amplitude spectrums. However, in this paper, with the proposed phase-key algorithm, we can make a series of images have the same amplitude spectrum but different phase key in the frequency domain. It is useful for image encryption, data compression, image transformation, and reducing the distortion in communication. Our algorithm can be applied for not only the FT but also all the operations with conjugate symmetric kernels. Moreover, we can also reverse the direction of the algorithm, i.e., make a series of images have the same phase spectrum and use the amplitude spectra to distinguish different images.

Recently, Candan introduced higher order DFT-commuting matrices whose eigenvectors are accurate approximations to the continuous Hermite-Gaussian functions (HGFs). However, the highest order 2k of the O(h2k) N×N DFT-commuting matrices proposed by Candan is restricted by 2k+1≤N. In this paper, we remove that restriction of order upper bound by developing a coefficient truncation technique to construct arbitrary-order DFT-commuting matrices. Exploiting that coefficient truncation technique, we also develop a method to construct n-diagonal arbitrary-order DFT-commuting matrices, whose number of nonzero diagonal bands n can be prespecified at will. Results of computer experiments show that the Hermite-Gaussian-like (HGL) eigenvectors of the new DFT-commuting matrices proposed in this paper outperform those of Candan’s.

A more accurate algorithm for corner and edge detections that is the improved form of the well-known Harris' algorithm is introduced. First, instead of approximating |L[m+x, n+y]-L[m, n]|2just in terms of x2, xy, and y2, we will approximate |L[m+x, n+y]-L[m, n]|(L[m+x, n+y]-L[m, n]) by the linear combination of x2, xy, y2, x, y, and 1. With the modifications, we can observe the sign of variation. It can avoid misjudging the pixel at a dot or on a ridge as a corner and is also helpful for increasing the robustness to noise. Moreover, we also use orthogonal polynomial expansion and table looking up and define the comity as the "integration" of the quadratic function to further improve the performance. From simulations, our algorithm can much reduce the probability of regarding a non-corner pixel as a corner. In addition, our algorithm is also effective for edge detection.

In this paper, we generalize the lifting scheme and the triangular matrix scheme. For the existing lifting scheme and the triangular matrix scheme, the entries on the diagonal line must be 1 or 2k. In this paper, we find that this constraint can be relaxed and the lifting or the triangular matrix is still reversible. Thus, the constraint that det(A) = 2Lis not required and we can convert a matrix into a reversible integer transform without pre-scaling even when det(A) ne 2L. Moreover, the proposed scaled schemes are also helpful for improving the accuracy and reducing the implementation complexity.

In this correspondence, we introduce a systematic algorithm that can convert any 3 x 3 color transform into a reversible integer-to-integer transform. We also discuss the ways to improve accuracy and reduce implementation complexity. We derive the integer RGB-to-KLA, IV1 V2, YCbCr, DCT, YUV, and YIQ transforms that are optimal in accuracy.

In this paper, we discuss the relations among the random process, the Wigner distribution function, the ambiguity function, and the fractional Fourier transform (FRFT). We find many interesting properties. For example, if we do the FRFT for a stationary process, although the result in no longer stationary, the amplitude of its covariance function is still independent of time. Moreover, for the FRFT of a stationary random process, the ambiguity function will be a radiant line passing through (0, 0) and the Wigner distribution function will be invariant along a certain direction. We also define the fractional stationary random process and find that a non-stationary random process can be expressed a summation of fractional stationary random processes. The proposed theorems will be useful for filter design, noise synthesis and analysis, system modeling, and communication.

This paper presents a modified approach to the successive mean quantization transform, which is called as the weighted histogram separation (WHS) for enhancement of color images. Property of WHS situates between the successive mean quantization transform and the histogram equalization. In addition, this approach is further applied to the local enhancement, which is similar to the adaptive histogram equalization, and it is termed as the adaptive weighted histogram separation (AWHS). A comparison with successive mean quantization transform and histogram equalization has been performed in the experiments.

A dual-domain bi-watermarking algorithm is proposed in this paper. This algorithm embeds bi-watermark in DCT domain, but bi-watermark can be extracted in both spatial and DCT domains. Therefore, it can implement on the DCT-based compressed image/frame. The dual-domain bi-watermarking algorithm is the extension of the spatial-domain bi-watermarking algorithm, which implements quantization index modulation with two quantization step sizes. These quantization step sizes construct the non-uniform quantization intervals. Additionally, the luminance quantization table of JPEG compression is considered in the algorithm. In the experimental result, two extracted watermarks show the capability for various compression rates, and the watermarks also reveal the different robustness against the global and the regional attacks.

Integer transform are the discrete transforms whose entries are summations of 2/sup -k/. If for an integer transform, we can perfectly recover the input from the output, we call it the reversible integer transform. In 2001, Hao and Shi developed an algorithm that can convert any reversible non-integer transform into a reversible integer transform. In this paper, we improve their works. First, we simplify the way of derivation. Then, we analyze the approximation error and introduce the way to reduce it. We also discuss the problem of bit constraint and how to reduce the number of time cycle in implementation.

It is known that one can use Shannon's theory to sample a bandlimited signal. In this paper, we introduce how to use prolate spheroidal wave functions (PSWFs) to sample a time-limited and nearly band-limited signal. PSWFs have the property of optimal energy concentration. Thus we can apply it to sampling theory to reduce the aliasing error of the recovered signal. We derive a theory that can estimate the upper bound of the error. With it, we can determine, to achieve certain accuracy, how many samples we should acquire. Moreover, we combine the proposed sampling theory with the fractional Fourier transform (FRFT). We also find an important theory, i.e., to achieve a certain degree of accuracy, the number of sampling points required for a signal is proportional to the 'area' of its time-frequency distribution.

Based on discrete Hermite-Gaussian like functions, a discrete fractional Fourier transform (DFRFT) which provides sample approximations of the continuous fractional Fourier transform was defined and investigated recently. In this paper, we propose a new nearly tridiagonal matrix which commutes with the discrete Fourier transform (DFT) matrix. The eigenvectors of the new nearly tridiagonal matrix are shown to be better discrete Hermite-Gaussian like functions than those developed before. Furthermore, by appropriately combining two linearly independent matrices which both commute with the DFT matrix, we develop a method to obtain even better discrete Hermite-Gaussian like functions. Then, new versions of DFRFT produce their transform outputs more close to the samples of the continuous fractional Fourier transform, and their application is illustrated.

In this paper, we introduce a new algorithm for corner detection. Instead of calculating the gradients along x and y-axes, which is the common step of many existed algorithms, we use the sum of differences to observe the variations along the adaptive vertical and tangent axes. We classify the variations into 36 types and use the 'case table' to determine whether a pixel is a corner. We do some experiments and show that our algorithm can detect almost all the corners of a complicated natural image (such as Lena image and Fruit image) successfully. In addition to corner detection, it is also possible to use our algorithm to detect the edges, ridges, valleys, isolated dots, saddles, and plain regions of a natural image.

In color image processing, the RGB color coordinate is usually transformed into another one (e.g., YIQ or KLA) for system fitting or other purposes. Most of the color transforms are done by 3/spl times/3 matrices. However, these matrices are always not fixed-point. In this paper, we use a systematic algorithm to convert every 3/spl times/3 color transform into a reversible integer-to-integer transform. The resulted transform can be implemented with only fixed-point processor and no floating-point processor is required. Moreover, with the use of ladder-truncation technique, we can make least bit of the output the same as that of the input, and the long bit-length problem that always occurs for other integer transforms can be avoided. We derive the integer color transforms of RGB-to-KLA, IV/sub 1/V/sub 2/, YCrCb, DCT, and YIQ successfully.

It is well-known that the Hilbert transform (HLT) is useful for generating analytic signals, and saving the bandwidth required, in communication. However, it is less known that the HLT is also a useful tool for edge detection. We introduce the generalized radiant Hilbert transform (GRHLT), and illustrate how to use it for edge detection. The GRHLT is the general form of the two-dimensional HLT. Together with some other techniques (such as section dividing and shorter impulse response modification), we can use the GRHLT to detect the edges of images exactly. The GRHLT used for edge detection has a higher capability for noise immunity than other edge detection algorithms. Besides, we can also use the GRHLT for directional edge detection, i.e., detecting edges with certain directions.

In this paper, we first discuss the singular value decomposition (SVD) of a quaternion matrix and propose an algorithm to calculate the SVD of a quaternion matrix using its equivalent complex matrix. The singular values of a quaternion matrix are still real and positive, but the two unitary matrices are quaternion matrices with quaternion entries. Then, applications for color image processing by the SVD of a quaternion matrix are given. Since a quaternion matrix can represent a color image, so we can use the SVD of a quaternion matrix to decompose a color image. Therefore, many useful image processing methods by SVD, such as eigen-images, image compression, image enhancement and denoise, can be extended to color image processing without separating the color image into three channel images.

The Fourier transform can be generalized into the fractional Fourier transform (FRFT), linear canonical transform (LCT), and simplified fractional Fourier transform (SFRFT). They extend the utilities of original Fourier transform, and can solve many problems that can not be solved well by original Fourier transform. We generalize the cosine transform. We derive the fractional cosine transform (FRCT), canonical cosine transform (CCT), and simplified fractional cosine transform (SFRCT). We show that they are very similar to the FRFT, LCT, and SFRFT, but they are much more efficient for dealing with the even, real even functions. For digital implementation, FRCT and CCT can save 1/2 of the real number multiplications, and SFRCT can save 3/4. We also discuss their applications, such as optical system analysis and space-variant pattern recognition.

It is popular to use the conventional correlation for pattern recognition. But when using the conventional correlation, the pattern should be the gray-level pattern. In this paper, we discuss how to use discrete quaternion correlation (DQCR) for the application of color pattern recognition. With the algorithm introduced here, we can detect the objects that have the same shape, color, and brightness as the reference pattern. Besides, we can also detect (a) the objects with the same shape, color, but different brightness, (b) the objects with the same shape, brightness, but different color, and (c) the objects just have the same shape as the reference. Our algorithm can classify the objects into 5 classes due to whether their shape, brightness, and color match those of the reference pattern. Besides, with our algorithm, the difference of the brightness and color can also be calculated at the same time.

The affine Fourier transform (AFT) also called as the canonical transform. It generalizes the fractional Fourier transform (FRFT), Fresnel transform, scaling operation, etc., and is a very useful tool for signal processing. We derive the eigenfunctions of the AFT. The eigenfunctions seems hard to be derived, but since the AFT can be represented by the time-frequency matrix (TF matrix), we can use the matrix operations to derive its eigenfunctions. Then, because many optical systems can be represented as a special case of the AFT, the eigenfunctions of the AFT are just the light distributions that will cause the self-imaging phenomena for some optical systems. We use the eigenfunctions we derive to discuss the self-imaging phenomena.

DFT has good quality of performance and fast algorithms. But when we implement the DFT, we require the floating-point multiplication. In this paper, we introduce the integer Fourier transform (ITFT). ITFT is approximated to the DFT, but all the entries in the transform matrix are integer numbers. So it only requires fixed-point multiplication, and the implementation can be much simplified, especially for VLSI. This new transform will work similarly to the original DFT, for example, the transform results are similar and the shifting-invariant property is also preserved for ITFT. We also introduce the general method to derive the integer transform. By this approach, we can derive many types of integer transforms (such as integer cosine, sine, and Hartley transforms).

The 2-D Fourier transform has been generalized into the 2-D separable fractional Fourier transform (replaces the 1-D Fourier transform by the l-D fractional Fourier transform for each variable) and the 2-D separable canonical transform (further replaces the fractional Fourier transform by canonical transform) of Sahin, Ozaktas and Mendlovic (see Appl. Opt., vol.37, no.11, p.2130-41, 1998). It also has been generalized into the 2-D unseparable fractional Fourier transform with 4 parameters of Sahin et al. (see Appl. Opt., vol.37, no.23, p.5444-53, 1998). In this paper, we introduce the 2-D affine generalized fractional Fourier transform (AGFFT). These 2-D transforms has been further generalized. We show it can deal with many problems that can not be dealt with by these 2-D transforms and extend their utility.

This paper is concerned with the definition of the continuous fractional Hartley transform. First, a general theory of the linear fractional transform is presented to provide a systematic procedure to define the fractional version of any well-known linear transforms. Then, the results of general theory are used to derive the definitions of the fractional Fourier transform (FRFT) and fractional Hartley transform (FRHT) which satisfy the boundary conditions and additive property simultaneously. Next, an important relationship between FRFT and FRHT is described. Finally, a numerical example is illustrated to demonstrate the transform results of the delta function of FRHT.