We propose PoseGaussian, a pose-guided Gaussian Splatting framework for high-fidelity human novel view synthesis. Human body pose serves a dual purpose in our design: as a structural prior, it is fused with a color encoder to refine depth estimation; as a temporal cue, it is processed by a dedicated pose encoder to enhance temporal consistency across frames. These components are integrated into a fully differentiable, end-to-end trainable pipeline. Unlike prior works that use pose only as a condition or for warping, PoseGaussian embeds pose signals into both geometric and temporal stages to improve robustness and generalization. It is specifically designed to address challenges inherent in dynamic human scenes, such as articulated motion and severe self-occlusion. Notably, our framework achieves real-time rendering at 100 FPS, maintaining the efficiency of standard Gaussian Splatting pipelines. We validate our approach on ZJU-MoCap, THuman2.0, and in-house datasets, demonstrating state-of-the-art performance in perceptual quality and structural accuracy (PSNR 30.86, SSIM 0.979, LPIPS 0.028).

Extrapolation plays a critical role in machine/deep learning (ML/DL), enabling models to predict data points beyond their training constraints, particularly useful in scenarios deviating significantly from training conditions. This article addresses the limitations of current convolutional neural networks (CNNs) in extrapolation tasks within image restoration and compressed sensing (CS). While CNNs show potential in tasks such as image outpainting and CS, traditional convolutions are limited by their reliance on interpolation, failing to fully capture the dependencies needed for predicting values outside the known data. This work proposes an extrapolation convolution (EC) framework that models missing data prediction as an extrapolation problem using linear prediction within DL architectures. The approach is applied in two domains: first, image outpainting, where EC in encoder-decoder (EnDec) networks replaces conventional interpolation methods to reduce artifacts and enhance fine detail representation; second, Fourier-based CS-magnetic resonance imaging (CS-MRI), where it predicts high-frequency signal values from undersampled measurements in the frequency domain, improving reconstruction quality and preserving subtle structural details at high acceleration factors. Comparative experiments demonstrate that the proposed EC-DecNet and FDRN outperform traditional CNN-based models, achieving high-quality image reconstruction with finer details, as shown by improved peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and kernel inception distance (KID)/Frechet inception distance (FID) scores. Ablation studies and analysis highlight the effectiveness of larger kernel sizes and multilevel semi-supervised learning in FDRN for enhancing extrapolation accuracy in the frequency domain.

We present a mcthod to lcarn a diverse group of object categories from an unordcrcd point set. We propose our Pyramid Point network, which uses a dense pyramid structure instead of thc traditional ’U’ shape, typically seen in semantic segmentation networks. This pyramid structure gives a second look, allowing thc network to revisit different layers creating from various leveis on the network, allowing for feature propagation. We introduce a Focused Kemel Point convolution (FKP Conv), which expands on the traditional point convolutions by adding an attention mcchanism to the kemel outputs. This FKP Conv increases our feature quality and allows us to weigh thc kemel outputs dynamically. These FKP Convs are the central part of our Recurrent FKP Bottlcneck block, which makes up thc backbonc of our encoder. With this distinct network, we demonstrate competitive performance on threc benchmark data sets.

Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Unfortunately, most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. Many state-of-the-art algorithms for event cameras rely on interpolated event representations-obscuring crucial timing information, increasing the data volume, and limiting overall network performance. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss. This bio-inspired design is memory efficient, computationally fast, avoids time-blocking (i.e., fixed and predefined frame rates), and contains "local memory" from past data. The design is evaluated on a wide range of challenging tasks (e.g., event denoising, image reconstruction, classification, and human pose estimation) and is shown to dramatically improve state-of-the-art performance. TORE volumes are an easy-to-implement replacement for any algorithm currently utilizing event representations.

This paper presents a novel method for labeling real-world neuromorphic camera sensor data by calculating the likelihood of generating an event at each pixel within a short time window, which we refer to as “event probability mask” or EPM. Its applications include (i) objective benchmarking of event denoising performance, (ii) training convolutional neural networks for noise removal called “event denoising convolutional neural network” (EDnCNN), and (iii) estimating internal neuromorphic camera parameters. We provide the first dataset (DVSNOISE20) of real-world labeled neuromorphic camera events for noise removal.

We propose a novel attention-based framework for 3D human pose estimation from a monocular video. Despite the general success of end-to-end deep learning paradigms, our approach is based on two key observations: (1) temporal incoherence and jitter are often yielded from a single frame prediction; (2) error rate can be remarkably reduced by increasing the receptive field in a video. Therefore, we design an attentional mechanism to adaptively identify significant frames and tensor outputs from each deep neural net layer, leading to a more optimal estimation. To achieve large temporal receptive fields, multi-scale dilated convolutions are employed to model long-range dependencies among frames. The architecture is straightforward to implement and can be flexibly adopted for real-time applications. Any off-the-shelf 2D pose estimation system, e.g. Mocap libraries, can be easily integrated in an ad-hoc fashion. We both quantitatively and qualitatively evaluate our method on various standard benchmark datasets (e.g. Human3.6M, HumanEva). Our method considerably outperforms all the state-of-the-art algorithms up to 8% error reduction (average mean per joint position error: 34.7) as compared to the best-reported results. Code is available at: (https://github.com/lrxjason/Attention3DHumanPose)

Given that there are numerous amounts of unlabeled data available for usage in training neural networks, it is desirable to implement a neural network architecture and training paradigm to maximize the ability of the latent space representation. Through multiple perspectives of the latent space using adversarial learning and autoencoding, data requirements can be reduced, which improves learning ability across domains. The entire goal of the proposed work is not to train exhaustively, but to train with multiperspectivity. We propose a new neural network architecture called Active Recall Network (ARN) for learning with less labels by optimizing the latent space. This neural network architecture learns latent space features of unlabeled data by using a fusion framework of an autoencoder and a generative adversarial network. Variations in the latent space representations will be captured and modeled by generation, discrimination, and reconstruction strategies in the network using both unlabeled and labeled data. Performance evaluations conducted on the proposed ARN architectures with two popular datasets demonstrated promising results in terms of generative capabilities and latent space effectiveness. Through the multiple perspectives that are embedded in ARN, we envision that this architecture will be incredibly versatile in every application that requires learning with less labels.

High-value-target tracking in full-motion-video is a difficult surveillance task due to model drift while online training. We propose a tracker that adaptively fuses detections from multiple target models trained using three memory types to overcome model drift and other challenges. The short-term memory uses correlation and histogram features to detect the target from its recent appearance. The working memory uses a deep extreme learning network with polynomial connectivity that is trained online using a set of target appearances and background from the recent past. The long-term memory uses an offline trained polynomial CNN as a vehicle detector. Occlusion detection along with image registration and motion estimation stages aid in tracking the target through occlusions. A motion detection module reduces the effects of model drift and the scale change detector keeps the boundary accurate to the target. The proposed tracker is evaluated against state-of-the-art tracking methods on a vehicle dataset that includes challenging scenarios such as tree canopy occlusion, shadow, and erratic vehicle motion.

An unmanned aerial system (UAS)-based imaging technology has gained great interests in modern photogrammetry and remote sensing. However, due to the limitations of UAS imaging devices, image enhancement (IE) has become a necessary process for improving the visual appearance of UAS images. Although a great amount of effort has been focused on improving image quality from different aspects, the major obstacles are from computational efficiency and complexity, such as manually adjusting the associated algorithmic parameters that account for various image luminance. To overcome these drawbacks, we propose a new adaptive yet highly efficient luminance enhancement method, namely, adaptive trigonometric transformation function (ATTF), for enhancing the visual quality of digital color images captured by a UAS. The ATTF is derived from a tangent-based transformation function whose characteristics adaptively change with respect to the variation of the image luminance. By combining ATTF with a Laplacian operator and a color restoration process, a well-balanced color image is obtained. The effectiveness of the proposed technique is evaluated on various UAS-based images and compared with other IE techniques.

To overcome visual object tracking challenges, various feature-based object trackers use feature combination. Each feature component is developed to overcome certain tracking challenges, but the interaction between the components may cause tracking errors. We propose a tracking solution based on human vision principles to reduce combination errors by adaptively fusing each feature using its previous performance. An adaptive fusion technique is developed to determine feature quality using feature likelihood map variance ratios. The proposed method is completely modular, while reducing the risk of tracker failure. Experimental results on the Visual Object Tracking database show the proposed tracker's robustness and its advantage over state-of-the-art trackers.

In this letter, we propose to use an enhanced version of volumetric directional pattern to efficiently extract rich spatial context information in the hyperspectral imagery (HSI). The proposed technique fuses the texture information from three consecutive bands in the input HSI. The extracted local image texture features for each pixel of interest are then fed into an extreme learning machine classifier to assign object category. The experimental results on three standard hyperspectral data sets demonstrate the effectiveness of the proposed method for HSI classification compared with that of a set of state-of-the-art spatial extraction methods.

In this paper, we present a new framework for building change detection from monocular aerial imagery that automatically predicts building candidates based on adaptive local textural features with successive background removal. An adaptive local entropy feature is developed based on quadratic regression and Random Sample Consensus (RANSAC) for extracting potential building candidates. Then a majority voting aggregation strategy is employed to accurately estimate shadow direction associated with building objects to aid in reducing false positives in the detected building candidates. A ground plane estimation method is proposed to distinguish building and non-building objects that share similar textural features. Finally, to better estimate changes in building area, a double convex hull based morphological merging technique is introduced. The evaluation results of the proposed framework performed on three-band (RGB) aerial images indicate its capability to successfully detect building changes in urban, suburban, and rural areas.

We present a deterministic object detection algorithm capable of detecting multiclass objects in hyperspectral imagery (HSI) without any training or preprocessing. The proposed method, which is named class-associative spectral fringe-adjusted joint transform correlation (CSFJTC), is based on joint transform correlation (JTC) between object and nonobject spectral signatures to search for a similar match, which only requires one query (training-free) from the object's spectral signature. Our method utilizes class-associative filtering, modified Fourier plane image subtraction, and fringe-adjusted JTC techniques in spectral correlation domain to perform the object detection task. The output of CSFJTC yields a pair of sharp correlation peaks for a matched target and negligible or no correlation peaks for a mismatch. Experimental results, in terms of receiver operating characteristic (ROC) curves and area-under-ROC (AUROC), on three popular real-world hyperspectral data sets demonstrate the superiority of the proposed CSFJTC technique over other well-known hyperspectral object detection approaches.

In this paper, we propose a self-organized learning based active contour model with a lattice Boltzmann convergence criteria for fast and effective segmentation preserving the precise details of the object's region of interest. A dual self-organizing map approach is being used to learn the object of interest and the background independently in order to guide the active contour to extract the target region. The lattice Boltzmann method is utilized to evolve the level-set function faster and terminate the evolution of the curve at the most optimum region, which segments objects in cluttered environments. Experiments performed on a challenging dataset (PSCAL 2011) show promising results in terms of time and quality of the segmentation and that our method is more than 53% faster than other state-of-the-art learning-based active contour model approaches.

The challenges existing for current intensity-based histogram feature tracking methods in wide area motion imagery include object structural information distortions and background variations, such as different pavement or ground types. All of these challenges need to be met in order to have a robust object tracker, while attaining to be computed at an appropriate speed for real-time processing. To achieve this we propose a novel method, Directional Ringlet Intensity Feature Transform (DRIFT), that employs Kirsch kernel filtering and Gaussian ringlet feature mapping. We evaluated the DRIFT on two challenging datasets, namely Columbus Large Image Format (CLIF) and Large Area Image Recorder (LAIR), to evaluate its robustness and efficiency. Experimental results show that the proposed approach yields the highest accuracy compared to state-of-the-art object tracking methods.

Extreme Learning Machine (ELM) has been introduced as a new algorithm for training single hidden layer feed-forward neural networks (SLFNs) instead of the classical gradient-based algorithms. Based on the consistency property of data, which enforce similar samples to share similar properties, ELM is a biologically inspired learning algorithm with SLFNs that learns much faster with good generalization and performs well in classification applications. However, the random generation of the weight matrix in current ELM based techniques leads to the possibility of unstable outputs in the learning and testing phases. Therefore, we present a novel approach for computing the weight matrix in ELM which forms a State Preserving Extreme Leaning Machine (SPELM). The SPELM stabilizes ELM training and testing outputs while monotonically increases its accuracy by preserving state variables. Furthermore, three popular feature extraction techniques, namely Gabor, Pyramid Histogram of Oriented Gradients (PHOG) and Local Binary Pattern (LBP) are incorporated with the SPELM for performance evaluation. Experimental results show that our proposed algorithm yields the best performance on the widely used face datasets such as Yale, CMU and ORL compared to state-of-the-art ELM based classifiers.

A discrete cosine transform (DCT)-based no-reference video quality prediction model is proposed that measures artifacts and analyzes the statistics of compressed natural videos. The model has two stages: 1) distortion measurement and 2) nonlinear mapping. In the first stage, an unsigned ac band, three frequency bands, and two orientation bands are generated from the DCT coefficients of each decoded frame in a video sequence. Six efficient frame-level features are then extracted to quantify the distortion of natural scenes. In the second stage, each frame-level feature of all frames is transformed to a corresponding video-level feature via a temporal pooling, then a trained multilayer neural network takes all video-level features as inputs and outputs, a score as the predicted quality of the video sequence. The proposed method was tested on videos with various compression types, content, and resolution in four databases. We compared our model with a linear model, a support-vector-regression-based model, a state-of-the-art training-based model, and a four popular full-reference metrics. Detailed experimental results demonstrate that the results of the proposed method are highly correlated with the subjective assessments.

Most detection algorithms are established by using well defined features. Since wide area imagery is low resolution and has features that are not well defined, a local intensity distribution based methodology seems a likely candidate. We propose a new methodology, Gaussian Ringlet Intensity Distribution (GRID), which is a derivative of the ring-partitioned histograms for local intensity distribution based object tracking in low-resolution environments, which deals with the issue of rotation invariance. We observed that the proposed algorithm produces the highest accuracy among other state of the art methodologies and provides robust features for rotationally invariant detection and tracking in wide area motion imagery.

This paper presents an approach to predict the quality of compressed videos with content of natural scenes. The method is focused on measuring the distortion of compressed video without reference. There are two main steps of the proposed method: measuring distortion and predicting video quality. Each frame of the distorted video sequence is first decomposed to an N-subband Laplacian pyramid, then their intra-subband and inter-subband statistical features are fully exploited. Three intra-subband features and three inter-subband features are taken as inputs of the prediction model. Its output is a single score as the predicted video quality. The performance of the proposed method is evaluated on the LIVE video database and the LIVE mobile video database. Results show that the predicted quality scores are well correlated with the mean opinion scores associated to the subjective assessment.

Automatic recognition of face sketches is a challenging problem with application in criminal investigations. We propose a method that allows face sketch recognition across modalities called Local Difference of Gaussian Binary Pattern (LDoGBP). LDoGBP is based on the fact that the sketches are similar to their corresponding photos even though they are prone to shape distoration. This similarity between sketch and photo is captured and used for recognition across modalities. In this method, the face image characteristics are captured in the Difference of Gaussian (DoG) representation of the image patches. The Local Binary Pattern(LBP) corresponding to the DoG representation is then generated. These histograms are concatenated to generate the feature vector corresponding to input image. These feature vectors are compared using Earth Mover's Distance for recognition. Experiments on the CUFS(Chinese University of Hong Kong (CUHK) Face Sketch Database) and CUFSF (CUHK Face Sketch FERET Database) datesets prove the effectiveness of this feature in Face Sketch Recognition.

In this paper, a novel approach for image spatial resolution enhancement based on multi-level local Fourier phase features is proposed. This method uses adaptive kernel regression technique based on multi-level local covariance to estimate the high resolution image from a low resolution input. However, this concept is similar to other regression and covariance based methods, our method uses multi-level Fourier image features to learn the local covariance from geometric similarity between low resolution image and its corresponding high resolution image. For each local region, four weighted integrated directional variances are estimated to adapt the interpolated pixels. This method is tested on various natural and aerial images at higher resolution scales. The results confirm that the proposed technique performs better especially at high resolution scales in comparison with other state of art techniques.

We present a 3D reconstruction algorithm designed to support various autonomous vehicle navigation applications. The algorithm presented focuses on the 3D reconstruction of a scene using only a single moving camera. Utilizing video frames captured at different points in time allows us to determine the relative depths in a scene. The original reconstruction process resulting in a point cloud was computed based on feature matching and depth triangulation analysis. In an improved version of the algorithm, we utilized optical flow features to create an extremely dense representation model. Although dense, this model is hindered due to its low disparity resolution. With the third algorithmic modification, we introduce the addition of the preprocessing step of nonlinear super resolution. With this addition, the accuracy and quantity of features is significantly increased since the number of features is directly proportional to the resolution and high frequencies of the input images. Our final contribution of additional pre and post processing steps are designed to filter noise points and mismatched features, completing the presentation of our Dense Point-cloud Representation (DPR) technique. We measure the success of DPR by evaluating the visual appeal, density, usability and computational expense of the reconstruction technique and compare with two state-of-the-art techniques.

A human action recognition framework is proposed which models motion variations corresponding to a particular class of actions without the need for sequence length normalization. The motion descriptors used in this framework are based on the optical flow vectors computed at every point on the silhouette of the human body. Histogram of flow(HOF) is computed from the optical flow vectors and these give the motion orientation in a local neighborhood. To get a relationship between the motion vectors at a particular instant, the magnitude and direction of the optical flow vector are coded with local binary patterns(LBP). The concatenation of these histograms(HOF-LBP) are considered as the action feature set to be used in the proposed framework. We illustrate that this motion descriptor is suitable for classifying various human actions when used in conjunction with the proposed action recognition framework which models the motion variations in time for each class using regression based techniques. The feature vectors extracted from the training set are suitably mapped to a lower dimensional space using Empirical Orthogonal Functional Analysis. A regression based technique such as Generalized Regression Neural Networks(GRNN), are used to compute the functional mapping from the action feature vectors to its reduced Eigenspace representation for each class, thereby obtaining separate action manifolds. The feature set obtained from a test sequence are compared with each of the action manifolds by comparing the test coefficients with the ones corresponding to the manifold (as estimated by GRNN) to determine the class using Mahalanobis distance.

In this paper we propose a new method to detect characteristic shape variations in infrared video using the concept of fractals. The proof of the concept is presented in terms of an application to detect whale blows in infrared video. This application will be of immense use to researchers who study whale behavior like its migration patterns. Whale blows appear as characteristic patterns with relatively higher intensity when compared to its surroundings. The pattern of whale blow formation is detected using fractal analysis. We have been able to experimentally prove that the fractal dimension can be used as an efficient tool to detect the presence of whale blows subject to some thresholding constraints. The development of thresholding constraints based on the variation of the coefficient of variation is also explained in the paper. We have been able to detect whale blows in infrared video with a very high degree of accuracy irrespective of the size of the whale blow and its distance from the camera.

Automatic recognition of human faces irrespective of the expression variations is a challenging problem. In this paper, we propose a novel method for face recognition based on `edge-strings'. Experimental studies on face perception have shown the significance of edge features in visual perception and learning. In the proposed technique, the edges of a face are identified, and a feature string is created from edge pixels. This forms a symbolic descriptor corresponding to the edge image referred to as `edge-string'. The `edge-strings' are then compared using the Smith-Waterman algorithm to match them. The class corresponding to each image is identified based on the number of string primitives that match. Local string alignment algorithm is more robust to noise than global alignment algorithm; it gives better performance even if the input image is noisy. In addition, this method needs only a single training image per class. The proposed technique is a good solution for expression invariant face recognition. The effectiveness of the proposed method is compared with state-of-the-art algorithms on the Yale Face database, the Japanese Female Face Expression database (JAFFE) and CMU AMP Face EXpression database.

In this paper we propose a novel tracking method in Wide Area Motion Imagery (WAMI) data based on local region histogram feature and a statistical distance measure. The aspects that make tracking particularly challenging are global camera motion, large movement of targets, poor gradient and texture information and absence of color information. Global camera motion is reduced or eliminated by registering the images from frame to frame employing SURF (Speeded Up Robust Feature). The proposed method is based on a variant of intensity histogram that encodes both spatial and intensity information. The method is evaluated on aerial WAMI data. The robustness of the feature eliminates the need for background subtraction in videos. A performance comparison of our feature descriptor with other descriptors such as HOG (Histogram of Gradients), SURF and SIFT (Scale Invariant Feature Transform) shows the effectiveness of the proposed method. We also show a comparison of our method with mean-shift tracking to show its effectiveness in tracking on WAMI data.

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this paper. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature space is developed. Least squares estimation approach that utilizes interdependency between points in training patterns is used to form the nonlinear region. A feature space encompassing multiple pattern classes can be trained by modeling a separate constraint region for each pattern class and obtaining a mean constraint region by averaging all the individual regions. Unlike most other nonlinear techniques, the proposed method provides an easy intuitive way to place new points onto a nonlinear region in the feature space. Classification accuracy is further improved by introducing the concepts of modularity and discriminant analysis into the proposed method.

A feature selection technique along with an information fusion procedure for improving the recognition accuracy of a visual and thermal image-based facial recognition system is presented in this paper. A novel modular kernel eigenspaces approach is developed and implemented on the phase congruency feature maps extracted from the visual and thermal images individually. Smaller sub-regions from a predefined neighborhood within the phase congruency images of the training samples are merged to obtain a large set of features. These features are then projected into higher dimensional spaces using kernel methods. The proposed localized nonlinear feature selection procedure helps to overcome the bottlenecks of illumination variations, partial occlusions, expression variations and variations due to temperature changes that affect the visual and thermal face recognition techniques. AR and Equinox databases are used for experimentation and evaluation of the proposed technique. The proposed feature selection procedure has greatly improved the recognition accuracy for both the visual and thermal images when compared to conventional techniques. Also, a decision level fusion methodology is presented which along with the feature selection procedure has outperformed various other face recognition techniques in terms of recognition accuracy.

A new wavelet-based image enhancement algorithm is proposed to improve performance of face detection in non-uniform lighting environment with high dynamic range. Wavelet transform is used for dimension reduction so that dynamic range compression with local contrast enhancement algorithm is applied only to the approximation coefficients. The normalized approximation coefficients are transformed using a hyperbolic sine curve which achieves dynamic range compression. Contrast enhancement is realized by tuning the magnitude of each coefficient with respect to its surroundings. The detail coefficients are also modified to prevent the edge deformation. Experimental results on the proposed algorithm show improvement on the performance of the Viola-Jones face detector when compared to other prominent enhancement techniques.

In this paper, an efficient design for the high performance, power-aware architecture to extract skinlike regions in the video stream is presented. Skin segmentation is an important step in many image processing and computer vision applications such as face detection and hand gesture recognition. The design utilizes the high correlation and similarity of neighboring pixels in video streams to reduce switching activity (hence reducing dynamic power dissipation) in the arithmetic unit. The proposed design is implemented and fitted in the Altera's Cyclone II FPGA which is available in the DE2 development and educational board. The pipelined system is capable of performing the skin segmentation procedure in real-time with a processing rate of 654 frames per second for video frames with standard size of 640*480. It is observed that the proposed design helps to reduce operations and switching activities in the processing unit up to 42 percent which results in lower dynamic power dissipation with low hardware overhead.

A new framework of recurrent neural network is proposed in this paper for automatic feature selection for tracking. The network is not designed particularly for conventional applications such as pattern classification, association, and recognition; instead, it captures parts of those ingredients for identification of unique features from given sets of data. The architecture extracts different types of textures defined by natural importance to the datasets. These textural layers are then fused into single layer feature where the neurons compete and converge with few iterations based on the criteria of uniqueness of the textually maximized features. The automatically selected features by winning neurons, if any, are determined once and applied for subsequent feature tracking within the same architecture. Experiments performed on video sequence showed that the framework for feature selection and tracking is acceptable to gradual in-plane rotation and some degree of scale and out-of-plane rotation.

In designing a recurrent neural network, it is usually of prime importance to guarantee the convergence in the dynamics of the network. We propose to modify this picture: if the brain remembers by converging to the state representing familiar patterns, it should also diverge from such states when presented with an unknown encoded representation of a visual image. We propose to capture this behavior using a nonlinear line attractor network. This model encapsulates attractive fixed points scattered in the state space representing patterns with similar characteristics as an attractive curved line. The dynamics of the nonlinear line attractor network is designed such that when the network is able to reach equilibrium (stable), the input is considered as one of the stored patterns. Conversely, when the network is unable to reach equilibrium (unstable), the input is considered to be dissimilar to the stored patterns and therefore is considered as pattern of another class. Several experiments on benchmark problems have shown that the proposed model can be very useful for discriminating patterns.

A novel architecture for performing color image enhancement using a machine learning algorithm called Ratio Rule is proposed in this paper. The threshold width of the activation function is automatically determined from the image characteristics. The approach promotes log-domain computation to eliminate all multiplications and divisions, utilizing approximation techniques for efficient estimation of the log2and inverse-log2. The design incorporates the dynamic thresholds of the activation functions and update rate. The improved quadrant symmetric architecture is also presented to provide very high throughput rate for homomorphic filters which is part of the pixel intensity enhancement across RGB components in the system. The pipelined design of the filter features the flexibility in reloading a wide range of kernels for different frequency responses. A new approach for the design of the uniform filters is also presented to reduce the processing element arrays (PEAs) from W PEAs to 2 PEAs for W × W window. This new concept is applied to assist in training the synaptic weights of the neural network for color balancing to restore the intensity enhanced image to its natural color existed in the original image. The concept of uniform filter is further extended to design max/min filters. It is observed that the performance of the system with parallel and pipelined architectures is able to achieve 139.3 million outputs per second (MOPS), or equivalently 54.7 billion operations per second on Xilinx's Virtex II XC2V2000-4ff896 FPGA at a clock frequency of 139.3 MHz.

We propose a linear attractor network based on the observation that similar patterns form a pipeline in the state space, which can be used for pattern association. To model the pipeline in the state space, we present a learning algorithm using a recurrent neural network. A least-squares estimation approach utilizing the interdependency between neurons defines the dynamics of the network. The region of convergence around the line of attraction is defined based on the statistical characteristics of the input patterns. Performance of the learning algorithm is evaluated by conducting several experiments in benchmark problems, and it is observed that the new technique is suitable for multiple-valued pattern association.

Actual observed scenes usually produce a wide dynamic range. Currently available visual systems use low dynamic range light detector that provide only 8 bits of brightness information at each pixel. This greatly limits what visual systems can do for surveillance applications such as face detection and face recognition. A color image enhancement procedure based on the concept of color characterization, enhancement, and color balancing is proposed in this paper. The enhancement technique directly operates on pixels using a hyperbolic tangent function to increase the dynamic range of the pixel. The global and local statistics of the image is used to control the curvature of the hyperbolic tangent function. The color characterization and color balancing processes are based on a new nonlinear line attractor network to create a color manifold to restore the relationship of red, green, and blue components of the pixels. The proposed enhancement approach greatly improves the dynamic range compression and color rendition of an image.

An efficient pipelined architecture for the real-time correction of barrel distortion in wide-angle camera images is presented in this paper. The distortion correction model is based on least-squares estimation to correct the nonlinear distortion in images. The model parameters include the expanded/corrected image size, the back-mapping coefficients, distortion center, and corrected center. The coordinate rotation digital computer (CORDIC) based hardware design is suitable for an input image size of 1028/spl times/1028 pixels and is pipelined to operate at a clock frequency of 40 MHz. The VLSI system will facilitate the use of a dedicated hardware that could be mounted along with the camera unit.

We present a new skin modeling technique based on SNoW (sparse network of Winnows) for accurate and robust skin region detection. A skin distribution map (SDM) representing the sparse network is trained with skin pixels to learn their distribution in a color space. We then train the SDM with non-skin pixels to unlearn the distribution of the non-skin pixels, which overlap with the skin pixels in the color space. This skin model can be used for skin detection on any color space. We have found the accuracy of skin detection using SDM to be slightly better than that using the skin probability map (SPM) method. The main advantage of using the SDM method over the SPM method is that the complexity, memory requirements and time for skin detection are reduced significantly.

A rotation invariant human face detection system in color images based on human skin color distribution and intensity is proposed in this paper. Skin color distribution typical to a human face is used as a feature along with the intensity variations to classify the candidate regions into faces and nonfaces. The detection process is carried out in YCbCr color space. Sparse Network of Winnows architecture is used to train three networks one for intensity and two for the color distributions for classification of candidate regions. Rotation invariance in detection of faces is achieved by training multiple classifiers, each to detect faces at a particular orientation. The detection process also implements a non linear luminance based lighting compensation method which is very efficient in enhancing and restoring the natural colors into the images which are taken in darker and varying lighting conditions. Experimental results show that the new face detection technique is highly efficient in terms of speed and accuracy in detecting frontal view faces at different orientations in complex environments.

Homomorphic filter is an illumination-reflectance model that can be used to develop a frequency domain procedure for improving the appearance of an image by simultaneous gray-level range compression and contrast enhancement. Many previously reported methods on homomorphic filter for color images shows that the homomorphic filter consistently provides excellent dynamic range compression but is lacking final color rendition. We present a novel color image enhancement process to overcome this limitation. The color image enhancement process involved using a neural network algorithm, namely ratio rule, to pre-process and post-process the color image in the homomorphic system. This method improves the appearance of images as perceived by the human eye, and/or to render these images more suitable for computer analysis. That is, both color rendition and dynamic range compression are achieved using this method.

Design and implementation of a fast parallel architecture based on an improved principal component analysis (PCA) method called Composite PCA suitable for real-time face recognition is presented in this paper. The proposed architecture performs the tasks of both feature extraction and classification. Composite PCA takes in to consideration the local features of face images, which do not vary widely between face images of the same person taken under varying expression, illumination and pose. Hence it leads to a better recognition rate than PCA. Composite PCA has more parallelism than conventional PCA and this parallelism is utilized to design an efficient architecture capable of performing real-time face recognition. The face recognition system is implemented in an FPGA environment and tested using standard databases. The system is able to recognize a person from a database of 110 images of 10 individuals in approximately 4 ms.

The consequence of reducing the impact of the synaptic weights from neurons farther away from the neuron under consideration on a modular two-dimensional Hopfield network using Hebbian learning rule is examined for image processing applications. A generalized modular architecture is developed by defining each module with a group of neighboring neurons and all modules communicating with each other. A spatially decaying distance factor is introduced into the Hebbian rule to reduce the effect of neurons from farther modules. A biologically inspired visual perception concept has been adopted for defining the variation of the distance factor. The performance of the new technique is evaluated by conducting several experiments on character images and it is observed that the proposed method increases the learning ability and convergence rate of the network. The nature of the distance factor helps the removal of several synaptic weights farther away from a particular neuron and this leads to the reduction of complexity of the network in terms of both software and hardware implementation.

A novel learning algorithm, named ratio rule, for association of multi-valued patterns in a recurrent neural network is proposed in this paper. The learning is performed based on the degree of similarity between the relative magnitudes of the output of each neuron with respect to that of all other neurons. The dynamics of the neural network functions as a line attractor as opposed to the common concept of point attractor. The limit of the convergence region around the line of attraction is defined based on the statistical characteristics of the input patterns. Theoretical analysis of the associativity of the network with the ratio rule confirms the authenticity of its learning ability. The performance of the ratio rule on associativity and convergence of the recurrent network is evaluated by conducting several experiments on face images. It is observed that the ratio rule is suitable for retention, reconstruction, and restoration of learned patterns with varying face expressions.

A new learning algorithm based on L2-norm approximation to define the relationship between two neurons in a recurrent neural network is proposed in this paper. The learning process utilizes the statistical relationship between each component of the input pattern with respect to every other component. The activation function of a neuron is a rectangular function whose position changes adaptively with respect to the input pattern and its left and right wings are decided by the mean of maximum variations of the training signals to that neuron. The new training algorithm is applied for recognition of faces images with varying expressions. 975 face images of 13 persons from the Carnegie Mellon University (CMU) face expression variant database are used for evaluating the performance of the network. The network has been trained with 5 images and tested with the remaining 70 images of each person. The recurrent neural network with the new learning algorithm recognized all the 13 persons in this database without error.

We consider an improved least square algorithm based on subspace rotations using Procrustes approximation. Robust estimation of the poles of a narrowband signal is achieved by reducing the bias of the estimates. The key idea is to consider the signal as a vector in multi-dimensional space, and to separate the “pure signal” and the noise into two mutually orthogonal components lying in different subspaces. Though the quality of the estimated signal is not much improved as compared to the conventional least square based subspace approaches, the advantage of using the proposed algorithm is the asymptotically unbiased nature of the estimates.

A technique for the training of multiple-valued neural networks based on a backpropagation learning algorithm employing a multilevel threshold function is proposed. The optimum threshold width of the multilevel function and the range of the learning parameter to be chosen for convergence are derived. Trials performed on a benchmark problem demonstrate the convergence of the network within the specified range of parameters.

Images captured with a typical endoscope show spatial distortion, which necessitates distortion correction for subsequent analysis. In this paper, a new methodology based on least squares estimation is proposed to correct the nonlinear distortion in the endoscopic images. A mathematical model based on polynomial mapping is used to map the images from distorted image space onto the corrected image space. The model parameters include the polynomial coefficients, distortion center, and corrected center. The proposed method utilizes a line search approach of global convergence for the iterative procedure to obtain the optimum expansion coefficients. A new technique to find the distortion center of the image based on curvature criterion is presented. A dual-step approach comprising token matching and integrated neighborhood search is also proposed for accurate extraction of the centers of the dots contained in a rectangular grid, used for the model parameter estimation. The model parameters were verified with different grid patterns. The distortion-correction model is applied to several gastrointestinal images and the results are presented. The proposed technique provides high-speed response and forms a key step toward online camera calibration, which is required for accurate quantitative analysis of the images.

An optimization technique for the design of two types of multiple-valued PLAs is described. In a type-I PLA, the multiple-valued function is realized directly, whereas in a type-II PLA, output encoding is used to encode the binary output of the PLA. In both types, multiple function literal circuits are used for the purpose of minimization. It is shown that the proposed technique leads to a considerably reduced size of PLA when compared to the earlier techniques.>