This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks.

In this paper, we propose a bioinspired model for human action recognition through modeling neural mechanisms of information processing in two visual cortical areas: the primary visual cortex (V1) and the middle temporal cortex (MT) dedicated to motion. This model, named V1-MT, is composed of V1 and MT models (layers) corresponding to their cortical areas, which are built with layered spiking neural networks (SNNs). Some neuron properties in V1 and MT, such as direction and speed selectivity, spatiotemporal inseparability, and center surround suppression, are integrated into SNNs. Based on speed and direction selectivity, V1 and MT models contain multiple SNN channels, each of which processes motion information in sequences with spatiotemporal tunings of neurons at a certain speed and different directions. Therefore, we propose two operations, input signal perceiving with 3-D Gabor filters and surround inhibition processing with 3-D differences of Gaussian functions, to perform this task according to the spatiotemporal inseparability and center surround suppression of neurons. Then, neurons are modeled with our simplified integrate-and-fire model and motion information is transformed into spike trains. Afterward, we define a new feature vector: a mean motion map computed from spike trains in all channels to represent human actions. Finally, a support vector machine is trained to classify actions represented by the feature vectors. We conducted extensive experiments on public action databases, and the results show that our model outperforms other bioinspired models and rivals the state-of-the-art approaches.

A model for detecting contours in natural images is presented by combining the visual perceptual mechanisms and machine learning. The surround stimuli will enhance the response of the central stimulus if they can form a precise spatial configuration. On the other hand, surround inhibition will reduce the responses to homogeneous elements. Facilitation and inhibition activities in the primary visual cortex (V1) are used to enhance the well-organized structures and to reduce the non-meaningful distractors engendering from texture fields, respectively. We approach the task of facilitatory and inhibitory cue integration as a supervised learning problem using the logistic regression model. Our experiments demonstrate that the model can dramatically reduce texture edges and spurious contours, and meanwhile can to some extent avoid ground-truth contours missed by the detector.

Recently, many vision applications tend to utilize saliency maps derived from input images to guide them to focus on processing salient regions in images. In this paper, we propose a simple and effective method to quantify the saliency for each pixel in images. Specially, we define the saliency for a pixel in a ratio form, where the numerator measures the number of dissimilar pixels in its center-surround and the denominator measures the total number of pixels in its center-surround. The final saliency is obtained by combining these ratios of dissimilarity over multiple scales. For images, the saliency map generated by our method not only has a high quality in resolution also looks more reasonable. Finally, we apply our saliency map to extract the salient regions in images, and compare the performance with some state-of-the-art methods over an established ground-truth which contains 1000 images.

In this paper, we propose a novel unsupervised clustering method for feature space analysis. We combine mean shift with a transductive learning method, semi-supervised discriminant analysis (SDA), in an incremental learning scheme. We use mean shift clustering to generate the class label, and use SDA to do subspace selection. Both these steps are performed alternately. Our clustering result could maintain good spatial consistency for all data in feature space. On image segmentation, we directly apply our clustering method to the L*a*b* color feature space generated from superpixels, and set each pixel with the clustering label of its superpixel. We test our image segmentation method on Berkeley image data set.

This paper proposes a new approach to describe the salient contours in cluttered scenes. No need to do the preprocessing, such as edge detection, we directly use a set of random straight line segments, as the intermediate level vision tokens, to approximate the salient contours. This line set is modeled by a stochastic framework, marked point process, in which the point denotes the center of lines, and the marker denotes the orientation and length of lines. Generic Gastalt factors of proximity and collinear continuity are embedded to constraint the geometrical inter-relations between lines. Different data likelihoods are used on synthetic and real images. Optimization is done by simulated annealing using Reversible Jump Markov chain Monte Carlo. Our results not only have a good approximation to the salient contours, also make other post-processing application more robust.

In this letter, we address the problem of modeling scene background for moving object detection. Although the per-pixel model has been extensively exploited in the literature, the pixel-pair relationship is still a nontrivial problem to be represented efficiently. To deal with this issue, we propose a spatiotemporal smooth model based on conditional random field. Besides the mutual influence on labels, data dependencies are encoded into the spatiotemporal smooth model effectively by employing the contextual constraints in terms of both spatial coherence and temporal persistency. Accurate extraction of foreground from nonstationary scenes can be achieved by the proposed model. Experiments conducted on various sequences demonstrate the property of the proposed method.