Completely automated public Turing test to tell humans apart (CAPTCHA) is an effective mechanism to protect websites and online applications from malicious bots programs. Image-based CAPTCHA is one of the most widely used schemes. However, deep learning techniques have significantly weakened the security of some image-based CAPTCHA schemes. Mooney images (MIs) are important research materials in the field of cognitive science. Compared to natural images, MI exhibits fewer visual cues, fragmented content, and greater ambiguity, leading to the perception of MI relying more on the iterative process between feedforward and feedback mechanisms. In this paper, we raise an intriguing question: can MIs be used to enhance the security of CAPTCHA? Before this study, we first propose a novel framework HiMI that generates the high-quality MIs from natural images and also allows flexible adjustment of the perceived difficulty. Based on MI, we design two MI-CAPTCHA schemes related to object detection and instance segmentation tasks, respectively. We experimentally demonstrate that HiMI performs better than other baseline methods in terms of both image quality and application potential in two MI-CAPTCHA schemes. Additionally, we conduct experiments to explore the solving performance of humans and CAPTCHA solvers under different parameter settings of schemes, providing valuable reference for the practical application.

Perceptual edge grouping is a technique for organizing the cluttered edge pixels into meaningful structures and further serves high-level vision tasks, which has long been a basic and critical task in computer vision. Existing methods usually have a poor performance when coping with the junctions caused by occlusion and noise in natural images. In this paper, we present GPGrouper, a perceptual edge grouping model based on gestalt theory and the primary visual cortex (V1). Different from the existing methods, GPGrouper leverages the edge representation and grouping matrix (ERGM), a functional structure inspired by V1 mechanisms, to represent edges in a way that can effectively reduce grouping errors caused by occlusion between objects. ERGM is trained with natural image contours and further provides a priori guidance for the construction of the edge connection graph (ECG) that is useful to minimize the impact of noise on grouping. In the experiment, we compared GPGrouper and the state-of-the-art (SOTA) method of perceptual grouping on the visual psychology pathfinder challenge. The results demonstrate that GPGrouper outperforms the SOTA method in grouping performance. Furthermore, in the grouping experiments involving line segments with varying lengths detected by the Line Segment Detector (LSD), as well as those involving superpixel segmentation results with significant levels of interfering noise using the SLIC algorithm, GPGrouper was superior to the existing methods in terms of grouping effect and robustness. Moreover, the results of applying the grouping results to the vision tasks objectness demonstrate that GPGrouper can contribute significantly to high-level visual tasks.

3D keypoint detection is of great interest to researchers in computer vision and graphics because it is an integral part of realizing many tasks, such as object tracking, 3D reconstruction, and shape registration. However, it is challenging to detect 3D keypoints quickly and stably due to the ambiguity of the keypoints and the presence of noise, density changes, and geometric distortions in the 3D point cloud. This paper proposes a novel 3D keypoint detection method based on point cloud structural saliency (PCSS) to realize stable and efficient 3D keypoint detection. First, we propose an effective point cloud feature descriptor called local spatial geometric feature, which can effectively combine spatial and geometric information to improve feature distinguishability. Second, we define a point cloud structural saliency representation that effectively characterizes the structured information in the point cloud. Finally, we generate 3D keypoints based on point cloud structural saliency using a non-maximum suppression method. We evaluate our method on five 3D keypoint benchmark datasets, and the experimental results demonstrate that it achieves state-of-the-art performance in 3D keypoint detection. Comparing it with previous keypoint detection methods further demonstrates the effectiveness and superiority of our method.

Existing view planning systems either adopt an iterative paradigm using next-best views (NBV) or a one-shot pipeline relying on the set-covering view-planning (SCVP) network. However, neither of these methods can concurrently guarantee both high-quality and high-efficiency reconstruction of 3-D unknown objects. To tackle this challenge, we introduce a crucial hypothesis: with the availability of more information about the unknown object, the prediction quality of the SCVP network improves. There are two ways to provide extra information: first, leveraging perception data obtained from NBVs, and second, training on an expanded dataset of multiview inputs. In this work, we introduce a novel combined pipeline that incorporates a single NBV before activating the proposed multiview-activated (MA-)SCVP network. The MA-SCVP is trained on a multiview dataset generated by our long-tail sampling method, which addresses the issue of unbalanced multiview inputs and enhances the network performance. Extensive simulated experiments substantiate that our system demonstrates a significant surface coverage increase and a substantial 45% reduction in movement cost compared to state-of-the-art systems. Real-world experiments justify the capability of our system for generalization and deployment.

Keypoint detection plays a fundamental role in many applications, such as 3D reconstruction, object registration, and shape retrieval, and has attracted significant interest from researchers in computer vision and graphics. However, due to the ambiguity of the keypoint and the complexity of 3D objects, it is still tricky for existing 3D keypoint detection methods to generate stable keypoints with good coverage, especially for unsupervised detection methods. This paper proposes a 3D keypoint detection method based on hierarchical point saliency. This method can effectively and accurately locate the keypoints of a 3D point cloud, and it does not require complex training processes. First, we propose a simple and effective point descriptor called the local geometric structure feature, which can effectively characterize the geometric structure changes of 3D point clouds and has a strong feature identification ability. Second, we define two saliency measures used to characterize the saliency of points in the point cloud, which are low-level and high-level saliency. Third, we hierarchically characterize the saliency of points by combining the low-level and high-level saliency, thus measuring the probability that a point belongs to a keypoint. Finally, we extensively test our method on three benchmark 3D point cloud datasets, and the experimental results demonstrate that our method achieves state-of-the-art performance in keypoint detection tasks, significantly superior to the prior hand-crafted and deep-learning-based 3D keypoint detection methods.

Neurobiology has made groundbreaking discoveries that have propelled the development of brain-inspired computing. This new field of research has led to the creation of neural pathways that mirror the human brain's visual system, providing numerous transformative insights for modern image processing. Brain-like vision offers remarkable processing efficiency and promises a range of exciting applications, from enhancing video surveillance to improving medical diagnosis. However, current brain-inspired computing models lack portability and efficient power consumption, which limits their widespread use. To address these challenges, we've designed a brain-like color opponent multi-layer neural network model on field programmable gate arrays (FPGAs). This innovative approach enhances portability, reduces power consumption, and increases parallelism in the brain-like computational model. We've conducted rigorous experiments on natural images, and the results are promising. FPGAs can further enhance this innovative approach by improving computational parallelism and enhancing the display of crucial structural information while downplaying trivial details. This integration of biological mechanisms into practical embedded systems is a significant step towards merging these two technologies. Our multi-layer neural network structure aligns with FPGAs' in-memory computing mechanism, bridging the gap between these two technologies. Our work represents a step towards merging them and utilizes physiological color opponent theory to enhance FPGAs' effectiveness in practical image processing applications. By combining the power of neurobiology with the flexibility of FPGAs, we can create more efficient and practical solutions for image processing tasks in various fields.

Abstract In this study, we construct a smoothing term structure, which is an essential part of the energy function in binocular matching. However, the existing energy models are mainly deterministic, which cannot adapt to processing low-quality images, especially when there exists a large proportion of vague areas. In order to perform better in processing these low-quality images, in this paper, we construct the smoothing term based on a fuzzy model, which includes fuzzy segmentation, the fuzzy network between the superpixels and the fuzzy relationship between the pixels. These can be compatible with the uncertainty in the image. In addition, to explain the rationality of the calculation of the degree of correlation between superpixels and further elaborate on the property of these degrees between each superpixel, we propose five corresponding theorems with proofs. After we solve the energy model combined with our proposed smoothing term, we compare our disparity results with the corresponding deterministic model and several state-of-the-art algorithms in the experiment. The results verify the effectiveness of the proposed algorithm.

In many brain network studies, brain functional connectivity data is extracted from neuroimaging data and then used for disease prediction. For now, brain disease data not only has a small sample but also has the problem of high dimensional and nonlinear. Therefore, deep clustering on brain functional connectivity data is very challenging. To solve these problems, we propose a Soft-orthogonal Constrained Dual-stream Encoder with Self-supervised clustering network (SSCDE), which consists of a pretext task and downstream task, which can fully mine the effective information in brain disease data. In the pretext task, we use two brain disease data under the same category to do cross-domain learning to obtain effective information from the same dataset. In the downstream task, to reduce redundancy and avoid negative coding, we propose a soft-orthogonal constrained dual-stream encoder to encode features separately. At the same time, we use the pseudo labels given by the pretext task as prior information for self-supervised learning. We conduct validation on different brain disease recognition tasks, and the result have proved that the proposed framework has achieved good performance compared with the unsupervised clustering analysis algorithms. To our knowledge, this is the first cross-domain assisted recognition study on brain functional connectivity data. The code is available at https://github.com/hulu88/SSCDE .

Point cloud registration is a crucial challenge in photogrammetry and computer vision, aimed at aligning adjacent point clouds optimally. In this article, we present a novel registration approach based on plane constraints for large-scale LiDAR point clouds, called PCGOR, effectively decoupling rotation estimation and translation estimation. The point cloud registration challenge is then bifurcated into two distinct parts: rotation estimation and translation estimation. For rotation estimation, we develop an outlier removal method combining coarse filtering with rotation-invariant constraints (RICs) and refined filtering based on computational geometric consistency checks, effectively pruning outliers and robustly estimating accurate relative rotations from plane normals. In translation estimation, we design a componentwise approach based on translation component constraints (TCCs) to efficiently estimate relative translations. Experimental findings validate the robustness and efficacy of our proposed approach on three popular LiDAR point cloud datasets, yielding state-of-the-art performance.

Automatic registration of point clouds is a fundamental research problem in 3D computer vision. In this paper, a sketch-based registration framework is proposed targeting 3D scenarios. It consists of two major modules: pairwise alignment and multi-view alignment. For the pairwise alignment, a point cloud is first abstracted into a sketch which greatly preserves the contour information in the scene; then an Entropy Point Pair Feature (EPPF) method that integrates contour shape features and point pair geometric features is applied to estimate transformation. For the multi-view alignment, the key is to combine the voting-based pairwise method with Simultaneous Localization and Mapping (SLAM) system, which ensures the robustness of the proposed framework in different scenarios. Experiments show that the proposed sketch-based method clearly outperforms the state-of-the-art methods.

Plant identification based on leaf images is a widely concerned application field in artificial intelligence and botany. The key problem is extracting robust discriminative features from leaf images and assigning a measure of similarity. This study proposes an effective, robust shape descriptor to identify plant species from images of their leaves, which we call the high-level triangle shape descriptor (HTSD). First, we extract a leaf image's external contour and internal salient point information. We then use triangle features to describe the leaf contour, which we call the contour point based on triangle features (CPTFs). The internal information of the leaf image is based on salient point triangle features (SPTFs). The third step is to apply the Fisher vector to encode the two kinds of point-based local triangle features into the HTSD. Finally, we employ the simple euclidean distance to calculate the dissimilarities between the HTSD characteristics of leaf images. We have extensively evaluated the proposed approach on several public leaf datasets successfully. Experimental results show that our method has superior recognition accuracy, outperforming current state-of-the-art shape-based and deep-learning plant identification approaches.

Point cloud registration is a crucial part of 3D computer vision. Existing point cloud registration methods primarily concentrate on utilizing features such as points, lines, and planes, disregarding the valuable contour cues inherent in the scene. In this article, we propose a novel sketch-based framework for point cloud registration that incorporates contour cues to enhance the point cloud registration task. To fully exploit the abundant information provided by contour cues in the scene, the point cloud is first abstracted into a sketch consisting of contour cues obtained through the utilization of planar features, which greatly preserves the inherent contour information. Subsequently, a local contour geometric descriptor is introduced to encode the contour cues in the sketch. Finally, a voting-based Contour Point Pair Feature (CPPF) framework is employed to fuse planar features, local contour geometric features and point pair geometric features, enabling precise estimation of the pose transformation between pairwise point clouds. Extensive experiments conducted on two large-scale outdoor point cloud datasets and two indoor point cloud datasets validate the effectiveness of the proposed sketch-based method. Our proposed method successfully suppresses rotation and translation errors, ultimately achieving state-of-the-art performance.

Abstract Although many algorithms perform very well on certain datasets, existing stereo matching algorithms still fail to obtain ideal disparity images with high precision in practical robotic applications with weak or untextured objects. This greatly limits the application of binocular vision for robotic arm guidance. Traditional stereo matching algorithms suffer from disparity loss, dilation and other problems, and deep learning algorithms have weakly generalization ability, making high‐accuracy results impossible with non‐training images. We propose an algorithm that uses segments and edges as matching units. We find the mapping relationship between two‐dimensional images and three‐dimensional scenes using segments. The algorithm obtains highly accurate results in industrial robotic applications with mainly planar facets. We combine it with a deep learning algorithm to obtain very good high‐accuracy results in both general scenes and applications of industrial robots. The algorithm effectively improves the non‐linear optimization ability of traditional algorithms and generalization ability of deep learning, and provides an effective method for the binocular vision guidance of industrial robotic scenes. We used the algorithm to guide the robot arm for threading with a success rate of 70%.

Abstract Stereo matching algorithms have been developed for many years but basically focus only on the implementation of existing datasets and are rarely applied to real scenarios, such as industrial robot scenarios. Traditional stereo matching algorithms have a high error rate, and deep learning algorithms are difficult to obtain good results in real scenarios because of their weak generalisation ability and difficult access to training data. In order to use stereo matching algorithms for industrial robot guidance, it is better to design a new traditional algorithm with low time complexity for the characteristics of industrial robot scenarios dominated by planar facets. This paper proposes a new matching method based on subrows of pixels, instead of individual pixels, in order to improve robustness of matching and reduce running time. First, the pixel strings from the same row of the left and right images are divided into several colour‐identical or colour‐gradient segments. Then, the colour and length of the two left and right pixel segment are used as clues to determine a matching relation and obtain the matching type. Then, all match types can be determined according to non‐crossing mapping. Each match type can reason backward to the corresponding spatial state of the stimulus source so that the disparity of pixels in pixel segments representing the spatial state can be calculated. This new matching method makes full use of the stimulus homology constraints and projective geometric constraints of row‐aligned images. The method can obtain good results in industrial robot scenarios and be applied for industrial robot guidance.

Delivery using autonomous vehicles for medical and emergency supplies is a potential way to avoid unsafe and unpredictable factors. However, its implementation is hindered due to several key issues. A major dilemma is understanding curved alleyways in street scenes. These can be seen as compositions of non-Manhattan structures, which can help us estimate their original posture in three-dimensional scenes. We propose a new methodology to understand curved alleyways, and to bridge the gap between two-dimensional scene understanding and three-dimensional environment reconstruction from a monocular camera. Angle projections are assigned to clusters. Coplanar surfaces, which can compose fold structures, are estimated. Curved alley scenes are approximately represented by Manhattan and non-Manhattan fold structures, and approximated in the reconstruction of alley scenes. With geometric features, the algorithm requires no prior training or knowledge of the camera’s internal parameters. We compared the estimated layout to the ground truth and measured the percentage of incorrectly classified pixels. The results showed that the algorithm can successfully understand alley scenes including both Manhattan and curved non-Manhattan structures.

The edges of an image contains rich visual cognitive cues. However, the edge information of a natural scene usually is only a set of disorganized unorganized pixels for a computer. In psychology, the phenomenon of quickly perceiving global information from a complex pattern is called the global precedence effect (GPE). For example, when one observes the edge map of an image, some contours seem to automatically “pop out” from the complex background. This is a manifestation of GPE on edge information and is called global contour precedence (GCP). The primary visual cortex (V1) is closely related to the processing of edges. In this article, a neural computational model to simulate GCP based on the mechanisms of V1 is presented. There are three layers in the proposed model: the representation of line segments, organization of edges, and perception of global contours. In experiments, the ability to group edges is tested on the public dataset BSDS500. The results show that the grouping performance, robustness, and time cost of the proposed model are superior to those of other methods. In addition, the outputs of the proposed model can also be applied to the generation of object proposals, which indicates that the proposed model can contribute significantly to high-level visual tasks.

Helping mobile robots understand curved corridor scenes has considerable value in computer vision. However, due to the diversity of curved corridor scenes, such as curved structures that do not satisfy Manhattan assumption, understanding them remains a challenge. Curved non-Manhattan structures can be seen as compositions of spatial right angles projected into two dimensional projections, which may help us estimate their original posture in 3D scenes. In this paper, we presented an approach for mobile robots to understand curved corridor scenes including Manhattan and curved non-Manhattan structures, from a single image. Angle projections can be assigned to different clusters via geometric inference. Then coplanar structures can be estimated. Fold structures consisting of coplanar structures can be estimated, and curved non-Manhattan structures can be approximately represented by fold structures. Based on understanding curved non-Manhattan structures, the method is practical and efficient for a navigating mobile robot in curved corridor scenes. The algorithm requires no prior training or knowledge of the camera's internal parameters. With geometric features from a monocular camera, the method is robust to calibration errors and image noise. We compared the estimated curved layout against the ground truth and measured the percentage of pixels that were incorrectly classified. The experimental results showed that the algorithm can successfully understand curved corridor scenes including both Manhattan and curved non-Manhattan structures, meeting the requirements of robot navigation in a curved corridor environment.

Helping robots understand indoor scenes has considerable value in computer vision. However, due to the diversity of indoor scenes, understanding them remains a big challenge. There are many spatial right-angles in indoor scenes. These spatial right-angles are projected into diverse 2D projections. These projections can be considered a composition of a pair of lines (line-pairs). Given the vanishing points (VPs), line segments can be assigned to 1 of 3 main orthogonal directions. The line-pairs (intersection of 2 lines), such that each of them converges to a different VP, are likely to be the projection of a spatial right-angle onto the image plane. These projections may enable us to estimate their original orientation and position in 3D scenes. In this paper, we presented a method to efficiently understand indoor scenes from a single image, without training or any knowledge of the camera's internal calibration. Through geometric inference of line-pairs, it is possible to find these spatial right-angle projections. Then, these projections can be assigned to different clusters, and the line that lies in the neighbor-cluster helps us estimate the layout of the indoor scene. The proposed approach required no prior training. We compared the room layout estimated by our algorithm against the room box ground truth, measuring the percentage of pixels that were correctly classified. These experiments showed that our method estimated not only room layout, but also details of the indoor scene.

Animals' decision-making behaviors are widely studied in two-alternative forced-choice tasks. Many models have been proposed to model the decision-making process and explain results of behavioral experiments. These models can fit the data of behavioral experiments well. However, the process of learning the correct decision with reward or punishment feedback is ignored in these models. Learning with reward or punishment feedback is very common in behavior experiments that are conducted to animals. It is closely linked to the decision-making process. In this paper, we investigated how to integrate learning process into two-choice decision-making models. We show that we can combine a two-choice decision-making model implemented with spiking neurons with synaptic plasticity to include the decision-learning process. The decision-making model can learn its output according to either reward or punishment feedback from the environment. And output of the decision-making model can be explained at the synapse level.

The shapes of plant leaves are very important to plant ecologists and botanists because these can help distinguish plant species as well as serve as health indicators. In this paper, we present a novel contour-based shape descriptor named multiscale triangular centroid distance (MTCD) for plant leaf recognition. MTCD features at different triangles are extracted from each contour point to provide a compact, multiscale shape descriptor. Both local and global features of a plant leaf are effectively captured by the proposed method. A simple cosine distance is used to calculate the dissimilarity measurement between MTCD descriptors. Therefore, MTCD is a rapid approach for shape matching and is suitable for real-time application. The proposed method has been evaluated using four publicly available plant leaf datasets, including the Swedish Leaf dataset, the Smithsonian Leaf dataset, the Flavia Leaf dataset, and the ImageCLEF2012 Leaf dataset. The experimental results show that this novel approach can achieve high recognition accuracy. Comparisons with other state-of-the-art shape-based plant leaf recognition methods further demonstrate the effectiveness and efficiency of MTCD.

We present a novel model to represent and match contour lines of closed shapes. This model is based on the mechanism of visual cortex. It extracts orientation features from input images with simple computation units that imitate simple cells in the visual cortex. The contour lines are accurately located by searching adjacent activated simple units. These activated simple units are concatenated in a chain to code the contour lines of closed shapes. In order to match between shapes, we propose a measure based on Fréchet distance and use dynamic programming to calculate the distance between different chains of simple units. The model is evaluated on the MPEG7 shape data set. We also demonstrate that this model can explain the shape selectivity of visual area V4.

A visual stimulus is represented by the biological visual system at several levels: in the order from low to high levels, they are: photoreceptor cells, ganglion cells (GCs), lateral geniculate nucleus cells and visual cortical neurons. Retinal GCs at the early level need to represent raw data only once, but meet a wide number of diverse requests from different vision-based tasks. This means the information representation at this level is general and not task-specific. Neurobiological findings have attributed this universal adaptation to GCs' receptive field (RF) mechanisms. For the purposes of developing a highly efficient image representation method that can facilitate information processing and interpretation at later stages, here we design a computational model to simulate the GC's non-classical RF. This new image presentation method can extract major structural features from raw data, and is consistent with other statistical measures of the image. Based on the new representation, the performances of other state-of-the-art algorithms in contour detection and segmentation can be upgraded remarkably. This work concludes that applying sophisticated representation schema at early state is an efficient and promising strategy in visual information processing.

Shape representation is an extremely important and longstanding problem in the field of pattern recognition. Closed contour, which refers to shape contour, plays a crucial role in the comparison of shapes. Because shape contour is the most stable, distinguishable, and invariable feature of an object, it is useful to incorporate it into the recognition process. This paper proposes a method based on genetic algorithms. The proposed method can be used to identify the most common contour fragments, which can be used to represent the contours of a shape category. The common fragments clarify the particular logics included in the contours. This paper shows that the explicit representation of the shape contour contributes significantly to shape representation and object recognition.

Studies in neuroscience and biological vision have shown that the human retina has strong computational power, and its information representation supports vision tasks on both ventral and dorsal pathways. In this paper, a new local image descriptor, termed distinctive efficient robust features (DERF), is derived by modeling the response and distribution properties of the parvocellular-projecting ganglion cells in the primate retina. DERF features exponential scale distribution, exponential grid structure, and circularly symmetric function difference of Gaussian (DoG) used as a convolution kernel, all of which are consistent with the characteristics of the ganglion cell array found in neurophysiology, anatomy, and biophysics. In addition, a new explanation for local descriptor design is presented from the perspective of wavelet tight frames. DoG is naturally a wavelet, and the structure of the grid points array in our descriptor is closely related to the spatial sampling of wavelets. The DoG wavelet itself forms a frame, and when we modulate the parameters of our descriptor to make the frame tighter, the performance of the DERF descriptor improves accordingly. This is verified by designing a tight frame DoG, which leads to much better performance. Extensive experiments conducted in the image matching task on the multiview stereo correspondence data set demonstrate that DERF outperforms state of the art methods for both hand-crafted and learned descriptors, while remaining robust and being much faster to compute.

The ability of visual cortex to accomplish object recognition tasks accurately and effortlessly makes it an attractive goal of computer vision to emulate the mechanism of the cortex. The neural process of object recognition in the brain follows a hierarchical scheme. In this paper, we present a novel model inspired by the visual pathway in primate brains. This multi-layer neural network model imitates the hierarchical convergent processing mechanism of the visual pathway. We show experimentally that local image features generated by this model exhibit robust discrimination and even better generalization ability compared with some existing image descriptors. We also demonstrate the application of this model to object recognition tasks. The result provides strong support for the potential of this model.

Detecting object in unseen images is an challenging task because of the strong clutter background, various scale of object and the deformation of class. In this paper, we present a shape-based object detection model using scale-invariant fragment feature which is approximated by conjunctive short straight segments. This is a novel shape descriptor for object detection by bypassing estimation of scale of object in natural scene. Utilizing those local and consistent segments, we improve the robustness of model to natural background and deformation of object. We experiment our model on two texture-less image datasets, INRIA horses dataset and Weizmann horses dataset. The results demonstrate our model outperform those state-of-the-art methods.

Existing studies have shown that neuronal functional networks (NFNs) exhibit small-world properties. However, the issue of whether NFNs have any other complex network topology properties remains unresolved. In this paper, we introduced a new hierarchical clustering-based method that can clearly indicate the hierarchical modular organization of NFNs. Based on the modularity function Q proposed by Newman, we can divide the NFNs into suitable sub-modules. We proposed a new measure function to calculate the correlations between pairs of spike trains without requiring binning of the spike trains through small time windows. This method can be used to analyze the level of synchronization between spike trains and functional connectivity relationships between neurons. We analyzed NFNs constructed from multi-electrode recordings in rat brain cerebral cortexes in vivo. These rats had been trained to perform different working memory cognitive tasks. The results show that NFNs exhibit a clear hierarchical modular organization in rat brains. These results provided evidence confirming that the brain networks are complex. This can also be used as a means of studying the relationship between neuronal functional organization and cognitive behavioral tasks.

Visual area V4 lies in the middle of the ventral visual pathway in the primate brain. It is an intermediate stage in the visual processing for object discrimination. It plays an important role in the neural mechanism of visual attention and shape recognition. V4 neurons exhibit selectivity for salient features of contour conformation. In this paper, we propose a novel model of V4 neurons based on a multilayer neural network inspired by recent studies on V4. Its low-level layers consist of computational units simulating simple cells and complex cells in the primary visual cortex. These layers extract preliminary visual features including edges and orientations. The V4 computational units calculate the entropy of the extracted features as a measure of visual saliency. The salient features are then selected and encoded with a layer of Restricted Boltzmann Machine to generate an intermediate representation of object shapes. The model was evaluated in shape distinction, handwritten digits classification, feature detection, and feature matching experiments. The results demonstrate that this model generates discriminative local representation of object shapes. It provides clues to understand the high level representation of visual stimuli in the brain.

Recognizing an object from its background is always a very challenging task for pattern recognition, especially when the size, pose, or illumination of the object or the background are changing. The most essential method of handling this classical problem is to learn and define the structure of an object using its topological or geometrical features and components. To create a data structure that can describe the spatial relationships of object components formally and join knowledge learning and applying in a seamless loop, a representation platform must be developed. This platform can serve as a shared workspace not only for learning but also for recognition. In this paper, the platform is established by simulating the primary functional modules in the biological primary visual cortex (V1). V1 is located at the middle level of the visual information processing system. As the conjunction of low-level data and high-level knowledge, it performs visual processing for general purposes. Orientation columns in V1 are simulated in our platform, and an array of such columns is designed to represent the orientation features of edges in an image. With this platform, formalized prototypes are designed to represent each typical view of an object and thus the object concept. Data- and concept-driven processing can shift iteratively on this platform. The processes of acquiring knowledge of an object and applying that knowledge coincide with each other perfectly. The experimental results show that our algorithm can learn from a small training set and can recognize the same types of objects in natural background without any preliminary information. This bioinspired representation platform offers a promising prospect for the handling of semantic-concerned problems that need prior knowledge.

Object recognition method based on Geometric characteristics is a key method to solve the visual pattern recognition problem. Contour feature is one of the most important geometric clues. Biological visual cortex can get fragmentary information of the object edges. How to combine the fragments to a longer, more complete contour becomes a key basic problem. Genetic algorithm is usually used to solve combination optimization problems. This paper uses a new kind of gene encoding based on graph structure and an improved algorithm to combine the short line segments. The experimental results show that using the formatting long contour lines can improve the performance and the long contour lines can promote the realization of recognition invariance. Meanwhile, there is no loss of the information and it takes less space to store the images. Large contour features have great significance for the definition of object structured semantics, the explicit definition of the knowledge of the object recognition and realization of the process of top-down processing.

The shape, or contour, of an object is usually stable and persistent, so it is a good basis for invariant recognition. For this purpose, two problems must be addressed. The first is to obtain clean edges, and the second is to organize those edges into a structured data form upon which the necessary manipulations and analysis may be performed. Simple cells in the primary visual cortex are specialized in orientation detection, so the neural mechanism can be simulated by a computational model, which can produce a fairly clean set of lines, and all of them in vectors rather than in pixels. Then a line-context descriptor was designed to describe geometrical distribution of lines in a local area. All lines were also recorded by a weighted graph, and its minimum spanning tree can be used to describe the topological features of an object. An iterative matching algorithm was developed by combining line-context descriptors and minimum spanning tree, and was shown to match objects of the same type but with different shapes very well. Our results suggest that key to representation efficiency of searchable trees is to apply a mid-level line-context. This once more confirms the crucial role played by simple cells in visual processing path, for its preprocessing can greatly ease the subsequent processing.

The vision system of primates could process colorful scenes very efficiently. This is because, in biological retina, there are three types of cone cells and several types of ganglion cells that possess highly complicated receptive fields. The central and the surrounding areas of a receptive field are usually composed of different types of cones. Typically, they form two classes, namely the red-green opponency and the blue-yellow opponency. In order to develop a new representation schema for colorful images, we simulated some physiological mechanisms in retina, such as the opponent color theory. Based on anatomical and electrophysiological findings of ganglion cells, we proposed a bio-inspired color processing method. We designed a neural network simulating retinal ganglion cells (GCs) and their classical receptive fields (CRF), and also raised a dynamic procedure to control receptive field's self-adjustment according to the characteristics of an image. A great number of experiments were conducted on natural images. The results showed that this new method could reserve crucial structural information of an image and suppress trivial information at the same time. Depending on these new representations, some upcoming processing, such as image segmentation, could be improved significantly. Image segmentation is very critical to ultimate image understanding. However, actual image stimuli are a little bit far from biological studies. Our work integrated them together and explained how the physiological opponent-color theory could facilitate image processing in real applications.

Graph theory is a very useful tool in the study of functional and anatomical network in the brain. It had been widely used in the functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) signals. Only very few studies analyzed the neuronal connections composed of individually recorded neurons. Particularly applying in the case of neuronal functional networks of animal behavior-dependent was rare. Scientists have found the small-world network properties in the functional network derived from fMRI and EEG signals. Whether there are existing small-world properties in the neuronal networks of multi-electrode recording? We use graph theory techniques to construct and analyze the neuronal networks. In the functional networks of a simultaneously recorded population of neurons in prefrontal cortex of the rat, in a Y-maze working memory task, we find that the neuronal connection density is highly relevant to rat behavior. We find there is a small-world effect in the neuronal functional network compared to a random graph with the same size and average connection density. We also find that small-world properties have a great relationship to correlation coefficient threshold selection. These findings indicate that neuronal functional networks of multi-electrode recordings are also small-world networks. Network connection topology and connection density are related to the working memory tasks in the rat.

The feedforward model proposed by Hubel and Wiesel partially explained orientation selectivity in simple cells. This classical hypothesis attributed orientation preference to idealized alignment of geniculate cell receptive fields. Many scholars have been either revising this model or putting forward new theories to account for more related phenomenon such as contrast invariant tuning. None of the previous neural models is complete in implementation details or involves strict computational strategies. This paper mathematically studied a detailed but vital question which has long been neglected: the possibility of massive variable-sized, unaligned geniculate cell receptive fields producing the orientation selectivity of a simple cell. The response curve of each afferent neuron is fully utilized to obtain a local constraint and a group-decision making approach is then applied to solve the constraint satisfaction problem. Our new model does not achieve just consistent experimental results with physiological data, but consistent interpretations of several illusions with observers' perceptions. The current work, which supplemented the previous models with necessary computational details, is based on ensemble coding in essence. This underlying mechanism helps to understand how visual information is processed in from the retina to the cortex.

This paper proposes a model for image representation and image analysis using a multi-layer neural network, which is rooted in the human vision system. Having complex neural layers to represent and process information, the biological vision system is far more efficient than machine vision system. The neural model simulate non-classical receptive field of ganglion cell and its local feedback control circuit, and can represent images, beyond pixel level, self-adaptively and regularly. The results of experiments, rebuilding, distribution and contour detection, prove this method can represent image faithfully with low cost, and can produce a compact and abstract approximation to facilitate successive image segmentation and integration. This representation schema is good at extracting spatial relationships from different components of images and highlighting foreground objects from background, especially for nature images with complicated scenes. Further it can be applied to object recognition or image classification tasks in future.