Visual motion detection for extremely tiny (ET-) targets is challenging, due to their category-independent nature and the scarcity of visual cues, which often incapacitate mainstream feature-based models. Natural architectures with rich interpretability offer a promising alternative, where STMD architectures derived from insect visual STMD (Small Target Motion Detector) pathways have demonstrated their effectiveness. However, previous STMD models are constrained to a narrow velocity range, hindering their efficacy in real-world scenarios where targets exhibit diverse and unstable dynamics. To address this limitation, we present vSTMD, a learning-free model for motion detection of ET-targets at various velocities. vSTMD proposes two key mechanisms: cross-Inhibition Dynamic Potential (cIDP) and Collaborative Directional Gradient Calculation (CDGC). Specifically, cIDP serves as a self-adaptive mechanism, efficiently capturing motion cues across a wide velocity spectrum. CDGC enhances orienting accuracy and robustness while reducing computational overhead to one-eighth of previously isolated strategies. Evaluated on the real-world dataset RIST, the proposed vSTMD and its feedback-facilitated variant vSTMD-F achieve relative $F_{1}$ gains of 30% and 58% over state-of-the-art STMD approaches, respectively. Furthermore, both models demonstrate competitive orientation estimation performance compared to SOTA deep learning-driven methods. Experiments also reveal the superiority of the natural architecture for ET-object motion detection - vSTMD is $60\times $ faster than contemporary data-driven methods, making it highly suitable for real-time applications in dynamic scenarios and complex backgrounds. Code is available at https://github.com/MingshuoXu/vSTMD.

Amari’s Dynamic Neural Field (DNF) framework provides a brain-inspired approach to modeling the average activation of neuronal groups. Leveraging a single field, DNF has become a promising foundation for low-energy looming perception module in robotic applications. However, the previous DNF methods face significant challenges in detecting incoherent or inconsistent looming features—conditions commonly encountered in real-world scenarios, such as collision detection in rainy weather. Insights from the visual systems of fruit flies and locusts reveal encoding ON/OFF visual contrast plays a critical role in enhancing looming selectivity. Additionally, lateral excitation mechanism potentially refines the responses of loom-sensitive neurons to both coherent and incoherent stimuli. Together, these offer valuable guidance for improving looming perception models. Building on these biological evidence, we extend the previous single-field DNF framework by incorporating the modeling of ON/OFF visual contrast, each governed by a dedicated DNF. Lateral excitation within each ON/OFF-contrast field is formulated using a normalized Gaussian kernel, and their outputs are integrated in the Summation field to generate collision alerts. Experimental evaluations show that the proposed model effectively addresses incoherent looming detection challenges and significantly outperforms state-of-the-art locust-inspired models. It demonstrates robust performance across diverse stimuli, including synthetic rain effects, underscoring its potential for reliable looming perception in complex, noisy environments with inconsistent visual cues.

Small moving objects at far distance always occupy only one or a few pixels in image and exhibit extremely limited visual features, which bring great challenges to motion detection. Highly evolved visual systems endow flying insects with remarkable ability to pursue tiny mates and prey, providing a good template to develop image processing method for small target motion detection. The insects' excellent sensitivity to small moving objects is believed to come from a class of specific neurons called small target motion detectors (STMDs). However, existing STMD-based methods often experience performance degradation when coping with complex natural scenes. In this paper, we propose a bio-inspired visual system with spatio-temporal feedback mechanism (called Spatio-Temporal Feedback STMD) to suppress false positive background movement while enhancing system responses to small targets. Specifically, the proposed visual system is composed of two complementary subnetworks and a feedback loop. The first subnetwork is designed to extract spatial and temporal movement patterns of cluttered background by neuronal ensemble coding. The second subnetwork is developed to capture small target motion information where its output and signal from the first subnetwork are integrated together via the feedback loop to filter out background false positives in a recurrent manner. Experimental results demonstrate that the proposed spatio-temporal feedback visual system is more competitive than existing methods in discriminating small moving targets from complex natural environments.

Being one of the most fundamental and crucial capacity of robots and animals, autonomous navigation that consists of goal approaching and collision avoidance enables completion of various tasks while traversing different environments. In light of the impressive navigational abilities of insects despite their tiny brains compared to mammals, the idea of seeking solutions from insects for the two key problems of navigation, i.e., goal approaching and collision avoidance, has fascinated researchers and engineers for many years. However, previous bio-inspired studies have focused on merely one of these two problems at one time. Insect-inspired navigation algorithms that synthetically incorporate both goal approaching and collision avoidance, and studies that investigate the interactions of these two mechanisms in the context of sensory-motor closed-loop autonomous navigation are lacking. To fill this gap, we propose an insect-inspired autonomous navigation algorithm to integrate the goal approaching mechanism as the global working memory inspired by the sweat bee's path integration (PI) mechanism, and the collision avoidance model as the local immediate cue built upon the locust's lobula giant movement detector (LGMD) model. The presented algorithm is utilized to drive agents to complete navigation task in a sensory-motor closed-loop manner within a bounded static or dynamic environment. Simulation results demonstrate that the synthetic algorithm is capable of guiding the agent to complete challenging navigation tasks in a robust and efficient way. This study takes the first tentative step to integrate the insect-like navigation mechanisms with different functionalities (i.e., global goal and local interrupt) into a coordinated control system that future research avenues could build upon.

Small target motion detection within complex natural environments is an extremely challenging task for autonomous robots. Surprisingly, the visual systems of insects have evolved to be highly efficient in detecting mates and tracking prey, even though targets occupy as small as a few degrees of their visual fields. The excellent sensitivity to small target motion relies on a class of specialized neurons, called small target motion detectors (STMDs). However, existing STMD-based models are heavily dependent on visual contrast and perform poorly in complex natural environments, where small targets generally exhibit extremely low contrast against neighboring backgrounds. In this article, we develop an attention-and-prediction-guided visual system to overcome this limitation. The developed visual system comprises three main subsystems, namely: 1) an attention module; 2) an STMD-based neural network; and 3) a prediction module. The attention module searches for potential small targets in the predicted areas of the input image and enhances their contrast against a complex background. The STMD-based neural network receives the contrast-enhanced image and discriminates small moving targets from background false positives. The prediction module foresees future positions of the detected targets and generates a prediction map for the attention module. The three subsystems are connected in a recurrent architecture, allowing information to be processed sequentially to activate specific areas for small target detection. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed visual system for detecting small, low-contrast moving targets against complex natural environments.

Collision detection is critical for autonomous vehicles or robots to serve human society safely. Detecting looming objects robustly and timely plays an important role in collision avoidance systems. The locust lobula giant movement detector (LGMD1) is specifically selective to looming objects which are on a direct collision course. However, the existing LGMD1 models cannot distinguish a looming object from a near and fast translatory moving object, because the latter can evoke a large amount of excitation that can lead to false LGMD1 spikes. This article presents a new visual neural system model (LGMD1) that applies a neural competition mechanism within a framework of separated ON and OFF pathways to shut off the translating response. The competition-based approach responds vigorously to monotonous ON/OFF responses resulting from a looming object. However, it does not respond to paired ON-OFF responses that result from a translating object, thereby enhancing collision selectivity. Moreover, a complementary denoising mechanism ensures reliable collision detection. To verify the effectiveness of the model, we have conducted systematic comparative experiments on synthetic and real datasets. The results show that our method exhibits more accurate discrimination between looming and translational events-the looming motion can be correctly detected. It also demonstrates that the proposed model is more robust than comparative models.

Discriminating small moving objects within complex visual environments is a significant challenge for autonomous micro-robots that are generally limited in computational power. By exploiting their highly evolved visual systems, flying insects can effectively detect mates and track prey during rapid pursuits, even though the small targets equate to only a few pixels in their visual field. The high degree of sensitivity to small target movement is supported by a class of specialized neurons called small target motion detectors (STMDs). Existing STMD-based computational models normally comprise four sequentially arranged neural layers interconnected via feedforward loops to extract information on small target motion from raw visual inputs. However, feedback, another important regulatory circuit for motion perception, has not been investigated in the STMD pathway and its functional roles for small target motion detection are not clear. In this article, we propose an STMD-based neural network with feedback connection (feedback STMD), where the network output is temporally delayed, then fed back to the lower layers to mediate neural responses. We compare the properties of the model with and without the time-delay feedback loop and find that it shows a preference for high-velocity objects. Extensive experiments suggest that the feedback STMD achieves superior detection performance for fast-moving small targets, while significantly suppressing background false positive movements which display lower velocities. The proposed feedback model provides an effective solution in robotic visual systems for detecting fast-moving small targets that are always salient and potentially threatening.

Collision detection is one of the most challenging tasks for unmanned aerial vehicles (UAVs). This is especially true for small or micro-UAVs due to their limited computational power. In nature, flying insects with compact and simple visual systems demonstrate their remarkable ability to navigate and avoid collision in complex environments. A good example of this is provided by locusts. They can avoid collisions in a dense swarm through the activity of a motion-based visual neuron called the Lobula giant movement detector (LGMD). The defining feature of the LGMD neuron is its preference for looming. As a flying insect's visual neuron, LGMD is considered to be an ideal basis for building UAV's collision detecting system. However, existing LGMD models cannot distinguish looming clearly from other visual cues, such as complex background movements caused by UAV agile flights. To address this issue, we proposed a new model implementing distributed spatial-temporal synaptic interactions, which is inspired by recent findings in locusts' synaptic morphology. We first introduced the locally distributed excitation to enhance the excitation caused by visual motion with preferred velocities. Then, radially extending temporal latency for inhibition is incorporated to compete with the distributed excitation and selectively suppress the nonpreferred visual motions. This spatial-temporal competition between excitation and inhibition in our model is, therefore, tuned to preferred image angular velocity representing looming rather than background movements with these distributed synaptic interactions. Systematic experiments have been conducted to verify the performance of the proposed model for UAV agile flights. The results have demonstrated that this new model enhances the looming selectivity in complex flying scenes considerably and has the potential to be implemented on embedded collision detection systems for small or micro-UAVs.

There is a growing interest in product aesthetics analytics and design. However, the lack of available large-scale data that covers various variables and information is one of the biggest challenges faced by analysts and researchers. In this paper, we present our multidisciplinary initiative of developing a comprehensive automotive dataset from different online sources and formats. Specifically, the created dataset contains 1.4 million images from 899 car models and their corresponding model specifications and sales information over more than ten years in the UK market. Our work makes significant contributions to: (i) research and applications in the automotive industry; (ii) big data creation and sharing; (iii) database design; and (iv) data fusion. Apart from our motivation, technical details and data structure, we further present three simple examples to demonstrate how our data can be used in business research and applications.

In this paper, a numerical neural network inspired by the lobula plate/lobula columnar type II (LPLC2), the ultra-selective looming sensitive neurons identified within visual system of Drosophila, is proposed utilising non-linear computation. This method aims to be one of the explorations towards solving the collision perception problem resulted from radial motion. Taking inspiration from the distinctive structure and placement of directionally selective neurons (DSNs) named T4/T5 interneurons and their post-synaptic neurons, the motion opponency along four cardinal directions is computed in a non-linear way and subsequently mapped into four quadrants. More precisely, local motion excites adjacent neurons ahead of the ongoing motion, whilst transfers inhibitory signals to presently-excited neurons with slight temporal delay. From comparative experimental results collected, the main contribution is established by sculpting the ultra-selective features of generating a vast majority of responses to dark centroid-emanated centrifugal motion patterns whilst remaining nearly silent to those starting from other quadrants of receptive field (RF). The proposed method also distinguishes relatively dark approaching objects against brighter background and light ones against dark background via exploiting ON/OFF parallel channels, which well fits the physiological findings. Accordingly, the proposed neural network consolidates the theory of non-linear computation in Drosophila's visual system, a prominent paradigm for studying biological motion perception. This research also demonstrates potential of being fused with attention mechanism towards utility in devices such as unmanned aerial vehicles (UAVs), protecting them from unexpected and imminent collision by calculating a safer flying pathway.

The perception of looming objects moving in depth is a basis of artificially dynamic vision system, which has been widely used in robots for autonomous obstacle detection-and-avoidance. How to reliably detect looming objects in chaotic environments is prerequisite, however, still a challenging problem. The current looming perception models or neural networks are greatly affected by input variability on visual contrast between looming object and its background. In this case, the responses of looming detection neurons in animals are robust, which suggests that contrast cues are well encoded in biological visual neural pathways. Considering the physiological homology between Drosophila and locust, this paper draws lessons from the progress of Drosophila physiology to improve the current locust-inspired looming perception model. Two contrast computation schemes herein are proposed: (1) In the early stage of visual processing, the instantaneous feedback mechanism based contrast normalisation dynamically suppress the preliminary motion signals with respect to time. (2) In the later stage of processing, a parallel channel dedicated to calculating local contrast of motion signal is converged to weaken high-contrast signals. Through the comparative tests against many pure and natural scenes, the proposed method works effectively and robustly to reduce fluctuation and variance of response against high input variability on contrast. Here we highlight the effectiveness of temporally dynamic suppression to motion signals in the proposed neural network model, which significantly improves the fidelity of looming perception. This study also shows its competitiveness in the repository of bio-inspired looming perception models.

In nature, crabs have a panoramic vision for the localization and perception of accelerating motion from local segments to global view in order to guide reactive behaviours including escape. The visual neuronal ensemble in crab plays crucial roles in such capability, however, has never been investigated and modelled as an artificial vision system. To bridge this gap, we propose an accelerating motion perception model (AMPM) mimicking the visual neuronal ensemble in crab. The AMPM includes two main parts, wherein the pre-synaptic network from the previous modelling work simulates 16 MLGI neurons covering the entire view to localize moving objects. The emphasis herein is laid on the original modelling of MLGIs' post-synaptic network to perceive accelerating motions from a global view, which employs a novel spatial-temporal difference encoder (STDE), and an adaptive spiking threshold temporal difference encoder (AT-TDE). Specifically, the STDE transforms “time-to-travel” between activations of two successive segments of MLG1 into excitatory post-synaptic current (EPSC), which decays with the elapse of time. The AT-TDE in two directional, i.e., counter-clockwise and clockwise accelerating detectors guarantees “non-firing” to constant movements. Accordingly, the accelerating motion can be effectively localized and perceived by the whole network. The systematic experiments verified the feasibility and robustness of the proposed method. The model responses to translational accelerating motion also fit many of the explored physiological features of direction selective neurons in the lobula complex of crab (i.e. lobula complex direction cells, LCDCs). This modelling study not only provides a reasonable hypothesis for such biological neural pathways, but is also critical for developing a new neuromorphic sensor strategy.

In low altitude flights, powerlines are one of the most dangerous threats for Unmanned Aerial Vehicles (UAVs), as they are widely spread in human society but too thin to be perceived effectively. With the thriving of UAV technology, powerline detection has become a hot issue in numerous researches. Most of the literature, however, extracted powerlines in static images while didn't from the perspective of dynamic scenes. In nature, the motion-sensitive neurons of various insects provide us with ideal examples of perceiving environmental information through the dynamic features of image motion. An outstanding one is the locusts' looming-sensitive neuron, namely the Lobula Giant Movement Detector (LGMD). The locusts being able to fly in a dense swarm without chaos collisions is believed largely owing to the LGMD neuron. Its specialized preference for looming threats and compact morphology inspired a series of robotic researches including the modeling of UAVs' collision detecting systems. However, existing LGMD models are more reactive to large size object movements while not that effective for small ones, making them incompetent for powerline detection. To address the above-mentioned challenges, we propose a neural computational model which fuses a line-attention module with the LGMD model in three different manners, i.e. (a) the preprocessing of input images, (b) processing motion information, and (c) inter-neuron synaptic mappings. Systematic experiments demonstrated that the attention mechanism, especially when discriminating features of image motion in the third manner, excellently extracts visual cues about powerlines and helps to improve collision avoidance performance even in complex backgrounds.

This paper describes a versatile platform for swarm robotics research. It integrates multiple pheromone communication with a dynamic visual scene along with real time data transmission and localization of multiple-robots. The platform has been built for inquiries into social insect behavior and bio-robotics. By introducing a new research scheme to coordinate olfactory and visual cues, it not only complements current swarm robotics platforms which focus only on pheromone communications by adding visual interaction, but also may fill an important gap in closing the loop from bio-robotics to neuroscience. We have built a controllable dynamic visual environment based on our previously developed ColCOSΦ (a multi-pheromones platform) by enclosing the arena with LED panels and interacting with the micro mobile robots with a visual sensor. In addition, a wireless communication system has been developed to allow transmission of real-time bi-directional data between multiple micro robot agents and a PC host. A case study combining concepts from the internet of vehicles (IoV) and insect-vision inspired model has been undertaken to verify the applicability of the presented platform, and to investigate how complex scenarios can be facilitated by making use of this platform.

This paper aims at addressing a challenging problem on reliably estimating image motion against highly variable natural signals which artificial dynamic vision systems are faced with. Previously, the visual system response always represents fluctuation and high variance influenced by spatial contrast, the local difference between neighbouring luminance values. Effective contrast computation is therefore a prerequisite for robust motion vision. In this regard, sighted animals such as flies are remarkably adept at estimating image motion regardless of image statistics by rapidly adjusting contrast sensitivity. Current artificial visual systems, however, cannot account for this capability. Learning from neuroscience, here we propose contrast vision computation to improve a state-of-the-art, bio-inspired neural network model for background motion estimation. This includes mainly two neural computation schemes of (1) an instantaneous, feedback divisive contrast normalisation prior to motion correlation in the ON and OFF pathways to reduce local contrast sensitivity, (2) parallel contrast pathways influencing ON/OFF motion signals, negatively, at the pooling output layer to suppress high-contrast optic flows. We created a dataset of many shifting natural images with high input variability to investigate the proposed method. The experiments have demonstrated the effectiveness and robustness of the proposed contrast vision computation to reduce response fluctuation and variance against natural signals. The fidelity of motion perception thus has been significantly increased. The proposed methods could be generic to other motion vision models dealing with high-contrast visual scenes.

Currently, collision detection methods based on visual cues are still challenged by several factors including ultrafast approaching velocity and noisy signal. Taking inspiration from nature, though the computational models of lobula giant movement detectors (LGMDs) in locust's visual pathways have demonstrated positive impacts on addressing these problems, there remains potential for improvement. In this paper, we propose a novel method mimicking neuronal refractoriness, i.e. the refractory period (RP), and further investigate its functionality and efficacy in the classic LGMD neural network model for collision perception. Compared with previous works, the two phases constructing RP, namely the absolute refractory period (ARP) and relative refractory period (RRP) are computationally implemented through a ‘link (L) layer’ located between the photoreceptor and the excitation layers to realise the dynamic characteristic of RP in discrete time domain. The L layer, consisting of local time-varying thresholds, represents a sort of mechanism that allows photoreceptors to be activated individually and selectively by comparing the intensity of each photoreceptor to its corresponding local threshold established by its last output. More specifically, while the local threshold can merely be augmented by larger output, it shrinks exponentially over time. Our experimental outcomes show that, to some extent, the investigated mechanism not only enhances the LGMD model in terms of reliability and stability when faced with ultra-fast approaching objects, but also improves its performance against visual stimuli polluted by Gaussian or Salt-Pepper noise. This research demonstrates the modelling of refractoriness is effective in collision perception neuronal models, and promising to address the aforementioned collision detection challenges.

Efficient and robust motion perception systems are important pre-requisites for achieving visually guided flights in future micro air vehicles. As a source of inspiration, the visual neural networks of flying insects such as honeybee and Drosophila provide ideal examples on which to base artificial motion perception models. In this paper, we have used this approach to develop a novel method that solves the fundamental problem of estimating angular velocity for visually guided flights. Compared with previous models, our elementary motion detector (EMD) based model uses a separate texture estimation pathway to effectively decode angular velocity, and demonstrates considerable independence from the spatial frequency and contrast of the gratings. Using the Unity development platform the model is further tested for tunnel centering and terrain following paradigms in order to reproduce the visually guided flight behaviors of honeybees. In a series of controlled trials, the virtual bee utilizes the proposed angular velocity control schemes to accurately navigate through a patterned tunnel, maintaining a suitable distance from the undulating textured terrain. The results are consistent with both neuron spike recordings and behavioral path recordings of real honeybees, thereby demonstrating the model's potential for implementation in micro air vehicles which have only visual sensors.

The LGMD1 neuron of locusts shows strong looming-sensitive property for both light and dark objects. Although a few LGMD1 models have been proposed, they are not reliable to inhibit the translating motion under certain conditions compare to the biological LGMD1 in the locust. To address this issue, we propose a bio-plausible model to enhance the collision-selectivity by inhibiting the translating motion. The proposed model contains three parts, the retina to lamina layer for receiving luminance change signals, the lamina to medulla layer for extracting motion cues via ON and OFF pathways separately, the medulla to lobula layer for eliminating translational excitation with neural competition. We tested the model by synthetic stimuli and real physical stimuli. The experimental results demonstrate that the proposed LGMD1 model has a strong preference for objects in direct collision course-it can detect looming objects in different conditions while completely ignoring translating objects.

Building an efficient and reliable collision perception visual system is a challenging problem for future robots and autonomous vehicles. The biological visual neural networks, which have evolved over millions of years in nature and are working perfectly in the real world, could be ideal models for designing artificial vision systems. In the locust's visual pathways, a lobula giant movement detector (LGMD), that is, the LGMD2, has been identified as a looming perception neuron that responds most strongly to darker approaching objects relative to their backgrounds; similar situations which many ground vehicles and robots are often faced with. However, little has been done on modeling the LGMD2 and investigating its potential in robotics and vehicles. In this article, we build an LGMD2 visual neural network which possesses the similar collision selectivity of an LGMD2 neuron in locust via the modeling of biased-ON and -OFF pathways splitting visual signals into parallel ON/OFF channels. With stronger inhibition (bias) in the ON pathway, this model responds selectively to darker looming objects. The proposed model has been tested systematically with a range of stimuli including real-world scenarios. It has also been implemented in a micro-mobile robot and tested with real-time experiments. The experimental results have verified the effectiveness and robustness of the proposed model for detecting darker looming objects against various dynamic and cluttered backgrounds.

Discriminating targets moving against a cluttered background is a huge challenge, let alone detecting a target as small as one or a few pixels and tracking it in flight. In the insect's visual system, a class of specific neurons, called small target motion detectors (STMDs), have been identified as showing exquisite selectivity for small target motion. Some of the STMDs have also demonstrated direction selectivity which means these STMDs respond strongly only to their preferred motion direction. Direction selectivity is an important property of these STMD neurons which could contribute to tracking small targets such as mates in flight. However, little has been done on systematically modeling these directionally selective STMD neurons. In this paper, we propose a directionally selective STMD-based neural network for small target detection in a cluttered background. In the proposed neural network, a new correlation mechanism is introduced for direction selectivity via correlating signals relayed from two pixels. Then, a lateral inhibition mechanism is implemented on the spatial field for size selectivity of the STMD neurons. Finally, a population vector algorithm is used to encode motion direction of small targets. Extensive experiments showed that the proposed neural network not only is in accord with current biological findings, i.e., showing directional preferences but also worked reliably in detecting the small targets against cluttered backgrounds.

With the increasing popularity of social media and smart devices, the face as one of the key biometrics becomes vital for person identification. Among those face recognition algorithms, video-based face recognition methods could make use of both temporal and spatial information just as humans do to achieve better classification performance. However, they cannot identify individuals when certain key facial areas, such as eyes or nose, are disguised by heavy makeup or rubber/digital masks. To this end, we propose a novel deep spiking neural network architecture in this paper. It takes dynamic facial movements, the facial muscle changes induced by speaking or other activities, as the sole input. An event-driven continuous spike-timing-dependent plasticity learning rule with adaptive thresholding is applied to train the synaptic weights. The experiments on our proposed video-based disguise face database (MakeFace DB) demonstrate that the proposed learning method performs very well, i.e., it achieves from 95% to 100% correct classification rates under various realistic experimental scenarios.

Monitoring small objects against cluttered moving backgrounds is a huge challenge to future robotic vision systems. As a source of inspiration, insects are quite apt at searching for mates and tracking prey, which always appear as small dim speckles in the visual field. The exquisite sensitivity of insects for small target motion, as revealed recently, is coming from a class of specific neurons called small target motion detectors (STMDs). Although a few STMD-based models have been proposed, these existing models only use motion information for small target detection and cannot discriminate small targets from small-target-like background features (named fake features). To address this problem, this paper proposes a novel visual system model (STMD+) for small target motion detection, which is composed of four subsystems-ommatidia, motion pathway, contrast pathway, and mushroom body. Compared with the existing STMD-based models, the additional contrast pathway extracts directional contrast from luminance signals to eliminate false positive background motion. The directional contrast and the extracted motion information by the motion pathway are integrated into the mushroom body for small target discrimination. Extensive experiments showed the significant and consistent improvements of the proposed visual system model over the existing STMD-based models against fake features.

Three-dimensional (3D) convolutional networks have been proven to be able to explore spatial context and spectral information simultaneously for super-resolution (SR). However, such kind of network can't be practically designed very `deep' due to the long training time and GPU memory limitations involved in 3D convolution. Instead, in this paper, spatial context and spectral information in hyperspectral images (HSIs) are explored using Two-dimensional (2D) and One-dimenional (1D) convolution, separately. Therefore, a novel 2D-1D generative adversarial network architecture (2D-1D-HSRGAN) is proposed for SR of HSIs. Specifically, the generator network consists of a spatial network and a spectral network, in which spatial network is trained with the least absolute deviations loss function to explore spatial context by 2D convolution and spectral network is trained with the spectral angle mapper (SAM) loss function to extract spectral information by 1D convolution. Experimental results over two real HSIs demonstrate that the proposed 2D-1D-HSRGAN clearly outperforms several state-of-the-art algorithms.

Super-resolution (SR) of hyperspectral images (HSIs) aims to enhance the spatial/spectral resolution of hyperspectral imagery and the super-resolved results will benefit many remote sensing applications. A generative adversarial network for HSIs super-resolution (HSRGAN) is proposed in this paper. Specifically, HSRGAN constructs spectral and spatial blocks with residual network in generator to effectively learn spectral and spatial features from HSIs. Furthermore, a new loss function which combines the pixel-wise loss and adversarial loss together is designed to guide the generator to recover images approximating the original HSIs and with finer texture details. Quantitative and qualitative results demonstrate that the proposed HSRGAN is superior to the state of the art methods like SRCNN and SRGAN for HSIs spatial SR.

Insects use visual information to estimate angular velocity of retinal image motion, which determines a variety of flight behaviours including speed regulation, tunnel centring and visual navigation. For angular velocity estimation, honeybees show large spatial-independence against visual stimuli, whereas the previous models have not fulfilled such an ability. To address this issue, we propose a bio-plausible model for estimating the image motion velocity based on behavioural experiments of the honeybee flying through patterned tunnels. The proposed model contains mainly three parts, the texture estimation layer for spatial information extraction, the delay-and-correlate layer for temporal information extraction and the decoding layer for angular velocity estimation. This model produces responses that are largely independent of the spatial frequency in grating experiments. And the model has been implemented in a virtual bee for tunnel centring simulations. The results coincide with both electro-physiological neuron spike and behavioural path recordings, which indicates our proposed method provides a better explanation of the honeybee's image motion detection mechanism guiding the tunnel centring behaviour.

The robust detection of small targets against cluttered background is important for future artificial visual systems in searching and tracking applications. The insects' visual systems have demonstrated excellent ability to avoid predators, find prey or identify conspecifics - which always appear as small dim speckles in the visual field. Build a computational model of the insects' visual pathways could provide effective solutions to detect small moving targets. Although a few visual system models have been proposed, they only make use of small-field visual features for motion detection and their detection results often contain a number of false positives. To address this issue, we develop a new visual system model for small target motion detection against cluttered moving backgrounds. Compared to the existing models, the small-field and wide-field visual features are separately extracted by two motion-sensitive neurons to detect small target motion and background motion. These two types of motion information are further integrated to filter out false positives. Extensive experiments showed that the proposed model can outperform the existing models in terms of detection rates.

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.

Human beings can achieve reliable and fast visual pattern recognition with limited time and learning samples. Underlying this capability, ventral stream plays an important role in object representation and form recognition. Modeling the ventral steam may shed light on further understanding the visual brain in humans and building artificial vision systems for pattern recognition. The current methods to model the mechanism of ventral stream are far from exhibiting fast, continuous, and event-driven learning like the human brain. To create a visual system similar to ventral stream in human with fast learning capability, in this paper, we propose a new spiking neural system with an event-driven continuous spike timing dependent plasticity (STDP) learning method using specific spiking timing sequences. Two novel continuous input mechanisms have been used to obtain the continuous input spiking pattern sequence. With the event-driven STDP learning rule, the proposed learning procedure will be activated if the neuron receive one pre- or post-synaptic spike event. The experimental results on MNIST database show that the proposed method outperforms all other methods in fast learning scenarios and most of the current models in exhaustive learning experiments.

Pansharpening has been an important technique to increase the spatial resolution of the multispectral (MS) images provided by many earth observation satellites. Since the different spatial resolutions exist between the multispectral and panchromatic (PAN) images, pansharpening usually upsamples the MS images to the same size as the PAN image and then injects the spatial details into the upscaled MS ones. In this paper, we propose a novel pansharpening method focusing on the structure injection into the MS images through a joint Gaussian guided upsampling. The original spectral information is transferred to the joint upsampling outputs by using the hyperspherical color transformation (HCT). The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.

The remote sensing images provided by satellites usually contain complex earth objects with different scales. In the fusion of such images, most of the existing filtering-based pansharpening methods often suffer from spectral and/or spatial information distortions due to the inaccuracy of the detail extraction. Motivated by this, we propose an effective and straightforward multi-scale structure perception pansharpening method, which uses the structure-preserving filter with great structure-aware ability to progressively perceive the structures and accurately extract the details. The experiment is carried out on GeoEye-1 satellite images. Visual and objective analysis show that our method can produce high-quality pansharpened results and outperform some existing methods.

Face is a vital biometric for personal identification. However, the current video-based face recognition methods could not cope with large variances such as heavy makeups, disguised faces with rubber/digital masks or faces with certain areas (eyes, nose, or mouth) invisible. In this paper, we proposed a deep spiking neural network (SNN) architecture with the dynamic facial movements (facial muscle changes caused by speaking) as the sole input for the video-based disguise face recognition application. An event-driven continuous spike-timing dependent plasticity (STDP) learning algorithm with adaptive thresholding has been applied to train the synaptic weights. The proposed video-based disguise face recognition (VDFR) learning method achieves 95% correct classification rate on our proposed video-based disguise face database (MakeFace DB).

The sense and avoid capability enables insects to fly versatilely and robustly in dynamic and complex environment. Their biological principles are so practical and efficient that inspired we human imitating them in our flying machines. In this paper, we studied a novel bio-inspired collision detector and its application on a quadcopter. The detector is inspired from Lobula giant movement detector (LGMD) neurons in the locusts, and modeled into an STM32F407 Microcontroller Unit (MCU). Compared to other collision detecting methods applied on quadcopters, we focused on enhancing the collision accuracy in a bio-inspired way that can considerably increase the computing efficiency during an obstacle detecting task even in complex and dynamic environment. We designed the quadcopter's responding operation to imminent collisions and tested this bio-inspired system in an indoor arena. The observed results from the experiments demonstrated that the LGMD collision detector is feasible to work as a vision module for the quadcopter's collision avoidance task.

In this paper, we proposed a pheromone-based aggregation method based on the state-of-the-art BEECLUST algorithm. We investigated the impact of pheromone-based communication on the efficiency of robotic swarms to locate and aggregate at areas with a given cue. In particular, we evaluated the impact of the pheromone evaporation and diffusion on the time required for the swarm to aggregate. In a series of simulated and real-world evaluation trials, we demonstrated that augmenting the BEECLUST method with artificial pheromone resulted in faster aggregation times.

Pansharpening technology has been an important tool in remote sensing applications. It aims at increasing the spatial resolution of multispectral (MS) image with the aid of panchromatic (PAN) image. A key point of pansharpening is spatial detail extraction and injection. Since MS and PAN images contain objects in different sizes and structures of various scales, scale-sensitive detail extraction is desired. In this paper, we present a scale-aware pansharpening method which uses rolling guidance filter to separate structure from details and injects the details through Gram-Schmidt transformation. The experimental results show that our proposed method can obtain high-quality sharpened results and outperforms some existing methods.

Pansharpening methods generally inject the missing spatial details from a high spatial resolution panchromatic (PAN) image into the corresponding co-registered low spatial resolution multispectral (MS) images while preserving the spectral information. However, most of methods extract the details only from PAN image, which may lead to distortions in the sharpened results. Motivated by this, we present a novel two-stage guided filtering based pansharpening method. In the preliminary stage, an injection model based on multi-channel guidance filtering is designed to keep the spectral fidelity of MS imagery. Then a single channel guidance filtering based injection model is proposed to enhance the details in the second stage. The proposed method is tested and verified by GeoEye-1 satellite images. Qualitative and quantitative analyses demonstrate the superiority of the proposed method compared with some state-of-the-art guided filtering based pansharpening methods.

The nature endows animals robust vision systems for extracting and recognizing different motion cues, detecting predators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In this paper, with respect to recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is filtered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: first, the neural network fulfilled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in vision-based micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been modeled and successfully applied in robotic vision system for perceiving potential collisions in an efficient and reliable manner. In this research, we conduct binocular neuronal models, for the first time combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions of this research: (1) The arena tests involving multiple robots verified the effectiveness and robustness of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models, which fulfill corresponding biological research. (3) The utilized micro robot may also benefit researches on other embedded vision systems as well as swarm robotics.

Automatic traffic sign detection is challenging due to the complexity of scene images, and fast detection is required in real applications such as driver assistance systems. In this paper, we propose a fast traffic sign detection method based on a cascade method with saliency test and neighboring scale awareness. In the cascade method, feature maps of several channels are extracted efficiently using approximation techniques. Sliding windows are pruned hierarchically using coarse-to-fine classifiers and the correlation between neighboring scales. The cascade system has only one free parameter, while the multiple thresholds are selected by a data-driven approach. To further increase speed, we also use a novel saliency test based on mid-level features to pre-prune background windows. Experiments on two public traffic sign data sets show that the proposed method achieves competing performance and runs 2~7 times as fast as most of the state-of-the-art methods.

All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.

Pansharpening has been an important tool in remote sensing field, which is a process of providing multispectral images with higher spatial resolution. When dealing with WorldView-2 satellite imagery having more bands and higher resolution, most existing methods are not effective. In this paper, we propose a novel and effective pansharpening methods combing guided filtering and hyperspherical color transformation (HCT) for WorldView-2 images. We use panchromatic image as the guidance to further refine the intensity of multispectral data and also to extract the sufficient details from the panchromatic image itself. Moreover, the guided filtering and HCT integrated scheme can inject the extracted details into the multispectral data and the multispectral images can be sharpened all at once with an arbitrary order. The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.

Neuroscience study shows mammalian brain only use millisecond scale time window to process complicated real-life recognition scenarios. However, such speed cannot be achieved by traditional rate-based spiking neural network (SNN). Compared with spiking rate, the specific spiking timing (also called spiking pattern) may convey much more information. In this paper, by using modified rank order coding scheme, the generated absolute analog features have been encoded into the first spike wave with specific spatiotemporal structural information. An intuitive yet powerful feed-forward spiking neural network framework has been proposed, along with its own unsupervised spike-timing-dependent plasticity (STDP) learning rule with dynamic post-synaptic potential threshold. Compared with other state-of-art spiking algorithms, the proposed method uses biologically plausible STDP learning method to learn the selectivity while the dynamic post-synaptic potential threshold guarantees no training sample will be ignored during the learning procedure. Furthermore, unlike the complicated frameworks used in those state-of-art spiking algorithms, the proposed intuitive spiking neural network is not time-consuming and quite capable of on-line learning. A satisfactory experimental result has been achieved on classic MNIST handwritten character database.

In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.

An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.

Most functional reaching activities in daily living generally require a hand to reach the functional position in appropriate orientation with invariant spatiotemporal profile. Effectively reproducing such spatiotemporal feature of hand configuration trajectory in real time is essential to understand the human motor control and plan human-like motion on anthropomorphic robotic arm. However, there are no novel computational models in literature toward reproducing hand configuration-to-configuration movement in spatiotemporal profile. In response to the problem, this paper presents a computational framework for hand configuration trajectory formation based on hierarchical principle of human motor control. The composite potential field is constructed on special Euclidean Group to induce time-varying configuration toward target. The dynamic behavior of hand is described by a second-order kinematic model to produce the external representation of high-level motor control. The multivariate regression relation between intrinsic and extrinsic coordinates of arm, is statistically analyzed for determining the arm orientation in real time, which produces the external representation of low-level motor control. The proposed method is demonstrated in an anthropomorphic arm by performing several highly curved self-reaching movements. The generated configuration trajectories are compared with actual human movement in spatiotemporal profile to validate the proposed method.

How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand.

Pheromone-based communication is one of the most effective ways of communication widely observed in nature. It is particularly used by social insects such as bees, ants and termites; both for inter-agent and agent-swarm communications. Due to its effectiveness; artificial pheromones have been adopted in multi-robot and swarm robotic systems for more than a decade. Although, pheromone-based communication was implemented by different means like chemical (use of particular chemical compounds) or physical (RFID tags, light, sound) ways, none of them were able to replicate all the aspects of pheromones as seen in nature. In this paper, we propose a novel artificial pheromone system that is reliable, accurate and it uses off-the-shelf components only - LCD screen and low-cost USB camera. The system allows to simulate several pheromones and their interactions and to change parameters of the pheromones (diffusion, evaporation, etc.) on the fly allowing for controllable experiments. We tested the performance of the system using the Colias platform in single-robot and swarm scenarios. To allow the swarm robotics community to use the system for their research, we provide it as a freely available open-source package.

We propose a cascade method for fast and accurate traffic sign detection. The main feature of the method is that mid-level saliency test is used to efficiently and reliably eliminate background windows. Fast feature extraction is adopted in the subsequent stages for rejecting more negatives. Combining with neighbor scales awareness in window search, the proposed method runs at 3~5 fps for high resolution (1360×800) images, 2~7 times as fast as most state-of-the-art methods. Compared with them, the proposed method yields competitive performance on prohibitory signs while sacrifices performance moderately on danger and mandatory signs.

The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments.

In this paper, we have proved that in every underdetermined linear system Ax = b, there corresponds a constant p*(A, b) > 0 such that every solution to the l p-norm minimization problem also solves the l0-norm minimization problem whenever 0 <; p <; p*(A, b). This phenomenon is named NP/CMP equivalence.

Nowadays, motion attention model is a controversial topic in the biological computer vision area. The computational attention model can be decomposed into a set of features via predefined channels. Here we designed a bio-inspired vision attention model, and added the rarity measurement onto it. The priority of rarity is emphasized under the assumption of weighting effect upon the features logic fusion. At this stage, a final saliency map at each frame is adjusted by the spatiotemporal and rarity values. By doing this, the process of mimicking human vision attention becomes more realistic and logical to the real circumstance. The experiments are conducted on the benchmark dataset of static images and video sequences. We simulated the attention shift based on several dataset. Most importantly, our dynamic scenes are mostly selected from the objects moving on the highway and dynamic scenes. The former one can be developed on the detection of car collision and will be a useful tool for further application in robotics. We also conduct experiment on the other video clips to prove the rationality of rarity factor and feature cues fusion methods. Finally, the evaluation results indicate our visual attention model outperforms several state-of-the-art motion attention models.

A binary Self Organizing Map (SOM) has been designed and implemented on a Field Programmable Gate Array (FPGA) chip. A novel learning algorithm which takes binary inputs and maintains tri-state weights is presented. The binary SOM has the capability of recognizing binary input sequences after training. A novel tri-state rule is used in updating the network weights during the training phase. The rule implementation is highly suited to the FPGA architecture, and allows extremely rapid training. This architecture may be used in real-time for fast pattern clustering and classification of binary features.

The sparse distributed memory (SDM) proposed by Kanerva provides a simple model for human long-term memory, with a strong underlying mathematical theory. However, there are problematic features in the original SDM model that affect its efficiency and performance in real world applications and for hardware implementation. In this paper, we propose modifications to the SDM model that improve its efficiency and performance in pattern recall. First, the address matrix is built using training samples rather than random binary sequences. This improves the recall performance significantly. Second, the content matrix is modified using a simple tri-state logic rule. This reduces the storage requirements of the SDM and simplifies the implementation logic, making it suitable for hardware implementation. The modified model has been tested using pattern recall experiments. It is found that the modified model can recall clean patterns very well from noisy inputs.

The lobula giant movement detector (LGMD) is a wide-field visual neuron that is located in the lobula layer of the locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement.

Reliably recognizing objects approaching on a collision course is extremely important. A synthetic vision system is proposed to tackle the problem of collision recognition in dynamic environments. The system combines the outputs of four whole-field motion-detecting neurons, each receiving inputs from a network of neurons employing asymmetric lateral inhibition to suppress their responses to one direction of motion. An evolutionary algorithm is then used to adjust the weights between the four motion-detecting neurons to tune the system to detect collisions in two test environments. To do this, a population of agents, each representing a proposed synthetic visual system, either were shown images generated by a mobile Khepera robot navigating in a simplified laboratory environment or were shown images videoed outdoors from a moving vehicle. The agents had to cope with the local environment correctly in order to survive. After 400 generations, the best agent recognized imminent collisions reliably in the familiar environment where it had evolved. However, when the environment was swapped, only the agent evolved to cope in the robotic environment still signaled collision reliably. This study suggests that whole-field direction-selective neurons, with selectivity based on asymmetric lateral inhibition, can be organized into a synthetic vision system, which can then be adapted to play an important role in collision detection in complex dynamic scenes.

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the images of an approaching object such as a predator. Its computational model can cope with unpredictable environments without using specific object recognition algorithms. In this paper, an LGMD-based neural network is proposed with a new feature enhancement mechanism to enhance the expanded edges of colliding objects via grouped excitation for collision detection with complex backgrounds. The isolated excitation caused by background detail will be filtered out by the new mechanism. Offline tests demonstrated the advantages of the presented LGMD-based neural network in complex backgrounds. Real time robotics experiments using the LGMD-based neural network as the only sensory system showed that the system worked reliably in a wide range of conditions; in particular, the robot was able to navigate in arenas with structured surrounds and complex backgrounds.

The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. A computational neural network model based on the structure of the LGMD and its afferent inputs is also able to detect approaching objects. In order for the LGMD network to be used as a robust collision detector for robotic applications, we proposed a new mechanism to enhance the feature of colliding objects before the excitations are gathered by LGMD cell. The new model favours grouped excitation but tends to ignore isolated excitation with selective passing coefficients. Experiments with a Khepera robot showed the proposed collision detector worked in real time in an arena surrounded with blocks.

It is difficult for robots to handle a vibrating deformable object. Even for human beings it is a high-risk operation to, for example, insert a vibrating linear object into a small hole. However, fast manipulation using a robot arm is not just a dream; it may be achieved if some important features of the vibration are detected online. We present an approach for fast manipulation using a force/torque sensor mounted on the robot's wrist. A template matching method is employed to recognize the vibrational phase of the deformable objects. Therefore, a fast manipulation can be performed with a high success rate, even if there is acute vibration. Experiments inserting a deformable object into a hole are conducted to test the presented method. Results demonstrate that the presented sensor-based online fast manipulation is feasible.