Abstract:In the last years, smart surveillance has been one of the most active research topics in computer vision because of the wide spectrum of promising applications. Its main point is about the use of automatic video analysis technologies for surveillance purposes. In general, a processing framework for smart surveillance consists of a preliminary motion detection step in combination with high‐level reasoning that allows automatic understanding of evolutions of observed scenes. In this paper, we propose a surveillance framework based on a set of reliable visual algorithms that perform different tasks: a motion analysis approach that segments foreground regions is followed by three procedures, which perform object tracking, homographic transformations and edge matching, in order to achieve the real‐time monitoring of forbidden areas and the detection of abandoned or removed objects. Several experiments have been performed on different real image sequences acquired from a Messapic museum (indoor context) and the nearby archaeological site (outdoor context) to demonstrate the effectiveness and the flexibility of the proposed approach.

In the last decade, soccer video analysis has received a lot of attention from the scientific community. This increasing interest is motivated by the possible applications over a wide spectrum of topics: indexing, summarization, video enhancement, team and players statistics, tactics analysis, referee support, etc. The application of computer vision methodologies in the soccer context requires many problems to be faced: ball and players have to be detected in the images in any light and weather condition, they have to be localized in the field, tracked over time and finally their interactions have to be detected and analyzed. The latter task is fundamental, especially for statistic and referee decision support purposes, but, unfortunately, it has not received adequate attention from the scientific community and a lot of research remains to be done. In this paper a multicamera system is presented to detect the ball player interactions during soccer matches. The proposed method extracts, by triangulation from multiple cameras, the 3D ball and player trajectories and, by estimating the trajectory intersections, detects the ball-player interactions. An inference process is then introduced to determine the player kicking the ball and to estimate the interaction frame. The system was tested during several matches of the Italian first division football championship and experimental results demonstrated that the proposed method is robust and accurate.

The railway maintenance is a particular application context required in order to prevent any dangerous situation. With the growing of the high-speed railway traffic, automatic inspection systems able to detect rail defects, sleepers' anomalies, as well as missing fastening elements, become strategic since they could increase the ability in the detection of defects and reduce the inspection time in order to guarantee more frequent maintenance of the railway network. This paper presents a patented fully automatic and configurable real-time vision system able to detect the presence/absence of the fastening bolts that fix the rails to the sleepers. It gets an accuracy of 99.9%, and, thanks to its parallel processing allowed by a Graphic Processing Unit, reaches an average throughput of 187 km/h, speeding up of about 287% the performance of a quadcore CPU implementation.

Passive Radio Frequency Identification (RFID) is being increasingly used in mobile robotics applications, as it provides inexpensive and effective solutions to data association issues in basic navigation tasks. Nonetheless, problems related to sensitivity of the signal to interference and reflections, and missing tag range and bearing information are open. In this paper, we propose a novel approach to passive RFID, which tackles those issues using fuzzy reasoning. Specifically, first, we present a fuzzy antenna model. Then, based on this model, we describe two fuzzy logic methods for tag localization. One allows us to accurately localize passive tags in the environment and to generate what we call an RFID-augmented map; the other is suited for estimating the bearing of a tag relative to the robot. The general use of both methods is in object localization, map building, environment monitoring, and robot pose estimation. Results of experimental tests demonstrate that fuzzy logic is appropriate to operate under uncertainty in RFID systems, and allows for accurate tag localization.

In this paper, we investigate on the feasibility of multiple camera system for automatic offside detection. We propose six fixed cameras, properly placed on the two sides of the soccer field (three for each side) to reduce perspective and occlusion errors. The images acquired by the synchronized cameras are processed to detect the players' position and the ball position in real-time; a multiple view analysis is carried out to evaluate the offside event, considering the position of all the players in the field, determining the players who passed the ball, and determining if active offside condition occurred. The whole system has been validated using real-time images acquired during official soccer matches, and quantitative results on the system accuracy were obtained comparing the system responses with the ground truth data generated manually on a number of extracted significant sequences.

Rail inspection is a very important task in railway maintenance, and it is periodically needed for preventing dangerous situations. Inspection is operated manually by trained human operator walking along the track searching for visual anomalies. This monitoring is unacceptable for slowness and lack of objectivity, as the results are related to the ability of the observer to recognize critical situations. The correspondence presents a patent-pending real-time Visual Inspection System for Railway (VISyR) maintenance, and describes how presence/absence of the fastening bolts that fix the rails to the sleepers is automatically detected. VISyR acquires images from a digital line-scan camera. Data are simultaneously preprocessed according to two discrete wavelet transforms, and then provided to two multilayer perceptron neural classifiers (MLPNCs). The "cross validation" of these MLPNCs avoids (practically-at-all) false positives, and reveals the presence/absence of the fastening bolts with an accuracy of 99.6% in detecting visible bolts and of 95% in detecting missing bolts. A field-programmable gate array-based architecture performs these tasks in 8.09 mus, allowing an on-the-fly analysis of a video sequence acquired at 200 km/h

This paper describes a technological solution for the remote fruition of an archaeological site, a cave hosting pre-historical heritage of relevant importance, using a properly equipped mobile robot. The goal is twofold: to increase the fruition of these precious paintings (both off-line and on-line with the robot moving in the cave) and to collect data (about the two-dimensional map of the complex cave and about the three-dimensional structure of the wall hosting the paints) that can be important to increase the understanding of the environment by the experts. The choice of the robot is suitable for this task before it does need a very light and minimal installation of mobile and reversible infrastructures: in fact in such an environment any intervention can cause significant damages to the archaeological site. The current modules are able to explore the cave to determine the most interesting zones (with a richer content of pre-historical wall paintings), to collect three-dimensional data (using a laser range finder) that allow the reconstruction of a planar map of the cave, the navigation between these relevant zones, the fine positioning with respect to the paintings and the reconstruction of a full three-dimensional model of the walls hosting the paints starting from a set of three images acquired from slightly different points of view.

In this work the complex behavior of localizing a mobile vehicle with respect to the door of the environment and then reaching the door has been developed. The robot uses visual information to detect and recognize the door and to determine its state with respect to it. This complex task has been divided into two separate behaviors: door-recognition and door-reaching. A supervised methodology based on learning by components has been applied for recognizing the door. Learning by components allows to recognize the door also in difficult situations such as partial occlusions and besides, it makes recognition independent of viewpoint variations and scale changes. An unsupervised methodology based on reinforcement learning has been used for the door-reaching behavior, instead. The image of the door gives information about the relative position of the vehicle with respect to the door. Then the Q-learning algorithm is used to generate the optimal state-action associations. The problem of defining the state and the action sets has been addressed with the aim of producing smooth paths, of reducing the effects of visual errors during real navigation, and of keeping low the computational cost during the learning phase. A novel way to obtain a continuous action set has been introduced: it uses a fuzzy model to evaluate the system state. Experimental results in real environment show both the robustness of the door-recognition behavior and the generality of the door-reaching behavior.

The paper presents a fast and reliable approach to estimate body postures in outdoor visual surveillance. It works on patches corresponding to people, recognized by two subsystems (motion detection and object recognition) on image sequences coming from a still camera. The proposed algorithm is based on an unsupervised clustering approach and is substantially independent from a-priori assumption about the possible output postures. Horizontal and vertical histograms of the binary shapes associated to humans are selected as features. The Manhattan distance is used for building clusters and for run-time classification. After experimental tests the BCLS (Basic Competitive Learning Scheme) algorithm has been selected for the construction of clusters. The whole approach has been verified on real sequences acquired while typical illegal activities involved in stealing were simulated in an archeological site.

The Q-learning algorithm, for its simplicity and well-developed theory, has been largely used in the last years in order to realize different behaviors for autonomous vehicles. The most frequent applications required the standard tabular formulation with discrete sets of state and action. In order to consider continues variables, function approximators such as neural networks are required. In this work we investigate the neural approach of Q-learning on the robot navigation task of wall following. Some issues have been addressed in order to deal with the convergence problem and the need of huge training sets. The experience replay paradigm has been also applied to reduce the unlearning problem. Two different neural network architectures, which use different spatial decompositions of the sensory input, have been compared. The aim is to investigate how different choices of architecture can affect the learning convergence, the optimality of the final controller and the generalization ability.

The main aim of this work is to present a simple and reliable approach to the estimation of human body postures. The applicative context is the visual surveillance of an archaeological site. Motion detection and object recognition subsystems process image sequences coming from a still camera. Whenever a human is detected, his postures are characterized by the proposed pose estimation module. Then the results are fed to a HMM subsystem that identifies the current activity of the examined subject The proposed algorithm is based on an unsupervised clustering approach that makes the system substantially independent from any a-priori assumption about the possible output postures. The features selected for posture estimation are the horizontal and vertical histograms of binary shapes. A modified version of the Manhattan distance is used for both cluster identification and for run-time classification. After extensive experimental tests with different clustering schema, the BCLS algorithm (basic competitive learning scheme) has been selected. The proposed approach makes possible to change the number of classes, during the classification phase, without repeating the training step. Moreover it provides a measure of the reliability of its results. The proposed method has been verified on sequences acquired while typical illegal activities involved in stealing were simulated in a real archaeological site.

This paper presents a vision-based technique for detecting targets of the environment which have to be reached by an autonomous mobile robot during its navigational tasks. The targets the robot has to reach are the doors of the authors' office building. The detection of the door has been performed by detecting its most significant components in the image and it is based on data classification. Two neural classifiers have been trained for recognizing single components of the door. Then a combining algorithm, based on heuristic considerations, checks that they are in the proper geometric configuration of the structure of the door. The novelty of this work is to use together colour and shape information for identifying features and for detecting the components of the target. The approach, based on learning by components, is able to cleverly solve the problems of scale changes, perspective variations and partial occlusions. The obtained detecting system has been tested on a large test set of real images showing a high reliability and robustness: doors of different rooms, under different illumination conditions and by different viewpoints have been successfully recognized. Results in terms of door detection rate and false positive rate are presented throughout the paper.

In this paper we describe a people tracking system addressing the visual surveillance of outdoor environments. This context involves the detection of objects moving in a sequence of images acquired by a TV camera and the recognition and tracking of human figures to allow the analysis of their gestures. The proposed tracking approach does not consider the human body parts as features to be tracked, nor uses a priori body models. Moreover, also classical prediction models are useless in the specific context (visual surveillance of archaeological sites) because the behaviors identified as illegal imply a sequence of unpredictable gestures performed in a small area. The proposed method is based on the estimation of similarity between human figures segmented in the temporal sequence. The similarity score is estimated as a function of the number of corresponding feature points. The system has been verified on real image sequences acquired by a static TV camera while the gestures normally performed by intruders were simulated in a real archaeological site.

Our application context is the visual surveillance of archeological sites. In this context the main aim is to detect the presence of people and to scan them in order to recognize intruders on the basis of their gestures. Since an intruder needs some utensils indispensable to perform the illegal actions of excavating on the ancient ruins, intruder detection involves first of all to ascertain if a person is carrying some objects and then recognizing the kind. In this paper we concentrate on the recognition of the objects carried by the detected moving persons. An example-based learning technique is used to first detect people and successively to scan them to recognize the possible carried objects. The patterns to be analysed are represented through the approximation coefficients of their three level wavelet decomposition. Pattern classification is performed through a supervised three layer neural network.

Inspection of the rail state in order to detect defects is one of the basic tasks in railway maintenance. Rail defects exhibit different properties and are divided in various categories relating to the type and position of flaws on the rail. We propose a technique, based on texture analysis of the rail surface, to detect and classify a particular class of defects: corrugation.

Realizing complex robotic tasks by combining elementary behaviors is simple, robust and effective. Learning controllers from examples avoids manual identification of system states and actions and direct definition of their relationships. Fuzzy controllers represent direct associations between input and output variables, typical of reactive behaviors, in a form understandable by both automatic tools and humans. The paper describes the application of a set of tools developed for supporting the automatic design of fuzzy controllers (recording of associations between control commands and system states, identification of the required controller's components). Safe navigation of a real vehicle in an unknown indoor environment is addressed by combining two behaviors, wall-following and door-crossing, using a third module for judging their competence in each situation. Its performance, driving a Scout in our offices, is shown and related to the limitations of the two basic controllers.

We present a learning algorithm that realizes a simple goal-reaching task for an autonomous vehicle when only visual information about the goal is provided. The robot has to reach a door from every position of the environment. The state of the system is based on visual information received by a TV camera placed on the mobile robot. The vision algorithm is able to determine the relative position between the vehicle and the door according to the slopes of the contour lines of the door. A learning phase is carried out in simulation to obtain the optimal state-action rules. The learned knowledge is then transferred on the real robot for the testing phase. Experimental results show the generality of the knowledge learned as the real robot is always able to execute its paths towards the door in different environments.

The goal of this work is to propose a method to solve the problem of underwater navigation of an autonomous vehicle with visual means. We refer to the ability of an autonomous system to determine its motion with respect to the environment. The method we propose is a two stage approach: matching of features extracted from 2D images acquired at different times and egomotion parameter computation. Both algorithms are based on optimization approaches minimizing appropriate energy functions. The matching stage recovers correspondences between sparse high interest feature points of two successive images useful to perform the second stage of egomotion parameter estimation. Experimental results obtained in real context show the robustness of the method.

This paper deals with oriented texture analysis applied to a problem of defect detection and classification in leather inspection, for industrial application. In this paper we present results obtained using a defects detector based on oriented texture analysis, which reveals itself useful for a few classes of leather defects, such as scars or folds. These defects can be detected by using a black and white camera running over the leather patch and by classifying textures, on the basis of their gradient orientations and local coherence. Leather defects are detected by segmenting the oriented texture map of the leather surface. The orientation in every point is expressed using a normalized B-spline basis whose knots are distributed on the leather surface. The resulting erector of coefficients can be used to perform the final segmentation and localize defective areas. A morphological segmentation procedure is applied to the regularized oriented texture field in order to extract probable defective areas.

Sparse optical flow estimated by an image sequence, often, is enough to obtain useful information about camera motion. Normally, matching among features on image sequence is estimated by radiometric similarity. When the camera is moving on a flat surface (planer navigation) additional constraints can be introduced in order to obtain a more robust estimation of the optical flow. In this paper, an energy based optimization method to compute a robust sparse optical flow using global constraints induced by camera motion, is proposed. Convergence is reached in few iterations, making the proposed method suitable for real-time applications.

The main aim of this paper is to describe an efficient approach for egomotion parameters estimation in order to solve the passive navigation problem. Passive navigation is the ability of an autonomous agent to determine its motion with respect to the environment. The two main egomotion parameters allowing to perform passive navigation are the heading direction and the time to collision with the environment. A lot of approaches have been proposed in literature in order to estimate the above parameters, most of which work well only if the motion is a predominant translation forward or small amounts of noise are present in the input data. In our work we propose a method computing in real time, using a least square approach, the egomotion parameters from same robust feature correspondences. The approach we propose is based on the observation that the radial structures of the 2D motion field generated by passive navigation can be approximated by a 2D motion field corresponding to the projection on the image plane of a translational motion on both X horizontal axis and Z axis. The aim is to show that the heading direction can be computed correctly from translational motion parameters, though large rotations are performed by the vehicle.

This paper deals with the problem of recovering directionality from compositional textures. It presents a new method that has proved to be effective for detection and classification of leather defects in the manufacturing industry. A flow field, estimated from texture, is represented as a set of projection coefficients onto suitable elementary basis vector fields. Since these bases are neither orthogonal nor complete, the coefficients must be estimated using a global optimization criterion. The method performs this optimization using a neural network. A region growing approach, based on single iterations of the same neural network, expands patches corresponding to oriented structures and produces the final segmentation map.

Goal-oriented navigation of a mobile robot by landmark based techniques is a straightforward and suitable approach. Most of the methods would normally give an estimation of the position and orientation of the vehicle, but often they are not able to provide a good estimate of the uncertainty in the measurement. That information is useful in application where multisensor fusion is requested. Our method permits to determine the vehicle location and relative uncertainty, when its orientation is obtained by a heading sensor, using a visual landmark based method. The approach is straightforward and suitable for real time performance on general purpose hardware. Experimental results are provided by implementation on our autonomous mobile vehicle SAURO.

Fuzzy logic is a useful tool for realizing a direct mapping between perceptual situations and control commands in robotic applications that do not require internal representation or planning. It allows explicit programming and automatic learning from suitable training data to be mixed in several ways in order to produce the necessary control rules. Automatic learning enables both the reduction of the annoying and error-prone explicit programming work and the evolution in time of the controller in order to cope with dynamically changing environments. At any time an expert can verify and eventually modify the knowledge of the fuzzy system according to its personal experience. Two methods for the automatic extraction of rules from training data have been proved on a fuzzy control system for wall-following. The system is intended to grow including all the behaviors required for the safe navigation of a autonomous mobile vehicle and their arbitration. Distance measures supplied by an ultrasonic sensor ring have been chosen as sensory data. The training data have been collected during operator-driven runs of the vehicle. The same data have been used by the two methods for building rule bases that have proved effectively in driving a real AGV (a TRC Labmate base) in an indoor environment.

The goal of this work is to propose a method to solve the problem of passive navigation with visual means. The method is a two stage approach: matching of feature extracted from 2D images of a sequence at different times and egomotion parameter computation. Both algorithms are based on a least-square-error technique to minimize appropriate energy functions. Experimental results obtained in real context show the robustness of the method.

In this paper a neural algorithm for defect detection in industrial inspection is proposed. One of the most difficult problems in process control and automated inspection is the identification, description and classification of surface defects and anomalies. A critical role in surface inspection is played by texture because most of the defects are rich in textural content. The goal of this work is to propose a quantitative and qualitative technique to detect surface defects with oriented texture structure. The neural algorithm, that the authors propose to classify an oriented flow field, can classify each kind of defect with textural characteristics: once the system has been set to recognize a limited number of patterns, it is able to identify and analyze a broader family of patterns. To the surface image to be analyzed is associated a vector field which computes the dominant local orientations of the gradients of the image smoothed with a Gaussian filter. The different defective surface regions are recovered and classified minimizing an energy function by means of a neural network. Experimental results of tests performed on ferromagnetic surfaces show as the proposed neural at algorithm works.

The development of new techniques in sequencing nuclei acids has produced a great amount of sequence data and has led to the discovery of new relationships. In this paper, we study a method for parallelizing the algorithm WORDUP, which detects the presence of statistically significant patterns in DNA sequences. WORDUP implements an efficient method to identify the presence of statistically significant oligomers in a non-homologous group of sequences. It is based on a modified version of the Boyer-Moore algorithm, which is one of the fastest algorithms for string matching available in the literature. The aim of the parallel version of WORDUP presented here is to speed up the computational time and allow the analysis of a greater set of longer nucleotide sequences, which is usually impractical with sequential algorithms.

This paper is concerned with the probleni of determining the position of the vehicle both in goal-oriented and free- path navigations. A vision system for self location of a mobile robot has been devel- oped. By means of few known landmarks, the system has the ability of continuously monitoring position and compare these es- timates with the measures provided by the odometers. Besides a procedure has been im- plemented that recovers the position of the vehicle when the other sensors completely fail.

Presents an approach to detect moving obstacles in order to define a collision zone around them. The approach uses a camera system to provide image sequences from which the dynamic aspect of the scene can be recovered. In fact, if the camera samples the observation space at some time intervals, a two-dimensional image sequence may be created, due to the relative motion. The analysis of these sequences can be useful to recovery information about the relative notion between a mobile vehicle and the navigation space. The appearance of an obstacle can be recovered using optical flow analysis. The authors compute optical flow maps to solve the problem of detecting discontinuities in the flow field in order to segment the image in different moving objects. An estimation of depth values for the segmented points is accomplished by means of optical flow and focus of expansion. Experimental results on image sequences of a real scene are presented.>

Recovering three-dimensional visible surfaces is an important cue in computer vision. Like most of the inverse problems it is hard to be solved due to the insufficient and inaccurate data that can be collected using passive sensors. Data fusion and integration over time can be used to overcome this problem. In this paper the depth and orientation are acquired from a scene and are used together to build a dense map of the visible surface for each of several points of view. An incremental estimator is used to integrate these maps, as soon as they become available, in a more reliable result.>