In this paper, we focus on the field of maritime simulation games, also known as serious games or simulators, which serve as a vital tool for maritime education and training. These simulations offer a controlled and risk-free platform to mimic real-world situations, thereby aiding seafarers in learning essential skills such as ship maneuverability, collision prevention, and understanding other naval agents’ intentions. We present an implementation of a Unity-based naval simulator that enables the development of complex, multi-vessel navigation scenarios and provides multiple key capabilities relevant to the naval domain. First, the agent vessels are equipped with mathematical models to assess their capabilities and vulnerabilities. Second, a vulnerability heatmap is developed to illustrate the sensor and defense coverage of an agent or a group of agents. Third, a Closest Point of Approach (CPA) based action heatmap is developed to explore potential threats from the surrounding agents. Furthermore, these heatmaps are fused into a threat heatmap that encodes in real time an agent’s overall coverage and potential threats. In addition, vessel agents are equipped with Hidden Markov Model-based intent recognition models, to analyze the behavior of other agents around them. This paper describes the naval simulator with its capabilities and illustrates its main capabilities in various naval scenarios.

In this paper, a modification of steady state genetic algorithm, called dual-population scheme, is proposed to improve its execution speed on electronic hardware. It utilizes two memories to store two interactive populations on them. The system, inherently interchanges chromosomes between these populations. In this manner, it can fully benefit from the pipeline processing on hardware. It is shown that the proposed method performs much faster than the standard steady state and canonical genetic algorithms on pipelined genetic hardware. Moreover, the searching performance, repeatability and convergence properties of the proposed technique were tested. They show dual-population scheme performs similarly to the regular genetic algorithms while achieves better results than the present hardware-oriented genetic algorithm models.

The ability to detect human actions or gestures is key for a wide range of applications that involve interactions between humans and robots. These actions are patterns that have a particular spatio-temporal structure. This paper presents an approach for encoding such patterns using spike-timing networks with axonal conductance delays. The proposed method brings the following contributions: first, it enables the encoding of patterns in an unsupervised manner. Second, it allows us to create models of specific patterns using a very small set of training samples, in contrast with standard pattern recognition approaches that typically require large amounts of training data. Based on these models, the method further enables classification of new patterns using a longest-common subsequence approach for matching between patterns of activated neurons. Third, the approach is invariant to scale and translation and thus it enables generalization across multiple scales and positions. Fourth, the approach also enables early recognition of patterns from only partial information about the pattern. The proposed method is validated on a set of gestures representing the digits from 0 to 9, extracted from video data of a human drawing the corresponding digits. The results are also compared with other state of the art pattern recognition algorithms.

This paper investigates three approaches to the problem of identity recognition in real-world unconstrained environments. We describe a new and challenging face recognition dataset captured in a laboratory environment with no strong constraints on lighting, motion, or subject pose, orientation, distance, or facial expression. We then evaluate three approaches to identity recognition on this new dataset. We find that a deep neural network with stacked denoising auto-encoders significantly outperforms a standard feedforward neural network and a baseline eigenfaces approach from the OpenCV library. Despite the 66 million plus parameters in the best trained deep network, it significantly outperforms the other two methods even on the relatively small number (relative to the number of deep network parameters) of 8,895 training samples. We believe our work adds to the growing empirical and theoretical evidence that deep networks provide a promising approach to unconstrained recognition problems.

In this paper, we propose a new approach for recognizing intentions of humans by observing their activities with an RGB-D camera. Activities and goals are modeled as a distributed network of inter-connected nodes in an Activation Spreading Network (ASN). Inspired by a formalism in hierarchical task networks, the structure of the network captures the hierarchical relationship between high-level goals and low-level activities that realize these goals. Our approach can detect intentions before they are realized and it can work in real-time. We also extend the formalism of ASNs to incorporate contextual information into intent recognition. A fully functioning system is developed for experimental evaluation. We implemented a robotic system that uses our intent recognition to naturally interact with the user. Our ASN based intent recognizer is tested against two different scenarios involving everyday activities performed by a subject, and our results show that the proposed approach is able to detect low-level activities and recognize high-level intentions effectively in real-time. Further analysis shows that contextual ASN is able to discriminate between otherwise ambiguous goals.

The ability to learn new concepts is essential for any robot to be successful in real-world applications. This is due to the fact that it is impractical for a robot designer to pre-endow it with all the concepts that it would encounter during its operational lifetime. In this context, it becomes necessary that the robot is able to acquire new concepts, in a real-world context, from cues provided in natural, unconstrained interactions, similar to a human-teaching approach. However, existing approaches on concept learning from visual images and abstract concept learning address this problem in a manner that makes them unsuitable for learning in an embodied, real-world environment. This paper presents a developmental approach to concept learning. The proposed system learns abstract, generic features of objects and associates words from sentences referring to those objects with the features, thus providing a grounding for the meaning of the words. The method thus allows the system to later identify such features in previously unseen images. The paper presents results obtained on data acquired with a Kinect camera and on synthetic images.

This paper addresses the problem of gender recognition by proposing a new feature descriptor to be used in classification. The contribution of this work is an extension to the local binary patterns traditionally used as descriptors. Local binary patterns include information about the relationship between a central pixel value and those of its neighboring pixels in a very compact manner. In the proposed method we incorporate into the descriptor more information from the neighborhood by using four predefined patterns, rather than just one, as in the classic model. We evaluate the performance of our method on the standard FERET database by comparing it to existing methods and show that we can extract more discriminative features and subsequently provide better gender recognition accuracy.

Effective human-robot interaction requires systems that can accurately infer and predict human intentions. In this paper, we introduce a system that uses stacked denoising autoencoders to perform intent recognition. We introduce the intent recognition problem, provide an overview of deep architectures in machine learning, and outline the components of our system. We also provide preliminary results for our system's performance.

As cameras and storage devices have become cheaper, the number of video surveillance systems has also increased. Video surveillance was (and mostly is) done by human operators on a need-to-know basis. The advent of new algorithms from the computer vision community, and increased computational power offered by new CPUs have shown a strong possibility of automating this task. Different approaches have been proposed by computer scientists to solve the difficult problem of content recognition from video data. They use many different videos to prove their usefulness and accuracy. A careful comparison and evaluation needs to be done to find the most suitable method under given conditions. To compare the results given by video surveillance applications, the ground truth needs to be established. In the case of computer vision, the ground truth needs to be provided by humans, making it one of the most time-consuming tasks in the evaluation process. This paper presents a tool (GTVT) that allows the user to establish the ground truth for a given video. GTVT presents a user-friendly interface to perform the cumbersome task of ground truth establishment and verification.

The paper presents an adaptive approach for person detection and tracking. The key feature of the proposed method is the ability to detect an object as a member of a generic class (in this case, a pedestrian) as it enters the scene, then dynamically build and maintain a model of its appearance that allows tracking of this specific instance of the class. Such an approach also addresses the problem of recognizing the object if it leaves and then re-enters the scene.

The paper proposes a new holistic representation for actions, called action signature, which uses both global and local motion variations to classify an action. The action signatures are generated by estimating a sequence of orientations from the motion history images. A mixture of von Mises distributions is used in the classification stage of the system.

The paper presents a method for the detection of abandoned luggage in video surveillance applications. The technique uses a foreground sampling method to localize abandoned luggage in the presence of cluttered scenes. A subsequent approach for person detection is proposed in order to detect the owners of the luggage, by using a combination of skin color detection and a generalized Hough transform for head-and-shoulders detection.

The paper presents a methodology for the detection of adverse weather conditions (snow and fog) in video sequences of traffic scenes. The snow falling is detected by tuning the learning parameter of the background model, while the fog is detected by analyzing the frequency spectrum from the Fourier Transform of the image.

Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among agents or detection of situations that can pose a threat. In this paper, we propose an approach that allows a robot to detect intentions of others based on experience acquired through its own sensory-motor capabilities, then using this experience while taking the perspective of the agent whose intent should be recognized. Our method uses a novel formulation of Hidden Markov Models designed to model a robot's experience and interaction with the world. The robot's capability to observe and analyze the current scene employs a novel vision-based technique for target detection and tracking, using a non-parametric recursive modeling approach. We validate this architecture with a physically embedded robot, detecting the intent of several people performing various activities.

We present a hierarchical feature fusion model for image classification that is constructed by an evolutionary learning algorithm. The model has the ability to combine local patches whose location, width and height are automatically determined during learning. The representational framework takes the form of a two-level hierarchy which combines feature fusion and decision fusion into a unified model. The structure of the hierarchy itself is constructed automatically during learning to produce optimal local feature combinations. A comparative evaluation of different classifiers is provided on a challenging gender classification image database. It demonstrates the effectiveness of these Feature Fusion Hierarchies (FFH).

Background modeling is an essential and important part of many high-level video processing applications. Recently, the Support Vector Data Description (SVDD) has been introduced for novelty detection when only one class of data is available, i.e. background pixels. This paper proposes a method to efficiently train an SVDD and compares the performance of this training algorithm with the traditional SVDD training techniques. We compare the performance of our method with traditional SVDD and other classification algorithms on various data sets including real video sequences.

This paper proposes an automatic framework for figure-ground segmentation of edged images in the presence of cluttered background. Our work employs perceptual grouping concepts to characterize image segments by means of their saliency, which is computed via tensor voting. The main innovation of our work is a case-based thresholding scheme which iteratively eliminates edge segments with low-saliency in multiple scales, preserving those that are more likely to belong to foreground. The key idea is classifying saliency histograms in several cases by considering the relative position of the modes of the figure/ground distributions and applying specific actions in each case. We have performed extensive experiments in order to evaluate our framework both quantitatively and qualitatively, including real images from the Berkeley dataset. 1

This paper describes a novel hand-based verification system based on palm-finger segmentation and fusion. The proposed system operates on 2D hand images acquired by placing the hand on a planar lighting table without any guidance pegs. The segmentation of the palm and the fingers is performed without requiring the extraction of any landmark points on the hand. First, the hand is segmented from the forearm using a robust, iterative methodology based on morphological operators. Then, the hand is segmented into six regions corresponding to the palm and the fingers using morphological operators again. The geometry of each component of the hand is represented using high order Zernike moments which are computed using an efficient methodology. Finally, verification is performed by fusing information from different parts of the hand. The proposed system has been evaluated on a database of 101 subjects illustrating high accuracy and robustness. Comparisons with competitive approaches that use the whole hand illustrate the superiority of the proposed, component-based, approach both in terms of accuracy and robustness. Qualitative comparisons with state of the art systems illustrate that the proposed system has comparable or better performance.

Recognizing human actions from video has been a challenging problem in computer vision. Although human actions can be inferred from a wide range of data, it has been demonstrated that simple human actions can be inferred by tracking the movement of the head in 2D. This is a promising idea as detecting and tracking the head is expected to be simpler and faster because the head has lower shape variability and higher visibility than other body parts (e.g., hands and/or feet). Although tracking the movement of the head alone does not provide sufficient information for distinguishing among complex human actions, it could serve as a complimentary component of a more sophisticated action recognition system. In this article, we extend this idea by developing a more general, viewpoint invariant, action recognition system by detecting and tracking the 3D position of the head using multiple cameras. The proposed approach employs Principal Component Analysis (PCA) to register the 3D trajectories in a common coordinate system and Dynamic Time Warping (DTW) to align them in time for matching. We present experimental results to demonstrate the potential of using 3D head trajectory information to distinguish among simple but common human actions independently of viewpoint.

Most approaches for motion analysis and interpretation rely on restrictive parametric models and involve iterative methods which depend heavily on initial conditions and are subject to instability. Further difficulties are encountered in image regions where motion is not smooth-typically around motion boundaries. This work addresses the problem of visual motion analysis and interpretation by formulating it as an inference of motion layers from a noisy and possibly sparse point set in a 4D space. The core of the method is based on a layered 4D representation of data and a voting scheme for affinity propagation. The inherent problem caused by the ambiguity of 2D to 3D interpretation is usually handled by adding additional constraints, such as rigidity. However, enforcing such a global constraint has been problematic in the combined presence of noise and multiple independent motions. By decoupling the processes of matching, outlier rejection, segmentation, and interpretation, we extract accurate motion layers based on the smoothness of image motion, then locally enforce rigidity for each layer in order to infer its 3D structure and motion. The proposed framework is noniterative and consistently handles both smooth moving regions and motion discontinuities without using any prior knowledge of the motion model.

Producing an accurate motion flow field is very difficult at motion boundaries. We present a noniterative approach for segmentation from image motion, based on two voting processes, in different dimensional spaces. By expressing the motion layers as surfaces in a 4D (four-dimensional) space, a voting process is first used to enforce the smoothness of motion and determine an estimation of pixel velocities, motion regions and boundaries. The boundary estimation is then combined with intensity information from the original images in order to locally define a boundary tensor field. The correct boundary is inferred by a 2D (two-dimensional) voting process within this field that enforces the smoothness of boundaries. Finally, correct velocities are computed for the pixels near boundaries, as they are reassigned to different regions. We demonstrate our contribution by analyzing several image sequences, containing multiple types of motion.

We address the problem of perceptual grouping from motion cues by formulating it as a motion layers inference from a sparse and noisy point set in a 4D space. Our approach is based on a layered 4D representation of data, and a voting scheme for token communication, within a tensor voting computational framework. Given two sparse sets of point tokens, the image position and potential velocity of each token are encoded into a 4D tensor. By enforcing the smoothness of motion through a voting process, the correct velocity is selected for each input point as the most salient token. An additional dense voting step allows for the inference of a dense representation in terms of pixel velocities, motion regions, and boundaries. Using a 4D space for this tensor voting approach is essential since it allows for a spatial separation of the points according to both their velocities and image coordinates. Unlike most other methods that optimize certain objective functions, our approach is noniterative and, therefore, does not suffer from local optima or poor convergence problems. We demonstrate our method with synthetic and real images, by analyzing several difficult cases-opaque and transparent motion, rigid and nonrigid motion, curves and surfaces in motion.

Choosing the appropriate type of video input is an important issue for any vision-based system and the right decision must take into account the specific requirements of the intended application. In the context of intelligent room systems, we establish several qualitative criteria to evaluate the video input component and we use them to compare three current solutions: mobile pan-tilt-zoom cameras, wide-angle lens cameras and electronic pan-tilt-zoom cameras. We show that electronic pan-tilt-zoom systems best satisfy our criteria. To support this claim, we present GlobeAll, a modular four-component prototype for a vision-based intelligent room: a video input component that uses an electronic pan-tilt-zoom camera array, a background learning and foreground extraction component, a tracking component and an interpretation component.