In computer vision, Local Binary Pattern (LBP) and Scale Invariant Feature Transform (SIFT) are two widely used local descriptors. In this paper, we propose to combine them effectively for scene categorization. First, LBP and SIFT features are regularly extracted from training images for constructing a LBP feature codebook and a SIFT feature codebook. Then, a two-dimensional table is created by combining the obtained codebooks. For creating a representation for an image, LBP and SIFT features extracted from the same positions of the image are encoded together based on sparse coding by using the two-dimensional table. After processing all features in the input image, we adopt spatial max pooling to determine its representation. Obtained image representations are forwarded to a Support Vector Machine classifier for categorization. In addition, in order to improve the scene categorization performance further, we propose a method to select correlated visual words from large codebooks for constructing the two-dimensional table. Finally, for evaluating the proposed method, extensive experiments are implemented on datasets Scene Categories 8, Scene Categories 15 and MIT 67 Indoor Scene. It is demonstrated that the proposed method is effective for scene categorization.

Recently, various applications of mobile devices supporting color vision deficiency are released. However, applications supporting color vision deficiency may not work as designed performance. Color calibration and personalization are needed when the user uses such an application, because a color representation performance of devices is distinct from each other, and users may also have individual differences in characteristics of color vision. Thus, confusion color pairs will change slightly. Conversely, it will be a clue for color calibration or personalization in oder to reduce changes. Here, we propose methods determining confusion color pairs by a simple operation to achieve it. We estimate color pairs by the procedure as a bisection method in order to determine them. A user is presented with the three visual stimuli (a reference and two targets). The user selects one of test targets whose color is perceived more similar to reference's color. We focused on how to select visual stimuli's colors. We propose two methods to select ones, which are represented the coordinates on u'v'-chromaticity diagram. One method uses colors on a circumference whose center corresponds to the white point. Another method uses colors on parallel lines to the line passing through primary colors (green and blue). We obtained results that it shows comparable performance determining color pairs to previous studies. Results of lines' method show that performance is better than the circumference method. And, its measurement time is reduced.

We propose a system helping blind people to get character information in their surrounding environment, such as merchandise information (name, price, and best-before/use-by date) and restaurant menu (name and price). The system consists of a computer, a wireless camera/scanner and an earphone. It processes images captured/scanned by a user and extracts character regions in the image by using Support Vector Machine (SVM). Applying Optical Character Recognition (OCR) to the extracted regions, the system outputs the character information as synthesized speech.

This paper proposes a multi-objective genetic programming (MOGP) for automatic construction of feature extraction programs (FEPs). The proposed method is modified from a well known non-dominated sorting evolutionary algorithm, i.e., NSGA-II. The key differences of the method are related with redundancies in program representation. We apply redundancy regulations in three main processes of the MOGP, i.e., population truncation, sampling, and offspring generation, to improve population diversity. Experimental results exhibit that the proposed MOGP-based FEPs construction system provides obviously better performance than the original non-dominated sorting approach.

Recently, evolutionary computation (EC) has been adopted to search for effective feature extraction programs for given image recognition problems. For this approach, feature extraction programs are constructed from a set of primitive operations (POs), which are usually general image processing and pattern recognition operations. In this paper, we focus on an approach based on a variation of linear genetic programming (LGP). We describe the causes of redundancies in LGP based representation, and propose a transformation that converts the redundant LGP representation into a canonical form, in which all redundancies are removed. Based on this transformation, we present a way to reduce computation time, i.e., the evolutionary search that avoids executions of redundant individuals. Experimental results demonstrate a success in computation time reduction; around 7–62% of total compuation time can be reduced. Also, we have experimented with an evolutionary search that prohibits existence of redundant individuals. When selection pressure is high enough, its search performance is better than that of conventional evoltuionary search.

In this research, we have developed a variable baseline stereo tracking vision system using a 1ms high-speed vision system and a high-speed linear slider. The vision system tracks a moving object, captures a stereo image pair and estimates its distance in only 1ms. It also calculates stereo matching and estimates its 3D shape every 2ms. The high speed linear slider moves stereo cameras horizontally and independently so the baseline length can be changed. We also propose a baseline control method which is adaptive to the distance to the object. When the object appears large in the image, the system reconstructs its 3D shape, and when the object appears small in the image, the system only estimates the 3D position of its center of gravity. The experimental results show that the proposed baseline control method improves the depth accuracy and the system can track and estimate the 3D shapes of fast moving objects.

Despite extensive research on image texture analysis, it is still problematic to characterize and segment texture images especially in the presence of complex patterns. Upon tremendous advancement of the internet and the digital technology, there is also a need for the development of simple but efficient algorithms, which can be adaptable to real-time systems. In this study, we propose such an approach based on multiresolution discrete wavelet transform (DWT). After the transform, we compute salient energy points from each directional sub-band (LH, HL, and HH) in the form of binary image by thresholding intermittency indices of wavelet coefficients. We then propose and extract two new texture features namely Salient Point Density (SPD) and Salient Point Distribution Nonuniformity (SPDN) based on the number and the distribution of salient pixels in the local neighborhood of every pixel of the multiscale binary images. We thus obtain a set of feature images, which are subsequently applied to the popular K-means algorithm for the unsupervised segmentation of texture images. Though the above representation appear simple and infrequent in the literature, it proves useful in the context of texture segmentation. Experimental results with the standard texture (Brodatz) and natural images demonstrate the robustness and potentiality of the proposed approach.

A number of approaches have been proposed for extracting a given speech from mixtures of sounds in the so-called cocktail party problem, in the aim of, for example, to enhance the speech recognition rate. The speech extraction can be carried out by either using single or multiple channel measurements. Some of those approaches explore the harmonicity of speech, and others make use of the redundancy among the channels, usually by independent component analysis. To extract one speech from a single channel, we propose here to use the characteristics of speech through the concept of efficient coding, which mimics the way the auditory cortex codes information. Simulations show that this technique can be used efficiently both to extract a single speech, as well as highlights a new way of approaching the speech recognition problem.

Redundancy reduction as a form of neural coding has been since the early sixties a topic of large research interest. A number of strategies has been proposed, but the one which is attracting most attention recently assumes that this coding is carried out so that the output signals are mutually independent. In this work we go one step further and suggest an strategy to deal also with non-orthogonal signals (i.e., "dependent" signals). Moreover, instead of working with the usual squared error, we design a neuron where the non-linearity is operating on the error. It is computationally more economic and, importantly, the permutation/scaling problem is avoided. The framework is given with a biological background, as we avocate throughout the manuscript that the algorithm fits well the single neuron and redundancy reduction doctrine. Moreover, we show that wavelet-like receptive fields emerges from natural images processed by this algorithm.

In this paper, we develop a system for enhancement of the speech signal with highest energy from a linear convolutive mixture of n statistically independent sound sources recorded by m microphones, where m

In this work we develop a system for enhancement of the speech signal with highest energy from a linear convolutive mixture of n statistically independent sound sources recorded by m microphones, where m

A human being understands the environment by integrating information obtained by sight, hearing and touch. To integrate information across different senses, a human being must find the correspondence of events observed by different senses. The paper seeks to relate the audio-visual events caused by more than one movement according to general physical laws without object-specific knowledge. As corresponding cues, we use Gestalt's grouping law; simultaneity of the occurrence of sound and change in movement, similarity of time variation between sound and movement, etc. We conducted experiments in a real environment and obtained satisfactory results showing the effectiveness of the proposed method.

We obtain an exact 3D model of a scene using multiple images from different camera positions. The recovery of a 3D shape using two images has problems such as being weak in noise. Therefore we present methods to improve the accuracy of 3D shapes with multiple images. The system is divided into three stages: (1) the recovery of 3D shapes from different viewpoints, (2) fusing the 3D shapes to obtain a para-ideal shape, and (3) removing the outlier shapes and feature points by an evaluation function, and fusing the rest of the shapes. We demonstrate a significant improvement in the recovered shape. Experimental results show that our system performs well in removing noise with robustness. The maximum noise reduction rate was 82% in real image experiments.

Object shape and camera motion recovery from an image sequence is a crucial issue in computer vision and many methods have been proposed by researchers. Theoretically, these methods are perfect, but they are sensitive to noise, so that in many practical situations, satisfactory results cannot be obtained. To solve this problem, we propose a shape and motion recovery method using a gyro sensor attached an a video camera for compensating images. We made an experimental system with a CCD camera and a gyro sensor. Using this system, we have examined the accuracy of our method and obtained good results.

This paper deals with blind separation that can extract original signals from their mixtures observed in a normal room. Our method achieves blind separation by making the mixed signals not correlating with each other. The validity of the proposed method has been confirmed by a computer simulation and an experiment in an anechoic room. In this paper, we apply our method to an experiment that extracts two source signals from their mixtures observed in a normal room.

We describe a method for navigating a mobile robot in the environment using signboards in scenes as landmarks. This method will enable us to realize human-like navigation and map-generation, and also can benefit human-robot communication. We choose signboards on walls and doors as landmarks. An environment map that contains the approximate position of each signboard in the environment is generated by the robot when the robot explores the environment the first time, and it will be used for re-localizing and navigating the robot afterward. First, characters in scene images are detected by using several heuristics of characters and character lines. Then, we calculate the relative orientation of the camera and a detected signboard from a single view using two methods based on the geometry of perspective projection. Finally, we reconstruct a distorted signboard image using two corresponding methods, and this makes the recognition of characters under a wider range of viewing condition much easier.

Shape recovery methods from an image sequence have been studied by many researchers. Theoretically, these methods are perfect, but they are sensitive to noise, so that, in many practical situations, we could not obtain satisfactory results. In addition, we could not obtain the scale of the recovered object because of the image-projection property. For solving these problems, we propose a method for shape and scale recovery which uses an acceleration-gyro sensor attached on a CCD camera for compensating images.

To create a more realistic soccer game derived from TV images, we developed an image synthesis system that generates an image sequence from the viewpoint of a player on the field. This system is based on the camera calibration theory. The system first determines the camera parameters of a TV image by using the intersection points of the white lines drawn on the soccer field. It then extracts players from each image and estimates their positons in the world coordinate system. Finally it applies a running motion to the players in their respective positions and generates computer graphics animation from the viewpoint of any player selected by a user. The system was tested over seven sequences of TV images and demonstrated satisfactory results.

Image processing systems, each consisting of massively parallel photodetectors and digital processing elements on a monolithic circuit, are being developed by several researchers. This approach is promising for applications in robot vision and visual inspection. Some early vision-like processing algorithms, such as edge-detection, smoothing and thinning, are installed in the vision systems. However, they are not sufficient for applications because their output is in the form of pattern information, so that, in order to respond to input, some feature values are required to be extracted from the pattern. We propose a method for extracting feature values associated with images in a massively parallel vision system. The feature can be a complex one, such as the rotation angle of an object, if the feature value changes continuously as the image changes. The method is based on the eigenspace method, but modified to be robust. In addition, we propose some computation accelerating methods and report some experimental results.

We discuss the uniqueness of 2-D shape recovery of a polyhedron from a single shading image. First, we analytically show that multiple convex (and concave) shape solutions usually exist for a simple polyhedron if interreflections are not considered. Then we propose a new approach to uniquely determine the concave shape solution using interreflections as a constraint. A numerical example, in which two convex shapes and two concave shapes exist for a trihedral corner, has been shown by Horn. However, it is difficult to prove the uniqueness using constraint equations. We analytically show that multiple convex (and concave) shape solutions usually exist for a pyramid using a reflectance map, if interreflection distribution is not considered. However, if interreflection distribution is used as a constraint that limits the shape solution (for a concave polyhedron), the polyhedral shape can be uniquely determined. Interreflections are used as a constraint to determine the shape solution in our approach.

As it has been observed in the behavior research, in reaching operation of animals, the hand performs a feedforward ballistic movement during which the vision system is watching only a target without seeing the hand simultaneously. We hypothesized that, for this, the two spaces, each of which is perceived by the eyes (eye-space) and is reached by the arm motion (arm-space) respectively, are fused into one. Human babies are supposed to gradually learn the fusion by themselves from their birth. This paper describes a self-organization of reaching operation in robots. We regarded the binocular movements as the main cue in the space perception for reaching rather than visual disparity. The eye-space as well as the arm-space is directly represented by a harmony of contractions of muscles controlling their movements. The fusion of the two spaces is obtained by a learn-by-doing manner via an interaction of the binocular and the arm movements, and is stored in a memory (CMAC) as a form of experience of these muscle contractions obtained through the interaction. During the learning process it is not necessary to know structure and parameters of robot systems at all.

The shape of articulated objects such as a human body can be simplified into a stick model which is represented by connections of several circular generalized cylinders (GCs). In this paper, we propose a new image feature called cue circles (CCs), and present a method how to obtain CCs from input images. Using CCs, the 3D motion of a human body is measured from a sequence of boundary contour image pairs, which is obtained by an active binocular sensor system. The body model used is represented by a set of cue spheres (CSs), each of which is located at ends of GCs. Stereo matching for recovering the body model is carried out by finding pairs of CCs between the pair of contour images under consideration: a CS is projected on the two image planes as its corresponding CCs.

This paper describes a mobile robot equipped with a real time sound localization system as well as a sonar system for obstacle detection. The sound localization method is based on a model of the precedence effect of the human auditory system to cope with echoes and reverberations. The sound localization and robot navigation experiments were conducted. The results show that the robot is capable of localizing sounding objects in a reverberant environment and approaching the objects without collisions, even when the objects were behind obstacles. Environment flexibility and error robustness of the system were discussed as well.

Image changes produced by a moving camera (optical flow) are an important information source of the observer's motion and structure of environment. Determination of the 3D motion and structure from optical flow is one of the most important problems in computer vision and intensively studied by researchers. However, previous works in this field have been unsatisfactory because they require the solution of nonlinear simultaneous equations using iterative search. In the present paper, we propose a method for the recovery of motion and structure from prospectively projected optical flow of feature points which move rigidly. In order to simplify the computation and obtain clear perspectives, we adopt a spherical image screen and arrange equations in linear form. As a result, our method does not require the solution of nonlinear simultaneous equations, but only requires solving linear ones. Simulation results of our method are also presented.

Regional image emphasis is often evident in paintings and illustrations. This technique increases local contrast while reducing global contrast by amplifying image intensity on shadowed surfaces, reducing intensity on illuminated surfaces, and then expanding contrast at intensity edges. The effects are assumed to result from the visual processing needed to interpolate the real world onto canvas. Therefore, we propose an intensity emphasis method based on human vision. This method simulates the adaptation of photoreceptor cells and the lateral inhibition of receptive fields. These attributes of a vision system are realized by computation of relative intensity and differential intensity in small areas. The proposed method can successfully generate painting‐like artifacts, which greatly improves the perception of visual elements displayed in an image. Since the method efficiently reduces the dynamic range of images, it can be used for displaying highlighted images on standard graphic monitors. Experiments on a computer‐generated image and a photograph confirm the advantages of our method.

A connectionist model which can store a sequence of various views of a 3D object is presented. The model consists of four components, each of which is realized as a network of neuron-like elements. The model can store several 3D objects in the same network in an overlapping manner. If the model is shown one view of a stored 3D object then a sequence of other views is recalled one by one. Since the storage is based on an associative matrix, the recall is possible even from an incomplete view. Instead of raw image data, a set of extracted features is stored