Classifying and modeling texture images, especially those with significant rotation, illumination, scale, and view-point variations, is a hot topic in the computer vision field. Inspired by local graph structure (LGS), local ternary patterns (LTP), and their variants, this paper proposes a novel image feature descriptor for texture and material classification, which we call Petersen Graph Multi-Orientation based Multi-Scale Ternary Pattern (PGMO-MSTP). PGMO-MSTP is a histogram representation that efficiently encodes the joint information within an image across feature and scale spaces, exploiting the concepts of both LTP-like and LGS-like descriptors, in order to overcome the shortcomings of these approaches. We first designed two single-scale horizontal and vertical Petersen Graph-based Ternary Pattern descriptors ($PGTP_{h}$and$PGTP_{v}$). The essence of$PGTP_{h}$and$PGTP_{v}$is to encode each$5\times 5$image patch, extending the ideas of the LTP and LGS concepts, according to relationships between pixels sampled in a variety of spatial arrangements (i.e., up, down, left, and right) of Petersen graph-shaped oriented sampling structures. The histograms obtained from the single-scale descriptors$PGTP_{h}$and$PGTP_{v}$are then combined, in order to build the effective multi-scale PGMO-MSTP model. Extensive experiments are conducted on sixteen challenging texture data sets, demonstrating that PGMO-MSTP can outperform state-of-the-art handcrafted texture descriptors and deep learning-based feature extraction approaches. Moreover, a statistical comparison based on the Wilcoxon signed rank test demonstrates that PGMO-MSTP performed the best over all tested data sets.

Research on content-based image retrieval (CBIR) has been under development for decades, and numerous methods have been competing to extract the most discriminative features for improved representation of the image content. Recently, deep learning methods have gained attention in computer vision, i ncluding CBIR. In this paper, we present a comparative investigation of different features, including low-level and high-level features, for CBIR. We compare the performance of CBIR systems using different deep features with state-of-the-art low-level features such as SIFT, SURF, HOG, LBP, and LTP, using different dictionaries and coefficient learning techniques. Furthermore, we conduct comparisons with a set of primitive and popular features that have been used in this field, including colour histograms and Gabor features. We also investigate the discriminative power of deep features using certain similarity measures under different validation approaches. Furthermore, we investigate the effects of the dimensionality reduction of deep features on the performance of CBIR systems using principal component analysis, discrete wavelet transform, and discrete cosine transform. Unprecedentedly, the experimental results demonstrate high (95% and 93%) mean average precisions when using the VGG-16 FC7 deep features of Corel-1000 and Coil-20 datasets with 10-D and 20-D K-SVD, respectively.

The paper presents a statistical learning system based visual solution developed and applied for fault detection in industrial environment. As a mobile vision system the area of use was the automatic detection of rare faults in complex assembled objects. The object detection, the fore- and background separation, and the multi-model database enables the system to manage irregular batches of the different objects. A multi-model database guarantees that the object is compared with the statistically most relevant model, therefore it reduces the number of false alarms. The developed system is able to detect faults with the size of 2% of the total picture based on previously learned models.

Sensor fusion is an important part of modern cyber-physical systems that observe and analyse real-world environments. Time-of-Flight depth cameras provide high frame rate low-resolution depth data that can be efficiently used in many applications related to cyber-physical systems. In this paper, we address the critical issue of upsampling and enhancing the low-quality depth data using a calibrated and registered high-resolution colour image or video. Two novel algorithms for image-guided depth upsampling are proposed based on different principles. A new method for video-guided upsampling is also presented. Initial test results on synthetic and real data are shown and discussed.

Abstract The separation of reflection components is an important issue in computer graphics, computer vision and image processing. It provides useful information for the applications that need consistent object surface appearance, such as stereo reconstruction, visual recognition, tracking, objects re‐illumination and dichromatic editing. In this paper we will present a brief survey of recent advances in separation of reflection components, also known as specularity (highlights) removal. Several techniques that try to tackle the problem from different points of view have been proposed so far. In this survey, we will overview these methods and we will present a critical analysis of their benefits and drawbacks.

Illumination-robust optical flow algorithms are needed in numerous machine vision applications such as vision-based intelligent vehicles, surveillance and traffic monitoring. Recently, we have proposed an implicit nonlinear scheme for variational optical flow that assumes no particular analytical form of energy functional and can accommodate various image components and data metrics. Using test data with brightness and colour illumination changes, we study different features and metrics and demonstrate that cross-correlation is superior to the L1metric for all combinations of the features.

We present an optical flow calculation method based on the variational approach. The method uses an implicit non-linear numerical scheme that makes no assumption on the analytical form of the optical constraints and combines local and global smoothness criteria in a natural way. Defining the optical constraints with normalised cross-correlation, we obtain a technique that is robust against illumination changes in colour and greyscale images.

Dynamic textures (DTs) are videos of natural or artificial processes, such as waves, smoke, fire, or walking crowd. While activities and motion events produced by moving shapes are well localised in space and time, the spatiotemporal extent of most natural DTs is less definite. The notion of periodicity, or regularity, has been extensively studied in static texture analysis and in activity analysis, where it proved very useful. At the same time, much less attention has been paid to temporal (quasi-) periodicity of dynamic textures, despite the obvious fact that this property is inherent in dynamic texture. We discuss the reasons, then present a framework for quantitative motion periodicity analysis of DTs. Using the optic flow and adapting the SVD-based algorithm for signal period estimation [17], we measure degree of periodicity of natural dynamic textures. Numerous test results are presented, including the application of the temporal periodicity features to DT classification. 1

Particle Image Velocimetry (PIV) is a popular approach to flow visualization and measurement in hydro- and aerodynamic studies and applications. The fluid is seeded with particles that follow the flow and make it visible. Traditionally, correlation techniques have been used to estimate the displacements of the particles in a digital PIV sequence. These techniques are relatively time-consuming and noise-sensitive. Recently, an optical flow estimation technique developed in machine vision has been successfully used in Particle Image Velocimetry. Feature tracking is an alternative approach to motion estimation, whose application to PIV is proposed and studied in this paper. Two efficient feature tracking algorithms are customized and applied to PIV. The algorithmic solutions of the application are described. In particular, techniques for coherence filtering and interpolation of a velocity field are developed. To assess the proposed and the previous approaches, velocity fields obtained by the different methods are quantitatively compared for numerous synthetic and real PIV sequences. It is concluded that the tracking algorithms offer Particle Image Velocimetry a good alternative to both correlation and optical flow techniques.

This paper addresses detection of imperfections in repetitive regular structures (textures). Humans can easily find such defects without prior knowledge of the 'good' pattern. In this study, it is assumed that structural defects are detected as irregularities, that is, locations of lower regularity. We define pattern regularity features and find defects by robust detection of outliers in the feature space. Two tests are presented to assess the approach. In the first test, diverse texture patterns are processed individually and outliers are searched in each pattern. In the second test, classified defects in a group of textiles are considered. Defect-free patterns are used to learn distance thresholds that separate defects.

In content-based image retrieval, a texture pattern may appear in a wide range of 3D views. Affine transformation is an approximation frequently used in practice to represent the variation of a pattern. The existing approaches to texture classification cannot cope with this variation. We use pattern regularity to define an affine-invariant feature vector and apply it to classification of structured textures under the orthographic projection, an important subset of affine transformations. The proposed approach achieves 85% accuracy for 18 patterns.

Digital particle image velocimetry (DPIV) aims at flow visualisation and measurement of flow dynamics in numerous applications. The fluid is seeded with particles that follow the flow and efficiently scattered light. Traditionally, FFT-based correlation techniques were used to estimate the displacements of the particles in a digital PIV sequence. Recently, an optical flow estimation technique developed in computer vision has been successfully applied to DPIV. In this paper we study the DPIV-efficiency of another group of tracking approaches, the feature tracking techniques. Velocity fields obtained by several methods are compared for synthetic and real PIV sequences. It is concluded that feature tracking algorithms applied to DPIV are a useful alternative to both the correlation and the optical flow algorithms.

Abstract This paper gives a summary of our research on pattern regularity. Periodic structures are perceived by humans as regular in a wide range of viewing angles. This observation motivates the development of a regularity based feature vector whose affine invariance is justified theoretically and tested experimentally. The vector is derived from the interaction map of a pattern. Several alternative but closely related definitions of the interaction map are discussed. The maximal regularity, a component of the feature vector, is shown to be consistent with human judgement on regularity. This feature can be implemented as a run filter, allowing for regularity based image filtering. Three applications of the regularity approach are presented. First, it is used for affine-invariant texture classification. Then, detection of periodic structures in aerial images is demonstrated. Finally, the texture inspection problem is addressed and structural defects are found as locations of low regularity.

A new algorithm is presented for feature point based motion tracking in long image sequences. Dynamic scenes with multiple, independently moving objects are considered in which feature points may temporarily disappear enter and leave the view field. The existing approaches to feature point tracking have limited capabilities in handling incomplete trajectories, especially when the number of points and their speeds are large, and trajectory ambiguities are frequent. The proposed algorithm was designed to efficiently resolve these ambiguities.

We have recently introduced a new tool for texture analysis called feature based interaction map (FBIM). The FBIM approach can be efficiently used to assess fundamental structural properties of textures such as anisotropy, symmetry, orientation and regularity. The FBIM is suitable for rotation-invariant texture classification of patterns with regular, weak regular, or linear structure. In this paper, we show how the interaction map can be applied as a structural filter for segmentation, detection of textured objects and texture defects, analysis of oriented structures and shape-from-texture. The power of the FBIM filter is in its unique capability to grasp the structure of pixel interactions typical for a given texture pattern. To efficiently use this capability, we propose a fast running implementation of the FBIM algorithm and present pilot experimental results demonstrating the potential of the FBIM approach in diverse tasks and applications.

We consider the problem of zone classification in document image processing. Document blocks are labelled as text or nontext using texture features derived from a feature based interaction map (FBIM), a recently introduced general tool for texture analysis. The zone classification procedure proposed is tested on the comprehensive document image database UW-I created at the University of Washington in Seattle. Different classification procedures are considered. The performance ranges from 96% to 98% using 6 FBIM texture features only.

We discuss a novel method for recovering fundamental, perceptually motivated structural features of a texture pattern: anisotropy, symmetry, and regularity. The method is based on extended spatial grey-level difference statistics which describe pairwise pixel interactions and yield an interaction map used to assess the overall two-dimensional structure of interactions and extract the significant short- and long-range interactions (intersample spacings). The new approach extends, in digital images, the notion of grey-level difference to arbitrary spacing vectors (i.e. any angle at any displacement). This provides the necessary background for precise anisotropy (or directionality) and symmetry analysis. Experimental results are shown with a set of Brodatz images that range from highly regular to patterns with weak regularity or anisotropy. A few especially interesting examples of recovering hardly visible structural features are given. Finally, the approach is applied to rotation-invariant texture classification.

Growing attention has been paid to the investigation of oriented (anisotropic) textures. Co-occurrence probability matrix (CPM) and gray-level difference histogram (GLDN) based features have been traditionally viewed as powerful texture analysis tools that are, however, less suitable for detailed anisotropy analysis because for small discrete spacing one cannot set fine angular resolution necessary for detailed directionality analysis. The author proposes a computationally efficient extension of CPM and GLDH to arbitrary angle and spacing and applies it to the analysis of texture anisotropy. Furthermore, the author considers the possibility of investigating the symmetry of a texture pattern via the symmetry properties of a polar diagram describing the anisotropy of the pattern. Results of pilot experiments with real-world textures are shown and discussed.

Gives a brief description of the operation principles of LeaVis, a prototype industrial machine vision system aimed at processing and segmentation of the images of hides marked by lines and other symbols that show defects and areas of different quality. A key feature of the vision system is its ability to correct line drawings by connecting broken lines and bridging gaps in line junctions. The goal of LeaVis is to provide a coherent visual input for a computer aided layout design system that creates trajectory descriptions by which the knife cutting the hides is controlled. The authors discuss the basic motivations of the vision system design, and present the ideas behind its core algorithms.>

A procedure is suggested for adaptive detection of texture imperfections in the presence of a slow spatial variation of the texture pattern. The results of initial experiments with the detector are reported as promising. The procedure seems to be efficient when the conventional one fails. the experiments also show that the shape and size of the detector window must be adjusted to the shape of imperfections.>