This paper addresses the challenge of data augmentation for monochrome images, which are still highly relevant in machine vision.Current augmentation methods designed for natural scenes focus on geometric transformation and often fall short in simulating real-world illumination variations.We propose a novel approach that leverages a two-step monochrome image processing technique consisting of image pseudo-colorization followed be a specific decolorization scheme.Our method generates diverse and unpredictable intensity variations, effectively emulating illumination changes and, if only a specific category of color maps are used, preserving the naturalness of the resulting images.To exclude the results which closely replicate images already in the dataset, we rank these maps, identifying a subset that significantly alters illumination characteristics of the originals.A popular SSIM measure is used for that purpose.The proposed technique enhances the illumination diversity of datasets, offering a valuable tool for improving the robustness of AI/ML models in monochrome imagery.NOTE: All figures are best viewed in high resolution.

Re-colorization of images or movies is a challenging problem due to the infinite RGB solutions for a monochrome object.In general, the process is assisted by humans, either by providing colorization hints or relevant training data for ML/AI algorithms.Our intention is to develop a mechanism for fully unguided (and with no training data used) colorization of movies.In other words, we aim to create acceptable colored counterparts of movies in domains where only monochrome visualizations physically exist (e.g.IR, UV, MRI, etc. data).Following our past approach to image colorization, the method assumes arbitrary rgb2gray models and utilizes a few probabilistic heuristics.Additionally, we maintain the temporal stability of colorization by locally using structural similarity (SSIM) between adjacent frames.The paper explains the details of the method, presents exemplary results and compares them to the state-of-the art solutions.NOTE: All figures are best viewed in color and high resolution.

The paper reports some results on neural architectures for learning numerical concepts from visual data.We use datasets of small images with single-pixel dots (one to six per image) to learn the abstraction of small integers, and other numerical concepts (e.g. even versus odd numbers).Both fully-connected and convolutional architectures are investigated.The obtained results indicate that two categories of numerical properties apparently exist (in the context of discussed problems).In the first category, the properties can be learned without acquiring the counting skills, e.g. the notion of small, medium and large numbers.In the second category, explicit learning of counting is embedded into the architecture so that the concepts are learned from numbers rather than directly from visual data.In general, we find that CNN architectures (if properly crafted) are more efficient in the discussed problems and (additionally) come with more plausible explainability.

The paper proposes mid-level features for socio-cognitive classification of crowd behavior in public spaces, particularly in the context of monitoring social interactions during, e.g., pandemic restrictions. The classification method follows a recently proposed categorization [37]. The features are built using statistics obtained from detection and tracking results forc crowd components, i.e. individuals and their groups (any typical detectors and trackers can be used). The features are defined by static (if obtained from the current frame) or dynamic (if derived from consecutive frames) parameters characterizing the crowd. Subsequently, the features extracted from a number of most recent frames are fed into a fully-connected shallow neural network to identify the type of social interactions in the monitored space. The experimental feasibility study shows encouraging performances of the approach. In particular, the results are far more discriminative than in the other solution (which, at the moment, is the only publicly known benchmark).

In this paper, we propose a novel modification to the Random Sampling Consensus (RANSAC) algorithm, called Iterative Comprehensive Sampling Consensus (ICSAC), enabling it to identify more than one transformation between images containing multiple matching partial near-duplicates (sub-images, objects), within a sufficient level of accuracy. The proposed method makes use of the k-means clustering algorithm in order to locate consensus in the parameter space of the transformation matrix. The method is developed primarily for outlier rejection after the feature matching step, where a geometric consistency assumption is enforced on all matching pairs. The method was verified against a variety of diversified images and demonstrates promising results and a clear ability of outlier rejection and accurate localization of multiple transformations within the same pair of matched images.

A CNN-based framework is presented for extracting and classifying from static images of crowd (acquired from surveillance systems) individuals, small groups and large groups. A novel approach to the network training has been investigated. Instead of manually outlined ground-truth data, we use automatic annotations by alternative baseline algorithms (which consider both motion and appearance). The proposed CNN detectors are initially trained over rather limited amounts of data. Nevertheless, the detectors are subsequently updated (fine-tuned) by using new batches of automatically annotated samples. Those test samples are periodically acquired by the baseline algorithms from the future surveillance data. Fine-tuning is performed when noticeable differences appear between results by the CNN-detectors and the results of baseline algorithms (which may indicate changes in visual conditions, scenarios or updates in the baseline algorithms). We preliminarily demonstrate that satisfactory performances of CNN-based detectors can be achieved, even if the baseline algorithms have limited accuracy. Actually, it was noticed that fine-tuned CNN-detectors can be superior to the baseline algorithms used for automatic annotation of training data (even though the baseline algorithms process both static images and video-sequences). Since only static images are used once the detectors are fully trained, the presented solution can simplify complexity of systems automatically evaluating structure and behavior of crowds.

In this paper, the problem of multi-class object recognition in thermal images is discussed. An alternative model of thermal objects is investigated, where an object is represented by multiple shapes extracted by MSER detectors. The shapes are nested within the largest MSER outlining the object (which might be the actual outline of the object, the outline of its thermal footprint or the outline of its largest prominent fragment). We show, using a multi-class dataset of thermal images captured in indoor environments, that the proposed methodology is a feasible solution for various object classification problems in thermal imaging. In particular, no object-specific algorithms are needed, so that the method is applicable to most of typical applications of thermal cameras (subject to general limitations of data captured by thermal imaging devices). The presented work is considered a preliminary feasibility study exploring potentials an limits of thermal image classification in more sophisticated machine vision problems.

In this paper, a temporally iterative Gaussian Mixture Model (GMM) of Dynamic Texture (DT) for target detection using a moving PTZ camera, is proposed. Camera movement in a PTZ sensor causes motion-based target detection techniques to fail for the periods affected by the scene change. This is because the whole scene is considered a representation of the target motion. When the camera is in motion, conventional background models remain invalid until the time that the model has adapted and updated its parameters to the newly perceived scene. The proposed model is based on an iterative modeling of spatio-temporal patches that represent the visual scene using GMM-of-DT. During the initial iteration of the proposed GMM-of-DT model, the input video is temporally segmented into clips in a manner that separates global from local motion. Further, parameters of the GMM-of-DT model are estimated for each temporal segment and in subsequent iterations updated adaptively to generate the final foreground mask. The proposed technique is tested and verified on video scenes from public datasets.

CBVIR approach to video-based surveillance is discussed. The objective is to detect in real time near-duplicates (e.g. similarly-looking objects) simultaneously appearing in concurrently captured/played videos. A novel method of keypoint matching is proposed, based on keypoint descriptions additionally incorporating visual and geometric contexts. Near-duplicate fragments can be identified by keypoint matching only. The analysis of geometric constraints (a bottleneck of typical CBVIR methods for sub-image retrieval) is not required. When the proposed method is fully implemented, high-speed and good performances can be achieved, as preliminarily shown in proof-of-concept experiments. The method is affine-invariant and employs typical keypoint detectors and descriptors (MSER and SIFT) as the low-level mechanisms.

Visual search for objects of interest in complex environment is an important (and still challenging) problem in mobile robotics. In particular, the usage of content-based visual information retrieval (CBVIR) methods, which are a natural choice for such tasks, is often constrained by the real-time requirements, and the mobility of searching agents is sometimes not sufficiently exploited in the search model. In this paper, a CBVIR-based scheme is proposed, which takes into account motion of the searching agents to achieve a low-cost and highspeed detection of objects of interest in cluttered scenes, with good overall performances. We combine standard CBVIR tools, i.e. MSER detector and SIFT descriptor (quantized into sufficiently large vocabularies) assuming additionally that objects become objects of interest only when approached closely enough by the mobile agent, i.e. when seen at an adequately large scale. Thus, an object of interest is considered detected only if a sufficient number of keypoints from the current video-frame are matched (including the corresponding matches of scales) to the keypoints from the database images of the object. Preliminary experiments on a limited-size dataset confirm performances of the scheme, although in the classical task of video-frame retrieval the scheme cannot compete with more sophisticated CBVIR algorithms. The scheme can prospectively become more flexible if combined with a range-finding device so that the approximate distances to the scene components within the currently inspected part of the image can be used to proportionally modify the scale correspondences.

This paper presents a novel hardware accelerator architecture for the linear-time Maximally Stable Extremal Regions (MSER) detector algorithm. In contrast to the standard MSER algorithm, the linear-time MSER implementation is more suitable for real-time applications of image retrieval in large-scale and high resolution datasets (e.g. satellite images). The linear-time MSER accelerator design is optimized by enhancing its flooding process (which is one of the major drawbacks of the standard linear-time MSER) using a structure that we called stack of pointers, which makes it memory-efficient as it reduces the memory requirement by nearly 90%. The accelerator is configurable and can be integrated with many image processing algorithms, allowing a wide spectrum of potential real-time applications to be realized even on small and power-limited devices.

This paper presents a novel implementation of the Maximally Stable Extremal Regions (MSER) detector on system-on-chip (SoC) using 65 nm CMOS technology. The novel SoC was developed following the Application Specific Integrated Circuit (ASIC) design flow which significantly enhanced its realization and fabrication, and overall performances. The SoC has very low area requirement (around 0.05 mm2) and is capable of detecting both bright and dark MSERs in a single run, while computing simultaneously their associated regions' moments, simplifying its interfacing with other image algorithms (e.g. SIFT and SURF). The novel MSER SoC is power-efficient (requires 2.25 mW) and memory-efficient as it saves more than 31% of the memory space reported in the state-of-the-art MSER implementation on FPGA, making it suitable for mobile devices. With 256×256 resolution and its operating frequency of 133 MHz, the SoC is expected to have a 200 frames/second processing rate, making it suitable (when integrated with other algorithms in the system) for time-critical real-time applications such as visual surveillance.

This paper presents a new fast and scalable Parallel Union-Find algorithm for image segmentation and its System-on-Chip (SoC) implementation using 65nm CMOS technology following the Application-Specific Integrated Circuit (ASIC) design flow. The algorithm is capable of labeling all foreground and background pixels, using the least possible pixels scanning. This contrasts the classical labeling algorithms that label only foreground (or background) pixels in a single run. The new algorithm utilizes only two memory blocks. In one memory block, it labels image segments using their seeds as the label and, simultaneously, the segments sizes are used as the other label in second memory block. By this parallel labeling, monitoring the image segments is very fast and efficient. With 350 MHz operating frequency, the processing rate estimated to be 2100 frames/sec, the total chip area of 15950.5 μm2 (off-chip memory) and very low-power of 0.3 mW, the SoC tends to be an excellent candidate for mobile devices and real-time applications.

The paper presents a novel technique to segment and extract streets from satellite images. This technique utilizes, for the first time in the known literature, the Maximally Stable Extremal Regions (MSER) algorithm to robustly identify and segment streets from satellite images. The technique extracts dark MSERs and then classifies them based on multiple metrics such as the intensity of the pixels, the region stability, and the major-to-minor axes ratio. Testing results under multiple scenarios corroborate the accuracy of the proposed technique. The technique will allow fast and accurate implementation of a wide spectrum of applications such as in Global Positioning System (GPS) driving guidance.

This paper introduces a dramatically novel traffic signs recognition (TSR) system that can perform traffic sign detection and tracking simultaneously. The proposed approach utilizes intensity images and the depth images, in parallel, to robustly detect and track traffic signs in real-time. Additionally, we suggest to supplement the ordinary traffic signs with the corresponding quick-response (QR) code plates that inherent the many advantages of the QR-codes, introducing the concept of QR-TSR systems.

The paper proposes CONSIFT descriptors which are the rotation-variant modification of SIFT (primarily for affine-invariant keypoints). CONSIFT of a keypoint K is its SIFT computed relatively to the orientation defined by the location of another keypoint L (and concatenated with similarly computed SIFT for keypoint L relatively to the location of K). It is additionally recommended that K and L are extracted by different detectors of complementary properties (e.g. Harris-Affine and Hessian-Affine). The paper evaluates CONSIFT performances over the benchmark dataset. Surprisingly, performances of CONSIFT are nearly equivalent to SIFT. However, the intersection of SIFT-based and CONSIFT-based matches is a highly discriminative tool which rejects most of correspondences which are incorrect in a wider context. Since the method does not require any consistency verification over groups of preliminary matched keypoints (and its memory usage is acceptable) it can be instrumental in large-scale CBVIR systems.

In this paper we propose a novel approach to introducing semantic relations into the bag-of-words framework. We use the latent semantic models, such as LSA and pLSA, in order to define semantically-rich features and embed the visual features into a semantic space. The semantic features used in LSA technique are derived from the low-rank approximation of word-document occurrence matrix by SVD. Similarly, by using the pLSA approach, the topic-specific distributions of words can be considered dimensions of a concept space. In the proposed space, the distances between words represent the semantic distances which are used for constructing a discriminative and semantically meaningful vocabulary. We have tested our approach on the KTH action database and on the Fifteen Scene database and have achieved very promising results on both.

Retrieval of near-duplicate image fragments is one of the most challenging problems is CBIR (content-based image retrieval). The objective is to identify almost the same fragments in random images of unpredictable contents. Such fragments usually represent identical object, though captured from a different viewpoint, under different photometric conditions and/or by a different camera. The paper presents techniques developed for such applications. In general, the proposed methods are based on statistical properties of keypoint similarities between compared images. In the first approach, we assume that near-duplicates are (approximately) related by affine transformations, i.e. the underlying objects are locally planar. In the second approach, a wider range of shape distortions is acceptable. Implementations (including online detection in realtime videos) are presented and their performances discussed. Additionally, an algorithm for a highly accurate segmentation of detected near-duplicate fragments is presented.

The scale-invariant feature transform (SIFT) descriptor has been widely applied in many fields due to its resistance to common image transformations. However, the dimension of SIFT is high which makes it not practical in limited-memory systems. Thus, some lower-dimension SIFTs are proposed by using subspace projection techniques. The most popular technique is Principle Component Analysis (PCA) which can produce two different lower-dimension SIFTs, PCA-SIFT and PSIFT. They apply PCA on gradient field of local patches or on a set of training descriptors, respectively. However, the other subspace techniques can be also used. This paper proposes two more low-dimensional SIFTs (namely LPP-SIFT and SPCA-SIFT) by incorporating manifold subspace and sparse eigenspace learning techniques (Locality Preserving Projection and Sparse PCA are used as the exemplary implementations). Although these techniques are not novel, our results demonstrate they can be used to produce low-dimensional SIFTs. More importantly, by comparing their performance to the existing low-dimension SIFTs, we show which of them are more suitable for image matching.

Local feature based methods have achieved a great success in the field of image matching due to its invariance under typical image transformations. However, local features are often not invariant under complex non-affine transformations, which makes the matching methods ineffective. To remove the effect of complex image transformations, this paper proposes alignment of images before they are compared. An automatic image alignment method is introduced based on thin plate spline (TPS) warping. Then, the aligned matching scheme is designed to utilize correct alignments and reject false alignments for the improvement of matching performance. The method is evaluated using scene retrieval and object categorization. Experiments show the proposed aligned matching outperforms two typical methods: voting scheme and histograms comparison (over a set of prototypes which must be found by clustering).

The paper presents fundamentals and preliminary results of a technique for defining, building and positioning novel local feature. The features are created by approximating the content of a scanning circular window by a collection of predefined patterns. Although basics of the technique have been discussed in previous papers, the major modification is the introduction of Hough transform as a part of the algorithm. By applying a modified Hough transform to the contents of scanning windows, approximations can be build more reliably (the algorithm is not sensitive to so-called ldquovisual intrusionsrdquo) more accurately (localization of features is more precise) and at lower computational costs (a part of complex mathematics in previously used moment-based approximations can be avoided).

We report development of a hardware accelerator for robot-soccer systems equipped with global vision. An FPGA-embedded preprocessor captures individual video frames and detects colour blobs (corresponding to the ball and the players). The numerical characteristics of the blobs (colour, size and location) are sent to the host computer so that the current game configuration can be immediately identified by the computer. By exploiting parallelism and pipelining techniques available in the FPGA, a real-time performance has been achieved at rates higher than TV standards. The prototype implementation of the module (using Celoxica™ RC203 board with Virtex II FPGA) is able to process frames and to transfer results to the host computer at over 50 frames per second rate. A software module (Visual C++, Windows® XP platform) run be the host computer is used to interface the vision hardware, and to analyze and visualize the current game configuration.

This paper proposes a method to localize a 3D object in cluttered environment. Model of the object is represented by local shape features, computed from some reference images. Localization is performed by stereo images captured simultaneously by a pair of calibrated cameras. First the object is recognized in both images, using the local shape features. Then common features are used for reconstruction of the 3D position of the object. The proposed localization method is robust to different kinds of geometric and photometric transformations in addition to cluttering, partial occlusions and background changes. The accuracy and efficiency of the method is good enough for many practical applications

We propose novel tools that reduce complexity and improve performances of visual detection and identification of known objects randomly located in complex cluttered environments. Generally, the propose mechanisms are based on local shape features (interest points, visual saliencies) detected in images and characterized by compact descriptors invariant to geometric and photometric transformations. In particular, a novel invariant for intensity changes is proposed, and the problem of over-exposed and under-exposed images is discussed. Both models of known objects and images of real scenes are represented using interest points, though in different scales (reference scale for models and relative scale for images). By matching interest point detected in images to interest points from the model database, known objects present in the scene can be detected and identified. The methodology can be used both for robot-mounted navigation modules and for distributed visual surveillance systems since the proposed mechanisms minimize the amount of visual data to be transmitted, thus preventing communicational saturation of such systems. In the paper, we focus on the image processing aspects of the problems. Image acquisition issues and high-level identification algorithms are only briefly mentioned

We present further results in a novel method of visual obstacle detection. In this method, scenes are illuminated by short laser pulses, and images are captured by a gated camera. By controlling the pulse width and the gating time, it is possible to obtain images that contain only objects within a predefined distance. The rest of the scene remains invisible. We previously presented preliminary results confirming the method's feasibility, in this paper, we focus on the results obtained for more complex scenes with large 3D objects. The experiments again confirm that the method can be prospectively used to create robust and reliable systems for vision-based navigation. However, we also observed many unexpected effects that require further experiments and theoretical studies.

We discuss potential applications of a novel visual sensing method that is particularly suitable for obstacle detection and/or path planning, in this method, scenes are illuminated by short laser pulses and images are captured by a gated camera. By controlling the pulse width and the gating time, it is possible to get images that contain only objects within a predefined distance from the camera. The rest of the scene remains invisible. The presented experimental results confirm that the method can be prospectively used to create very fast, robust and reliable systems for vision-based navigation. Many significant constraints of the existing algorithms are overcome.

The paper introduces hierarchical image transformations that can be used for detecting various image features of gradually increased complexity. The major prospective application of the method is in (semi-) autonomous vision-guided robotic systems and, therefore, the local operators that can be prospectively hardware-implemented are the core component of the proposed algorithms. A feature map is a grey-level digital image with a vector attached to each pixel. Pixel intensities represent "the feature intensity", i.e. the estimated confidence that a feature of interest is located at the pixel. The vector components are characterizing the feature configuration. The low-level "intensity map" is the original grey-level image with the "feature intensity" being just the brightness value. A transformation from the current feature map to the map of a higher level is obtained by applying a local operator (with a circular scanning window). For each location of the window, the operator determines the template instance of a higher-level feature prospectively existing at this location. Then, the template is matched to the actual content of the window and - based on their similarity - the feature intensity value for the higher-level map pixel is determine. The associated vectors are containing the configuration parameters of the templates extracted by the operator. The paper contains the theoretical foundations of the proposed method, but exemplary results illustrating the method's principles are also provided.

A new methodology in contour analysis (based on the observation that image derivatives are dangerously amplifying noises) has been recently gaining popularity. Instead, contour features are directly detected from grey-level images. The paper presents a novel technique of contour feature extraction (including features more complex than usually discussed edges and corners) based on locally computed moments of intensity function. The scanning circular window is rather large (7-15 pixel radius) which allows feature extraction in noised and/or textured images. For each location of the window, a moment-based parameter fitting is used to determine the model contour feature prospectively existing at this location. Then, the model feature is matched with the actual content of the window. The results are either shown as a "feature intensity" map or can be used for further image processing. The prospects of hardware implementation are also briefly discussed.

A multi-level technique of segmenting digital contours using multiple segmentation primitives is presented. At each level, the prospective instances of the primitives are detected. Then, the detected instances are ranked according to how accurately they approximate the corresponding fragment of the contour. Finally, the top-rank instances are selected for the segmentation output. The remaining parts of the contour are segmented at the next level (using primitives of smaller size), etc.