The integration of human-robot interaction (HRI) technologies with industrial automation has become increasingly essential for enhancing productivity and safety in manufacturing environments. In this paper, we propose a novel approach to address these challenges by using stereo vision and gesture control in cooperative robotic cells. Our system enables seamless authentication of operators and real-time verification of task execution, ensuring compliance with established protocols and safety standards.Key features of our system include its gesture-based operation with gesture recognition algorithms, allowing operators to interact with robotic systems intuitively and efficiently. By leveraging stereo vision, our system accurately tracks the operators’ movement within the workspace, facilitating precise task execution and object manipulation.We present a detailed description of our system architecture, experimental configuration, and real-world performance assessment. Our results demonstrate the effectiveness and feasibility of our approach in enhancing operational efficiency, ensuring quality, and improving the overall user experience in industrial automation.

Motion estimation of surrounding objects is indispensable to any mobile machinery. The paper proposes a method to solve the estimation and reconstruction problem of dynamic objects with a mono camera. Using the relative camera motion and detected rigidly moving objects on the image, we estimate their movement up to a scale factor. Utilization priors about their moving direction are used to estimate the transformation, which maps the 3D object from the previous frame to the actual one. Our two-frame method works twice the speed or more as other methods using three frames or more for the estimation, and we do this without any constraints. We evaluate our method on various traffic scenarios of different autonomous driving datasets.

In this paper, we propose a novel algorithm to compute the initial structure of pose-graph based Simultaneous Localization and Mapping (SLAM) systems. We perform a Breadth-First Search (BFS) on the graph in order to obtain multiple votes regarding the location of a certain robot position from all of its previously processed neighbors. Next, we define the initial location of a pose as the average of the multiple alternatives. By adopting the proposed initialization approach, the number of iterations needed for optimization is significantly reduced while the computational complexity remains lightweight. We perform quantitative evaluation on various 2D and 3D benchmark datasets to demonstrate the advantages of the proposed method.

Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512 × 384). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.

This paper presents a blind image quality assessment (BIQA) method based on deep learning with convolutional neural networks (CNN). Our method is trained on full and arbitrarily sized images rather than small image patches or resized input images as usually done in CNNs for image classification and quality assessment. The resolution independence is achieved by pyramid pooling. This work is the first that applies a fine-tuned residual deep learning network (ResNet-101) to BIQA. The training is carried out on a new and very large, labeled dataset of 10, 073 images (KonIQ-10k) that contains quality rating histograms besides the mean opinion scores (MOS). In contrast to previous methods we do not train to approximate the MOS directly, but rather use the distributions of scores. Experiments were carried out on three benchmark image quality databases. The results showed clear improvements of the accuracy of the estimated MOS values, compared to current state-of-the-art algorithms. We also report on the quality of the estimation of the score distributions.

LIDAR sensors are part of the sensor system of several intelligent vehicles and transportation systems providing both object and free-space detection capabilities. In this paper a recognition method is proposed for LIDARs with only a few detection planes. Our method is especially useful in the case when angular resolution of the scan is sufficient, but in the vertical direction the planes are far from each other. The proposed method uses new features including Fourier based descriptor, deep learning classification and exploits additional 3D information if it is available. We tested the method on ten thousands of samples from a large public database. This paper gives an effective solution for a hard problem of LIDAR based recognition problems, namely the far-object detection in case of mobile LIDARs of limited or poor vertical resolution.

Environment analysis of automatic vehicles needs the detection from 3-D point cloud information. This paper addresses this task when only partial scanning data are available. Our method develops the detection capabilities of autonomous vehicles equipped with 3-D range sensors for navigation purposes. In industrial practice, the safety scanners of automated guided vehicles (AGVs) and a localization technology provide an additional possibility to gain 3-D point clouds from planar contour points or low vertical resolution. Based on this data and a suitable evaluation algorithm, intelligence of vehicles can be significantly increased without the need for installation of additional sensors. In this paper, we propose a solution for an obstacle categorization problem for partial point clouds without shape modeling. The approach is tested for a known database, as well as for real-life scenarios. In case of AGVs, real-time run is provided by on-board computers of usual complexity.

Subjective video quality assessment (VQA) strongly depends on semantics, context, and the types of visual distortions. Currently, all existing VQA databases include only a small number of video sequences with artificial distortions. The development and evaluation of objective quality assessment methods would benefit from having larger datasets of real-world video sequences with corresponding subjective mean opinion scores (MOS), in particular for deep learning purposes. In addition, the training and validation of any VQA method intended to be ‘general purpose’ requires a large dataset of video sequences that are representative of the whole spectrum of available video content and all types of distortions. We report our work on KoNViD-1k, a subjectively annotated VQA database consisting of 1,200 public-domain video sequences, fairly sampled from a large public video dataset, YFCC100m. We present the challenges and choices we have made in creating such a database aimed at ‘in the wild’ authentic distortions, depicting a wide variety of content.

This paper deals with automatic image colorization. This is a very difficult task, since it is an ill-posed problem that usually requires user intervention to achieve high quality. A fully automatic approach is proposed that is able to produce realistic colorization of an input grayscale image. Motivated by the recent success of deep learning techniques in image processing, we propose a feed-forward, two-stage architecture based on Convolutional Neural Network that predicts the U and V color channels. Unlike most of the previous works, this paper presents a fully automatic colorization which is able to produce high-quality and realistic colorization even of complex scenes. Comprehensive experiments and qualitative and quantitative evaluations were conducted on the images of SUN database and on other images. We have found that Quaternion Structural Similarity (QSSIM) gives in some degree a good base for quantitative evaluation, that is why we chose QSSIM as an index-number for the quality of colorization.

Due to the explosive increase of online images, content-based image retrieval has gained a lot of attention. The success of deep learning techniques such as convolutional neural networks have motivated us to explore its applications in our context. The main contribution of our work is a novel end-to-end supervised learning framework that learns probability-based semantic-level similarity and feature-level similarity simultaneously. The main advantage of our novel hashing scheme that it is able to reduce the computational cost of retrieval significantly at the state-of-the-art efficiency level. We report on comprehensive experiments using public available datasets such as Oxford, Holidays and ImageNet 2012 retrieval datasets.

Classifying segments and detecting changes in terrestrial areas are important and time-consuming efforts for remote sensing image analysis tasks, including comparison and retrieval in repositories containing multitemporal remote image samples for the same area in very different quality and details. We propose a multilayer fusion model for adaptive segmentation and change detection of optical remote sensing image series, where trajectory analysis or direct comparison is not applicable. Our method applies unsupervised or partly supervised clustering on a fused-image series by using cross-layer similarity measure, followed by multilayer Markov random field segmentation. The resulted label map is applied for the automatic training of single layers. After the segmentation of each single layer separately, changes are detected between single label maps. The significant benefit of the proposed method has been numerically validated on remotely sensed image series with ground-truth data.

This letter addresses the automatic detection of urban area in remotely sensed images. As manual administration is time consuming and unfeasible, researchers have to focus on automated processing techniques, which can handle various image characteristics and huge amount of data. The applied method extracts feature points in the first step, which is followed by the construction of a voting map to represent urban areas. Finally, an adaptive decision making is performed to find urban areas. This letter presents methodological contributions in two key issues to the algorithm: 1) An automatically extracted Harris-based feature point set is introduced for the first step, which is able to represent urban areas more precisely. 2) An improved orientation-sensitive voting technique is proposed, exploiting the orientation information calculated in the local neighborhood of points. Evaluation results show that the proposed contributions increase the detection accuracy of urban areas.

This paper introduces a novel method to detect structural changes between MRI scans, without using prior knowledge. After a simple registration step, the method calculates a difference image, based on modified Harris saliency function, which is then used to define change candidates. Localization step filters out false hits with local contour descriptors featuring the neighborhood of candidates. Finally, boundary of the lesion is detected by integration of contour point extraction and Chan-Vese active contour method. Tests on simulated and real data show that the results are very promising.

In this paper a new decomposition method is introduced that splits the image into geometric (or cartoon) and texture parts. Following a total variation based preprocesssing, the core of the proposed method is an anisotropic diffusion with an orthogonality based parameter estimation and stopping condition. The quality criterion is defined by the theoretical assumption that the cartoon and the texture components of an image should be orthogonal to each other. The presented method has been compared to other decomposition algorithms through visual and numerical evaluation to prove its superiority.

In this paper we present a method for reconstructing static scene viewed through thick smoke using multiple images. Based on spatiotemporal statistical approach our method works well on noisy videos containing swirling smoke. We apply statistical analysis on regions of color input images, and show the way to reconstruct scene by transforming images to alter mean and deviation locally. We introduce a method to extract necessary parameters using multiple frames of a video. We verify our method with the widely used physical model of aerosols, highlighting some differences from removing haze and fog - a widely studied area. Furthermore, our approach eliminates the need for complex optimization, making real-time processing possible. Results show that our method is capable of reconstructing scene in challenging cases.

Parametric active contours are efficient tools for boundary detection. However, existing external-energy-inspired methods have difficulties when detecting high curvature, noisy or low contrasted contours and they often suffer from initialization sensitivity. To address these issues, this paper introduces Harris-based Vector Field Convolution (HVFC), operating with the modified characteristic function of Harris corner detector used in the feature map of the external force component. Initial contour is calculated as the convex hull of the most salient points of the map. Experimental results show that HVFC outperforms other state-of-the-art methods, when tested on high curvature, noisy or low-contrasted contours.

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

The paper introduces a video surveillance and event detection framework and application for semi-supervised surveillance use. The systempsilas intended use is in automatic mode on camera feeds that are not actively watched by surveillance personnel, and should raise alarms when unusual events occur. We present the current detector filters, and the extendable modular interface. Filters include local and global unusual motion detectors, left/stolen object detector, motion detector, tampering/failure detector, etc. The system stores the events and associated data, which can be organized, searched, annotated and (re)viewed. It has been tested in real life situation for police street surveillance.

Describing local patches to register image keypoints is an important task for building a huge database from video frames. When searching for an efficient descriptor, task is twofold: features must describe the featuring patches at a high efficiency, while the dimensionality should be kept at a manageable low value. The main assumption in finding local descriptors is the defect of continuity in the discrete neighborhood or the imperfectness of local shape formats. Curve fitting methods for noisy shapes are called: active contours are generated around keypoints. Local contours are characterized by a small number of Fourier descriptors, resulting a new feature set of low dimensionality. Similarity among different images are searched through these descriptors. The method was tested on 22 real-life video frames made by an outdoor surveillance camera of a city police central.

In this paper, we propose a probabilistic model for detecting relevant changes in registered aerial image pairs taken with the time differences of several years and in different seasonal conditions. The introduced approach, called the conditional mixed Markov model, is a combination of a mixed Markov model and a conditionally independent random field of signals. The model integrates global intensity statistics with local correlation and contrast features. A global energy optimization process ensures simultaneously optimal local feature selection and smooth observation-consistent segmentation. Validation is given on real aerial image sets provided by the Hungarian Institute of Geodesy, Cartography and Remote Sensing and Google Earth.

We propose a new Bayesian method for detecting the regions of object displacements in aerial image pairs. We use a robust but coarse 2-D image registration algorithm. Our main challenge is to eliminate the registration errors from the extracted change map. We introduce a three-layer Markov random field (L(3)MRF) model which integrates information from two different features, and ensures connected homogenous regions in the segmented images. Validation is given on real aerial photos.

Extraction of foreground is a basic task in surveillance video analysis. In most real cases, its performance is heavily based on the efficiency of shadow detection and on the analysis of lighting conditions and reflections caused by mirrors or other reflective surfaces. This correspondence is focused on the improvement of foreground extraction in the case of planar reflective surfaces. We show that the geometric model of a scene with a planar reflective surface is reduced to the estimation of vanishing-point for the case of an auto-epipolar (skew-symmetric) fundamental matrix. The correspondences for the vanishing-point estimation are extracted from motion statistics. The knowledge of the position of the vanishing point allows us to integrate the geometric model and the motion statistics into image foreground-extraction to separate foreground from reflections, and thus to achieve better performance. The experiments confirm the accuracy of the vanishing point and the improvement of the foreground image mask by removing reflected object parts.

In the paper we propose a novel multi-layer Mixed Markov model for detecting relevant changes in registered aerial images taken with significant time differences. The introduced approach combines global intensity statistics with local correlation and contrast features. A global energy optimization process simultaneously ensures optimal local feature selection and smooth, observation-consistent classification. Validation is given on real aerial photos.

In in this paper, we propose a new model regarding foreground and shadow detection in video sequences. The model works without detailed a priori object-shape information, and it is also appropriate for low and unstable frame rate video sources. Contribution is presented in three key issues: 1) we propose a novel adaptive shadow model, and show the improvements versus previous approaches in scenes with difficult lighting and coloring effects; 2) we give a novel description for the foreground based on spatial statistics of the neighboring pixel values, which enhances the detection of background or shadow-colored object parts; 3) we show how microstructure analysis can be used in the proposed framework as additional feature components improving the results. Finally, a Markov random field model is used to enhance the accuracy of the separation. We validate our method on outdoor and indoor sequences including real surveillance videos and well-known benchmark test sets.

In this paper, we give a probabilistic model for automatic change detection on airborne images taken with moving cameras. To ensure robustness, we adopt an unsupervised coarse matching instead of a precise image registration. The challenge of the proposed model is to eliminate the registration errors, noise and the parallax artifacts caused by the static objects having considerable height (buildings, trees, walls etc.) from the difference image. We describe the background membership of a given image point through two different features, and introduce a novel three-layer Markov random field (MRF) model to ensure connected homogenous regions in the segmented image.

We present an automatic focus area estimation method, working with a single image without a priori information about the image, the camera, or the scene. It produces relative focus maps by localized blind deconvolution and a new residual error-based classification. Evaluation and comparison is performed and applicability is shown through image indexing.

This paper presents a robust walk-detection algorithm, based on our symmetry approach which can be used to extract gait characteristics from video-image sequences. To obtain a useful descriptor of a walking person, we temporally track the symmetries of a person's legs. Our method is suitable for use in indoor or outdoor surveillance scenes. Determining the leading leg of the walking subject is important, and the presented method can identify this from two successive walk steps (one walk cycle). We tested the accuracy of the presented walk-detection method in a possible application: Image registration methods are presented which are applicable to multicamera systems viewing human subjects in motion.

A new motion-based method is presented for automatic registration of images in multicamera systems, to permit synthesis of wide-baseline composite views. Unlike existing static-image and motion-based methods, our approach does not need any a priori information about the scene, the appearance of objects in the scene, or their motion. We introduce an entropy-based preselection of motion histories and an iterative Bayesian assignment of corresponding image areas. Finally, correlated point-histories and data-set optimization lead to the matching of the different views. The method is validated by demonstrating its successful use on several real-life indoor and outdoor stereo video image-sequence pairs.

In this paper we address the problem of estimating the position of the vanishing point in an outdoor camera-mirror environment, making use of motion statistics derived from a video sequence. Knowledge of the vanishing-point position is the key for the geometrical modeling of reflective surfaces or cast shadows; this allows the (future) integration of the model into foreground-extraction methods to achieve better performance. Thanks to the use of co-motion statistics in the correspondence-detection step, our approach gives robust results. The presented optimization method for final parameter estimation utilizes motion statistics to determine an improved fitting function in case of wide-baseline views and in cases where the correspondences are corrupted with considerable amounts of noise. Results are presented for several video sequences representing a variety of scene layouts and object types. The outcomes show that our approach gives robust results in the context of widely different environments.

In the paper a new Bayesian method is presented for the automatic extraction of common areas of images in multi-camera systems through the detection of concurrently changing pixels. Unlike existing still-image and motion-based methods our approach does not need any a priori information about the scene, the appearance of objects in the scene, or their motion. The method is validated by demonstrating its successful use on several real-life outdoor stereo video image-sequence pairs.

We outline our latest methods and algorithmic solutions for detection of concurrent motions and registering cameras in real-life surveillance systems. We show that cameras can be registered in several rather unfavorable conditions, based on: (1) unpredictable motion without structured background or defined object shapes or (2) walking persons of undefined silhouettes and short detectable walking distances or (3) shadows of undefined structures in front of flickering background. These methods are tested in real-life sequences and we show that flexible 3D camera registration is possible even in bad lighting conditions and in the lack of any known structures or motions.

In this paper we present a symmetry-based approach which can be used to detect humans and to extract biometric characteristics from video image-sequences. The method employs a simplified symmetry-feature extracted from the images. To obtain a useful descriptor of a walking person, we track temporally the symmetries which result from the movements of the person's legs. In a further processing stage these patterns are filtered, then re-sampled using Bezier-splines to generate an invariant and noise-cleaned signature or "feature". In our detection method the extracted spatio-temporal feature with a large number of dimensions (800) is transformed to a space with a much smaller number of dimensions (3), which we call the "eigenwalks space"; the method uses principal component analysis (PCA) to reduce the dimensionality, and the support vector machine (SVM) method in the eigenspace for recognition purposes. Finally we present a method by which we can estimate the gait-parameters (the beginning and end of a walk-cycle, identification of the leading leg) from the symmetry-patterns of the walking person, without camera calibration, based on two successive detected walk-steps.

We have developed a new semi-automatic neural network based method to detect blotches with low false alarm rate on archive films. Blotches can be modeled as temporal intensity discontinuities, hence false detection results originate from object motion (e.g. occlusion), non-rigid objects or erroneous motion estimation. In practice, usually, after the automatic detection step the false alarms are removed manually by an operator, significantly decreasing the efficiency of the restoration process. Our post-processing method classifies each detected blotch by its image features to minimize false results and the necessity of human intervention. The proposed method is tested on real archive sequences.

In the paper we introduce an algorithm for matching partially overlapping image-pairs where the object of interest is in motion, even if the motion is discontinuous and in an unstructured environment. In our previous work [Z. Szlavik et al, 2004] we have shown that by using co-motion statistics matching of overlapping views can be done and then the projective geometry can be estimated. Here, we show how to optimize searching for concurrently moving pixels. The robust algorithm we describe here finds point correspondences in two images by using entropy-based thresholding and without searching for any structures and without the need for tracking continuous motion. Our method makes it possible to (re)calibrate multicamera systems without human assistance.

We propose a new content based image retrieval method. The novelty of our approach lies in the applied image similarity measure: unlike traditional features, such as color, texture or shape, our measure is based on a painted representation of the original image. We use paintbrush stroke parameters as features. These strokes are produced by a stochastic paintbrush algorithm which simulates a painting process. Stroke parameters include color, orientation and location. Therefore, it provides information not only about the color content but also about the structural properties of an image. Experimental results on a database of more than 500 images show that the CBIR method using paintbrush features has a higher retrieval rate than methods using color features only.

In this paper some applications of motion analysis are investigated for a compact panoramic optical system (panoramic annular lens). Panoramic image acquisition makes multiple or mechanically controlled camera systems needless for many applications. Panoramic annular lens' main advantage to other omnidirectional monitoring systems is that it is a cheap, small, compact device with no external hyperboloidal, spherical, conical or paraboloidal reflecting surface as in other panoramic optical devices. By converting the annular image captured with an NTSC camera to a rectangular one, we get a low-resolution (2.8 pixels/degrees horizontally and 3 pixels/degree vertically) image. We developed algorithms which can analyze this low-resolution image to yield motion information for surveillance and smoke detection.

In this paper we outline a fully parallel and locally connected computation model for the spatio-temporal segmentation of motion events in video sequences. We are searching for a new algorithm, which can be easily implemented in one-pixel/one-processor cell-array VLSI architectures at high-speed. Our proposed algorithm starts from an oversegmented image, then the segments are merged by applying the information coming from the spatial and temporal auxiliary data: motion fields and motion history, which is calculated from consecutive image frames. This grouping process is defined through a similarity measure of neighboring segments, which is based on the values of intensity, speed and the time-depth of motion history. As for checking the merging process there is a feedback implemented, by that we can accept or refuse the cancellation of a segment-border. Our parallel approach is independent of the number of segments and objects, since instead of graph representation and serial processing of these components, image features are defined on the pixel-level. We use simple functions, easily realizable in VLSI, like arithmetic and logical operators, local memory transfers and convolution.

A paintbrush-like image transformation is proposed. It is based on a random searching to insert brush-strokes into a generated image at decreasing scale of brush-sizes, without predefined models or interaction. One of the goals of the method is to transform the image into a representation that is very similar to the human sensation of artistic images. We introduce a sequential multiscale image decomposition method, based on simulated rectangular-shaped paintbrush strokes. The resulting images look like good-quality paintings with well-defined contours, at an acceptable distortion compared to the original image. The image can be described with the parameters of the consecutive paintbrush strokes, resulting in a parameter-series that can be used for compression. The painting process can be used for scale-space image representation, segmentation and contour detection, and image representation for retrieval purposes.

Anisotropic diffusion is an image enhancement method. It is a nonlinear process which removes noise and irrelevant details while preserving the edges, i.e. it "extracts" the essential visual information. The paper proposes a useful application of anisotropic diffusion in image data compression. We argue that for high compression an anisotropic diffusion preprocessing results in better quality of the decoded image.

Due to the large computation power needed for Markovian random field (MRF) based image processing, new variations of the basic MRF model are implemented. The transportation of the model to the very fast cellular neural networks (CNN) gave new tasks and opportunities to improve the technique, since the CNN has a special local architecture. This CNN architecture can be implemented in real VLSI circuits of superior speed in image processing. A type of MRF image segmentation with modified metropolis dynamics (MMD) can be well implemented in the CNN architecture. In this paper we address the improvement of this existing CNN method by introducing anisotropic diffusion as the smoothing process in the model. We suggest that this new feature with the MRF representation will give a new approach to solving early vision problems in the future.

A new one-chip texture-classifier system is demonstrated with an execution time of about a few /spl mu/secs. Cellular neural networks (CNN) provide a new fast parallel computational method for VLSI image processing. CNN contain one or more layers of 2D cell-arrays with local cell-interconnections and in-cell dynamics. In this paper it is demonstrated that many of the feature mapping early vision effects can be simultaneously executed as convolution/deconvolution, cross-correlation, pattern-shifting, halftoning etc. The parameters of the CNN (a template) are trained through a genetic-like learning algorithm. Several types of Brodatz textures have been examined to test our method for classification and segmentation. Choosing 4 Brodatz textures which are close to each other in their main characteristics, they can be discriminated at a classification error of about 1-5% and segmentation error of about 5-10 pixels by using only one CNN template, even in the case of noisy or nonuniform CNN VLSI parameters. Using a 44*44 CNN array and one template, 16 Brodatz textures can be successfully discriminated.

A new statistical pattern recognition method has been developed for detection, recognition or measurement of patterns which are (much) smaller than the measure of the elementary pixel windows in the image screen. In this measurement the gray-level histogram of the objects examined is compared with the simulated histograms of different (in type or size) possible objects, and the recognition (of shape or measure) is taken on the basis of the comparison. This method does not need ultra-precise movement of the scanning sensors or any additional hardwares. Moreover, the examined pattern should be randomly distributed on the screen, or a random movement of camera (or target or both) is needed. Effect of noises are analyzed, and filtering processes are suggested in the histogram domain. Several examples of different shapes are presented through simulations and experiments.>