This paper presents a novel approach for unsupervised video summarization using reinforcement learning (RL), addressing limitations like unstable adversarial training and reliance on heuristic-based reward functions. The method operates on the principle that reconstruction fidelity serves as a proxy for informativeness, correlating summary quality with reconstruction ability. The summarizer model assigns importance scores to frames to generate the final summary. For training, RL is coupled with a unique reward generation pipeline that incentivizes improved reconstructions. This pipeline uses a generator model to reconstruct the full video from the selected summary frames; the similarity between the original and reconstructed video provides the reward signal. The generator itself is pre-trained self-supervisedly to reconstruct randomly masked frames. This two-stage training process enhances stability compared to adversarial architectures. Experimental results show strong alignment with human judgments and promising F-scores, validating the reconstruction objective. The code for this project will be available online1.

Split-Federated (SplitFed) learning is an extension of federated learning that places minimal requirements on the clients’ computing infrastructure, since only a small portion of the overall model is deployed on the clients’ hardware. In SplitFed learning, feature values, gradient updates, and model updates are transferred across communication channels. In this paper, we study the effects of noise in the communication channels on the learning process and the quality of the final model. We propose a smart averaging strategy for SplitFed learning with the goal of improving resilience against channel noise. Experiments on a segmentation model for embryo images shows that the proposed smart averaging strategy is able to tolerate two orders of magnitude stronger noise in the communication channels compared to conventional averaging, while still maintaining the accuracy of the final model.

In this paper, we present a new process for creating video summaries in an unsupervised manner. Our approach involves training a transformer encoder model to reconstruct missing frames in a video in a self-supervised way using the partially masked video as input. We then introduce an algorithm that utilizes the above-trained encoder to generate an importance score for each frame. Such frame importance scores are used to create the summary of the video. We show that the reconstruction loss of the model for a video with masked frames correlates with the representativeness of the remaining frames in the video. We validate the effectiveness of our approach on two benchmark datasets of TVSum and SumMe. We demonstrate that it outperforms state-of-the-art (SOTA) methods. Additionally, our approach is more stable during the training process compared to SOTA techniques based on generative adversarial learning. Our source code is publicly available1.

Video analysis requires both spatial and temporal data analysis, unlike image analysis, which is limited to processing only spatial information. The added complexity for analyzing videos translates into the requirement for more sophisticated algorithms with diverse data to optimize a model. Creating large video datasets for analysis tasks is challenging, especially in the medical field, where data accessibility is limited. As a result, many computationally complex methods, such as 3D-CNN models, are not well-suited for medical applications. Therefore, innovative strategies are required to train deep learning (DL)-based models for limited video data. This paper proposes a system to predict human embryo implantation outcome in In Vitro Fertilization (IVF) process by analyzing image sequences captured during incubation. However, data availability is restricted for this task as acquiring and annotating data involves several complex steps. The proposed approach focuses on utilizing morphological changes of embryos over time and linking them to the outcome. We convert embryonic microscopic videos into multivariate time series arrays and apply state-of-the-art time series classifiers to predict growth patterns and outcomes. However, these classifiers fail to utilize the temporal patterns in the data and result in poor performance. Therefore, we propose to modify the time series classifiers with attention mechanisms that can capture both short- and long-term dependencies and improve the accuracy of predicting the success of the IVF procedure. The proposed method11https://githuh.com/mehryar72/Emhryo-TSC demonstrates promising results, improving the prediction accuracy by 3.3% for Day 3 and 3.1 % for Day 5 embryo time-lapse videos. Our ensemble classifier achieved a prediction accuracy of 77.5%, a 5.2% improvement over the state-of-the-art.

Several deep learning-based models for single image super-resolution (SISR) have been presented in recent years. For different image content, however, the performance of each Super Resolution (SR) model varies. For example, one model may outperform others on images with high and intricate textures, while another may outperform others on images of structured scenes. We present a method that uses image content to select the most suitable model for SR reconstruction by taking advantage of each image's dominant content. The obtained results confirm that the proposed model delivers better results for scene-specific content.

Identification of clouds and their shadows are two major preprocessing steps for providing an effective interpretation of remotely sensed optical images. Although deep learning-based methods have proved to deliver competent performance for detecting clouds and cloud shadows, improving their generalization ability for reliable results requires a large number of training images and accurate ground truths. As creating ground truth of cloud and cloud shadow is expensive, a practical way to generate more images and their ground truths is to use data augmentation methods. We propose two new data augmentation approaches (one for generating synthetic clouds in scenes and the other one for creating cloud shadows with different levels of shade) so as to achieve natural-looking images with little or no effort required for creating their ground truths. Our experiments show that while each of these approaches is capable of boosting cloud and cloud shadow segmentation individually, the fusion of them improves detecting cloud and shadow in a simultaneous learning pipeline, leading to outperforming the state-of-the-art results.

Various protocols have been developed to improve the success rate of In Vitro Fertilization (IVF). Earlier protocols were based on embryonic cell quality on embryos’ third day. Newer protocols rely on the blastocyst quality (day-5 embryo). Artificial intelligence (AI) systems for automatic human embryo quality assessment seem to be the natural trend towards improving IVF’s outcome. AI systems can potentially reveal hidden relationships between embryos’ various attributes. To this date, most AI systems assess single blastocyst images. This paper proposes a novel approach that predicts the embryo implantation outcome from their time-lapse images. This approach consists of two models. One model evaluates each embryo based on its day-3 attributes, while the second model assesses the same embryo’s day-5 image sequence. A Data Length Schedular (DLS) algorithm is developed addressing variations in blastocyst stage sequences’ lengths. With an accuracy of 76.9%, the proposed system beats state of the art by 6%.

Identifying embryos with the highest implantation potential is one of the most critical tasks in In Vitro Fertilization (IVF) treatment. We propose a classifier for predicting an embryo's implantation outcome by analyzing time-lapse images during the blastocyst stage. We report a novel data regularization method called Timed Data Incrementation (TDI) to address the length variation in time-lapse image sequences. Sequences with variable length could add significant bias in any training-based method and ultimately lead to an over-fitting or a non-converging system. Our proposed system outperforms the reported state-of-the-art by 2.18% in accuracy (73.08%). The current state-of-the-art is only trained and tested on a single blastocyst image from many embryos. However, to the best of our knowledge, we are the first to utilize time-lapse sequences for embryo implantation outcome prediction. Finally, We show that TDI could benefit other AI-based systems requiring analyzing videos with different capture frequencies or lengths in various applications.

The impact of marine life on the oceans of our planet is undeniable and overfishing is a serious threat to marine ecosystems worldwide. Maritime domain awareness calls for continuous monitoring and tracking of fisheries using data from maritime intelligence sources to detect illegal fishing activities. Marine traffic data from vessel tracking services is a promising source for identifying, locating, and capturing vessel information. Given the volume of such data, manual processing is impossible, raising an immediate need for autonomous and smart systems to follow the footprints of vessels and detect their activity types in near real-time. To achieve this goal, we propose FishNET, a simple yet effective convolutional neural network (CNN) model for vessel trajectory classification. The model is trained using a set of invariant spatiotemporal feature sequences extracted from the behavioral characteristics of vessel movements.

Couples suffering from infertility issues often use In Vitro Fertilization (IVF) treatment to give birth. Continuous embryo monitoring with time-lapse imaging enables time-based development metrics alongside visual features to assess an embryo's quality before transfer. Tracking embryonic cell development provides valuable information about its likelihood of leading to a positive pregnancy. Automating this task is challenging due to cell overlap, occlusion, and variation. In this paper, cell stage is identified by counting detected cell centroids in early embryo time-lapse sequences. A convolutional regression network is trained on Gaussian-annotated centroid maps to localize cell centroids. Added network attention blocks encode spatio-temporal relationship in time-lapse sequences to emphasize relevant features in the current frame based on previous frame and cell movement. The proposed approach was applied to 108 embryo sequences including 1- to 4-cell stage, achieving cell centroid localization distance error of 3.98 pixels, cell detection rate 80.9%, and cell counting accuracy of 80.2%.

Cloud segmentation is a vital task in applications that utilize satellite imagery. A common obstacle in using deep learning-based methods for this task is the insufficient number of images with their annotated ground truths. This work presents a content-aware unpaired image-to-image translation algorithm. It generates synthetic images with different land cover types from original images, while preserving the locations and the intensity values of the cloud pixels. Therefore, no manual annotation of ground truth in these images is required. The visual and numerical evaluations of the generated images by the proposed method prove that their quality is better than that of competitive algorithms.

Components of a human blastocyst (day-5 embryo) and their morphological attributes highly correlate with the embryo's potentials for a viable pregnancy. Automatic semantic segmentation of human blastocyst components is a crucial step toward achieving objective quality assessment of such blastocyst. In this paper, a semantic segmentation system is proposed for human blastocyst components in microscopic images. The proposed Blast-Net features two novel components: a Cascaded Atrous Pyramid Pooling (CAPP) module to incorporate multi-scale global contextual priors, and a Dense Progressive Sub-pixel Upsampling (DPSU) module to recover the high-resolution prediction map. Experimental results confirm that the proposed method achieves the best-reported segmentation performance to date with a mean Jaccard Index of 82.85 % for microscopic images of the human blastocyst.

Cloud detection in satellite images is an important first-step in many remote sensing applications. This problem is more challenging when only a limited number of spectral bands are available. To address this problem, a deep learning-based algorithm is proposed in this paper. This algorithm consists of a fully convolutional network (FCN) that is trained by multiple patches of Landsat 8 images. This network, which is called Cloud-Net, is capable of capturing global and local cloud features in an image using its convolutional blocks. Since the proposed method is an end-to-end solution no complicated pre-processing step is required. Our experimental results prove that the proposed method outperforms the state-of-the-art method over a benchmark dataset by 8.7% in Jaccard Index.

Quality scores assigned to blastocyst inner cell mass (ICM), trophectoderm (TE), and zona pellucida (ZP) are critical markers for predicting implantation potential of a human blastocyst in IVF treatment. Deep Convolutional Neural Networks (CNNs) have shown success in various image classification tasks, including classification of blastocysts into two quality categories. However, the problem of multi-label multi-class classification for blastocyst grading remains unsolved. In this paper, a single CNN is proposed to automatically classify human microscopic blastocyst images in three grading labels, each with multiple quality classes. Additionally, network training with severely imbalanced data (i.e. minority class representing <5% of total samples) is addressed using parameter sharing and stratified sampling schemes. Network parameters associated with each output label are trained jointly, achieving accuracy of 73.9% for ICM grading, 67.3% for TE grading, and 81.8% for ZP grading. This represents accuracy improvement of 3.1%, 18.3%, and 18.2% for ICM, TE, and ZP grading, respectively, over three baseline models.

Automatic identification of shadow regions in an image is a basic and yet very important task in many computer vision applications such as object detection, target tracking, and visual data analysis. Although shadow detection is a well-studied topic, current methods for identification of shadow are not as accurate as required. In this work, we propose a deep-learning method for shadow detection at a pixel-level that is suitable for single RGB images. The proposed CNN-based method benefits from a novel architecture through which global and local shadow attributes are identified using a new and efficient mapping scheme in the skip connection. It extracts and preserves shadow context in multiple layers and utilizes them gradually in multiple blocks to generate final shadow masks. The training phase of the network is simple and can be directly and easily adapted for other image segmentation tasks. The performance of the proposed system is evaluated on three publicly available datasets sbudataset,stcgan,ucf, where it outperforms the state-of-the-art Balanced Error Rates (BER) by 3%, 6.2%, and 11.4%.

Identifying different components of a developing human embryo is a crucial step toward achieving automatic objective quality assessment of such embryo. Inner Cell Mass (ICM), part of the embryo that will eventually develop into a fetus, is one of the most important components of a human blastocyst and its morphological attributes are highly correlated with the overall quality of the embryo. In this paper, a deep learning based semantic segmentation approach is proposed to take on the challenging task of ICM segmentation. Particularly, multi-resolutional ensemble of stacked dilated U-Net is proposed for accurate segmentation of this region. Experimental results confirm that the proposed method achieves the best reported results to date with average Precision of 88.6%, Recall of 91.5%, Accuracy of 98.3%, Dice Coefficient of 89.5% and Jaccard Index of 81.6%.

Super-resolution reconstruction based on deep learning has come a long way since the first proposed method in 2015. Numerous methods have been developed for this task using deep learning approaches. Among these methods, residual deep learning algorithms have shown better performance. Although all early proposed deep learning based super-resolution frameworks used bicubic upsampled versions of low resolution images as the main input, most of the current ones use the low resolution images directly by adding up-sampling layers to their networks. In this work, we propose a new method by using both low resolution and bicubic upsampled images as the inputs to our network. The final results confirm that decreasing the depth of the network in lower resolution space and adding the bicubic path lead to almost similar results to those of the deeper networks in terms of PSNR and SSIM, yet making the network computationally inexpensive and more efficient.

This paper presents a deep-learning based framework for addressing the problem of accurate cloud detection in remote sensing images. This framework benefits from a Fully Convolutional Neural Network (FCN), which is capable of pixel-level labeling of cloud regions in a Landsat 8 image. Also, a gradient-based identification approach is proposed to identify and exclude regions of snow/ice in the ground truths of the training set. We show that using the hybrid of the two methods (threshold-based and deep-learning) improves the performance of the cloud identification process without the need to manually correct automatically generated ground truths. In average the Jaccard index and recall measure are improved by 4.36% and 3.62%, respectively.

Automatic detection of shadow regions in an image is a difficult task due to the lack of prior information about the illumination source and the dynamic of the scene objects. To address this problem, in this paper, a deep-learning based segmentation method is proposed that identifies shadow regions at the pixel-level in a single RGB image. We exploit a novel Convolutional Neural Network (CNN) architecture to identify and extract shadow features in an end-to-end manner. This network preserves learned contexts during the training and observes the entire image to detect global and local shadow patterns simultaneously. The proposed method is evaluated on two publicly available datasets of SBU and UCF. We have improved the state-of-the-art Balanced Error Rate (BER) on these datasets by 22 % and 14 %, respectively.

The time of the first cell cleavage in the embryonic development of a human embryo is an important indicator of the embryo's potential for developing into a healthy baby. The time and synchronicity of following cleavages are also linked to the quality of an embryo. In this paper, a deep learning based framework is proposed to take on the challenging task of automatic counting and centroid localization of embryonic cells (blastomeres) in microscopic images of human embryos. In particular, ensemble of residual dilated UNet is proposed to count blastomeres and localize their centroids. Experimental results confirm that the proposed framework is capable of counting blastomeres in a densely occupied and overlapping space of human embryo by an average accuracy of 88.2% for embryos of 1 - 5 cells.

The success of In-Vitro Fertilization (IVF) greatly relies on the quality of the Inner Cell Mass (ICM) obtained at day 5 of embryo development. Unfortunately, ICM segmentation is difficult due to its shape variability and unconstrained profile. This paper proposes a two-stage pipeline that first uses a preprocessing step to remove artifacts from the input images which are then used by the Fully Convolutional Networks (FCN) to produce the ICM segmentation. The paper also proposes a novel data augmentation technique, specific to this application. The pre-processing step is shown to accelerate the learning of the FCN while data augmentation avoids overfitting and lead to better generalization. The performance of the proposed pipeline is evaluated on pixel classification accuracy and Jaccard index, on a dataset of 8460 images augmented from 235 images. The proposed method outperforms the state-of-art by about 28% on Jaccard index.

Fertility specialists have linked the size, shape and position of blastomeres in humans embryos with the viability of such embryos. We propose an automatic blastomere identification and modeling approach in an attempt to aid physicians in determining embryo's viability. The proposed method applies isoperimetric graph partitioning, succeeded by a novel region merging algorithm to Hoffman Modulation Contrast (HMC) embryo images, to approximate blastomeres positions. Ellipsoidal models are then used to approximate the shape and the size of each blastomere. We discuss experimental results on a dataset of 40 embryo images, and expand on the advantages and drawbacks of our method while comparing our method to other approaches.

Building profile extraction from aerial imagery constitutes a key element in numerous geospatial applications. Rooftop detection has been addressed through a variety of approaches that are, however, rarely capable of coping with conditions such as arbitrary illumination, variant reflections, and complex building profiles. This paper proposes a new method for extracting 2-D rooftop footprints from nadir aerial imagery through a fully automatic approach that handles arbitrary illumination, variant reflections, and complex building profiles without shape priors. The proposed method combines the strength of energy-based approaches with distinctiveness of corners. Corners are assessed using multiple color and color-invariance spaces. A rooftop outline is generated from selected corner candidates and further refined to fit the best possible boundaries through level-set curve evolution that is enhanced via a mean squared error map. Experimental results confirm the ability of the presented system to effectively extract rooftop profiles with an overall average shape accuracy of 84%, correctness of 94%, completeness of 92 %, and quality of 88%.

This paper introduces a novel system for automatic detection and height estimation of buildings with polygonal shape roofs in singular satellite images. The system is capable of detecting multiple flat polygonal buildings with no angular constraints or shape priors. The proposed approach employs image primitives such as lines, and line intersections, and examines their relationships with each other using a graph-based search to establish a set of rooftop hypotheses. The height (mean height from rooftop edges to the ground) of each rooftop hypothesis is estimated using shadows and acquisition geometry. The potential ambiguities in identification of shadows in an image and the uncertainty in identifying true shadows of a building have motivated for a fuzzy logic-based approach that estimates buildings heights according to the strength of shadows and the overlap between identified shadows in the image and expected shadows according to the building profile. To reduce the time complexity of the implemented system, a maximum number of eight sides for polygonal rooftops is assumed. Promising experimental results verify the effectiveness of the presented system with overall mean shape accuracy of 94% and mean height error of 0.53 m on QuickBird satellite (0.6 m/pixel) imageries.

This paper presents an automatic point matching algorithm for establishing accurate match correspondences in two or more images. The proposed algorithm utilizes a group of feature points to explore their geometrical relationship in a graph arrangement. The algorithm starts with a set of matches (including outliers) between the two images. A set of nondirectional graphs is then generated for each feature and its K nearest matches (chosen from the initial set). Using the angular distances between edges that connect a feature point to its K nearest neighbors in the graph, the algorithm finds a graph in the second image that is similar to the first graph. In the case of a graph including outliers, the algorithm removes such outliers (one by one, according to their strength) from the graph and re-evaluates the angles until the two graphs are matched or discarded. This is a simple intuitive and robust algorithm that is inspired by a previous work. Experimental results demonstrate the superior performance of this algorithm under various conditions, such as rigid and nonrigid transformations, ambiguity due to partial occlusions or match correspondence multiplicity, scale, and larger view variation.

In this paper we present a novel perceptually-based algorithm for color quantization that produces images that consume less energy than conventionally quantized images when displayed on modern energy-adaptive displays. To evaluate the performance of the proposed algorithm, we performed a subjective study on a standard Kodak color image database. Experimental results indicate that the proposed algorithm is able to reduce the energy consumption by 4.25% on average, while achieving the same or better subjective image quality as conventional color quantization.

In this paper, we propose an unsupervised segmentation algorithm for extracting moving regions from compressed video using global motion estimation (GME) and Markov random field (MRF) classification. First, motion vectors (MVs) are compensated from global motion and quantized into several representative classes, from which MRF priors are estimated. Then, a coarse segmentation map of the MV field is obtained using a maximum a posteriori estimate of the MRF label process. Finally, the boundaries of segmented moving regions are refined using color and edge information. The algorithm has been validated on a number of test sequences, and experimental results are provided to demonstrate its advantages over state-of-the-art methods.

This paper presents a system for object recognition and localization within unknown indoor environments. The system includes a GUI design through which the user may describe an object of interest by means of color, size, and shape. A novel coarse to fine identification mechanism that incorporates multiple views of an object is then used to locate the described object within an unknown environment. The system includes a training stage in which representative information is extracted from database images. A stereo vision system, mounted on an indoor robot platform (Fig. 1), is used to retrieve the 3D location of potential match candidates in the scene and to inspect possible matches from three distinct viewpoints. Experimental evaluation is performed for indoor environments and promising results are shown for the application of this system.

This paper describes a semi-automatic algorithm for extracting liver masks of CT scan volumes. The proposed method relies on two types of information: liver's shape and its intensity characteristics. Here the liver shape information is retained by measuring the shape similarities between consecutive slices of the liver's CT scans. This is done through a deformable registration scheme. The liver intensity is utilized by a multi-layer image segmentation algorithm that emphasizes on the true boundaries of the liver. The proposed algorithm is tested for MICCAI 2007 grand challenge workshop dataset. The average results for volumetric overlap error and relative volume difference is 11.12% and 2.21% respectively.

This paper presents a novel height estimation method for buildings with complex rooftop contours in the presence of partial occlusions or interference by neighboring buildings in monocular satellite/aerial images. The proposed method employs fuzzy rules to estimate the most accurate heights even for buildings with partial occlusions using projected shadows of each building. The system utilizes the genetic algorithm for finding the best solution in a search space that is not limited to the traditional incremental height steps. The proposed method is independent of the acquisition method and can be used for both satellite and aerial imageries. Experimental results verify the effectiveness of the proposed method with mean height error of 27 cm for satellite and 15 cm for aerial images.

In this paper, we propose an unsupervised segmentation algorithm for extracting moving objects/regions from compressed video using Markov Random Field (MRF) classification. First, motion vectors (MVs) are quantized into several representative classes, from which MRF priors are estimated. Then, a coarse segmentation map of the MV field is obtained using a maximum a posteriori estimate of the MRF label process. Finally, the boundaries of segmented moving regions are refined using color and edge information. The algorithm has been validated on a number of test sequences, and experimental results are provided to demonstrate its superiority over state-of-the-art methods.

This paper introduces a novel automatic building detection method for aerial images. The proposed method incorporates a hierarchical multilayer feature based image segmentation technique using color. A number of geometrical/regional attributes are defined to identify potential regions in multiple layers of segmented images. A tree-based mechanism is utilized to inspect segmented regions using their spatial relationships with each other and their regional/geometrical characteristics. This process allows the creation of a set of candidate regions that are validated as rooftops based on the overlap between existing and predicted shadows of each region according to the image acquisition information. Experimental results show an overall shape accuracy and completeness of 96%.

We present a novel method for correctly identifying and straightening one dimensional agarose electrophoretic lanes. Our method has been shown to yield comparable accuracy with manual lane tracking results, and to successfully process 98% of DNA fingerprinting gels with no human intervention.

In this paper a new method is presented for finding data clusters centroids. This method, called Force, is based on the concepts of electrostatic fields in which the centroids are positioned at locations where an electrostatic equilibrium or balance could be achieved. After determining the centroids locations, criteria such as minimum distance to centroid can be used for clustering data points. The performance of the proposed method is compared against the k-means algorithm through simulation experiments. Experimental results show that the Force algorithm does not suffer from problems associated with k-means, such as sensitivity to noise and initial selection of centroids, and tendency to converge to poor local optimum. In fact, we show that this algorithm always converges to global equilibrium points, regardless of the initial guesses, and even in presence of high levels of noise.

This paper presents a new solution for automatic polygonal rooftop extraction (with no angular constraint) in satellite/aerial imageries based on line intersections. The proposed approach uses edge definitions and their relationships with each other to create a set of potential vertices. Using a graph representation, the relationship between potential vertices are studied in an efficient way. Polygonal rooftops correspond to closed loops in this graph. The experimental results for images acquired from Google Earth show that this solution has a high precision in detecting polygonal rooftops.

Automatic creation of 3D urban city maps could be an innovative way for providing geometric data for varieties of applications such as civilian emergency situations, natural disaster management, military situations, and urban planning. Reliable and consistent extraction of quantitative information from remotely sensed imagery is crucial to the success of any of the above applications. This paper describes the development of an automated roof detection system from single monocular electro-optic satellite imagery. The system employs a fresh approach in which each input image is segmented at several levels. The border line definition of such segments combined with line segments detected on the original image are used to generate a set of quadrilateral rooftop hypotheses. For each hypothesis a probability score is computed that represents the evidence of true building according to the image gradient field and line segment definitions. The presented results demonstrate that the system is capable of detecting small gabled residential rooftops with variant light reflection properties with high positional accuracies.

In this paper, a novel algorithm for foreground detection and shadow removal is presented. The proposed method employs a region-based approach by processing two foregrounds resulted from gradient-and color-based background subtraction methods. The performance of the system is compared against conventional approaches for five indoor and outdoor video sequences. Experimental results confirm that the detection rate exceeds 90%, and the robustness is greatly improved.

This paper describes a vision-based system for 3-D localization of a mobile robot in a natural environment. The system includes a mountable head with three on-board charge-coupled device cameras that can be installed on the robot. The main emphasis of this paper is on the ability to estimate the motion of the robot independently from any prior scene knowledge, landmark, or extra sensory devices. Distinctive scene features are identified using a novel algorithm, and their 3-D locations are estimated with high accuracy by a stereo algorithm. Using new two-stage feature tracking and iterative motion estimation in a symbiotic manner, precise motion vectors are obtained. The 3-D positions of scene features and the robot are refined by a Kalman filtering approach with a complete error-propagation modeling scheme. Experimental results show that good tracking and localization can be achieved using the proposed vision system.

This paper describes a vision-based system for 3D localization and tracking of a mobile robot in an unmodified environment. The system includes a mountable head with three on-board stereo CCD cameras that can be installed on the robot. There the main emphasis is on the ability to estimate the geometric information of the robot independently from any prior scene knowledge, landmark or extra sensory device. Distinctive scene features are identified using a novel algorithm and their 3D locations are estimated with a stereo algorithm. Using multi-stage feature tracking and motion estimation in a symbolic manner, precise motion vectors are obtained. The 3D positions of the scene features are updated by a Kalman filtering process. Experimental results show that robust tracking and localization can be achieved using our vision system.

Corner extraction is an important task in many computer vision systems. The quality of the corners and the efficiency of the detection method are two very important aspects that can greatly impact the accuracy, robustness and real-time performance of the corresponding corner-based vision system. In this paper we introduce a fast corner detector based on local binary-image regions. We verify the performance of the proposed method by measuring the repeatability rate under various illumination, scale and motion conditions. Our experimental results show that while the quality of the features is comparable with other conventional methods, ours delivers a faster performance.

This paper presents a vision-based tracking system suitable for autonomous robot vehicle guidance. The system includes a head with three on-board CCD cameras, which can be mounted anywhere on a mobile vehicle. By processing consecutive trinocular sets of precisely aligned and rectified images, the local 3D trajectory of the vehicle in an unstructured environment can be tracked. First, a 3D representation of stable features in the image scene is generated using a stereo algorithm. Next, motion is estimated by trading matched features over time. The motion equation with 6-DOF is then solved using an iterative least squares fit algorithm. Finally, a Kalman filter implementation is used to optimize the world representation of scene features.