Indoor localization has been an active research area in the last decade. Various localization systems have been proposed based on different types of signals, including but not limited to WiFi, ultrawideband, inertial measurements, and visual signals. Fingerprinting-based methods are among the most popular methods due to their accuracy and ease of deployment. However, a disadvantage to fingerprinting-based methods is the training phase in which fingerprints have to be collected at known locations and stored for future localization inquiries. Recently a few methods have been proposed to alleviate this problem. In this paper, we focus on the possibility of reducing the burden of the training phase for visual (image-based) indoor localization systems by proposing a system that automatically generates the image-location database. The proposed system will be referred to as automatic visual fingerprinting that can be paired with any indoor image-based localization method. We will also validate the proposed system through extensive experiments.

Many popular fingerprinting-based indoor localization methods, such as WiFi-based localization systems, rely on a dataset of fingerprints labelled by known locations (fingerprint-location dataset) to be able to localize a user by comparing the queried fingerprint with the fingerprints in the dataset. Generating and updating such a dataset is a burden and requires significant amount of time, human effort and expertise. In this work, we propose a system to build such a dataset from scratch using crowdsourced data. Consequently we reduce the required time and effort, by leveraging the information shared by the crowd. The proposed system is based on localization of user motion, hence called LocaMotion. LocaMotion takes a supervised learning perspective towards localization of user-sent motion data. In other words, LocaMotion is formalised as a classifier that extracts and assigns features from a user motion data to a sequence of points in the building. Training and testing procedures for LocaMotion will be discussed in details followed by extensive experiments to demonstrate its validity and success to localize motion patterns and generate useful fingerprint-location datasets.

A fast and robust algorithm for motion estimation based on stereo vision is proposed. The algorithm consists of two consecutive stages. First, interest points are found and tracked between four images from two consecutive time steps. A novel robust method for tracking features is proposed for this step to address shortcomings of the traditional methods for indoor hand-held/wearable scenarios. Next, motion of the camera in time is estimated via optimizing the total reprojection error associated with the robust features found using the first stage. A key assumption is that motion between the two time steps is sufficiently small or equivalently, the frames are captured close enough in time. The proposed Visual Ego-motion Estimation (VEE), is robust due to robust feature tracking in the first step, and fast due to efficient reprojection error minimization used to estimate the six-degree-of-freedom (6DoF) motion of the camera. Experiments prove comparable/superior results compared to the state-of-the-art methods in the literature especially for indoor handheld scenarios.

A new method of image resolution up-conversion (image interpolation) based on maximum a posteriori sequence estimation is proposed. Instead of making a hard decision about the value of each missing pixel, we estimate the missing pixels in groups. At each missing pixel of the high resolution (HR) image, we consider an ensemble of candidate interpolation methods (interpolation functions). The interpolation functions are interpreted as states of a Markov model. In other words, the proposed method undergoes state transitions from one missing pixel position to the next. Accordingly, the interpolation problem is translated to the problem of estimating the optimal sequence of interpolation functions corresponding to the sequence of missing HR pixel positions. We derive a parameter-free probabilistic model for this to-be-estimated sequence of interpolation functions. Then, we solve the estimation problem using a trellis representation and the Viterbi algorithm. Using directional interpolation functions and sequence estimation techniques, we classify the new algorithm as an adaptive directional interpolation using soft-decision estimation techniques. Experimental results show that the proposed algorithm yields images with higher or comparable peak signal-to-noise ratios compared with some benchmark interpolation methods in the literature while being efficient in terms of implementation and complexity considerations.

A new method of de-interlacing is proposed. De-interlacing is revisited as the problem of assigning a sequence of interpolation methods (interpolators) to a sequence of missing pixels of an interlaced frame (field). With this assumption, our de-interlacing algorithm (de-interlacer), undergoes transitions from one interpolation method to another, as it moves from one missing pixel position to the horizontally adjacent missing pixel position in a missing row of a field. We assume a discrete countable-state Markov-chain model on the sequence of interpolators (Markov-chain states) which are selected from a user-defined set of candidate interpolators. An estimation of the optimum sequence of interpolators with the aforementioned Markov-chain model requires the definition of an efficient cost function as well as a global optimization technique. Our algorithm introduces for the first time using a nonlocal cost (NLC) scheme. The proposed algorithm uses the NLC to not only measure the fitness of an interpolator at a missing pixel position, but also to derive an approximation for transition matrix (TM) of the Markov-chain of interpolators. The TM in our algorithm is a frame-variate matrix, i.e., the algorithm updates the TM for each frame automatically. The algorithm finally uses a Viterbi algorithm to find the global optimum sequence of interpolators given the cost function defined and neighboring original pixels in hand. Next, we introduce a new MAP-based formulation for the estimation of the sequence of interpolators this time not by estimating the best sequence of interpolators but by successive estimations of the best interpolator at each missing pixel using Forward-Backward algorithm. Simulation results prove that, while competitive with each other on different test sequences, the proposed methods (one using Viterbi and the other Forward-Backward algorithm) are superior to state-of-the-art de-interlacing algorithms proposed recently. Finally, we propose motion compensated versions of our algorithm based on optical flow computation methods and discuss how it can improve the proposed algorithm.

A new method of image resolution up-conversion based on maximum a posteriori sequence estimation is proposed. At each missing pixel of the high resolution (HR) image we consider an ensemble of candidate interpolation methods (interpolator). The interpolators are interpreted as states of a finite-state machine (FSM). Accordingly, the up-scaling problem is converted to the problem of estimating the optimal sequence of interpolators corresponding to the sequence of missing HR pixel positions. We derive a parameter-free probabilistic model for this FSM to solve the estimation problem using trellis diagrams and Viterbi algorithm. The experimental results prove that the proposed algorithm results sharper HR images and higher peak signal-to-noise ratios (PSNR) comparing to many algorithms in this domain.