Registration of deformable 3D shapes is a fundamental problem for several applications of computer vision including shape analysis, object detection, tracking, etc. Existing methods often require initial correspondences to match shapes with large deformations. However state-of-the-art geometric models fail when the initial correspondence set is not accurate. In this paper, we present a framework which is able to perform dense registration of pairwise shapes from inexact correspondence set. The proposed method is based on two stage refinements. In the initial stage we consider the local geometrical structure of the shapes and extract the surface patches based on shortest path algorithm from the initial correspondences. The surface patches are then rigidly aligned by a global optimization algorithm, which is further refined by computing functional map on the local structure. Transformed local patches are re-initialized to perform the next stage map computation at the global level. We demonstrate the effectiveness of the approach by performing experiments on standard datasets by comparing with several state-of-the-art methods.

Reconstruction of 3D point cloud from single view image is an extremely demanding as well as challenging task for several applications of computer vision and robotics. Although several works have been done in last few years, the problem is still not fully solved. In this work, we introduce BicycleDualNet, a novel architecture exploiting BicycleGAN and dual image encoder for 3D reconstruction. While existing methods overlook the depth or normal information in the reconstruction process, we demonstrate that incorporating the depth/normal information in the network can lead to better results. A base encoder using EfficientNet is first trained to match a pre-trained point cloud autoencoder’s latent space. To further improve its performance, BicycleGAN is then used to generate synthetic depth and normal images from input RGB images. A dual encoder is proposed with two branches for RGB and depth/normal images. This dual encoder is trained to predict 3D point clouds through the fixed point cloud decoder. Qualitative and quantitative experiments on both synthetic and real datasets demonstrate that the dual encoder performs better than the state-of-the-art in 3D reconstruction in terms of Chamfer distance and Earth Mover’s distance loss.

Tracking of plant organs over spatio-temporal sequence of point cloud data is one of the demanding tasks of agricultural robotics for automated plant monitoring and growth analysis. Due to the complex geometry of plants, it is extremely difficult to identify and track the individual organs in different growth stages of plants. In this paper, we present an approach to perform correspondence estimation of different plant organs over a series of spatio-temporal data. The approach is based on two stages. In the first stage we develop a robust skeleton extraction method from unstructured plant point cloud data by adopting Hausdorff distance metric and modified breadth first search algorithm. The proposed skeletonization method is shown to be performing better than state-of-the-art, especially in handling very thin and delicate branches. We also address an overlooked problem of connecting skeleton points in the form of a graph, and demonstrate that different types of plant phenotype parameters can be obtained in a fully automatic manner from the skeleton graph. In the second stage, we exploit the skeleton graphs in developing an algorithm to perform correspondence estimation among the skeleton nodes using a cosine similarity based approach. We demonstrate the effectiveness of the proposed skeletonization technique in tracking different organs of the plant by finding good quality correspondences. Experiments are performed on three datasets on real and synthetic sequence of spatio-temporal plant point cloud data to demonstrate the effectiveness of the proposed method.

Despite the significant success of document image analysis techniques, efficient Optical Character Recognition (OCR) of degraded document images still remains an open problem. Although a body of work has been reported on degraded document recognition for English language, only little attention has been paid to Indic scripts. In this work, we focus on developing a degraded OCR for Bangla, a major Indian language. In general, an OCR system includes segmentation of the foreground text part from the background followed by recognition of the extracted text. The text segmentation module aims to assign the foreground or background label to each pixel of the document image. In this paper, we present a new OCR system which is particularly suitable for degraded quality Bangla document images. The contribution is two fold. In the first phase, we use a semi-supervised Markov Random Field (MRF)- based Generative Adversarial Network (GAN) model (which we call MRF-GAN ) for foreground segmentation of texts from degraded text. In the proposed MRF-GAN , we extend the concept of GAN to a multitask learning mechanism where discriminator-classifier networks differentiate between real/fake images and also assign a foreground or background label to each pixel. In the second phase, we propose to use a new encoder-decoder based recognizer that incorporates an attention-based character to a word prediction model, which has the capability of minimizing Word Error Rate (WER) . We optimize this network using a Multitask based Transfer Learning scheme (MTTL) . We perform experiments on a publicly available degraded Bangla document image dataset as well as on a new degraded printed Hindi document image dataset, which has been created as a part of the present study. Results of the experimentations demonstrate the efficacy of the proposed OCR.

We present an approach to identify the plant species from the contour information from occluded leaf image using a database of full plant leaves. Although contour based 2D shape matching has been studied extensively in the last couple of decades, matching occluded leaves with full leaf databases is an open and little worked on problem. Classifying occluded plant leaves is even more challenging than full leaf matching because of large variations and complexity of leaf structures. Matching an occluded contour with all the full contours in a database is an NP-hard problem, so our algorithm is necessarily suboptimal. First, we represent the 2D contour points as a β-Spline curve. Then, we extract interest points on these curves via the Discrete Contour Evolution (DCE) algorithm. We use subgraph matching using the DCE points as graph nodes, which produces a number of open curves for each closed leaf contour. Next, we compute the similarity transformation parameters (translation, rotation, and uniform scaling) for each open curve. We then "overlay" each open curve with the inverse similarity transformed occluded curve and use the Fréchet distance metric to measure the quality of the match, retaining the best η matched curves. Since the Fréchet metric is cheap to compute but not perfectly correlated with the quality of the match, we formulate an energy functional that is well correlated with the quality of the match, but is considerably more expensive to compute. The functional uses local and global curvature, Shape Context descriptors, and String Cut features. We minimize this energy functional using a convex-concave relaxation framework. The curve among these best η curves, that has the minimum energy, is considered to be the best overall match with the occluded leaf. Experiments on three publicly available leaf image database shows that our method is both effective and efficient, outperforming other current state-of-the-art methods. Occlusion is measured as the percentage of the overall contour (and not leaf area) that is missing. We show that our algorithm can, even for leaves with a high amounts of occlusion (say 50 percent occlusion), still identify the best full leaf match from the databases.

Handling deformation is one of the biggest challenges associated with point cloud registration. When deformation happens due to the motion of an animated object which actively changes its location and general shape, registration of two instances of the same object turns out to be a challenging task. The focus of this work is to address the problem by leveraging the complementary attributes of local and global geometric structures of the point clouds. We define an energy function which consists of local and global terms, as well as a semi-local term to model the intermediate level geometry of the point cloud. The local energy estimates the transformation parameters at the lowest level by assuming a reduced deformation model. The parameters are estimated in a closed form solution, which are then used to assign the initial probability of a stochastic model working at the intermediate level. The global energy term estimates the overall transformation parameters by minimizing a nonlinear least square function via Gauss-Newton optimization framework. The total energy is optimized in a block coordinate descent fashion, updating one term at a time while keeping others constant. Experiments on three publicly available datasets show that the method performs significantly better than several state-of-the-art algorithms in registering pairwise point cloud data.

Machine vision for plant phenotyping is an emerging research area for producing high throughput in agriculture and crop science applications. Since 2D based approaches have their inherent limitations, 3D plant analysis is becoming state of the art for current phenotyping technologies. We present an automated system for analyzing plant growth in indoor conditions. A gantry robot system is used to perform scanning tasks in an automated manner throughout the lifetime of the plant. A 3D laser scanner mounted as the robot's payload captures the surface point cloud data of the plant from multiple views. The plant is monitored from the vegetative to reproductive stages in light/dark cycles inside a controllable growth chamber. An efficient 3D reconstruction algorithm is used, by which multiple scans are aligned together to obtain a 3D mesh of the plant, followed by surface area and volume computations. The whole system, including the programmable growth chamber, robot, scanner, data transfer, and analysis is fully automated in such a way that a naive user can, in theory, start the system with a mouse click and get back the growth analysis results at the end of the lifetime of the plant with no intermediate intervention. As evidence of its functionality, we show and analyze quantitative results of the rhythmic growth patterns of the dicot Arabidopsis thaliana (L.), and the monocot barley (Hordeum vulgare L.) plants under their diurnal light/dark cycles.

Laser scanner-captured 3-D point cloud data analysis is becoming more commonly used for remote sensing and plant science applications. Because of nonrigidity and complexity, reconstructing a 3-D model of a plant is extremely challenging. Existing algorithms often fail to find correct correspondences for plantlike thin structures. We address the problem of finding 3-D junction points in plant point cloud data as a first step of this correspondence matching process. Temporarily, we transform the 3-D problem into 2-D by performing appropriate coordinate transformations to the neighborhood of each 3-D point. Our proposed method has two steps. First, a statistical dip test of multimodality is performed to detect the nonlinearity of the local 2D structure. Then, each branch is approximated by sequential random-sample-consensus line fitting and a Euclidean clustering technique. The straight line parameters of each branch are extracted using total-least-squares estimation. Finally, the straight line equations are solved to determine if they intersect in the local neighborhood. Such junction points are good candidates for subsequent correspondence algorithms. Using these detected junction points, we formulate a correspondence algorithm as a subgraph matching problem and show that, without using traditional descriptor similarity-based matching, good correspondences can be obtained by simply considering geodesic distances among graph nodes. Experiments on synthetic and real ( Arabidopsis plant) data show that the proposed method outperforms the state of the art.