Current methods for embedding-based query answering over incomplete Knowledge Graphs (KGs) only focus on inductive reasoning, i.e., predicting answers by learning patterns from the data, and lack the complementary ability to do deductive reasoning, which requires the application of domain knowledge to infer further information. To address this shortcoming, we investigate the problem of incorporating ontologies into embedding-based query answering models by defining the task of embedding-based ontology-mediated query answering. We propose various integration strategies into prominent representatives of embedding models that involve (1) different ontology-driven data augmentation techniques and (2) adaptation of the loss function to enforce the ontology axioms. We design novel benchmarks for the considered task based on the LUBM and the NELL KGs and evaluate our methods on them. The achieved improvements in the setting that requires both inductive and deductive reasoning are from 20% to 55% in HITS@3.

This paper introduces a novel algorithm for transductive inference in higher-order MRFs, where the unary energies are parameterized by a variable classifier. The considered task is posed as a joint optimization problem in the continuous classifier parameters and the discrete label variables. In contrast to prior approaches such as convex relaxations, we propose an advantageous decoupling of the objective function into discrete and continuous subproblems and a novel, efficient optimization method related to ADMM. This approach preserves integrality of the discrete label variables and guarantees global convergence to a critical point. We demonstrate the advantages of our approach in several experiments including video object segmentation on the DAVIS data set and interactive image segmentation.

This paper addresses the problem of simultaneous estimation of different linear deformations, resulting in a global non-linear transformation, between an original object and its broken fragments. A general framework is proposed without using correspondences, where the solution of a polynomial system of equations directly provides the parameters of the alignment. We quantitatively evaluate the proposed algorithm on a large synthetic dataset containing 2D and 3D images, where linear (rigid-body and affine) transformations are considered. We also conduct an exhaustive analysis of the robustness against segmentation errors and the numerical stability of the proposed method. Moreover, we present experiments on 2D real images as well as on volumetric medical images.

In this paper we address the problem of semantic segmentation. Inspired by the significant role of the context information, in this task our solution makes use of semantically meaningful overlapping object hypotheses augmented by contextual information, which is obtained from a novel background mining procedure. More precisely, a fully connected conditional random field is considered over a set of overlapping segment hypotheses and the unlabeled background regions are learned from a training set and applied in the unary terms corresponding to the foreground regions. The final segmentation result is obtained via maximum a posteriori inference, in which the segments are merged based on a sequential aggregation followed by morphological hole filling and superpixel refinement serving as postprocessing. Moreover, by incorporating other kinds of contextual cues, such as global image classification and object detection cues, a new state-of-the-art performance is achieved by our proposed solution as experimentally verified on the challenging PASCAL Visual Object Class (VOC) Challenge 2012 and MSRC-21 object segmentation data sets.

This paper addresses the problem of semantic segmentation, where the possible class labels are from a predefined set. We exploit top-down guidance, i.e., the coarse localization of the objects and their class labels provided by object detectors. For each detected bounding box, figure-ground segmentation is performed and the final result is achieved by merging the figure-ground segmentations. The main idea of the proposed approach, which is presented in our preliminary work, is to reformulate the figure-ground segmentation problem as sparse reconstruction pursuing the object mask in a nonparametric manner. The latent segmentation mask should be coherent subject to sparse error caused by intra-category diversity; thus, the object mask is inferred by making use of sparse representations over the training set. To handle local spatial deformations, local patch-level masks are also considered and inferred by sparse representations over the spatially nearby patches. The sparse reconstruction coefficients and the latent mask are alternately optimized by applying the Lasso algorithm and the accelerated proximal gradient method. The proposed formulation results in a convex optimization problem; thus, the global optimal solution is achieved. In this paper, we provide theoretical analysis of the convergence and optimality. We also give an extended numerical analysis of the proposed algorithm and a comprehensive comparison with the related semantic segmentation methods on the challenging PASCAL visual object class object segmentation datasets and the Weizmann horse dataset. The experimental results demonstrate that the proposed algorithm achieves a competitive performance when compared with the state of the arts.

In this paper we address the problem of automatically parsing the fashion images with weak supervision from the user-generated color-category tags such as “red jeans” and “white T-shirt”. This problem is very challenging due to the large diversity of fashion items and the absence of pixel-level tags, which make the traditional fully supervised algorithms inapplicable. To solve the problem, we propose to combine the human pose estimation module, the MRF-based color and category inference module and the (super)pixel-level category classifier learning module to generate multiple well-performing category classifiers, which can be directly applied to parse the fashion items in the images. Besides, all the training images are parsed with color-category labels and the human poses of the images are estimated during the model learning phase in this work. We also construct a new fashion image dataset called Colorful-Fashion, in which all 2,682 images are labeled with pixel-level color-category labels. Extensive experiments on this dataset clearly show the effectiveness of the proposed method for the weakly supervised fashion parsing task.

Numerous existing object segmentation frameworks commonly utilize the object bounding box as a prior. In this paper, we address semantic segmentation assuming that object bounding boxes are provided by object detectors, but no training data with annotated segments are available. Based on a set of segment hypotheses, we introduce a simple voting scheme to estimate shape guidance for each bounding box. The derived shape guidance is used in the subsequent graph-cut-based figure-ground segmentation. The final segmentation result is obtained by merging the segmentation results in the bounding boxes. We conduct an extensive analysis of the effect of object bounding box accuracy. Comprehensive experiments on both the challenging PASCAL VOC object segmentation dataset and GrabCut-50 image segmentation dataset show that the proposed approach achieves competitive results compared to previous detection or bounding box prior based methods, as well as other state-of-the-art semantic segmentation methods.

In this paper, we propose a novel framework to estimate the parameters of a diffeomorphism that aligns a known shape and its distorted observation. Classical registration methods first establish correspondences between the shapes and then compute the transformation parameters from these landmarks. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly gives the parameters of the aligning transformation. The proposed method provides a generic framework to recover any diffeomorphic deformation without established correspondences. It is easy to implement, not sensitive to the strength of the deformation, and robust against segmentation errors. The method has been applied to several commonly used transformation models. The performance of the proposed framework has been demonstrated on large synthetic data sets as well as in the context of various applications.

Images taken from different views of a planar object are related by planar homography. Recovering the parameters of such transformations is a fundamental problem in computer vision with various applications. This paper proposes a novel method to estimate the parameters of a homography that aligns two binary images. It is obtained by solving a system of nonlinear equations generated by integrating linearly independent functions over the domains determined by the shapes. The advantage of the proposed solution is that it is easy to implement, less sensitive to the strength of the deformation, works without established correspondences and robust against segmentation errors. The method has been tested on synthetic as well as on real images and its efficiency has been demonstrated in the context of two different applications: alignment of hip prosthesis X-ray images and matching of traffic signs.

A direct approach for parametric estimation of 2D affine deformations between compound shapes is proposed. It provides the result as a least-square solution of a linear system of equations. The basic idea is to fit Gaussian densities over the objects yielding covariant functions, which preserves the effect of the unknown transformation. Based on these functions, linear equations are constructed by integrating nonlinear functions over appropriate domains. The main advantages are: linear complexity, easy implementation, works without any time consuming optimization or established correspondences. Comparative tests show that it outperforms state-of-the-art methods both in terms of precision, robustness and complexity.

A novel approach is proposed to estimate the parameters of a diffeomorphism that aligns two binary images. Classical approaches usually define a cost function based on a similarity metric and then find the solution via optimization. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly provides the parameters of the aligning transformation. The proposed method works without any time consuming optimization step or established correspondences. The advantage of our algorithm is that it is easy to implement, less sensitive to the strength of the deformation, and robust against segmentation errors. The efficiency of the proposed approach has been demonstrated on a large synthetic dataset as well as in the context of an industrial application.

We consider the problem of planar object registration on binary images where the aligning transformation is restricted to the group of affine transformations. Previous approaches usually require established correspondences or the solution of nonlinear optimization problems. Herein we show that it is possible to formulate the problem as the solution of a system of up to third order polynomial equations. These equations are constructed in a simple way using some basic geometric information of binary images. It does not need established correspondences nor the solution of complex optimization problems. The resulting algorithm is fast and provides a direct solution regardless of the magnitude of transformation.