Learning image representations that capture rich semantic relationships remains a significant challenge. Existing approaches are either contrastive, lacking robust theoretical guarantees, or struggle to effectively represent the partial orders inherent to structured visual-semantic data. In this paper, we introduce a nuclear norm-based loss function, grounded in the same information theoretic principles that have proved effective in self-supervised learning. We present a theoretical characterization of this loss, demonstrating that, in addition to promoting class orthogonality, it encodes the spectral geometry of the data within a subspace lattice. This geometric representation allows us to associate logical propositions with subspaces, ensuring that our learned representations adhere to a predefined symbolic structure.

Cycling is critical for cities to transition to more sustainable transport modes. Yet, safety concerns remain a critical deterrent for individuals to cycle. If individuals perceive an environment as unsafe for cycling, it is likely that they will prefer other means of transportation. Yet, capturing and understanding how individuals perceive cycling risk is complex and often slow, with researchers defaulting to traditional surveys and in-loco interviews. In this study, we tackle this problem. We base our approach on using pairwise comparisons of real-world images, repeatedly presenting respondents with pairs of road environments and asking them to select the one they perceive as safer for cycling, if any. Using the collected data, we train a siamese-convolutional neural network using a multi-loss framework that learns from individuals’ responses, learns preferences directly from images, and includes ties (often discarded in the literature). Effectively, this model learns to predict human-style perceptions, evaluating which cycling environments are perceived as safer. Our model achieves good results, showcasing this approach has a real-life impact, such as improving interventions’ effectiveness. Furthermore, it facilitates the continuous assessment of changing cycling environments, permitting short-term evaluations of measures to enhance perceived cycling safety. Finally, our method can be efficiently deployed in different locations with a growing number of openly available street-view images.

Head pose estimation has become a crucial area of research in computer vision given its usefulness in a wide range of applications, including robotics, surveillance, or driver attention monitoring. One of the most difficult challenges in this field is managing head occlusions that frequently take place in real-world scenarios. In this paper, we propose a novel and efficient framework that is robust in real world head occlusion scenarios. In particular, we propose an unsupervised latent embedding clustering with regression and classification components for each pose angle. The model optimizes latent feature representations for occluded and non-occluded images through a clustering term while improving fine-grained angle predictions. Experimental evaluation on in-the-wild head pose benchmark datasets reveal competitive performance in comparison to state-of-the-art methodologies with the advantage of having a significant data reduction. We observe a substantial improvement in occluded head pose estimation. Also, an ablation study is conducted to ascertain the impact of the clustering term within our proposed framework.

The precise estimation of camera poses within large camera networks is a foundational problem in computer vision and robotics, with broad applications spanning autonomous navigation, surveillance, and augmented reality. In this paper, we introduce a novel methodology that extends state-of-the-art Pose Graph Optimization (PGO) techniques. Departing from the conventional PGO paradigm, which primarily relies on camera-camera edges, our approach centers on the introduction of a dynamic element - any rigid object free to move in the scene - whose pose can be reliably inferred from a single image. Specifically, we consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step. This shift not only offers a solution to the challenges encountered in directly estimating relative poses between cameras, particularly in adverse environments, but also leverages the inclusion of numerous object poses to ameliorate and integrate errors, resulting in accurate camera pose estimates. Though our framework retains compatibility with traditional PGO solvers, its efficacy benefits from a custom-tailored optimization scheme. To this end, we introduce an iterative primal-dual algorithm, capable of handling large graphs. Empirical benchmarks, conducted on a new dataset of simulated indoor environments, substantiate the efficacy and efficiency of our approach.

Recent research in representation learning has shown that hierarchical data lends itself to low-dimensional and highly informative representations in hyperbolic space. However, even if hyperbolic embeddings have gathered attention in image recognition, their optimization is prone to numerical hurdles. Further, it remains unclear which applications stand to benefit the most from the implicit bias imposed by hyperbolicity, when compared to traditional Euclidean features. In this paper, we focus on prototypical hyperbolic neural networks. In particular, the tendency of hyperbolic embeddings to converge to the boundary of the Poincaré ball in high dimensions and the effect this has on few-shot classification. We show that the best few-shot results are attained for hyperbolic embeddings at a common hyperbolic radius. In contrast to prior benchmark results, we demonstrate that better performance can be achieved by a fixed-radius encoder equipped with the Euclidean metric, regardless of the embedding dimension.

A cornerstone of geometric reconstruction, rotation averaging seeks the set of absolute rotations that optimally explains a set of measured relative orientations between them. In spite of being an integral part of bundle adjustment and structure-from-motion, averaging rotations is both a non-convex and high-dimensional optimization problem. In this paper, we address it from a maximum likelihood estimation standpoint and make a twofold contribution. Firstly, we set forth a novel initialization-free primal-dual method which we show empirically to converge to the global optimum. Further, we derive what is to our knowledge, the first optimal closed-form solution for rotation averaging in cycle graphs and contextualize this result within spectral graph theory. Our proposed methods achieve a significant gain both in precision and performance.

Pose Graph Optimization (PGO) is an important problem in Computer Vision, particularly in motion estimation, whose objective consists of finding the rigid transformations that achieve the best global alignment of visual data on a common reference frame. The vast majority of PGO approaches rely on iterative techniques which refine an initial estimate until convergence is achieved. On the other hand, recent works have identified a global constraint which has cast this problem into the matrix completion domain. The success which both these formulations have had in computing accurate solutions efficiently has been overshadowed by large-scale industrial applications such as autonomous flight, self-driving cars and smart-cities, where it is necessary to fuse numerous images covering large areas but where each one of them has few pairwise observations. We propose a highly efficient algorithm to solve PGO which leverages the sparsity of the data by combining the Krylov- Schur method for spectral decomposition with Cholesky LDL factorization. Our method allows for high scalability, low computational cost and high precision, simultaneously.

We propose a method to reconstruct and cluster incomplete high-dimensional data lying in a union of low-dimensional subspaces. Exploring the sparse representation model, we jointly estimate the missing data while imposing the intrinsic subspace structure. Although we have a non-convex problem, we propose an algorithm robust to initialization. Extensive experiments with synthetic and real data show that our approach leads to significant improvements in the reconstruction and segmentation, outperforming current state of the art for both low and high-rank data.

In this paper, we aim to monitor the flow of people in large public infrastructures. We propose an unsupervised methodology to cluster people flow patterns into the most typical and meaningful configurations. By processing 3-D images from a network of depth cameras, we build a descriptor for the flow pattern. We define a data-irregularity measure that assesses how well each descriptor fits a data model. This allows us to rank flow patterns from highly distinctive (outliers) to very common ones. By discarding outliers, we obtain more reliable key configurations (classes). Synthetic experiments show that the proposed method is superior to standard clustering methods. We applied it in an operational scenario during 14 days in the X-ray screening area of an international airport. Results show that our methodology is able to successfully summarize the representative patterns for such a long observation period, providing relevant information for airport management. Beyond regular flows, our method identifies a set of rare events corresponding to uncommon activities (cleaning, special security, and circulating staff).

Due to economic and environmental issues, bicycles have been regaining their significance as a transportation vehicle in urban scenarios. To further drive this desirable trend, policy makers must have the tools to access current bicycle infrastructures and road safety concerns. Fundamental for this assessment is a deeper understanding of how cyclists use current infrastructures, if the cycling experience results in stressful events, and the conditions of the current infrastructure. We here introduce a new platform, SMARTcycling, that, by taking advantage of the mobile power available to a smartphone, captures and stores data from several sensors, namely an action camera, a cardio signal acquisition belt, and smartphone's Global Positioning System (GPS) coordinates. The data is further processed and, through visual cues, we access the cyclist driving events and road condition cues. SMARTcycling also detects the cyclist stress using the electrocardiograms (ECG) from the belt. We further contribute by making available a dataset containing the sensors data from 10 paths over two cities in Portugal. On this dataset, we show our initial promising results on event detection, road condition identification and stress assessment.

In this paper we propose a point matching algorithm that computes correspondences between images and/or 3D objects in affine camera settings. We formulate the point correspondence problem by minimizing an error function over the set of all binary decisions. This function has two components: a geometric error (akin to retro-projection error) and an image dissimilarity component. Originally a combinatorial problem we obtain its exact solution through a convex relaxation. Hinging on a recent theorem, this solution is derived by carefully designing the minimizing function. The large scale optimization problem is handled by an extremely fast algorithm based on Nesterov's projected gradient method. Experimental results show both computational efficiency and high robustness to large deviations and we demonstrate that it can cope with very hard real situations such as scenes with repetitive patterns. The methodology can be applied seamlessly to both 2D or/and 3D cameras.

Finding correspondences between feature points is one of the most relevant problems in the whole set of visual tasks. In this paper we address the problem of matching a feature vector (or a matrix) to a given subspace. Given any vector base of such a subspace, we observe a linear combination of its elements with all entries swapped by an unknown permutation. We prove that such a computationally hard integer problem is uniquely solved in a convex set resulting from relaxing the original problem. Also, if noise is present, based on this result, we provide a robust estimate recurring to a linear programming-based algorithm. We use structure-from-motion and object recognition as motivating examples.

The 3D object recognition from a single or multiple 2D images is a very important problem in the computer vision field with a wide range of real applications. Considering the affine camera model, the main issue in solving this problem is the matching process between the object's 3D points and their 2D projections. In this work, we tackle the 3D-2D matching problem. It is formulated as a finite set of independent linear programs, solved efficiently. The 2D-2D and 3D-3D are also discussed. To show the validity of the proposed method, synthetic and real experiments are performed.

In this article we explore the use of methodologies for 3D reconstruction from multiple images to recognize faces. We try to devise a strategy to tackle the problem of recognizing faces from images exhibiting strong pose (rotation and occlusion) and without prior knowledge (uncalibrated cameras, images from different sources). We do so by framing the recognition in the context of 3D structure from motion with missing data problems. In fact, recently, there has been a strong trend towards using 3D information to verify and recognize faces. However most of the state of the art works are developed over 3D sensors (3D range finders, stereo).Here we propose to do recognition by measuring the likelihood that one or more images were generated by a given 3D shape. In other words, given the subject's 3D shape, we compute the required transformation to match the probe images and measure its deviation from rigidity. This process is not straightforward due to the strong pose: face points are not visible in all images so there is a ldquomissing datardquo problem intrinsic to the formulation. One key assumption is that all images (and shape) correspond to neutral expression. Though limited in scale due to lack of large databases, a set of tests demonstrate the adequacy and good performance of the approach.