A crucial step for improving the usability of panoramas is the accurate alignment of the upright vector to match the direction of gravity. This paper introduces an anchor-based method for estimating the upright vector in panoramas. Our method, named AnchorsUp, combines the strengths of classification and regression by selecting a rough estimate of the upright vector from a predefined set of anchor vectors, followed by a fine-grained adjustment through regression to achieve precise alignment. We evaluate AnchorsUp on the SUN360 dataset and show superior performance compared to state-of-the-art methods, particularly for angular errors below 5 and 12 degrees, considered the most relevant in practical applications. The source code and dataset information are available at https://github.com/mabergmann/anchorsup.

In the first half of 2024, thousands of pets were separated from their guardians due to the floods in Rio Grande do Sul, Brazil. Tools that help guardians search for their lost animals are pivotal in such situations. Such a tool might allow users (in this case, guardians or volunteers from shelters) to send images of the animals, allowing manual or automated searches to match lost and rescued animals. This work presents a novel dataset – the PetsRS dataset1– containing organized tuples of pictures of dogs and cats for pet retrieval. We show pet recognition results using pre-trained off-the-shelf image encoders and the K-NN algorithm with embeddings as data points. In our results, DINOv2 achieves the best recall@K on all tests. This work aims to create a benchmark and baseline for evaluating techniques that tackle pet recognition.

This paper presents a single-image approach for classifying 3D objects represented as meshes. Our method centers a virtual spherical camera at the object’s centroid and casts omnidirectional rays. Then, it computes local geometry information of each ray’s first and last intersection points, generating a single multi-channel equirectangular (ERP) image per object. We propose a convolutional block named Horizontally Stacked Dilated Convolution (HSDC) to handle ERP distortions and introduce a classifier built upon these blocks. Our experiments in popular datasets show that the results produced by our method are competitive or better than state-of-the-art voxel-and point-based methods, being the best among single-view approaches. Code is available at https://github.com/rmstringhini/HSDCNet.

Aligned color and depth images allow exploring a scene by projecting the virtual camera to arbitrary positions. Depth-image-based rendering (DIBR) methods are often adopted in practical solutions, requiring fewer data than other approaches that explore stereo or multi-view setups. As the virtual camera moves, artifacts are expected to appear, which must be tackled adequately. For video sequences, not only spatial but also temporal inconsistencies affect the user experience. This paper proposes a background model that can be coupled to reduce the flickering of state-of-the-art still-image DIBR methods applied to video sequences captured by static cameras. Qualitative and quantitative results show the potential of the use of our method in practical applications.

The discrete cosine transform (DCT) is a relevant tool in signal processing applications, mainly known for its good decorrelation properties. Current image and video coding standards—such as JPEG and HEVC—adopt the DCT as a fundamental building block for compression. Recent works have introduced low-complexity approximations for the DCT, which become paramount in applications demanding real-time computation and low-power consumption. The design of DCT approximations involves a trade-off between computational complexity and performance. This paper introduces a new multiparametric transform class encompassing the round-off DCT (RDCT) and the modified RDCT (MRDCT), two relevant multiplierless 8-point approximate DCTs. The associated fast algorithm is provided. Four novel orthogonal low-complexity 8-point DCT approximations are obtained by solving a multicriteria optimization problem. The optimal 8-point transforms are scaled to lengths 16 and 32 while keeping the arithmetic complexity low. The proposed methods are assessed by proximity and coding measures with respect to the exact DCT. Image and video coding experiments and hardware realization are performed. The novel transforms perform close to or outperform the current state-of-the-art DCT approximations.

View synthesis allows observers to explore static scenes using aligned color images and depth maps captured in a preset camera path. Among the options, depth-image-based rendering (DIBR) approaches have been effective and efficient since only one pair of color and depth map is required, saving storage and bandwidth. The present work proposes a novel DIBR pipeline for view synthesis that properly tackles the different artifacts that arise from 3D warping, such as cracks, disocclusions, ghosts, and out-of-field areas. A key aspect of our contributions relies on the adaptation and usage of a hierarchical image superpixel algorithm that helps to maintain structural characteristics of the scene during image reconstruction. We compare our approach with state-of-the-art methods and show that it attains the best average results in two common assessment metrics under public still-image and video-sequence datasets. Visual results are also provided, illustrating the potential of our technique in real-world applications.

This paper presents a perturbation analysis for the estimate of epipolar matrices using the 8-Point Algorithm (8-PA). Our approach explores existing bounds for singular subspaces and relates them to the 8-PA, without assuming any kind of error distribution for the matched features. In particular, if we use unit vectors as homogeneous image coordinates, we show that having a wide spatial distribution of matched features in both views tends to generate lower error bounds for the epipolar matrix error. Our experimental validation indicates that the bounds and the effective errors tend to decrease as the camera Field of View (FoV) increases, and that using the 8-PA for spherical images (that present 360°x180° FoV) leads to accurate essential matrices. As an additional contribution, we present bounds for the direction of the translation vector extracted from the essential matrix based on singular subspace analysis.

In this paper we compare the quality of synthesized views produced by four DIBR methods when fed by depth maps estimated by five state-of-the-art stereo matching algorithms. Also, we compute the correlation between four popular metrics for ranking stereo matching algorithms and two metrics commonly used to evaluate synthesized views (PSNR and SSIM) plus one specific for DIBR. Among our findings, we highlight that (i) PSNR and SSIM have a weak correlation with common stereo matching metrics, (ii) using ground-truth depth does not lead necessarily to the best DIBR result; and (iii) estimated depth maps present artifacts that affect differently DIBR methods.

We propose a novel method for estimating the 3D geometry of indoor scenes based on multiple spherical images. Our technique produces a dense depth map registered to a reference view so that depth-image-based-rendering (DIBR) techniques can be explored for providing three-degrees-of-freedom plus immersive experiences to virtual reality users. The core of our method is to explore large displacement optical flow algorithms to obtain point correspondences, and use cross-checking and geometric constraints to detect and remove bad matches. We show that selecting a subset of the best dense matches leads to better pose estimates than traditional approaches based on sparse feature matching, and explore a weighting scheme to obtain the depth maps. Finally, we adapt a fast image-guided filter to the spherical domain for enforcing local spatial consistency, improving the 3D estimates. Experimental results indicate that our method quantitatively outperforms competitive approaches on computer-generated images and synthetic data under noisy correspondences and camera poses. Also, we show that the estimated depth maps obtained from only a few real spherical captures of the scene are capable of producing coherent synthesized binocular stereoscopic views by using traditional DIBR methods.

In this paper we propose a framework for inferring depth from a single spherical image, which can be coupled to any generic planar image monocular depth estimation algorithm. It consists of first inferring depth from overlapping planar patches extracted from the spherical image, and then using a regularized minimization scheme to stitch the patches back to the sphere. We test three state-of-the-art convolutional neural network (CNN)-based methodologies as baseline methods, and for all of them the proposed approach presented better results than applying the CNN directly to the equirectangular projection and to disjoint sections of the sphere according to the scale-invariant mean squared error (SIMSE) metric.

The growing need for urban mobility implies new challenges for traffic management. Since vehicular networks (VANETs) will be available in future, new solutions can be applied to intersection control, offering more agility to transportation networks. While VANETs are not still widely deployed, this work explores the application of different intersection control policies on urban roads using ubiquitous simulation as support. Implementation of different policies to intersection control were made. Experimental evaluation shown that the deployment of VANETs and the use of other policies — other than those based on signal-timing plans — offer benefits to traffic management.