Our goal in this article is to propose a fast and easy to implement BRDF modeling method that provides both accurate and compact representations for all types of BRDF, i.e., isotropic or anisotropic. To achieve this objective, we use a Bayesian regression method with a Gaussian process prior which allows obtaining compact BRDF representations in a purely analytical way. For this purpose, we use a generalzed distance covariance kernel which is much better suited to BRDF features than the usual Gaussian kernel. To speed up the processing, we adapt this method to the specificities of BRDFs through an appropriate input data structure and distribution of observations so as to drastically reduce the problem dimensionality through an efficient factorization method. In this way, all calculations at both fitting and rendering steps are reduced to basic matrix products and the computation of a BRDF representation with our modeling method takes only a few seconds. Furthermore, rather than using a systematic approach as in state-of-the-art methods, the size and complexity of the BRDF representation can be adapted to the application requirements as regards the fitting accuracy and rendering constraints. Besides, our BRDF representation can be easily converted to spherical harmonics expansions, which allows easier integration in usual rendering algorithms. We also propose importance sampling methods derived from our BRDF modeling method that leads to fast and easy implementations. Experimental applications of our method to various types of isotropic and anisotropic BRDFs show that state-of-the-art methods can be outperformed in most cases by using a small set of observations for the regression.

Food segmentation, including in videos, is vital for addressing real-world health, agriculture, and food biotechnology issues. Current limitations lead to inaccurate nutritional analysis, inefficient crop management, and suboptimal food processing, impacting food security and public health. Improving segmentation techniques can enhance dietary assessments, agricultural productivity, and the food production process. This study introduces the development of a robust framework for high-quality, near-real-time segmentation and tracking of food items in videos, using minimal hardware resources. We present FoodMem, a novel framework designed to segment food items from video sequences of 360-degree unbounded scenes. FoodMem can consistently generate masks of food portions in a video sequence, overcoming the limitations of existing semantic segmentation models, such as flickering and prohibitive inference speeds in video processing contexts. To address these issues, FoodMem leverages a two-phase solution: a transformer segmentation phase to create initial segmentation masks and a memory-based tracking phase to monitor food masks in complex scenes. Our framework outperforms current state-of-the-art food segmentation models, yielding superior performance across various conditions, such as camera angles, lighting, reflections, scene complexity, and food diversity. 2 2 More details in the supplementary material. This results in reduced segmentation noise, elimination of artifacts, and completion of missing segments. We also introduce a new annotated food dataset encompassing challenging scenarios absent in previous benchmarks. Extensive experiments conducted on MetaFood3D, Nutrition5k, and Vegetables & Fruits datasets demonstrate that FoodMem enhances the state-of-the-art by 2.5% mean average precision in food video segmentation and is 58 × faster on average. The source code is available at: 3 3 https://amughrabi.github.io/foodmem . . • Introduces FoodMem, the first near-real-time food video segmentation framework. • Leverages a segmentation transformer and memory model for mask refinement. • Outperforms FoodSAM, the state-of-the-art, across diverse conditions with 2.5% higher mAP. • Achieves 58x faster processing with minimal hardware resources. • Provides a novel annotated food dataset with challenging scenarios.

Leveraging inexpensive and human intervention-based annotating methodologies, such as crowdsourcing and web crawling, often leads to datasets with noisy labels. Noisy labels can have a detrimental impact on the performance and generalization of deep neural networks. Robust models that are able to handle and mitigate the effect of these noisy labels are thus essential. In this work, we explore the open challenges of neural network memorization and uncertainty in creating robust learning algorithms with noisy labels. To overcome them, we propose a novel framework called "Bayesian DivideMix++" with two critical components: (i) DivideMix++, to enhance the robustness against memorization and (ii) Monte-Carlo MixMatch, which focuses on improving the effectiveness towards label uncertainty. DivideMix++ improves the pipeline by integrating the warm-up and augmentation pipeline with self-supervised pre-training and dedicated different data augmentations for loss analysis and backpropagation. Monte-Carlo MixMatch leverages uncertainty measurements to mitigate the influence of uncertain samples by reducing their weight in the data augmentation MixMatch step. We validate our proposed pipeline using four datasets encompassing various synthetic and real-world noise settings. We demonstrate the effectiveness and merits of our proposed pipeline using extensive experiments. Bayesian DivideMix++ outperforms the state-of-the-art models by considerable differences in all experiments. Our findings underscore the potential of leveraging these modifications to enhance the performance and generalization of deep neural networks in practical scenarios.

The creation of large-scale datasets annotated by humans inevitably introduces noisy labels, leading to reduced generalization in deep-learning models. Sample selection-based learning with noisy labels is a recent approach that exhibits promising upbeat performance improvements. The selection of clean samples amongst the noisy samples is an important criterion in the learning process of these models. In this work, we delve deeper into the clean-noise split decision and highlight the aspect that effective demarcation of samples would lead to better performance. We identify the Global Noise Conundrum in the existing models, where the distribution of samples is treated globally. We propose a per-class-based local distribution of samples and demonstrate the effectiveness of this approach in having a better clean-noise split. We validate our proposal on several benchmarks - both real and synthetic, and show substantial improvements over different state-of-the-art algorithms. We further propose a new metric, classiness to extend our analysis and highlight the effectiveness of the proposed method. Source code and instructions to reproduce this paper are available at https://github.com/aldakata/CCLM/

Physically-based synthesis of high quality imagery, including global illumination light transport phenomena, results in a significant workload, which makes interactive rendering a very challenging task. We propose a VPL-based ray tracing approach that runs entirely in the GPU and achieves interactive frame rates while handling global illumination light transport phenomena. This approach is based on clustering both shading points and VPLs and computing visibility only among clusters’ representatives. A new massively parallel K-means clustering algorithm, enables efficient execution in the GPU. Rendering artifacts, that could result from the piecewise constant approximation of the VPLs/shading points visibility function introduced by the clustering, are smoothed away by resorting to an innovative approach based on fuzzy clustering and weighted interpolation of the visibility function. The effectiveness of the proposed approach is experimentally verified for a collection of scenes, with frame rates larger than 3 fps and up to 25 fps being demonstrated.

Abstract Simulating crowds requires controlling a very large number of trajectories of characters and is usually performed using crowd steering algorithms. The question of choosing the right algorithm with the right parameter values is of crucial importance given the large impact on the quality of results. In this paper, we study the performance of a number of steering policies (i.e., simulation algorithm and its parameters) in a variety of contexts, resorting to an existing quality function able to automatically evaluate simulation results. This analysis allows us to map contexts to the performance of steering policies. Based on this mapping, we demonstrate that distributing the best performing policies among characters improves the resulting simulations. Furthermore, we also propose a solution to dynamically adjust the policies, for each agent independently and while the simulation is running, based on the local context each agent is currently in. We demonstrate significant improvements of simulation results compared to previous work that would optimize parameters once for the whole simulation, or pick an optimized, but unique and static, policy for a given global simulation context.

Abstract Efficient approximation of incident radiance functions from a set of samples is still an open problem in physically based rendering. Indeed, most of the computing power required to synthesize a photo‐realistic image is devoted to collecting samples of the incident radiance function, which are necessary to provide an estimate of the rendering equation solution. Due to the large number of samples required to reach a high‐quality estimate, this process is usually tedious and can take up to several days. In this paper, we focus on the problem of approximation of incident radiance functions on the sphere. To this end, we resort to a Gaussian Process (GP), a highly flexible function modelling tool, which has received little attention in rendering. We make an extensive analysis of the application of GPs to incident radiance functions, addressing crucial issues such as robust hyperparameter learning, or selecting the covariance function which better suits incident radiance functions. Our analysis is both theoretical and experimental. Furthermore, it provides a seamless connection between the original spherical domain and the spectral domain, on which we build to derive a method for fast computation and rotation of spherical harmonics coefficients.

Spherical Fibonacci grids (SFG) yield extremely uniform point set distributions on the sphere. This feature makes SFGs particularly well-suited to a wide range of computer graphics applications, from numerical integration, to vector quantization, among others. However, the application of SFGs to problems in which further refinement of an initial point set is required is currently not possible. This is because there is currently no solution to the problem of adding new points to an existing SFG while maintaining the point set properties. In this work, we fill this gap by proposing the extensible spherical Fibonacci grids (E-SFG). We start by carrying out a formal analysis of SFGs to identify the properties which make these point sets exhibit a nearly-optimal uniform spherical distribution. Then, we propose an algorithm (E-SFG) to extend the original point set while preserving these properties. Finally, we compare the E-SFG with a other extensible spherical point sets. Our results show that the E-SFG outperforms spherical point sets based on a low discrepancy sequence both in terms of spherical cap discrepancy and in terms of root mean squared error for evaluating the rendering integral.

We propose a theoretical framework, based on the theory of Sobolev spaces, that allows for a comprehensive analysis of quadrature rules for integration over the sphere. We apply this framework to the case of shading integrals in order to predict and analyze the performances of quadrature methods. We show that the spectral distribution of the quadrature error depends not only on the samples set size, distribution and weights, but also on the BRDF and the integrand smoothness. The proposed spectral analysis of quadrature error allows for a better understanding of how the above different factors interact. We also extend our analysis to the case of Fourier truncation-based techniques applied to the shading integral, so as to find the smallest spherical/hemispherical harmonics degree L (truncation) that entails a targeted integration error. This application is very beneficial to global illumination methods such as Precomputed Radiance Transfer and Radiance Caching. Finally, our proposed framework is the first to allow a direct theoretical comparison between quadrature- and truncation-based methods applied to the shading integral. This enables, for example, to determine the spherical harmonics degree L which corresponds to a quadrature-based integration with N samples. Our theoretical findings are validated by a set of rendering experiments.

Abstract This paper proposes optimal quadrature rules over the hemisphere for the shading integral. We leverage recent work regarding the theory of quadrature rules over the sphere in order to derive a new theoretical framework for the general case of hemispherical quadrature error analysis. We then apply our framework to the case of the shading integral. We show that our quadrature error theory can be used to derive optimal sample weights (OSW) which account for both the features of the sampling pattern and the bidirectional reflectance distribution function (BRDF). Our method significantly outperforms familiar Quasi Monte Carlo (QMC) and stochastic Monte Carlo techniques. Our results show that the OSW are very effective in compensating for possible irregularities in the sample distribution. This allows, for example, to significantly exceed the regular convergence rate of stochastic Monte Carlo while keeping the exact same sample sets. Another important benefit of our method is that OSW can be applied whatever the sampling points distribution: the sample distribution need not follow a probability density function, which makes our technique much more flexible than QMC or stochastic Monte Carlo solutions. In particular, our theoretical framework allows to easily combine point sets derived from different sampling strategies (e.g. targeted to diffuse and glossy BRDF). In this context, our rendering results show that our approach overcomes MIS (Multiple Importance Sampling) techniques.

Most recent crowd simulation algorithms equip agents with a synthetic vision component for steering. They offer promising perspectives through a more realistic simulation of the way humans navigate according to their perception of the surrounding environment. In this paper, we propose a new perception/motion loop to steering agents along collision free trajectories that significantly improves the quality of vision-based crowd simulators. In contrast with solutions where agents avoid collisions in a purely reactive (binary) way, we suggest exploring the full range of possible adaptations and retaining the locally optimal one. To this end, we introduce a cost function, based on perceptual variables, which estimates an agent's situation considering both the risks of future collision and a desired destination. We then compute the partial derivatives of that function with respect to all possible motion adaptations. The agent then adapts its motion by following the gradient. This paper has thus two main contributions: the definition of a general purpose control scheme for steering synthetic vision-based agents; and the proposition of cost functions for evaluating the perceived danger of the current situation. We demonstrate improvements in several cases.

Abstract Quasi‐Monte Carlo (QMC) methods exhibit a faster convergence rate than that of classic Monte Carlo methods. This feature has made QMC prevalent in image synthesis, where it is frequently used for approximating the value of spherical integrals (e.g. illumination integral). The common approach for generating QMC sampling patterns for spherical integration is to resort to unit square low‐discrepancy sequences and map them to the hemisphere. However such an approach is suboptimal as these sequences do not account for the spherical topology and their discrepancy properties on the unit square are impaired by the spherical projection. In this paper we present a strategy for producing high‐quality QMC sampling patterns for spherical integration by resorting to spherical Fibonacci point sets. We show that these patterns, when applied to illumination integrals, are very simple to generate and consistently outperform existing approaches, both in terms of root mean square error (RMSE) and image quality. Furthermore, only a single pattern is required to produce an image, thanks to a scrambling scheme performed directly in the spherical domain.

The Monte Carlo method has proved to be very powerful to cope with global illumination problems but it remains costly in terms of sampling operations. In various applications, previous work has shown that Bayesian Monte Carlo can significantly outperform importance sampling Monte Carlo thanks to a more effective use of the prior knowledge and of the information brought by the samples set. These good results have been confirmed in the context of global illumination but strictly limited to the perfect diffuse case. Our main goal in this paper is to propose a more general Bayesian Monte Carlo solution that allows dealing with nondiffuse BRDFs thanks to a spherical Gaussian-based framework. We also propose a fast hyperparameters determination method that avoids learning the hyperparameters for each BRDF. These contributions represent two major steps toward generalizing Bayesian Monte Carlo for global illumination rendering. We show that we achieve substantial quality improvements over importance sampling at comparable computational cost.