Automating garment assembly from sewing patterns remains a significant challenge due to the lack of standardized annotation protocols and the frequent absence of semantic cues. Existing methods often rely on panel labels or handcrafted heuristics, which limit their applicability to real-world, non-conforming patterns. We present AutoSew, a fully automatic, geometry-based approach for predicting stitch correspondences directly from 2D pattern contours. AutoSew formulates the problem as a graph matching task, leveraging a Graph Neural Network to capture local and global geometric context, and employing a differentiable optimal transport solver to infer stitching relationships—including multi-edge connections. To support this task, we update the GarmentCodeData dataset modifying over 18k patterns with realistic multi-edge annotations, reflecting industrial assembly scenarios. AutoSew achieves 96% F1-score and successfully assembles 73.3% of test garments without error, outperforming existing methods while relying solely on geometric input. Our results demonstrate that geometry alone can robustly guide stitching prediction, enabling scalable garment assembly without manual input. Our dataset and code are available online.1

Estimating the surface of translucent objects from photometric data poses significant challenges due to complex internal light scattering. We introduce a novel method that computes a depth map from single-viewpoint photographs of a material sample, captured under multiple illuminations. Our approach leverages inverse rendering to derive a volumetric representation, including density, albedo, and phase function, from which a surface mesh is reconstructed. Beyond validation with synthetic and 3D-printed physical models, we illustrate our technique's power by successfully applying it to the digitization of fabrics, a notoriously difficult material due to its intricate translucent structure. This work advances the state-of-the-art texture stack acquisition via enhanced surface reconstruction.

We present a novel approach that combines intrinsic decomposition of outdoor scenes with real-time rendering of new views under unknown illumination. Building on top of the state of the art, our method tackles the challenges of limited information in single-illumination scenarios by introducing pixel-level regularization terms aligning inferred material segmentation labels with albedo consistency estimators. For outdoor illumination, we adopt a physically-based sky model which increases the intrinsic decomposition robustness by relying on a reduced set of expressive parameters. Our approach enables partial retraining of 2DGS/3DGS models to render de-illuminated scenes in real time, with seamless integration into rendering engines for enhanced scene lighting, achieving better decomposition results than the state of the art. We show several experiments, including ablation studies and material segmentation source comparisons, proving our method’s advantages over previous work, despite remaining challenges in handling fine shadow details and view-dependent effects due to the limitations of the Lambertian shading model.

Flatbed scanners have emerged as promising devices for high-resolution, single-image material capture. However, existing approaches assume very specific conditions, such as uniform diffuse illumination, which are only available in certain high-end devices, hindering their scalability and cost. In contrast, in this work, we introduce a method inspired by intrinsic image decomposition, which accurately removes both shading and specularity, effectively allowing captures with any flatbed scanner. Further, we extend previous work on single-image material reflectance capture with the estimation of opacity and transmittance, critical components of full material appearance (SVBSDF), improving the results for any material captured with a flatbed scanner, at a very high resolution and accuracy. • We introduce a generative model for digitizing materials using any flatbed scanners, capable of removing undesirable shading and specular highlights. • We expand the realism of digital replicas of material by including opacity and transmittance in the material model, both of which are key attributes for thin-layer materials like fabrics. • We provide an extensive and thorough experimentation using image-based and render aware metrics. Our results show that our method works with a wide range of scanning devices.

We introduce TexTile, a novel differentiable metric to quantify the degree upon which a texture image can be concatenated with itself without introducing repeating artifacts (i.e., the tileability). Existing methods for tileable texture synthesis focus on general texture quality, but lack explicit analysis of the intrinsic repeatability properties of a texture. In contrast, our TexTile metric effectively evaluates the tileable properties of a texture, opening the door to more in-formed synthesis and analysis of tileable textures. Under the hood, TexTile is formulated as a binary classifier carefully built from a large dataset of textures of different styles, semantics, regularities, and human annotations. Key to our method is a set of architectural modifications to baseline pretrain image classifiers to overcome their shortcomings at measuring tileability, along with a custom data augmen-tation and training regime aimed at increasing robustness and accuracy. We demonstrate that TexTile can be plugged into different state-of-the-art texture synthesis methods, in-cluding diffusion-based strategies, and generate tileable textures while keeping or even improving the overall texture quality. Furthermore, we show that TexTile can objectively evaluate any tileable texture synthesis method, whereas the current mix of existing metrics produces uncorrelated scores which heavily hinders progress in the field.

Abstract Environment maps are commonly used to represent and compute far‐field illumination in virtual scenes. However, they are expensive to evaluate and sample from, limiting their applicability to real‐time rendering. Previous methods have focused on compression through spherical‐domain approximations, or on learning priors for natural, day‐light illumination. These hinder both accuracy and generality, and do not provide the probability information required for importance‐sampling Monte Carlo integration. We propose NEnv, a deep‐learning fully‐differentiable method, capable of compressing and learning to sample from a single environment map. NEnv is composed of two different neural networks: A normalizing flow, able to map samples from uniform distributions to the probability density of the illumination, also providing their corresponding probabilities; and an implicit neural representation which compresses the environment map into an efficient differentiable function. The computation time of environment samples with NEnv is two orders of magnitude less than with traditional methods. NEnv makes no assumptions regarding the content (i.e. natural illumination), thus achieving higher generality than previous learning‐based approaches. We share our implementation and a diverse dataset of trained neural environment maps, which can be easily integrated into existing rendering engines.

Existing devices for measuring material appearance in spatially-varying samples are limited to a single scale, either micro or mesoscopic. This is a practical limitation when the material has a complex multi-scale structure. In this paper, we present a system and methods to digitize materials at two scales, designed to include high-resolution data in spatially-varying representations at larger scales. We design and build a hemispherical light dome able to digitize flat material samples up to 11x11cm. We estimate geometric properties, anisotropic reflectance and transmittance at the microscopic level using polarized directional lighting with a single orthogonal camera. Then, we propagate this structured information to the mesoscale, using a neural network trained with the data acquired by the device and image-to-image translation methods. To maximize the compatibility of our digitization, we leverage standard BSDF models commonly adopted in the industry. Through extensive experiments, we demonstrate the precision of our device and the quality of our digitization process using a set of challenging real-world material samples and validation scenes. Further, we demonstrate the optical resolution and potential of our device for acquiring more complex material representations by capturing microscopic attributes which affect the global appearance: we characterize the properties of textile materials such as the yarn twist or the shape of individual fly-out fibers. We also release the SEDDIDOME dataset of materials, including raw data captured by the machine and optimized parameteres.

The estimation of the optical properties of a material from RGB-images is an important but extremely ill-posed problem in Computer Graphics. While recent works have successfully approached this problem even from just a single photograph, significant simplifications of the material model are assumed, limiting the usability of such methods. The detection of complex material properties such as anisotropy or Fresnel effect remains an unsolved challenge. We propose a novel method that predicts the model parameters of an artist-friendly, physically-based BRDF, from only two low-resolution shots of the material. Thanks to a novel combination of deep neural networks in a nested architecture, we are able to handle the ambiguities given by the non-orthogonality and non-convexity of the parameter space. To train the network, we generate a novel dataset of physically-based synthetic images. We prove that our model can recover new properties like anisotropy, index of refraction and a second reflectance color, for materials that have tinted specular reflections or whose albedo changes at glancing angles.

Abstract Accurately modeling how light interacts with cloth is challenging, due to the volumetric nature of cloth appearance and its multiscale structure, where microstructures play a major role in the overall appearance at higher scales. Recently, significant effort has been put on developing better microscopic models for cloth structure, which have allowed rendering fabrics with unprecedented fidelity. However, these highly‐detailed representations still make severe simplifications on the scattering by individual fibers forming the cloth, ignoring the impact of fibers' shape, and avoiding to establish connections between the fibers' appearance and their optical and fabrication parameters. In this work we put our focus in the scattering of individual cloth fibers; we introduce a physically‐based scattering model for fibers based on their low‐level optical and geometric properties, relying on the extensive textile literature for accurate data. We demonstrate that scattering from cloth fibers exhibits much more complexity than current fiber models, showing important differences between cloth type, even in averaged conditions due to longer views. Our model can be plugged in any framework for cloth rendering, matches scattering measurements from real yarns, and is based on actual parameters used in the textile industry, allowing predictive bottom‐up definition of cloth appearance.

Abstract We present new methods for uniformly sampling the solid angle subtended by a disk. To achieve this, we devise two novel area‐preserving mappings from the unit square [0,1]2 to a spherical ellipse (i.e. the projection of the disk onto the unit sphere). These mappings allow for low‐variance stratified sampling of direct illumination from disk‐shaped light sources. We discuss how to efficiently incorporate our methods into a production renderer and demonstrate the quality of our maps, showing significantly lower variance than previous work.

Abstract Most popular methods in cloth rendering rely on volumetric data in order to model complex optical phenomena such as sub‐surface scattering. These approaches are able to produce very realistic illumination results, but their volumetric representations are costly to compute and render, forfeiting any interactive feedback. In this paper, we introduce a method based on the Graphics Processing Unit (GPU) for voxelization and visualization, suitable for both interactive and offline rendering. Recent features in the OpenGL model, like the ability to dynamically address arbitrary buffers and allocate bindless textures, are combined into our pipeline to interactively voxelize millions of polygons into a set of large three‐dimensional (3D) textures (>109 elements), generating a volume with sub‐voxel accuracy, which is suitable even for high‐density woven cloth such as linen.

Cloth is made of yarns that are stitched together forming semi-regular patterns. Due to the complexity of stitches and patterns, the macroscopic behavior of cloth is dictated by the contact interactions between yarns, not by the mechanical properties of yarns alone. The computation of cloth mechanics at the yarn level appears as a computationally complex and costly process at first sight, due to the need to resolve many fine-scale contact interactions. We propose instead an efficient representation of cloth at the yarn level that treats yarn-yarn contacts as persistent, but with the possibility to slide, thereby avoiding expensive contact handling altogether. We introduce a compact representation of yarn geometry and kinematics, capturing the essential deformation modes of yarn crossings, loops, stitches, and stacks, with a minimum cost. Based on this representation, we design force models that reproduce the characteristic macroscopic behavior of yarn-based fabrics. Our approach is suited for both woven and knitted fabrics. We demonstrate the efficiency of our method on simulations with millions of degrees of freedom (hundreds of thousands of yarn loops), almost one order of magnitude faster than previous techniques. We also compare the different macroscopic behavior under woven and knitted patterns with the same yarn density.

We introduce a method to compute intrinsic images for a multiview set of outdoor photos with cast shadows, taken under the same lighting. We use an automatic 3D reconstruction from these photos and the sun direction as input and decompose each image into reflectance and shading layers, despite the inaccuracies and missing data of the 3D model. Our approach is based on two key ideas. First, we progressively improve the accuracy of the parameters of our image formation model by performing iterative estimation and combining 3D lighting simulation with 2D image optimization methods. Second, we use the image formation model to express reflectance as a function of discrete visibility values for shadow and light, which allows to introduce a robust visibility classifier for pairs of points in a scene. This classifier is used for shadow labeling, allowing to compute high-quality reflectance and shading layers. Our multiview intrinsic decomposition is of sufficient quality to allow relighting of the input images. We create shadow-caster geometry which preserves shadow silhouettes and, using the intrinsic layers, we can perform multiview relighting with moving cast shadows. We present results on several multiview datasets, and show how it is now possible to perform image-based rendering with changing illumination conditions.

Abstract We present an interactive approach for decompositing bitmap drawings and studio photographs into opaque and semi‐transparent vector layers. Semi‐transparent layers are especially challenging to extract, since they require the inversion of the non‐linear compositing equation. We make this problem tractable by exploiting the parametric nature of vector gradients, jointly separating and vectorising semi‐transparent regions. Specifically, we constrain the foreground colours to vary according to linear or radial parametric gradients, restricting the number of unknowns and allowing our system to efficiently solve for an editable semi‐transparent foreground. We propose a progressive workflow, where the user successively selects a semi‐transparent or opaque region in the bitmap, which our algorithm separates as a foreground vector gradient and a background bitmap layer. The user can choose to decompose the background further or vectorise it as an opaque layer. The resulting layered vector representation allows a variety of edits, such as modifying the shape of highlights, adding texture to an object or changing its diffuse colour.

The large-scale mechanical behavior of woven cloth is determined by the mechanical properties of the yarns, the weave pattern, and frictional contact between yarns. Using standard simulation methods for elastic rod models and yarn-yarn contact handling, the simulation of woven garments at realistic yarn densities is deemed intractable. This paper introduces an efficient solution for simulating woven cloth at the yarn level. Central to our solution is a novel discretization of interlaced yarns based on yarn crossings and yarn sliding, which allows modeling yarn-yarn contact implicitly, avoiding contact handling at yarn crossings altogether. Combined with models for internal yarn forces and inter-yarn frictional contact, as well as a massively parallel solver, we are able to simulate garments with hundreds of thousands of yarn crossings at practical frame-rates on a desktop machine, showing combinations of large-scale and fine-scale effects induced by yarn-level mechanics.

Abstract Many high‐level image processing tasks require an estimate of the positions, directions and relative intensities of the light sources that illuminated the depicted scene. In image‐based rendering, augmented reality and computer vision, such tasks include matching image contents based on illumination, inserting rendered synthetic objects into a natural image, intrinsic images, shape from shading and image relighting. Yet, accurate and robust illumination estimation, particularly from a single image, is a highly ill‐posed problem. In this paper, we present a new method to estimate the illumination in a single image as a combination of achromatic lights with their 3D directions and relative intensities. In contrast to previous methods, we base our azimuth angle estimation on curve fitting and recursive refinement of the number of light sources. Similarly, we present a novel surface normal approximation using an osculating arc for the estimation of zenith angles. By means of a new data set of ground‐truth data and images, we demonstrate that our approach produces more robust and accurate results, and show its versatility through novel applications such as image compositing and analysis.

Vector graphics represent images with compact, editable and scalable primitives. Skillful vector artists employ these primitives to produce vivid depictions of material appearance and lighting. However, such stylized imagery often requires building complex multi-layered combinations of colored fills and gradient meshes. We facilitate this task by introducing vector shade trees that bring to vector graphics the flexibility of modular shading representations as known in the 3D rendering community. In contrast to traditional shade trees that combine pixel and vertex shaders, our shade nodes encapsulate the creation and blending of vector primitives that vector artists routinely use. We propose a set of basic shade nodes that we design to respect the traditional guidelines on material depiction described in drawing books and tutorials. We integrate our representation as an Adobe Illustrator plug-in that allows even inexperienced users to take a line drawing, apply a few clicks and obtain a fully colored illustration. More experienced artists can easily refine the illustration, adding more details and visual features, while using all the vector drawing tools they are already familiar with. We demonstrate the power of our representation by quickly generating illustrations of complex objects and materials.

Abstract Decomposing an input image into its intrinsic shading and reflectance components is a long‐standing ill‐posed problem. We present a novel algorithm that requires no user strokes and works on a single image. Based on simple assumptions about its reflectance and luminance, we first find clusters of similar reflectance in the image, and build a linear system describing the connections and relations between them. Our assumptions are less restrictive than widely‐adopted Retinex‐based approaches, and can be further relaxed in conflicting situations. The resulting system is robust even in the presence of areas where our assumptions do not hold. We show a wide variety of results, including natural images, objects from the MIT dataset and texture images, along with several applications, proving the versatility of our method.

Abstract We present a novel method to estimate an approximation of the reflectance characteristics of optically thick, homogeneous translucent materials using only a single photograph as input. First, we approximate the diffusion profile as a linear combination of piecewise constant functions, an approach that enables a linear system minimization and maximizes robustness in the presence of suboptimal input data inferred from the image. We then fit to a smoother monotonically decreasing model, ensuring continuity on its first derivative. We show the feasibility of our approach and validate it in controlled environments, comparing well against physical measurements from previous works. Next, we explore the performance of our method in uncontrolled scenarios, where neither lighting nor geometry are known. We show that these can be roughly approximated from the corresponding image by making two simple assumptions: that the object is lit by a distant light source and that it is globally convex, allowing us to capture the visual appearance of the photographed material. Compared with previous works, our technique offers an attractive balance between visual accuracy and ease of use, allowing its use in a wide range of scenarios including off‐the‐shelf, single images, thus extending the current repertoire of real‐world data acquisition techniques.

We present a powerful technique to simulate and approximate caustics in images. Our algorithm is designed to produce good results without the need to painstakingly paint over pixels. The ability to edit global illumination through image processing allows interaction with images at a level which has not yet been demonstrated, and significantly augments and extends current image-based material editing approaches. We show by means of a set of psychophysical experiments that the resulting imagery is visually plausible and on par with photon mapping, albeit without the need for hand-modeling the underlying geometry.