Most modern web scrapers use an embedded browser to render web pages and to simulate user actions. Such scrapers (or wrappers) are therefore expensive to execute, in terms of time and network traffic. In contrast, it is magnitudes more resource-efficient to use a "browserless" wrapper which directly accesses a web server through HTTP requests, and takes the desired data directly from the raw replies. However, creating and maintaining browserless wrappers of high precision requires specialists, and is prohibitively labor-intensive at scale. In this paper, we demonstrate the principal feasibility of automatically translating browser-based wrappers into "browserless" wrappers. We present the first algorithm and system performing such an automated translation on suitably restricted types of web sites. This system works in the vast majority of test cases and produces very fast and extremely resource-efficient wrappers. We discuss research challenges for extending our approach to a general method applicable to a yet larger number of cases.

We describe ArtForm, a tool for exploring the codebase of dynamic data-driven websites where users enter data via forms. ArtForm extends an instrumented browser, so it can directly implement user interactions, adding in symbolic and concolic execution of JavaScript. The tool supports a range of exploration modes with varying degrees of user intervention. It includes a number of adaptations of concolic execution to the setting of form-based web programs.

We present a visualization tool aimed specifically at the development and optimization of photon map denoising methods. Our tool allows the rapid testing of hypotheses and algorithms through the use of parallel coordinates, domain‐specific scripting, colour mapping and point plots. Interaction is carried out by brushing, adjusting parameters and focus‐plus‐context and yields interactive visual feedback and debugging information. We demonstrate the use of the tool to explore high‐dimensional photon map data, facilitating the discovery of novel parameter spaces which can be used to dissociate complex caustic illumination. We then show how these new parametrizations may be used to improve upon pre‐existing noise removal methods in the context of the photon relaxation framework.

Abstract Stream compaction is an important parallel computing primitive that produces a reduced (compacted) output stream consisting of only valid elements from an input stream containing both invalid and valid elements. Computing on this compacted stream rather than the mixed input stream leads to improvements in performance, load balancing and memory footprint. Stream compaction has numerous applications in a wide range of domains: e.g. deferred shading, isosurface extraction and surface voxelization in computer graphics and visualization. We present a novel In‐Kernel stream compaction method, where compaction is completed before leaving an operating kernel. This contrasts with conventional parallel compaction methods that require leaving the kernel and running a prefix sum kernel followed by a scatter kernel. We apply our compaction methods to ray‐tracing‐based visualization of volumetric data. We demonstrate that the proposed In‐Kernel compaction outperforms the standard out‐of‐kernel Thrust parallel‐scan method for performing stream compaction in this real‐world application. For the data visualization, we also propose a novel multi‐kernel ray‐tracing pipeline for increased thread coherency and show that it outperforms a conventional single‐kernel approach.

This paper presents a novel approach to detecting and preserving fine illumination structure within photon maps. Data derived from each photon's primal trajectory is encoded and used to build a high-dimensional kd-tree. Incorporation of these new parameters allows for precise differentiation between intersecting ray envelopes, thus minimizing detail degradation when combined with photon relaxation. We demonstrate how parameter-aware querying is beneficial in both detecting and removing noise. We also propose a more robust structure descriptor based on principal components analysis that better identifies anisotropic detail at the sub-kernel level. We illustrate the effectiveness of our approach in several example scenes and show significant improvements when rendering complex caustics compared to previous methods.

We introduce a novel algorithm for progressively removing noise from view-independent photon maps while simultaneously minimizing residual bias. Our method refines a primal set of photons using data from multiple successive passes to estimate the incident flux local to each photon. We show how this information can be used to guide a relaxation step with the goal of enforcing a constant, per-photon flux. Using a reformulation of the radiance estimate, we demonstrate how the resulting blue noise photon distribution yields a radiance reconstruction in which error is significantly reduced. Our approach has an open-ended runtime of the same order as unbiased and asymptotically consistent rendering methods, converging over time to a stable result. We demonstrate its effectiveness at storing caustic illumination within a view-independent framework and at a fidelity visually comparable to reference images rendered using progressive photon mapping.

Abstract The photon mapping method is one of the most popular algorithms employed in computer graphics today. However, obtaining good results is dependent on several variables including kernel shape and bandwidth, as well as the properties of the initial photon distribution. While the photon density estimation problem has been the target of extensive research, most algorithms focus on new methods of optimising the kernel to minimise noise and bias. In this paper we break from convention and propose a new approach that directly redistributes the underlying photons. We show that by relaxing the initial distribution into one with a blue noise spectral signature we can dramatically reduce background noise, particularly in areas of uniform illumination. In addition, we propose an efficient heuristic to detect and preserve features and discontinuities. We then go on to demonstrate how reconfiguration also permits the use of very low bandwidth kernels, greatly improving render times whilst reducing bias.

Photon mapping is an efficient method for producing high-quality, photorealistic images with full global illumination. In this paper we present a more accurate and efficient approach to final gathering using the photon map based upon hierarchical evaluation of the photons over each surface. We use the footprint of each gather ray to calculate the irradiance estimate area rather than deriving it from the local photon density. We then describe an efficient method for computing the irradiance from the photon map given an arbitrary estimate area. Finally, we demonstrate how the technique may be used to reduce variance and increase efficiency when sampling diffuse and glossy-specular BRDFs.