Shows such as The Mandalorian have used Virtual Production (VP) to great creative advantage. This combination, of tracked cameras and virtual environments, gives film crews the flexibility to rapidly iterate ideas and discover the best telling of their story within the time constraints of movie production. We explore if the same ideas can translate to pre-production, specifically if similar advantages can be found during the storyboarding phase. Developing a novel virtual production interface named Camera Chameleon, we compare user interaction against traditional keyboard/mouse peripherals when creating storyboards. Similar expert-rated quality is measured on storyboard output, however, users report increased enjoyment and complete shots more quickly. Novice users explore more when using our approach, demonstrating its creative potential.

Texture synthesis has grown into a mature field in computer graphics, allowing the synthesis of naturalistic textures and images from photographic exemplars. Surprisingly little work, however, has been dedicated to synthesizing tileable textures, that is, textures that when laid out in a regular grid of tiles form a homogeneous appearance suitable for use in memory-sensitive real-time graphics applications. One of the key challenges in doing so is that most natural input exemplars exhibit uneven spatial variations that, when tiled, show as repetitive patterns. We propose an approach to synthesize tileable textures while enforcing stationarity properties that effectively mask repetitions while maintaining the unique characteristics of the exemplar. We explore a number of alternative measures for texture stationarity and show how each measure can be integrated into a standard texture synthesis method (PatchMatch) to enforce stationarity at user-controlled scales. We demonstrate the efficacy of our approach using a database of 118 exemplar images, both from publicly available sources as well as new ones captured under uncontrolled conditions, and we quantitatively analyze alternative stationarity measures for their robustness across many test runs using different random seeds. In conclusion, we suggest a novel synthesis approach that employs local histogram matching to reliably turn input photographs of natural surfaces into tiles well suited for artifact-free tiling.

There are many scenarios where we wish to imitate a specific author’s pen-on-paper handwriting style. Rendering new text in someone’s handwriting is difficult because natural handwriting is highly variable, yet follows both intentional and involuntary structure that makes a person’s style self-consistent. The variability means that naive example-based texture synthesis can be conspicuously repetitive. We propose an algorithm that renders a desired input string in an author’s handwriting. An annotated sample of the author’s handwriting is required; the system is flexible enough that historical documents can usually be used with only a little extra effort. Experiments show that our glyph-centric approach, with learned parameters for spacing, line thickness, and pressure, produces novel images of handwriting that look hand-made to casual observers, even when printed on paper.

Video analysis often begins with background subtraction. This problem is often approached in two steps-a background model followed by a regularisation scheme. A model of the background allows it to be distinguished on a per-pixel basis from the foreground, whilst the regularisation combines information from adjacent pixels. We present a new method based on Dirichlet process Gaussian mixture models, which are used to estimate per-pixel background distributions. It is followed by probabilistic regularisation. Using a non-parametric Bayesian method allows per-pixel mode counts to be automatically inferred, avoiding over-/under- fitting. We also develop novel model learning algorithms for continuous update of the model in a principled fashion as the scene changes. These key advantages enable us to outperform the state-of-the-art alternatives on four benchmarks.

Real-world classification problems, such as visual surveillance and network intrusion detection, often contain common yet uninteresting background classes and rare but interesting classes, that need to be both discovered and classified. Active learning offers a suitable solution to joint rare class discovery and classification, by minimising the manual labelling of training data. A novel active learning approach is proposed, which automatically balances the competing goals of new class discovery and improving classification. Crucially it is free of tuneable parameters. Using Dirichlet processes a new active learning criterion is formulated, based on first computing the probability that unlabelled exemplars are from a new class, in addition to existing classes, and subsequently the probability of misclassification, which is then used for query selection. The proposed approach works with any probabilistic classification model and its effectiveness is demonstrated on multiple problems.

In the security domain a key problem is identifying rare behaviours of interest. Training examples for these behaviours may or may not exist, and if they do exist there will be few examples, quite probably one. We present a novel weakly supervised algorithm that can detect behaviours that either have never before been seen or for which there are few examples. Global context is modelled, allowing the detection of abnormal behaviours that in isolation appear normal. Pragmatic aspects are considered, such that no parameter tuning is required and real time performance is achieved.

In this paper we present a method of combining stereo and shape-from-shading information, taking account of the local reliability of each shape estimate. Local estimates of disparity and orientation are modelled using Gaussian distributions. A Gaussian-Markov random field is used to represent the disparity-map, taking into account interactions between disparity measurements and surface orientation, and the MAP estimate found using belief propagation. Local estimates of the precision of disparities and surface normals are found and used to control the process so that the most accurate data source is used in each region. We assess the performance of our approach using both synthetic and real stereo pairs, and compare against ground truth.

Binocular stereo has been extensively studied for extracting the shape of a scene. The challenge is in matching features between two images of a scene; this is the correspondence problem. Shape from shading (SfS) is another method of extracting shape. This models the interaction of light with the scene surface(s) for a single image. These two methods are very different; stereo uses surface features to deliver a depth-map, SfS uses shading, albedo and lighting information to infer the differential of the depth-map. In this paper we develop a framework for the integration of both depth and orientation information. Dedicated algorithms are used for initial estimates. A Gaussian-Markov random field then represents the depth-map, Gaussian belief propagation is used to approximate the MAP estimate of the depth-map. Integrating information from both stereo correspondences and surface normals allows fine surface details to be estimated.