Fine-tuning is a popular approach to solve the few-shot object detection problem. In this paper, we attempt to introduce a new perspective on it. We formulate the few-shot novel tasks as a type of distribution shifted from its ground-truth distribution. We introduce the concept of imaginary placeholder masks to show that this distribution shift is essentially a composite of in-distribution(ID) and out-of-distribution(OOD) shifts. Our empirical investigation results show that it is significant to balance the trade-off between adapting to the available few-shot distribution and keeping the distribution-shift robustness of the pre-trained model. We explore improvements in the few-shot fine-tuning transfer in the few-shot object detection(FSOD) settings from three aspects. First, we explore the LinearProbe-Finetuning(LP-FT) technique to balance this trade-off to mitigate the feature distortion problem. Second, we explore the effectiveness of utilizing the protection freezing strategy for query-based object detectors to keep their OOD robustness. Third, we try to utilize ensembling methods to circumvent the feature distortion. All these techniques are integrated into a whole method called BIOT(Balanced ID-OOD Transfer). Evaluation results show that our method is simple yet effective and general to tap the FSOD potential of query-based object detectors. It outperforms the current SOTA method in many FSOD settings and has a promising scaling capability.

We compared face identification by humans and machines using images taken under a variety of uncontrolled illumination conditions in both indoor and outdoor settings. Natural variations in a person's day-to-day appearance (e.g., hair style, facial expression, hats, glasses, etc.) contributed to the difficulty of the task. Both humans and machines matched the identity of people (same or different) in pairs of frontal view face images. The degree of difficulty introduced by photometric and appearance-based variability was estimated using a face recognition algorithm created by fusing three top-performing algorithms from a recent international competition. The algorithm computed similarity scores for a constant set of same-identity and different-identity pairings from multiple images. Image pairs were assigned to good , moderate , and poor accuracy groups by ranking the similarity scores for each identity pairing, and dividing these rankings into three strata. This procedure isolated the role of photometric variables from the effects of the distinctiveness of particular identities. Algorithm performance for these constant identity pairings varied dramatically across the groups. In a series of experiments, humans matched image pairs from the good, moderate, and poor conditions, rating the likelihood that the images were of the same person (1: sure same - 5: sure different). Algorithms were more accurate than humans in the good and moderate conditions, but were comparable to humans in the poor accuracy condition. To date, these are the most variable illumination- and appearance-based recognition conditions on which humans and machines have been compared. The finding that machines were never less accurate than humans on these challenging frontal images suggests that face recognition systems may be ready for applications with comparable difficulty. We speculate that the superiority of algorithms over humans in the less challenging conditions may be due to the algorithms' use of detailed, view-specific identity information. Humans may consider this information less important due to its limited potential for robust generalization in suboptimal viewing conditions.

The intended applications of automatic face recognition systems include venues that vary widely in demographic diversity. Formal evaluations of algorithms do not commonly consider the effects of population diversity on performance. We document the effects of racial and gender demographics on the accuracy of algorithms that match identity in pairs of face images. In particular, we focus on the effects of the “background” population distribution of non-matched identities against which identity matches are compared. The algorithm we tested was created by fusing three of the top performers from a recent US Government competition. First, we demonstrate the variability of algorithm performance estimates when the non-matched identities were demographically “yoked” by race and/or gender (i.e., “yoking” constrains non-matched pairs to be of the same race or gender). We also found a shift in the match threshold required to maintain a stable false positive rate when demographic control scenarios varied. These results were verified with two independent data sets that differed in demographic characteristics. In a second experiment, we explored the effects of progressive increases in population diversity on algorithm performance. We found systematic, but non-general, effects when the balance between majority and minority populations of non-matched identities shifted. Finally, we show that identity match accuracy differs substantially when the non-match identity population varied by race. The results indicate the importance of the demographic composition and modeling of the background population in predicting the accuracy of face recognition algorithms.

We propose a method for retargeting measured materials, where a source measured material is edited by applying the reflectance functions of a template measured dataset. The resulting dataset is a material that maintains the spatial patterns of the source dataset, while exhibiting the reflectance behaviors of the template. Compared to editing materials by subsequent selections and modifications, retargeting shortens the time required to achieve a desired look by directly using template data, just as color transfer does for editing images. With our method, users have to just mark corresponding regions of source and template with rough strokes, with no need for further input. This paper introduces AppWarp , an algorithm that achieves retargeting as a user-constrained, appearance-space warping operation, that executes in tens of seconds. Our algorithm is independent of the measured material representation and supports retargeting of analytic and tabulated BRDFs as well as BSSRDFs. In addition, our method makes no assumption of the data distribution in appearance-space nor on the underlying correspondence between source and target. These characteristics make AppWarp the first general formulation for appearance retargeting. We validate our method on several types of materials, including leaves, metals, waxes, woods and greeting cards. Furthermore, we demonstrate how retargeting can be used to enhance diffuse texture with high quality reflectance.

Abstract This paper presents an image editing framework where users use reference images to indicate desired color edits. In our approach, users specify pairs of strokes to indicate corresponding regions in both the original and the reference image that should have the same color “style”. Within each stroke pair, a nonlinear constrained parametric transfer model is used to transfer the reference colors to the original. We estimate the model parameters by matching color distributions, under constraints that ensure no visual artifacts are present in the transfer result. To perform transfer on the whole image, we employ optimization methods to propagate the model parameters defined at each stroke location to spatially‐close regions of similar appearance. This stroke‐based formulation requires minimal user effort while retaining the high degree of user control necessary to allow artistic interpretations. We demonstrate our approach by performing color transfer on a number of image pairs varying in content and style, and show that our algorithm outperforms state‐of‐the‐art color transfer methods on both user‐controllability and visual qualities of the transfer results.

We present an intuitive and efficient method for editing the appearance of complex spatially-varying datasets, such as images and measured materials. In our framework, users specify rough adjustments that are refined interactively by enforcing the policy that similar edits are applied to spatially-close regions of similar appearance. Rather than proposing a specific user interface, our method allows artists to quickly and imprecisely specify the initial edits with any method or workflow they feel most comfortable with. An energy optimization formulation is used to propagate the initial rough adjustments to the final refined ones by enforcing the editing policy over all pairs of points in the dataset. We show that this formulation is equivalent to solving a large linear system defined by a dense matrix. We derive an approximate algorithm to compute such a solution interactively by taking advantage of the inherent structure of the matrix. We demonstrate our approach by editing images, HDR radiance maps, and measured materials. Finally, we show that our framework generalizes prior methods while providing significant improvements in generality, robustness and efficiency.

Abstract We present an efficient approach that merges the virtual objects into video sequences taken by a freely moving camera in a realistic manner. The composition is visually and geometrically consistent through three main steps. First, a robust camera tracking algorithm based on key frames is proposed, which precisely recovers the focal length with a novel multi‐frame strategy. Next, the concerned 3D models of the real scenes are reconstructed by means of an extended multi‐baseline algorithm. Finally, the virtual objects in the form of 3D models are integrated into the real scenes, with special cares on the interaction consistency including shadow casting, occlusions, and object animation. A variety of experiments have been implemented, which demonstrate the robustness and efficiency of our approach. Copyright © 2006 John Wiley & Sons, Ltd.