Anomaly detection aims to identify abnormal data that deviates from the normal ones, while typically requiring a sufficient amount of normal data to train the model for performing this task. Despite the success of recent anomaly detection methods, performing anomaly detection in an unseen domain remain a challenging task. In this paper, we address the task of domain-generalized textured surface anomaly detection. By observing normal and abnormal surface data across multiple source domains, our model is expected to be generalized to an unseen textured surface of interest, in which only a small number of normal data can be observed during testing. Although with only image-level labels observed in the training data, our patch-based meta-learning model exhibits promising generalization ability: not only can it generalize to unseen image domains, but it can also localize abnormal regions in the query image. Our experiments verify that our model performs favorably against state-of-the-art anomaly detection and domain generalization approaches in various settings.

In this paper, we propose a framework called Trust-MAE to address the problem of product defect classification. Instead of relying on defective images that are difficult to collect and laborious to label, our framework can accept datasets with unlabeled images. Moreover, unlike most anomaly detection methods, our approach is robust against noises, or defective images, in the training dataset. Our framework uses a memory-augmented auto-encoder with a sparse memory addressing scheme to avoid over-generalizing the auto-encoder, and a novel trust-region memory updating scheme to keep the noises away from the memory slots. The result is a framework that can reconstruct defect-free images and identify the defective regions using a perceptual distance network. When compared against various state-of-the-art baselines, our approach performs competitively under noise-free MVTec datasets. More importantly, it remains effective at a noise level up to 40% while significantly outperforming other baselines.

Motion synthesis in a dynamic environment has been a long-standing problem for character animation. Methods using motion capture data tend to scale poorly in complex environments because of their larger capturing and labeling requirement. Physics-based controllers are effective in this regard, albeit less controllable. In this paper, we present CARL, a quadruped agent that can be controlled with high-level directives and react naturally to dynamic environments. Starting with an agent that can imitate individual animation clips, we use Generative Adversarial Networks to adapt high-level controls, such as speed and heading, to action distributions that correspond to the original animations. Further fine-tuning through the deep reinforcement learning enables the agent to recover from unseen external perturbations while producing smooth transitions. It then becomes straightforward to create autonomous agents in dynamic environments by adding navigation modules over the entire process. We evaluate our approach by measuring the agent's ability to follow user control and provide a visual analysis of the generated motion to show its effectiveness.

VRAPS (Visual Rhythm-based Audio Playback System) is an interactive multimedia application that uses a novel visual rhythm detection technique to allow a user to control the playback speed of an audio signal by moving in front of a video camera. As traditionally defined in the context of music, a beat represents a distinctive musical event such as the hitting of a drum or the start of a new melodic note. Similarly, we define a visual beat to indicate distinctive visual events such as rhythmic gestures and dance motions. In this paper, we propose a method for automatically detecting visual beats and tempo from a video signal. As a demonstration of this technique, we present an interactive, real-time audio playback system that continuously adjusts the playback speed of an audio signal based on the visual rhythm detected from a real-time video capture device.

Multi-core processors represent a major development in computing technology. For example, IntelregCoretrade 2 Quad processors, IBM Cell processors, and Nvidia GeForce 9800 GX2, are widely used. However, most applications struggle to make the best use of the power provided by many-core processors. Easy-to-use software tools are hard to find. Furthermore, it's not clear what changes need to be made to algorithms to fully utilize many-core CPUs or GPUs. In this paper, we try to offer a bird's eye view of the opportunities lying ahead in two folds: (1) software tools and (2) workload analysis. With good software tools and insightful workload analysis, software and algorithm developers can not only harness the power of many computing cores, but also innovate new algorithms that best utilize the many computing cores. New algorithms and applications are thus made possible with the computing power not available before.

Current approaches to pose estimation and tracking can be classified into two categories: generative and discriminative. While generative approaches can accurately determine human pose from image observations, they are computationally expensive due to search in the high dimensional human pose space. On the other hand, discriminative approaches do not generalize well, but are computationally efficient. We present a hybrid model that combines the strengths of the two in an integrated learning and inference framework. We extend the Gaussian process latent variable model (GPLVM) to include an embedding from observation space (the space of image features) to the latent space. GPLVM is a generative model, but the inclusion of this mapping provides a discriminative component, making the model observation driven. Observation Driven GPLVM (OD-GPLVM) not only provides a faster inference approach, but also more accurate estimates (compared to GPLVM) in cases where dynamics are not sufficient for the initialization of search in the latent space. We also extend OD-GPLVM to learn and estimate poses from parameterized actions/gestures. Parameterized gestures are actions which exhibit large systematic variation in joint angle space for different instances due to difference in contextual variables. For example, the joint angles in a forehand tennis shot are function of the height of the ball (Figure 2). We learn these systematic variations as a function of the contextual variables. We then present an approach to use information from scene/objects to provide context for human pose estimation for such parameterized actions.

The recent emergence of multi-core processors enables a new trend in the usage of computers. Computer vision applications, which require heavy computation and lots of bandwidth, usually cannot run in real-time. Recent multi-core processors can potentially serve the needs of such workloads. In addition, more advanced algorithms can be developed utilizing the new computation paradigm. In this paper, we study the performance of an articulated body tracker on multi-core processors. The articulated body tracking workload encapsulates most of the important aspects of a computer vision workload. It takes multiple camera inputs of a scene with a single human object, extracts useful features, and performs statistical inference to find the body pose. We show the importance of properly parallelizing the workload in order to achieve great performance: speedups of 26 on 32 cores. We conclude that: (1) data-domain parallelization is better than function-domain parallelization for computer vision applications; (2) data-domain parallelism by image regions and particles is very effective; (3) reducing serial code in edge detection brings significant performance improvements; (4) domain knowledge about low/mid/high level of vision computation is helpful in parallelizing the workload.

Streaming of precoded video, which is both source and channel coded, over wireless networks faces challenges of time-varying packet loss rate and fluctuating bandwidth. Rate shaping has been proposed (Chen, T.P.-C. and Chen, T., ICASSP, 2002) to "shape" the precoded video to adapt to the real-time bandwidth requirement and the packet loss rate. We now propose a novel "fine-grained rate shaping" (FGRS) scheme to allow for bandwidth adaptation over a wide range of bandwidth and packet loss rate. The video is precoded with fine granularity scalability (FGS) followed by forward error correction (FEC) coding with erasure codes. Utilizing the fine granularity property of FGS and FEC, FGRS selectively drops part of the precoded video and still yields a decodable bitstream at the decoder. A new two-stage rate-distortion (R-D) optimization, with model-based hyper-plane and hill-climbing based refinement, is proposed to select which part of the precoded video to drop. Promising results for FGRS are shown.

Streaming of preceded video, which is both source- and channel-coded, over packet-loss networks faces challenges of the time-varying packet loss rate and fluctuating bandwidth. Rate shaping has been proposed to reduce the bit rate of a preceded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. In practice, the reconstructed result of the previous frame will affect the following frames if the video is predictive coded, and/or the error concealment method performed at the receiver utilizes temporal information. However, none prior work considers rate shaping for pre source- and channel-coded video with such frame dependency. In order to incorporate frame dependency into the rate shaping process, we propose to send the location (and mean) of the corrupted macroblock back to the sender, and use such feedback information to determine the distortion measure in the rate-distortion optimized rate shaping. Experiments have shown improved performance with the aids of the feedback information.

We present in this paper an adaptive joint source-channel coding scheme using rate shaping on pre-coded video data. Rate shaping selectively drops portions of the video bitstream before transmitting them in order to satisfy the network bandwidth requirement. In wireless multimedia transport over heterogeneous networks, limited bandwidth is not the only issue. The high error rate of the channel should be considered as well, so channel coding is often applied. We propose a rate shaping method that drops not only the source-coding segments of the video bitstream, but also the channel-coding segments of the video bitstream, adaptively according to the network condition, in order to achieve the optimal rate-distortion performance. The proposed method is based on “discrete rate-distortion combination” to accomplish joint source-channel coding. We consider both the simulcast and multicast scenarios and show promising results.

Video transport over error prone channels may result in loss or erroneous decoding of the video. Error concealment is an effective mechanism to reconstruct the video content. In this paper, we review different error concealment methods and introduce a new framework, which we refer to as second-generation error concealment. All the error concealment methods reconstruct the lost video content by making use of some a priori knowledge about the video content. First-generation error concealment builds such a priori in a heuristic manner. The proposed second-generation error concealment builds the a priori by modeling the statistics of the video content. Context-based models are trained with the correctly decoded video content, and then used to replenish the lost video content. Trained models capture the statistics of the video content and thus reconstruct the lost video content better than reconstruction by heuristics.

We present a new statistical modeling technique called "updating mixture of principal components" (UMPC). UMPC specifically captures the non-stationary as well as the multi-modal characteristics of the data. Real-world data such as video data typically have these two characteristics. The video content changes over time and has a multi-modal probability distribution. We apply UMPC to perform error concealment for video data transmitted over networks with losses, and show that UMPC outperforms conventional error concealment methods.

Abstract Video transport over error‐prone channels may result in loss or erroneous decoding of the video data. Error concealment is an effective mechanism to reconstruct the video data. In this paper, we review error‐concealment methods and introduce a new framework, which we refer to assecond‐generation error concealment. All the error‐concealment methods reconstruct the lost video data by making use of certaina prioriknowledge about the video content.First‐generation error concealmentbuilds sucha prioriin a heuristic manner. The proposed second‐generation error concealment builds thea prioriby modeling the statistics of the video content explicitly, typically in the region of interest (ROI). Context‐based models are trained with the correctly received video data and then used to replenish the lost video data. Trained models capture the statistics of the video content and thus reconstruct the lost video data better than reconstruction by heuristics. A new dynamic model ‘updating principal components’ (UPC) is proposed as a model for second‐generation error concealment. UPC can be applied to pixel values to conceal loss of pixel data. In addition, UPC can be applied to motion vectors, which results in ‘updating eigenflows’ (U‐Eigenflow), to conceal loss of motion vectors. With UPC applied to both pixel values and motion vectors, hybrid temporal/spatial error concealment can be achieved. The proposed second‐generation error‐concealment method provides superior performances to first‐generation error‐concealment methods. Copyright © 2002 John Wiley & Sons, Ltd.