Video anomaly detection (VAD) methods analyze untrimmed videos to make temporal decisions at the frame level to identify abnormal events. An important challenge of VAD approaches is the accurate spatial segmentation of the anomalous regions within frames to provide interpretability of anomalies. In this paper, we introduce FUVAS, a fast few-shot unsupervised VAD method ideal for low-data scenarios. FUVAS efficiently identifies temporal anomalies and spatially segments them within each frame of the input video. Our approach harnesses rich video features extracted from pre-trained 3D deep neural networks (DNNs) and performs out-of-distribution detection in the spatiotemporal deep feature space induced by short temporal segments of the video input using low-rank factorization techniques. The proposed approach is agnostic to the choice of 3D DNN backbone architecture and supports both convolutional and transformer models. We present comprehensive results and ablation studies across popular datasets, demonstrating the quality, computational efficiency, and wide applicability of our method.Our code is available at: https://github.com/openvinotoolkit/anomalib/tree/main/src/anomalib/models/video/fuvas

In hardware accelerator design, software-hardware co-optimization requires intricate trade-offs and tight integration between software algorithms and hardware design to optimize performance, power efficiency, and area (PPA) while ensuring high accuracy. Furthermore, the inherent non-ideality in some emerging hardware technologies poses extra challenges to the co-optimization problem. This paper proposes a novel software-hardware co-optimization framework for hyperdimensional (HD) computing accelerators with emerging ReRAM-based processing in-memory (PIM) technologies, which have shown superior performance and energy efficiency over conventional machine learning accelerators. We first comprehensively characterize the non-trivial trade-offs between design parameters in HD-PIM and PPA and accuracy metrics in HD-PIM. Then, we develop a multi-objective noise-aware Bayesian optimization algorithm to find the Pareto set (optimal trade-offs between metrics) of the HD-PIM design. Our methodology uniquely addresses the stochastic nature of ReRAM by integrating error characteristics into the optimization process, thereby enhancing the quality of the generated designs. Experimental results show that our configurations achieve up to 4.28% accuracy improvement, 35.38% power reduction, 49x timing improvement, and 10% area reduction over a non-optimized design.

In the hardware design space exploration process, it is critical to optimize both hardware parameters and algorithm-to-hardware mappings. Previous work has largely approached this simultaneous optimization problem by separately exploring the hardware design space and the mapspace—both individually large and highly nonconvex spaces—independently. The resulting combinatorial explosion has created significant difficulties for optimizers.

The hardware design space is high-dimensional and discrete. Systematic and efficient exploration of this space has been a significant challenge. Central to this problem is the intractable search complexity that grows exponentially with the design choices and the discrete nature of the search space. This work investigates the feasibility of learning a meaningful low-dimensional continuous representation for hardware designs to reduce such complexity and facilitate the search process. We devise a variational autoencoder (VAE)-based design space exploration framework called VAESA, to encode the hardware design space in a compact and continuous representation. We show that black-box and gradient-based design space exploration algorithms can be applied to the latent space, and design points optimized in the latent space can be reconstructed to high-performance realistic hardware designs. Our experiments show that performing the design space search on the latent space consistently leads to the optimal design point under a fixed number of samples. In addition, the latent space can improve the sample efficiency of the original algorithm by 6.8$\times$ and can discover hardware designs that are up to 5% more efficient than the optimal design searched directly in the high-dimensional input space.

Stochastic variational inference for Bayesian deep neural network (DNN) requires specifying priors and approximate posterior distributions over neural network weights. Specifying meaningful weight priors is a challenging problem, particularly for scaling variational inference to deeper architectures involving high dimensional weight space. We propose MOdel Priors with Empirical Bayes using DNN (MOPED) method to choose informed weight priors in Bayesian neural networks. We formulate a two-stage hierarchical modeling, first find the maximum likelihood estimates of weights with DNN, and then set the weight priors using empirical Bayes approach to infer the posterior with variational inference. We empirically evaluate the proposed approach on real-world tasks including image classification, video activity recognition and audio classification with varying complex neural network architectures. We also evaluate our proposed approach on diabetic retinopathy diagnosis task and benchmark with the state-of-the-art Bayesian deep learning techniques. We demonstrate MOPED method enables scalable variational inference and provides reliable uncertainty quantification.

Deep neural networks (DNNs) provide state-of-the-art results for a multitude of applications, but the approaches using DNNs for multimodal audiovisual applications do not consider predictive uncertainty associated with individual modalities. Bayesian deep learning methods provide principled confidence and quantify predictive uncertainty. Our contribution in this work is to propose an uncertainty aware multimodal Bayesian fusion framework for activity recognition. We demonstrate a novel approach that combines deterministic and variational layers to scale Bayesian DNNs to deeper architectures. Our experiments using in- and out-of-distribution samples selected from a subset of Moments-in-Time (MiT) dataset show a more reliable confidence measure as compared to the non-Bayesian baseline and the Monte Carlo dropout (MC dropout) approximate Bayesian inference. We also demonstrate the uncertainty estimates obtained from the proposed framework can identify out-of-distribution data on the UCF101 and MiT datasets. In the multimodal setting, the proposed framework improved precision-recall AUC by 10.2% on the subset of MiT dataset as compared to non-Bayesian baseline.

Blur is an important attribute in the study and modeling of the human visual system. In the blur discrimination experiments, just-noticeable additional blur required to differentiate from the reference blur level is measured. The past studies on blur discrimination have measured the sensitivity of the human visual system to blur using two-dimensional (2D) test patterns. In this study, subjective tests are performed to measure blur discrimination thresholds using stereoscopic 3D test patterns. Specifically, how the binocular disparity affects the blur sensitivity is measured on a passive stereoscopic display. A passive stereoscopic display renders the left and right eye images in a row interleaved format. The subjects have to wear circularly polarized glasses to filter the appropriate images to the left and right eyes. Positive, negative, and zero disparity values are considered in these experiments. A positive disparity value projects the objects behind the display screen, a negative disparity value projects the objects in front of the display screen, and a zero disparity value projects the objects at the display plane. The blur discrimination thresholds are measured for both symmetric and asymmetric stereo viewing cases. In the symmetric viewing case, the same level of additional blur is applied to the left and right eye stimulus. In the asymmetric viewing case, different levels of additional blur are applied to the left and right eye stimuli. The results of this study indicate that, in the symmetric stereo viewing case, binocular disparity does not affect the blur discrimination thresholds for the selected 3D test patterns. As a consequence of these findings, we conclude that the models developed for 2D blur discrimination can be used for 3D blur discrimination. We also show that the Weber model provides a good fit to the blur discrimination threshold measurements for the symmetric stereo viewing case. In the asymmetric viewing case, the blur discrimination thresholds decreased, and the decrease in threshold values is found to be dominated by eye observing the higher blur.

Texture granularity is an important attribute to quantify the level of details present in the image. This work presents a no-reference texture granularity index and shows using subjective experiments that the proposed granularity index correlates well with the perceived granularity of textures. In addition, a subjective study is conducted to assess the effect of compression on textures with varying degrees of granularity. It is shown that a measure of texture granularity can predict the compression quality.

The shape information of objects is perceptually very significant and can aid in the recognition of objects. Depending upon the bit budget, Internet browsing applications can take advantage of an arbitrary-shape region-of-interest (ROI) coding method by sending only the shape information first, and then progressively transmitting the ROI texture, followed by the less important non-ROI texture information. Unfortunately, the existing ROI coding methods, including the JPEG2000-based methods, do not support separate coding and decoding of the shape information. In particular, the maxshift and the scaling-based methods of JPEG2000 are limited in that the former does not allow the coding of any non-ROI bitplanes prior to the coding of the ROI bitplanes, and the latter supports only rectangular and circular ROIs. The paper presents a novel ROI-based coding method that extends the scaling-based method of JPEG2000, and allows the coding of arbitrary-shape ROIs, as well as the separate coding and transmission of the ROI shape information prior to the texture information. Coding results are presented to illustrate the performance of the proposed ROI-based coding method.

The JPEG2000 standard supports two methods to code regions-of-interest (ROIs) in an image-the maxshift method and the scaling-based method. Compared to the maxshift method, the scaling-based method is preferable in many applications because it allows the nonROI (background) region to be partially coded prior to coding all of the ROIs. This is accomplished by supporting arbitrary bitplane shift factors and by filling the most significant bit-planes of the nonROI region with zeros after the ROI bitplanes have been shifted up in value. In both of these methods, the embedded block coding with optimum truncation (EBCOT) algorithm is then applied to code the wavelet coefficients. However, when multiple regions-of-interest need to be coded, the performance of the scaling-based method degrades significantly since the EBCOT coder is no longer able to efficiently code the nonROI region, which is now less contiguous as compared to the single-ROI case. Also, the large header information that is required in JPEG2000 for each ROI degrades the coding performance. This paper presents an improved scaling-based method for the efficient coding of multiple, arbitrarily-shaped ROIs using a modified EBCOT algorithm. The proposed method utilizes the shape information of the different ROIs to generate a stripe mask, which is then used to optimize the coding performance. Coding results and comparisons with the JPEG2000 scaling-based method are presented to illustrate the improved performance of the proposed scheme.