Vision-Language Models (VLMs) have demonstrated remarkable capabilities in visual classification tasks. Existing methods for enhancing VLMs on this task often rely heavily on direct category-to-image matching, which limits generalization and results in suboptimal performance. In addition, these methods provide no understanding of why a specific category is chosen. To address these limitations, we introduce a new deliberative visual classification task that decomposes the classification process into multiple deliberative steps and leverages Large Language Models (LLMs) to perform explicit reasoning before the final decision. Specifically, we propose a Retrieval-driven Reasoning model (RdR) with two components, i.e., retrieval database construction and deliberative category prediction. The first component leverages LLMs to extract category-relevant descriptors and constructs a retrieval database for effective image–descriptor matching. The second component facilitates multiple deliberative steps and performs explicit reasoning based on the retrieved descriptors to augment the category prediction. Extensive experiments on multiple datasets demonstrate that RdR consistently outperforms strong baselines, highlighting its robustness and generalization ability.

Despite its wide applications in criminal investigations and clinical communications with patients suffering from autism, automatic micro-expression recognition remains a challenging problem because of the lack of training data and imbalanced classes problems. In this study, we proposed a meta-learning-based multi-model fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on the metric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable for model-level fusion. The frame difference and optical flow features were fused, deep features were extracted from the fused feature, and finally in the meta-learning-based framework, weighted sum model fusion method was applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-art methods on four datasets. The code is available at https://github.com/wenjgong/meta-fusion-based-method .

Abstract Object tracking is a very important step in building an intelligent video monitoring system that can protect people’s lives and property. In recent years, although visual tracking has made great progress in terms of speed and accuracy, there are still few real-time high-precision tracking algorithms. Although discriminative correlation filters have excellent performance in tracking speed, there are deficiencies in handling fast motion. This leads to the inability to achieve long-term stable tracking results. The long-time tracking with discriminative correlation filter (LT-DCF) was proposed to solve these deficiencies. We use larger size detection image blocks and smaller size filters to increase the proportion of real samples to solve the boundary effects of fast motion. And we combine the histogram of oriented gradient (HOG) feature detection and scale-invariant feature transform (SIFT) key point detection to solve the obstacles caused by scale variations. The detector with deep feature flow is then incorporated into the tracker to detect key frames to improve tracking accuracy. This method has achieved more than 75% of the distance accuracy and 70% of the overlapping success rate on the VOT2015 and VOT2016 datasets, and the stable tracking video length can reach 6895 frames.

In example-based human pose estimation, the configuration of an evolving object is sought given visual evidence, having to rely uniquely on a set of sample images. We assume here that, at each time instant of a training session, a number of feature measurements is extracted from the available images, while ground truth is provided in the form of the true object pose. In this scenario, a sensible approach consists in learning maps from features to poses, using the information provided by the training set. In particular, multivalued mappings linking feature values to set of training poses can be constructed. To this purpose we propose a belief modeling regression (BMR) approach in which a probability measure on any individual feature space maps to a convex set of probabilities on the set of training poses, in a form of a belief function. Given a test image, its feature measurements translate into a collection of belief functions on the set of training poses which, when combined, yield there an entire family of probability distributions. From the latter either a single central pose estimate or a set of extremal ones can be computed, together with a measure of how reliable the estimate is. Contrarily to other competing models, in BMR the sparsity of the training samples can be taken into account to model the level of uncertainty associated with these estimates. We illustrate BMR's performance in an application to human pose recovery, showing how it outperforms our implementation of both relevant vector machine and Gaussian process regression. Finally, we discuss motivation and advantages of the proposed approach with respect to its most direct competitors.

Summary Accurate resource requests prediction is essential to achieve optimal job scheduling and load balancing for cloud Computing. Existing prediction approaches fall short in providing satisfactory accuracy because of high variances of cloud metrics. We propose a deep belief network (DBN)‐based approach to predict cloud resource requests. We design a set of experiments to find the most influential factors for prediction accuracy and the best DBN parameter set to achieve optimal performance. The innovative points of the proposed approach is that it introduces analysis of variance and orthogonal experimental design techniques into the parameter learning of DBN. The proposed approach achieves high accuracy with mean square error of [10−6,10−5], approximately 72%reduction compared with the traditional autoregressive integrated moving average predictor, and has better prediction accuracy compared with the state‐of‐art fractal modeling approach. Copyright © 2016 John Wiley & Sons, Ltd.

There arises the needs for fast processing of continuous video data using embedded devices, for example the one needed for UAV aerial photography. In this paper, we proposed a distributed embedded platform built with NVIDIA Jetson TX1 using deep learning techniques for real time video processing, mainly for object detection. We design a Storm based distributed real-time computation platform and ran object detection algorithm based on convolutional neural networks. We have evaluated the performance of our platform by conducting real-time object detection on surveillance video. Compared with the high end GPU processing of NVIDIA TITAN X, our platform achieves the same processing speed but a much lower power consumption when doing the same work. At the same time, our platform had a good scalability and fault tolerance, which is suitable for intelligent mobile devices such as unmanned aerial vehicles or self-driving cars.

A large number of video applications require real-time response. The high-speed video processing then requires a distributed and parallelized framework utilizing all possible computing resources, i.e., both Central Processing Unit (CPU) and Graphics Processing Unit (GPU) at their best. The CPU-GPU collaboration may cause resource imbalance where GPU-based jobs consume less computing resources while occupying more memory compared with CPU-based jobs. In this paper, we propose a load-aware pluggable cloud framework for real-time video processing where CPU-GPU switching based on workload status can be performed at runtime. Furthermore, we design aspect-oriented monitors to collect framework metrics and propose a distance coverage algorithm to detect performance degradation in order to make sure that the framework runs optimally to achieve good performance when a load-aware task switching is made. We have comprehensively evaluated the framework and the evaluation results show that the proposed framework has good performance, reusability, pluggability, and scalability.