State-of-the-art Active Learning (AL) methods often encounter challenges associated with a hysteretic learning process and an expensive data sampling mechanism. The former implies that data selection in the ($i+1$)-th round is solely based on the learned model’s results in the$i$-th round. The latter involves using model inference to calculate data value (e.g., uncertainty estimation based on model inference), which can be cumbersome, particularly when working with large datasets or Deep Neural Networks (DNNs). To address these challenges, we propose FastAL, an efficient and dynamic deep AL framework. Our approach includes an efficient method for calculating data value from the frequency domain perspective, generating multiple candidates. Then, we introduce the Fast Evaluation Module, which directly calculates each candidate’s contribution to future model training and selects the best options. In addition, current AL methods, particularly those based on uncertainty, are susceptible to data bias, which implies that selected data may not represent the original unlabeled data adequately. To alleviate this issue, we propose the De-similar Module, which removes partially similar data. The above three modules are model-agnostic and thus can be seamlessly integrated into any Active Learning framework. We conducted rigorous experiments on various benchmark datasets to validate our approach’s effectiveness. Our results demonstrate that FastAL outperforms other state-of-the-art methods by a significant margin, including those based on uncertainty, diversity, and expected model change.

Despite the large-scale adoption of Artificial Intelligence (AI) models in healthcare, there is an urgent need for trustworthy tools to rigorously backtrack the model decisions so that they behave reliably. Counterfactual explanations take a counter-intuitive approach to allow users to explore "what if" scenarios gradually becoming popular in the trustworthy field. However, most previous work on model's counterfactual explanation cannot generate in-distribution attribution credibly, produces adversarial examples, or fails to give a confidence interval for the explanation. Hence, in this paper, we propose a novel approach that generates counterfactuals in locally smooth directed semantic embedding space, and at the same time gives an uncertainty estimate in the counterfactual generation process. Specifically, we identify low-dimensional directed semantic embedding space based on Principal Component Analysis (PCA) applied in differential generative model. Then, we propose latent space smoothing regularization to rectify counterfactual search within in-distribution, such that visually-imperceptible changes are more robust to adversarial perturbations. Moreover, we put forth an uncertainty estimation framework for evaluating counterfactual uncertainty. Extensive experiments on several challenging realistic Chest X-ray and CelebA datasets show that our approach performs consistently well and better than the existing several state-of-the-art baseline approaches.

Active learning (AL) aims to sample the most valuable data for model improvement from the unlabeled pool. Traditional works, especially uncertainty-based methods, are prone to suffer from a data bias issue, which means that selected data cannot cover the entire unlabeled pool well. Although there have been lots of literature works focusing on this issue recently, they mainly benefit from the huge additional training costs and the artificially designed complex loss. The latter causes these methods to be redesigned when facing new models or tasks, which is very time-consuming and laborious. This article proposes a feature-matching-based uncertainty that resamples selected uncertainty data by feature matching, thus removing similar data to alleviate the data bias issue. To ensure that our proposed method does not introduce a lot of additional costs, we specially design a unsupervised fusion feature matching (UFFM), which does not require any training in our novel AL framework. Besides, we also redesign several classic uncertainty methods to be applied to more complex visual tasks. We conduct rigorous experiments on lots of standard benchmark datasets to validate our work. The experimental results show that our UFFM is better than the similar unsupervised feature matching technologies, and our proposed uncertainty calculation method outperforms random sampling, classic uncertainty approaches, and recent state-of-the-art (SOTA) uncertainty approaches.

Recent transformer-based models, especially patch-based methods, have shown huge potentiality in vision tasks. However, the split fixed-size patches divide the input features into the same size patches, which ignores the fact that vision elements are often various and thus may destroy the semantic information. Also, the vanilla patch-based transformer cannot guarantee the information communication between patches, which will prevent the extraction of attention information with a global view. To circumvent those problems, we propose an Efficient Attention Pyramid Transformer (EAPT). Specifically, we first propose the Deformable Attention, which learns an offset for each position in patches. Thus, even with split fixed-size patches, our method can still obtain non-fixed attention information that can cover various vision elements. Then, we design the Encode-Decode Communication module (En-DeC module), which can obtain communication information among all patches to get more complete global attention information. Finally, we propose a position encoding specifically for vision transformers, which can be used for patches of any dimension and any length. Extensive experiments on the vision tasks of image classification, object detection, and semantic segmentation demonstrate the effectiveness of our proposed model. Furthermore, we also conduct rigorous ablation studies to evaluate the key components of the proposed structure.

Abstract Saliency detection has been a hot topic in the field of computer vision. In this paper, we propose a novel approach that is based on multiscale segmentation and fuzzy broad learning. The core idea of our method is to segment the image into different scales, and then the extracted features are fed to the fuzzy broad learning system (FBLS) for training. More specifically, it first segments the image into superpixel blocks at different scales based on the simple linear iterative clustering algorithm. Then, it uses the local binary pattern algorithm to extract texture features and computes the average color information for each superpixel of these segmentation images. These extracted features are then fed to the FBLS to obtain multiscale saliency maps. After that, it fuses these saliency maps into an initial saliency map and uses the label propagation algorithm to further optimize it, obtaining the final saliency map. We have conducted experiments based on several benchmark datasets. The results show that our solution can outperform several existing algorithms. Particularly, our method is significantly faster than most of deep learning-based saliency detection algorithms, in terms of training and inferring time.

Rain removal from a single image is a challenging problem and has attracted much attention in recent years. In this paper, we revisit the single image deraining problem, and present a novel solution. The central idea of our solution is to merge the merits of two-phase processing methods and the Fuzzy Broad Learning System (FBLS). Specifically, our solution first uses the dehazing algorithm to preprocess the input rainy image and separates it into the detail layer and the base layer. After that, it puts the Y-channel image of the detail layer into the FBLS to obtain the derained Y channel image, which is then combined with the Cb and Cr channel images to obtain the derained detail layer. Later, it fuses the derained detail layer and the base layer to get a preliminary derained image. Finally, it superimposes the details extracted from the dehazed image with some transparency on the preliminary result, obtaining the final result. Experimental results based on both real and synthetic rainy images demonstrate that our proposed solution can outperform several state-of-the-art algorithms, while it consumes much less running time and training time, compared against the competitors.

The saliency detection technologies are very useful to analyze and extract important information from given multimedia data, and have already been extensively used in many multimedia applications. Past studies have revealed that utilizing the global cues is effective in saliency detection. Nevertheless, most of prior works mainly considered the single-scale segmentation when the global cues are employed. In this paper, we attempt to incorporate the multi-scale global cues for saliency detection problem. Achieving this proposal is interesting and also challenging (e.g., How to obtain appropriate foreground and background seeds effectively? How to merge rough saliency results into the final saliency map efficiently?). To alleviate the challenges, we present a three-phase solution that integrates several targeted strategies, first, a self-adaptive strategy for obtaining appropriate filter parameters; second, a cross-validation scheme for selecting appropriate background and foreground seeds; and third, a weight-based approach for merging the rough saliency maps. Our solution is easy to understand and implement, but without loss of effectiveness. Extensive experimental results based on benchmark datasets demonstrate the feasibility and competitiveness of our proposed solution.

Saliency detection has been a hot topic in computer vision. Among existing approaches, a representative one is to use the convex hull prior to find the salient object in the image; and there are many variants that are based on the convex hull prior. Most of these works used a single center to construct the convex hull center prior map, while few attention has been made on the use of multiple centers. In this paper, we propose a multi-center convex hull prior based solution for saliency detection. Particularly, our solution also integrates two non-trivial optimizations: one is for obtaining an enhanced global color distinction prior map, and another is for refining the preliminary saliency map. We experimentally evaluate our solution through comparing against state-of-the-art algorithms. The results demonstrate the effectiveness and superiorities of the proposed solution.

Saliency detection has been a hot topic in computer vision and image processing communities. Utilizing the global cues has been shown effective in saliency detection, whereas most of prior works mainly considered the single-scale segmentation when the global cues are employed. In this paper, we attempt to incorporate the multi-scale global cues (MSGC) for saliency detection. Achieving this proposal is interesting and also challenging (e.g., how to obtain appropriate foreground and background seeds; how to merge rough saliency results into the final saliency map efficiently). To alleviate various challenges, we present a solution that integrates three targeted techniques: (i) a self-adaptive approach for obtaining appropriate filter parameters; (ii) a cross-validation approach for selecting appropriate background and foreground seeds; and (iii) a weight-based approach for merging the rough saliency maps. Our solution is easy-to-understand and implement, but without loss of effectiveness. We have validated its competitiveness through widely used benchmark datasets.

Many state-of-the-art saliency detection algorithms rely on the boundary prior, but these algorithms simply suppose the boundaries around an image as background regions. Here we propose a fast and effective algorithm for salient object detection. First, a novel method is proposed to approximately locate the foreground object by using the convex hull from Harris corner. On this basis, we divide the saliency values of different regions into two parts and generate the corresponding cue maps (foreground and background), which are combined into a convex hull prior map. Then a new prior based on distance to the convex hull center is proposed to replace the center prior. Finally, the convex hull prior map and the convex hull center-biased map are combined to be the saliency map, which is then optimized to get the final result. Compared with eighteen existing algorithms and tested on several datasets, the present algorithm performs well in terms of precision and recall.

Image resizing is a key technique for displaying images on different devices, and has attracted much attention in the past few years. This paper reviews the image resizing methods proposed in recent years, gives a detailed comparison on their performance, and reveals the main challenges raised in several important issues such as preserving an important region, minimizing distortions, and improving efficiency. Furthermore, this paper discusses the research trends and points out the possible hotspots in this field. We believe this survey can give some guidance for researchers from relevant research areas, offering them an overall and novel view.

ABSTRACT In this paper, we propose a new video composition method by 3D mean‐value coordinate (MVC). 2D MVCs have been widely used in image composition; however, when 2D MVC is applied to a video sequence directly, because of over‐blending and the lack of temporal consistency, some unnatural effects may appear in the final composite results. Although 3D Poisson editing can maintain spatial and temporal consistency, it also leads to high algorithm complexity. Instead of 3D Poisson editing, we use the 3D MVC to seamlessly blend a given source video patch into a target video sequence; this approach is able to achieve high‐performance blending with less computation. We show that the combination of alpha matte‐based approaches and our method can further refine the produced video when the boundaries of the source object and the target object are very different. Our algorithm can be paralleled and run on a graphics processing unit. The experimental results show that our method is effective and efficient. Copyright © 2012 John Wiley & Sons, Ltd.

Many important alpha matting algorithms are proposed in recent years, which focus on how to solve the alpha value on image interactively. However, few papers discuss about why the border of opaque object in image is half-transparent. In this paper, we show how the optical principles produce the translucent effect on photography. We denote the matting problem in a unified form, by which form, we can analyze and compare different matting algorithm by comparing their sampling function and matting matrix. Our work can help us to propose and evaluate new matting algorithm. We also expand our framework to video matting and get an enhanced video matting algorithm.