Previous research on the diagnosis of Bipolar disorder has mainly focused on resting-state functional magnetic resonance imaging. However, their accuracy can not meet the requirements of clinical diagnosis. Efficient multimodal fusion strategies have great potential for applications in multimodal data and can further improve the performance of medical diagnosis models. In this work, we utilize both sMRI and fMRI data and propose a novel multimodal diagnosis model for bipolar disorder. The proposed Patch Pyramid Feature Extraction Module extracts sMRI features, and the spatio-temporal pyramid structure extracts the fMRI features. Finally, they are fused by a fusion module to output diagnosis results with a classifier. Extensive experiments show that our proposed method outperforms others in balanced accuracy from 0.657 to 0.732 on the OpenfMRI dataset, and achieves the state of the art.

The deep neural networks (DNNs) trained by adversarial training (AT) usually suffered from significant robust generalization gap, i.e., DNNs achieve high training robustness but low test robustness. In this paper, we propose a generic method to boost the robust generalization of AT methods from the novel perspective of attribution span. To this end, compared with standard DNNs, we discover that the generalization gap of adversarially trained DNNs is caused by the smaller attribution span on the input image. In other words, adversarially trained DNNs tend to focus on specific visual concepts on training images, causing its limitation on test robustness. In this way, to enhance the robustness, we propose an effective method to enlarge the learned attribution span. Besides, we use hybrid feature statistics for feature fusion to enrich the diversity of features. Extensive experiments show that our method can effectively improves robustness of adversarially trained DNNs, outperforming previous SOTA methods. Furthermore, we provide a theoretical analysis of our method to prove its effectiveness.

Standard classification models rely on the assumption that all the classes of interest are equally represented in training datasets. However, visual phenomena exhibit a long-tailed distribution, such that many standard approaches fail to properly model and result in a considerable degeneration on accuracy. The recent methods have produced encouraging results, but their efforts only seek to simulate the statistical relationship between data and labels and compensate for imbalanced data-related issues, without addressing the underlying causal mechanisms. In this paper, a comprehensive structural causal model is developed to excavate the intrinsic causal mechanism between data and labels. Specifically, we assume that each input is constructed from a mix of causal factors and non-causal factors, and only the causal factors cause the classification judgments. In order to extract such causal factors from inputs and then reconstruct the invariant causal mechanisms, we propose a Causal Invariant Transformation algorithm for Long-tailed recognition (CITL), which generates diverse data to avoid the over-fitting on the tail classes and enforces the learnt representations to maintain the causal factors and eliminate the non-causal factors. Our extensive experimental results on several widely used datasets have demonstrated the effectiveness of our proposed CITL approach.

The estimation of Individual Treatment Effect (ITE) is a challenging problem in causal inference, due to the missing counterfactual data and the selection bias. In this paper, we propose a novel representation learning framework via the Learnable Query based transformer for Treatment Effect Estimation (LQTEE). A certain number of queries are learned in a sample-agnostic way and extract global critical features from covariate and treatment data separately. We also propose the hierarchical propensity score regularized adversarial loss to obtain balanced covariate representations, and the mutual orthogonal constraint to force queries to focus on diverse parts of covariates, thus the impact of instrumental variables can be adaptively reduced. Treatment representation learning enables our estimator to support general-purpose treatments, and more importantly, it can reveal the underlying patterns of data-generation process efficiently. Extensive experiments show that our ITE estimator significantly outperforms the state-of-the-art methods.

Accurate polyp segmentation from colonoscopy images is essential for identifying colorectal cancer. Recently, segmentation methods based on convolutional neural networks and transformers have represented excellent performance for image polyp segmentation. However, these methods are mostly designed for individual images rather than the entire video datasets, which results in the absence of sequential relationships among lesion images and neglects the significant intrinsic property of continuous video. In this work, we propose a temporal correlation network (TC-Net) for video polyp segmentation. In TC-Net, the temporal correlation is unprecedentedly modeled based on the relationship between the original video and the captured frames to be adaptable for video polyp segmentation, and the network is also calibrated for the corresponding time correlation output. Furthermore, we design a dual-track learning strategy for the optimization method in TC-Net to ensure the independence of TC-Net during the learning process to adequately exploit the optimization effect of temporal correlation. The network’s effectiveness is demonstrated by extensive experiments on five publicly available biomedical datasets, and TC-Net achieves state-of-the-art (SOTA) performance.

Non-parallel voice conversion (VC) is a technique of transfer-ring voice from one style to another without using a parallel corpus in model training. Various methods are proposed to approach non-parallel VC using deep neural networks. Among them, CycleGAN-VC and its variants have been widely accepted as benchmark methods. However, there is still a gap to bridge between the real target and converted voice and an increased number of parameters leads to slow convergence in training process. Inspired by recent advancements in unsupervised image translation, we propose a new end-to-end unsupervised framework U-GAT-VC that adopts a novel inter- and intra-attention mechanism to guide the voice conversion to focus on more important regions in spectrograms. We also introduce disentangle perceptual loss in our model to capture high-level spectral features. Subjective and objective evaluations shows our proposed model outperforms CycleGAN-VC2/3 in terms of conversion quality and voice naturalness.

While deep learning makes significant achievements in Artificial Intelligence (AI), the lack of transparency has limited its broad application in various vertical domains. Explainability is not only a gateway between AI and society, but also a powerful feature to detect flaws of the models and bias of the data. Local Interpretable Model-agnostic Explanation (LIME) is a widely-accepted technique that explains the predictions of any classifier faithfully by learning an interpretable model locally around the predicted instance. However, the sampling operation in the standard implementation of LIME is defective. Perturbed samples are generated from an uniform distribution, ignoring the complicated correlation between features. Moreover, as the local decision boundary is non-linear for most complex networks, linear approximation may produce serious errors. This paper proposes an high-interpretability and high-fidelity local explanation method, known as Kernel-based LIME with Feature Dependency Sampling (KLFDS). KLFDS enhances interpretability by feature sampling with intrinsic dependency. Besides, KLFDS improves the local explanation fidelity by approximating nonlinear boundary of local decision. We evaluate our method on image classification tasks and results show that KLFDS's explanation of the black-box model achieves much better performance than original LIME.

With the coming age of big data, the image signals play more and more important role in our life due to the extraordinary advance of network communication technology, and the corresponding high efficiency image processing techniques are demanded urgently. The Fourier transform is an important image processing tool which is used in a wide range of applications. Traditional Fourier transform algorithm computes on the value of each point of image, regardless of their properties in frequency domain. However, most image signals possess sparsity in frequency domain. In this paper, we present a new fast two-dimensional Fourier transform based on image sparsity. With hash function including a series of procedures such as random spectrum permutation, filtering and subsampling in frequency domain, the algorithm could identify and estimate the k largest coefficients quickly. In most sparse cases, the resulting algorithm performs faster than state-of-the-art fast Fourier transform algorithm, FFTW.

Compressive sensing allows a signal to be sampled at sub-Nyquist rate and still get recovered exactly, if the signal is sparse in some domain. Block compressive sensing (BCS) is advocated for practical image compressive sensing, since it processes image at block level and significantly reduces the memory requirement for storing projection matrix. However, existing BCS methods process blocks separately, which breaks the continuity between blocks and usually produces blocking artifacts. This paper proposes a new image compressive sensing scheme using overlapped-block projection and reconstruction (OBPR), in which the sampling is performed on overlapped blocks. During reconstruction, the sparsity constraint in transform domain is also enforced on the overlapped blocks. An augmented Lagrangian method is used to solve the optimization problem efficiently. Experimental results show that the proposed OBPR scheme achieves significantly better results than the existing BCS schemes in reconstruction quality.

With the coming age of big data, the cloud technology, referred to as the computations or applications through the Internet, is dramatically developed. The screen content has become one of the most common data form due to the extraordinary advance of network communication technology, and the JCT-VC has started to develop new standard focusing on improving the efficiency of screen content based on High Efficiency Video Coding (HEVC). Nevertheless, the research on the quality assessment of screen content is still quite limited at the current stage. In this paper, we present a study on subjective quality assessment of the Screen Content Images (SCIs) and investigate whether the existing objective Image Quality Assessment (IQA) methods can effectively evaluate the quality of distorted SCIs. We construct a new Screen Content Database (SCD) including 24 source SCIs and 492 compressed ones with two codecs including HEVC as well as the HEVC extension. The Single Comparison (SC) method is employed for the subjective viewing to guarantee the reliability of the results. In our experiment, the correlations of eight popular IQA methods with the obtained Mean Opinion Score (MOS) values are evaluated. The result indicates that visual information fidelity method can achieve highest consistency with human visual perception.