Video anomaly detection (VAD) is a significant computer vision problem. Existing deep neural network (DNN) based VAD methods mostly follow the route of frame reconstruction or frame prediction. However, the lack of mining and learning of higher-level visual features and temporal context relationships in videos limits the further performance of these two approaches. Inspired by video codec theory, we introduce a brand-new VAD paradigm to break through these limitations: First, we propose a new task of video event restoration based on keyframes. Encouraging DNN to infer missing multiple frames based on video keyframes so as to restore a video event, which can more effectively motivate DNN to mine and learn potential higher-level visual features and comprehensive temporal context relationships in the video. To this end, we propose a novel U-shaped Swin Transformer Network with Dual Skip Connections (USTN-DSC) for video event restoration, where a cross-attention and a temporal upsampling residual skip connection are introduced to further assist in restoring complex static and dynamic motion object features in the video. In addition, we propose a simple and effective adjacent frame difference loss to constrain the motion consistency of the video sequence. Extensive experiments on benchmarks demonstrate that USTN-DSC outperforms most existing methods, validating the effectiveness of our method.

Evolutionary multitask optimization (EMTO) has recently attracted widespread attention in the evolutionary computation community, which solves two or more tasks simultaneously to improve the convergence characteristics of tasks when individually optimized. Effective knowledge between tasks is transferred by taking advantage of the parallelism of population-based search. Without any prior knowledge about tasks, it is a challenging problem of how to adaptively transfer effective knowledge between tasks and reduce the impact of negative transfer in EMTO. However, these two issues are rarely studied simultaneously in the existing literature. Besides, in complex many-task environments, the potential relationships among individuals from highly diverse populations associated with tasks directly determine the effectiveness of cross-task knowledge transfer. Keeping those in mind, we propose a multitask evolutionary algorithm based on anomaly detection (MTEA-AD). Specifically, each task is assigned a population and an anomaly detection model. Each anomaly detection model is used to learn the relationship among individuals between the current task and the other tasks online. Individuals that may carry negative knowledge are identified as outliers, and candidate transferred individuals identified by the anomaly detection model are selected to assist the current task, which may carry common knowledge across the current task and other tasks. Furthermore, to realize the adaptive control of the degree of knowledge transfer, the successfully transferred individuals that survive to the next generation through the elitism are used to update the anomaly detection parameter. The fair competition between offspring and candidate transferred individuals can effectively reduce the risk of negative transfer. Finally, the empirical studies on a series of synthetic benchmarks and a practical study are conducted to verify the effectiveness of MTEA-AD. The experimental results demonstrate that our proposal can adaptively adjust the degree of knowledge transfer through the anomaly detection model to achieve highly competitive performance compared to several state-of-the-art EMTO methods.

Taxi arrival time prediction is essential for building intelligent transportation systems. Traditional prediction methods mainly rely on extracting features from traffic maps, which cannot model complex situations and nonlinear spatial and temporal relationships. Therefore, we propose Multi-View Spatial-Temporal Model (MVSTM) to capture the mutual dependence of spatial-temporal relations and trajectory features. Specifically, we use graph2vec to model the spatial view, dual-channel temporal module to model the trajectory view, and structural embedding to model traffic semantics. Experiments on large-scale taxi trajectory data have shown that our approach is more effective than the existing novel methods. The source code can be found at https://github.com/775269512/SIGSPATIAL-2021-GISCUP-4th-Solution.

Accurate classification and comprehensive explanation is crucial to build a computer aided diagnosis (CAD) system of diffuse lung disease (DLD). Although deep neural networks (DNNs) have been applied to this task, the classification performance and reliability are not satisfied for medical clinical requirements. Specifically, DNNs are regarded as unexplainable “black-box” in general, and, thus, are not deemed reliable by expects. In this paper, we propose a neural network structure search approach based on evolutionary algorithm to improve the DNN's effectiveness and interpretability, and applied to the pulmonary textures classification problem. Through this network structure search approach, we find out how a DNN's subnet recognize the pulmonary textures features, then filter out the redundant subnets, and retain the most distinctive feature subnets. Besides, we utilize the method of feature visualization and the fine-grained heat map of the activation to interpret network's decision-making process. Finally, through quantitatively and qualitatively evaluate on a real dataset of diffuse lung disease, we verify the effectiveness of this neural network structure search approach on VggNet and ResNet, and achieve the state-of-the-art performance. We can classify the pulmonary textures on high-resolution computed tomography (HRCT) images.

The existing 3D point cloud classification/segmentation networks directly use Convolutional Neural Networks (C-NNs) to extract features from indoor data and have no advantages handling complex outdoor scenes. This is mainly due to the segmentation of large scale outdoor scenes depends on global context information. Inspired by Global Attention (GA) mechanism, we design a Global Point Attention module (GPA) by regarding high-level features, which usually contain rich semantic information, as a guidance to low-level features. In this paper, we embed GPA in PointSIFT to accomplish segmentation and call this new network PointSIFT-GPA. Experimental results on the US3D dataset demonstrate the performance of GPA and the superior performance of PointSIFT-GPA. In particular, PointSIFT-GPA ranks the 2-nd place on 2019 IEEE GRSS Data Fusion Contest 3D Point Cloud Classification Challenge with mIoU 0.9454.

This paper proposes a novel algorithm to estimate the noise level function (NLF) of signal-dependent noise (SDN) from a single image based on the sparse representation of NLFs. Noise level samples are estimated from the high-frequency discrete cosine transform (DCT) coefficients of nonlocal-grouped low-variation image patches. Then, an NLF recovery model based on the sparse representation of NLFs under a trained basis is constructed to recover NLF from the incomplete noise level samples. Confidence levels of the NLF samples are incorporated into the proposed model to promote reliable samples and weaken unreliable ones. We investigate the behavior of the estimation performance with respect to the block size, sampling rate, and confidence weighting. Simulation results on synthetic noisy images show that our method outperforms existing state-of-the-art schemes. The proposed method is evaluated on real noisy images captured by three types of commodity imaging devices, and shows consistently excellent SDN estimation performance. The estimated NLFs are incorporated into two well-known denoising schemes, nonlocal means and BM3D, and show significant improvements in denoising SDN-polluted images.

This paper proposes a noise estimation method for signal-dependent sensor noise based on sparse representation of noise level functions (NLFs). Homogeneous blocks are detected by an image structure analyzer, and grouped to estimate noise levels for various image intensities with confidences. The noise level function is recovered from the incomplete and noisy estimated samples by solving its sparse representation under a trained basis. Experimental results show that our proposed method accurately estimates NLFs for both smooth and highly-textured images over various noise levels.