We propose a novel method for aerial visual localization over low Level-of-Detail (LoD) city models. Previous wireframe-alignment-based method LoD-Loc has shown promising localization results leveraging LoD models. However, LoD-Loc mainly relies on high-LoD (LoD3 or LoD2) city models, but the majority of available models and those many countries plan to construct nationwide are low-LoD (LoD1). Consequently, enabling localization on low-LoD city models could unlock drones' potential for global urban localization. To address these issues, we introduce LoD-Loc v2, which employs a coarse-to-fine strategy using explicit silhouette alignment to achieve accurate localization over low-LoD city models in the air. Specifically, given a query image, LoD-Loc v2 first applies a building segmentation network to shape building silhouettes. Then, in the coarse pose selection stage, we construct a pose cost volume by uniformly sampling pose hypotheses around a prior pose to represent the pose probability distribution. Each cost of the volume measures the degree of alignment between the projected and predicted silhouettes. We select the pose with maximum value as the coarse pose. In the fine pose estimation stage, a particle filtering method incorporating a multi-beam tracking approach is used to efficiently explore the hypothesis space and obtain the final pose estimation. To further facilitate research in this field, we release two datasets with LoD1 city models covering 10.7 km , along with real RGB queries and ground-truth pose annotations. Experimental results show that LoD-Loc v2 improves estimation accuracy with high-LoD models and enables localization with low-LoD models for the first time. Moreover, it outperforms state-of-the-art baselines by large margins, even surpassing texture-model-based methods, and broadens the convergence basin to accommodate larger prior errors.

Abstract With the popularity and advancement of 3D point cloud data acquisition technologies and sensors, research into 3D point clouds has made considerable strides based on deep learning. The semantic segmentation of point clouds, a crucial step in comprehending 3D scenes, has drawn much attention. The accuracy and effectiveness of fully supervised semantic segmentation tasks have greatly improved with the increase in the number of accessible datasets. However, these achievements rely on time‐consuming and expensive full labelling. In solve of these existential issues, research on weakly supervised learning has recently exploded. These methods train neural networks to tackle 3D semantic segmentation tasks with fewer point labels. In addition to providing a thorough overview of the history and current state of the art in weakly supervised semantic segmentation of 3D point clouds, a detailed description of the most widely used data acquisition sensors, a list of publicly accessible benchmark datasets, and a look ahead to potential future development directions is provided.

Abstract Image denoising aims to restore a clean image from a noisy image. Traditional methods utilizing convolutional neural networks (CNN) for denoising are trained using pairs of noisy and clean images to comprehend the transformation from a noisy image to a clean one. However, the acquisition of such image pairs in real‐world scenarios presents a challenge. Hence, numerous self‐supervised denoising techniques have been developed that do not require clean images for training. This study demonstrates that a straightforward loss design, concentrating on variance, can effectively train a standard CNN denoiser in a self‐supervised fashion. A novel theoretical framework is introduced for training a basic CNN denoising model using three constraints: mean, variance, and augmentation. The variance constraint is crucial as it prevents the trained model from converging to trivial solutions such as identity or zero mapping. This theory provides valuable insights for the development of new self‐supervised denoising methods. Furthermore, a method that applies this theory to proposed dual networks is developed, which consist of two standard CNN models predicting both the clean image and the noise. This approach enhances model capacity during training while minimizing computational costs during inference. This method exemplifies the implementation of the variance constraint and introduces a data constraint for dual networks. Notably, the proposed method only assumes the presence of additive white noise, irrespective of the noise distribution. This minimal assumption enhances the model's robustness against noise with complex or unknown distributions in real‐world distorted images. Experimental results indicate that the proposed Noise2Variance method exhibits commendable performance on peak signal noise ratio and structural similarity metrics compared to existing self‐supervised denoising techniques. Visual comparison of results further substantiates the efficacy of the proposed method. A comparison of model complexity reveals that the method is efficient among the compared CNN‐based techniques.

Building-based damage assessment usually involves two different tasks, i.e., localizing buildings before disasters and scoring the damage levels after disasters. Recently, Convolutional Neural Network based machine learning methods achieve impressive performance by directly fusing the feature from pre- and post-disaster satellite imageries, which are collected at totally different dates and climatic conditions. The discrepancy of input sources may disturb the learning process, especially when simultaneously detecting buildings and assessing the damage levels by a unified model become a new trend. In this paper, we propose a Dynamic Cross Fusion Network (DCFNet) that makes each task bidirectionally access to multi-levels features from a shared backbone network. Moreover, a task-shared head is designed to adaptively exchange information cross tasks and avoid under-fitting for one specific task. Without bells and whistles, the proposed DCSNet achieves state-of-the-art single-model performance in the xView2 Challenge, which provides a new and large-scale damage assessment dataset xBD.

The goal of image denoising is to recover a clean image from noisy input(s). For single image denoising, utilising similarities (or priors) within and across an image dataset helps recover clean images. As the noise level increases, using multiple frames become feasible, which is defined as burst denoising. In this study, the authors propose a deep residual model with squeeze‐and‐excitation (SE) modules for the burst denoising. Unlike previous methods, the authors' model does not need an explicit aligning procedure, which is light‐weighted and fast. The network contains a noise estimation convolutional neural network, which makes it capable of blind denoising. Besides, by inverting the image processing pipeline and simulating real noise in bursts, their model can suppress real noise blindly. Since denoising performance is closely related to the noise level, frame displacement, and the number of frames (burst length), intensive experiments including ablation study are performed. Quantitative results show that the proposed method performs significantly better than previous state‐of‐the‐art methods V‐BM4D and KPN in removing Gaussian noise. Qualitative results show that the proposed method is also effective in removing real noise using bursts and the SE module is key to reduce blur in results.

We propose a new method of no-reference (NR) image sharpness assessment. Our method is based on a multiscale decomposition of non-overlapping blocks, and on analyzing the statistics of Local-Mean Magnitude (LMM) maps. With the detected high-activity blocks, a set of inter-scale and inter-direction sharpness ratios are found. These sharpness ratios are closely correlated with the level of blurring, and is capable of measuring image sharpness effectively from a multiscale and a directional view. They are linearly combined using automatic estimated weighting parameters to induce an overall effective sharpness model. To enhance accuracy, a sharpness factor based on the blur effect on DCT edges and a log-energy sharpness model are also incorporated into the method. Experiments on a large number of public images have shown that our method consistently produces predictions that are highly correlated with human perceptual of image sharpness, and outperforms the current state-of-the-art algorithms.

Image demosaicing and denoising are import steps of image signal processing. Sequential executions of demosaicing and denoising have essential drawbacks that they degrade the results of each other. Joint demosaicing and denoising overcomes the difficulties by solving the two problems in one model. This paper introduces a unified object function with hidden priors and a variant of ADMM to recover a full-resolution color image with a noisy Bayer input. Experimental results demonstrate that our method performs better than state-of-the-art methods in both PSNR comparison and human vision. In addition, our method is much more robust to variations of noise level.

Fast abnormal events detection in video is important for intelligent analysis of video. This paper proposes a fast anomaly detection algorithm based on sparse optical flow. We improve the efficiency of optical flow computation with foreground mask and spacial sampling and increase the robustness of optical flow with good feature (TK) points selecting and forward-backward filtering. A foreground channel is also added to the feature vector to help detect static or low speed objects. The algorithm is validated on real-life traffic surveillance to prove its effectiveness. It is also evaluated on a benchmark dataset and achieve detection results comparable to state-of-art methods and outperforms them at pixel-level when the false alarm rate is low. The strength of our algorithm is that it runs real-time on the benchmark dataset which is hundreds of times faster than comparative methods.

This paper proposes an approach for fast and parallel video processing on MapReduce-based clusters such as Apache Hadoop. By utilizing clusters, the approach is able to handle large-scale of video data and the processing time can be significantly reduced. Technique details of performing video analysis on clusters are revealed, including method of porting typical video processing algorithms designed for a single computer to the proposed system. As case studies, face detection and motion detection and tracking algorithms have been implemented on clusters. Performance experiments on an Apache Hadoop cluster of six computers show that the system is able to reduce the running time of the two implemented algorithms to below 25% of that of a single computer. The applications of the system include smart city video surveillance, services provided by video sites and satellite image processing.