Satisfactory extraction of buildings from aerial images has long been a challenging task. In the recently fully convolutional network (FCN)-based methods, the locality of the convolution operation is detrimental to handling global long-range dependencies for complex buildings, such as those shaded by trees, obscured by the shadow of high-rise ones, and blurred by the high-similarity pixels. Although graph neural networks (GNN) show clear-cut advantages in modeling semantic correlations between different segments or instances, existing FCN-GNN methods cannot directly perform graph reasoning for spatial features in an end-to-end framework due to two chief limitations: 1) it is costly to construct large fully-connected graphs; 2) there is not a standard pixel-based graph reasoning paradigm. We therefore developed a hybrid end-to-end FCN-like network to perform better building extraction on complex scenes, termed the Dual Spatial-Graph Refinement Network (DSRNet). Specifically, we first proposed a spatial-graph reasoning module (SR), which effectively constructs adjacent relations for dense pixels, to make fast graph reasoning for grid-based features possible. Considering SR as the basis, we further developed a dual spatial-graph refinement module (DSR), consisting of body and structure SRs (BSR and SSR), to make SR attentively perceive global buildings’ spatial-semantic relationships from two complementary perspectives. BSR was designed to enhance consistency within buildings and SSR models correlations of buildings’ structural information, to extract buildings with more coherent bodies and tight-fitting edge contours. Finally, we devised a Contour Alignment loss function (CA) to encourage the segmentation result to align correctly with the contours of the ground truths. DSRNet outperforms state-of-the-art FCN-based methods on Christchurch and Tokyo high-resolution building datasets and consistently shows improvements in building segmentation and contour extraction, especially in the case of complex scenes.

Without reducing the spectral resolution, hyperspectral image super-resolution has achieved remarkable progress thanks to the success of deep neural networks. However, existing methods can not fully excavate the latent high-frequency details only in the single spatial domain. Different from existing methods that only achieves the super-resolution task in spatial domain, we optimize the amplitude spectrum and phase spectrum in frequency domain to obtain high resolution hyperspectral image (HR-HSI). We propose a new unsupervised framework to reconstruct HR-HSI using only the observed low resolution HSI and HR multispectral image. Based on triple-level modeling, the encoder-decoder learns abundant features including contextual information from multiple scales. In addition, we propose iterative across domain consistency-difference (ADCD) module, which is embedded between encoder and decoder. In ADCD module, three parallel convolution streams, (amplitude spectrum adjustment branch, phase spectrum adjustment branch and spatial domain branch) are used to explore the consistency-difference between each other, which is preserved by memory units within the module. Particularly, we embed the dilated causal convolution in the frequency domain processing branch, which is convenient to flexibly adjust the receptive field and adapt to different domains. Extensive experiments are conducted on widely-used datasets in comparison with state-of-the-art models, demonstrating the advantage of the proposed method.

To date immunity to disruptions of multi-scale urban road networks (URNs) has not been effectively quantified. This study uses robustness as a meaningful - if partial - representation of immunity. We propose a novel Relative Area Index (RAI) based on traffic assignment theory to quantitatively measure the robustness of URNs under global capacity degradation due to three different types of disruptions, which takes into account many realistic characteristics. We also compare the RAI with weighted betweenness centrality, a traditional topological metric of robustness. We employ six realistic URNs as case studies for this comparison. Our analysis shows that RAI is a more effective measure of the robustness of URNs when multi-scale URNs suffer from global disruptions. This improved effectiveness is achieved because of RAI’s ability to capture the effects of realistic network characteristics such as network topology, flow patterns, link capacity, and travel demand. Also, the results highlight the importance of central management when URNs suffer from disruptions. Our novel method may provide a benchmark tool for comparing robustness of multi-scale URNs, which facilitates the understanding and improvement of network robustness for the planning and management of URNs.

This paper introduces a cross-scale attention based end-to-end learning framework for tree crown detection via UAV imagery. Given that UAV images covered a large forests, the illumination variations, shadow obstacles and texture repetition always lead to inaccurate tree crown detection results. We introduce a cross-scale attention based mechanism to address the above issues, enabling the tree crown detection framework to reason about the RGB texture information and depth information introduced by the automatically generated depth map jointly. Compared to traditional image based tree crown detection methods, our approach learns prior over geometrical structure information from the real 3D world, which is robust to the texture repetition and small tree crowns. The experimental results demonstrated that the proposed approach outperforms the traditional CNN based method.

In recent years, hyperspectral image (HSI) super-resolution has made significant progress by leveraging convolution neural network. Existing methods with spectral or spatial attention, which only consider the spectral similarity or pixel-pixel similarity, ignore sample-sample correlations and sparsity. Therefore, based on the fusion of HSI and multispectral image, we propose a new HSI super-resolution model with external-internal attention. Instead of considering a single sample, external attention module is employed to exploit the incorporating correlations between different samples to get a better feature representation. In addition, an internal attention module based on non-local operation is designed to explore the long-range dependencies information. Particularly, oriented to high mapping precision and low computational cost inference, spherical locality sensitive hashing is used to divide features into different hash buckets so that every query point is calculated in the hash bucket assigned to it, rather than based a weight sum of features across all positions. The sequential external-internal attention greatly improves the generalization ability and robustness of the model by modeling at the dataset level and at the sample level. Extensive experiments are conducted on five widely-used datasets in comparison with state-of-the-art models, demonstrating the advantage of the method we proposed.

We consider long-term path forecasting problems in crowds, where future sequence trajectories are generated given a short observation. Recent methods for this problem have focused on modeling social interactions and predicting multi-modal futures. However, it is not easy for machines to successfully consider social interactions, such as avoiding collisions while considering the uncertainty of futures under a highly interactive and dynamic scenario. In this paper, we propose a model that incorporates multiple interacting motion sequences jointly and predicts multi-modal socially acceptable distributions of futures. Specifically, we introduce a new aggregation mechanism for social interactions, which selectively models long-term inter-related dynamics between movements in a shared environment through a message passing mechanism. Moreover, we propose a loss function that not only accesses how accurate the estimated distributions of the futures are but also considers collision avoidance. We further utilize mixture density functions to describe the trajectories and learn the multi-modality of future paths. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast socially acceptable distributions in complex scenarios.

Accurate human path forecasting in complex and crowded scenarios is critical for collision avoidance of autonomous driving and social robots navigation. It still remains as a challenging problem because of dynamic human interaction and intrinsic multimodality of human motion. Given the observation, there is a rich set of plausible ways for an agent to walk through the circumstance. To address those issues, we propose a spatio-temporal model that can aggregate the information from socially interacting agents and capture the multimodality of the motion patterns. We use mixture density functions to describe the human path and predict the distribution of future paths with explicit density. To integrate more factors to model interacting people, we further introduce a coordinate transformation to represent the relative motion between people. Extensive experiments over several trajectory prediction benchmarks demonstrate that our method is able to forecast various plausible futures in complex scenarios and achieves state-of-the-art performance.

Recently, owing to the outstanding capability of deep learning in solving ill-posed problems, the single-frame super-resolution (SR) researches tend to focus on deep learning methods largely. However, related researches are implemented and evaluated through various datasets and different deep learning frameworks, which hinders the comparison of performance among different methods and heavily hampers the progress of SR techniques. In this study, we present GeoSR, an open source computer vision package for deep learning based single-frame remote sensing imagery super-resolution to facilitate the development of the SR community. As a unified, simple, and flexible package, GeoSR contains pipeline-like integrated tools from data retrieval to final result evaluation, which enables users to develop self-defined models conveniently; several state-of-the-art models trained through the same high-quality dataset are provided as the baseline in the package as well. Moreover, the proposed package could potentially serve as a viable backend for other related packages such as image segmentation with high efficiency.

Recently, deep learning algorithms, especially fully convolutional network based methods, have become very popular in the field of remote sensing. However, these methods are implemented and evaluated through various datasets and deep learning frameworks. There has not been a package that covers these methods in a unified manner. In this study, we introduce a computer vision package termed Geoseg that focuses on building segmentation and outline extraction. Geoseg implements nine state-of-the-art models as well as utility scripts needed to conduct model training, logging, evaluation, and visualization. The implementation of Geoseg emphasizes unification, simplicity, and flexibility. The performance and computational efficiency of all implemented methods are evaluated by a comparison experiment using a unified, high-quality aerial image dataset.

In this study, we present population distribution map and automated human settlement map for Maputo. To identify buildings from high-resolution remote sensing (HRRS) images of highly dense area, we used pixel-based classification frame with Ensemble Convolution Neural Network (ECNN) algorithm for building detection and used these identified buildings with zone based population data provided by the Japan International Cooperation Agency (JICA) to create population distribution map. Because of enormous HRRS image data, the training was performed on TSUBAME, a supercomputer operated by Tokyo Institute of Technology. The system used up to 28 Physical CPU cores, 128 GB of system memory and 16 GB of GPU memory for training. The result from building detection algorithm achieved overall accuracy of 96.8% and the average kappa of 70% and the population map created from it has a resolution of 10m × 10m.

The automatic digitizing of paper maps is a significant and challenging task for both academia and industry. As an important procedure of map digitizing, the semantic segmentation section is mainly relied on manual visual interpretation with low efficiency. In this study, we select urban planning maps as a representative sample and investigate the feasibility of utilizing U-shape fully convolutional based architecture to perform end-to-end map semantic segmentation. The experimental results obtained from the test area in Shibuya district, Tokyo, demonstrate that our proposed method could achieve a very high Jaccard similarity coefficient of 93.63% and an overall accuracy of 99.36%. For implementation on GPGPU and cuDNN, the required processing time for the whole Shibuya district can be less than three minutes. The results indicate the proposed method can serve as a viable tool for urban planning map semantic segmentation task with high accuracy and efficiency.

Based on inverse hydraulic-thermodynamic transient analyses and on an improved particle swarm optimization (PSO), a leak detection and localization method is proposed for liquid pipelines. The finite volume method is employed to numerically model the continuity, momentum, and energy equations. To determine the optimum-improved PSO algorithm and corresponding parameters, four types of algorithms were used for the analyses of a virtual pipeline. Accuracy, stability, robustness, and false alarm rate tests were conducted. The SIPSO algorithm has been shown to outperform the other algorithms. Two oil pipelines were utilized as example cases in conjunction with the use of the SIPSO algorithm. One of these pipelines was involved in a field opening experiment, while the other was involved in a real leak accident. The results demonstrated that smaller relative errors were achieved with the proposed method for the estimations of the location, coefficient, and starting time of the leak.