Detecting earthquake-induced landslides in remote sensing images is challenging due to the varying sizes of landslides, uneven distribution, and the prevalence of small targets. This study proposes a novel approach, the TLSTMF-YOLO model, which combines a C3-Swin-Transformer and multiscale feature fusion techniques to enhance detection accuracy and efficiency. Key innovations include the use of a convolutional block attention module (CBAM) to improve feature representation, and a bidirectional feature pyramid network (BiFPN) for optimized cross-scale feature fusion. To address data scarcity, a transfer learning strategy is applied, supported by an AdamW optimizer and cosine learning rate strategy for faster convergence. Evaluations on the Jiuzhaigou and Luding landslide datasets demonstrate the model’s effectiveness, achieving precision, recall, and mean average precision (mAP)@0.5 of 95.7%, 89.9%, and 90.5% on the Jiuzhaigou dataset, and 96.0%, 90.9%, and 94.5% on the Luding dataset, respectively. In addition, the model processes frames efficiently, with times of 6.61 and 12.2 ms on the two datasets. These results confirm the model’s capability for accurate and efficient landslide detection, highlighting its potential for real-world applications.

Binary segmentation is the main task underpinning several remote sensing applications, which are particularly interested in identifying and monitoring a specific category/object. Although extremely important, such a task has several challenges, including huge intra-class variance for the background and data imbalance. Furthermore, most works tackling this task partially or completely ignore one or both of these challenges and their developments. In this paper, we propose a novel method to perform imbalanced binary segmentation of remote sensing images based on deep networks, prototypes, and contrastive loss. The proposed approach allows the model to focus on learning the foreground class while alleviating the class imbalance problem by allowing it to concentrate on the most difficult background examples. The results demonstrate that the proposed method outperforms state-of-the-art techniques for imbalanced binary segmentation of remote sensing images while taking much less training time.

Remote sensing image captioning is a research domain that aims to automatically generate natural language descriptions of the contents within remote sensed images. Providing accurate depictions of image contents holds great significance for downstream applications such as image retrieval and image understanding. While there is a pressing need for reliable results, current research predominantly focuses on single captioning algorithms, striving to enhance their performance on specific target-oriented datasets. Undoubtedly, this research trajectory is highly important. However, we believe that relying solely on the output of a single captioner may introduce a vulnerability from a robustness standpoint. This concern is particularly relevant in remote sensing, where the scarcity of large-scale datasets can limit the robustness and reliability of resulting algorithms. In this paper, we propose an approach that harnesses the advantages of ensembles to enhance accuracy and reliability in the context of image captioning. Our method introduces a novel technique for utilizing an ensemble of diverse captioning algorithms and automatically selecting the most suitable caption from the set of predictions. By decoupling the description generation and selection phases, this approach enables high flexibility of integration of architecturally different captioning algorithms in the pipeline.

Texture analysis is a classical yet challenging task in computer vision for which deep neural networks are actively being applied. Most approaches are based on building feature aggregation modules around a pre-trained backbone and then fine-tuning the new architecture on specific texture recognition tasks. Here we propose a new method named R andom encoding of A ggregated D eep A ctivation M aps (RADAM) which extracts rich texture representations without ever changing the backbone. The technique consists of encoding the output at different depths of a pre-trained deep convolutional network using a Randomized Autoencoder (RAE). The RAE is trained locally to each image using a closed-form solution, and its decoder weights are used to compose a 1-dimensional texture representation that is fed into a linear SVM . This means that no fine-tuning or backpropagation is needed for the backbone. We explore RADAM on several texture benchmarks and achieve state-of-the-art results with different computational budgets. Our results suggest that pre-trained backbones may not require additional fine-tuning for texture recognition if their learned representations are better encoded.

Interactive segmentation plays an essential role in several tasks involving point clouds. However, existing methods suffer from low segmentation accuracy and cannot adjust the segmentation results according to the user’s personal demands. This paper presents a novel deep learning-based interactive segmentation method, named Click Rough Segmentation Network (CRSNet), designed to handle point clouds. The method allows users to iteratively click to segment interesting objects. CRSNet consists of two key parts: a CRS module and a feature extraction module. First, the CRS module transforms click operations into an appropriate representation to input into the feature extraction module. The CRS module takes raw point clouds and click operations as input and outputs 3D Gaussian vectors and roughly segmented blocks, which adapt to different-sized and densely-distributed objects in complex environments. Second, the feature extraction module, which uses a novel mix loss-based analysis algorithm, extracts deep features and obtains instance segmentation results. The module is highly compatible because its backbones can be replaced by different deep learning architectures. Experimental results on the KITTI, Apolloscape, Roadmarking, Scannet, and SemanticKITTI datasets show that our method outperforms state-of-the-art semantic segmentation methods with one click. Moreover, our method can generalize well to unseen objects and datasets.

Automatic building extraction from remote sensing imagery is crucial to urban construction and management. To address the main challenges of diverse building scale and appearance, this letter proposes an automatic building instance extraction method based on an improved hybrid task cascade (HTC). Our method consists of three components by obtaining high-resolution representation, defining guided anchor, and forming focal loss to boost the adaptability of automatic building instance extraction. Comprehensive experimental results on WHU aerial building data set demonstrated that compared with the mainstream Mask R-CNN method, our method increased AP and AR in bounding box branch and mask branch by 9.8%–6.5% and 10.7%–8.0% respectively, especially AP$_{S}$and AP$_{L}$in the two branches by 10.1%–6.9% and 3.4%–2.4%, respectively. We evaluated the effectiveness and complexity of these components separately and discussed the universality and practicability of deep learning method in automatic building extraction.

Robust road boundary extraction and completion play an important role in providing guidance to all road users and supporting high-definition (HD) maps. The significant challenges remain in remarkable and accurate road boundary recovery from poor road boundary conditions. This paper presents a novel deep learning framework, named BoundaryNet, to extract and complete road boundaries by using both mobile laser scanning (MLS) point clouds and high-resolution satellite imagery. First, road boundaries are extracted by conducting a curb-based extraction method. Such extracted 3D road boundary lines are used as inputs to feed into a U-shaped network for erroneous boundary denoising. Then, a convolutional neural network (CNN) model is proposed to complete the road boundaries. Next, to achieve more complete and accurate road boundaries, a conditional deep convolutional generative adversarial network (c-DCGAN) with the assistance of road centerlines extracted from satellite images is developed. Finally, according to the completed road boundaries, the inherent road geometries are calculated. The proposed methods were evaluated using satellite imagery and four MLS point cloud datasets with varying densities and road conditions in urban environments. The quality evaluation metrics of 82.88%, 82.43%, 88.86%, and 84.89% were achieved for four data sets. The experimental results indicate that the BoundaryNet model can provide a promising solution for road boundary completion and road geometry estimation.

3D vehicle detectors based on point clouds generally have higher detection performance than detectors based on multi-sensors. However, with the lack of texture information, point-based methods get many missing detection of occluded and distant vehicles, and false detection with high-confidence of similarly shaped objects, which is a potential threat to traffic safety. Therefore, in the long run, fusion-based methods have more potential. This paper presents a multi-level fusion network for 3D vehicle detection from point clouds and images. The fusion network includes three stages: data-level fusion of point clouds and images, feature-level fusion of voxel and Bird’s Eye View (BEV) in the point cloud branch, and feature-level fusion of point clouds and images. Besides, a novel coarse-fine detection header is proposed, which simulates the two-stage detectors, generating coarse proposals on the encoder, and refining them on the decoder. Extensive experiments show that the proposed network has better detection performance on occluded and distant vehicles, and reduces the false detection of similarly shaped objects, proving its superiority over some state-of-the-art detectors on the challenging KITTI benchmark. Ablation studies have also demonstrated the effectiveness of each designed module.

Deep learning (DL) methods, integrated with imagery obtained by remote sensors, are considered a novel source of information in the field of hydrometry. Their results can be a support for standard gauging systems. In this research, three different model definitions based on the SegNet architecture were developed: training a new model, using a pre-trained model, and performing transfer learning. The main goal of this approach is the performance assessment of convolutional neural network (CNN) generalization to segment images containing different water bodies automatically. Diverse sensors were used to obtain RGB images from different areas of the world. The effectiveness of the CNN was estimated using pixel accuracy and IoU metrics. Training a new model and using transfer learning revealed similar high accuracies that were at least twice as accurate compared to the pre-trained model. However, the transfer learned model is preferred due to significantly lower training expenses.

The potential of integrating deep learning and Google Earth Engine (GEE) is a few explored in the literature. Here, we investigated their potential in the context of Eucalyptus mapping in Brazilian Savanah. Based on GEE API using python language, it is possible to integrate it with Google Colab. The experiments were conducted using the U-Net semantic segmentation method. A total of 704, 88, and 88 patches were used for training, validation, and test, respectively. The overall accuracy obtained in the test dataset was 96.88%, while the Jaccard index was 0.84. These results demonstrated the applicability of these platforms for using deep learning techniques for mapping based on satellite images.

The integration of photogrammetry and deep learning methods can be powerful for Earth observation applications. Photogrammetry techniques allow the achievement of detailed geospatial products with em-level positional accuracy. Deep learning enables automatic image classification, segmentation, and object detection. For instance, when dealing with a large data set, photogrammetric processing steps, such as image orientation and dense point cloud generation, results in high computational costs. In contrast, deep learning methods are fast in the inference step. Here, we explore the complementarity of deep learning and photogrammetry, aiming to generate accurate and fast geospatial information. The main aim is to discuss the possibilities of using deep learning in the photogrammetric process. We conduct experiments to present the potential of the Mask R-CNN method trained on the COCO dataset to generate masks, essential to remove image observations from moving objects during the orientation (alignment) step.

Object detection and image segmentation are essential for environmental monitoring. This task can be performed and automated using machines with the processing capabilities of convolutional neural networks (CNN), achieving outstanding performance thanks to the current computing capacity and data available. Still, there are advances to perceive and questions on how to get the optimal performance and improve the results. With this aim, we assessed a state-of-the-art CNN, the Dynamic Dilated Convolution Neural Network (DDCN), to segment trees inside an urban environment. We chose DDCN because it exploits the paradigm of multi-context without increasing the number of trainable parameters of the network and defines, while training, the best patch size that should be used by the network in the test phase, helping to tune the CNN. With this technique, we achieved: pixel accuracy of 95.86%; average accuracy of 90.63%; F1-score of 90.93%; Kappa index of 81.87% and IoU of 72.78%. We are showing the capabilities of this CNN to segment complex images.

Brazilian eucalyptus plantations are largely used to produce wood products and play an important role in the country's economy. The presence of termites in these plantations can cause serious damage, requiring their identification and control. Here, we investigated the RetinaNet object detection deep learning method's potential to identify termite mounds in eucalyptus forests automatically. The experiments were conducted using terrestrial images acquired with a GoPro 360, composed of two fisheye cameras, providing a challenging task due to non-conventional geometry. In the tests, the model presented an Average Precision (AP) of 0.721, and after stipulating thresholds from which the objects would be used in training, the AP was 0.867.

Maintaining vegetation cover in cities is a key component to keep cities safe and resilient. The monitoring of trees is usually done with LiDAR data or multi and hyperspectral images. In this sense, remote sensing RGB images are presented as a cheaper and easier processing solution. Here, we proposed to evaluate deep learning-based methods combined with high-resolution RGB images to detect single-trees in the urban environment. Three state-of-the-art methods are tested: Faster-RCNN, RetinaNet, and ATSS. A total of 220 images were used, in which we manually labeled 3382 trees. For the proposal task, our findings show that ATSS is 3% more accurate than Faster-RCNN and 4% than RetinaNet. However, in a qualitative inspection, Faster-RCNN and RetinaNet seems to be better at this task. Our findings shows the need of further research for developing suitable tools for urban tree detection. This tools can help cities top achieve a more sustainable and resilient environment especially to face climate change.

Accurate road marking extraction and classification play a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Due to point density and intensity variations from mobile laser scanning (MLS) systems, most of the existing thresholding-based extraction methods and rule-based classification methods cannot deliver high efficiency and remarkable robustness. To address this, we propose a capsule-based deep learning framework for road marking extraction and classification from massive and unordered MLS point clouds. This framework mainly contains three modules. Module I is first implemented to segment road surfaces from 3D MLS point clouds, followed by an inverse distance weighting (IDW) interpolation method for 2D georeferenced image generation. Then, in Module II, a U-shaped capsule-based network is constructed to extract road markings based on the convolutional and deconvolutional capsule operations. Finally, a hybrid capsule-based network is developed to classify different types of road markings by using a revised dynamic routing algorithm and large-margin Softmax loss function. A road marking dataset containing both 3D point clouds and manually labeled reference data is built from three types of road scenes, including urban roads, highways, and underground garages. The proposed networks were accordingly evaluated by estimating robustness and efficiency using this dataset. Quantitative evaluations indicate the proposed extraction method can deliver 94.11% in precision, 90.52% in recall, and 92.43% in F1-score, respectively, while the classification network achieves an average of 3.42% misclassification rate in different road scenes.

Texture analysis has attracted increasing attention in computer vision due to its power in describing images and the physical properties of objects. Among the methods for texture analysis, complex network (CN)-based ones have emerged to model images because of their flexibility. In image modeling, each pixel is mapped to a vertex of the CN and two vertices are connected if they are spatially close in the image. Then measurements are extracted from the CN to characterize its topology and therefore characterize the image content. Despite the promising results, the accuracy of these methods depends on the suitability of the measurement for the application. In texture analysis, simple measurements have been used, such as those based on vertex degree and shortest paths. Motivated by these issues, this paper proposes a new method for texture analysis based on the CN and a new measurement that calculates the importance of each vertex within its neighborhood. For calculating the importance of vertices, we extend the pagerank to CN in order to correlate the vertex importance with its degree and show that this new measurement extracts texture properties. Experimental results on well-known datasets and in the recognition of soybean diseases using leaf texture show the effectiveness of the proposed method for texture recognition.

The Landsat 1-5 multispectral scanner system (MSS) collected records of land surface mainly during 1972-1992. Investigations on MSS have been relatively limited compared with the numerous investigations on its successors, such as Thematic Mapper (TM) and Enhanced TM Plus (ETM+). The benefits of the Landsat program are not fully accomplished without the inclusion of MSS archives. Investigations on the Landsat 1-5 MSS channel reflectance characteristics wereperformed followed by derived vegetation spectral indices and the Tasseled Cap (TC) transformed features mainly using a collection of synthesized records. On average, the Landsat 4 MSS is generally comparable to the Landsat 5 MSS. The Landsat 1-3 MSSs show disagreement in channel reflectance compared with the Landsat 5 MSS, especially for the red channel (600-700 nm) and the near-infrared channel (700-800 nm). Meanwhile, the relative differences for vegetation spectral indices of the Landsat 3 MSS are mainly from -16% to -5% with the median about -11.5%, while those of the Landsat 2 MSS are mainly from -15% to -7%. Cross-validation tests and two case applications suggested that between-sensor consistency was improved generally through the transformation models generated by ordinary least-squares regression. To improve the consistency of the vegetation indices and the TC greenness, direct strategy employing respective transformation models was more effective than calculations based on the transformed channel reflectance. Considering the shortages of the Landsat MSS archives, further efforts are needed to improve its comparability with observations by other successive Landsat sensors.

Transportation infrastructure needs constant maintenance. Pavement management systems requires reliable and detailed data of the current state of the roads to make effective decisions. Currently, pavement condition evaluation methods are mostly performed manually with visual inspection and interpretations in situ, which is labor intensive, time consuming and expensive. In this paper an experiment was conducted where a set of different configurations and parameters for Convolutional Neural Networks (CNNs) were applied to automatically detect potholes from images taken by an Unmanned Aerial Vehicle (UAV). Results showed that the pre-trained Faster-RCNN Inception ResNet model with reduced anchor box stride and image augmentation applied provides better accuracy compared to several other models tested, obtaining accuracy for this experiment of 70.4% across five-fold cross validation.

Unmanned aerial vehicles (UAVs) are platforms suitable for obtaining information utilizing sensors in a great variety of areas and enviroments, thus in this context, this paper objective was to identify trees in images collected using UAV high-resolution imagery, with the digital approach of superpixel segmentation and convolutional neural networks. A forest environment was analyzed in the form of an orthomosaic that was produced using 423 images. Superpixels were generated using the Simple Linear Iterative Clustering (SLIC) method, considering only two classes for the classification: trees and background. The generation of superpixels (segmentation) and classification were performed considering the configurations: 2000, 3000 and 4000 segments, sigma 5 and compactness 10 for SLIC and 100% transfer of learning. For the purposes of classification, the deep convolutional networks were adopted through ResNet-50 architecture, the weights of this network were previously trained in an Image bank and later the bank of images of superpixels underwent a fine-tuning. The experiments obtained a maximum classification accuracy of 89% with 3000 superpixels distinction between canopies and Background.

Pantanal is one of the most important biomes of the world, with a large number of wild animal species, some of them are in extinction. The automatic identification of wild animals is extremely important for the estimation of the species' population within Pantanal. However, digital processing techniques for the identification and tracking of species have faced great challenges due to clumsy light and pose conditions present in images taken in the wild. To overcome such problems, we propose a methodology that, by combining regular RGB images and thermal images, improves the identilication of species even in images taken in rough circumstances. We use the SLIC segmentation algorithm to identify the regions of the images where animals are present; after that, we apply convolutional neural networks to classify the identified regions according to eight possible animal species. We experiment on a real-world dataset composed of 1,600 images. Our results showed an average gain between 6% and 10% when compared to the method Fast R-CNN.

We introduce a method for shape recognition based on the angular analysis of Complex Networks. Our method models shapes as Complex Networks defining a more descriptive representation of the inner angularity of the shape's perimeter. The result is a set of measures that better describe shapes if compared to previous approaches that use only the vertices' degree. We extract the angle between the Complex Network edges, and then we analyze their distribution along with a network dynamic evolution. The proposed approach, named Angular Descriptors of Complex Networks (ADCN), presents a high discriminatory power, as evidenced by experiments conducted in five datasets. It is rotation invariant, presents high robustness against scale changes and degradation levels, overcoming traditional methods such as Zernike moments, Multiscale Fractal dimension, Fourier, Curvature and the degree-based descriptors of Complex Networks.

Face recognition has became one of the most popular application in computer vision, due to a large demand for security. In recent years, major advances have been reported in the face recognition and, therefore, the methods are becoming more accurate and efficient. Recent research pointed that feature extraction using activity in directed graphs showed excellent results in feature extraction. In this paper, we proposed a face recognition method based on activities of directed graphs in spatial pyramid. First, the image is represented by a graph, where each pixel is mapped into a vertex and then, the activity of each vertex is estimated. Finally, histograms of activities in spatial pyramid are calculated to characterize the face. According to the results, it was confirmed the efficiency of the proposed method in facial recognition using widely used datasets.

Much work has been done in the field of texture analysis and classification. While promising classification methods have been proposed, most of them rely on classical image analysis approaches. This paper presents a texture classification method based on diffusion in directed networks. First, an image is modeled as a directed network by mapping each pixel as a node and connecting two nodes up to a maximum distance r. To reveal texture properties, links between two nodes are removed based on the pixel intensity difference. Once such a network is obtained, the activity of each node is estimated by random walks and combined into a histogram to describe the image. The main contribution of this paper is the use of directed networks, which tends to provide better performance than in undirected cases. Also, we have shown that the activity induced on these networks can be effectively used as texture descriptor. Experimental results show that the proposed method is favorably compared to traditional texture methods on widely used texture datasets. The proposed method is also found to be promising for plant species classification using samples of leaf texture.