Multi-view stereo (MVS) implicitly encodes photometric and geometric cues into the cost volume for multi-view correspondence matching, transferring insufficient geometric cues essential to depth estimation and reconstruction. This paper proposes GE-MVS, a novel multi-view stereo network with geometric encoding for more accurate and complete depth estimation and point cloud reconstruction. First, the cross-view adaptive cost volume aggregation module is proposed to strengthen multi-view geometric cues encoding during cost volume construction. Then, the depth consistency optimization is performed in the 3D point space during learning by invoking ground-truth depth cues from adjacent views. Finally, the surface normal geometries are explicitly encoded to refine the sampled depth hypotheses to be consistent in the local neighbor regions. Extensive experiments on the standard MVS benchmarks including DTU, Tanks and Temples, and BlendedMVS demonstrate the state-of-the-art depth estimation and point cloud reconstruction performance of GE-MVS. The GE-MVS is further deployed in real-world experiments for UAV-based large-scale reconstruction, where our method outperforms the prevalent industrial reconstruction solutions concerning reconstruction efficiency and efficacy. Our project page is: https://cuhk-usr-group.github.io/GE-MVS/

Recent advancements in learning-based multi-view stereo (MVS) have demonstrated significant improvements over traditional counterpart, primarily due to the extensive availability of multi-view training images with ground-truth metric depths in the terrestrial in-air domain. However, underwater multi-view stereo (UwMVS) faces substantial challenges arising from the domain gap between in-air and underwater environments, leading to degraded performance when applying in-air MVS models to underwater scenarios. Furthermore, the progress of learning-based UwMVS methods has been hindered by the scarcity of underwater multi-view images with ground-truth depth maps and point clouds. In this paper, we address these challenges by introducing a physically-guided approach for synthesizing underwater multi-view images and present the first large-scale UwMVS dataset for end-to-end training and evaluation of learning-based UwMVS methods. Furthermore, we propose a novel UwMVS network that enhances geometric cue encoding to achieve more accurate and complete point cloud reconstruction. Extensive experiments on our dataset and real-world underwater scenes demonstrate that our dataset enables the trained models for underwater dense reconstruction and that our method achieves state-of-the-art performance in underwater reconstruction. Dataset, code and appendix are available at: https://cuhk-usr-group.github.io/UwMVS/

Defect diagnosis in urban infrastructure is crucial for public safety. Traditional manual inspections face significant challenges in terms of accuracy and cost-effectiveness. In this paper, we propose a lightweight and hardware-friendly large-scale infrastructure detector, CUPID, highly suitable for unmanned aerial vehicles (UAVs). Given the significant challenges in automatically detecting defects of varying intensity and size within complex infrastructure, along with the tendency of lightweight models to lose detail and fail to fully capture features during the defect extraction process, we propose the CUPID_Block, a multi-level information fusion block to construct the backbone, featuring the CUPID_Conv module equipped with our proposed CCA (CrissCross Attention). Furthermore, CUPID features an auxiliary training branch that assimilates lower feature maps, helping to recover details lost in deeper convolutional layers. To verify the effectiveness of CUPID and to address the lack of a suitable dataset in the community, we establish a multi-scenario infrastructure defect dataset, CUBIT2024, to conduct extensive experiments. Finally, to assess the efficiency and adaptability of CUPID in UAV for online infrastructure inspection, we design a compact autonomous drone, CU-Astro, where the proposed CUPID is deployed on the Jetson Orin NX computer onboard to evaluate the speed and power consumption of the inference.

Accurately predicting ocean subsurface temperature is vital for advancing ocean and climate research, particularly given the sparse and costly nature of subsurface observations. This study introduces sparse-to-dense prediction of ocean subsurface temperature using multi-level spatiotemporal (ST) information fusion. The framework integrates interpretable ST decoupling, adaptive feature updating, and sparse-to-dense information fusion modules to address the challenge of sparse observations and ever-evolving dynamic environments. Comprehensive experiments focused on the Pacific demonstrate the superiority of the proposed methodology over peer methods. The proposed methodology achieves high-resolution predictions with a root mean square error of 0.2230, accuracy of 0.9846, and point-wise prediction errors below 0.5°C under 10% online random sparse observations (ORSO). Analyses of spatial and temporal temperature dynamics reveal long-term warming trends in the Pacific, including a temperature rise of up to 2.8°C at -100 m in low-latitude regions over the past 40 years, and identify the latitudinal slope of thermocline dynamics. This study advances the understanding of multi-scale thermal processes and variability in the Pacific, demonstrating the potential of application in climate studies, marine resource management, and environmental monitoring.

Underwater optical remote sensing is crucial for geoscience applications but often suffers from image degradation due to complex underwater environments. While learning-based methods have advanced underwater image enhancement (UIE), their efficacy in real-world UIE applications still faces challenges. This limitation arises from training predominantly on synthetic underwater images, resulting in a significantinter-domain gap when applied to real-world data. Additionally, diverse underwater conditions introduceintra-domain challenges, such as color casts and haze, further complicating the UIE process. To address these issues, we propose SSD-UIE, a semi-supervised domain-adaptive framework designed to mitigate bothinter- andintra-domain gaps. Our approach employs a systematic synthesis pipeline to reduce visualinter-domain discrepancies and introduces a Large Synthetic-Real Underwater Image Dataset (LSRUID) to facilitate the training of the framework. The Semantic-Blender is developed to handle semanticinter-domain differences, while the Intra-domain-aware Feature Extraction (IFE) branch and feature alignment strategy effectively addressintra-domain variability. Furthermore, the Dual-Trans Block is introduced to enhance the UIE performance while maintaining computational efficiency. Extensive experiments demonstrate that SSD-UIE outperforms state-of-the-art (SOTA) UIE methods in both qualitative and quantitative evaluations on real-world underwater images. Codes and dataset will be publicly available at https://github.com/RockWenJJ/SSD-UIE.git.

Multi-view stereo (MVS) enables accurate and complete 3D reconstruction from multi-view imagery, serving as a core methodology in remote sensing applications across terrestrial and underwater domains. Recent advancements in learning-based MVS have demonstrated significant improvements over traditional counterparts, primarily due to the extensive availability of multi-view training images with ground-truth metric depths in the terrestrial in-air domain. However, underwater multi-view stereo (UwMVS) faces substantial challenges arising from the domain gap between in-air and underwater environments, leading to degraded performance when applying in-air MVS models to underwater scenarios. Furthermore, the progress of learning-based UwMVS methods has been hindered by the scarcity of underwater multi-view images with ground-truth depth maps and point clouds. In this paper, we address these challenges by introducing a physically-guided approach for synthesizing underwater multi-view images and presenting the first large-scale synthetic UwMVS dataset preserving real-world underwater degradation properties for end-to-end training and evaluation of learning-based UwMVS methods. Furthermore, we propose a novel UwMVS network that enhances geometric cue encoding to achieve more accurate and complete point cloud reconstruction. Extensive experiments on the dataset and real-world underwater scenes demonstrate that our dataset enables the trained models for underwater dense reconstruction and that our method achieves state-of-the-art performance in underwater reconstruction. Dataset, appendix, and supplementary video are available at https://yang-sober.github.io/UnderMVS/.

This paper proposes SANet, a small but accurate detector for aerial flying objects. The detector introduces an attention module into the feature extraction module (FEM) for enhancing the accuracy. This FEM with fewer convolutional kernel channels can reduce the parameters, speed up the inference time, and mitigate the computational burden. Furthermore, we optimize the Spatial Pyramid Pooling (SPP) module to enhance both the accuracy and speed. By analyzing the structure characteristic of the ResNet and RepVGG network that are usually utilized to extract features, a feature fusion module named RepNeck is designed to comprehensively fuse features extracted by the FEM, further enhancing the speed and accuracy. Eventually, we develop a neural network with an impressively small model size of only 4.5M. This network can achieve the state-of-the-art performance on three challenging datasets. Apart from its superior performance, our approach enjoys a real-time detection speed of 14.8 frames per second (fps) and power consumption of only 2.9W while the CPU and GPU temperatures are maintained below 50◦C even on an edge-computing device, highlighting the practicality of our approach for long-duration flying object detection and monitoring tasks.

Visual inspection plays a predominant role in inspecting infrastructure surface. However, the generalization of existing visual inspection systems to large-scale real-world scenes remains challenging. In this paper, we introduce Det-Recon-Reg, an intelligent framework separating the complex inspection procedure into three stages: Detect, Reconstruct, and Register. (1) For defect detection (Detect), we present the first high-resolution defect dataset tailored for large-scale defect detection. Based on the dataset, we evaluate the most effective real-time object detection algorithms and push the boundary by proposing CUBIT-Net for real-world defect inspection. (2) For infrastructure reconstruction (Reconstruct), we propose a learning-based multi-view stereo (MVS) network to adapt to large-scale scenes, taking as input the multi-view images and outputting the point cloud reconstruction, where its performance has been validated on the standard MVS datasets, including BlendedMVS, DTU, and Tanks and Temples datasets. (3) For defect localization (Register), we propose an effective registration method based on the geographic information system that registers the detected defects onto the reconstructed infrastructure model to establish a global reference for maintenance measures. The real-world experiments further verify the effectiveness and efficiency of our proposed framework. More details about our proposed dataset, code, and appendix are available on our project page: https://cuhk-usr-group.github.io/large-scale-inspect-framework/.

The misuse of drones can jeopardize public safety and privacy. The detection and catching of intruding drones are crucial and urgent issues to be investigated. This work proposes VDTNet, an accurate, lightweight, and fast network for visually detecting and tracking intruding drones. We first incorporate an SPP module into the first head of YOLOv4 to enhance detection accuracy. Model compression is utilized to shrink the model size and concurrently speed up inference. We then propose and insert an SPPS module and a ResNeck module into the neck, and introduce an effective attention module for the backbone to compensate for the accuracy drop brought on by compression. With the above strategies, we present the accurate and compact VDTNet with a model size of merely 3.9 MB, ensuring low computational cost and fast detection and tracking performance in real time. Extensive experiments on four challenging public datasets show that our proposed network outperforms state-of-the-art approaches. In real-world scenarios, the comparative ground-to-air detection testing proves the generalization ability of the VDTNet, and we further demonstrate the portability and practicability of the network by deploying it on drone onboard edge-computing devices for air-to-air real-time detection of the intruding drones.

Beyond programming robots to accomplish a single high-level task at a time, people also hope robots follow instructions and complete a series of tasks while meeting their requirements. This paper presents an interactive software system that consists of a multiple-task linear temporal logic (LTL) path planner and a human-machine interface (HMI). The HMI transforms human oral instructions into task commands that can be understood by the machine. The planner grows a rapid random exploring tree to search for solutions for multiple tasks. When switching tasks, the search tree is re-initialized and reconnected to utilize the information gathered during the exploration of the workspace. The feasibility of the improved planner is theoretically guaranteed, and profiling in simulation shows an acceleration in planning. An experiment with a quadcopter is conducted to show that the combination of the multiple-task LTL planner and the HMI results in a synergistic effect in real-world applications.

In this paper, we present LCM-MVSNet, a novel multi-view stereo (MVS) network with learnable cost metric (LCM) for more accurate and complete depth estimation and dense point cloud reconstruction. To adapt to the scene variation and improve the reconstruction quality in non-Lambertian low-textured scenes, we propose LCM to adaptively aggregate multi-view matching similarity into the 3D cost volume by leveraging sparse points hints. The proposed LCM benefits the MVS approaches in four folds, including depth estimation enhancement, reconstruction quality improvement, memory footprint reduction, and computational burden alleviation, allowing the depth inference for high-resolution images to achieve more accurate and complete reconstruction. Moreover, we improve the depth estimation by enhancing the propagation of shallow features via a bottom-up path and strengthen the end-to-end supervision by adapting the focal loss to reduce ambiguity caused by sample imbalance. Extensive experiments on two benchmark datasets show that our network achieves state-of-the-art performance on the DTU dataset and exhibits strong generalization ability with a competitive performance on the Tanks and Temples benchmark. Furthermore, we deploy our LCM-MVSNet into the real-world application for large-scale 3D reconstruction based on multi-view aerial images collected by self-developed UAV, demonstrating the robustness and scalability of our method. More detailed results are available in the Appendix11shorturl.at/rBG28

We propose a lightweight, effective, and efficient anti-drone network, namely ADMNet, for visually detecting and monitoring unfriendly drones with a constrained view field, flying against a complex environment. We merge an SPP module to the first head of YOLOv4 to improve accuracy and perform network compression to reduce inference latency and model size. To compensate for the accuracy loss caused by condensation, we propose an SPPS module and a ResNeck module for the neck of the network and implement an effective attention module for the backbone. Eventually, we present an accurate and compact ADMNet with barely 3.9 MB, ensuring low computational cost and real-time detection. Our method achieves state-of-the-art performance on three challenging real-world datasets (Average Precision @0.5IoU): Det-Fly 96.2%, NPS-Drones 92.0%, and TIBNet 89.7%. The throughput is higher than the prior work, in addition to its superior performance. The comparative testing in real-world scenarios proves that our method exhibits strong reliability and generalization ability. Deploying the network on drone onboard edge-computing devices enables real-time detection and monitoring of flying drones, highlighting the portability and viability of the ADMNet.