This paper studies the task of SatStreet-view synthesis, which aims to render photorealistic street-view panorama images and videos given a satellite image and specified camera positions or trajectories. Our approach involves learning a satellite image conditioned neural radiance field from paired images captured from both satellite and street viewpoints, which comes to be a challenging learning problem due to the sparse-view nature and the extremely large viewpoint changes between satellite and street-view images. We tackle the challenges based on a task-specific observation that street-view specific elements, including the sky and illumination effects, are only visible in street-view panoramas, and present a novel approach, Sat2Density++, to accomplish the goal of photo-realistic street-view panorama rendering by modeling these street-view specific elements in neural networks. In the experiments, our method is evaluated on both urban and suburban scene datasets, demonstrating that Sat2Density++ is capable of rendering photorealistic street-view panoramas that are consistent across multiple views and faithful to the satellite image.

This paper addresses metric 3D reconstruction of indoor scenes by exploiting their inherent geometric regularities with compact representations. Using planar 3D primitives -- a well-suited representation for man-made environments -- we introduce PLANA3R, a pose-free framework for metric $\underline{Plana}$r $\underline{3}$D $\underline{R}$econstruction from unposed two-view images. Our approach employs Vision Transformers to extract a set of sparse planar primitives, estimate relative camera poses, and supervise geometry learning via planar splatting, where gradients are propagated through high-resolution rendered depth and normal maps of primitives. Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision, enabling scalable training on large-scale stereo datasets using only depth and normal annotations. We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments across diverse tasks under metric evaluation protocols, including 3D surface reconstruction, depth estimation, and relative pose estimation. Furthermore, by formulating with planar 3D representation, our method emerges with the ability for accurate plane segmentation. The project page is available at: \url{https://lck666666.github.io/plana3r/}.

This paper investigates the effective utilization of unlabeled data for large-area cross-view gee-localization (CVGL), encompassing both unsupervised and semi-supervised settings. Common approaches to CVGL rely on ground-satellite image pairs and employ label-driven supervised training. However, the cost of collecting precise cross-view image pairs hinders the deployment of CVGL in real-life scenarios. Without the pairs, CVGL will be more challenging to handle the significant imaging and spatial gaps between ground and satellite images. To this end, we propose an unsupervised framework including a cross-view projection to guide the model for retrieving initial pseudo-labels and a fast re-ranking mechanism to refine the pseudo-labels by leveraging the fact that “the perfectly paired ground-satellite image is located in a unique and identical scene”. The framework exhibits competitive performance compared with supervised works on three open-source benchmarks. Our code and models will be released on https://github.com/liguopeng0923/UCVGL.

We present Multi-View Attentive Contextualization (MvACon), a simple yet effective method for improving 2D-to-3D feature lifting in query-based multi-view 3D (MV3D) object detection. Despite remarkable progress witnessed in the field of query-based MV3D object detection, prior art often suffers from either the lack of exploiting high-resolution 2D features in dense attention-based lifting, due to high computational costs, or from insufficiently dense grounding of 3D queries to multi-scale 2D features in sparse attention-based lifting. Our proposed MvACon hits the two birds with one stone using a representationally dense yet computationally sparse attentive feature contextualization scheme that is agnostic to specific 2D-to-3D feature lifting approaches. In experiments, the proposed MvA-Con is thoroughly tested on the nuScenes benchmark, using both the BEVFormer and its recent 3D deformable attention (DFA3D) variant, as well as the PETR, showing consistent detection performance improvement, especially in enhancing performance in location, orientation, and velocity prediction. It is also tested on the Waymo-mini benchmark using BEVFormer with similar improvement. We qualitatively and quantitatively show that global cluster-based contexts effectively encode dense scene-level contexts for MV3D object detection. The promising results of our proposed MvA-Con reinforces the adage in computer vision - “(contextualized) feature matters”.

This paper aims to develop an accurate 3D geometry representation of satellite images using satellite-ground image pairs. Our focus is on the challenging problem of 3D-aware ground-views synthesis from a satellite image. We draw inspiration from the density field representation used in volumetric neural rendering and propose a new approach, called Sat2Density. Our method utilizes the properties of ground-view panoramas for the sky and non-sky regions to learn faithful density fields of 3D scenes in a geometric perspective. Unlike other methods that require extra depth information during training, our Sat2Density can automatically learn accurate and faithful 3D geometry via density representation without depth supervision. This advancement significantly improves the ground-view panorama synthesis task. Additionally, our study provides a new geometric perspective to understand the relationship between satellite and ground-view images in 3D space.

Defocus blur detection (DBD) separates in-focus and out-of-focus regions in an image. Previous approaches mistakenly mistook homogeneous areas in focus for defocus blur regions, likely due to not considering the internal factors that cause defocus blur. Inspired by the law of depth, depth of field (DOF), and defocus, we propose an approach called D-DFFNet, which incorporates depth and DOF cues in an implicit manner. This allows the model to understand the defocus phenomenon in a more natural way. Our method proposes a depth feature distillation strategy to obtain depth knowledge from a pre-trained monocular depth estimation model and uses a DOF-edge loss to understand the relationship between DOF and depth. Our approach outperforms state-of-the-art methods on public benchmarks and a newly collected large benchmark dataset, EBD. Source codes and EBD dataset are available at: github.com/yuxinjin-whu/D-DFFNet.

Cloud and cloud shadow segmentation is one of the most important problems in remote sensing image processing. Due to the vulnerability to ground object interference, noise interference and other factors, and the lack of generalization ability, traditional deep learning network would inevitably lose details and spatial information, resulting in imprecise segmentation of cloud and cloud shadow boundaries, missing detection and false detection. In order to solve these problems, a multi-scale attention feature aggregation network is proposed, which extracts semantic information at different levels based on residual network. A multi-scale strip pooling attention module is designed to extract multi-scale context information and deep spatial channel information. In order to improve the feature extraction of the model, the deep multi-head feedforward transfer attention module is introduced to enable the adjacent two layers of the backbone network to guide each other for feature mining. Then, a bilateral feature fusion module is used to guide the fusion of low-level semantic information and high-level detail information. Finally, in order to enhance the repair of boundary details of cloud and cloud shadow, a boundary refinement boosting model is designed. The experimental results show that our model can handle complex cloud cover scenes, and has excellent performance on cloud, cloud shadow dataset and CSWV dataset. Its segmentation accuracy is superior to the existing methods, which is of great significance to the related work of cloud detection.

Traditional building and water segmentation methods are vulnerable to noise interference, and hence they could not avoid missed and false detections in the detection process. Excessive deep learning downsampling would lead to significant loss of feature map information, and image location information offset, and the overall effect of falling apart. To address these issues, a Multi-Scale Location Attention Network (MSLA) is proposed. Location-spatial information and channel information are particularly important for edge detail segmentation in building and water cover. The network includes a Location Channel Attention Unit (LCA) to focus on tributary details of rivers and segmentation of building edge eaves. Moreover, this paper builds a Dual-Branch Multi-Scale Aggregation Unit (DBMSA) to obtain deeper multi-scale semantic information. Finally, the Multi-Scale Fusion Unit (MSF) is used to guide the information merging of multiple stages, and the boundary information is improved by splicing the acquired deep multi-scale information with the information of the relevant feature extraction layer in the downsampling. The experimental results on several datasets show that the proposed approach outperforms other methodologies in segmentation accuracy.

Cloud and cloud shadow segmentation is one of the most important issues in remote sensing image processing. Most of the remote sensing images are very complicated. In this work, a dual-branch model composed of Transformer and convolution network is proposed to extract semantic and spatial detail information of the image respectively to solve the problems of false detection and missed detection. To improve the model’s feature extraction, a Mutual Guidance Module is introduced so that the Transformer Branch and the Convolution Branch can guide each other for feature mining. Finally, in view of the problem of rough segmentation boundary, this work uses different features extracted by the Transformer Branch and the Convolution Branch for decoding, and repairs the rough segmentation boundary in the decoding part to make the segmentation boundary clearer. Experimental results on the Landsat-8, Sentinel-2 data, the public dataset HRC_WHU of Wuhan University and the public dataset SPARCS demonstrate the effectiveness of our method and its superiority to existing state-of-the-art Cloud and Cloud Shadow Segmentation approaches.

Wireless capsule endoscopy has opened a new era by enabling remote diagnostic assessment of the gastrointestinal tract in a painless procedure. Video capsule endoscopy is currently commercially available worldwide. However, it is limited to visualization of superficial tissue. Ultrasound (US) imaging is a complementary solution as it is capable of acquiring transmural information from the tissue wall. This paper presents a mechanical scanning device incorporating a high-frequency transducer specifically as a proof of concept for US capsule endoscopy (USCE), providing information that may usefully assist future research. A rotary solenoid-coil-based motor was employed to rotate the US transducer with sectional electronic control. A set of gears was used to convert the sectional rotation to circular rotation. A single-element focused US transducer with 39-MHz center frequency was used for high-resolution US imaging, connected to an imaging platform for pulse generation and image processing. Key parameters of US imaging for USCE applications were evaluated. Wire phantom imaging and tissue phantom imaging have been conducted to evaluate the performance of the proposed method. A porcine small intestine specimen was also used for imaging evaluation in vitro. Test results demonstrate that the proposed device and rotation mechanism are able to offer good image resolution ( [Formula: see text]) of the lumen wall, and they, therefore, offer a viable basis for the fabrication of a USCE device.

Spatial reuse protocol (SRP) is a media access control (MAC)-layer protocol that operates over a double counter-rotating ring network topology. SRP is designed to enhance the SONET network so that it can handle data traffic more efficiently. We study the all-terminal reliability of an SRP ring implementing intelligent protection switch (IPS) and enhanced intelligent protection switch (E-IPS). We calculate reliability for all scenarios under which all-terminal connectivity is maintained. By summing up the results arising from individual mutually exclusive events, all-terminal reliabilities for IPS and E-IPS are obtained. By further assuming constant and time-dependent component failure rates, mean time to failure (MTTF) and mean time to failure improvement factor (MTIF) measures are calculated. Results show that MTTF/MTIF vary as functions of network size, component reliability and coverage factor. The advantage of E-IPS over IPS is more obvious when network size is big and concentrator failure rate is small. Our analysis could be helpful to network designers who wish to determine whether the additional cost/complexity added by the E-IPS scheme is justified.