Deep learning methods, renowned for their ability to discern physical features from images, are frequently used in the semantic segmentation of remote sensing images. However, objects with different functional attributes may exhibit similar physical characteristics, resulting in comparable spectral reflectance and visual features. This issue, known as the “different categories with the same spectra” problem, limits the ability to discriminate between objects, thereby increasing the difficulty of differentiation. Studies based on Euclidean space often struggle to distinguish between objects due to the limited information available. To improve differentiation, additional information—such as neighborhood relationships—needs to be incorporated. The geographical scenario, i.e., the environmental context of the object—which includes crucial neighborhood categories and their spatial relationships, provides spatial relationship information for objects. This information provides an important context for object differentiation and becomes the key to distinguishing these similar objects. Following this idea, geographical scenarios are represented as graphs in graph space, with categories as nodes and adjacency relationships as edges, and a geographical knowledge graph is created based on all the scenarios. We convert the remote sensing images into graphs to match with the geographical scenarios and propose a knowledge-based semantic segmentation network for remote sensing, graph structure attention network (GSAN). In GSAN, a graph structure attention (GSAT) is designed based on the graph kernel. This allows it to discern graph structures corresponding to different geographical scenarios. GSAT serves as a link between the fine-grained visual objects and the coarse-grained semantic knowledge. Experiment results indicate that GSAN outperforms other attention networks in semantic segmentation on our sea and land remote sensing (SLRS) dataset. This demonstrates its advantages in geographical scenario recognition and remote sensing semantic segmentation.

The recognition of complex geographic scenes requires to simultaneously describe different kinds of relations such as hyponymy and spatial relations among multiple-scale categories. The former is hierarchical structure and the latter is called eigen structure. Current researches focus on representing these relations in Euclidean space. However, the key characteristic of the Euclidean spaces is that it expands polynomially with respect to the radius, which makes it difficult to describe both the hierarchical and eigen structure within finite computable dimensions simultaneously. To address the above issues, a Hyperbolic Embedding Model for Representing complex geographic scenes (HEMR) is first proposed. First, the visual features of remote sensing objects are extracted through a deep neural network in Euclidean space and mapped onto the n-dimensional Poincare ball in hyperbolic space by an exponential projection. Second, a two-stage contrastive learning mechanism based on Möbius transformation is designed, which uses the property of exponential expansion to obtain the ability to represent different kinds of relations. Finally, a new quaternion loss is designed to describe the relations between feature categories in remote sensing scenarios by hyperbolic distance. The experiments indicate that our model can leverage the characteristic of spatial exponential growth with the radius in hyperbolic space, simultaneously describing the hierarchical and eigen structures of complex remote sensing scenes in finite dimensions. This provides support for downstream tasks such as semantic segmentation of remote sensing, knowledge graph representation, and link prediction.

In the above article[1], the legends ofFigs. 7–9and11–20, andTables I,II, andIVshould have used the word “carrier” instead of “battleship.” Each figure is provided here with the corrected terminology.

Facial symmetry can be used as the prior knowledge to supplement shape information in face techniques such as shadow compensation or pose correction. In this paper, we demonstrate the application of the facial symmetry assumption in 3D face reconstruction. We horizontally mirror non-frontal illumination face images and align them to the original images using rigid transformation and optical flow based non-rigid registration. We observe that the resultant images can be used as reasonable estimates of the images of the face under new lighting conditions. We used them in the illumination cone based 3D face reconstruction of which the original method requires seven face images under different illuminations as input. We reveal that by utilizing the mirrored face images, the illumination cone based 3D face reconstruction can actually be performed by using no more than three differently illuminated face images. Experimental results on the YaleB database show that the reconstructed 3D face model is highly accurate. Fundamentally, the mirrored face images improve the support of the illumination cone which can be useful to not only 3D face reconstruction but also other subspace based face techniques such as face recognition.

In this paper, we present a new sparse to dense 3D face reconstruction method using monocular video sequences. Structure from motion (SFM) is an effective method to reconstruct sparse 3D facial shape; however, its performance degrades drastically when tracking errors caused by self-occlusion or image noise exist. To address the problem, we propose a reliable point selection method to automatically evaluate the reliability of corresponding points obtained by optical flow. The gray level cooccurrence matrix (GLCM) is applied to the texture-based correlation evaluation and those points whose correlation coefficients are lower than a threshold will be removed. Benefiting from the SFM's capacity of dealing with missing data, our method is more robust to tracking point correspondence errors and accordingly achieves a lower 3D reconstruction error compared with traditional SFM methods without correlation checking.

We describe a novel variational level set based method for delineating the sub thalamic nucleus (STN) region of human brains from magnetic resonance (MR) images. Based on the understanding of specific imaging characteristic of STN, we apply a narrow band limitation in the region-based energy function for localizing the STN from initial contour information provided by doctors. The validity of the algorithm was tested on practical Parkinson's Patients' T1-weighted magnetic resonance images. Comparing to traditional edge-based and global region-based level set methods, our method achieves more reliable segmentation results. Also, our method is insensitive to the initial contour placement, making it applicable in practical surgical navigations.