Electroencephalography (EEG) has emerged as a widely utilized signal in motor imagery (MI) brain-computer interfaces(BCI) due to its convenience and safety. Recently, deep learning methods have rapidly developed in the field of brain computer interfaces. However, traditional EEG classification methods often face challenges related to limited generalization capability across subjects. To address this issue, this paper proposes a multi-source discriminant dynamic domain adaptation model(MSD-DDA) aimed at fully leveraging domain adaptation to enhance the accuracy of motor imagery classification. The model adeptly handles global and local disparities in motor imagery classification by dynamically minimizing differences between global domain and local subdomain. Furthermore, to ensure discriminability and diversity in the target domain, we introduce batch kernel norm maximization of the difference, thereby enhancing the model's discriminability in the target domain while preserving prediction diversity. To tackle variations in similarity between different source domains and the target domain, we devise a weighted joint prediction mechanism. This mechanism automatically adjusts the contribution weight of each source domain based on its similarity to the target domain, facilitating more precise discriminant prediction and improved adaptability to scenarios with multiple source domains. To evaluate our approach, we conducted a large number of experiments on datasets 1 and 2a of the Fourth BCI Competition and on the openBMI dataset, with average classification accuracy of 92.43%, 79.24% and 71.96%, respectively.Finally, we compare the proposed method with several classical and recent algorithms, and prove that its performance is better than the existing methods.

Weakly-supervised temporal action localization aims to identify action instances using only video-level labels, and localize the action position in untrimmed videos. Due to the temporal continuity of video data, most methods that use single scale convolution kernel cannot model against the characterization of video data effectively, and lead to a decrease in accuracy. However, simply using multiscale features can introduce redundant information and noise, reducing model efficiency while also affecting the accurate judgement of the model during training process. To alleviate this problem, a video complicated-information extraction and filtering network (VCEF-Net) is proposed. It contains two main modules. The first multi-scale feature extraction module is developed to enrich the information that model received. The second pseudo-label filtering module inhibits redundant information interference. VCEF-Net introduces these two modules for achieving a better utilization of video information. Experiments tested on THUMOS14 and ActivityNet1.2 demonstrate better performances of the proposed VCEF-Net and validate its effectiveness. Code is available at https: // github. com/ JXL119/ VCEF

As spatial resolution increases in remote-sensing imagery, the challenge of semantic segmentation intensifies due to the need to discern intricate changes in terrain. Terrain, a composite of diverse geographic elements arranged in specific spatial patterns, demands a higher level of abstraction in semantic categorization. Achieving accurate semantic segmentation in high-resolution remote-sensing images necessitates a profound understanding of the semantic structures within complex scenes. In response to this imperative, this article introduces the object Markov random field with hierarchical semantics (OMRF-HSs) method. The primary contributions of this work are twofold: 1) effective representation of layered semantic information within images is achieved, enhancing the understanding of complex scenes and 2) unified under an object Markov random field (MRF) model, the method enables the joint modeling of structured semantics and spatial context information, facilitating more robust segmentation outcomes. Experimental evaluations conducted on multiscene high-resolution remote-sensing images from the aerial sensor, SPOT5, GeoEye, and Gaofen-2 satellites demonstrate that the proposed method outperforms state-of-the-art techniques, yielding superior segmentation accuracy. The availability of code and example data further facilitates the reproducibility and adoption of the OMRF-HS method, accessible athttps://github.com/FHY-146/OMRF-HS.

Remote sensing images have a large imaging field, covering a wide range of land surfaces and encompassing complex geographical scenes. More diverse land cover types and detailed structural information within geographical scenes can be observed with improved image spatial resolution. However, due to the high cost of annotating fine semantic scales, only a few semantic segmentation methods can fully utilize multi-level semantic information. This paper introduces a novel multi-semantic Markov Random Field (MRF-MS) model for semantic segmentation of high-resolution remote sensing images. This method extends the classical object-level Markov Random Field model by introducing a sub-label field to capture semantic hierarchical information in high-resolution images. Assuming the initial categories follow a mixture of Gaussian distribution, this method models the land cover types or structural details of geographical objects within the scene by decomposing the mixture of Gaussian distribution into sub-components. Furthermore, it establishes an interactive contextual structure between the sub-label and initial label fields to model the interaction across different semantic scales. Experimental results on multi-scene high-resolution remote sensing images demonstrate that this approach yields more accurate classification results compared to the latest MRF methods.

Crop identification is a fundamental task in remote sensing image interpretation. The rapid development of Unmanned Aerial Vehicle (UAV) has revolutionized the acquisition of super-high-resolution images. Compared with remote sensing ones, the fact that UAV images are easier to be flexibly acquired and contain more information brings opportunities for refined semantic segmentation. Recently deep learning methods have gained substantial popularity in the field of semantic segmentation. However, the practical application of deep learning methods is often hindered by heavy labeling tasks and computational resources. The object-based Markov random field (OMRF) offers a both time and labor cost-effective unsupervised approach. Nevertheless, the high-spatial heterogeneity exhibited by crops in UAV images brings great difficulties to the application of this method. To address these challenges, this letter introduces RA-OMRF model, an unsupervised approach that improves OMRF model through our newly proposed pre-processing step named Region Aid (RA). RA is used to increase the amount of data for categories with fewer samples to alleviate the problem of high-spatial heterogeneity of ground object categories, thereby improving the performance of the OMRF model. Compared to five deep learning models, our method achieves matching or even higher segmentation accuracy (for instance an overall accuracy of 97.43% on the UAV Dataset DWST) in crop identification tasks of UAV images, while reducing time and labor costs.

Plant stomatal phenotype traits play an important role in improving crop water use efficiency, stress resistance and yield. However, at present, the acquisition of phenotype traits mainly relies on manual measurement, which is time-consuming and laborious. In order to obtain high-throughput stomatal phenotype traits, we proposed a real-time recognition network SLPA-Net for stomata localization and phenotypic analysis. After locating and identifying stomatal density data, ellipse fitting is used to automatically obtain phenotype data such as apertures. Aiming at the problems of small stomata and high similarity to background, we introduced ECANet to improve the accuracy of stoma and aperture location. In order to effectively alleviate the unbalance problem in bounding box regression, we replaced the Loss function with a more effective Focal EIoU Loss. The experimental results show that SLPA-Net has excellent performance in the migration generalization and robustness of stomata and apertures detection and identification, as well as the correlation between stomata phenotype data obtained and artificial data.

Semantic segmentation is one of the most important tasks in the field of remote sensing. As the spatial resolution increases, the remote sensing images can capture more detailed information and make hierarchical semantic interpretation possible. However, hierarchical semantic segmentation encounters high heterogeneity not only within the intra-layer classes but also among inter-layer classes. It brings challenges to semantic segmentation methods such as the convolutional neural network (CNN). In this article, a hierarchical self-learning knowledge inference model (HSKIM) based on the Markov random field (MRF) model is proposed for hierarchical semantic segmentation of remote sensing images. The HSKIM model introduces a new framework that integrates the advantages of CNN-based data feature learning and MRF-based hierarchical semantic inference. It contains three modules: data learning module ($\boldsymbol {D}$), inference units generation module ($\boldsymbol {I}$), and self-learning knowledge inference module ($\boldsymbol {S}$). The module$\boldsymbol {D}$uses CNN to learn specific data features layer by layer and extract preliminary geographical objects as the initial results. The module I refines the geographical objects using a novel boundary-preservation trick to generate more accurate inference units with clear geographical meaning. The module S introduces a hierarchical object-based MRF model to implement semantic inference among intra-layer and inter-layer inference units, guided by the spatial interactions and geographical criteria. This module can self-learn and update the relationship between classes iteratively and provide the final result. Experiments on the GID dataset with hierarchical classes, alongside 12 state-of-the-art CNN-based methods, validate the effectiveness and robustness of the proposed HSKIM model. The code of this article is available athttps://github.com/iichengzi/HSKIM.

Convolutional neural network (CNN) has been widely used in semantic segmentation for remote sensing images, and it has achieved great success. Due to the diversity of the spatial distribution of terrestrial objects in remote sensing images, it is difficult to effectively learn general geographical laws and apply them to a specific image. To introduce geographical knowledge into the CNN model more effectively, a self-learning-update CNN model (SLU-CNN) is proposed in this letter. It learns the representation of specific spatial dependence among different objects according to the CNN result, and then incorporates it with the CNN result to make semantic inference available. The proposed method mainly involves two modules. First, geographical objects generated from the CNN result are used as inference units. Second, the spatial dependence between inference units is learned to build a specific adaptive geographical relationship. And then, it is embedded as an adaptive penalty term into an object-based Markov random field model to achieve the collaboration between the CNN result and the semantic inference. Our method provides a general data-knowledge dual-driven framework for the deep neural network. Experiments of the GID and Sentinel-2 datasets validate the effectiveness of the proposed method by comparing it with different state-of-the-art CNN methods.

Deep learning methods have been widely studied in the semantic segmentation field of the remote sensing image. Training images play an important role in these methods; however, each training image usually contains not only the generalization information of each land category but also the specific interclass context between different categories. The specific interclass context prevents deep learning methods from focusing on generalization information learning during training and limits the performance on different data distributions. This article proposes a generalization sampling learning method of deep convolutional neural network (GSL-CNN) to emphasize generalization information learning for the semantic segmentation of remote sensing images. The proposed method develops a new CBR sampling strategy that contains three modules: category grouping ($\mathbf {\boldsymbol {C}}$), basic unit extraction ($\mathbf {\boldsymbol {B}}$), and random combination ($\mathbf {\boldsymbol {R}}$). Module$\mathbf {\boldsymbol {C}}$collects each land category map and strips away the specific interclass context from the raw annotated image. Module$\mathbf {\boldsymbol {B}}$extracts basic units with different granularities from each land category map, and each basic unit can keep the generalization information of this category. Module$\mathbf {\boldsymbol {R}}$aims to enhance the robustness against different data distributions by randomly picking basic units of different categories and randomly generating their interclass context. The new GSL-CNN method integrates the CBR sampling strategy with the convolutional neural network (CNN) model for semantic segmentation. Experiments on different remote sensing datasets and 15 state-of-the-art CNN models validated that the proposed method has the potential of improving the generalization ability of the CNN method from a sampling perspective.

Semantic segmentation is one of the most important tasks in the field of remote sensing image processing. Many methods have been proposed to realize it at the pixel granularity or object granularity. Specifically, the pixel-based methods usually can effectively extract the detailed information and edges, and the object-based methods can keep the internal consistency of each land cover or land use. The Markov random field (MRF) model provides a statistical way to combine the advantages of both pixel and object granularities together. However, current MRF-based methods still face a problem, that is, how to ensure that the advantages of different granularities will complement each other, not that disadvantages will affect advantages. To solve this problem, a new multigranularity edge-preservation optimization is proposed in this letter. The proposed method first represents the image with a series of granularities from the object to the pixel by downsampling. Then, the MRF model is defined on each granularity. By defining an edge set for each granularity, during the process of downsampling, the proposed method can continuously correct edges while maintaining intraclass consistency. Experiments of Gaofen-2 and SPOT5 demonstrate the effectiveness of the proposed method. Moreover, the proposed method can be also used as the postprocessing step for deep learning. The experiment of the Pavia University hyperspectral image illustrates it for an instance of DeepLab v3+.

Loss function is an important topic in the field of face recognition, while the margin-based loss function is one of the most useful methods to enhance discriminability. Recently, the method of dividing the samples into easy and hard ones effectively optimizes the margin-based loss function by emphasizing these two parts at different training stages. However, the hard samples contain not only intra-personal variations that can be easily misclassified but also wrong-labeled samples. To capture and distinguish their difference, an EnhanceFace method is proposed with a new adaptive weighted loss function in this letter to improve the accuracy of face recognition. Firstly, this method partitions the hard samples into semi-hard and harder samples. Then, a new loss function is developed to adaptively adjust corresponding weights for both semi-hard samples and harder samples during the training process. With this new loss function, their effects can be well balanced according to their importance. The EnhanceFace method is tested on different popular benchmarks. Finally, better performance is shown by the experimental results comparing with other state-of-the-art methods.

Semantic segmentation is one of the most important tasks in remote sensing. However, as spatial resolution increases, distinguishing the homogeneity of each land class and the heterogeneity between different land classes are challenging. The Markov random field model (MRF) is a widely used method for semantic segmentation due to its effective spatial context description. To improve segmentation accuracy, some MRF-based methods extract more image information by constructing the probability graph with pixel or object granularity units, and some other methods interpret the image from different semantic perspectives by building multilayer semantic classes. However, these MRF-based methods fail to capture the relationship between different granularity features extracted from the image and hierarchical semantic classes that need to be interpreted. In this article, a new MRF-based method is proposed to incorporate the multigranularity information and the multilayer semantic classes together for semantic segmentation of remote sensing images. The proposed method develops a framework that builds a hybrid probability graph on both pixel and object granularities and defines a multiclass-layer label field with hierarchical semantic over the hybrid probability graph. A generative alternating granularity inference is suggested to provide the result by iteratively passing and updating information between different granularities and hierarchical semantics. The proposed method is tested on texture images, different remote sensing images obtained by the SPOT5, Gaofen-2, GeoEye, and aerial sensors, and Pavia University hyperspectral image. Experiments demonstrate that the proposed method shows a better segmentation performance than other state-of-the-art methods.

Markov random field model (MRF) is one widely used method in the field of segmentation of remote sensing images. The basic unit of the MRF model can be either the pixel or the over-segmented region, known as the object. The pixel-level and the object-level MRF models are often studied independently. But, advantages of these two types of MRF models are complementary. In order to combine their advantages, a new MRF model with alternating granularities (MRF-AG) is proposed in this paper. The MRF-AG model first uses two granular units, the pixel and the object, to build the probability graphs of the MRF model, and defines feature fields and label fields on these graphs to describe the feature patterns and the corresponding class labels. Then, a new probability inference is developed to integrate the pixel-level and the object-level information by alternately using one granular label result as the prior information of the other. With iterations, the convergence label result is achieved as the final segmentation result. Experiments on different remote sensing images demonstrate that the proposed method can provide a better performance than other state-of-the-art MRF-based methods.

The Markov random field (MRF) model has attracted great attention in the field of image segmentation. However, most MRF-based methods fail to resolve segmentation misclassification problems for high spatial resolution remote sensing images due to insufficiently using the hierarchical semantic information. In order to solve such a problem, this paper proposes an object-based MRF model with auxiliary label fields that can capture more macro and detailed information and apply it to the semantic segmentation of high spatial resolution remote sensing images. Specifically, apart from the label field, two auxiliary label fields are first introduced into the proposed model for interpreting remote sensing images from different perspectives, which are implemented by setting a different number of auxiliary classes. Then, the multilevel logistic model is used to describe the interactions within each label field, and a conditional probability distribution is developed to model the interactions between label fields. A net context structure is established among them to model the interactions of classes within and between label fields. A principled probabilistic inference is suggested to solve the proposed model by iteratively renewing the label field and auxiliary label fields, in which different information of auxiliary label fields can be integrated into the label field during iterations. Experiments on different remote sensing images demonstrate that our model produces more accurate segmentation than the state-of-the-art MRF-based methods. If some prior information is added, the proposed model can produce accurate results even in complex areas.

The multiresolution technique is one of the most important techniques for image segmentation. Wavelet transformation is a pixel-based method and is widely used for multiresolution segmentation approaches, but it suffers the deficiency of modeling the macrotexture pattern of a given image. In order to overcome such a problem, this letter extends the multiresolution technique from the pixel level to the region level and proposes a new image segmentation model by incorporating the multiregion-resolution and the Markov random field model. Experiments are conducted using synthetic-aperture-radar data and remote sensing images, which demonstrates that our method can improve the segmentation accuracy compared with the multiresolution method based on the pixel level.