This work investigates generative facial expression interfaces for intelligent agents from a meta-design perspective. We propose the Generative Personalized Facial Expression Interface (GPFEI) framework, which organizes rule-bounded spaces, character identity, and context–expression mapping to address challenges of control, coherence, and alignment in run-time facial expression generation. To operationalize this framework, we developed GenFaceUI, a proof-of-concept tool that enables designers to create templates, apply semantic tags, define rules, and iteratively test outcomes. We evaluated the tool through a qualitative study with twelve designers. The results show perceived gains in controllability and consistency, while revealing needs for structured visual mechanisms and lightweight explanations. These findings provide a conceptual framework, a proof-of-concept tool, and empirical insights that highlight both opportunities and challenges for advancing generative facial expression interfaces within a broader meta-design paradigm.

It is highly challenging to register large-scale, heterogeneous SAR and optical images, particularly across platforms, due to significant geometric, radiometric, and temporal differences, which most existing methods struggle to address. To overcome these challenges, we propose Grid-Reg, a grid-based multimodal registration framework comprising a domain-robust descriptor extraction network, Hybrid Siamese Correlation Metric Learning Network (HSCMLNet), and a grid-based solver (Grid-Solver) for transformation parameter estimation. In heterogeneous imagery with large modality gaps and geometric differences, obtaining accurate correspondences is inherently difficult. To robustly measure similarity between gridded patches, HSCMLNet integrates a hybrid Siamese module with a correlation metric learning module (CMLModule) based on equiangular unit basis vectors (EUBVs), together with a manifold consistency loss to promote modality-invariant, discriminative feature learning. The Grid-Solver estimates transformation parameters by minimizing a global grid matching loss through a dual-loop search strategy to reliably find patch correspondences across entire images. Furthermore, we curate a challenging benchmark dataset for SAR-to-optical registration using UAV MiniSAR data and Google Earth optical imagery. Extensive experiments demonstrate that our proposed approach achieves superior performance over state-of-the-art methods.

This work investigates the integration of generative visual aids in human-robot task communication. We developed GenComUI, a system powered by large language models that dynamically generates contextual visual aids (such as map annotations, path indicators, and animations) to support verbal task communication and facilitate the generation of customized task programs for the robot. This system was informed by a formative study that examined how humans use external visual tools to assist verbal communication in spatial tasks. To evaluate its effectiveness, we conducted a user experiment (n = 20) comparing GenComUI with a voice-only baseline. The results demonstrate that generative visual aids, through both qualitative and quantitative analysis, enhance verbal task communication by providing continuous visual feedback, thus promoting natural and effective human-robot communication. Additionally, the study offers a set of design implications, emphasizing how dynamically generated visual aids can serve as an effective communication medium in human-robot interaction. These findings underscore the potential of generative visual aids to inform the design of more intuitive and effective human-robot communication, particularly for complex communication scenarios in human-robot interaction and LLM-based end-user development.

Metacognition has been recognized as a crucial factor in enhancing human learning and problem-solving capabilities. While robots can serve as assistive mediators to improve human metacognition in specific tasks, existing research lacks a comprehensive framework for implementing metacognitive theories in human-robot interaction design. In addressing that, we propose a two-level state-based framework that integrates metacognition regulation cycle with specific intervention strategies. Through analyzing metacognitive theories, we present a hierarchical structure where Level 1 states represent the metacognition cycle and Level 2 states provide strategic interventions. Our work bridges the gap between metacognitive theory and human-robot interaction design, contributing a flexible approach for developing metacognition-enhanced robotic interfaces.

Single-modal change detection methods based on optical or Synthetic Aperture Radar (SAR) images face challenges such as degradation due to adverse weather or noise interference. In contrast, multimodal change detection struggles with significant domain gaps between different modalities. Inspired by the SAM2 model’s temporal memory mechanism for video segmentation, this paper introduces the concept of memory into change detection and proposes a novel approach called Memory-based Multimodal Change Detection (MMCD). By treating change detection as a temporal problem and modeling remote sensing images as video sequences, the proposed method integrates historical optical images with current SAR images to enhance detection accuracy. Additionally, a difference map enhancement module is introduced to mitigate false changes caused by modality discrepancies. Experimental results show that this approach achieves state-of-the-art performance in multimodal change detection, demonstrating the effectiveness of the proposed method.

Aircraft detection in synthetic aperture radar (SAR) images is one challenging task due to the discreteness of aircraft scattering, the diversity of aircraft size, and the interference of background. In order to deal with these problems, a novel method named scattering enhancement and feature fusion network (SEFFNet) is here proposed to detect aircraft via combining traditional image processing and deep learning together. At first, a scattering information extraction and enhancement module (SIEEM) is proposed to highlight the scattering points of aircraft targets. Then, to more effectively focus on the location of aircraft targets, a space-to-depth coordinate attention module (SDCAM) is further designed, following which an efficient multi-scale feature fusion pyramid (FFP) is also introduced to fuse the semantic information of different layers. At last, a contextual fusion head (CFH) is built to improve the receptive field for better detecting aircraft. The experiments carried out on the popular datasets SADD and SAR-AIRcraft-1.0 show that SEFFNet is more appropriate for aircraft detection, especially the small-size aircraft detection, in comparison with other state-of-the-art (SOTA) methods. Taking the dataset SADD for example, on average, the precision, recall, F1-score, and APs values are respectively 2.8%, 2.6%, 2.7%, and 2.0% higher than the baseline network YOLOv5.

The application of deep learning in the synthetic aperture radar (SAR) field is becoming increasingly widespread, but its black-box nature and the existence of adversarial samples limit its practical utility. In order to enhance the explainability and security of deep learning models, we initially selected 3 different deep convolutional neural network (DCNN) structures for training in SAR image classification. Subsequently, we applied Madry Defense Method to obtain robust models, and used 2 adversarial attack methods to attack DCNN classifiers. Finally, we employed 3 explainable artificial intelligence (XAI) methods to explain the predictions of different DCNN classifiers. Across different datasets and DCNNs, the Madry Defense Method helps classification models to reduce Infidelity and focus more on feature regions rich in semantic information when making decisions. The experimental results provide new insights into the adversarial robustness of DCNN classifiers in SAR image classification from an explainability viewpoint.

Deep learning models have shown excellent performance in synthetic aperture radar (SAR) automatic target recognition (ATR) tasks. However, the opacity of the decision-making mechanisms within these models hamper their credibility in practical applications. Therefore, numerous explainable artificial intelligence (XAI) methods have been developed to interpret the models. Among them, the randomized input sampling for explanation (RISE) method introduces input random pixel perturbations to observe resulting changes in the output by generating saliency heatmaps. However, unlike optical images, SAR images have their unique physical properties. This paper introduces the attributed scattering center (ASC) into the RISE method, known as the ASC-RISE method guided by physical information, to explain the model. Experimental results demonstrate that the heatmaps generated by ASC-RISE effectively locate the model’s decision features and provide corresponding physical information.

To mitigate the overfitting in the few-shot high-resolution range profile (HRRP) recognition, we introduce the Mahalanobis based statistical ProtoNet (MSP) with regularization, inspired by the prototypical network (ProtoNet). MSP leverages regularized feature covariance matrix to enhance the ProtoNet’s Euclidean distance metric based on the isotropic Gaussian distribution. Additionally, we propose a simplified MSP, the normalized statistical ProtoNet (NSP) for the faster inference of the statistical ProtoNet. Experiments demonstrate that statistical distance metrics enhance the few-shot recognition performance in scenarios with varying signal-to-noise ratios (SNR) and domain bias.

Land use land cover classification (LULC) plays a pivotal role in comprehending and managing the Earth’s surface. Numerous efforts have concentrated on deep learning methods utilizing supervised techniques, but their performance often relies on extensive well-annotated data, incurring significant time and labour expenses. This paper introduces MAE-PixelSeg, a novel self-supervised framework for LULC, leveraging a worldwide dataset of unlabeled Sentinel-2 satellite images from the Google Earth Engine (GEE) platform. MAE-PixelSeg employs the Vision Transformer (ViT) backbone, initialized by the MAE encoder, with a Shuffle Neck for high-resolution hierarchical feature map extraction and an ASPP Head for precise segmentation. Results demonstrate MAE-PixelSeg’s superiority over baseline methods, achieving a remarkable mean Intersection over Union (mIoU) of 71.07% on WorldCover and 77.43% on GID. Furthermore, we show that self-supervised pre-training yields robust feature representations, enabling commendable performance when transferred to other datasets, particularly in situations with severely limited annotated data.

The high-resolution range profile (HRRP) plays an important role in radar automatic target recognition (RATR) due to its rich target structure information and small data volume. However, it is difficult for HRRP to obtain sufficient data, especially for non-cooperative target, so few-shot HRRP target recognition methods have emerged. Inspired by human cognitive neuroscience, we compensate for the sample shortage by introducing semantic knowledge, focusing on key distinguishing features of categories while ignoring noise through knowledge guidance. To this end, we propose a knowledge-guided BiTCN prototypical network, where semantic knowledge is presented in the form of knowledge graph embeddings, HRRP feature extraction is completed using the temporal model BiTCN (bidirectional temporal convolutional network), and then knowledge-guided attention is used to enhance class prototypes. The experimental results demonstrate the effectiveness of our method.

With the rapid development of deep learning, significant progress has been made in radar high-resolution range profile (HRRP) target recognition methods based on deep neural networks. However, these closed set recognition methods assume that the categories of all targets are known during the model training phase, while in practice seen and unseen class targets coexist. Therefore, we propose a novel method for generalized zero-shot HRRP target recognition. Specifically, we use knowledge graph as auxiliary semantic information, utilizing two variational auto-encoders (VAEs) for cross-modal alignment and distribution alignment of semantic embeddings and HRRP features, and adopting a joint learning method of feature alignment and classification to enhance the separability of HRRP features for different categories. The experimental results demonstrate that our method is superior to existing methods.

Deep learning has significantly enhanced the performance of automatic target recognition (ATR) in synthetic aperture radar (SAR). However, the concept of model overinterpretation, characterized by classifiers discerning strong class evidence within image regions that lack semantically salient features related to target (e.g., background clutter in SAR images), has undermined confidence in the reliability of deep learning models. Previous studies predominantly relied solely on one interpretability method to qualitatively identify the key input pixels, without assessing the efficiency of these features in decision-making process, posing a significant hurdle in evaluating the model overinterpretation. In this paper, we propose necessity-sufficiency index (NSI) to select models’ decision-making basis among all the key regions identified by multiple interpretability methods and segmentation algorithm. Furthermore, we propose weighted composition ratio statistic (WCRS) method to quantitatively analyze the model overinterpretation by incorporating the NSI as weighted average weights. The experimental results indicate that our methods are capable of accurately identifying the decision-making features and quantitatively analyzing the models’ tendency towards overinterpretation.

Water body extraction is an important issue in flood surveillance and environmental protection. With the development of neural network, deep learning has been widely used in the water body extraction task because of its powerful feature extraction ability. Even so, most of existing deep learning networks only take into account the translation equivariance of convolution kernel for water body extraction. Actually, in terms of water body, its orientations imaged by optical sensor are usually various. So, when the orientated images are not well contained in the training set, the networks may yield some unsatisfactory extraction results. To solve this problem, in this paper, we propose an information-expanding network IE-Unet based on the traditional network U-net, where the rotation equivariant convolution, rotation-based channel attention mechanism, and the optimized Batchnorm are adopted jointly. To quantitatively evaluate its edge extraction capability, a new edge index AOD is proposed as well. The experimental results on one public dataset of water body demonstrate the effectiveness of IE-Unet. Compared with the original U-net, the IOU value of IE-Unet is increased by 7%, and the A0D value is reduced by 0.76.

Ship classification is usually a challenging task due to the small sizes of ship targets and the lack of significant differences between different categories. In terms of synthetic aperture radar (SAR) images, most existing deep learning-based methods are not designed from the angle of polarimetric characteristics to achieve ship classification. Thus, when facing the ship classification task of dual-polarized SAR images, these networks are often unsatisfactory. To cure this shortcoming, we here propose a novel dual branch deep network DBDN specifically designed for dual-polarized SAR ship classification. Our approach consists of three key modules: the image construction module ICM, the feature extraction module FEM, and the feature fusion and classifier module FFCM. In ICM, two novel pseudo RGB images are constructed for the first time, i.e., the polarimetric features-guided pseudo RGB image (PF-RGB) and the texture features-guided pseudo RGB image (TF-RGB), which can more accurately and comprehensively reflect ships’ characteristics. FEM enables the network to focus on important ship features and suppress irrelevant noise through transferred layers and designed ConvNeXt-Attention block (CNABlock), enhancing the discriminative capability of different ships. Finally, FFCM extracts and combines various ship features for classification, wherein the enhanced inverted residual block (EIRBlock) and the channel spatial attention module (CSAM) components are proposed as well. The performance of DBDN is evaluated on the OpenSARShip2.0 dataset, and experimental results show that DBDN achieves excellent performance in all evaluation metrics in comparison with some state-of-the-art (SOTA) algorithms. For example, compared to the recently proposed method DSN, DBDN further improves the accuracy by 4.74% and 4.07% in the three-class and six-class classification tasks, respectively.

With the increasing number of mobile devices, mobile crowdsensing (MCS) has garnered significant attention in research. However, computing infrastructures such as edge/cloud nodes, which are necessary for processing sensor data, are not always readily available. To address this issue, we propose a cooperative computing framework that enables the offloading of sensor data to nearby mobile devices with unused computational resources (known as helpers) for processing. Our approach considers a scenario with multiple sources and multiple helpers, where computational tasks can be partially offloaded to several helpers. We jointly optimize task offloading strategy, communication resources, and computational resources to minimize the weighted sum energy consumption of mobile devices. We model the optimization problem as a mixed- integer nonlinear programming (MINLP), with the source-helper assignment solved using a distributed algorithm based on matching theory, and the joint task partition and resource allocation problem solved using an alternating optimization (AO) method. Simulation results demonstrate the efficacy of our cooperative computing framework and scheduling scheme, which offer significant advantages over local computing in terms of reducing the weighted sum energy consumption and improving the task completion ratio.

During the portfolio selection process, investors often have individual behavioral preferences that may be fuzzy, uncertain, and incomplete. For the first time, this article combines fuzzy preference relations (FPRs) with portfolio theory to study the portfolio selection problem faced by investors. First, we improve the relations between judgment elements and priority vectors in FPRs and develop a new definition of additively consistent FPRs under priority vectors. On the basis of this and Markowitz portfolio models, we then propose two portfolio models considering FPRs with unknown preference information and three types of portfolio models considering FPR with partially known preference information. We conduct a comparative analysis of these models and derive a flowchart of investment selection under different conditions. Finally, we use an empirical example to compare the results of all models and analyze their sensitivity to various parameters. The results demonstrate that increasing the distance threshold between judgment elements has a positive effect on the objective function of portfolio models considering FPRs, although this effect gradually becomes saturated. Additionally, the models proposed in this article exhibit high robustness to the consistency index of FPRs.

Most of the conventional ship detection methods based on synthetic aperture radar (SAR) intends to process the focused images, which does not take full advantages of the intermediate data in the SAR imaging process. In this paper, we introduce a new framework that treats a two-dimensional point target as multiple one-dimensional sequences in the range-compressed domain, and then employs a Long Short-Term Memory (LSTM)-based network to perform the ship detection, thus reducing the computational burden and improving efficiency significantly. To validate the effectiveness of our proposed method, we conduct experiments on real SAR data. The results demonstrate the superiority of our framework in ship detection tasks.

Nowadays, deep learning has been widely used for water body detection because of its high precision databased water segmentation ability. Although the networks based on deep learning have shown higher automation, applicability and extraction accuracy than the traditional threshold methods in water body detection, only the translation equivariance of the convolution kernel is considered in these networks. Actually, for the detection of water body, its rotation equivariance also needs to be considered. For this goal, we here propose a new convolutional neural network by improving the U-Net with the rotation equivariant convolution and attention mechanism, which is simplified as GACNN. Experimental results on optical water body images demonstrate the effectiveness of the improved network based on U-net.

Ship plays an important role in human marine production and living activities at sea. In this paper, we design a ship detection method for polarimetric synthetic aperture radar (Pol-SAR) images. In brief, one nonlocal neighborhood polarimetric covariance matrix [NC] is first built by improving the neighborhood polarimetric covariance matrix [N] with a new similarity parameter rI. Then, the proposed method PWFNCis achieved through directly computing the polarimetric whiten filter (PWF) with [NC]. Experiments carried out on two real PolSAR datasets show that, compared to the recently proposed matrix [N], [NC] can better improve the ship detection performance of PWF.

Deep neural networks are widely used in SAR image classification and recognition, achieving state-of-the-art performance. But it remains a challenging task to recognize occluded targets. In this letter, we propose a novel robust SAR recognition method against occlusion. Specifically, we design a scattering excitation learning module that encourages the network to learn more robust features responding to the scattering centers of targets. In addition, we adopt a random feature channel dropout technique which can further improve robustness to occlusion. Our method makes the network more robust against occlusion but without any occlusion-simulated data for training. Experimental results on MSTAR dataset shows that our proposed method achieves remarkably improved robustness even under severe occlusions. Code is made available at https://github.com/koervcor/SEL-CD.

In recent years, deep learning has been extensively used in hyperspectral image (HSI) classification. The representative method is the convolutional neural network (CNN). However, due to the limitations of its inherent network backbone, CNNs still easily fail to mine some important information of HSIs, such as the sequence attributes of spectral signatures. To deal with this problem and make full use of the spectral-spatial information of HSIs, we propose a novel network named Multi-dimensional Information Expansion and Processing Network (MIEPN) for HSI classification, which is mainly composed of one information expansion module (IEM), one feature information expansion and extraction module (FEEM), and one ViT module. Briefly speaking, IEM expands and fuses HSI information in a three-dimensional (3D) space, yet FEPM pays more attention to digging deeper information. After these, the extracted information is input into the ViT module for HSI classification. Experiments carried out on several typical datasets demonstrate that the proposed network MIEPN can provide competitive results compared to the other state-of-the-art CNN-based methods.

The remote sensing community has shown increasingly interest in self-supervised learning for its ability to learn representations without labeled data. These representations can be easily adapted to downstream tasks through pre-training and fine-tuning. Recently, Masked Autoencoders (MAE) achieve better semantic representation by masking out a significant portion of the input image. However, the original design of MAE for RGB natural images may not be optimal for remote sensing (RS) images, which exhibit considerable variation between modalities like SAR and optical. To address this, we propose a 3D mask that enhances feature extraction along the vertical dimension. After fine-tuning, our 3DMAE model outperforms state-of-the-art contrastive and MAE-based models on BigEarthNet-MM classification and significantly reduces input data volume by at least 50% with the vertical mask, resulting in a more efficient model. Generalization experiments show a 5.9% F1-score improvement when applied to the SEN12MS dataset, which has diverse data distributions.

Supervised Deep Neural Networks (DNNs) have proven to be powerful tools for SAR image interpretation tasks. However, they present a formidable challenge in acquiring a substantial amount of labeled data. In this paper, we investigate the promising technique of contrastive self-supervised learning for SAR image classification. This approach allows us to take advantage of a large number of available unlabeled images to pre-train a SAR image classification model. Our novel contrastive learning framework conducts both instance-level and cluster-level pretext tasks, which not only enforce consistency between the images and their augmented "views" at the instance level but also their representation within clusters. Besides generating different views through random, low-level image transformations, we proposed two new strategies to construct positive sample pairs to improve contrastive SAR image feature learning: middle-level optical assistance and high-level graph searching. The middle-level optical assistance strategy is inspired by the observation that domain experts typically interpret SAR images with the aid of optical images. This insight spurs us to generate intermediate SAR images as positive samples from geographically matched optical data using CycleGAN. Furthermore, we augment the positive samples of the image with their KNN (K-Nearest Neighbor) counterparts, following the idea that the KNN samples should belong to the same cluster. Extensive experimental results conducted on the SEN12MS land cover classification benchmark dataset demonstrate that our method is competitive with state-of-the-art self-supervised methods for SAR image classification. Even with only a small amount of labeled data for fine-tuning the model, our method rapidly improves classification performance, surpassing models pre-trained on natural image datasets.

In the last decades, how to detect ships with polarimetric synthetic aperture radar (PolSAR) has become one hot topic. Unfortunately, most of the existing ship detection methods cannot well detect small ships with weak backscattering. To deal with this issue, a ship detection matrix named complete polarimetric covariance matrix [CP] was recently proposed from the perspective of spatial information utilization. Although it is able to improve small ships’ target-to-clutter ratio (TCR) values, its calculation strategy still needs to be rethought due to the possible information loss of some ships. Besides, its mathematical characteristic (i.e., not positive semidefinite) also limits the successful applications of some existing polarimetric theories to it. To overcome these two drawbacks, we here develop an information reconstruction-based polarimetric covariance matrix [IC]. In brief, one new difference calculation strategy is first performed on the Sinclair matrix [$S$], so as to reconstruct its information, by which a feature vector$v$is subsequently extracted with the Lexicographic matrix basis. Then, via further performing an outer product operation on$v$, the matrix [IC] is proposed. Meanwhile, to demonstrate the effectiveness of [IC] in ship detection, two different [IC]-based intensity detectors, respectively, named SPANIC and PEDIC, are designed as well. Experiments carried out on three GF-3 PolSAR datasets show that: 1) the proposed matrix [IC] has a better performance than [CP] and the original polarimetric covariance matrix [$C$] in ship detection and 2) compared to the total power detector SPAN and geometrical perturbation-polarimetric notch filter (GP-PNF), both SPANIC and PEDIC can better detect ships, especially the small ships.

The influence of mutual interaction behaviors on the opinions and consensus process has gradually emerged due to the information sharing in social networks. Currently, the cost of making individual decisions and the similarity between experts’ decision behaviors are relatively less addressed in the social network decision process. Therefore, in this study, a consensus approach with the trust relationships and adjustment cost is proposed to fill in such a gap. The method is divided into three stages: 1) trust propagation; 2) weight allocation; and 3) consensus reaching. In the trust propagation phase, uninorm is extended to the uncertain theory and employed in the transmission and integration problems of trust relationships. In the weight allocation stage, the comprehensive weight is assigned based on network structure and strength of relationship. In the consensus-reaching process, two levels of consensus are considered: 1) consensus among individuals and 2) consensus between individuals and the collective group, and chance-constrained programming models are constructed to obtain collective decision opinions. Moreover, a comparative analysis is performed to clarify the effectiveness and advancement of the proposed consensus method.

In this paper, we deal with the issue of discovering data of novel categories for Synthetic Aperture Radar (SAR) images under open-set conditions. The traditional SAR image classification methods are trained under the closed-set setting where all categories in testing data are seen in training data. It does not always meet the requirements of the real SAR imagery interpretation applications. With a labelled SAR image dataset, we propose a multi-stage approach to effectively pick out images belonging to new classes in another unlabelled dataset and then cluster them into correct number of novel categories. To do so, our pipeline is composed of three major steps: (1) train a powerful feature extractor leveraging both the labelled and unlabelled dataset by semi-supervised inference; (2) identify the unknown data by openset detection; (3) cluster these unknown data based on the features generated by the extractor to discover novel categories. The proposed method is validated on a Sentinel-1 SAR image dataset OpenSARUrban [1].

In this paper, we deal with the problem of heterogeneous image classifiers transferring between SAR and optical im-agery through a novel multi-stage domain adaptation technique. The problem of transferring the optical image classifier to SAR and vice-versa is of practical importance because it allows us to leverage plenty of labelled data in the source do-main for the target domain task, but gains little attention. Be-cause there is a drastic distribution-gap between both the opti-cal and SAR imaging modalities, it is non-trivial to apply do-main adaption directly. We propose a multi-stage adversarial feature alignment procedure that firstly performs global ad-versarial feature alignment and then a class-conditional adver-sarial feature alignment is conducted to further enable class-discriminative feature adaption. The proposed method is val-idated on the SEN12MS dataset, and some discussions are provided about heterogeneous domain adaption between SAR and optical imagery.

Deep learning methods exhibit outstanding performance in synthetic aperture radar (SAR) image interpretation tasks. However, these are black box models that limit the com-prehension of their predictions. Therefore, to meet this challenge, we have utilized explainable artificial intelli-gence (XAI) methods for the SAR image classification task. Specifically, we trained state-of-the-art convolutional neural networks for each polarization format on OpenSARUrban dataset and then investigate eight explanation methods to analyze the predictions of the CNN classifiers of SAR images. These XAI methods are also evaluated qualitatively and quantitatively which shows that Occlusion achieves the most reliable interpretation performance in terms of Max-Sensitivity but with a low-resolution explanation heatmap. The explanation results provide some insights into the in-ternal mechanism of black-box decisions for SAR image classification.

In this paper, we focus on synthetic aperture radar automatic target recognition for ships, and modify the Simple Siamese (SimSiam) framework, a contrastive self-supervised representation learning method, to improve ship classification accuracy. We design a novel sampling method that takes polarization information into account, in addition to the image augmentation based positive pair sampling method that is commonly used in contrastive learning approaches. The results of the experiments show that positive pairs of VH and VV polarized images can provide complementary information about ship targets to strengthen the classifiers. Besides, different similarity measurement functions are analyzed in the experiments.

Polarimetric synthetic aperture radar (PolSAR) designed to obtain the polarimetric information of scenes is a crucial tool for microwave remote sensing. Recently, a complete polarimetric covariance difference matrix [CP] was built to detect ships of PolSAR image. Along this work, this paper extends its application to the spectrum domain. Briefly speaking, four sub-aperture images are first separated from the original PolSAR data. Then, four different power values corresponding to the [CP] matrices of these sub-aperture images are respectively calculated. At last, via multiplying these values together, a PolSAR ship detector named MPS (Multiplicative Polarimetric SPAN) is proposed. The experiment carried out on one real PolSAR image demonstrates that, compared to traditional power detectors SPAN and$SPAN_{CP,}$MPS holds a better ability to detect small ships.

Aiming at the problem of ship classification in Synthetic Aperture Radar (SAR) images crossing different imaging satellites, we propose a novel domain adaptation (DA) network. For the proposed DA network, we consider one labeled SAR image dataset as source domain and another unlabeled dataset acquired by a different imaging satellite as target do-main. First, the structural regularization of the source domain is achieved by jointly training the feature classifier and the domain classifier. Then, by minimizing the KL-divergence between the label distribution predicted by the network and the introduced auxiliary distribution, the cluster alignment of the target domain is further realized. The experimental results on the datasets obtained from different satellites verify the performance of proposed method is better than state-of-the-art DA method.

Clustering analysis is a key technique in reducing the dimensionality of high volume irregular data containing large-scale group decision-making (LSGDM) information. Uncertainty theory is suitable for subjective estimation or situation, such as lack of historical data, and it can be employed to effectively express the uncertainty of trust and preference information in LSGDM problems. This paper studies the dimensionality reduction and subgroup optimization in LSGDM by utilizing linear uncertain variables in social networks. A clustering method is proposed to decompose the large group into several subgroups of higher consilience degrees and higher preference similarities, and lower the dimension of information for LSGDM. In the clustering process, two measurement attributes, trust relationship and preference relationship of decision-makers, are combined, and information consistent degree is utilized as the clustering indicator. This approach does not need to preset the threshold and the number of subgroups, and can be employed to obtain subgroups with similar preferences and stable trust relationship. Through the clustering reliability evaluation of subgroups, the rationality of large-scale group clustering results is verified. Subgroup consensus contribution is used to identify superior subgroups and quantify the role of subgroups in improving the consensus level. An example of emergency decision-making and comparative analysis is provided to explain the feasibility and advantages of the proposed method.

This letter focuses on remote sensing image interpretation and aims to promote the use of contrastive self-supervised learning in varied applications of remote sensing image classification. The proposed method is a contrastive self-supervised pre-training framework that encourages the network to learn image representations by comparing image embeddings extracted by different encoders and predictors. Experiments were carried out on a variety of remote sensing image datasets to determine the efficacy of the proposed method for classification tasks. Results show that the proposed framework exploits the capabilities of encoders and outperforms the supervised learning method in terms of classification accuracy. Besides, it takes a few pre-training epochs to find a suboptimal initialization of network weights, and the pre-trained encoders use a little training data to get outstanding classification results, which shows the time and data efficiency of the proposed framework. Code is available at https://github.com/yinxu98/MECo.

Ship detection of polarimetric synthetic aperture radar (PolSAR) plays an important role in marine monitoring and ocean protection. Over the past years, local spatial information around pixels has been successfully applied to this task. However, few works have been done on PolSAR ship detection using the nonlocal spatial information (NSI). Within this context, we here propose one NSI-guided ship detection method PMR. Briefly speaking, the feature power difference (PD) is first constructed by computing the total power difference between the center pixelcand its most similar nonlocal pixeliwithin a 7×7 window. Then, the polarimetric feature reflection symmetry (RS) is introduced into PD to construct the method PMR (i.e., PD Multiply RS) for further enhancing the target-to-clutter ratio (TCR) and improving the ship detection accuracy. Experiments carried out on three real PolSAR datasets show that, in comparison with some other methods, especially the recently proposed local neighborhood information-based ship detector PWFN, PMR is more apt for ship detection. On average, its figure of merit (FoM) and TCR values respectively surpass PWFN0.24 and 12.83 dB.

Synthetic aperture radar (SAR) image classification task when the training and test sets have different distributions can be initially solved using existing domain adaptation (DA) methods. However, considering that none of their classification accuracy is high, this letter proposes an active learning DA classification method to further solve this task. First, an adversarial learning-based DA pipeline is put forth, using labeled source and unlabeled target domains to conduct adversarial learning in order to narrow the domain gap. A prototype regularization process is then built, which further enhances the target domain data clusters’ ability to discriminate between them. In order to fully improve SAR image classification accuracy, we then propose a dynamic hard sample selection process to choose hard samples to supplement into the subsequent stage of training samples. This process involves moving the gradient direction of the query function closer to the gradient direction of the class margin objective function. Extensive experiments on SAR image datasets with different distributions from different imaging platforms and optical remote sensing datasets have verified the effectiveness and superiority of the proposed method.

For synthetic aperture radar (SAR) image classification problem, we need to take into account unlabeled datasets containing unknown classes crossing different satellites. In this letter, a spherical space domain adaptation (DA) network under open set condition is proposed to solve this problem. First, we transform the prior Euclidean feature space into the spherical space to construct a classification network such that features of the same class of SAR images are clustered together and features of different or unknown classes are separated on the hypersphere. Second, a correction module is designed to increase the accuracy of the pseudo-label obtained by the classifier. Then, based on the adversarial learning strategy, we introduce the gradient alignment module to achieve better alignment of the source and target domains. Finally, tests on two SAR benchmark datasets from distinct satellites show that the proposed network outperforms state-of-the-art (SOTA) approaches in terms of classification accuracy.

Ship detection using polarimetric SAR (PolSAR) images has recently been an active topic in the Earth observation field. There, how to detect small ships is an open and challenging issue. Within this context, we put forward a two-stage ship detection model, by which a novel ship detection method is proposed as well. Briefly, in the first stage, a suppression manipulation is adopted to suppress sea clutter, where the feature SVVSOis built on the intensity information with the orientation angle compensation (OAC). In the second stage, an enhancement manipulation is further executed to highlight ships from the suppressed sea clutter, where the features PID (polarimetric intensity difference) and NsD (nonsurface degree) are first constructed with SVVSOand a series of theoretical derivations. Then, via fusing PID and NsD together, the two-stage-based method FPAN is proposed to detect ships. To demonstrate its performance, we apply FPAN to four different L-Band PolSAR datasets. Experimental results reveal that, compared to other state-of-the-art methods, especially the DBSPCPmethod, FPAN is more effective in detecting small ships. On average, its figure-of-merit (FoM) and target-to-clutter ratio (TCR) values are, respectively 9.40% and 25.18% greater than those of DBSPCP, while the time consumption is just 58.67% of the latter.

In the face of Synthetic Aperture Radar (SAR) image object detection with different distributions of training and test data, traditional supervised learning methods cannot achieve good detection performance. Domain adaptation (DA) method has been shown to have the ability to solve this problem, but existing DA object detection algorithms all use adversarial DA theory for the detection task, which is ineffective in solving object regression localization in the detection task. In this article, to better solve the above problem, an automatic SAR image ship detection method based on feature decomposition crossing different satellites is proposed. The feature extraction layer of backbone network is divided into low level and high level, where domain-invariant feature extractors are designed for the local features extracted from the low level and the global features extracted from the high level, respectively. We argue that the local and global features extracted from source domain and target domain contain domain-specific features (DSF) for adversarial DA and domain-invariant features (DIF) that contribute to object regression localization. Then, we decompose the local features and global features into DSF and DIF via vector decomposition method. For DSF counterpart, we introduce adversarial DA attention for feature alignment. DIF from the local features are fused into the backbone network for high-level global feature extraction. Finally, by using region proposal network and adversarial domain classifier, we can get the accurate bounding box and object class of SAR image objects. Extensive experiments prove that the proposed method outperforms state-of-the-art methods in terms of detection performance.

Traditional deep learning Synthetic Aperture Radar (SAR) image object detection methods fail to provide effective detection results when faced with SAR image datasets with different joint probability distributions obtained from multiple imaging satellites. In this article, an automatic SAR image object detection method based on domain adaptation is proposed to adapt to unlabeled target domain datasets acquired by different satellites. On the basis of introducing an adversarial domain adaptation learning strategy, we propose Adversarial Learning Attention (ALA) and Compensation Loss Module (CLM) on the baseline network. In ALA, considering the great difference in the scattering intensity of SAR images, the entropy vector can be used to distinguish the high-entropy and low-entropy regions among them and assign the corresponding weights, based on which the adversarial domain adaptation learning attention is proposed to achieve instance-level feature alignment and pixel-level feature alignment in source domain and target domain, respectively. In CLM, the domain alignment of pixel-level feature and instance-level feature of SAR image objects is first implemented, and then to make the feature alignment of both domains more accurate, better aggregation of proposals of different classes of prototype objects in the same domain is required, and further compensation loss is proposed to further restrict the prototype alignment of SAR object features in both domains. We conduct experiments on two SAR datasets obtained from different satellites whose results show the superiority of the proposed method over the state-of-the-art (SOTA) methods and the effectiveness of the proposed module in improving the detection accuracy.

LinkedIn connects the world's professionals to make them more productive and successful. People Search plays an important role in fulfilling this goal by helping members find the most relevant and personalized results through a broad range of queries like names, job titles, skills, companies, locations, etc. It is one of the biggest search verticals at LinkedIn both in terms of engineering footprint and search traffic. In this paper, we present an overview of the People Search system, and discuss how we build and serve deep neural network (DNN) models, leveraging state-of-the-art deep natural language processing (NLP) techniques (e.g., convolutional neural networks (CNN) and Bidirectional Encoder Representations from Transformers (BERT)). We describe our journey of applying deep neural ranking models to a real-life product, including the modeling and system bottleneck challenges, crucial design choices, and lessons learned along the way. We hope a story of our endeavors and successes will provide meaningful insights to other similar systems.

Accurate and efficient ship detection in remote sensing images still remains a challenging task due to the large variations of scales, orientations and distributions. In this paper, we propose an anchor-free ship detector that directly regresses ship localization parameters, offering a simpler pipeline over the previous methods. The detection network is then trained in a multi-task fashion which contains not only the ship center-point maps and oriented bounding boxes but the ship masks. Instead of fixing the weights among the multiple task losses, we adopt a curriculum learning strategy which gradually adapts the loss weights during the training process so that the network can learn the discriminative ship features at the early stage and obtain more localization information while training continues. Experimental results on real dataset demonstrate the effectiveness and efficiency of our proposed method.

In Synthetic Aperture Radar (SAR) image classification tasks, the performance depends on both the classifier and the dataset itself. However, in comparison with plenty of SAR classification methods, there is little work aimed at analyzing the distinguishability of the dataset. In the classification dataset, some classes are semantically different but their distinguishability is low, the classes are hard to be classified especially in some more practical cases that there are unknown classes without supervision exist. Referring to open set recognition (OSR), in this paper, we proposed the SAR Distinguishability Analysor (SAR-DA) to evaluate the distinguishability of the OpenSARUrban dataset. By modeling each class as a multivariate Gaussian distribution in latent space, SAR-DA can not only classify the classes having been seen in training phase, but also can recognize unknown samples if a test sample is out of each known distribution. Each class in OpenSARUr-ban is set unknown in turn, then we apply the SAR-DA on the split dataset in OSR and supervised setting. The distinguishability can be reflected by the unknown recognition recall rate. The experimental results show that the unknown recognition recall rate in OSR setting significantly decreased compared with those in supervised setting, indicating that even though the classes in OpenSARUrban are semantically different from each other, the latent distributions of some classes are quite similar and hard to be classified, thus these classes are of low distinguishability.

In this paper, we present a self-supervised learning method of Auto-Encoding Multi-Transformations (AEMT) for airplane classification. In this method, the image features are learned in an unsupervised way by simultaneously estimating multiple image transformations from the features of original and transformed images instead of reconstructing the input images. Besides, we propose two structure variants of the AEMT method: composite and parallel modes of which the former transforms the images in a composite fashion while the latter does it in parallel. The experimental results demonstrate that the proposed method outperforms the state-of-the-art self-supervised learning methods for the airplane classification task.

Practical large-scale recommender systems usually contain thousands of feature fields from users, items, contextual information, and their interactions. Most of them empirically allocate a unified dimension to all feature fields, which is memory inefficient. Thus it is highly desired to assign various embedding dimensions to different feature fields according to their importance and predictability. Due to the large amounts of feature fields and the nuanced relationship between embedding dimensions with feature distributions and neural network architectures, manually allocating embedding dimensions in practical recommender systems can be challenging. To this end, we propose an AutoML-based framework (AutoDim) in this paper, which can automatically select dimensions for different feature fields in a data-driven fashion. Specifically, we first proposed an end-to-end differentiable framework that can calculate the weights over various dimensions in a soft and continuous manner for feature fields, and an AutoML-based optimization algorithm; then, we derive a hard and discrete embedding component architecture according to the maximal weights and retrain the whole recommender framework. We conduct extensive experiments on benchmark datasets to validate the effectiveness of AutoDim.

The use of consistency methods to supplement or generate missing information so as to obtain a rational and logical ranking list has been a key point in studies of the incomplete preference relation. Existing works on incomplete interval fuzzy preference relations (IFPRs) utilize only endpoints of the original intervals and generate missing values using interval operations, resulting in the loss and distortion of information. This article considers IFPRs as linear uncertain preference relations (LUPRs), and introduces belief degree and inverse uncertainty distributions to explore the consistency and ranking problems with LUPRs. Definitions related to uncertain preference relations (UPRs) as well as their additive and multiplicative consistency are first given. Then, models to solve weight vectors are proposed under both consistency scenarios. Finally, algorithms that generate missing information in incomplete LUPRs are provided. By building mathematical relationships between the minimum deviation and the belief degree, analytic formulas for missing values are obtained when the minimum deviation is achieved. It is verified that consistent UPRs and the proposed weight-vector solving models under incomplete preference relations are extensions of the traditional IFPRs.

Ranking is the most important component in a search system. Most search systems deal with large amounts of natural language data, hence an effective ranking system requires a deep understanding of text semantics. Recently, deep learning based natural language processing (deep NLP) models have generated promising results on ranking systems. BERT is one of the most successful models that learn contextual embedding, which has been applied to capture complex query-document relations for search ranking. However, this is generally done by exhaustively interacting each query word with each document word, which is inefficient for online serving in search product systems. In this paper, we investigate how to build an efficient BERT-based ranking model for industry use cases. The solution is further extended to a general ranking framework, DeText, that is open sourced and can be applied to various ranking productions. Offline and online experiments of DeText on three real-world search systems present significant improvement over state-of-the-art approaches.

As the largest professional network, LinkedIn hosts millions of user profiles and job postings. Users effectively find what they need by entering search queries. However, finding what they are looking for can be a challenge, especially if they are unfamiliar with specific keywords from their industry. Query Suggestion is a popular feature where a search engine can suggest alternate, related queries. At LinkedIn, we have productionized a deep learning Seq2Seq model to transform an input query into several alternatives. This model is trained by examining search history directly typed by users. Once online, we can determine whether or not users clicked on suggested queries. This new feedback data indicates which suggestions caught the user's attention. In this work, we propose training a model with both the search history and user feedback datasets. We examine several ways to incorporate feedback without any architectural change, including adding a novel pairwise ranking loss term during training. The proposed new training technique produces the best combined score out of several alternatives in offline metrics. Deployed in the LinkedIn search engine, it significantly outperforms the control model with respect to key business metrics.

Query Auto Completion (QAC), as the starting point of information retrieval tasks, is critical to user experience. Generally it has two steps: generating completed query candidates according to query prefixes, and ranking them based on extracted features. Three major challenges are observed for a query auto completion system: (1) QAC has a strict online latency requirement. For each keystroke, results must be returned within tens of milliseconds, which poses a significant challenge in designing sophisticated language models for it. (2) For unseen queries, generated candidates are of poor quality as contextual information is not fully utilized. (3) Traditional QAC systems heavily rely on handcrafted features such as the query candidate frequency in search logs, lacking sufficient semantic understanding of the candidate.

Crowd counting plays an increasingly important role in public security. Recently, many crowd counting methods for a single image have been proposed but few studies have focused on using temporal information from image sequences of videos to improve prediction performance. In the existing methods using videos for crowd estimation, temporal features and spatial features are modeled jointly for the prediction, which makes the model less efficient in extracting spatiotemporal features and difficult to improve the performance of predictions. In order to solve these problems, this paper proposes a Parasite-Host Network (PHNet) which is composed of Parasite branch and Host branch to extract temporal features and spatial features respectively. To specifically extract the transform features in the time domain, we propose a novel architecture termed as “Relational Extractor”(RE) which models the multiplicative interaction features of adjacent frames. In addition, the Host branch extracts the spatial features from a current frame which can be replaced with any model that uses a single image for the prediction. We conducted experiments by using our PHNet on four video crowd counting benchmarks: Venice, UCSD, FDST and CrowdFlow. Experimental results show that PHNet achieves superior performance on these four datasets to the state-of-the-art methods.

Satellite and Aerial Image Time Series contain tremendous amounts of information of ground targets in space and time and have been commonly used in temporal pattern analysis tasks of ground targets. It has a wide range of applications including environment monitoring, urban planning, hazard assessment, etc. PV (photovoltaic) field construction monitoring is a new topic with increasing attentions. In this paper, we proposed a simple yet effective network based on LSTM (long short term memory) to detect PV field construction event. We build image time-series dataset from Sentinel-2 data of an Egypt PV field under construction to monitor the project progress. Compared with clustering model and CNNs (convolutional neural networks), our network achieves the better accuracy. Although low resolution and mislabeled pixels limit the accuracy cap, the simple network is still robust and easy to transfer for other applications.

Oil is an essential asset for every country, and plays a key role in world trade system. The detection of oil tanks is a very important task for both military and commerce. Recently, researchers have shown an increasing interest in oil tanks detection in remote sensing imagery. However, the previous works almost used the methods of circle detection, but the real oil tanks in remote sensing imagery are more close to ellipses. In this paper, we propose an oil tanks detector base on an U-shape Fully Convolutional Network(FCN) in optical remote sensing images. The structure of our network consists of three parts: feature extraction part, feature merge part and the output layer. The output layer consists of two output branches, one branch is a score map branch, which generates confidence score to indicate the region of oil tanks at pixel wise, and the other ends up with several channels which regress the ellipse geometric parameters (center, horizontal axis and vertical axis). In addition, we also design a novel loss function adapted to our network. The experimental results conducted on our dataset collected from Google Earth show that this method achieves promising performance on oil tanks detection in terms of both efficiency and accuracy in high-resolution optical remote sensing images.

In this talk, we will go over the components of personalized search and recommender systems and demonstrate the applications of various deep learning techniques along the way.

With the exponential growth of information on the internet, users have been relying on search engines for finding the precise documents. However, user queries are often short. The inherent ambiguity of short queries imposes great challenges for search engines to understand user intent. Query suggestion is one key technique for search engines to augment user queries so that they can better understand user intent. In the past, query suggestions have been relying on either term-frequency--based methods with little semantic understanding of the query, or word-embedding--based methods with little personalization efforts. Here, we present a sequence-to-sequence-model--based query suggestion framework that is capable of modeling structured, personalized features and unstructured query texts naturally. This capability opens up the opportunity to better understand query semantics and user intent at the same time. As the largest professional network, LinkedIn has the advantage of utilizing a rich amount of accurate member profile information to personalize query suggestions. We applied this framework in the LinkedIn production traffic and showed that personalized query suggestions significantly improved member search experience as measured by key business metrics at LinkedIn.

Automatic building extraction from aerial or satellite images is a dense pixel prediction task for many applications. It demands a large number of clean label data to train a deep neural network for building extraction. But it is labor expensive to collect such pixel-wise annotated data manually. Fortunately, the building footprint data of geographic information system (GIS) maps provide a cheap way of generating building label data, but these labels are imperfect due to misalignment between the GIS maps and images. In this letter, we consider the task of learning a deep neural network to label images pixel-wise from such noisy label data for building extraction. To this end, we propose a general label noise-adaptive (NA) neural network framework consisting of a base network followed by an additional probability transition modular (PTM) which is introduced to capture the relationship between the true label and the noisy label. The parameters of the PTM can be estimated as part of the training process of the whole network by the off-the-shelf backpropagation algorithm. We conduct experiments on real-world data set to demonstrate that our proposed PTM can better handle noisy labels and improve the performance of convolutional neural networks (CNNs) trained on the noisy label data generated by GIS maps for building extraction. The experimental results indicate that being armed with our proposed PTM for fully CNN, it provides a promising solution to reduce manual annotation effort for the labor-expensive object extraction tasks from remote sensing images.

Search and recommender systems share many fundamental components including language understanding, retrieval and ranking, and language generation. Building powerful search and recommender systems requires processing natural language effectively and efficiently. Recent rapid growth of deep learning technologies has presented both opportunities and challenges in this area. This tutorial offers an overview of deep learning based natural language processing (NLP) for search and recommender systems from an industry perspective. It first introduces deep learning based NLP technologies, including language understanding and language generation. Then it details how those technologies can be applied to common tasks in search and recommender systems, including query and document understanding, retrieval and ranking, and language generation. Applications in LinkedIn production systems are presented. The tutorial concludes with discussion of future trend.

Deep learning models have been very successful in many natural language processing tasks. Search engine works with rich natural language data, e.g., queries and documents, which implies great potential of applying deep natural language processing on such data to improve search performance. Furthermore, it opens an unprecedented opportunity to explore more advanced search experience, such as conversational search and chatbot. This tutorial offers an overview on deep learning based natural language processing for search systems from an industry perspective. We focus on how deep natural language processing powers search systems in practice. The tutorial introduces basic concepts, elaborates associated challenges, reviews the state-of-the-art approaches, covers end-to-end tasks in search systems with examples, and discusses the future trend.

Ship detection from high-resolution optical remote sensing images has been a prevalent domain in recent years. Unlike objects in natural images, ships of interest can be anywhere in optical remote sensing images with multi-scale and multi-oriented which makes it more different to be detected. In this paper, we propose a novel method based on the fully convolutional network to detect ships. Our method has three important components: 1) we design a network merging different levels of feature map to fuse multi-scale information. Determining the existence of large ship require features from deep layers in the network, while predicting rotated bounding box enclosing small ships need shallow layers information; 2) The network can be trained end-to-end to generate score maps which indicates the confidence score for the ship region of interest in pixel-wise level through all locations and scaled of an image; 3) We design a rotated bounding box regression model to localize the ships. The experimental results on our dataset collected from Google Earth has demonstrated our proposed method achieves promising performance on ship detection in terms of both efficiency and accuracy in high-resolution optical remote sensing images.

Ship detection from remote sensing images can provide important information for maritime reconnaissance and surveillance and is also a challenging task. Although previous detection methods including some advanced ones based on deep convolutional neural network expertize in detecting horizontal or nearly horizontal targets, they cannot give satisfying detection results for arbitrary-oriented ship detection. In this letter, we introduce a novel ship detection system that can detect arbitrary-oriented ships. In this method, a rotated region proposal networks (R2PN) is proposed to generate multiorientated proposals with ship orientation angle information. In R2PN, the orientation angles of bounding boxes are also regressed to make the inclined ship region proposals generated more accurately. For ship discrimination, a rotated region of interest pooling layer is adopted in the following classification subnetwork to extract discriminative features from such inclined candidate regions. The proposed whole ship detection system can be trained end to end. Experimental results conducted on our rotated ship data set and HRSD2016 benchmark demonstrate that our proposed method outperforms state-of-the-art approaches for the arbitrary-oriented ship detection task.

In this paper, we present a method of ship size extraction for Sentinel-1 synthetic aperture radar (SAR) images, which is composed of the image processing stage and the regression stage. In order to achieve extraction with high accuracy, considering the data characteristics of Sentinel-1 images, we propose to use the dual-polarization fusion and the nonlinear regression with the gradient boosting. The experiments and analyses on a relatively large data set show that: 1) compared with the existing and related studies, the proposed method achieves an improved performance. The extraction errors are pushed under one pixel, and they are 4.66% (8.80 m) and 7.01% (2.17 m) for length and width, respectively; 2) the dual-polarization information fusion does improve the size extraction accuracy; and 3) the nonlinear regression does exploit the relationship between the influential factors and the size parameters and provide a better performance than the linear regression. The experimental results verify that the proposed design is suitable for ship size extraction in Sentinel-1 SAR images.

When people recall and digest what they have read for writing summaries, the important content is more likely to attract their attention.Inspired by this observation, we propose a cascaded attention based unsupervised model to estimate the salience information from the text for compressive multi-document summarization.The attention weights are learned automatically by an unsupervised data reconstruction framework which can capture the sentence salience.By adding sparsity constraints on the number of output vectors, we can generate condensed information which can be treated as word salience.Fine-grained and coarse-grained sentence compression strategies are incorporated to produce compressive summaries.Experiments on some benchmark data sets show that our framework achieves better results than the state-of-the-art methods.

In this paper, we propose a superpixel generation method for synthetic aperture radar (SAR) images by using the density-based spatial clustering of applications with noise (DBSCAN) algorithm. The pixels is firstly grouped to generate initial superpixels by using probabilistic patch-based (PPB) dissimilarity. Then, small clusters are combined into their neighbor superpixels to get final results through a distance measurement defined by statistical models. Experiments on simulated data sets exhibit high boundary adherence of the generated superpixels and demonstrate the availability and efficiency of the proposed method.

Performance of ship detection is influenced by synthetic aperture radar (SAR) imaging characteristics and environmental conditions. In this paper, aiming at evaluating vessel detectability for Sentinel-1 SAR data, a model based on a large-scale Sentinel-1A vessel chips database is established. The model sensitivity is analyzed by simulation data. In the experiment, by inputting the parameters of imaging characteristics (incidence angle, polarization, spatial resolution) and environmental conditions (wind speed, wind direction, sea state) of a specific Sentinel-1 image, the minimum detectable vessel length can be estimated. Further validations demonstrate the availability of the estimated minimum detectable vessel length.

This paper presents a new algorithm for recognizing patterns of densely arrayed targets in high resolution (HR) SAR images, serving the increasing demand for target detection and recognition in HR/VHR SAR images. The novelty of our work is to formulate the problem of pattern extarction as a jigsaw puzzle with similar target patches. Compared to existing work, our algorithm has multiple advantages, including: 1) extracting densely arrayed targets region from full scale SAR images automaticly; 2) predicting accurate displacement vector between neighboring similar patches; 3) synthetising an uniform target pattern with similar patches. These advantages are demonstratced by results of pattern extraction from a study case in the Davis-Monthan air force base.

Synthetic Aperture Radar (SAR) image land cover classification is an important task in SAR image interpretation. Supervised learning, such as Convolutional Neural Network (CNN), demands instances which are accurately labeled. However, a large amount of accurately labeled SAR images are difficult to produce. In this paper, a Probability Transition CNN (PTCNN) is proposed for patch-level SAR image land cover classification with noisy labels. Firstly, deep features are extracted by a CNN model, followed by a probabilistic transition model, where true labels are treated as hidden variables and the posterior probabilities of true labels are transferred into their noisy versions. The whole network is trained with Caffe in a uniform fashion and a land cover database is used to produce noisy labels, which are randomly chosen with various proportions. Experimental results demonstrate that the proposed PTCNN model is robust to noise and gives a promising classification performance. Therefore, the PTCNN model may lower the standards for the quality of image labels, and shows its availability in practical applications.

Fine-scale classification in form of object extraction or segmentation for high resolution SAR images is a challenging task due to the existing local noises, object deformation and part missing. A novel SAR classification method based on CRFs which combines low-level features, label context and object structure priors is presented in this paper. Local label pattern is proposed in this paper to model the object structures by measuring the local label configuration on the grid layer of SAR images. We build a new CRFs model with label context and object structure priors for image classification. Besides, we adopt Mean Field approximation for efficient inference of our CRFs model. This work intends to implement an efficient classification framework by integrating high-level label context and object priors and apply it to fine-scale object extraction of SAR images. The framework demonstrates good performance in both accuracy and efficiency for object extraction or segmentation of simulated images and high resolution SAR images.

In this paper, we propose a fast PolSAR image superpixel segmentation method. This method takes a simple coarse-to-fine optimization technique to minimize a Markov-Random-Field (MRF) like energy function which integrates the Pol- SAR image statistic, spatial position and boundary smoothing. It updates boundary of superpixels staring with a large block level and iterates down to the final pixel level. We demonstrate the performance of our approach both on the synthetic and real full polarimetric images , showing that our proposed approach can achieve significantly faster convergence than SLIC method, and make a good compromise between accuracy and computation speed.

Convolutional Neural Network (CNN) has attracted much attention for feature learning and image classification, mostly related to close range photography. As a benchmark work, we trained a relatively large CNN to classify SAR image patches into five different categories, where the image patches tiled and annotated from a typical TerraSAR-X spotlight scene of Wuhan, China. The neural network designed in this paper consists of seven layers, including one input layer, two convolutional layers where each followed by a max-pooling layer, as well as two fully-connected layers with a final five-class softmax. Using the toolkit caffe, we achieved the training and testing accuracy of 85.7% and 85.6% respectively, which is considerably better than the traditional feature extraction and classification based SVM method and shows great potential of CNN used for SAR image interpretation. In order to accelerate the training process, a very efficient GPU implementation was employed.

Web search queries provide a surprisingly large amount of information, which can be potentially organized and converted into a knowledgebase. In this paper, we focus on the problem of automatically identifying brand and product entities from a large collection of web queries in online shopping domain. We propose an unsupervised approach based on adaptor grammars that does not require any human annotation efforts nor rely on any external resources. To reduce the noise and normalize the query patterns, we introduce a query standardization step, which groups multiple search patterns and word orderings together into their most frequent ones. We present three different sets of grammar rules used to infer query structures and extract brand and product entities. To give an objective assessment of the performance of our approach, we conduct experiments on a large collection of online shopping queries and intrinsically evaluate the knowledgebase generated by our method qualitatively and quantitatively. In addition, we also evaluate our framework on extrinsic tasks on query tagging and chunking. Our empirical studies show that the knowledgebase discovered by our approach is highly accurate, has good coverage and significantly improves the performance on the external tasks.

Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, Rebecca Passonneau. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Sarcasm is generally characterized as a figure of speech that involves the substitution of a literal by a figurative meaning, which is usually the opposite of the original literal meaning.We re-frame the sarcasm detection task as a type of word sense disambiguation problem, where the sense of a word is either literal or sarcastic.We call this the Literal/Sarcastic Sense Disambiguation (LSSD) task.We address two issues: 1) how to collect a set of target words that can have either literal or sarcastic meanings depending on context; and 2) given an utterance and a target word, how to automatically detect whether the target word is used in the literal or the sarcastic sense.For the latter, we investigate several distributional semantics methods and show that a Support Vector Machines (SVM) classifier with a modified kernel using word embeddings achieves a 7-10% F1 improvement over a strong lexical baseline.

This paper proposes a semi-supervised nonnegative matrix factorization algorithm for face and gait recognition. The proposed algorithm imposes hard constraints on the labelled data points, such that the data points that belong to the same class are projected to the same lower dimensional point. In addition, it introduces a graph Laplacian regularization term that preserves the local geometry structure of the data by penalising large distances between the projections of points that are close in the original space. This results in a constrained optimization problem, that is solved using block coordinate descent with multiplicative update rules. Experimental results on several publicly available datasets demonstrate that proposed method performs in par or considerably better than state of the art methods.

In this paper, we exploit the advantages of tensorial representations and propose several tensor learning models for regression. The model is based on the canonical/parallel-factor decomposition of tensors of multiple modes and allows the simultaneous projections of an input tensor to more than one direction along each mode. Two empirical risk functions are studied, namely, the square loss and ε -insensitive loss functions. The former leads to higher rank tensor ridge regression (TRR), and the latter leads to higher rank support tensor regression (STR), both formulated using the Frobenius norm for regularization. We also use the group-sparsity norm for regularization, favoring in that way the low rank decomposition of the tensorial weight. In that way, we achieve the automatic selection of the rank during the learning process and obtain the optimal-rank TRR and STR. Experiments conducted for the problems of head-pose, human-age, and 3-D body-pose estimations using real data from publicly available databases, verified not only the superiority of tensors over their vector counterparts but also the efficiency of the proposed algorithms.