Vanilla convolution and window-based self-attention have shown significant success in image dehazing. However, they are constrained by limited receptive fields and ignore frequency gaps between dehazed and clear images. The former hampers the modeling of global dependencies, while the latter impedes the learning of high-frequency features, leading to suboptimal performance. In this paper, we propose the Joint Spatial and Fourier Convolutional Network (JSFC-Net), which leverages Fourier transformation to simultaneously address the two aforementioned problems with low computational overhead. We introduce the Frequency-Spatial Promoted and Physical Learning Block, which extracts high-level features from the spatial domain and frequency domain in parallel. We design a simple yet effective solution that uses spatial features to promote and modulate frequency features in a multi-scale manner, achieving refinement of frequency features and addressing robustness issue caused by global sensitivity. Additionally, we present the Receptive Field Selection Module to facilitate improved fusion of spatial and frequency domain features. Finally, we introduce frequency loss to further narrow frequency gaps. Comprehensive experiments on multiple datasets demonstrate that JSFC-Net is significantly superior to SOTA dehazing methods.

In satellite-based tactical communication systems, non-cooperative radio frequency (RF) communication signals are widely used, but their detection remains highly challenging in complex electromagnetic environments characterized by low signal-to-noise ratios, multi-signal coexistence, and dynamic interference. This paper focuses on the most widely used non-cooperative communication protocols—Link 11 and Link 4A—and proposes a cross-modal detection approach that maps signals into the image domain through high-resolution time-frequency analysis, enhancing detection robustness and interpretability. Additionally, we construct SC2SM, the first benchmark dataset specifically designed for communication signal detection, comprising 14,791 time-frequency images and over 85,000 annotated bounding boxes, comprehensively covering aliasing, strong interference, and other complex environmental scenarios. Furthermore, we introduce YOLO-Link, the first optimized object detection framework for this task, which enables efficient deployment on Internet of Things (IoT) edge computing platforms. Extensive experimental results demonstrate that, after training on the SC2SM dataset, YOLO-Link achieves state-of-the-art performance in communication signal detection, outperforming existing methods by 4.1% in Recall and achieving real-time inference at 40 FPS on IoT edge computing platforms, striking an optimal balance between detection accuracy and computational efficiency. This study provides technical support for intelligent detection and monitoring of non-cooperative communication signals and promotes the application of cross-modal learning in complex electromagnetic environments. The code and dataset are available at.

Satellite video images contain temporal contextual information that is unavailable in single-frame images. Therefore, using a sequence of frames for super-resolution can significantly enhance the reconstruction effect. However, most existing satellite Video Super-Resolution (VSR) methods focus on improving the network’s presentation ability, overlooking the complex degradation processes present in real-world satellite videos which appear as a blind SR problem. In this paper, we propose an effective satellite VSR method based on a unidirectional recurrent network named URD-VSR. Simultaneously, a network independent of the SR structure is utilized to model the degradation process. Experiments on real satellite video datasets and integration with object detection demonstrate the effectiveness of the proposed method.

Semantic segmentation of remote sensing images is crucial for various applications, including land use mapping and environmental monitoring. However, most CNNs lack the ability to capture long-range context due to their limited receptive fields. While Transformers adopt multi-head self-attention mechanism to capture long-range context for better accuracy, it often leads to high parameter volume and computational complexity. In this paper, we propose PCNeXt, a lightweight pure-convolutional neural network for semantic segmentation of remote sensing images. For the encoder part, we design a pure-convolutional lightweight module named MSWCA based on MSCA, which is the key component of the encoder of SegNeXt. For the decoder part, we design a purely Convolutional Global-Local Block (CGLB) to replace the GLTB module of UNetFormer, which utilizes the MSWCA module instead of self-attention to capture global context and a simple convolutional operation to capture local context. The experiments prove that instead of using self-attention in transformer, the proposed PCNeXt is capable of achieving competitive accuracy by using only convolutional structure. Notably, our PCNeXt achieves 91.53% aAcc and 83.9 % mIoU on the Potsdam dataset, with only 4.42MB parameters and 6.87G FLOPs calculations.

The traditional on-board software architecture lacks adaptability and scalability, with a high degree of customization, significant software-hardware coupling, and slow technological evolution. To enhance utilization efficiency and intelligent application capabilities of satellite, this paper proposes an application-oriented on-board software management system for micro-nano satellite. By analyzing the research status and technical characteristics of traditional software architectures, an open software management system based on Linux is implemented, including key components and functions. In line with the system’s operational mode, we propose a method for on-board software dynamic refactoring. Furthermore, we validate the effectiveness of the software management system by a remote sensing image processing algorithm, which has a wide application prospect.

Traditional object detection requires extensive annotation, consuming considerable time and manpower. In recent years, semi-supervised object detection (SSOD) methods, which utilize a blend of unlabeled and labeled data to train object detectors, have been extensively researched. SSOD has made significant progress and can achieve similar levels of accuracy as the fully supervised methods. However, existing SSOD approaches primarily focus on horizontal objects in natural scenes, with scant research on other scenarios such as remote sensing images. This paper proposes a novel semi-supervised oriented object detection algorithm based on a two-stage object detector. For oriented objects in remote sensing, we particularly emphasize the utilization of oriented information of remote sensing objects to generate precise pseudo-label and improve the learning capability of the student network. Iterative labeled data filtering is performed by incorporating metrics with designed measurements. Valuable annotated samples can enhance the quality of pseudo-label generation. A strong augmentation method has been designed to utilize rotational information, enabling the student network to learn more diverse features. Additionally, we investigate the long-tail distribution problem in remote sensing images and mitigate the bias brought by category imbalance through phased training and post-processing in detection. Our experiments demonstrate that the designed semi-supervised oriented object detection method surpasses existing methods in the DOTAv1.5 benchmark, culminating in state-of-the-art performance.

Semantic segmentation of remote sensing images (RSIs) plays a crucial role in various applications, including urban planning and environmental monitoring. However, the high cost and complexity of obtaining detailed annotations for RSIs pose significant challenge. This issue necessitates the exploration of weakly supervised learning as an effective alternative, which utilizes more readily available, less granular forms of labeling. Yet, weakly supervised approaches face their own set of challenges, primarily due to scarcity of precise pixel-level labels which significantly hampers the model’s ability to learn accurate representations. In this article, we introduce a weakly supervised semantic segmentation (WSSS) approach for RSIs that leverages self-supervised learning (SSL) and pseudo-label noise mitigation to address these challenges. Our method leverages a self-supervised encoder for providing similarity information, which enhances feature representation in RSIs and enables the generation of more accurate pseudo-labels, thus reducing the noise in the pseudo-labels. Furthermore, we propose a refined loss function that incorporates gradient clipping and label smoothing to mitigate the impact of noisy labels, thereby improving the robustness and accuracy of the segmentation results. Extensive experiments on the ISPRS Potsdam, ISPRS Vaihingen, and iSAID datasets demonstrate that our approach achieves state-of-the-art (SOTA) performance, closely matching that of fully supervised methods. Our method not only reduces the dependency on expensive pixel-level annotations but also showcases the potential of SSL in enhancing WSSS tasks.

Despite recent advances in object detection, closed-set detectors with fixed training classes often overlook or misclassify unannotated objects during testing. To address this, open world object detection (OWOD) algorithms identify and label these objects as unknown, better aligning with real-world scenarios and human learning. However, remote sensing images, with their arbitrary object orientations and large interclass feature disparities, pose significant challenges for these algorithms. To tackle this, we propose OPODet, an Open-world Potential Oriented object Detection framework for remote sensing images. Specifically, we incorporate the oriented unknown-aware region proposal network (OUA-RPN) into traditional oriented object detection models, enabling the network to predict potential oriented objects. To address the significant interclass feature differences among potential unknown classes, we propose a multiunknown-class clustering aligning prototype (MCAP) learning method to prevent feature collapse in the feature space. In addition, to address the lack of rotation information for potential objects, we introduce a rotation potential target consistency (RPTC) algorithm to impose explicit rotation constraints for generating more accurate potential unknown proposals. Extensive experiments on DIOR-R, DOTA-v1.0, and HRSC2016 datasets demonstrate the effectiveness of our approach in detecting potential oriented objects.

In recent years, the implementation of infrared small-target detection using convolutional neural networks (CNNs) has garnered widespread attention due to its high performance. Since this issue is often addressed through fully supervised image segmentation networks, the training process requires significant human effort and time to annotate pixel-level mask labels. Consequently, employing single-point supervision as a form of weak supervision for model training has aroused widespread interest in saving annotation costs. However, the class imbalance issue caused by single-point supervision in the early stages of training, along with inaccuracies in pseudo-label updates, and the difficulty in achieving convergence simultaneously, have made it challenging for such methods to attain satisfactory performance. In this article, we introduce a label evolution framework based on local contrast measure (LELCM) to address these issues. Before training, we expand the single-point labels into initial pseudo-labels based on the inherent information of the targets, which mitigates the problem of class imbalance. Furthermore, in the process of updating pseudo-labels, we employ a strategy that utilizes confidence contrast for updates, not only enabling more stable updates of pseudo-labels based on target characteristics but also facilitating adaptive cessation of updates. Our experimental results reveal that our approach not only attains target detection rates (Pd) on par with full supervision models but also achieves 80% of the full supervisory effect in terms of intersection over union (IoU).

Ship detection in optical remote sensing (RS) images remains a persistent challenge in current research. While prevailing methods achieve satisfactory outcomes in detecting large ship objects within RS images, the identification of small ship objects poses greater difficulty due to their limited pixel information. To address this challenge, the utilization of generative adversarial network-based (GAN-based) super-resolution (SR) techniques proves effective. Therefore, in this article we present a high-resolution feature generator (HRFG) specifically tailored for small ship detection. Different from previous GAN-based methods which rely on image-level SR or feature sharing between SR and detection, we design a new architecture that uses an additional network branch, i.e., high-resolution feature extractor (HRFE), to extract real high-resolution (HR) feature as a feature-level supervisory signal. The intuition is that real HR features may guide the generator network to extract HR feature from low-resolution (LR) image directly. Consequently, the feature for detection is extracted and enhanced at the same time so that large amount of calculation brought by image-level SR is avoided. Additionally, we introduce a background degradation strategy within the HRFE to improve the performance of small object recognition. Extensive experiments on a self-assembled ship dataset and two other public datasets show superiority of the proposed method in small ship detection task.

Unsupervised super-resolution aims to enhance the quality of images without high-resolution (HR) labels during the training stage, making it applicable to real-world scenarios. However, unsupervised super-resolution methods face the challenge of effectively learning the internal structure of images due to the absence of high-quality HR images as references. Moreover, multi-spectral remote sensing images often contain stochastic features caused by cloud and fog occlusions. These occlusions make it difficult to achieve accurate reconstruction of occluded areas through direct modeling of deep features in multi-spectral images. In this paper, we propose a method inspired by conditional variational autoencoders to address the issue of stochastic features in unsupervised multi-spectral super-resolution. Additionally, we introduce a channel attention feature fusion module to combine two types of features. We evaluated our unsupervised multi-spectral image super-resolution method using a real satellite remote sensing dataset. Experimental results demonstrate the qualitative and quantitative effectiveness of our approach.

Semantic segmentation of high-resolution remote sensing images (RSIs) is developing rapidly. Multispectral images can provide rich spectral information for semantic segmentation, while 3D LiDAR point cloud data can provide depth information. Thus, semantic segmentation accuracy could be improved by fusing multispectral images and 3D LiDAR point cloud. In this paper, we propose a method titled Direct LiDAR-Aerial Fusion Network (DLAFNet) which directly uses RSIs and LiDAR point cloud for semantic segmentation tasks. In particular, owing to the fact that sparse features extracted from the KPConv branch are not as essential as features from RSIs, we design LiDAR Assisted Attention Module (L-AAM). Our experiments on the modified GRSS18 dataset prove that our method is proper and can obtain the best results by comparing with its components and other methods.

The emergence of Semi-Supervised Object Detection (SSOD) techniques has led to notable improvements in object detection capabilities by leveraging a restricted quantity of labeled data and a copious amount of unlabeled data. However, there are two challenging issues that need to be addressed in remote sensing images. Firstly, the complex background and large variation in target scales in remote sensing images can result in poor quality of pseudo-labels. Secondly, the long-tailed distribution problem, where some categories have a large number of instances while others have very few, is also common in remote sensing images. In this paper, we address SSOD in remote sensing images characterized by a long-tailed distribution. We propose an active learning strategy for selecting labeled data in the process of semi-supervised learning. The model training is decoupled into the training of backbone and detector. This idea contributes to favorable improvement in the regression branch and our method can achieve significant results on DOTA-v1.0 dataset.

The rapid development of remote sensing technology has provided convenient conditions for obtaining abundant research data. The use of visible light remote sensing images for ship detection has profound significance in the fields of port management, maritime rescue, and military investigation. Our paper focuses on the shortcomings of current way of ship positioning expression and uses deep learning to study the rotated ship detection task. A kind of ship detection algorithm based on dense points related to the position of ships is proposed to solve the problem of angle boundary and vertex sorting ambiguity in current rotated object detection methods. Our method achieves 92.4 mAP on the HRSC2016 dataset, ranking among the top in similar studies.

In contrast to natural objects, aerial targets are usually non-axis aligned with arbitrary orientations. However, mainstream weakly supervised object detection (WSOD) methods can only predict horizontal bounding boxes (HBBs) from existing proposals generated by offline algorithms. To predict oriented bounding boxes (OBBs) for aerial targets while testing images end-to-end without proposals, WSODet is designed leveraging on layerwise relevance propagation (LRP) and point set representation (RepPoints). To be specific, based on the mainstream WSOD framework, LRP on multiple instance learning branch (MIL-LRP) is conducted to decrease the uncertainty and ambiguity of feature map. Then, a pseudo oriented label generation algorithm is designed to obtain OBB pseudolabels, which serve as supervision to train an oriented RepPoint Net under the guidance of improved oriented loss function (IOLF). During the test, input images are sent to oriented RepPoint branch (ORB) to obtain OBB predictions without proposals. Extensive experiments on the detection in optical remote sensing images (DIOR), Northwestern Polytechnical University (NWPU) VHR-10.v2, and HRSC2016 datasets demonstrate the effectiveness of our method to predict precise oriented aerial objects, achieving 22.2%, 46.5%, and 43.3% mAP, respectively. Moreover, training jointly with ORB boosts the results of the original WSOD framework compared with the existing WSOD methods even if there is no specific design for the original structure.

In this paper we propose a novel model towards multi-frame super-resolution, which leverages multiple RAW images and yields a super-resolved RGB image. To facilitate the pixel misalignment in burst photography, we apply a refined Pyramid Cascading and Deformable Convolution (PCD) feature alignment module. A new 3D deformable convolution fusion module is proposed subsequently to merge the information from all frames adaptively. In addition, we employ an encoder-decoder network to restore color and details in sRGB space after super-resolving images in linear space. Extensive experiments demonstrate the superiority of our architecture and the strength of multi-frame super-resolution with RAW images.

The integration technology on micro-nano satellite for high performance remote sensing of H4L (high resolution, high coverage, high intelligence, high applicable flexibility and low cost) is discussed in this paper. The integration of information, energy and mechatronics in platform and payload is the key problem. The information flow processing of platform and payload, the efficient and dynamic management of energy flow, the micro-nano functional units and the multi-functional structure slabs are adopted to achieve high level integration and functionality. Payloads include the optical remote sensing payload unit and micro sensors for space environment detection. The satellite is capable of on-board processing and rapid information transmission, which can improve the flexibility of satellite applications. The techniques presented in this paper can be applied to low-cost constellation networking.

Thin cloud removal from remote sensing (RS) images is challenging. Recently, deep-learning-based methods have achieved excellent results using supervised training on paired image data. However, in practice, real paired image data are unavailable. Therefore, in this letter, we propose a novel thin cloud removal method, a physical-model-based CycleGAN (PM-CycleGAN), which can be trained using only unpaired data. The PM-CycleGAN training process comprises forward and backward loops. The forward loop first decomposes a cloudy image into a cloud-free image, thin cloud thickness map, and thickness coefficient using three generators. Then, it combines these three components using a physical model to reconstruct the original cloudy image to obtain the cycle consistency constraint. The backward loop first uses the physical model to synthesize a cloud-free image, thin cloud thickness map, and thickness coefficient into a cloudy image, which are then decomposed into the original three components using the three generators. Visual and quantitative comparisons against several state-of-the-art (SOTA) methods on a cloudy image dataset demonstrated the superiority of PM-CycleGAN.

Convolution neural networks (CNNs) and graph representation learning are two common methods for hyperspectral image (HSI) classification. Recently, graph convolutional neural networks, a combination of CNN and graph representation learning, have shown great potential in the HSI classification problem. However, the existing graph convolution network (GCN)-based methods have many problems, such as overdependence on the adjacency matrix, usage of a single modal feature, and lower accuracy than the mature CNN method. In this article, we propose a feature fusion hypergraph neural network (F2HNN) for HSI classification. F2HNN first generates hyperedges from features of different modalities to construct a hypergraph representing multimodal features in HSI. Then, the HSI and the extracted hypergraph are input into the hypergraph convolutional neural network for learning. In addition, we propose three feature fusion strategies. The first strategy is the most basic spatial and spectral feature fusion. The second strategy fuses the spectral features extracted by a pretrained multilayer perceptron (MLP) with the spatial features to reduce the redundant information of the original spectral features. The third strategy uses the fusion of CNN features, spectral features, and spatial features to explore the capabilities of F2HNN. Sufficient experiments on four datasets have proved the effectiveness of F2HNN.

Single image super-resolution (SISR) is an important procedure to improve many remote sensing applications. Global features play an important role in pixel generation of SISR. In this paper, we proposed a self-attention fusion module named as SAF module which combines spatial attention and channel attention in parallel to handle this problem. Our self-attention fusion module can be flexibly added to many popular deep-learning-based SISR models to further improve their representation ability and learn global features. Experiments on UC Merced dataset indicate that SAF module can improve the performance of classic SISR models and achieve state-of-the-art super-resolution results.

Micro-nano satellite has developed rapidly in recent years. It shares the advantages of small size, low power consumption, short development cycle, and the ability to complete complex space missions at a lower cost, which is important for scientific research, national defense and commercial fields. The great capacity of imaging data brings heavy pressure to the satellite-earth data transmission, therefore it is necessary and significant to develop intelligent processing on the satellite to transform remote sensing images into concerned information. In this paper, we introduce an integration technology on micronano satellite with high performance of resolution, coverage, intelligence, applicable flexibility and low cost. The designed micro-nano satellite is deployed with a small optical payload. The imaging capability is 5k pixels x5k pixels and the resolution is 1.6m@500km. We apply Commercial Off-The-Shelf (COTS) component equipped with deep learning algorithm to extract the information of target, including position, size, course, etc. We evaluate the proposed method on the ship detection task. The experimental results have indicated a great prospect for on-board intelligent processing on the micro-nano satellite.

Single image super-resolution (SISR) is to recover the high spatial resolution image from a single low spatial resolution one, which is a useful procedure for many remote sensing applications. Most previous convolutional neural network (CNN)-based methods adopt supervised learning. However, paired high-resolution and low-resolution remote sensing images are actually hard to acquire for supervised learning SR methods. To handle this problem, we propose a novel cycle convolutional neural network (Cycle-CNN). Our network consists of two generative CNNs for down-sampling and SR separately and can be trained with unpaired data. We perform comprehensive experiments on panchromatic and multispectral images of the GaoFen-2 satellite and the UC Merced land use data set. Experimental results indicate that our method achieves state-of-the-art CNN-based SR results and is robust against noise and blur in remote sensing images. Comprehensively considering super-resolved image quality and time costs, our proposed method outperforms the compared learning-based SISR approaches.

Color consistency is crucial to developing robust deep learning methods for histopathological image analysis. With the increasing application of digital histopathological slides, the deep learning methods are probably developed based on the data from multiple medical centers. This requirement makes it a challenging task to normalize the color variance of histopathological images from different medical centers. In this paper, we propose a novel color standardization module named stain standardization capsule based on the capsule network and the corresponding dynamic routing algorithm. The proposed module can learn and generate uniform stain separation outputs for histopathological images in various color appearance without the reference to manually selected template images. The proposed module is light and can be jointly trained with the application-driven CNN model. The proposed method was validated on three histopathology datasets and a cytology dataset, and was compared with state-of-the-art methods. The experimental results have demonstrated that the SSC module is effective in improving the performance of histopathological image analysis and has achieved the best performance in the compared methods.

The development of whole slide imaging techniques and online digital pathology platforms have accelerated the popularization of telepathology for remote tumor diagnoses. During a diagnosis, the behavior information of the pathologist can be recorded by the platform and then archived with the digital case. The browsing path of the pathologist on the WSI is one of the valuable information in the digital database because the image content within the path is expected to be highly correlated with the diagnosis report of the pathologist. In this article, we proposed a novel approach for computer-assisted cancer diagnosis named session-based histopathology image recommendation (SHIR) based on the browsing paths on WSIs. To achieve the SHIR, we developed a novel diagnostic regions attention network (DRA-Net) to learn the pathology knowledge from the image content associated with the browsing paths. The DRA-Net does not rely on the pixel-level or region-level annotations of pathologists. All the data for training can be automatically collected by the digital pathology platform without interrupting the pathologists' diagnoses. The proposed approaches were evaluated on a gastric dataset containing 983 cases within 5 categories of gastric lesions. The quantitative and qualitative assessments on the dataset have demonstrated the proposed SHIR framework with the novel DRA-Net is effective in recommending diagnostically relevant cases for auxiliary diagnosis. The MRR and MAP for the recommendation are respectively 0.816 and 0.836 on the gastric dataset. The source code of the DRA-Net is available at https://github.com/zhengyushan/dpathnet.

Single image super-resolution (SISR) is a useful procedure for many remote sensing applications. However, paired high-resolution and low-resolution remote sensing images are actually hard to acquire for supervised learning SR methods. In this paper, we propose an unsupervised network named Cycle-CNN to handle this problem. Our network consists of two generative CNNs for down-sampling and super-resolution separately, and can be trained with unpaired data. Experiments on panchromatic and multi-spectral images of GaoFen-2 satellite indicate that our method achieves state-of-the-art SR results and is robust against noise and blur in the remote sensing images.

Inshore ship detection is a popular research domain for optical remote sensing image understanding with many applications in harbor management. However, recent approaches on inshore ship detection depend heavily on hand-crafted features, which need a complicated procedure. In this paper, we propose a new method to achieve inshore ship detection based on Mask R-CNN. We introduce Soft-Non-Maximum Suppression (Soft-NMS) into our framework to improve the robustness to nearby inshore ships. Both battleships and merchantships can be detected in our framework. Furthermore, our framework can also obtain the binary masks of inshore ships. Experimental results on a dataset collected from Google Earth have quantitatively and qualitatively demonstrated the effectiveness of our approach.

In this paper, a novel solution is introduced for accurate subpixel star centroiding of the focused geostationary Earth observing sensor on a three-axis stabilized satellite. A small 2-dimensional array is utilized to better capture the star spot than the linear array of detectors. The popular center of mass method is used to compute star centroid in a single frame. Then the subpixel accuracy of star centroiding can be improved by fitting the linear trajectory of the observed star according to exact imaging time produced from time awarding system of the satellite. Experimental results on simulated star images in various conditions validate the effectiveness and robustness of our star centroiding method.

Convolutional neural network obtains remarkable achievements on target detection, due to its prominent capability on feature extraction. However, it still needs further study for aircraft detection task, since intraclass variation still restricts the accuracy of aircraft detection in remote sensing images. In this letter, we adopt regularity of aircraft circle response to design our end-to-end fully convolutional network (FCN), and embed online exemplar mining into our network to handle intraclass variation. The mined exemplars are employed to capture different intraclass characteristics, which effectively reduces the burden of network training. Specifically, we first select basic exemplars based on labeled information and initialize the relationships between exemplars and aircraft examples. Then, these relationships will be updated by the similarity of these examples in high-level features space. Finally, aircraft examples will be used to train different exemplar detectors according to updated relationships. Motivated by the geometric shape of aircraft, a circle response map is developed to construct our FCN to achieve more efficient aircraft detection. The comparative experiments indicate that superior performance of our network in accurate and efficient aircraft detection.

Content-based image retrieval (CBIR) has been widely researched for histopathological images. It is challenging to retrieve contently similar regions from histopathological whole slide images (WSIs) for regions of interest (ROIs) in different size. In this paper, we propose a novel CBIR framework for database that consists of WSIs and size-scalable query ROIs. Each WSI in the database is encoded into a matrix of binary codes. When retrieving, a group of region proposals that have similar size with the query ROI are firstly located in the database through an efficient table-lookup approach. Then, these regions are ranked by a designed multi-binary-code-based similarity measurement. Finally, the top relevant regions and their locations in the WSIs as well as the corresponding diagnostic information are returned to assist pathologists. The effectiveness of the proposed framework is evaluated on a fine-annotated WSI database of epithelial breast tumors. The experimental results have proved that the proposed framework is effective for retrieval from database that consists of WSIs. Specifically, for query ROIs of 4096 4096 pixels, the retrieval precision of the top 20 return has reached 96% and the retrieval time is less than 1.5 s.

Histopathological image classification (HIC) and content-based histopathological image retrieval (CBHIR) are two promising applications for the histopathological whole slide image (WSI) analysis. HIC can efficiently predict the type of lesion involved in a histopathological image. In general, HIC can aid pathologists in locating high-risk cancer regions from a WSI by providing a cancerous probability map for the WSI. In contrast, CBHIR was developed to allow searches for regions with similar content for a region of interest (ROI) from a database consisting of historical cases. Sets of cases with similar content are accessible to pathologists, which can provide more valuable references for diagnosis. A drawback of the recent CBHIR framework is that a query ROI needs to be manually selected from a WSI. An automatic CBHIR approach for a WSI-wise analysis needs to be developed. In this paper, we propose a novel aided-diagnosis framework of breast cancer using whole slide images, which shares the advantages of both HIC and CBHIR. In our framework, CBHIR is automatically processed throughout the WSI, based on which a probability map regarding the malignancy of breast tumors is calculated. Through the probability map, the malignant regions in WSIs can be easily recognized. Furthermore, the retrieval results corresponding to each sub-region of the WSIs are recorded during the automatic analysis and are available to pathologists during their diagnosis. Our method was validated on fully annotated WSI data sets of breast tumors. The experimental results certify the effectiveness of the proposed method.

The geometrical characteristic and low-level manually designed features are usually used to detect airports in optical remote sensing images. But it is insufficient to describe airport in low resolution and illumination environment. This paper presents a hard example mining algorithm to train the end-to-end deep convolutional neural network for airport detection in complex situation. Compared with conventional airport detection methods which design specific low-level manually designed features for high-resolution remote sensing images, an end-to-end network can mine the general characteristic among the training samples and learn high-level features in multi-scale and multi-view remote sensing images. Meanwhile, an automatic hard example mining principle is introduced to make training more efficiently and accurately. The proposed method is validated on a multi-scale and multi-view dataset collected from Google Earth. The experimental results demonstrate that the proposed method is robust and efficient, and superior to the state-of-the-art airport detection models.

Remote sensing images are with the characteristics of large width and sparse distribution of specific targets, so that the extraction of region proposal is necessary before detection. In this paper, we propose a new ship detection method on sea-background remote sensing images, which are generally influenced by clouds, waves and other inhomogeneities. Instead of exhaustive search, the core of our method is that the region proposals are obtained from edge detection based on structured forests, which makes our method accurate and efficient. This edge detection method only demands a small training set and then produces contours with the background suppressed. After some morphological processing on the contours, we obtained ship proposals by connected domain detection. Adopting support vector machine(SVM) as classifier, we finally acquire ship detection results. The remote sensing images in our datasets are downloaded from Google Earth map. In our experiments, the proposed method is feasible and effective, and it shows better performance than other methods especially in various illumination and interference conditions.

The persistent haze weather in North China has aroused extensive attention to environmental protection. Among all pollution resources, the anthropogenic emission by fossil fuel power plants plays an important role. To assist the environmental protection administration monitoring fossil fuel power plants, we propose an effective approach in this paper to learn an integrated model for chimney and condensing tower detection based on Faster R-CNN in high resolution remote sensing images. Our method can detect chimneys and condensing towers under different imaging condition efficiently and accurately. Experimental results on a self-collected dataset demonstrate the effectiveness of the proposed method.

In the field of pathology, whole slide image (WSI) has become the major carrier of visual and diagnostic information. Content-based image retrieval among WSIs can aid the diagnosis of an unknown pathological image by finding its similar regions in WSIs with diagnostic information. However, the huge size and complex content of WSI pose several challenges for retrieval. In this paper, we propose an unsupervised, accurate, and fast retrieval method for a breast histopathological image. Specifically, the method presents a local statistical feature of nuclei for morphology and distribution of nuclei, and employs the Gabor feature to describe the texture information. The latent Dirichlet allocation model is utilized for high-level semantic mining. Locality-sensitive hashing is used to speed up the search. Experiments on a WSI database with more than 8000 images from 15 types of breast histopathology demonstrate that our method achieves about 0.9 retrieval precision as well as promising efficiency. Based on the proposed framework, we are developing a search engine for an online digital slide browsing and retrieval platform, which can be applied in computer-aided diagnosis, pathology education, and WSI archiving and management.

Conditional Random Field(CRF) has been successfully applied to the hyperspectral image classification. However, it suffers from the availability of large amount of labeled pixels, which is labor- and time-consuming to obtain in practice. In this paper, a semi-supervised CRF(ssCRF) is proposed for hyperspectral image classification with limited labeled pixels. Laplacian Support Vector Machine(LapSVM), after extended into the composite kernel type, is defined as the association potential. And the Potts model is utilized as the interaction potential. The ssCRF is evaluated on the two benchmarks and the results show the effectiveness of ssCRF.

Ship target detection in optical remote sensing images has attracted more and more attention in the field of remote sensing. The ship target detection technology of optical remote sensing images is vulnerable to many factors, while the real data are difficult to contain various elements. In order to obtain the various situations in the large sea scenes, we develop a simulation system for high-resolution optical remote sensing image of ship targets. The simulated images with different sea states, cloud conditions, target types and imaging conditions can support the evaluation and comparison of ship detection algorithms as well as other tasks in remote sensing image analysis.

Object recognition is a key precursory challenge in the fields of object manipulation and robotic/AI visual reasoning in general. Recognizing object categories, particular instances of objects and viewpoints/poses of objects are three critical subproblems robots must solve in order to accurately grasp/manipulate objects and reason about their environ- ments. Multi-view images of the same object lie on intrinsic low-dimensional manifolds in descriptor spaces (e.g. visual/depth descriptor spaces). These object manifolds share the same topology despite being geometrically different. Each object manifold can be represented as a deformed version of a unified manifold. The object manifolds can thus be parametrized by its homeomorphic mapping/reconstruction from the unified manifold. In this work, we construct a manifold descriptor from this mapping between homeomorphic manifolds and use it to jointly solve the three challenging recognition sub-problems. We extensively experiment on a challenging multi-modal (i.e. RGBD) dataset and other object pose datasets and achieve state-of-the-art results.

Acquiring optical images is a basic task of space based surveillance system. However, these images are hard to obtain in real space condition or are classified for security reason. To solve this problem, a simulation system, based on STK and OpenGL, is established. Using data generated by STK, 3D models of satellites and a star catalogue, we can render the celestial background and space object in OpenGL. Then some post-processing is done to the acquired images to make the final results look real. The simulation results, with high reality and fidelity, can provide data support for space information processing technology, such as object detection, tracking, recognition or for the space surveillance system design.