Fine-grained ship recognition in remote sensing imagery is essential for maritime applications. However, its development is hindered by two challenges: 1) the limited granularity of existing ship detection datasets, and 2) the disturbance of complex maritime conditions as well as the arbitrary ship orientations and distributions. To address the first issue, we annotated a large-scale fine-grained ship instance detection dataset (LAFI), comprising 48,717 ship instances worldwide with 49 categories. To tackle the challenges of marine disturbance and diverse ship status, we proposed a controllable generative knowledge-driven ship detection framework (COSD). It employs a controllable diffusion model guided by ship-marine textual prompt to generate millions of synthetic images that not only preserve ship structures but also cover diverse sea and weather conditions for robust pretraining. The pretraining stage then utilizes masked reconstruction to learn component-level cues under occlusion, clutter, fog, and illumination changes. Furthermore, a heterogeneous feature alignment decoder is designed to align multi-modal metrics of orientation and distribution features in the latent space, allowing for accurate representation of diverse ship status. Extensive experiments on two benchmark datasets showed that our method respectively increased 0.011 and 0.030 mean average precision (mAP@50) over SOTA methods, particularly in scenarios involving small, densely packed and arbitrary oriented ships.

Auto-segmentation quality or accuracy influences their clinical usefulness. However, currently widely utilized segmentation metrics (e.g., Dice Coefficient (DC) and Hausdorff Distance (HD)) cannot effectively express the manual mending effort required when utilizing auto-segmentation results in clinical practice. In this article, we explore ways of evaluating auto-segmentations with clinical efficiency considerations in mind. The time required for correcting auto-segmentations by experts is recorded to indicate ground-truth mending effort. Extended from our previous work, five explicitly-defined metrics are studied in detail for their ability to predict mending effort. More importantly, we explore the use of deep learning networks to provide an implicit metric, which predict mending effort using auto-segmentation masks and original images as input. A 3-institution evaluation is conducted with 7 different anatomic organs in the setting of auto-contouring for radiation therapy planning. Among the five explicit metrics, one form of the proposed Mendability Index (MIhd) shows the best performance to indicate the mending effort for sparse objects with 6.2-14.4% error, while one form of HD (sHD) performs best when assessing large non-sparse objects. Interestingly, while the explicit metrics all require ground truth segmentations for estimating mending effort, the implicit models obtained via deep learning are effective in predicting mending efforts (with 2.9-12.9% error) without the need for ground-truth segmentations and directly from the given image plus the auto-segmentations. We conclude that once effort-predicting deep models are created, it is feasible to assess the clinical usability of new segmentation models, going beyond bench technical evaluation commonly done via explicit metrics.

Intraplaque neovascularization (IPN) within carotid plaque is a crucial indicator of plaque vulnerability. Contrast-enhanced ultrasound (CEUS) is a valuable tool for assessing IPN by evaluating the location and quantity of microbubbles within the carotid plaque. However, this task is typically performed by experienced radiologists. Here we propose a deep learning-based multi-task model for the automatic segmentation and IPN grade classification of carotid plaque on CEUS images and videos. We also compare the performance of our model with that of radiologists. To simulate the clinical practice of radiologists, who often use CEUS videos with dynamic imaging to track microbubble flow and identify IPN, we develop a workflow for plaque vulnerability assessment using CEUS videos. Our multi-task model outperformed individually trained segmentation and classification models, achieving superior performance in IPN grade classification based on CEUS images. Specifically, our model achieved a high segmentation Dice coefficient of 84.64% and a high classification accuracy of 81.67% . Moreover, our model surpassed the performance of junior and medium-level radiologists, providing more accurate IPN grading of carotid plaque on CEUS images. For CEUS videos, our model achieved a classification accuracy of 80.00% in IPN grading. Overall, our multi-task model demonstrates great performance in the automatic, accurate, objective, and efficient IPN grading in both CEUS images and videos. This work holds significant promise for enhancing the clinical diagnosis of plaque vulnerability associated with IPN in CEUS evaluations.

Fraud is increasingly prevalent, and its patterns are frequently changing, posing challenges for fraud detection methods such as random forests and Graph Neural Networks (GNNs), which rely on bin-based and mixture features separately. The former may lose crucial graph-associated features, while the latter face incorrect feature fusion. To overcome these limitations, we propose an approach based on attribute-association pattern that leverages the distinct attribute and association patterns differentiating fraudulent from benign behaviors, to enhance fraud detection capabilities. Attribute features are adaptively split into separate bins to eliminate incorrect attribute fusion and combine association patterns through graph neighbor message passing, thereby deriving attribute-association pattern features. Using the learned attribute-association patterns, the fraud patterns between a single pattern and the patterns across the entire graph are globally aggregated. Extensive experiments comparing our approach with 24 methods on 7 datasets demonstrate that the proposed method achieves SOTA performance.

Performing surgical tasks safely and reliably presents significant challenges, including obstacle avoidance, joint limit constraints, and motion smoothness during the tool-target alignment (T-TA) stage, as well as precise tracking of preoperative plans during the execution of the preoperative planning surgery path (EPSP). The traditional inverse kinematics methods fall short in addressing these complex motion planning and control issues within the unstructured and time-varying surgical environment. Therefore, a novel spatial–temporal varying primal–dual neural network (STV-PDNN) that incorporates guided obstacle avoidance and null space optimization to address spatial–temporal constraints during surgery is proposed. Firstly, a velocity control quadratic programming (QP) framework based on target distance and orientation metrics is constructed by considering the relationships among the surgical robot, the environment, and the surgical target. Then, the STV-PDNN enables real-time problem-solving across two specific stages, employing velocity vector projection for obstacle avoidance and joint space obstacle avoidance velocity superposition to enhance the obstacle avoidance guidance. Furthermore, the joint null space optimization and maximum manipulability, along with a preoperative planning path velocity feed-forward and feedback velocity control mechanism, are integrated into the STV-PDNN structure. The improvement facilitates smoother, lower-energy joint movements and effective motion singularity avoidance during the T-TA stage, as well as precise motion control in the EPSP stage. The experiments conducted on the redundant robot Diana7 Med validate the effectiveness of the proposed method in autonomously executing T-TA and EPSP for pedicle screw implantation, offering a promising solution for the task autonomy of surgical robot.

Benefiting from long-term time series and large spatial coverage, Sentinel-2 has been widely used in urban tree cover retrieval. However, mixed pixel effects in Sentinel-2 imagery make it challenging to accurately identify urban tree covers. To address this problem, Sub-Pixel Mapping (SPM) is developed to reconstruct a high-resolution urban tree cover from medium-resolution imagery. While deep-learning-based SPM seeks fine-grained patterns solely within medium-resolution feature spaces and spatiotemporal fusion-based SPM leverages additional high-resolution imagery from different times at the same location, both face limitations: the former lacks detailed spatial constraints, and the latter struggles with acquiring geographically aligned imagery. To address these challenges, this study proposes a Geographic Prior guided Sub-pixel Mapping (GPSPM) approach for urban tree cover reconstruction. The geographic prior is grounded in the scaling law of geography, a fundamental principle of spatial heterogeneity stating that high-resolution imagery contains far more detailed features (e.g., small tree parcels) than lower-resolution imagery. These fine-grained features enhance SPM by providing robust cross-scale spatial prior based on a “teacher-student” domain adaptation training framework. Besides, considering the geometric feature discrepancy and long-tail distribution exists across different geographic scales, cross-scale image mosaicking and resampling strategy are further developed. Experiments on public urban tree cover dataset demonstrate that the proposed method improves the Intersection over Union (IoU) of urban tree cover by approximately 5% compared to traditional unsupervised SPM and shows significant improvements in spatial detail quality.

Subpixel mapping (SPM) is a crucial technique in remote sensing imagery analysis, aimed at characterizing subpixel distribution within the mixed pixels. Traditional SPM methods and convolutional neural network (CNN)-based SPM methods primarily rely on local spatial autocorrelation, which limits their ability to capture long-range dependencies between distant locations or objects. To address this limitation, we propose a global-local spatial dependence integrator for the SPM method (GLSDSPM) that employs both CNN and the vision transformer as dual-path structures to model global context and local spatial dependencies efficiently. Besides, the previous SPM methods often struggle to accurately reconstruct high-quality spatial patterns for linear features, such as slender rivers and roads, due to insensitivity to textures and sharp, high-frequency details. To overcome this challenge, we integrate a linear pattern refinement module (LPRM) into GLSDSPM, which adaptively focuses on thin and long local structures to accurately capture high-frequency features and detailed information. Two experiments conducted on the Pavia and Houston hyperspectral images prove that the proposed method achieves superior performance, outperforming the state-of-the-art by 4.13% and 3.95% in overall accuracy (OA), respectively.

With the widespread application of remote sensing images in military and civilian fields, remote sensing object detection (RSOD) has become an important research direction. However, limited generalization and complex background interference have long been persistent challenges that hinder the development of RSOD. To address these issues, we propose a Pre-trained Scene-aware Object Detection Network (PSODNet). Firstly, we design an Enhanced Object Network (EON), which leverages a multi-head pretraining strategy to jointly train data from diverse sources, thereby expanding the scale of dataset and improve the generalization ability. Secondly, we introduce scenario-object relationship module to learn a multi-scale relationship map between objects and scenes, which is used to constrain the solution space of object detection, thereby enhancing performance in complex scenarios. Lastly, by using Label Smoothing Loss, PSODNet leverages mutual information of label to prevent extreme distributions of classification probabilities and reduce the risk of overfitting. In the experiment part, PSODNet was pretrained on multiple datasets and then fine-tuned on three datasets for validation. Results on three public datasets demonstrate that PSODNet outperforms existing models in detection performance by up to 2.4%, achieving a maximum mAP of up to 95%. Through visual interpretation of the relationship map, we found that PSODNet is able to bridge the semantic relevance between objects and scenes, demonstrating its potential in object detection in complex scenarios. Code is available at: https://github.com/creature-compound/PSODNet.

Convolutional-based sub-pixel mapping (SPM) is a cutting-edge approach to solve the mixed pixel problem in land cover mapping. However, the feature modeling process of convolutional neural networks (CNN) lacks interpretability, and thus, it is difficult to learn spatial correlation that is helpful for sub-pixel location reasoning, and usually fail in reconstruction of fragmented land parcels. Therefore, this study constructs a SPM network that combines data-driven and model-driven approaches (DEMON). It uses nested self-attention mechanisms to simulate spatial dependency modeling, which learns spatial correlations between sub-pixels and pixels of different land covers, the learned spatial correlations can then be used to explicitly infer the spatial positions to achieve interpretability. Three public datasets are used for validation, and we found that our proposed method outperforms SOTA methods by circa 6%, and the visualized spatial correlations verifies the interpretability of the model.

Subpixel mapping (SPM) aims to reconstruct a subpixel-level class distribution map from the pixel-level abundance maps, which is an under-determined problem that has nonunique solutions. To address this, the spatiotemporal SPM uses the abundance, spatial, and temporal constraints to reduce the uncertainty of the mapping solutions, so the spatiotemporal SPM is essentially a constrained optimization problem. However, it is hard to find the optimal weighting parameters to combine the three joint constraints. In addition, the existing spatiotemporal SPM methods mainly use the temporal information either for the unchanged subpixels detection or for the subpixel classification, which is insufficient in the utilization of the temporal information. In this article, a novel spatiotemporal SPM algorithm based on multiobjective optimization (STSPM_MO) is proposed. STSPM_MO is composed of an unchanged subpixels detection stage and a multiobjective spatiotemporal mapping stage. In the former stage, the historical thematic map is used for identifying the unchanged subpixels. In the latter stage, the historical thematic map is further used for providing the temporal dependence, so that the temporal information can be more fully utilized. Moreover, to solve the constrained optimization problem of the spatiotemporal SPM, the abundance, spatial, and temporal constraints are modeled as three objectives and are dynamically fused through the subfitness-based multiobjective evolution, to generate the optimal subpixel classification map. Both synthetic and real-data experiments have been conducted, and the results show the proposed method, and its two variants are superior, stable, and effective.

Super-resolution mapping (SRM) is a way to solve the mixed-pixel problem in urban land use/land cover caused by the limited spatial-resolving ability of satellite sensors, through resolution enhancement of the classification map. Recently, deep learning-based super-resolution mapping (DLSM) networks have been boomed, which can automatically learn a mapping pattern from low-resolution (LR) image to high-resolution (HR) land cover distribution to alleviate mixed-pixel problem. However, the urban compositions like buildings, trees, and roads exhibit a multiscale distribution with different size or orientation, which makes the traditional single-scale DLSM failed for an appropriate recognition. In addition, the urban compositions also show significant spatial heterogeneity with irregular distribution and intricate morphological shape, which are difficult to learn by simple convolutional layer. Therefore, it is necessary to explore the cue of these distribution characteristic to constrain the learning behavior of the network for better detail restoration. In this article, a deep hierarchical pyramid sub-pixel mapping network (HiSMNet) with high-frequency-aware differential architecture is proposed, which establishes an HP architecture to achieve explicit multiscale supervision of the feature map and prompt the network to learn a multiscale representation. In addition, a differential architecture is designed to enforce the network to intensify the learning of the high-frequency details. The validation experiments demonstrate that HiSMNet achieves superior performances in detailed delineation and outperformed the state-of-the-art DLSM models by up to 10% in terms of overall accuracy.

As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is an image processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove noise from PAM images without manual selection of settings for different noisy images. An attention enhanced generative adversarial network is used to extract image features and adaptively remove various levels of Gaussian, Poisson, and Rayleigh noise. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. In the in vivo experiments using synthetic datasets, our method achieves the improvement of 6.53 dB and 0.26 in peak signal-to-noise ratio and structural similarity metrics, respectively. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising processing speed of 0.016 s is achieved for an image with 256×256 pixels, which has the potential for real-time applications. Our approach is effective and practical for the de-noising of PAM images.

Photoacoustic microscopy (PAM) is a promising imaging modality because it is able to reveal optical absorption contrast in high resolution on the order of a micrometer. It can be applied in an endoscopic approach by implementing PAM into a miniature probe, termed photoacoustic endoscopy (PAE). Here we develop a miniature focus-adjustable PAE (FA-PAE) probe characterized by both high resolution (in micrometers) and large depth of focus (DOF) via a novel optomechanical design for focus adjustment. To realize high resolution and large DOF in a miniature probe, a 2-mm plano-convex lens is specially adopted, and the mechanical translation of a single-mode fiber is meticulously designed to allow the use of multi-focus image fusion (MIF) for extended DOF. Compared with existing PAE probes, our FA-PAE probe achieves high resolution of [Formula: see text] within unprecedentedly large DOF of 3.2 mm, more than 27 times the DOF of the probe without performing focus adjustment for MIF. The superior performance is first demonstrated by imaging both phantoms and animals including mice and zebrafish in vivo by linear scanning. Further, in vivo endoscopic imaging of a rat's rectum by rotary scanning of the probe is conducted to showcase the capability of adjustable focus. Our work opens new perspectives for PAE biomedical applications.

Sub-pixel mapping (SPM) has been widely adopted to alleviate the mixed pixel problem in hyperspectral image, as an extension of spectral unmixing (SU), providing a way to observe the spatial location of the endmember within mixed pixel. However, most of the SPM methods are unmixing-then-mapping (UTM), i.e., SPM process relies on the abundance images generated from SU, in which process uncertainty inherently exists and would be propagated to SPM. Furthermore, the prior knowledge toward the sub-pixel scale distribution is mainly model-driven/handcrafted, which has limitation for geographical-realistic distribution representation. In this letter, we proposed spectral–spatial fusion SPM based on deep neural network (SSNET), to realize the integrative modeling of SU and SPM problem in a unified network fashion to avoid uncertainty accumulation in UTM process, and it can simultaneously generate SU result and SPM result. Besides, SSNET provides a supervised manner to learn prior knowledge with external exemplar pairs of low- and high-resolution images for a geographical-realistic distribution representation. The experiment with two hyperspectral images validated the superiority of the proposed SSNET.

Change detection (CD) aims to distinguish surface changes based on bitemporal images. Since high-resolution (HR) images cannot be typically acquired continuously over time, bitemporal images with different resolutions are often adopted for CD in practical applications. Traditional subpixel-based methods for CD using images with different resolutions may lead to substantial error accumulation when the HR images are employed, which is because of intraclass heterogeneity and interclass similarity. Therefore, it is necessary to develop a novel method for CD using images with different resolutions that are more suitable for the HR images. To this end, we propose a super-resolution-based change detection network (SRCDNet) with a stacked attention module (SAM). The SRCDNet employs a super-resolution (SR) module containing a generator and a discriminator to directly learn the SR images through adversarial learning and overcome the resolution difference between the bitemporal images. To enhance the useful information in multiscale features, a SAM consisting of five convolutional block attention modules (CBAMs) is integrated to the feature extractor. The final change map is obtained through a metric learning-based change decision module, wherein a distance map between bitemporal features is calculated. Ablation study and comparative experiments on two large datasets, building change detection dataset (BCDD) and season-varying change detection dataset (CDD), and a real-image experiment on the Google dataset fully demonstrate the superiority of the proposed method. The source code of SRCDNet is available athttps://github.com/liumency/SRCDNet.

Deep sub-pixel mapping network (DSMNet) is a state-of-the-art approach in the field of sub-pixel mapping (SPM, also called super resolution mapping), combining deep learning theory, to solve the mixed pixel problem, which is ubiquitous in remote sensing images due to the spatial-resolving limitation. However, traditional DSMNet usually do not consider the multi-scale distribution characteristics of the real geographical distribution exposed in urban landscape. Furthermore, the heterogeneous distribution characteristics (high-frequency components) are the most important for SPM, but are difficult to learn and usually ignored in the tradition network models. In this paper, the high-frequency component aware (HFCA) module was proposed, based on the hierarchical supervised deep sub-pixel mapping network (HiDSMNet). HiDSMNet establishes a hierarchical supervised architecture for explicit multi-scale supervision to prompt the network to learn a multi-scale representation. Besides, HFCA module is integrated to prompt the network to intensify the learning of the high-frequency representation. The experimental results with three public datasets validated the superiority of the proposed HiDSMNet.

Mixed pixel problem is omnipresent in remote sensing images for urban land use interpretation due to the hardware limitations. Subpixel mapping (SPM) is a usual way to solve this problem by improving the observation scale and realizing a finer spatial resolution land cover mapping. Recently, deep learning-based subpixel mapping network (DLSMNet) was proposed, benefited from its strong representation and learning ability, to restore a visually pleasing finer mapping. However, the spatial context features of artifacts are usually aggregated and progressively lost during the forward pass of the network without sufficient representation, which make it difficult to be learned and restored. In this article, a semantic information modulated (SIM) deep subpixel mapping network (SIMNet) is proposed, which uses low-resolution semantic images as prior, to reinforce the representation of spatial context features. In SIMNet, SIM module is proposed to parametrically incorporate the semantic prior into the state-of-the-art (SOTA) feed forward network architecture in an end-to-end training fashion. Furthermore, stacked SIM module with residual blocks (SIM_ResBlock) is adopted to pass the representation of spatial context feature to the deep layers, to get it fully learned during backpropagation. Experiments have been implemented on three public urban scenario data sets, and the SIMNet generates a clearer outline of artificial facilities with sufficient spatial context, and is distinctive for even individual building, which is challenging for other SOTA DLSMNet. The results demonstrate that the proposed SIMNet is a promising way for high-resolution urban land use mapping from easily available lower resolution remote sensing images.Mixed pixel problem is omnipresent in remote sensing images for urban land use interpretation due to the hardware limitations. Subpixel mapping (SPM) is a usual way to solve this problem by improving the observation scale and realizing a finer spatial resolution land cover mapping. Recently, deep learning-based subpixel mapping network (DLSMNet) was proposed, benefited from its strong representation and learning ability, to restore a visually pleasing finer mapping. However, the spatial context features of artifacts are usually aggregated and progressively lost during the forward pass of the network without sufficient representation, which make it difficult to be learned and restored. In this article, a semantic information modulated (SIM) deep subpixel mapping network (SIMNet) is proposed, which uses low-resolution semantic images as prior, to reinforce the representation of spatial context features. In SIMNet, SIM module is proposed to parametrically incorporate the semantic prior into the state-of-the-art (SOTA) feed forward network architecture in an end-to-end training fashion. Furthermore, stacked SIM module with residual blocks (SIM_ResBlock) is adopted to pass the representation of spatial context feature to the deep layers, to get it fully learned during backpropagation. Experiments have been implemented on three public urban scenario data sets, and the SIMNet generates a clearer outline of artificial facilities with sufficient spatial context, and is distinctive for even individual building, which is challenging for other SOTA DLSMNet. The results demonstrate that the proposed SIMNet is a promising way for high-resolution urban land use mapping from easily available lower resolution remote sensing images.

Subpixel mapping (SPM) is an effective way to solve the mixed pixel problem, which is a ubiquitous phenomenon in remotely sensed imagery, by characterizing subpixel distribution within the mixed pixels. In fact, the majority of the classical and state-of-the-art SPM algorithms can be viewed as a convolution process, but these methods rely heavily on fixed and handcrafted kernels that are insufficient in characterizing a geographically realistic distribution image. In addition, the traditional SPM approach is based on the prerequisite of abundance images derived from spectral unmixing (SU), during which process uncertainty inherently exists and is propagated to the SPM. In this article, a kernel-learnable convolutional neural network (CNN) framework for subpixel mapping (SPMCNN-F) is proposed. In SPMCNN-F, the kernel is learnable during the training stage based on the given training sample pairs of low- and high-resolution patches for learning a geographically realistic prior, instead of fixed priors. The end-to-end mapping structure enables direct subpixel information extraction from the original coarse image, avoiding the uncertainty propagation from the SU. In the experiments undertaken in this study, two state-of-the-art super-resolution networks were selected as application demonstrations of the proposed SPMCNN-F method. In experiment part, three hyperspectral image data sets were adopted, two in a synthetic coarse image approach and one in a real coarse image approach, for the validation. Additionally, a new data set with pairs of Moderate-resolution Imaging Spectroradiometer (MODIS) and Landsat images were adopted in a real coarse image approach, for further validation of SPMCNN-F in large-scale area. The restored fine distribution images obtained in all the experiments showed a perceptually better reconstruction quality, both qualitatively and quantitatively, confirming the superiority of the proposed SPM framework.

The maximum a posteriori (MAP) estimation model-based sub-pixel mapping (SPM) method is an alternative way to solve the ill-posed SPM problem. The MAP estimation model has been proven to be an effective SPM approach and has been extensively developed over the past few years, as a result of its effective regularization capability that comes from the spatial regularization model. However, various spatial regularization models do not always truly reflect the detailed spatial distribution in a real situation, and the over-smoothing effect of the spatial regularization model always tends to efface the detailed structural information. In this article, under the scenario of time-series observation by remote sensing imagery, the joint spectral-spatial-temporal MAP-based (SST_MAP) model for SPM is proposed. In SST_MAP, a newly developed temporal regularization model is added to the MAP model, based on the prerequisite for a temporally close fine image covering the same study region. This available fine image can provide the specific spatial structures most closely conforming to the ground truth for a more precise constraint, thereby reducing the over-smoothing effect. Furthermore, the three dimensions are mutually balanced and mutually constrained, to reach an equilibrium point and achieve restoration of both smooth areas for the homogeneous land-cover classes and a detailed structure for the heterogeneous land-cover classes. Four experiments were designed to validate the proposed SST_MAP: three synthetic-image experiments and one real-image experiment. The restoration results confirm the superiority of the proposed SST_MAP model. Notably, under the background of time-series observation, SST_MAP provides an alternative way of land-cover change detection (LCCD), achieving both detailed spatial-scale and high-frequency temporal LCCD observation for the study case of urbanization analysis within the city of Wuhan in China.

In recent decades, spatiotemporal subpixel mapping (SSM) approaches have been extensively developed to deal with the mixed-pixel problem by incorporating fine spatial resolution images with the same field of view from different acquisition times. This is an alternative to the conventional subpixel mapping (SPM) method, which is based on only monotemporal images. SSM has become one of the state-of-the-art SPM approaches, and has been widely applied in urban management and ecological monitoring. However, in the traditional SSM methods, the spatial correlation within the multitemporal images is insufficiently exploited and is ignored in the spatiotemporal model construction. In addition, the contribution of the land covers' spatial distribution in the multitemporal images is incompletely considered, and the geographic variation during the time interval is ignored, which underutilizes the spatiotemporal information. In this paper, an SSM algorithm based on a geographically weighted regression (GWR) model and evolutionary algorithm theory, called spatiotemporal subpixel geographical evolution mapping (STGEM), is proposed for multitemporal remote sensing images. The proposed algorithm considers the spatiotemporal dependence not only between the current subpixel and the corresponding fine pixel, but also with the neighboring fine distribution patterns within the fine image. Moreover, the potential temporal information of the geospatial variation is fully realized by considering not only the time interval between the bitemporal images, but also the ratio of changed area between them, based on the GWR model. Two synthetic-image experiments with bitemporal Landsat 8 images and bitemporal QuickBird images were carried out to validate the proposed algorithm. Furthermore, a real-image experiment using a bitemporal pair of Gaofen-2 images and a Landsat 8 image was also undertaken. A comparison was made with several traditional SPM methods, as well as the state-of-the-art SSM approaches, and the experimental results confirmed the superiority of the proposed STGEM algorithm. The proposed STGEM achieves a fine spatial and temporal resolution thematic map, both qualitatively and quantitatively, and has great potential for fine-scale and frequent time-series observation and monitoring.

In the past decades, land cover change detection (LCCD) has been dramatically developed, since it provides corroborative support for policy decision, regulatory actions, and subsequent urban-rural activities. Satellite remote sensing image is the major source of LCCD since it is able to revisit the Earth's surface regularly and provide time series images for monitoring and space-time analysis. However, there is always a trade-off between spatial scale and temporal scale, i.e., finer spatial resolution image generally has a lower revisit frequency, leading to an observation omission; while higher revisit frequency image usually has a lower spatial resolution, resulting in a deficiency in detecting finer scale change information. In this paper, a spatial-temporal sub-pixel mapping (SSM) algorithm is proposed on the premise that one pair of fine spatial resolution image with low frequency revisit period and coarse spatial resolution with high frequently revisit period are available, and SSM is taken to restore the coarse image to a finer scale thematic map which can be then compared to the fine image, realizing a frequency and detailed LCCD. SSM is an extension of traditional mono-temporal sub-pixel mapping (SPM) algorithm, and is improved by incorporating temporally fine distribution patterns for a more appropriate restoration of coarse image. A study case for urban expansion LCCD were carried out to verify the ability of the proposed algorithm to handle change detection based on one pair of china-made Gaofen-2 image (GF-2) and Landsat-8 image, the result demonstrate that the proposed SSM algorithm outperform the other traditional SPM, achieving both fine temporal resolution and spatial resolution LCCD for further applications.

The hyperspectral subpixel mapping (SPM) technique can generate a land-cover map at the subpixel scale by modeling the relationship between the abundance map and the spatial distribution image of the subpixels. However, this is an inverse ill-posed problem. The most widely used way to resolve the problem is to introduce additional information as a regularization term and acquire the unique optimal solution. However, the regularization parameter either needs to be determined manually or it cannot be determined in a fully adaptive manner. Thus, in this paper, the multiobjective subpixel land-cover mapping (MOSM) framework for hyperspectral remote sensing imagery is proposed, in which the two function terms [the fidelity term and the prior term (i.e., the regularization term)] can be optimized simultaneously, and there is no need to determine the regularization parameter explicitly. In order to achieve this goal, two strategies are designed in MOSM: 1) a high-resolution distribution image-based individual encoding strategy is designed in order to calculate the prior term accurately and 2) a subfitness-based individual comparison strategy is designed in order to generate subpixel land-cover mapping solutions with a high quality to update the population. Four data sets (one simulated, two synthetic, and one real hyperspectral image) were used to test the proposed method. The experimental results show that MOSM can perform better than the other subpixel land-cover mapping methods, demonstrating the effectiveness of MOSM in balancing the fidelity term and prior term in the SPM model.

Mixed pixel is a ubiquitous phenomenon in remotely sensed imagery, especially in moderate and low spatial resolution imagery, which compromise the hard land cover classification since the dominant class will shadow the information of other vulnerable classes, bringing trouble to imagery interpretation. Since the past decades, sub-pixel mapping (SPM) approaches were developed to deal with the mixture problem, on the basis of soft classification, to retrieval the pure components and its geospatial distribution within mixed pixels. Recently, SPM integrated with auxiliary information is gradually been a state-of-the-art method for mixed pixel problem, and has been proved effectively. However, few works has been dedicated to explore the geostatistic inter-correlation between spatial and temporal among the time sequences imageries. In this paper, a novel SPM algorithm based on swarm intelligence theory, considering spatiotemporal geographical attraction among multi-temporal imageries, called spatiotemporal attraction based sub-pixel evolution mapping (SASEM), is proposed for remote sensing imagery, Experiments were carried out to verify the proposed algorithm, and the result illustrate that the proposed algorithm outperform the traditional SPM, achieving a fine spatial resolution thematic map for further applications.

Sparse representation theory has become a powerful tool since it can obtain the sparsest or the unique solution for the underdetermined problem with the development of linear algebra, optimization, scientific computing and more. As subpixel information extraction encountered in hyperspectral remote sensing, which contains many mixed pixels, are famous under-determined ill-posed problem. In addition, there is no unified model to conquer the problems with the subpixel analysis techniques, i.e., spectral unmixing and subpixel mapping. To cope with this under-determined problem, a unified sparse subpixel information extraction framework was proposed in this paper, which connects sparse unmixing and sparse subpixel mapping methods in a unified theoretical system as a serious of sparse regression problem. The experimental results with hyperspectral images indicate that the proposed sparse representation framework outperforms the previous subpixel analysis approaches, hence, provides an effective option for subpixel information extraction idea for hyperspectral remote sensing imagery.

Nowadays, Dynamic Power Management and Dynamic Voltage (and Frequency) Scaling are well accepted for adjusting the trade-off between the performance and power dissipation. In this article, we investigate the problem of combined application of DPM and DVS in the context of hard real-time systems on cluster-based multi-core processor platforms. We propose a heuristic algorithm based on simulated annealing and its online execution. Our approach considers multiple low power states with non-negligible state switching overhead. The experimental results show that our algorithm can significantly reduce the power consumption in comparison with existing algorithms.

The energy efficiency becomes one of the most important concerns in mobile electronic systems design with mandatory requirements for low energy consumption, long battery life and low heat dissipation. Dynamic Power Management (DPM) and Dynamic Voltage and Frequency Scaling (DVFS or DVS) are two widely applied system level techniques to conserve system-wide power consumption. In the context of hard real-time systems, however, DPM and DVS have to be used with great caution in terms of timing constraints. In this article, we study the combined application of DPM and DVS for component oriented systems with hard real-time tasks and propose a simulated annealing based optimization algorithm and its online execution with constant complexity at each scheduling point. Additionally, our approach considers multiple low power states (sleep states) with non-negligible switching overhead. The experimental results show that our approach can achieve almost an optimal solution.

UML is widely applied for the specification and modeling of software and some studies have demonstrated that it is applicable for HW/SW codesign. However, in this area there is still a big gap from UML modeling to SystemC-based verification and synthesis environments. This paper presents an efficient approach to bridge this gap in the context of Systems-on-a-Chip (SoC) design. We propose a framework for the seamless integration of a customized SysML entry with code generation for HW/SW cosimulation and high-level FPGA synthesis. For this, we extended the SysML UML profile by SystemC and synthesis capabilities. Two case studies demonstrate the applicability of our approach.