Temporal graphs are essential for modeling complex real-world systems, such as social interactions, financial transactions, and recommendation systems, but the high computational cost and model complexity of dynamic graph neural networks (DGNNs) pose significant challenges for practical deployment. Although various pruning and sampling techniques have proven effective in accelerating static GNNs, they fall short in dynamic settings due to temporal dependencies in evolving graph structures. To address these challenges, we propose TrimDG, a general framework that accelerates DGNNs by eliminating both static and runtime redundancies. For static redundancy, we introduce a novel node influence metric, Temporal Personalized PageRank (TPP), to prune less informative nodes, and employ temporal binning to remove redundant events. For runtime redundancy during training, we develop an adaptive sampling strategy guided by graph information bottleneck and further reduce sampling frequency through temporal batch selector and sampling cache. Theoretical analysis supports our design, and experiments on real-world datasets show that TrimDG reduces runtime by an average of 83.49% across diverse DGNN backbones, while maintaining strong predictive performance, demonstrating both its efficiency and generalizability.

Remote sensing image spatiotemporal fusion (STF) algorism plays an important role by supplementing the lack of original high-resolution remote sensing satellite images in the study scenarios of dense time-series data. In recent years, the deep-learning-based STF algorithm has become a research hotspot with comparatively higher accuracy and robustness. However, due to the lack of sufficient high-quality images for training and the huge resolution gap between low-resolution images and high-resolution images, it is difficult to recover detailed information, especially for areas of land-cover change. In this paper, we propose a two-layers super-resolution based generation adversarial spatiotemporal fusion model(TLSRSTF) using smaller inputs to reduce pressure on data requirements and a mutual affine convolution to reduce model parameters. Specifically, we only use a pair of high-resolution and low-resolution images and a high-resolution image at any time. A spatial degradation consistency is constructed to adaptively determine the ratio of two layers of the super-resolution STF model. The quantitative and qualitative experimental results on public spatiotemporal fusion datasets demonstrate our superiority over the state-of-the-art methods.

Head pose estimation suffers from several problems, including low pose tolerance under different disturbances and ambiguity arising from common head pose representation. In this study, a robust three-branch model with triplet module and matrix Fisher distribution module is proposed to address these problems. Based on metric learning, the triplet module employs triplet architecture and triplet loss. It is implemented to maximize the distance between embeddings with different pose pairs and minimize the distance between embeddings with same pose pairs. It can learn a highly discriminate and robust embedding related to head pose. Moreover, the rotation matrix instead of Euler angle and unit quaternion is utilized to represent head pose. An exponential probability density model based on the rotation matrix (referred to as the matrix Fisher distribution) is developed to model head rotation uncertainty. The matrix Fisher distribution can further analyze the head pose, and its maximum likelihood obtained using singular value decomposition provides enhanced accuracy. Extensive experiments executed over AFLW2000 and BIWI datasets demonstrate that the proposed model achieves state-of-the-art performance in comparison with traditional methods.

CNN-based Spatiotemporal image fusion (STIF) methods have achieved better performance than traditional researches. However, most CNN-based methods fail to make full use of hierarchical features, and ignore the quality and distribution characteristics of feature maps in fine-grained STIF. In this paper, we propose a network with channel attention and feature super-resolution for STIF (CAFSRNet). First, our method uses the low resolution time-domain changing images as input to extract changes more accurately and simplify computational overhead. Second, channel attention mechanism is introduced into Cross-spatial Resolution Mapping module to make the network pay more attention to informative features. Third, by adding feature super-resolution into the supervision process, we enhance the distribution of feature maps and the quality of mapping results. The qualitative and quantitative experimental results on various datasets demonstrate the superiority of our proposed method over the state-of-the-art methods.

For hyperspectral classification, mixed pixels are usually the biggest reason to reduce the classification accuracy. To solve the problem, we apply the multi-label classification technique to the hyperspectral image classification. The approach Label-specific Features (LIFT) achieves state-of-the-art performance because the most distinctive features are constructed for each label. Clustering centers play an essential role in label-specific feature conversion. However, LIFT does not consider the relationship between positive and negative instances, and clustering centers on the training dataset is inconsistent with that of the original instances. In this paper, we propose a new algorithm called SMOTE_DFL, which can get better clustering centers through two strategies: 1) The spectral clustering algorithm SIA is introduced to focus on the local structure between positive and negative instances; 2) By oversampling training sample, the clustering center of training sample is close to that of the whole. Extensive experiments are conducted on three datasets. The results validate the superiority of SMOTE_DFL to other algorithms.

Convolutional neural network (CNN) is one of the most powerful tools to deal with computer vision tasks such as hyperspectral image (HSI) classification. While many studies using CNN focus on classification precision, few of them pay attention to the model size and running time. Some studies focus on lightweight neural networks for traditional RGB image processing tasks and achieve fantastic results, but none of them are designed for hyperspectral image processing. In this paper, a novel lightweight neural network designed for hyperspectral image classification is proposed to do fast HSI processing while maintaining high classification precision. The network uses the idea of feature reuse to reduce parameter size and improve convergence. Expansion convolution is adopted to overcome the defects which are brought by parameter reduction. The experiments show that the proposed network has SOTA level classification accuracy while maintaining high processing speed.

When Darwin visited the Galapagos archipelago, he observed that, in spite of the islands' physical similarity, members of species that had dispersed to them recently were beginning to diverge from each other. He postulated that these divergences must have resulted primarily from interactions with sets of other species that had also diverged across these otherwise similar islands. By extrapolation, if Darwin is correct, such complex interactions must be driving species divergences across all ecosystems. However, many current general ecological theories that predict observed distributions of species in ecosystems do not take the details of between-species interactions into account. Here we quantify, in sixteen forest diversity plots (FDPs) worldwide, highly significant negative density-dependent (NDD) components of both conspecific and heterospecific between-tree interactions that affect the trees' distributions, growth, recruitment, and mortality. These interactions decline smoothly in significance with increasing physical distance between trees. They also tend to decline in significance with increasing phylogenetic distance between the trees, but each FDP exhibits its own unique pattern of exceptions to this overall decline. Unique patterns of between-species interactions in ecosystems, of the general type that Darwin postulated, are likely to have contributed to the exceptions. We test the power of our null-model method by using a deliberately modified data set, and show that the method easily identifies the modifications. We examine how some of the exceptions, at the Wind River (USA) FDP, reveal new details of a known allelopathic effect of one of the Wind River gymnosperm species. Finally, we explore how similar analyses can be used to investigate details of many types of interactions in these complex ecosystems, and can provide clues to the evolution of these interactions.

Existing deep learning-based pan-sharpening methods mainly learn spatial information from a high-resolution (HR) panchromatic (PAN) image for each spectral channel. However, due to the own characteristics of remote sensing image data, the spatial information of PAN image often shows weak correlation with some spectral channel, especially for channels non-overlapped by PAN channel. In this paper, we propose a parallel pyramid network (PPN) for pan-sharpening. First, a three-branch parallel structure is proposed for dealing with PAN image detail, multispectral (MS) images detail and spectral property respectively. Second, pyramid network structure is introduced in two detail branches to solve the problem of weak correlation due to scale difference. Third, the feature level fusion in two detail branches is implemented, which utilizes redundancy between channels to solve detail representation of channels non-overlapped by PAN channel. The qualitative and quantitative experimental results on various data sets demonstrate the superiority of our proposed method over the state-of-the-art methods.

Imaging photoplethysmography (iPPG) can be used to detect heart rates from facial videos. However, it is sensitive to motion disturbances in realistic environments. To address this problem, a frequency-constrained multilayer sparse coding (FCMSC) algorithm is proposed in this paper. Specifically, FCMSC learns a dictionary about iPPG signals from a large set of clean iPPG signals in the training phase, and then reconstructs distorted iPPG signals with the learned dictionary in the testing phase. Compared to previous methods, FCMSC has stronger learning power due to its multilayer structure and better generalization ability because of its frequency constraint. A total of 3630 clips are cut out from the MAHNOB-HCI (a public video dataset) to train and test FCMSC. Experimental results show that FCMSC outperforms state-of-the-art methods in extracting heart rates from facial videos involving motion disturbances.

With integrated LED lamps and cameras, smartphones are capable of extracting pulse waves from fingertip videos using image photoplethysmography technique. This paper presents a novel method for detecting pulse waves based on smartphone videos, with a focus on preserving vital details in pulse waves (such as limbs and dicrotic waves). Chrominance features are first extracted from videos to obtain a raw pulse wave, whose primary frequency is then used to build its fundamental pulsatile wave. After extracting pulse details from the raw pulse wave, a smooth pulse wave with clear details can be reconstructed by combining the fundamental pulsatile wave with pulse details. Experiments are conducted on a dataset involving 40 videos from 10 subjects under ambient lighting environments. Results have demonstrated the proposed method outperforms state-of-the-art ones in both pulse rate detection and pulse detail preservation especially in cases of motion artifacts.

In order to promote progress on fusion and analysis methodologies for multi-source remote sensing data, The Image Analysis and Data Fusion Technical Committee organized the 2018 IEEE GRSS Data Fusion contest. In this contest, we proposed a two-branch convolution network for hyperspectral image classification with a data re-sampling strategy and semi-supervised learning to address three existing problems, i.e. multi-scale feature learning, data imbalance, and small size of the dataset. The contest showed that our proposal achieved the best performance on two metrics: the overall accuracy of 77.39% and a kappa coefficient of 0.76 on the hyperspectral images provided by 2018 IEEE GRSS Data Fusion Contest.

Multimodal remote sensing images contain complementary information, thus, could potentially benefit many remote sensing applications. To this end, the image registration is a common requirement for utilizing the multimodal images. However, due to the rather different imaging mechanisms, multimodal image registration becomes much more challenging than ordinary registration, particular for optical and synthetic aperture radar (SAR) images. In this work, we design a deep matching network to exploit the latent and coherent features between multimodal patch pairs for inferring their matching labels. But, the network requires immense data for training, which is not usually met. To address this issue, we propose a generative matching network (GMN) to generate the coupled optical and SAR images, hence, improve the quantity and diversity of the training data. The experimental results show that our proposal significantly improves the registration performance of optical and SAR image registration, and achieves subpixel or close to subpixel error.

In this paper, we address a haze removal problem from a single nighttime image, even in the presence of varicolored and non-uniform illumination. The core idea lies in a novel maximum reflectance prior. We first introduce the nighttime hazy imaging model, which includes a local ambient illumination item in both direct attenuation term and scattering term. Then, we propose a simple but effective image prior, maximum reflectance prior, to estimate the varying ambient illumination. The maximum reflectance prior is based on a key observation: for most daytime haze-free image patches, each color channel has very high intensity at some pixels. For the nighttime haze image, the local maximum intensities at each color channel are mainly contributed by the ambient illumination. Therefore, we can directly estimate the ambient illumination and transmission map, and consequently restore a high quality haze-free image. Experimental results on various nighttime hazy images demonstrate the effectiveness of the proposed approach. In particular, our approach has the advantage of computational efficiency, which is 10-100 times faster than state-of-the-art methods.

Depth estimation from single image is an important component of many vision systems, including robot navigation, motion capture and video surveillance. In this paper, we propose to apply a structure forest framework to infer depth information from single RGB image. The core idea of our approach is to exploit the structure properties exhibit in local patches of depth map to learn the depth level for each pixel. We formulate the problem of depth estimation in a structured learning framework based on random decision forests. Each trained forest infers a patch of structured labels that are accumulated across the image to obtain the final depth map. Moreover, we systematically investigate a variety of depth-relevant features and the regression forest framework automatically determines the best feature combination and uses as input of structure forest. Our approach achieves quasi real-time performance that is orders of magnitude faster than state-of-the-art approaches, while also achieving state-of-the-art depth estimation results on the Make3D dataset.

This letter presents a novel method for ship classification that uses synthetic-aperture-radar images to distinguish ships based on superstructure scattering features. The ratio of dimensions, which combines the 2-D and 3-D properties of scattering, is explored as an effective and credible means to describe the scattering features of ships. The proposed method consists of three main stages: 1) ship isolation from the sea; 2) parametric vector (F) estimation; and 3) categorization using a support vector machine (SVM) classifier. To depict ship features more accurately and reduce feature redundancy, we propose employing peak extraction to divide a ship into bow, middle, and stern instead of into three equal parts. The classification method is tested with RadarSat-2 images, and ground-truth information is supplied by an automatic identification system. The experimental results show that the proposed method can achieve satisfactory ship-classification performance compared with existing methods, with an overall accuracy exceeding 80%.

Stereo image applications are becoming more and more prevalent. However, there has been little research on stereo image enhancement. In this paper, we address the challenging problem of underwater stereo image enhancement. A new underwater imaging model is proposed and it can better describe the degradation of underwater images including color distortion and contrast attenuation. In addition, a novel observation that the intensity of the water part within the image is mainly contributed by the scattering light is also proposed. Coupling the proposed model and prior together, the parameters of scattering light can be estimated. Then an iterative approach to process stereo matching and stereo image enhancement alternatively is presented, which can significantly improve the quality of the images and depth maps. The experimental results demonstrate that the proposed method can significantly enhance the image visibility and achieve better depth perception.

The demand to restore all-in-focus images from defocused images and produce photographs focused at different depths is emerging in more and more cases, such as low-end hand-held cameras and surveillance cameras. In this paper, we manage to solve this challenging multi-focusing problem with a single image taken with an uncalibrated conventional camera. Different from all existing multi-focusing approaches, our method does not need to include a deconvolution process, which is quite time-consuming and will cause ringing artifacts in the focused region and low depth-of-field. This paper proposes a novel systematic approach to realize multi-focusing from a single photograph. First of all, with the optical explanation for the local smooth assumption, we present a new point-to-point defocus model. Next, the blur map of the input image, which reflects the amount of defocus blur at each pixel in the image, is estimated by two steps. 1) With the sharp edge prior, a rough blur map is obtained by estimating the blur amount at the edge regions. 2) The guided image filter is applied to propagate the blur value from the edge regions to the whole image by which a refined blur map is obtained. Thus far, we can restore the all-in-focus photograph from a defocused input. To further produce photographs focused at different depths, the depth map from the blur map must be derived. To eliminate the ambiguity over the focal plane, user interaction is introduced and a binary graph cut algorithm is used. So we introduce user interaction and use a binary graph cut algorithm to eliminate the ambiguity over the focal plane. Coupled with the camera parameters, this approach produces images focused at different depths. The performance of this new multi-focusing algorithm is evaluated both objectively and subjectively by various test images. Both results demonstrate that this algorithm produces high quality depth maps and multi-focusing results, outperforming the previous approaches.

In this paper, we address a challenging problem of multi-focusing image from a single photograph taken with an uncalibrated conventional camera. In order to achieve this, we firstly derive an optical degradation model which enables us to adopt a point operation scheme to realize image multi-focusing. This scheme can effectively reduce halo artifacts in the refocused image and greatly improve the computational efficiency. Then, a two-step approach is applied to estimate the blur map of the input image. i). A sparse blur map is obtained by estimating the amount of defocus blur at edge locations. ii). The guided image filtering method is applied to propagate the value from edge locations into the unknown regions. In order to obtain the depth map of the whole scene to realize the multi-focusing, we adopt a simple geometry prior of photograph to eliminate the ambiguity over the focal plane. Based on the obtained depth map, we can directly produce different styles of images by multi-focusing with the adjustment to the camera parameters. Experimental results on a variety of images show that our method can acquire visual pleasing multi-focusing results. Moreover, our method can also extract the depth map of the scene with fairly good extent of accuracy.

In the hazy weather, the image of outdoor scene is degraded by suspended particles. Scattering and absorption hinder scene radiance and bring in environment light into camera. In this work, a novel algorithm is introduced to restore the clear day image by the segmented hazy image. First, the existing visibility restoration model is analyzed and a conclusion is drawn that the model will violate the contrast enhancement constraint in some specific situations. Next, the graph-based image segmentation method is applied to segment the hazed image by choosing the optimal parameter. Then, the transmission maps prior are obtained according to the blackbody theory. After that, a bilateral filter is designed to amend the transmission map, which can make up the deficiency of restoration model and ensure the transmission map smooth under the contrast enhancement constraint. Last, the experimental results show that the method achieves rather good dehazing results.

Multi-target detection and tracking are the two key issues of video surveillance. This paper presents the methods to solve the two problems. Firstly, a moving object detection algorithm based on background modeling is given, which can effectively deal with the background disturbance and illumination changes. Then, a multi-targets tracking algorithm based on particle filter is described. Some special problems in multi-targets tracking include computational complexity, new target entry, target exit and occlusion are also discussed. Experiment results show that presented method is effective.