Abstract The quality of images we perceive visually is heavily impacted by the settings used for camera exposure. When these settings are imbalanced, it can result in an undesired prominent phenomenon known as blur effects. To address this problem, an ExposureNet project has been undertaken, which aims to develop an autonomous camera exposure settings control system for blur effects prevention. The proposed ExposureNet model is a CNN/Transformer hybrid neural structure, created and trained in a comprehensive manner to effectively predict the ideal exposure settings based on the semantic features of the scene being captured. This system is designed to learn the necessary steps for processing, such as identifying relevant scene features, using only two camera exposure parameters (shutter speed (SHS) and ISO) as training signals. As a result, this system can associate the semantic features of a scene with the appropriate exposure parameter adjustments, customized to the scene's dynamics and lighting conditions. By simultaneously optimizing all processing steps and bypassing traditional post‐processing stages, the proposed system is designed to achieve faster performance, reduced computational cost, and lower power consumption. Experimental results demonstrate that the proposed system significantly outperforms existing methods and achieves cutting‐edge performance.

The imaging of maneuvering targets may be defocused by the range cell and Doppler frequency migrations initiated by the targets’ complex motions in a long Coherent Processing Interval (CPI). To settle these problems, a generalized matched filtering method driven by the Bayesian motion trajectory inference (BMTI) is presented in this paper. Analyses show that any motion trajectory evolutions caused by targets’ movements can be described by the state equations derived from a polynomial prediction model. It is then demonstrated the uncertainties of the targets entering/leaving the radar beam coverage area will make the state equations into a state labeled multi-Bernoulli random finite set (LMBRFS). With the scale-transformation-based processing pipeline, a measurement random finite set (RFS) taking both the statistical properties of the maneuvering target with random appearance and disappearance and the noise/clutter into consideration is coined for the state LMBRFS. In this way, a state space model for describing the target instantaneous motions in a CPI is established. Once the target motion trajectories are inferred by a modified Bayesian filter, it is shown that the matched filter bank designed by the inferred trajectories can refocus the echo energies. Numerical simulation results verify the correctness of our analytical ones, while it is illustrated that our model and method are universal for indicating high-speed and maneuvering targets with any complex and/or even unknown motion forms in a CPI.

Hyperspectral image classification with limited training samples has become a hot research topic recently. Though deep convolution neural network shows powerful ability for feature extraction, its good performance often relies on sufficient training data. In this paper, we propose a multitask learning framework based on siamese network and auto-encoder to fully exploit limited labeled samples’ information and obtain discriminative features for classification of hyperspectral images. A low intraclass and high interclass variability of features can be learned by metric learning using our framework. And superpixel-based 3D sample preprocessing is applied to improve the classification accuracy on the hyperspectral images’ boundaries. The experimental results demonstrate that our framework can achieve competitive results compared with the state-of-the-art methods.

The imbalance between high dimensionality and limited labeled samples has been a great challenge for classification task of hyperspectral images (HSIs). In this paper, a novel semi-supervised classification method for HSIs is proposed. This method contains two major parts: representation using spatial-spectral graph model and graph convolutional networks (GCN) using edge-conditioned convolution. For the proposed method, spatial-spectral information is considered simultaneously during the process of graph construction, and then GCN is used to extract the feature from input data and learn their topology relationships with edge label involved. Experimental results on multiple hyperspectral datasets with various contexts and resolutions demonstrate that the proposed classifier outperforms several graph-based methods.

In human vision, it has been well understood that the red, green, and blue (RGB) trichromatic system, whose colors can be well expressed by an RGB hue ring with complementary color relations, is the most accessible and extensible color representation. However, such color relations rarely play a role in color image/video processing tools such as wavelets. In this paper, the gap between wavelets and complementary color relations is filled by designing a novel color image/video processing tool called the complementary color wavelet transform. Our novel wavelets consist of a family of well-designed 2-D complex wavelets with 2π/3 phase differences copied from the R, G, and B angle relations on the RGB hue ring. Analyses show that our new wavelets can not only utilize the complementary color relations but also inherit and improve upon the advantages of classical complex wavelets, such as their multi-resolution, directional selectivity, and shift-invariance properties. By the newly designed wavelets and by properly adding and/or subtracting the wavelet coefficients of a color image/video, some color-related image/video processing applications that are typically seen as infeasible or difficult to achieve by traditional methods, for instance, capturing and classifying local color changes, statistical modeling of the wavelet coefficients among the color channels, and multi-resolution directional color filtering, can be performed very easily. The cost for this is simply a transform redundancy increase (2.25 times that of classical complex wavelets). We provide three application examples of our method: color filtering, texture retrieval, and video quality assessment. The results show that our new wavelet tool greatly outperforms traditional methods.

Wavelength dependent nonlinearity is an essential issue in hyperspectral unmixing, which was overlooked in the past. In this paper, a band-wise nonlinear unmixing method is presented. An extended multilinear mixing model is adopted for interpreting different degrees of nonlinear contributions per band. Moreover, regularizers including abundances' sparsity and nonlinear parameters' smoothness are exploited to formulate the optimization problem and obtain better unmixing results. Finally, unmixing is implemented in the scheme of alternating direction method of multipliers. Experimental results on both simulated and real hyperspectral data validate that the proposed method can improve the unmixing accuracy and reveal the change of nonlinearity at each band as well.

In today WSNs, sensor nodes are able to obtain energy from ambient with energy-harvesting components. However, the energy consumption are diverse across these nodes due to functional or geographical variation, which may lead to potential energy imbalance in network. In virtue of recent wireless power transfer (WPT) technology, the imbalance can be alleviated if all sensor nodes share energy with each other. In order to achieve the maximum energetically sustainable workload, we design an energy cooperation strategy in network by WPT, named energy routing, which should be jointly optimized with data routing simultaneously. An iterative distributed algorithm is developed to achieve the optimal data routing and energy routing solutions, where all sensors only need to exchange local information with neighbors. Simulation results show that the proposed algorithm can achieve higher workload than algorithms without energy cooperation.

Change detection for multitemporal hyperspectral images (HSIs) involves two major steps: change feature extraction and classification. For the first part, conventional methods mostly consider spectral features but neglect spatial patterns. Since multitemporal HSIs consist of four dimensions (one for time, one for spectral domain and two for spatial domain), we propose using 4-dimensional Higher Order Singular Value Decomposition (4D-HOSVD) based on tensor algebra to capture the details in all the dimensions simultaneously and thus producing comprehensive change features. To emphasize on the effectiveness of the change feature extraction method, this paper reduces the change classification to a simple binary problem: a pixel is either changed or unchanged. Experimental results show that 4D-HOSVD can outperform its matrix counterpart, Principal Component Analysis (PCA), as well as some other widely adopted method.

There are two great challenges for classification of hyperspectral images (HSIs): lack in prior knowledge and serious internal-class variability. To address the issues, we propose a novel semisupervised method based on affinity scoring (AS). It can harness the fuzzy state of the contributions of spectral and spatial features to classification. The method consists of three major steps: over-segmentation, semisupervised classification and modification. First, superpixels are generated to maintain local class consistency, which can balance spectral variability. Then unlabeled samples are classified by AS in an iterative manner, whereas precious labeled samples are made most use of. Finally, AS is adopted again to refine the classification map, which further exploits spatial smoothness in HSIs. Experiments show that the proposed method can largely outperform several state-of-the-art classifiers.

Sparse representation has been introduced to tackle the target detection problem in hyperspectral imagery. While using windows to build the sparse dictionary, there exists target contamination problem. In our approach, we utilize a learning method based on convex optimization to build a dictionary for sparse target detection. Through its application, prior information such as the size of windows can be spared, while considerably reducing the occurrence of contamination. To verify the efficacy of using the learned dictionary, the dictionary built through the dual-window method is used as a comparison and two sparse target detection methods are employed afterward. Experimental results show that, by using the learned dictionary, a better result is obtained compared to the methods using traditional dual-window background dictionary.

Hyperspectral unmixing is an important issue to analyze hyperspectral data. Based on the present mixing models, this paper proposes a new nonlinear unmixing framework for hyperspectral imagery. The proposed framework transforms the hyperspectral unmixing problem to a constrained nonlinear least squares problem by introducing the abundance nonnegative constraint, abundance sum-to-one constraint and the bound constraints of nonlinear parameters. Accordingly, an alternating iterative optimization algorithm is developed to solve the arising nonlinear least squares problem. The method decomposes the nonlinear unmixing problem into two sub-problems, which obtain alternately the abundance vectors and nonlinear parameters of the observation pixels. The experimental results on synthetic and real hyperspectral dataset demonstrate that the proposed algorithm can effectively overcome the inherent limitations of the linear mixing model. Meanwhile, the proposed algorithm performs well for noisy data, and can also be used as an effective technique for the nonlinear unmixing of hyperspectral imagery.

A novel algorithm is proposed to perform object tracking with multiple cameras in the Bayesian Inference framework. The key contribution is the exploitation of Bayesian network to fuse spatial-temporal position and object template feature in multiple cameras. Firstly, Bayesian network is used to model the multiple static cameras' tracking system. Then, the high-dimensional joint posterior is propagated spatiotemporally. Finally, the estimation of the target location in each camera view is achieved by using sequential Monte Carlo Approximation. The robust tracking algorithm efficiently fuses information from different views and is capable of dealing with partial and full occlusion. Besides, the distributed tracking algorithm is implemented on OMAP3530 platform. Both qualitative and quantitative experiments have demonstrated the effectiveness and robustness of the proposed algorithm.

We propose a complete solution to robust and accurate object tracking in face of various types of occlusions which pose many challenges to correct judgment of occlusion situation and proper update of target template. In order to tackle those challenges, we first propose a content-adaptive progressive occlusion analysis (CAPOA) algorithm. By combining the information provided by spatiotemporal context, reference target, and motion constraints together, the algorithm makes a clear distinction between the target and outliers. Accurate tracking of an occluded target is achieved by rectifying the target location using the variant-mask template matching (VMTM). In order to deal with template drift in the process of template update, we propose a drift-inhibitive masked Kalman appearance filter (DIMKAF) which accurately evaluates the influence of template drift when updating the masked template. Finally, we devise a local best match authentication (LBMA) algorithm to handle complete occlusions, so that we can achieve a much more trustworthy detection of the end of an arbitrarily long complete occlusion. Experimental results show that our proposed solution tracks targets reliably and accurately no matter when they are under: short-term, long-term, partial or complete occlusions.

How to decompose mixed pixels effectively and precisely for remote sensing images is a critical issue for the quantitative remote sensing research. In this paper we propose a new method for decomposition of mixed pixels of multispectral or hyperspectral remote sensing images. The proposed method introduces the algorithm of Bayesian Self-Organizing Map (BSOM) into the problem of the decomposition of mixed pixels. It estimates Gaussian parameters by minimizing the Kullback- Leibler information metric, and finishes the unmixing with Gaussian Mixture Model (GMM). In order to obtain a high unmixing precision, we need to extend the range of Gaussian distributions, and thus propose the 3 σ variances adjustment method to solve this problem. In addition, the used unmixing model satisfies two constraints which are demanded for the problem of the decomposition of mixed pixels automatically: abundances non-negative constraint (ANC) and abundances summed-to-one constraint (ASC). Experimental results on simulated and practical remote sensing images demonstrate that the proposed method can get good unmixing results for the decomposition of mixed pixels, and is more robust to noise than other methods.

The simplex-based methods are a kind of the most important and widely used methods for the decomposition of mixed pixels in multispectral and hyperspectral remote sensing images, which need a strong basic hypothesis that there is at least one pure pixel for every endmember existing in the images. N- FINDR is of typical sense in this kind of methods, and it also needs this basic hypothesis. Unfortunately, the precision for the decomposition of mixed pixels will be seriously influenced if this hypothesis cannot be met in practice. This paper presents a new scheme based on Nonnegative Matrix Factorization (NMF) to solve this problem. In addition, some appropriate constrains are introduced into NMF for the decomposition of mixed pixels. Experimental results obtained from both artificial simulated and real-world remote sensing data demonstrate that the proposed scheme for decomposition of mixed pixels has excellent analytical performance.

A new algorithm for decomposition of mixed pixels based on orthogonal bases of data space is proposed in this paper. It extracts endmembers sequentially by adding a new vertex of the simplex with the maximum volume every time. It can avoid the dilemma in traditional simplex-based endmember extraction algorithms such as N-FINDR that it generally produces different sets of final endmembers if different initial conditions are used. Meanwhile, at each step of the searching process, the calculation of a simplex volume based on determinant of a matrix is replaced by the one based on the norm products of a set of orthogonal bases, which results in a tremendous reduction of computational complexity. Moreover, with this set of orthogonal bases, the proposed algorithm can also determine the proper number of endmembers and finish the unmixing of the original images which cannot be done by the traditional endmember extraction algorithms. Experimental results of practical remote sensing images demonstrate that the algorithm proposed in this paper is a fast and accurate algorithm for the decomposition of mixed pixels.

In this paper, a hypercomplex principle component weighted approach to multiplespectral and panchromatic images fusions is presented. Since it takes the vectorial property of RGB components into account, there is not the distortion of visible spectrums in the images fused by the proposed method. The evaluation results from the various methods representing the state of the art ones also show that the fusion results of presented approach is better than those of IHS(hue-intensity-saturation), PCA (principle component analysis) and wavelet-based methods.