Abstract Internet of Things (IoT) applications continue to expand into new applications with a growing need for image processing on the edge. Many edge devices are resource limited microcontrollers, which significantly prohibits many of the mature image processing algorithms. This paper proposes an approach optimized, for resource constrained processors removing the need for computationally expensive floating point arithmetic by using a framework based on unsigned integer arithmetic for image processing. The proposed framework (OptInt) is demonstrated using edge detection algorithms evaluated on two typical low-power IoT-ready micro-controllers and for comparison a more powerful Raspberry Pi. Results indicate that the OptInt approach for basic image processing in resource constrained devices reduces the computation time as well as the memory requirements, thereby allowing for more edge computing capabilities in these devices. Furthermore, the images produced using OptInt produce results of similar quality to mature edge detection algorithms.

Abstract Depth information is useful in many image processing applications. However, since taking a picture is a process of projection of a 3D scene onto a 2D imaging sensor, the depth information is embedded in the image. Extracting the depth information from the image is a challenging task. A guiding principle is that the level of blurriness due to defocus is related to the distance between the object and the focal plane. Based on this principle and the widely used assumption that Gaussian blur is a good model for defocus blur, we formulate the problem of estimating the spatially varying defocus blurriness as a Gaussian blur classification problem. We solve the problem by training a deep neural network to classify image patches into one of the 20 levels of blurriness. We have created a dataset of more than 500,000 image patches of size $$32\times 32$$ 32×32 which does not require human labelling. The dataset is used to train and test several well-known network models. We find that MobileNetV2 is suitable for this application due to its low memory requirement and high accuracy. The trained model is used to determine the patch blurriness which is then refined by applying an iterative weighted guided filter. The result is a defocus map that carries the information of the degree of blurriness for each pixel. We compare the proposed method with state-of-the-art techniques and we demonstrate its successful applications in adaptive image enhancement and defocus magnification limited to images that present a clear distinction between defocus levels.

Current state-of-the-art detectors achieved impressive performance in detection accuracy with the use of deep learning. However, most of such detectors cannot detect objects in real time due to heavy computational cost, which limits their wide application. Although some one-stage detectors are designed to accelerate the detection speed, it is still not satisfied for task in high-resolution remote sensing images. To address this problem, a lightweight one-stage approach based on YOLOv3 is proposed in this paper, which is named Squeeze-and-Excitation YOLOv3 (SE-YOLOv3). The proposed algorithm maintains high efficiency and effectiveness simultaneously. With an aim to reduce the number of parameters and increase the ability of feature description, two customized modules, lightweight feature extraction and attention-aware feature augmentation, are embedded by utilizing global information and suppressing redundancy features, respectively. To meet the scale invariance, a spatial pyramid pooling method is used to aggregate local features. The evaluation experiments on two remote sensing image data sets, DOTA and NWPU VHR-10, reveal that the proposed approach achieves more competitive detection effect with less computational consumption.

Edge‐aware smoothing has proved to be a fundamental technique for various image processing and computer vision tasks. In this study, the authors introduce a local, non‐iterative, and effective edge‐preserving filter namely guided adaptive interpolation filter (GAIF). GAIF can be used as a post‐processing step after any smoothing filter to improve its edge preservation performance without reformulation. GAIF has an computation complexity, where N is the total number of pixels in the image. To further increase the efficiency of GAIF at edge‐preservation, two techniques are introduced and demonstrated. GAIF efficiency is demonstrated and compared to state‐of‐the‐art techniques on a number of tasks including image smoothing, flash/no‐flash image denoising/fusion, single image dehazing, and image details enhancement.

With the development of artificial intelligence, robot swarm systems also frequently appear in complex tasks of different situation. One of the important research directions is the formation of multi-robots. This paper analyzes the limitations of existing algorithms for large-scale mobile robot swarm formation control problems and proposes a consensus control algorithm with two-layer nearest neighbor information. It carries out experimental simulation to verify its convergence performance. At the same time, combined with a distributed structure control strategy that can change the number of robot formation members, the formation control experiment is carried out on the experimental platform consisted of robot state information detection device and multiple mobile robots,to further verify its feasibility.

The goal of blind image quality assessment (IQA) is to predict the quality of an image as perceived by human observers without using a reference image. The authors explore a new approach which predicts the image quality based on the conformity of the first digit distribution (FDD) of natural images in the transform domain with Benford's law (BL). The conformity is measured by the symmetric Kullback–Leibler divergence. They first show that while in the transform domain the FDD of a natural image conforms with BL well, the FDD of a distorted natural image violates this conformity. They then train a non‐linear regression model which maps features derived from the FDD to the subjective evaluation score of an image. The non‐linear mapping is trained using Gaussian process regression with a rational quadratic kernel. The selection of this particular non‐linear regression tool is based on extensive experiments and evaluations of many regression tools. They conduct experiments to test the proposed technique using five databases. Results demonstrated that its performance is competitive with those state‐of‐the‐art blind IQA algorithms. In particular, the overall performance of the proposed technique is among the best in all algorithms tested.

The bilateral filter (BF) is a non‐linear filter that spatially smooths images with awareness of large structures such as edges. The level of smoothness applied to a pixel is constrained by a photometric weight, which can be obtained from the same image to be filtered (in case of the original BF) or from a guided image (in case of the joint/cross BF). In this study, the authors propose a new filter called the semi‐guided BF which is derived from solving a non‐linear constraint least square problem. The proposed filter's photometric weight incorporates information from the image to be filtered and the guided image. They propose a fast implementation of the filter based on layer approximation. They also study the iterative application of the proposed filter and show that the filter can preserve large structures while smoothing out small structures. This makes the proposed filter an efficient and effective tool for structure‐aware image smoothing. Experimental results have demonstrated that performance of the proposed filter is comparable to those of the state‐of‐the‐art algorithms.

Edge-aware smoothing has been extensively studied due to its wide range of applications in computer vision and graphics. Most published works have been focused on formulating the smoothing problem in the spatial domain. In this paper, we propose a new edge-aware smoothing framework called guided wavelet shrinkage (GWS), which is formulated in the wavelet domain as a maximum a posterior estimation problem. We impose a number of desirable properties on the statistical models and the associated parameters in order to derive an effective and computationally efficient algorithm. We compare the proposed GWS with classical image denoising in the wavelet domain. We have also compared different wavelet representations and found that the double-density dual-tree wavelet transform is the best choice. We show that the GWS can be configured as either self-guidance or external guidance. It can also be configured to operate in an iterative or non-iterative way. Experimental results and comparison with many state-of-the-algorithms demonstrate that the GWS-based algorithm can produce competitive results with O(N) computational complexity.

A new computationally efficient algorithm for two‐dimensional sliding‐window least‐squares prediction is presented in this study. The fast algorithm is based on a recursive update of the Cholesky decomposition. Compared with the state‐of‐the‐art algorithm, the proposed algorithm reduces the computational complexity from O ( D 3 ) to O ( D 2 h ), where D is the predictor order and h is the height of the prediction patch. The computational improvement is made at the stage of solving the normal equations for which an update algorithm for the Cholesky decomposition of the covariance matrix is proposed. It is shown that a large part of the Cholesky decomposition at location n can be efficiently calculated by performing orthonormal updates on the Cholesky decomposition at n − 1. The computational improvement is made without requiring additional storage space. Extensive experiments using causal and non‐causal predictors of varying shapes and sizes have confirmed that the proposed algorithm is consistently faster than the state‐of‐the‐art algorithm and produces identical prediction images. The efficiency of the proposed algorithm is shown to be affected by the order in which pixels are sampled, thus an ordering procedure is proposed to minimise the number of numerical operations.

There has been continuous research in edge-aware filters which have found many applications in computer vision and image processing. In this paper, we propose a principled-approach for the development of edge-aware filters. The proposed approach is based on two well-established principles: 1) optimal parameter estimation and 2) Bayesian model averaging (BMA). Using this approach, we formulate the problem of filtering a pixel in a local pixel patch as an optimal estimation problem. Since a pixel belongs to multiple local patches, there are multiple estimates of the same pixel. We combine these estimates into a final estimate using BMA. We demonstrate the versatility of this approach by developing a family of BMA filters based on different settings of cost functions and log-likelihood and log-prior functions. We also present a new interpretation of the guided filter and develop a BMA guided filter which includes the guided filter as a special case. We show that BMA filters can produce similar smoothing results as those of the state-of-the-art edge-aware filters. Two BMA filters are computationally as efficient as the guided filter which is one of the fastest edge-aware filters. We also demonstrate that the BMA guided filter is better than the guided filter in preserving sharp edges. A new feature of the BMA guided filter is that the filtered image is similar to that produced by a clustering process.

Image enlargement is a typical application of image processing. Many algorithms have been proposed and have found successful applications. Most of these algorithms are based on the theory of interpolation and make use of local structure of natural images. In this paper, we propose a new algorithm which is inspired by a new imaging sensor called the giga-vision sensor. In the proposed algorithm, an image is converted to an “imaginary” sensor domain in which the size of the image is many times bigger than that of the original image. Low pass filtering and ensemble filtering are then applied to the sensor domain image to produce an enlarged image. To evaluate the quality of the enlarged image, we proposed a spectral and an information based measures. Results show that the quality of the image produced by the proposed algorithm is comparable to that of cubic-spline based interpolation.

By using the triangular norm, we propose two methods for the construction of generalized linear systems, and show new insights into the relationship between typical systems. Using the Hamacher and Frank t-norm, we propose a parametric log-ratio model, which is a generalization of the log-ratio model and is more flexible in algorithmic development. We develop a generalized linear contrast enhancement algorithm based on the proposed parametric log-ratio model. We show that the performance of the proposed algorithm is effective and robust for different types of images.

Adaptive predictor combination (APC) is a framework for combining multiple predictors for lossless image compression and is often at the core of state-of-the-art algorithms. In this paper, a Bayesian parameter estimation scheme is proposed for APC. Extensive experiments using natural, medical, and remote sensing images of 8–16 bit/pixel have confirmed that the predictive performance is consistently better than that of APC for any combination of fixed predictors and with only a marginal increase in computational complexity. The predictive performance improves with every additional fixed predictor, a property that is not found in other predictor combination schemes studied in this paper. Analysis and simulation show that the performance of the proposed algorithm is not sensitive to the choice of hyper-parameters of the prior distributions. Furthermore, the proposed prediction scheme provides a theoretical justification for the error correction stage that is often included as part of a prediction process.

The logarithmic image processing (LIP) model is a mathematical theory providing generalized linear operations for image processing. The gigavision sensor (GVS) is a new imaging device that can be described by a statistical model. In this paper, by studying these two seemingly unrelated models, we develop a generalized LIP (GLIP) model. With the LIP model being its special case, the GLIP model not only provides new insights into the LIP model but also defines new image representations and operations for solving general image processing problems that are not necessarily related to the GVS. A new parametric LIP model is also developed. To illustrate the application of the new scalar multiplication operation, we propose an energy-preserving algorithm for tone mapping, which is a necessary step in image dehazing. By comparing with results using two state-of-the-art algorithms, we show that the new scalar multiplication operation is an effective tool for tone mapping.

Enhancement of contrast and sharpness of an image is required in many applications. Unsharp masking is a classical tool for sharpness enhancement. We propose a generalized unsharp masking algorithm using the exploratory data model as a unified framework. The proposed algorithm is designed to address three issues: (1) simultaneously enhancing contrast and sharpness by means of individual treatment of the model component and the residual, (2) reducing the halo effect by means of an edge-preserving filter, and (3) solving the out-of-range problem by means of log-ratio and tangent operations. We also present a study of the properties of the log-ratio operations and reveal a new connection between the Bregman divergence and the generalized linear systems. This connection not only provides a novel insight into the geometrical property of such systems, but also opens a new pathway for system development. We present a new system called the tangent system which is based upon a specific Bregman divergence. Experimental results, which are comparable to recently published results, show that the proposed algorithm is able to significantly improve the contrast and sharpness of an image. In the proposed algorithm, the user can adjust the two parameters controlling the contrast and sharpness to produce the desired results. This makes the proposed algorithm practically useful.

The DNA microarray may contain missing expression data. Estimation of missing values is a necessary step in microarray analysis, because data mining procedures require a complete expression as their input. In this paper, we propose a missing data estimation algorithm, named KPCAimpute, based on kernel principal component analysis. We consider a family of heavy-tailed kernel functions, which is a generalization of the famous Gaussian kernel. The performance of the proposed KPCAimpute algorithm is compared with two state-of-the-art linear regression methods, i.e., Bayesian principal component analysis imputation (BPCA) and local least squares imputation (LLSimpute). The KPCAimpute outperforms the LL-Simpute when the missing percentage increases. The performance of the KPCAimpute is similar to that of the BPCA imputation. Therefore, it is an effective and promising algorithm in estimating missing values for DNA microarray profiles.

The logarithmic image processing (LIP) model is a mathematical theory that provides new operations for image processing. The contrast definition has been shown to be consistent with some important physical laws and characteristics of human visual system. In this paper, we establish an information-theoretic interpretation of the contrast definition. We show that it can be expressed as a combination of the relative entropy and Shannon's information content. Based on this new interpretation, we propose an adaptive algorithm for enhancing the contrast and sharpness of noisy images.

In this paper we propose an alternative way to developing a robust and adaptive sequential algorithm for estimating the unknown impulse response of a linear system. Our approach is based on formulating the problem as a maximum penalized likelihood (MPL) problem. We use the Fair penalty function as the generalized log-likelihood and a quadratic function to play a regularization role. The MPL formulation also leads naturally to adaptive schemes for learning the regularization and scale parameters. The robustness of the proposed algorithm to impulsive noise is demonstrated through mathematical analysis and numerical simulations.

We develop an iterative algorithm based on minorization - maximization optimization to determine the maximum a posteriori estimate of the signal. We focus on linear Gaussian signal model with a family of heavy-tailed prior distributions which can be represented as scale mixture of Gaussian. We then modify the proposed algorithm for wavelet domain image denoising. Experimental results show that using complex wavelet representations, the performance of the proposed algorithm is very competitive with that of the state-of-the-art algorithms.

Signal processing using over-complete representations has been an active research field in recent years. In this article, we study the following two related problems: (1) given two wavelets and the Gaussian observation model, what is the optimal estimate of the signal which is corrupted by additive noise? and (2) to minimize the variance of the estimate, what is the relationship between the phase responses of the two scaling filters? Based on a study of these two problems, we develop a denoising algorithm. We test the proposed algorithm in image denoising and show that its performance is comparable to that of the state-of-the-art.

A fundamental problem in signal processing is to estimate signal from noisy observations. When some prior information about the statistical models of the signal and noise is available, the estimation problem can be solved by using the maximum a posteriori (MAP) principle. In this paper, we develop an EM algorithm for the MAP estimate of signals modeled by a family of heavy-tail prior distributions: Laplacian, student-t and slash. We establish links between the EM algorithm and the Wiener estimation. We then modify the EM algorithm and propose two generalized Wiener estimation algorithms for image denoising. Experimental results show that the performance of the proposed algorithms is better than that of the bi-shrinkage algorithm which is arguably one of the best in recent publications.

We present a study of a general framework for second-level adaptive prediction which is formed from a group of predictors. It is a natural extension to that of the first-level which is formed directly from a group of pixels. The proposed framework offers a greater degree of freedom for adaptation and addresses some of the tough problems such as model uncertainty that is inherent to the first-level prediction methods. We show that the proposed methods of taking weighted average (WAVE) and weighted median (WMED) of a group of predictions are alternative and competitive adaptive image prediction methods. We have achieved better compression performance than that of TMW/sup Lego/ by combining a group of linear predictors.

A family of biorthogonal wavelet filters for digital watermarking application is presented here. The filters are explicitly parametrized by two free parameters and can be used to provide diversity in watermarking. Diversity can be used to improve the security of the watermarking system from hostile attacks. Each filter has at least two vanishing moments which is important for ensuring some degree of smoothness in the resulting wavelet function.

In lossless image coding, it has been demonstrated that using the method of ordinary least squares (OLS) to design a linear predictor for each pixel results in better compression performance than that of the state-of-the-art. In previous studies, the order of the predictor is chosen empirically and fixed for the whole image. Since images are nonstationary signals, the order should be adapted to the local characteristics of the image. In this paper, we tackle this problem by using a model averaging approach. We show that by averaging over a group of OLS predictors, the effective number of parameter of the resultant predictor is adjusted adaptively. We show that the proposed method is robust to changes in the size of the training block. It also leads to better performance than the OLS predictor.

A study of a family of biorthogonal wavelet filters for use in digital watermarking is presented. The filters are explicitly parametrized by two free parameters and can be used to provide diversity in watermarking. Diversity can be used to improve the security of the watermarking system from hostile attacks. Each filter has at least two vanishing moments, which is important for ensuring some degree of smoothness in the resulting wavelet function. Along with robustness and security, other factors, which impact upon the successful implementation of the filters in a watermarking application are analysed. The relationship between the strength of the inserted watermark and wavelet energy is also discussed.

An efficient and robust approach to adaptive prediction is presented which uses a local causal area to evaluate a number of individual fixed sub-predictors. Various schemes are proposed to utilise this information, including a rank-order based approach, a two stage adaptive selection technique utilising median filtering, a technique for adaptive combination and a technique incorporating adaptive selection followed by adaptive combination. The respective selection and combination schemes display superior results for particular image types which, coupled with prediction error feedback and adaptive arithmetic coding, produce results superior to CALIC. To produce a more robust predictor a secondary stage based on one of the proposed selection schemes is applied resulting in an adaptive prediction scheme which effectively utilises the principles of predictor combination and selection.

We present a new prediction method for lossless image coding. In this prediction scheme, the prediction for each pixel is formed by adaptively combining the predicted values from a set of least squares based linear predictors. The combination scheme follows the main idea of the Bayesian model averaging to reduce prediction error due to model uncertainty. A lossless image coding algorithm based on this prediction method is also presented. Experimental results show that the compression performance of this algorithm is better than that of the TMW 0.51. It is also close to that of a newly improved version of TMW.

Common components in recently published lossless image compression algorithms include adaptive prediction, context-based error feedback and adaptive entropy coding. Each component has a number of building blocks. In this paper, we present a new algorithm which uses three new building blocks: an adaptive predictor, a symbol mapping scheme and a context filtering scheme. Experimental results show that the compression performance of the proposed algorithm is better than those of the three state-of-the-art algorithms: CALIC, HBB and LOGO. Experimental results also show that the proposed new building blocks are promising tools for lossless image compression.

Key building blocks of lossless image compression algorithms include adaptive prediction, context-based error feedback and adaptive entropy coding. This paper presents a new algorithm which includes two other building blocks-symbol mapping and context filtering. Experimental results show that the compression performance of the proposed algorithm is very close to that of CALIC and is better than that of LOGO and S+P. It is different from CALIC in the following aspects: (1) an adaptive median-FIR predictor, (2) a new error representation scheme using symbol mapping, (3) a different context calculation scheme for the prediction error, and (4) a new context filtering scheme.

Presented in this paper are two new adaptive prediction techniques for lossless, DPCM based, image compression. The first prediction scheme is derived from weighted median filter theory and extends on the median adaptive prediction (MAP) technique first presented by Martucci (1990). The second prediction technique is context modelling based and determines the optimum prediction by establishing the dominance of a particular predictor for each image context. The paper draws a direct comparison between MAP and the two new methods, and also between median predictor selection and context based predictor selection.

Although subband and wavelet coding have been proved to be very efficient for lossy image compression, it is generally not possible to represent the output of most subband/wavelet filters by a limited number of bits and yet guarantee perfect reconstruction. This is a fundamental problem when applying subband and wavelet coding to lossless image compression. This paper presents a solution to this problem by an investigation of interpolative subband coding. Based on an evaluation of the decorrelation performance of some typical image interpolation techniques, a new interpolative coding algorithm has been proposed. The new algorithm has been shown to be superior to two related algorithms, the hierarchical interpolation algorithm and the recently published minimum entropy pyramid algorithm.