Understanding leaf traits is crucial for the early detection of plant stress. Remote sensing offers the potential to estimate these traits non-destructively. This research explores the inversion of the PROSPECT Radiative Transfer Model for retrieving structural and biochemical properties of olive tree leaves from reflectance spectra within the visible-Near Infra-Red domain. To achieve this, we investigate the impact of fixed and variable Refractive Index (RI) on the estimation accuracy using two inversion methods: Look-Up Table (LUT) and Particle Swarm Optimization (PSO). Field-measured reflectance spectra using a spectrometer are used to evaluate the performance of the inversion methods. The Mean Squared Error (MSE) metric is used for this assessment. The results of the study demonstrate that the use of a variable RI significantly improves the accuracy of leaf trait estimation compared to a fixed RI approach for both inversion methods, which produce similar precisions.

In driver-car interaction scenarios, effective communication is crucial for ensuring safe and efficient transportation. Traditional methods of communication, such as audio speech commands or manual input, may not always be practical or safe, particularly in dynamic and noisy driving environments. Lipreading presents a promising alternative modality for communication, leveraging visual cues from facial expressions and lip movements. However, current lipreading methods rely on RGB cameras, which struggle with varying lighting in car cockpits, making near-infrared more suitable. Due to limited nearinfrared data, simple fine-tuning of RGB models is insufficient. Therefore, in order to transfer Learning for Limited-Data InfraRed Lipreading Model: We propose a knowledge distillation method that transfers features from pre-trained RGB models to near-infrared models, improving performance with limited nearinfrared data. In addition, to backup our finding we introduce LR-CAR, the first dataset for driver-car interaction with both modalities. The obtained results demonstrates that our approach improves lipreading performance showing significant gains by 16.78% over naive fine-tuning,

In this study, we propose an innovative approach to segment images of olive tree branches taken by smartphones to detect olives and leaves. The methodology relies on the self-creation of a labelled database (e.g., olives, leaves, and others) using Grounding-Dino and SAM for detection and segmentation, respectively. We exploit the performance of DINOv2 to compute embeddings for each class. These embeddings are then used to train a Support Vector Machine (SVM) that classifies objects in photos (olives/leaves). Finally, the image is segmented into objects using SAM, and each object is classified by the trained SVM to determine their respective classes. The proposed approach not only ensures the automatic construction of our specific database but also demonstrates high performances that surpass RESNET50.

Predicting building heights based solely on RGB images is a complex task due to a range of challenges, including limited data availability, data quality issues, diverse building types, and the cost of acquiring multi-view geospatial imagery. In response to these challenges, we have designed a novel multi-task learning model for estimating elevations from a single remotely sensed image where a segmentation step is associated to the height estimation model to enhance height predictions. Our experimental results, conducted on the GeoNRW reference dataset, demonstrate the superiority of our multi-task model when involving instance segmentation in terms of Mean Square Error (MSE). Indeed, we obtain a MSE of 3.84, in comparison to the single-task approach where no segmentation step is considered (MSE = 10.32).

In Tunisia, olive tree cultivation is a significant agricultural asset. Ensuring the sustainability and optimal yield, both in terms of quality and quantity, of these tree orchards is therefore crucial. It needs careful monitoring due to its susceptibility to various anomalies, water stress, and nutrient deficiency. This can be achieved by observing the biophysical properties of the trees, such as chlorophyll content (Cab) and leaf area index (LAI), to assess their growth and health. This is accomplished by utilizing both time-series imagery from the high spectral resolution Sentinel-2 and high spatial resolution Pleiades sensors. Establishing the relationship between the images from one side and the biophysical properties from the other requires the inversion of radiative transfer models (RTM). RTM are time-consuming, thus their inversion is impractical. Our approach involves designing a fast RTM emulator based on an artificial neural network (ANN). Our inversion technique is based on the multi-scale variational approach to accurately retrieve the required properties from the satellite image time series. It allows convergence towards the global optimal solution since it is able to avoid local optima. The suggested approach for inverting RTM Emulator promises superior retrieval performance, achieving remarkably low RMSE values of 0.03 and $1.57\mu g|cm^{2}$ for LAI and Cab, respectively.

Thermal imaging of unmanned aerial vehicles (UAVs) can sometimes suffer from a lack of information due to their small sizes. Therefore, the ability to understand and analyze such images of drones will be limited. Nevertheless, high-resolution (HR) visible images are often available and could be useful for improving the resolution of thermal images from UAVs. In recent years, Deep learning has been increasingly used in several computer vision tasks such as super-resolution (SR), where it has shown promising results for image resolution enhancement as it allows for creating high-quality detailed images. In this paper, we propose a Guidance Super-Resolution Network (GSRNet), that improves the spatial resolution of thermal UAVs images by taking advantage of the textures of the visible images. We adapt a Convolutional Neural Network (CNN) model that has an encoder-decoder architecture to translate visible images into thermal images as well as an auto-attention mechanism to allow the network to selectively focus on relevant structures of the image while ignoring irrelevant parts. Moreover, to preserve the low frequency information such as the brightness level and the body that are present in the low-resolution (LR) thermal image, we propose to merge the letter image with the translated one, such that the obtained HR thermal image when downsampled it equals the original LR one. Experimental results on the custom UAVs image dataset prove the higher performance of the proposed model on both qualitative and quantitative evaluations when compared to several state-of-the-art (SOTA) methods.

Monitoring vegetation growth, phenology and health in agriculture requires very high spatial and spectral resolution sensors. Sentinel-2 is among the spaceborne sensors that tried to meet these requirements. However, due to physical constraints, few visible bands are present with relatively large spectral responses which limits the leaf pigment content estimation. In this work, we propose to fuse Sentinel-2 bands with a Sentinel-3 image which has a lower spatial resolution but contains several spectral narrower bands in both the visible and near-infrared domains. The fusion procedure consists in sharpening the Sentinel-3 bands to match the Sentinel-2 spatial resolution leading to a higher spatial resolution Sentinel-3 image. The proposed fusion technique follows a physics-based approach based on the use of a radiative transfer model for establishing a correspondence between Sentinel-2 and Sentinel-3 images. In greater details, the main spectrally pure constituent of each high resolution Sentinel-2 pixel is identified for each pixel and it is then related to the corresponding low resolution Sentinel-3 pixel. Pure materials are the barycenters of clusters obtained by an unsupervised classification of the Sentinel-2 image. Sentinel-3 signatures are obtained using coarse resolution pixel matching with the Sentinel-2 image. The latter result is then corrected using radiative transfer modeling allowing the production of more realistic signatures. Validation was done using real Sentinel-2/Sentinel-3 images taken with a delay of only one day or less and by comparing the sharpened Sentinel-3 bands Oa04, Oa06, Oa08, and Oa17 with the corresponding Sentinel-2 bands B2, B3, B4, and B8A, as they are respectively spectrally close. For all the bands, the RMS is lower than 0.007, and compared to the state-of-the-art techniques it shows competitive results and robustness against scene heterogeneity. Besides, our findings prove that the retrieved Sentinel-3 signatures at full resolution are physically consistent and in good agreement with the Sentinel-2 data.

Precise monitoring of the crop growth is useful for agriculture management systems that allows to improve and sustain food security. Measuring the biophysical properties such as Leaf area index (LAI) and chlorophyll (Cab) content are the key to properly follow the growth cycle of crops. In this perspective, we propose an innovative approach for monitoring landscape-scale biophysical properties of wheat and barley field crops using Sentinel-2 time-series imagery based on accurate 3D direct and inverse radiative transfer modeling. It relies on an original 3D radiative transfer model emulator architecture based on a residual artificial neural network (ResNet). We employed the well-known Discrete Anisotropic Radiative Transfer (DART) model to simulate Sentinel-2 images and generate a database that serves to train and validate the ResNet emulator. To do it, two realistic mockups both composed of soil and 3D wheat plants are set up. The first mockup mimics the wheat and barley from their start of growth to their full growth and the second reproduce the features of the wheat and barley plant when they turn yellowish and mature for harvesting. Besides, we extract the bare soil spectra signature from Sentinel-2 images acquired before sowing. Results of the proposed emulator show its similarity to DART simulation while it is much faster. These emulator features let it possible to adapt an inversion technique based on the Markov Chain Monte Carlo (MCMC), specifically, as novelty, a continuous MCMC. The latter exploits the emulator to perform the iterative sampling processing on all continuous values in allowed intervals according to each sought biophysical property, until converging to its respective posterior distribution. The proposed scheme of RTM Emulator inversion based on continues MCMC and using Sentinel-2 Imagery is called REMI. It ensures high retrieval performances which reach a RMSE equal to 0.06 and 0.1 for LAI and Cab, respectively.

In Tunisian agriculture, olive tree cultivation plays an important role. It is affected by different stresses that jeopardize its sustainability. In this context, our objective is to enhance the resilience of this crop. To achieve this goal, our work consists of detecting anomalies at early stage starting from the tree to the field scale. The proposed solution takes advantage of the emergence of satellites with high spatial and temporal resolution. In particular, the Sentinel-2 sensor which is well-adapted to monitor the vegetation. It is characterized by ten spectral bands allowing to access to key vegetation properties such as leaf area index (LAI), chlorophyll content (Cab) and water content (Cw), etc. Direct estimation of these parameters for the image is not practical as the signal is convolved. For that, we use artificial intelligence techniques to separate the effects of the different properties. We develop an Artificial Neural Network (ANN) that learn to estimate the vegetation properties given the pixel signature. The learning is done using a database of simulated data produced by a radiative transfer model that simulates the satellite image given the vegetation cover properties. The stress detection is based on a threshold on tree LAI and Cab. Comparison with ground truth with healthy and stressed plots has shown the validity of our approach.

International audience

Pan-sharpening is a fusion task, which aims to combine a low spatial resolution multispectral (MS) image with a high spatial resolution single band panchromatic (PAN) image to produce a high spatial and spectral Pan-sharpened image. The success of a Pan-sharpening technique depends on its ability to boost the spatial quality of the MS image while preserving its spectral feature. To this end, we propose in this article a new two-stage detail injection approach allowing to reconstruct fine structures based on convolutional neural networks (CNNs). First, generalized Laplacian pyramid gain injections CNN is performed to estimate the optimal values of the injection gains for each MS band to inject spatial details extracted from the PAN image. Next, the result is enhanced by injecting the details missing using the power of deep residual learning. The quantitative and qualitative results on data sets from different satellites show that the proposed approach can achieve higher performances in both spatial and spectral qualities compared to the state of the art as well as the new CNN-based methods.

Physical models simulating the radiative budget (RB) and remote sensing (RS) observation of three-dimensional (3D) landscapes are critical to better understand human and natural components of the Earth system and further develop RS technology. DART is one of the most comprehensive 3D models of Earth-atmosphere optical radiative transfer (RT), from ultraviolet (UV) to thermal infrared (TIR). It simulates the optical signal of proximal, aerial and satellite imaging spectrometers and laser scanners, the 3D RB and solar induced chlorophyll fluorescence (SIF) signal, for any urban or natural landscape and any experimental or instrument configuration. It is freely available for research and teaching activities (https://dart.omp.eu). Here, five recent advances are presented. 1) Atmosphere RT. 2) RT in non repetitive topography. 3) Monte Carlo modelling for fast RS image simulation of large landscapes. 4) SIF modelling for vegetation simulated as facets and turbid cells. 5) RS image inversion for mapping the optical properties of urban material and the urban radiative budget.

This paper presents new approaches to forecast the plant growth based on statistical methods: autoregressive and Markov chain models, using a time series of the normalized different vegetation index. Here, a monthly normalized different vegetation index time series was derived from Sentinel-2 over Limaya olive tree fields from January 2016 to November 2019. To ensure consistent prediction, processing is done over homogeneous clusters of vegetation. Finally, the performance of our approach is evaluated by means of the root mean square error between the predicted and true values.

The potential of solar-induced chlorophyll fluorescence (SIF) to monitor photosynthesis and plant stress has attracted considerable interest in SIF remote sensing (RS). However, canopy SIF and RS observations are impacted by topography, vegetation three dimension (3D) structure, leaf orientation, non foliar elements (e.g., tree woody skeleton), ... Physically based downscaling of canopy SIF RS data to leaf-level (i.e., to leaf photosynthesis) requires 3D radiative transfer (RT) models simulating canopy SIF and its observation. These models are necessary to better exploit the potential of SIF, by linking leaf SIF and SIF in RS observations as a function of canopy 3D architecture and experimental configurations (sun and viewing directions, etc.). The Discrete Anisotropic Radiative Transfer (DART) model is a comprehensive 3D radiative transfer (RT) model for urban and natural landscapes. This paper presents its SIF modeling for vegetation simulated with facets, its validation with the SCOPE/mSCOPE 1D models, and its recent extension to SIF modelling for landscapes simulated with 3D turbid medium.

Pan-sharpening aims to fuse a low-spatial-resolution multispectral (MS) image with an associated higher resolution panchromatic image (PAN) in order to produce a high-resolution MS (HRMS) image to overcome physical limitation of satellite sensors. In this letter, we propose a new generalized Laplacian pyramid gain injection prediction based on convolutional neural networks (GIP-CNN) for pan-sharpening, which estimates the values of the injection gains for each MS band to complement it with spatial details extracted from the PAN image. The experimental results on images from different satellites show that GIP-CNN can achieve higher performances with respect to the state-of-the-art and new CNN-based methods in both spatial and spectral qualities.

This work proposes a new approach to estimate two canopy structure properties: leaf area index (LAI) and leaf angle distribution (LAD) using terrestrial LiDAR system (TLS) data. Our methodology consists of two steps. First, a forward model was developed to simulate TLS observations of a vegetation scene having known structure variables (i.e. LAI and LAD) which permit obtaining 3D point cloud representing the studied scene. Second, a backward model was designed to retrieve LAI and LAD based on the relationship between light transmittance and foliage density. Our approach was validated with results obtained with different homogenous vegetation covers.

In this paper, we study the estimation of olive tree biophysical properties driven by Sentinel-2 (S2) image inversion. The latter is based on the forward/backward radiative transfer (RT) model. The forward step is done simulating DART on a realistic tree mock-up, whereas the backward is done based on a coupling between the Look UP Table (LUT) and the Markov Chain Monte Carlo (MCMC). The parameters Leaf area index (LAI), chlorophyll (Cab) water (Cw) contents and mesophyll structure (N) are derived. The results are promising, in particular LAI and Cab values are close to those found in literature.

In this paper, we investigate the estimation of crop biophysical properties from small footprint LiDAR waveforms inversion. Due to crop heterogeneity within the same agricultural field, a classification on similar waveform clusters is performed before inversion. A Look up table (LUT) approach was adapted then, to derive the height and LAI of maize and wheat crops. The LUT was generated using the Discrete Anisotropic Radiative Transfer (DART) by simulating the LiDAR observations which are used to search for the suitable set of biophysical properties describing the different crops clusters. The results are promising. Crops height is accurately estimated with a root mean square error (RMSE) of 0.06m and 0.03m for maize and wheat, respectively. LAI was well estimated with RMSE of 0.07 and 0.43 for maize and wheat, respectively.

Among others, the wavelet-based pansharpening approach tries to enhance the resolution of the multispectral (MS) image by injection of spatial details extracted from the high-resolution panchromatic (PAN) image. The problem is presented as follows, the inputs are a coarse-resolution MS image and a high-resolution detail image provided from the PAN image; therefore, one would think that the wavelet reconstruction allows combining approximations and details to construct the high-resolution MS image. However, the wavelet transform (WT) assumes that details and approximations are calculated using the same wavelet decomposition. Now, in the pansharpening case, the MS low-resolution image is assumed to be aliased and blurred due to the imaging system modulation transfer function (MTF) that is approximated as a specific low-pass filter. Meanwhile, there are no constraints about details that can be extracted from PAN using discrete WT (DWT). Approximation and details are not any more orthogonal as needed in the reconstruct of the MS high-resolution image based on DWT. For that, we propose in this paper a new fusion schema [coupled multiresolution decomposition model (CMD)] allowing the reconstruction of a high-resolution MS given its approximation and details obtained by MTF-tailored downsampling and wavelet decomposition, respectively. For validation, CMD is applied to Pléiades, GeoEye-1, and SPOT 6 images. Compared to other approaches [i.e., Gram-Schmidt (GS) adaptive, GS mode 2 (GS2), “À trous' WT (AWT), generalized Laplacian pyramid (GLP), DWT, and PCI Geomatics software algorithm], our method performs generally better.

This study focuses on the problem of object reconstruction through several frames of a video sequence. Elementary detections on which this reconstruction is based are assumed to be fragments of the objects. Belief function framework allows then the modelling of the uncertain and imprecise location of these object fragments within the image. We show that the two competing mechanisms for object reconstruction, namely the data accumulation and their temporal removal or weighting, can be implemented using belief function operators. Results illustrate the robustness of the proposed approach to object partial occultation and crossing.

The spatial resolution of thermal infrared (TIR) instruments is often not sufficient for many applications, but this low resolution is counterbalanced by the high temporal resolution (for example the SEVIRI instrument onboard the European Meteosat 8 and 9 presents a spatial resolution of 3 km$ \times$3 km at nadir and a temporal resolution of 15 mn). At kilometric scales, the observed pixel is generally heterogeneous in terms of land cover, and the temperatures of the different components may present large discrepancies. This paper presents a methodology to infer the temperatures of the various land cover/use classes composing a mixed pixel, from a whole pixel measurement. To infer intra-pixel temperature, information on the mixture within each low resolution pixel, e.g., the proportions of the land cover types derived from high spatial resolution imaging, account for a first constraint. However, in the absence of supplementary constraints, the number of unknown variables is greater than the number of measurements, and there is not uniqueness of the solution. Thus, we propose to take advantage of a priori knowledge provided by a land surface model (LSM), and of the temporal and spatial correlation features of the surface temperature. We propose a new downscaling method for estimating sub pixel signal. It applies to TIR data and: the inversion procedure provides as a result, the land surface temperature (LST) temporal series of each land cover/use class (called endmember) constituting the coarse resolution pixel. Three kinds of a priori information have been introduced, namely 1) a first guess subpixel temperature derived from the SEtHyS LSM; 2) a Markov Random Chain model of the surface temperature temporal dependencies from times$t$to$t + 1$; 3) a Markov Random Field model of the spatial dependencies between endmember temperatures. Then, the “Maximum A Posteriori” estimator provides the most likely endmember temperatures, given 1) the observed coarse resolution temperatures, 2) the composition of the pixels in terms of “land cover/land use,” and 3) the LSM first guess subpixel temperature values, 4) the a priori spatial and temporal Markov models. The performance of this new method has been first evaluated on simulated data (random Gaussian variables with means equal to endmember temperatures simulated using LSM). The method accuracy versus the observation errors and the number of endmembers was analyzed. The algorithm was then run on actual data, namely Meteosat SEVIRI Land Surface products acquired over an agricultural region in southeastern France. The performance evaluation was done by comparing the subpixel LST estimations to the high-resolution temperatures provided by the Terra/ASTER instrument. Due to the huge bias between sensors ($ \sim$4 K), an intercalibration preprocessing between SEVIRI and ASTER was done. In this case, the achieved RMSE is lower than 2 K.

The paper presents a new methodology adapted to the downscaling of low resolution IRT signals, i.e. the estimation of subpixel temperatures. The approach is based on the inversion of subpixel variables by multilinear regressions constrained by a priori temperature estimates provided by a physical land surface model. The method was developed and validated against a synthetic database built on model simulations. The precision of the methodology was analysed in terms of errors on the subpixel temperature estimations according to model and observation uncertainties. The impact of the number of observations used (i.e. the number of low resolution pixels considered) as well as the influence of the pixel heterogeneity were studied.

The goal of this paper is to propose a methodology based on vegetation index fusion to provide an accurate estimation of the fraction of vegetation cover (fCover). Because of the partial and imprecise nature of remote-sensing data, we opt for the evidential framework that allows us to handle such kind of information. The defined fCover belief functions are continuous with the interval [0, 1] as a discernment space. Since the vegetation indices are not independent (e.g., perpendicular vegetation index and weighted difference vegetation index are linearly linked), we define a new combination rule called “cautious adaptive” to handle the partial “nondistinctness” between the sources (vegetation indices). In this rule, the “nondistinctness” is modeled by a factor$\varrho$varying from zero (distinct sources) to one (totally correlated sources), and the fusion rule varies accordingly from the conjunctive rule to the cautious one. In terms of results, both in the cases of simulated data and actual data, we show the interest of the combination of two or three vegetation indices to improve either the accuracy of fCover estimation or its robustness.

The SAIL model (proposed by Verhoef) is largely used in the remote sensing community to calculate the canopy Bidirectional Reflectance Distribution Function. The simulation results appear acceptable compared to observations especially for not very dense planophile vegetation. However, for erectophile dense crops (e.g. corn) the simulations appear less accurate. This inadequacy is due to the assumption that the multiple scattered are isotropically distributed. The SAIL parameters are interpretable at the level of elementary layer components. Now, the Adding method (initially proposed by Van de Hulst) provides a good framework to model the radiative transfer inside a vegetation layer, but its parameter estimation lies on very simple geometric modeling of the canopy. In this paper, we propose an adaptation of the Adding method using the SAIL model canopy representation. Such an approach allows both to overcome the isotropy assumption and to take into account the multi hot spot effect. It also allows to check the energy conservation in both turbid and discrete case.

Ant colony optimization (ACO) has been proposed as a promising tool for regularization in image classification. The algorithm is applied here in a different way than the classical transposition of the graph color affectation problem. The ants collect information through the image, from one pixel to the others. The choice of the path is a function of the pixel label, favoring paths within the same image segment. We show that this corresponds to an automatic adaptation of the neighborhood to the segment form, and that it outperforms the fixed-form neighborhood used in classical Markov random field regularization techniques. The performance of this new approach is illustrated on a simulated image and on actual remote sensing images.

Digital change detection deals with the quantification, from multi-date imagery, of temporal phenomena, such as Aforestation-Reforestation-Deforestation, agricultural field rotation, abnormal evolution of the land surface. Despites the numerous change detection indices already proposed, none is sufficiently precise and reliable. We propose then to detect changes by considering not only one but several change indices, as well as information available from other source than remote sensing, i.e. derived from surface evolution model or a priori. For fusion, we chose the framework of the Dempster-Shafer evidence theory. It allows for some global ignorance, which is either present at the borders between the 'No-Change' and 'Change' classes, or is due to the poor quality of some change indices. The performance of the Non Remote Sensing (NRS) data change prediction, when known (e.g. statistical error of a model), can be taken into account the discounting of the mass functions. We present the results obtained in two different cases of application: forest logging and winter vegetation cover of fields in intensive farming areas. Remote sensing data are SPOT/HRV images. Considering the performance in terms of Non-Detection and False Detection rates, the interest of combining at least two change indices was clearly stated. The interest of NRS information has been then evaluated in the case of the field winter coverage application.