Neural Ordinary Differential Equations (NODEs) serve as continuous-time analogs of residual networks. They provide a system-theoretic perspective on neural network architecture design and offer natural solutions for time series modeling, forecasting, and applications where invertible neural networks are essential. However, these models suffer from slow performance due to heavy numerical solver overhead. For instance, a popular solution for training and inference of NODEs consists in using adaptive step size solvers such as the popular Dormand-Prince 5(4) (DOPRI). These solvers dynamically adjust the Number of Function Evaluations (NFE) as the equation fits the training data and becomes more complex. However, this comes at the cost of an increased number of function evaluations, which reduces computational efficiency. In this work, we propose a novel approach: making the parameters of the numerical integration scheme trainable. By doing so, the numerical scheme dynamically adapts to the dynamics of the NODE, resulting in a model that operates with a fixed NFE. We compare the proposed trainable solvers with state-of-the-art approaches, including DOPRI, for different benchmarks, including classification, density estimation, and dynamical system modeling. Overall, we report a state-of-the-art performance for all benchmarks in terms of accuracy metrics, while enhancing the computational efficiency through trainable fixed-step-size solvers. This work opens up new possibilities for practical and efficient modeling applications with NODEs.

Data-driven approaches, particularly those based on deep learning, are rapidly advancing Earth system modeling. However, their application to ocean forecasting remains limited despite the ocean's pivotal role in climate regulation and marine ecosystems. To address this gap, we present OceanBench, a benchmark designed to evaluate and accelerate global short-range (1–10 days) data-driven ocean forecasting.OceanBench is constructed from a curated dataset comprising first-guess trajectories, nowcasts, and atmospheric forcings from operational physical ocean models, typically unavailable in public datasets due to assimilation cycles. Matched observational data are also included, enabling realistic evaluation in an operational-like forecasting framework.The benchmark defines three complementary evaluation tracks: (i) Model-to-Reanalysis, where models are compared against the reanalysis dataset commonly used for training; (ii) Model-to-Analysis, assessing generalization to a higher-resolution physical analysis; and (iii) Model-to-Observations, Intercomparison and Validation (IV-TT) CLASS-4 evaluation against independent observational data. The first two tracks are further supported by process-oriented diagnostics to assess the dynamical consistency and physical plausibility of forecasts.OceanBench includes key ocean variables: sea surface height, temperature, salinity, and currents, along with standardized metrics grounded in physical oceanography. Baseline comparisons with operational systems and state-of-the-art deep learning models are provided. All data, code, and evaluation protocols are openly available at https://github.com/mercator-ocean/oceanbench, establishing OceanBench as a foundation for reproducible and rigorous research in data-driven ocean forecasting.

Monitoring optical properties of coastal and open ocean waters is crucial to assessing the health of marine ecosystems. Deep learning offers a promising approach to address these ecosystem dynamics, especially in scenarios where gap-free ground-truth data is lacking, which poses a challenge for designing effective training frameworks. Using an advanced neural variational data assimilation scheme (called 4DVarNet), we introduce a comprehensive training framework designed to effectively train directly on gappy data sets. Using the Mediterranean Sea as a case study, our experiments not only highlight the high performance of the chosen neural network in reconstructing gap-free images from gappy datasets but also demonstrate its superior performance over state-of-the-art algorithms such as DInEOF and end-to-end neural mapping schemes based CNN or UNet architectures.

In oceanography, the acquisition and processing of observations are crucial to improve the understanding of complex oceanic processes. Considering an idealized model of the North Atlantic ocean circulation, we propose to implement a variational data assimilation method optimized by deep learning to reconstruct abrupt changes in ocean circulation, representing Dansgaard-Oeschger climate events. We show that this assimilation method leads to improved reconstruction performances, particularly at low sampling frequencies. Focusing on this difficult latter case, four sampling strategies are studied more specifically. Our experiments highlight that clusters of three consecutive observations regularly sampled leads to a better monitoring of the ocean circulation regime shifts. These results pave the way for further research in optimal ocean observation.

When considering the data-driven identification of non-linear differential equations, the choice of the integration scheme to use is far from being trivial and may dramatically impact the identification problem. In this work, we discuss this aspect and propose a novel architecture that jointly learns Neural Ordinary Differential Equations (NODEs) as well as the corresponding integration schemes that would minimize the forecast of a given sequence of observations. We demonstrate its relevance with numerical experiments on non-linear dynamics, including chaotic systems.

The monitoring of waves at the ocean surface is critical for both operational needs (e.g., maritime traffic) and scientific studies (e.g., air-sea interactions). Synthetic aperture radar (SAR) Satellites provide one of the only remote sensing observations to retrieve ocean wave information on a global scale. However state-of-the-art SAR processing schemes often lead to poor inversion performance due to overly-simplistic assumptions. Here we leverage deep learning schemes to address these shortcomings. We state the targeted measurement of the ocean wave spectrum at sea surface as a neural mapping from SAR satellite observations. We exploit supervised deep learning schemes trained from a large-scale collocation dataset between real SAR observations and Wavewatch III model data. Our results emphasize for the first time how deep learning schemes can outperform the state-of-the-art analytical SAR-based inversion with an improvement in terms of mean square error greater than 65%. We analyse and discuss further the key features of the trained neural processing.

This study presents an enhanced approach to ocean colour L4 product generation through the Adaptive Spatial and Multi-Variable Generalization of 4DVarNet - an innovative integration of deep neural networks with variational data assimilation proposed in [1]. We explore the model’s capabilities in generalizing across various geographical regions and bio-optical variables using datasets of the North Sea and the Mediterranean Sea. Our analysis and visualization show that 4DVarNet demonstrates a notable ability to adapt and scale, reducing significantly training cost thanks to this generalization ability.

Remote sensing of rainfall events is critical for both operational and scientific needs, including for example weather forecasting, extreme flood mitigation, water cycle monitoring, etc. Ground-based weather radars, such as NOAA’s Next-Generation Radar (NEXRAD), provide reflectivity and precipitation estimates of rainfall events. However, their observation range is limited to a few hundred kilometers, prompting the exploration of other remote sensing methods, particularly over the open ocean, that represents large areas not covered by land-based radars. Here we propose a deep learning approach to deliver a three-class segmentation of SAR observations in terms of rainfall regimes. SAR satellites deliver very high resolution observations with a global coverage. This seems particularly appealing to inform fine-scale rain-related patterns, such as those associated with convective cells with characteristic scales of a few kilometers. We demonstrate that a convolutional neural network trained on a collocated Sentinel-1/NEXRAD dataset clearly outperforms state-of-the-art filtering schemes such as the Koch’s filters. Our results indicate high performance in segmenting precipitation regimes, delineated by thresholds at 24.7, 31.5, and 38.8 dBZ. Compared to current methods that rely on Koch’s filters to draw binary rainfall maps, these multi-threshold learning-based models can provide rainfall estimation. They may be of interest in improving high-resolution SAR-derived wind fields, which are degraded by rainfall, and provide an additional tool for the study of rain cells.

Sea surface height (SSH) is a key geophysical parameter for monitoring and studying meso-scale surface ocean dynamics. For several decades, the mapping of SSH products at regional and global scales has relied on nadir satellite altimeters, which provide one-dimensional-only along-track satellite observations of the SSH. The Surface Water and Ocean Topography (SWOT) mission deploys a new sensor that acquires for the first time wide-swath two-dimensional observations of the SSH. This provides new means to observe the ocean at previously unresolved spatial scales. A critical challenge for the exploitation of SWOT data is the separation of the SSH from other signals present in the observations. In this paper, we propose a novel learning-based approach for this SWOT calibration problem. It benefits from calibrated nadir altimetry products and a scale-space decomposition adapted to the structure of the different processes in play in the SWOT’s swath geometry. In a supervised setting, our method reaches the state-of-the-art residual error of ≈ 1.4cm while proposing a correction on the entire spectrum from 10km to 1000km and with less restrictive constraints on the modeled error signal.

We address Lagrangian drift simulation in geophysical dynamics and explore Deep Learning approaches to overcome known limitations of state-of-the-art model-based and Markovian approaches in terms of computational complexity and error propagation. We introduce a novel architecture, referred to as DriftNet, inspired from the Eulerian Fokker-Planck representation of Lagrangian dynamics. Numerical experiments for Lagrangian drift simulation at the sea surface demonstrates the relevance of DriftNet w.r.t. state-of-the-art schemes. Benefiting from the convolutional nature of DriftNet, we explore through a neural inversion how to diagnose model-derived velocities w.r.t. real drifter trajectories.

Optical remote sensing is increasingly used to assess various sea surface biogeochemical parameters (e.g., Chl-a [1] , turbidity [2] ). If today's systems offer a better spatiotemporal coverage, it still depends on both the satellite revisit period and the cloud cover at the time of the acquisition. The resulting sea surface observations generally present large proportions of missing data, limiting their use. Typically, in our experiments, we worked with datasets containing up to 98% of missing data.

Characterization of suspended sediment dynamics in the coastal ocean provides essential information for scientific studies and operational challenges concerning, among others, turbidity, water transparency and the development of microorganisms using photosynthesis, which is critical for primary production. The complexity of the processes involved in sediment dynamics makes it difficult to predict surface dynamics. In the continuity of previous experiments, the 4DVarNet model having shown encouraging results with SSSC interpolations, it is tested in a 20-day forecasting problem. In addition to the learning architecture including the missing observation data, a protocol has been conceptualized to integrate different types of forcing to improve the reconstructions. The results of the method show that it is possible to produce satisfactory results. The results of the method show that it is possible to produce satisfactory results. The contribution of the input forcing is notable improving of 20% precision horizons. The study also highlights a characterization of the different input forcing and their effect on the system.

The ocean is a crucial component of the Earth's system. It profoundly influences human activities and plays a critical role in climate regulation. Our understanding has significantly improved over the last decades with the advent of satellite remote sensing data, allowing us to capture essential sea surface quantities over the globe, e.g., sea surface height (SSH). Despite their ever-increasing abundance, ocean satellite data presents challenges for information extraction due to their sparsity and irregular sampling, signal complexity, and noise. Machine learning (ML) techniques have demonstrated their capabilities in dealing with large-scale, complex signals. Therefore we see an opportunity for these ML models to harness the full extent of the information contained in ocean satellite data. However, data representation and relevant evaluation metrics can be the defining factors when determining the success of applied ML. The processing steps from the raw observation data to a ML-ready state and from model outputs to interpretable quantities require domain expertise, which can be a significant barrier to entry for ML researchers. In addition, imposing fixed processing steps, like committing to specific variables, regions, and geometries, will narrow the scope of ML models and their potential impact on real-world applications. OceanBench is a unifying framework that provides standardized processing steps that comply with domain-expert standards. It is designed with a flexible and pedagogical abstraction: it a) provides plug-and-play data and pre-configured pipelines for ML researchers to benchmark their models w.r.t. ML and domain-related baselines and b) provides a transparent and configurable framework for researchers to customize and extend the pipeline for their tasks. In this work, we demonstrate the OceanBench framework through a first edition dedicated to SSH interpolation challenges. We provide datasets and ML-ready benchmarking pipelines for the long-standing problem of interpolating observations from simulated ocean satellite data, multi-modal and multi-sensor fusion issues, and transfer-learning to real ocean satellite observations. The OceanBench framework is available at https://github.com/jejjohnson/oceanbench and the dataset registry is available at https://github.com/quentinf00/oceanbench-data-registry.

The space-time reconstruction of sea surface dynamics from satellite observations is a challenging inverse problem due to the associated irregular sampling. Satellite altimetry provides a direct observation of the sea surface height (SSH), which relates to the divergence-free component of sea surface currents. The associated sampling pattern prevents operational schemes from retrieving fine-scale dynamics, typically below 10 days. By contrast, other satellite sensors provide higher-resolution observations of sea surface tracers such as sea surface temperature (SST). Multimodal inversion schemes then arise as appealing approaches. Though theoretical evidence supports the existence of an explicit relationship between sea surface temperature and sea surface dynamics under specific dynamical regimes, the generalization to the variety of upper ocean dynamical regimes is complex. Here, we investigate this issue from a physics-informed learning perspective. We introduce a trainable multimodal inversion scheme for the reconstruction of sea surface dynamics from multi-source satellite-derived observations, namely satellite-derived SSH and SST data. The proposed multimodal 4DVarNet schemes combine a variational formulation involving trainable observation anda prioriterms with a trainable gradient-based solver. An observing system simulation experiment for a Gulf Stream region supports the relevance of our approach compared with state-of-the-art schemes. We report a relative improvement greater than 60% compared with the operational altimetry product in terms of root mean square error and resolved space-time scales. We discuss further the potential and the limitations of the proposed approach for the reconstruction and forecasting of geophysical dynamics from irregularly-sampled satellite observations.

Satellite radar altimeters are a key source of observation of ocean surface dynamics. However, current sensor technology and mapping techniques do not yet allow to systematically resolve scales smaller than 100km. With their new sensors, upcoming wide-swath altimeter missions such as SWOT should help resolve finer scales. Current mapping techniques rely on the quality of the input data, which is why the raw data go through multiple preprocessing stages before being used. Those calibration stages are improved and refined over many years and represent a challenge when a new type of sensor start acquiring data. Here we show how a data-driven variational data assimilation framework could be used to jointly learn a calibration operator and an interpolator from non-calibrated data . The proposed framework significantly outperforms the operational state-of-the-art mapping pipeline and truly benefits from wide-swath data to resolve finer scales on the global map as well as in the SWOT sensor geometry.

In this work, we focus on the estimation of sea surface current using Automated identification system (AIS) data streams in the Mediterranean sea. We propose to use deep learning techniques to solve the associated ill-posed inverse problem, for methodological purpose we compare two differ-ent approaches, the first one relies on a physical constrained unsupervised technique whereas the seconds exploit a super-vised framework and a dataset of in-situ observation from HF Radar. Performances are evaluated using ground-truth measurement provided by drifting buyos and HF Radar over area of the Sicily channel. We show that both AIS-derived product outperform satellite-altimetry derived ones in terms of reconstruction criterion. When comparing the two learning framework, the use of supervised learning algorithms leads to the best performances.

Precipitation measurement is an important prior for several operational and scientific applications, including weather forecasting, hazard prevention, agriculture, etc. Weather radars, such as NEXRAD, observe the air volume reflectivity and infer precipitation intensity at high resolution. However, their capabilities are limited over the ocean. C-band SAR imagery, which is sensitive to ocean surface roughness, is known to be sensitive to the effect of rain. In this study, we improve existing NEXRAD/Sentinel-1 collocations and train a U-Net deep learning model to estimate NEXRAD radar reflectivity from Sentinel-1 observations. Precipitation fore-casts are returned as segmentations with thresholds at 1, 3 and 10 mm/hr. The results indicate high performance over a wide range of wind speeds and thus can provide an accurate estimate of precipitation in the absence of weather radar.

At-sea behaviour of seabirds have received significant attention in ecology over the last decades as it is a key process in the ecology and fate of these populations. It is also, through the position of top predator that these species often occupy, a relevant and integrative indicator of the dynamics of the marine ecosystems they rely on. Seabird trajectories are recorded through the deployment of GPS, and a variety of statistical approaches have been tested to infer probable behaviours from these location data. Recently, deep learning tools have shown promising results for the segmentation and classification of animal behaviour from trajectory data. Yet, these approaches have not been widely used and investigation is still needed to identify optimal network architecture and to demonstrate their generalization properties. From a database of about 300 foraging trajectories derived from GPS data deployed simultaneously with pressure sensors for the identification of dives, this work has benchmarked deep neural network architectures trained in a supervised manner for the prediction of dives from trajectory data. It first confirms that deep learning allows better dive prediction than usual methods such as Hidden Markov Models. It also demonstrates the generalization properties of the trained networks for inferring dives distribution for seabirds from other colonies and ecosystems. In particular, convolutional networks trained on Peruvian boobies from a specific colony show great ability to predict dives of boobies from other colonies and from distinct ecosystems. We further investigate accross-species generalization using a transfer learning strategy known as 'fine-tuning'. Starting from a convolutional network pre-trained on Guanay cormorant data reduced by two the size of the dataset needed to accurately predict dives in a tropical booby from Brazil. We believe that the networks trained in this study will provide relevant starting point for future fine-tuning works for seabird trajectory segmentation.

Representing maritime traffic patterns and detecting anomalies from them are key to vessel monitoring and maritime situational awareness. We propose a novel approach—referred to asGeoTrackNet—for maritime anomaly detection from AIS data streams. Our model exploits state-of-the-art neural network schemes to learn a probabilistic representation of AIS tracks anda contrariodetection to detect abnormal events. The neural network provides a new means to capture complex and heterogeneous patterns in vessels’ behaviours, while thea contrariodetector takes into account the fact that the learnt distribution may be location-dependent. Experiments on a real AIS dataset comprising more than 4.2 million AIS messages demonstrate the relevance of the proposed method compared with state-of-the-art schemes.

Space oceanography missions, especially altimeter missions, have considerably improved the observation of sea surface dynamics over the last decades. They can however hardly resolve spatial scales below ~ 100km. Meanwhile the AIS (Automatic Identification System) monitoring of the maritime traffic implicitly conveys information on the underlying sea surface currents as the trajectory of ships is affected by the current. Here, we show that an unsupervised variational learning scheme provides new means to elucidate how AIS data streams can be converted into sea surface currents. The proposed scheme relies on a learnable variational framework and relate to variational auto-encoder approach coupled with neural ODE (Ordinary Differential Equation) solving the targeted ill-posed inverse problem. Through numerical experiments on a real AIS dataset, we demonstrate how the proposed scheme could significantly improve the reconstruction of sea surface currents from AIS data compared with state-of-the-art methods, including altimetry-based ones.

Variational models are among the state-of-the-art formulations for the resolution of ill-posed inverse problems. Following recent advances in learning-based variational settings, we investigate the end-to-end learning of variational models, more precisely of the regularization term given some observation model, jointly to the associated solver, so that we can optimize the reconstruction performance. In the proposed end-to-end setting, both the variational cost and the gradient-based solver are stated as neural networks using automatic differentiation for the latter. We consider an application to inverse problems with incomplete datasets (image inpainting and multivariate time series interpolation). We experimentally illustrate that this framework can lead to a significant gain in terms of reconstruction performance, including w.r.t. the direct minimization of the variational formulation derived from the known generative model.

The reconstruction of better-resolved sea surface currents is a key challenge in space oceanography. Besides the upcoming SWOT wide-swath altimeter mission, new algorithms are explore to produce improved gap-free gridded products. Based on the recent development of a generic end-to-end deep learning scheme for inverse problems backed on a variational formulation, we investigate how this framework applies to the space-time interpolation of satellite-derived SSH fields. We consider different parameterization of the proposed end-to-end learning scheme, especially regarding the embedded variational solver. Using an Observing System Simulation Experiment based on high-resolution numerical simulations in the Gulf Stream region, we show that the later may significantly outperform the state-of-the-art, including DUACS optimal interpolation product, when jointly considering nadir along-track altimeter data and upcoming SWOT wide-swath data.

Segmentations of ocean SAR images (Sentinel-1 A and B) into 10 classes of metoceanic phenomena are for the first time presented, with a 400 m resolution. Ocean SAR images segmentation differs from classic deep learning problems with a high variety of shapes and a particular importance of high-frequency patterns. To this end, an assessment of deep learning frameworks is performed, with a focus on the comparison between weakly supervised and supervised methods. Metrics based on the Wassertein distance indicate best performances by the supervised segmentation (U-Net) given operational constraints, thus highlighting the significance of properly annotated data sets. While available training data sets are made of small$20 \times 20 \text{km}$imagettes, the extension of the inference from imagettes to wide swath images, with a wider variety of incidence angles, presents promising results and opens the way to more extensive oceanographic applications in SAR imagery.

The reconstruction of sea surface geophysical variables typically relies either on Optimal Interpolation (OI) or on model-based approaches which explicitly exploit a dynamical model. While the optimal interpolation suffers from smoothing issues making it unreliable in retrieving fine scale variability, the selection and parametrization of a dynamical model, when considering model based data assimilation strategies, remains a complex issue since several trade-offs between the models complexity and its applicability in sea surface data assimilation need to be carefully addressed. In this work, and motivated by the success of artificial intelligence algorithms in various signal processing fields as well as the increasing amounts of observations and simulation datasets, we explore a data driven Koopman model and formulate the reconstruction problem as solution of the classical Kalman filter in a space of observables. The proposed architecture although linear, is shown to outperform state-of-the-art non linear data-driven filtering schemes such as the Analog Data Assimilation (AnDA).

Due to complex natural and anthropogenic forcings, the dynamics of suspended sediments within the ocean water column remains difficult to monitor. Nowadays however, more and more available information is coming from in situ and satellite measurements, as well as from simulation models. Data assimilation methods propose to combine all this information to produce the most precise results, allowing better analyzes of the processes in play. Here a comparison of data-driven methods is presented. Optimal Interpolation (OI), Empirical Orthogonal Function (EOF) based and Kalman Filter based methods are compared to a new one using neural networks. The latter is a Data Interpolation method based on convolutional AutoEncoders (DinAE). Present results show that DinAE better performs compared to other methods, having the lowest error budget and the highest learning of high frequency events.

Hyperspectral image unmixing is an inverse problem aiming at recovering the spectral signatures of pure materials of interest (called endmembers) and estimating their proportions (called abundances) in every pixel of the image. However, in spite of a tremendous applicative potential and the avent of new satellite sensors with high temporal resolution, multitemporal hyperspectral unmixing is still a relatively underexplored research avenue in the community, compared to standard image unmixing. In this paper, we propose a new framework for multitemporal unmixing and endmember extraction based on a state-space model, and present a proof of concept on simulated data to show how this representation can be used to inform multitemporal unmixing with external prior knowledge, or on the contrary to learn the dynamics of the quantities involved from data using neural network architectures adapted to the identification of dynamical systems.

Despite some promising results under ideal conditions (i.e. noise-free and complete observation), learning chaotic dynamical systems from real life data is still a very challenging task. We propose a novel framework, which combines data assimilation schemes and neural network representation, namely Auto-Encoders and Ensemble Kalman Smoother, to learn the governing equations of dynamical systems. By treating the learning as a Bayesian estimation problem, our framework can deal with noisy and partial observations. Experiments on the chaotic Lorenz-63 dynamics with different noise settings demonstrate the advantages of our method over the state-of-the-art.

The upcoming Surface Water Ocean Topography (SWOT) satellite altimetry mission is expected to yield two-dimensional high-resolution measurements of Sea Surface Height (SSH), thus allowing for a better characterization of the mesoscale and submesoscale eddy field. However, to fulfill the promises of this mission, filtering the tidal component of the SSH measurements is necessary. This challenging problem is crucial since the posterior studies done by physical oceanographers using SWOT data will depend heavily on the selected filtering schemes. In this paper, we cast this problem into a supervised learning framework and propose the use of convolutional neural networks (ConvNets) to estimate fields free of internal tide signals. Numerical experiments based on an advanced North Atlantic simulation of the ocean circulation (eNATL60) show that our ConvNet considerably reduces the imprint of the internal waves in SSH data even in regions unseen by the neural network. We also investigate the relevance of considering additional data from other sea surface variables such as sea surface temperature (SST).

The forecasting and assimilation of sea surface dynamics from satellite-derived data is a challenging issue. Data-driven approaches have arisen as promising schemes to fully exploit satellite observations. A key feature of sea surface dynamics is that they relate to partially-observed dynamics. Here, guided by physical and mathematical considerations of the underlying dynamics of a given system, we propose a novel neural networks architecture for the identification of ordinary differential equations (ODE) of partially observed systems. Numerical experiments for toy models and a sea level anomaly dynamics illustrate the relevance of the proposed scheme for forecasting and assimilation issues w.r.t. other state-of-the-art data-driven models.

The constant growth of maritime traffic leads to the need of automatic anomaly detection, which has been attracting great research attention. Information provided by AIS (Automatic Identification System) data, together with recent outstanding progresses of deep learning, make vessel monitoring using neural networks (NNs) a very promising approach. This paper analyse a novel neural network we have recently introduced -GeoTrackNet- regarding operational contexts. Especially, we aim to evaluate (i) the relevance of the abnormal behaviours detected by GeoTrackNet with respect to expert interpretations, (ii) the extent to which GeoTrackNet may process AIS data streams in real time. We report experiments showing the high potential to meet operational level of the model.

In this paper, we adapt Recurrent Neural Networks with Stochastic Layers, which are the state-of-the-art for generating text, music and speech, to the problem of acoustic novelty detection. By integrating uncertainty into the hidden states, this type of network is able to learn the distribution of complex sequences. Because the learned distribution can be calculated explicitly in terms of probability, we can evaluate how likely an observation is then detect low-probability events as novel. The model is robust, highly unsupervised, end-to-end and requires minimum preprocessing, feature engineering or hyperparameter tuning. An experiment on a benchmark dataset shows that our model outperforms the state-of-the-art acoustic novelty detectors.

In the last years, Neural Networks have enriched the state-of-the-art in probabilistic modeling. This is principally due to the advances in deep learning which allow a better understanding of complex systems. However, the stochastic representation of spatio-temporal fields is still an open challenge that may benefit from the recent advances in probabilistic modelization. In this work, we explore neural network to derive a stochastic representation of spatio-temporal dynamical systems based on ensemble forecasting. Trough the implementation of our stochastic model in a classical Kalman filtering scheme, we demonstrate the relevance of the proposed architecture in the reconstruction of geophysical fields with respect to the state-of-the-art approaches.

In this work, we investigate residual neural network representations for the identification and forecasting of dynamical systems. We propose a novel architecture that jointly learns the dynamical model and the associated Runge-Kutta integration scheme. We demonstrate the relevance of the proposed architecture with respect to learning-based state-of-the-art approaches in the identification and forecasting of chaotic dynamics when provided with training data with low temporal sampling rates.

This work deals with data-driven and learning-based approaches to fill space-time sampling gaps in the observation of sea surface tracers such as Sea Surface Height (SSH), Sea Surface temperature (SST), Ocean Colour,... More precisely, we jointly address field super-resolution and the prediction of subgrid-scale features, which is novel to our knowledge. From a methodological point of view, we consider deep learning architectures with a view to learning geophysically-sound differential operators (i.e. trasnport operators). Based on an Observing System Simulation Experiment representative of SWOT fast sampling phase using NATL60 simulation data, we illustrate the relevance of the proposed methodological framework which reconstructs more than 90% of the variance of the high-resolution SSH anomaly field and above 90% of the subgrid-scale variance of this anomaly. We also illustrate significant gain w.r.t. other baseline neural network architectures and further discuss the relevance of the reported contribution for other tracer fields and case studies.

In this work, we propose an alternative to the Ensemble Kalman filter through the implementation of a neural networks filtering scheme based on a parametric stochastic model. From our numerical experiment, we prove the relevance of the proposed architecture in the reconstruction of geophysical fields with respect to the state-of-the-art schemes.

Recent studies have investigated the identification of governing equations of geophysical systems from data. Here, we investigate such identification issues for ocean surface dy-namcis from ocean remote sensing data. From a methodological point of view, we address the learning of data-driven dynamical models when only provided with a noisy training dataset. We propose a novel architecture that relies on data assimilation schemes to learn the underlying dynamical model through the minimization of a reconstruction cost. We demonstrate the relevance of the proposed architecture with respect to the state-of-the-art approaches in the identification and forecasting of synthetic and real case-studies.

In a world of global trading, maritime safety, security and efficiency are crucial issues. We propose a multi-task deep learning framework for vessel monitoring using Automatic Identification System (AIS) data streams. We combine recurrent neural networks with latent variable modeling and an embedding of AIS messages to a new representation space to jointly address key issues to be dealt with when considering AIS data streams: massive amount of streaming data, noisy data and irregular time-sampling. We demonstrate the relevance of the proposed deep learning framework on real AIS datasets for a three-task setting, namely trajectory reconstruction, anomaly detection and vessel type identification.

This work presents EddyNet, a deep learning based architecture for automated eddy detection and classification from Sea Surface Height (SSH) maps provided by the Copernicus Marine and Environment Monitoring Service (CMEMS). EddyNet consists of a convolutional encoder-decoder followed by a pixel-wise classification layer. The output is a map with the same size of the input where pixels have the following labels {`0': Non eddy, `1': anticyclonic eddy, `2': cyclonic eddy}. Keras Python code, the training datasets and EddyNet weights files are open-source and freely available on https://github.com/redouanelg/EddyNet.

The ever increasing availability of in situ, remote sensing and simulation data supports the development of data-driven alternatives to classical model-driven methods for the interpolation of sea surface geophysical fields from partial satellite-derived observations. In this respect, we recently introduced the Analog Data Assimilation (AnDA), which exploits patch-based analog forecasting operators within a classic Kalman-based data assimilation framework. In this work, we consider the application of AnDA to the spatio-temporal interpolation of SLA (Sea Level Anomalies) from two types of satellite altimetry data, namely from along-track nadir data [1] and data from the upcoming wide-swath SWOT mission [2]. We report a sensitivity analysis w.r.t. the main parameters of the proposed AnDA scheme. Overall, the reported benchmarking analysis supports the relevance of the proposed AnDA scheme for an improved reconstruction of mescoscale structures for horizontal scales ranging from ~ 20km to ~ 100km, with an gain of 42% (12%) in terms of SLA RMSE (correlation) with respect to Optimal Interpolation (OI) [3]. Results suggest an additional potential improvement from the joint assimilation of SWOT and along-track nadir observations.

The forecasting and reconstruction of ocean and atmosphere dynamics from satellite observation time series are key challenges. While model-driven representations remain the classic approaches, data-driven representations become more and more appealing to benefit from available large-scale observation and simulation datasets. In this work we investigate the relevance of recently introduced bilinear residual neural network representations, which mimic numerical integration schemes such as Runge-Kutta, for the forecasting and assimilation of geophysical fields from satellite-derived remote sensing data. As a case-study, we consider satellite-derived Sea Surface Temperature time series off South Africa, which involves intense and complex upper ocean dynamics. Our numerical experiments demonstrate that the proposed patch-level neural-network-based representations outperform other data-driven models, including analog schemes, both in terms of forecasting and missing data interpolation.

Multisatellite measurements of altimeter-derived sea surface height (SSH) have provided a wealth of information on the ocean. Yet, horizontal scales below 100 km remain scarcely resolved. Especially, in the Mediterranean Sea, an important fraction of the mesoscale range, characterized by a small Rossby radius of deformation of 15-20 km, is not properly retrieved by altimeter-derived gridded products. Here, we investigate a novel processing of AVISO products with a view to resolving the horizontal scales sensed by current along-track altimeter data. The key feature of our framework is the use of linear convolutional operators to model the fine-scale SSH detail as a function of different sea surface fields, especially optimally interpolated SSH and sea surface temperature (SST). The proposed model embeds the surface quasi-geostrophic SST-SSH synergy as a special case. Using an observing system simulation experiment with simulated SSH data from model outputs in the Western Mediterranean Sea, we show that the proposed approach has the potential for improving current optimal interpolations of gridded altimeter-derived SSH fields by more than 20% in terms of relative SSH and kinetic energy mean square error, as well as in terms of spectral signatures for horizontal scales ranging from 30 to 100 km. Our results also suggest that SST-SSH relationship may only play a secondary role compared with the interscale SSH cascade. We further discuss the relevance of the proposed approach in the context of future altimetric satellite missions.

We address in this paper the reconstruction of irregurlarly-sampled image time series with an emphasis on geophysical remote sensing data. We develop a data-driven approach, referred to as an analog assimilation and stated as an ensemble Kalman method. Contrary to model-driven assimilation models, we do not exploit a physically-derived dynamic prior but we build a data-driven dynamic prior from a representative dataset of the considered image dynamics. Our contribution is here to extend analog assimilation to images, which involve high-dimensional state space. We combine patch-based representations to a multiscale PCA-constrained decomposition. Numerical experiments for the interpolation of missing data in satellite-derived ocean remote sensing images demonstrate the relevance of the proposed scheme. It outperforms the classical optimal interpolation with a relative RMSE gain of about 50% for the considered case study.

Super-resolution is a classical problem in image processing, with numerous applications to remote sensing image enhancement. Here, we address the super-resolution of irregularly-sampled remote sensing images. Using an optimal interpolation as the low-resolution reconstruction, we explore locally-adapted multimodal convolutional models and investigate different dictionary-based decompositions, namely based on principal component analysis (PCA), sparse priors and non-negativity constraints. We consider an application to the reconstruction of sea surface height (SSH) fields from two information sources, along-track altimeter data and sea surface temperature (SST) data. The reported experiments demonstrate the relevance of the proposed model, especially locally-adapted parametrizations with non-negativity constraints, to outperform optimally-interpolated reconstructions.

This paper addresses the reconstruction of high-resolution Sea Surface Height (SSH) from the synergy between along-track altimeter data, OI-interpolated SSH fields and satellite-derived high-resolution Sea Surface Temperature (SST) fields. We aim at better resolving the fine-scale range, typically below 100km, which remains scarcely resolved by operational optimal interpolation schemes. The proposed scheme relies on multi-tracer convolutional models and on their calibration from the observed along-track data. We explore a dictionary-based decomposition of the convolutional models to improve the robustness of the calibration. We report a numerical evaluation using an Observation Simulation System Experiment (OSSE) for a case study region in the western Mediterranean sea. Our numerical experiments demonstrate that we can improve reconstruction performance by about 20%, in terms of mean square error, compared to optimally-interpolated fields. Dictionary-based decompositions also resort to similar potential improvement. We further analyze different parameterizations of the convolution models in relation to physical priors (e.g., SGQ dynamics, isotropical transfer functions).

The reconstruction of high-resolution gridded altimetry maps from irregularly sampled along-track data remains a key challenge in ocean remote sensing science. Operational products use optimal Interpolation (OI) techniques, which may not deal with nonlinear dynamics at short space-time scales. Here, we investigate an analog data assimilation scheme to improve the reconstruction of fine-scale structures. The analog data assimilation combines an ensemble Kalman model and a dataset of exemplars issued from high-resolution numerical simulations to perform an exemplar-based spatio-temporal interpolation of along-track data. As a case-study, we consider a region in the South China Sea and demonstrate the proposition analog data assimilation outperforms the classical OI by about ≃ 20% in terms of mean square reconstruction error.

The identification and separation of contributions associated with different sources or processes is a general problem in signal and image processing. Here, we focus on the decomposition of multiple linear relationships and introduce a non-negative formulation. The proposed models can be viewed as generalizations of latent class regression models and account for possibly varying magnitudes of the linear transfer functions. Along with these models, we present model calibration algorithms. We first demonstrate their performance on simulated data. We also report an application to the analysis of upper ocean dynamics from remote sensing data (namely, satellite-derived Sea Surface Height (SSH) and Sea Surface temperature (SST) image series). This application further stresses the proposed formulation's relevance compared to state-of-the-art regression models.

Deep Learning (DL), especially Convolutional Neural Networks (CNN), has become the state-of-the-art for a variety of pattern recognition issues. Technological developments have allowed the use of high-end General Purpose Graphic Processor Units (GPGPU) for accelerating numerical problem solving. They resort no only to lower computational time, but also allow considering much larger networks. Hence, nowadays computers are able to drive deeper, wider and more powerful models. State of the art CNNs have achieved human-like performance in several recognition tasks such as: handwritten character recognition, face recognition, scene labelling, object detection and image classification among others. Meanwhile, mobile devices have become powerful enough to handle the computations required for deploying CNNs models in near real-time. Here, we investigate the implementation of light-weight CNN schemes on mobile devices for domain-specific objection recognition tasks.

We here address the super-resolution of a high-resolution image involving missing data given that a low-resolution image of the same scene is available. This is a typical issue in the remote sensing of geophysical parameters from different spaceborne sensors. Such super-resolution application involves large downscaling factor (typically from 10 to 20) and the super-resolution model should account for both texture patterns and specific statistical features, especially the spectral and non-Gaussian features. In this context, we propose a novel non-local approach and formally states the solution as the joint minimization of several projection constraints. We illustrate the relevance of the proposed model on real ocean remote sensing data, namely sea surface temperature fields, as well on visual textures.

In this work, we propose a novel non parametric method for the temporally-consistent inpainting of dynamic texture sequences. The inpainting of texture image sequences is stated as a stochastic assimilation issue, for which a novel model-free and data-driven Ensemble Kalman method is introduced. Our model is inspired by the Analog Ensemble Kalman Filter (AnEnKF) recently proposed for the assimilation of geophysical space-time dynamics, where the physical model is replaced by the use of statistical analogs or nearest neighbours. Such a non-parametric framework is of key interest for image processing applications, as prior models are seldom available in general. We present experimental evidence for real dynamic texture that using only a catalog database of historical data and without having any assumption on the model, the proposed method provides relevant dynamically-consistent interpolation and outperforms the classical parametric (autoregressive) dynamical prior.

From the multispectral top-of-atmosphere observations, ocean colour inversion aims at separating atmosphere and water contribution. In this context, we propose a novel Bayesian model with a focus on the definition of non-homogeneous priors on the aerosol and water multispectral signatures. The considered priors are set conditionally to observed covariates, typically geometry acquisition conditions and pre-estimates by a standard algorithm. We demonstrate from numerical experiments performed for real data the relevance of our non-homogeneous Bayesian setting to retrieve geophysically-consistent ocean colour images, in particular when dealing with complex coastal waters where standard algorithms perform poorly. Using a groundtruthed dataset, quantitative comparisons with operational schemes stress the overall improvement on the relative absolute error (respectively, 67% compared with the standard ESA MEGS algorithm and 9% compared with the ESA C2R neural network, for 12 bands ranging from 412 to 865 nm).

This paper addresses the reconstruction of sea surface currents from satellite ocean sensing data. Whereas the classical surface currents derived from the SSH (Sea Surface Height) products are rather low space-time resolution fields (typically, 50 km and 12-day actual space-time grid resolution), we investigate the extent to which we can retrieve sea surface currents at higher resolution using daily SST (Sea Surface Temperature) satellite observations. State-of-the-art methods, which exploit classical optical flow schemes or nonlinear regression techniques, do not provide satisfactory results due to the space-time variabilities of the relationships between the SST and the sea surface current. Motivated by our recent joint SST-SSH identification of characterization of upper ocean dynamical modes, we here show that a multiregime model, formally stated as a Markov-switching latent class regression model, provides a relevant model to capture the above-mentioned variabilities and reconstruct SST-driven sea surface currents. The considered case study within the Agulhas current demonstrates that our model retrieves highresolution space-time details which cannot be resolved by the classical SSH-derived products.

This paper addresses the simulation of high-resolution geophysical fields from low-resolution satellite observations in the context of the remote sensing of the ocean surface. Within a super-resolution framework, we investigate texture-based stochastic models while controlling high-resolution spectral, geometrical and statistical features. We introduce a novel model stated as the solution of a SPDE (Stochastic Partial Differential Equation) associated with conditional 2D Gaussian field. We address both model parameter inference from real high-resolution images and simulation issues. Experiments for Sea Surface Temperature fields demonstrate the relevance of our model compared to classical stationary schemes.

This paper addresses the reconstruction of high-resolution (HR) sea surface wind fields (typically, at a spatial resolution of 1 km). The availability of such HR fields is critical for numerous issues, e.g. coastal management, offshore structures, oil spill disaster tracking, etc. Satellites, especially from Synthetic Aperture Radar (SAR) systems, can monitor the ocean surface at a spatial resolution of a few meters. SAR wind fields are operationally produced with spatial resolutions of less than 1 km [1, 2]. However, satellite SAR systems involve a highly irregular sampling of the ocean surface and, for a given region, SAR wind fields may be delivered with a low temporal resolution, typically every 7-to-10 days for temperate zones. By contrast, model predictions, such as European Center for Medium-range Weather Forecast (ECMWF) wind fields, are typically delivered with a high temporal resolution (e.g. every 3 h or 6 h), but with a low spatial resolution (~50km × 50km). The question of the combination of numerical model predictions and SAR wind fields naturally arises to deliver HR wind fields at sea surface anywhere and anytime. Here, we state this issue as the statistical learning of transfer functions between low-resolution (LR) model predictions and the associated HR SAR fields. We investigate the extent to which such regression functions can be learnt from a set of co-located HR and LR fields. Both local and non-local schemes as well as linear and non-linear regression methods are considered. As a case-study, we carry out numerical experiments for a coastal area off Norway, which involves complex LR-to-HR situations.

The availability of daily satellite Sea Surface Temperature (SST) data and theoretical results (see e.g., [1]) advocate for new methods to retrieve the Sea Surface Height (SSH) and the surface geostrophic currents from SST observations. The underlying hypothesis comes to assume that the local variations of the SST relate to the surface currents. Ocean turbulence models, such as the Surface Quasi Geostrophic (SQG) theory (cf. [2], [3]) or statistical methods like neural networks (cf. [4]) or latent class regressions (cf. [5]) provide different means to state the SST-SSH relationships. This later approach has the advantage to be completely parametric and to account for different transfer functions between SST and SSH. It relies on a conditional setting with respect to a hidden variable related to different dynamical modes at the surface of the ocean. In this paper, we aim at further developing such latent models with an emphasis on two aspects: (i) the modeling and learning of the spatio-temporal dynamics of the hidden dynamical modes using Markovian priors, (ii) the reconstruction of daily SSH fields from a joint analysis of microwave SST and altimetry observation series. We evaluate the proposed model both qualitatively and quantitatively with respect to the reference altimetry product.

The development of animal tracking technologies (including GPS and ARGOS satellite systems) and the increasing resolution of remote-sensing observations call for tools extracting and describing the geometric patterns along a track or within an image over a wide range of spatial scales. Whereas shape analysis has largely been addressed over the last decades, the multiscale analysis of the geometry of opened planar curves has received little attention. We here show that classical multiscale techniques cannot properly address this issue and propose an original wavelet-based scheme. To highlight the generic nature of our multiscale wavelet technique, we report applications to two different observation data sets, namely, wild animal movement paths recorded by electronic tags and satellite observations of sea-surface geophysical fields.

Multisatellite measurements of altimeter-derived sea surface height (SSH) and sea surface temperature (SST) provide a wealth of information about ocean circulation, particularly mesoscale ocean dynamics which may involve strong spatiotemporal relationships between SSH and SST fields. Within an observation-driven framework, we investigate the extent to which mesoscale ocean dynamics may be decomposed into a mixture of dynamical modes, characterized by different local regressions between SSH and SST fields. Formally, we develop a novel latent class regression model to identify dynamical modes from joint SSH and SST observation series. Applied to the highly dynamical Agulhas region, we demonstrate and discuss the geophysical relevance of the proposed mixture model to achieve a spatiotemporal segmentation of the upper ocean dynamics.

This paper addresses stochastic geometry-driven image models and its application to super-resolution issues. Whereas most stochastic image models rely on some priors on the distribution of grey-level configurations (e.g., patch-based models, Markov priors, multiplicative cascades,...), we here focus on geometric priors. We aim at simulating texture samples while controlling high-resolution geometrical features. In this respect, we introduce a stochastic model for texture orientation fields stated as a 2D Orstein-Uhlenbeck process. We show that this process resorts in the stationary case to priors on orientation statistics. We exploit this model to state image super-resolution as a geometry-driven variational minimization, where the geometry is sampled from the proposed conditional 2D Orstein-Uhlenbeck process. We demonstrate the relevance of this approach for real images associated with the remote sensing of ocean surface dynamics.

The use of satellite Sea Surface Temperature (SST) fields to retrieve zonal and meridional surface currents (U, V) is now a widespread idea. Since the classical approach involves temporal differencing of SST fields, we investigate in this paper the extent to which mesoscale ocean dynamics may be decomposed into a superposition of dynamical modes, characterized by different linear relationships between surface currents and temperature fields. Based on a completely observation-driven approach, we propose a latent class regression model from local satellite surface currents and patches of SST measurements. Applied to the highly dynamical Agulhas region, we demonstrate and discuss the geophysical relevance of the proposed mixture model to achieve a spatio-temporal segmentation and tracking of the ocean surface dynamical modes. Moreover, we show the accuracy of the proposed model to predict mesoscale surface currents from SST single maps.

In many biological or medical applications, images that contain sequences of shapes are common. The existence of high inter-individual variability makes their interpretation complex. In this paper, we address the computer-assisted interpretation of such images and we investigate how we can remove or reduce these image variabilities. The proposed approach relies on the development of an efficient image registration technique. We first show the inadequacy of state-of-the-art intensity-based and feature-based registration techniques for the considered image datasets. Then, we propose a robust variational method which benefits from the geometrical information present in this type of images. In the proposed non-rigid geodesics-based registration, the successive shapes are represented by a level-set representation, which we rely on to carry out the registration. The successive level sets are regarded as elements in a shape space and the corresponding matching is that of the optimal geodesic path. The proposed registration scheme is tested on synthetic and real images. The comparison against results of state-of-the-art methods proves the relevance of the proposed method for this type of images.

To evaluate the proposed model we used SST and SSS observations. High resolution SST observations are taken from the METOP dataset and corresponding low resolution observations from the REMSS database. High and low resolution SSS observations are taken from the ESA/SMOS (European Space Agency/Soil Moisture and Ocean Salinity) database.

In this letter, we evaluate to which extent the activity of ocean fronts can be retrieved from the geometric regularity of ocean tracer observations. Applied to sea surface temperature (SST), we propose a method for the characterization of this geometric regularity from curvature-based statistics along temperature level lines in front regions. To assess the effectiveness of the proposed descriptors, we used six years (from 2003 to 2008) of daily SST observations of the regions of Agulhas in the South of Africa and of Malvinas off the southern Brazilian coast. These experiments stress the relevance of geometric regularity features of tracer observation at ocean surface to characterize seasonal variations in ocean regimes.

In this letter, we address the characterization of objects in 3-D sonar images of the water column obtained by a multibeam echo sounder. Compared with classic 2-D images from a monobeam echo sounder, these 3-D images provide finer scale observation of the pelagic biomasses and new tools to characterize 3-D distributions. By viewing object patterns as realizations of spatial point processes, we investigate descriptive spatial statistics. This method is then applied to 3-D fisheries acoustics data set for characterization of the distribution of pelagic fish schools. Reported experiments illustrate the relevance of the proposed descriptors. The comparison of our method with 2-D sonar data analysis further demonstrates the information gain from using 3-D sonar imagery.

In this paper, we address seabed characterization and recognition in sonar images using keypoint-based approaches. Keypoint-based texture recognition has recently emerged as a powerful framework to address invariances to contrast change and geometric distortions. We investigate here to which extent keypoint-based techniques are relevant for sonar texture analysis which also involves such invariance issues. We deal with both the characterization of the visual signatures of the keypoints and the spatial patterns they form. In this respect, spatial statistics are considered. We report a quantitative evaluation for sonar seabed texture data sets comprising six texture classes such as mud, rock, and gravely sand. We clearly demonstrate the improvement brought by keypoint-based techniques compared to classical features used for sonar texture analysis such as cooccurrence and Gabor features. In this respect, we demonstrate that the joint characterization of the visual signatures of the visual keypoints and their spatial organization reaches the best recognition performances (about 97% of correct classification w.r.t. 70% and 81% using cooccurrence and Gabor features). Furthermore, the combination of difference of Gaussian keypoints and scale-invariant feature transform descriptors is recommended as the most discriminating keypoint-based framework for the analysis of sonar seabed textures.

In this paper, we address invariant keypoint-based texture characterization and recognition. Viewing keypoint sets associated with visual textures as realizations of point processes, we investigate probabilistic texture models from multivariate log-Gaussian Cox processes. These models are parameterized by the covariance structure of the spatial patterns. Their implementation initially rely on the construction of a codebook of the visual signatures of keypoints. We discuss invariance properties of the proposed models for texture recognition applications and report a quantitative evaluation for three texture datasets, namely: UIUC, KTH-TIPs and Brodatz. These experiments include a comparison of the performance reached using different methods for keypoint detection and characterization and demonstrate the relevance of the proposed models w.r.t. state-of-the-art methods. We further discuss the main contribution of proposed approach, including the key features of a statistical model and complexity aspects.

In this paper, invariant sonar texture characterization for seabed classification is addressed from the spatial distribution of image keypoints using log-Gaussian Cox processes. Considering the categorized visual keypoints, the spatial statistical properties of keypoint sets are expressed by the intensity and the pair correlation function of the log-Gaussian Cox model to define a novel invariant texture descriptor. Reported results of an application to sonar texture classification validate the proposed descriptor compared to previous work. We further discuss the main contribution of proposed approach, including the key features of a statistical model and complexity aspects.

In this paper we address the joint interpolation of missing data and estimation of ocean surface velocities from multi-resolution sea surface satellite observations. A variational assimilation model is proposed. Using synthetic simulation and real SST data, we conducted experiments to evaluate the relevance of the proposed model, in particular the relevance of the fusion of observations at different resolutions and the improvement issued from the consideration of dynamics prior in the assimilation model. Numerical and qualitative results assessed the effectiveness of the proposed methods.

In this paper, we address invariant scene classification from images. We propose a novel descriptor based on the statistical characterization of the spatial patterns formed by elementary objects in images. Elementary objects are defined from a tree of shapes of the topology map of the image and each object is characterized by shape context feature vector. Viewing the set of elementary objects as a realization of a random spatial process, we investigate a statistical analysis using log- Gaussian Cox model to define an invariant image descriptor. An application to natural scene recognition is described. Re- ported results validate the proposed descriptor with respect to previous work.

This paper addresses the training of classification trees for weakly labelled data. We call “weakly labelled data”, a training set such as the prior labelling information provided refers to vector that indicates the probabilities for instances to belong to each class. Classification tree typically deals with hard labelled data, in this paper a new procedure is suggested in order to train a tree from weakly labelled data. Resulting tree is different than usual in the sense that weak labels are taking into account and affected to test instances. Considering a forest, we show how trees can be associated in the test step. The proposed method is compared with typical models such as generative and discriminative methods for object recognition and we show that our model can outperform the two previous. The considered models are evaluated on standard datasets from UCI and an application to fisheries acoustics is considered.

This paper addresses the development of invariant descriptors for sonar texture based on spatial statistics of local features. We suggest using a hierarchical clustering algorithm to construct a set of codebook from the vector descriptor of keypoints. Spatial point process model allows us to estimate a co-occurrence statistics of the marks of neighboring keypoints in various study region. The resulting descriptor is applied to texture classification using a discriminative method (K-NN or SVM). Experiments were carried out on a set of real sidescan sonar images aiming to compare our proposed descriptor with other texture descriptors.

Nowadays, high resolution sea surface temperature (SST) observations recorded from orbital satellites are available. Because SST fronts appearing at the ocean surface convey information about the dynamics of deeper ocean layers, their study is of high interest in oceanography. In this paper we present a variational method for fronts tracking in SST images. The proposed method integrates into the variational data assimilation framework a variational method for fronts detection using the level set formulation. This allows our method to extract temporally consistent fronts in SST images sequences. The proposed method is validated on two sequences of SST images of two regions, the region of Malvinas and the region of Aghulas-Benguela, which host very active oceanic fronts.

This paper presents static and dynamic variational data assimilation methods for missing data interpolation in sea surface temperatures (SST) images. Evaluation using 50 AVHRR METOP SST images assesses the effectiveness of the proposed methods.

Monitoring sea surface temperature (SST) is of high interest. The dynamics of climatic events such as tropical storms or cyclones are closely related to the sea surface temperature. An important research topic in oceanography is about the understanding of the interactions between the ocean's surface and it's deeper layers. In Lapeyre et al. show that in the case of baroclinic unstable flows, the potential vorticity of mesoscale and submesoscale structures in the ocean interior are strongly correlated to the surface density structures. Thus, in frontal regions, knowledge of the ocean surface structures regularity gives insights about the 3D dynamics of the ocean. In this paper we focus on the analysis of the SST images level lines regularity from satellites images in regions in the neighborhood of oceanic fronts. Two regions of interest are considered: the region of Benguela and the region of Malvinas. Our investigations suggest that SST fronts belong to the class of statistical self similar curves. Taking into account self similar curves properties, we propose local curvature based descriptors for SST front region regularity characterization. We experimentally assess the efficiency of our descriptors by measuring their ability to capture SST image regularity seasonal variations.

This paper investigates variational region-level criterion for supervised and unsupervised texture-based image segmentation. The focus is given to the demonstration of the effectiveness and robustness of this region-based formulation compared to most common variational approaches. The main contributions of this global criterion are twofold. First, the proposed methods circumvent a major problem related to classical texture based segmentation approaches. Existing methods, even if they use different and various texture features, are mainly stated as the optimization of a criterion evaluating punctual pixel likelihoods or similarity measure computed within a local neighborhood. These approaches require sufficient dissimilarity between the considered texture features. An additional limitation is the choice of the neighborhood size and shape. These two parameters and especially the neighborhood size significantly influence the classification performances: the neighborhood must be large enough to capture texture structures and small enough to guarantee segmentation accuracy. These parameters are often set experimentally. These limitations are mitigated with the proposed variational methods stated at the region-level. It resorts to an energy criterion defined on image where regions are characterized by nonparametric distributions of their responses to a set of filters. In the supervised case, the segmentation algorithm consists in the minimization of a similarity measure between region-level statistics and texture prototypes and a boundary based functional that imposes smoothness and regularity on region boundaries. In the unsupervised case, the data-driven term involves the maximization of the dissimilarity between regions. The proposed similarity measure is generic and permits optimally fusing various types of texture features. It is defined as a weighted sum of Kullback-Leibler divergences between feature distributions. The optimization of the proposed variational criteria is carried out using a level-set formulation. The effectiveness and the robustness of this formulation at region-level, compared to classical active contour methods, are evaluated for various Brodatz and natural images.

This paper addresses weakly supervised object recognition. We show how the combination of an image-level inference, in terms of image-level object class priors, can lead to better training of object recognition models. Stated within a probabilistic setting, the proposed approach is applied to fisheries acoustics and fish school recognition.

In this paper we define a distance between shapes based on geodesics in shape space. The proposed distance, robust to outliers, uses shape matching to compare shapes locally. Multiscale analysis is introduced in order to avoid problems of local and global variabilities. The resulting similarity measure is invariant to translation, rotation and scaling independently of constraints or landmarks, but constraints can be added to the approach formulation when needed. An evaluation of the proposed approach is reported for shape classification and retrieval on a complex benchmark shape database. It demonstrates in both cases that previous work is outperformed.

In this paper, we propose and compare two supervised algorithms for the segmentation of textured sonar images, with respect to seafloor types. We characterize seafloors by a set of empirical distributions estimated on texture responses to a set of different filters. Moreover, we introduce a novel similarity measure between sonar textures in this feature space. Our similarity measure is defined as a weighted sum of Kullback-Leibler divergences between texture features. The weight setting is twofold. First, each filter is weighted according to its discrimination power: The computation of these weights are issued from a margin maximization criterion. Second, an additional weight, evaluated as an angular distance between the incidence angles of the compared texture samples, is considered to take into account sonar-image acquisition process that leads to a variability of the backscattered value and of the texture aspect with the incidence-angle range. A Bayesian framework is used in the first algorithm where the conditional likelihoods are expressed using the proposed similarity measure between local pixel statistics and the seafloor prototype statistics. The second method is based on a variational framework as the minimization of a region-based functional that involves the similarity between global-region texture-based statistics and the predefined prototypes.

This paper addresses the inference of probabilistic classification models using weakly supervised learning. In contrast to previous work, the use of proportion-based training data is investigated in combination to non-linear classification models. An application to fisheries acoustics and fish school classification is considered and experiments are reported for synthetic and real datasets.

This paper deals with seabed imaging issued from sonar systems. Such imaging systems produce images of backscattering (BS) strength relative to physical seabed characteristics. However, these Bs measurements are not only seabed-related but also dependent on the incident angle. Therefore, to enhance the quality of such seabed imaging systems, we develop an unsupervised approach to compensate for these seabed-related angular dependencies. Our approach combines robust estimation and hidden Markov random fields. Results on real data demonstrate the relevance of our approach to improve seabed observation

We propose a novel multi-class method for texture segmentation. The segmentation issue is stated as the minimization of a region-based functional that involves a weighted Kullback-Leibler measure between distributions of local texture features and a regularization term that imposes smoothness and regularity of region boundaries. The proposed approach is implemented using level-set methods, and partial differential equations (PDE) are expressed using shape derivative tools introduced in S. Jehan-Besson et al. (2003). As an application, we have tested the method using cooccurrence distributions to segment synthetic mosaics of textures from the Brodatz album, as well as real textured sonar images. These results prove the relevance of the proposed approach for supervised and unsupervised texture segmentation

Psychovision have shown that many grouping laws come into play to structure human vision. They use informations of different kinds, not only gray (or color)-level values. Here we will show how an orientation interpolation operator working in S1(angle in [0, 2π]) can be used to recover geometrical information in images. The operator is presented and is used to produce fields that drive a fast marching contour extraction algorithm and a LIC-based smoothing method. Experiment on real images are reported to validate the proposed approach.

This paper copes with the reconstruction of accretionary morphogenesis within a given observation plane from an image depicting successive (typically seasonal or daily) growth structures. Modeling accretionary growth shapes as the level-sets of a potential function, a variational framework is derived from geometric criteria. It resorts to minimizing an energy functional involving two terms: a regularization term and a data-driven term which constrain the evolution of the shapes with respect to a growth orientation field. Experiments carried out on real data (e.g., fish otoliths) validate the proposed approach, which opens new research directions for information extraction and decoding from biological archives.

This paper presents a general framework for the automated estimation of age and growth from images of biological materials depicting concentric ring-like structures such as tree trunks, corals, bivalve seashells, fish scales or otoliths. This interpretation task can be seen as a ring segmentation issue, where growth rings are associated to image ridge and valley structures. This is stated as the Bayesian selection of a subset of partial ring curves extracted using a semi-local template-based growth-adapted scheme. The application to fish otolith interpretation provides a consistent and convincing validation of the proposed framework.

This paper presents a new technique for noise removal in images. It benefits both from the recent advances in wavelet-based and variational denoising. Whereas wavelet-based analysis tends to strongly depend on the selected wavelet basis, we propose to combine and fuse several mono-wavelet analysis within a variational framework. The associated energy function involves M-estimator in order to guarantee the robustness to outliers and to preserve image structures (edges, ridges, etc.). An experimental evaluation for a Gaussian additive noise validates the proposed approach and an application to speckle removal in sonar sea-bed images highlights the interest of this approach for real images.

We present a robust method for time-frequency model estimation. It involves a robust Leclerc's estimator to ensure robustness w.r.t. noise and interferences present in time- frequency representations. This scheme is applied to fish age and growth analysis from otolith images. This application involves the estimation of the parameters of a priori fish growth models using this robust time-frequency analysis. We present a quantitative experimental validation over a large set of real images of Plaice otoliths.

A new registration method for ultrasound volumes relying on on a statistical texture-based similarity measure is investigated. Texture information is given by spatial Gabor filters and represented by statistical kernel-based distributions. The registration similarity measure is then defined as a probabilistic distance, derived from Bhattacharyya coefficient, between two statistical distributions. Given this similarity measure, parametric ultrasound image registration is stated as a robust minimization issue. We also exploit frequency properties of spatial Gabor filters to propose a multiresolution approach to perform this minimization. We provide a preliminary evaluation of the new registration technique on clinical data.

A new approach for motion characterization in image sequences is presented. It relies on the probabilistic modeling of temporal and scale co-occurrence distributions of local motion-related measurements directly computed over image sequences. Temporal multiscale Gibbs models allow us to handle both spatial and temporal aspects of image motion content within a unified statistical framework. Since this modeling mainly involves the scalar product between co-occurrence values and Gibbs potentials, we can formulate and address several fundamental issues: model estimation according to the ML criterion (hence, model training and learning) and motion classification. We have conducted motion recognition experiments over a large set of real image sequences comprising various motion types such as temporal texture samples, human motion examples, and rigid motion situations.

This paper describes an original approach for content-based video indexing and retrieval. We aim at providing a global interpretation of the dynamic content of video shots without any prior motion segmentation and without any use of dense optic flow fields. To this end, we exploit the spatio-temporal distribution, within a shot, of appropriate local motion-related measurements derived from the spatio-temporal derivatives of the intensity function. These distributions are then represented by causal Gibbs models. To be independent of camera movement, the motion-related measurements are computed in the image sequence generated by compensating the estimated dominant image motion in the original sequence. The statistical modeling framework considered makes the exact computation of the conditional likelihood of a video shot belonging to a given motion or more generally to an activity class feasible. This property allows us to develop a general statistical framework for video indexing and retrieval with query-by-example. We build a hierarchical structure of the processed video database according to motion content similarity. This results in a binary tree where each node is associated to an estimated causal Gibbs model. We consider a similarity measure inspired from Kullback-Leibler divergence. Then, retrieval with query-by-example is performed through this binary tree using the maximum a posteriori (MAP) criterion. We have obtained promising results on a set of various real image sequences.

We present an original approach for non parametric motion analysis in image sequences. It relies on the statistical modeling of distributions of local motion-related measurements computed over image sequences. Contrary to previously proposed methods, the use of temporal multiscale Gibbs models allows us to handle in a unified statistical framework both spatial and temporal aspects of motion content. The important feature of our probabilistic scheme is to make the exact computation of conditional likelihood functions feasible and simple. It enables us to straightforwardly achieve model estimation according to the ML criterion and to benefit from a statistical point of view for classification issues. We have conducted motion recognition experiments over a large set of real image sequences comprising various motion types such as temporal texture samples, human motion examples and rigid motion situations.

This paper deals with the detection of moving objects in videos, directly from MPEG coded data, with a view to content-based video indexing. The detection is stated as a Markovian labeling issue in terms of macroblocks conforming or not to the estimated dominant image motion assumed to be due to the camera motion. The dominant motion estimation and the moving object detection stages only utilize MPEG motion vectors and DC coefficients of the discrete cosine transform (DCT) directly extracted from the MPEG bit stream of the processed video. Therefore, our method implies a very low computational cost. Experimental results have demonstrated the interest of the proposed approach.

This paper describes an original approach for non parametric motion analysis in image sequences. It relies on a statistical modeling of distributions of local motion-related measurements, computed over image sequences, resulting from spatio-temporal random walks. It handles in a single probabilistic framework both spatial and temporal properties of motion content. The important feature of our method is to make feasible the exact computation of conditional likelihood functions. We have carried out motion recognition experiments over a large set of real image sequences comprising various motion types.

In this paper we propose an original approach for content-based video indexing and retrieval. It relies on the tracking of entities of interest and the analysis of their apparent motion. To characterize the dynamic information attached to these objects, we consider a probabilistic modeling of the spatio-temporal distribution of the optic flow field computed within the tracked area after canceling the estimated dominant motion due to camera movement. This leads to a general statistical framework for motion-based video classification and retrieval. We have obtained promising results on a set of various real image sequences.

We aim at detecting moving objects in color image sequences acquired with a mobile camera. This issue is of key importance in many application fields. To accurately recover motion boundaries, we exploit a fine spatial image partition supplied by a MRF-based color segmentation algorithm. We introduce a region-level graph modeling embedded in a Markovian framework to detect moving objects in the scene viewed by a mobile camera. This is stated as the binary segmentation into regions conforming or not conforming to the dominant image motion assumed to be due to the camera movement. The method is validated on real image sequences.

This paper describes an original approach for motion interpretation with a view to content-based video indexing. We exploit a statistical analysis of the temporal distribution of appropriate local motion-based measures to perform a global motion characterization. We consider motion features extracted from temporal cooccurrence matrices, and related to properties of homogeneity, acceleration or complexity. Results on various real video sequences are reported and provide a first validation of the approach.