This study proposes and benchmarks a lightweight pipeline for large-scale parcel-level crop classification using time-series data. The key contributions include a scalable SITS download strategy, a novel parcel-level crop classification dataset for the USA States of Texas and California, and a benchmark of classification techniques tested under varied data availability scenarios, highlighting the pipeline’s potential for enhancing agricultural monitoring systems. The methodology extracts basic descriptive statistics integrating with agricultural and cloud filtering, leveraging field boundaries delineation and USDA Cropland Data Layer datasets. Experimental results show that pretrained and fine-tuned models, such as SITS-BERT, outperform Random Forest and Support Vector Machine approaches, achieving an F1 score of 98.2% and overall accuracy of 99.3% on the Texas dataset for the more abundant training data scenarios. The pipeline has a projected computational speed-up of at leastat least 2, 500× compared to pixel-based methods for the tested datasets. The download pipeline is available on AgriGEE.lite library in https://pypi.org/project/agrigee-lite. The full benchmark results and related code are available at https://github.com/mateuspinto/light-crop-classification.

When it comes to technology in agriculture, one of the most important aspects is farmland crop monitoring. However, in most cases, only the main crops are needed to be monitored by satellite images, due to their high territorial extension. Therefore, a semantic segmentation model for identifying plantations should correctly classify the majority classes and also automatically identify other unknown crops. Open set recognition (OSR) aims to embrace both of these causes, so that the model can be more robust in the wild. This work adapts the framework of outlier exposure (OE) for open set image segmentation. OE was evaluated by adding it to three distinct methods for open set segmentation: softmax thresholding, OpenPCS and OpenPCS++. We conducted several experiments to enrich the discussion of the impact of OE on the semantic segmentation of crop imagery. Our methodology achieved a consistent increase for OpenPCS and OpenPCS++ methods, with an improvement of up to 7.5% in terms of area under the receiver operating characteristic (AUROC) curve if compared to previous work.

In traditional semantic segmentation, knowing about all existing classes is essential to yield effective results with the majority of existing approaches. However, these methods trained in a Closed Set of classes fail when new classes are found in the test phase, not being able to recognize that an unseen class has been fed. This means that they are not suitable for Open Set scenarios, which are very common in real-world computer vision and remote sensing applications. In this paper, we discuss the limitations of Closed Set segmentation and propose two fully convolutional approaches to effectively address Open Set semantic segmentation: OpenFCN and OpenPCS. OpenFCN is based on the well-known OpenMax algorithm, configuring a new application of this approach in segmentation settings. OpenPCS is a fully novel approach based on feature-space from DNN activations that serve as features for computing PCA and multi-variate gaussian likelihood in a lower dimensional space. In addition to OpenPCS and aiming to reduce the RAM memory requirements of the methodology, we also propose a slight variation of the method (OpenIPCS) that uses an iteractive version of PCA able to be trained in small batches. Experiments were conducted on the well-known ISPRS Vaihingen/Potsdam and the 2018 IEEE GRSS Data Fusion Challenge datasets. OpenFCN showed little-to-no improvement when compared to the simpler and much more time efficient SoftMax thresholding, while being some orders of magnitude slower. OpenPCS achieved promising results in almost all experiments by overcoming both OpenFCN and SoftMax thresholding. OpenPCS is also a reasonable compromise between the runtime performances of the extremely fast SoftMax thresholding and the extremely slow OpenFCN, being able to run close to real-time. Experiments also indicate that OpenPCS is effective, robust and suitable for Open Set segmentation, being able to improve the recognition of unknown class pixels without reducing the accuracy on the known class pixels. We also tested the scenario of hiding multiple known classes to simulate multimodal unknowns, resulting in an even larger gap between OpenPCS/OpenIPCS and both SoftMax thresholding and OpenFCN, implying that gaussian modeling is more robust to settings with greater openness.

Semantic segmentation is a classic computer vision task with multiple applications, which includes medical and remote sensing image analysis. Despite recent advances with deep-based approaches, labeling samples (pixels) for training models is laborious and, in some cases, unfeasible. In this paper, we present two novel meta-learning methods, named WeaSeL and ProtoSeg, for the few-shot semantic segmentation task with sparse annotations. We conducted an extensive evaluation of the proposed methods in different applications (12 datasets) in medical imaging and agricultural remote sensing, which are very distinct fields of knowledge and usually subject to data scarcity. The results demonstrated the potential of our method, achieving suitable results for segmenting both coffee/orange crops and anatomical parts of the human body in comparison with full dense annotation.

Open set segmentation is a relatively new and unexplored task, with just a handful of methods proposed to model such tasks. We propose a novel method called CoReSeg that tackles the issue using class conditioned reconstruction of the input images according to their pixelwise mask. Our method conditions each input pixel to all known classes, expecting higher errors for pixels of unknown classes. It was observed that the proposed method produces better semantic consistency in its predictions than the baselines, resulting in cleaner segmentation maps that better fit object boundaries. CoReSeg outperforms state-of-the-art methods on the Vaihingen and Potsdam ISPRS datasets, while also being competitive on the Houston 2018 IEEE GRSS Data Fusion dataset. Our official implementation for CoReSeg is available at: https://github.com/iannunes/CoReSeg.

Multitemporal remote-sensing images play a key role as a source of information for automated crop mapping and monitoring. The spatial/spectral pattern evolution along time provides information about the dynamics of the crops and are very useful for productivity estimation. Although the multitemporal mapping of crops has progressed considerably with the advent of deep learning in recent years, the classification models obtained still have limitations when exposed to unknown classes in the prediction phase, reducing their usefulness. In other words, these models are trained to identify a closed set of crops (e.g., soy and sugar cane) and are therefore unable to recognize other types of crops (e.g., maize). In this letter, we deal with the challenges of multitemporal crop recognition by proposing a new approach called OpenPCS++ that is not only able to learn known classes but is also capable of identifying new crops in the predicting phase. The proposed approach was evaluated in two challenging public datasets located in tropical climates in Brazil. Results showed that OpenPCS++ achieved increases of up to 0.19 in terms of area under the receiver-operating characteristic (ROC) curve in comparison with baselines. Code is available athttps://github.com/DiMorten/osss-mcr.

Current deep learning methods for interpreting seismic images require large amounts of labeled data, and due to strategic and economic interests, these data are not plenty available. In this scenario, seismic interpretation can benefit from self-supervised learning by relying on prior training without manually-annotated labels within the target data domain and subsequent fine-tuning with few-shot. To demonstrate the potential of such an approach, we conducted experiments with three classic context-based pretext tasks: rotation, jigsaw, and frame order prediction. Our results for 1, 5, 10 and 20-shots showed significant improvement for mean Intersection-over-Union (mIoU) measurements for semantic segmentation in most scenarios, outperforming the baseline method in 38% in the 1-shot scenario for the F3 Netherlands Dataset, and 16.4% in the New Zealand Parihaka dataset, and this gap grows even higher after performing ensemble modeling. These experiments suggest that applying SSL methods can also bring great benefits in seismic interpretation when few labeled data are available.

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identifying inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.

The constant need of sharing data in information systems leads to the development of more complex and creative solutions to the physical or cost limitations of the nowadays technology. The main problems of a distributed system include: the huge information volume by time interval carried over the network infrastructure, and the confidentiality of the ongoing data. Going into the media transmission sub area, there are even more restrictions to be considered. Error or delay, for example, can drastically impact the user experience in real-time transmission. In this context, this paper proposes a tool for performing efficient and secure distribution and encryption of video streams. This tool was implemented and applied in several contexts. In order to validate the tool in a set of possible situations, tests of the video reflector were made using several sets of parameters evolving variations of video codecs and presence or absence of cryptography.