Abstract Monitoring solar phenomena, such as sunspots and active regions, is crucial for ensuring astronaut safety, telecommunications reliability, and predicting terrestrial events like auroras. Traditional methods for detecting these phenomena have limitations in accuracy and baseline maintenance. This paper presents a novel deep learning object detection method that leverages multispectral image data from satellites to enhance the detection of "sunspots" and active regions. Utilizing images from the SDO satellite and annotations from the DeepSDO dataset, we constructed a new dataset composed of aligned observations from HMI Ic, AIA 211\,\AA, and AIA 335\,\AA. We adapted and developed a stock YOLOv5-based model capable of handling and fusing any number of input images. Two fusion methodologies, early and late fusion, and three different fusion modules --- CatFuse (simple concatenation), CBAMC (CBAM-based module), and TransEnc (transformer encoder) --- were implemented and tested. Our critical evaluation of the models, supported by statistical analysis, proved the developed models to be statistically significantly different among themselves at a p-value of 0.05, and helped us to identify the best-performing model: CatFuse with early fusion, which achieved a [email protected]:0.95 of 0.52 and a [email protected] of 0.94. This result was marginally better than the best baseline (YOLOv5 with a single HMI image) and comparable to other state-of-the-art models, demonstrating a modest but consistent improvement of multispectral image fusion for this task.

Contrastive Analysis (CA) detects anomalies by contrasting patterns unique to a target group (e.g., unhealthy subjects) from those in a background group (e.g., healthy subjects). In the context of brain MRIs, existing CA approaches rely on supervised contrastive learning or variational autoencoders (VAEs) using both healthy and unhealthy data, but such reliance on target samples is challenging in clinical settings. Unsupervised Anomaly Detection (UAD) learns a reference representation of healthy anatomy, eliminating the need for target samples. Deviations from this reference distribution can indicate potential anomalies. In this context, diffusion models have been increasingly adopted in UAD due to their superior performance in image generation compared to VAEs. Nonetheless, precisely reconstructing the anatomy of the brain remains a challenge. In this work, we bridge CA and UAD by reformulating contrastive analysis principles for the unsupervised setting. We propose an unsupervised framework to improve the reconstruction quality by training a self-supervised contrastive encoder on healthy images to extract meaningful anatomical features. These features are used to condition a diffusion model to reconstruct the healthy appearance of a given image, enabling interpretable anomaly localization via pixel-wise comparison. We validate our approach through a proof-of-concept on a facial image dataset and further demonstrate its effectiveness on four brain MRI datasets, outperforming baseline methods in anomaly localization on the NOVA benchmark. • Unsupervised framework enhancing reconstruction quality in brain MRIs. • Target-invariant contrastive encoder capturing meaningful anatomical features. • Conditional diffusion model to reconstruct the healthy appearance of a given image. • Outperforms competing methods in anomaly localization on the NOVA benchmark.

3D human pose estimation aims to reconstruct the human skeleton of all the individuals in a scene by detecting several body joints. The creation of accurate and efficient methods is required for several real-world applications including animation, human–robot interaction, surveillance systems or sports, among many others. However, several obstacles such as occlusions, random camera perspectives, or the scarcity of 3D labelled data, have been hampering the models’ performance and limiting their deployment in real-world scenarios. The higher availability of cameras has led researchers to explore multi-view solutions due to the advantage of being able to exploit different perspectives to reconstruct the pose. Most existing reviews focus mainly on monocular 3D human pose estimation and a comprehensive survey only on multi-view approaches to determine the 3D pose has been missing since 2012. Thus, the goal of this survey is to fill that gap and present an overview of the methodologies related to 3D pose estimation in multi-view settings, understand what were the strategies found to address the various challenges and also, identify their limitations. According to the reviewed articles, it was possible to find that most methods are fully-supervised approaches based on geometric constraints . Nonetheless, most of the methods suffer from 2D pose mismatches, to which the incorporation of temporal consistency and depth information have been suggested to reduce the impact of this limitation, besides working directly with 3D features can completely surpass this problem but at the expense of higher computational complexity . Models with lower supervision levels were identified to overcome some of the issues related to 3D pose, particularly the scarcity of labelled datasets. Therefore, no method is yet capable of solving all the challenges associated with the reconstruction of the 3D pose. Due to the existing trade-off between complexity and performance, the best method depends on the application scenario. Therefore, further research is still required to develop an approach capable of quickly inferring a highly accurate 3D pose with bearable computation cost. To this goal, techniques such as active learning, methods that learn with a low level of supervision, the incorporation of temporal consistency, view selection, estimation of depth information and multi-modal approaches might be interesting strategies to keep in mind when developing a new methodology to solve this task.

Deep Learning models are tremendously valuable in several prediction tasks, and their use in the medical field is spreading abruptly, especially in computer vision tasks, evaluating the content in X-rays, CTs or MRIs. These methods can save a significant amount of time for doctors in patient diagnostics and help in treatment planning. However, these models are significantly sensitive to confounders in the training data and generally suffer a performance hit when dealing with out-of-distribution data, affecting their reliability and scalability in different medical institutions. Deep Learning research on Medical datasets may overlook essential details regarding the image acquisition procedure and the preprocessing steps. This work proposes a data-centric approach, exploring the potential of attention maps as a regularisation technique to improve robustness and generalisation. We use image metadata and explore self-attention maps and contrastive learning to promote feature space invariance to image disturbance. Experiments were conducted using Chest X-ray datasets that are publicly available. Some datasets contained information about the windowing settings applied by the radiologist, acting as a source of variability. The proposed model was tested and outperformed the baseline in out-of-distribution data, serving as a proof of concept.

An autonomous vehicle relying on LiDAR data should be able to assess its limitations in real time without depending on external information or additional sensors. The point cloud generated by the sensor is subjected to significant degradation under adverse weather conditions (rain, fog, and snow), which limits the vehicle's visibility and performance. With this in mind, we show that point cloud data contains sufficient information to estimate the weather accurately and present Mo-bileWeatherNet, a LiDAR-only convolutional neural network that uses the bird's-eye view 2D projection to extract point clouds' weather condition and improves state-of-the-art performance by 15% in terms of the balanced accuracy while reducing inference time by 63%. Moreover, this paper demonstrates that among common architectures, the use of the bird's eye view significantly enhances their performance without an increase in complexity. To the extent of our knowledge, this is the first approach that uses deep learning for weather estimation using point cloud data in the form of a bird's-eye-view projection.

This paper presents the results of an experimental study of the popular Bag-of-Features (BoF) model for the application of unsupervised learning of images, or image clustering.Although this method has been extensively applied for image classification and scene recognition, there has been few works which employ it in an unsupervised way.Also, due to the fact that the BoF model requires a great amount of steps, algorithms and parameter settings, we felt like there was a lack of detailed studies about the subject.We implemented testing routines in Python which we made publicly available in GitHub.In order to assess the performance of the model, three image datasets were used, namely, Coil-20 dataset, Natural and Urban dataset and Event dataset.The results obtained indicate that the BoF method provides a good representation of simple image collections for the purpose of clustering.However, it requires fine tunning of the parameters and algorithms for each dataset and obtains poor results for more complex scene datasets.We can therefore conclude that more advanced techniques are required in order to be able to effectively extract information from large image collections.

While video surveillance systems are spreading everywhere, extracting meaningful information from what they are recording is still prohibitively expensive. There is a major effort under way in order to make this process economical by including an intelligent software that eases the burden of the system. In this paper, we introduce an incremental learning framework to classify parallel data streams generated in a multi-camera surveillance scenario. The framework exploits active learning strategies in order to interact wisely with operators to address various problems that exist in such non-stationary environments, such as concept drift and concept evolution. If we look at the problem as mining parallel streams, the framework can address learning from uneven parallel streams applying a class-based ensemble, a problem that has not been addressed before. Favourable results indicate the success of the framework.

In this paper we present a complete system for object tracking over multiple uncalibrated cameras with or without overlapping fields of view. We employ an approach based on the bag-of-visterms technique to represent and match tracked objects. The tracks are compared with a global object model based on an ensemble of individual object models. The system can globally recognise objects and minimise common tracking problems such as track drift or split. The output is a timeline representing the objects present in a given multi-camera scene. The methods employed in the system are online and can be optimized to operate in real-time.

The design and implementation of multimedia signal processing systems is challenging especially when efficiency and real-time performance is desired. In many modern applications, software systems must be able to handle multiple flows of various types of multimedia data such as audio and video. Researchers frequently have to rely on a combination of different software tools for each modality to assemble proof-of-concept systems that are inefficient, brittle and hard to maintain. Marsyas is a software framework originally developed to address these issues in the domain of audio processing. In this paper we describe MarsyasX, a new open-source cross-modal analysis framework that aims at a broader score of applications. It follows a dataflow architecture where complex networks of processing objects can be assembled to form systems that can handle multiple and different types of multimedia flows with expressiveness and efficiency.