International audience

International audience

Over the past few years, notable advancements have been made through the adoption of self-attention mechanisms and perceptual optimization, which have proven to be successful techniques in enhancing the overall quality of image reconstruction.Self-attention mechanisms in Vision Transformers have been widely used in neural networks to capture long-range dependencies in image data, while perceptual optimization has been shown to enhance the perceptual quality of reconstructed images.In this paper, we present a novel approach to image reconstruction by bridging the capabilities of Vision Transformer and Perceptual Compressive Sensing Networks.Specifically, we use a self-attention mechanism to capture the global context of the image and guide the sampling process, while optimizing the perceptual quality of the sampled image using a pretrained perceptual loss function.Our experiments demonstrate that our proposed approach outperforms existing state-of-the-art methods in terms of reconstruction quality and achieves visually pleasing results.Overall, our work contributes to the development of efficient and effective techniques for image sampling and reconstruction, which have potential applications in a wide range of domains, including medical imaging and video processing.

The progress achieved in transportation systems and artificial intelligence has amplified the use of intelligent transportation systems and Autonomous Vehicles (AVs). Indeed, AV systems have attracted much research in recent years, which enabled multiple autonomous driving tasks, including scene understanding, visual prediction, decision-making, and communication. The latter may create a bottleneck in low-resource autonomous driving systems that send the collected images to remote edge servers for processing and decision-making. Such an issue can be addressed by compressing the images in the AV and ensuring a good-quality reconstruction at the edge. In this paper, we propose a deep neural network for Compressed Sensing (CS) based image reconstruction that integrates image semantic perception to improve the reconstruction process for visual prediction tasks. The reconstruction process is optimized using a perception-inspired loss in an end-to-end model learning process. The trained model is evaluated on autonomous driving car datasets. Obtained experimental results outperform state-of-the-art approaches in terms of both image reconstruction quality and processing time. Finally, we perform semantic urban scene segmentation on the reconstructed image to evaluate reconstruction quality for visual task prediction. Obtained results on three semantic urban scene datasets demonstrate the efficiency of the proposed approach.

The optimal transmit power that maximizes energy efficiency (EE) in Longe Range (LoRa) networks is investigated by using the deep learning (DL) approach. Particularly, the proposed artificial neural network (ANN) is trained two times; in the first phase, the ANN is trained by the model-based data which are generated from the simplified system model while in the second phase, the pre-trained ANN is re-trained by the practical data. Numerical results show that the proposed approach outperforms the conventional one which directly trains with the practical data. Moreover, the performance of the proposed ANN under both partial and full optimum architecture are studied. The results depict that the gap between these architectures is negligible. Finally, our findings also illustrate that instead of fully re-trained the ANN in the second training phase, freezing some layers is also feasible since it does not significantly decrease the performance of the ANN.

Recently, Computer Aided Diagnosis (CAD) systems based on Convolutional Neural Networks (CNNs) have shown promising performance in Alzheimer’s Disease (AD) diagnosis using imaging biomarkers. However, the detection of early AD stages namely Mild Cognitive Impairment (MCI) and Mild Alzheimer Disease (MAD) remains hard to achieve using imaging features. Proton Magnetic Resonance Spectroscopy (1H-MRS), a powerful non-invasive technique for early diseases diagnosis, provides a promising solution for early biological brain changes detection. In this paper, we propose an explainable classification framework for early AD detection using1H-MRS. The proposed method consists of an end-to-end One Dimensional-CNN model integrating a novel decision interpretation method. Data used in this paper are collected in the University Hospital of Poitiers, which contain 1111H-MRS samples divided into 3 classes namely Normal Control (NC), MCI and MAD. The proposed framework achieves an accuracy of 82% between for the most challenging classification task (MCI vs. MAD classification). Yet, the proposed AD detection framework is explainable, highlighting the clinically relevant brain metabolites for early AD subjects discrimination.

International audience

Fifth generation and beyond networks are envisioned to support multi industrial Internet of Things (IIoT) applications with a diverse quality-of-service (QoS) requirements. Network slicing is recognized as a flagship technology that enables IIoT networks with multiservices and resource requirements by allowing the network-as-infrastructure transition to the network-as-service. Motivated by the increasing IIoT computational capacity, and taking into consideration the QoS satisfaction and private data sharing challenges, federated reinforcement learning (RL) has become a promising approach that distributes data acquisition and computation tasks over distributed network agents, exploiting local computation capacities and agent's self-learning experiences. This article proposes a novel deep RL scheme to provide a federated and dynamic network management and resource allocation for differentiated QoS services in future IIoT networks. This involves IIoT slices resource allocation in terms of transmission power (TP) and spreading factor (SF) according to the slices QoS requirements. Toward this goal, the proposed deep federated Q-learning (DFQL) is reached into two main steps. First, we propose a multiagent deep Q-learning-based dynamic slices TP and SF adjustment process that aims at maximizing self-QoS requirements in term of throughput and delay. Second, the deep federated learning is proposed to learn multiagent self-model and enable them to find an optimal action decision on the TP and the SF that satisfy IIoT virtual network slice QoS reward, exploiting the shared experiences between agents. Simulation results show that the proposed DFQL framework achieves efficient performance compared to the traditional approaches.

In this paper, we propose a multimodal framework for video segment interestingness prediction based on the genre and affective impact of movie content. We hypothesize that the emotional characteristic and impact of a video infer its genre, which can in turn be a factor for identifying the perceived interestingness of a particular video segment (shot) within the entire media. Our proposed approach is based on audio-visual deep features for perceptual content analysis. The multimodal content is quantified in a mid-level representation which consists in describing each audio-visual segment as a distribution over various genres (action, drama, horror, romance, sci-fi for now). Some segment might be more characteristic of a media and therefore be more interesting than a segment containing content with a neutral genre. Having determined the genre of individual video segments, we trained a classifier to produce an interestingness factor which is then used to rank segments. We evaluate our approach on the MediaEval2017 Media Interestingness Prediction Task Dataset (PMIT). We demonstrate that our approach outperforms the existing video interestingness approaches on the PMIT dataset in terms of Mean Average Precision.

In this paper, we study the importance of spectral sensitivity functions in constructing discriminative representation of hyperspectral images (HSI). The main goal of a such representation is to improve image content recognition by focusing the processing only on the most relevant spectral channels. The underlying hypothesis is that for a given category, each image content is better extracted through a specific set of spectral sensitivity functions. In this study, we fixed the number of spectral sensitivity functions to 3 for displaying purposes. Deep features are then extracted from the obtained trichromatic representation of HSI data to build a discriminative image signature. Finally, spectral sensitivity functions are compared in a Content-Based Image Retrieval (CBIR) paradigm. Exhaustive experiments have been conducted on a hyperspectral dataset. Obtained results show the usefulness of the whole spectrum to obtain a discriminative image representation compared to the RGB representation.

In this paper, we propose a multimodal deep learning architecture for emotion recognition in video regarding our participation to the audio-video based sub-challenge of the Emotion Recognition in the Wild 2017 challenge. Our model combines cues from multiple video modalities, including static facial features, motion patterns related to the evolution of the human expression over time, and audio information. Specifically, it is composed of three sub-networks trained separately: the first and second ones extract static visual features and dynamic patterns through 2D and 3D Convolutional Neural Networks (CNN), while the third one consists in a pretrained audio network which is used to extract useful deep acoustic signals from video. In the audio branch, we also apply Long Short Term Memory (LSTM) networks in order to capture the temporal evolution of the audio features. To identify and exploit possible relationships among different modalities, we propose a fusion network that merges cues from the different modalities in one representation. The proposed architecture outperforms the challenge baselines (38.81 % and 40.47 %): we achieve an accuracy of 50.39 % and 49.92 % respectively on the validation and the testing data.

Second screen applications are becoming key for broadcasters exploiting the convergence of TV and Internet. Authoring such applications however remains costly. In this paper, we present a second screen authoring application that leverages multimedia content analytics and social media monitoring. A back-office is dedicated to easy and fast content ingestion, segmentation, description and enrichment with links to entities and related content. From the back-end, broadcasters can push enriched content to front-end applications providing customers with highlights, entity and content links, overviews of social network, etc. The demonstration operates on political debates ingested during the 2017 French presidential election, enabling insights on the debates.

In this paper, we propose a feature-based classification framework for Alzheimer's disease (AD) recognition using Tensor Diffusion Imaging (DTI). The main contribution consists in considering the visual pattern of water molecules diffusion in the most involved region in AD (hippocampal area). We use the Circular Harmonic Functions (CHFs) and the Bag-of-Visual-Words approach to build an AD related-signature. The experiments were accomplished first with a subset of participants from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and then with the DTI scans of a French epidemiological study: ”Bordeaux-3City”. Experimental results demonstrate that our features-based method applied on the MD maps is able to capture the AD-related atrophy and then classify between AD subjects.