Large language models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks. This research introduces a novel “Prompting-in-a-Series” algorithm, termed psychology-informed content embeddings for personality recognition (PICEPR), featuring two pipelines: 1) contents; and 2) embeddings. The approach demonstrates how a modularised decoder-only LLM can summarize or generate content, which can aid in classifying or enhancing personality recognition functions as a personality feature extractor and a generator for personality-rich content. We conducted various experiments to provide evidence to justify the rationale behind the PICEPR algorithm. Meanwhile, we also explored closed-source models such asgpt4ofrom OpenAI andgeminifrom Google, along with open-source models such asmistralfrom Mistral AI, to compare the quality of the generated content. The PICEPR algorithm has achieved a new state-of-the-art performance for personality recognition by 5–15% improvement. The work repository and models’ weight can be found at:https://research.jingjietan.com/?q=PICEPR.

This paper presents our Cityscape Aerial image Dataset for Object deTection (CADOT), a new benchmark dataset for object detection, based on raw aerial images sourced from the IGN website and focused on a specific area within the Paris department. CADOT consists of cropped sub-images with irrelevant areas carefully excluded. Due to image complexity, an algorithm with human validation was developed for accurate annotation. CADOT contains 106691 object annotations across 14 categories in COCO format, spanning transportation infrastructure to urban features. We provide distinct splits for training, validation, and testing. The dataset’s complexity is highlighted through comparative performance evaluations against well-established detection models using other existing aerial image datasets such as DIOR. Our results show that CADOT offers a significantly greater challenge than other benchmark aerial image datasets. The CADOT dataset is accessible via CADOT .

The Versatile Video Coding (VVC) standard integrates advanced tools to improve coding efficiency, including Reference Picture Resampling (RPR). The latter enables dynamic spatial resolution adjustments without requiring the insertion of additional intra-coded frames. However, implementing per-frame adaptive resampling in hardware-based VVC encoders poses challenges, leading to potential visual artifacts, especially with frequent resolution changes within a single scene. Nevertheless, maintaining a constant resolution throughout an entire sequence may not always be optimal, as varying frame content could benefit from different coding resolutions. This paper explores Full-Scene Resampling-based Coding (FsRC), a hardware-friendly resampling coding method that dynamically adjusts spatial video resolution at scene transitions. We analyze scaling ratios of 1.5 and 2, resulting in sample count reductions of 56% and 75%, respectively, and measure the energy savings and the potential coding efficiency gains. Coding efficiency is evaluated through bitrate savings at equivalent objective metric, the assessed objective metrics including MS-SSIM, VMAF, and PSNR. An exhaustive search establishes the coding efficiency upper bound for FsRC by systematically testing all scaling ratios and selecting the optimal for each encoding, simulating ideal decision-making.

Compressing high-resolution video content under strict bitrate constraint is a major challenge, particularly when aiming to preserve visual quality. The core difficulty arises from the increase in pixel count with resolution, despite limited bitrate availability. Resampling-based approaches offer a practical trade-off between coding efficiency and computational cost in this context. In this paper, we propose a scene-level resolution adaptation strategy for Versatile Video Coding (VVC), in which resampling scales are dynamically selected at scene boundaries, where content changes are typically most pronounced. This adaptation is guided by a lightweight machine learning model that informs the decision-making process at these transitions, enabling efficient compression without degrading the decoded video quality. Experimental results on JVET CTC sequences under standard VVC conditions show bitrate savings of up to −14.97% (PSNR) and −9.71% (VMAF) in ultra-low bitrate scenarios, approaching the upper bound defined by exhaustive search. Across very-low and low bitrate settings, the proposed method consistently outperforms parametric state-of-the-art baselines in terms of BD-rate. With an average inference time of only 1.02ms per sample in batched mode, our approach is also well suited for real-time encoding scenarios.

International audience

Recently, Deep Reinforcement Learning (DRL) has been applied to enhance the Quality of Experience (QoE) of Adaptive Bitrate Streaming (ABR) by adjusting the video quality level in real time based on instantaneous network conditions. To build a state-of-the-art DRL-based ABR (DRLABR) algorithm, it must learn from the clients’ actual network and video streaming behavior. However, collecting such data directly from clients introduces several challenges, including privacy concerns, high bandwidth consumption, and the straggler effect—where poor network conditions of certain clients delay the training process, as DRLABR’s performance is highly dependent on network interactions. To overcome these limitations, we propose a decentralized training approach for DRLABR using a Federated Learning (FL) framework. Instead of gathering raw data, clients train their local DRLABR models independently and send only model updates to the central server. To address the straggler issue, we propose to desynchronize the FL update rules, allowing clients to contribute their model updates at their own pace, regardless of varying network conditions. In addition, we design a DRL-based client selection mechanism to prevent oversampling of high-bandwidth clients, which could lead to model divergence, thereby ensuring balanced participation and improving the overall training efficiency. We validate our approach through a comprehensive simulation encompassing diverse video content and real-world network traces, simulating a wide range of streaming activities. Our results show that the proposed framework significantly outperforms conventional FedAvg and FedAsync methods, achieving the highest average QoE score of 2.02 and reducing the total training latency by 21.26%.

Deep learning has significantly advanced the field of building extraction from remote sensing images, providing robust solutions for identifying and delineating building footprints. However, a major challenge persists in the form of domain adaptation, particularly when addressing cross-city variations. The primary challenge lies in the significant differences in building appearances across cities, influenced by variations in building shapes and environmental characteristics. Consequently, models trained on data from one city often struggle to accurately identify buildings in another city. In this paper, we address this challenge from a data-centric perspective, focusing on diversifying the training set. Our empirical results show that improving data diversity via open-source datasets and diffusion augmentation significantly improved the performance of the segmentation model. Our baseline model, trained with no extra dataset, only achieved a private F1 score of 0.663. On the other hand, our model trained with the additional Las Vegas building footprints extracted from the Microsoft Building Footprint dataset, achieved a high private F1 score of 0.703. Surprisingly, we found that diffusion augmentation helps improve our model score to 0.681 without requiring an extra dataset, which is higher than the baseline model. Finally, we also experimented with the Non-Maximal Suppression (NMS) hyperparameter to improve the model’s performance in segmenting dense and small objects, which gave us a high private F1 score of 0.897. These techniques ultimately led our solution to rank 1st in the competition. Our source code and the pretrained models are publicly available at https://github.com/DoubleY-BEGC2024/OurSolution.

Object detection in Remote Sensing Images (RSI) is a critical task for numerous applications in Earth Observation (EO). Differing from object detection in natural images, object detection in remote sensing images faces challenges of scarcity of annotated data and the presence of small objects represented by only a few pixels. Multi-modal fusion has been determined to enhance the accuracy by fusing data from multiple modalities such as RGB, infrared (IR), lidar, and synthetic aperture radar (SAR). To this end, the fusion of representations at the mid or late stage, produced by parallel subnetworks, is dominant, with the disadvantages of increasing computational complexity in the order of the number of modalities and the creation of additional engineering obstacles. Using the cross-attention mechanism, we propose a novel multi-modal fusion strategy for mapping relationships between different channels at the early stage, enabling the construction of a coherent input by aligning the different modalities. By addressing fusion in the early stage, as opposed to mid or late-stage methods, our method achieves competitive and even superior performance compared to existing techniques. Additionally, we enhance the SWIN transformer by integrating convolution layers into the feed-forward of non-shifting blocks. This augmentation strengthens the model’s capacity to merge separated windows through local attention, thereby improving small object detection. Extensive experiments prove the effectiveness of the proposed multimodal fusion module and the architecture, demonstrating their applicability to object detection in multimodal aerial imagery. Our code is available at here.

Object Detection models are difficult to adapt to real use cases as they require large and expensive datasets during training. Few-Shot Object Detection (FSOD) tries to solve detection with only limited annotated data. It relies on extensive pre-training and elaborated fine-tuning strategy and model design. Most FSOD approaches are designed and evaluated only on natural images which result in consistent performance drop when applied to other kind of images such as aerial ones. We propose, Few-Shot DiffusionDet (FSDD), a novel approach using a recent diffusion-based object detector. FSDD largely outperforms existing work on aerial images while remaining competitive with natural images. The versatility of FSDD is demonstrated with thorough experiments in the FSOD setting and with multiple kinds of images in the more challenging scenario of Cross-Domain FSOD.

Transformers have led to learning-based image compression methods that outperform traditional approaches. However, these methods often suffer from high complexity, limiting their practical application. To address this, various strategies such as knowledge distillation and lightweight architectures have been explored, aiming to enhance efficiency without significantly sacrificing performance. This paper proposes a State Space Model-based Image Compression (SSMIC) architecture. This novel architecture balances performance and computational efficiency, making it suitable for real-world applications. Experimental evaluations confirm the effectiveness of our model in achieving a superior BD-rate while significantly reducing computational complexity and latency compared to competitive learning-based image compression methods.

Small object detection is a challenging task in computer vision. In the few-shot regime, this problem is reinforced. Leveraging useful information from only a few examples is difficult, in particular with small objects. We hypothesize that features extracted from small objects are noisy and often dominated by background information. In addition, recent detectors rely on multi-scale features and visually similar objects of different sizes may have unaligned representations. We address these issues with Cross-Scale Query-Support Alignment (XQSA), a novel attention mechanism that combines features from query and support images at different scales. This allows matching objects of different sizes and therefore improves Few-Shot Object Detection (FSOD) performance. Extensive experiments are conducted on four distinct datasets, including natural images (Pascal VOC and MS COCO) and aerial images (DOTA and DIOR). XQSA improves the detection of small objects on all tested datasets. In aerial images, which contain smaller objects, it yields significant gains for the overall detection and outperforms the state-of-the-art results on DOTA and DIOR.

This paper proposes a few-shot method based on Faster R-CNN and representation learning for object detection in aerial images. The two classification branches of Faster R-CNN are replaced by prototypical networks for online adaptation to new classes. These networks produce embeddings vectors for each generated box, which are then compared with class prototypes. The distance between an embedding and a prototype determines the corresponding classification score. The networks are trained in an episodic manner. A new detection task is randomly sampled at each epoch, consisting in detecting only a subset of the classes annotated in the dataset. This strategy encourages the network to adapt to new classes as it would at test time. In addition, several ideas are explored to improve the proposed method such as a hard negative examples mining strategy and self-supervised clustering for background objects. The performance of our method is assessed on DOTA, a large-scale remote sensing images dataset. The experiments conducted provide a broader understanding of the capabilities of representation learning. It highlights in particular some intrinsic weaknesses for the few-shot object detection task. Finally, some suggestions and perspectives are formulated according to these insights.

This paper deals with semi-blind channel estimation in Single-Input Single-Output (SISO) Orthogonal Frequency Division Multiplexing (OFDM) communications system. The proposed algorithm proceeds in two main stages. The first one addresses the pilot-based Time-Of-Arrival (TOA) estimation using subspace methods and then estimates the channel through its specular model. In the second stage, one considers a decision feedback equalizer that is used to refine the channel parameters estimates. Simulation results show that good performance can be reached with only one OFDM pilot symbol with appropriate windowing using only one iteration. A significant performance improvement as compared to the pilot-based TOA method is observed.

This study deals with semi‐blind (SB) channel estimation of multiple‐input multiple‐output orthogonal frequency‐division multiplexing (MIMO‐OFDM) system using maximum likelihood (ML) technique. For the ML cost optimisation function, new expectation maximisation (EM) algorithms for the channel taps estimation are introduced. Different approximation/simplification approaches are proposed for the algorithm's computational cost reduction. The first approach consists of decomposing the MIMO‐OFDM system into parallel multiple‐input single‐output OFDM systems. The EM algorithm is then applied to estimate the MIMO channel in a parallel way. The second approach takes advantage of the SB context to reduce the EM cost from exponential to linear complexity by reducing the size of the search space. Finally, the last proposed approach uses a parallel interference cancellation technique to decompose the MIMO‐OFDM system into several single‐input multiple‐output OFDM systems. The latter are identified in a parallel scheme and with a reduced complexity. The performance of the proposed approaches are discussed, assessed through numerical experiments and compared with respect to the Cramèr Rao Bound and to other EM‐based solutions reported in the literature.

Channel estimation, which is a key task in massive multiple‐input multiple‐output orthogonal‐frequency division‐multiplexing systems (MIMO‐OFDM), is severely affected by the problem of pilot contamination during the uplink transmission. Thus, the aim of this study is to investigate, via the Cramér‐Rao Bound tool, the effectiveness of semi‐blind (SB) methods for pilot contamination mitigation. For synchronous cells, these analyses demonstrate the possibility to efficiently solve the pilot contamination problem, with SB approaches, when considering a finite‐alphabet (non‐Gaussian) communications signal. However, considering only the signal's Second Order Statistics is not enough for solving such an issue even if the SB approach is adopted. Moreover, the analyses show that it is possible to get close to the optimal performance with a SB approach even if the pilots are non‐orthogonal as long as they are not fully coherent. For the asynchronous cells case, it has been demonstrated that the pilot contamination still occurs under small inter‐cell delays, but can be strongly mitigated with large inter‐cell delays.

This paper deals with semi-blind (SB) channel estimation of Multiple-Input Multiple-Output Orthogonal Frequency-Division Multiplexing (MIMO-OFDM) wireless communications system in the uplink transmission. Herein, we propose a new channel estimation approach using the well known EM technique. More precisely, we derive first the SB EM algorithm in the MIMO case. Then, a parallelizable version of this algorithm is introduced relying on the decomposition of the MIMO system into several MISO systems. Finally, we propose a reduced cost EM version where only the lattice points in the neighboring of the pilot-based detected symbols are considered.

This paper deals with semi-blind channel estimation Cramèr Rao Bound (CRB) performance of a multiuser Multiple-Input Multiple-Output Orthogonal Frequency-Division Multiplexing (MIMO-OFDM) wireless communication system in the uplink transmission. The first contribution shows that the Carrier Frequency Offset (CFO) impacts advantageously the CRB of the semi-blind channel estimation mainly due to the CFO cyclostationarity propriety. The second contribution states that when the relation between the subcarrier channel coefficients is not taken into account, i.e. without resorting to the inherent OFDM ‘channel structure’ during the channel estimation, results in a loss of the estimation performance. An evaluation of the significant performance loss resulting from this approach is provided.

Most communications systems require channel estimation for equalization and symbol detection. Currently, this is achieved by using dedicated pilot symbols, which consume a non-negligible part of the throughput and power resources, especially for large dimensional systems. The main objective of this paper is to quantify the rate of reduction of this overhead due to the use of a semi-blind channel estimation. Different data models and different pilot design schemes have been considered in this paper. By using the Cramér Rao Bound (CRB) tool, the estimation error variance bounds of the pilot-based and semi-blind based channel estimators for a multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) system are compared. In particular, for large MIMO-OFDM systems, a direct computation of the CRB is prohibitive, and hence, a dedicated numerical technique for its fast computation has been developed. Many key observations have been made from this comparative study. The most important one is that, thanks to the semi-blind approach, one can skip about 95% of the pilot samples without affecting the channel estimation quality.

With the recent advances in stereoscopic display technologies, there is a growing demand for designing efficient stereo image compression techniques. For this reason, a great attention should be paid to the disparity/estimation process used to generate the residual image. In this paper, we propose to improve the disparity compensation process in a typical closed-loop-based stereo image coding scheme. A new formulation of this process, based on a block dependent dictionary, is developed. More specifically, the main idea aims to link together the disparities yielding similar compensations and assign a common disparity candidate to each subset of disparities. Experimental results have shown the interest of the proposed method in terms of bitrate saving and quality of reconstruction.

This paper deals with synchronization problem in IEEE 802.11a wireless system. In addition to traditional training sequences, the SIGNAL field of the physical frame can be considered as a new source of information. Indeed the receiver is able to predict the SIGNAL unknown parts relying on the knowledge provided by the CSMA/CA protocol during the negotiation of the transmission medium reservation. The exchanged RtS control frame used jointly with the bit-rate adaptation algorithm to the channel helps the receiver not only to predict the SIGNAL field but also to get information on the channel state. Based on this knowledge, joint MAP channel, time and frequency synchronization algorithm is carried out. Moreover to estimate the residual time offset, a timing metric in frequency domain is performed by minimizing the average of transmission errors in the presence of all channel estimation errors. The performance in terms of probability of synchronization failure is shown to be improved compared to existing algorithms.

This paper addresses the time synchronization problem in IEEE 802.11a OFDM wireless systems. To enhance the coarse time synchronization mechanism, recent methods exploit not only traditional training sequences as specified by the standard but also additional knowledge available when the Carrier Sense Multiple Access with Collision Avoidance mechanism (CSMA/CA) is triggered. In this case, additional information can be used as training sequences based on the protocol knowledge training sequence known by the receiver. This step is followed by a time synchronization and channel estimation which results in the smallest Channel Estimate Errors (CEE) according to the selected criterion (e.g. LS, MAP). However it was found that the synchronization failure probability heavily depends on the channel estimate quality. Therefore to improve the performance of this class of algorithms, we propose an optimal time synchronization metric that minimizes the average of the transmission error over all CEE. Simulation results show a strongly improved performance in terms of synchronization failure probability in comparison with the existing algorithms.

This paper proposes an algorithm for the robust reception of compressed HTML files transmitted over a noisy mobile radio channel. Both source encoders and transmission systems are assumed to be standard compliant. The source encoder follows the HTTP1.1 protocol specifications, i.e. the HTML files are encoded by the deflate algorithm, a combination of Lempel-Ziv and Huffman algorithms. The transmission scheme follows IEEE 802.11a (and 802.11n) standard as an example. The proposed receiver is based on an iterative joint source-channel decoding approach. The Soft-Input Soft-Output outer source decoder is based on a sequential M-algorithm, which has been modified to improve the decoding performance by exploiting the specific grammatical and syntax rules of (i) Huffman codes; (ii) Lempel-Ziv codes; and (iii) HTML language. Simulation results following the IEEE 802.11a (and 802.11n) standard over additive white Gaussian noise and Rayleigh fading channels show that the proposed receiver drastically reduces the number of errors occurring in the received HTML files compared to the classical receivers. An EXIT chart analysis illustrates some properties of this combination of source and channel decoders.

Image quality assessment is still an active field of research. The main objective of the developed image quality metric is to offer an index of quality that is consistent with the human subjective judgment of image quality. Despite the great number of developed metrics, there is still a need for image analysis tools that is able to extract the most perceptual relevant characteristics of an image. The goal of this work is then to propose a more advanced analysis and representation tools to extract more effective features that could be incorporated in the design of the image quality metric. In this paper, we propose a novel objective metric based on wave atoms transform. This new transform is half multiscale and half multi-directional. It offers a better representation of images containing oscillatory patterns and textures than the others known transforms. In this work, we propose a new full reference image quality metric based on wave atom transform and exploiting some properties of the human visual system. The consistency of the proposed metric with subjective evaluation is performed on LIVE database. The obtained correlation of this metric with the MOS provided by the database is better than other known metrics confirming thus the efficiency of this new image quality measure in predicting image quality.

This paper focuses on the bidirectional motion estimation problem for videoconferencing applications at very low bit rate. The missed frames are predicted by the decoder using only the transmitted frames with no additional information. The proposed approach is based on spatio-temporal interpolation. On each of the two selected reference frames, the same chosen moving objects are meshed using deformable block quad-tree decomposition. The positions of the mesh nodes are related to the content of the objects in such a way that the quadratic spatial reconstruction error of the objects is as small as possible. From these nodes, a temporal cubic spline interpolation predicts the mesh nodes of the moving object in the missed frame reconstructing then the meshed object. The proposed approach is integrated in the H.264/AVC video coding standard. Simulation results present the performances of the proposed bidirectional motion estimation, at very low bit rate.

This paper concerns the error correction of corrupted compressed HTML pages during their transmission via a noisy mobile channel. The proposed receiver is based on an iterative joint source channel decoding approach similar to the turbo decoding of two serial concatenated codes. We propose a soft-input soft-output Lempel-Ziv inner decoder based on the modified version of the traditional sequential decoding M-algorithm. This new algorithm exploits the specific grammatical rules of the Lempel-Ziv-77 codes which are combined to the syntax of the HTML language. This source decoder is combined to a Soft-Input Soft-Output channel decoder of convolutional codes. Simulation results, over an additive white Gaussian noise channel, show that the proposed method drastically reduces the number of files in error compared to any conventional channel decoding.

This paper proposes a new still image codec. The encoder has the following structure: a set of pixels of the image is selected and transmitted, together with their position. Then, the value of the image at other places is obtained by a prediction algorithm at the decoder. A useful theoretical covariance model adapted to the image to be encoded is proposed, avoiding the transmission of additional information. The selected pixels and their corresponding positions on the image are encoded using lossless coding algorithms. The computational time of the decoding process is significantly reduced according to the efficient structured memory organization of (i) the image covariance values and (ii) the ordered distances concerning the search of nearest pixels. Experimental results performed on a set of test images show that the rate- distortion results are competitive to the best coders JPEG 2000 and SPIHT with arithmetic coding.

This paper investigates the mathematical framework of the two-dimensional multiresolution analysis adapted to irregularly spaced data. This two-dimensional multiresolution is related on separable multiresolution analysis using non-uniform B-spline functions. For any arbitrary degree of the spline function, we propose an orthonormalization procedure of the scaling and wavelet bases. These orthonormal basis functions satisfy the important features required by a traditional multiresolution analysis such as: (i) the continuity conditions of the scaling and wavelet functions and (ii) the compact support of the scaling and wavelets functions. We show that the orthogonal decomposition is implemented using filter banks where the coefficients depend on the location of the samples on the image grid.

This paper proposes an efficient implementation of fast FIR filtering algorithms with useful characteristics for real-time application. They maintain a low processing delay, independent of the filter length. The difficulty is to keep as much as possible of the improvement brought by the reduction of the arithmetic complexity of these fast FIR filtering algorithms without exceeding the Digital Signal Processor (DSP) resources (number of registers, pointers, memory, ...). A particular attention is devoted to the heavy use of pointers which represents a crucial problem. It is solved in this paper by an optimal organisation of data in memory. Improvements of more than 70% in actual timings on an ADSP-2100 compared to the classical algorithm of convolution are obtained, even for very short blocks.

The paper proposes an efficient implementation of fast composite length FIR algorithms on the digital signal processor (DSP) "ADSP-2100", which closely follows the mathematical structure of the algorithm. The difficulty is to keep as much as possible the improvement brought by the reduction of the complexity without exceeding the DSP resources (number of registers, pointers, memory,...). This difficulty was already pointed out in previous papers, in which the algorithm was restricted to a single decomposition due to these constraints. Particular attention was devoted to this problem. The solution is to structure the algorithm in such a way that the organization of data in the memory is optimised. The authors propose an implementation requiring only five pointers whatever the number of iterations. An improvement of more than 50% in terms of actual throughput (number of cycles per point) compared to the implementation of the direct convolution is achieved.>