Claim-level Uncertainty Quantification (UQ) is a promising approach to mitigate the lack of reliability in Large Language Models (LLMs). We introduce MUCH, the first claim-level UQ benchmark designed for fair and reproducible evaluation of future methods under realistic conditions. It includes 4,873 samples across four European languages (English, French, Spanish, and German) and four instruction-tuned open-weight LLMs. Unlike prior claim-level benchmarks, we release 24 generation logits per token, facilitating the development of future white-box methods without re-generating data. Moreover, in contrast to previous benchmarks that rely on manual or LLM-based segmentation, we propose a new deterministic algorithm capable of segmenting claims using as little as 0.2% of the LLM generation time. This makes our segmentation approach suitable for real-time monitoring of LLM outputs, ensuring that MUCH evaluates UQ methods under realistic deployment constraints. Finally, our evaluations show that current methods still have substantial room for improvement in both performance and efficiency.

Visual Language Models require substantial computational resources for inference due to the additional input tokens needed to represent visual information. However, these visual tokens often contain redundant and unimportant information, resulting in an unnecessarily high number of tokens. To address this, we introduce PACT, a method that reduces inference time and memory usage by pruning irrelevant tokens and merging visually redundant ones at an early layer of the language model. Our approach uses a novel importance metric to identify unimportant tokens without relying on attention scores, making it compatible with FlashAttention. We also propose a novel clustering algorithm, called Distance Bounded Density Peak Clustering, which efficiently clusters visual tokens while constraining the distances between elements within a cluster by a predefined threshold. We demonstrate the effectiveness of PACT through extensive experiments.

Large Language Models have received significant attention due to their abilities to solve a wide range of complex tasks. However these models memorize a significant proportion of their training data, posing a serious threat when disclosed at inference time. To mitigate this unintended memorization, it is crucial to understand what elements are memorized and why. This area of research is largely unexplored, with most existing works providing a posteriori explanations. To address this gap, we propose a new approach to detect memorized samples a priori in LLMs fine-tuned for classification tasks. This method is effective from the early stages of training and readily adaptable to other classification settings, such as training vision models from scratch. Our method is supported by new theoretical results, and requires a low computational budget. We achieve strong empirical results, paving the way for the systematic identification and protection of vulnerable samples before they are memorized.

As the usage of user-private-data is increasingly monitored by regulatory institutions for security purposes, its transfer becomes more constrained. Synthetic data has recently emerged as a viable alternative to prevent the disclosure of user-protected information that complies with data sharing regulations. Both public and private sectors commonly use a combination of tabular and time-series tables that often contains user-related sensitive information. They are usually intrinsically interlinked as they describe the users and their behaviors over different perimeters. Moreover, it contains both numerical and categorical features, adding complexity to the anonymization task. State of the art generative methods, specialized either in tabular or time-series data, are able to generate high quality synthetic data. However, if each table is generated independently, it becomes impossible to link them. As a result, the usability of such synthetic data is impacted. To address this issue, we not only propose a coherent multi-table generative model that uses Generative Adversarial Networks (GANs) to sample both tabular and time-series tables, but also a conditional time-series generative model that handles both numerical and categorical features. Additionally, many experiments are conducted to analyse the inner modules of our model and evaluate it on an in-house private dataset in order to prove the viability of the synthetic data generated for machine learning tasks.

Points of interest (POIs) are a core component of geographical databases and of location based services. POI acquisition was performed by domain experts but associated costs and access difficulties in many regions of the world reduce the coverage of manually built geographical databases. With the availability of large geotagged multimedia datasets on the Web, a sustained research effort was dedicated to automatic POI discovery and characterization. However, in spite of its practical importance, POI localization was only marginally addressed. To compute POI coordinates an assumption was made that the more data were available, the more precise the localization will be. Here we shift the focus of the process from data quantity to data quality. Given a set of geotagged Flickr photos associated to a POI, close-up classification is used to trigger a spatial clustering process. To evaluate the newly introduced method against different other localization schemes, we create an accurate ground truth. We show that significant localization error reductions are obtained compared to a coordinate averaging approach and to a X-Means clustering scheme.

Improving coding and spatial pooling for bag-of-words based feature design have gained a lot of attention in recent works addressing object recognition and scene classification. Regarding the coding step in particular, properties such as sparsity, locality and saliency have been investigated. The main contribution of this work consists in taking into acount the local spatial context of an image into the usual coding strategies proposed in the state-of-the-art. For this purpose, given an imgae, dense local features are extracted and structured in a lattice. The latter is endowed with a neighborhood system and pairwise interactions. We propose a new objective function to encode local features, which preserves locality constraints both in the feature space and the spatial domain of the image. In addition, an appropriate efficient optimization algorithm is provided, inspired from the graph-cut framework. In conjunction with the maximum-pooling operation and the spatial pyramid matching, that reflects a global spatial layout, the proposed method improves the performances of several state-of-the-art coding schemes for scene classification on three publicly available benchmarks (UIUC 8-sport, Scene-15 and Caltech-101).

The automatic attribution of semantic labels to unlabeled or weakly labeled images has received considerable attention but, given the complexity of the problem, remains a hard research topic. Here we propose a unified classification framework which mixes textual and visual information in a seamless manner. Unlike most recent previous works, computer vision techniques are used as inspiration to process textual information. To do so, we consider two types of complementary tag similarities, respectively computed from a conceptual hierarchy and from data collected from a photo sharing platform. Visual content is processed using recent techniques for bag-of visual-words feature generation. A central contribution of our work is to infer the coding step of the general bag-of-word framework with such similarities and to aggregate these tag-codes by max-pooling to obtain a single representative vector (signature). Final image annotations are obtained via late fusion, where the three modalities (two text-based and one visual-based) are merged during the classification step. Experimental results on the Pascal VOC 2007 and MIR Flickr datasets show an improvement over the state-of-the-art methods, while significantly decreasing the computational complexity of the learning system.

Accurate digital elevation model (DEM) estimation using synthetic aperture radar interferometry still remains a challenging problem in the geographical information science community, particularly in dealing with a high noise rate and atmospheric disturbances. Such task suffers from the lack of efficient and reliable methods to overcome these artifacts. This work provides a method that aims to solve this problem through a Bayesian formulation with the Markovian energy minimization framework. The DEM is generated from a set of multifrequency/multibaseline interferograms using a multichannel phase unwrapping algorithm combined with an estimation method of the atmospheric artifacts. A set of experimental results illustrates the effectiveness and robustness of the proposed approach.

Interferometric synthetic aperture radar (SAR) (InSAR) systems allow 3-D reconstruction of observed scene. In this paper, an innovative approach for phase unwrapping and digital elevation model (DEM) generation using multichannel InSAR data is presented. The proposed algorithm, exploiting both the amplitude and phase of the available complex data, is able to unwrap and simultaneously regularize the observed data. In particular, the exploitation of amplitude data within the unwrapping chain helps in preserving sharp discontinuities typical of urban areas. As a result, the technique provides accurate DEM reconstructions. For this aim, a Markovian approach, together with a new graph-cut-based optimization algorithm, has been considered. The method has been developed specifically to work in urban areas with very high resolution InSAR image stacks, being able to automatically compensate possible phase offsets. Results on both simulated and real case studies are reported, showing the effectiveness of the method.

In this letter, we propose a Markovian approach for interferometric synthetic aperture radar (InSAR) phase reconstruction. Recently, Markovian models based on multichannel InSAR likelihood statistics and total variation prior have been proposed to reconstruct the noisy and wrapped phase. Efficient discrete optimization algorithms based on the graph-cut technique are used to efficiently minimize the energy. Our contribution consists in extending these works to cope with continuous label sets providing more precise and accurate reconstructed profiles. The proposed approach also provides a good way to estimate local hyperparameters to adjust the prior model and preserve well discontinuities in profiles. This task is useful when working with real InSAR data where the quantization of the continuous label set leads to a loss of some physical information. The proposed method is compared to other Markovian approaches with discrete multilabel optimization algorithms. Experiments show better quality results both on simulated and real InSAR data.

The aim of this paper is the three dimensional reconstruction of urban areas using Very High Resolution (VHR) images. The proposed innovative approach for the three dimensional reconstruction is based on the joint exploitation of both amplitude and interferometric phase images of a multichannel SAR system. The information provided by the amplitude data is added to the 3D reconstruction chain, considering that in urban areas edges of amplitude image are likely also present in the interferometric phase one and conversely. Differently from other works present in literature, the proposed technique exploits the amplitude image, not only to improve the phase regularization, but also to improve the phase unwrapping step. The results will show the effectiveness of the method.

This paper copes with the approximate minimization of Markovian energy with pairwise interactions. We extend previous approaches that rely on graph-cuts and move making techniques. For this purpose, a new move is introduced that permits us to perform better approximate optimizations. Some experiments show that very good local minima are obtained while keeping the memory usage low.

Markovian approaches have proven to be effective for solving the multichannel phase-unwrapping (PU) problem, particularly when dealing with noisy data and big discontinuities. This letter presents a Markovian approach to solve the PU problem based on a newapriorimodel, the total variation, and graph-cut-based optimization algorithms. The proposed method turns out to be fast, simple, and robust. Moreover, compared with other approaches, the proposed algorithm is able to unwrap and restore the solution at the same time, without any additional filtering. A set of experimental results on both simulated and real data illustrates the effectiveness of our approach.

With the development of remotely-sensed multisensor satellites like Pleiades Cosmo-Skymed that have the particularity of providing both SAR and optic data, new techniques in image processing are needed. These techniques must take into account the complementarities and differences in nature of these data. A preliminary operation for advanced techniques that use multisensor images such as fusion, classification, etc. is registration. In the case of SAR and optic data, we can do automatic registration if we exactly know the sensor parameters and have a digital terrain model (DTM) or a digital elevation model (DEM) at our disposal. If we do not have an exact knowledge of these parameters, the registration becomes difficult. Another approach to achieve the automatic registration which does not need sensor parameters will rely on comparison measures between both data. In this paper, we present a comparison of several similarity measures between multisensor SAR and optic images used in matching algorithms. An evaluation of these measures for synthetic data based on their distributions is given. Then results on real images are analyzed.