Non-experts increasingly engage in user-driven algorithm auditing, interacting directly with AI systems to probe, document, and reflect on biased behavior. Yet, auditing remains challenging due to model opacity and limited support for navigating and interpreting outputs. This paper explores the design and evaluation of interfaces grounded in the sensemaking framework to support non-experts in auditing gender bias in image captioning. In a between-subjects study, 60 participants audited an image captioning model using one of three interface conditions: a Baseline interface, a Masking Tool for image manipulation, or a Filtering Tool for organizing captions. Our findings show that interface design shaped what participants noticed, how they interpreted model behavior, and supported their hypotheses. The Image Masking Tool enabled fine-grained testing of visual cues and context, while the Text Filtering Tool revealed broader asymmetries in gendered language. We argue that incorporating sensemaking into auditing practices can advance accountability and transparency in machine learning systems.

While human-centered approaches to machine learning explore various human roles within the interaction loop, the notion of Interactive Machine Teaching (IMT) emerged with a focus on leveraging the teaching skills of humans as a teacher to build machine learning systems. However, most systems and studies are devoted to single users. In this article, we study collaborative interactive machine teaching in the context of image classification to analyze how people can structure the teaching process collectively and to understand their experience. Our contributions are threefold. First, we developed a web application called TeachTOK that enables groups of users to curate data and train a model together incrementally. Second, we conducted a study in which ten participants were divided into three teams that competed to build an image classifier in nine days. Qualitative results of participants’ discussions in focus groups reveal the emergence of collaboration patterns in the machine teaching task, how collaboration helps revise teaching strategies and participants’ reflections on their interaction with the TeachTOK application. From these findings we provide implications for the design of more interactive, collaborative and participatory machine learning-based systems.

Abnormal event detection in videos is a challenging problem, partly due to the multiplicity of abnormal patterns and the lack of their corresponding annotations. In this paper, we propose new constrained pretext tasks to learn object level normality patterns. Our approach consists in learning a mapping between down-scaled visual queries and their corresponding normal appearance and motion characteristics at the original resolution. The proposed tasks are more challenging than reconstruction and future frame prediction tasks which are widely used in the literature, since our model learns to jointly predict spatial and temporal features rather than reconstructing them. We believe that more constrained pretext tasks induce a better learning of normality patterns. Experiments on several benchmark datasets demonstrate the effectiveness of our approach to localize and track anomalies as it outperforms or reaches the current state-of-the-art on spatio-temporal evaluation metrics.

This paper addresses video anomaly detection problem for videosurveillance. Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy, in which our model learns object-centric normal patterns without seeing anomalous samples during training. The main contributions consist in coupling pre-trained object-level action features prototypes with a cosine distance-based anomaly estimation function, therefore extending previous methods by introducing additional constraints to the mainstream reconstruction-based strategy. Our framework leverages both appearance and motion information to learn object-level behavior and captures prototypical patterns within a memory module. Experiments on several well-known datasets demonstrate the effectiveness of our method as it outperforms current state-of-the-art on most relevant spatio-temporal evaluation metrics.

While the research in automatic Sign Language Processing (SLP) is growing, it has been almost exclusively focused on recognizing lexical signs, whether isolated or within continuous SL production. However, Sign Languages include many other gestural units like iconic structures, which need to be recognized in order to go towards a true SL understanding. In this paper, we propose a newer version of the publicly available SL corpus Dicta-Sign, limited to its French Sign Language part. Involving 16 different signers, this dialogue corpus was produced with very few constraints on the style and content. It includes lexical and non-lexical annotations over 11 hours of video recording, with 35000 manual units. With the aim of stimulating research in SL understanding, we also provide a baseline for the recognition of lexical signs and non-lexical structures on this corpus. A very compact modeling of a signer is built and a Convolutional-Recurrent Neural Network is trained and tested on Dicta-Sign-LSF-v2, with state-of-the-art results, including the ability to detect iconicity in SL production.

This paper presents MEDIAPI-SKEL, a 2D-skeleton database of French Sign Language videos aligned with French subtitles. The corpus contains 27 hours of video of body, face and hand keypoints, aligned to subtitles with a vocabulary size of 17k tokens. In contrast to existing sign language corpora such as videos produced under laboratory conditions or translations of TV programs into sign language, this database is constructed using original sign language content largely produced by deaf journalists at the media company Média-Pi. Moreover, the videos are accurately synchronized with French subtitles. We propose three challenges appropriate for this corpus that are related to processing units of signs in context: automatic alignment of text and video, semantic segmentation of sign language, and production of video-text embeddings for cross-modal retrieval. These challenges deviate from the classic task of identifying a limited number of lexical signs in a video stream.

This study proposes a strategy to refine optical flow based on the estimated reliability maps. These maps are firstly estimated a posteriori after the motion estimation by the well‐known Kanade–Lucas–Tomasi (KLT). With two new defined criteria based, respectively, on the optical flow local variance and the temporal evolution of the KLT residuals, a global refinement of the motion map is then carried out through two stages under the control of the reliability measures and the colour local homogeneousness. According to the experiments performed on the Middlebury dataset, the authors' reliability measures prove to be a good indicator for the quality of the estimation. Indeed, the correction process increases the global reliability measures and reduces the global errors in a significant way. The experiments show that the quality is higher than classical estimation methods and ranked at 88/168 on Middlebury website.

International audience

This paper presents a preliminary study which aims to investigate the tangible navigation in 3D Virtual Environments with new tablets (ex. Google Tango) that provide a self-contained 3D full tracking (ex. Google Tango, Intel RealSense). The tangible navigation was compared with tactile navigation techniques used in standard 3D applications (ex. video games, CAD, data visualization). Four conditions were compared: classic multi-touch interaction, tactile interaction with sticks, tangible interaction with a 1:2.5 scale factor and tangible interaction with a 1:5 scale factor. The study focuses on the subjective evaluation users make of the different interaction techniques in terms of usability and acceptability, and the fidelity of representation of the explored scene. Participants were asked to explore a virtual appartment using one of the four interaction techniques. Then, they were requested to draw a 2D sketch map of the explored appartment and to complete a questionnaire of usability and acceptabilty. In order to go further into the investigation, the participant's activity has also been recorded. First results show that exploring a virtual environment using only the device's movements is more usable and acceptable compared to tactile interaction technique, but the difference seems to become smaller as the used scale factor increases.

Projector-camera systems, used in Spatial Augmented Reality, automatically adapt the video projections to the scene objects according to the visualization conditions. This paper introduces a novel non-invasive (without Structured Light) method based on a combination of traditional Feature Matching (FM) and more computationally effective Optical Flow (OF). It requires only one projected and one acquired image at a time in the most difficult case when both projected content and geometric transformations change every frame. It detects scene changes when OF fails and thus should be replaced by FM. In the experiments, we show that the method yields a more precise and less shaky compensation for different types of projected videos, and is up to 2.8 times faster than previous FM-based works.

This paper presents a mobile platform that uses a new generation of tablets equipped with a depth sensor to perform a real time 3D reconstruction of an indoor environment. The platform generates a 3D model where the strucutral elements are identified: ground, ceiling, walls and openings. The 3D model is used for the evaluation of both the geometric features of the building, but also for the assessment of the building's energetic performance. Also, a series of edition and visualization tools are proposed within the platform to assist the user in modifying and exploring the 3D reconstructed environment.

One of the main challenges in a projector-camera system is to be able to project on an arbitrary surface and compensate color changes due to the color of the projection surface. This task becomes dramatically more difficult in case the projection surface changes dynamically. Even a slight displacement of the surface breaks the photometric compensation and visible artifacts may occur. An approach that is able to compensate for a dynamic scene is of great importance since it expands the application range of such a system. In this paper, a method that requires only a single frame in order to adapt its photometric compensation to the new surface is demonstrated. Furthermore, it is compared with another state-of-the-art method that claims one-frame delay compensation. In addition, the complex characterization of a DLP projector is addressed. The results prove that not only our approach outperforms the pre-existing method but also it is robust against challenging surfaces with sharp and saturated color patches, as the ones in a real environment.

This paper proposes a novel definition of color lines and sets, based on the dichromatic model for lambertian objects. The ends of the body vectors are robustly detected, from the clearest to the darkest through to a multi-level 2D histogram analysis. Finally, instead of classically defining the topographic map along one sole luminance direction, our body lines are designed along each body vector. Compared to existing topographic maps, our method is more compact while better preserving the color quality. Furthermore, it is faster to compute than.

This paper proposes an extension to the mean shift tracking by using XY projection-histograms to model the object. More than providing statistical information about the target to track, they embed information about the spatial arrangement of pixels. This approach, without any complexity increase, provides a better robustness and quality of the tracking. That is asserted by the experiments performed on several sequences showing either vehicles or pedestrians in various contexts.

This paper proposes an extension to the mean shift tracking. We introduce the color connectedness degrees (CCD) which, more than providing statistical information about the target to track, embeds information about the amount of connectedness of the color intervals which compose the target. With a low increase of complexity, this approach provides a better robustness and quality of the tracking compared to the use of the RGB space. This is asserted by the experiments performed on several sequences showing vehicles and pedestrians in various contexts.

Unlike edges, level lines are closed and less sensitive to external parameters. They provide a compact geometrical representation of images and they are, to some extent, robust to contrast changes. This paper proposes a novel and vectorial representation of color lines which does not require any color conversion. The topographic map is defined along each dominant color vector, from the body reflection, instead of the sole luminance direction in the RGB space. Experimental results show that this approach provides a better trade-off between compactness and quality. In addition, the lines extracted are quasi-invariant to illuminant changes.

This paper addresses the problem of robust feature points tracking by using specific color invariants -robust to specular reflections, lighting changes and to some extent to color lighting changes-when they are relevant and photometric normalization in the opposite case. Indeed, most color invariants become noisy or irrelevant for low saturation and/or low intensity. They can even make tracking fail. Combining them with luminance information yields to a more performant tracking, whatever the lighting conditions are. A few experiments on real image sequences prove the efficiency of this procedure.

This article proposes a local photometric model that compensates for specular highlights and lighting variations due to position and intensity changes. We define clearly on which assumptions it is based, according to widely used reflection models. Moreover, its theoretical validity is studied according to few configurations of the scene geometry (lighting, camera and object relative locations). Next, this model is used to improve the robustness of points tracking in luminance images with respect to specular highlights and lighting changes.

In this article, we study the color appearance of the ink printed on a background, according to both its concentration and the background color. We find some attributes, the concentration quotients ratios, that are more invariant to the ink concentration than simple color attributes. Our work deals with traceability of porcine products. We have to detect the animal identifier, printed with ink on the pork rind. Using the concentration quotients ratios, our segmentation technique succeeds for any quantity of ink and any hue of pork rind. This technique could be applied to segment any set of pixels, that are colorimetrically and spatially close, but not necessarily all connected.