Soccer video understanding has motivated the creation of datasets for tasks such as temporal action localization, spatiotemporal action detection, jersey recognition, and multi-object tracking. Automating the annotation of structured sequences of events (who does what, when, and where) used in soccer analytics requires a holistic integration of these computer-vision subtasks. However, current action recognition methods remain insufficient for producing reliable play-by-play data and are typically used to assist rather than fully automate annotation. In parallel, research in tactical modeling and trajectory forecasting has highlighted the strong tactical and temporal regularities that structure soccer, motivating the use of tactical knowledge as a prior to support vision-based predictions and enable more automated and reliable play-by-play reconstruction. We introduce Footovision Play-by-Play Action Spotting in Soccer Dataset (FOOTPASS), the first benchmark for play-by-play action spotting over entire soccer matches in a multi-modal, multi-agent tactical context. It supports the development of player-centric action spotting methods that combine computer-vision outputs with long-term tactical regularities inherent to the game. In addition, the dataset benchmarks several spatiotemporal action detection methods, highlighting the limitations of purely perceptual approaches and demonstrating the benefits of incorporating tactical reasoning for improving the reliability and completeness of play-by-play predictions.

PosePilot is a Web-Based Application for the analysis and visualization of human motion data in a Biovision Hierarchy (BVH) format. It allows users to upload, view, and analyze human motion data directly within a web browser, eliminating the need for specialized software or programming knowledge. It displays 3D skeleton animation in real-time, in addition to interactive 2D and 3D plots of human joints’ positions and rotations. With its adaptable interface and intuitive design, PosePilot offers a scalable solution, valuable for researchers, educators, and professionals involved in human motion studies.

Human gesture recognition plays an important role in professional environments and applications such as industrial automation and human-machine collaboration. Standard Single-Task Learning (STL) architectures, while effective for gesture recognition, appear robust only in curated settings, lacking the ability to generalize in real-world scenarios due to the inherent variability of human movement. Multi-Task Learning (MTL) appears as method able to enhance generalization by exchanging knowledge among tasks, without taking into consideration the temporal dependencies and stochasticity of human movement. This work addresses these limitations, by introducing a hierarchical structure that decomposes professional movements into three levels and an Autoregressive State-Space Loss (ASSL) function that introduces stochasticity to the model. Experiments on real-world datasets demonstrate that MTL with both the hierarchical structure and the ASSL function provide stability and pave the way for new learning approaches.

International audience

This paper introduces an innovative web-based platform for visualizing, analyzing, and generating human movement, leveraging explainable artificial intelligence (AI). It addresses the challenge of limited data by offering one-shot and data-intensive training approaches, facilitates data augmentation, and provides insights into human dexterity through user interaction with the learned movement models. The paper outlines the prototype's implementation, highlighting its tools for analyzing and visualizing complex movements, and discusses its application in analyzing professional movements within the heritage crafts and manufacturing industry. Future work aims to expand its motion capture technology compatibility and improve analytical functionalities. This enhancement seeks to extend the platform's reach, making it a valuable tool for researchers and professionals across various disciplines.

Extensive research has been conducted on analyzing human movements, driven by its diverse practical applications such as human–robot interaction, human learning, and clinical diagnosis. However, the current state-of-the-art still encounters scientific challenges when it comes to modeling human movements. There are two key aspects that need to be addressed. Firstly, new models should consider the stochastic nature of human movement and the physical structure of the human body to accurately predict the patterns in full-body motion descriptors over time. Secondly, while deep learning algorithms have been utilized, they lack explainability in terms of predicting body posture sequences, making it essential to improve their comprehensible representation of human movement. This paper aims to tackle these challenges by presenting three innovative methods for creating explainable representations of human movement. The study formulates human body movement as a state-space model based on the Gesture Operational Model (GOM). Model parameters are estimated through either one-shot training employing Kalman Filters or data-intensive training utilizing artificial neural networks. The trained models are utilized for analyzing the dexterity of expert professionals in full-body movements, enabling the identification of dynamic associations between body joints and gesture recognition. Additionally, these models are employed to generate artificial professional movements.

The high prevalence of work-related musculoskeletal disorders (WMSDs) could be addressed by optimizing Human-Robot Collaboration (HRC) frameworks. In this context, this paper proposes a methodology for ergonomically effective task delegation and HRC for manufacturing applications based on two hypotheses. The first hypothesis states that it is possible to rapidly quantify ergonomically professional tasks using motion data from a few wearable sensors and then delegate the high-risk tasks to a collaborative robot. The second hypothesis is that, compared to typical HRC frameworks involving physical interaction, the ergonomics and safety of an HRC scenario can be enhanced by combining gesture recognition and pose estimation. These remove unnecessary motions that could expose operators to ergonomic risks, decrease the amount of physical effort, and make the robot aware and responsive to the presence and gestures of the operators. The methodology is evaluated by optimizing the HRC scenario of a television manufacturing process, yet it is described how it can be reconfigured for other industrial scenarios. The effect of the temporal and spatial adaptation on the operator's range of motion was analyzed through three separate experiments. The effectiveness of HRC is measured through the standard key performance indicators (KPIs); however, to evaluate the collaboration and required physical demand, two KPIs are proposed in this paper. These are the rate of spatial adaptation and the rate of reduction in the operator's motion. The results demonstrated that the methodology enhanced the ergonomics and efficiency of the production process. First, the robot was delegated two tasks identified as the most ergonomically dangerous for human operators. Then the optimized HRC achieved an average rate of spatial adaptation of 29.37% and a decrease in operator movement of 28.8% across 14 subjects compared to HRC frameworks that do not include spatial and temporal adaptation.

To prevent work-related musculoskeletal disorders (WMSD) the ergonomists apply manual heuristic methods to determine when the worker is exposed to risk factors. However, these methods require an observer and the results can be subjective. This paper proposes a method to automatically evaluate the ergonomic risk factors when performing a set of postures from the ergonomic assessment worksheet (EAWS). Joint angle motion data have been recorded with a full-body motion capture system. These data modeled the motion patterns of four different risk factors, with the use of hidden Markov models (HMMs). Based on the EAWS, automated scores were assigned by the HMMs and were compared to the scores calculated manually. Because the method proposed here is intrusive and requires expensive equipment, kinematic data from a reduced set of two sensors was also evaluated.

The acquisition of gestural know-how in manual professions constitutes a real challenge since it passes from master to learner, through a many years long « in person » transmission. However this binding transmission is not always possible for practical reasons; the learner must train himself alone, by using traditional Knowledge Management tools such as e-documentation and multimedia contents. These tools present important limitations, only providing the learner expert knowledge in a descriptive way, with a low attractiveness and interaction level, without any sensorimotor feedback. It thus becomes crucial to find novel ways to preserve and transmit know-how. In this work we present the idea of a methodological framework for gestural know-how management in wheel throwing pottery, based on motion capture and gesture recognition technologies. In combination with machine learning techniques, they permit to model the practical, cinematic aspects of potter's expertise. These technologies can be used to compare experts' and learners' simulated performances and to provide real-time feedback to the learner, guiding him in the adjustment of his gestures. The final goal is to propose a novel and highly interactive embodied pedagogical application for gestural know-how transmission, supporting « self » trainings, and making them more efficient.

Voice, text or music recognition systems have quite a lot of elements in common with gesture recognition systems. They recognize "something" moving in orbit in space and in time. Hidden Markov Models (HMM) is an advisable mathematic theory managing stochastic processes such as human voice and gestures. This paper proposes a system retrieving music information from the right hand of a pianist using computer vision technology. Finger movement recognition is included in a general procedure, the gesture recognition. The results from a tentative analysis led to the use of a low cost video camera. The study on the pianist fingering retrieval has been effectuated on four aspects: preparation and capture of video signal, finger segmentation and tracking, feature vector exportation and HMM classifier. The recognition has not been effectuated in real time. Video signal processing (masks, filters, threshold etc) as well as the application of HMM was done with the use of suitable MATLAB toolboxes.

This paper presents the specifications of an open - source Learning Management System using ICT, which was implemented on a Greek High School. This effort began because of our interest to see the impact of the use of Multimedia and Internet resources having e-learning content, in a classroom. Greece lags far behind in use of Internet towards other European Countries. This paper also presents how pupils evaluated the use of this innovative pedagogical framework. Our research was extended to the new role that teachers called to adapt in teaching.