Vision-based methods for gesture analysis in sports can provide valuable insights to players and coaches, without any disturbing body-worn sensors. In table tennis, accurately predicting the trajectories of the ball and its 3D kinematics is crucial for assessing the effectiveness of a stroke. While several computer vision methods have investigated this area, they often focus on trajectory prediction solely, without considering physical parameters. Additionally, these studies are typically limited to controlled laboratory settings, which can be invasive and require extensive data collection. In this paper, we present a novel vision-based approach to 3D ball trajectory analysis. Our proposed method relies on Physics-Informed Learning to predict the 3D trajectory of a table tennis ball and to infer its 3D kinematics parameters using a small amount of data acquired from monocular video sequences. To the best of our knowledge, this is the first attempt to integrate data-driven approaches with physical constraints in order to predict the 3D trajectory and kinematics parameters of a table tennis ball.

Human behaviour sensing and understanding with the analysis of human intentions is a mandatory part of many assistive and healthcare IT scenarios. In the present work we propose an integrative approach for understanding human intentions and its context for vision-assisted control of upper limb prostheses. In particular, we focus on the estimation of the 6D pose, i.e. the spatial coordinates and the rotation angles around the axes of the egocentric system of first-person-view camera mounted on the glasses. We propose a solution based on the DenseFusion backbone, which is applicable to a real-world prosthesis-assisting scenario due to its fairly good accuracy. Indeed, the error in the object position measured as the mean error of object model points with a fine-tuned model on a controllable synthetic dataset is of 0.8 cm, which is an acceptable range for Robotic API in our scenario of assisting amputees wearing upper limb prostheses.

Vision-based methods for gesture analysis in sports can provide valuable insights to players and coaches, without any disturbing body-worn sensors. In table tennis, accurately predicting the trajectories of the ball and its 3D kinematics is crucial for assessing the effectiveness of a stroke. While several computer vision methods have investigated this area, they often focus on trajectory prediction solely, without considering physical parameters. Additionally, these studies are typically limited to controlled laboratory settings, which can be invasive and require extensive data collection. In this paper, we present a novel visionbased approach to 3D ball trajectory analysis. Our proposed method relies on Physics-Informed Learning to predict the 3D trajectory of a table tennis ball and to infer its 3D kinematics parameters using a small amount of data acquired from video sequences. To the best of our knowledge, this is the first attempt to integrate data-driven approaches with physical constraints in order to predict the 3D trajectory and kinematics parameters of a table tennis ball.

The context of this research is the use of computer vision to assess the quality of sport gestures in non-intrusive conditions, i.e. without any body-worn sensors. This paper addresses the estimation of Table Tennis ball kinematic parameters from single-view videos. These parameters are important for analyzing effects given on the ball by the players, a key factor in the Table Tennis game. We introduce 3D ball trajectories extraction and analysis with very few acquisition constraints. To obtain ball to camera distance, the estimation of the apparent ball size is performed with a 2D CNN trained on a generated dataset. By formulating the problem of trajectory estimation as the solution of an Ordinary Differential Equation (ODE) with initial conditions, we can extract the ball kinematic parameters such as tangential and rotation speeds. Validation experiments are presented on both a synthetic dataset and on real video sequences.

Vision is the first indicator for coaches to assess the quality of a sport gesture. However, gesture analysis using computer vision is often restricted to laboratory experiments, far from the real conditions in which athletes train on a daily basis. In this perspective, we introduce 3D ball trajectories analysis using a single camera with very few acquisition constraints. A key point of the proposal is the estimation of the apparent ball size for obtaining ball to camera distance. For this purpose, a 2D CNN is trained using a generated dataset that enables a reliable ball size extraction, even in case of high motion blur. The final objective is not only to be able to determine ball trajectories, but most importantly to retrieve their relevant physical parameters. Those kinematics parameters could help coaches and players to better analyze strokes with strong rotation effects during training sessions or matches. With a precise estimation of those trajectories, it is indeed possible to extract the ball tangential and rotation speed, related to the so-called Magnus effect. Validation experiments for characterizing table tennis strokes are presented on both a synthetic dataset and on real video sequences.

The paper addresses the problem of recognition of actions in video with low inter-class variability such as Table Tennis strokes. Two stream, “twin” convolutional neural networks are used with 3D convolutions both on RGB data and optical flow. Actions are recognized by classification of temporal windows. We introduce 3D attention modules and examine their impact on classification efficiency. In the context of the study of sportsmen performances, a corpus of the particular actions of table tennis strokes is considered. The use of attention blocks in the network speeds up the training step and improves the classification scores up to 5% with our twin model. We visualize the impact on the obtained features and notice correlation between attention and player movements and position. Score comparison of state-of-the-art action classification method and proposed approach with attentional blocks is performed on the corpus. Proposed model with attention blocks outperforms previous model without them and our baseline.

Human action recognition in videos is one of the key problems in visual data interpretation. Despite intensive research, the recognition of actions with low inter-class variability remains a challenge. To answer this problem, my thesis focus on fine-grained classification challenge using a Siamese Spatio-Temporal Convolutional Neural Network and apply it to a new dataset we have introduced TTStroke-21. Our model take as input data RGB images and Optical Flow and is able to reach an accuracy of 91.4% against 43.1% for our baseline on temporal segmented videos. Detection and classification in videos using a sliding temporal window leads to a score of 81.3% over the whole dataset.

Detecting and classifying human actions in videos is one of the current challenges in visual content analysis and mining. This paper presents a method for performing a finegrained classification of sport actions using a Siamese SpatioTemporal Convolutional Neural Network (SSTCNN) model. This model takes RGB images and Optical Flow field as input data. Our first contribution is the comparison of different Optical flow methods and a study of their influence on the classification score. We also present different normalization methods for the optical flow that drastically impact results, boosting performances from 44% to 74% of accuracy. Our second contribution is the detection and classification of actions in videos performed using a sliding temporal window. It leads to a satisfying score of 81.3% over the whole dataset TTStroke-21.

Human action recognition in video is one of the key problems in visual data interpretation. Despite intensive research, the recognition of actions with low inter-class variability remains a challenge. This paper presents a new Siamese Spatio-Temporal Convolutional neural network (SSTC) for this purpose. When applied to table tennis, it is possible to detect and recognize 20 table tennis strokes. The model has been trained on a specific dataset, TTStroke-21, recorded in natural condition (markerless) at the Faculty of Sports of the University of Bordeaux. Our model takes as inputs a RGB image sequence and its computed Optical Flow. After 3 spatio-temporal convolutions, data are fused in a fully connected layer of a proposed siamese network architecture. Our method reaches an accuracy of 91.4% against 43.1% for our baseline.

This paper presents an original approach for recognizing human activities in video sequences. A human activity is seen as a temporal sequence of elementary action probabilities. Actions are first generically learned using a robust action recognition method based on optical flow estimation and a cross-dataset training process. Activities are then projected as trajectories on the semantic simplex in order to be characterized and discriminated. A new trajectory attribute based on the total curvature Fourier descriptor is introduced. This attribute takes into account the induced geometry of the simplex manifold. Experiments on labelled datasets of human activities prove the efficiency of the proposed method for discriminating complex actions.

This paper focuses on human action recognition in video sequences. A method based on the optical flow estimation is presented, where critical points of the flow field are extracted. Multi-scale trajectories are generated from those points and are characterized in the frequency domain. Finally, a sequence is described by fusing this frequency information with motion orientation and shape information. Experiments show that this method has recognition rates among the highest in the state of the art on the KTH dataset. Contrary to recent dense sampling strategies, the proposed method only requires critical points of motion flow field, thus permitting a lower computation time and a better sequence description. Results and perspectives are then discussed.

The research context of this paper is dynamic texture analysis and characterization. Many dynamic textures can be modeled as large scale propagating wavefronts and local oscillating phenomena. After introducing a formal model for dynamic textures, the morphological component analysis (MCA) approach with a well-chosen dictionary is used to retrieve the components of dynamic textures. We define two new strategies for adaptive thresholding in the MCA framework, which greatly reduce the computation time when applied on videos. Tests on real image sequences illustrate the efficiency of the proposed method. An application to global motion estimation is proposed and future prospects are finally exposed.

The research context of this work is dynamic texture analysis and characterization. Many dynamic textures can be modeled as a large scale propagating wave and local oscillating phenomena. The Morphological Component Analysis algorithm is used to retrieve these components using a well chosen dictionary. We define a new strategy for adaptive thresholding in the Morphological Component Analysis framework, which greatly reduces the computation time when applied on videos. Tests on synthetic and real image sequences illustrate the efficiency of the proposed method and future prospects are finally exposed.

We present the DynTex database of high-quality dynamic texture videos. It consists of over 650 sequences of dynamic textures, mostly in everyday surroundings. Additionally, we propose a scheme for the manual annotation of the sequences based on a detailed analysis of the physical processes underlying the dynamic textures. Using this scheme we describe the texture sequences in terms of both visual structure and semantic content. The videos and annotations are made publicly available for scientific research.

This paper presents a new approach for segmenting a video sequence containing dynamic textures. The proposed method is based on a 2D+T curvelet transform and an octree hierarchical representation. The curvelet transform enables to outline spatio-temporal structures of a given scale and orientation. The octree structure based on motion coherence enables a better spatio-temporal segmentation than a direct application of the 2D+T curvelet transform. Our segmentation method is successfully applied on video sequences of dynamic textures. Future prospects are finally exposed.

This article addresses the problem of road extraction from new high resolution satellite images. The proposed algorithm is divided in two sequential modules : a topologically correct graph of the road network is first extracted, and roads are then extracted as surface elements. The graph of the network is extracted by a following algorithm which minimizes a cost function. The extraction algorithm makes use of specific active contours (snakes) combined with a multiresolution analysis (MRA) for minimizing the problem of geometric noise. This reconstruction phase is composed of two steps : the extraction of road segments and the extraction of road intersections. Results of the road network extraction are presented in order to illustrate the different steps of the method and future prospects are exposed.

International audience