Sports analytics has been revolutionized by advanced tracking technologies, yet the integration of human pose estimation into performance metrics remains underexplored in football. Estimating the probability of scoring from a shot is a central task in football analytics and is commonly approached through expected goals (xG) models. Progress in this area, however, is often constrained by the limited availability of publicly accessible datasets that include fine-grained biomechanical information. In this work, we present xGHub, an open-source dataset of football shots enriched with player pose estimation, body orientation, and contextual features extracted from broadcast video. The dataset is generated using an automated pipeline for player detection and tracking, followed by an external verification process to ensure annotation reliability. As a use case, we analyze how pose- and orientation-related features can be incorporated into a standard xG modeling framework. Our results indicate that 3D orientation information is informative for specific subsets of shots, while its contribution is limited in others, reflecting the inherently non-linear nature of angular representations. This analysis serves to illustrate the potential and limitations of the released annotations. By making this dataset publicly available, we aim to support future research on the role of player biomechanics in shot analysis and related football analytics tasks.Sports analytics has been revolutionized by advanced tracking technologies, yet the integration of human pose estimation into performance metrics remains underexplored in football. Estimating the probability of scoring from a shot is a central task in football analytics and is commonly approached through expected goals (xG) models. Progress in this area, however, is often constrained by the limited availability of publicly accessible datasets that include fine-grained biomechanical information. In this work, we present xGHub, an open-source dataset of football shots enriched with player pose estimation, body orientation, and contextual features extracted from broadcast video. The dataset is generated using an automated pipeline for player detection and tracking, followed by an external verification process to ensure annotation reliability. As a use case, we analyze how pose- and orientation-related features can be incorporated into a standard xG modeling framework. Our results indicate that 3D orientation information is informative for specific subsets of shots, while its contribution is limited in others, reflecting the inherently non-linear nature of angular representations. This analysis serves to illustrate the potential and limitations of the released annotations. By making this dataset publicly available, we aim to support future research on the role of player biomechanics in shot analysis and related football analytics tasks.

Abstract Predictive modeling plays a crucial role in machine learning, data analysis, and statistics. In sports, predictive modeling methods have emerged to provide insights and evaluate performances based on key performance metrics. However, most existing models tend to focus on predicting only partial aspects of an event, such as the outcome, action type, or location, while neglecting the temporal factors involved. To address this gap, this study introduces the Transformer-Based Neural Marked Spatio-Temporal Point Process (NMSTPP) model, specifically designed for football event data. The NMSTPP model predicts a comprehensive set of future event components, including inter-event time, zone, and action. Additionally, it features a dependent prediction layers architecture to enhance model performance. The Holistic Possession Utilization Score (HPUS) metric is also proposed to evaluate the effectiveness and efficiency of possession periods in football based on the NMSTPP model. With open-source football event data, the NMSTPP model successfully predicted the aforementioned three components of future events, with an improvement of up to 4% overall and 9% for individual components compared to baseline models. The HPUS demonstrated a 0.9 correlation with existing performance metrics, highlighting its utility in performance evaluation. The NMSTPP and HPUS were applied to the Premier League to demonstrate their practical feasibility.

Evaluation of intervention in a multiagent system, for example, when humans should intervene in autonomous driving systems and when a player should pass to teammates for a good shot, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multiagent relationships and covariate counterfactual prediction. This may lead to erroneous assessments of ITE and difficulty in interpretation. Here, we propose an interpretable, counterfactual recurrent network in multiagent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks (GVRNNs) and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multiagent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle and biological agents with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates and the most effective treatment timing than the baselines. Furthermore, using real basketball data, our methods performed realistic counterfactual predictions and evaluated the counterfactual passes in shot scenarios.

This study aims to enhance the training efficiency of guide dogs by employing computer vision to collect training data and analyze the movements of both trainers and dogs. This task is challenging, owing to the constant movement of cameras and unstable reference points for camera calibration, which stem from the complexities of the guide dog training process and the surrounding environment. In addition, trainers and dogs walk side by side, making it difficult for 2D videos to capture complete interactions without obstacles. We present a comprehensive system that starts with 2D video footage from multicamera setups, proceeds to extrinsic camera calibration from the moving trainer's joints, and reconstructs the 3D poses of guide dogs and trainers. This process includes human 2D/3D pose estimation, camera calibration, and dog 2D/3D pose estimation. A novel aspect of the proposed approach involves modifying the existing calibration method for multiple cameras. This modification is designed to achieve extrinsic camera calibration and accommodating complex camera settings in real-world situations, including both fixed and moving multicamera setups without calibration objects. We can create a 3D representation of the training sessions by detecting the trainer's 2D and 3D skeletons and using calibrated cameras to triangulate the dog's 3D pose. This allows for a detailed analysis and adjustment of guide dog training methods based on the 3D pose data of trainers and guide dogs, thereby improving the overall training process.

Abstract The application of visual tracking to the performance analysis of sports players in dynamic competitions is vital for effective coaching. In doubles matches, coordinated positioning is crucial for maintaining control of the court and minimizing opponents’ scoring opportunities. The analysis of such teamwork plays a vital role in understanding the dynamics of the game. However, previous studies have primarily focused on analyzing and assessing singles players without considering occlusion in broadcast videos. These studies have relied on discrete representations, which involve the analysis and representation of specific actions (e.g., strokes) or events that occur during the game while overlooking the meaningful spatial distribution. In this work, we present the first annotated drone dataset from top and back views in badminton doubles and propose a framework to estimate the control area probability map, which can be used to evaluate teamwork performance. We present an efficient framework of deep neural networks that enables the calculation of full probability surfaces. This framework utilizes the embedding of a Gaussian mixture map of players’ positions and employs graph convolution on their poses. In the experiment, we verify our approach by comparing various baselines and discovering the correlations between the score and control area. Additionally, we propose a practical application for assessing optimal positioning to provide instructions during a game. Our approach offers both visual and quantitative evaluations of players’ movements, thereby providing valuable insights into doubles teamwork. The dataset and related project code is available at https://github.com/Ning-D/Drone_BD_ControlArea

Evaluation of intervention in a multi-agent system, e.g., when humans should intervene in autonomous driving systems, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multi-agent relationships and covariate counterfactual prediction. Here we propose an interpretable, counterfactual recurrent network in multi-agent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multi-agent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates.

Recent advances in reinforcement learning (RL) have made it possible to develop sophisticated agents that excel in a wide range of applications. Simulations using such agents can provide valuable information in scenarios that are difficult to scientifically experiment in the real world. In this paper, we examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players. We explore what can be learnt from the use of simulated environments by using aggregated statistics and social network analysis (SNA). As a result, we found that (1) there are strong correlations between the competitiveness of an agent and various SNA metrics and (2) aspects of the RL agents play style become similar to real world footballers as the agent becomes more competitive. We discuss further advances that may be necessary to improve our understanding necessary to fully utilise RL for the analysis of football.

Trajectory analysis has been a central problem in applications of location tracking systems. Recently, the (discrete) Fréchet distance becomes a popular approach for measuring the similarity of two trajectories because of its high feature extraction capability. Despite its importance, the Fréchet distance has several limitations: (i) sensitive to noise as a trade-off for its high feature extraction capability; and (ii) it cannot be incorporated into machine learning frameworks due to its non-smooth functions. To address these problems, we propose the Fréchet kernel (FRK), which is associated with a smoothed Fréchet distance using a combination of two approximation techniques. FRK can adaptively acquire appropriate extraction capability from trajectories while retaining robustness to noise. Theoretically, we find that FRK has a positive definite property, hence FRK can be incorporated into the kernel method. We also provide an efficient algorithm to calculate FRK. Experimentally, FRK outperforms other methods, including other kernel methods and neural networks, in various noisy real-data classification tasks.

Extracting the interaction rules of biological agents from movement sequences pose challenges in various domains. Granger causality is a practical framework for analyzing the interactions from observed time-series data; however, this framework ignores the structures and assumptions of the generative process in animal behaviors, which may lead to interpretational problems and sometimes erroneous assessments of causality. In this paper, we propose a new framework for learning Granger causality from multi-animal trajectories via augmented theory-based behavioral models with interpretable data-driven models. We adopt an approach for augmenting incomplete multi-agent behavioral models described by time-varying dynamical systems with neural networks. For efficient and interpretable learning, our model leverages theory-based architectures separating navigation and motion processes, and the theory-guided regularization for reliable behavioral modeling. This can provide interpretable signs of Granger-causal effects over time, i.e., when specific others cause the approach or separation. In experiments using synthetic datasets, our method achieved better performance than various baselines. We then analyzed multi-animal datasets of mice, flies, birds, and bats, which verified our method and obtained novel biological insights.

Massive datasets of spatial trajectories representing the mobility of a diversity of moving objects are ubiquitous in research and industry. Similarity search of a large collection of trajectories is indispensable for turning these datasets into knowledge. Locality sensitive hashing (LSH) is a powerful technique for fast similarity searches. Recent methods employ LSH and attempt to realize an efficient similarity search of trajectories; however, those methods are inefficient in terms of search time and memory when applied to massive datasets. To address this problem, we present the trajectory-indexing succinct trit-array trie (tSTAT), which is a scalable method leveraging LSH for trajectory similarity searches. tSTAT quickly performs the search on a tree data structure called trie. We also present two novel techniques that enable to dramatically enhance the memory efficiency of tSTAT. One is a node reduction technique that substantially omits redundant trie nodes while maintaining the time performance. The other is a space-efficient representation that leverages the idea behind succinct data structures (i.e., a compressed data structure supporting fast data operations). We experimentally test tSTAT on its ability to retrieve similar trajectories for a query from large collections of trajectories and show that tSTAT performs superiorly in comparison to state-of-the-art similarity search methods.

Accurate extraction of foregrounds in videos is one of the challenging problems in computer vision. In this study, we propose dynamic mode decomposition via dictionary learning (dl-DMD), which is applied to extract moving objects by separating the sequence of video frames into foreground and background information with a dictionary learned using block patches on the video frames. Dynamic mode decomposition (DMD) decomposes spatiotemporal data into spatial modes, each of whose temporal behavior is characterized by a single frequency and growth/decay rate and is applicable to split a video into foregrounds and the background when applying it to a video. And, in dl-DMD, DMD is applied on coefficient matrices estimated over a learned dictionary, which enables accurate estimation of dynamical information in videos. Due to this scheme, dl-DMD can analyze the dynamics of respective regions in a video based on estimated amplitudes and temporal evolution over patches. The results on synthetic data exhibit that dl-DMD outperforms the standard DMD and compressed DMD (cDMD) based methods. Also, the results of an empirical performance evaluation in the case of foreground extraction from videos using publicly available dataset demonstrates the effectiveness of the proposed dl-DMD algorithm and achieves a performance that is comparable to that of the state-of-the-art techniques in foreground extraction tasks.

Understanding dynamical systems by extracting spatiotemporal patterns from data is fundamental in a variety of fields of engineering and science. Dynamic mode decomposition (DMD) has recently attracted attention in these fields as a way of obtaining a global modal description of a nonlinear dynamical system from data, without requiring explicit prior knowledge. However, DMD is in principle an unsupervised dimensionality reduction algorithm; it is not endowed with the mechanism to utilize label information even if a set of data with different labels is given. In this paper, we propose the algorithm that incorporates label information into DMD via multitask learning by solving sparse-group Lasso. To this end, we estimate sparse weights over dynamic modes in a label-wise manner by regarding data with different labels as different tasks. Modal descriptions estimated by this approach share a part of the global modes, resulting in the extraction of label-specific and common (or mixed) dynamical structures, which could be useful in understanding mechanisms in the spatiotemporal behavior behind data. We investigate the empirical performance using synthetic and real-world datasets, and validate that our algorithm can extract and visualize common and label-specific spatiotemporal structures.

The development of a metric for structural data is a long-term problem in pattern recognition and machine learning. In this paper, we develop a general metric for comparing nonlinear dynamical systems that is defined with Perron-Frobenius operators in reproducing kernel Hilbert spaces. Our metric includes the existing fundamental metrics for dynamical systems, which are basically defined with principal angles between some appropriately-chosen subspaces, as its special cases. We also describe the estimation of our metric from finite data. We empirically illustrate our metric with an example of rotation dynamics in a unit disk in a complex plane, and evaluate the performance with real-world time-series data.

Modeling the complex collective behavior is a challenging issue in several material and life sciences. The collective motion has been usually modeled by simple interaction rules and explained by global statistics. However, it remains difficult to bridge the gap between the dynamic properties of the complex interaction and the emerging group-level functions. Here we introduce decomposition methods to directly extract and classify the latent global dynamics of nonlinear dynamical systems in an equation-free manner, even including complex interaction in few data dimensions. We first verified that the basic decomposition method can extract and discriminate the dynamics of a well-known rule-based fish-schooling (or bird-flocking) model. The method extracted different temporal frequency modes with spatial interaction coherence among three distinct emergent motions, whereas these wave properties in multiple spatiotemporal scales showed similar dispersion relations. Second, we extended the basic method to map high-dimensional feature space for application to actual small-dimensional systems complexly changing the interaction rules. Using group sports human data, we classified the dynamics and predicted the group objective achievement. Our methods have a potential for classifying collective motions in various domains which obey in non-trivial dominance law known as active matters.