This paper introduces a framework for monocular 3D reconstruction of cheetah movements, leveraging a combination of data-driven and physics-based modeling as well as trajectory optimization. Unlike traditional methods that rely solely on kinematics, our approach integrates dynamic motion principles, enhancing the plausibility and generalization of motion estimates. Validated on the cheetah running dataset, AcinoSet, we achieve mean per-joint position errors of 78.8 mm and 72.5 mm, showcasing significant advancements over the existing model used in AcinoSet. By addressing the challenge of absent ground truth data, this work not only advances animal motion capture techniques but also informs the development of bio-inspired robotic systems, offering a robust solution for accurately capturing complex animal locomotion in natural settings.

Tracking the 3D motion of agile animals in the wild will enable new insight into the design of robotic controllers. However, in-field 3D pose estimation of high-speed wildlife such as cheetahs is still a challenge [1]. In this work, we aim to solve two of these challenges: unnatural pose estimates during highly occluded sequences and synchronization error between multi-view data. We expand on our previous Full Trajectory Estimation (FTE) method with two significant additions: Pairwise FTE (PW-FTE) and Shutter-delay FTE (SD-FTE). The PW-FTE expands on image-dependent pairwise terms, produced by a convolutional neural network (CNN), to infer occluded 2D keypoints, while SD-FTE uses shutter delay estimation to correct the synchronization error. Lastly, we combine both methods into PW-SD-FTE and perform a quantitative and qualitative analysis on a subset of AcinoSet, the video dataset of rapid and agile motions of cheetahs. We found that SD-FTE has significant benefits in tracking the position of the cheetah in the world frame, while PW-FTE provided a more robust 3D pose estimate during events of high occlusion. The PW-SD-FTE was found to retain both advantages, resulting in an improved baseline for AcinoSet. Code and data can be found at https://github.com/African-Robotics-Unit/AcinoSet/tree/pw_sd_fte.

Animals are capable of extreme agility, yet understanding their complex dynamics, which have ecological, biomechanical and evolutionary implications, remains challenging. Being able to study this incredible agility will be critical for the development of next-generation autonomous legged robots. In particular, the cheetah (acinonyx jubatus) is supremely fast and maneuverable, yet quantifying its wholebody 3D kinematic data during locomotion in the wild remains a challenge, even with new deep learning-based methods. In this work we present an extensive dataset of free-running cheetahs in the wild, called AcinoSet, that contains 119, 490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7, 588 human-annotated frames. We utilize markerless animal pose estimation to provide 2D keypoints. Then, we use three methods that serve as strong baselines for 3D pose estimation tool development: traditional sparse bundle adjustment, an Extended Kalman Filter, and a trajectory optimization-based method we call Full Trajectory Estimation. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided. We believe this dataset will be useful for a diverse range of fields such as ecology, neuroscience, robotics, biomechanics as well as computer vision. Code and data can be found at: https://github.com/African-Robotics-Unit/AcinoSet.

Graph cuts have proven useful for image segmentation and for volumetric reconstruction in multiple view stereo. However, solutions are biased: the cost function tends to favour either a short boundary (in 2D) or a boundary with a small area (in 3D). This bias can be avoided by instead minimising the cut ratio, which normalises the cost by a measure of the boundary size. This paper uses ideas from discrete differential geometry to develop a linear programming formulation for finding a minimum ratio cut in arbitrary dimension, which allows constraints on the solution to be specified in a natural manner, and which admits an efficient and globally optimal solution. Results are shown for 2D segmentation and for 3D volumetric reconstruction.

The uniformly most powerful invariant (UMPI) test is derived for detecting a target with unknown location in a noise sequence. This test has the property that for each possible target location it has the greatest power of all tests which are invariant to cyclic permutations of the observations. The test is compared to the generalised likelihood ratio test (GLRT), which is commonly used as a solution to this detection problem. Monte-Carlo simulations show that the powers of the two tests are comparable, thereby justifying near-optimality of the GLRT.

The EM algorithm is a commonly cited solution in the literature for the problem of maximum likelihood estimation of covariance matrices under a Toeplitz constraint. In this paper, the solution is extended to the case of two-dimensional signals, where spatial stationarity enforces a Toeplitz-block-Toeplitz structure on the covariance matrix. A further generalisation which is presented involves the estimation of the covariance when the observations are subject to subspace interference. It is shown that this situation is amenable to a missing data interpretation, and can be incorporated into the EM iteration with moderate ease. The solution shares all the characteristics of the I-D Toeplitz estimate. The need to solve this problem arises in many invariance applications, where it is required to fit a stationary multivariate normal model to data which is subject to a certain type of interference. The case of unknown DC offset is included in this class.