In this paper, we propose a novel structure for a multi-modal data association referred to as Associative Variational Auto-Encoder (AVAE). In contrast to the existing models using a shared latent space among modalities, our structure adopts distributed latent spaces for multi-modalities which are connected through cross-modal associators. The proposed structure successfully associates even heterogeneous modality data and easily incorporates the additional modality to the entire network via the associator. Furthermore, in our structure, only a small amount of supervised (paired) data is enough to train associators after training auto-encoders in an unsupervised manner. Through experiments, the effectiveness of the proposed structure is validated on various datasets including visual and auditory data.

We propose an online tracking algorithm for multiple target tracking with multiple cameras. In this paper, we suggest a multiple hypothesis tracking (MHT) framework to find an unknown number of multiple tracks through the spatio-temporal association between tracklets generated from multiple cameras. In this framework, the MHT is realized online by solving the maximum weighted clique problem (MWCP) at every frame to estimate the 3D trajectories of the targets. To handle the NP-hard issue of the MWCP, we propose a novel online scheme that formulates the MWCP using feedback information from the previous frame's result to find optimal tracks at every frame. This scheme enables the MWCP to be formulated by multiple subproblems and will significantly reduce the computation. The experiments show that the proposed algorithm performs comparably with the state-of-the-art batch algorithms, even though it adopts an online scheme.

This paper proposes a novel multi-target tracking method which jointly solves a data association problem using images from multiple cameras. In this work, the spatiotemporal data association problem is formulated as a multidimensional assignment problem (MDA). To achieve a fast, efficient, and easily implementable approximation algorithm, we solve the MDA problem approximately by solving a sequence of bipartite matching problems using random splitting and merging operations. In this formulation, we design a new cost function, considering the accuracy in 3D reconstruction, motion smoothness, visibility from cameras, starting/ending at entrance and exit zone, and false positive. Our approach reconstructs 3D trajectories that represent people’s movement as 3D cylinders whose locations are estimated considering all adjacent frames. The experiments illustrate the proposed method shows the state-of-the-art performance in challenging multi-camera datasets and the computational efficiency with 8 times faster computation than the existing BIP approach.

In this paper, we aim to find the matches of head detections from different cameras. Matching of heads detected in different cameras and assigning the same label to the detection results which are actually from the same person can help to track the person robustly in severe occlusion because matching process can give more clues when the tracked person is even totally occluded in one camera. We present two similarity measurement methods for head matching based on geometry information: homography similarity and epipolar similarity. Each similarity is based on homography and fundamental matrix respectively. Experimental results show that our similarity measures have validity to use for matching head detections. We also compare two similarities and analyze what strong point of each method is.