Multi-object tracking (MOT) is a critical task in computer vision, requiring the accurate identification and continuous tracking of multiple objects across video frames. However, current state-of-the-art methods mainly rely on a global optimization technique and multi-stage cascade association strategy, and those approaches often overlook the specific characteristics of assignment task in MOT and useful detection results that may represent occluded objects. To address these challenges, we propose a novel Track-Focused Online Multi-Object Tracker (TrackTrack) with two key strategies: Track-Perspective-Based Association (TPA) and Track-Aware Initialization (TAI). The TPA strategy associates each track with the most suitable detection result by choosing the one with the minimum distance from all available detection results in a track-perspective manner. On the other hand, TAI precludes the generation of spurious tracks in the track-aware aspect by suppressing track initialization of detection results that heavily overlap with current active tracks and more confident detection results. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate that our TrackTrack outperforms current state-of-the-art trackers, offering improved robustness and accuracy across diverse and challenging tracking scenarios.

Multi-object tracking (MOT), a crucial task in computer vision, has broad applicability, and recently, tracking-by-detection-based trackers, which separate the processes of object detection and association, are showing state-of-the-art performance. However, while techniques like feature enhancement and distance measures have been extensively explored, the matching strategy itself remains an area that requires more in-depth study. As a result, many trackers still require manual adjustment of sensitive hyper-parameters for each tracking scenario, limiting their adaptability and robustness in dynamic environments. To address these limitations, we introduce CMTrack, a new tracker featuring a novel confidence-aware matching strategy comprised of three modules: confidence-aware cascade matching (CCM), confidence-aware metric fusion (CMF), and confidence-aware feature update (CFU). Our matching strategy enables the tracker to be a generalized and practical solution for various tracking scenarios within a unified framework while obviating manual calibration of hyper-parameters. The effectiveness of CMTrack is demonstrated through comprehensive assessments of three prominent MOT datasets: MOT17, MOT20, and DanceTrack. Notably, our CMTrack consistently surpasses existing state-of-the-art trackers, showcasing its superior generalization capabilities. The source codes and models are open at https://github.com/kamkyu94/CMTrack.

Multi-object tracking (MOT) plays a pivotal role in various computer vision domains with recent tracking-by detection algorithms that treat MOT as distinct detection and association tasks. However, the existing trackers often rely on sensitive thresholds to associate previous tracks and current detection results while forming complete trajectories across a video. These thresholds are crucial for tracking performance and require manual tuning for each dataset or even sequence, limiting the adaptability in real-world applications. To tackle this problem, in this paper, we introduce AdapTrack, a novel MOT algorithm designed to enable adaption on varying scenarios without handcrafted threshold configuration. With a carefully designed matching strategy, our tracker can adaptively select proper thresholds for each frame and correctly associate detected objects. Consequently, AdapTrack shows outperforming results on standard MOT benchmarks, MOT17 and MOT20, compared to existing state-of-the-art methods. Every source code is available at https://github.com/kamkyu94/AdapTrack.

Despite its recent advancements, multi-object tracking (MOT), one of the major research areas in video technology, still faces various challenges, including severe occlusion and diversity of tracking targets. In this paper, we introduce a novel strategy, Temporal Feature Mix (TFM), that can improve the overall robustness of multi-object trackers in diverse scenarios. More specifically, our approach simulates new and challenging scenes that can train networks to better localize the targets by blending high-level features from temporally adjacent frames with the insights that the high-level features are mainly activated on salient targets and the targets on the adjacent frames are nearly located. Therefore, our TFM can offer novel and diversified training experiences to the networks, achieved through the intensive augmentation of the high-level features of each target. As a result, our approach demonstrates notable performance improvement with three major MOT benchmarks and a newly constructed corruption dataset for MOT, underscoring its potential to enhance the robustness of MOT systems in real-world scenarios. Every related source code is released at https://github.com/kamkyu94/Temporal Feature Mix.

Deep neural networks are vulnerable to maliciously crafted inputs called adversarial examples. Research on unprecedented adversarial attacks is significant since it can help strengthen the reliability of neural networks by alarming potential threats against them. However, since existing adversarial attacks disturb models unconditionally, the resulting adversarial examples increase their detectability through statistical observations or human inspection. To tackle this limitation, we propose hidden conditional adversarial attacks whose resultant adversarial examples disturb models only if the input images satisfy attackers’ pre-defined conditions. These hidden conditional adversarial examples have better stealthiness and controllability of their attack ability. Our experimental results on the CIFAR-10 and ImageNet datasets show their effectiveness and raise a serious concern about the vulnerability of CNNs against the novel attacks.

Siamese-based trackers have recently demonstrated impressive performance and high speed. Despite their great success, conventional siamese trackers are prone to be fooled when facing appearance variations of target objects because they refer to fixed templates captured from first frames to track target objects in the rest of videos. To address this issue, we propose a novel siamese-based tracking framework utilizing a dual template which consists of a static template and a dynamic template. The dynamic template is updated every update interval and allows the tracker to catch appearance variations of the target over time. Furthermore, we introduce a reliability score which prevents incorrect dynamic templates from degrading tracking performance to ensure reliable dynamic template updates. Experimental results show that our method possesses better discriminability and robustness than the baseline, which utilizes a single static template.

In order to make Visual Question Answering (VQA) explainable, previous studies not only visualize the attended region of a VQA model, but also generate textual explanations for its answers. However, when the model’s answer is “no,” existing methods have difficulty in revealing detailed arguments that lead to that answer. In addition, previous methods are insufficient to provide logical bases when the question requires common sense to answer. In this paper, we propose a novel textual explanation method to overcome the aforementioned limitations. First, we extract keywords that are essential to infer an answer from a question. Second, we utilize a novel Variable-Constrained Beam Search (VCBS) algorithm to generate explanations that best describe the circumstances in images. Furthermore, if the answer to the question is “yes” or “no,” we apply Natural Langauge Inference (NLI) to determine if contents of the question can be inferred from the explanation using common sense. Our user study, conducted in Amazon Mechanical Turk (MTurk), shows that our proposed method generates more reliable explanations compared to the previous methods. Moreover, by modifying the VQA model’s answer through the output of the NLI model, we show that VQA performance increases by 1.1% from the original model.

Multiple object tracking (MOT), a popular subject in computer vision with broad application areas, aims to detect and track multiple objects across an input video. However, recent learning-based MOT methods require strong supervision on both the bounding box and the ID of each object for every frame used during training, which induces a heightened cost for obtaining labeled data. In this paper, we propose a weakly-supervised MOT framework that enables the accurate tracking of multiple objects while being trained without object ID ground truth labels. Our model is trained only with the bounding box information with a novel masked warping loss that drives the network to indirectly learn how to track objects through a video. Specifically, valid object center points in the current frame are warped with the predicted offset vector and enforced to be equal to the valid object center points in the previous frame. With this approach, we obtain an MOT accuracy on par with those of the state-of-the-art fully supervised MOT models, which use both the bounding boxes and object ID as ground truth labels, on the MOT17 dataset.

Crowd counting is one of the most important tasks in visual surveillance applications since it provides useful information such as the number of crowds and their distribution. However, it is very challenging due to severe occlusions, large geometrical deformations, and high visual clutter. To tackle this problem, we propose a novel CNN-based crowd density estimation network consisting of a backbone, decoder, and mapper, and also a multi-step quantization scheme to train the network more effectively. As a backbone network, ResNet is adopted, then the decoder and mapper are added to deal with multi-scale problems of crowd counting and to generate high-resolution density maps. Finally, a multi-step quantization scheme discretizes the continuous space of both predictions and ground truth density maps, and it reduces the search scope of the network and raises their matching ratio. As a result, our method outperforms recent methods in four major datasets.