Multi-camera Multiple People Tracking (MMPT) is a challenging task in advanced visual monitoring systems. The main challenge of MMPT is how to accurately match the single-camera trajectories generated from different viewpoints and establish one global and complete cross-camera trajectory for each target, i.e., the multi-camera trajectory matching problem. In this paper, we propose a novel framework to solve this problem using a scene-aware multiple object tracking. Furthermore, unlike most existing methods that purely use single-camera trajectories for multiple object tracking, we introduce a new multiple camera compensation mechanism 2D-to-3D Mutual Iterative Optimization for MMPT (MIO-MMPT) to enhance the person tracking results, which exploits the crucial multi-camera relationships among the human trajectories appearing in different cameras both robustly and automatically. Based on the camera calibration, we can project the 2D coordinate into 3D coordinate to achieve more reliable tracking results for each person. Once we have the 3D tracking results, we can re-project to 2D coordinate of each camera to solve the missing detection issues from occlusion or the blind spot of the camera. According to our experimental results, the proposed method achieves a new state-of-the-art on ICCV 2021 MMPT dataset with MOTA of 95% and IDF1 of 96%.

Few-shot audio classification is an emerging topic that attracts more and more attention from the research community. Most existing work ignores the specificity of the form of the audio spectrogram and focuses largely on the embedding space borrowed from image tasks, while in this work, we aim to take advantage of this special audio format and propose a new method by hallucinating high-frequency and low-frequency parts as structured concepts. Extensive experiments on ESC50 and our curated balanced Kaggle18 dataset show the proposed method outperforms the baseline by a notable margin. The way that our method hallucinates high-frequency and low-frequency parts also enables its interpretability and opens up new potentials for the few-shot audio classification.

In the rapidly evolving landscape of smart home automation, the potential of IoT devices is vast. In this realm, rules are the main tool utilized for this automation, which are predefined conditions or triggers that establish connections between devices, enabling seamless automation of specific processes. However, one significant challenge researchers face is the lack of comprehensive datasets to explore and advance the field of smart home rule recommendations. These datasets are essential for developing and evaluating intelligent algorithms that can effectively recommend rules for automating processes while preserving the privacy of the users, as it involves personal information about users' daily lives. To bridge this gap, we present the Wyze Rule Dataset, a large-scale dataset designed specifically for smart home rule recommendation research. Wyze Rule encompasses over 1 million rules gathered from a diverse user base of 300,000 individuals from Wyze Labs, offering an extensive and varied collection of real-world data. With a focus on federated learning, our dataset is tailored to address the unique challenges of a cross-device federated learning setting in the recommendation domain, featuring a large-scale number of clients with widely heterogeneous data. To establish a benchmark for comparison and evaluation, we have meticulously implemented multiple baselines in both centralized and federated settings. Researchers can leverage these baselines to gauge the performance and effectiveness of their rule recommendation systems, driving advancements in the domain. The Wyze Rule Dataset is publicly accessible through HuggingFace's dataset API.

The successful application of deep learning to many visual recognition tasks relies heavily on the availability of a large amount of labeled data which is usually expensive to obtain. The few-shot learning problem has attracted increasing attention from researchers for building a robust model upon only a few labeled samples. Most existing works tackle this problem under the meta-learning framework by mimicking the few-shot learning task with an episodic training strategy. In this paper, we propose a new transfer-learning framework for semi-supervised few-shot learning to fully utilize the auxiliary information from labeled base-class data and unlabeled novel-class data. The framework consists of three components: 1) pre-training a feature extractor on base-class data; 2) using the feature extractor to initialize the classifier weights for the novel classes; and 3) further updating the model with a semi-supervised learning method. Under the proposed framework, we develop a novel method for semi-supervised few-shot learning called TransMatch by instantiating the three components with imprinting and MixMatch. Extensive experiments on two popular benchmark datasets for few-shot learning, CUB-200-2011 and miniImageNet, demonstrate that our proposed method can effectively utilize the auxiliary information from labeled base-class data and unlabeled novel-class data to significantly improve the accuracy of few-shot learning task, and achieve new state-of-the-art results.

With the widespread use of depth sensors, it is crucial to provide an effective and efficient solution for human action analysis applications upon the informative depth data. In this paper, we present a generic framework of modeling the human action by deep architecture enhanced local features with depth data. To introduce robust higher-level representations, we augment the adaptive and scalable local depth features in a deep feature learning manner. Specifically, a Deep Architecture of Comparative Coding Descriptor (DA-CCD) is proposed to represent the depth action data. Our approach obtains consistently superior recognition precisions on view specific/non-specific scenarios compared with other leading action representations of depth data on the Huawei/3DLife Dataset.

In real application scenarios, the visual observations of the same type of action vary significantly from one view to another. This paper addresses the action recognition problem under the view changes, especially when no labels are available in the target view. A novel feature, called Sequential Motion Accumulation (SMA), is proposed to characterize actions. The SMA descriptor depicts the temporal structure of motion property to explore the distinguishing action characteristics and their invariances across views. Moreover, we propose a weakly supervised categorization approach to generate target-view categorical prior for learning a cross-view metric, which can further improve the recognition accuracy of the SMA descriptor. Our method is verified on the multiview IXMAS dataset, and it achieves superior performance compared with the state-of-the-art methods.

Identifying individuals in multi-view camera network, known as person re-identification, becomes an emerging topic for video surveillance. In this paper, we address person re-identification as a set-based classification problem and introduce mutual-information to fully utilize gallery information. Firstly, we define a set-based structure that contains pairwise features between query image and gallery images. Then these features are fed into a set-class model, which exploits the relationship between set and class label (person identity) using mutual-information. Finally, we estimate and rank the mutual-information scores, and the corresponding label of the highest score is assigned to the query image. Our method has gained a superior performance compared with the state-of-the-art in the benchmark datasets i-LIDS and ETHZ.

Human activity analysis is an important and challenging task in video content analysis and understanding. In this paper, we focus on the activity of small human group, which involves countable persons and complex interactions. To cope with the variant number of participants and inherent interactions within the activity, we propose a hierarchical model with three layers to depict the characteristics at different granularities. In traditional methods, group activity is represented mainly based on motion information, such as human trajectories, but ignoring discriminative appearance information, e.g. the rough sketch of a pose style. In our approach, we take advantage of both the motion and the appearance information in the spatiotemporal activity context under the hierarchical model. These features are inhomogeneous. Therefore, we employ multiple kernel learning methods to fuse the features for group activity recognition. Experiments on a surveillance-like human group activity database demonstrate the validity of our approach and the recognition performance is promising.

Human action recognition has been well studied recently, but recognizing the activities of more than three persons remains a challenging task. In this paper, we propose a motion trajectory based method to classify human group activities. Gaussian Processes are introduced to represent human motion trajectories from a probabilistic perspective to handle the variability of people's activities in group. With respect to the relationships of persons in group activities, three discriminative descriptors are designed, which are Individual, Dual and Unitized Group Activity Pattern. We adopt the Bag of Words approach to solve the problem of unbalanced number of persons in different activities. Experiments are conducted on the human group-activity video database, and the results show that our approach outperforms the state-of-the-art.