Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on cameras and microphones, which can cause significant disturbances and often raise privacy concerns. In this paper, we sense floor vibration, which provides a less disruptive and more non-intrusive way of crowd sensing, to predict crowd behavior. However, since the vibration-based crowd monitoring approach is newly developed, one main challenge is the lack of training data due to sports stadiums are usually large public spaces with complex physical activities.

This paper addresses challenges in monitoring human behavior in crowds through floor vibration sensing, overcoming limitations like subjective manual observation, visual occlusions, and audio interference. Our approach involves tackling limited-data vibration signal tasks by conducting pre-training across modalities, leveraging publicly available audio datasets. By leveraging self-supervised representation learning to pre-train on publicly available audio datasets, our approach reduces data requirements, improves robustness, and minimizes the need for human labeling efforts. Evaluation using in-game stadium vibration data with YouTube audio dataset demonstrates up to 5.8 × error reduction for crowd behavior.

Computer vision has been used more ubiquitously in recent years to understand and measure the environment around us, particularly in our neighborhoods. However, many city-wide sensing applications using vision require large labeling efforts, making various applications difficult on a wide scale. We propose a framework for labeling and self-training of in-car video to detect trash on the roads. Our approach requires minimal manual labeling to identify items not meant to be in the street, sidewalk, or public places, from a front-viewing car camera. Our system provides each frame of a video with a score indicating the amount of trash. To prevent overfitting, due to minimal available data, we remove data with high certainty of trash from the training dataset. The results show that our prediction with manually labeled ground truth yield an R2 of 0.66.

Excellent text recognition results have been obtained by training recognition models with synthetic images. However, recognizing text from real-world images still faces challenges due to the domain shift between synthetic and real-world text images. One strategy to eliminate this domain difference without manual annotation is unsupervised domain adaptation (UDA). Due to the characteristics of sequential labeling tasks, most popular UDA methods cannot be directly applied to text recognition. To tackle this problem, we proposed a UDA method that minimizes latent entropy on sequence-to-sequence attention-based models with class-balanced self-paced learning. Experimental results show that our proposed framework achieves better recognition results than the existing methods on most UDA text recognition benchmarks. All codes are publicly available1.

Polyphone disambiguation is the most crucial task in Mandarin grapheme-to-phoneme (g2p) conversion.Previous studies have approached this problem using pre-trained language models, restricted output, and extra information from Part-Of-Speech (POS) tagging.Inspired by these strategies, we propose a novel approach, called g2pW, which adapts learnable softmax-weights to condition the outputs of BERT with the polyphonic character of interest and its POS tagging.Rather than using the hard mask as in previous works, our experiments show that learning a soft-weighting function for the candidate phonemes benefits performance.In addition, our proposed g2pW does not require extra pre-trained POS tagging models while using POS tags as auxiliary features since we train the POS tagging model simultaneously with the unified encoder.Experimental results show that our g2pW outperforms existing methods on the public CPP dataset.All codes, model weights, and a user-friendly package are publicly available.

Dance movements recognition is a challenging research issue, due to the complexity of movement, complex attitude transition, and the movement interpretation diversity between human subjects. Moreover, for a static camera, dancer's behavior is often occluded by a human's body parts or other objects. A real-time prototype is proposed to verify that the complicated dancing behaviors can be recognized.