Acoustic eavesdropping against private or confidential spaces is a significant threat in the realm of privacy protection. While the presence of soundproof material would weaken such an attack, current eavesdropping technology may be able to bypass these protections. Fortunately, existing studies either inadequately cover the full spectrum of human speech due to low-frequency responses or rely heavily on the prior knowledge used to train a model. To address these challenges, this paper introduces mmEcho, a new acoustic eavesdropping method that utilizes millimeter-wave signals to sense vibration induced by sound precisely. Through signal processing techniques such as the intra-chirp scheme and phase calibration algorithm, mmEcho achieves micrometer-level vibration extraction without requiring target-related data. To improve the range of eavesdropping attacks while reducing noise, we optimize radar signals by leveraging the widespread availability of multiple antennas on commercial off-the-shelf radars. We comprehensively evaluate the performance of mmEcho in different real-world settings. Experimental results demonstrate that, with the aid of multi-antenna technology, mmEcho can more effectively reconstruct the audio from the target at various distances, directions, sound insulators, reverberating objects, sound levels, and languages. Compared to existing methods, our approach provides better effectiveness without prior knowledge, such as the speech data from the target.

Rapid urban development necessitates advanced methodologies for efficiently acquiring and analyzing detailed building information. This study proposes an automated framework, named Urban Street Buildings Scanner (USBS), which combines geospatial data and deep learning techniques to analyze street-level imagery, thereby providing valuable insights into building features. Specifically, we leverage the OSMnx library for geospatial data retrieval before exploiting coordinate sorting and interpolation techniques to extract information about buildings on both roadsides from street images. After that, Segment Anything Model (SAM) and GPT are utilized to segment each building in street-view images for analysis. YOLOv7 is used to further identify the windows of individual buildings and to analyze the buildings in a more detailed way. Meanwhile, GPT can also estimate the numbers of floors and the heights of the buildings, providing a more comprehensive street view analysis. The experimental results demonstrate the efficacy of the proposed framework in effectively extracting and analyzing building characteristics from street-level images. Visual representations and statistical data illustrate the successful application of the framework, providing valuable information for urban planning and development.

Cameras have been widely used in traffic operations. While many technologically smart camera solutions in the market can be integrated into Intelligent Transport Systems (ITS) for automated detection, monitoring and data generation, many Network Operations (a.k.a Traffic Control) Centres still use legacy camera systems as manual surveillance devices. In this paper, we demonstrate effective use of these older assets by applying computer vision techniques to extract traffic data from videos captured by legacy cameras. In our proposed vision-based pipeline, we adopt recent state-of-the-art object detectors and transfer-learning to detect vehicles, pedestrians, and cyclists from monocular videos. By weakly calibrating the camera, we demonstrate a novel application of the image-to-world homography which gives our monocular vision system the efficacy of counting vehicles by lane and estimating vehicle length and speed in real-world units. Our pipeline also includes a module which combines a convolutional neural network (CNN) classifier with projective geometry information to classify vehicles. We have tested it on videos captured at several sites with different traffic flow conditions and compared the results with the data collected by piezoelectric sensors. Our experimental results show that the proposed pipeline can process 60 frames per second for pre-recorded videos and yield high-quality metadata for further traffic analysis.

Temporal action proposal generation, which aims to locate temporal segments that may contain actions, is a key prepositive step of various video analysis tasks, like temporal action detection. In this letter, we present Self-Similarity Action Proposal (SSAP), a simple method that generates action proposals using the self-similarity of videos. Specifically, a basic low-level index, structural similarity, is adopted to measure the similarity between adjacent frames. Potential action boundaries are located by thresholding the similarity values and candidate action segments are successively generated by grouping the boundaries. A segment evaluation module (SEM) is further employed to score and refine the segments. The framework achieves state-of-the-art performance on THUMOS14 and competitive results on ActivityNet v1.3. Notably, on THUMOS14, it achieves over 4% improvement on the average recall at 50 proposals and 3.3% gain in [email protected] when combined with an existing action classifier for temporal action detection.

Existing deep learning methods of video recognition usually require a large number of labeled videos for training. But for a new task, videos are often unlabeled and it is also time-consuming and labor-intensive to annotate them. Instead of human annotation, we try to make use of existing fully labeled images to help recognize those videos. However, due to the problem of domain shifts and heterogeneous feature representations, the performance of classifiers trained on images may be dramatically degraded for video recognition tasks. In this paper, we propose a novel method, called Hierarchical Generative Adversarial Networks (HiGAN), to enhance recognition in videos (i.e., target domain) by transferring knowledge from images (i.e., source domain). The HiGAN model consists of a \emph{low-level} conditional GAN and a \emph{high-level} conditional GAN. By taking advantage of these two-level adversarial learning, our method is capable of learning a domain-invariant feature representation of source images and target videos. Comprehensive experiments on two challenging video recognition datasets (i.e. UCF101 and HMDB51) demonstrate the effectiveness of the proposed method when compared with the existing state-of-the-art domain adaptation methods.

During action recognition in videos, irrelevant motions in the background can greatly degrade the performance of recognizing specific actions with which we actually concern ourself here. In this paper, a novel deep neural network, called factorized action-scene network (FASNet), is proposed to encode and fuse the most relevant and informative semantic cues for action recognition. Specifically, we decompose the FASNet into two components. One is a newly designed encoding network, named content attention network (CANet), which encodes local spatial-temporal features to learn the action representations with good robustness to the noise of irrelevant motions. The other is a fusion network, which integrates the pretrained CANet to fuse the encoded spatial-temporal features with contextual scene feature extracted from the same video, for learning more descriptive and discriminative action representations. Moreover, different from the existing deep learning based tasks for generic action recognition, which applies softmax loss function as the training guidance, we formulate two loss functions for guiding the proposed model to accomplish more specific action recognition tasks, i.e., the multilabel correlation loss for multilabel action recognition and the triplet loss for complex event detection. Extensive experiments on the Hollywood2 dataset and the TRECVID MEDTest 14 dataset show that our method achieves superior performance compared with the state-of-the-art methods.