We present a novel duplicate face detection method inspired by a psychological memory model that explains the human cognitive process through sensory memory, short-term memory, and long-term memory to improve the accuracy of visitor counting. To achieve this, we propose a detection array (DA), a short-stay queue (SQ), and a long-stay queue (LQ), corresponding to the three memory types in the cognitive model. Two deep learning models, MobileNet and ResNet, are utilized to detect face areas and extract features, respectively. Detected faces in the current video frame are associated with the registered faces in the DA to identify counting targets. Subsequently, duplicate face removal was performed by sequentially matching the target faces with the registered faces in the SQ (managed by stay time) and the LQ (managed by queue length). Unmatched target faces are newly registered in the SQ, and the visitor count was then updated accordingly. In the experiments, the proposed method was evaluated using the QMUL and ChokePoint datasets. The experimental results demonstrate improved counting performance, with the proposed method achieving 100% accuracy for the QMUL dataset and 94.82% accuracy for the ChokePoint dataset.

This paper presents a novel approach to explain scoring results of infant visual-motor integration sketches utilized in developmental tests by training predefined rules for each test item. To address the performance issues caused by limited data, we employ a pre-trained model that uses supervised contrastive learning based on item scores. To ensure effective training, a memory bank structure is proposed to accumulate diverse embeddings over multiple iterations and prevent the encoder that processes item information from being trained to prevent collapsing in the Siamese network. Experiments demonstrate that the proposed method improves performance in both score and rule inferences, achieving an accuracy of approximately 75.95% in rule inference. In addition, an ablation study validates the effectiveness of the proposed approach in enhancing performance, confirming its potential as a reliable tool for early developmental screenings and clinical assessments. As such, the proposed approach could enhance clinical decision-making by providing essential interpretability for developmental tests.

Facial expressions are one of the most essential channels to communicate a person’s emotional state. In social interaction, the capability to accurately read subtle changes in facial expressions, which reveal emotional fluctuations, is critical for 1) comprehending others’ emotions in context and background situations, 2) identifying responsiveness to others’ emotions, and 3) developing social skills in human–computer interaction. In this paper, we first introduce automatic emotion change detection via facial expression that discovers timings or temporal locations in a video where facial expression significantly changes. We propose a weakly-supervised deep emotion change detection framework that does not require facial expression videos with expensive temporal annotations and instead learns static images for training. Incorporating these ideas, we performed extensive experiments to demonstrate fundamental insights into emotion change detection and the efficacy of our framework using three video datasets, i.e., CASME II, MMI, and our YoutubeECD. Furthermore, we modified our framework for temporal spotting, which is the most similar task to emotion change detection, and showed comparable results with state-of-the-art methods on CAS(ME)2, proving justification for the problem. Even though we only employed the AffectNet to train our framework rather than the CASME II, MMI, YoutubeECD, and CAS(ME)2, experimental results demonstrate its exceptional generalization capability in cross-dataset environments.

The Alberta Infant Motor Scale (AIMS) is a well-known assessment scheme that evaluates the gross motor development of infants by recording the number of specific poses achieved. With the aid of the image-based pose recognition model, the AIMS evaluation procedure can be shortened and automated, providing early diagnosis or indicator of potential developmental disorder. Due to limited public infant-related datasets, many works use the SMIL-based method to generate synthetic infant images for training. However, this domain mismatch between real and synthetic training samples often leads to performance degradation during inference. In this paper, we present a CNN-based model which takes any infant image as input and predicts the coarse and fine-level pose labels. The model consists of an image branch and a pose branch, which respectively generates the coarse-level logits facilitated by the unsupervised domain adaptation and the 3D keypoints using the HRNet with SMPLify optimization. Then the outputs of these branches will be sent into the hierarchical pose recognition module to estimate the fine-level pose labels. We also collect and label a new AIMS dataset, which co—tains 750 real and 4000 synthetic infants images with AIMS pose labels. Our experimental results show that the proposed method can significantly align the distribution of synthetic and real-world datasets, thus achieving accurate performance on fine-grained infant pose recognition.

Conventional studies dealing with repetition detection have mainly focused on tasks of temporal localization or counting the number of repetitions in videos. However, direct discrimination between repetitive and non-repetitive behaviors in videos, called repetitive behavior classification (RBC), has attracted less attention despite its great potential and advantages of: 1) filling the demands in the fields such as classification of repetitive behaviors in children with autism spectrum disorder (ASD) and helping to alleviate manual and time-consuming diagnostic procedures, 2) directly learning representation of differences between repetition and non-repetition patterns along the temporal dimension, and 3) being an effective alternative to the existing repetition counting and temporal segmentation tasks that are struggling with insufficient data and laborious manual annotation effort. In this paper, to the best of our knowledge, we firstly cast the problem of the RBC using deep learning frameworks. For this, we propose a simple yet effective add-on network, SiRepNet, that exploits the Siamese network structure to learn the inherent properties of repetitive behaviors. We also composed the RBC dataset by re-purposing and re-organizing Kinetics and Countix datasets for training our method. To validate our ideas, we carried out extensive experiments on RBC datasets, which showed performance improvement over state-of-the-art video classification algorithms by simply attaching our scheme to them for the RBC task.

Based on Alberta Infant Motor Scale (AIMS), a questionnaire that tracks an infant’s motor function, an infant’s mental development can be evaluated by recording poses a baby can achieve. Therefore, it is meaningful to propose a systematic image-based pose classifier to classify infant actions based on AIMS to provide early diagnosis of a potential develop-mental disorder such as Autism. This paper presents a hierarchical pose classifier, given a baby image frame that com-bines the benefits of 3D human pose estimation and scene context information. Due to privacy policies, we cannot collect enough real infant images/videos for experiments. In-stead, we generate synthetic baby images with the help of the Skinned Multi-Infant Linear (SMIL) model. Images are first fed into a ResNet-50 for coarse-level pose classification. A stacked hourglass CNN and a hierarchical 3D pose estimation scheme are used for 2D/3D pose estimation. Finally, an innovative Hierarchical Infant Pose Classifier (HIPC) takes the estimated 3D keypoints and coarse-level pose classification confidence scores to give the fine-level baby pose classification results. Our experimental results show that our hierarchical pose classifier achieves accurate and stable performance on infant pose recognition.

We propose an ensemble of invariant features for person re-identification. The proposed method requires no domain learning and can effectively overcome the issues created by the variations of human poses and viewpoint between a pair of different cameras. Our ensemble model utilizes both holistic and region-based features. To avoid the misalignment problem, the test human object sample is used to generate multiple virtual samples, by applying slight geometric distortion. The holistic features are extracted from a publically available pre-trained deep convolutional neural network. On the other hand, the region-based features are based on our proposed Two-Way Gaussian Mixture Model Fitting and the Completed Local Binary Pattern texture representations. To make better generalization during the matching without additional learning processes for the feature aggregation, the ensemble scheme combines all three feature distances using distances normalization. The proposed framework achieves robustness against partial occlusion, pose and viewpoint changes. In addition, the experimental results show that our method exceeds the state of the art person re-identification performance based on the challenging benchmark 3DPeS.

Camera networks have been deployed to facilitate human tracking across multi-cameras in the modern surveillance systems. However, human privacy is an important concern on security surveillance. More specifically, in the real world, surveillance cameras are commonly installed by different entities (such as departments or companies), and any recorded video by one entity should not be shared with others to protect the privacy of tracked humans, while maintaining the knowledge of those moving trajectories of tracked humans in a centralized cloud server. This tracking across multi-cameras information can serve as a very powerful analysis tool for locating crime suspects or collecting business statistics. This paper is the first to aim at the importance of privacy-preserving in a multiple-camera tracking system. We address the problems of privacy-preserving human tracking based on Paillier encryption without revealing any recorded video or data, and introduce the secure multiple-camera system which consists of two stages: training and testing stages. Finally, the security analyses and simulations show the effectiveness of the proposed scheme.

ABSTRACT Since Rathaet al. introduced the functional transform for cancelable fingerprint templates, a few simulation attacks to this method have been proposed. The attacks are based on the fact that the transform depends only on the predefined parameters. That is, the attacker may fully simulate the transform and partially invert it if the parameters are available. Although an original template is transformed using different parameters for different systems, even the compromise of only one of these systems may reveal the original template, which may be a serious potential threat from a practical viewpoint. In this paper, we propose an improved functional transform, whose parameters are decided by the original template, as well as predefined user‐specific keys. Because the information on the original template will not be available to the attacker even when a system is compromised, the proposed method significantly improves the security of the original template by preventing the attacker from reconstructing the transform. Copyright © 2013 John Wiley & Sons, Ltd.

In this paper, we propose a novel feature extraction method for face recognition. This method is based on discrete cosine transform (DCT), energy probability (EP), and linear discriminant analysis (LDA). We define an energy probability as magnitude of effective information. It is used to create a frequency mask in DCT domain. Our method consists of three steps. First, the spatial domain of face images is transformed into the frequency domain called DCT domain. Second, for energy probability is applied on DCT domain which acquires from face image, dimension reduction of data and optimization of valid information. At last, in order to obtain the most significant feature of face images, LDA is applied to the extracted data using frequency mask. Our experimental results show that the proposed method improves on the dimension reduction of feature space and the face recognition over the previously proposed methods

We describe a new method for analyzing and extracting human gait motion by combining statistical methods with image processing. The periodic motion of human gait is modeled by trigonometric-polynomial interpolant functions. The gait description is derived by topological analysis guided by medical studies that selects areas from which joint angles are derived by regression analysis. Then, the interpolant functions are fitted to the gait data and whilst showing fidelity to earlier medical studies, also show recognition capability. As such, a new combination of medical knowledge, image processing and regression analysis can be used to label human motion in image sequences.