The incidence of autism spectrum disorders (ASD), a neurodevelopmental condition associated with challenges in social communication, has witnessed a remarkable surge in recent years, with adverse effects on individuals, families, and society at large. Early screening for autism ensures timely access to interventions, yet screening lacks systematic and methodical approaches for objectively quantifying social behaviors. In response to this, we propose a protocol for early assistive screening, termed the Express-Needs-with-Pointing (ENP), which employs a multi-sensor platform to quantify the one of the social skills of toddler. A vision-based pointing behavior detection method is proposed, combining gaze estimation and pointing estimation, where the pointing estimation integrates forearm orientation and finger direction. We conduct an experiment involving twenty toddlers aged between 16 and 32 months, 4 of whom are typically developing (TD) children, 6 diagnosed with ASD, 8 diagnosed with global developmental delay (GDD), and 5 diagnosed with language disorders (LD). The results demonstrate that the automated assessment methods for pointing behavior achieved an impressive accuracy rate of 93.9%. These findings provide compelling evidence that the ENP is one of the highly effective protocols and holds significant implications for assisting in early autism screening.

Cognitive assessment can reveal a person's cognitive processing and behavioral patterns, making it an indispensable component of autism intervention and prognosis. Existing machine-assisted cognitive assessment methods primarily focus on children's performance outcomes, overlooking distinctive behavioral models, particularly characteristics of eye movement behavior, which have been demonstrated as the most direct indicators of cognitive abilities. In this study, we explore eye-tracking biomarkers for assisting cognitive assessment through a series of meticulously designed multi-level human-computer interaction protocols, encompassing three cognitive abilities: pairing and categorization, emotion recognition, and social interaction. A platform embedded with an eye-tracking module has been developed to reliably collect and analyze eye movement data, even in the presence of unrestricted large head movements in children. Experimental results indicate that there are significant group differences between autism and typically developing children in the eye-tracking features of total fixation duration, response latency, time to first fixation, mean fixation duration, and visit count in the absence of significant intergroup differences in the Wechsler Preschool and Primary Scale of Intelligence (WPPSI) and Wechsler Intelligence Scale for Children (WISC) assessment results. In addition, certain eye-tracking features in each group are correlated with WPPSI/WISC scale scores, enabling clinical cognitive assessments within each group based on these eye movement features. This study suggests that using eye-tracking features as biomarkers to assist detailed cognitive assessments holds significant potential for the intervention and prognosis of autism.

Developmental and behavioral disorders (DBD) have a significant impact on children's neurological activity and behavioral performance. Early diagnosis and treatment are known to be beneficial for improving DBD outcomes, yet existing unimodal neurophysiological assessment tools for DBD yield significant heterogeneity in results, highlighting the urgent need for exploring novel assessment tools. Cortico-ocular coupling (COC) refers to the information interaction between the cerebral cortex and eyes, and COC analysis is a technique for quantitatively measuring the correlation of neural oscillations and eye movements as biomarkers for assessment and mechanism disclosure. This review focuses on COC analysis for DBD from four perspectives: neural substrates, research paradigms, analysis methods, and applications. First, this review provides a comprehensive overview of the neural substrates and evocation paradigms related to COC analysis, aiming at helping target brain region selection, experimental result analysis, and paradigm design. The neural substrates and evocation paradigms are categorized according to functional domains, including social functioning, attention, cognition, early visual processing, and motor function. Then, this review summarizes the EEG and eye-tracking features, the analysis methods, and the validation datasets involved in COC analysis, aiming at helping implement COC analysis. Next, this review presents the applications of COC analysis in DBD, proving the validity and advance of COC analysis. In the end, the limitations, challenges, and future directions of COC analysis are discussed.

Estimating children's engagement levels improves their understanding of their social behaviors, since they can reflect their devotion to social interaction with others. This paper proposes an automatic method to recognize children's engagement levels in a triadic social interaction context. First, an overall metric function containing behavior, cognition, and affective dimensions is proposed to estimate children's multidimensional engagement levels. Then, the automatic feature extraction method based on gaze estimation, facial expression recognition, pose estimation, and object recognition models is illustrated to extract features to compute the engagement levels. Videos of 24 children, including 13 children with autism spectrum disorder (ASD), in triadic social interaction were collected for the engagement recognition experiment and cross-group analysis. The experimental results validate the effectiveness of the proposed automatic feature extraction method compared to human observations. Cross-group analyses revealed significant differences in affective engagement between children with ASD and typical developmental (TD) children.

Autism Spectrum Disorders (ASD) remain a healthcare challenge and gain considerable attention due to the increasing prevalence rates and insupportable burden on families and society. It is noted that the recognition of children’s emotional states plays an important role in the evaluation and intervention process of ASD. In this paper, we aim to address the problem of automatic recognition of the emotional states of ASD children in social interactive scenarios. Since the child can be unconstrained in realistic scenarios, the face occlusion under pose variations and uncertain backgrounds become challenges of this task. To tackle this problem, we employ both facial expressions as well as body poses as cues to recognize the emotional states while most traditional methods only leverage the former. Firstly for the facial information, spatial features are extracted through convolutional neural networks followed by a temporal transformer to extract temporal information. Then for the body pose information, graph convolutional networks combined with the self-attention part are used to represent spatial features and temporal convolutional layers for temporal counterparts. Finally, different multimodal fusion ways are explored to generate final recognition results. We evaluate this method on a challenging database collected by us in real-world child-clinician interactive scenarios and the proposed method achieved significantly better results than baselines using only facial information. Thus it is suggested that there is a potential to assist in clinical practice by providing the recognized emotion as feedback.

Gaze is a vital feature in analyzing natural human behavior and social interaction. Existing gaze target detection studies learn gaze from gaze orientations and scene cues via a neural network to model gaze in unconstrained scenes. Though achieve decent accuracy, these studies either employ complex model architectures or leverage additional depth information, which limits the model application. This article proposes a simple and effective gaze target detection model that employs dual regression to improve detection accuracy while maintaining low model complexity. Specifically, in the training phase, the model parameters are optimized under the supervision of coordinate labels and corresponding Gaussian-smoothed heatmap labels. In the inference phase, the model outputs the gaze target in the form of coordinates as prediction rather than heatmaps. Extensive experimental results on within-dataset and cross-dataset evaluations on public datasets and clinical data of autism screening demonstrate that our model has high accuracy and inference speed with solid generalization capabilities.

Interpersonal communication facilitates symptom measures of autistic sociability to enhance clinical decision-making in identifying children with autism spectrum disorder (ASD). Traditional methods are carried out by clinical practitioners with assessment scales, which are subjective to quantify. Recent studies employ engineering technologies to analyze children's behaviors with quantitative indicators, but these methods only generate specific rule-driven indicators that are not adaptable to diverse interaction scenarios. To tackle this issue, we propose a Computational Interpersonal Communication Model (CICM) based on psychological theory to represent dyadic interpersonal communication as a stochastic process, providing a scenario-independent theoretical framework for evaluating autistic sociability. We apply CICM to the response-to-name (RTN) with 48 subjects, including 30 toddlers with ASD and 18 typically developing (TD), and design a joint state transition matrix as quantitative indicators. Paired with machine learning, our proposed CICM-driven indicators achieve consistencies of 98.44% and 83.33% with RTN expert ratings and ASD diagnosis, respectively. Beyond outstanding screening results, we also reveal the interpretability between CICM-driven indicators and expert ratings based on statistical analysis.

Obtaining accurate and stable results in gaze target detection is vital for the subsequent analysis of gaze meaning. However, existing image-based methods, which focus solely on enhancing accuracy, demonstrate poor stability when directly applied on videos. Especially when the video frame rate is low, even though the actual gaze target positions do not differ significantly between adjacent frames, the detected positions vary considerably. This inconsistency, stemming from the lack of temporal information, makes dynamic detection challenging and can lead to jarring outcomes. To reduce the jitter in gaze target detection in videos, we introduce an approach that integrates spatial and temporal modules to combine spatial with temporal information. Additionally, we propose a Jitter loss function to capture significant jitter and impose a strong penalty during training, which empowers our model with increased stability for dynamic detection. Based on a self-collected dataset, experiments demonstrate that our approach exhibits superior stability without compromising accuracy.

The major challenge of Facial Expression Recog-nition (FER) is to learn class discriminative representations, and the existing works mainly address it by designing various classification networks from class level. However, learning representations at class level is limited due to the inconspicuous class discrimination among different facial expressions. Thus, in this paper, we propose a Weak Supervised Contrastive learning FER (WSCFER) method to improve facial expression representations by simultaneously learning instance-level representations which are highly complementary to the general class-level representations. Specifically, our proposed WSCFER consists of three components: a major task for FER classification, an auxiliary task for Weak Supervised Contrastive (WSC) learning which pulls augmented samples of the same image together while pushing apart instance samples from different classes, and a Partial Consistency Loss (PCL) for optimizing the two embedding spaces from both the class level and the instance level. We compare WSC with some state-of-the-art contrastive methods and find that it can efficiently learn instance-level representations but avoid overemphasizing irrelevant parts, which is crucial for FER. WSCFER achieves superior performance on several in-the-wild databases, and it also shows the promising potential for learning representations under noisy annotations.

Early screening for ASD (Autism Spectrum Disorder) is crucial and also challenging due to the limited medical resource. Expressing Needs with Pointing (ENP) is a low-cost yet effective protocol for early screening. However, the current methods need to manually trim video for analyzing ENP protocol, which is labour-intensive. Also, they detect discriminative signs with separately high-level clues (e.g., pose, object detection), but ignore the temporal action relationships between child and clinician, which usually leads to invalid detection. In contrast to previous approaches, we propose an Auto Rating Pipeline for Expressing Needs via Pointing Protocol, named AutoENP. Specifically, we introduce action segmentation into early screening, to capture temporal interaction relationships without manually intervention. To detect fine-grained hand motions, we fuse global, local and fine-grained features to fully understand the screening scene. Besides, we integrate focal loss and center loss to improve the detection accuracy for rare actions. To evaluate the proposed pipeline, we collected 22 ENP videos containing 7 actions with above 40,000 frames. Experimental results demonstrate that our model achieves 82.1% and 84.7% action accuracy for child and clinician, respectively. Moreover, 18 in 22 children’s ENP levels are reported correctly against the clinician’s diagnoses.

We consider the problem of clustering a set of uncertain data, where each data consists of a point-set indicating its possible locations. The objective is to identify the representative for each uncertain data and group them into k clusters so as to minimize the total clustering cost. Different from other models, our model does not assume that there is a probability distribution for each uncertain data. Thus, all possible locations need to be considered to determine the representative. Existing methods for this problem are either impractical or have difficulty to handle large-scale datasets due to their pairwise-distance based global search strategy and expensive optimization computation. In this paper, we propose a novel sparse Non-negative Matrix Factorization (NMF) method which measures the similarity of uncertain data by their most commonly shared features. A divide-and-conquer approach is adopted to remarkably improve the efficiency. A novel diagonal l0-constraint and its l1 relaxation are proposed to overcome the challenge of determining the representatives. We give a detailed analysis to show the correctness of our method, and provide an effective initialization and peeling strategy to enhance the ability of processing large-scale datasets. Experimental results on some benchmark datasets confirm the effectiveness of our method.

Early screening of autism spectrum disorder (ASD) is crucial since early intervention evidently confirms significant improvement of functional social behavior in toddlers. This article attempts to bootstrap the response-to-instructions (RTIs) protocol with vision-based solutions in order to assist professional clinicians with an automatic autism diagnosis. The correlation between detected objects and toddler's emotional features, such as gaze, is constructed to analyze their autistic symptoms. Twenty toddlers between 16-32 months of age, 15 of whom diagnosed with ASD, participated in this study. The RTI method is validated against human codings, and group differences between ASD and typically developing (TD) toddlers are analyzed. The results suggest that the agreement between clinical diagnosis and the RTI method achieves 95% for all 20 subjects, which indicates vision-based solutions are highly feasible for automatic autistic diagnosis.

In recent years, face recognition technology has been rapidly promoted and used because of its advantages of convenience, non-contact and good recognition performance, and achieved good results. But for the polluted situation, such as expression change, illumination influence, posture change, occlusion, etc., because the collected image lost the original intrinsic characteristics, seriously affect the effect of face recognition. In recent years, the SRC method and its variants have good robustness to shaded and contaminated samples. However, when certain training samples and test samples are contaminated, the performance will degrade. The method based on low rank matrix recovery (LRMR) can effectively deal with the simultaneous contamination of training samples and test samples. However, they either ignore relationships between similar samples or fail to learn a compact dictionary from contaminated training data. To solve these problems, a locally constrained low-rank representation and dictionary learning algorithm (LCLRRDL) is developed and applied to robust face recognition. In this paper, the theoretical and experimental research of locally constrained low-rank representation and dictionary learning algorithm (LCLRRDL) is carried out. A low-rank representation is introduced to deal with possible contamination of training and test data, and local constraints are introduced to identify the inherent manifold structure of training data. Under local constraints, similar samples often have similar representations, and the learned representations can be directly used for classification. At the same time, this method can learn a compact dictionary with better refactoring and recognition ability. The superiority of LCLRRDL algorithm is verified by comparing it with support vector machine (SVM) and Sparse representation (SRC) for contaminated small sample face recognition. By comparing the best recognition effect and classification time of the three algorithms, it is found that the three algorithms are robust to contaminated face recognition for two different face databases. LCLRRDL algorithm has the highest recognition rate, but its disadvantage is that the classification recognition time is long. Therefore, when the actual application does not require the recognition speed, the recognition rate of LCLRRDL algorithm is more suitable for high-precision face recognition.

Incidence of children with autism spectrum disorder (ASD) has increased with an average rate of 1% worldwide. Clinical ASD screening, especially for children screening is a laborious and skilled task; however, there is no objective and effective method automating ASD children screening. Analyzing children ASD characteristics in predefined motion behavior protocols is attempted to provide automatic solutions to children ASD screening. A novel protocol, response to name (RTN), is proposed in this article for ASD clinical validation and diagnosis. The RTN method is jointly designed with clinical partners, and novel gaze estimation is developed for validating ASD characteristic behavior. Seventeen subjects including ten adults and seven children (five ASD subjects and two healthy subjects) have participated the experiment. The experiment results show that the proposed RTN system achieves an average classification score of 92.7% fully demonstrating that the principle of motion protocol based ASD screening has the potential to have early ASD screening automated.

In order to improve the location accuracy of DV-Hop algorithm, an improved algorithm based on iterative computation and two communication ranges was proposed. This algorithm first selects a suitable communication radius for the current network topology and then uses it to estimate the average per hop distance of beacon nodes with the default communication radius of the node. Finally, an iterative algorithm was used to revise the average per hop distance obtained in the previous step so as to select the minimum average per hop distance to calculate the distance between the unknown nodes and beacon nodes. The simulation result indicates that the improved algorithm can greatly improve the location accuracy without obviously increasing algorithm complexity and communication traffic.