Autonomous driving holds transformative potential but remains fundamentally constrained by the limited perception and isolated decision-making with standalone intelligence. While recent multi-agent approaches introduce cooperation, they often focus merely on perception-level tasks, overlooking the alignment with downstream planning and control, or fall short in leveraging the full capacity of the recent emerging end-to-end autonomous driving. In this paper, we present UniMM-V2X, a novel end-to-end multi-agent framework that enables hierarchical cooperation across perception, prediction, and planning. At the core of our framework is a multi-level fusion strategy that unifies perception and prediction cooperation, allowing agents to share queries and reason cooperatively for consistent and safe decision-making. To adapt to diverse downstream tasks and further enhance the quality of multi-level fusion, we incorporate a Mixture-of-Experts (MoE) architecture to dynamically enhance the BEV representations. We further extend MoE into the decoder to better capture diverse motion patterns. Extensive experiments on the DAIR-V2X dataset demonstrate our approach achieves state-of-the-art (SOTA) performance with a 39.7% improvement in perception accuracy, a 7.2% reduction in prediction error, and a 33.2% improvement in planning performance compared with UniV2X, showcasing the strength of our MoE-enhanced multi-level cooperative paradigm.

Eye movement classification can decode cognitive processes, offering valuable insights for a wide range of applications. However, existing eye movement classification models primarily focus on static fixation-based features and often neglect the encoding of spatiotemporal eye movement features, which are crucial for accurately reconstructing visual attention. To address this limitation, we propose a spatiotemporal attention encoding (STAE) model that jointly captures both spatial and temporal features for eye-tracking classification. First, we utilize a Vision Transformer (ViT) to extract spatial features from fixations by taking global competition into consideration. We then introduce a global weighting Gated Recurrent Unit (GRU) model to capture temporal correlations from the feature sequence. Specifically, we propose a hidden-state-based weighting to fuse the influence of different fixations on the current fixation. In the experiment, we evaluated our model on three tasks: autism spectrum disorder (ASD) identification, visual task classification, and age classification. Experiential results across three databases demonstrate that our model outperforms existing methods and shows strong adaptability across various eye movement classification tasks. The code is available at https://github.com/HectorTo/spatiotemporal-attention-encoding-STAE-.

Eye-tracking is a reliable method for quantifying visual information processing and holds significant potential for group recognition, such as identifying autism spectrum disorder (ASD). However, eye-tracking research typically faces the heterogeneity of stimuli and is time-consuming due to the large number of observed stimuli. To address these issues, we first mathematically define the stimulus selection problem and introduce the concept of stimulus discrimination ability to reduce the computational complexity of the solution. Then, we construct a scanpath-based recognition model to mine the stimulus discrimination ability. Specifically, we propose cross-subject entropy and cross-subject divergence scores for quantitatively evaluating stimulus discrimination ability, effectively capturing differences in intra-group collective trends and inter-subject consistency within a group. Furthermore, we propose an iterative learning mechanism that employs stimulus-wise attention to focus on discriminative stimuli for discrimination purification. In the experiment, we construct an ASD eye-tracking dataset with diverse stimulus types and conduct extensive tests on three representative models to validate our approach. Remarkably, our method demonstrates superior performance using only 10 selected stimuli compared to models utilizing 220 stimuli. Additionally, we perform experiments on another eye-tracking task, gender prediction, to further validate our method. We believe that our approach is both simple and flexible for integration into existing models, promoting large-scale ASD screening and extending to other eye-tracking research domains.

Recognizing autism spectrum disorder (ASD) has faced great challenges due to insufficient professional clinicians and complex procedures. Automated data-driven ASD recognition models can reduce the subjectivity and physician dependency of traditional evaluation methods. Facial data, which can encode important perceptual and social behaviors, have emerged in ASD research to explore novel biomarkers for screening, diagnosing, and treating ASD. However, existing research mainly focuses on extracting low-level hand-crafted facial features for analysis and classification. Determining how to learn discriminative deep representations from dynamic facial data for computational model construction remains an unresolved challenge. In this study, we propose an ASD recognition model based on facial videos to fill the lack of temporal correlation learning of facial features. First, we utilize a vision transformer to extract frame-based global facial features. Then, we use a Longformer to establish the correlation of facial features over time. In the experiment, we recruited 146 subjects between 2 and 8 years of age to record their facial videos under a computer-based eye-tracking experiment and 76 subjects to conduct a smartphone-based experiment. Quantitative comparisons have shown the effectiveness and reliability of the proposed model. Furthermore, we have confirmed the correlation between facial and eye-tracking modalities in visual attention.

Urban intersections, dense with pedestrian and vehicular traffic and compounded by positioning signal obstructions, are among the most challenging areas in urban traffic systems. Traditional single-vehicle intelligence systems often perform poorly in such environments due to a lack of global scene observations and the inherent uncertainty in predicting other agents’ intentions. Vehicle-to-Everything (V2X) technology, through real-time communication between vehicles (V2V) and vehicles to infrastructure (V2I), offers a robust solution. However, practical applications still face numerous challenges. Spatial registration among vehicle and infrastructure endpoints with different configurations in multi-end sensing systems is crucial for ensuring the accuracy of perception system data. Most existing multi-end spatial registration methods rely on initial extrinsic values provided by positioning systems, but the instability of GNSS signals due to high buildings in urban canyons poses severe challenges to these methods. To address this issue, this paper proposes a novel multi-end spatial registration method that does not require positioning priors to determine initial external parameters and meets real-time requirements. Our method introduces an innovative multi-end perception object association technique that leverages a newOverall Distance(oDist) metric to measure the spatial association between perception objects, subsequently using this metric as the foundation for an optimal transport formulation. By this means, we can extract co-observed targets from object association results for further external parameter computation and optimization. Extensive comparative and ablation experiments conducted on the simulated dataset V2X-Sim and the real dataset DAIR-V2X confirm the effectiveness and efficiency of our method. The code for this method can be accessed at:https://github.com/MassimoQu/v2i-calib.

Saccadic scanpath, a data representation of human visual behavior, has received broad interest in multiple domains. Scanpath is a complex eye-tracking data modality that includes the sequences of fixation positions and fixation duration, coupled with image information. However, previous methods usually face the spatial misalignment problem of fixation features and loss of critical temporal data (including temporal correlation and fixation duration). In this study, we propose a Transformer-based scanpath model, SpFormer, to alleviate these problems. First, we propose a fixation-centric paradigm to extract the aligned spatial fixation features and tokenize the scanpaths. Then, according to the visual working memory mechanism, we design a local meta attention to reduce the semantic redundancy of fixations and guide the model to focus on the meta scanpath. Finally, we progressively integrate the duration information and fuse it with the fixation features to solve the problem of ambiguous location with the Transformer block increasing. We conduct extensive experiments on four databases under three tasks. The SpFormer establishes new state-of-the-art results in distinct settings, verifying its flexibility and versatility in practical applications. The code can be obtained from https://github.com/wenqizhong/SpFormer.

Gaze estimation is an important fundamental task in computer vision and medical research. Existing works have explored various effective paradigms and modules for precisely predicting eye gazes. However, the uncertainty for gaze estimation, e.g., input uncertainty and annotation uncertainty, have been neglected in previous research. Existing models use a deterministic function to estimate the gaze, which cannot reflect the actual situation in gaze estimation. To address this issue, we propose a probabilistic framework for gaze estimation by modeling the input uncertainty and annotation uncertainty. We first utilize probabilistic embeddings to model the input uncertainty, representing the input image as a Gaussian distribution in the embedding space. Based on the input uncertainty modeling, we give an instance-wise uncertainty estimation to measure the confidence of prediction results, which is critical in practical applications. Then, we propose a new label distribution learning method, probabilistic annotations, to model the annotation uncertainty, representing the raw hard labels as Gaussian distributions. In addition, we develop an Embedding Distribution Smoothing (EDS) module and a hard example mining method to improve the consistency between embedding distribution and label distribution. We conduct extensive experiments, demonstrating that the proposed approach achieves significant improvements over baseline and state-of-the-art methods on two widely used benchmark datasets, GazeCapture and MPIIFaceGaze, as well as our collected dataset using mobile devices.

Atypical eye movement is one of the critical symptoms of autism spectrum disorder (ASD). Automatic quantification of eye-tracking data can provide an objective, convenient, and non-invasive way to identify subjects with ASD, which can develop scalable screening tools for ASD to apply in areas with limited medical resources. However, existing eye-tracking-based ASD classification models usually calculated the score under each image separately and averaged the scores under different images in a post-processing manner to achieve ASD recognition. Determining how to utilize all eye-tracking data of each subject to globally integrate perceptual information and establish a subject-based visual preference for ASD screening is still an unresolved challenge. To address this issue, we propose a novel ASD screening model based on global visual preference encoding. First, we utilize the segment anything model (SAM) and vision transformer (ViT) to extract semantic label regions from all test images. Then, we establish a personalized visual preference graph for each subject based on the saccadic shifts between different semantic regions. Finally, we apply a graph convolutional network (GCN) to learn the mapping between the visual preference graph and classification labels for ASD recognition. In the experiment, we recruited 28 children with ASD and 30 typically developing (TD) children between 2 and 8 years of age to record their eye-tracking data under 220 test images from four types. The experimental results have shown that the proposed model can outperform the state-of-the-art eye-tracking-based ASD recognition models. Furthermore, the evaluation results have indicated the potential to extend the proposed model to other eye-tracking applications, resulting in progress in visual research, accessibility, and healthcare.

In recent years, predicting the saccadic scanpaths of humans has become a new trend in the field of visual attention modeling. Given various saccadic algorithms, determining how to evaluate their ability to model a dynamic saccade has become an important yet understudied issue. To our best knowledge, existing metrics for evaluating saccadic prediction models are often heuristically designed, which may produce results that are inconsistent with human subjective assessment. To this end, we first construct a subjective database by collecting the assessments on 5,000 pairs of scanpaths from ten subjects. Based on this database, we can compare different metrics according to their consistency with human visual perception. In addition, we also propose a data-driven metric to measure scanpath similarity based on the human subjective comparison. To achieve this goal, we employ a long short-term memory (LSTM) network to learn the inference from the relationship of encoded scanpaths to a binary measurement. Experimental results have demonstrated that the LSTM-based metric outperforms other existing metrics. Moreover, we believe the constructed database can be used as a benchmark to inspire more insights for future metric selection.

We propose the task of unsupervised morphological paradigm completion.Given only raw text and a lemma list, the task consists of generating the morphological paradigms, i.e., all inflected forms, of the lemmas.From a natural language processing (NLP) perspective, this is a challenging unsupervised task, and high-performing systems have the potential to improve tools for low-resource languages or to assist linguistic annotators.From a cognitive science perspective, this can shed light on how children acquire morphological knowledge.We further introduce a system for the task, which generates morphological paradigms via the following steps: (i) EDIT TREE retrieval, (ii) additional lemma retrieval, (iii) paradigm size discovery, and (iv) inflection generation.We perform an evaluation on 14 typologically diverse languages.Our system outperforms trivial baselines with ease and, for some languages, even obtains a higher accuracy than minimally supervised systems. 1 Sé vigilante y confirma las otras cosas que están para morir , porque no he hallado tus obras bien acabadas delante de Dios .Acuérdate , pues , de lo que has recibido y oído ; guárdalo y arrepiéntete , pues si no velas vendré sobre ti como ladrón y no sabrás a qué hora vendré sobre ti .El vencedor será vestido de vestiduras blancas , y no borraré su nombre del libro de la vida , y confesaré su nombre delante de mi Padre y delante de sus ángeles .El que tiene oído , oiga lo que el Espíritu dice a las iglesias .

Major depression disorder (MDD) has been proved to have difficulty in emotional conflict processing. The objective of this study is to investigate the different attention deployment patterns in emotional conflict processing between MDDs and Healthy Controls (HCs) using eye tracking data. A face-word Stroop task was used, and 49 MDDs and 50 healthy controls (HCs) were recruited in the experiment. Finally, our results indicated that MDDs demonstrated lower accuracy (ACC) compared with HCs during the process of emotional conflict. Moreover, we found attention bias in the process of attention maintenance but not vigilance, and it may be one of the possible reasons for different ability of emotional conflict processing between MDDs and HCs.

In existing big data stacks, the processes of analytical processing and knowledge serving are usually separated in different systems. In Alibaba, we observed a new trend where these two processes are fused: knowledge serving incurs generation of new data, and these data are fed into the process of analytical processing which further fine tunes the knowledge base used in the serving process. Splitting this fused processing paradigm into separate systems incurs overhead such as extra data duplication, discrepant application development and expensive system maintenance. In this work, we propose Hologres, which is a cloud native service for hybrid serving and analytical processing (HSAP). Hologres decouples the computation and storage layers, allowing flexible scaling in each layer. Tables are partitioned into self-managed shards. Each shard processes its read and write requests concurrently independent of each other. Hologres leverages hybrid row/column storage to optimize operations such as point lookup, column scan and data ingestion used in HSAP. We propose Execution Context as a resource abstraction between system threads and user tasks. Execution contexts can be cooperatively scheduled with little context switching overhead. Queries are parallelized and mapped to execution contexts for concurrent execution. The scheduling framework enforces resource isolation among different queries and supports customizable schedule policy. We conducted experiments comparing Hologres with existing systems specifically designed for analytical processing and serving workloads. The results show that Hologres consistently outperforms other systems in both system throughput and end-to-end query latency.

Depression, influencing millions of people, has become a major disease in the past decade. However, the assessment methods of diagnosing depression almost exclusively rely on patient-reported or clinical judgments of symptom severity, which are associated with subjective biases and intensive labor. Some bio-signals such as EEG and eye movements are used for automatic detection but their accuracies are not accurate enough for the real application, further improvements are needed. This research proposes a content based ensemble method (CBEM) to promote the depression detection accuracy, generating data subsets by the content of the experiment, then using the majority vote of subsets to determine the subjects' label. The validation of the method is testified by two different experiments which included free viewing eye tracking and task-state EEG and these two experiments have 36, 40 subjects respectively. In these two experiments CBEM gains accuracies of 82.5% and 92.73% respectively. The results show that CBEM outperform traditional classification methods. Our findings provide an effective solution for promoting the accuracy of depression identification, and give an objective and quantitative evaluation of depression, which in the future could be used for the auxiliary diagnosis of depression.

Visual attention is a dynamic process of scene exploration and information acquisition. However, existing research on attention modeling has concentrated on estimating static salient locations. In contrast, dynamic attributes presented by saccade have not been well explored in previous attention models. In this paper, we address the problem of saccadic scanpath prediction by introducing an iterative representation learning framework. Within the framework, saccade can be interpreted as an iterative process of predicting one fixation according to the current representation and updating the representation based on the gaze shift. In the predicting phase, we propose a Bayesian definition of saccade to combine the influence of perceptual residual and spatial location on the selection of fixations. In implementation, we compute the representation error of an autoencoder-based network to measure perceptual residuals of each area. Simultaneously, we integrate saccade amplitude and center-weighted mechanism to model the influence of spatial location. Based on estimating the influence of two parts, the final fixation is defined as the point with the largest posterior probability of gaze shift. In the updating phase, we update the representation pattern for the subsequent calculation by retraining the network with samples extracted around the current fixation. In the experiments, the proposed model can replicate the fundamental properties of psychophysics in visual search. In addition, it can achieve superior performance on several benchmark eye-tracking data sets.

Visual attention is a dynamic search process of acquiring information. However, most previous studies have focused on the prediction of static attended locations. Without considering the temporal relationship of fixations, these models usually cannot explain the dynamic saccadic behavior well. In this paper, an iterative representation learning framework is proposed to predict the saccadic scanpath. Within the proposed framework, saccade can be explained as an iterative process of finding the most uncertain area and updating the representation of scenes. In implementation, a deep autoencoder is employed for representation learning. The current fixation is predicted to be the most salient pixel, with saliency estimated by the reconstruction residual of the deep network. Image patches around this fixation are then sampled to update the network for the selection of subsequent fixations. Compared with existing models, the proposed model shows the state-of-the-art performance on several public data sets.

Research on visual perception indicates that the human visual system is sensitive to center-surround (C-S) contrast in the bottom-up saliency-driven attention process. Different from the traditional contrast computation of feature difference, models based on reconstruction have emerged to estimate saliency by starting from original images themselves instead of seeking for certain ad hoc features. However, in the existing reconstruction-based methods, the reconstruction parameters of each area are calculated independently without taking their global correlation into account. In this paper, inspired by the powerful feature learning and data reconstruction ability of deep autoencoders, we construct a deep C-S inference network and train it with the data sampled randomly from the entire image to obtain a unified reconstruction pattern for the current image. In this way, global competition in sampling and learning processes can be integrated into the nonlocal reconstruction and saliency estimation of each pixel, which can achieve better detection results than the models with separate consideration on local and global rarity. Moreover, by learning from the current scene, the proposed model can achieve the feature extraction and interaction simultaneously in an adaptive way, which can form a better generalization ability to handle more types of stimuli. Experimental results show that in accordance with different inputs, the network can learn distinct basic features for saliency modeling in its code layer. Furthermore, in a comprehensive evaluation on several benchmark data sets, the proposed method can outperform the existing state-of-the-art algorithms.

The center-surround comparison principle is widely used in existing bottom-up saliency estimation models. However, most of them are based on local image processing techniques which are hard to handle texture regions well as a relatively large neighborhood is required to represent textures. In this paper, we propose a nonlocal patch-based reconstruction approach to reformulate the center-surround comparison. In the proposed approach, the saliency is measured by the reconstruction residual of representing the central patch with a linear combination of its surrounding patches. As a generalization of Itti et al.'s classical center-surround comparison scheme, the proposed approach performs well on images with symmetric structures where Itti et al.'s method fails, as well as on general natural images. Numerical experiments show the proposed approach produces better results compared to the state-of-the-art algorithms on several public databases.

A simplified Doppler diversity technique for orthogonal frequency division multiplexing (OFDM) high-speed mobile communication systems has been proposed in this paper. And SIR has been analyzed for OFDM receiver with the proposed Doppler diversity without losing the diversity performance. The optimal configuration of this receiver has also been researched to improve the system performance. Finally, simulation results of SIR for OFDM receiver and comparison have been given.

Mobile ad hoc networks (MANET) works properly only if the participating nodes cooperate in routing and forwarding. However, for saving battery life, the selfish node simply does not cooperate in network operation. So in this paper, we analyze the harm of selfishness behavior and its solution, and provide an improved scheme, which is based on trusty center server and neighbor monitor cooperative enhancing. It may solve the spoofing attacks caused by selfish nodes changing their identities. Finally, the simulation results conclude that this cooperation enhancement scheme is effective to solve the selfish nodes secure problem, and it is easy to be implement.

A multiple access/multiple user system called collision frequency shift-keying (CFSK) is investigated. Maximum-likelihood sequence detection (MLSD) provides optimal multiuser detection for the CFSK system but is too complex for practical applications so iterative detection is investigated as a suboptimal approach. The MAP-consensus decoder is given as an example of iterative detection and studied for both the synchronous and asynchronous system. A channel interleaver is introduced so that the performance of the MAP-consensus decoder is asymptotically optimal. The metric of the CFSK system has nonpolynomial complexity so suboptimal metrics with linear complexity are considered. Simulation results show that iterative detection with reduced complexity metrics achieve good performance for moderate spreading factors.