Large Language Models (LLMs) demonstrate human-level or even superior language abilities, effectively modeling syntactic structures, yet the specific computational units responsible remain unclear. A key question is whether LLM behavioral capabilities stem from mechanisms akin to those in the human brain. To address these questions, we introduce the Hierarchical Frequency Tagging Probe (HFTP), a tool that utilizes frequency-domain analysis to identify neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP) neurons) and cortical regions (via intracranial recordings) encoding syntactic structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama 2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human brain relies on distinct cortical regions for different syntactic levels. Representational similarity analysis reveals a stronger alignment between LLM representations and the left hemisphere of the brain (dominant in language processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows greater brain similarity than Gemma, while Llama 3.1 shows less alignment with the brain compared to Llama 2. These findings offer new insights into the interpretability of LLM behavioral improvements, raising questions about whether these advancements are driven by human-like or non-human-like mechanisms, and establish HFTP as a valuable tool bridging computational linguistics and cognitive neuroscience. This project is available at https://github.com/LilTiger/HFTP.

Providing potential next point-of-interest (POI) suggestions for users has become a prominent task in location-based social networks, which receives more and more attention from the industry and academia and it remains challenging due to highly dynamic and personalized interactions in user movements. Currently, state-of-the-art works develop various graph- and sequential-based learning methods to model user-POI interactions and transition regularities. However, there are still two significant shortcomings in these works: (1) ignoring personalized spatial and temporal-aspect interactive characteristics capable of exhibiting periodic interests of users and (2) insufficiently leveraging the sequential patterns of interactions for beyond-pairwise high-order collaborative signals among users’ sequences. To jointly address these challenges, we propose a novel multi-view hypergraph learning with spatial-temporal periodic interests for next POI recommendation (MvStHgL). In the local view, we attempt to learn the POI representation of each interaction via jointing periodic characteristics of spatial and temporal aspects. In the global view, we design a hypergraph by regarding interactive sequences as hyperedges to capture high-order collaborative signals across users, for further POI representations. More specifically, the output of POI representations in the local view is used for the initialized embedding, and the aggregation and propagation in the hypergraph are performed by a novel Node-to-Hypergraph-to-Node scheme. Furthermore, the captured POI embeddings are applied to achieve sequential dependency modeling for next POI prediction. Extensive experiments on three real-world datasets demonstrate that our proposed model outperforms the state-of-the-art models.

Pre-trained natural language processing models on a large natural language corpus can naturally transfer learned knowledge to protein domains by fine-tuning specific in-domain tasks. However, few studies focused on enriching such protein language models by jointly learning protein properties from strongly-correlated protein tasks. Here we elaborately designed a multi-task learning (MTL) architecture, aiming to decipher implicit structural and evolutionary information from three sequence-level classification tasks for protein family, superfamily and fold. Considering the co-existing contextual relevance between human words and protein language, we employed BERT, pre-trained on a large natural language corpus, as our backbone to handle protein sequences. More importantly, the encoded knowledge obtained in the MTL stage can be well transferred to more fine-grained downstream tasks of TAPE. Experiments on structure- or evolution-related applications demonstrate that our approach outperforms many state-of-the-art Transformer-based protein models, especially in remote homology detection.