The continuous emergence of new entities, relations, triples, and multimodal information drives the dynamic evolution of multimodal knowledge graph (MMKG). However, existing MMKG embedding models follow a static setting, where training from scratch for growing MMKG wastes learned knowledge, while fine-tuning on new knowledge easily leads to catastrophic forgetting, severely limiting their applicability in real-world scenarios. To address this, we propose a multimodal continual representation learning framework (MoFot) for growing MMKG. Unlike existing static multimodal embedding methods, MoFot focuses on alleviating catastrophic forgetting rather than retraining to adapt to new knowledge. Specifically, MoFot effectively mitigates catastrophic forgetting caused by parameter updates and differing forgetting rates across modalities through a multimodal collaborative modulation mechanism. The mechanism ensures consistent retention of previously learned multimodal knowledge across snapshots through multimodal weight modulation and multimodal feature modulation. MoFot outperforms existing MMKG embedding, KG continual learning, and MMKG inductive models. Experimental results demonstrate that MoFot not only avoids forgetting but also enhances old knowledge by learning new knowledge, achieving adaptation to new knowledge while mitigating forgetting of old knowledge.

A reliable autonomous driving system requires a high-precision perception module. Collaborative perception is emerging as a web-scale information-sharing paradigm for autonomous driving, enabling multiple vehicles to collectively achieve a broader perception field than any single vehicle. However, existing approaches necessitate frequent one-to-many communication, which increases network load and leads to information redundancy. This paper presents Octopus, an innovative vehicle-to-road collaboration framework that leverages the computational capabilities of roadside units. Instead of frequent one-to-many communication, vehicles interact only with roadside units, which significantly reduces communication overhead and improves real-time processing efficiency. While this design alleviates communication burdens, vehicles may still struggle to achieve comprehensive situational awareness in highly dynamic environments. To further address this limitation, our framework incorporates global fusion results as prior knowledge, enabling closed-loop fusion to refine vehicle-side perception. Extensive experiments on OPV2V and V2V4Real datasets demonstrate that Octopus excels at collaborative perception, outperforming the state-of-the-art approach up to 11.58% on [email protected], 12.74% on [email protected] and 5514× reduction in communication volume.

• A simple and effective multi-site brain disease recognition framework based on tensor decomposition and personalized federated learning is proposed to quickly integrate samples from different hospitals/sites while enabling personalized feature extraction at each site. • A designed Dynamic Prototype Aggregation (DPA) module utilizes a sliding window technique to capture the intrinsic characteristics of time-varying BOLD signals. • A dual-feature aggregation module is designed to aggregate coarse-grained shared features and fine-grained prototype representation features, respectively, to facilitate efficient knowledge sharing among sites. Brain diseases significantly impact physical and mental health, making the development of models to identify biomarkers for early diagnosis essential. However, building high-quality models typically relies on large-scale datasets, while the privacy-sensitive nature of medical data often restricts its sharing and utilization. Multi-site studies provide a potential solution by integrating data from various sources, yet existing methods frequently neglect site-specific private features, such as demographic information. Therefore, in this paper, we propose a simple yet effective framework based on Tensor Decomposition and Personalized Federated Learning (TDPFL) for multi-site brain disease recognition, while protecting these private features. On the central server, we designed a dual feature aggregation module to facilitate efficient knowledge sharing among sites. On the client side, we introduced a personalized branch to safeguard private information ( i.e. , age, gender, and education) and developed a tensor decomposition module to extract features from subjects’ brain scan data. Furthermore, we developed a dynamic prototype aggregation module to monitor evolving brain features over time. This mechanism enhances the model’s capacity to capture these dynamics, thereby improving classification and prediction accuracy. Experiments on two publicly available rs-fMRI datasets across six sites showed that TDPFL outperformed baseline methods with a 4 % improvement in average classification accuracy. Additionally, we identified site-specific brain disease-related biomarkers, offering novel insights into early diagnosis. Code is available at https://github.com/ChaojunZ/TDPFL.git

Recently, most image segmentation methods exhibit an extreme trade-off between performance and efficiency, resulting in approaches with high performance typically having low computational efficiency, while efficient methods compromise on segmentation accuracy. To address this dual challenge, this study introduces a simple yet efficient segmentation framework based on Multi-scale Prototype matching and visual Sparse Attention mechanisms (MPSA), which is a transformer-based architecture designed to optimize the balance between performance and efficiency. The proposed MPSA integrates a novel lightweight cross-attention mechanism and prototype selection and filtering strategy to accurately correlate category queries with corresponding visual objects with a multi-scale Feature Pyramid Network (FPN). Within the pixel decoder, our Axial Convolution Enhanced (ACE) module mitigates lost global context by combining depth-wise separable convolutions with deformable convolutions, thereby recovering global semantics while preserving fine-grained spatial details. Through this innovative design, MPSA demonstrates outstanding performance in both semantic and panoptic segmentation tasks across multiple datasets. Remarkably, MPSA achieves surprising 83.9% mIoU with only 114M parameters on the Cityscapes dataset while compared to some state-of-the-art architectures, highlighting its ability to deliver exceptional results with significantly reduced resource consumption. Our code is released at https://github.com/zxqing01/MPSA.

Text-based visual question answering (TextVQA) aims to answer questions by understanding scene text in images. However, many current methods overly depend on the accuracy of Optical Character Recognition (OCR) systems, while overlooking the significance of visual objects. They tend to perform poorly when the question involves the relationships between visual objects and scene text. To address the above issues, we focus on raising the status of visual objects and innovatively propose a hierarchical semantic reasoning network (CT-HSR) based on the cascade transformer architecture, achieving fine-grained cross-modal reasoning and visual semantic enhancement. Specifically, the visual representations containing rich semantic information of the question modality are obtained through the cross-modal transformer-based vision-language pre-training model firstly. Then, the uni-modal transformer for unified modality encoding module is utilized to capture visual objects that are more semantically related to OCR texts. In addition, we further alleviate the cross-modal noise interference through the feature filtering strategy. Finally, we better align the three modalities by introducing TextVQA pre-training tasks and generate prediction answers through multi-step iterative prediction during fine-tuning. Extensive experiments on the TextVQA, ST-VQA, and OCR-VQA datasets have demonstrated the effectiveness of our proposed model compared to the state-of-the-art methods. The code will be released at https://github.com/FTFWO/CT-HSR .

Multimodal Knowledge Graphs (MMKG) models integrate multimodal contexts to improve link prediction performance. All existing MMKG models follow the transductive setting with a fixed predefined set, meaning that all the entities, relations, and multimodal information in the test graph are observed during training. This hinders their generalization to real-world MMKG with unseen entities and relations. Intuitively, a MMKG model trained on DBpedia cannot infer on Freebase. To address above limitations, we make the first attempt towards inductive learning for MMKG and propose a multimodal Inductive MMKG model (IndMKG) that is universal and transferable to any MMKG. Distinct from existing transductive methods, our model does not rely on specific trained embeddings; instead, IndMKG generates adaptive embeddings conditioned on any new MMKG via multimodal prototypes. Specifically, we construct class-adaptive prototypes to appropriately characterize the multimodal feature distribution of the given graph and equip IndMKG with robust adaptability to multimodal information across MMKGs. In addition, IndMKG learns non-specific structural embeddings based on meta relations. Such strategies tackle the challenge of notable multimodal feature discrepancies in cross-graph induction and allow the pre-trained IndMKG model to effectively zero-shot generalize to any MMKG. The strong performance in both inductive and transductive settings, across more than 20+ different scenarios, confirms the effectiveness and robustness of IndMKG. Our code is released at https://github.com/MMKGer/IndMKG/.

As a prevalent computing paradigm, graph computing provides an effective service strategy for the analysis of big data within the Internet of Everything (IoE), particularly for clustering graph-structured data in the IoE. Among various methods, graph contrastive clustering, which is devoted to revealing the intrinsic structure of graphs and efficiently grouping nodes into distinct clusters by contrasting positive-negative counterparts, has attracted widely attention in recent years. However, the existing methods seriously ignore the graph topology and node attributes when setting positive and negative sample pairs, which further leads to node semantic inconsistency. To this end, we design a novel tensor representation-based multiview contrastive graph representation learning framework, including adaptive data augmentation, high-confidence sample pairs construction, and a simple yet effective self-optimizing module guided by clustering objective function, to address issues of graph contrastive learning in ignoring complementary information among the topology and attributes. Specifically, by jointly modeling the graph structure and multiview node attributes, the new proposed clustering model can concurrently mine both hard positive and negative samples, and dynamically enhance the weight allocation of the hard samples during the learning process. Then leveraging the characteristics of graph-structured data, we incorporate a small subset of nodes with the highest similarity as additional positive samples to improve the discriminative power of the proposed model. Furthermore, a self-optimizing clustering module is introduced to enhance the algorithm’s performance. Experimental results on four commonly used IoE-related data sets validate that our proposed approach can achieve state-of-the-art clustering performance.

Ultra-short-term prediction of photovoltaic power generation is the prerequisite for real-time dispatch of the power system, in order to further improve the accuracy of ultra-short-term prediction of photovoltaic power, this paper proposes an ultra-short-term prediction method of photovoltaic power based on an improved Generalized Regression Neural Network(GRNN)-Long Short-Term Memory (LSTM) combination model. Firstly, in the GRNN model, an improved whale optimization algorithm based on grid constraints, quasi-reverse learning and local perturbations is proposed to solve the optimal smoothing factor of the model and improve the prediction accuracy. Secondly, in the LSTM model, a CNN convolutional layer and attention mechanism are introduced to better exploit temporal information and enhance the predictive capabilities for complex weather changes. Finally, a combination model threshold setting method based on an improved Dynamic Time Warping (DTW) algorithm is proposed. This method dynamically selects the best model's prediction results based on thresholds during different time periods, fully leveraging the advantages of each model to meet the diverse weather conditions in photovoltaic power prediction. Experimental results demonstrate that the improved GRNN model excels in predicting sunny weather power, the improved LSTM model excels in predicting complex weather power, and the combination model performs well across various meteorological conditions, providing a robust solution for photovoltaic power ultra-short-term prediction problems.

Link prediction aims to infer missing valid triplets to complete knowledge graphs, with recent inclusion of multimodal information to enrich entity representations. Existing methods project multimodal information into a unified embedding space or learn modality-specific features separately for later integration. However, performance was limited in such studies due to neglecting the modalities compatibility and conflict semantic carried by entities in valid and invalid triplets. In this paper, we aim at modeling inter-entity modality interactions and thus propose a novel Modality Circular fusion approach (MoCi), which interweaves multimodal contextual of entities. Firstly, unlike most methods in this task that directly fuse modalities, we design a triplets-prompt modality contrastive pre-training to align modality semantics beforehand. Moreover, we propose a modality circular fusion model using a simple yet efficient multilinear transformation strategy. This allows explicit inter-entity modality interactions, distinguishing it from methods confined to fuse within individual entities. To the best of our knowledge, MoCi presents one of the pioneering frameworks that tailored to grasp inter-entity modality semantics for better link prediction. Extensive experiments on seven datasets demonstrate our model yields SOTA performance, confirming the efficacy of MoCi in modeling inter-entity modality interactions. Our code is released at https://github.com/MoCiGitHub/MoCi.

The focus of this research lies in wireless gesture recognition, a prominent model of human-computer interaction that has garnered significant attention in recent years. This study aims to achieve efficient gesture recognition on smartphones using ultrasonic signals via speakers and microphones. Firstly, the original one-dimensional audio sequence is transformed into a two-dimensional spectrogram through the data preprocessing module, which facilitates capturing gesture patterns and changes while unifying the data format to enhance subsequent processing operability. Secondly, an algorithm for contour extraction is devised to mitigate signal interference caused by multipath effects. This algorithm enhances feature representation while reducing feature dimensionality and improving model robustness in adapting to interference and changes in various environments. By incorporating the spatial attention mechanism into the CNN model, it allows the model to focus on key areas, eliminate distracting information more effectively, better understand gesture features and shapes, and achieve more accurate recognition results. In testing basic number gestures, our method achieved an accuracy rate of 94%-96%, demonstrating its effectiveness. This would further enrich the spectrum of human-computer interaction, enhance user experiences, and drive the advancement of technological innovation.

The dual problem of image super‐resolution (SR), which is referred to as compact‐resolution (CR), and the corresponding image reconstruction are studied. These two problems have been studied independently by the researchers. In this study, a novel model for image CR and the corresponding reconstruction using the reversible network has been proposed. The reversible network has two properties, the first property, lossless information forwarding, which makes the compact‐resolved image retain more information from the original HR image. The second property, bidirectional mapping, by which the forward and reverse propagation of a reversible network can be utilised to implement image CR and reconstruction, respectively, i.e. using the reverse process of image CR to guide the reconstruction. In addition, the utilisation of a reversible network may reduce the size of the model. The superiority of the proposed model was demonstrated by comparing its performance with the state‐of‐the‐art methods on four well‐known benchmark datasets.

feature selection is a very important role in text classification. In this paper, a new idea is put forward, that is, the feature contribution degree to each category of text data set is taken into account in the feature selection and dimension reduction stage of text preprocessing, and then the features obtained by feature selection algorithm are further analyzed and filtered. In the end, the number of features that represent each category of the data set is identical. In this paper, a Naive Bayes classifier is used to verify this idea on three benchmark datasets: 20-newgroups, Reuters and WebKB. The experimental results show that the proposed method can effectively improve the accuracy of text classification.

Feature selection is often considered as a key step in text categorization. In this paper, we proposed a new feature selection algorithm, named AD, which comprehensively measures the degree of relevance and distinction of terms occur in document set. We evaluated AD on three benchmark document collections, 20-Newsgroups, Reuters-21578 and WebKB, using two classification algorithms, Naive Bayes and Support Vector Machines. The experimental results, comparing AD with six classic feature-selection algorithms, show that the proposed method AD is significantly superior to Information Gain, Mutual Information, Odds Ratio, DIA association factor, Orthogonal Centroid Feature Selection and Ambiguity Measure when Naive Bayes classifier is used and significantly outperforms IG,MI,OR,DIA,OCFS and AM when Support Vector Machines is used.

The most important characteristic of text categorization is the high dimensionality even for the moderate size dataset. Feature selection, which can reduce the size of the dimensionality without sacrificing the performance of the categorization and avoid over-fitting, is a commonly used approach in dimensionality reduction. In this paper, we proposed a new feature selection, which evaluates the deviation from the centroid based on both inter-category and intra-category. We compared the proposed method with four well-known feature selection algorithms using support vector machines on three benchmark datasets (20-newgroups, reuters-21578 and webkb). The experimental results show that the proposed method can significantly improve the performance of the classifier.