As a fundamental problem in transfer learning, model selection aims to rank off-the-shelf pretrained models and select the most suitable one for the new target task. Existing model selection techniques are often constrained in their scope and tend to overlook the nuanced relationships between models and tasks. In this paper, we present a pragmatic framework Fennec, delving into a diverse, large-scale model repository while meticulously considering the intricate connections between tasks and models. The key insight is to map all models and historical tasks into a transfer-related subspace, where the distance between model vectors and task vectors represents the magnitude of transferability. A large vision model, as a proxy, infers a new task's representation in the transfer space, thereby circumventing the computational burden of extensive forward passes and reliance on labels. We also investigate the impact of the inherent inductive bias of models on transfer results and propose a novel method called archi2vec to encode the intricate structures of models. The transfer score is computed through straightforward vector arithmetic with a constant time complexity of O(k) per model inference (where k is the dimension of the latent space). Finally, we make a substantial contribution to the field by releasing a comprehensive benchmark that includes 105 different models. We validate the effectiveness of Fennec through rigorous testing on two benchmarks. Our framework achieves the best evaluation accuracy on both benchmarks. On the PARC benchmark, the feature extraction time is 6.5× faster than the baseline at the same level, and the inference time is 5.7× faster than the fastest baseline. On the larger benchmark, the performance gains are even more pronounced, with feature extraction time being 52.3× faster and inference time 1.8× faster than the fastest baseline, demonstrating remarkable performance efficiency. The code has been made publicly available at: https://github.com/Fay-why/Fennec.

In this paper, we present a novel transformer architecture tailored for learning robust power system state representations, which strives to optimize power dispatch for the power flow adjustment across different transmission sections. Specifically, our proposed approach, named Powerformer, develops a dedicated section-adaptive attention mechanism, separating itself from the self-attention employed in conventional transformers. This mechanism effectively integrates power system states with transmission section information, which facilitates the development of robust state representations. Furthermore, by considering the graph topology of power system and the electrical attributes of bus nodes, we introduce two customized strategies to further enhance the expressiveness: graph neural network propagation and multi-factor attention mechanism. Extensive evaluations are conducted on three power system scenarios, including the IEEE 118-bus system, a realistic China 300-bus system, and a large-scale European system with 9241 buses, where Powerformer demonstrates its superior performance over several popular baseline methods. The code is available at: https://github.com/Cra2yDavid/Powerformer

Abstract Personalized content synthesis technologies based on diffusion models have achieved significant breakthroughs, allowing for the creation of specific images from just a few reference photos. However, when these technologies are used to create fake news or unsettling content targeting individuals, they pose a substantial risk to society. To address this issue, current methods generate adversarial samples by adversarially maximizing the training loss, thus disrupting any personalized generation model trained with these samples. However, these methods are not efficient and do not fully consider the intrinsic mechanisms of successful personalization attacks. In this paper, we introduce an innovative Disruptive Text-Image Alignment (DTIA) framework. Based on the analysis that existing methods succeed in their attacks due to an overfitting of text to noise, which results in inaccurate face information matching, we have designed a Text-Image Mis-Match Attack framework. This framework aims to disrupt the model’s learning of associations between input faces and specific texts, thereby reducing unnecessary computational load. We also studied how the choice of timestep in diffusion models affects adversarial attacks and proposed a step schedule strategy to enhance algorithm efficiency. Extensive experiments on facial benchmarks have demonstrated that our DTIA framework not only disrupts personalized generation models but also significantly improves model efficiency.

Graph-level representation learning is the pivotal step for downstream tasks that operate on the whole graph. The most common approach to this problem is graph pooling, where node features are typically averaged or summed to obtain the graph representations. However, pooling operations like averaging or summing inevitably cause severe information missing, which may severely downgrade the final performance. In this paper, we argue what is crucial to graph-level downstream tasks includes not only the topological structure but also thedistributionfrom which nodes are sampled. Therefore, powered by existing Graph Neural Networks (GNN), we propose a new plug-and-play pooling module, termed asDistribution Knowledge Embedding(DKEPool), where graphs are viewed as distributions on top of GNNs and the pooling goal is to summarize the entire distribution information instead of retaining a certain feature vector by simple predefined pooling operations. A DKEPool networkde factodisassembles representation learning into two stages,structure learninganddistribution learning. Structure learning follows a recursive neighborhood aggregation scheme to update node features where structure information is obtained. Distribution learning, on the other hand, omits node interconnections and focuses more on the distribution depicted by all the nodes. Extensive experiments on graph classification tasks demonstrate that the proposed DKEPool significantly and consistently outperforms the state-of-the-art methods. The code is avaliable athttps://github.com/chenchkx/dkepool

Graph disentangling is a new promising direction that can help us to discover the latent patterns in the data and understand the behaviors of a graph learning model. Despite the many efforts in disentangling representation learning, few works focus on disentangling the latent factors behind a graph. Most current foci are mainly on studying node-level semantics in the graphs. Compared with node-level, the structure-level view can provide a new interpretable and in-depth insight into graph data. The study of structure-level relations enables us to reveal the high-order structural semantics in the data. To explore the complex high-order structural semantics in the data, we propose the High-order Structural Semantic Disentangled Neural Network (HSDN) to model the graph structure units and disentangle structural semantics. It's the first attempt to hypergraph disentangled networks. Unlike prior methods that disentangle factor graphs based on pair-wise relations only, we introduce hyperedges on pair-wise graphs to model structure units and disentangle the complex high-order structural semantics between different structures. Extensive experiments demonstrate that HSDN achieves state-of-the-art performances in terms of both disentangling and downstream tasks.

Temporal graphs exhibit dynamic interactions between nodes over continuous time, whose topologies evolve with time elapsing. The whole temporal neighborhood of nodes reveals the varying preferences of nodes. However, previous works usually generate dynamic representation with limited neighbors for simplicity, which results in both inferior performance and high latency of online inference. Therefore, in this paper, we propose a novel method of temporal graph convolution with the whole neighborhood, namely Temporal Aggregation and Propagation Graph Neural Networks (TAP-GNN). Specifically, we first analyze the computational complexity of the dynamic representation problem by unfolding the temporal graph in a message-passing paradigm. The expensive complexity motivates us to design the AP (aggregation and propagation) block, which significantly reduces the repeated computation of historical neighbors. The final TAP-GNN supports online inference in the graph stream scenario, which incorporates the temporal information into node embeddings with a temporal activation function and a projection layer besides several AP blocks. Experimental results on various real-life temporal networks show that our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency.