3D Gaussians, as an explicit scene representation, typically involve thousands to millions of elements per scene. This makes it challenging to control the scene in ways that reflect the underlying semantics, where the number of independent entities is typically much smaller. Especially, if one wants to animate or edit objects in the scene, as this requires coordination among the many Gaussians involved in representing each object. To address this issue, we develop a mutual information shaping technique that enforces resonance and coordination between correlated Gaussians via a Gaussian attribute decoding network. Such correlations can be learned from putative 2D object masks in different views. By approximating the mutual information with the gradients concerning the network parameters, our method ensures consistency between scene elements and enables efficient scene editing by operating on network parameters rather than massive Gaussians. In particular, we develop an effective learning pipeline named ***InfoGS*** with lightweight optimization to shape the attribute decoding network ,while ensuring that the shaping (consistency) is maintained during continuous edits, avoiding re-shaping after parameter changes. Notably, our training only touches a small fraction of all Gaussians in the scene yet attains the desired correlated behavior according to the underlying scene structure. The proposed technique is evaluated on challenging scenes and demonstrates significant performance improvements in 3D object segmentation and promoting scene interactions, while inducing low computation and memory requirements. Our code is available at: https://github.com/StylesZhang/InfoGS.

Yunchao Zhang, Zonglin Di, Kaiwen Zhou, Cihang Xie, Xin Wang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

In recent years, graph pre-training has gained significant attention, focusing on acquiring transferable knowledge from unlabeled graph data to improve downstream performance. Despite these recent endeavors, the problem of negative transfer remains a major concern when utilizing graph pre-trained models to downstream tasks. Previous studies made great efforts on the issue of what to pre-train and how to pre-train by designing a variety of graph pre-training and fine-tuning strategies. However, there are cases where even the most advanced "pre-train and fine-tune" paradigms fail to yield distinct benefits. This paper introduces a generic framework W2PGNN to answer the crucial question of when to pre-train (.e., in what situations could we take advantage of graph pre-training) before performing effortful pre-training or fine-tuning. We start from a new perspective to explore the complex generative mechanisms from the pre-training data to downstream data. In particular, W2PGNN first fits the pre-training data into graphon bases, each element of graphon basis (i.e., a graphon) identifies a fundamental transferable pattern shared by a collection of pre-training graphs. All convex combinations of graphon bases give rise to a generator space, from which graphs generated form the solution space for those downstream data that can benefit from pre-training. In this manner, the feasibility of pre-training can be quantified as the generation probability of the downstream data from any generator in the generator space. W2PGNN offers three broad applications: providing the application scope of graph pre-trained models, quantifying the feasibility of pre-training, and assistance in selecting pre-training data to enhance downstream performance. We provide a theoretically sound solution for the first application and extensive empirical justifications for the latter two applications.

In recent years, prompt tuning has sparked a research surge in adapting pre-trained models. Unlike the unified pre-training strategy employed in the language field, the graph field exhibits diverse pre-training strategies, posing challenges in designing appropriate prompt-based tuning methods for graph neural networks. While some pioneering work has devised specialized prompting functions for models that employ edge prediction as their pre-training tasks, these methods are limited to specific pre-trained GNN models and lack broader applicability. In this paper, we introduce a universal prompt-based tuning method called Graph Prompt Feature (GPF) for pre-trained GNN models under any pre-training strategy. GPF operates on the input graph's feature space and can theoretically achieve an equivalent effect to any form of prompting function. Consequently, we no longer need to illustrate the prompting function corresponding to each pre-training strategy explicitly. Instead, we employ GPF to obtain the prompted graph for the downstream task in an adaptive manner. We provide rigorous derivations to demonstrate the universality of GPF and make guarantee of its effectiveness. The experimental results under various pre-training strategies indicate that our method performs better than fine-tuning, with an average improvement of about 1.4% in full-shot scenarios and about 3.2% in few-shot scenarios. Moreover, our method significantly outperforms existing specialized prompt-based tuning methods when applied to models utilizing the pre-training strategy they specialize in. These numerous advantages position our method as a compelling alternative to fine-tuning for downstream adaptations.

Manufacturing activities dominate the energy consumption and greenhouse emissions of the industrial sector. With the increasing concerns of greenhouse gas (GHG) emissions and climate change in recent years, the significance of the performance in terms of sustainability of manufacturing has been gradually recognized by both academia and industry. Various researches have been implemented to analyze, model, and reduce the energy consumption of manufacturing activities toward sustainable manufacturing. In a typical manufacturing system with multiple machines and buffers, the state of a certain machine is not only determined by the machine itself, but also the states of the adjacent machines and buffers. Therefore, machines may be in idle states due to nonincoming part from the upstream section of the manufacturing system or noncapacity to hold the delivered part to the downstream section of the manufacturing system. Those idle machines consume energy without production if there is no appropriate energy control strategy. In this article, we focus on the reduction of the energy waste for those idle machines in a typical multi-machine and multi-buffer manufacturing system. A Gaussian mixture model (GMM) integrated neural network is proposed to predict the duration of the idle periods for the idle machines, during which optimal energy control action can be identified and implemented under the constraint of production throughput of the manufacturing system. A manufacturing system simulator is built to provide the training dataset including the information, such as production throughput, energy consumption, buffer content, and failure rate, to the proposed neural network. A numerical case study for a five-machine-and-four-buffer manufacturing system is conducted to validate the effectiveness of the proposed prediction model in terms of the energy waste reduction for the idle machines.

Spatial representation learning (SRL) is to automatically learn feature representations that characterize spatial entities. In this paper, we study the problem of improving spatial representation learning using spatial structure knowledge. We consider two types of structure knowledge: (1) spatial autocorrelations refer to the pattern that similar spatial entities are more likely to share similar roles and configurations. (2) spatial complementarity refers to the effect that the role of a spatial entity can be complemented and augmented by other different yet compatible spatial entities. Along this line, we develop a step-by-step SRL framework to integrate spatial autocorrelations and complementarity. This framework includes four testable steps. First, we construct multi-view POI-POI(Point of Interest) graphs to characterize the static and dynamic patterns of each spatial region. We then use the graphs as inputs to train an adversarial autoencoder (AAE) that can preserve the spatial autocorrelation property and learn representations of spatial entities. Later, with the learned representations extracted from AAE inputs, a Graph Convolutional Network (GCN) is trained in an unsupervised fashion in order to overcome label sparsity and capture the spatial complementarity effect. In this way, we significantly improve the quality of spatial representations. In addition, we apply the proposed method to characterize residential communities for predicting real estate prices. Finally, we present intensive experimental results with real-world real estate data to demonstrate the proposed method effectiveness.

Unsupervised spatial representation learning aims to automatically identify effective features of geographic entities (i.e., regions) from unlabeled yet structural geographical data. Existing network embedding methods can partially address the problem by: (1) regarding a region as a node in order to reformulate the problem into node embedding; (2) regarding a region as a graph in order to reformulate the problem into graph embedding. However, these studies can be improved by preserving (1) intra-region geographic structures, which are represented by multiple spatial graphs, leading to a reformulation of collective learning from relational graphs; (2) inter-region spatial autocorrelations, which are represented by pairwise graph regularization, leading to a reformulation of adversarial learning. Moreover, field data in real systems are usually lack of labels, an unsupervised fashion helps practical deployments. Along these lines, we develop an unsupervised Collective Graph-regularized dual-Adversarial Learning (CGAL) framework for multi-view graph representation learning and also a Graph-regularized dual-Adversarial Learning (GAL) framework for single-view graph representation learning. Finally, our experimental results demonstrate the enhanced effectiveness of our method.