Large language models (LLMs) have garnered increasing attention owing to their powerful comprehension and generation capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful decision-making capability for new situations. Consequently, S-LLMs are helpless when it comes to continuous decision-making tasks that require logical reasoning. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, LD fine-tunes S-LLMs based on the function base to learn the logic employed by L-LLMs in decision-making. During testing, S-LLMs will yield decision-making outcomes, function by function, based on current states. Experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in continuous decision-making tasks, comparable to, or even surpassing, those of L-LLMs. The code and data for the proposed method are provided for research purposes https://github.com/Anfeather/Logic-Distillation.

The Heterogeneous Capacitated Vehicle Routing Problem (HCVRP) is an NP-hard combinatorial optimization problem. State-of-the-art neural solvers face difficulties in generalizing to large-scale scenarios after training on small-scale instances. Our experiments reveal that performance degradation is primarily due to the low-rank nature of attention matrix in large-scale instances. This results in insufficient distinction among node features, impacting the accuracy of Markov Decision Processes. Additionally, these models utilize self-attention for vehicle information interaction, but overly incorporate features from others, which suppresses individual features and leads to a deviation from the optimal route. To address these challenges, we propose the Rank-Augmented Neural Solver (RANS), which introduces two key innovations: 1) A simple yet effective mechanism to increase and approximate the upper bound of the attention matrix's rank, enabling the generation of more distinctive node features. 2) A Dual Cross-Attention Module within the vehicle encoder that accurately captures each vehicle's optimal routes while maintaining balanced vehicle collaboration. The experimental results show that RANS performs favorably against the baselines. Notably, when applied to instances with up to 10,000 nodes, RANS achieves an inference time that is merely 13.42% of the best baseline among the neural solvers, while simultaneously reducing the min-max travel time by 23.72%.

Dunhuang mural inpainting aims to fill in the missing regions of damaged murals with realistic content. Denoising probabilistic diffusion model (DDPM) has made great strides in semantic generation and shown promising results in image inpainting. However, three potential challenges prevent existing diffusion-based methods from restoring the Dunhuang murals: 1) effective visual information cannot be accurately extracted due to historical reasons, with most of the pixels being faded; 2) there are semantic discrepancy between damaged and visible regions in the inpainting results; and 3) the original structure and style of the damaged regions cannot be adequately restored. To this end, we propose a novel adversarial diffusion model for mural inpainting, which consists of: 1) a mural enhancement module named pixel-enhanced fire-controlled pulse-coupled neural network (PEFCPCNN), designed to enhance faded pixels to accurately extract the visual features of the mural; 2) a novel adversarial diffusion framework that optimizes the sampling prediction of mural over time steps; and 3) line drawing and different loss functions to constrain the reconstructed content to approximate the structure and style of original mural. The variational transform layer (VTL) and multi-scale contextual feature aggregation (MCFA) module are proposed to reconstruct content that is structurally coherent and texturally reasonable. Experiments on the Dunhuang mural dataset demonstrate that the proposed method outperforms state-of-the-art methods in terms of both the semantic reasonableness and global semantic consistency of inpainting content.

In 3D point cloud-based object detection, attention mechanism in Group-Free [1] learns direct relationships between proposals and all seed points, providing each proposal with a global context in the form of a cross-attention map. However, our analysis and experimental comparison show that the attention mechanism assigns inappropriately large attention weights to certain seed points far from a proposal, which is not conducive to detecting objects correctly. In this work, we alleviate the above problem by proposing a mask method. For an initial proposal, our method first calculates a spatial distance-based mask, which measures the spatial relationship between all seed points and the proposal. Then, we fuse the mask into cross-attention layers in stacked attention modules and get a refined cross-attention map. In essence, our mask gives each proposal a local context; after it is fused with the global context given by the attention mechanism, the refined cross-attention map could suppress the negative impact of some distant seed points on a proposal. We present two alternative strategies to compute the mask, a hard mask, and a soft mask. Experimental results demonstrate that the soft mask brings better performance. In the soft mask, for each initial proposal's 3D-box shape, we use a parametric approximate ellipsoid as the basis of the mask's calculation, which has only two learnable parameters. Experimental results show our work could outperform Group-Free 0.7 [email protected] at the cost of increasing inference time by less than 1%. The performance of our algorithm on the public dataset SUN RGB-D is 63.7 [email protected] and 45.5 [email protected], which is the best performance among algorithms that preserve the irregular of seed points.

Graph Convolutional Networks (GCNs) derive inspiration from recent advances in computer vision, by stacking layers of first-order filters followed by a nonlinear activation function to learn entity or graph embeddings. Although GCNs have been shown to boost the performance of many network analysis tasks, they still face tremendous challenges in learning from Heterogeneous Information Networks (HINs), where relations play a decisive role in knowledge reasoning. What's more, there are multiaspect representations of entities in HINs, and a filter learned in one aspect do not necessarily apply to another. We address these challenges by proposing the Aspect-Aware Graph Attention Network (AGAT), a model that extends GCNs with alternative learnable filters to incorporate entity and relational information. Instead of focusing on learning the general entity embeddings, AGAT learns the adaptive entity embeddings based on prediction scenario. Experiments of link prediction and semi-supervised classification verify the effectiveness of our algorithm.

Exploring and establishing artificial neural networks with electrophysiological characteristics and high computational efficiency is a popular topic that has been explored for many years in the fields of pattern recognition and computer vision. Inspired by the working mechanism of the primary visual cortex, pulse-coupled neural networks (PCNNs) can exhibit the characteristics of synchronous oscillation, refractory period, and exponential decay. These characteristics empower the PCNN model to group pixels with similar spatiality and gray values and to process digital images without training. However, electrophysiological evidence shows that the neurons exhibit highly complex nonlinear dynamics when stimulated by external periodic signals. This chaos phenomenon, also known as the ‘butterfly effect,” cannot be explained by all PCNN models. In this work, we analyze the main obstacle preventing PCNN models from imitating a real primary visual cortex. We consider neuronal excitation as a stochastic process. We then propose a novel neural network of the primary visual cortex, called a continuous-coupled neural network (CCNN). Theoretical analysis indicates that the dynamic behavior of the CCNN is distinct from the PCNN. Numerical results show that the CCNN model exhibits periodic behavior under a DC stimulus, and exhibits chaotic behavior under an AC stimulus, which is consistent with the testing results of primary visual cortex neurons. Furthermore, the image and video processing mechanisms of the CCNN model are analyzed. For image processing tasks, this model encodes the pixel intensity as the frequency of output signals so that it can group pixels with similar gray values. This image processing method can reduce the local gray level difference of the image, and compensate for small local discontinuities in the image. For video processing tasks, the CCNN encodes changing pixels as non-periodic chaotic signals, and it encodes static pixels as periodic signals. It thusachieves the purpose of moving target object recognition by distinguishing the dynamic states corresponding to different neuron clusters in the video. Experimental results on image segmentation indicate that the CCNN model has better performance than the state-of-the-art of visual cortex neural network models.

Existing network embedding algorithms based on generative adversarial networks (GANs) improve the robustness of node embeddings by selecting high-quality negative samples with the generator to play against the discriminator. Since most of the negative samples can be easily discriminated from positive samples in graphs, their poor competitiveness weakens the function of the generator. Inspired by the sales skills in the market, in this article, we present tripartite adversarial training for network embeddings (TriATNE), a novel adversarial learning framework for learning stable and robust node embeddings. TriATNE consists of three players: 1) producer; 2) seller; and 3) customer. The producer strives to learn the representation of each sample (node pair), making it easy for the customer to differentiate between the positive and the negative, while the seller tries to confuse the customer by selecting realistic-looking samples. The customer, a biased evaluation metric, provides feedback for training the producer and the seller. To further enhance the robustness of node embedding, we model the customer as a two-layer neural network, where each unit in the hidden layer can be regarded as a customer with different preferences. TriATNE also plays against the producer by adjusting the weight of each customer. We test the performance of TriATNE on two common tasks: classification as well as link prediction. The experimental results on various publicly available datasets show that TriATNE can exploit the network structure well.

Network embeddings aim to learn representations of nodes in a network with both the first- and the high-order proximities preserved. The first-order proximity corresponds to network reconstruction, while the high-order proximity is in tune with network inference. Since the tradeoff between the two proximities varies on scenarios, we propose an adjustable network embedding (ANE) algorithm for adjusting the weight between the first- and the high-order proximities. ANE is based on two hypotheses: 1) nodes in closed triplets are more important than nodes in open triplets and 2) closed triplets with higher degrees are more important. In addition, we change the bidirectional sampling of Word2vec into directional sampling to preserve the frequency of node pairs in the training set. Three common tasks, network reconstruction, link prediction, and classification are conducted on various publicly available data sets to validate the abovementioned statements.

Minimax similarity stresses the connectedness of points via mediating elements rather than favoring high mutual similarity. The grouping principle yields superior clustering results when mining arbitrarily-shaped clusters in data. However, it is not robust against noises and outliers in the data. There are two main problems with the grouping principle: first, a single object that is far away from all other objects defines a separate cluster, and second, two connected clusters would be regarded as two parts of one cluster. In order to solve such problems, we propose robust minimum spanning tree (MST)-based clustering algorithm in this letter. First, we separate the connected objects by applying a density-based coarsening phase, resulting in a low-rank matrix in which the element denotes the supernode by combining a set of nodes. Then a greedy method is presented to partition those supernodes through working on the low-rank matrix. Instead of removing the longest edges from MST, our algorithm groups the data set based on the minimax similarity. Finally, the assignment of all data points can be achieved through their corresponding supernodes. Experimental results on many synthetic and real-world data sets show that our algorithm consistently outperforms compared clustering algorithms.