Autoregressive neural language models (LMs) generate a probability distribution over tokens at each time step given a prompt.In this work, we attempt to systematically understand the probability distributions that LMs can produce, showing that some distributions are significantly harder to elicit than others.Specifically, for any target next-token distribution over the vocabulary, we attempt to find a prompt that induces the LM to output a distribution as close as possible to the target, using either soft (Li and Liang, 2021) or hard (Wallace et al., 2019) gradient-based prompt tuning.We find that (1) in general, distributions with very low or very high entropy are easier to approximate than those with moderate entropy; (2) among distributions with the same entropy, those containing "outlier tokens" are easier to approximate; (3) target distributions generated by LMs-even LMs with different tokenizers-are easier to approximate than randomly chosen targets.These results offer insights into the expressiveness of LMs and the challenges of using them as probability distribution proposers.

As an important sub-field in clustering analysis, deep graph clustering is receiving more and more attention from academia and industry. The goal of deep graph clustering is to learn effective embeddings for all nodes in the graph. Such node embeddings can effectively perform clustering task and thus be extended to various real-world application scenarios. However, existing graph clustering methods usually focus on technical-level improvements and ignore data-level information augmentation. In fact, data augmentation on graph data can effectively improve the receptive field and feature richness of model, and its effectiveness has been verified in many deep learning fields. Based on this motivation, we proposed a simple structure enhanced method for graph clustering, called SEGC. Such method only needs to construct a simple deep clustering network at the technical-level to achieve better performance by performing data augmentation at the data-level. Specifically, we leverage the underexplored potential of node activeness to perform edge-increasing and edge-decreasing operations on the original graph data, thereby generating different views to enhance the model's learning ability and receptive field. Extensive experiments on multiple real-world datasets demonstrate the effectiveness of our method.

We investigate the problem of discovering approximate denial constraints (DCs), for finding DCs that hold with some exceptions to avoid overfitting real-life dirty data and facilitate data cleaning tasks. Different methods have been proposed to address the problem, by following the same framework consisting of two phases. In the first phase a structure called evidence set is built on the given instance, and in the second phase approximate DCs are found by leveraging the evidence set. In this paper, we present novel and more efficient techniques under the same framework. (1) We optimize the evidence set construction by first building a condensed structure called clue set and then transforming the clue set to the evidence set. The clue set is more memory-efficient than the evidence set and facilitates more efficient bit operations and better cache utilization, and the transformation cost is usually trivial. We further study parallel clue set construction with multiple threads. (2) Our solution to approximate DC discovery from the evidence set is a highly non-trivial extension of the evidence inversion method for exact DC discovery. (3) Using a host of datasets, we experimentally verify our approximate DC discovery approach is on average 8.2 and 7.5 times faster than the two state-of-the-art ones that also leverage parallelism, respectively, and our methods for the two phases are up to an order of magnitude and two orders of magnitude faster than the state-of-the-art methods, respectively.

We consider the degrees of freedom (DoF) of the 2 × 2 × 2 multiple-input multiple-output (MIMO) interference network with arbitrary antenna configuration and practical rank-deficiency. Different from existing researches on the 2 × 2 × 2 MIMO interference model, the antenna number can be arbitrarily different at the transmitters, relays and receivers simultaneously, and the ranks of the channel matrices can be arbitrarily different. The exact DoF sum of this scenario is obtained. For different configurations of antennas and ranks, the outer bound is proved mainly by the min-cut outer bound and the achievability is presented with effective schemes, which combine zero-forcing (ZF) scheme, ZF over broadcast channel, interference alignment in X channel, and aligned interference neutralization (AIN) to match the outer bound. The traditional AIN scheme is modified in this paper to maximize the utilization of signal space overlap. Some related earlier works can be seen as special cases of our research.