Nihal Jain, Dejiao Zhang, Wasi Uddin Ahmad, Zijian Wang, Feng Nan, Xiaopeng Li, Ming Tan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Xiaofei Ma, Bing Xiang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Question answering over semi-structured tables has attracted significant attention in the NLP community. However, most of the existing work focus on questions that can be answered with short-form answer, i.e. the answer is often a table cell or aggregation of multiple cells. This can mismatch with the intents of users who want to ask more complex questions that require free-form answers such as explanations. To bridge the gap, most recently, pre-trained sequence-to-sequence language models such as T5 are used for generating free-form answers based on the question and table inputs. However, these pre-trained language models have weaker encoding abilities over table cells and schema. To mitigate this issue, in this work, we present an intermediate pre-training framework, Generation-focused Table-based Intermediate Pre-training (GENTAP), that jointly learns representations of natural language questions and tables. GENTAP learns to generate via two training objectives to enhance the question understanding and table representation abilities for complex questions. Based on experimental results, models that leverage GENTAP framework outperform the existing baselines on FETAQA benchmark. The pre-trained models are not only useful for free-form question answering, but also for few-shot data-to-text generation task, thus showing good transfer ability by obtaining new state-of-the-art results.

Large pretrained language models offer powerful generation capabilities, but cannot be reliably controlled at a sub-sentential level. We propose to make such fine-grained control possible in pretrained LMs by generating text directly from a semantic representation, Abstract Meaning Representation (AMR), which is augmented at the node level with syntactic control tags. We experiment with English-language generation of three modes of syntax relevant to the framing of a sentence - verb voice, verb tense, and realization of human entities - and demonstrate that they can be reliably controlled, even in settings that diverge drastically from the training distribution. These syntactic aspects contribute to how information is framed in text, something that is important for applications such as summarization which aim to highlight salient information.

Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, Bing Xiang. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Conversation structure is useful for both understanding the nature of conversation dynamics and for providing features for many downstream applications such as summarization of conversations. In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to. Previous work usually took a pair of utterances to decide whether one utterance is the parent of the other. We believe the entire ancestral history is a very important information source to make accurate prediction. Therefore, we design a novel masking mechanism to guide the ancestor flow, and leverage the transformer model to aggregate all ancestors to predict parent utterances. Our experiments are performed on the Reddit dataset (Zhang, Culbertson, and Paritosh 2017) and the Ubuntu IRC dataset (Kummerfeld et al. 2019). In addition, we also report experiments on a new larger corpus from the Reddit platform and release this dataset. We show that the proposed model, that takes into account the ancestral history of the conversation, significantly outperforms several strong baselines including the BERT model on all datasets.

Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

At present, with the rapid development of society, concrete has been widely used in various construction. For example, building construction, bridge construction and so on, almost all construction can not be separated from concrete. Therefore, the use of concrete is also profoundly related to the quality and safety of the building. In the process of using concrete, the temperature field of concrete has a great influence on concrete. In the process of concrete construction, cracks are likely to appear due to excessive temperature difference between inside and outside, which has a serious impact on building quality and safety. Therefore, it is particularly important to study the temperature field of concrete and effectively control its temperature. Thermodynamic parameters and boundary conditions have the greatest influence on the accuracy of concrete temperature control experimental calculation. However, obtaining these relevant parameters by means of traditional experiments requires high experimental cost, and is often inconsistent with the actual situation of concrete. Therefore, on the basis of simulated annealing algorithm, this paper makes relevant improvements. Through implementation, it can be found that the improved simulated annealing algorithm can be applied in the back analysis of concrete temperature field, and relevant Suggestions and measures can be put forward for the temperature control of concrete.

In view of the defects of traditional BP algorithm in neural network-based function fitting, such as low accuracy and slow convergence speed, this paper proposes an improved and optimized BP neural network algorithm (igsaa-bp) in the application of UHPC axial tension performance and crack width control ability prediction. The algorithm in high performance concrete (UHPC) axial tensile properties and fracture width control prediction is adopted in the control variable method, the four kinds of different fiber content of the shaft and characteristic parameters of high performance concrete and crack width control parameters for the comparison and analysis, based on the measured curve, puts forward the relationship between the stress - strain relationship and crack width - contains the basic tensile stress of three phase structure. The improved and optimized BP neural network algorithm can avoid falling into local optimization more effectively, and modify all individuals of the new population species according to the situation, so as to improve the global optimization ability of the algorithm and increase the diversity of individual population. The research results in this paper show that igsaa-bp algorithm not only improves the convergence speed of BP network, but also effectively improves the fitting ability of the network. When the steel fiber content increases from 2.3% to 3.5%, the axial tension performance strain increases by 230%, which indicates that the constitutive relation model of UHPC and the experimental data are in line with expectations.

We propose a novel neural topic model in the Wasserstein autoencoders (WAE) framework.Unlike existing variational autoencoder based models, we directly enforce Dirichlet prior on the latent document-topic vectors.We exploit the structure of the latent space and apply a suitable kernel in minimizing the Maximum Mean Discrepancy (MMD) to perform distribution matching.We discover that MMD performs much better than the Generative Adversarial Network (GAN) in matching high dimensional Dirichlet distribution.We further discover that incorporating randomness in the encoder output during training leads to significantly more coherent topics.To measure the diversity of the produced topics, we propose a simple topic uniqueness metric.Together with the widely used coherence measure NPMI, we offer a more wholistic evaluation of topic quality.Experiments on several real datasets show that our model produces significantly better topics than existing topic models.

We propose a novel adaptive approximation approach for test-time resource-constrained prediction motivated by Mobile, IoT, health, security and other applications, where constraints in the form of computation, communication, latency and feature acquisition costs arise. We learn an adaptive low-cost system by training a gating and prediction model that limits utilization of a high-cost model to hard input instances and gates easy-to-handle input instances to a low-cost model. Our method is based on adaptively approximating the high-cost model in regions where low-cost models suffice for making highly accurate predictions. We pose an empirical loss minimization problem with cost constraints to jointly train gating and prediction models. On a number of benchmark datasets our method outperforms state-of-the-art achieving higher accuracy for the same cost.

We point out an issue with one of the results in Bellala et al.[1] that invokes a main result on adaptive stochastic minimum cost cover problem (Theorem 5.8) of Golovin and Krause. We construct an example that shows that the proof of Theorem 5.8 of Golovin and Krause is invalid, and therefore, the proof in Bellala et al. about the near-optimum performance of their algorithm is also invalid.

We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program with linear constraints that encourages feature re-use. We establish total unimodularity of the constraint set to prove that the corresponding LP relaxation solves the original integer program. We then exploit connections to combinatorial optimization and develop an efficient primal-dual algorithm, scalable to large datasets. In contrast to our bottom-up approach, which benefits from good RF initialization, conventional methods are top-down acquiring features based on their utility value and is generally intractable, requiring heuristics. Empirically, our pruning algorithm outperforms existing state-of-the-art resource-constrained algorithms.

We seek decision rules for \it prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified \it average feature acquisition budget. While random forests yield strong generalization performance, they do not explicitly account for feature costs and furthermore require low correlation among trees, which amplifies costs. Our random forest grows trees with low acquisition cost and high strength based on greedy minimax cost-weighted-impurity splits. Theoretically, we establish near-optimal acquisition cost guarantees for our algorithm. Empirically, on a number of benchmark datasets we demonstrate competitive accuracy-cost curves against state-of-the-art prediction-time algorithms.

We present a novel classification algorithm for learning with test time budgets. In this setting, the goal is to reduce feature acquisition cost while maintaining classification accuracy. For every decision, our approach dynamically selects features based on previously observed information. Once a desired confidence of a decision is achieved, the acquisition stops and the test instance is classified. Our approach can be used in conjunction with many popular margin based classification algorithms. We use margin information from training data in the partial feature neighborhood of a test point to compute a probability of correct classification. This estimate is used to either select the next feature or to stop. We compare our algorithm to other cost-sensitive methods on real world datasets. The experiments demonstrate that our algorithm provides an accurate estimate of classification confidence and outperforms other approaches while being significantly more efficient in computation.

Most of the target tracking algorithms proposed for wireless sensor networks (WSNs) so far have been relying on sensors of single modality. To integrate multiple sensing modalities (e.g., by using the proximity sensors and ranging sensors together) to improve the tracking performance, the tracking algorithm shall be capable to deal with non-linear and non-Gaussian nature of the tracking problem. In this paper, we present a particle filter algorithm for multi-modality target tracking in WSNs. Simulation results show that the proposed algorithm can provide a good balance among sensor costs and tracking accuracy.