We present UniDetox, a universally applicable method designed to mitigate toxicity across various large language models (LLMs). Previous detoxification methods are typically model-specific, addressing only individual models or model families, and require careful hyperparameter tuning due to the trade-off between detoxification efficacy and language modeling performance. In contrast, UniDetox provides a detoxification technique that can be universally applied to a wide range of LLMs without the need for separate model-specific tuning. Specifically, we propose a novel and efficient dataset distillation technique for detoxification using contrastive decoding. This approach distills detoxifying representations in the form of synthetic text data, enabling universal detoxification of any LLM through fine-tuning with the distilled text. Our experiments demonstrate that the detoxifying text distilled from GPT-2 can effectively detoxify larger models, including OPT, Falcon, and LLaMA-2. Furthermore, UniDetox eliminates the need for separate hyperparameter tuning for each model, as a single hyperparameter configuration can be seamlessly applied across different models. Additionally, analysis of the detoxifying text reveals a reduction in politically biased content, providing insights into the attributes necessary for effective detoxification of LLMs.

Big data analysis is increasingly being used as a decision-making support tool for science and technology policy and technology management through the understanding of the trends and structure of innovation activities and the prediction of their changes. Since diversity and convergence are generally regarded as opportunities for groundbreaking innovation, analysis of academia-industry convergence, a type of fusion among entities, using big data analysis is expected to provide useful suggestions for policy formation and technology management. Nevertheless, little quantitative analysis has been conducted to date. With the development of academia-industry collaboration in a broad sense, it has become possible to observe a certain number of cases where researchers themselves bridge academia and industry with two faces, one in academia and the other in business. In this study, we focus on academic research conducted through academia-industry collaboration and researchers who conduct academia-industry collaboration themselves ("ambidextrous researchers") and clarify the magnitude and changes of their academic impact by analyzing article citation data. Specifically, using Scopus as a dataset, we extracted papers co-authored by academia-industry and "ambidextrous researchers" using our definition and analyzed their publication and citation numbers comprehensively and over time. The results showed, first, that the average number of citations of the academia-industry co-authored papers is higher than the overall average, indicating that academia-industry collaboration enhances the quality of academic research. On the other hand, it was found that, at least up to now, academia-industry collaboration is not necessarily a suitable environment for producing home-run papers, as we cannot find such papers at the top of the citation count. Second, for papers whose authors include "ambidextrous researchers" who belong to both academic institutions and companies, we found that the number of publications and the academic attention of papers whose authors include "ambidextrous researchers" is increasing over time, especially in the field of AI (artificial intelligence), where we can safely say that the attention is structurally higher. Third, as a system in which "ambidextrous researchers" act as intermediaries for continuous technology transfer to companies has been established, it was confirmed that companies, especially platform companies, have come to play a major role in advanced AI research. This is because research resources such as computational resources and data possessed by platforms are effective in further developing academic research and so are quite attractive for researchers at universities and other institutions, which can be an incentive for them to be "ambidextrous researchers."

Abstract This paper presents a novel unsupervised abstractive summarization method for opinionated texts. While the basic variational autoencoder-based models assume a unimodal Gaussian prior for the latent code of sentences, we alternate it with a recursive Gaussian mixture, where each mixture component corresponds to the latent code of a topic sentence and is mixed by a tree-structured topic distribution. By decoding each Gaussian component, we generate sentences with tree-structured topic guidance, where the root sentence conveys generic content, and the leaf sentences describe specific topics. Experimental results demonstrate that the generated topic sentences are appropriate as a summary of opinionated texts, which are more informative and cover more input contents than those generated by the recent unsupervised summarization model (Bražinskas et al., 2020). Furthermore, we demonstrate that the variance of latent Gaussians represents the granularity of sentences, analogous to Gaussian word embedding (Vilnis and McCallum, 2015).

This paper presents a tree-structured neural topic model, which has a topic distribution over a tree with an infinite number of branches.Our model parameterizes an unbounded ancestral and fraternal topic distribution by applying doubly-recurrent neural networks.With the help of autoencoding variational Bayes, our model improves data scalability and achieves competitive performance when inducing latent topics and tree structures, as compared to a prior tree-structured topic model (Blei et al., 2010).This work extends the tree-structured topic model such that it can be incorporated with neural models for downstream tasks.

This paper focuses on the end-to-end abstractive summarization of a single product review without supervision.We assume that a review can be described as a discourse tree, in which the summary is the root, and the child sentences explain their parent in detail.By recursively estimating a parent from its children, our model learns the latent discourse tree without an external parser and generates a concise summary.We also introduce an architecture that ranks the importance of each sentence on the tree to support summary generation focusing on the main review point.The experimental results demonstrate that our model is competitive with or outperforms other unsupervised approaches.In particular, for relatively long reviews, it achieves a competitive or better performance than supervised models.The induced tree shows that the child sentences provide additional information about their parent, and the generated summary abstracts the entire review.

The need for automatic document summarization that can be used for practical applications is increasing rapidly.In this paper, we propose a general framework for summarization that extracts sentences from a document using externally related information.Our work is aimed at single document summarization using small amounts of reference summaries.In particular, we address document summarization in the framework of multitask learning using curriculum learning for sentence extraction and document classification.The proposed framework enables us to obtain better feature representations to extract sentences from documents.We evaluate our proposed summarization method on two datasets: financial report and news corpus.Experimental results demonstrate that our summarizers achieve performance that is comparable to stateof-the-art systems.

Abstract In this article, we build models to predict the existence of citations among papers by formulating link prediction for 5 large‐scale datasets of citation networks. The supervised machine‐learning model is applied with 11 features. As a result, our learner performs very well, with the F1 values of between 0.74 and 0.82. Three features in particular, link‐based Jaccard coefficient difference in betweenness centrality, and cosine similarity of term frequency–inverse document frequency vectors, largely affect the predictions of citations. The results also indicate that different models are required for different types of research areas—research fields with a single issue or research fields with multiple issues. In the case of research fields with multiple issues, there are barriers among research fields because our results indicate that papers tend to be cited in each research field locally. Therefore, one must consider the typology of targeted research areas when building models for link prediction in citation networks.

As academic disciplines are segmented and specialized, it becomes more difficult to capture relevant research areas precisely by common retrieval strategies using either keywords or journal categories. This paper proposes a method of measuring the relatedness among sets of academic papers in order to detect unrelated communities which are not related to target topic. A citation network, extracted by given keywords, is divided into communities based on the density of links. We measured and compared four measures of relatedness between two communities in a citation network for three large-scale citation datasets. We used both link and semantic similarities. The topological distance from the center in a citation network is a more efficient measure for removing the unrelated communities than the other three measures: the ratio of the number of intercluster links over the all links, the ratio of the number of common terms over all terms, cosine similarity of tf-idf vectors.