MOTIVATION: Mention-agnostic biomedical concept recognition (MA-BCR) requires inferring ontology concepts directly from passages, without relying on explicit mention spans. Prior work has mainly focused on generative and classification-based approaches. Ranking-based methods typically use a retrieve-rerank pipeline, and this paradigm has not been systematically studied for MA-BCR. Consequently, it remains unclear how ranking-based approaches compare with existing paradigms and what types of supervision are most beneficial for ranker training under limited annotation settings. RESULTS: Through a systematic comparison of ranking-, generative-, and classification-based paradigms, we show that a two-stage retrieve-rerank architecture is the most robust and scalable backbone for MA-BCR. Building on this finding, we propose ENR, an error-aware negative-enhanced ranking framework that augments training with false positives collected from heterogeneous recognizers, improving reranking performance without increasing inference-time cost. Experiments on MM-HPO and MM-GO (two datasets derived from MedMentions-ST21pv) demonstrate that ENR substantially outperforms prior approaches. AVAILABILITY AND IMPLEMENTATION: The code and data underlying this article are available in Github at https://github.com/sl-633/enr-recognizer or in Zenodo at https://doi.org/10.5281/zenodo.20730803.

Recent advancements in large language models (LLMs) have enabled their application across various domains. However, in the field of patent translation, Transformer encoder-decoder based models remain the standard approach, and the potential of LLMs for translation tasks has not been thoroughly explored. In this study, we conducted patent claim translation using an LLM fine-tuned with parallel data through continual pre-training and supervised fine-tuning, following the methodology proposed by Guo et al. (2024) and Kondo et al. (2024). Comparative evaluation against the Transformer encoder-decoder based translations revealed that the LLM achieved high scores for both BLEU and COMET. This demonstrated improvements in addressing issues such as omissions and repetitions. Nonetheless, hallucination errors, which were not observed in the traditional models, occurred in some cases and negatively affected the translation quality. This study highlights the promise of LLMs for patent translation while identifying the challenges that warrant further investigation.

In patent documents, patent claims represent a particularly important section as they define the scope of the claims. However, due to the length and unique formatting of these sentences, neural machine translation (NMT) systems are prone to translation errors, such as omissions and repetitions. To address these challenges, this study proposes a translation method that first segments the source sentences into multiple shorter clauses using a clause segmentation model tailored to facilitate translation. These segmented clauses are then translated using a clause translation model specialized for clause-level translation. Finally, the translated clauses are rearranged and edited into the final translation using a reordering and editing model. In addition, this study proposes a method for constructing clause-level parallel corpora required for training the clause segmentation and clause translation models. This method leverages word alignment tools to create clause-level data from sentence-level parallel corpora. Experimental results demonstrate that the proposed method achieves statistically significant improvements in BLEU scores compared to conventional NMT models. Furthermore, for sentences where conventional NMT models exhibit omissions and repetitions, the proposed method effectively suppresses these errors, enabling more accurate translations.

In this paper, we filter negative comments on videos using large language models (LLMs). In comment filtering using LLMs such as GPT-4 (zero-shot learning), over filtering occurs frequently. This over filtering can be improved by using few-shot learning based on chain-of-thought reasoning. Few-shot learning is a strategy to improve the performance of LLMs by presenting a small number of examples to LLMs. Chain-of-thought (CoT) [18] is one of the reasoning strategy which allows LLMs to engage in more logical reasoning by generating the process of inference. In this paper, we propose and evaluate an effective method of few-shot learning based on CoT in the task of negative comment filtering. The results showed that over filtering can be effectively reduced by selecting appropriate few-shot comments from the clusters created by GPT-4, and providing them for the prompt as demonstrations of CoT.

We propose a method that utilizes a large language model in the task of lyrics emotion classification. We especially employ GPT-4o, which is expected to deliver high performance to conduct emotion classification of lyrics, where we propose a few-shot prompt of GPT-4o for classification into 6 classes. We developed a dataset of 181 lyrics categorized into six classifications to evaluate GPT-4O’s performance. This dataset maintains a reasonable level of validity as it was carefully curated by the first author, then independently reclassified by six annotators, with final classifications determined through majority voting. Performing detailed classification, we achieved over 75% classification performance for the total accuracy. This accuracy was achieved through performing extractive summarization of the lyrics into optimal number of characters. In other words, rather than feeding the collected lyrics directly into GPT-4o, we implemented a two-stage process where GPT-4o first automatically summarizes the lyrics before they are used. Using a confusion matrix, we analyzed the tendencies of classification errors and showed the possibility of improving the total accuracy.

Retrieve-edit-rerank is a text generation framework composed of three steps: retrieving for sentences using the input sentence as a query, generating multiple output sentence candidates, and selecting the final output sentence from these candidates. This simple approach has outperformed other existing and more complex methods. This paper focuses on the retrieving and the reranking steps. In the retrieving step, we propose retrieving similar target language sentences from a target language monolingual translation memory using language-independent sentence embeddings generated by mSBERT or LaBSE. We demonstrate that this approach significantly outperforms existing methods that use monolingual inter-sentence similarity measures such as edit distance, which is only applicable to a parallel translation memory. In the reranking step, we propose a new reranking score for selecting the best sentences, which considers both the log-likelihood of each candidate and the sentence embeddings based similarity between the input and the candidate. We evaluated the proposed method for English-to-Japanese translation on the ASPEC and English-to-French translation on the EU Bookshop Corpus (EUBC). The proposed method significantly exceeded the baseline in BLEU score, especially observing a 1.4-point improvement in the EUBC dataset over the original Retrieve-Edit-Rerank method.

Recently, there has been a growing interest in pretraining models in the field of natural language processing. As opposed to training models from scratch, pretrained models have been shown to produce superior results in low-resource translation tasks. In this paper, we introduced the use of pretrained seq2seq models for preordering and translation tasks. We utilized manual word alignment data and mBERT-based generated word alignment data for training preordering and compared the effectiveness of various types of mT5 and mBART models for preordering. For the translation task, we chose mBART as our baseline model and evaluated several input manners. Our approach was evaluated on the Asian Language Treebank dataset, consisting of 20,000 parallel data in Japanese, English and Hindi, where Japanese is either on the source or target side. We also used in-house 3,000 parallel data in Chinese and Japanese. The results indicated that mT5-large trained with manual word alignment achieved a preordering performance exceeding 0.9 RIBES score on Ja-En and Ja-Zh pairs. Moreover, our proposed approach significantly outperformed the baseline model in most translation directions of Ja-En, Ja-Zh, and Ja-Hi pairs in at least one of BLEU/COMET scores.

This paper describes an automatic fluency evaluation of spontaneous speech. Although we regularly observe a variety of different disfluencies in spontaneous speech, we focus on two types of phenomena, i.e., filled pauses and word fragments. This paper aims to reveal that these two types of disfluencies have effects on speech fluency evaluation differently. To this end, we conduct a series of SVM classification experiments on the Japanese spontaneous speech corpus. The experimental results show that the features derived from word fragments are effective in evaluating disfluent speech especially when combined with prosodic features such as speech rate and pauses/silence, while the features from filled pauses are not effective in evaluating fluency.

This paper describes an automatic fluency evaluation of spontaneous speech. In the task of automatic fluency evaluation, we integrate diverse features of acoustics, prosody, and disfluency-based ones. Then, we attempt to reveal the contribution of each of those diverse features to the task of automatic fluency evaluation. Although a variety of different disfluencies are observed regularly in spontaneous speech, we focus on two types of phenomena, i.e., filled pauses and word fragments. The experimental results demonstrate that the disfluency-based features derived from word fragments and filled pauses are effective relative to evaluating fluent/disfluent speech, especially when combined with prosodic features, e.g., such as speech rate and pauses/silence. Next, we employed an LSTM based framework in order to integrate the disfluency-based and prosodic features with time sequential acoustic features. The experimental evaluation results of those integrated diverse features indicate that time sequential acoustic features contribute to improving the model with disfluency-based and prosodic features when detecting fluent speech, but not when detecting disfluent speech. Furthermore, when detecting disfluent speech, the model without time sequential acoustic features performs best even without word fragments features, but only with filled pauses and prosodic features.

This paper proposes techniques of automatically discovering tips Web sites from a large collection of Web pages using a topic model and support vector machine (SVM). Tips refer to practical knowledge or expertise that is used to help accomplish certain tasks in a particular field. We designed several approaches of extracting features with respect to domain names based on their distribution among Web pages and candidate tips Web sites. In addition, search engine suggests, the query keywords used to fetch Web pages from the search engine are also considered to present patterns that can be potential features. It was discovered from our dataset that domain names of tips Web sites (Web sites containing tips on a certain specific theme) are more likely dispersed among topics and Web pages. These domain names also tend to correspond to a larger number of search engine suggests. This paper verifies such observed patterns by training an SVM using those extracted features. Evaluation is performed in precision and recall to measure correctness of classifying whether or not a domain name belongs to a tips Web site.

This paper proposes a method of assisting movie summarization using plotinformation. A plot of a movie available at Wikipedia contains a majorstory of the movie. From such a plot of a movie, we extract severalimportant sentences as the content of summary. For summarizing movie, the key work is finding the best alignment between sentences of plot andshots which are segmented from a movie. There are two cues used tomeasure the similarity between a sentence and a shot. One is based oncharacter appearing in both sentence and shot, another is based on wordsmatching. Then an alignment method based on dynamic programming is applyto optimize the alignment. Finally an experiment on movie "RomanHoliday" and "Alice in the wonderland" show the effectiveness of this method.

The background of this paper is the issue of how to overview the knowledge of a given query keyword. Especially, we focus on concerns of those who search for Web pages with a given query keyword. The Web search information needs of a given query keyword is collected through search engine suggests. Given a query keyword, we collect up to around 1,000 suggests, while many of them are redundant. We cluster redundant search engine suggests based on a topic model. However, one limitation of the topic model based clustering of search engine suggests is that the granularity of the topics, i.e., the clusters of search engine suggests, is too coarse. In order to overcome the problem of the coarse-grained clusters of search engine suggests, this paper further applies the word embedding technique to the Web pages used during the training of the topic model, in addition to the text data of the whole Japanese version of Wikipedia. Then, we examine the word embedding based similarity between search engines suggests and further classify search engine suggests within a single topic into finer-grained subtopics based on the similarity of word embeddings. Evaluation results prove that the proposed approach performs well in the task of subtopic clustering of search engine suggests.

While playing the communication game “Are You a Werewolf”, a player always guesses other players' roles through discussions, based on one's own role and other players' crucial utterances. The underlying goal of this paper is to construct an agent that can analyze the participating players' utterances and play the werewolf game as if it is a human. For the first step of this underlying goal, given a specific player participating in the wolf game, this paper studies how to generate a digest of inferring other players' roles from the viewpoint of the given specific player. In this inference process, we regard the werewolf game rules as well as certain common sense as inference rules. Then, we develop a set of inference rules and apply them to infer the participating players' roles from a real werewolf game log.

Pointing behavior in childhood is typical developmental sign having strong correlation to his or her language development. This paper focuses on the pointing behavior accompanied by utterance during picture book reading. With this respect, we make use of picture books' review data amounting to approximately 320 thousand, and analyze the reviews reflecting the pointing behavior with children's utterance. The results show patterns of the pointing with utterance change corresponding to children's developmental stages. Also, one of the pointing patterns is found to have strong relationship with a certain type of picture books.

We organized a machine translation (MT) task at the Seventh NTCIR Workshop. Participating groups were requested to machine translate sentences in patent documents and also search topics for retrieving patent documents across languages. We analyzed the relationship between the accuracy of MT and its effects on the retrieval accuracy.

This paper presents speech-driven Web retrieval models which accepts spoken search topics (queries) in the NTCIR-3 Web retrieval task. We experimentally evaluate the techniques of combining outputs of multiple LVCSR models with a language model(LM) with a 60,000 vocabulary size in recognition of spoken queries. As model combination techniques, we use the SVM learning. We show that the techniques of multiple LVCSR model combination can achieve improvement both in speech recognition and retrieval accuracies in speech-driven text retrieval. Comparing with the retrieval accuracies when a LM with a 20,000/60,000 vocabulary size is used in LVCSRs, the LM that has larger size of the vocabulary improves also retrieval accuracies.

For many practical applications of speech recognition systems, it is quite desirable to have an estimate of confidence for each hypothesized word. Unlike previous works on confidence measures, we have proposed features for confidence measures that are extracted from outputs of more than one LVCSR models. For further analysis of the proposed confidence measure, this paper examines the correlation between each word's confidence and the word's features such as its part-of-speech and syllable length. We then apply SVM learning technique to the task of combining outputs of multiple LVCSR models, where, as features of SVM learning, information such as the pairs of the models which output the hypothesized word are useful for improving the word recognition rate. Experimental results show that the combination results achieve a relative word error reduction of up to 72 % against the best performing single model and that of up to 36 % against ROVER.

This paper studies speech-driven Web retrieval models which accepts spoken search topics (queries) in the NTCIR-3 Web retrieval task. The major focus of this paper is on improving speech recognition accuracy of spoken queries and then improving retrieval accuracy in speech-driven Web retrieval. We experimentally evaluate the techniques of combining outputs of multiple LVCSR models in recognition of spoken queries. As model combination techniques, we compare the SVM learning technique and conventional voting schemes such as ROVER. We show that the techniques of multiple LVCSR model combination can achieve improvement both in speech recognition and retrieval accuracies in speech-driven text retrieval. We also show that model combination by SVM learning outperforms conventional voting schemes both in speech recognition and retrieval accuracies.

In this paper, we propose a method for learning a classifier which combines outputs of more than one Japanese named entity extractors. The proposed combination method belongs to the family of stacked generalizers, which is in principle a technique of combining outputs of several classifiers at the first stage by learning a second stage classifier to combine those outputs at the first stage. Individual models to be combined are based on maximum entropy models, one of which always considers surrounding contexts of a fixed length, while the other considers those of variable lengths according to the number of constituent morphemes of named entities. As an algorithm for learning the second stage classifier, we employ a decision list learning method. Experimental evaluation shows that the proposed method achieves improvement over the best known results with Japanese named entity extractors based on maximum entropy models.

For many practical applications of speech recognition systems, it is quite desirable to have an estimate of confidence for each hypothesized word. Unlike previous works on confidence measures, this paper studies features for confidence measures that are extracted from outputs of more than one LVCSR models. More specifically, this paper experimentally evaluates the agreement among the outputs of multiple Japanese LVCSR models, with respect to whether it is effective as an estimate of confidence for each hypothesized word. The results of experimental evaluation show that the agreement between the outputs with two LVCSR models with different decoders and acoustic models can achieve quite reliable confidence. Furthermore, among various features of acoustic models based on Gaussian mixture HMMs, it is concluded that ones such as whether or not to have short pause models, as well as different units in HMMs (e.g., triphone model or syllable model) are the most effective in achieving highly reliable confidence.

For many practical applications of speech recognition systems, it is quite desirable to have an estimate of confidence for each hypothesized word. Unlike previous works on confidence measures, this paper studies features for confidence measures that are extracted from outputs of more than one LVCSR models. More specifically, this paper experimentally evaluates the agreement among the outputs of multiple Japanese LVCSR models, with respect to whether it is effective as an estimate of confidence for each hypothesized word. The results of experimental evaluation show that the agreement between the outputs with two acoustic models which have different units in HMMs, such as phonemes and syllables, can achieve quite reliable confidence. 1.

A sharable software repository for Japanese LVCSR (Large Vocabulary Continuous Speech Recognition) is introduced. It is designed as a baseline platform for research and developed by researchers of different academic institutes under a governmental support. The repository consists of a recognition engine (Julius), Japanese acoustic models and statistical language models as well as Japanese morphological analysis tools. These modules can be easily integrated and replaced under a plug-and-play framework, which makes it possible to fairly evaluate components and to develop specific application systems. Assessment of these modules and systems in a 20000-word dictation task is reported. The software repository is freely available to the public.

The project of Japanese LVCSR (Large Vocabulary Continuous Speech Recognition) platform is introduced. It is a collaboration of researchers of different academic institutes and intended to develop a sharable software repository of not only databases but also models and programs. The platform consists of a standard recognition engine, Japanese phone models and Japanese statistical language models. A set of Japanese phone HMMs are trained with ASJ (Acoustic Society of Japan) databases of 20K sentence utterances per each gender. Japanese word N-gram (2-gram and 3-gram) models are constructed with a corpus of Mainichi newspaper of four years. The recognition engine JULIUS is developed for assessment of both acoustic and language models. The modules are integrated as a Japanese LVCSR system and evaluated on 5000-word dictation task. The software repository is available to the public.

This paper describes a method for finding structural matching between parallel sentences of two languages, (such as Japanese and English). Parallel sentences are analyzed based on unification grammars, and structural matching is performed by making use of a similarity measure of word pairs in the two languages. Syntactic ambiguities are resolved simultaneously in the matching process. The results serve as a useful source for extracting linguistic and lexical knowledge.