In recent years, there has been a rapid growth in the volume of textual data generated from various sources, including industries, news media, and social media, across various fields worldwide. It contains valuable information and knowledge, but its sheer volume requires effective summarization techniques to make it useful. Text summarization has thus become an important technology for distilling large amounts of information and capturing key insights. With advancements in large language models (LLMs), most existing abstractive summarization models leverage pretrained models such as BART. These models are based on Transformer architectures and undergo unsupervised pretraining, followed by fine-tuning on specific datasets for downstream tasks, particularly text summarization. These models have demonstrated superiority in this field. Real-world data often contains noise, and most pretrained language models suffer from exposure bias and a discrepancy between their training objectives and real evaluation metrics. In this paper, we introduce sentence-level data augmentation to simulate real-world text variations and enhance the model's denoising capability. Furthermore, contrastive learning is employed to learn to distinguish the differences between the original article with candidate summaries and among candidate summaries themselves. Finally, the highest-scoring article is output as the resulting summary. The experimental results on the CNN/DailyMail and XSum datasets indicate that this model shows improvement in generating text summaries compared to pretrained models and existing methods with contrastive learning. When comparing to the multitask learning framework BRIO variant with contrastive loss, our proposed method achieves a better ROUGE-1 of 48.52 and comparable ROUGE-2/L of 24.84 and 39.78. This shows the potential of our proposed method. Further investigation is needed to verify the performance in real-world texts in various domains.

Solving math word problems is a popular topic in natural language processing. We not only need to classify the grammatical structures in the questions, but also understand the mathematical logic expressed between words. Errors in semantic understanding may lead to the failure to generate correct solution equations. Thus, the correct answer cannot be calculated. Previous studies mainly used sequence-to-sequence recurrent neural networks (RNNs) to obtain meaning in words, or combined graph neural networks to capture more information in questions to achieve better results. In addition, recent studies also showed that, a tree-based decoder leads to better results than a decoder of RNNs. In this paper, we propose to combine transformers and tree-based decoders for solving math word problems. Firstly, we use a transformer encoder to read math word problems, whose outputs are given to two different decoders, including Transformer decoder, and a treebased decoder. Secondly, from the answer equations generated from the two decoders, the better solution is selected. The experimental results on the two commonly used math word problem datasets, MAWPS and ASDiv-A, show that our model achieves 89.3% and 81.9% accuracy, which are 2.2% and 4.2% higher than the vanilla transformer model, respectively. For the MAWPS dataset the performance is comparable to state-of-the-art model Graph2Tree. This shows the effectiveness of our proposed method.

Huge amount of user generated contents are being created in major social media everyday. It’s very difficult for users to quickly grasp the most important topics from such big data. Document clustering techniques are often used for topic detection from news, but it’s still challenging for social media. Firstly, since social media posts are usually very short, it’s hard to capture their semantic meanings. Secondly, given huge amount of social media posts, clustering effectiveness becomes unacceptable. Among agglomerative hierarchical clustering methods, centroid clustering is much more efficient, but with the issue of inversion or reversals. In this paper, we propose to detect popular topics from social media posts using dot-product similarity based centroid clustering of their word embeddings. Firstly, we extract keywords in posts with word segmentation, where documents are represented by word embedding of keywords. Secondly, various topics are extracted from the clustering results of social media posts by calculating dot-product similarity of their word embeddings. Finally, the popularity of each topic is estimated by the aggregate sentiment ratings from user replies. From the experimental results on PTT discussion forum, centroid clustering with dot-product similarity of word embeddings achieves better clustering efficiency with comparable effectiveness in terms of Adjusted Rand Index (ARI) and Adjusted Mutual Information (AMI). Further investigation is needed to verify the effectiveness in different social media sources.

With the development of Industrial Internet of Things (IIoT) technologies, there are more and more diverse smart devices and sensors connected in smart factories. Since these devices are only designed for connection with each other, they usually have very limited security mechanism. Also, due to the diverse behaviors for different devices, it would be difficult to design individual security mechanism manually. To detect potential threats on these devices, machine learning methods might be helpful to learn the diverse behaviors from their generated packets for identifying device types. In this paper, we propose a machine learning approach to automatic device identification and anomaly detection through network traffic analysis. First, we utilize both unsupervised and supervised learning for identifying different types of IoT devices. Second, based on the model learned from device identification module, we conduct feature selection to improve classification performance for anomaly detection. In our experiments on real data in a smart factory, the performance of device identification using supervised learning outperforms that of unsupervised learning. The best performance can be obtained for XGBoost with the best accuracy of 97.6% and micro-averaging F1 score of 97.6%. Also, in the emulated attacks on real devices, gradient boosted decision trees were found useful in anomaly detection, which gives an accuracy of 99.997% with the F1 score of 99.995%. This shows the potential of the proposed approach for anomaly detection in smart factories. Further investigation is needed to verify the proposed approach using more types of devices and network attacks.

K-Means algorithm is one of the most popular methods for flat clustering, but it's time-consuming in similarity calculation for big data, which causes lower performance in practice. Previous studies proposed improvements for finding better initial centroids to facilitate effective assignment of the data points to suitable clusters with reduced time complexity. However, in vector space representation, as the data volume increases, the dimension of vector space becomes higher which takes more time in similarity calculation. In this paper, we propose a SimHash-based K-Means clustering algorithm that used locality-sensitive hashing and dimensionality reduction to improve the efficiency in big data analytics. The experimental results showed that our proposed method greatly reduces the processing time of K-Means clustering without significantly affecting the effectiveness. Further investigation is needed to verify the performance for data in larger scale.

Huge numbers of documents are being generated on the Web, especially for news articles and social media. How to effectively organize these evolving documents so that readers can easily browse or search is a challenging task. Existing methods include classification, clustering, and chronological or geographical ordering, which only provides a partial view of the relations among news articles. To better utilize cross-document relations in organizing news articles, in this paper, we propose a novel approach to organize news archives by exploiting their near-duplicate relations. First, we use a sentence-level statistics-based approach to near-duplicate copy detection, which is language independent, simple but effective. Since content-based approaches are usually time consuming and not robust to term substitutions, near-duplicate detection approach can be used. Second, by extracting the cross-document relations in a block-sharing graph, we can derive a near-duplicate clustering by cross-document relations in which users can easily browse and find out unnecessary repetitions among documents. From the experimental results, we observed high efficiency and good accuracy of the proposed approach in detecting and clustering near-duplicate documents in news archives.

Social media present a user-friendly way of communication and sharing, which brings new chances to understand users and their social communication patterns. With the popularity of microblogging services, the huge volume of very short texts makes it difficult to track the latest updates or breaking news. In this paper, we propose a novel social influence model for estimating the popularity score for each short text in plurk. First, the degrees of user participation and user propagation are estimated by the number of replies, replurks, likes, and URIs. Then, we measure the influence persistence by the duration of the initial post and the last response, and the influence score can be derived from a linear combination of these simple statistics. Our experimental results on more than 300 thousand plurks collected from 1,750 users showed a good performance in determining popular messages, with the best F-measure of 0.86. From our case studies, top-ranked messages can accurately reflect the popular discussions on important events. This shows the effectiveness of our proposed approach. Further investigation of applying the influence model in event detection is needed.

Blog has become one of the most popular ways of communication among social communities since blog posts can be replied, commented, and even shared to other users in a convenient way. All posts and comments, no matter good or bad, have to be manually coordinated by blog owners. In order to prevent comment spam, most blog sites provide challenge-response tests such as CAPTCHA to ensure that the response is from human, instead of automatically generated by a computer. However, these tests cannot prohibit spammers from manually leaving spam messages. Existing studies of Chinese blog comment spam only focus on comments containing hyperlinks, which only stand for a small portion of blog comment spam. In this paper, we propose to include inter-comment Jaccard similarity in the features in addition to the post-comment similarity, stop words ratio, and comment length for blog comment classification. In order to verify the effects of inter-comment similarity features, we compared several classification algorithms such as C4.5, Naïve Bayes, and Neural Network. Experimental results showed that the feature combination of inter-comment and post-comment similarity under the classification of C4.5 achieves the best performance. This shows the effectiveness of the proposed inter-comment similarity feature for Chinese blog comment spam classification.

In online resources, such as news and weblogs, authors often extract articles, embed content, and comment on existing articles related to a popular event. Therefore, it is useful if authors can check whether two or more articles share common parts for further analysis, such as cocitation analysis and search result improvement. If articles do have parts in common, we say the content of such articles is event-relevant. Conventional text classification methods classify a complete document into categories, but they cannot represent the semantics precisely or extract meaningful event-relevant content. To resolve these problems, we propose a near-duplicate detection approach for finding event-relevant content in Web documents. The efficiency of the approach and the proposed duplicate set generation algorithms make it suitable for identifying event-relevant content. The experiment results demonstrate the potential of the proposed approach for use in weblogs.

Abstract Users' cross‐lingual queries to a digital library system might be short and the query terms may not be included in a common translation dictionary (unknown terms). In this article, the authors investigate the feasibility of exploiting the Web as the multilingual corpus source to translate unknown query terms for cross‐language information retrieval in digital libraries. They propose a Web‐based term translation approach to determine effective translations for unknown query terms by mining bilingual search‐result pages obtained from a real Web search engine. This approach can enhance the construction of a domain‐specific bilingual lexicon and bring multilingual support to a digital library that only has monolingual document collections. Very promising results have been obtained in generating effective translation equivalents for many unknown terms, including proper nouns, technical terms, and Web query terms, and in assisting bilingual lexicon construction for a real digital library system.

It is crucial for cross-language information retrieval (CLIR) systems to deal with the translation of unknown queries due to that real queries might be short. The purpose of this paper is to investigate the feasibility of exploiting the Web as the corpus source to translate unknown queries for CLIR. We propose an online translation approach to determine effective translations for unknown query terms via mining of bilingual search-result pages obtained from Web search engines. This approach can alleviate the problem of the lack of large bilingual corpora, translate many unknown query terms, provide flexible query specifications, and extract semantically-close translations to benefit CLIR tasks -- especially for cross-language Web search.