Numerous text classification tasks inherently possess hierarchical structures among classes, often overlooked in traditional classification paradigms. This study introduces novel approaches for hierarchical text classification using Large Language Models (LLMs), exploiting taxonomies to improve accuracy and traceability in a zero-shot setting. We propose two hierarchical classification methods, namely (i) single-path and (ii) path-traversal, which all leverage the hierarchical class structures inherent in the target classes (e.g., a bird is a type of animal that belongs to a species) and improve naïve hierarchical text classification from literature. We implement them as prompts for generative models such as OpenAI GPTs and benchmark them against discriminative language models (BERT and RoBERTa). We measure the classification performance (precision, recall, and F1-score) vs. computational efficiency (time and cost). Throughout the evaluations of the classification methods on two diverse datasets, namely ComFaSyn, containing mental health patients’ diary entries, and DBpedia, containing structured information extracted from Wikipedia, we observed that our methods, without any form of fine-tuning and few-shot examples, achieve comparable results to flat classification and existing methods from literature with minimal increases in the prompts and processing time.

The exponential growth of data production emphasizes the importance of database management systems (DBMS) for managing vast amounts of data. However, the complexity of writing Structured Query Language (SQL) queries requires a diverse range of skills, which can be a challenge for many users. Different approaches are proposed to address this challenge by aiding SQL users in mitigating their skill gaps. One of these approaches is to design recommendation systems that provide several suggestions to users for writing their next SQL queries. Despite the availability of such recommendation systems, they often have several limitations, such as lacking sequence-awareness, session-awareness, and context-awareness. In this paper, we propose TRANSQLATION, a session-aware and sequence-aware recommendation system that recommends the fragments of the subsequent SQL query in a user session. We demonstrate that TRANSQLATION outperforms existing works by achieving, on average, 22% more recommendation accuracy when having a large amount of data and is still effective even when training data is limited. We further demonstrate that considering contextual similarity is a critical aspect that can enhance the accuracy and relevance of recommendations in query recommendation systems.

In large software-intensive organizations, trouble reports (TRs) are heavily involved in reporting, analyzing, and resolving faults. Due to the scale of modern organizations and products, multiple people independently often identify faults, leading to duplicate TRs. To mitigate the additional manual effort to identify and resolve these duplicate TRs, prior work at Ericsson focused on developing a 2-stage BERT-based retrieval system for identifying similar TRs when provided a new fault observation. This approach, although powerful, struggled to generalize to out-of-domain TRs. In this paper, we evaluate several fine-tuning strategies to integrate domain knowledge further, notably telecommunications knowledge, into the BERT-based TR retrieval models to (i) attain better performance on duplicate TR retrieval/identification and (ii) improve model generalizability to out-of-domain TR data. We find that adding domain-specific data into the fine-tuning models led to improved results on both overall model performance and model generalizability.

Transformer models have great potential in Graph Representation Learning (GRL) for efficiently scaling the learning process on large datasets and solving many challenges presented in Graph Neural Networks, e.g., oversmoothing and suspended animation. To represent each node of a graph, Transformer models as input usually take a node together with the node context, i.e., a set of other nodes that serve as learning context for the target node. However, current GRL Transformer models mainly consider the graph topology when selecting the node context for each target node. In this work, we demonstrate the important role of node features in selecting the node context. Specifically, we propose a hybrid approach for selecting node context that considers both the graph topology and the semantic similarities between node features. Through the empirical evaluations, we show the advantages of our hybrid node context selection method for a downstream classification task on various datasets compared to selection methods that only consider graph topology or semantic similarities. The best classification accuracy improvements of our proposed hybrid methods over the baseline methods on each dataset range from 0.77% to 6.05%.

In the era of big data, considerable research focus is being put on designing efficient algorithms capable of learning and extracting high-level knowledge from ubiquitous data streams in an online fashion. While, most existing algorithms assume that data samples are drawn from a stationary distribution, several complex environments deal with data streams that are subject to change over time. Taking this aspect into consideration is an important step towards building truly aware and intelligent systems. In this paper, we propose GNG-A, an adaptive method for incremental unsupervised learning from evolving data streams experiencing various types of change. The proposed method maintains a continuously updated network (graph) of neurons by extending the Growing Neural Gas algorithm with three complementary mechanisms, allowing it to closely track both gradual and sudden changes in the data distribution. First, an adaptation mechanism handles local changes where the distribution is only non-stationary in some regions of the feature space. Second, an adaptive forgetting mechanism identifies and removes neurons that become irrelevant due to the evolving nature of the stream. Finally, a probabilistic evolution mechanism creates new neurons when there is a need to represent data in new regions of the feature space. The proposed method is demonstrated for anomaly and novelty detection in non-stationary environments. Results show that the method handles different data distributions and efficiently reacts to various types of change.