We propose a Competitive Low-Rank Adaptation (ComLoRA) framework to address the limitations of the LoRA method, which either lacks capacity with a single rank-$r$ LoRA or risks inefficiency and overfitting with a larger rank-$Kr$ LoRA, where $K$ is an integer larger than 1. The proposed ComLoRA method initializes $K$ distinct LoRA components, each with rank $r$, and allows them to compete during training. This competition drives each LoRA component to outperform the others, improving overall model performance. The best-performing LoRA is selected based on validation metrics, ensuring that the final model outperforms a single rank-$r$ LoRA and matches the effectiveness of a larger rank-$Kr$ LoRA, all while avoiding extra computational overhead during inference. To the best of our knowledge, this is the first work to introduce and explore competitive learning in the context of LoRA optimization. The ComLoRA's code is available at https://github.com/hqsiswiliam/comlora.

We propose Hadamard High-Rank Adaptation (HiRA), a parameter-efficient fine-tuning (PEFT) method that enhances the adaptability of Large Language Models (LLMs). While Low-rank Adaptation (LoRA) is widely used to reduce resource demands, its low-rank updates may limit its expressiveness for new tasks. HiRA addresses this by using a Hadamard product to retain high-rank update parameters, improving the model capacity. Empirically, HiRA outperforms LoRA and its variants on several tasks, with extensive ablation studies validating its effectiveness. Our code is available at https://github.com/hqsiswiliam/hira.

Persona-based dialogue systems aim to generate consistent responses based on historical context and predefined persona. Unlike conventional dialogue generation, the persona-based dialogue needs to consider both dialogue context and persona, posing a challenge for coherent training. Specifically, this requires a delicate weight balance between context and persona. To achieve that, in this paper, we propose an effective framework with Persona-Adaptive Attention (PAA), which adaptively integrates the weights from the persona and context information via our designed attention. In addition, a dynamic masking mechanism is applied to the PAA to not only drop redundant information in context and persona but also serve as a regularization mechanism to avoid overfitting. Experimental results demonstrate the superiority of the proposed PAA framework compared to the strong baselines in both automatic and human evaluation. Moreover, the proposed PAA approach can perform equivalently well in a low-resource regime compared to models trained in a full-data setting, which achieve a similar result with only 20% to 30% of data compared to the larger models trained in the full-data setting. To fully exploit the effectiveness of our design, we designed several variants for handling the weighted information in different ways, showing the necessity and sufficiency of our weighting and masking designs.

Personalized dialogue generation, focusing on generating highly tailored responses by leveraging persona profiles and dialogue context, has gained significant attention in conversational AI applications.However, persona profiles, a prevalent setting in current personalized dialogue datasets, typically composed of merely four to five sentences, may not offer comprehensive descriptions of the persona about the agent, posing a challenge to generate truly personalized dialogues.To handle this problem, we propose Learning Retrieval Augmentation for Personalized DialOgue Generation (LAPDOG), which studies the potential of leveraging external knowledge for persona dialogue generation.Specifically, the proposed LAPDOG model consists of a story retriever and a dialogue generator.The story retriever uses a given persona profile as queries to retrieve relevant information from the story document, which serves as a supplementary context to augment the persona profile.The dialogue generator utilizes both the dialogue history and the augmented persona profile to generate personalized responses.For optimization, we adopt a joint training framework that collaboratively learns the story retriever and dialogue generator, where the story retriever is optimized towards desired ultimate metrics (e.g., BLEU) to retrieve content for the dialogue generator to generate personalized responses.Experiments conducted on the CONVAI2 dataset with ROCStory as a supplementary data source show that the proposed LAPDOG method substantially outperforms the baselines, indicating the effectiveness of the proposed method.The LAPDOG model code is publicly available for further exploration.

Audio captioning aims to generate text descriptions of audio clips.In the real world, many objects produce similar sounds.How to accurately recognize ambiguous sounds is a major challenge for audio captioning.In this work, inspired by inherent human multimodal perception, we propose visuallyaware audio captioning, which makes use of visual information to help the description of ambiguous sounding objects.Specifically, we introduce an off-the-shelf visual encoder to extract video features and incorporate the visual features into an audio captioning system.Furthermore, to better exploit complementary audio-visual contexts, we propose an audio-visual attention mechanism that adaptively integrates audio and visual context and removes the redundant information in the latent space.Experimental results on AudioCaps, the largest audio captioning dataset, show that our proposed method achieves state-of-theart results on machine translation metrics.

Punctuation is critical in understanding natural language text. Currently, most automatic speech recognition (ASR) systems do not generate punctuation, which affects the performance of downstream tasks, such as intent detection and slot filling. This gives rise to the need for punctuation restoration. Recent work in punctuation restoration heavily utilizes pre-trained language models without considering data imbalance when predicting punctuation classes. In this work, we address this problem by proposing a token-level supervised contrastive learning method that aims at maximizing the distance of representation of different punctuation marks in the embedding space. The result shows that training with token-level supervised contrastive learning obtains up to 3.2% absolute F1 improvement on the test set.

We propose a novel approach to generate a binary descriptor optimized for each image patch independently. The approach is inspired by the linear discriminant embedding that simultaneously increases inter and decreases intra class distances. A set of discriminative and uncorrelated binary tests is established from all possible tests in an offline training process. The patch adapted descriptors are then efficiently built online from a subset of features which lead to lower intra-class distances and thus, to a more robust descriptor. We perform experiments on three widely used benchmarks and demonstrate improvements in matching performance, and illustrate that per-patch optimization outperforms global optimization.

In this paper we propose a novel approach to generate a binary descriptor optimized for each image patch independently. The approach is inspired by the linear discriminant embedding that simultaneously increases inter and decreases intra class distances. A set of discriminative and uncorrelated binary tests is established from all possible tests in an offline training process. The patch adapted descriptors are then efficiently built online from a subset of tests which lead to lower intra class distances thus a more robust descriptor. A patch descriptor consists of two binary strings where one represents the results of the tests and the other indicates the subset of the patch-related robust tests that are used for calculating a masked Hamming distance. Our experiments on three different benchmarks demonstrate improvements in matching performance, and illustrate that per-patch optimization outperforms global optimization.

In recent years, the performance and accuracy of automatic license plate number recognition (ALPR) systems have greatly improved, however the increasing number of applications for such systems have made ALPR research more challenging than ever. The inherent computational complexity of search dependent algorithms remains a major problem for current ALPR systems. This paper proposes a novel search-free method of localization based on the estimation of saliency and local variance. Gabor functions are then used to validate the choice of candidate license plate. The algorithm was applied to three image datasets with different levels of complexity and the results compared with a number of benchmark methods, particularly in terms of speed. The proposed method outperforms the state of the art methods and can be used for real time applications.

Context provides additional information in detection and tracking and several works proposed online trained trackers that make use of the context. However, the context is usually considered during tracking as items with motion patterns significantly correlated with the target. We propose a new approach that exploits context in tracking-by-detection and makes use of persistent false positive detections. True detection as well as repeated false positives act as pointers to the location of the target. This is implemented with a generalised Hough voting and incorporated into a state-of-the art online learning framework. The proposed method presents good performance in both speed and accuracy and it improves the current state of the art results in a challenging benchmark.

With the highly increasing availability of text data on the Internet, the process of selecting an appropriate set of features for text classification becomes more important, for not only reducing the dimensionality of the feature space, but also for improving the classification performance. This paper proposes a novel feature selection approach to improve the performance of text classifier based on an integration of Ant Colony Optimization algorithm (ACO) and Trace Oriented Feature Analysis (TOFA). ACO is metaheuristic search algorithm derived by the study of foraging behavior of real ants, specifically the pheromone communication to find the shortest path to the food source. TOFA is a unified optimization framework developed to integrate and unify several state-of-the-art dimension reduction algorithms through optimization framework. It has been shown in previous research that ACO is one of the promising approaches for optimization and feature selection problems. TOFA is capable of dealing with large scale text data and can be applied to several text analysis applications such as text classification, clustering and retrieval. For classification performance yet effective, the proposed approach makes use of TOFA and classifier performance as heuristic information of ACO. The results on Reuters and Brown public datasets demonstrate the effectiveness of the proposed approach.

Hidden Markov Model (HMM) is a technique highly capable of modelling the structure of an observation sequence. In this paper, HMM is used to provide the contextual information for detecting clinical signs present in diabetic retinopathy screen images. However, there is a need to determine a feature set that best represents the complexity of the data as well as determine an optimal HMM. This paper addresses these problems by automatically selecting the best feature set while evolving the structure and obtaining the parameters of a Hidden Markov Model. This novel algorithm not only selects the best feature set, but also identifies the topology of the HMM, the optimal number of states, as well as the initial transition probabilities.

This paper describes a method of training an artificial neural network, specifically a multilayer perceptron (MLP), to act as a tool to help identify plants using morphological characters collected automatically from images of botanical herbarium specimens. A methodology is presented here to provide a practical way for taxonomists to use neural networks as automated identification tools, by collating results from a population of neural networks. A case study is provided using data extracted from specimens of the genus Tilia in the Herbarium of the Royal Botanic Gardens, Kew, UK. A classification accuracy of 44% was achieved on this challenging multiclass problem.

The demand for automatically recognizing and retrieving medical images for screening, reference, and management is growing faster than ever. In this paper, we present an intelligent content-based image retrieval system called I-Browse, which integrates both iconic and semantic content for histological image analysis. The I-Browse system combines low-level image processing technology with high-level semantic analysis of medical image content through different processing modules in the proposed system architecture. Similarity measures are proposed and their performance is evaluated. Furthermore, as a byproduct of semantic analysis, I-Browse allows textual annotations to be generated for unknown images. As an image browser, apart from retrieving images by image example, it also supports query by natural language.

The I-Browse project aimed to develop a prototype system to provide facilities for supporting intelligent retrieval of medical images through a combination of iconic and semantic content. The resulting prototype system, I-Browse, is able to extract and represent relevant iconic and semantic information from input images and to automatically generate textual annotations for images. Techniques were also developed for retrieving relevant images from the database given either an example image query or a textual query. The facilities provided by I-Browse were evaluated by medical colleagues and judged to have the potential of alleviating some of the time-consuming tasks that doctors now have to perform daily and providing a means of identifying previously unknown relationships between visual appearance and histological events.

We present a semantic content representation scheme and the associated techniques for supporting (a) query by image examples or by natural language in a histological image database and (b) automatic annotation generation for images through image semantic analysis. In this research, various types of query are analysed by either a semantic analyser or a natural language analyser to extract high level concepts and histological information, which are subsequently converted into an internal semantic-content representation structure code-named "Papillon". Papillon serves not only as an intermediate representation scheme but also stores the semantic content of the image that will be used to match against the semantic index structure within the image database during query processing. During the image database population phase, all images that are going to be put into the database will go through the same processing so that every image would have its semantic content represented by a Papillon structure. Since the Papillon structure for an image contains high level semantic information of the image, it forms the basis of the technique that automatically generates textual annotation for the input images. Papillon bridges the gap between different media in the database, allows complicated intelligent browsing to be carried out efficiently and also provides a well-defined semantic content representation scheme for different content processing engines developed for content-based retrieval.

Medical images are usually composed of different kinds of texture components which are always so much varied that a conventional single window approach cannot capture enough salient information for comparison. This paper applies the widely used multi-channel Gabor filters to demonstrate how a multi-window approach can improve the classification accuracy rate of histological labels. In addition, a most confident window method is proposed to further increase the accuracy rate of the multi-window approach.

A gastro-intestinal (GI) tract histological image is usually composed of texture components with different dimensions and properties. To analyze a histological image, we divide it into an array of sub-images. A feature vector comprising a set of Gabor filters and the intensity statistics is computed in order to classify each sub-image to one of 63 histological labels. To retrieve an image from the database, we compare three similarity measures, shape, neighbour and sub-image frequency distribution. It is found that both neighbour and sub-image frequency distribution similarity measures perform similarly well but the shape similarity measure yields the worst result when retrieving images of different GI tract organs. In general, the sub-image frequency distribution measure is the best choice because it requires less time to compute than the neighbour measure.

Reports a generic object-oriented framework for content-based image retrieval (CBIR) systems. It is designed so that the basic data structures and functionality of a typical CBIR system are provided without sacrificing speed and flexibility. The framework is based on a five-tier architecture that allows modules in different tiers to be developed independently, and thus flexibility is ensured. We show that our framework is able to adapt to a wide range of CBIR applications by applying the framework to the development of two on-going projects: a trademark image retrieval system and a medical (histological) image retrieval system. These applications are briefly discussed.