Leveraging artificial intelligence (AI) techniques in medical applications is helping our world to deal with the shortage of healthcare workers and improve the efficiency and productivity of healthcare delivery. Intelligent pre-consultation (IPC) is a relatively new application deployed on mobile terminals for collecting patient's information before a face-to-face consultation. It takes advantages of state-of-the-art machine learning techniques to assist doctors on clinical decision-making. One of key functions of IPC is to detect medical symptoms from patient queries. By extracting symptoms from patient queries, IPC is able to collect more information on patient's health status by asking symptom-related questions. All collected information will be summarized as a medical record for doctors to make clinical decision. This saves a great deal of time for both doctors and patients. Detecting symptoms from patient's query is challenging, as most patients lack medical background and often tend to use colloquial language to describe their symptoms. In this work, we formulate symptom detection as a retrieval problem and propose a bi-directional hard-negative enforced noise contrastive estimation method (Bi-hardNCE) to tackle the symptom detection problem. Bi-hardNCE has both forward contrastive estimation and backward contrastive estimation, which forces model to distinguish the true symptom from negative symptoms and meanwhile distinguish true query from negative queries. To include more informative negatives, our Bi-hardNCE adopts a hard-negative mining strategy and a false-negative eliminating strategy, which achieved a significant improvement on performance. Our proposed model outperforms commonly used retrieval models by a large margin.

Junjie Luo, Xi Chen, Jichao Sun, Yuejia Xiang, Ningyu Zhang, Xiang Wan. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

This article presents an efficient and totally unsupervised content-based similarity search method for multimedia data objects represented by high-dimensional feature vectors. The assumption is that the similarity measure is applicable to feature vectors of arbitrary length. During the offline process, different sets of features are selected by a generalized version of the Laplacian Score in an unsupervised way for individual data objects in the database. Online retrieval is performed by ranking the query object in the feature spaces of candidate objects. Those candidates for which the query object is ranked highly are selected as the query results. The ranking scheme is incorporated into an automated query expansion framework to further improve the semantic quality of the search result. Extensive experiments were conducted on several datasets to show the capability of the proposed method in boosting effectiveness without losing efficiency.

Neighborhood graphs are an essential component of many established methods for content-based image retrieval and automated image annotation. The performance of such methods relies heavily on the semantic quality of the graphs, which can be measured as the proportion of neighbors sharing the same class label as their query images. In this paper, we propose a new framework for the efficient construction of K-nearest neighbor (K-NN) graphs based on nearest-neighbor descent (NN-Descent), in which selective sparsification of object feature vectors is interleaved with neighborhood refinement operations in an effort to improve the semantic quality of the result. A local variant of the Laplacian Score is used to identify noisy features with respect to individual images, whose values are then set to 0 (the global mean value after standardization). We show through extensive experiments that our graph construction method is able to increase the proportion of semantically-related images over unrelated images within the neighbor sets.

In image applications, direct representations of images typically involve hundreds or thousands of features and not all the features are relevant for any given object. Errors introduced into similarity measurements by irrelevant or noisy features are detrimental to the semantic performance of content-based image retrieval. Feature selection techniques can be used to identify indiscriminative features from the entire image database. However, such global approach neglects the possibility that the feature importance may vary across different images or classes of images. We propose several techniques for the local selection of features for image databases. By checking the local neighborhood of each image, our methods determine the feature importance with respect to the image and select different feature sets for individual images. We also design methods based on the proposed local selection schemes for KNN graph construction, image search, and graph-based image annotation. We provide experimental results on two image datasets to demonstrate the effectiveness of our methods.

The practicality of large-scale image indexing and querying methods depends crucially upon the availability of semantic information. The manual tagging of images with semantic information is in general very labor intensive, and existing methods for automated image annotation may not always yield accurate results. The aim of this paper is to reduce to a minimum the amount of human intervention required in the semantic annotation of images, while preserving a high degree of accuracy. Ideally, only one copy of each object of interest would be labeled manually, and the labels would then be propagated automatically to all other occurrences of the objects in the database. To this end, we propose an influence propagation strategy, SW-KProp , that requires no human intervention beyond the initial labeling of a subset of the images. SW-KProp distributes semantic information within a similarity graph defined on all images in the database: each image iteratively transmits its current label information to its neighbors, and then readjusts its own label according to the combined influences of its neighbors. SW-KProp influence propagation can be efficiently performed by means of matrix computations, provided that pairwise similarities of images are available. We also propose a variant of SW-KProp which enhances the quality of the similarity graph by selecting a reduced feature set for each prelabeled image and rebuilding its neighborhood. The performances of the SW-KProp method and its variant were evaluated against several competing methods on classification tasks for three image datasets: a handwritten digit dataset, a face dataset and a web image dataset. For the digit images, SW-KProp and its variant performed consistently better than the other methods tested. For the face and web images, SW-KProp outperformed its competitors for the case when the number of prelabeled images was relatively small. The performance was seen to improve significantly when the feature selection strategy was applied.

This paper presents the prototype of a personal photo album manager that we build over three years. It integrates state of the art techniques of both database systems and computer vision to propose to publishers a privacy preserving sharable photo album manager where the publisher decides what each user is allowed to see. In addition, the photo album manager provides tools for an easy semantic annotation, classification and querying of the images.

The aim of this paper is to reduce to a minimum the level of human intervention in the semantic annotation process of images. Ideally, only one copy of each object of interest would be labeled manually, and the labels would then be propagated automatically to all other occurrences of the objects in the database. To that end, we propose a neighbor-based influence propagation approach KProp which builds a voting model and propagates the knowledge associated to some objects to similar objects. We show that KProp can perform efficiently through matrix computations and achieve better performance with fewer labeled examples per object.