With the proliferation of large language models (LLMs) in the medical domain, there is increasing demand for improved evaluation techniques to assess their capabilities. However, traditional metrics like F1 and ROUGE, which rely on token overlaps to measure quality, significantly overlook the importance of medical terminology. While human evaluation tends to be more reliable, it can be very costly and may as well suffer from inaccuracies due to limits in human expertise and motivation. Although there are some evaluation methods based on LLMs, their usability in the medical field is limited due to their proprietary nature or lack of expertise. To tackle these challenges, we present AutoMedEval, an open-sourced automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs. The overarching objective of AutoMedEval is to assess the quality of responses produced by diverse models, aspiring to significantly reduce the dependence on human evaluation. Specifically, we propose a hierarchical training method involving curriculum instruction tuning and an iterative knowledge introspection mechanism, enabling AutoMedEval to acquire professional medical assessment capabilities with limited instructional data. Human evaluations indicate that AutoMedEval surpasses other baselines in terms of correlation with human judgments.

As multimodal large language models (MLLMs) gain prominence in the medical field, the need for precise evaluation methods to assess their effectiveness has become critical. While benchmarks provide a reliable means to evaluate the capabilities of MLLMs, traditional metrics like ROUGE and BLEU employed for open domain evaluation only focus on token overlap and may not align with human judgment. While human evaluation is more reliable, it is labor-intensive, costly, and not scalable. LLM-based evaluation methods have proven promising, but to date, there is still an urgent need for open-source multimodal LLM-based evaluators in the medical field. To address this issue, we introduce ACE-M^3, an open-sourced Automatic Capability Evaluator for Multimodal Medical Models that specifically designed to assess the question answering abilities of medical MLLMs. It first utilizes a branch-merge architecture to provide both detailed analysis and a concise final score based on standard medical evaluation criteria. Subsequently, a reward token-based direct preference optimization (RTDPO) strategy is incorporated to save training time without compromising performance of our model. Extensive experiments have demonstrated the effectiveness of our ACE-M^3 model in evaluating the capabilities of medical MLLMs.

Smart wearable devices (SWDs) collect and store sensitive daily information of many people. Its primary method of identification is still the password unlocking method. However, several studies have shown serious security flaws in that method, which makes the privacy and security concerns of SWDs particularly urgent. Gait identification is well suited for SWDs because its built-in sensors can provide data support for identification. However, existing gait identification methods have low accuracy and neglect to protect the privacy of gait features. In addition, the SWD can be used as an internet of things device for users to share data. But few studies have used gait feature-based encryption schemes to protect the privacy of message interactions between SWDs and other devices. In this paper, we propose a gait identification network, a bi-directional long short-term memory network with an attention mechanism (ABLSTM), to improve the identification accuracy and a stochastic orthogonal transformation (SOT) scheme to protect the extracted gait features from leakage. In the experiments, ABLSTM achieves an accuracy of 95.28%, reducing previous error rate by 19.3%. The SOT scheme is proved to be resistant to the chosen plaintext attack (CPA) and is 30% faster than previous methods. A biometric-based encryption scheme is proposed to enable secure message interactions using gait features as keys after the gait identification stage is passed, and offers better protection of the gait features compared to previous schemes.

In this article, we consider the partial quantum consensus problem of a qubit network in a distributed view. The local quantum operation is designed based on the Hamiltonian by using the local information of each quantum system in a network of qubits. We construct the unitary transformation for each quantum system to achieve the partial quantum consensus, that is, the directions of the quantum states in the Bloch ball will reach an agreement. A simple case of two-qubit quantum systems is considered first, and a minimum completing time of reaching partial consensus is obtained based on the geometric configuration of each qubit. Furthermore, we extend the approaches to deal with the more general N -qubit networks. Two partial quantum consensus protocols, based on the Lyapunov method for chain graphs and the geometry method for connected graphs, are proposed. The geometry method can be utilized to deal with more general connected graphs, while for the Lyapunov method, the global consensus can be obtained. The numerical simulation over a qubit network is demonstrated to verify the validity and the effectiveness of the theoretical results.

Modern recommender systems are increasingly expected to provide informative explanations that enable users to understand the reason for particular recommendations. However, previous methods struggle to interpret the input IDs of user--item pairs in real-world datasets, failing to extract adequate characteristics for controllable generation. To address this issue, we propose disentangled conditional variational autoencoders (CVAEs) for explainable recommendation, which leverage disentangled latent preference factors and guide the explanation generation with the refined condition of CVAEs via a self-regularization contrastive learning loss. Extensive experiments demonstrate that our method generates high-quality explanations and achieves new state-of-the-art results in diverse domains.

Link prediction is of fundamental importance in network science and machine learning. Early methods consider only simple topological features, while subsequent supervised approaches typically rely on human-labeled data and feature engineering. In this work, we present a new representation learning-based approach called SEMAC that jointly exploits fine-grained node features as well as the overall graph topology. In contrast to the SGNS or SVD methods espoused in previous representation-based studies, our model represents nodes in terms of subgraph embeddings acquired via a form of convex matrix completion to iteratively reduce the rank, and thereby, more effectively eliminate noise in the representation. Thus, subgraph embeddings and convex matrix completion are elegantly integrated into a novel link prediction framework. Experimental results on several datasets show the effectiveness of our method compared to previous work.

Recurrent neural networks (RNNs) have enjoyed great success in speech recognition, natural language processing, etc. Many variants of RNNs have been proposed, including vanilla RNNs, LSTMs, and GRUs. However, current architectures are not particularly adept at dealing with tasks involving multi-faceted contents. In this work, we solve this problem by proposing Multiple-Weight RNNs and LSTMs, which rely on multiple weight matrices in an attempt to mimic the human ability of switching between contexts. We present a framework for adapting RNN-based models and analyze the properties of this approach. Our detailed experimental results show that our model outperforms previous work across a range of different tasks and datasets.

Morphological segmentation, which aims to break words into meaning-bearing morphemes, is an important task in natural language processing. Most previous work relies heavily on linguistic preprocessing. In this paper, we instead propose novel neural network architectures that learn the structure of input sequences directly from raw input words and are subsequently able to predict morphological boundaries. Our architectures rely on Long Short Term Memory (LSTM) units to accomplish this, but exploit windows of characters to capture more contextual information. Experiments on multiple languages confirm the effectiveness of our models on this task.

Relation classification is a crucial ingredient in numerous information extraction systems seeking to mine structured facts from text.We propose a novel convolutional neural network architecture for this task, relying on two levels of attention in order to better discern patterns in heterogeneous contexts.This architecture enables endto-end learning from task-specific labeled data, forgoing the need for external knowledge such as explicit dependency structures.Experiments show that our model outperforms previous state-of-the-art methods, including those relying on much richer forms of prior knowledge.

Linlin Wang, Kang Liu, Zhu Cao, Jun Zhao, Gerard de Melo. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Bagging (Breiman 1996) and its variants is one of the most popular methods in aggregating classifiers and regressors. Originally, its analysis assumed that the bootstraps are built from an unlimited, independent source of samples, therefore we call this form of bagging ideal-bagging. However in the real world, base predictors are trained on data subsampled from a limited number of training samples and thus they behave very differently. We analyze the effect of intersections between bootstraps, obtained by subsampling, to train different base predictors. Most importantly, we provide an alternative subsampling method called design-bagging based on a new construction of combinatorial designs, and prove it universally better than bagging. Methodologically, we succeed at this level of generality because we compare the prediction accuracy of bagging and design-bagging relative to the accuracy ideal-bagging. This finds potential applications in more involved bagging-based methods. Our analytical results are backed up by experiments on classification and regression settings.