In recent years, large language models (LLMs) have demonstrated outstanding capabilities in various tasks. However, LLMs also have various drawbacks, especially hallucination. Hallucination refers to the generation of content that does not align with the user input, contradicts previously generated content or world knowledge. Current research on hallucination mainly include knowledge retrieval, prompt engineering, training data improvement, reinforcement learning, etc. However, these methods do not involve different categories of hallucinations which is important on hallucination analysis, and make detailed investigation for the internal state of LLMs which indicates the direction on hallucination occurrence. Therefore, in our research, we introduce an attribution framework to trace the origins of hallucinations based on the internal signals of LLMs. To support this framework, we develop a new benchmark named RelQA-Cate, which includes eight categories of hallucinations for the answers generated by LLMs. After that, we present a novel Differential Penalty Decoding (DPD) strategy for reducing hallucinations through adjusting post-probabilities of each answer. We conduct a series of experiments and the performance on answer reliability has significant improvement, achieving 28.25% at most, which demonstrates the effectiveness of our proposed DPD and its generalization in mitigating hallucination in LLMs.

In medical research, clinical trials are pivotal. While prospective clinical research provides a systematic approach to collecting patient data, it grapples with challenges like long durations, increased costs, and most crucially, data scarcity. To address above-mentioned challenge, this paper introduces a novel approach: using cross-table generation to create relevant data. Unlike existing work focused on single-table operations, our method leverages data from multiple sources across various tables, integrating diverse data types and ensuring data consistency across multiple tables. We develop a new framework, MedTransTab, tailored for cross-table tabular data generation in the medical context. This framework extends our previous efforts and is built upon the newly constructed PMC-Struct, derived from an unstructured PMC-patient dataset. Our MedTransTab can generate high-quality patient records, synthesizing detailed biomedical information to align with real or simulated tables from multiple sources. The experiments show that the proposed method significantly improves performance in cross-table tasks. On the PMC-Struct-Plus dataset, we observe an average improvement of 28.85% in data generation and prediction. Similarly, on the Out-Of-Domain (OOD) dataset, there's an average improvement of 22.56%, indicating substantial progress in medical data analysis.