Large language models (LLMs) have demonstrated great potential for automating the evaluation of natural language generation.Previous frameworks of LLM-as-a-judge fall short in two ways: they either use zero-shot setting without consulting any human input, which leads to low alignment, or fine-tune LLMs on labeled data, which requires a non-trivial number of samples.Moreover, previous methods often provide little reasoning behind automated evaluations.In this paper, we propose HYPO-EVAL, Hypothesis-guided Evaluation framework, which first uses a small corpus of human evaluations to generate more detailed rubrics for human judgments and then incorporates a checklist-like approach to combine LLM's assigned scores on each decomposed dimension to acquire overall scores 1 .With only 30 human evaluations, HypoEval achieves stateof-the-art performance in alignment with both human rankings (Spearman correlation) and human scores (Pearson correlation), on average outperforming G-Eval by 11.86% and finetuned LLAMA-3.1-8B-INSTRUCT with at least 3 times more human evaluations by 11.95%.Furthermore, we conduct systematic studies to assess the robustness of HYPOEVAL, highlighting its effectiveness as a reliable and interpretable automated evaluation framework.

Large language models (LLMs) often incorporate multiple text chunks in their inputs to provide the necessary contexts. To speed up the prefill of the long LLM inputs, one can pre-compute the KV cache of a text and re-use the KV cache when the context is reused as the prefix of another LLM input. However, the reused text chunks are not always the input prefix, which makes precomputed KV caches not directly usable since they ignore the text's cross-attention with the preceding texts. Thus, the benefits of reusing KV caches remain largely unrealized.

As large language models (LLMs) take on complex tasks, their inputs are supplemented with longer contexts that incorporate domain knowledge. Yet using long contexts is challenging as nothing can be generated until the whole context is processed by the LLM. While the context-processing delay can be reduced by reusing the KV cache of a context across different inputs, fetching the KV cache, which contains large tensors, over the network can cause high extra network delays.

Patent Entity and Relation Extraction (PERE) aims to extract entities and entity-relation triples from unstructured patent texts. PERE is one of the fundamental tasks in patent text mining, providing crucial technical support for patent retrieval and technology opportunity discovery. Previous works struggle to capture the implicit semantic information hidden within overlapping triples, especially a large number of overlapping triples existing in patent texts. A Patent Entity and Relation Extraction model based on Context query and Axial attention is proposed, named PERE-CA. As for entity recognition, the text segment is regarded as candidate entity span and entity types are acquired by span classification. Subsequently, the semantic context related to an entity pair is calculated by a context query method. And the semantic context is integrated into entity pair representation. For relation extraction, axial attention is implemented to get the implicit semantic information among overlapping entity pairs. And then, the model outputs all valid entity-relation triples. Experimental results on the patent dataset TFH-2020 and the public dataset SciERC demonstrate that the implementation of context query and axial attention can effectively improve extraction performance.