Conversational Question Answering (CQA) aims to provide natural language answers to users in information-seeking dialogues.Existing CQA benchmarks often evaluate models using pre-collected human-human conversations.However, replacing the model-predicted dialogue history with ground truth compromises the naturalness and sustainability of CQA evaluation.While previous studies proposed using predicted history and rewriting techniques to address unresolved coreferences and incoherencies, this approach renders the question self-contained from the conversation.In this paper, we propose a novel automatic evaluation approach, interview evaluation.Specifically, ChatGPT acts as the interviewer (Q agent) with a set of carefully designed prompts, and the CQA model under test serves as the interviewee (A agent).During the interview evaluation, questions are dynamically generated by the Q agent to guide the A agent in predicting the correct answer through an interactive process.We evaluated four different models on QuAC and two models on CoQA in our experiments.The experiment results demonstrate that our interview evaluation has advantages over previous CQA evaluation approaches, particularly in terms of naturalness and coherence.The source code is made publicly available.

We tackle Open-Domain Conversational Question Answering (abbr., ODCQA), a task of answering questions in multi-turn conversations by mining clues from a large passage collection. Recent progress in deep learning and large-scale Pre-trained Language Model (PLM) is driving fast-paced advances in ODCQA. However, the typical retriever-reranker-reader (3R) pipeline framework of the existing ODCQA systems suffers from a potential flaw, which is referred to the unavoidable interference from the reader to reranker. Briefly, there are a large number of shareable parameters between the reader and reranker because they share the same PLM-based encoder. During the gradient backpropagation, their directions of updating PLM parameters are most probably inconsistent, which leads to interference in seeking for the optimal solution. In addition, the recent retriever in 3R framework merely utilizes dense representation. Though, dense retrievers are generally weaker than sparse retrievers in lexical matching for rare entities. To address the aforementioned two issues, we first propose to utilize a weak reader to alleviate the interference, and explore three different methods to pursue the goal, including (1) masking gradient, (2) dropping inputs and (3) early stopping. Besides, we propose to hybridize a semantic-sensitive dense retriever and a keyword -sensitive sparse retriever, so as to enhance the robustness of retriever in dealing with rare entities. We conduct extensive experiments on the benchmark corpus OR-QUAC. Experimental results show that our approach outperforms the State-of-The-Art (SoTA) models, yeilding an improvement of 6.1 % F1-score.