State-of-the-art translation Quality Estimation (QE) models are proven to be biased.More specifically, they over-rely on monolingual features while ignoring the bilingual semantic alignment.In this work, we propose a novel method to mitigate the bias of the QE model and improve estimation performance.Our method is based on the contrastive learning between clean and noisy sentence pairs.We first introduce noise to the target side of the parallel sentence pair, forming the negative samples.With the original parallel pairs as the positive sample, the QE model is contrastively trained to distinguish the positive samples from the negative ones.This objective is jointly trained with the regression-style quality estimation, so as to prevent the QE model from overfitting to monolingual features.Experiments on WMT QE evaluation datasets demonstrate that our method improves the estimation performance by a large margin while mitigating the bias 1 .

The accuracy of Automatic Speech Recognition (ASR) is critical to speech-based products, such as subtitling, speech translation, spoken dialogue. We aim to enhance ASR accuracy by correcting errors in ASR hypothesis with sequence-to-sequence (seq2seq) models. In this paper, we propose to boost Transformer-based ASR error correction by fusing the pre-trained BERT [1] in encoder and copying mechanism in decoder, which exploits externally well-learned token representation and copying correct tokens in ASR transcript respectively. In addition, we leverage Text-to-Speech (TTS) synthesized data and ASR 5-best hypotheses to augment the training data and make the data more diverse. We evaluate our approach on two internal test sets and two public ASR test sets. Experimental results show that the proposed approach decreases the average Character Error Rate (CER) from 9.36% to 7.30% compared with ASR hypothesis without correction, and outperforms Transformer-based model by a large margin.