In this paper, we investigate the instability in the standard dense retrieval training, which iterates between model training and hard negative selection using the being-trained model.We show the catastrophic forgetting phenomena behind the training instability, where models learn and forget different negative groups during training iterations.We then propose ANCE-Tele, which accumulates momentum negatives from past iterations and approximates future iterations using lookahead negatives, as "teleportations" along the time axis to smooth the learning process.On web search and OpenQA, ANCE-Tele outperforms previous state-of-theart systems of similar size, eliminates the dependency on sparse retrieval negatives, and is competitive among systems using significantly more (50x) parameters.Our analysis demonstrates that teleportation negatives reduce catastrophic forgetting and improve convergence speed for dense retrieval training.

We present a new zero-shot dense retrieval (Ze-roDR) method, COCO-DR, to improve the generalization ability of dense retrieval by combating the distribution shifts between source training tasks and target scenarios.To mitigate the impact of document differences, COCO-DR continues pretraining the language model on the target corpora to adapt the model to target distributions via COtinuous COtrastive learning.To prepare for unseen target queries, COCO-DR leverages implicit Distributionally Robust Optimization (iDRO) to reweight samples from different source query clusters for improving model robustness over rare queries during fine-tuning.COCO-DR achieves superior average performance on BEIR, the zero-shot retrieval benchmark.At BERT Base scale, COCO-DR Base outperforms other ZeroDR models with 60× larger size.At BERT Large scale, COCO-DR Large outperforms the giant GPT-3 embedding model which has 500× more parameters.Our analysis show the correlation of COCO-DR's effectiveness in combating distribution shifts and improving zero-shot accuracy.Our code and model can be found at https://github.com/OpenMatch/COCO-DR.

Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Photo service providers are facing critical challenges of dealing with the huge amount of photo storage, typically in a magnitude of billions of photos, while ensuring national-wide or world-wide satisfactory user experiences. Distributed photo caching architecture is widely deployed to meet high performance expectations, where efficient still mysterious caching policies play essential roles. In this work, we present a comprehensive study on internet-scale photo caching algorithms in the case of QQPhoto from Tencent Inc., the largest social network service company in China. We unveil that even advanced cache algorithms can only perform at a similar level as simple baseline algorithms and there still exists a large performance gap between these cache algorithms and the theoretically optimal algorithm due to the complicated access behaviors in such a large multi-tenant environment. We then expound the reasons behind this phenomenon via extensively investigating the characteristics of QQPhoto workloads. Finally, in order to realistically further improve QQPhoto cache efficiency, we propose to incorporate a prefetcher in the cache stack based on the observed immediacy feature that is unique to the QQPhoto workload. The prefetcher proactively prefetches selected photos into cache before they are requested for the first time to eliminate compulsory misses and promote hit ratios. Our extensive evaluation results show that with appropriate prefetching we improve the cache hit ratio by up to 7.4 percent, while reducing the average access latency by 6.9 percent at a marginal cost of 4.14 percent backend network traffic compared to the original system that performs no prefetching.

Photo service providers are facing critical challenges of dealing with the huge amount of photo storage, typically in a magnitude of billions of photos, while ensuring national-wide or world-wide satisfactory user experiences. Distributed photo caching architecture is widely deployed to meet high performance expectations, where efficient still mysterious caching policies play essential roles. In this work, we present a comprehensive study on internet-scale photo caching algorithms in the case of QQPhoto from Tencent Inc., the largest social network service company in China. We unveil that even advanced cache algorithms can only perform at a similar level as simple baseline algorithms and there still exists a large performance gap between these cache algorithms and the theoretically optimal algorithm due to the complicated access behaviors in such a large multi-tenant environment. We then expound the behind reasons for that phenomenon via extensively investigating the characteristics of QQPhoto workloads. Finally, in order to realistically further improve QQPhoto cache efficiency, we propose to incorporate a prefetcher in the cache stack based on the observed immediacy feature that is unique to the QQPhoto workload. Evaluation results show that with appropriate prefetching we improve the cache hit ratio by up to 7.4%, while reducing the average access latency by 6.9% at a marginal cost of 4.14% backend network traffic compared to the original system that performs no prefetching.

China has overtaken India and the U.S. as host to the largest diabetic population in the world. Many problems exist in the Chinese healthcare system and very small number of diabetes patients receives treatment. Our paper reports on a case study through the lens of an online diabetes patient community, Sweet Home. We conducted participant observations, text analysis, and interviews, to understand the health management of patients at Sweet Home. Our findings reveal that patients' understanding of diabetes, their choice of treatments, their routine management, and their interactions with others (in the physical world) and among themselves (in the online world) are influenced by many factors: belief in traditional Chinese versus western medicine, cultural and social norms regarding social eating and drinking, conflicts over self-images, and responses to comments and pressures of coworkers. That is, social context may significantly affect patients' behaviors and each individual patient's actions may also help reshape the social context. We draw out implications for how our society as a whole may respond to these issues, from the perspective of public health, education, and information technology design.

The patient portal is a relatively new healthcare information technology that enables patients more convenient access to their healthcare information and allows them to send messages to their doctors. Our study examines the themes discussed in these messages and the different ways in which patients communicate with their providers via a portal employed in a large medical center. We also explore the differences between the patient portal and more traditional communication media, and investigated the advantages and potential problems of the portal system. Our findings show a wide variety of topics discussed in the communication messages (such as medication, appointments, laboratory tests, etc.) and how patients provide information, consult with their providers, and express psychosocial and emotional needs. We argue that the patient portal improves the accuracy of communication and could facilitate illness management for patients, especially over a longer term. However, messaging through the patient portal is not popular among patients and the simultaneous use of multiple communication media may create information gaps. More research is needed to better elucidate barriers to the use of patient portals and the optimal methods of communication and information integration given different contexts.