Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encoder at large scales. We demonstrate that the scaling law is present in user representation learning areas, where the training error scales as a power-law with the amount of computation. Our Contrastive Learning User Encoder (CLUE), optimizes task-agnostic objectives, and the resulting user embeddings stretch our expectation of what is possible to do in various downstream tasks. CLUE also shows great transferability to other domains and companies, as performances on an online experiment shows significant improvements in Click-Through-Rate (CTR). Furthermore, we also investigate how the model performance is influenced by the scale factors, such as training data size, model capacity, sequence length, and batch size. Finally, we discuss the broader impacts of CLUE in general.

Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, Sang-Woo Lee. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Video streaming applications based on Peer-to-Peer (P2P) systems are popular for their scalability, which is hard to achieve with traditional client-server approaches. In particular, layered video streaming has been much-studied due to its ability to differentiate users' streaming qualities in heterogeneous user environments. Previous work, however, has shown that user misbehavior (e.g., free-riding and protocol deviation) poses a serious threat to P2P systems that are not equipped with proper incentive mechanisms. We propose a method to disincentivize such misbehavior. Our SVC-TChain is a layered P2P video streaming method based on scalable video coding (SVC), which uses the recently proposed T-Chain incentive mechanism to discourage free-riding. After introducing T-Chain, we present the first analytical framework to study SVC piece selection with multiple video layers, using it to efficiently choose SVC-TChain's optimal piece selection parameters and thus discourage deviations from the piece selection policy. Extensive experimental results show that SVC-TChain outperforms layered extensions of BiTos and Give-to-Get, two popular P2P video streaming approaches, both in the absence of user misbehavior and when some users misbehave.

In this paper, we propose a simple, distributed, but highly efficient fairness-enforcing incentive mechanism for cooperative computing. The proposed mechanism, called triangle chaining (T-Chain), enforces reciprocity to avoid the exploitable aspects of the schemes that allow free-riding. In T-Chain, symmetric key cryptography provides the basis for a lightweight, almost-fair exchange protocol, which is coupled with a pay-it-forward mechanism. This combination increases the opportunity for multi-lateral exchanges and further maximizes the resource utilization of participants, each of whom is assumed to operate solely for his or her own benefit. T-Chain also provides barrier-free entry to newcomers with flexible resource allocation, allowing them to immediately benefit, and, therefore, is suitable for dynamic environments with high churn (i.e., turnover). T-Chain is distributed and simple to implement, as no trusted third party is required to monitor or enforce the scheme, nor is there any reliance on reputation information or tokens.

As more devices gain Internet connectivity, more information needs to be exchanged between them. For instance, cloud servers might disseminate instructions to clients, or sensors in the Internet of Things might send measurements to each other. In such scenarios, information spreads faster when users have an incentive to contribute data to others. While many works have considered this problem in peer-to-peer scenarios, none have rigorously theorized the performance of different design choices for the incentive mechanisms. In particular, different designs have different ways of "bootstrapping" new users (distributing information to them) and preventing "free-riding" (receiving information without uploading any in return). We classify incentive mechanisms in terms of reciprocity-, altruism-, and reputation-based algorithms, and then analyze the performance of these three basic and three hybrid algorithms. We show that the algorithms lie along a tradeoff between fairness and efficiency, with altruism and reciprocity at the two extremes. The three hybrids all leverage their component algorithms to achieve similar efficiency. The reputation hybrids are the most fair and can nearly match altruism's bootstrapping speed, but only the reciprocity/reputation hybrid can match reciprocity's zero-tolerance for free-riding. It therefore yields better fairness and efficiency when free-riders are present. We validate these comparisons with extensive experimental results.

In this paper, we propose a simple, distributed, but highly efficient fairness-enforcing incentive mechanism for cooperative computing. The proposed incentive scheme, called Triangle Chaining (T-Chain), enforces reciprocity to minimize the exploitable aspects of other schemes that allow free-riding. In T-Chain, symmetric key cryptography provides the basis for a lightweight, almost-fair exchange protocol, which is coupled with a pay-it-forward mechanism. This combination increases the opportunity for multi-lateral exchanges and further maximizes the resource utilization of participants, each of whom is assumed to operate solely for his or her own benefit. T-Chain also provides barrier-free entry to newcomers with flexible resource allocation, providing them with immediate benefits, and therefore is suitable for dynamic environments with high churn (i.e., Turnover). TChain is distributed and simple to implement, as no trusted third party is required to monitor or enforce the scheme, nor is there any reliance on reputation information or tokens.

In P2P file sharing systems, free-riders who use others' resources without sharing their own cause system-wide performance degradation. Existing techniques to counter free-riders are either complex (and thus not widely deployed), or easy to bypass (and therefore not effective). This paper proposes a simple yet highly effective free-rider prevention scheme using (t, n) threshold secret sharing. A peer must upload encrypted file pieces to obtain the subkeys necessary to decrypt a file which has been downloaded, i.e., subkeys are swapped for file pieces. No centralized monitoring or control is required. This scheme is called “treat-before-trick” (TBeT). TBeT penalizes free-riding with increased file completion times (time to download file and necessary subkeys). TBeT counters known free-riding strategies, incentivizes peers to donate more upload bandwidth, and increases the overall system capacity for compliant peers. TBeT has been implemented as an extension to BitTorrent, and results of experimental evaluation are presented.