Job seekers often initiate search with short, underspecified queries. At LinkedIn, over 80% of job-related queries contain three or fewer keywords, making accurate user intent inference and relevant job retrieval particularly challenging. We present dynamic facet suggestion (DFS), an interactive query-refinement mechanism that facilitates intent disambiguation by surfacing personalized semantic attributes conditioned on the joint user-query context in real time. We propose a policy-grounded, retrieval-augmented ranking framework for facet suggestion, comprising offline taxonomy curation, embedding-based retrieval of top-K candidates, and a distilled small language model (SLM) based candidate scoring. The system is optimized for real-time serving via point-wise single-token scoring and batching/prefix caching. Offline evaluation demonstrates high precision for generated suggestions, and online A/B tests show significant lifts in suggestion engagement and job search outcomes.

Query understanding is essential in modern relevance systems, where user queries are often short, ambiguous, and highly context-dependent. Traditional approaches often rely on multiple task-specific Named Entity Recognition models to extract structured facets as seen in job search applications. However, this fragmented architecture is brittle, expensive to maintain, and slow to adapt to evolving taxonomies and language patterns. In this paper, we introduce a unified query understanding framework powered by a Large Language Model (LLM), designed to address these limitations. Our approach jointly models the user query and contextual signals such as profile attributes to generate structured interpretations that drive more accurate and personalized recommendations. The framework improves relevance quality in online A/B testing while significantly reducing system complexity and operational overhead. The results demonstrate that our solution provides a scalable and adaptable foundation for query understanding in dynamic web applications.

LinkedIn, one of the world's largest platforms for professional networking and job seeking, encounters various modeling challenges in building recommendation systems for its job matching product, including cold-start, filter bubbles, and biases affecting candidate-job matching. To address these, we developed the STAR (Signal integration for Talent And Recruiters) system, leveraging the combined strengths of Large Language Models (LLMs) and Graph Neural Networks (GNNs). LLMs excel at understanding textual data, such as member profiles and job postings, while GNNs capture intricate relationships and mitigate cold-start issues through network effects. STAR integrates diverse signals by uniting LLM and GNN capabilities with industrial-scale paradigms including adaptive sampling and version management. It provides an end-to-end solution for developing and deploying embeddings in large-scale recommender systems. Our key contributions include a robust methodology for building embeddings in industrial applications, a scalable GNN-LLM integration for high-performing recommendations, and practical insights for real-world model deployment.

We present LinkSAGE, an innovative framework that integrates Graph Neural Networks (GNNs) into large-scale personalized job matching systems, designed to address the complex dynamics of LinkedIn's extensive professional network. Our approach capitalizes on a novel job marketplace graph, the largest and most intricate of its kind in industry, with billions of nodes and edges. This graph is not merely extensive but also richly detailed, encompassing member and job nodes along with key attributes, thus creating an expansive and interwoven network. A key innovation in LinkSAGE is its training and serving methodology, which effectively combines inductive graph learning on a heterogeneous, evolving graph with an encoder-decoder GNN model. This methodology decouples the training of the GNN model from that of existing Deep Neural Network (DNN) models, eliminating the need for frequent GNN retraining while maintaining up-to-date graph signals in near real-time, allowing for the effective integration of GNN insights through transfer learning. The subsequent nearline inference system serves the GNN encoder within a real-world setting, significantly reducing online latency and obviating the need for costly real-time GNN infrastructure. Validated across multiple online A/B tests in diverse product scenarios, LinkSAGE demonstrates marked improvements in member engagement, relevance matching, and member retention, confirming its generalizability and practical impact.

Web-scale search and recommendation systems depend on efficient retrieval to manage massive datasets and user traffic.This paper chronicles our evolutionary path in building the retrieval layer at LinkedIn, progressing from a CPU-based inverted index system to a GPU-accelerated embedding-based retrieval system.Initially anchored by traditional term-based retrieval, we enhanced relevance and productivity through learning-to-retrieve approaches by generating mappings among inferred attributes.As these early efforts encountered limitations in inferring and matching attributes at scale, we transitioned to embedding-based retrieval for greater flexibility and performance, but found that existing infrastructure couldn't support large-scale production needs.This led us to develop a GPUbased retrieval system designed for high performance, flexible modeling, and multi-objective business optimization.We present the infrastructure innovations, optimizations, and key lessons learned throughout this transition, offering practical insights for building scalable, flexible retrieval systems.

Web-scale search systems typically tackle the scalability challenge with a two-step paradigm: retrieval and ranking. The retrieval step, also known as candidate selection, often involves extracting entities, creating an inverted index, and performing term matching for retrieval. Such traditional methods require manual and time-consuming development of retrieval models. In this paper, we propose a framework for constructing a graph that integrates human knowledge with user activity data analysis. The learned links are utilized for retrieval purposes. The model is easy to explain, debug, and tune. The system implementation is straightforward and can directly leverage existing inverted index systems. We applied this retrieval framework to enhance the job search and recommendation systems on a large professional networking portal, resulting in significant performance improvements.

Modern data storage systems are characterized by a distributed architecture as well as the presence of multiple storage tiers and caches. Both system developers and operators are challenged with the complexity of such systems as it is hard to evaluate how a configuration change will impact the workload or system performance and identify the best configuration to satisfy some performance objective. DITIS is a new simulator that models the end-to-end execution of file requests on distributed tiered storage systems that addresses the aforementioned challenges efficiently without any costly system redeployments. The demonstration will showcase the key functionalities and benefits offered by DITIS, including (i) analyzing workload traces to understand their characteristics and the behavior of the underlying storage system; (ii) running simulations with different configurations to evaluate their impact on performance; and (iii) running optimizations over custom search spaces to find the best configuration that satisfies a given objective.

The goal of Jobs Marketplace at LinkedIn is to match members to promoted job postings such that both job posters' ROI is optimized (amount of money spent per job clicks and applications) and the members are presented with relevant jobs that they are interested in and qualified for. This is achieved via a first-price auction mechanism where each job provides a bid for the member that comes to the job recommendations page. This bid depends on the match of the member to the job, as well as the daily budget that remains for the job, and its capability to spend it via clicks (e.g. some jobs might have more demand and have it easier to spend their budgets via clicks than others). In such a scheme, budget pacing, i.e. the capability of a job to spend its daily budget evenly, or according to a preset plan, is extremely important towards efficient utilization of its budget via reaching a higher number of candidates, and obey a variety of spending plans optimizing for different events such as clicks and applications.

The field of online advertising, in essence, deals with the problem of presenting ads to online users in the most appropriate contexts to achieve a multitude of advertiser goals. A vast amount of work in online advertising has been focused on optimizing banner display advertising campaigns where the main goal lies in direct response metrics, often as clicks or conversions. In this paper, we explore the newly popularized space of online video advertising, where brand recognition is the key focus. We propose a framework based on a feedback mechanism where we optimize multiple video specific performance indicators while making sure the delivery constraints (budget and user reach) of advertisers are satisfied. While our main focus is on improving metrics such as engagement (amount of view time), and viewability (whether a campaign is within eyesight of a user), we also discuss the possibilities of expanding to other metrics. We demonstrate the benefit of our framework via empirical results in multiple real-world advertising campaigns. To the best of our knowledge, this is the first paper that deals with the unique challenges arising from the nature of online video advertising.

Real-Time Bidding allows an advertiser to purchase media inventory through an auction system that unfolds in the order of milliseconds. Media providers are increasingly being integrated into such programmatic buying platforms. It is typical for a contemporary Real-Time Bidding system to receive millions of bid requests per second at peak time, and have a large portion of these to be irrelevant to any advertiser. Meanwhile, given a valuable bid request, tens of thousands of advertisements might be qualified for scoring. We present our efforts in building selection models for both bid requests and advertisements to handle this scalability challenge. Our bid request model treats the system load as a hierarchical resource allocation problem and directs traffic based on the estimated quality of bid requests. Next, our exploration/exploitation advertisement model selects a limited number of qualified advertisements for thorough scoring based on the expected value of a bid request to the advertiser given its features. Our combined bid request and advertisement model is able to win more auctions and bring more value to clients by stabilizing the bidding pipeline. We empirically show that our deployed system is capable of handling 5x more bid requests.

In digital advertising, advertisers want to reach the right audience over media channels such as display, mobile, video, or social at the appropriate cost. The right audience for an advertiser consists of existing customers as well as valuable prospects, those that can potentially be turned into future customers. Identifying valuable prospects is called the audience extension problem because advertisers find new customers by extending the desirable criteria for their starting point, which is their existing audience or customers. The complexity of the audience extension problem stems from the difficulty of defining desirable criteria objectively, the number of desirable criteria (such as similarity, diversity, performance) to simultaneously satisfy, and the expected runtime (a few minutes) to find a solution over billions of cookie-based users. In this paper, we formally define the audience extension problem, propose an algorithm that extends a given audience set efficiently under multiple desirable criteria, and experimentally validate its performance. Instead of iterating over individual users, the algorithm takes in Boolean rules that define the seed audience and returns a new set of Boolean rules that corresponds to the extended audience that satisfy the multiple criteria.

Penalty function method is one of the most popular used Constraint Handling Techniques for Evolutionary Algorithms (EAs) solution selecting, whose performance is mainly determined by penalty parameters. This paper tries to study the penalty parameter from the aspect of problem characteristics, i.e., to construct a corresponding relationship between the problems and the penalty parameters. The experimental results confirm the relationship, which provides valuable reference for future algorithm design.

A method to convert binary to pseudo-ternary sequences is proposed. The method keeps run-length constraints imposed on finite-length binary sequence and also provides a unit value of a channel transition. The resulting finite-length pseudo-ternary sequence has exponential sums falling within specified regions. This allows spectrum shaping. The average power spectral density of infinite sequences obtained by concatenating spectrally-constrained sequences is examined. Thus we obtain the pseudo-ternary spectrum shaped code, which generally yields higher capacity than the binary one.

Systems for assessing and tutoring reading skills place unique requirements on underlying ASR technologies. This paper presents VersaReader, a system automatically measuring children's oral reading fluency skills. Critical techniques that improve the recognition accuracy and make the system practical are discussed in detail. We show that using a set of linguistic rules learned from a collection of transcriptions, the proposed rule-based language model outperformed traditional n-gram language models. Combined with a specific acoustic model with explicit long silence modeling, plus adaptation, a WER 7.25% was achieved in our test set. The impact of different kinds of rules on performance is also discussed. We demonstrate that VersaReader can provide highly accurate Words Correct Per Minute scores automatically, which are virtually indistinguishable from scores provided by careful human analysis.

Intelligent desktop assistants could provide more help for users if they could learn models of the users' workflows. However, discovering desktop workflows is difficult because they unfold over extended periods of time (days or weeks) and they are interleaved with many other workflows because of user multi-tasking. This paper describes an approach to discovering desktop workflows based on rich instrumentation of information flow actions such as copy/paste, SaveAs, file copy, attach file to email message, and save attachment. These actions allow us to construct a graph whose nodes are files, email messages, and web pages and whose edges are these information flow actions. A class of workflows that we call work procedures can be discovered by applying graph mining algorithms to find frequent subgraphs. This paper describes an algorithm for mining frequent closed connected subgraphs and then describes the results of applying this method to data collected from a group of real users.

The TaskTracer system allows knowledge workers to define a set of activities that characterize their desktop work. It then associates with each user-defined activity the set of resources that the user accesses when performing that activity. In order to correctly associate resources with activities and provide useful activity-related services to the user, the system needs to know the current activity of the user at all times. It is often convenient for the user to explicitly declare which activity he/she is working on. But frequently the user forgets to do this. TaskTracer applies machine learning methods to detect undeclared activity switches and predict the correct activity of the user. This paper presents TaskPredictor2, a complete redesign of the activity predictor in TaskTracer and its notification user interface. TaskPredictor2 applies a novel online learning algorithm that is able to incorporate a richer set of features than our previous predictors. We prove an error bound for the algorithm and present experimental results that show improved accuracy and a 180-fold speedup on real user data. The user interface supports negotiated interruption and makes it easy for the user to correct both the predicted time of the task switch and the predicted activity.

Real-time problems, in which the learning must be fast and the importance of the features might be changing, pose a challenge to machine learning algorithms. To learn robust classifiers in such nonstationary environments, it is essential not to assign too much weight to any single feature. We solve the problems by combining regularization mechanisms with online large margin learning algorithms. We prove bounds on their error and show that removing features with small weights has little influence on the accuracy, suggesting that these methods exhibit feature selection ability. We show that such regularized learning algorithms automatically decrease the influence of the old training instances and focus on the more recent ones. This makes them especially attractive in the dynamic environments. We present experimental results on real datasets to show the merit of our algorithms.

Knowledge workers perform many different activities daily. Each activity defines a distinct work context with different information needs. In this paper we leverage users' activity representations, stored in an activity management system, to automatically recommend resources to support knowledge workers in their current activity. We developed a collaborative activity predictor to both predict the current work activity and measure a resource's relevance to a specific activity. Relevant resources are then displayed in a contextual side bar on the desktop. We describe the system, our new activity-centric search algorithm, and experimental results based on the data from 50 real users.

Intelligent desktop environments allow the desktop user to define a set of projects or activities that characterize the user's desktop work. These environments then attempt to identify the current activity of the user in order to provide various kinds of assistance. These systems take a hybrid approach in which they allow the user to declare their current activity but they also employ learned classifiers to predict the current activity to cover those cases where the user forgets to declare the current activity. The classifiers must be trained on the very noisy data obtained from the user's activity declarations. Instead of asking the user to review and relabel the data manually, we employ an active EM algorithm that combines the EM algorithm and active learning. EM can be viewed as retraining on its own predictions. To make it more robust, we only retrain on those predictions that are made with high confidence. For active learning, we make a small number of queries to the user based on the most uncertain instances. Experimental results on real users show this active EM algorithm can significantly improve the prediction precision, and that it performs better than either EM or active learning alone.

The TaskTracer system seeks to help multi-tasking users manage the resources that they create and access while carrying out their work activities. It does this by associating with each user-defined activity the set of files, folders, email messages, contacts, and web pages that the user accesses when performing that activity. The initial TaskTracer system relies on the user to notify the system each time the user changes activities. However, this is burdensome, and users often forget to tell TaskTracer what activity they are working on. This paper introduces TaskPredictor, a machine learning system that attempts to predict the user’s current activity. TaskPredictor has two components: one for general desktop activity and another specifically for email. TaskPredictor achieves high prediction precision by combining three techniques: (a) feature selection via mutual information, (b) classification based on a confidence threshold, and (c) a hybrid design in which a Naive Bayes classifier estimates the classification confidence but where the actual classification decision is made by a support vector machine. This paper provides experimental results on data collected from Task-Tracer users.