Document retrieval, designed to recall query-relevant documents from expansive collections, is essential for information-seeking tasks, such as web search and open-domain question-answering. Advances in representation learning and pretrained language models (PLMs) have driven a paradigm shift from traditional sparse retrieval methods to more effective dense retrieval approaches, forging enhanced semantic connections between queries and documents and establishing new performance benchmarks. However, reliance on extensive annotated document-query pairs limits their competitiveness in low-resource scenarios. Recent research efforts employing the few-shot capabilities of large language models (LLMs) and prompt engineering for synthetic data generation have emerged as a promising solution. Nonetheless, these approaches are hindered by the generation of lower-quality data within the conventional dense retrieval training process. To this end, in this paper, we introduce iGFT, a framework aimed at enhancing low-resource dense retrieval by integrating a three-phase process --- Generation, Filtering, and Tuning --- coupled with an iterative optimization strategy. Specifically, we first employ supervised fine-tuning on limited ground truth data, enabling an LLM to function as the generator capable of producing potential queries from given documents. Subsequently, we present a multi-stage filtering module to minimize noise in the generated data while retaining samples poised to significantly improve the dense retrieval model's performance in the follow-up fine-tuning process. Furthermore, we design a novel iterative optimization strategy that dynamically optimizes the query generator for producing more informative queries, thereby enhancing the efficacy of the entire framework. Finally, extensive experiments conducted on a series of publicly available retrieval benchmark datasets have demonstrated the effectiveness of the proposed iGFT.

Compiler auto-tuning optimizes pass sequences to improve performance metrics such as Intermediate Representation (IR) instruction count. Although recent advances leveraging Large Language Models (LLMs) have shown promise in automating compiler tuning, two significant challenges still remain: the absence of high-quality reasoning datasets for agents training, and limited effective interactions with the compilation environment. In this work, we introduce Compiler-R1, the first reinforcement learning (RL)-driven framework specifically augmenting LLM capabilities for compiler auto-tuning. Compiler-R1 features a curated, high-quality reasoning dataset and a novel two-stage end-to-end RL training pipeline, enabling efficient environment exploration and learning through an outcome-based reward. Extensive experiments across seven datasets demonstrate Compiler-R1 achieving an average 8.46\% IR instruction count reduction compared to opt -Oz, showcasing the strong potential of RL-trained LLMs for compiler optimization. Our code and datasets are publicly available at https://github.com/Panhaolin2001/Compiler-R1.

Code based adversarial attacks play a crucial role in revealing vulnerabilities of software system. Recently, pre-trained programming language models (PLMs) have demonstrated remarkable success in various significant software engineering tasks, progressively transforming the paradigm of software development. Despite their impressive capabilities, these powerful models are vulnerable to adversarial attacks. Therefore, it is necessary to carefully investigate the robustness and vulnerabilities of the PLMs by means of adversarial attacks. Adversarial attacks entail imperceptible input modifications that cause target models to make incorrect predictions. Existing approaches for attacking PLMs often employ either identifier renaming or the greedy algorithm, which may yield sub-optimal performance or lead to high inference times. In response to these limitations, we propose CARL, an unsupervised black-box attack model that leverages reinforcement learning to generate imperceptible adversarial examples. Specifically, CARL comprises a programming language encoder and a perturbation prediction layer. In order to achieve more effective and efficient attack, we cast the task as a sequence decision-making process, optimizing through policy gradient with a suite of reward functions. We conduct extensive experiments to validate the effectiveness of CARL on code summarization, code translation, and code refinement tasks, covering various programming languages and PLMs. The experimental results demonstrate that CARL surpasses state-of-the-art code attack models, achieving the highest attack success rate across multiple tasks and PLMs while maintaining high attack efficiency, imperceptibility, consistency, and fluency.

In the contemporary era of widespread online recruitment, resume understanding has been widely acknowledged as a fundamental and crucial task, which aims to extract structured information from resume documents automatically. Compared to the traditional rule-based approaches, the utilization of recently proposed pre-trained document understanding models can greatly enhance the effectiveness of resume understanding. The present approaches have, however, disregarded the hierarchical relations within the structured information presented in resumes, and have difficulty parsing resumes in an efficient manner. To this end, in this paper, we propose a novel model, namely ERU, to achieve efficient resume understanding. Specifically, we first introduce a layout-aware multi-modal fusion transformer for encoding the segments in the resume with integrated textual, visual, and layout information. Then, we design three self-supervised tasks to pre-train this module via a large number of unlabeled resumes. Next, we fine-tune the model with a multi-granularity sequence labeling task to extract structured information from resumes. Finally, extensive experiments on a real-world dataset clearly demonstrate the effectiveness of ERU.

Job interviews are the most widely accepted method for companies to select suitable candidates, and a critical challenge is finding the right questions to ask job candidates. Moreover, there is a lack of integrated tools for automatically generating interview questions and recommending the right questions to interviewers. To this end, in this paper, we propose an intelligent system for assisting job interviews, namely, DuerQues. To build this system, we first investigate how to automatically generate skill-oriented interview questions in a scalable way by learning external knowledge from online knowledge-sharing communities. Along this line, we develop a novel distantly supervised skill entity recognition method to identify skill entities from large-scale search queries and web page titles with less need for human annotation. Additionally, we propose a neural generative model for generating skill-oriented interview questions. In particular, we introduce a data-driven solution to create high-quality training instances and design a learning algorithm to improve the performance of question generation. Furthermore, we exploit click-through data from query logs and design a recommender system for recommending suitable questions to interviewers. Specifically, we introduce a graph-enhanced algorithm to efficiently recommend suitable questions given a set of queried skills. Finally, extensive experiments on real-world datasets demonstrate the effectiveness of our DuerQues system in terms of the quality of generated skill-oriented questions and the performance of question recommendation.

Understanding the semantic structure of resumes plays an important role for various intelligent recruitment related applications. However, due to the unique characteristics of resume documents (e.g., diverse writing styles and multi-page) and the lack of labeled data, it has been a long-standing challenge to effectively extract the structural information of resumes through machine learning models. While considerable efforts have been made in this direction, existing methods only focus on the textual information in the document where the rich multi-modal information (e.g., the visual and layout information) is largely ignored. To this end, in this paper, we propose ResuFormer for understanding the semantic structure of resumes. Specifically, ResuFormer focuses on two typical tasks in this direction, namely resume block classification and intra-block information extraction respectively. For the first task, we propose a multi-modal pre-training model with a hierarchical Transformer encoder, in which we design three self-supervised training objectives, i.e., masked layout-language model, self-supervised contrastive learning and dynamic next-sentence prediction, to pre-train the model parameters, and fine-tune the model only using a small amount of training data. For the second task, we introduce a self-distillation based self-training learning framework to make the distantly supervised model more robust to the noise data. Finally, extensive experiments conducted on real-world resume datasets have clearly validated the performance of our ResuFormer compared with state-of-the-art (SOTA) baselines.

Recent years have witnessed the rapid development of machine-learning-based intelligent recruitment services. Along this line, a large number of emerging models have been proposed, achieving remarkable performance in various tasks, such as person-job fit, job classification and salary prediction. However, existing studies are usually domain/task specific, which significantly hinders the adaptation of models for different industries/tasks with limited training data. To this end, in this paper, we propose a novel skill-aware prompt-based pretraining framework, namely RecruitPro, which is capable of learning unified representations on the recruitment data and adapting for various downstream tasks of intelligent recruitment services. To be specific, we first present a contextualized embedding model that is pretrained on a large-scale recruitment dataset. Then, we construct 13 downstream benchmark tasks that are representative in the recruitment process. Along this line, we propose a skill-aware prompt learning module to enhance the adaptability of the pretrained model on downstream tasks. This module includes a skill-related prompt, which is designed to explore key semantic information (i.e., skills) from recruitment text, and a task-related prompt, which is designed to bridge the gap between the pretrained model and different downstream tasks. Moreover, we propose a strategy for extracting potential skills to further improve the performance of our skill-aware prompt learning module. Finally, extensive experiments have clearly demonstrated the effectiveness of RecruitPro. In addition, a case study has been presented to discuss the privacy preserving issue of our RecruitPro.

Recent years have witnessed the rapid development of heterogeneous graph neural networks (HGNNs) in information retrieval (IR) applications. Many existing HGNNs design a variety of tailor-made graph convolutions to capture structural and semantic information in heterogeneous graphs. However, existing HGNNs usually represent each node as a single vector in the multi-layer graph convolution calculation, which makes the high-level graph convolution layer fail to distinguish information from different relations and different orders, resulting in the information loss in the message passing. Then we propose a novel heterogeneous graph neural network with sequential node representation, namely Seq-HGNN. To avoid the information loss caused by the single vector node representation, we first design a sequential node representation learning mechanism to represent each node as a sequence of meta-path representations during the node message passing. Then we propose a heterogeneous representation fusion module, empowering Seq-HGNN to identify important meta-paths and aggregate their representations into a compact one. We conduct extensive experiments on four widely used datasets from Heterogeneous Graph Benchmark (HGB) and Open Graph Benchmark (OGB). Experimental results show that our proposed method outperforms state-of-the-art baselines in both accuracy and efficiency. The source code is available at https://github.com/nobrowning/SEQ_HGNN.

A job description shows the responsibilities of the job position and the skill requirements for the job. An effective job description will help employers to identify the right talents for the job, and give a clear understanding to candidates of what their duties and qualifications for a particular position would be. In this paper, we investigate how to automate the process to generate job descriptions with less human intervention. We propose an end-to-end capability-aware neural job description generation framework, namely Cajon, to facilitate the writing of job description. Specifically, we first propose a novel capability-aware neural topic model to distill the various capability information from the larger-scale recruitment data. Also, an encoder-decoder recurrent neural network is designed for enabling the job description generation. In particular, the capability-aware attention and copy mechanisms are proposed to guide the generation process to ensure the generated job descriptions can comprehensively cover relevant and representative capability requirements for the job. Moreover, we propose a capability-aware policy gradient training algorithm to further enhance the rationality of the generated job description. Finally, extensive experiments on real-world recruitment data clearly show our Cajon framework can help to generate more effective job descriptions in an interpretable way

As an essential task of talent recruitment, person-job fit aims to measure the matching degree between talent qualifi-cation and the job requirements of a position. Existing studies usually formulate this task as a long text matching problem with a focus on learning effective representations of both job postings and resumes. However, it is commonly known that there exists a semantic gap between textual job postings and textual resumes. Therefore, in this paper, we study how to improve person-job fit by bridging this semantic gap with the help of prior knowledge. To this end, we first design a distantly supervised skill extraction model to identify the skill entities from the given job postings and resumes using only unlabeled data and skill entity dictionaries. The identified skill entities will be used to construct a skill knowledge graph (KG) on the global corpus, which can provide the prior knowledge. Also, we propose a knowledge enhanced person-job fit approach for talent recruitment. Here, we model job postings and resumes as two graphs and fuse the prior external knowledge into the graph representation learning. Specifically, we first build the graphs from job posting and resume text. Then, we design a knowledge-aware graph encoder that can not only capture the contextual word relationships within each job posting or resume, but also incorporate the prior knowledge into node representation learning. In addition, we propose an interactive learning method to perform effective graph matching in both graph-level and node-level, respectively. Meanwhile, a multi-task learning strategy is introduced to facilitate the graph representation learning. Finally, extensive experiments conducted on real-world datasets have clearly validated the effectiveness of our approaches compared with state-of-the-art baselines.

As a fundamental task of document content analysis, keyphrase extraction (KE) aims at predicting a set of lexical units that conveys the core information of the document. In this paper, we study the problem of KE in the talent recruitment. This problem is critical for the development of a variety of intelligent recruitment services, such as person-job fit, market trend analysis and course recommendation. However, unlike traditional textual data, the texts from the recruitment domain, such as resume and job postings, often have unique characteristics of abbreviation and succinctness, resulting in massive keyphrases consisting of inconsecutive words that are hard to be fully captured by existing KE methods. To this end, we propose an interactive neural network approach, INKE, for facilitating KE in the talent recruitment. To be specific, we first introduce a novel keyphrase indicator that captures the explicit hint information for each keyphrase. Then, we design a dynamically-initialized decoder which can generate keyphrases in an interactive manner. Moreover, we propose a hierarchical reinforcement learning algorithm to enhance the interaction between the hint information capture and keyphrase generation. Finally, extensive experiments on real-world data clearly validate the effectiveness and interpretability of INKE compared with state-of-the-art baselines.

Recurrent neural network-based sequence-to-sequence attentional models have proven effective in abstractive text summarization. In this paper, we model abstractive text summarization using a dual encoding model. Different from the previous works only using a single encoder, the proposed method employs a dual encoder including the primary and the secondary encoders. Specifically, the primary encoder conducts coarse encoding in a regular way, while the secondary encoder models the importance of words and generates more fine encoding based on the input raw text and the previously generated output text summarization. The two level encodings are combined and fed into the decoder to generate more diverse summary that can decrease repetition phenomenon for long sequence generation. The experimental results on two challenging datasets (i.e., CNN/DailyMail and DUC 2004) demonstrate that our dual encoding model performs against existing methods.

We propose a novel neural network model that aims to generate diverse and human-like natural language questions. Our model not only directly captures the variability in possible questions by using a latent variable, but also generates certain types of questions by introducing an additional observed variable. We deploy our model in the generative adversarial network (GAN) framework and modify the discriminator which not only allows evaluating the question authenticity, but predicts the question type. Our model is trained and evaluated on a question-answering dataset SQuAD, and the experimental results shown the proposed model is able to generate diverse and readable questions with the specific attribute.