Over the last year, a digital initiative has been focused on reengineering the search engine for kp.org, a health web portal serving over 12 million members. However, traditional software testing techniques that rely on limited use cases and consistent behavior are neither comprehensive nor specific for capturing complex user search behaviors. To support system deployment, we utilize information retrieval (IR) technologies to monitor search performance, identify areas of improvement and suggest actionable items. In this case study we share industrial experience on building an IR evaluation pipeline and its usage to inform deployment and improve system development. The work emphasizes domain specific challenges, best practices and lessons learned during system deployment in a healthcare setting. It features the ability of IR techniques to strengthen collaboration between data scientists, software engineers and product managers in making data-driven decisions.

Several frameworks are available for processing and analyzing the bigdata. But there is limited support available in integrating such heterogeneous bigdata programs to create a bigdata pipeline. In this paper, we introduce BBC, a Domain Specific Language (DSL) designed to create a bigdata pipeline from the heterogeneous programs. The paper focuses on Spark jobs but using a similar approach the DSL can be extended for other frameworks. We have successfully used BBC DSL to design the The Home Depot (THD) search evaluation pipeline that is instrumental in deploying every search feature to the THD website.

Named Entity Recognition (NER) and Named Entity Normalization (NEN) refer to the recognition and normalization of raw texts to known entities. From the perspective of recruitment innovation, professional skill characterization and normalization render human capital data more meaningful both commercially and socially. Accurate and detailed normalization of skills is the key for the predictive analysis of labor market dynamics. Such analytics help bridge the skills gap between employers and candidate workers by matching the right talent for the right job and identifying in-demand skills for workforce training programs. This can also work towards the social goal of providing more job opportunities to the community. In this paper we propose an automated approach for skill entity recognition and optimal normalization. The proposed system has two components: 1) Skills taxonomy generation, which employs vocational skill related sections of resumes and Wikipedia categories to define and develop a taxonomy of professional skills; 2) Skills tagging, which leverages properties of semantic word vectors to recognize and normalize relevant skills in input text. By sampling based end-user evaluation, the current system attains 91% accuracy on the taxonomy generation and 82% accuracy on the skills tagging tasks. The beta version of the system is currently applied in various big data and business intelligence applications for workforce analytics and career track projections at CareerBuilder.

Building a knowledge base (KB) describing domain-specific entities is an important problem in industry, examples including KBs built over companies (e.g. Dun & Bradstreet), skills (LinkedIn, CareerBuilder) and people (inome). The task involves several engineering challenges, including devising effective procedures for data extraction, aggregation and deduplication. Data extraction involves processing multiple information sources in order to extract domain-specific data instances. The extracted instances must be aggregated and deduplicated; that is, instances referring to the same underlying entity must be identified and merged. This paper describes a pipeline developed at CareerBuilder LLC for building a KB describing employers, by first extracting entities from both global, publicly available data sources (Wikipedia and Freebase) and a proprietary source (Infogroup), and then deduplicating the instances to yield an employer-specific KB. We conduct a range of pilot experiments over three independently labeled datasets sampled from the extracted KB, and comment on some lessons learned.

Named entity sense disambiguation is a problem with important natural language processing applications. In the online recruitment industry, normalization and recognition of occupational skills play a key role in linking the right candidate with the right job. The disambiguation of multisense skills will help improve this normalization and recognition process. In this paper we discuss an automatic large-scale system to identify and disambiguate multi-sense skills, including: (1) Feature Selection: employing word embedding to quantify the skills and their contexts into vectors; (2) Clustering: applying Markov Chain Monte Carlo (MCMC) methods to aggregate vectors into clusters that represent respective senses; (3) Large-scale: implementing parallelization to process text blobs on a large-scale; (4) Pruning: cluster cleaning by analyzing intra-cluster cosine similarities. Based on experiments on sample datasets, the MCMC-based clustering algorithm outperforms other clustering algorithms for the disambiguation problem. Also based on data-driven in-house evaluations, our disambiguation system achieves 84% precision.

Domain-agnostic signature discovery entails study across multiple scientific disciplines. The cross-disciplinary nature and breadth of this work requires that existing executable applications be integrated with new capabilities into workflows, representing a wide range of user tasks. An algorithm may be written in multiple programming languages for various hardware platforms, and so workflow composition requires integrating executables from any number of remote hosts. This raises an engineering issue on how to generate web service wrappers for these heterogeneous executables and to compose them into a scientific workflow environment (e.g., Taverna). In this position paper, we summarize our work on two simple Domain-Specific Languages (DSLs) that automate these processes. Our Service Description Language (SDL) describes key elements of a signature discovery service and automatically generates its implementation code. The Workflow Description Language (WDL) describes the pipeline of services and generates deployable artifacts for the Taverna workflow management system. We demonstrate our approach with a real-world workflow composed of services wrapping remote executables.

Graphical Processing Unit (GPU) programming languages are used extensively for general-purpose computations. However, GPU programming languages are at a level of abstraction suitable only for use by expert parallel programmers. This paper presents a new approach through which `C' or Java programmers can access these languages without having to focus on the technical or language-specific details. A prototype of the approach, named CUDACL, is introduced through which a programmer can specify one or more parallel blocks in a file and execute in a GPU. CUDACL also helps the programmer to make CUDA or OpenCL kernel calls inside an existing program. Two scenarios have been successfully implemented to assess the usability and potential of the tool. The tool was created based on a detailed analysis of the CUDA and OpenCL programs. Our evaluation of CUDACL compared to other similar approaches shows the efficiency and effectiveness of CUDACL.

Programmers copy and paste code. As a result, similar code fragments (clones) are added into software systems. Like other software artifacts, clones require attention and effort from programmers so that they can be found, understood, and correctly adapted and evolved. In addition to what clone-detection-based tools can offer, other automated support can be developed to better assist programmers in these activities, for example, to compare and contrast code clones, or help edit (a group of) clones consistently and quickly. We describe several such features currently being developed in the CnP project on top of Eclipse and for Java.

Programmers copy and paste code for many reasons. Regardless of the specific reasons, similar code fragments (clones) are introduced into software systems. Like other software artifacts, clones may require attention and effort from programmers so that they can be understood, and correctly adapted and evolved. More specifically, when understanding and maintaining clones, programmers need to know where the clones are. Programmers also need to compare and contrast code clones in order to figure out how they correspond and differ. Finally, they also need to edit or remove clones. In addition to what clone detection-based tools can offer, more automated support is needed to better assist programmers in these activities. In this paper, we introduce a toolkit CnP that is aimed to support and manage clones proactively as they are created and evolved. We describe the initial features and the design decisions taken in CnP. We also discuss possible future design extension.