Logging is a common practice in software development, widely used for debugging, testing, and performance monitoring. However, recording sensitive user data can introduce severe privacy risks. Past incidents involving leaked logs have prompted platforms such as Android to publish strict guidelines discouraging developers from logging personally identifiable information (PII) and other ''linkable'' or ''ambiguous'' data unless strictly required for core functionality. To evaluate real-world compliance, we examined the logging practices of 500 Android applications across six categories. Our findings reveal that 264 apps contain logging violations, from which we identified 864 instances of sensitive data exposure. Notably, 54% of these violations stem from debugging logs that should have been removed before release. The recorded data includes PII such as email addresses and phone numbers, as well as linkable information such as shopping history, health records, and private messages each in direct violation of Android's privacy guidelines. Moreover, removing these logging statements from application source code does not affect app functionality, raising questions about their necessity. Our analysis shows that most violations originate from debugging practices, third-party analytics tracking, and HTTP request logging. Further, by applying a large language model (LLM) to inspect an additional set of 300 applications, we found 240 apps exhibiting sensitive data logging violations. We also discovered that many apps share log data with third-party services, often contradicting their own privacy policies. To mitigate these risks, we provide practical recommendations for both app developers and mobile platforms to enforce responsible and privacy-preserving logging practices.

Keystroke dynamics is a popular behavioral biometric for user authentication. This paper explores the potential of user-specific feature selection in performance improvement beyond existing template adaptation strategies. Four experiments with CMU and GreyC datasets evaluated the approach. Two impostor sample selection methods were tested: the head method versus random sampling. Experiments 1 and 2 used the head method for the CMU and GreyC datasets, while Experiments 3 and 4 employed random sampling. GreyC had shown better EER improvement. The CMU dataset achieved a better improvement using random sampling, while the GreyC dataset did the same with the head method, likely due to differences in password characteristics and familiarity.

In this research paper, we conducted a comparative study to measure the effectiveness of the provided technical support in computing project-based learning (PjBL) courses. Students learn much better by solving authentic real-world problems through PjBL. PjBL in computing education has proven to boost student motivation and engagement while enhancing academic performance. Crucial to PjBL in computing is the technical support that the instructors can provide to students, which is required for sustained, successful learning during project tasks. Without adequate support, PjBL will fall short of accomplishing its goals, leading to a rise in student frustration, a loss of motivation and engagement, and compromised learning outcomes. Measuring the impacts and effectiveness of the provided support is imperative for fostering continuous improvement, informed decision-making, and student success. It enables instructors to assess the impacts of their strategies, improve their approaches, and utilize their resources more effectively. To measure the impacts of technical support on students during PjBL, we performed a comparative study on two undergraduate computing courses in Spring 2024, Fundamentals of Software Engineering and Database Systems. In both courses, students work on two assigned projects, one with little and inadequate support and the other with adequate support. We administered a post-survey after each project was completed. We analyzed students' selfreflection responses across four sub-scales, support satisfaction, motivation, self-efficacy, and project satisfaction. The results show a statistically significant increase in the supported project in the Fundamentals of Software Engineering course and no difference in the Database Systems course. This finding is likely due to other differences between the two projects for the Database Systems course beyond support, such as project scale. Qualitative analysis of students' responses also indicates the need for support by students in the less supported projects. Based on our experience, we reflect on the question of what would constitute a good design for studies that seek to compare two different student learning experiences.

In this research paper, drawing from our own and other computing instructors' experiences, we highlight common technical challenges faced by students in software engineering project-based learning (PjBL) and discuss ways in which instructors can support students in overcoming them so that motivation is summoned and sustained. Through the use of practical hands-on experiences, PjBL has been shown to be an effective educational approach. However, unless projects are intentionally designed and supported in a way that summons and sustains student motivation, PjBL is likely to fail to accomplish its goals. Several factors influence student motivation, including their perception of the project's value and how confident they are in their ability to complete it. In particular, challenges that students perceive as insurmountable during the project can significantly weaken their motivation. On the other hand, supporting students to overcome such hurdles can be troublesome, especially in large classes as well as classes with diversity in student backgrounds. To generalize from our own experience, we designed a questionnaire targeted at PjBL computing instructors that contained closed and open questions on technical challenges faced by students, support instructors provided to overcome such challenges, and lessons learned by instructors on the effectiveness of their support. A total of 47 responses were collected from instructors with diverse backgrounds in terms of courses taught, students' years, and class sizes. We categorized the technical challenges into three main categories, namely (a) challenges in installing and configuring software packaged tools, (b) lack of prerequisite knowledge, and (c) challenges while completing project tasks. In this paper, we present the survey results from the three categories of technical challenges, their frequencies, importance, and effective support strategies instructors use to alleviate them.

As more types of transactions move online, there is an increasing need to verify someone’s identity remotely. Remote identity verification (RIdV) technologies have emerged to fill this need. RIdV solutions typically use a smart device to validate an identity document like a driver’s license by comparing a face selfie to the face photo on the document. Recent research has been focused on ensuring that biometric systems work fairly across demographic groups. This study assesses five commercial RIdV solutions for equity across age, gender, race/ethnicity, and skin tone across 3,991 test subjects. This paper employs statistical methods to discern whether the RIdV result across demographic groups is statistically distinguishable. Two of the RIdV solutions were equitable across all demographics, while two RIdV solutions had at least one demographic that was in-equitable. For example, the results for one technology had a false negative rate of 10.5% +/- 4.5% and its performance for each demographic category was within the error bounds, and, hence, were equitable. The other technologies saw either poor overall performance or inequitable performance. For one of these, participants of the race Black/African American (B/AA) as well as those with darker skin tones (Monk scale 7/8/9/10) experienced higher false rejections. Finally, one technology demonstrated more favorable but inequitable performance for the Asian American and Pacific Islander (AAPI) demographic. This study confirms that it is necessary to evaluate products across demographic groups to fully understand the performance of remote identity verification technologies.

Utilization of Internet in everyday life has made us vulnerable in terms of security and privacy of our data and systems. For example, large-scale data breaches have occurred at Yahoo and Equifax because of lacking of robust and secure data protection within systems. Therefore, it is imperative to find solutions to further boost data security and protect privacy of our systems. To this end, we propose to authenticate users by utilizing score-level fusions based on mouse dynamics (e.g., mouse movement on a screen) and widget interactions (e.g., when clicking or hovering over different icons on a screen) on two novel datasets. In this study, we focus on two common applications, PayPal (a money transaction website) and Facebook (a social media platform). Though we fuse the same modalities for both applications, the purpose of investigating PayPal is to demonstrate how we can authenticate users when the users interact with the app for only a short period of time, while the purpose of investigating Facebook is to authenticate users based on social media browsing activities. We have a total of 10 users for PayPal with an average of 12 minutes of data per user and a total of 15 users for Facebook with an average of 2 hours of data per user. By fusing a single mouse trajectory with the associated widget interactions that occur during the trajectory, our mean EERs (Equal Error Rates) with a score-level fusion of mouse dynamics and widget interactions are 7.64% (SVM-rbf) and 3.25% (GBM), for PayPal, and 5.49% (SVM-rbf) and 2.54% (GBM), for Facebook. To further improve the performance of our fusion, we combine decision scores from multiple consecutive trajectories, which yields a 0% mean EER after 11 decision scores across all the users for both PayPal and Facebook.

The ubiquity of mobile devices nowadays necessitates securing the apps and user information stored therein. However, existing one-time entry-point authentication mechanisms and enhanced security mechanisms such as Multi-Factor Authentication (MFA) are prone to a wide vector of attacks. Furthermore, MFA also introduces friction to the user experience. Therefore, what is needed is continuous authentication that once passing the entry-point authentication, will protect the mobile devices on a continuous basis by confirming the legitimate owner of the device and locking out detected impostor activities. Hence, more research is needed on the dynamic methods of mobile security such as behavioral biometrics-based continuous authentication, which is cost-effective and passive as the data utilized to authenticate users are logged from the phone's sensors. However, currently, there are not many mobile authentication datasets to perform benchmarking research. In this work, we share two novel mobile datasets (Clarkson University (CU) Mobile datasets I and II) consisting of multi-modality behavioral biometrics data from 49 and 39 users respectively (88 users in total). Each of our datasets consists of modalities such as swipes, keystrokes, acceleration, gyroscope, and pattern-tracing strokes. These modalities are collected when users are filling out a registration form in sitting both as genuine and impostor users. To exhibit the usefulness of the datasets, we have performed initial experiments on selected individual modalities from the datasets as well as the fusion of simultaneously available modalities.

As account compromises and malicious online attacks are on the rise, multi-factor authentication (MFA) has been adopted to defend against these attacks. OTP and mobile push notification are just two examples of the popularly adopted MFA factors. Although MFA improve security, they also add additional steps or hardware to the authentication process, thus increasing the authentication time and introducing friction. On the other hand, keystroke dynamics-based authentication is believed to be a promising MFA for increasing security while reducing friction. While there have been several studies on the usability of other MFA factors, the usability of keystroke dynamics has not been studied. To this end, we have built a web authentication system with the standard features of signup, login and account recovery, and integrated keystroke dynamics as an additional factor. We then conducted a user study on the system where 20 participants completed tasks related to signup, login and account recovery. We have also evaluated a new approach for completing the user enrollment process, which reduces friction by naturally employing other alternative MFA factors (OTP in our study) when keystroke dynamics is not ready for use. Our study shows that while maintaining strong security (0% FPR), adding keystroke dynamics reduces authentication friction by avoiding 66.3% of OTP at login and 85.8% of OTP at account recovery, which in turn reduces the authentication time by 63.3% and 78.9% for login and account recovery respectively. Through an exit survey, all participants have rated the integration of keystroke dynamics with OTP to be more preferable to the conventional OTP-only authentication.

In this workshop, we introduce participants to the accomplishments and lessons learned from our ongoing NSF IUSE education research project, which is focused on supporting undergraduate project-based learning in computing education by developing and piloting a set of scaffolded course projects. The workshop has two main goals. One is to facilitate exchange of experiences on project-based learning among workshop participants. The other is to encourage adoption of the developed course projects by the broader computing education community.

Teaching students complex problem-solving skills using large-scale, real-world problems is challenging for both students and teachers alike. As a result, most courses use small, well-specified, toy-like problems, which are not representative of what students will encounter in the workforce. One approach that allows teachers to use large-scale problems in class is by introducing scaffolding. Scaffolding breaks a larger problem into smaller steps, which students can solve independently, while deemphasizing tangential concepts such as the complex configuration files needed to compile open-source software systems. Strong scaffolding supports student learning, preventing them from getting bogged down with unnecessary tasks or overwhelmed by complexity. This work investigates a scaffolded problem-based-learning module for computing courses, using a realistically-sized project with characteristics representative of the industry. The project was implemented in a computer science course with roughly 100 students, and the results speak to the importance of scaffolding for student success. In fact, there were two student assignments that lacked sufficient scaffolding, compared with other tasks, and the reduction in student scoring and persistence shows that project scaffolding is necessary when implementing these types of assignments. Most students felt the project helped prepare them for a job in their chosen field.

This qualitative research performs a thematic analysis of the learning objectives in existing project-based undergraduate software engineering courses to align them with the competency model defined in the Computing Curricula 2020 reports (CC2020). This study identifies the trends, strengths, and gaps in how the reviewed course learning objectives cover the knowledge, skill, and disposition components of the CC2020 competency model. The learning objectives were categorized according to knowledge elements, skills, and dispositions as defined in the CC2020 competency model. Our analysis shows that 54% of knowledge elements from the reviewed learning objectives do not have any skill level specified and overall, only two out of the eleven dispositions in CC2020 are specified (“Collaboration” and “Professional”). We also find that technical knowledge elements from the software development category (e.g., software process, software design, and software quality, verification & validation) and systems modeling category (e.g., systems analysis & design, and requirements analysis and specification), probably unsurprisingly, are covered the most often. Similarly, collaboration & teamwork, and oral & written communication are unsurprisingly the most common professional & foundational knowledge elements in the reviewed course's learning objectives as they are essential to project-based learning. Although they are essential for the completion of a successful software project, knowledge elements such as time management, security technology & implementation, and user experience design are rarely mentioned. We discuss the implications of our findings on course design.

Keystroke dynamics has been shown to be a promising method for user authentication based on a user's typing rhythms. Over the years, it has seen increasing applications such as in preventing transaction fraud, account takeovers, and identity theft. However, due to the variable nature of keystroke dynamics, a user's typing patterns may vary on a different keyboard or in a different keyboard language setting, which may affect the system accuracy. In other words, an algorithm modeled with data collected using a mechanical keyboard may perform significantly differently when tested with an ergonomic keyboard. Similarly, an algorithm modeled with data collected in one language may perform significantly differently when tested with another language. Hence, there is a need to study the impact of multiple keyboards and multiple languages on keystroke dynamics performance. This motivated us to develop two free-text keystroke dynamics datasets. The first is a multi-keyboard keystroke dataset comprising of four (4) physical keyboards - mechanical, ergonomic, membrane, and laptop keyboards - and the second is a bilingual keystroke dataset in both English and Chinese languages. Data were collected from a total of 86 participants using a non-intrusive web-based keylogger in a semi-controlled setting. To the best of our knowledge, these are the first multi-keyboard and bilingual keystroke datasets, as well as the data collection software, to be made publicly available for research purposes. The usefulness of our datasets was demonstrated by evaluating the performance of two state-of-the-art free-text algorithms.

Facebook has become an important part of our daily life. From knowing the status of our relatives, showing off a new car, to connecting with a high school classmate, abundant personally identifiable information (PII) are made visible to others by posts, images and news. However, this free flow of information has also created significant cyber-security challenges that make us vulnerable to social engineering and cyber crimes. To confront these challenges, we propose a new behavioral biometric that verifies a user based on his or her widget interaction behavior when using Facebook. Specifically, we monitor activities on the user's Facebook account using our own logging software and verify the user's claimed identity by binary classifiers trained with two algorithms (SVM-rbf and the GBM- Gradient Boosting Machines). Our novel dataset consists of eight users over a month of data collection with an average of 2.95k rows of data per user. We convert these activities data into meaningful features such as day-of-week, hour-of-day, and widget types and duration of mouse staying on a widget. The performance shows that our novel widget interaction modality is promising for authentication. The SVM-rbf classifiers achieve a mean Equal Error Rate (EER) and mean Accuracy (ACC) of 3.91% and 97.79%, while the GBM classifiers a mean EER and ACC of 2.76% and 97.88%, respectively. In addition, we perform an ablation study to understand the impact of individual features on authentication performance. The importance of features are ranked in the descending order of hour-of-day, day-of-week, and widget types and duration.

Occupancy detection is a process of inferring the presence of persons in a space but without necessarily knowing their identities or quantity. Knowledge of occupancy is essential in building information modeling, building energy demand forecast, energy simulations, security, and smart control. Unlike other approaches that require the installation of additional sensors, such as cameras, infrared sensors, optical devices, sound sensors, within the targeted areas, we propose to improve a state of the art approach by Kleiminger, Beckel, and Santini that infers occupancy based on the aggregated power consumption data readily available in many residential houses. Our new approach integrates both aggregated and appliance specific electricity consumption data for occupancy detection. Evaluation of our new approach using the publicly available ECO dataset shows that our best approach can achieve an overall accuracy between 84% and 93%, which improves the baseline accuracy by 1% to 8%. However, our second experiment with a dataset that does not include data about HVAC yields much poorer performance than the first. Our central conclusion is that while appliances are usually strong predictors of occupancy, individual appliances rarely offer adequate coverage of a whole day as they tend to be used rather sporadically. They work well when combined with HVAC, as in the first experiment, but not so well when there is no HVAC, as in the second experiment. Therefore, the key for this approach to work well is when there are sufficient coverage from appliances, or perhaps to complement other data sources.

Conventional one-stop authentication of a computer terminal takes place at a user's initial sign-on. In contrast, continuous authentication protects against the case where an intruder takes over an authenticated terminal or simply has access to sign-on credentials. Behavioral biometrics has had some success in providing continuous authentication without requiring additional hardware. However, further advancement requires benchmarking existing algorithms against large, shared datasets. To this end, we provide a novel large dataset that captures not only keystrokes, but also mouse events and active programs. Our dataset is collected using passive logging software to monitor user interactions with the mouse, keyboard, and software programs. Data was collected from 103 users in a completely uncontrolled, natural setting, over a time span of 2.5 years. We apply Gunetti & Picardi's algorithm, a state-of-the-art algorithm in free text keystroke dynamics, as an initial benchmarkfor the new dataset.

Home appliance monitoring provides useful information about appliance usage, which can be used to better inform users about their consumption habits and promote energy conservation. However, metering all loads is cost prohibitive. Instead, many have tried to disaggregate loads from aggregate power measurements. Existing approaches that require submetering high resolution power signals of individual appliances during training and testing, are impractical and economically infeasible. In this paper, we introduce a low-cost approach for home appliance monitoring based on observations made on a single circuit. Our approach consists of three steps. First, we apply a neural network classifier to segment the input power signals. Then, we apply another classifier to label each segment as an individual appliance or multiple appliances. Finally, we iteratively disaggregate the multi-appliance segments with the classifier in the previous step. Our proposed approach is evaluated in two experiments. The first experiment uses a fully labeled public dataset consisting of 1,211 segments from 25 student bedrooms on a university campus. The second experiment uses 1,563 segments from the REDD public dataset. The evaluation shows that our approach can accurately detect those appliances that dominate energy consumption (overall accuracy of 86.6% and 91.2%, respectively). We also present an in-depth analysis of the failures and conjecture why they are harder to detect.

The use of software frameworks enables the delivery of common functionality but with significantly less effort than when developing from scratch. To meet application specific requirements, the behavior of a framework needs to be customized via extension points. A common way of customizing framework behavior is by passing a framework related object as an argument to an API call. Such an object can be created by subclassing an existing framework class or interface, or by directly customizing an existing framework object. However, to do this effectively requires developers to have extensive knowledge of the framework's extension points and their interactions. To aid the developers in this regard, we propose and evaluate a graph mining approach for extension point management. Specifically, we propose a taxonomy of extension patterns to categorize the various ways an extension point has been used in the code examples. Our approach mines a large amount of code examples to discover all extension points and patterns for each framework class. Given a framework class that is being used, our approach aids the developer by following a two-step recommendation process. First, it recommends all the extension points that are available in the class. Once the developer chooses an extension point, our approach then discovers all of its usage patterns and recommends the best code examples for each pattern. Using five frameworks, we evaluate the performance of our two-step recommendation, in terms of precision, recall, and F-measure. We also report several statistics related to framework extension points.

Software frameworks enable developers to reuse existing well tested functionalities instead of taking the burden of implementing everything from scratch. However, to meet application specific requirements, the frameworks need to be customized via extension points. This is often done by passing a framework related object as an argument to an API call. To enable such customizations, the object can be created by extending a framework class, implementing an interface, or changing the properties of the object via API calls. However, it is both a common and non-trivial task to find all the details related to the customizations. In this paper, we present a tool, called FEMIR, that utilizes partial program analysis and graph mining technique to detect, group, and rank framework extension examples. The tool extends existing code completion infrastructure to inform developers about customization choices, enabling them to browse through extension points of a framework, and frequent usages of each point in terms of code examples. A video demo is made available at https://asaduzzamanparvez.wordpress.com/femir.

In current practice, smart housing environments often over-install smart sensors on every fixture and log data from them at high sampling rates, resulting in more data being collected than is necessary. Fixture identification offers a possible alternative to reduce the number of sensors installed and the amount of data collected in smart housing. Fixture identification applies classifiers to label utility consumption data aggregated at the apartment level by the specific fixture that actually contributes the data, such as the shower or the kitchen sink. Successful fixture identification can be used to educate tenants, optimize the resource supply strategy, and offer a smart solution for detecting abnormal usage activities. In this paper, we report a case study of water fixture identification by using support vector machines (SVMs) to perform fixture classification. We use the Smart Housing Dataset from Clarkson University, which comprises of one academic year of tenant activities from 12 student apartments. Our results show that the proposed approach achieves an average accuracy between 78% to 87.8% for identifying hot water fixtures including kitchen sink, bathroom sink and shower. As a result, the number of smart meters per apartment is reduced from 7 to 3, one for hot water, one for cold water, and the third for toilet. The novelty of our study lies in the feature selection process, which is guided by our domain knowledge of water fixture characteristics and the correlation between water fixture usage and other user behavior in the apartments. We describe our proposed features, their rationale, and their effect on classification performance.

Abstract Code completion helps developers use application programming interfaces (APIs) and frees them from remembering every detail. In this paper, we first describe a novel technique called Context‐sensitive Code Completion (CSCC) for improving the performance of API method call completion. CSCC is context sensitive in that it uses new sources of information as the context of a target method call. CSCC indexes method calls in code examples by their context. To recommend completion proposals, CSCC ranks candidate methods by the similarities between their contexts and the context of the target call. Evaluation using a set of subject systems and five popular state‐of‐the‐art techniques suggests that CSCC performs better than existing type or example‐based code completion systems. We conduct experiments to find how different contextual elements of the target call benefit CSCC. Next, we investigate the adaptability of the technique to support another form of code completion, i.e., field completion. Evaluation with eight different subject systems suggests that CSCC can easily support field completion with high accuracy. Finally, we compare CSCC with four popular statistical language models that support code completion. Results of statistical tests from our study suggest that CSCC not only outperforms those techniques that are based on token level language models, but also in most cases performs better or equally well with GraLan, the state‐of‐the‐art graph‐based language model. Copyright © 2016 John Wiley & Sons, Ltd.

Feature location is a critical step in the software maintenance process where a developer identifies the software artifacts that need to be changed in order to fulfill a new feature request. Much progress has been made in understanding the feature location process and in creating new tools to help a developer in performing this task. However, there is still lack of support for locating architectural features, ones that require a developer to touch on more than one architectural component. We demonstrate a tool called ArchFLoc that can be used to discover and highlight architectural level features that are otherwise hidden in a software system. ArchFLoc is integrated into user interfaces, so the developer can express a feature query by directly interacting with user interface elements at runtime. Based on the user query, ArchFLoc discovers relevant code artifacts and dependencies, and assembles documentation to explain their roles in the overall architectural design.

Keystroke authentication can help significantly improve computer security by hardening passwords or offering active, continuous authentication. Over the years, many keystroke authentication algorithms have been reported to produce promising results. However, these results are tested on proprietary datasets with varying numbers of subjects and amounts of text, making it difficult to compare and improve the state of art. We describe a new dataset that we have developed with the goal to serve as a shared common testbed to enable future improvements. The new dataset includes keystroke data for short pass-phrases, fixed text (transcription of long proses), and free text. It also includes video of a subject's facial expression and hand movement during the data collection sessions, allowing for a deeper understanding of why an algorithm works the way it does, for example, by finding out whether a subject is a touchtypist or not. As a baseline for benchmarking, we also include the results of replicating two existing algorithms using the new dataset.

Code Completion helps developers learn APIs and frees them from remembering every detail. In this paper, we describe a novel technique called CSCC (Context Sensitive Code Completion) for improving the performance of API method call completion. CSCC is context sensitive in that it uses new sources of information as the context of a target method call. CSCC indexes method calls in code examples by their contexts. To recommend completion proposals, CSCC ranks candidate methods by the similarities between their contexts and the context of the target call. Evaluation using a set of subject systems and five popular state of-the-art techniques suggests that CSCC performs better than existing type or example-based code completion systems. We also investigate how the different contextual elements of the target call benefit CSCC.

Developers depend on APIs of frameworks and libraries to support the development process. Due to the large number of existing APIs, it is difficult to learn, remember, and use them during the development of a software. To mitigate the problem, modern integrated development environments provide code completion facilities that free developers from remembering every detail. In this paper, we introduce CSCC, a simple, efficient context-sensitive code completion tool that leverages previous code examples to support method completion. Compared to other existing code completion tools, CSCC uses new sources of contextual information together with lightweight source code analysis to better recommend API method calls.

Online technical forums are valuable sources for mining useful software engineering information. LDA (Latent Dirichlet Allocation) is an unsupervised machine learning method which can be used for extracting underlying topics out of such large forums. However, the main output of LDA forum learning are usually huge matrices that contain millions of numbers, which is impossible for researchers to directly scrutinize the numerical distribution and semantically evaluate the relationship between the extracted topics and large collection of unorganized documents. In this paper, we present LDAAnalyzer, an LDA visualization tool that makes the hidden topic-document structures rise to the surface. From the functionality point of view, LDA Analyzer consists of (1) LDA modeling (2) LDA output analysis and (3) new corpus training. With the help of LDAAnalyzer, our semantic topic-modeling evaluation based on large technical forums becomes feasible.

Text categorization, automatically labeling natural language text with pre-defined semantic categories, is an essential task for managing the abundant online data. An example of such data in Software Engineering is the large, ever-growing volume of forum discussions on how to use particular APIs. We have conducted a study to explore the question as to how well machine learning algorithms can be applied to categorize API discussions based on their content. Our goal is two-fold: (1) Can a relatively straightforward algorithm such as Naive Bayes work sufficiently well for this task? (2) If yes, how can we control its performance? We have achieved the best test accuracy mean (TAM) of 94.1% with our largest training data set for the AWT/Swing API, which consists of 833 forum discussions distributed over eight categories/topics. We have also investigated factors that impact classification accuracy, with the most important two being the size of the training set and multi-label documents (the phenomenon that some discussions involve more than one category).

Software engineering activities often produce large amounts of unstructured data. Useful information can be extracted from such data to facilitate software development activities, such as bug reports management and documentation provision. Online forums, in particular, contain extensive valuable information that can aid in software development. However, no work has been done to extract problematic API features from online forums. In this paper, we investigate ways to extract problematic API features that are discussed as a source of difficulty in each thread, using natural language processing and sentiment analysis techniques. Based on a preliminary manual analysis of the content of a discussion thread and a categorization of the role of each sentence therein, we decide to focus on a negative sentiment sentence and its close neighbors as a unit for extracting API features. We evaluate a set of candidate solutions by comparing tool-extracted problematic API design features with manually produced golden test data. Our best solution yields a precision of 89%. We have also investigated three potential applications for our feature extraction solution: (i) highlighting the negative sentence and its neighbors to help illustrate the main API feature; (ii) searching helpful online information using the extracted API feature as a query; (iii) summarizing the problematic features to reveal the “hot topics” in a forum.

Java objects are required to honor an equality contract in order to participate in standard collection data structures such as List, Set, and Map. In practice, the implementation of equality can be error prone, resulting in subtle bugs. We present a checker called EQ that is designed to automatically detect such equality implementation bugs. The key to EQ is the automated extraction of a logical model of equality from Java code, which is then checked, using Alloy Analyzer, for contract conformance. We have evaluated EQ on four open-source, production code bases in terms of both scalability and usefulness. We discuss in detail the detected problems, their root causes, and the reasons for false alarms.

Learning to use a software framework and its API (Application Programming Interfaces) can be a major endeavor for novices. To help, we have built a critic to advise the use of an API based on the formal semantics of the API. Specifically, the critic offers advice when the symbolic state of the API client code triggers any API usage rules. To assess to what extent our critic can help solve practical API usage problems and what kinds of API usage rules can be formulated, we manually analyzed 150 discussion threads from the Java Swing forum. We categorize the discussion threads according to how they can be helped by the critic. We find that API problems of the same nature appear repeatedly in the forum, and that API problems of the same nature can be addressed by implementing a new API usage rule for the critic. We characterize the set of discovered API usage rules as a whole. Unlike past empirical studies that focus on answering why frameworks and APIs are hard to learn, ours is the first designed to produce systematic data that have been directly used to build an API support tool.

It is well-known that APIs can be hard to learn and use. Although search tools can help find related code examples, API novices still face other significant challenges such as evaluating the relevance of the search results. To help address the broad problems of finding, understanding, and debugging API-based solutions, we have built a critic system that offers recommendations, explanations, and criticisms for API client code. Our critic takes API usage rules as input, performs symbolic execution to check that the client code has followed these rules properly, and generates advice as output to help improve the client code. We demonstrate our critic by applying it to a real-world example derived from the Java Swing Forum.

Code Completion is one of the most popular IDE features for accessing APIs, freeing programmers from remembering specific details about an API and reducing keystrokes. We propose three ways to enhance the current code-completion systems to work more effectively with large APIs. First, we propose two methods for sorting APIs, by type hierarchy and by use count, and show that their use significantly reduces the number of API proposals a user must navigate while using Code Completion. Second, we show that context-specific filtering of inappropriate proposals can also reduce the number of proposals a user must navigate. Third, we propose to group API proposals by their functional roles, which can help maintain a well-ordered, meaningful list of API proposals in the presence of dynamic reordering. These functionalities are grouped into a research prototype, BCC (Better Code Completion). We evaluated fourteen configurations of BCC by simulating Code Completion nearly three million times on nine open-source Java projects that utilize AWT/Swing.

Objects in Object-Oriented languages such as Java are required to implement an equality predicate using the equals(Object) method in order to be compared with each other. This is particularly important when these objects interact with lists, sets, and maps from the Java Collection Framework. There are several considerations that must be taken in the implementation of this method, which, if ignored, will lead to subtle bugs. We present a tool called EQ that analyzes the source code to find such bugs.

Large software frameworks and APIs can be hard to learn and use, impeding software productivity. But what are the specific challenges that programmers actually face when using frameworks and APIs in practice? What makes APIs hard to use, and what can be done to alleviate the problems associated with API usability and learnability? To explore these questions, we conducted an exploratory study in which we manually analyzed a set of newsgroup discussions about specific challenges that programmers had about a software framework. Based on this set of data, we identified several categories of obstacles in using APIs. We discussed what could be done to help overcome these obstacles.

Programmers encounter many difficulties in using an API to solve a programming task. To cope with these difficulties, they browse the Internet for code samples, tutorials, and API documentation. In general, it is time-consuming to find relevant help from the plethora of information on the web. While programmers can use search-based tools to help locate code snippets or applications that may be relevant to the APIs they are using, they still face the significant challenge of understanding and assessing the quality of the search results. We propose to investigate a proactive help system that is integrated in a development environment to provide contextual suggestions to the programmers as the code is being read and edited in the editor.

Copying and pasting source code results in code duplication. A common form of software reuse involves modifying the new duplicate to fit a current task. The similar code fragments (code clones) may be edited inconsistently by the programmer, for various reasons, leaving a bug in the software that may remain undetected by both the programmer and the compiler. A previously published tool, CReN, helps the programmer by automatically renaming all instances of the same identifier consistently within a clone when one is edited. In this tool demo, we introduce an extension of CReN, an Eclipse plug-in named LexId, which renames the same parts of different identifiers consistently together within code clones.

When programmers copy, paste, and then modify source code, the once-identical code fragments (code clones) can become indistinguishable as the software evolves over time. In this paper, we present three features of our software tool, a set of Eclipse plug-ins named CnP (CnP's clone visualization, CReN, and LexId), which aids the programmer during copy-and-paste programming. We believe that the clone-awareness that the tool provides can help programmers benefit from this clone information during debugging and modification tasks, develop software more efficiently, and prevent inconsistent identifier renaming within clones. We tested these hypotheses with a user study and present our results.

Programmers copy and paste code. As a result, similar code fragments (clones) are added into software systems. Like other software artifacts, clones require attention and effort from programmers so that they can be found, understood, and correctly adapted and evolved. In addition to what clone-detection-based tools can offer, other automated support can be developed to better assist programmers in these activities, for example, to compare and contrast code clones, or help edit (a group of) clones consistently and quickly. We describe several such features currently being developed in the CnP project on top of Eclipse and for Java.

Integrated Development Environments (IDEs) help increase programmer productivity by automating much clerical and administrative work. Thus, it is of great research and practical interest to learn about the characteristics on how IDE features change and mature. To this end, we have conducted an empirical study, analyzing a total of 645 ldquoWhat's Newrdquo release note entries in 7 releases of the Eclipse IDE both quantitatively and qualitatively. It is found that majority of the changes are refinements or incremental additions to the feature architecture set up in early releases (1.0 and 2.0). Motivated by this, a further analysis on usability is performed to characterize how these changes impact programmers effectiveness in using the IDE. We summarize our study methodology and lessons learned.

Nowadays, programmers spend much of their workday dealing with code libraries and frameworks that are bloated with APIs. One common way of interacting with APIs is through Code Completion inside the code editor. By default, Code Completion presents in a scrollable list, in alphabetical order, all accessible members available in the apparent type and supertypes of a receiver expression. This default behavior for Code Completion should and can be further improved because (1) not all public methods are APIs and presenting non-API public members to a programmer is misleading, (2) certain APIs are meant to be accessible only in some limited contexts but not others, and (3) the alphabetical order separates otherwise logically related APIs, making it hard to see their connection. BCC (Better Code Completion) addresses these problems by enhancing Code Completion so that programmers can control how specific API elements should be sorted, filtered, and grouped.

Programmers copy and paste code for many reasons. Regardless of the specific reasons, similar code fragments (clones) are introduced into software systems. Like other software artifacts, clones may require attention and effort from programmers so that they can be understood, and correctly adapted and evolved. More specifically, when understanding and maintaining clones, programmers need to know where the clones are. Programmers also need to compare and contrast code clones in order to figure out how they correspond and differ. Finally, they also need to edit or remove clones. In addition to what clone detection-based tools can offer, more automated support is needed to better assist programmers in these activities. In this paper, we introduce a toolkit CnP that is aimed to support and manage clones proactively as they are created and evolved. We describe the initial features and the design decisions taken in CnP. We also discuss possible future design extension.

Scattered concerns, design features whose implementations span multiple program units, can pose extra difficulty for developers to locate, understand, modify, and extend. In particular, since successful application frameworks tend to be widely used by many developers, the impact of scattered concerns on framework usability can be especially significant. Ideally, every scattered concern present at the framework interface should be carefully evaluated to justify its introduction. If it cannot be avoided, it should at least be well-documented to facilitate its use. To gain insights into how scattered concerns are actually designed and documented in industrial frameworks, a method for documenting and evaluating scattered concerns is proposed, and a manual pilot study on the JFC swing JTree is performed. In this method, concerns are identified and documented in a structured manner, listing not only those items which an application must directly depend on, but other methods and classes that indirectly contribute to the design of a concern. The documented concerns are compared with the framework architecture in order to justify their existence. Concerns that map nicely to the architecture are deemed to be acceptable as they fit within the design and were explicitly traded off for other quality attributes. Those that show a mismatch indicate a place where the concern could cause problems for application developers, and thus should be either refactored or well-documented. Concerns are also evaluated using design criteria like cohesion and coupling. While it is not always possible to identify design flaws in a scattered concern, at the least, documenting these concerns will make them easier to use. The JTree study results in 12 concerns, 7 of which are missing or only partially documented in the official swing tutorial. 4 framework usability flaws are identified, which, if addressed, would lead to the removal of 2 of the 12 documented concerns and improvements to another 2. Thus, the proposed method has the potential to be useful to framework development teams.

Studying example applications is a common approach to learning software frameworks. However, to be truly effective in adapting example solutions with high confidence and accuracy, a developer needs to learn enough about the framework designs. The empirical study described in this paper investigates the effectiveness of a new approach to framework learning, where example-based learning is augmented with instruction on framework designs. Learning framework designs up-front from an instructor helps developers acquire the necessary design knowledge and avoid the time-consuming task of recovering such knowledge from code and other artifacts. The particular question of interest in this study is how characteristics of the framework designs influence project outcome. 11 student projects are analyzed using both qualitative and quantitative methods to characterize the overall reuse practice and to detect salient patterns that address the question. The contribution of this paper is a set of well-supported hypotheses that can be tested in future studies as well as their implications.

The usefulness and usability of programming tools (for example, languages, libraries, and frameworks) may greatly impact programmer productivity and software quality. Ideally, these tools should be designed to be both useful and usable.But in reality, there always exist some tools or features whose essential characteristics can be fully understood only after they have been extensively used. The study described in this paper is focused on discovering how C++'s function overloading is used in production code using an instrumented g++ compiler. Our principal finding for the system studied is that the most 'advanced' subset of function overloading tends to be defined in only a few utility modules, which are probably developed and maintained by a small number of programmers, the majority of application modules use only the 'easy' subset of function overloading when overloading names,and most overloaded names are used locally within rather than across module interfaces.We recommend these as guidelines to software designers.

Developers often fail to respect the intentions behind a design due to poor communication of design intent. SCL (Structural Constraint Language) helps capture and confirm aspects of design intent by using structural constraints on a program model extracted through static analysis. The original designer expresses design intent in terms of constraints on the program model using the SCL language, and the SCL conformance checking tool examines developer code to confirm that the code honors these constraints. This paper presents the design of the SCL language and its checker, a set of practical examples where SCL has been applied, and our experience. SCL has a formal foundation, supports a wide range of design intent, is extensible for additional expressive power and checking capabilities, scales to a million lines of code, and is relatively easy to use

Facts extracted from source code have been used to support a variety of software engineering activities, ranging from architectural understanding, through detection of design patterns, to program exploration. Several fact extractors have been developed and published in the literature, but most of them extract facts only from individual compilation units. Linking multiple fact-bases is largely overlooked. Source-level linkage is different from compilation linkage. Its goal is to assist a software engineer, not to produce an executable program. Thus a source-level linker needs to collect as many as possible facts that may be potentially helpful to a software engineer's task, many of which are not available from a compiler linker. We present the design of a source-level linker for C++. This linker has been used to analyze a dozen of Microsoft Foundation Classes (MFC) programs and over 200 C++ programs that cover an extensive subset of C++ features, including templates from the standard template library (STL). As a further validation, we design a structural constraint language, SCL, to express and machine-check a wide range of constraints on the abstract semantics graph (ASG) produced by the linker

Software systems are often written in more than one programming language. During development, programmers need to understand not only the dependencies among code in a particular language, but dependencies that span languages. In this paper, we focus on the problem of scripting languages (such as Perl) and their extension mechanisms to calling functions with a C interface. Our general approach involves building a fact extractor for each scripting language, by hooking into the language interpreter itself. The produced facts conform to a common schema, and an analyzer is extended to recognize the cross-language dependencies. We present how these statically discovered dependencies can be represented, visualized, and explored in the Eclipse environment

Software developers often fail to respect the intentions of designers due to missing or ignored documentation of design intent. SCL (Structural Constraint Language) addresses this problem by enabling designers to formalize and confirm compliance with design intent. The designer expresses his intent as constraints on the program model using the SCL language. The SCL conformance checking tool examines developer code to confirm that the code honors these constraints. This paper presents the design of the SCL language and its checker, a set of practical examples of applying SCL, and our experience with using it both in an industrial setting and on open-source software

Frameworks are usually large and complex, and typically reusers need to understand them well enough to effectively use them. This research concentrates on verifying applications built on top of OO frameworks. The idea is to get framework builders to specify a set of constraints for the correct usage of the framework and check them using static analysis techniques.