Automatic speech scoring is crucial in language learning, providing targeted feedback to language learners by assessing pronunciation, fluency, and other speech qualities. However, the scarcity of human-labeled data for languages beyond English poses a significant challenge in developing such systems. In this work, we propose a Language-Independent scoring approach to evaluate speech without relying on labeled data in the target language. We introduce a multilingual speech scoring system that leverages representations from the wav2vec 2.0 XLSR model and a force-alignment technique based on CTC-Segmentation to construct speech features. These features are used to train a machine learning model to predict pronunciation and fluency scores. We demonstrate the potential of our method by predicting expert ratings on a speech dataset spanning five languages - English, French, Spanish, German and Portuguese, and comparing its performance against Language-Specific models trained individually on each language, as well as a jointly-trained model on all languages. Results indicate that our approach shows promise as an initial step towards a universal language independent speech scoring.

The most popular way to assess talent around the world is through interviews. Interviewers contribute substantially to candidate experience in many organizations' hiring strategies. There is a lack of comprehensive understanding of what makes for a good interview experience and how interviewers can conduct candidate-centric interviews. An exploratory study with 123 candidates revealed critical metrics about interviewer behavior that affects candidate experience. These metrics informed the design of our AI-driven SmartView system that provides automated post-interview feedback to Interviewers. Real-world deployment of the system was conducted for three weeks with 35 interviewers. According to our study, most interviewers found that SmartView insights helped identify areas for improvement and could assist them in improving their interviewing skills.

Evaluators wish to test candidates on their ability to propose the correct algorithmic approach to solve programming problems. Recently, several automated systems for grading programs have been proposed, but none of them address uncompilable codes. We present the first approach to grade uncompilable codes and provide semantic feedback on them using machine learning. We propose two methods that allow us to derive informative semantic features from programs. One of this approach makes the program compilable by correcting errors, while the other relaxes syntax/grammar rules to help parse uncompilable codes. We compare the relative efficacy of these approaches towards grading. We finally combine them to build an algorithm which rivals the accuracy of experts in grading programs. Additionally, we show that the models learned for compilable codes can be reused for uncompilable codes. We present case studies, where companies are able to hire more efficiently by deploying our technology.

Data-driven decision making is fast becoming a necessary skill in jobs across the board. The industry today uses analytics and machine learning to get useful insights from a wealth of digital information in order to make decisions. With data science becoming an important skill needed in varying degrees of complexity by the workforce of the near future, we felt the need to expose school-goers to its power through a hands-on exercise. We organized a half-day long data science tutorial for kids in grades 5 through 9 (10-15 years old). Our aim was to expose them to the full cycle of a typical supervised learning approach - data collection, data entry, data visualization, feature engineering, model building, model testing and data permissions. We discuss herein the design choices made while developing the dataset, the method and the pedagogy for the tutorial. These choices aimed to maximize student engagement while ensuring minimal pre-requisite knowledge. This was a challenging task given that we limited the pre-requisites for the kids to the knowledge of counting, addition, percentages, comparisons and a basic exposure to operating computers. By designing an exercise with the stated principles, we were able to provide to kids an exciting, hands-on introduction to data science, as confirmed by their experiences. To the best of the authors' knowledge, the tutorial was the first of its kind. Considering the positive reception of such a tutorial, we hope that educators across the world are encouraged to introduce data science in their respective curricula for high-schoolers and are able to use the principles laid out in this work to build full-fledged courses.

Motor skills are required in a large number of vocational jobs today. However, no automated means exist to test and provide feedback on these skills. In this paper, we explore the use of touch-screen surfaces and tablet-apps to measure these skills. We design novel gamified apps to predict the performance of candidates in doing manual tasks in the industry. We demonstrate two important results - we use the information captured on a touch-screen device to successfully predict the scores of traditional, non-automated motor skill tests. Further, we show that this information successfully predicts the performance of workers in their respective jobs. The results presented in this work make a strong case for using such automated, touchscreen based apps in job selection and to provide automatic feedback. To the best of the authors' knowledge, this is the first attempt at using touch-screen devices to scalably and reliably measure motor skills.

Learning supervised models to grade open-ended responses is an expensive process. A model has to be trained for every prompt/question separately, which in turn requires graded samples. In automatic programming evaluation specifically, the focus of this work, this issue is amplified. The models have to be trained not only for every question but also for every language the question is offered in. Moreover, the availability and time taken by experts to create a labeled set of programs for each question is a major bottleneck in scaling such a system. We address this issue by presenting a method to grade computer programs which requires no manually assigned labeled samples for grading responses to a new, unseen question. We extend our previous work [25] wherein we introduced a grammar of features to learn question specific models. In this work, we propose a method to transform those features into a set of features that maintain their structural relation with the labels across questions. Using these features we learn one supervised model, across questions for a given language, which can then be applied to an ungraded response to an unseen question. We show that our method rivals the performance of both, question specific models and the consensus among human experts while substantially outperforming extant ways of evaluating codes. We demonstrate the system single s value by deploying it to grade programs in a high stakes assessment. The learning from this work is transferable to other grading tasks such as math question grading and also provides a new variation to the supervised learning approach.

Vinay Shashidhar, Nishant Pandey, Varun Aggarwal. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

In this paper, we address the problem of grading spontaneous speech using a combination of machine learning and crowdsourcing. Traditional machine learning techniques solve the stated problem inadequately as automatic speaker-independent speech transcription is inaccurate. The features derived from it are also inaccurate and so is the machine learning model developed for speech evaluation. We propose a framework that combines machine learning with crowdsourcing. This entails identifying human intelligence tasks in the feature derivation step and using crowdsourcing to get them completed. We post the task of speech transcription to a large community of online workers (crowd). We also get spoken English grades from the crowd. We achieve 95% transcription accuracy by combining transcriptions from multiple crowd workers. Speech and prosody features are derived by force aligning the speech samples on these highly accurate transcriptions. Additionally, we derive surface and semantic level features directly from the transcription. We demonstrate the efficacy of our approach by predicting expert graded speech sample of 566 adult non-native speakers across two different countries - India and Philippines. Using the regression modeling technique, we are able achieve a Pearson correlation of 0.79 on the Philippines set and 0.74 on the Indian set with expert grades, an accuracy much higher than any previously reported machine learning approach. Our approach has an accuracy that rivals that of expert agreement. We show the value of the system through a case study in a real-world industrial deployment. This work is timely given the huge requirement of spoken English training and assessment.

In the current paper, it has been demonstrated through experimental results, historical citations and technical arguments that coherer (including cat's whisker) is the elusive memristor. The current paper draws parallel between the research pursued by scientists in the field of memristor and that in coherer, around a century back. The paper discusses various important milestones in the research of that era along with memristor research and how the scientific community missed to identify the memristor despite being very close to the discovery. The paper demonstrates that the first radios were made of memristor and provides novel applications of the same.

The automatic evaluation of computer programs is a nascent area of research with a potential for large-scale impact. Extant program assessment systems score mostly based on the number of test-cases passed, providing no insight into the competency of the programmer. In this paper, we present a system to grade computer programs automatically. In addition to grading a program on its programming practices and complexity, the key kernel of the system is a machine-learning based algorithm which determines closeness of the logic of the given program to a correct program. This algorithm uses a set of highly-informative features, derived from the abstract representations of a given program, that capture the program's functionality. These features are then used to learn a model to grade the programs, which are built against evaluations done by experts. We show that the regression models provide much better grading than the ubiquitous test-case-pass based grading and rivals the grading accuracy of other open-response problems such as essay grading . We also show that our novel features add significant value over and above basic keyword/expression count features. In addition to this, we propose a novel way of posing computer-program grading as a one-class modeling problem and report encouraging preliminary results. We show the value of the system through a case study in a real-world industrial deployment. To the best of the authors' knowledge, this is the first time a system using machine learning has been developed and used for grading programs. The work is timely with regard to the recent boom in Massively Online Open Courseware (MOOCs), which promises to produce a significant amount of hand-graded digitized data.

Digital integrators (DIs) and digital differentiators (DDs) of second, third and fourth‐order based on particle swarm optimisation (PSO) algorithm are presented. A modified particle swarm optimisation (MPSO) algorithm with reducing maximum velocity has been used to optimise the mean square error of the digital operators. Statistical and simulation results have been presented for comparing quality of optimal operators obtained by MPSO, genetic algorithm (GA), two variants of PSO and PSO‐GA hybrid techniques. The results obtained for best solutions by the proposed algorithm are either superior or at par with the basic PSO variants and hybrid techniques. The proposed digital operators have also been simulated using MATLAB, and the results have been compared with that of existing DIs and DDs derived by different optimisation algorithms, to demonstrate the effectiveness of the use of proposed MPSO. The relative magnitude errors (dB) obtained for digital integrators and differentiators are as low as −40 and −35 dB, respectively, which are valid for almost the full band of normalised frequency.

The automatic evaluation of computer programs is a nascent area of research with a potential for large-scale impact. Extant program assessment systems score mostly based on the number of test-cases passed, providing no insight into the competency of the programmer. In this paper, we present a machine learning framework to automatically grade computer programs. We propose a set of highly-informative features, derived from the abstract representations of a given program, that capture the program's functionality. These features are then used to learn a model to grade the programs, which are built against evaluations done by experts on the basis of a rubric. We show that regression modeling based on the given features provide much better grading than the ubiquitous test-case-pass based grading and rivals the grading accuracy of other open-response problems such as essay grading. We also show that our novel features add significant value over and above basic keyword/expression count features. In addition to this, we propose a novel way of posing computer-program grading as a one-class modeling problem. Our preliminary investigations in the same show promising results and suggest an implicit correlation of our features with the proposed grading-levels (rubric). To the best of the authors' knowledge, this is the first time machine learning has been applied to the problem of grading programs. The work is timely with regard to the recent boom in Massively Online Open Courseware (MOOCs), which promises to produce a significant amount of hand-graded digitized data.

The paper presents an algorithm to automatically design posynomial models for parameters of the MOS transistors using simulation data. These models improve the accuracy of the geometric programming flow for automatic circuit sizing. The models are reusable for multiple circuits on a given silicon technology and hence don't adversely affect the scalability of the geometric programming approach. The proposed method is a combination of genetic algorithms and quadratic programming. It is the only approach for posynomial modeling with real-valued exponents which is easily extensible to different error metrics. The authors compare the proposed technique with state-of-art posynomial/monomial modeling techniques and show its superiority

This contribution is the first to discover exploitable structural features within circuit optimization problems (COP) and discuss how it is indicative of a general structure and possibly a 'measure of hardness' in real-world multi-objective optimization problems. We then present a methodology to exploit this structure in a multi-objective evolutionary algorithm by designing a novel Correlation Sensitive Mutation Operator, COSMO. COSMO is, at the least, universally applicable in the domain of circuits and we discuss how it can be easily extended to other domains. We discuss the rationale behind COSMO and interpret it in context of dimensional locality. We compare COSMO's performance with the traditional operators used for multi-objective optimization. For two instances of circuits, we show that COSMO gives significantly faster and better optimization than conventional operators. The paper also takes the first steps in thinking and interpreting how operators for MO-EAs should be designed.

We present a genetic algorithm which is distributed in two novel ways: along genotype and temporal axes. Our algorithm first distributes, for every member of the population, a subset of the genotype to each network node, rather thana subset of the population to each. This genotype distribution is shown to offer a significant gain in running time. Then, for efficient use of the computational resources in the network, our algorithm divides the candidate solutions intopipelined sets and thus the distribution is in the temporal domain, rather that in the spatial domain. This temporal distribution may lead to temporal inconsistency in selection and replacement, however our experiments yield better efficiency in terms of the time to convergence without incurring significant penalties.

Abstract — We consider the problem of minimizing the resources used for network coding while achieving the desired throughput in a multicast scenario. Since this problem is NPhard, we seek a method for quickly finding sufficiently good solutions. To this end, we take evolutionary approaches based on a Genetic Algorithm. In this paper, we extend the evolutionary algorithm that we previously proposed in three perspectives. First, whereas the previous algorithm can be applied to only acyclic networks, we devise a modified evaluation method that works also with networks with cycles. Second, we introduce a new set of GA components that in our experiments outperforms the one used in the previous algorithm. Third, we present a new framework of the evolutionary approach, where fitness evaluation and population management are done in a decentralized manner with a limited amount of coordination. The new framework enables a network coding protocol where the resources used for coding are optimized in the setup phase as the proposed evolutionary algorithm being loaded and run at each node of the network. We demonstrate the effectiveness of our algorithms by carrying out simulations on a number of different sets of network topologies. I.

We demonstrate that particle swarm optimization (PSO) can be successfully used to evolve high performance filter approximations. These evolved approximations use sets of quantitative specifications which conventional analytically derived approximations can not directly employ. The conventional derivations use only a subset of the quantitative specifications in their algorithm and the remaining specifications are side-effect results of the algorithm. Thus, with PSO, instead of a filter designer having access to a limited set of “ specification knobs ” that directly and indirectly achieve performance, a designer has a ”knob ” for each specification that consequently drives the approximation to the desired performance. Categories and Subject Descriptors B.7.2 [Hardware]: Integrated Circuits-Design Aids The primary performance criterion of a filter is its frequency domain magnitude response (henceforth called, magnitude response, defined in Section 3.4). A second set of performance criteria are time domain characteristics. These can be tested by examining the step and impulse responses of the filter. Ideally there should be no oscillations in the step response, a small overshoot in case of oscillations, a low rise time and settling time. A filter meeting the brickwall characteristic has ideal frequency band selection but will oscillate on being activated by a step (in addition to voiding causality). Another criterion is a linear phase response in frequency domain (henceforth called phase response). Finally, because there is a tradeoff between filter complexity (i.e. order) and implementation feasibility, complexity is a performance criterion. 1 0

Abstract Background Protein sequence alignment is one of the basic tools in bioinformatics. Correct alignments are required for a range of tasks including the derivation of phylogenetic trees and protein structure prediction. Numerous studies have shown that the incorporation of predicted secondary structure information into alignment algorithms improves their performance. Secondary structure predictors have to be trained on a set of somewhat arbitrarily defined states (e.g. helix, strand, coil), and it has been shown that the choice of these states has some effect on alignment quality. However, it is not unlikely that prediction of other structural features also could provide an improvement. In this study we use an unsupervised clustering method, the self-organizing map, to assign sequence profile windows to "structural states" and assess their use in sequence alignment. Results The addition of self-organizing map locations as inputs to a profile-profile scoring function improves the alignment quality of distantly related proteins slightly. The improvement is slightly smaller than that gained from the inclusion of predicted secondary structure. However, the information seems to be complementary as the two prediction schemes can be combined to improve the alignment quality by a further small but significant amount. Conclusion It has been observed in many studies that predicted secondary structure significantly improves the alignments. Here we have shown that the addition of self-organizing map locations can further improve the alignments as the self-organizing map locations seem to contain some information that is not captured by the predicted secondary structure.