Sockpuppet accounts pose significant threats to the integrity of online platform, but research on their detection in Arabic-language content remains limited due to the lack of suitable datasets. This paper introduces the first publicly available Arabic sockpuppet dataset, addressing a critical gap in the current research. The dataset comprises approximately 3,000 distinct accounts labeled as normal, Sockpuppet, or personal spammer. A Human-in-the-Loop labeling approach was adopted, involving 12 trained students as labelers over a six-week period. The labeling process included structured training, batch creation and scheduling, dataset labeling and peer review, and coordinator oversight. Additionally, students feedback was collected through a survey to evaluate the clarity and effectiveness of the labeling framework and instruction. Future work will involve labeling additional accounts, validating the labeling with another cohort and refining the labeling instructions based on the feedback received.

Detecting troll accounts in Online Social Networks (OSN) has become increasingly critical for maintaining reliable online interactions. This study presents a comprehensive analysis of machine learning methods for troll detection on Reddit platform, evaluating seven classifiers, Support Vector Machine, Logistic Regression, Naive Bayes, K-Nearest Neighbors, Random Forest, AdaBoost, and XGBoost, along with hard and soft voting classifiers. Aggregating the outputs of the last three classifiers yielded the best results, achieving 94% accuracy with an F1-score of 94% for hard voting and 96% accuracy with an F1-score of 95% for soft voting, both surpassing prior works. Furthermore, a deep learning model (Deep Neural Network) was implemented and achieved high performance, reaching up to 95% accuracy and F1-score of 93%. In addition to this automated approach, this paper emphasizes the importance of Human-in-the-Loop (HITL) analysis for improving model performance through hyperparameter tuning, feature engineering, and misclassification analysis. Additionally, we propose a hybrid model that integrates automated analysis with a cognitive assistant trained by human expertise. This system facilitates continuous learning of troll patterns and Sockpuppet behaviors broadly. Future work will focus on implementing and validating this framework across larger datasets, aiming to develop a more accurate, adaptive, and resilient troll detection system.

Student failure and dropout in universities are detrimental in many ways, costing $1700 on average in tuition fees, risking future financial aid, and prompting issues in mental health and motivation. Considering the 220 million active users and sub-5% course completion rate in Massive Open Online Courses (MOOCs), billions of dollars are lost in wasted tuition and millions of student dropouts occur each year. Current industry systems and academic research are targeting high performance but providing little meaningful feedback for at-risk students, which is especially needed in large mandatory university courses or MOOCs. This paper proposes EduBoost, a novel, intrinsically interpretable solution boasting comparable performance to Black-Box models. EduBoost contains a voting ensemble model formed using the best 3 classifiers out of 4 tested Black-Box models (SVM, RF, MLP, KNN), a White-Box classifier (CART), and two distinct Grey-Box model pipelines that combine the joint model and White-Box. The first Grey-Box pipeline is a basic interpretable mimic learning approach that augments labels of the training data using a Black-Box model. The second Grey-Box pipeline is an iterative modeling approach that uses a confidence threshold to filter out extraneous instances that could occur from using the first pipeline. EduBoost is tested on two datasets: 1) student data based on a mandatory university course and 2) a MOOC provided in the Open University Learning Analytics Dataset (OULAD), containing various features that gauge student performance and effort in class, i.e., grades, attempts, procrastination, etc. EduBoost showed increases of 9–11% in accuracy, precision, recall, and f1 on the OULAD over the baseline White-Box model and increases of 2–3% in those same metrics on the university dataset. Our results yield a few major findings. First, our joint Black-Box model successfully outperforms all individual learners. Second, the proposed modeling approach in EduBoost can display performance comparable to Black-Box models, consistently outperforming White-Box models while also maintaining interpretability. Though the first Grey-Box pipeline was shown to always improve performance on both datasets, the second Grey-Box pipeline has poor performance on the smaller university dataset but beats out the first Grey-Box pipeline on the OULAD, suggesting that there is a minimum amount of data instances for the second Grey-Box pipeline to function well. Finally, EduBoost's intrinsic interpretability helps to maintain IF-THEN interpretability, making it simpler and more comprehensible than post-hoc or correlative methods of interpretability.

This paper overviews an end-to-end learning-based approach to the rapid development of practical cognitive assistants for intelligence analysis. A learning agent shell has been trained by a knowledge engineer with general evidence-based reasoning knowledge for intelligence analysis. This agent is further trained by an expert analyst how to analyze complex hypotheses from a given intelligence analysis domain. The resulting cognitive assistant is used by a typical analyst to rapidly analyze hypotheses from agent's area of expertise. During its use, the agent continues to learn reasoning patterns from its user. This approach has been implemented and practical agents have been developed and used. This is a significant application of machine learning to agents development in intelligence analysis that can be generalized to many other domains involving evidence-based reasoning, including medicine, law, and science.

Our research addresses the issue of developing knowledge-based agents that capture and use the problem solving knowledge of subject matter experts from diverse application domains. This paper emphasizes the use of negative examples in agent learning by presenting several strategies for capturing expert's knowledge when the agent fails to correctly solve a problem. These strategies have been implemented into the disciple learning agent shell and used in complex application domains such as intelligence analysis, center of gravity determination, and emergency response planning.

This paper presents recent results on developing learning agents that can be taught by subject matter experts how to solve problems, through examples and explanations. It introduces the lazy rule refinement method where the expert modifies an example generated by a learned rule. In this case the agent has to decide whether to modify the rule (if the modification applies to all the previous positive examples) or to learn a new rule. However, checking the previous examples would be disruptive or even impossible. The lazy rule refinement method provides an elegant solution to this problem, in which the agent delays the decision whether to modify the rule or to learn a new rule until it accumulated enough examples during the follow-on problem solving process. This method has been incorporated into the disciple learning agent shell and used in the complex application areas of center of gravity analysis and intelligence analysis

Over the years we have developed the Disciple theory, methodology, and family of tools for building knowledge-based agents. This approach consists of developing an agent shell that can be taught directly by a subject matter expert in a way that resembles how the expert would teach a human apprentice when solving problems in cooperation. This paper presents the most recent version of the Disciple approach and its implementation in the Disciple–RKF (rapid knowledge formation) system. Disciple–RKF is based on mixed-initiative problem solving, where the expert solves the more creative parts of the problem and the agent solves the more routine ones, integrated teaching and learning, where the agent helps the expert to teach it, by asking relevant questions, and the expert helps the agent to learn, by providing examples, hints, and explanations, and multistrategy learning, where the agent integrates multiple learning strategies, such as learning from examples, learning from explanations, and learning by analogy, to learn from the expert how to solve problems. Disciple–RKF has been applied to build learning and reasoning agents for military center of gravity analysis, which are used in several courses at the US Army War College.

This paper presents the Disciple-RKF methodology for rapid development of large knowledge bases which relies on importing ontological knowledge from existing knowledge repositories, on parallel development of separate knowledge bases by subject matter experts, and on the merging of these knowledge bases into a high performance integrated knowledge base. The paper discusses several issues related to ontology import and merging, and presents the results of a successful knowledge base development and integration experiment performed at the US Army War College.

This paper presents current results in developing a practical approach, methodology and tool, for the development of knowledge bases and agents by subject matter experts, with limited assistance from knowledge engineers. This approach is based on mixed-initiative reasoning that integrates the complementary knowledge and reasoning styles of a subject matter expert and a learning agent, and on a division of responsibilities for those elements of knowledge engineering for which they have the most aptitude. The approach was evaluated at the US Army War College, demonstrating very good results and a high potential for overcoming the knowledge acquisition bottleneck.