Despite Siamese trackers’ substantial potential, they offer sub-optimal tracking performance in low-resolution (LR) contexts. We introduce a Random Noise Salient Feature Fusion Learning Network to address this issue. This method integrates random noise-infused feature maps into a similaritylearning matching model. This integration acts as an effective regularization technique, enhancing the network’s generalization capabilities in LR environments. Additionally, by integrating attention mechanisms, we enhance the discriminative ability of the network, assigning more weights to important features. This directs the network’s focus toward the most salient regions of the feature map, ensuring improved accuracy without a significant increase in parameter overhead, and maintaining a high operating speed. To validate the effectiveness of our method, we performed qualitative and quantitative comparisons with state-of-the-art (SOTA) trackers.

Motor skills are crucial for young children’s academic and social growth, making early detection of motor skill delays critical. Although the current work in digital assessments and tablets in classrooms offer accessible diagnostic tools, they only provide general diagnostics. This leaves teachers to identify specific deficiencies, create lesson plans, and monitor progress, which is challenging due to high teacher-to-student ratios. We take a step towards an assessment that can identify development levels in specific instances of writing through the analysis of the relationship between shape drawings and the letter development levels of children in specific instances. In this study, we collected 133 letter writings and 339 shape drawings from 58 unique participants ages 3-5. Every letter writing was categorized as either developing or developed, and each letter was paired with the corresponding set of Rubine on Steroid features extracted from a series of circle, square, and triangle drawings. We grouped each of the features by development level of letter for the A, B, and C and calculated the Pearson Correlation Coefficient for each feature. We then took the top 15 correlated features and ran Unpaired T-tests to assess whether each feature exhibited significant differences among the different developmental groups. We found a weak to moderate relationship between the features and 40 out of 45 features demonstrated statistically significant differences when comparing them across the different developmental groups. Additionally, A further look into these features yielded insight into different aspects of shape drawings and their respective characteristics and their possible relationship to the developmental level of each letter.

Contribution: In this work-in-progress paper we describe the process of creating and validating a conceptual assessment in the field of sedimentology for undergraduate geoscience courses. The mechanism can aid future geoscience educators and researchers in the process of academic assessment development aligned with learning objectives in these courses. Background: Prior literature review supports the benefits of using active learning tools in STEM (Science, Technology, Engineering, and Mathematics) courses. This paper is part of a larger project to develop and incorporate research-based active learning software in sedimentology and other geoscience courses to improve grade point average (GPA) and time to graduation for Hispanic students at Texas A&M University. To evaluate the novel tool, we designed and validated the conceptual assessment instrument presented in this work. Research Question: What is the process to develop and validate a conceptual assessment for sedimentology? Methodology: This paper follows quantitative analysis and the assessment triangle approach and focuses on cognition, observation, and interpretation to design and evaluate the conceptual assessment. In the cognition element of the triangle, we explain the mechanism for creating the assessment instrument using students' learning objectives. The observation element explains the mechanism of data collection and the instrument revision. The interpretation element explains the results of the validation process using item response theory and reliability measures. We collected the conceptual assessment data from 17 participants enrolled in two courses where sedimentology topics are taught. Participants were geology majors in one of the courses and engineering majors in the other. Findings: The team developed a conceptual assessment that included eight multiple-choice (MCQ) and four open-ended response questions. The results of the design process described the conceptualization of questions and their validation. Also, the validity of created rubrics was established using inter-rater reliability measures, which showed good agreement between raters. Additionally, the results of the validation process indicated that the conceptual assessment was designed for students with average abilities.

This is a WIP progress paper belongs to innovative practice section. According to Bandura, self-efficacy refers to “beliefs in one's capabilities to organize and execute the courses of action required to produce given attainments.” Self-efficacy plays an essential role in skill development and engineering education. Sketching is a critical skill for engineering students, and it has various benefits, especially in developing spatial visualization skills. Sketching enables students to effectively communicate graphically, represent ideas, and brainstorm ideas, especially in the early design stages. No formal research has been conducted concerning the role of self-efficacy in learning to sketch. An instrument that measures sketching self-efficacy is beneficial for gaining insight into the levels of sketching self-efficacy of engineering students. A sketching self-efficacy instrument was developed for high school students, however it needs revision and validation for use in undergraduate engineering classrooms. This work-in-progress study aims to revise the sketching self-efficacy survey instrument items based on expert feedback to ensure the validity of the instrument's content. We have gathered feedback from nine engineering instructors from various disciplines who are experts in sketching on the items of the current sketching self-efficacy instrument. Our research question is: In what areas do engineering students need to develop self-efficacy in sketching according to engineering instructors, and how do these areas vary across mechanical and industrial engineering disciplines? We conducted five interviews virtually and semi-structured through the Zoom platform. The interview transcripts were anonymized, cleaned, and loaded into qualitative data analysis software, MAXQDA. Qualitative contnet analysis was utilized to analyze the interview data. Based on the interview analysis results, we will discuss the recommended changes to the sketching self-efficacy instrument. Our final study will include psychometric analysis, such as exploratory Factor analysis and Confirmatory factor analysis, to test the instrument's validity for college students.

The past several decades have seen significant advances in monitoring older adults' health and well-being. However, creating viable, practical monitoring systems for informing caregivers requires understanding which behaviors and signs to track and what approaches best present that information. To investigate how technology can be leveraged to better augment caregivers' workflows, we take a multi-stage, qualitative approach to gain insights into the needs of caregivers and the older adults receiving care. Specifically, we use a series of domain expert interviews, cognitive walkthroughs, and semi-structured interviews with residents, and we synthesize our takeaways using thematic analysis at each phase. Our results show that this type of monitoring technology has great potential to reduce the effort needed by caregivers to complete their responsibilities and communicate with their teams. Additionally, we found that older adults are receptive to the technology but their privacy and autonomy must be prioritized for the sake of their mental wellbeing. These insights will facilitate greater intelligent interface development for Person-Centered Care by identifying important design considerations and vital features that require system support.

This paper represents a survey of Artificial Emotional Intelligence (AEI) for cooperative social human-machine interactions. Additionally, it explores the potential benefits and challenges of integrating AEI into various applications, including virtual assistants, customer service, and healthcare. To imbue machines with the ability to understand human emotions, interest in AEI has grown to develop machines capable of natural human-machine interactions. The rapid advancement of Artificial Intelligence (AI) has enabled machines to perform tasks that were once exclusive to humans. However, current AI systems cannot understand emotions and social cues, which limits their ability to interact with humans. Social cues refer to the various nonverbal and verbal signals that people use to convey their emotions, intentions, and thoughts during social interactions. These cues include facial expressions, tone of voice, gestures, body language, and other subtle indicators that humans use to communicate with each other. There are several challenges associated with developing AEI systems that can navigate the complexity of human social interactions and emotions. One of the biggest challenges is developing machine learning algorithms that can accurately recognize and interpret human emotions. Emotions are complex, often subtle, and heavily context-dependent. Machines must recognize and respond to them in a way similar to how humans do through contextual, nonverbal, and verbal cues. Another challenge is developing natural language processing algorithms that accurately recognize the sentiment behind spoken or written communication. Sentiment analysis is a challenging problem because language is inherent ambiguity, and the meaning of a word or phrase can depend on situational, historical, and cultural context. Sentiment can depend on subtle factors such as tone of voice, facial expressions, and body language. Despite these challenges, there are also significant opportunities for improving the quality of human-machine interactions. By developing machines that recognize and respond to human emotions, we can achieve the goal of creating more natural and satisfying interactions between humans and machines. This work-in-progress paper is a preliminary analysis of the technical challenges and opportunities of developing AEI and assesses potential implications for human-machine interaction (e.g., human trust in technology). The paper presents future research directions for AEI, highlighting the need for interdisciplinary collaborations to develop a more human-centered and responsible AI system, while surveying the technology and the potential impacts of human trust in technology. In this literature review, the guiding question is: Can a method for AI in combination with interdisciplinary collaborations be developed to take a logical leap to make a determination of a human's emotional state for a more human-centered and responsible AI system.

Students taking an introductory class in engineering, especially those who are required to take a “coding class” for their non-computer science major can be very intimidated. The thought of learning to program or being grouped with peers who have coding experience or have otherwise been involved with computers can leave many students feeling behind. Some engineering courses, like Computer Science (CS) service classes, will continue to encompass students from many diverse majors, as more degree plans require a coding class. Coding does not always come easy for these students. This paper examines assignment choice in an undergraduate CS-1 course but can be applied to many engineering service courses. Guided by the Self Determination Theory, the authors will implement assignment choice as a means for students to plan a path choosing different assignments to meet the course learning objectives. We will integrate the choices in the curriculum into Canvas, the Learning Management System, and provide a course grade calculator to students. Students receive a customized course road map to visualize their path through the assignments and to plan their own road to success. The approach applies to students' own internal motivation, whether intrinsic or extrinsic, by allowing students to choose their own path or set of assignments to complete the course. By allowing choice, the assignment set is more likely to meet the various interests of the students. This work-in-progress paper will show preliminary results for semesters where assignment choice was part of the course vs. semesters where all students followed the same path. We have observed a decrease in the DFQ (students who earn a D, an F, or Q-drop a class) rate in current semesters with a population size of 200 students with the traditional course delivery vs. 300 students with the Assignment Choice delivery. We will define future areas to study and improve. Our goal of Research-to-Practice is to come up with a framework that allows the instructor to ensure all learning objectives of a class are met, while students still have some choice in the assignments. We believe the output of this paper will provide a framework that will allow instructors to meet the goal of students learning the course objectives and passing their course so that they can continue in their chosen field of study.

This Work In Progress research investigates sketching and visualization experts' perspectives on the definitions and alignment of object assembly sketching exercises with experience from their professional practice. Learning to sketch is a key skill for developing strong visualization and spatial reasoning skills, as well as communication, representation, idea generation, and idea fluency during engineering design. However, manual sketching has largely been replaced by computer graphics tools in undergraduate engineering classrooms. The expert feedback of architecture, civil engineering, and mechanical engineering instructors are reported on relative importance of eight sketching skills, as well as grading practices and discipline-specific practices. Experts generally valued shape quality metrics over line quality, and suggested new interpretations of rubric levels and criteria. We discuss recommended changes to the rubric and exercises.

In this work-in-progress paper we explored the effect of active learning software FossilSketch, and variations in learning modes on Hispanic students' learning. Our institutional data showed that Hispanic students have consistently lower GPAs at graduation, and their average grades are lower across all geological courses, compared to all other ethnicities. However, in 2021, when students were in hybrid learning mode, we observe improvement in grades across all geological courses for Hispanic students. FossilSketch is an educational software application for teaching undergraduate micropaleontology, introduced at the University starting in 2021. Analysis of Hispanic student survey responses showed that their perception of learning micropaleontology improved with the introduction of FossilSketch. When looking at class mode, we conclude that hybrid mode was the most favourable learning mode for Hispanic students, when the flexibility of class mode during 2021–2022 and use of FossilSketch improved Hispanic students' attitudes to learning, which is additionally supported by improved assessment scores in the class where FossilSketch was used, and across all geological courses.

Despite the role that Software Quality Assurance (SQA) plays in developing a quality software product, the techniques taught in undergraduate software engineering (SE) courses are learned superficially. This is evident from new professionals entering industry who lack the skills to consistently implement SQA practices [2] The purpose of this study is to assess the effectiveness of a technique called audits in improving students' ability to learn the SQA process. Organizations such as the International Organization for Standardization (ISO) use audits to assess an organization's development process maturity [17]. These audits follow a systematic process which involves a verbal inquiry of the workflow of the specified development process to produce product feature(s). In this work-in-progress (WIP) paper, we hypothesize that the use of audits as a pedagogy technique improves students' ability to learn SQA techniques and will allow them to apply them in industry projects. We also hypothesize that students' learning of SQA through audits will have a positive correlation with their ability to develop quality products. This paper advances the literature on the use of audits, used in industry to institutionalize processes effective in producing quality products, as a pedagogical technique in an undergraduate SE course at Texas A&M University, a large R1 university.

Understanding the current curricular state of undergraduate engineering program curricula is essential to the growth and evolution of engineering as a discipline. By characterizing various institutions that exist in our nation we can better understand how those programs might be uniquely affecting the training of engineering students. Currently little work has been done to explicitly compare engineering degree program curricula in mechanical, electrical, and civil engineering, but the work that has been done is limited in scope and depth. More importantly, no work as of yet has attempted to examine whether there are significant differences in how minority serving universities might be uniquely preparing their students or if there are innate advantages at more “prestigious”, degree programs of some universities. Understanding the differences between degree programs of universities of varying minority serving levels and university types could bring some insight as to how all universities might better serve diverse student populations on their campuses. This work will examine an assortment of university types that provide mechanical, civil, and electrical engineering program options for their students. These universities span a range of institutional types such as the Morrill Act of 1862 Land Grant institutions, Historically Black College and Universities (HBCUs), Hispanic Serving Institutions (HSIs), and Association of American Universities (AAU) membership universities. This work in progress aims to begin an inquiry into this topic with an ultimate goal of developing a repeatable methodology to assess engineering curricula from the three largest degree granting disciplines of Mechanical, Electrical, and Civil engineering.

A group of engineering educators at Texas A&M University participating in a book club professional development experience. This book club has listened to and discussed several books and movies including the books “Uncomfortable Conversations With a Black Man” by Emmanuel Acho, “Pedagogy of the Oppressed” by Paulo Friere, “Inclusive Conversations” by Mary-Frances Winters, and the movies “Picture a Scientist” and “Coded Bias.” Engineering in the academe is at a critical juncture considering the frequency and acceptance of difficult conversations related to society and social justice having a place in the discipline. Historically, Engineering has steered away from uncomfortable difficult conversations, but it is time to shed our old skin of a habitual depoliticization and emerge free from this restriction to develop our next generation of engineers. Not only are these types of discussions and conversations essential to ensure that the disci-pline continues to become more diverse, inclusive, and equitable, but these discussions better prepare engineers to consider societal factors when they ultimately are designing products, services, and tools for society as a whole. Several thousand engineers are graduating from undergraduate programs every year with little to no discussion of social justice in their programs This study will identify the impact participation in this book club has had in professionally developing faculty participants to engage in uncomfortable discussions and how they might have attempted to integrate societal issues into the courses that they teach at Texas A&M University. Interviews were conducted with members of this book club via Zoom and transcribed automatically. Those transcripts were then analyzed using thematic analysis to identify the major recurrent themes amongst participants. This work builds upon a previous publication by the authors that utilized a focus group design paired with a quantitative Likert scale survey to gauge perceptions of the book club itself, but did not focus on how faculty participants were taking this information and experiential learning into their own classrooms This work will expand the researcher's understanding of the impact of this book club to include this concept of classroom integration of societal challenges that often is overlooked by engineering as a discipline.

From an early age, children begin developing critical motor skills, such as fine motor control, that contribute significantly to reading, writing, drawing, and more, all of which are important for communication and school readiness. Pediatricians can evaluate a child’s motor skills using activities and questionnaires. Sometimes these involve adults drawing with their child, but it can be difficult to fully evaluate a child’s drawings through a handful of sketches from limited direct assessments. We propose creating a sketching system that will collect free-form drawing data from parents and children that can then automatically differentiate a child’s sketch from an adult’s using only the pen strokes of their drawing. In this paper, we describe our study that collected sketches from 14 children aged 2 to 5 and 25 adults over 18. We contribute a machine learning classifier based on sketch recognition features from free-hand drawings capable of distinguishing children’s sketches from those made by adults with an F-measure of 0.906. These results indicate the potential of creating sketch-based applications for assessing children’s fine motor development.

This Research Work In Progress Paper examines empirical evidence on the impacts of feedback from an intelligent tutoring software on sketching skill development. Sketching is a vital skill for engineering design, but sketching is only taught limitedly in engineering education. Teaching sketching usually involves one-on-one feedback which limits its application in large classrooms. To meet the demands of feedback for sketching instruction, SketchTivity was developed as an intelligent tutoring software. SketchTivity provides immediate personalized feedback on sketching freehand practice. The current study examines the effectiveness of the feedback of SketchTivity by comparing students practicing with the feedback and without. Students were evaluated on their motivation for practicing sketching, the development of their skills, and their perceptions of the software. This work in progress paper examines preliminary analysis in all three of these areas.

This Research Work-In-Progress reports the implementation of an Object Assembly Test for sketching skills in an undergraduate mechanical engineering graphics course. Sketching is essential for generating and refining ideas, and for communication among team members. Design thinking is supported through sketching as a means of translating between internal and external representations, and creating shared representations of collaborative thinking. While many spatial tests exist in engineering education, these tests have not directly used sketching or tested sketching skill. The Object Assembly Test is used to evaluate sketching skills on 3-dimensional mental imagery and mental rotation tasks in 1- and 2-point perspective. We describe revisions to the Object Assembly Test skills and grading rubric since its pilot test, and implement the test in an undergraduate mechanical engineering course for further validation. We summarize inter-rater reliability for each sketching exercise and for each grading metric for a sample of sketches, with discussion of score use and interpretation.

Increased virtual meeting software usage has allowed people to meet remotely in a more seamless fashion. However, compared to in-person meetings, valuable interaction cues such as impromptu group polling are less optimally executed due to increased difficulty in gauging remote participants, while also requiring prior meeting setup for automated counting with built-in polling tools. We propose a novel intelligent user interface approach for virtual meeting software that supports impromptu polling interactions by leveraging real-time hand gesture recognition and video filter feedback. We conducted studies to design and evaluate this intuitive gesture-based polling system with visual feedback. Our results demonstrated that our system was able to recognize attendees’ gestures and poll responses with reasonable accuracy, and showed improvements in hosts’ task workload performance. From our findings, our interface informs hosts of valuable results while maintaining organic gestural interaction cues with attendees similar to in-person meetings.

The implementation of temporal dynamic variables improves the accuracy of spatial accessibility measurements. However, in previous studies, the temporal dynamics were partially incorporated, and a few snapshots were subjectively selected to demonstrate temporal changes of spatial accessibility, which may not represent the entire variation. In this study, we proposed a conceptual framework to leverage spatial accessibility temporal fluctuation, facilitating decision-making. Not only was the full implementation of time-dependent inputs, but the framework also employed the Gaussian two-step floating catchment area (G2SFCA) method and measured hourly spatial accessibility over 24 hours. Then, a temporal clustering with K-means and hierarchical clustering methods was performed, detecting a few distinctive temporal changes. Lastly, the significance of dynamic accessibility measurement and temporal clustering was validated using Pearson’s correlations. We took electric vehicle (EV) charging stations in Seoul, South Korea, as a case study. The results presented that neglecting temporal dynamics could fail to predict accessibility during the daytime. Additionally, temporal clustering summarized the 24-hour changes of accessibility into five temporal phases and showed the need for additional resources for insufficient accessibility locations in the afternoon. Consequently, our framework elicited the temporal changes of accessibility measures and identified a specific space and time for supplementary infrastructure.

Computer science ( CS ) majors are in high demand and account for a large part of national computer and information technology job market applicants. Employment in this sector is projected to grow 12% between 2018 and 2028, which is faster than the average of all other occupations. Published data are available on traditional non-computer science-specific hiring processes. However, the hiring process for CS majors may be different. It is critical to have up-to-date information on questions such as “what positions are in high demand for CS majors?,” “what is a typical hiring process?,” and “what do employers say they look for when hiring CS graduates?” This article discusses the analysis of a survey of 218 recruiters hiring CS graduates in the United States. We used Atlas.ti to analyze qualitative survey data and report the results on what positions are in the highest demand, the hiring process, and the resume review process. Our study revealed that a software developer was the most common job the recruiters were looking to fill. We found that the hiring process steps for CS graduates are generally aligned with traditional hiring steps, with an additional emphasis on technical and coding tests. Recruiters reported that their hiring choices were based on reviewing resume’s experience, GPA, and projects sections. The results provide insights into the hiring process, decision making, resume analysis, and some discrepancies between current undergraduate CS program outcomes and employers’ expectations.

The shift to virtual has changed our society and left an impact on nearly every part of our lives. Although it brought many challenges, global remote access also opened up a world of educational and professional opportunities for many people who did not have them before. In this work, we detail the findings of a study collecting feedback on inclusivity in virtual, in-person, and hybrid spaces, with the goal of building a greater understanding of the issues and personal challenges faced by access and equity stakeholders. From a survey of 104 individuals and detailed interviews with 12, we have used a mix of qualitative and quantitative methods to discover key challenges. The diversity of experiences and opinions was striking, with many winners and losers going into the virtual space. We propose some modifications to the online environment and in-person practices with the aim of furthering equality.

introduction Share on Special Issue on Highlights of IUI 2021: Introduction Authors: Tracy Hammond Texas A&M University, College Station, Texas, United States Texas A&M University, College Station, Texas, United StatesView Profile , Bart Knijnenburg Clemson University, Clemson, South Carolina, United States Clemson University, Clemson, South Carolina, United StatesView Profile , John O’Donovan University of California, Santa Barbara, California, United States University of California, Santa Barbara, California, United StatesView Profile , Paul Taele Texas A&M University, College Station, Texas, United States Texas A&M University, College Station, Texas, United StatesView Profile Authors Info & Claims ACM Transactions on Interactive Intelligent SystemsVolume 12Issue 403 February 2023Article No.: 25pp 1–4https://doi.org/10.1145/3561516Published:03 February 2023Publication History 0citation47DownloadsMetricsTotal Citations0Total Downloads47Last 12 Months47Last 6 weeks9 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Sketchtivity is a stylus-based intelligent tutoring system that can help instructors automatically provide feedback to their students, saving them the time and effort of providing personalized feedback themselves. The system uses a generic evaluation of perspective, direction, and accuracy to give students feedback on the quality of their sketches. If instructors want to personalize the metrics, the system would require them to provide multiple sets of samples. Therefore, instructors may use instructional team members such as teaching and graduate teaching assistants to provide feedback on the required samples. Compared to that of assistants, the feedback they produce might vary due to expertise and create noise in the training data. To address this problem, we implement a deep neural network that leverages learning to reweight algorithms. The data collected by the instructor from undergraduate and graduate-level rectilinear perspectives sketching is considered the validated sample. In this study, we analyzed the training size requirement for a Multi-Layer Perceptron (MLP) to accurately predict whether or not a stroke was a perspective stroke. We observed that the training data required to predict stroke accuracy is small. In addition, the performance of the algorithm in terms of accuracy was good even under extreme conditions such as having highly unbalanced data and having a small valid set of data. The results from the study support the use of these types of algorithms for future system personalizing to support scalable feedback systems in education.

With the onset of the COVID-19 pandemic, faculty are suddenly thrown into a world where they have to teach virtually, forcing them to innovate in their teaching practice without the ability to “chat with their colleagues” next door. Additionally, many faculty do not know how to create an inclusive classroom, which is crucial for students who have also lost their learning community and support structure. Many faculty are afraid to include issues of inclusion and diversity, feeling ill equipped and unsupported. As tensions rise across the US, there is a critical need for engineering students to be able to discuss issues involving inclusion in the classroom as well as apply their abilities as engineers to make a significant impact (both positively or negatively) on inclusion through their applications and innovations. Without the proper support for the faculty to innovate and foster inclusive classrooms, millions of students will suffer in their education. Additionally, without a trusting support group, most engineering faculty would not choose to assume the risk. In response, we developed a community of practice with six teaching fellows who shared, watched, commented on, and emulated each other's classroom recordings. The teaching fellows learned a significant amount from each other, courageously implemented activities focused on inclusion and awareness in their classrooms, observed greater awareness among their students on inclusion issues, and were able to mentor faculty and publish about the data driven best practices that they completed.

This research Work In Progress systematically reviews the current literature on sketching assessment in engineering education. Sketching is an integral part of the engineering curriculum for conceptual understanding, communication, and design. Sketching enables designers to offload, view, share, and test their ideas. In addition, sketching serves as a tool to increase students' spatial reasoning skills, which is critical to retention and success in engineering. Due to its impact, sketching has been studied in a variety of ways and settings and there are a wide array of methods for assessing sketching. Researchers often assess sketching skill through expert judgment, and when actual sketches are assessed, there are many different metrics that are used. This study is a systematic literature review of sketching assessment exploring applications, cognitive dimensions, and metrics. Databases namely Engineering Village, APA PsycInfo, and Education Source were searched for finding relevant literature related to sketching assessment. Data collection criteria included papers at the high school and college level in engineering, design, architecture, and art. In this paper, our search strings and summary of the final literature sample at the abstract level in terms of publication sources, year, and reviewer decisions are presented. Future directions include continuation of content analysis at the full paper level and assigning quality rankings. The end goal of the project is to provide the design and education communities with a succinct recommendation on sketching assessment to unify efforts in sketching research across the literature.

Nearly half of people prescribed medication to treat chronic or short-term conditions do not take their medicine as prescribed. This leads to worse treatment outcomes, higher hospital admission rates, increased healthcare costs, and increased morbidity and mortality rates. While some instances of medication non-adherence are a result of problems with the treatment plan or barriers caused by the health care provider, many are instances caused by patient-related factors such as forgetting, running out of medication, and not understanding the required dosages. This presents a clear need for patient-centered systems that can reliably increase medication adherence. To that end, in this work we describe an activity recognition system capable of recognizing when individuals take medication in an unconstrained, real-world environment. Our methodology uses a modified version of the Bagging ensemble method to suit unbalanced data and a classifier trained on the prediction probabilities of the Bagging classifier to identify when individuals took medication during a full-day study. Using this methodology we are able to recognize when individuals took medication with an F-measure of 0.77. Our system is a first step towards developing personal health interfaces that are capable of providing personalized medication adherence interventions.

Engineering students need practical, open-ended problems to help them build their problem-solving skills and design abilities. However, large class sizes create a grading challenge for instructors as there is simply not enough time nor support to provide adequate feedback on many design problems. In this work, we describe an intelligent user interface to provide automated real-time feedback on hand-drawn free body diagrams that is capable of analyzing the internal forces of a sketched truss to evaluate open-ended design problems. The system is driven by sketch recognition algorithms developed for recognizing trusses and a robust linear algebra approach for analyzing trusses. Students in an introductory statics course were assigned a truss design problem as a homework assignment using either paper or our software. We used conventional content analysis on four focus groups totaling 16 students to identify key aspects of their experiences with the design problem and our software. We found that the software correctly analyzed all student submissions, students enjoyed the problem compared to typical homework assignments, and students found the problem to be good practice. Additionally, students using our software reported less difficulty understanding the problem, and the majority of all students said they would prefer the software approach over pencil and paper. We also evaluated the recognition performance on a set of 3000 sketches resulting in an f-score of 0.997. We manually reviewed the submitted student work which showed the handful of student complaints about recognition were largely due to user error.

Kanji script writing is a skill that is often introduced to novice Japanese foreign language students for achieving Japanese writing mastery, but often poses difficulties to students with primarily English fluency due to their its vast differences with written English. Instructors often introduce various pedagogical methods—such as visual structure and written techniques—to assist students in kanji study, but may lack availability providing direct feedback on students' writing outside of class. Current educational applications are also limited due to lacking richer instructor-emulated feedback. We introduce Kanji Workbook, a writing-based intelligent tutoring system for students to receive intelligent assessment that emulates human instructor feedback. Our interface not only leverages students' computing devices for allowing them to learn, practice, and review the writing of prompted characters from their course's kanji script lessons, but also provides a diverse set of writing assessment metrics—derived from instructor interviews and classroom observation insights—through intelligent scoring and visual animations. We deployed our interface onto novice- and intermediate-level university courses over an entire academic year, and observed that interface users on average achieved higher course grades than their peers and also reacted positively to our interface's various features.

Sketching is a practical and useful skill that can benefit communication and problem solving. However, it remains a difficult skill to learn because of low confidence and motivation among students and limited availability for instruction and personalized feedback among teachers. There is an need to improve the educational experience for both groups, and we hypothesized that integrating technology could provide a variety of benefits. We designed and developed an intelligent tutoring system for sketching fundamentals called Sketchtivity, and deployed it in to six existing courses at the high school and university level during the 2017-2018 school year. 268 students used the tool and produced more than 116,000 sketches of basic primitives. We conducted semi-structured interviews with the six teachers who implemented the software, as well as nine students from a course where the tool was used extensively. Using grounded theory, we found ten categories which unveiled the benefits and limitations of integrating an intelligent tutoring system for sketching fundamentals in to existing pedagogy.

Providing opportunities for students to exercise their creative skills in large, entry engineering classes challenges most faculty. This paper presents a study of a large statics class provided with a homework problem that asks them to design a truss structure. Automatic grading was done by Mechanix, an AI tutor-based software package that can automatically recognize a free-body diagram or a planar, 2d, statically determinate truss structure. The paper presents a study done in two different semesters, comparing the students using Mechanix to a control (problem on paper). To ease grading, the control group's trusses were analyzed by Mechanix after submission. No mean homework grade differences were observed, but students in the Mechanix group produced trusses that could withstand higher loads. This is despite the fact the only guidance or feedback Mechanix provides was if the students' calculated max load was correct, and if it was not, which member failed. This study occurred in Fall 2019 and Spring 2020. Students also submitted more attempts in Mechanix than the control. It may be students in the control group only submitted correct answers despite being asked to submit all attempts. Future work will provide more incentive for students to submit all attempts on paper. Mechanix automatically records all attempts. During high stress (Covid-19), more students in the Mechanix group submitted the assignment indicating that students may find this system less mentally taxing to use, less stressful, or something else led to this difference. It will be explored with focus groups in the future. AI tools have the potential to provide automatic grading for open-ended, creativity required, design problems, and to engage students more, allowing universities to develop more innovative engineers while also deepening their knowledge.

Sketching in perspective is a valuable skill for art, and for professional disciplines like industrial design, architecture, and engineering. However, it tends to be a difficult skill to grasp for novices. We have developed an algorithm that can classify rectilinear perspective strokes, as well as classify which of those strokes are accurate on a stroke-by-stroke basis. We also developed an intelligent user interface which can provide real-time accuracy feedback on a user's free-hand digital perspective sketch. To evaluate the system, we conducted a between-subjects user study with 40 novice participants which involved sketching city street corners in 2-point perspective. We discovered that the participants who received real-time intelligent feedback improved their perspective accuracy significantly (p < 0.0005) in a subsequent unassisted sketch, suggesting a mild learning effect and knowledge transfer. Through qualitative feedback we also derived principles for how intelligent real-time feedback can best assist novices in learning perspective.

In this workshop, the focus will be on providing a forum for researchers in the various interdisciplinary fields to engage in and explore research outcomes and discussions on sketching interactions, specifically for interfaces that promote creativity and cognition. Attendees of the workshop will actively experience sessions related to the presentation, discussion, and sharing of discoveries and innovations that target or leverage sketching interactions for interfaces in domains that include the learning sciences, education, artificial, and engineering. The workshop will serve as a platform to encourage opportunities of new collaborations and inspire growth for leveraging sketching interactions towards creativity and cognition.

We present DrawMyPhoto, an interactive system that can assist a drawing novice in producing a quality drawing by automatically parsing a photograph in to a step-by-step drawing tutorial. The system utilizes image processing to produce distinct line work and shading steps from the photograph, and offers novel real-time feedback on pressure and tilt, along with grip suggestions as the user completes the tutorial. Our evaluation showed that the generated steps and real-time assistance allowed novices to produce significantly better drawings than with a more traditional grid-based approach, particularly with respect to accuracy, shading, and details. This was confirmed by domain experts who blindly rated the drawings. The participants responded well to the real-time feedback, and believed it helped them learn proper shading techniques and the order in which a drawing should be approached. We saw promising potential in the tool to boost the confidence of novices and lower the barrier to artistic creation.

Recent trends in computer-mediated communication (CMC) have not only led to expanded instant messaging through the use of images and videos but have also expanded traditional text messaging with richer content in the form of visual communication markers (VCMs) such as emoticons, emojis, and stickers. VCMs could prevent a potential loss of subtle emotional conversation in CMC, which is delivered by nonverbal cues that convey affective and emotional information. However, as the number of VCMs grows in the selection set, the problem of VCM entry needs to be addressed. Furthermore, conventional means of accessing VCMs continue to rely on input entry methods that are not directly and intimately tied to expressive nonverbal cues. In this work, we aim to address this issue by facilitating the use of an alternative form of VCM entry: hand gestures. To that end, we propose a user-defined hand gesture set that is highly representative of a number of VCMs and a two-stage hand gesture recognition system (trajectory-based, shape-based) that can identify these user-defined hand gestures with an accuracy of 82%. By developing such a system, we aim to allow people using low-bandwidth forms of CMCs to still enjoy their convenient and discreet properties while also allowing them to experience more of the intimacy and expressiveness of higher-bandwidth online communication.

Gaze gesture-based interactions on a computer are promising, but the existing systems are limited by the number of supported gestures, recognition accuracy, need to remember the stroke order, lack of extensibility, and so on. We present a gaze gesture-based interaction framework where a user can design gestures and associate them to appropriate commands like minimize, maximize, scroll, and so on. This allows the user to interact with a wide range of applications using a common set of gestures. Furthermore, our gesture recognition algorithm is independent of the screen size, resolution, and the user can draw the gesture anywhere on the target application. Results from a user study involving seven participants showed that the system recognizes a set of nine gestures with an accuracy of 93% and a F-measure of 0.96. We envision, this framework can be leveraged in developing solutions for situational impairments, accessibility, and also for implementing rich a interaction paradigm.

Interdisciplinary research always yields an enhanced and different perspective on the problem. Solving a problem using many modalities and technologies gives us the complete picture of the problem and the best solution possible. My thesis is about solving the various problem in different domains using the Eye and Gaze Tracking. My research area currently limits to the application of Eye tracking to three domains i.e. Identification and Authentication, Page Ranking based on the Gaze Interaction, and Data Analytics based on the Gaze Interaction data. In the Authentication and Identification, I have developed different authentication systems which use Gaze and Eye tracking to authenticate a user with 97% accuracy. For Page ranking of the pages, I have used the eye-tracking interaction to enhance the ranking of the page/link on Google and Bing backed Search Emulator. This project is in progress and will be mostly done in coming two months. The third and the last project is related to using the Eye tracking and Gaze Interaction data for inferences which range from detecting the unique eye moment pattern of the user, the saccades and micro-saccades in both horizontal as well as vertical directions, moreover detecting the eye diseases using the Gaze Interaction data.

Design sketching is an important skill for designers, engineers, and creative professionals, as it allows them to express their ideas and concepts in a visual medium. Being a critical and versatile skill for many different disciplines, courses on design sketching are often taught in universities. Courses today predominately rely on pen and paper; however, this traditional pedagogy is limited by the availability of human instructors, who can provide personalized feedback. Using a stylus-based intelligent tutoring system called SketchTivity , we aim to eventually mimic the feedback given by an instructor and assess student-drawn sketches to give students insight into areas for improvement. To provide effective feedback to users, it is important to identify what aspects of their sketches they should work on to improve their sketching ability. After consulting with several domain experts in sketching, we came up with several classes of features that could potentially differentiate expert and novice sketches. Because improvement on one metric, such as speed, may result in a decrease in another metric, such as accuracy, the creation of a single score may not mean much to the user. We attempted to create a single internal score that represents overall drawing skill so that the system can track improvement over time and found that this score correlates highly with expert rankings. We gathered over 2,000 sketches from 20 novices and four experts for analysis. We identified key metrics for quality assessment that were shown to significantly correlate with the quality of expert sketches and provide insight into providing intelligent user feedback in the future.

Searching is a necessary tool for managing and navigating the massive amounts of data available in today's information age. While new searching methods have become increasingly popular and reliable in recent years, such as image-based searching, these methods may be more limited than text-based means in that they do not allow generic user input. Sketch-based searching is a method that allows users to draw generic search queries and return similar drawn images, giving more user control over their search content. In this paper, we present SketchSeeker, a system for indexing and searching across a large number of sketches quickly based on their similarity. SketchSeeker introduces a technique for indexing sketches in extremely compressed representations, which allows for fast, accurate retrieval augmented with a multilevel ranking subsystem. SketchSeeker was tested on a large set of sketches against existing sketch similarity metrics, and it shows significant improvements in terms of storage requirements, speed, and accuracy.

Intelligent tutoring systems (ITS) empower instructors to make teaching more engaging by providing a platform to tutor, deliver learning material, and to assess students' progress. Despite the advantages, existing ITS do not automatically assess how students engage in problem solving? How do they perceive various activities? and How much time they spend on each activity leading to the solution? In this research, we present an eye tracking framework that, based on eye movement data, can assess students' perceived activities and overall engagement in a sketch based Intelligent tutoring system, "Mechanix" [Valentine et al. 2012]. Based on an evaluation involving 21 participants, we present the key eye movement features, and demonstrate the potential of leveraging eye movement data to recognize students' perceived activities, "reading, gazing at an image, and problem solving," with an accuracy of 97.12%.

Recent developments in eye tracking technology are paving the way for gaze-driven interaction as the primary interaction modality. Despite successful efforts, existing solutions to the "Midas Touch" problem have two inherent issues: 1) lower accuracy, and 2) visual fatigue that are yet to be addressed. In this work we present GAWSCHI: a Gaze-Augmented, Wearable-Supplemented Computer-Human Interaction framework that enables accurate and quick gaze-driven interactions, while being completely immersive and hands-free. GAWSCHI uses an eye tracker and a wearable device (quasi-mouse) that is operated with the user's foot, specifically the big toe. The system was evaluated with a comparative user study involving 30 participants, with each participant performing eleven predefined interaction tasks (on MS Windows 10) using both mouse and gaze-driven interactions. We found that gaze-driven interaction using GAWSCHI is as good (time and precision) as mouse-based interaction as long as the dimensions of the interface element are above a threshold (0.60" x 0.51"). In addition, an analysis of NASA Task Load Index post-study survey showed that the participants experienced low mental, physical, and temporal demand; also achieved a high performance. We foresee GAWSCHI as the primary interaction modality for the physically challenged and a means of enriched interaction modality for the able-bodied demographics.

Learning music theory not only has practical benefits for musicians to write, perform, understand, and express music better, but also for both non-musicians to improve critical thinking, math analytical skills, and music appreciation. However, current external tools applicable for learning music theory through writing when human instruction is unavailable are either limited in feedback, lacking a written modality, or assuming already strong familiarity of music theory concepts. In this paper, we describe Maestoso, an educational tool for novice learners to learn music theory through sketching practice of quizzed music structures. Maestoso first automatically recognizes students’ sketched input of quizzed concepts, then relies on existing sketch and gesture recognition techniques to automatically recognize the input, and finally generates instructor-emulated feedback. From our evaluations, we demonstrate that Maestoso performs reasonably well on recognizing music structure elements and that novice students can comfortably grasp introductory music theory in a single session.

A national study by the Australian Transport Safety Bureau revealed that motorcyclist deaths were nearly thirty times more prevalent than that of drivers of other vehicles. These fatalities represent approximately 5% of all highway deaths each year, yet motorcycles account for only 2% of all registered vehicles in the USA. Motorcyclists are highly exposed on the road, so maintaining situational awareness at all times is crucial. Route guidance systems enable users to efficiently navigate between locations using dynamic visual maps and audio directions, and have been well tested with motorists, but remain unsafe for use by motorcyclists. Audio/visual routing systems decrease motorcyclists' situational awareness and vehicle control, and thus elevate chances of an accident. To enable motorcyclists to take advantage of route guidance while maintaining situational awareness, we created HaptiMoto, a wearable haptic route guidance system. HaptiMoto uses tactile signals to encode the distance and direction of approaching turns, thus avoiding interference with audio/visual awareness. Our evaluations demonstrate that HaptiMoto is both intuitive and a safer alternative for motorcyclists compared to existing solutions.

Soldiers, to guard themselves from enemy assault, have to maintain visual and auditory awareness of their environment. Their visual and auditory senses are thus saturated. This makes these channels less usable for communication. The tactile medium of communication with users is appropriate for displaying information in such situations. Research in interpersonal communication among soldiers shows that the most common form of communication between soldiers involves the use of verb phrases. In this article, we have developed a three-by-three tactile display and proposed a method for mapping the components of a verb phrase to two dimensions of tactile codes—shape and waveform. Perception of tactile codes by users depends on the ability of users to distinguish shape and waveform of the code. We have proposed a measure to rate the distinguish-ability of any two shapes and created a graph-based user-centric model using this measure to select distinguishable shapes from a set of all presentable shapes. We conducted two user studies to evaluate the ability of users to perceive tactile information. The results from our first study showed users' ability to perceive tactile shapes, tactile waveforms, and form verb phrases from tactile codes. The recognition accuracy and time taken to distinguish were better when the shapes were selected from the graph model than when shapes were chosen based on intuition. The second user study was conducted to test the performance of users while performing a primary visual task simultaneously with a secondary audio or haptic task. Users were more familiar with perceiving information from an auditory medium than from a haptic medium, which was reflected in their performance. Thus the performance of users in the primary visual task was better while using an audio medium of communication than while using a haptic medium of communication.

Research in multi-touch interaction has typically been focused on direct spatial manipulation; techniques have been create to result in the most intuitive mapping between the movement of the hand and the resultant change in the virtual object. However, as we attempt to design for more complex operations, the expectation of spatial manipulation becomes infeasible.

Introductory engineering courses within large universities often have annual enrollments which can reach up to a thousand students. It is very challenging to achieve differentiated instruction in classrooms with class sizes and student diversity of such great magnitude. Professors can only assess whether students have mastered a concept by using multiple choice questions, while detailed homework assignments, such as planar truss diagrams, are rarely assigned because professors and teaching assistants would be too overburdened with grading to return assignments with valuable feedback in a timely manner. In this paper, we introduce Mechanix, a sketch-based deployed tutoring system for engineering students enrolled in statics courses. Our system not only allows students to enter planar truss and free body diagrams into the system just as they would with pencil and paper, but our system checks the student’s work against a hand-drawn answer entered by the instructor, and then returns immediate and detailed feedback to the student. Students are allowed to correct any errors in their work and resubmit until the entire content is correct and thus all of the objectives are learned. Since Mechanix facilitates the grading and feedback processes, instructors are now able to assign free response questions, increasing teacher’s knowledge of student comprehension. Furthermore, the iterative correction process allows students to learn during a test, rather than simply displaying memorized information.

This paper presents an acoustic sound recognizer to recognize what people are writing on a table or wall by utilizing the sound signal information generated from a key, pen, or fingernail moving along a textured surface. Sketching provides a natural modality to interact with text, and sound is an effective modality for distinguishing text. However, limited research has been conducted in this area. Our system uses a dynamic time- warping approach to recognize 26 hand-sketched characters (A-Z) solely through their acoustic signal. Our initial prototype system is user-dependent and relies on fixed stroke ordering. Our algorithm relied mainly on two features: mean amplitude and MFCCs (Mel-frequency cepstral coefficients). Our results showed over 80% recognition accuracy.

Mechanix is a sketch recognition tool that provides an efficient means for engineering students to learn how to draw truss free-body diagrams (FBDs) and solve truss problems. The system allows students to sketch these FBDs into a tablet computer or by using a mouse just as they would by hand, a mouse can also be used to draw the sketch using a regular computer and monitor. Mechanix is then able to provide immediate feedback to the students and tell them if they are missing any components of the FBD, without providing answers. The program is also able to tell them whether their solved reaction or member forces are correct or not. This paper presents a study to evaluate the effectiveness and advantages of using Mechanix in the classroom, as a supplement to traditional teaching and learning methods. Current results demonstrate that students believe Mechanix enhances their learning and are highly engaged when using it. Future work on the refinement of the program is also discussed.

In an introductory Engineering course with an annual enrollment of over 1000 students, a professor has little option but to rely on multiple choice exams for midterms and finals. Furthermore, the teaching assistants are too overloaded to give detailed feedback on submitted homework assignments. We introduce Mechanix, a computer-assisted tutoring system for engineering students. Mechanix uses recognition of freehand sketches to provide instant, detailed, and formative feedback as the student progresses through each homework assignment, quiz, or exam. Free sketch recognition techniques allow students to solve free-body diagram and static truss problems as if they were using a pen and paper. The same recognition algorithms enable professors to add new unique problems simply by sketching out the correct answer. Mechanix is able to ease the burden of grading so that instructors can assign more free response questions, which provide a better measure of student progress than multiple choice questions do.

The 2011 IUI workshop on Sketch Recognition was a daylong event held on February 13th, 2011 in Palo Alto, California. The workshop consisted of an invited talk by Dr. Sharon Oviatt, several short and long talks, a student research poster session, and a series of discussions about pertinent topics in and about the field of sketch recognition.

Sketch recognition attempts to interpret the hand-sketched markings made by users on an electronic medium. Through recognition, sketches and diagrams can be interpreted and sent to simulators or other meaningful analyzers. Primitives are the basic building block shapes used by high-level visual grammars to describe the symbols of a given sketch domain. However, one limitation of these primitive recognizers is that they often only support basic shapes drawn with a single stroke. Furthermore, recognizers that do support multistroke primitives place additional constraints on users, such as temporal timeouts or modal button presses to signal shape completion. The goal of this research is twofold. First, we wanted to determine the drawing habits of most users. Our studies found multistroke primitives to be more prevalent than multiple primitives drawn in a single stroke. Additionally, our studies confirmed that threading is less frequent when there are more sides to a figure. Next, we developed an algorithm that is capable of recognizing multistroke primitives without requiring special drawing constraints. The algorithm uses a graph-building and search technique that takes advantage of Tarjan's linear search algorithm, along with principles to determine the goodness of a fit. Our novel, constraint-free recognizer achieves accuracy rates of 96% on freely-drawn primitives.

Sketch recognition is automated understanding of hand-drawn diagrams. Current sketch recognition systems exist for only a handful of domains, which contain on the order of 10--20 shapes. Our goal was to create a generalized method for recognition that could work for many domains, increasing the number of shapes that could be recognized in real-time, while maintaining a high accuracy. In an effort to effectively recognize shapes while allowing drawing freedom (both drawing-style freedom and perceptually-valid variations), we created the shape description language modeled after the way people naturally describe shapes to 1) create an intuitive and easy to understand description, providing transparency to the underlying recognition process, and 2) to improve recognition by providing recognition flexibility (drawing freedom) that is aligned with how humans perceive shapes. This paper describes the results of a study performed to see how users naturally describe shapes. A sample of 35 subjects described or drew approximately 16 shapes each. Results show a common vocabulary related to Gestalt grouping and singularities. Results also show that perception, similarity, and context play an important role in how people describe shapes. This study resulted in a language (LADDER) that allows shape recognizers for any domain to be automatically generated from a single hand-drawn example of each shape. Sketch systems for over 30 different domains have been automatically generated based on this language. The largest domain contained 923 distinct shapes, and achieved a recognition accuracy of 83% (and a top-3 accuracy of 87%) on a corpus of over 11,000 sketches, which recognizes almost two orders of magnitude more shapes than any other existing system.

When asked to draw, many people are hesitant because they consider themselves unable to draw well. This paper describes the first system for a computer to provide direction and feedback for assisting a user to draw a human face as accurately as possible from an image. Face recognition is first used to model the features of a human face in an image, which the user wishes to replicate. Novel sketch recognition algorithms were developed to use the information provided by the face recognition to evaluate the hand-drawn face. Two design iterations and user studies led to nine design principles for providing such instruction, presenting reference media, giving corrective feedback, and receiving actions from the user. The result is a proof-of-concept application that can guide a person through step-by-step instruction and generated feedback toward producing his/her own sketch of a human face in a reference image.

Military course-of-action (COA) diagrams are used to depict battle scenarios and include thousands of unique symbols, complete with additional textual and designator modifiers. We have created a real-time sketch recognition interface that recognizes 485 freely-drawn military course-of-action sym- bols. When the variations (not allowable by other systems) are factored in, our system is several orders of magnitude larger than the next biggest system. On 5,900 hand-drawn symbols, the system achieves an accuracy of 90% when con- sidering the top 3 interpretations and requiring every aspect of the shape (variations, text, symbol, location, orientation) to be correct.

Freehand drawing on a computer screen allows users to provide input through a natural mode of human interaction. With this freedom of expression, however, there exists a paradoxical limitation: the user is bound through the existing interface to the fixed drawing surface. In this work, we overcome this limitation by presenting a surfaceless pen-based interface with an application in the field of sketch recognition. A pilot study was conducted to examine the usability of the surfaceless pen-based interface. Results indicated that learning to use the device is relatively straightforward, but that interaction difficulty increases in a directly proportionally manner with drawing complexity.

No abstract available.

This paper describes the IUI'09 workshop on Sketch Recognition.

Computer-based games and technologies can be significant aids for helping children learn. However, most computer-based games simply address the learning styles of visual and auditory learners. Sketch-based interfaces, however, can also address the needs of those children who learn better through tactile and kinesthetic approaches. Furthermore, sketch recognition can allow for automatic feedback to aid children without the explicit need for teacher to be present. In this paper, we present various sketch-based tools and games that promote tactile learning and entertainment for children.

Sketching is a natural form of human communication and has become an increasingly popular tool for interacting with user interfaces. In order to facilitate the integration of sketching into traditional user interfaces, we must first develop accurate ways of recognizing users' intentions while providing feedback to catch recognition problems early in the sketching process. One approach to sketch recognition has been to recognize low-level primitives and then hierarchically construct higher-level shapes based on geometric constraints defined by the user, however, current low-level recognizers only handle a few number of primitive shapes. We propose a new low-level recognition and beautification system that can recognize eight primitive shapes, as well as combinations of these primitives, with recognition rates at 98.56%. Our system also automatically generates beautified versions of these shapes to provide feedback early in the sketching process. In addition to looking at geometric perception, much of our recognition success can be attributed to two new features, along with a new ranking algorithm, which have proven to be significant in distinguishing polylines from curved segments.

Diagrams are widely used as abstract models of systems. The act of creating the initial diagram is a common problem solving strategy. Most diagrams are conceived and created with an old-fashioned pen on a piece of blank paper or a whiteboard. Current widget based tools, in contrast, are unsuitable for early design support because of the constraints they place on the user. Sketch diagramming tools provide the unrestrained interface of paper and the document preparation support of computers. However there are many outstanding challenges before we can provide really good sketch tools for tasks such as diagramming.

Sketch interfaces provide more natural interaction than the traditional mouse and palette tool, but can be time consuming to build if they have to be built anew for each new domain. A shape description language, such as the LADDER language we created, can significantly reduce the time necessary to create a sketch interface by enabling automatic generation of the interface from a domain description. However, structural shape descriptions, whether written by users or created automatically by the computer, are frequently over- or under- constrained. We present a technique to debug over- and under-constrained shapes using a novel form of active learning that generates its own suspected near-miss examples. Using this technique we implemented a graphical debugging tool for use by sketch interface developers.