Generative AI is increasingly shaping early-stage software design by enabling the rapid creation of synchronized artifacts such as personas, user journeys, wireframes, and evaluation prompts within integrated workspaces. Prevailing accounts of Human-in-the-Loop (HITL), however, remain largely control-centric, framing the human primarily as a supervisor of automated output. This paper reports an exploratory qualitative study of an integrated AI-supported design environment across two settings: non-computing domain experts collaborating directly with the system, and professional teams using the same environment to mediate participatory sessions with institutional stakeholders. Drawing on triangulated data from structured surveys, semi-structured interviews, and screen recordings, we examine how AI-mediated interaction shapes reflection, articulation, and negotiation during early-stage design. The analysis indicates that the AI acted as a reflective interlocutor, supporting the externalization of intent, the identification of ambiguities, and the synchronization of artifacts. On this basis, Reflection-in-the-Loop (RITL) is proposed as an epistemic extension of HITL, highlighting how such reflective dynamics promote semantic alignment, rationale articulation, and the production of evaluation-ready artifacts. We discuss implications for AI-augmented design, emphasizing DesignOps support for meaning making and rationale traceability, and propose metrics such as reflective turn rate, semantic alignment, and iteration latency for future study.

ABSTRACT Automatic Speech Recognition (ASR) systems have been widely used as a practical method of interaction between humans and devices. They are typically employed to enhance the accessibility of devices and to improve the security of systems, among other purposes. However, the design of speech‐based systems imposes many challenges due to their particularities. Currently, the majority of ASR systems is based on the Hidden Markov Model (HMM), and, more recently, on Convolutional Neural Networks (CNN). The present research evaluates the performance of Hidden Markov Model (HMM) and Convolutional Neural Network (CNN) algorithms in speech recognition and proposes a novel hybrid approach that combines both methods. The study assesses various performance metrics, including accuracy, precision, recall, F1‐score, response time, and computational cost. The experimental tests show that the integration between HMM and CNN increased the accuracy by 6% and 8% when compared to HMM and CNN isolated, respectively, in accordance with results presented in previous papers. However, the results of the ANOVA test revealed that the difference in question is not statistically significant, and the HMM‐only approach still being an interesting option for embedded systems due to its lesser demanded computational effort.

Context. Given the need for digital transformation in education during the pandemic, this study focused on the Dell Accessible Learning (DAL) platform and its adaptation to the metaverse environment to improve learning for People with Disabilities (PwD). Goal. To analyze the viability of the metaverse environment in providing collaborative and accessible interaction for DAL students. Method: We conducted a prior literature review to align requirements with the State of the Art, considering collaborative learning and accessibility. Interviews and usability analysis were conducted with 9 PwD, DELL employees, corroborating the information in the literature. After usability evaluations and interviews, it was found that the third-party metaverse environment did not fully meet the accessibility criteria and usability. In response, DALverse was developed, a proof of concept integrated with DAL. Result. The results indicate benefits, including increased resources, multimodal interactions, and student retention. Future research should explore the use of DALverse in parallel with DAL courses, aiming to measure changes in indicators such as interaction, engagement, and performance. Furthermore, we provide an architectural model used in DAL and DALverse and implications for education, researchers, industry professionals, students and tutors.

Adaptive teaching, mediated by technology, has become a reality and a great ally in distance teaching and learning. Artificial intelligence, through machine learning, has been an important tool for the development of innovative solutions for education. One of the approaches for adaptive teaching has been the student emotion analysis in distance learning to propose didactic solutions that best fit the profile of each student. In this sense, this work aims to present the preliminary studies carried out with SVM models and the AffectNet database in order to propose a solution for the analysis of emotions in the context of distance learning. As a result of this study, it is expected to propose an additional resource for the Homero platform (A Learning Management System) capable of adapting content based on information about the students' emotional states. In addition, this information can also be used to predict potential evasion risks. The development and preliminary tests point to a result with an average accuracy of 73% for emotion recognition based on video. With the tests, it was also possible to verify some necessary adjustments for the development of an efficient model for the recognition and analysis of emotions.

This work presents the research to identify the most feasible way to deliver an augmented reality-based learning object to promote skill development among the deaf to prepare them to job positions in a computer manufacturing assembly line. A mobile app with augmented reality was developed to facilitate such training. To identify the best form of interaction the application had two forms of natural interaction and installed on two kinds of wearable devices. It allowed for the definition of four interaction set configurations. For identify the best arrangement, an experiment was carried out with 80 people (40 deaf and 40 hearing people) that allowed the analysis of quantitative and qualitative factors. After that, we evolved the tool to cope with real objects in the assembly line and assure. sustainable use.

This Innovate Practice Full Paper document presents a technological ecosystem for classroom management that we call GSA. GSA is meant to be used in any class/ of any subject/k-college and supports electronically mediated in class, engaging activities learning strategies to reduce dropout rates and create more learning opportunities. In the previous study, we interviewed 123 8th and 9th-grade students and 15 teachers who had been using the GSA for two weeks. The results were promising. An increase in the level of motivation was observed. Being able to mediate in real time, teacher/student interactions create opportunities to capture valuable data that can be used to feed performance/dropout predictors. In this paper, we discuss the introduction of these predictors and how they can be used to foster preemptive measures to mitigate the problem.

This Research Full Paper presents a literature review about overcoming accessibility barriers for People with Severe vision impairment in Web-based Learning Environments.People with vision impairment face several barriers while interacting with web-based environments.Among them, one can cite the lack of suitability of these environments for the use of screen readers, e.g. if the description of the environment elements are not readable by the screen reader.These barriers are also present in web-based learning environments. In this work we present, through a literature review, the most common accessibility barriers faced by people with vision impairment while interacting with web-based learning environments. In addition, we propose solutions that can mitigate this problem, allowing broad access to distance education environments and a better learning experience with them.

This innovative practice full paper presents a Metaphorical Debugger Model (JAD) to support deaf and hearing impaired in the process of learning how to program in Java. In Brazil, 9.7 million people are deaf or hearing impaired (DHI). DHI people usually face several barriers to get proper education. During a series of Java classes conducted by our team, we verified that these difficulties were also related to the use of software that is not tailored for their impairment. We also saw that DHI students encountered many problems while developing and evolving their codes using the traditional tools, for instance, visual signs to perform tasks, can be difficult to understand, in addition, the high degree of abstraction required for the programming logic, can be an accessibility barrier for them. Thus, we decided to propose JAD aiming to provide accessible debugging for DHI. The JAD uses the concept of metaphorical interfaces, adopting appropriate symbols and signs borrowed from traffic to aid in the process of code debugging and evolution. In this paper, we present some user studies with JAD. The results suggest that Java programmers, DHI e non-DHI, had similar performance in task related to software evolution when JAD was used.

To use a computer, users with spinal cord injury must employ special tools that, in most cases, demand a great effort. In online courses, such tools often become a distraction, which might hinder the learning process. Solutions such as tongue mouses, smart glasses and computer vision systems, although promising, still face considerable usability problems. On the other hand, ANA, our conversational agent that can hear and listen to the student, has proven quite promising [1]. Tests have shown that, while using our assistant, tetraplegia were able to focus more on their studies than on the system's interface. In this paper, we present our latest advances on ANA's architecture, introducing a computational grammar that allows for the insertion in online courses of “learning objects” (LOs) to which students can talk in a natural way. Those LOs respond by performing the requested action. We present the grammar used for processing commands in natural language and show how it is integrated with Google's conversational AI assistant.

IT job market seems to be an interesting alternative for those who are deaf or hearing impaired (DHI). All accessible (interface+content+learning objects) online courses have the reach and the means to, potentially, impact the status quo. All lab offers such courses to both DHI and non-DHI communities. Still, DHI graduates legs behind non-DHI in debugging-related tasks when the industry-standard Eclipse (Development Environment) is used [1]. Visual debuggers improve DHI performance [2]. In this paper, we present Java Accessible Visual Debugger (JAD). JAD was conceived to be the first accessible visual debugger and to leverage debugging performance between DHI and non-DHI. This paper presents a study involving DHI/non-DHI and JAD. Three metrics were used to assess performance: 1) number of fully completed tasks (TFC); 2) time to task completion (TCT); 3) number of external help requests (HR). Non-paired t-Student's tests show no significant difference in any of the above criteria.

Teaching a programming language to a person who is deaf or hearing impaired (DHI) has, along with other challenges, the lack of signs to convey basic concepts in computing. This shortage of signs makes learning even harder for the DHI.

Growing demand for computer programmers presents a good opportunity for better paying jobs for those who are deaf. Our online java programming course designed for the deaf has staggering dropout rates. We introduce the JLoad, an e-learning object designed to smooth the learning curve of a remote deaf Java learner. JLoad has a simple IDE and collaboration tools to enable asynchronous and remote situated student monitoring and assistance. Such collaboration involves student, tutors and translators. We presented JLoad's prototypes to a community of deaf programmers and tutor for early validation. We discuss how they received JLoad and improved its prototypes.

Human discourse is an embodied activity emerging from the embodied imagery and construction of our talk. Gesture and speech are coexpressive, conveying this imagery and meaning simultaneously. Mathematics instruction and discourse typically involve two modes of communication: speech and graphical presentation. Our goal is to assist Individuals who are Blind or Severely Visually Impaired (IBSVI) to access such instruction/communication. We employ a haptic glove interface to furnish the IBSVI with awareness of the deictic gestures performed by the instructor over the graphic in conjunction with speech. We present a series of studies spanning two years where we show how our Haptic Deictic System (HDS) can support learning in inclusive classrooms where IBSVI receive instruction alongside sighted students. We discuss how the introduction of the HDS was advantageous to all parties: IBSVI, instructor, and sighted students. The HDS created more learning opportunities, increasing mutual understanding and promoting greater engagement.

Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of mathematics instructors. This impedes the ability of blind students and their teachers to maintain situated communication. In this paper, we set forth the relevant phenomenological analyses to support this claim. We show that mathematical communication and instruction are inherent embodied; that the blind are able to conceptualize visuo-spatial information; and argue that uptake of embodied behavior is critical to receiving relevant mathematical information. Based on this analysis, we advance an approach to provide students who are blind with awareness of their teachers' deictic gestural activity via a set of haptic output devices. We lay forth a set of open research question that researcher in multimodal interfaces may address.