In this paper, we propose MixA-Q, a mixed-precision activation quantization framework that leverages intra-layer activation sparsity (a concept widely explored in activation pruning methods) for efficient inference of quantized window-based vision transformers. For a given uniform-bit quantization configuration, MixA-Q separates the batched window computations within Swin blocks and assigns a lower bit width to the activations of less important windows, improving the trade-off between model performance and efficiency. We introduce a Two-Branch Swin Block that processes activations separately in high- and low-bit precision, enabling seamless integration of our method with most quantization-aware training (QAT) and post-training quantization (PTQ) methods, or with simple modifications. Our experimental evaluations over the COCO dataset demonstrate that MixA-Q achieves a training-free 1.35x computational speedup without accuracy loss in PTQ configuration. With QAT, MixA-Q achieves a lossless 1.25x speedup and a 1.53x speedup with only a 1% mAP drop by incorporating activation pruning. Notably, by reducing the quantization error in important regions, our sparsity-aware quantization adaptation improves the mAP of the quantized W4A4 model (with both weights and activations in 4-bit precision) by 0.7%, reducing quantization degradation by 24%.

As the demand for disassembling end-of-life products grows, limitations in traditional disassembly line design, low efficiency, and high resource consumption become increasingly evident. Particularly in large-scale disassembly tasks, where the cost of conventional remanufacturing rises and the technologies fail to meet high-efficiency requirements. The integration of robots into disassembly lines is a promising solution to alleviate these issues. This work presents a multi-product hybrid disassembly line balancing problem that considers machine wear rates and establishes a mixed-integer programming model guided by profit maximization to address it. An improved fireworks algorithm is used in the proposed approach. The developed solution is compared with genetic and ant colony algorithms. Evaluation results and analysis demonstrated the competitive efficiency and stability of our approach.

Robots have been substantially employed across various avenues over the years. Yet, their application has been largely limited to controlled environments where variables are few and predictable. To address this challenge, we propose Robo-CSK-Organizer, a novel system that enhances robotic perception by integrating commonsense knowledge (CSK) for improved object organization, classification, and decision-making. By combining ConceptNet for semantic reasoning, DETIC for object identification, and BLIP for contextual analysis, Robo-CSK-Organizer achieves superior ambiguity resolution, task adaptation, and explainability compared to models without CSK. Testing in real-world robotics settings demonstrates notable gains in transparency, user trust, and error handling, making the approach valuable for advancing AI transparency and the development of versatile robotic applications in automation and engineering. Future directions of this work are comprehensively discussed.

Efficient disassembly is essential for the reintegration of end-of-life products into the remanufacturing process. Previous studies utilize human–robot collaboration and parallel workstations to enhance disassembly efficiency. However, the disassembly lines in these studies are typically independent of each other. As the number of disassembly lines in a plant increases, labor resources such as workers and robots become redundant, leading to low resource utilization and decreased disassembly revenue. This study proposes a novel disassembly scheme aimed at achieving high efficiency by leveraging parallelization and human–robot collaboration to share labor resources on a hybrid disassembly line. Specifically, this work develops a mixed-integer programming model to maximize disassembly profit. A discrete aquila optimizer algorithm, incorporating uniform variation and two-point crossover methods, provides the solution for the problem. Furthermore, the correctness of the proposed model and algorithm is verified within the solvable range of the commercial solver CPLEX. Finally, a comparative analysis of the proposed algorithm with the salp swarm algorithm, the fireworks algorithm, and the whale optimization algorithm demonstrates its superiority in solving the problem.

Understanding the link between visual attention and users’ information needs when visually exploring information visualisations is under-explored due to a lack of large and diverse datasets to facilitate these analyses. To fill this gap we introduce SalChartQA – a novel crowd-sourced dataset that uses the BubbleView interface to track user attention and a question-answering (QA) paradigm to induce different information needs in users. SalChartQA contains 74,340 answers to 6,000 questions on 3,000 visualisations. Informed by our analyses demonstrating the close correlation between information needs and visual saliency, we propose the first computational method to predict question-driven saliency on visualisations. Our method outperforms state-of-the-art saliency models for several metrics, such as the correlation coefficient and the Kullback-Leibler divergence. These results show the importance of information needs for shaping attentive behaviour and pave the way for new applications, such as task-driven optimisation of visualisations or explainable AI in chart question-answering.

This research-to-practice full paper describes a study on the performance of feature-matching algorithms in constrained computational environments, focusing on autonomous driving using low-end hardware like the Raspberry Pi 4B. We evaluate algorithms such as ORB, AKAZE, BRISK, and SIFT, examining their efficiency, accuracy, and robustness under various conditions. While ORB offers speed, AKAZE and BRISK demonstrate more consistent performance. To mitigate the gap between theoretical analysis and practical application, we integrate these findings into a robotics course through a project-based learning (PBL) approach. The comparison analysis provides the instructor with the necessary insights to guide students, as the research setting closely mirrors the course project. This hands-on project not only deepens students' understanding of computer vision but also hones critical problem-solving skills essential for modern engineering challenges. Future work will extend this study to other single-board computers and explore advanced computational techniques like parallel computing and GPU acceleration.

Multimedia data plays an important role in medicine and healthcare since EHR (Electronic Health Records) entail complex images and videos for analyzing patient data. In this article, we hypothesize that transfer learning with computer vision can be adequately harnessed on such data, more specifically chest X-rays, to learn from a few images for assisting accurate, efficient recognition of COVID. While researchers have analyzed medical data (including COVID data) using computer vision models, the main contributions of our study entail the following. Firstly, we conduct transfer learning using a few images from publicly available big data on chest X-rays, suitably adapting computer vision models with data augmentation . Secondly, we aim to find the best fit models to solve this problem, adjusting the number of samples for training and validation to obtain the minimum number of samples with maximum accuracy. Thirdly, our results indicate that combining chest radiography with transfer learning has the potential to improve the accuracy and timeliness of radiological interpretations of COVID in a cost-effective manner . Finally, we outline applications of this work during COVID and its recovery phases with future issues for research and development. This research exemplifies the use of multimedia technology and machine learning in healthcare.

As robotics and artificial intelligence (AI) technologies have become increasingly relevant over the past couple of years, they will inevitably be key components for industries of all aspects which continue to expand to technological solutions. Particularly, the agricultural industry has progressed to using such means to minimize human involvement and reduce tasks that are time-consuming and costly. Motivated by this, we developed a robot-assisted crop maturity recognition and harvest system to accurately classify and detect the stages of ripeness the crops are in—ripe, medium ripe, and not ripe. Our proposed approach integrates computer vision, image processing, collaborative robotics, and a subcategory of artificial intelligence—transfer learning. The transfer learning-based model is trained to classify and recognize the crop in its maturity stages and locate the crop during real-time detection. Experimental results and analysis in real-world robot-assisted smart agriculture environments successfully demonstrated crop ripeness recognition accuracy, proving transfer learning could be utilized to effectively improve the efficiency and productivity of harvesting processes in the agricultural industry. The future work of this study is also discussed.

As technology has advanced, society has witnessed and participated in the creation of robots that can walk, talk, and recognize speech. To facilitate communication and collaboration between humans and humanoid robots, we develop a teaching-learning framework for human beings to teach humanoid robots to complete object identification and operation tasks. The robots learn from their human partners based on the transfer learning approach and can assist humans using their learned knowledge. Experimental results and evaluations suggest the success and efficiency of the developed approach in smart service contexts for human-robot partnerships. The future work of this study is also discussed.

BACKGROUND: Fluorescence image analysis in biochemical science often involves the complex tasks of identifying samples for analysis and calculating the desired information from the intensity traces. Analyzing giant unilamellar vesicles (GUVs) is one of these tasks. Researchers need to identify many vesicles to statistically analyze the degree of molecular interaction or state of molecular organization on the membranes. This analysis is complicated, requiring a careful manual examination by researchers, so automating the analysis can significantly aid in improving its efficiency and reliability. RESULTS: We developed a convolutional neural network (CNN) assisted intelligent analysis routine based on the whole 3D z-stack images. The programs identify the vesicles with desired morphology and analyzes the data automatically. The programs can perform protein binding analysis on the membranes or state decision analysis of domain phase separation. We also show that the method can easily be applied to similar problems, such as intensity analysis of phase-separated protein droplets. CNN-based classification approach enables the identification of vesicles even from relatively complex samples. We demonstrate that the proposed artificial intelligence-assisted classification can further enhance the accuracy of the analysis close to the performance of manual examination in vesicle selection and vesicle state determination analysis. CONCLUSIONS: We developed a MATLAB based software capable of efficiently analyzing confocal fluorescence image data of giant unilamellar vesicles. The program can automatically identify GUVs with desired morphology and perform intensity-based calculation and state decision for each vesicle. We expect our method of CNN implementation can be expanded and applied to many similar problems in image data analysis.

Human-robot collaboration (HRC), where humans and robots work together to handle specific tasks, requires designing robots that can effectively support human beings. Robots need to conduct reasoning using commonsense knowledge (CSK), e.g., fundamental knowledge that humans possess and use subconsciously, in order to assist humans in challenging and dynamic environments. Currently, there are several effective CSK systems used for organizing information and facts, along with detecting objects and determining their properties. HRC is employed in various manufacturing tasks, such as paint spraying and assembly, in order to keep humans safe while increasing efficiency. Although there is a large array of research on HRC and on CSK, there is minimal research linking the two concepts together. This paper presents a novel system on human-robot collaboration guided by commonsense reasoning for automation in manufacturing tasks. This fits within the general realm of smart manufacturing. The primary focus is on improving the efficacy of human-robot co-assembly tasks. Evaluations conducted with online simulations and real-world experiments indicate that reasoning using CSK-based robot priorities enhances HRC as compared to simpler robot priorities, e.g., merely handling nearby objects. This system is modifiable and can be used for larger and more complex real-world tasks, thereby leading to improved automation in manufacturing. This paper demonstrates the scope of combining HRC and CSK, while future works will be able to further utilize the benefits of combining the two fields with significant impacts.Note to Practitioners—This paper is motivated by the human-robot collaboration problem in smart manufacturing. Robots operating by reasoning with commonsense priorities in human-robot collaboration enable faster task execution and better human work life. This can help balance work for humans and prevent injury. Adding robots to tasks accordingly does not necessarily decrease costs, but can limit human exposure to danger which is significant (and can also lower costs overall). Simulations and real-world experiments in our research using commonsense reasoning demonstrate how work is easier and better with human-robot collaboration. These factors are highly significant when tasks are repeated multiple times. The system is presented within automated manufacturing and is scalable for different real-world applications. Such automation is particularly helpful during recent times in the aftermath of the COVID-19 pandemic.

In human–robot shared manufacturing contexts, product parts or tools hand-over between the robot and the human is an important collaborative task. Facilitating the robot to figure out and predict human hand-over intentions correctly to improve the task efficiency in human–robot collaboration is therefore a necessary issue to be addressed. In this study, a teaching-learning-prediction (TLP) framework is proposed for the robot to learn from its human partner’s multimodal demonstrations and predict human hand-over intentions. In this approach, the robot can be programmed by the human through demonstrations utilizing natural language and wearable sensors according to task requirements and the human’s working preferences. Then the robot learns from human hand-over demonstrations online via extreme learning machine (ELM) algorithms to update its cognition capacity, allowing the robot to use its learned policy to predict human intentions actively and assist its human companion in hand-over tasks. Experimental results and evaluations suggest that the human may program the robot easily by the proposed approach when the task changes, as the robot can effectively predict hand-over intentions with competitive accuracy to complete the hand-over tasks.Note to Practitioners—This article is motivated by human–robot hand-over problems in smart manufacturing contexts. Product parts or tools delivery in worker–robot partnerships is an important collaborative task. We develop a teaching-learning-prediction (TLP) framework for the robot to learn from its human partner’s multimodal demonstrations and predict human hand-over intentions. The robot can be taught by human through natural language and wearable sensing information. The extreme learning machine (ELM) approach is employed for the robot to build its cognition capacity to predict human intentions actively and assist its human companion in hand-over tasks. We demonstrate that the proposed approach presents distinct and effective advantages to facilitate human–robot hand-over tasks in collaborative manufacturing contexts.

Human–robot collaborative assembly has been one of the next-generation manufacturing paradigms in which superiorities of humans and robots can be fully leveraged. To enable robots effectively collaborate with humans, similar to human–human collaboration, robot learning from human demonstrations has been adopted to learn the assembly tasks. However, existing feature-based approaches require critical feature design and extraction process and are usually complex to incorporate task contexts. Existing learning-based approaches usually require a large amount of manual effort for data labeling and also rarely consider task contexts. This article proposes a dual-input deep learning approach to incorporate task contexts into the robot learning from human demonstration process to assist human in assembly. In addition, online automated data labeling during human demonstration is proposed to reduce the training efforts for learning. The experimental validations on a realistic human–robot model car assembly task with safety-concerned execution designs demonstrate the effectiveness and advantages of the proposed approaches.

In human-robot collaboration, current widely used human-robot communication is mainly based on audio and haptic mediums. However, this kind of communication is stiff and mechanical. Inspired by human-human communication in which vision and hearing contribute over 88% for human perception, we propose a knowledge-driven audio-visual virtual agent system, which allows collaborative robots to present its knowledge and feelings in a human-like way. During the collaboration training process, the virtual agent will build its assembly knowledge of how to work with the co-worker based on inverse reinforcement learning. To deploy a co-assembly task with its human partner, the virtual agent will also be able to produce assembly knowledge-based responses, which include knowledge-driven speech and speech synchronized facial animations. By leveraging the proposed knowledge-driven virtual agent, the collaborative robot not only can fulfill the co-assembly task but also can communicate with human partner in a more natural way.

This Innovative Practice Work-In-Progress Paper presents a case of learning autonomous driving in tangible practice. As technology sustainably enhances the quality of life, intelligent systems continue to contribute solutions to some of the biggest challenges faced by humans. Autonomous vehicles offer humans the opportunity to increase transportation safety by reducing human errors on the road, preventing accidents, improving human productivity by reducing commuting time, and possibly mitigating air pollution. There is a critical shortage of educational and training programs in autonomous vehicles due to the high cost of full-size vehicles, computing and sensor equipment, and big lab space needed. To address this problem, we develop a 1/10-scale autonomous vehicle powered by pre-collision detection, lane tracking, and road sign recognition systems. The pre-collision system is built using ultrasonic sensors, and the Proportional-Integral-Derivative (PID) control is implemented to manipulate the vehicle's safety response. The Open-Source Computer Vision Library (OpenCV) is exploited to detect and process real-time on-road streaming video to enable lane-tracking and road sign recognition. AI techniques are utilized for the model training. Preliminary results of this work are presented and analyzed. We also discuss the future directions of this study.

This paper addresses commonsense knowledge (CSK) to enhance human-robot collaboration (HRC) in large scale smart manufacturing. As big data in collaborative robotics grows, CSK in useful to achieve task optimization as depicted in our simulation studies and laboratory experiments, extendable to real-world applications.

The trust of a robot in its human partner is a significant issue in human-robot interaction, which is seldom explored in the field of robotics. This study addresses a critical issue of robots' trust in humans during the human-robot collaboration process based on the data of human motions, past interactions of the human-robot pair, and the human's current performance in the co-carry task. The trust level is evaluated dynamically throughout the collaborative task that allows the trust level to change if the human performs false positive actions, which can help the robot avoid making unpredictable movements and causing injury to the human. Experimental results showed that the robot effectively assisted the human in collaborative tasks through the proposed computational trust model.

The novel coronavirus (Covid-19) has spread rapidly amongst countries all around the globe. Compared to the rise in cases, there are few Covid-19 testing kits available. Due to the lack of testing kits for the public, it is useful to implement an automated AI-based E-health decision support system as a potential alternative method for Covid-19 detection. As per medical examinations, the symptoms of Covid-19 could be somewhat analogous to those of pneumonia, though certainly not identical. Considering the enormous number of cases of Covid-19 and pneumonia, and the complexity of the related images stored, the data pertaining to this problem of automated detection constitutes big data. With rapid advancements in medical imaging, the development of intelligent predictive and diagnostic tools have also increased at a rapid rate. Data mining and machine learning techniques are widely accepted to aid medical diagnosis. In this paper, a huge data set of X-ray images from patients with common bacterial pneumonia, confirmed Covid-19 disease, and normal healthy cases are utilized for AI-based decision support in detecting the Coronavirus disease. The transfer learning approach, which enables us to learn from a smaller set of samples in a problem and transfer the discovered knowledge to a larger data set, is employed in this study. We consider transfer learning using three different models that are pre-trained on several images from the ImageNet source. The models deployed here are VGG16, VGG19, and ResNet101. The dataset is generated by gathering different classes of images. We present our approach and preliminary evaluation results in this paper. We also discuss applications and open issues.

This Innovative Practice Work-In-Progress Paper presents a situated learning-based robotics education pedagogy for computing students. Different from traditional textbook-directed robotics courses, in this work, we develop a hands-on, project-oriented robotics curriculum based on the situated learning methodology for undergraduate and graduate students. The innovative pedagogical framework and curriculum development are presented. Preliminary results and evaluations suggest that our situated learning-based pedagogy and developed robotics curriculum provide an effective solution for computing students to learn robotics.

This Innovative Practice Full Paper presents several educational simulation designs to transform learning in the earth and environmental sciences. In recent years, K-12 education has seen a widespread pedagogical shift from traditional textbook-based learning to a student-centered interactive learning environment. Innovative teaching technologies including games and simulations are exploited to make learning fun and engaging for students at various grades. As technology continues to advances, stakeholders such as teachers, parents, and educational policymakers are motivated to know the most effective technology platform to engage students in active learning. In this paper, we discuss some useful simulation techniques from the perspectives of users' experience, system deployment and developers' point of view. In addition, we also present the design and implementation of some Earth and Environmental Science simulations that are developed using NetLogo and JavaScript. Some student assessment results are also provided to compare the learning effectiveness between the traditional textbooks and computer simulation-based lessons. Our results show that computer simulation can help enhance the student's content retention and graph interpretation skills.

Human-robot collaborative assembly consists of humans and automated robots, who cooperate with each other to accomplish complex assembly tasks, which are difficult for either humans or robots to accomplish alone. There has been some success in statistics-based and optimization-based approaches to realize human-robot collaboration. However, they usually need a set of complex modeling and setup efforts and the robots usually need to be programmed by a well-trained expert. In this paper, we take a new approach by introducing convolutional neural networks (CNN) into the teaching- learning-collaboration (TLC) model for collaborative assembly tasks. The proposed approach can alleviate the need for complex modeling and setup compared to the existing approaches. It can collect and automatically label the data from human demonstrations and then train a CNN-based robot assistance model to make the robot assist humans in the assembly process in real-time. We have experimentally verified our proposed approach on a human-robot collaborative assembly platform and the results suggest that the robot can successfully learn from human demonstrations to automatically generate right actions to assist human in accomplishing assembly tasks.

Collaborative robots are widely employed in strict hybrid assembly tasks involved in intelligent manufacturing. In this paper, we develop a teaching-learning-collaboration (TLC) model for the collaborative robot to learn from human demonstrations and assist its human partner in shared working situations. The human could program the robot using natural language instructions according to his/her personal working preferences via this approach. Afterward, the robot learns from human assembly demonstrations by taking advantage of the maximum entropy inverse reinforcement learning algorithm and updates its task-based knowledge using the optimal assembly strategy. In the collaboration process, the robot is able to leverage its learned knowledge to actively assist the human in the collaborative assembly task. Experimental results and analysis demonstrate that the proposed approach presents considerable robustness and applicability in human-robot collaborative tasks. Note to Practitioners-This paper is motivated by the human-robot collaborative assembly problem in the context of advanced manufacturing. Collaborative robotics makes a huge shift from the traditional robot-in-a-cage model to robots interacting with people in an open working environment. When the human works with the robot in the shared workspace, it is significant to lessen human programming effort and improve the human-robot collaboration efficiency once the task is updated. We develop a TLC model for the robot to learn from human demonstrations and assist its human partner in collaborative tasks. Once the task is changed, the human may code the robot via natural language instructions according to his/her personal working preferences. The robot can learn from human assembly demonstrations to update its task-based knowledge, which can be leveraged by the robot to actively assist the human to accomplish the collaborative task. We demonstrate the advantages of the proposed approach via a set of experiments in realistic human-robot collaboration contexts.

With the deployment of collaborative robots in intelligent manufacturing, object hand-over between humans and robots plays a significant role in human-robot collaborations. In most collaboration studies, human hand-over intentions were usually assumed to be known by the robot, and the research mainly focused on robot motion planning and control during the hand-over process. Several approaches have been developed to control the human-robot hand-over, such as vision-based approach and physical contact-based approach, but their applications in manufacturing environments are limited due to various constraints, such as limited human working ranges and safety concerns. In this paper, we develop a practical approach using a wearable sensory system, which has a natural and simple configuration and can be easily utilized by humans. This approach could make a robot recognize a human's hand-over intentions and enable the human to effectively and naturally control the hand-over process. In addition, the approach could recognize the attribute classes of the objects in the human's hand using the wearable sensing and enable the robot to actively make decisions to ensure that graspable objects are handed over from the human to the robot. Results and evaluations illustrate the effectiveness and advantages of the proposed approach in human-robot hand-over control.

Human following is an important feature in various human-mobile-robot collaboration applications. Vision-based human following approaches employing visual servoing controls to achieve human following are commonly adopted. Such approaches, however, require to pre-define the desired human-following parameters and need to online extract features from the acquired images to calculate the human-following parameters which serve as the feedback of visual servoing controls. This paper proposes a novel visual servoing control using the non-vector space control theory, which makes the robot be able to personalize its desired human-following parameters as a desired image learnt from human-driven demonstrations. The approach provides an easy and intuitive way for humans to personalize mobile robots to complete human-following tasks in the manners that humans prefer. Experimental results demonstrate the effectiveness and advantage of the proposed approach.