Learning effective molecular representations is crucial for accurate property prediction in AI-aided drug discovery. However, most existing molecular pre-training methods are still primarily based on 2D topological graphs, limiting their ability to exploit 3D geometric information. Moreover, methods that do incorporate 3D geometry often do not distinguish between the roles of atom-centered and bond-centered representations. To address these limitations, we propose GDGraph, a geometryenhanced dual-view framework for molecular representation learning. GDGraph models molecular geometry from two complementary structural perspectives: an atom view for capturing global spatial dependencies and a bond view for modeling local geometric patterns. To support this dual-view design, we introduce a multi-scale geometric feature encoding scheme and a view-specific geometry-aware learning strategy, enabling each view to focus on the geometric dependencies it is best suited to capture. Extensive experiments demonstrate that GDGraph achieves strong and stable performance on molecular property prediction benchmarks, and effectively predicts geometrysensitive quantum chemical properties on the QM9 dataset.

The paradigm of web search is currently shifting from reactive information retrieval to Agentic AI, where proactive systems autonomously synthesise information to assist users. However, for these agents to be effective, they must understand which user parameters drive behaviour to resolve the personalisation cold-start problem. While cognitive architecture is a recognised factor, empirical evidence linking specific traits to web search interaction remains unclear. This paper investigates the Verbal-Imagery (V-I) cognitive style dimension and its influence on proactive search behaviour, mental workload (MWL), and overall search user interface (SUI) alignment. Through a controlled user study (N = 20), web search behaviours were evaluated using interaction logs and think-aloud protocols, while MWL and usability were assessed via NASA-TLX and the System Usability Scale (SUS). Our findings reveal that verbalisers and imagers adopt statistically distinct navigational preferences: ver-balisers prefer sporadic, reactive interactions, while imagers rely on structured, proactive synthesis such as the Knowledge Panel. It is worth noting that qualitative data identifies a "trust gap" in AI-generated overviews based on cognitive modality preferences. These results demonstrate that the V-I dimension is a critical parameter for user modelling in agentic systems. We conclude by proposing requirements for user-aware agentic information retrieval, providing a framework for agents to dynamically adapt their representation strategies to minimise cognitive friction and enhance trust in proactive computing environments. This work is licensed under a Creative Commons "Attribution 4.0 International" license.

Cognitive style, a user's habitual approach to information processing, offers a promising approach to personalising information searching (IS) systems, yet the underlying style-related search behaviours remain poorly understood. This study investigates how the Wholist–Analytic and Verbal–Imagery dimensions of cognitive style influence search behaviour and prefrontal cortex (PFC) activation during the Post-focus stage of exploratory search. Forty participants completed comparison search tasks while we recorded behavioural metrics, subjective workload ratings, and functional near-infrared spectroscopy (fNIRS) data. Our results demonstrate that cognitive style significantly predicted search engine results page (SERP) interaction patterns: analytics and imagers adopted structured navigation with detailed reading, whereas wholists and verbalisers preferred sporadic navigation with rapid scanning. Critically, fNIRS revealed distinct PFC activation patterns, specifically in the Ventrolateral PFC (VLPFC) and Dorsolateral PFC (DLPFC), corresponding to these behavioural differences. By mapping neuro-cognitive profiles to IR behaviours, this work provides empirical grounding for designing ''style-aware'' adaptive IR interfaces that tailor information density and navigational support to individual cognitive profiles.

The robust segmentation of different targets in multiple modality images is challenging due to factors such as low contrast, variations in target size and shape, and interference from diseases, which may lead to segmentation ambiguity. In addition, the assessment of the reliability of artificial intelligence is crucial for its clinical application. This paper proposes the Online Bayesian approximation based Uncertainty-aware Network (OBU-Net) for robust ophthalmic image segmentation. Our approach introduces an efficient online Bayesian method to update a spatial uncertainty map during training continuously. Then, the Spatial Uncertainty Aware Block (SUA-B) leverages the uncertainty map to localize and prioritize attention to ambiguous regions. Additionally, we extract pixel-wise confidence from multi-scale predictions to integrate hierarchical predictions. We compare OBU-Net with state-of-the-art (SOTA) methods on six datasets. The experimental results demonstrate that our method achieves the best overall performance across different modalities and segmentation tasks, highlighting the robustness of our approach. Additionally, metamorphic testing experiments were conducted, exploring the algorithm's stability against random perturbations. Lastly, we propose an image-level uncertainty score and demonstrate its effectiveness for evaluating the model's segmentation reliability.

Testing web forms is an essential activity for ensuring the quality of web applications. It typically involves evaluating the interactions between users and forms. Automated test-case generation remains a challenge for web-form testing: Due to the complex, multi-level structure of Web pages, it can be difficult to automatically capture their inherent contextual information for inclusion in the tests. Large Language Models (LLMs) have shown great potential for contextual text generation. This motivated us to explore how they could generate automated tests for web forms, making use of the contextual information within form elements. To the best of our knowledge, no comparative study examining different LLMs has yet been reported for web-form-test generation. To address this gap in the literature, we conducted a comprehensive empirical study investigating the effectiveness of 11 LLMs on 146 web forms from 30 open source Java web applications. In addition, we propose three HTML-structure-pruning methods to extract key contextual information. The experimental results show that different LLMs can achieve different testing effectiveness, with the GPT-4, GLM-4, and Baichuan2 LLMs generating the best web-form tests. Compared with GPT-4, the other LLMs had difficulty generating appropriate tests for the web forms: Their Successfully Submitted Rates (SSRs)—the proportions of the LLMs-generated web-form tests that could be successfully inserted into the web forms and submitted—decreased by 9.10% to 74.15%. Our findings also show that, for all LLMs, when the designed prompts include complete and clear contextual information about the web forms, more effective web-form tests were generated. Specifically, when using Parser-Processed HTML for Task Prompt (PH-P), the SSR averaged 70.63%, higher than the 60.21% for Raw HTML for Task Prompt (RH-P) and 50.27% for LLM-Processed HTML for Task Prompt (LH-P). With RH-P, GPT-4’s SSR was 98.86%, outperforming models like LLaMa2 (7B) with 34.47% and GLM-4V with 0%. Similarly, with PH-P, GPT-4 reached an SSR of 99.54%, the highest among all models and prompt types. Finally, this article also highlights strategies for selecting LLMs based on performance metrics, and for optimizing the prompt design to improve the quality of the web-form tests.

The adoption of immersive virtual reality (IVR) for gamified rehabilitation is increasing. However, a significant challenge relates to perceptions of virtual environments as empty and isolating, potentially increasing stress, particularly among older users (patients). This paper explores the use of a virtual peer mentor (VPM) in a custom IVR-rehabilitation game to provide guidance and companionship. This game specifically targets patients recovering from acute myocardial infarction (AMI), commonly known as a heart attack. Grounded in social support theory, the VPM provides three types of support: (1) informational support, through pre-exercise narratives detailing a shared medical history; (2) instrumental support, through real-time demonstrations of clinically-validated exercise movements; and (3) emotional support, through positive feedback and encouragement. A within-subjects study involving 30 hospitalized AMI patients (all over 47 years old) evaluated the effectiveness of the VPM-integrated IVR-rehabilitation game. Each participant experienced a baseline (no VPM) and VPM-integrated version of the game on separate days. The results from the Intrinsic Motivation Inventory (IMI) and the social presence module of the Game Experience Questionnaire (GEQ-SPM) show that the VPM resulted in significant increases in engagement, and statistically significant lower pressure/tension. Furthermore, participants exhibited high user acceptance (76.7%) and task-completion rates (98.5%), with minimal cybersickness. The findings demonstrate that a psychologically-grounded VPM can effectively reduce stress in middle- and older-age patients in an IVR rehabilitation setting.

Virtual Production (VP) is a novel approach to filmmaking that blends real-time computer graphics, live-action footage, and cutting-edge technologies (such as LED volumes and game engines like Unreal Engine). As China’s higher education (HE) landscape continues to grow and diversify, Sino- foreign higher education institutions (SfHEIs) — partnerships between local Chinese universities and international institutions — have become increasingly prominent: They were created as an innovative solution to challenges such as insufficient domestic capacity, outdated curricula, regional imbalances, limited global engagement, and quality-assurance gaps, and have themselves become centers of pedagogical, research, and institutional innovation. This paper examines the experience of a team of students and faculty at an SfHEI who collaborated on a new approach to support VP, using in-camera virtual effects (ICVFX). The paper explores the background to the project, its core technical innovations, and the team dynamics. Parallel to the technical development work, an Open Educational Resource (OER) was also created. This OER contains not only the technical elements of the project, thus helping future VP/ICVFX workers, but also the team-development experiences. The paper will be of interest not only to the VP/ICVFX community, but also to SfHEI students and staff, and to the OER community.

Learning effective molecular representations is crucial for accurate property prediction in artificial intelligence (AI)-aided drug discovery. Graph and fingerprint representations have been widely used to encode molecular topological structures and chemical substructures. To enhance the feature embedding of each modality and leverage their complementary strengths, we propose a novel Chemically-aware Attention-based Multi-modal Fusion Framework (CAMFF) for molecular representation learning, which integrates molecular graphs and extended-connectivity fingerprints by exploiting various attention mechanisms. Specifically, the proposed CAMFF consists of three modules: 1) a graph embedding module incorporating multi-head attention to capture local heterogeneous interactions and all-pair self-attention to capture long-range atomic dependencies from molecular graph representations; 2) a fingerprint embedding module using a pre-trained Mol2Vec model to generate dense chemical substructure representations; and 3) a chemically-aware feature interaction and fusion module incorporating self-attention to enable interactions between various chemical substructures and cross-attention to ensure effective multi-modal alignment and fusion. To evaluate the effectiveness of CAMFF, we compare it with 14 state-of-the-art methods across 9 molecular property prediction benchmarks. CAMFF demonstrates competitive predictive performance and improves interpretability through attention-based visualization, showing its potential for real-world drug discovery.

The rapid development of Large Language Models (LLMs), such as ChatGPT and DeepSeek, has revolutionized software development, particularly in the domain of automated code generation. These models, built on architectures like the Transformer, have demonstrated remarkable capabilities in generating human-like text and source code, significantly enhancing developer productivity and reducing development time. However, the widespread adoption of LLMs for code generation raises concerns regarding the reliability, quality, and potential risks associated with the generated code. This article illustrates and analyzes the state of the art in evaluating LLM-generated code, summarizing research findings, and application areas. This paper highlights the challenges in distinguishing between machine-generated and human-written code, as well as the potential for LLMs to introduce security vulnerabilities and maintainability issues. We discuss the implications of these findings for both researchers and practitioners, emphasizing the need for continued research in the evaluation of LLM-generated code. Finally, we identify gaps in the literature and propose future research directions, such as the development of more robust benchmarks and improved evaluation metrics. By providing a thorough overview of the current landscape, this paper provides a valuable resource for researchers and practitioners interested in LLM’s code generation capabilities and limitations. We also highlight the importance of ongoing evaluation and refinement of these models to ensure their safe and effective integration into software-development practices.

Large language models (LLMs) have rapidly become a powerful tool in automated code generation, yet most research has focused on their correctness and efficiency rather than the stylistic patterns of their outputs. In this preliminary study, we analyze the code patterns generated by five popular LLMs—ChatGPT, Gemini, Claude, Grok, and DeepSeek—in their free versions, across three LeetCode problems, one top-ranking each from the easy, medium, and hard categories. Our evaluation employs key metrics including inline comment density, naming conventions, and edge case handling, highlighting both similarities and differences in verbosity, comprehensibility, and robustness among the codes generated by models. The findings of this study have important implications for software engineering and education, suggesting that LLM-generated code can serve as both a tool for rapid prototyping and an effective learning resource for beginners. Our future work will extend this analysis to a broader set of coding challenges and compare LLM outputs with human-written code to develop robust criteria for evaluating automated code generation.

The increasing complexity of deep learning models, especially in black-box scenarios, presents significant challenges to traditional software testing methods. Due to the lack of transparency in neural networks’ decision-making processes and the non-deterministic nature of model outputs, traditional test oracle approaches become inadequate. To address this problem, Metamorphic Testing (MT) and its extended approach, Metamorphic Exploration (ME), provide new ideas for validating deep learning systems by defining Metamorphic Relations (MR) between inputs and outputs. However, existing image transformation-based MR faces new challenges in image synthesis scenarios, as these operations may destroy the contextual information and affect the model’s performance. This paper proposes a novel ME design for deep learning image synthesis networks and demonstrates its effectiveness using a visible-infrared image fusion network as the case study. The result identifies the performance degradation problem due to the tensor dimension manipulation error, which indicates that the ME not only detects defects but also helps developers deeply understand the internal mechanisms of complex systems through the Hypothesized Metamorphic Relation (HMR), thus providing unique value for software quality assurance (SQA) of AI-driven software.

Context: The rapid development of Machine Learning (ML) has led to the proposal of various ML models to improve credit score assessment, creating a need for effective validation methods to ensure their performance aligns with business expectations. Objective: This paper introduces a novel approach for validating credit scoring models by focusing on user-hypothesized business expectations, enabling testers to predict how input changes affect outputs and assess alignment with business intuition. Methods: The approach uses Metamorphic Testing (MT), applying Metamorphic Relations (MRs) to examine input–output relationships, and Metamorphic Exploration (ME), an advanced extension of MT that constructs MRs based on user expectations. A case study evaluates and contrasts three popular ML models, neural networks, random forests, and gradient boosting tree, using both traditional evaluation metrics in credit scoring and ME. The study investigates how models selected based on traditional metrics perform when evaluated against MRs. Results: Empirical findings reveal that all three models often violate MRs, with violations becoming more extensive as model complexity increases. Neural networks have low number of MR violations on average but tends to be less robust. Interestingly, random forests exhibit most MR violations relative to the other two models. Traditional metrics fail to capture these violations, highlighting their limitations in ensuring alignment with business expectations. Conclusions: ME is proposed as a complementary validation method for model selection and post-deployment monitoring, ensuring models adhere to business intuition. The study underscores the importance of combining traditional metrics with ME, particularly for complex models like neural networks, to improve reliability in real-world applications.

ABSTRACT With the increasingly frequent attacks on software systems, software security is an issue that must be addressed. Within software security, automated detection of software vulnerabilities is an important subject. Most existing vulnerability detectors rely on the features of a single code type (e.g., source code or intermediate representation [IR]), which may lead to both the global features of the code slices and the memory operation information not being captured or considered. In particular, vulnerability detection based on source‐code features cannot usually include some macro or type definition content. In this paper, we propose a vulnerability‐detection method that combines the semantic features of source code and the low level virtual machine (LLVM) IR. Our proposed approach starts by slicing (C/C++) source files using improved slicing techniques to cover more comprehensive code information. It then extracts semantic information from the LLVM IR based on the executable source code. This can enrich the features fed to the artificial neural network (ANN) model for learning. We conducted an experimental evaluation using a publicly‐available dataset of 11,381 C/C++ programs. The experimental results show the vulnerability‐detection accuracy of our proposed method to reach over 96% for code slices generated according to four different slicing criteria. This outperforms most other compared detection methods.

ABSTRACT Metamorphic testing (MT) is effective in detecting software failures; it detects failures by examining the metamorphic relations (MRs) among source test cases (STCs), follow‐up test cases (FTCs) and their respective outputs. The STCs together with the corresponding FTCs, considered as a whole, are called metamorphic groups (MGs). MT performance relies heavily on the MRs and MGs. Previous studies have mainly focused on improving MT performance by identifying effective MRs, or through generation of MGs with high quality, but have somewhat neglected the selection of MRs and MGs from existing ones. In this paper, we address this issue by introducing a new metric for guiding the selection of effective MR‐MG pairs from a new perspective: The MR‐MG pair is chosen such that the MR makes the current MG as far away as possible from the executed MGs. We design an MR‐MG pair selection algorithm, named metamorphic relation and group selection based on adaptive random testing (MRGS‐ART), to implement our metric. The intuition behind MRGS‐ART is that we attempt to improve MT performance by achieving an even distribution of STCs and FTCs in their corresponding input domains for all the MRs used. Experimental results indicate that MRGS‐ART can enhance MT performance. We believe that this is the first comprehensive and systematic demonstration, from the perspective of both MRs and MGs, that making STCs and FTCs evenly distributed in their corresponding input domains can improve MT performance. Finally, by analysing the experimental results, we provide guidance on how to most effectively implement MRGS‐ART.

Metamorphic testing has become one mainstream technique to address the notorious oracle problem in software testing, thanks to its great successes in revealing real-life bugs in a wide variety of software systems. Metamorphic relations, the core component of metamorphic testing, have continuously attracted research interests from both academia and industry. In the last decade, a rapidly increasing number of studies have been conducted to systematically generate metamorphic relations from various sources and for different application domains. In this article, based on the systematic review on the state of the art for metamorphic relations’ generation, we summarize and highlight visions for further advancing the theory and techniques for identifying and constructing metamorphic relations and discuss promising research directions in related areas.

Software developers can only obtain a very small amount of information from the individual failure-causing inputs, which makes debugging difficult. Therefore, it is necessary to explore additional failure-causing inputs (failure regions) using the known failure-causing inputs. In order to accurately and efficiently identify the failure region, we propose a novel two-stage search algorithm, TS-FRI. In the initial exploration stage, a round-robin search identifies several boundary failure-causing points, and the failure region's centroid is estimated. During the main search stage, the boundary failure-causing points are identified through iterative division of the input domain with an equally sized partitioning strategy. This results in the boundary points being as dispersed as possible around the failure-region boundary, with the polytope formed by the points approximating the failure region (e.g., a polygon in two dimensions). The proposed algorithm is validated through simulation and empirical analysis: The experimental results show that the TS-FRI accuracy is at least comparable to the best accuracy of the compared three algorithms, and can be ten times better. In addition, TS-FRI only takes a quarter of the computation time and half the failure-validation cost of the other algorithms.

In recent years, deep learning (DL) systems have been applied in many areas, including image processing and autonomous driving. Software testing is an important way to ensure the quality of software. Among various testing methods, random testing (RT) has been widely used for DL systems, due to its simplicity and efficiency. However, it has been criticized for its poor fault-detection effectiveness. As an enhancement of RT, adaptive random testing (ART) attempts to evenly spread test cases over the input domain, aiming to improve the distribution diversity. However, current ART methods for DL systems have low testing efficiency, particularly for image-based DL systems. This is because of the current reliance on visual geometry group network-16 (VGGNet-16) to extract image features to represent image inputs—VGGNet-16 is a 16-layer, deep convolutional neural network that has been widely used for image classification and feature extraction. Feature extraction with VGGNet-16 is very time-consuming, with each image being extracted as a high-dimensional vector. The (dis)similarity calculations for images with such high-dimensional vectors incur heavy computational overheads. To overcome these challenges, we propose a new ART approach:image-hashing-based ART(IHART). IHART uses image hashing to quickly extract features from each image, storing them as a low-dimensional binary vector. This can significantly reduce the computational costs for dissimilarity calculations during test-case generation. We report on a series of experiments, using several well-known datasets and DL systems, to evaluate the IHART performance. Our results show that, of the three mainstream image-hashing strategies studied, perceptual hashing delivers the best ART test-case generation performance—perceptual hashing, which is used in image deduplication and content searching, uses content features in the hashing process. Compared with current approaches, IHART performs very well in fault-detection effectiveness across most datasets and models, and significantly better fault-detection efficiency.

This study reports on the development and implementation of the Interactive Design Research Toolkit (IDRT), a mobile application that supports the teaching and learning (T&L) of Human-Centered Design (HCD) practices. Guided by the Attention, Relevance, Confidence, and Satisfaction (ARCS) Model of Motivational Design and leveraging mobile microlearning (MML) principles, the IDRT integrates multimodal instructions and gamification features to encourage students to employ HCD. Implemented in a design-project-based class, 37 students' perceptions and engagement were evaluated using the Instructional Materials Motivation Survey. Results indicate positive feedback, high engagement levels, and increased comprehension of HCD. Analysis of design reports reveals a more diverse application of HCD, particularly among higher-performing students, indicating the IDRT's influence on intrinsic motivation. However, challenges were identified among lower-performing students, indicating the need for further research to enhance engagement. Overall, the study underscores the potential of MML as a viable instruction method to influence students' motivation and proficiency positively and to scaffold the T&L of HCD.

Immersive Virtual Reality (IVR) exergaming presents a promising avenue to integrate physical exercise with engaging virtual experiences, potentially encouraging sustained physical activity. However, maintaining user motivation over extended periods poses a significant challenge. Recent research has introduced the Dynamic Difficulty Adjustment (DDA) mechanism, dynamically regulating exergame difficulty based on specific conditions to enhance user adaptation. While prior studies have predominantly focused on gaming performance to adjust difficulty, they often overlook the emotional impact on user motivation. This study investigates users' emotional responses to the game timer change (TC) as a DDA mechanism during IVR exergaming. Results indicate that subjects in the TC-implemented game displayed more neutral emotions, concomitant with improved gaming performance. Conversely, subjects in the game without TC exhibited a broader range of detected emotions (sad and happy), suggesting difficulties in adapting to in-game difficulty levels incongruent with their gaming abilities. Overall, this study establishes a foundation for future research in affective computing-based IVR exergaming, aiming to develop an intelligent autonomous DDA mechanism tailored to users' physical and mental conditions.

Graph Neural Networks (GNNs) have been widely used to model molecules with a graph representation. However, GNNs face inherent challenges in accurately modeling long-range atomic interactions and identifying complex molecular substructures. This research proposes a novel Three-branch Molecular Representation Learning Framework (TMRLF) for predicting molecular properties: it integrates one branch of a GNN that extracts local molecular structural information with two branches of fully connected networks that capture the chemical substructure based on two fingerprints. Specifically, to better capture the long-range interactions, the GNN is designed with an attention mechanism to enhance the atomic interactions. As the Morgan fingerprint effectively captures functional groups of molecules and another well-used molecular fingerprint in the field of drug discovery, the Extended Reduced Graph (ErG) Fingerprint specifically targets molecular features with pharmacological relevance. These two fingerprints are both utilized to complement the chemical information and long-range information processing at the level of key structural features that GNNs lack. The proposed TMRLF extracts a robust feature representation of molecules, crucial for accurately predicting molecular properties and identifying potential drug candidates. Our proposed TMRLF is compared against six state-of-the-art models on eight benchmark datasets. It demonstrates superior capability in predicting molecular properties. Its effectiveness is further highlighted through proof-of-concept validation in identifying potential inhibitors for the Son of Sevenless Homolog 1 (SOSI) protein in real-world drug discovery scenarios.

Observational studies are widely used in Human-Computer Interaction (HCI) research to evaluate usability and user experience with technologies. However, the act of observation may influence participant behaviour and performance, threatening the validity of study findings. This paper investigates the impact of three observation types on participant outcomes in a simulated HCI study context. Participants completed Sudoku puzzles under baseline (no observation), human observation, sensor-based observation, and combined human/sensor conditions. Performance was assessed by puzzle completion rates. The mental workload was measured via NASA-TLX surveys, heart rate, galvanic skin response, and infrared thermal imaging. Results showed observations negatively impacted performance versus baseline, with human observers inducing the greatest distraction. Experienced participants were more influenced than novices. Task medium also affected engagement and observation reactivity. Findings demonstrate observations introduce bias in HCI research, emphasising careful consideration of observation methods to improve result validity.

Purpose: This paper reports on an ongoing study examining the implementation of image-based generative AI in higher education to study the impacts and changes to learning behaviors and academic performance of architecture undergraduate students. The findings will be part of a methodological framework to evaluate whether or not AI is a high-value digital tool in this context. Approach: The study is designed through a series of workshops with architecture students, which aim to identify the role of image-based AI in the architecture design process for under-graduate students in Sino-foreign higher education institutions in order to assess the potential of using AI in the future architecture industry, especially for junior architects. Findings: The preliminary findings of this ongoing study indicate that AI increases the creativity of architecture design concepts, especially via better visual presentation for junior students. However, AI does not seem to be able to comprehend basic architecture design principles when it is implemented. Originality/value: The outcomes of this study will aid the larger teaching community when adopting AI in their teaching while mitigating the potential negative impacts on the students' learning experiences.

The recent release of the Apple Vision Pro has reignited interest in the metaverse, showcasing the intensified efforts of technology giants in developing platforms and devices to facilitate its growth. As the metaverse continues to proliferate, it is foreseeable that everyday environments will become increasingly saturated with its presence. Consequently, uncovering links to these metaverse items will be a crucial first step to interacting with this new augmented world. In this paper, we address the problem of establishing connections with virtual worlds within everyday environments, especially those that are not readily discernible through direct visual inspection. We introduce a vision-based approach leveraging Artcode visual markers to uncover hidden metaverse links embedded in our ambient surroundings. This approach progressively localises the access points to the metaverse, transitioning from coarse to fine localisation, thus facilitating an exploratory interaction process. Detailed experiments are conducted to study the performance of the proposed approach, demonstrating its effectiveness in Artcode localisation and enabling new interaction opportunities.

Metamorphic Testing (MT) has been repeatedly proven effective in detecting software faults. MT detects faults by checking the Metamorphic Relations (MRs) among Source Test Cases (STCs) and Follow-up Test Cases (FTCs) and the corresponding outputs. Metamorphic Groups (MGs) denote the associated STCs and FTCs. The performance of MT relates strongly to the MRs and MGs. However, previous studies that on MG generation mainly focused on improving the effectiveness (i.e. fault-detection capability) of MT, but to some extent overlooked the efficiency. This paper proposes a new kind of MG generation algorithms called Metamorphic-Testing-based Adaptive Random Testing through Partitioning (MT-PART). These algorithms at-tempt to improve both the effectiveness and the efficiency of MT by dynamically partitioning the input domain and generating new STCs and FTCs that are uniformly distributed over their corresponding input domains. Through empirical experiments, we found that our algorithms are able to significantly outper-form other existing MG generation algorithms in terms of test efficiency, while maintaining good test effectiveness.

This paper presents a pilot study that examines the capacity of novice testers to generate Metamorphic Relations (MRs) for autonomous driving systems (ADSs), specifically fo-cusing on parking functions. By comparing MRs generated by human participants with those generated by artificial intelligence (AI), we seek to understand the variances in quality, particularly in terms of correctness, applicability, novelty, and utility. Our findings indicate that despite receiving only minimal training, human participants were capable of producing MRs with a wide range of effectiveness. Notably, humans exhibited a potential for creative thinking, contrasting with AI's ability to generate MRs that adhere closely to technical and applicability standards. The study underscores the need for improved educational strategies aimed at enhancing the quality and confidence of MRs produced by humans. Future research directions will explore the optimization of training approaches, particularly within a constrained timeframe to create a positive learning experience and maintain participant engagement, to fully harness the creative capabilities of human learners in the context of ADS testing.

This study introduces a website serving as an Open Educational Resource (OER), dedicated to Metamorphic Testing (MT) and Metamorphic Relation (MR) generation, with a specific focus on Autonomous Driving Systems (ADSs). It offers a comprehensive introduction to MT and ADSs, and presents a specially designed scenario template that simplifies the MR generation process for ADS functions. This template enhances accessibility, making it more user-friendly for a wider audience, and facilitates systematic application, ensuring that users can apply test case and MR generation in a structured and organized manner. The MR generation guidelines that work with the template lower the learning barrier for beginners in MT, thus facilitating easier adoption and application of MT to ADSs.

Visible-infrared image fusion is a technique that extracts information from different sensors. It could be used to enhance human visual perception of video surveillance under low-light conditions, and provide rich information for subsequent tasks. Vision Transformer (ViT) based fusion algorithms require standardizing input images to a specific height and width that could be divided into a series of blocks of fixed size. Consequently, a scaling operation must be performed on the original image, which frequently decreases the quality of fusion results. This paper proposes a visible-infrared image fusion neural network that is insensitive to input size, by first utilizing a fixed-size image pre-fusion framework to generate lossless instructive fusion results (IFRs), followed by a size-insensitive enhancing framework that refines these preliminary fused images under the guidance of IFRs. It also has potential applicability to other image fusion algorithms, like multi-focus image fusion.

We introduce a web-based SQL query visualisation tool aimed at improving students' understanding of SQL query execution. The tool, available as an Open Educational Resource (OER), allows users to visualise the stages of a SQL query on an uploaded SQLite database. The web interface enables students to interactively explore each stage of the query and view results in a dynamically updated table. This initial version of the tool is intended to inspire further development and enhancement while providing educators with a valuable resource for teaching SQL concepts.

Summary When selecting services from a cloud center to build applications, the quality of service (QoS) is an important nonfunctional attribute to be considered. However, in actual application scenarios, the QoS details for many services may not be available. This has led to a situation where prediction of the missing QoS records for services has become a key problem for service selection. This article presents a selective ensemble learning (SEL) framework for prefilling‐based matrix factorization (PFMF) predictors. In each PFMF predictor, the improved collaborative filtering is defined by examining the stability of the QoS records when measuring the similarity of users (or services), and then used to prefill empty records in the initial QoS matrix. To ensure the diversity of the basic PFMF predictors, various prefilled QoS matrices are constructed for the matrix factorization. In this process, different reference weights are assigned to the original and the prefilled QoS records. Finally, particle swarm optimization is used to set the ensemble weights for the basic PFMF predictors. The proposed SEL on PFMF (SEL‐PFMF) algorithm is validated on a public dataset, where its prediction performance outperforms the state‐of‐the‐art algorithms, and also shows good stability.

The performance of metamorphic testing relates strongly to the quality of test cases. However, most related research has only focused on source test cases, ignoring follow-up test cases to some extent. In this paper, we identify a potential problem that may be encountered with existing metamorphic group generation algorithms. We then propose a possible solution to address this problem. Based on this solution, we design a new algorithm for generating effective source and follow-up test cases. To improve the performance (test effectiveness and efficiency) of metamorphic testing. We introduce the concept of the input-domain difference problem, which is likely to affect the performance of metamorphic group generation algorithms. We propose a new test-case distribution criterion for metamorphic testing to address this problem. Based on our proposed criterion, we further present a new metamorphic group generation algorithm, from a black-box perspective, with new distance metrics to facilitate this algorithm. Our algorithm performs significantly better than existing algorithms, in terms of test effectiveness, efficiency and test-case diversity. Through experiments, we find that the input-domain difference problem is likely to affect the performance of metamorphic group generation algorithms. The experimental results demonstrate that our algorithm can achieve good test efficiency, effectiveness, and test-case diversity.

Although many of today’s object detectors (ODs) are fairly powerful and advanced, most of them still suffer from high detection failure rates. To address this issue, we have developed an innovative, multiple-object detection validation method using a complex-network-community-based relevance metric. This metric aims to measure the relevance of multiple objects in the same OD output, based on our observation that a faulty OD output generally includes objects that are irrelevant or unrelated to each other. To verify the effectiveness of our method, we formulated four research questions, and performed an experiment with statistical analyses to address these questions. Our experiment provides strong support that our method (particularly the relevance metric) is highly effective at helping human testers in identifying faulty OD outputs.

Abstract Distributed systems have been widely used in many safety‐critical areas. Any abnormalities (e.g., service interruption or service quality degradation) could lead to application crashes or decrease user satisfaction. These things may cause serious economic losses. Among the various quality assurance approaches for distributed systems, log‐based anomaly detection (LAD) has become a popular research topic. Its popularity relates to system logs being able to record and reveal important run‐time information. This paper presents a general LAD framework for distributed systems. Log grouping and feature‐pattern mining are two crucial LAD components that impact on the anomaly‐detection effectiveness. We also present a systematic survey of techniques in these two directions; propose classification frameworks for log grouping and feature patterns; and summarize four log‐grouping techniques and five feature patterns (which refer to invariant relationships among logs that can be used for anomaly detection). To evaluate their applicability, we report on the findings when applying existing techniques to Ray, a popular industrial distributed system. Based on these findings, several open issues are identified, which provide potential guidance for future research and development.

ABSTRACT The proliferation of driver‐assistance features in vehicles has resulted in a growing interest among the public in fully autonomous driving systems (ADSs). However, the integration of software and hardware in these complex systems presents significant testing challenges, particularly with respect to ensuring passenger safety. To address these challenges, simulation has emerged as a crucial step in the testing of ADSs. This paper presents a solution to the challenges faced in testing ADSs, with a focus on the validation of ADS simulators. The proposed approach involves using simulations and metamorphic testing (MT) to generate multiple concrete metamorphic relations (MRs) for testing ADS simulators. In order to accomplish this goal, we introduce three metamorphic relation patterns (MRPs). Each MRP is accompanied by a metamorphic relation input pattern (MRIP) that aids in generating detailed MRs. These MRs are designed to identify potential issues within the ADS simulator. To simplify the testing process and facilitate MT for testers, a self‐evolving scenario‐testing framework is also presented. The framework allows testers to improve test cases and MRs iteratively until issues detected are confirmed. The benefits and limitations of the framework are demonstrated using an industry case study. Overall, this study offers a practical solution to the challenges in testing ADSs and provides useful insights into improving testing efficiency for researchers and practitioners in the field.

The segmentation of blurred cell boundaries in cornea endothelium microscope images is challenging, which affects the clinical parameter estimation accuracy. Existing deep learning methods only consider pixel-wise classification accuracy and lack of utilization of cell structure knowledge. Therefore, the segmentation of the blurred cell boundary is discontinuous. This paper proposes a structural prior guided network (SPG-Net) for corneal endothelium cell segmentation. We first employ a hybrid transformer convolution backbone to capture more global context. Then, we use Feature Enhancement (FE) module to improve the representation ability of features and Local Affinity-based Feature Fusion (LAFF) module to propagate structural information among hierarchical features. Finally, we introduce the joint loss based on cross entropy and structure similarity index measure (SSIM) to supervise the training process under pixel and structure levels. We compare the SPG-Net with various state-of-the-art methods on four corneal endothelial datasets. The experiment results suggest that the SPG-Net can alleviate the problem of discontinuous cell boundary segmentation and balance the pixel-wise accuracy and structure preservation. We also evaluate the agreement of parameter estimation between ground truth and the prediction of SPG-Net. The statistical analysis results show a good agreement and correlation.

Adaptive Random Testing(ART) enhances the testing effectiveness (including fault-detection capability) ofRandom Testing(RT) by increasing the diversity of the random test cases throughout the input domain. Many ART algorithms have been investigated such asFixed-Size-Candidate-Set ART(FSCS) andRestricted Random Testing(RRT), and have been widely used in many practical applications. Despite its popularity, ART suffers from the problem of high computational costs during test-case generation, especially as the number of test cases increases. Although several strategies have been proposed to enhance the ART testing efficiency, such as theforgetting strategyand thek-dimensional tree strategy, these algorithms still face some challenges, including: (1) Although these algorithms can reduce the computation time, their execution costs are still very high, especially when the number of test cases is large; and (2) To achieve low computational costs, they may sacrifice some fault-detection capability. In this paper, we propose an approach based onApproximate Nearest Neighbors(ANNs), calledLocality-Sensitive Hashing ART(LSH-ART). When calculating distances among different test inputs, LSH-ART identifies the approximate (not necessarily exact) nearest neighbors for candidates in an efficient way. LSH-ART attempts to balance ART testing effectiveness and efficiency.

Artificial intelligence (AI) has revolutionized software engineering (SE) by enhancing software development efficiency. The advent of pre-trained models (PTMs) leveraging transfer learning has significantly advanced AI for SE. However, existing PTMs that operate on individual code tokens suffer from several limitations: They are costly to train and fine-tune; and they rely heavily on labeled data for fine-tuning on task-specific datasets.In this paper, we present TransformCode, a novel framework that learns code embeddings in a contrastive learning manner. Our framework is encoder-agnostic and language-agnostic, which means that it can leverage any encoder model and handle any programming language.We also propose a novel data-augmentation technique called abstract syntax tree (AST) transformation, which applies syntactic and semantic transformations to the original code snippets, to generate more diverse and robust samples for contrastive learning. Our framework has several advantages over existing methods: (1) It is flexible and adaptable, because it can easily be extended to other downstream tasks that require code representation (such as code-clone detection and classification); (2) it is efficient and scalable, because it does not require a large model or a large amount of training data, and it can support any programming language; (3) it is not limited to unsupervised learning, but can also be applied to some supervised learning tasks by incorporating task-specific labels or objectives; and (4) it can also adjust the number of encoder parameters based on computing resources. We evaluate our framework on several code-related tasks, and demonstrate its effectiveness and superiority over the state-of-the-art methods such as SourcererCC, Code2vec, and InferCode.

Android compilers play a crucial role in Android app development. The correctness of the apps relies on the compilers because the source code of the app is translated into the target language by the compilers. The use of obfuscators is becoming the standard in app development to prevent reverse engineering or code tampering. Despite their importance, both compilers and obfuscators lack an oracle, which is the mechanism to determine the correctness of the execution, and hence they can be called untestable software. Metamorphic Testing (MT) is a state-of-the-art testing method that can test untestable software. MT tests software based on Metamorphic Relations (MRs). Recent studies have shown that program transformation, an MT-based compiler-testing strategy, is highly effective in revealing bugs in compilers. However, this strategy requires sophisticated tools that could take significant time to develop. Therefore, program transformation using obfuscators is proposed. Based on research into testing obfuscators using MT, it is suggested that an MT-based compiler-testing strategy could be achieved by using obfuscators. In addition, this method has the potential to detect bugs in both compilers and obfuscators. This paper reports on our experience using MT techniques to test compilers and obfuscators. We present three related MRs, two of which uncover evidence of faults.

Concerns have been growing over fake news and its impact. Software that can automatically detect fake news is becoming more popular. However, the accuracy and reliability of such fake-news detection software remains questionable, partly due to a lack of testing and verification. Testing this kind of software may face the oracle problem, which refers to difficulty (or inability) of identifying the correctness of the software’s output in a reasonable amount of time. Metamorphic testing (MT) has a record of effectively alleviating the oracle problem, and has been successfully applied to testing fake-news detection software. This paper reports on a study, extending previous work, exploring the use of MT for fake-news detection software. The study includes new metamorphic relations and additional experimental results and analysis. Some alternative MR-generation approaches are also explored. The study targets software where the output is a real/fake news decision, enhancing the applicability of MT to current fake-news detection software. The paper also explores the impact of the prediction accuracy of the fake-news detection software on the MT process. The study demonstrates the validity and applicability of MT to fake-news detection software. The prediction accuracy of the software has a greater impact on MT experiments with greater changes between the source and follow-up inputs, and less dependence on prediction stability. Some possible factors affecting the experimental results are discussed, and directions for future work are provided.

Automated Driving Systems (ADSs) have gained popularity recently. However, the unstable and unsafe ADSs have caused many traffic accidents and received widespread attention. One way to alleviate such issues is to enhance the correctness and efficiency of testing ADSs. Due to the difficulty of checking ADSs’ behavior such as parking the car, confirming the correctness of the actual behavior may be non-trivial or impossible. This kind of problem is called the test oracle problem. Unlike traditional software testing, Metamorphic Testing (MT) does not focus on the correctness of the actual strategy but examines whether or not the inputs and outputs of multiple executions of a Software Under Test (SUT) satisfy certain relations of the SUT, called Metamorphic Relations (MRs). The paper also implements Mutation Analysis (MA) on Baidu Apollo ADS to evaluate our MT. MA involves small modifications to a program’s source code to see if test-cases can detect these changes. This work was part of a larger endeavour to create an Open Educational Resource (OER) to support learning about how to apply MT to ADSs. This paper reports on an experience of implementing MT to test the Automated Parking System (APS) of Apollo ADS and applying MA to evaluate the MT.

This document1is an extended abstract of a Science of Computer Programming paper, "Candidate Test Set Reduction for Adaptive Random Testing: An Overheads Reduction Technique," presented as a J1C2 (Journal publication first, Conference presentation following) at the 30th IEEE International Conference on Software Analysis, Evolution and Reengineering (Saner 2023).The paper presents a candidate set reduction strategy to enhance the Fixed-Sized-Candidate-Set version of Adaptive Random Testing (FSCS-ART). The proposed method reduces the number of randomly-generated candidate test cases by retaining valuable, unused candidates from previous iterations. As the computational costs associated with a stored/retained candidate are less than the costs associated with a randomly-generating one, the overall computational overheads of FSCS-ART are reduced. The reported experimental studies show that the proposed method has a comparable failure-detection effectiveness to FSCS-ART, but less computational overheads.

Adaptive random testing(ART) is an enhancement ofrandom testing(RT), and aims to improve the RT failure-detection effectiveness by distributing test cases more evenly in the input domain. Many ART algorithms have been proposed, withfixed-size-candidate-setART (FSCS-ART) being one of the most effective and popular. FSCS-ART ensures high failure-detection effectiveness by selecting as the next test case the candidate farthest from previously executed test cases. Although FSCS-ART has good failure-detection effectiveness, it also faces some challenges, including heavy computational overheads. In this article, we propose an enhanced version of FSCS-ART,vantage point partitioning ART(VPP-ART). VPP-ART addresses the FSCS-ART computational overhead problem using VPP, while maintaining the failure-detection effectiveness. VPP-ART partitions the input domain space using amodified vantage point tree(VP-tree) and finds the approximate nearest executed test cases of a candidate test case in the partitioned subdomains—thereby significantly reducing the time overheads compared with the searches required for FSCS-ART. To enable the FSCS-ART dynamic insertion process, we modify the traditional VP-tree to support dynamic data. The simulation results show that VPP-ART has a much lower time overhead compared to FSCS-ART, but also delivers similar (or better) failure-detection effectiveness, especially in the higher dimensional input domains. According to statistical analyses, VPP-ART can improve on the FSCS-ART failure-detection effectiveness by approximately 50–58%. VPP-ART also compares favorably with theKD-tree-enhanced fixed-size-candidate-set ART(KDFC-ART) algorithms (a series of enhanced ART algorithms based on the KD-tree). Our experiments also show that VPP-ART is more cost-effective than FSCS-ART and KDFC-ART.

IEEE COMPSAC Workshops series is an established research event that complements the main focus of the IEEE COMPSAC conference, by focusing on specific topics that gain momentum in the research community. This paper introduces the overview of the IEEE COMPSAC 2022 co-located workshops, and discusses some open research issues that have emerged from the different workshops' topics and selected contributions.

The recent Facebook rebranding to Meta has drawn renewed attention to the metaverse. Technology giants, amongst others, are increasingly embracing the vision and opportunities of a hybrid social experience that mixes physical and virtual interactions. As the metaverse gains in traction, it is expected that everyday objects may soon connect more closely with virtual elements. However, discovering this “hidden” virtual world will be a crucial first step to interacting with it in this new augmented world. In this paper, we address the problem of connecting phys-ical objects with their virtual counterparts, especially through connections built upon visual markers. We propose a unified recognition framework that guides approaches to the metaverse access points. We illustrate the use of the framework through experimental studies under different conditions, in which an interactive and visually attractive decoration pattern, an Artcode, is used as the approach to enable the connection. This paper will be of interest to, amongst others, researchers working in Interaction Design or Augmented Reality who are seeking techniques or guidelines for augmenting physical objects in an unobtrusive, complementary manner.

Metamorphic testing (MT) has been growing in pop-ularity, but it can still be quite challenging and time-consuming to assess its performance. Typical approaches to performance assessment can require a series of steps, and depend on a variety of factors, often requiring serendipity. This can be a bottleneck for some aspects of MT research. Central to MT, metamorphic relations (MRs) represent necessary properties of the system under test (SUT). In traditional software testing, simulations are often employed to examine and compare the performance of dif-ferent testing strategies. However, these simulations are typically designed based on the assumed availability (and applicability) of a test oracle - a mechanism to decide the correctness of the SUT output or behaviour. A key reason for the popularity of MT is its proven record of effective software testing, without the need for a test oracle. This strength, however, also means that traditional ways of using simulations to analyse software testing approaches are not applicable for MT. This lack of cheap and fast ways to conduct simulation analyses of MT is a hurdle for many aspects of MT research, and may be an obstacle to its more widespread adoption. To address this, in this paper we introduce the concept of MR-violation regions (MRVRs), and show how they can be used for a certain category of MRs, Deterministic MRs (DMRs), to build simulation tools for MT. We analyse the differences between MRVRs and traditional, oracle-defined failure regions; and report on a preliminary case study exploring MRVRs in numerical-input-domain systems from previous MT studies. We anticipate that the proposed MT simulation framework may facilitate more research into MT, and may help lead to its more widespread adoption.

Computing systems are becoming increasingly complex and sophisticated. Technologies such as artificial intelligence, big data, and autonomous vehicles are pushing the boundaries of system size, complexity, and comprehensibility beyond anything seen before. These advances, however, have left the associated software quality assurance (SQA) tools and processes behind. This is compounded by many training and education programs also not attempting to address this inadequacy in the preparation of future software engineering professionals. We face a situation of extensively-deployed advanced computing systems, many of which lack sufficient SQA support. Metamorphic Testing (MT) and Metamorphic Exploration (ME) are SQA approaches that have a record of being able to alleviate some of the challenges associated with the advanced computer systems. This paper reports on an MT/ME experience with the Baidu Apollo autonomous driving system (ADS). The experience included identifying an apparent problem in Apollo, which was later confirmed to be a misunderstanding, but which illustrated the potential for ME to scaffold learning how to perform SQA on such complex systems. The report will be of benefit not only to other ADS developers and testers, but also to other SQA professionals, and especially to SQA trainers and educators.

This extended abstract presents an approach to enhance the Fixed-Sized-Candidate-Set Adaptive Random Testing (FSCS-ART) sampling strategy. SWFC-ART, the proposed approach, stores the previously-executed, non-failure-causing test cases into a Hierarchical Navigable Small World Graph (HNSWG) data structure and uses an efficient and consistent Nearest Neighbor Search (NNS) mechanism, especially for high-dimensional input domains. Our experiments show that SWFC-ART reduces the computational overhead of FSCS-ART from quadratic to log-linear order while retaining the failure-detection effectiveness of FSCS-ART.

[J1C2 Presentation Abstract at IEEE SERVICES 2022 for IEEE Transactions on Services Computing, 2022, 15(2):736-751. DOI: 10.1109/TSC.2019.2960496].

Abstract Test case prioritization (TCP) aims at scheduling test case execution so that more important test cases are executed as early as possible. Many TCP techniques have been proposed, according to different concepts and principles, with dissimilarity‐based TCP (DTCP) prioritizing tests based on the concept of test case dissimilarity: DTCP chooses the next test case from a set of candidates such that the chosen test case is farther away from previously selected test cases than the other candidates. DTCP techniques typically only use one aspect/granularity of the information or features from test cases to support the prioritization process. In this article, we adopt the concept of data fusion to propose a new family of DTCP techniques, data‐fusion‐driven DTCP (DDTCP), which attempts to use different information granularities for prioritizing test cases by dissimilarity. We performed an empirical study involving 30 versions of five subject programs, investigating the testing effectiveness and efficiency by comparing DDTCP against DTCP techniques that use a dissimilarity granularity. The experimental results show that not only does DDTCP have better fault‐detection rates than single‐granularity DTCP techniques, but it also appears to only incur similar prioritization costs. The results also show that DDTCP remains robust over multiple system releases.

In recent years, service oriented architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this article, we propose a dynamic random testing (DRT) technique for web services, which is an improvement over the widely-practiced random testing (RT) and partition testing (PT) approaches. We examine key issues when adapting DRT to the context of SOA, including a framework, guidelines for parameter settings, and a prototype for such an adaptation. Empirical studies are reported where DRT is used to test three real-life web services, and mutation analysis is employed to measure the effectiveness. Our experimental results show that, compared with the three baseline techniques, RT, Adaptive Testing (AT) and Random Partition Testing (RPT), DRT demonstrates higher fault-detection effectiveness with a lower test case selection overhead. Furthermore, the theoretical guidelines of parameter setting for DRT are confirmed to be effective. The proposed DRT and the prototype provide an effective and efficient approach for testing web services.

Since the popularization of social media, news has entered our lives digitally. While news is spreading broader and faster, fake news is becoming an increasingly popular topic. Fake news detection is therefore important in both social media and research areas. With artificial intelligence technology, software engineers have developed a lot of fake news detection systems. One of the biggest challenges for such systems is that they may face the oracle problem, which means that there may not be a way, or it may take too long time, to confirm the correctness of a specific output. Metamorphic Testing has been applied successfully to alleviate the oracle problem in many different areas, including in artificial intelligence. In this paper, we propose several metamorphic relations for fake news detection and report on experiments using metamorphic testing on fake news detection applications.

Awareness of web accessibility issues is necessary for, amongst other things, good website design. Good website design can mean the difference between disabled users being able to access the website content, or not. This paper describes the impact of a student-led project to develop a VR application, as an Open Education Resource (OER), to increase users’ knowledge and awareness of accessibility. An evaluation for the intervention delivered by the VR application was conducted, according to which, the VR application generally increased knowledge and awareness of web accessibility, but also had a negative impact on some users. The project targetted a Human Computer Interaction (HCI) class taught at the first Sino-foreign higher education institution, University of Nottingham Ningbo China (UNNC). UNNC has already been involved in research into flipped classrooms, technology-enhanced teaching, and the development of several OERs. This paper introduces the background, motivation and objectives, design, and specification of the VR application. The impact of the application on users is evaluated, and some possible future work is discussed.

In today’s digital era, the number of digital artefacts produced by students in universities around the world continues to rise. This paper describes a project that developed a digital artefacts platform to showcase students’ projects at an international level, aiming to increase the visibility of students’ work to global audiences. Due to the COVID-19 pandemic, a large proportion of the software engineering project development took place remotely, with students and staff at University of Nottingham Ningbo China (UNNC), a Sino-foreign Higher Education Institution in Mainland China, adopting remote collaboration tools and techniques. This paper presents the background, software engineering development, and the project’s unique characteristics. In addition, the challenges to project completion and remote collaboration, future recommendations, and the potential to extend this project into an Open Educational Resource (OER) are also discussed.

Information is indispensable in modern society. People’s daily communication and work depend on information transmission. Unsafe storage or transmission of data may result in privacy and security problems. One way to attempt to prevent such issues is to use encryption algorithms to transform information into encrypted forms. Because the encryption steps of most encryption algorithms are complex, deciding the correctness of the encrypted output may take a long time in practice. This kind of problem is called the Test Oracle problem. In contrast to traditional software testing, Metamorphic Testing (MT) does not focus on the correctness of each individual output, but examines whether the inputs and outputs of multiple executions of a Program Under Test (PUT) satisfy necessary relations of the PUT, called metamorphic relations. This paper reports on an experience of applying MT to test three encryption algorithms — Data Encryption Standard (DES), Triple Data Encryption Standard (3DES), and Advanced Encryption Standard (AES).

COVID-19 has caused heart-break and disruption. The People's Republic of China (PRC) was one of the first severely-impacted countries, but also one of the first to attempt to return to normalcy. Education, including Higher Education-the context of this paper-was disrupted, and the impact of these disruptions continues to be felt. A small consolation, or silver lining, has been the incredible efforts and innovations made by teachers to overcome the COVID-19 challenges. One such innovation is described in this paper: Drawing on the traditions of Metamorphic Exploration and Testing, the authors applied two different approaches to resolving a technical administration problem related to the management of assessments and marks for undergraduate software engineering team projects. This took place at University of Nottingham Ningbo China (UNNC), a Sino-foreign Higher Education Institution (HEI) with a history of staff-student collaboration, and education innovation. This paper outlines the experience and the insights learned from it. In addition to being of general interest to education administrators, especially those involved with project and marking administration, the innovative use of Metamorphic Testing ideas to develop the solution will be particularly interesting to computer scientists and software engineers.

Abstract Combinatorial interaction testing (CIT) aims at constructing a covering array (CA) of all value combinations at a specific interaction strength, to detect faults that are caused by the interaction of parameters. CIT has been widely used in different applications, with many algorithms and tools having been proposed to support CA construction. To date, however, there appears to have been no studies comparing different CA constructors when only some of the CA test cases are executed. In this paper, we present an investigation of five popular CA constructors: ACTS, Jenny, PICT, CASA and TCA. We conducted empirical studies examining the five programs, focusing on interaction coverage and fault detection. The experimental results show that when there is no preference or special justification for using other CA constructors, then Jenny is recommended—because it achieves better interaction coverage and fault detection than the other four constructors in many cases. Our results also show that when using ACTS or CASA, their CAs must be prioritized before testing. The main reason for this is that these CAs can result in considerable interaction coverage or fault detection capabilities when executing a large number of test cases; however, they may also produce the lowest rates of fault detection and interaction coverage.

This paper presents an automated, domain-independent, metamorphic testing platform called MTKeras. In this paper, we report on an investigation demonstrating the effectiveness and usability of MTKeras through five case studies in the four domains of image classification, sentiment analysis, search engines and database management systems. We also report on the effectiveness of combining metamorphic relation (input) patterns in individual metamorphic relations, enhancing the failure-finding abilities of the individual relations. The results of our experiments support combining patterns, and the use of MTKeras. The research reported in this paper shows the applicability of metamorphic relation patterns, and introduces a practical tool for the research community.

Random testing (RT) is a well-studied testing method that has been widely applied to the testing of many applications, including embedded software systems, SQL database systems, and Android applications. Adaptive random testing (ART) aims to enhance RT's failure-detection ability by more evenly spreading the test cases over the input domain. Since its introduction in 2001, there have been many contributions to the development of ART, including various approaches, implementations, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on ART, classifying techniques, summarizing application areas, and analyzing experimental evaluations. This paper also addresses some misconceptions about ART, and identifies open research challenges to be further investigated in the future work.

Recently, businesses and government bodies in the Architecture, Engineering, and Construction (AEC) sector have faced a key challenge to the delivery of effective project outcomes. Asset Information Modelling (AIM) is a process aimed to serve the asset process once the project is handed over. Considering that Blockchain is a new data management technology that is intended to provide security within a decentralized culture, this could be the foundation for improving asset processes. The research is achieved through a systematic review of the relevant literature. Several studies have been identified in Building Information Modelling (BIM) and Blockchain in the construction domain. However, there is little research conducted in the context of Asset Management Processes (AMP). Skilled people are the main challenge for AIM and Blockchain: findings show that a lack of technological awareness and its link to the adoption of using AIM and Blockchain in the asset management life cycle is the main problem.

There is a lack of accurate and valid data provision during the construction project lifecycle, and throughout the in-use phase of building assets. Construction needs to adopt a new technology to support building assets. Due to the current limitation of construction technology, Blockchain can be considered a new revolution within the most advanced construction applications. This paper proposes a framework for assembling Asset Information Models (AIMs) through private Blockchain to allow the owners, consultants, contractors, and suppliers to upload and visualise the data anywhere at any time throughout the lifecycle of the building to support project owners in making decisions. This paper introduces a conceptual framework through the processes of data analysis between the construction stages. The framework will improve current management efficiency in many ways such as: model upload/download, lack of data, partial model exchange, clash detection, conflict resolution, multiple data model formats, and data security. This helps project owners and maintenance teams to enhance their decision-making processes and improve the trust between stakeholders.

This paper describes an on-going project to develop a Virtual Reality platform to deliver phobia-inspired experiences. These experiences could induce a reaction in the user that may help the user overcome, or alleviate, the phobia. The platform includes monitoring sensors that could be used to measure how much impact the experience is having. The project development has been taking place at a Sino-foreign Higher Education Institution in Mainland China, University of Nottingham Ningbo China (UNNC). UNNC has already been host to a number of OER (Open Educational Resource) development projects, and the current project is also anticipated to eventually be released to the OER community. This paper presents the background, development, and current state of the project. Challenges to project completion, and future work are also outlined.

This paper describes an on-going project to develop diagnostic profiling tools for healthcare professionals to identify potential speech and language developmental problems of Chinese-speaking children. The tools aim to provide a technical advance in helping children who may have speech impairment or language delay. The project is currently being carried out as a collaboration with Chinese healthcare professionals, and a multidisciplinary team including applied linguists, speech and language pathologists, and computer scientists (staff and students) at the first Sino-foreign higher education institution in China, University of Nottingham Ningbo China (UNNC). The paper presents the background, development, and current state of the project. Challenges to project completion, including difficulties encountered due to the Covid-19 outbreak and subsequent world-wide emergency and lock-down are also discussed.

Adaptive random testing (ART) aims at enhancing the testing effectiveness of random testing (RT) by more evenly spreading test cases over the input domain. Many ART methods have been proposed, based on various, different notions. For example, distance-based ART (DART) makes use of the concept of distance to implement ART, attempting to generate new test cases that are far away from previously executed ones. The Euclidean distance has been a popular choice of distance metric, used in DART to evaluate the differences between test cases. However, is the Euclidean distance the most suitable choice for DART? To answer this question, we conducted a series of simulations to investigate the impact that the Euclidean distance, and its many variations, has on the testing effectiveness of DART. The results show that when the dimensionality of the input domain is low, the Euclidean distance may indeed be a good choice. However, when the dimensionality is high, it appears to be less suitable.

This paper reports on the use of program slicing concepts and partial execution tracing at Adobe Analytics to address a major limitation of differential testing --- namely, how to deal with the large numbers of differences typically produced by this regression testing technique. Manual verification, typically used to verify detected differences, is tedious, time-consuming and error-prone. This severely limits the volume of testing that can be done and thereby reduces adoption of differential testing. It is hoped that, by sharing this experience, researchers with expertise in program slicing might be motivated to help solve some of the issues and limitations encountered during this novel application of slicing to a real-world industrial problem.

Summary Web vulnerability scanners (WVSs) are tools that can detect security vulnerabilities in web services. Although both commercial and open‐source WVSs exist, their vulnerability detection capability and performance vary. In this article, we report on a comparative study to determine the vulnerability detection capabilities of eight WVSs (both open and commercial) using two vulnerable web applications: WebGoat and Damn vulnerable web application. The eight WVSs studied were: Acunetix; HP WebInspect; IBM AppScan; OWASP ZAP; Skipfish; Arachni; Vega; and Iron WASP. The performance was evaluated using multiple evaluation metrics: precision; recall; Youden index; OWASP web benchmark evaluation; and the web application security scanner evaluation criteria. The experimental results show that, while the commercial scanners are effective in detecting security vulnerabilities, some open‐source scanners (such as ZAP and Skipfish) can also be effective. In summary, this study recommends improving the vulnerability detection capabilities of both the open‐source and commercial scanners to enhance code coverage and the detection rate, and to reduce the number of false‐positives.

Abstract test cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (e.g., in regression testing), test case prioritization (TCP) has been proposed to improve the testing quality, aiming at ordering test cases to increase the speed with which faults are detected. One intuitive and extensively studied TCP technique for ATCs is λ-wise Level-combination Coverage based Prioritization (λLCP), a static, black-box prioritization technique that only uses the ATC information to guide the prioritization process. A challenge facing λLCP, however, is the necessity for the selection of the fixed prioritization strength λ before testing-testers need to choose an appropriate λ value before testing begins. Choosing higher λ values may improve the testing effectiveness of λLCP (e.g., by finding faults faster), but may reduce the testing efficiency (by incurring additional prioritization costs). Conversely, choosing lower λ values may improve the efficiency, but may also reduce the effectiveness. In this paper, we propose a new family of λLCP techniques, Repeated Small-strength Level-combination Coverage-based Prioritization (RSLCP), that repeatedly achieves the full combination coverage at lower strengths. RSLCP maintains λLCP's advantages of being static and black box, but avoids the challenge of prioritization strength selection. We have performed an empirical study involving five different versions of each of five C programs. Compared with λLCP, and Incremental-strength LCP (ILCP), our results show that RSLCP could provide a good tradeoff between testing effectiveness and efficiency. Our results also show that RSLCP is more effective and efficient than two popular techniques of Similarity-based Prioritization (SP). In addition, the results of empirical studies also show that RSLCP can remain robust over multiple system releases.

Modern information technology paradigms, such as online services and off-the-shelf products, often involve a wide variety of users with different or even conflicting objectives. Every software output may satisfy some users, but may also fail to satisfy others. Furthermore, users often do not know the internal working mechanisms of the systems. This situation is quite different from bespoke software, where developers and users typically know each other. This paper proposes an approach to help users to better understand the software that they use, and thereby more easily achieve their objectives-even when they do not fully understand how the system is implemented. Our approach borrows the concept of metamorphic relations from the field of metamorphic testing (MT), using it in an innovative way that extends beyond MT. We also propose a “symmetry” metamorphic relation pattern and a “change direction” metamorphic relation input pattern that can be used to derive multiple concrete metamorphic relations. Empirical studies reveal previously unknown failures in some of the most popular applications in the world, and show how our approach can help users to better understand and better use the systems. The empirical results provide strong evidence of the simplicity, applicability, and effectiveness of our methodology.

The issue of validity threats in empirical software engineering research is important. However, some authors overlook this, focusing on validating their work through application of fundamental testing techniques, instead. However, testing is different to empirical validation, with the latter being more concerned about how experimental conclusions are justified. An important factor that can render an experimental conclusion incorrect is researcher's bias, which can be especially relevant when setting the experimental parameters. Therefore, consideration of validity threats is essential to enable confidence in research results and assure the research quality. This paper provides a practical approach for mitigating threats to validity in empirical software engineering using a sequence of software activities. The paper is based on a real-world traceability case study for illustration purposes.

Smart pervasive sensor networks are becoming an important part of our daily lives. Low-power, high-availability and high-throughput 5G mobile networks provide the necessary communication means for highly pervasive sensor networks, introducing a technological disruption to health monitoring. The meaningful use of large concurrent sensor networks in healthcare requires multi-level health knowledge integration with sensor data streams. In this paper, we highlight some software engineering and data-processing issues that can be addressed by metamorphic testing. The proposed solution combines data streaming with filtering and cross-calibration, use of medical knowledge for system operation and data interpretation, and IoT-based calibration using certified linked diagnostic devices.

This document is an extended abstract of an IEEE Software paper, "Metamorphic Testing: Testing the Untestable," presented as a J1C2 (Journal publication first, Conference presentation following) at the IEEE Computer Society signature conference on Computers, Software and Applications (COMPSAC 2019), hosted by Marquette University, Milwaukee, Wisconsin, USA.

Test‐case prioritisation (TCP) attempts to schedule the order of test‐case execution such that faults can be detected as quickly as possible. TCP has been widely applied in many testing scenarios such as regression testing and fault localisation. Abstract test cases (ATCs) are derived from models of the system under test and have been applied to many testing environments such as model‐based testing and combinatorial interaction testing. Although various empirical and analytical comparisons for some ATC prioritisation (ATCP) techniques have been conducted, to the best of the authors’ knowledge, no comparative study focusing on the most current techniques has yet been reported. In this study, they investigated 18 ATCP techniques, categorised into four classes. They conducted a comprehensive empirical study to compare 16 of the 18 ATCP techniques in terms of their testing effectiveness and efficiency. They found that different ATCP techniques could be cost‐effective in different testing scenarios, allowing us to present recommendations and guidelines for which techniques to use under what conditions.

One goal of software testing may be the identification or generation of a series of test cases that can detect a fault with as few test executions as possible. Motivated by insights from research into failure-causing regions of input domains, the even-spreading (even distribution) of tests across the input domain has been identified as a useful heuristic to more quickly find failures. This finding has encouraged a shift in focus from traditional random testing (RT) to its enhancement, adaptive random testing (ART), which retains the randomness of test input selection, but also attempts to maintain a more evenly distributed spread of test inputs across the input domain. Given that there are different ways to achieve the even distribution, several different ART methods and approaches have been proposed. This paper presents a new ART method, called ART by orthogonal recursive bisection (ART-ORB), which explores the advantages of repeated geometric bisection of the input domain, combined with restriction regions, to evenly spread test inputs. Experimental results show a better performance in terms of fewer test executions than RT to find failures. Compared with other ART methods, ART-ORB has comparable performance (in terms of required test executions), but incurs lower test input selection overheads, especially in higher dimensional input space. It is recommended that ART-ORB can be used in testing situations involving expensive test input execution.

Abstract test cases are derived by modeling the system under test, and have been widely applied in practice, such as for software product line testing and combinatorial testing. Abstract test case prioritization (ATCP) is used to prioritize abstract test cases and aims at achieving higher rates of fault detection. Many ATCP algorithms have been proposed, using different prioritization criteria and information. One ATCP approach makes use of fixed-strength level-combinations information covered by abstract test cases, and is called fixed-strength interaction coverage based prioritization (FICBP). Before using FICBP, the prioritization strength λ needs to be decided. Previous studies have generally focused on λ values ranging between 1 and 6. However, no study has investigated the appropriateness of such a range, nor how to assign the prioritization strength for FICBP. To answer these questions, this paper reports on an empirical study involving four real-life programs (each of which with six versions). The experimental results indicate that λ should be set approximately equal to a value corresponding to half of the number of parameters, when testing resources are sufficient. Our results also show that when testing resources are limited or insufficient, either small or large λ values are suggested for FICBP.

In Systems Engineering (SE), development of complex systems involves a collaboration of expertise from different domains. Heterogeneous artifacts are generated using different modeling tools. Capturing the traceability information among these artifacts helps serve many purposes, including change impact analysis; validation and verification; and requirements tracking. However, creating trace links among these heterogeneous artifacts is problematic. No precise semantics exist for the trace links that relate them. This paper shows how to capture traceability information in a system with heterogeneous artifacts, illustrated here using an avionics case study that uses a traceability model and a trace links taxonomy that we constructed and published previously.

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

Test case prioritization (TCP) attempts to order test cases such that those which are more important, according to some criterion or measurement, are executed earlier. TCP has been applied in many testing situations, including, for example, regression testing. An abstract test case (also called a model input) is an important type of test case, and has been widely used in practice, such as in configurable systems and software product lines. Similarity-based test case prioritization (STCP) has been proven to be cost-effective for abstract test cases (ATCs), but because there are many similarity measures which could be used to evaluate ATCs and to support STCP, we face the following question: How can we choose the similarity measure(s) for prioritizing ATCs that will deliver the most effective results? To address this, we studied fourteen measures and two popular STCP algorithms - local STCP (LSTCP), and global STCP (GSTCP). We also conducted an empirical study of five realworld programs, and investigated the efficacy of each similarity measure, according to the interaction coverage rate and fault detection rate. The results of these studies show that GSTCP outperforms LSTCP - in 61% to 84% of the cases, in terms of interaction coverage rates; and in 76% to 78% of the cases with respect to fault detection rates. Our studies also show that Overlap, the simplest similarity measure examined in this study, could obtain the overall best performance for LSTCP; and that Goodall3 has the best performance for GSTCP.

Open educational resources (OERs) are essentially resources made freely available to the public for the purpose of enabling education. This paper examines the experiences of a collaborative team of interdisciplinary teachers and computer science students in their research and development of an OER designed to facilitate student note-making, and research. As expected, the team encountered, and overcame, a number of challenges, with both teachers and students learning from the experiences. The OER has been completed, and has been deployed in a full research study. The reflections from the team may help guide similar projects and other forms of student-centeredcollaboration and education.

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

The process of component security testing can produce massive amounts of monitor logs. Current approaches to detect implicit security exceptions (those which cannot be identified by visual inspection alone) compare correct execution sequences with fixed patterns mined from the execution of sequential patterns in the monitor logs. However, this is not efficient and is not suitable for mining large monitor logs. To enable effective mining of implicit security exceptions from large monitor logs, this paper proposes a method based on improved variable-length sequential pattern mining. The proposed method first mines the variable-length sequential patterns from correct execution sequences and from actual execution sequences, thus reducing the number of patterns. The sequential patterns are then detected using the Sunday string-searching algorithm. We conducted an experimental study based on this method, the results of which show that the proposed method can efficiently detect the implicit security exceptions of components.

Purpose This paper presents an initial development of a personal data attitude (PDA) measurement instrument based on established psychometric principles. The aim of the research was to develop a reliable measurement scale for quantifying and comparing attitudes towards personal data that can be incorporated into cybersecurity behavioural research models. Such a scale has become necessary for understanding individuals’ attitudes towards specific sets of data, as more technologies are being designed to harvest, collate, share and analyse personal data. Design/methodology/approach An initial set of 34 five-point Likert-style items were developed with eight subscales and administered to participants online. The data collected were subjected to exploratory and confirmatory factor analyses and MANOVA. The results are consistent with the multidimensionality of attitude theories and suggest that the adopted methodology for the study is appropriate for future research with a more representative sample. Findings Factor analysis of 247 responses identified six constructs of individuals’ attitude towards personal data: protective behaviour, privacy concerns, cost-benefit, awareness, responsibility and security. This paper illustrates how the PDA scale can be a useful guide for information security research and design by briefly discussing the factor structure of the PDA and related results. Originality/value This study addresses a genuine gap in research by taking the first step towards establishing empirical evidence for dimensions underlying personal data attitudes. It also adds a significant benchmark to a growing body of literature on understanding and modelling computer users’ security behaviours.

Random testing (RT) has been identified as one of the most popular testing techniques, due to its simplicity and ease of automation. Adaptive random testing (ART) has been proposed as an enhancement to RT, improving its fault-detection effectiveness by evenly spreading random test inputs across the input domain. To achieve the even spreading, ART makes use of distance measurements between consecutive inputs. However, due to the nature of object-oriented software (OOS), its distance measurement can be particularly challenging: Each input may involve multiple classes, and interaction of objects through method invocations. Two previous studies have reported on how to test OOS at a single-class level using ART. In this study, we propose a new similarity metric to enable multiclass level testing using ART. When generating test inputs (for multiple classes, a series of objects, and a sequence of method invocations), we use the similarity metric to calculate the distance between two series of objects, and between two sequences of method invocations. We integrate this metric with ART and apply it to a set of open-source OO programs, with the empirical results showing that our approach outperforms other RT and ART approaches in OOS testing.

Combinatorial interaction testing is a well-studied testing strategy that aims at constructing an effective interaction test suite (ITS) of a specific generation strength to identify interaction faults caused by the interactions among factors. Due to limited testing resources in practice, for example in combinatorial interaction regression testing, interaction test suite prioritization (ITSP) has been proposed to improve the efficiency of testing. An intuitive ITSP strategy that has been widely used in practice is fixed-strength interaction coverage based prioritization (FICBP). FICBP makes use of a property of the ITS: interaction coverage at a fixed prioritization strength. However, a challenge facing FICBP is that, when the ITS is large, the prioritization cost can be very high. In this paper, we propose a new FICBP method that, by repeatedly using base choice coverage (i.e., one-wise coverage) during the prioritization process, improves testing efficiency while maintaining testing effectiveness. The empirical studies show that our method has fault detection capability comparable to current FICBP methods, but obtains more stable results in many cases. Additionally, our method requires considerably less prioritization time than other FICBP methods at different prioritization strengths.

Unified Modeling Language (UML) activity diagrams are widely used to model concurrent interaction among multiple objects. In this paper, we propose a transformation-based approach to generating scenario-oriented test cases for applications modeled by UML activity diagrams. Using a set of transformation rules, the proposed approach first transforms a UML activity diagram specification into an intermediate representation, from which it then constructs test scenarios with respect to the given concurrency coverage criteria. The approach then finally derives a set of test cases for the constructed test scenarios. The approach resolves the difficulties associated with fork and join concurrency in the UML activity diagram and enables control over the number of the resulting test cases. We further implemented a tool to automate the proposed approach and studied its feasibility and effectiveness using a case study. Experimental results show that the approach can generate test cases on demand to satisfy a given concurrency coverage criterion and can detect up to 76.5% of seeded faults when a weak coverage criterion is used. With the approach, testers can not only schedule the software test process earlier, but can also better allocate the testing resources for testing concurrent applications. Copyright © 2015 John Wiley & Sons, Ltd.

Fault Localization Prioritization (FLP) aims at reordering existing test cases so that the location of detected faulty components can be identified earlier, using certain fault localization techniques. Although some researchers have proposed adaptive prioritization strategies with white-box code coverage information, such information may not always be available. In this paper, we address the FLP problem using black-box information derived from partitioning the input domain. Based on the well-known technique of Spectra-Based Fault Localization (SBFL), three test case prioritization strategies are designed following some basic SBFL heuristics. The implementation of these proposed strategies relies only on the partition information, and does not require any test case execution history. Experiments show that our strategies, when compared with pure random selection, result in a faster localization of faulty statements, reducing the number of test case executions required. Here, we analyze the characteristics and merits of the three proposed strategies.

Software Testing is an approach to ensuring the quality of software systems. Testing of safety-critical systems often requires conformance to certain code coverage criteria, including for example, in aviation, Modified Condition/Decision Coverage (MC/DC). In some situations, however, access to the actual code may be restricted with black Box approaches, and testers may only be able to use models of the system, such as those in MATLAB's Simulink. Without access to the code, exact code coverage measurement may not be possible. This paper presents a method of identifying and using the Simulink model's constraints to generate test cases which can achieve high coverage of the actual source code. A case study confirming the relationship between the model's coverage and the code coverage is also presented.

In software testing, something which can verify the correctness of test case execution results is called an oracle. The oracle problem occurs when either an oracle does not exist, or exists but is too expensive to be used. Metamorphic testing is a testing approach which uses metamorphic relations, properties of the software under test represented in the form of relations among inputs and outputs of multiple executions, to help verify the correctness of a program. This paper presents new empirical evidence to support this approach, which has been used to alleviate the oracle problem in various applications and to enhance several software analysis and testing techniques. It has been observed that identification of a sufficient number of appropriate metamorphic relations for testing, even by inexperienced testers, was possible with a very small amount of training. Furthermore, the cost-effectiveness of the approach could be enhanced through the use of more diverse metamorphic relations. The empirical studies presented in this paper clearly show that a small number of diverse metamorphic relations, even those identified in an ad hoc manner, had a similar fault-detection capability to a test oracle, and could thus effectively help alleviate the oracle problem.

Combinatorial interaction testing is a well-recognized testing method, and has been widely applied in practice, often with the assumption that all test cases in a combinatorial test suite have the same fault detection capability. However, when testing resources are limited, an alternative assumption may be that some test cases are more likely to reveal failure, thus making the order of executing the test cases critical. To improve testing cost-effectiveness, prioritization of combinatorial test cases is employed. The most popular approach is based on interaction coverage, which prioritizes combinatorial test cases by repeatedly choosing an unexecuted test case that covers the largest number of uncovered parameter value combinations of a given strength (level of interaction among parameters). However, this approach suffers from some drawbacks. Based on previous observations that the majority of faults in practical systems can usually be triggered with parameter interactions of small strengths, we propose a new strategy of prioritizing combinatorial test cases by incrementally adjusting the strength values. Experimental results show that our method performs better than the random prioritization technique and the technique of prioritizing combinatorial test suites according to test case generation order, and has better performance than the interaction-coverage-based test prioritization technique in most cases.

Adaptive random testing (ART) methods are software testing methods which are based on random testing, but which use additional mechanisms to ensure more even and widespread distributions of test cases over an input domain. Restricted random testing (RRT) is a version of ART which uses exclusion regions and restriction of test case generation to outside these regions. RRT has been found to perform very well, but incurs some additional computational cost in its restriction of the input domain. This paper presents a method of reducing overheads called forgetting, where the number of test cases used in the restriction algorithm can be limited, and thus the computational overheads reduced. The motivation for forgetting comes from its importance as a human strategy for learning. Several implementations are presented and examined using simulations. The results are very encouraging

Restricted Random Testing (RRT) is a new method of testing software that improves upon traditional Random Testing (RT) techniques. Research has indicated that failure patterns (portions of an input domain which, when executed, cause the program to fail or reveal an error) can influence the effectiveness of testing strategies. For certain types of failure patterns, it has been found that a widespread and even distribution of test cases in the input domain can be significantly more effective at detecting failure compared with ordinary RT. Testing methods based on RT, but which aim to achieve even and widespread distributions, have been called Adaptive Random Testing (ART) strategies. One implementation of ART is RRT. RRT uses exclusion zones around executed, but non-failure-causing, test cases to restrict the regions of the input domain from which subsequent test cases may be drawn. In this paper, we introduce the motivation behind RRT, explain the algorithm and detail some empirical analyses carried out to examine the effectiveness of the method. Two versions of RRT are presented: Ordinary RRT (ORRT) and Normalized RRT (NRRT). The two versions share the same fundamental algorithm, but differ in their treatment of non-homogeneous input domains. Investigations into the use of alternative exclusion shapes are outlined, and a simple technique for reducing the computational overheads of RRT, prompted by the alternative exclusion shape investigations, is also explained. The performance of RRT is compared with RT and another ART method based on maximized minimum test case separation (DART), showing excellent improvement over RT and a very favorable comparison with DART.

Adaptive random testing is a black box testing method based on the intuition that random testing failure-finding efficiency can be improved upon, in certain situations, by ensuring a more widespread and evenly distributed spread of test cases in the input domain. One way of achieving this distribution is through the use of exclusion zones and restriction, resulting in a method called restricted random testing (RRT). Recent investigations into the RRT method have revealed several interesting and significant insights. A method of reducing the computational overheads of testing methods by partitioning an input domain, and applying the method to only one of the subdomains, mapping the test cases to other subdomains, has recently been introduced. This method, called mirroring, in addition to alleviating computational costs, has some properties which fit nicely with the insights into RRT, offering solutions to some possible shortcomings of RRT. In this paper we discuss the RRT method and additional insights; we explain mirroring; and we detail applications of mirroring to RRT. The mirror RRT method proves to be a very attractive variation of RRT.