In safety-critical engineering applications, such as robust prediction against adversarial noise, it is necessary to quantify neural networks' uncertainty. Interval neural networks (INNs) are effective models for uncertainty quantification, giving an interval of predictions instead of a single value for a corresponding input. This article formulates the problem of training an INN as a chance-constrained optimization problem. The optimal solution of the formulated chance-constrained optimization naturally forms an INN that gives the tightest interval of predictions with a required confidence level. Since the chance-constrained optimization problem is intractable, a sample-based continuous approximate method is used to obtain approximate solutions to the chance-constrained optimization problem. We prove the uniform convergence of the approximation, showing that it gives the optimal INN consistently with the original ones. Additionally, we investigate the reliability of the approximation with finite samples, giving the probability bound for violation with finite samples. Through a numerical example and an application case study of anomaly detection in wind power data, we evaluate the effectiveness of the proposed INN against existing approaches, including Bayesian neural networks, highlighting its capability to significantly improve the performance of applying INNs for regression and unsupervised anomaly detection.

As an factor influencing data quality, out-of-distribution (OOD) data detection plays a critical role in AI quality assurance. This paper presents an advanced OOD detection method based on Granular Computing (GrC) and dictionary learning, specifically designed for detecting textual OOD in natural language processing (NLP) systems. First, informative data structure descriptors (information granules) are generated through GrC, which are aimed to reduce the computation overhead in big data analysis. Next, granular dictionary is constructed from these granules and used to represent original data through dictionary learning and data reconstruction. Finally, OOD detection is formalized by analyzing differences between original and reconstructed data. Finally, the proposed method formulate OOD detection via the difference between original and reconstructed data. Experiments conducted on a sentiment analysis system based on a large language model (LLM) and three OOD datasets are implemented. The constructed granular dictionary is firstly demonstrated to have good representation ability supporting effective OOD detection. Furthermore, the proposed method’s effectiveness, efficiency and scalability in textual OOD detection are validated through comprehensive comparative analysis.

Data outside the problem domain poses significant threats to the security of AI-based intelligent systems. Aiming to investigate the data domain and out-of-distribution (OOD) data in AI quality management (AIQM) study, this paper proposes to use deep learning techniques for feature representation and develop a novel statistical measure for OOD detection. First, to extract low-dimensional representative features distinguishing normal and OOD data, the proposed research combines the deep auto-encoder (AE) architecture and neuron activation status for feature engineering. Then, using local conditional probability (LCP) in data reconstruction, a novel and superior statistical measure is developed to calculate the score of OOD detection. Experiments and evaluations are conducted on image benchmark datasets and an industrial dataset. Through comparative analysis with other common statistical measures in OOD detection, the proposed research is validated as feasible and effective in OOD and AIQM studies.

Large Language Models (LLMs) have been garnering significant attention of AI researchers, especially following the widespread popularity of ChatGPT. However, due to LLMs' intricate architecture and vast parameters, several concerns and challenges regarding their quality assurance require to be addressed. In this paper, a fine-tuned GPT-based sentiment analysis model is first constructed and studied as the reference in AI quality analysis. Then, the quality analysis related to data adequacy is implemented, including employing the content-based approach to generate reasonable adversarial review comments as the wrongly-annotated data, and developing surprise adequacy (SA)-based techniques to detect these abnormal data. Experi-ments based on Amazon.com review data and a fine-tuned GPT model were implemented. Results were thoroughly discussed from the perspective of AI quality assurance to present the quality analysis of an LLM model on generated adversarial textual data and the effectiveness of using SA on anomaly detection in data quality assurance.

This article introduces a neural approximation-based method for solving continuous optimization problems with probabilistic constraints. After reformulating the probabilistic constraints as the quantile function, a sample-based neural network model is used to approximate the quantile function. The statistical guarantees of the neural approximation are discussed by showing the convergence and feasibility analysis. Then, by introducing the neural approximation, a simulated annealing-based algorithm is revised to solve the probabilistic constrained programs. An interval predictor model (IPM) of wind power is investigated to validate the proposed method.

Considering mismatched data have harmful influence on the quality assurance in sentiment analysis, therefore this paper proposed an effective method to detect these mismatches. This study considered data adequacy and model’s confidence together, and proposed to use the surprise adequacy metric for mismatch detection. Experiments were implemented on Amazon.com review data. Performances of mismatched data detection and model retraining were evaluated. The proposed method using Mahalanobis-distance-based surprise adequacy was verified feasible and effective to detect mismatched data in the studied dataset. Moreover, after mismatch detection, retrained model was illustrated useful to improve AI model’s quality.

Deep Learning (DL) based self-driving systems are vigorously developing in big companies. While, as several serious accidents were reported with life- and property-loss, the issue of robustness in DL-based self-driving systems inspires great attention, especially facing with some high-risk cases, like adversarial inputs or corner case scenarios in driving. Considering the existing methods are cumbersome in real driving, therefore this paper proposed a novel and simple way which studies data's uncertainty to rise alarm for manual checking. This method developed a metric describing corner case with respect to DL models, and subsequently evaluated data uncertainty. Experiments on a self-driving system verified the feasibility and usefulness of the proposed method. Code in this paper is released [1].

Rule-based models are applicable to model the behavior of complex and nonlinear systems. Due to limited experience and randomness involving constructing information granules, an insufficient credible rules division could reduce the model's accuracy. This paper proposes a new rule-based modeling approach, which utilizes density-based spatial clustering of applications with noise (DBSCAN)-based information granules to construct the rules. First, bear in mind the advantages of density-based clustering, DBSCAN is proposed to generate data structures. Based on these data structures, two rule-based models are constructed: 1) models using DBSCAN clusters to construct granules and rules directly and 2) models generating subgranules in each DBSCAN cluster for rule formation. Experiments involving these two models are completed, and obtained results are compared with those generated with a traditional model involving fuzzy C -means-based granules. Numerical results show that the rule-based model, which builds rules from subgranules of DBSCAN structures, performs the best in analyzing system behaviors.

The study is concerned with a description of large numeric data with the aid of building a limited collection of representative information granules with the objective of capturing the structure of the original data. The proposed development scheme consists of two steps. First, a clustering algorithm characterized by high flexibility of coping with the diverse geometry of data structure and efficient computational overhead is invoked. At the second step, a clustering algorithm applied to the clusters already formed during the first phase, yielding a collection of numeric prototypes is involved and the numeric prototypes produced there are then generalized into their granular prototypes. The quality of granular prototypes is quantified while their build-up is supported by the mechanisms of granular computing such as the principle of justifiable granularity. In this paper, the clustering algorithms of DBSCAN and fuzzy C -means were used in successive phases of the processed approach. The experimental studies concerning synthetic data and publicly available data are covered and the performance of the developed approach is assessed along with a comparative analysis.