Feature selection is a critical aspect of eXplainable Artificial Intelligence (XAI), and it has implications for model interpretability and predictive performance. CoFE (Consistency-driven Feature Elimination) framework was introduced recently using a frequentist approach. CoFE eliminates features with inconsistent coefficient signs in Linear Regression models by estimating the Sign Entropy (variability of the sign) of the coefficients using Bootstrapping. However, the uncertainty associated with estimating Sign Entropy using bootstrapping leads to slower convergence and inconsistency in feature subset selection in CoFE. In this paper, we present Bay-CoFE, 1 a Bayesian reformulation of CoFE, to solve the slow convergence and inconsistency issues of CoFE while retaining the benefits of selecting features with lower Sign Entropy in a Bayesian framework. We provide theoretical justifications and empirical evidence to prove Bay-CoFE’s superior convergence properties. Across all datasets, Bay-CoFE achieves significantly superior sign stability compared to traditional feature selection methods (Mann-Whitney U test, p-value < = 1.63e-03 and mean Cliff’s delta ≈ 0.83), with minimal predictive performance differences (Mann-Whitney U test, p-value > 0.1 and mean Cliff’s delta ≈ 0.32), demonstrating a highly favorable trade-off for interpretable modeling.

Saliency maps are widely used as a post-hoc approach to explain the decision-making process of Deep Learning (DL) based image classification models, but evaluating their fidelity remains a complex problem. While saliency metrics have been introduced to evaluate the fidelity of saliency maps, existing saliency metrics, such as perturbation-based saliency metrics, have been previously reported to demonstrate statistical inconsistency. Although inconsistencies have been noted in different works, there exists no mechanism for estimating the same, i.e., Inconsistency Estimation (IE). Our primary objective is to address this limitation, and therefore, we propose a framework to estimate the inconsistency of saliency metrics for any given DL model. The framework enables building IE models for estimating the inconsistency by employing a set of perturbation types and schemes. The framework’s modular architecture provides flexibility across (i) perturbation types (Inpainting, Uniform, and Gaussian blur), (ii) perturbation schemes (pixel-wise and patch-wise), (iii) learning mechanisms (Convolutional Neural Networks and Vision Transformers) and (iv) IE modeling techniques (bagging and boosting). Extensive experimental results are shown on three well-known DL architectures (Inception-V3, Xception, and ResNet-50) on three different public datasets, including the Imagenette, Oxford-IIIT Pets Dataset, and PASCAL VOC 2007, along with results on ViTs for Oxford-IIIT Pets Dataset, and PASCAL VOC 2007. With a comprehensive evaluation of seven different perturbation types that include two inpainting, two Gaussian blur (with kernel widths of 0.9 and 1.5), and three uniform perturbations, our work shows the effectiveness of the proposed approach in estimating inconsistency. Statistically founded tests such as repeated cross-validation and the Permutation Test further validate the idea of the proposed framework for estimating the inconsistency of saliency metrics across unseen perturbations, making it useful in real-world scenarios.

Frequency shortcuts refer to specific frequency patterns that models heavily rely on for correct classification. Previous studies have shown that models trained on small image datasets often exploit such shortcuts, potentially impairing their generalization performance. However, existing methods for identifying frequency shortcuts require expensive computations and become impractical for analyzing models trained on large datasets. In this work, we propose the first approach to more efficiently analyze frequency shortcuts at a large scale. We show that both CNN and transformer models learn frequency shortcuts on ImageNet. We also expose that frequency shortcut solutions can yield good performance on out-of-distribution (OOD) test sets which largely retain texture information. However, these shortcuts, mostly aligned with texture patterns, hinder model generalization on rendition-based OOD test sets. These observations suggest that current OOD evaluations often overlook the impact of frequency shortcuts on model generalization. Future benchmarks could thus benefit from explicitly assessing and accounting for these shortcuts to build models that generalize across a broader range of OOD scenarios. Codes are available at https://github.com/nis-research/hfss.

Explanations of Local Interpretable Model-agnostic Explanations (LIME) are often inconsistent across different runs making them unreliable for eXplainable AI (XAI). The inconsistency stems from sign flips and variability in ranks of the segments for each different run. We propose a Bayesian Regularization approach to reduce sign flips, which in turn stabilizes feature rankings and ensures significantly higher consistency in explanations. The proposed approach enforces sparsity by incorporating a Sign Entropy prior on the coefficient distribution and dynamically eliminates features during optimization. Our results demonstrate that the explanations from the proposed method exhibit significantly better consistency and fidelity than LIME (and its earlier variants). Further, our approach exhibits comparable consistency and fidelity with a significantly lower execution time than the latest LIME variant, i.e., SLICE (CVPR 2024).

ABSTRACT We propose a tool that can replace one or more specific facial components in images for face analysis. The tool can replace the texture and shape of facial components such as eyes, nose, and mouth. The source and destination of the components can be real faces or an average face computed from a dataset. A seamless method is applied to smooth the component boundaries after replacement. The tool is developed using the Python language and is available in open source and online, with a web interface. We also provide a desktop version that can manage multiple files or a dataset as input. The tool can, for instance, be used to investigate the contribution of face components to face recognition, face perception analysis, the change of identity, and fun applications. Some illustrative examples are provided.

Local Interpretable Model-agnostic Explanations (LIME) - a widely used post-ad-hoc model agnostic ex-plainable AI (XAI) technique. It works by training a simple transparent (surrogate) model using random samples drawn around the neighborhood of the instance (image) to be explained (IE). Explanations are then extracted for a black-box model and a given IE, using the surrogate model. However, the explanations of LIME suffer from inconsistency across different runs for the same model and the same IE. We identify two main types of inconsistencies: variance in the sign and importance ranks of the segments (superpixels). These factors hinder LIME from obtaining consistent explanations. We analyze these inconsistencies and propose a new method, Stabilized LIME for Consistent Explanations (SLICE). The proposed method handles the stabilization problem in two aspects: using a novel feature selection technique to eliminate spurious superpixels and an adaptive perturbation technique to generate perturbed images in the neighborhood of IE. Our results demonstrate that the explanations from SLICE exhibit significantly better consistency and fidelity than LIME (and its variant BayLime).

Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection. Leveraging the dynamic sparse training (DST) algorithms within SNNs has demonstrated promising feature selection capabilities while drastically reducing computational overheads. Despite these advancements, several critical aspects remain insufficiently explored for feature selection. Questions persist regarding the choice of the DST algorithm for network training, the choice of metric for ranking features/neurons, and the comparative performance of these methods across diverse datasets when compared to dense networks. This paper addresses these gaps by presenting a comprehensive systematic analysis of feature selection with sparse neural networks. Moreover, we introduce a novel metric considering sparse neural network characteristics, which is designed to quantify feature importance within the context of SNNs. Our findings show that feature selection with SNNs trained with DST algorithms can achieve, on average, more than 50% memory and 55% FLOPs reduction compared to the dense networks, while outperforming them in terms of the quality of the selected features. Our code and the supplementary material are available on GitHub (https://github.com/zahraatashgahi/Neuron-Attribution).

Recent advancements in deep learning for face recognition have led to concerns regarding privacy and algorithmic bias, particularly in inferring demographic attributes from facial templates. Existing methods often struggle to balance privacy preservation with utility and operational efficiency. In this paper, we propose a filter-based privacy-enhancing method inspired by the information bottleneck concept. Our approach involves training a filter estimator that assigns scores to intermediate-layer elements based on their sensitivity to target attributes. By selectively replacing sensitive elements with noise while allowing less sensitive ones to pass through, our method aims to enhance privacy while managing the trade-off with verification performance. Most importantly, our approach allows for post-training tuning of the privacy-utility trade-off, providing flexibility for different operational requirements. Evaluation across multiple face recognition networks and datasets demonstrates that our approach can achieve substantial gains in the gender obfuscation task while maintaining adequate verification performance and computational efficiency suitable for real-time applications.

Partial face recognition (PFR) is challenging as the appearance of the face changes significantly with occlusion. In particular, these occlusions can be due to any item and may appear in any position that seriously hinders the extraction of discriminative features. Existing methods deal with PFR either by training a deep model with existing face databases containing limited occlusion types or by extracting un-occluded features directly from face regions without occlusions. Limited training data (i.e., occlusion type and diversity) can not cover the real-occlusion situations, and thus training-based methods can not learn occlusion robust discriminative features. The performance of occlusion region-based method is bounded by occlusion detection. Different from limited training data and occlusion region-based methods, we propose to use multi-label attributes for Partial Face Recognition (Attr4PFR). A novel data augmentation is proposed to solve limited training data and generate occlusion attributes. Apart from occlusion attributes, we also include soft biometric attributes and semantic attributes to explore more rich attributes to combat the loss caused by occlusions. To train our Attr4PFR, we propose an implicit attributes loss combined with a softmax loss to enforce Attr4PFR to learn discriminative features. As multi-label attributes are our auxiliary signal in the training phase, we do not need them in the inference. Extensive experiments on public benchmark AR and IJB-C databases show our method is 3% and 2.3% improvement compared to the state-of-the-art.

Abstract Face shape priors such as landmarks, heatmaps, and parsing maps are widely used to improve face super resolution (SR). It is observed that face priors provide locations of high‐frequency details in key facial areas such as the eyes and mouth. However, existing methods fail to effectively exploit the high‐frequency information by using the priors as either constraints or inputs. This paper proposes a novel high frequency highway () framework to better utilize prior information for face SR, which dynamically decomposes the final SR face into a coarse SR face and a high frequency (HF) face. The coarse SR face is reconstructed from a low‐resolution face via a texture branch, using only pixel‐wise reconstruction loss. Meanwhile, the HF face is directly generated from face priors via an HF branch that employs the proposed inception–hourglass model. As a result, allows the face priors to have a direct impact on the SR face by adding the outputs of both branches as the final result and provides an extra face editing function. Extensive experiments show that significantly outperforms state‐of‐the‐art face SR methods, is general for different texture branch models and face priors, and is robust to dataset mismatch and pose variations.

Face inpainting, the technique of restoring missing or damaged regions in facial images, is pivotal for applications like face recognition in occluded scenarios and image analysis with poor-quality captures. This process not only needs to produce realistic visuals but also preserve individual identity characteristics. The aim of this paper is to inpaint a face given periocular region (eyes-to-face) through a proposed new Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net). The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders have been used. The extracted features are then mapped to the latent space of a pre-trained StyleGAN generator to benefit from its state-of-the-art performance and its rich, diverse and expressive latent space without any additional training. We further improve the StyleGAN's output to find the optimal code in the latent space using a new optimization for GAN inversion technique. Our E2F-Net requires a minimum training process reducing the computational complexity as a secondary benefit. Through extensive experiments, we show that our method successfully reconstructs the whole face with high quality, surpassing current techniques, despite significantly less training and supervision efforts. We have generated seven eyes-to-face datasets based on well-known public face datasets for training and verifying our proposed methods. The code and datasets are publicly available1.

The remarkable success of face recognition (FR) has endangered the privacy of internet users particularly in social media. Recently, researchers turned to use adversarial examples as a countermeasure to privacy attacks. In this paper, we assess the effectiveness of using two widely known adversarial methods (BIM and ILLC) for de-identifying personal images. We discovered, unlike previous claims in the literature, that it is not easy to get a high protection success rate (suppressing identification rate) with imperceptible adversarial perturbation to the human visual system. Finally, we found out that the transferability of adversarial examples is highly affected by the training parameters of the network with which they are generated

Privacy-preserving biometric template protection schemes (BTPs) preserve biometric data by hiding biometric representations via a privacy-preserving mechanism (such as homomorphic encryption) and comparing the protected templates while conserving the recognition scores as in an embedding space. However, it is often tolerated to reveal these scores after performing a biometric comparison to gain efficiency and perform the score comparison directly on cleartext data. Through this work, we demonstrate that this cleartext score tolerance can lead to privacy breaches and bypass recognition systems, threatening those BTPs in the case of inner product-based facial template comparisons. We propose a template recovery attack that requires no training and a few random fake templates with their corresponding scores, from which we are able to recover the unprotected target template using the Lagrange multiplier optimization method. We evaluate our attack by verifying whether the recovered template is deemed similar to the target template held by recognition systems set to accept 0.1%, 0.01%, and 0.001% FMR. We estimate that between 60 to 165 revealed scores and fake templates can lead to a template recovery with a 100% success rate. We analyzed the impact of recovered templates by measuring the amount of gender information they contain, as well as their resemblance to the reconstructed images of their target templates.

Accurate face recognition systems are increasingly important in sensitive applications like border control or migration management. Therefore, it becomes crucial to quantify the quality of facial images to ensure that lowquality images are not affecting recognition accuracy. In this context, the current draft of ISO/IEC 29794-5 introduces the concept of component quality to estimate how single factors of variation affect recognition outcomes. In this study, we propose a quality measure (NeutrEx) based on the accumulated distances of a 3D face reconstruction to a neutral expression anchor. Our evaluations demonstrate the superiority of our proposed method compared to baseline approaches obtained by training Support Vector Machines on face embeddings extracted from a pre-trained Convolutional Neural Network for facial expression classification. Furthermore, we highlight the explainable nature of our NeutrEx measures by computing per-vertex distances to unveil the most impactful face regions and allow operators to give actionable feedback to subjects1.

Frequency analysis is useful for understanding the mechanisms of representation learning in neural networks (NNs). Most research in this area focuses on the learning dynamics of NNs for regression tasks, while little for classification. This study empirically investigates the latter and expands the understanding of frequency shortcuts. First, we perform experiments on synthetic datasets, designed to have a bias in different frequency bands. Our results demonstrate that NNs tend to find simple solutions for classification, and what they learn first during training depends on the most distinctive frequency characteristics, which can be either low- or high-frequencies. Second, we confirm this phenomenon on natural images. We propose a metric to measure class-wise frequency characteristics and a method to identify frequency shortcuts. The results show that frequency shortcuts can be texture-based or shape-based, depending on what best simplifies the objective. Third, we validate the transferability of frequency shortcuts on out-of-distribution (OOD) test sets. Our results suggest that frequency shortcuts can be transferred across datasets and cannot be fully avoided by larger model capacity and data augmentation. We recommend that future research should focus on effective training schemes mitigating frequency shortcut learning. Codes and data are available at https://github.com/nis-research/nn-frequency-shortcuts.

The cutting-edge biometric recognition systems extract distinctive feature vectors of biometric samples using deep neural networks to measure the amount of (dis-)similarity between two biometric samples. Studies have shown that personal information (e.g., health condition, ethnicity, etc.) can be inferred, and biometric samples can be reconstructed from those feature vectors, making their protection an urgent necessity. State-of-the-art biometrics protection solutions are based on homomorphic encryption (HE) to perform recognition over encrypted feature vectors, hiding the features and their processing while releasing the outcome only. However, this comes at the cost of those solutions' efficiency due to the inefficiency of HE-based solutions with a large number of multiplications; for (dis-)similarity measures, this number is proportional to the vector's dimension. In this paper, we tackle the HE performance bottleneck by freeing the two common (dis-)similarity measures, the cosine similarity and the squared Euclidean distance, from multiplications. Assuming normalized feature vectors, our approach pre-computes and organizes those (dis-)similarity measures into lookup tables. This transforms their computation into simple table-lookups and summation only. We study quantization parameters for the values in the lookup tables and evaluate performances on both synthetic and facial feature vectors for which we achieve a recognition performance identical to the non-tabularized baseline systems. We then assess their efficiency under HE and record runtimes between 28.95ms and 59.35ms for the three security levels, demonstrating their enhanced speed.

We explore a new method to hide identity information in a facial image from face recognition (FR) systems, while only minimally changing the appearance of the image as perceived by humans. We train a decoder network that reverses the mapping of an FR system and use the dissimilarity score function of this FR system to teach the decoder to return images with as little identity information as possible, while using a visual loss to change the image as little as possible visually. We show that these obfuscation attacks are also successful when the FR system is unknown. We analyse the obfuscated images in latent space and show that our approach as well as an existing method can be easily circumvented by applying the same obfuscation method to the enrolled faces as to the probe images. We suggest an adaptation that can help prevent this circumvention.

Abstract Presentation attack detection (PAD) is used to mitigate the dangers of the weakest link problem in biometric recognition, in which failure modes of one application affect the security of all other applications. Strong PAD methods are therefore a must, and we believe biometric challenge-response protocols (BCRP) form an underestimated part of this ecosystem. In this paper, we conceptualize what BCRPs are, and we propose a descriptive formalism and categorization for working with them. We validate the categorization against existing literature that we classified to be describing BCRPs. Lastly, we discuss how strong BCRPs provide advantages over PAD methods, specifically in the protection of individual applications and the protection of other applications from inadvertent leaks in BCRP applications. We note that research in BCRPs is fragmented, and our intent for the proposed formalism and categorization are to give focus and direction to research efforts into biometric challenge-response protocols.

Abstract In this paper, new insights in the near infrared imaging process used in finger‐vein recognition by developing a physical model are presented. A realistic phantom finger that mimics the living human finger and also includes veins has been developed to validate this model. NIR phantom finger images show that the phantom can emulate the optical properties of a living human finger and can provide ground truth for the locations of the veins. Through physical modeling, it is particularly learned that—besides blood and soft tissue—bone also plays an important role in generating reliable NIR finger‐vein images.

Abstract Sparse neural networks attract increasing interest as they exhibit comparable performance to their dense counterparts while being computationally efficient. Pruning the dense neural networks is among the most widely used methods to obtain a sparse neural network. Driven by the high training cost of such methods that can be unaffordable for a low-resource device, training sparse neural networks sparsely from scratch has recently gained attention. However, existing sparse training algorithms suffer from various issues, including poor performance in high sparsity scenarios, computing dense gradient information during training, or pure random topology search. In this paper, inspired by the evolution of the biological brain and the Hebbian learning theory, we present a new sparse training approach that evolves sparse neural networks according to the behavior of neurons in the network. Concretely, by exploiting the cosine similarity metric to measure the importance of the connections, our proposed method, “Cosine similarity-based and random topology exploration (CTRE)”, evolves the topology of sparse neural networks by adding the most important connections to the network without calculating dense gradient in the backward. We carried out different experiments on eight datasets, including tabular, image, and text datasets, and demonstrate that our proposed method outperforms several state-of-the-art sparse training algorithms in extremely sparse neural networks by a large gap. The implementation code is available on Github.

Abstract Major complications arise from the recent increase in the amount of high-dimensional data, including high computational costs and memory requirements. Feature selection, which identifies the most relevant and informative attributes of a dataset, has been introduced as a solution to this problem. Most of the existing feature selection methods are computationally inefficient; inefficient algorithms lead to high energy consumption, which is not desirable for devices with limited computational and energy resources. In this paper, a novel and flexible method for unsupervised feature selection is proposed. This method, named QuickSelection (The code is available at: https://github.com/zahraatashgahi/QuickSelection), introduces the strength of the neuron in sparse neural networks as a criterion to measure the feature importance. This criterion, blended with sparsely connected denoising autoencoders trained with the sparse evolutionary training procedure, derives the importance of all input features simultaneously. We implement QuickSelection in a purely sparse manner as opposed to the typical approach of using a binary mask over connections to simulate sparsity. It results in a considerable speed increase and memory reduction. When tested on several benchmark datasets, including five low-dimensional and three high-dimensional datasets, the proposed method is able to achieve the best trade-off of classification and clustering accuracy, running time, and maximum memory usage, among widely used approaches for feature selection. Besides, our proposed method requires the least amount of energy among the state-of-the-art autoencoder-based feature selection methods.

Biometric verification has been widely deployed in current authentication solutions as it proves the physical presence of individuals. Several solutions have been developed to protect the sensitive biometric data in such systems that provide security against honest-but-curious (a.k.a. semi-honest) attackers. However, in practice, attackers typically do not act honestly and multiple studies have shown severe biometric information leakage in such honest-but-curious solutions when considering dishonest, malicious attackers. In this paper, we propose a provably secure biometric verification protocol to withstand malicious attackers and prevent biometric data from any leakage. The proposed protocol is based on a homomorphically encrypted log likelihood-ratio (HELR) classifier that supports any biometric modality (e.g., face, fingerprint, dynamic signature, etc.) encoded as a fixed-length real-valued feature vector. The HELR classifier performs an accurate and fast biometric recognition. Furthermore, our protocol, which is secure against malicious adversaries, is designed from a protocol secure against semi-honest adversaries enhanced by zero-knowledge proofs. We evaluate both protocols for various security levels and record a sub-second speed (between 0.37s and 0.88s) for the protocol secure against semi-honest adversaries and between 0.95s and 2.50s for the protocol secure against malicious adversaries.

Morphing attacks have posed a severe threat to Face Recognition System (FRS). Despite the number of advancements reported in recent works, we note serious open issues such as independent benchmarking, generalizability challenges and considerations to age, gender, ethnicity that are inadequately addressed. Morphing Attack Detection (MAD) algorithms often are prone to generalization challenges as they are database dependent. The existing databases, mostly of semi-public nature, lack in diversity in terms of ethnicity, various morphing process and post-processing pipelines. Further, they do not reflect a realistic operational scenario for Automated Border Control (ABC) and do not provide a basis to test MAD on unseen data, in order to benchmark the robustness of algorithms. In this work, we present a new sequestered dataset for facilitating the advancements of MAD where the algorithms can be tested on unseen data in an effort to better generalize. The newly constructed dataset consists of facial images from 150 subjects from various ethnicities, age-groups and both genders. In order to challenge the existing MAD algorithms, the morphed images are with careful subject pre-selection created from the contributing images, and further post-processed to remove morphing artifacts. The images are also printed and scanned to remove all digital cues and to simulate a realistic challenge for MAD algorithms. Further, we present a new online evaluation platform to test algorithms on sequestered data. With the platform we can benchmark the morph detection performance and study the generalization ability. This work also presents a detailed analysis on various subsets of sequestered data and outlines open challenges for future directions in MAD research.

Along with the deployment of the Face Recognition Systems (FRS), concerns were raised related to the vulnerability of those systems towards various attacks including morphed attacks. The morphed face attack involves two different face images in order to obtain via a morphing process a resulting attack image, which is sufficiently similar to both contributing data subjects. The obtained morphed image can successfully be verified against both subjects visually (by a human expert) and by a commercial FRS. The face morphing attack poses a severe security risk to the e-passport issuance process and to applications like border control, unless such attacks are detected and mitigated. In this work, we propose a new method to reliably detect a morphed face attack using a newly designed demising framework. To this end, we design and introduce a new deep Multi-scale Context Aggregation Network (MS-CAN) to obtain denoised images, which is subsequently used to determine if an image is morphed or not. Extensive experiments are carried out on three different morphed face image datasets. The Morphing Attack Detection (MAD) performance of the proposed method is also benchmarked against 14 different state-of-the-art techniques using the ISO-IEC 30107-3 evaluation metrics. Based on the obtained quantitative results, the proposed method has indicated the best performance on all three datasets and also on cross-dataset experiments.

Accurate pulmonary nodule detection is a crucial step in lung cancer screening. Computer-aided detection (CAD) systems are not routinely used by radiologists for pulmonary nodule detection in clinical practice despite their potential benefits. Maximum intensity projection (MIP) images improve the detection of pulmonary nodules in radiological evaluation with computed tomography (CT) scans. Inspired by the clinical methodology of radiologists, we aim to explore the feasibility of applying MIP images to improve the effectiveness of automatic lung nodule detection using convolutional neural networks (CNNs). We propose a CNN-based approach that takes MIP images of different slab thicknesses (5 mm, 10 mm, 15 mm) and 1 mm axial section slices as input. Such an approach augments the two-dimensional (2-D) CT slice images with more representative spatial information that helps discriminate nodules from vessels through their morphologies. Our proposed method achieves sensitivity of 92.7% with 1 false positive per scan and sensitivity of 94.2% with 2 false positives per scan for lung nodule detection on 888 scans in the LIDC-IDRI dataset. The use of thick MIP images helps the detection of small pulmonary nodules (3 mm-10 mm) and results in fewer false positives. Experimental results show that utilizing MIP images can increase the sensitivity and lower the number of false positives, which demonstrates the effectiveness and significance of the proposed MIP-based CNNs framework for automatic pulmonary nodule detection in CT scans. The proposed method also shows the potential that CNNs could gain benefits for nodule detection by combining the clinical procedure.

Application‐specific data for certain biometric applications are often not sufficiently available. The authors present a solution for face recognition with limited application‐specific data. Existing methods often use a classifier with convolutional neural networks (CNNs) as feature extractors. The CNNs are trained with massive general (i.e. not application specific) data and the classifier is trained with application‐specific data. Alternatively, the authors propose a combined training strategy to train the classifier on a balanced mixture of general and application‐specific data, such that the recognition performance is maximised. The proposed method largely alleviates the needs for application‐specific data. To prove its effectiveness, they apply the proposed method to low‐resolution face recognition. Specifically, they use the heterogeneous joint Bayesian (HJB) classifier that is capable of comparing features from the same modality but with different characteristics. To further boost performance, the authors augment the training data by pre‐processing it to resemble application‐specific data. They conducted extensive experiments on challenging datasets, namely, SCface and COX. The results show that the proposed method improves the true match rate on SCface at a false match rate of 10% by ∼11% and the true match rate on COX at a false match rate of 1% by ∼12%.

Facial marks have been studied before, either as a complement to face recognition systems or for their suitability as a single biometric modality. In this paper, we use a subset of the FRGCv2 data set (12307 images and 568 subjects) to study the properties of facial marks, their spatial patterns, and classifiers acting upon these patterns. We observe differences between age and ethnic groups in the number of facial marks. Also, facial marks tend to be clustered. We present six forensically relevant aspects with respect to the design and evaluation of classifiers. These aspects help to systematically study factors that influence performance characteristics (discriminating power and calibration loss) of these classifiers. Calibration loss is of particular forensic importance; it essentially measures how well the classifier output can be used as strength of evidence in a court of law. We use various facial mark grids to which the facial mark spatial patterns are assigned. We find that a classifier that utilizes the facial mark grid of a specific subject outperforms all other classifiers. We also observe that the calibration loss of such subject-based classifier indicates that small grid cell sizes should be avoided.

Law enforcement agencies around the world use ridge-based biometrics, especially fingerprints, to fight crime. Fingermarks that are left at a crime scene and identified as potentially having evidential value (EV) in a court of law are recorded for further forensic analysis. Here, we test our evidential value algorithm (EVA) which uses image features trained on forensic expert decisions for 1428 fingermarks to produce an EV score for an image. First, we study the relationship between whether a fingermark is assessed as having EV, either by a human expert or by EVA, and its correct and confident identification by an automatic identification system. In particular, how often does an automatic system achieve identification when the mark is assessed as not having evidential value? We show that when the marks are captured by a mobile phone, correct and confident automatic matching occurs for 257 of the 1428. Of these, 236 were marked as having sufficient EV by experts and 242 by EVA thresholded on equal error rate. Second, we test four relatively challenging ridge-based biometric databases and show that EVA can be successfully applied to give an EV score to all images. Using EV score as an image quality value, we show that in all databases, thresholding on EV improves performance in closed set identification. Our results suggest an EVA application that filters fingermarks meeting a minimum EV score could aid forensic experts at the point of collection, or by flagging difficult latents objectively, or by pre-filtering specimens before submission to an AFIS.

Quality of a pair of facial images is a strong indicator of the uncertainty in decision about identity based on that image pair. In this paper, we describe a Bayesian approach to model the relation between image quality (like pose, illumination, noise, sharpness, etc) and corresponding face recognition performance. Experiment results based on the MultiPIE data set show that our model can accurately aggregate verification samples into groups for which the verification performance varies fairly consistently. Our model does not require similarity scores and can predict face recognition performance using only image quality information. Such a model has many applications. As an illustrative application, we show improved verification performance when the decision threshold automatically adapts according to the quality of facial images.

The increase of the dimensionality of data sets often leads to problems during estimation, which are denoted as the curse of dimensionality. One of the problems of second-order statistics (SOS) estimation in high-dimensional data is that the resulting covariance matrices are not full rank, so their inversion, for example, needed in verification systems based on the likelihood ratio, is an ill-posed problem, known as the singularity problem. A classical solution to this problem is the projection of the data onto a lower dimensional subspace using principle component analysis (PCA) and it is assumed that any further estimation on this dimension-reduced data is free from the effects of the high dimensionality. Using theory on SOS estimation in high-dimensional spaces, we show that the solution with PCA is far from optimal in verification systems if the high dimensionality is the sole source of error. For moderate dimensionality, it is already outperformed by solutions based on euclidean distances and it breaks down completely if the dimensionality becomes very high. We propose a new method, the fixed-point eigenwise correction, which does not have these disadvantages and performs close to optimal.

This paper presents a novel method of fusing multiple biometrics on the matching score level. We estimate the likelihood ratios of the fused biometric scores, via individual receiver operating characteristics (ROC) which construct the Naive Bayes classifier. Using a limited number of operation points on the ROC, we are able to realize reliable and robust estimation of the Naive Bayes probability without explicit estimation of the genuine and impostor score distributions. Different from previous work, the method takes into consideration a particular characteristic of the matching score: its quantitative value is already an indication of the sample's likelihood of being genuine. This characteristic is integrated into the proposed method to improve the fusion performance while reducing the inherent algorithmic complexity. We demonstrate by experiments that the proposed method is reliable and robust, suitable for a wide range of matching score distributions in realistic data and public databases.

Cryptography may mitigate the privacy problem in biometric recognition systems. However, cryptography technologies lack error-tolerance and biometric samples cannot be reproduced exactly, rising the robustness problem. The biometric template protection system needs a good feature extraction algorithm to be a good classifier. But, an even effective feature extractor can give a very low-quality biometric channel (i.e. high Bit Error Rate (BER)). Using the Spectral Minutiae method to identify fingerprints is one of the examples, which gives a BER of 40 ~ 50% to most of the matching channels. Therefore, we propose a concatenated coding scheme based on erasure codes to achieve a robust and secure biometric recognition system. The key idea is to transmit more packets than needed for decoding and allow the erasure-encoded packet suffering high BER to be discarded. The erasure decoder can reconstruct the secret key by collecting enough surviving packets. By applying the spectral minutiae method in the FVC2000-DB2 fingerprint database, the unprotected system achieves an EER of 3.7% and our proposed coding scheme reaches an EER of 4.6% with a 798-bit secret key.

Template protection techniques are used within biometric systems in order to protect the stored biometric template against privacy and security threats. A great portion of template protection techniques are based on extracting a key from, or binding a key to the binary vector derived from the biometric sample. The size of the key plays an important role, as the achieved privacy and security mainly depend on the entropy of the key. In the literature, it can be observed that there is a large variation on the reported key lengths at similar classification performance of the same template protection system, even when based on the same biometric modality and database. In this work, we determine the analytical relationship between the classification performance of the fuzzy commitment scheme and the theoretical maximum key size given as input a Gaussian biometric source. We show the effect of the system parameters such as the biometric source capacity, the number of feature components, the number of enrolment and verification samples, and the target performance on the maximum key size. Furthermore, we provide an analysis of the effect of feature interdependencies on the estimated maximum key size and classification performance. Both the theoretical analysis, as well as an experimental evaluation using the MCYT fingerprint database showed that feature interdependencies have a large impact on performance and key size estimates. This property can explain the large deviation in reported key sizes in literature.

Spectral Minutiae (SM) representation enables the combination of minutiae-based fingerprint recognition systems with template protection schemes based on fuzzy commitment, but it requires error-correcting codes that can handle high bit error rates (i.e. above 40%). In this paper, we propose a 3-Layer coding scheme based on erasure codes for the SM-based biometric recognition system. Our approach is inspired by the fact that the Packet Error Rate (PER) is proportional to the Bit Error Rate (BER). Each packet is encoded by an Error Detection Code (EDC) and an Error Correction Code (ECC). The packet can only survive if it successfully passes the ECC and EDC decoder. With the erasure code, the system can reconstruct the secret key by only using the survived packets. By applying SM to the FVC2000-DB2 fingerprint database, the unprotected system achieves an EER of around 6% while our proposed coding scheme reaches an EER of approximately 6.5% with a 1032-bit secret key.

Fuzzy commitment is an efficient template protection algorithm that can improve security and safeguard privacy of biometrics. Existing theoretical security analysis has proved that although privacy leakage is unavoidable, perfect security from information-theoretical points of view is possible when bits extracted from biometric features are uniformly and independently distributed. Unfortunately, this strict condition is difficult to fulfill in practice. In many applications, dependency of binary features is ignored and security is thus suspected to be highly overestimated. This paper gives a comprehensive analysis on security and privacy of fuzzy commitment regarding empirical evaluation. The criteria representing requirements in practical applications are investigated and measured quantitatively in an existing protection system for 3D face recognition. The evaluation results show that a very significant reduction of security and enlargement of privacy leakage occur due to the dependency of biometric features. This work shows that in practice, one has to explicitly measure the security and privacy instead of trusting results under non-realistic assumptions.

Extracting binary strings from real-valued biometric templates is a fundamental step in template compression and protection systems, such as fuzzy commitment, fuzzy extractor, secure sketch, and helper data systems. Quantization and coding is the straightforward way to extract binary representations from arbitrary real-valued biometric modalities. In this paper, we propose a pairwise adaptive phase quantization (APQ) method, together with a long-short (LS) pairing strategy, which aims to maximize the overall detection rate. Experimental results on the FVC2000 fingerprint and the FRGC face database show reasonably good verification performances.

Template protection techniques are used within biometric systems in order to safeguard the privacy of the system's subjects. This protection also includes unlinkability, i.e., preventing cross-matching between two or more reference templates from the same subject across different applications. In the literature, the template protection techniques based on fuzzy commitment, also known as the code-offset construction, have recently been investigated. Recent work presented the decodability attack vulnerability facilitating cross-matching based on the protected templates and its theoretical analysis. First, we extend the theoretical analysis and include the comparison between the system and cross-matching performance. We validate the presented analysis using real biometric data from the MCYT fingerprint database. Second, we show that applying a random bit-permutation process secures the fuzzy commitment scheme from cross-matching based on the decodability attack.

Verification decisions are often based on second order statistics estimated from a set of samples. Ongoing growth of computational resources allows for considering more and more features, increasing the dimensionality of the samples. If the dimensionality is of the same order as the number of samples used in the estimation or even higher, then the accuracy of the estimate decreases significantly. In particular, the eigenvalues of the covariance matrix are estimated with a bias and the estimate of the eigenvectors differ considerably from the real eigenvectors. We show how a classical approach of verification in high dimensions is severely affected by these problems, and we show how bias correction methods can reduce these problems.

A fixed-length binary representation of a fingerprint has the advantages of a fast operation and a small template storage. For many biometric template protection schemes, a binary string is also required as input. The spectral minutiae representation is a method to represent a minutiae set as a fixed-length real-valued feature vector. In order to be able to apply the spectral minutiae representation with a template protection scheme, we introduce two novel methods to quantize the spectral minutiae features into binary strings: Spectral Bits and Phase Bits. The experiments on the FVC2002 database show that the binary representations can even outperformed the spectral minutiae real-valued features.

In recent years, the protection of biometric data has gained increased interest from the scientific community. Methods such as the fuzzy commitment scheme, helper-data system, fuzzy extractors, fuzzy vault, and cancelable biometrics have been proposed for protecting biometric data. Most of these methods use cryptographic primitives or error-correcting codes (ECCs) and use a binary representation of the real-valued biometric data. Hence, the difference between two biometric samples is given by the Hamming distance (HD) or bit errors between the binary vectors obtained from the enrollment and verification phases, respectively. If the HD is smaller (larger) than the decision threshold, then the subject is accepted (rejected) as genuine. Because of the use of ECCs, this decision threshold is limited to the maximum error-correcting capacity of the code, consequently limiting the false rejection rate (FRR) and false acceptance rate tradeoff. A method to improve the FRR consists of using multiple biometric samples in either the enrollment or verification phase. The noise is suppressed, hence reducing the number of bit errors and decreasing the HD. In practice, the number of samples is empirically chosen without fully considering its fundamental impact. In this paper, we present a Gaussian analytical framework for estimating the performance of a binary biometric system given the number of samples being used in the enrollment and the verification phase. The error-detection tradeoff curve that combines the false acceptance and false rejection rates is estimated to assess the system performance. The analytic expressions are validated using the Face Recognition Grand Challenge v2 and Fingerprint Verification Competition 2000 biometric databases.

Eigenvalue analysis is an important aspect in many data modeling methods. Unfortunately, the eigenvalues of the sample covariance matrix (sample eigenvalues) are biased estimates of the eigenvalues of the covariance matrix of the data generating process (population eigenvalues). We present a new method based on bootstrapping to reduce the bias in the sample eigenvalues: the eigenvalue estimates are updated in several iterations, where in each iteration synthetic data is generated to determine how to update the population eigenvalue estimates. Comparison of the bootstrap eigenvalue correction with a state of the art correction method by Karoui shows that depending on the type of population eigenvalue distribution, sometimes the Karoui method performs better and sometimes our bootstrap method.

Most fingerprint recognition systems are based on the use of a minutiae set, which is an unordered collection of minutiae locations and orientations suffering from various deformations such as translation, rotation, and scaling. The spectral minutiae representation introduced in this paper is a novel method to represent a minutiae set as a fixed-length feature vector, which is invariant to translation, and in which rotation and scaling become translations, so that they can be easily compensated for. These characteristics enable the combination of fingerprint recognition systems with template protection schemes that require a fixed-length feature vector. This paper introduces the concept of algorithms for two representation methods: the location-based spectral minutiae representation and the orientation-based spectral minutiae representation. Both algorithms are evaluated using two correlation-based spectral minutiae matching algorithms. We present the performance of our algorithms on three fingerprint databases. We also show how the performance can be improved by using a fusion scheme and singular points.

We propose a novel method to correct for arbitrary illumination variation in the face images. The main purpose is to improve recognition results of face images taken under uncontrolled illumination conditions. We correct the illumination variation in the face images using a face shape model, which allows us to estimate the face shape in the face image. Using this face shape, we can reconstruct a face image under frontal illumination. These reconstructed images improve the results in face identification. We experimented both with face images acquired under different controlled illumination conditions in a laboratory and under uncontrolled illumination conditions.

The classical likelihood ratio classifier easily collapses in many biometric applications especially with independent training-test subjects. The reason lies in the inaccurate estimation of the underlying user-specific feature density. Firstly, the feature density estimation suffers from insufficient number of user-specific samples during the enrollment phase. Even if more enrollment samples are available, it is most likely that they are not reliable enough. Furthermore, it may happen that enrolled samples do not obey the Gaussian density model. Therefore, it is crucial to properly estimate the underlying user-specific feature density in the above situations. In this paper, we give an overview of several data modeling methods. Furthermore, we propose a discretized density based data model. Experimental results on FRGC face data set has shown reasonably good performance with our proposed model.

We show that there is a direct relation between the maximum length of the keys extracted from biometric data and the error rates of the biometric system. The length of the bio-key depends on the amount of information that can be extracted from the source data. This information can be used a-priori to evaluate the potential of the biometric data in the context of a specific cryptographic application. We model the biometric data more naturally as a continuous distribution and we give a new definition for fuzzy extractors that works better for this type of data.

By developing demonstrators and performing small-scale user trials, we found various opportunities and pitfalls for deploying personal networks (PNs) on a commercial basis. The demonstrators were created using as many as possible legacy devices and proven technologies. They deal with applications in the health sector, home services, tourism, and the transportation sector. This paper describes the various architectures and our experiences with the end users and the technology. We conclude that context awareness, service discovery, and content management are very important in PNs and that a personal network provider role is necessary to realize these functions under the assumptions we made. The PNPay Travel demonstrator suggests that PN service platforms provide an opportunity to develop true trans-sector services.

Information fusion in fingerprint recognition has been studied in several papers. However, only a few papers have been focused on sensor interoperability and sensor fusion. In this paper, these two topics are studied using a multisensor database acquired with three different fingerprint sensors. Authentication experiments using minutiae and ridge-based matchers are reported. Results show that the performance drops dramatically when matching images from different sensors. We have also observed that fusing scores from different sensors results in better performance than fusing different instances from the same sensor

In this paper we investigate the effect of image resolution on the error rates of a face verification system. We do not restrict ourselves to the face recognition algorithm only, but we also consider the face registration. In our face recognition system, the face registration is done by finding landmarks in a face image and subsequent alignment based on these landmarks. To investigate the effect of image resolution we performed experiments where we varied the resolution. We investigate the effect of the resolution on the face recognition part, the registration part and the entire system. This research also confirms that accurate registration is of vital importance to the performance of the face recognition algorithm. The results of our face recognition system are optimal on face images with a resolution of 32 times 32 pixels

For user verification on a personal digital assistant (PDA), a fast and simple system is developed. In the enrollment phase, face detection and registration are done by a Viola-Jones based method, taking advantage of its accuracy and speed. The face feature vectors obtained this way are then used to build up a face space specific to the user by principal component analysis (PCA). Furthermore, the face variations caused by small registration shifts are also modeled, in order to better capture the variation in the face space, and simplify the enrollment. Current experiments show that this system is fast, efficient, and accurate

Secure access is prerequisite for a mobile personal device (MPD) in a personal network (PN). An authentication method using biometrics, specifically face, is proposed in this paper. A fast face detection and registration method based on a Viola-Jones detector is implemented, and a face-authentication method based on subspace metrics is developed. Experiments show that the authentication method is effective with an equal error rate (EER) of 1.2%, despite its simplicity

The paper presents results on optimal similarity measures for biometric verification based on fixed-length feature vectors. First, we show that the verification of a single user is equivalent to the detection problem, which implies that, for single-user verification, the likelihood ratio is optimal. Second, we show that, under some general conditions, decisions based on posterior probabilities and likelihood ratios are equivalent and result in the same receiver operating curve. However, in a multi-user situation, these two methods lead to different average error rates. As a third result, we prove theoretically that, for multi-user verification, the use of the likelihood ratio is optimal in terms of average error rates. The superiority of this method is illustrated by experiments in fingerprint verification. It is shown that error rates below 10/sup -3/ can be achieved when using multiple fingerprints for template construction.

Efficient algorithms for the exact and approximate computation of the symmetrical Kullback-Leibler (1998) measure for spectral distances are presented for linear predictive coding (LPC) spectra. A interpretation of this measure is given in terms of the poles of the spectra. The performances of the algorithms in terms of accuracy and computational complexity are assessed for the application of computing concatenation costs in unit-selection-based speech synthesis. With the same complexity and storage requirements, the exact method is superior in terms of accuracy.

This paper discusses the computation of the centroid induced by the symmetrical Kullback-Leibler distance. It is shown that it is the unique zeroing argument of a function which only depends on the arithmetic and the normalized geometric mean of the cluster. An efficient algorithm for its computation is presented. Speech spectra are used as an example.

The perceptual relevance of changes to glottal-pulse parameters is studied. First, it is demonstrated that a distance measure based on excitation patterns can predict audibility discrimination thresholds for small changes to the R parameters of the Liljencrants-Fant (LF) model. Next, by using this measure the perceptual relevance of the LF parameters is quantified. Results are presented for a number of sets of glottal-pulse parameters that were taken from literature, representing distinct voice qualities.

This paper describes a new open source architecture for unit-selection based speech synthesis called BOSS (Bonn Open Synthesis System). It is built up modularly, with communications between modules taking place in a fixed format. This makes the addition, deletion and substitution of modules very easy. The strict separation between data and algorithms allows for the simple creation of new speech corpora for different domains and languages. 1.

A common problem in diphone synthesis is discussed, viz., the occurrence of audible discontinuities at diphone boundaries. Informal observations show that spectral mismatch is the most likely the clause of this phenomenon. We first set out to find an objective spectral measure for discontinuity. To this end, several spectral distance measures are related to the results of a listening experiment. Then, we studied the feasibility of extending the diphone database with context-sensitive diphones to reduce the occurrence of audible discontinuities. The number of additional diphones is limited by clustering consonant contexts that have a similar effect on the surrounding vowels on the basis of the best performing distance measure. A listening experiment has shown that the addition of these context-sensitive diphones significantly reduces the amount of audible discontinuities.

This research describes which modalities are preferred in particular contexts when interacting with a multi-modal dialogue system. The trade-off between three factors is investigated: (i) speech recognition performance, (ii) efficiency of input modality and (iii) the system 's output modality. Four versions were developed of a multimodal examinator to be used in elementary school. The versions differed in recognition performance (`perfect' vs. realistic) and output modality (speech or text). In all systems, subjects could provide input via speaking or typing. Answer length in characters was used as a measure of efficiency. Results show that both speech recognition performance and efficiency have a strong impact on preferred modalities. No effect was found of the system's output modality. 1. INTRODUCTION "Speech is the bicycle of user-interface design," according to Shneiderman (1998:328), "it is great fun to use (. . . ), but it can carry only a light load. Sober advocates know that ...

One problem with speech synthesis impeding high quality is the occurrence of audible discontinuities at segment boundaries. Formant jumps across concatenation points suggest the problem to be due to spectral differences. The problem is most apparent in vowels and semi-vowels. We propose to reduce the number of audible discontinuities by adding context-sensitive diphones to the database. The number of additional diphones is limited by clustering contexts with similar spectral effects on the neighbouring vowels, using the Kullback-Leibler distance. A listening experiment has shown that the percentage of perceived discontinuities has significantly decreased. 1.

The pitch-marking algorithm presented in this paper avoids inconsistency errors between\npitch markers in subsequent fundamental periods by adding the requirements of waveform\nand pitch consistency. The approach is as follow s. Candidate pitch markers satisfying userdefined properties for pitch marking are selected first. Dynamic programming is then used to find the sequence of candidate pitch markers that optimally satisfies the waveform and pitchconsistency requirements. The algorithm is described in detail and results are presented.

The paper analyses how variations of the parameters of the Liljencrants-Fant (1985) model of glottal flow influence the speech spectrum, in order to determine the spectral relevance of these parameters. The effects of small parameter variations are described analytically. This analysis also gives an indication to what extent the LF parameters can be estimated reliably from the speech spectrum. The effects of larger parameter variations are discussed with the help of figures. Results are presented for a number of sets of estimated glottal-pulse parameters that were taken from the literature. The main conclusion is that the LF model, which, given the fundamental period, is a three-parameter model, actually operates as a one- or a two-parameter model.

One well-known problem with diphone concatenation is the occurrence of audible discontinuities at diphone boundaries, which are most prominent in vowels and semi-vowels. Significant formant jumps at certain boundaries suggest that the problem is of a spectral nature. We have examined this hypothesis by correlating the results of a listening experiment with spectral distances measured across diphone boundaries. The aim is to find a spectral distance measure that best predicts when discontinuities are audible in order to find out how the diphone database can best be extended with context-sensitive diphones. The results show that the KullbackLeibler measure is the best predictor. 1. INTRODUCTION Most speech synthesis systems available today are based on diphone concatenation. One well-known problem with diphone concatenation is the occurrence of audible discontinuities at diphone boundaries, which are most prominent in vowels and semi-vowels and are caused by contextual influences. Our ...

We propose methods to track natural variations in the characteristics of the vocal-tract system from speech signals. We are especially interested in the cases where these characteristics vary over time, as happens in dynamic sounds such as consonant-vowel transitions. We show that the selection of appropriate analysis segments is crucial in these methods, and we propose a selection based on estimated instants of significant excitation. These instants are obtained by a method based on the average group-delay property of minimum-phase signals. In voiced speech, they correspond to the instants of glottal closure. The vocal-tract system is characterized by its formant parameters, which are extracted from the analysis segments. Because the segments are always at the same relative position in each pitch period, in voiced speech the extracted formants are consistent across successive pitch periods. We demonstrate the results of the analysis for several difficult cases of speech signals.

After a brief introduction to two-mass models for the vocal chords, it is shown how these can be implemented efficiently with digital signal-processing techniques. This facilitates their use in speech-synthesis hardware. An example of such an implementation and the glottal pulses obtained with it are presented.

The goal is to introduce and solve the audio coding optimization problem. Psychoacoustic results such as masking and excitation pattern models are combined with results from rate distortion theory to formulate the audio coding optimization problem. The solution of the audio optimization problem is a masked error spectrum, prescribing how quantization noise must be distributed over the audio spectrum to obtain a minimal bit rate and an inaudible coding errors. This result cannot only be used to estimate performance bounds, but can also be directly applied in audio coding systems. Subband coding applications to magnetic recording and transmission are discussed in some detail. Performance bounds for this type of subband coding system are derived.>

The exploitation of left-right correlation in a subband code for stereophonic audio signals is investigated. A transform of left and right signals into decorrelated intensity and error signals is presented. Although this can be seen as the optimal exploitation of redundancy, it yields only marginal gain in bit rate. If the reduced phase-sensitivity of the human observer can be exploited by encoding only the intensity signal, a substantial gain can be obtained. Preliminary results of a stereo codec are promising: at 192 kb/s good coding results have been obtained.>

A subband coding system for high quality digital audio signals is described. To achieve low bit rates at a high quality level, it exploits the simultaneous masking effect of the human ear. It is shown how this effect can be used in an adaptive bit-allocation scheme. The proposed approach has been applied in two coding systems, a complex system in which signal is split into 26 subbands, each approximately one third of an octave wide, and a simpler 20-band system. Both systems have been designed for coding stereophonic 16-bit compact disk signals with a sampling frequency of 44.1 kHz. With the 26-band system high-quality results can be obtained at bit rates of 220 kb/s. With the 20-band system, similar results can be obtained at bit rates of 360 kb/s.>

In this paper algorithms for the restoration of unknown samples at known positions embedded in a neighbourhood of known samples are discussed. First this restoration problem is treated as a (non-adaptive) linear minimum variance estimation problem. It is shown that the optimal linear minimum variance interpolator for unknown samples from an autoregressive process uses only a finite neighbourhood of known samples, whereas in general this neighbourhood is infinite. Secondly, for signals that can be modelled as autoregressive processes an adaptive solution to the restoration problem is given.