Communication through whispering is crucial for individuals who have undergone laryngectomy, a surgical procedure involving the partial or total removal of the voice box or larynx. Whispering, which lacks fundamental frequency, often results in hushed and unintelligible speech for these patients, necessitating prosthetics or treatments. These treatments pose a risk of getting infections, and the resulting speech sounds unnatural.Recently, in a computational framework, deep learning-based methods have been developed for a whisper-to-speech conversion that could be useful for laryngectomy patients as well. In this paper, we analyse GAN-based methods for the reconstruction of laryngectomised speech, using a laryngectomy dataset that we collected in New Zealand. Based on this analysis, we propose modified architectures and loss functions specifically tailored to laryngectomy patients. The proposed models are trained on both the wTIMIT and laryngectomy patient datasets. Evaluations show improved performance in both spectral features and speech intelligibility compared to existing models.

The rise of generative artificial intelligence (GenAI) has made it increasingly possible to use Deepfakes technology to generate fake pictures and videos. While this technology has benefits, it also has downsides such as spreading misinformation and endangering public interests. To address this issue, researchers have proposed various deep forgery detection algorithms and have achieved remarkable results. However, a common problem regarding these detection methods is that while in-library detection can usually achieve high accuracy, their performance is significantly degraded in cross-library detection. This indicates a severe problem of insufficient generalisation ability.To better compare the performance differences between various detection methods, this paper analyses the detection performance of the six established models of Two-stream, MesoNet, HeadPose, FWA, VA, and Multi-task. To ensure consistency, we employ a uniform evaluation framework as a benchmark for comparison. We conduct extensive intra-library and cross-library tests to evaluate these methods’ generalisation ability by utilising accuracy and error rate as key evaluation criteria for our experiments. Additionally, we further explore areas for improvement by analysing the impact of data augmentation, dataset partitioning, and threshold selection on the performance of these detection methods. Our comparative experiments are conducted on three existing fake face video datasets, including FaceForensics++, DeepfakeTIMIT, and Celeb-DF.Our research findings indicate the database partitioning method has a direct impact on the detector’s performance, and to enhance generalisation performance, the database should be divided person-based manually. The effectiveness of data augmentation techniques in improving cross-library performance is generally limited, and setting the threshold directly using source domain data often leads to a high error rate in the target domain. The findings of this paper provide insights into the development of more effective detection methods to combat the harmful effects of Deepfakes.

Dysarthria is a speech disorder disease that can have a significant impact on a person's daily life. Early detection of the disease can put the patient into therapy sessions more quickly. Researchers have established various approaches to detect the disease automatically. Traditional computational approaches commonly analysed acoustic features like Mel-Frequency Cepstral Coefficients (MFCC), Spectral Centroid, Linear Prediction Cepstral (LPC) coefficients and Perceptual Linear Prediction (PLP) from speech samples of patients to detect dysarthric speech characters like slow speech rate, short pauses, mis-articulated sounds, etc. Recent research has shown that some machine learning algorithms can also be deployed to extract speech features and detect the severity level automatically. \n \n In machine learning, feature extraction is a crucial step in dealing with classification and prediction problems. For different data formats, different well-established frameworks have been developed to extract and classify the corresponding features. For example, for an image data processing system, Convolution Neural Network (CNN) can provide the underlying network structure for the system to analyse the video data to obtain the visual features. In contrast, for audio data processing system, Natural Language Processing (NLP) algorithms can be applied to obtain acoustic features. Therefore, the selection of the framework to be used mainly depends on the modality of the input. As early steps in development of machine learning approaches for automatic assessment of dysarthric patients, classification systems based on audio features have been considered in literature; however, recent research efforts in other fields have shown that using an audio-video cross-modal framework can improve performance of the classification systems. \n \nIn this thesis, for the first time, an audio-video cross-modal framework is proposed using deep-learning algorithm that the network takes both audio and video data as input to detect severity levels of dysarthria. Within the deep-learning framework, we also propose two network architectures using audio-only or video-only input to detect dysarthria severity levels automatically. Comparing with current one-modality systems, the deep-learning framework yields satisfying results. More importantly, comparing with systems based only on audio data for automatic dysarthria severity level assessment, the audio-video deep-learning cross modal system proposed in this research can accelerate the training speed, improve accuracy and reduce the amount of required training data.

Child sexual abuse is a serious global problem that has gained public attention in recent years. Due to the popularity of digital cameras, many perpetrators take images of their sexual activities. Traditionally, it has been difficult to use vein patterns in evidence images for forensic identification, because they were nearly invisible in colour images. State-of-the-art techniques, and computational methods including optical-based vein uncovering or artificial neural networks have recently been introduced to extract vein patterns for identification purposes. However, these methods are still not mature due to limitations such as lack of reliable feature extraction, efficient uncovering algorithms, and matching difficulties. In this paper, we propose two new schemes to overcome some of these limitations by using sparse auto-encoder and adaptive contrast enhancement. Specifically, an adjustment sparse auto-encoder parameters scheme is used for optimising parameters, and then optimised parameters are automatically trained to enhance the robustness of vein visualisation and feature extraction. We also use a pair of synchronised colour and near infrared NIR images to generate the skeletonised vein patterns for verifying the outcome of the proposed method. The proposed algorithm was examined on a database with 100 pairs of colour and NIR images collected from different parts of the body such as forearms, thighs, chests and ankles. The experimental results are encouraging and indicate that the proposed method improves the feature extraction procedure, which can lead to better uncovering results compared with current methods.

Authentication methods based on some human traits, including fingerprint, face, iris, and palmprint, have been developed significantly, and currently, they are mature enough which have been reliably considered for person identification purposes. Recently, as a new research area, few methods based on non-facial skin features such as vein patterns have been developed. This extended abstract briefly explores some key features of biometric traits whereas vein pattern recognition is also outlined.

This paper recasts the problem of online secondary path modeling in the form of a statistical inverse problem. A statistical and, in particular, a Bayesian approach towards secondary path modeling is developed and the computational issues that emerge from this approach are discussed. All signals and parameters are modeled as random variables and the degree of information concerning them is coded in their probability density functions. An abstract solution is formulated in the form of a probability density function for the secondary path model. For extracting point estimates, common statistical estimation methods are investigated. It is shown that maximum likelihood estimation is not stable; however, Bayesian method of maximum a posteriori gives a reliable solution. An adaptive algorithm is then developed to compute this solution in a computationally efficient manner. This algorithm has three advantages, compared to the traditional secondary path modeling algorithms. First, it does not cause any interference with the main active noise control algorithm. Second, it does not require any additive-noise to be injected into the secondary path. Third, it does not require any off-line initiation. The convergence of the proposed algorithm is analyzed theoretically. The validity of the theoretical results is investigated by using computer simulation. Finally successful integration of the proposed algorithm into a real-time ANC system is reported.

This paper extends the existing work on the root locus analysis of FxLMS algorithm by considering secondary path modeling errors. Rules for sketching FxLMS root locus are set out. An analytic convergence condition is then derived from the root locus plot. A deliberately-misaligned secondary path model is proposed to be used as the data preparation filter in the FxLMS algorithm. The proposed filter increases the computational efficiency of the algorithm, without changing its convergence behavior. The theoretical results are verified in practice by busing an experimental system.

Forensic investigation methods based on some human traits, including fingerprint, face, and palm print, have been developed significantly, but some major aspects of particular crimes such as child pornography still lack of notable research efforts. Unlike common forensic identification methods, techniques for identifying criminals in child pornographic images should be developed based on partial non-facial skin observable in the images because criminals always hide their faces. Few methods published recently have shown the potential of vein patterns visualized from color images as a criminal and victim identification tool. However, these methods have two weaknesses: 1) they use single model to visualize vein patterns hidden in color images, which neglects the diversity of skin properties and 2) even though their parameters are determined automatically by an optimization, they do not adapt to fit local image characteristics. To address these weaknesses, this paper proposes an algorithm composed of a bank of mapping models which transform color images to near infrared (NIR) images for visualizing vein patterns and a local parameter estimation scheme for handling different image characteristics in different regions. Imbalanced data regression is also used to systematically construct the model bank. The proposed algorithm is examined and compared with the previous methods on a database of 920 thigh images from 230 subjects. It outperforms the previous methods.

Ultrasonic speech is a novel research area with significant applications: as a speech-aid prosthesis for patients with voice box difficulties, silent speech interfaces, secure mode of communication in mobile phones and as a communication medium in high noise industrial environments. Feature extraction is a critical part of the ultrasonic speech system. Linear prediction analysis (LPA) has been recently proven to be viable for extracting features from the three dimensional ultrasonic propagation in the vocal tract (VT). A one-dimensional autoregressive model based on averaging the LP coefficients, analysed in different recording positions has been investigated by the authors to fit the LF ultrasonic resonances of the VT. To reach a state of maturity for the LPA of ultrasonic speech and in continuum of the previous work, this paper compares the application of two major conventional methods of averaging and least squares error - already applied in room acoustics - for deriving the coefficients in autoregressive modelling of ultrasonic speech.