Digital audio watermarking is an important technique to secure and authenticate audio media. This paper provides a comprehensive review of the twenty years’ research and development works for digital audio watermarking, based on an exhaustive literature survey and careful selections of representative solutions. We generally classify the existing designs into time domain and transform domain methods , and relate all the reviewed works using two generic watermark embedding equations in the two domains. The most important designing criteria, i.e., imperceptibility and robustness, are thoroughly reviewed. For imperceptibility , the existing measurement and control approaches are classified into heuristic and analytical types, followed by intensive analysis and discussions. Then, we investigate the robustness of the existing solutions against a wide range of critical attacks categorized into basic, desynchronization, and replacement attacks, respectively. This reveals current challenges in developing a global solution robust against all the attacks considered in this paper. Some remaining problems as well as research potentials for better system designs are also discussed. In addition, audio watermarking applications in terms of US patents and commercialized solutions are reviewed. This paper serves as a comprehensive tutorial for interested readers to gain a historical, technical, and also commercial view of digital audio watermarking.

Fingerprint-based authentication systems have developed rapidly in the recent years. However, current fingerprint-based biometric systems are vulnerable to spoofing attacks. Moreover, single feature-based static approach does not perform equally over different fingerprint sensors and spoofing materials. In this paper, we propose a static software approach. We propose to combine low-level gradient features from speeded-up robust features, pyramid extension of the histograms of oriented gradient and texture features from Gabor wavelet using dynamic score level integration. We extract these features from a single fingerprint image to overcome the issues faced in dynamic software approaches, which require user cooperation and longer computational time. A experimental analysis done on LivDet 2011 data produced an average equal error rate (EER) of 3.95% over four databases. The result outperforms the existing best average EER of 9.625%. We also performed experiments with LivDet 2013 database and achieved an average classification error rate of 2.27% in comparison with 12.87% obtained by the LivDet 2013 competition winner.

Recently, the electric network frequency (ENF), a natural signature embedded in many audio recordings, has been utilized as a criterion to examine the authenticity of audio recordings. ENF-based audio authentication system involves extraction of the ENF signal from a questioned audio recording, and matching it with the reference signal stored in an ENF database. This establishes a popular application of audio timestamp verification. In this paper, we explore another important application, i.e., ENF-based audio tampering detection, which has received less research attention. Specifically, we introduce the absolute-error-map (AEM) between the ENF signals obtained from the testing audio recording and the database. The AEM serves as an ensemble of the raw data associated with the ENF matching process. Through intensive analysis of the AEM, we propose two algorithms to jointly deal with timestamp verification and tampering detection, including insertion, deletion, and splicing attacks, respectively. The first algorithm is based on exhaustive point search and measurement, while the second algorithm leverages the image erosion technique to achieve fast detection of tampering type and tampered region, thus the second algorithm sacrifices some accuracy for speed. The authentication mechanism is that the system first determines if the testing data have been tampered with, and then outputs the timestamp information if no tampering is detected. Otherwise, it outputs the tampering type and tampered region. We demonstrate the effectiveness of the proposed solution via both synthetic and practical examples from our practically deployed audio authentication system.

We present a time-spread echo-based audio watermarking scheme with optimized imperceptibility and robustness. Specifically, convex optimization based finite-impulse-response (FIR) filter design is utilized to obtain the optimal echo filter coefficients. The desired power spectrum of the echo filter is shaped by the proposed maximum power spectral margin (MPSM) and the absolute threshold of hearing (ATH) of human auditory system (HAS) to ensure the optimal imperceptibility. Meanwhile, the auto-correlation function of the echo filter coefficients is specified as the constraint in the problem formulation, which controls the robustness in terms of watermark detection. In this way, a joint optimization of imperceptibility and robustness can be quantitatively performed. As a result, the proposed watermarking scheme is superior to existing solutions such as the ones based on pseudo noise (PN) sequence or modified pseudo noise (MPN) sequence. Note that the designed echo kernel is also highly secure in that only with the same filter coefficients can one successfully detect the watermark. Experimental results are provided to evaluate the imperceptibility and robustness of the proposed watermarking scheme.

Cepstral analysis is an important signal processing procedure for audio watermark detection in echo-based audio watermarking systems. However, with the use of two common versions, i.e., complex and real cepstra, this procedure is usually treated as a very standard routine. This paper starts from noting inappropriate cepstral analysis from existing works, and provides rigorous derivations to reveal the advantages of using real cepstrum than complex cepstrum in echo-based audio watermark detection. Furthermore, we introduce two alternatives, termed as real part and imaginary part cepstrum, respectively, based on which a joint detection scheme is proposed. This is achieved by noting that both real part and imaginary part cepstra contain a full version of the echo kernel coefficients, which can be appropriately combined to obtain a composite cepstrum to further suppress the interferences. The advantages of the joint detection scheme over conventional approach using real cepstrum are illustrated via both performance analysis and experimental results. The accuracies of the mathematical approximations for each version of cepstrum are evaluated by normalized misalignment. The detection robustness is evaluated using the peak-to-average power ratio. The relationships among echo length, echo delays, and scaling factor, during watermark detection phase, are also discussed. Experimental results of watermark detection rate are provided to compare the performance of complex, real, and composite cepstra, respectively.

The electric network frequency (ENF) criterion is a recently developed technique for audio timestamp identification, which involves the matching between extracted ENF signal and reference data. For nearly a decade, conventional matching criterion has been based on the minimum mean squared error (MMSE) or maximum correlation coefficient. However, the corresponding performance is highly limited by low signal-to-noise ratio, short recording durations, frequency resolution problems, and so on. This paper presents a threshold-based dynamic matching algorithm (DMA), which is capable of autocorrecting the noise affected frequency estimates. The threshold is chosen according to the frequency resolution determined by the short-time Fourier transform (STFT) window size. A penalty coefficient is introduced to monitor the autocorrection process and finally determine the estimated timestamp. It is then shown that the DMA generalizes the conventional MMSE method. By considering the mainlobe width in the STFT caused by limited frequency resolution, the DMA achieves improved identification accuracy and robustness against higher levels of noise and the offset problem. Synthetic performance analysis and practical experimental results are provided to illustrate the advantages of the DMA.

Hidden Markov Model (HMM) is a technique highly capable of modelling the structure of an observation sequence. In this paper, HMM is used to provide the contextual information for detecting clinical signs present in diabetic retinopathy screen images. However, there is a need to determine a feature set that best represents the complexity of the data as well as determine an optimal HMM. This paper addresses these problems by automatically selecting the best feature set while evolving the structure and obtaining the parameters of a Hidden Markov Model. This novel algorithm not only selects the best feature set, but also identifies the topology of the HMM, the optimal number of states, as well as the initial transition probabilities.