In this paper, we propose a system for 3D face model reconstruction. Earlier studies on reconstruction methods included the software modeling methods or the instrument scanning modeling methods. But both of the above methods require a lot of development resources and time costs. Therefore, we develop a reconstruction system using a weakly supervised approach combining Convolutional Neural Networks (CNN) and 3D Morphable Face Models (3DMM). Given a sufficient number of 2D face images to train and learn the main features of the face, our system is capable of rapidly constructing 3D face models. The proposed method enhances the efficiency of preprocessing and improves the performance of loss function through image depth feature extraction and regression coefficients. Using two datasets for model evaluation and analysis, this study efficiently reconstructs faces without ground-truth labels.

Music information retrieval is of great interest in audio signal processing. However, relatively little attention has been paid to the playing techniques of musical instruments. This work proposes an automatic system for classifying guitar playing techniques (GPTs). Automatic classification for GPTs is challenging because some playing techniques differ only slightly from others. This work presents a new framework for GPT classification: it uses a new feature extraction method based on spectral-temporal receptive fields (STRFs) to extract features from guitar sounds. This work applies a supervised deep learning approach to classify GPTs. Specifically, a new deep learning model, called the hierarchical cascade deep belief network (HCDBN), is proposed to perform automatic GPT classification. Several simulations were performed and the datasets of: 1) data on onsets of signals; 2) complete audio signals; and 3) audio signals in a real-world environment are adopted to compare the performance. The proposed system improves upon the F-score by approximately 11.47% in setup 1) and yields an F-score of 96.82% in setup 2). The results in setup 3) demonstrate that the proposed system also works well in a real-world environment. These results show that the proposed system is robust and has very high accuracy in automatic GPT classification.

The next-generation video compression standard H.266/Future Video Coding (FVC) provides high compression efficiency in terms of the cost of computing the optimal intra mode from 67 modes. We propose an intra mode prediction method based on a convolutional neural network (CNN). An input image set of 20 × 20 blocks is used to train the CNN; the CNN is used to predict the best classes of intra mode direction. The CNN architecture comprises two convolutional layers and a fully connected layer. Compared with the default fast search method in FVC, the proposed method can achieve a 0.033% decrease in Bjøntegaard delta bit rate (BDBR) with only a slight increase in time.

In this work, we propose a system for speech emotion recognition based on regression models and classification models jointly. This speech emotion recognition technology can achieve the accuracy of 64.70% in the dataset of script and improvised mixed scenes. The accuracy can be up to 66.34% in the dataset with only improvised scenes. Compared to the state-of-art technology without the mental states, the accuracy of the proposed method is increased by 2.95% and 2.09% respect to improvised and mixed scenes. The results show that the characteristics of mental states can effectively improve the performance of speech emotion recognition.

Diacritics are very important in diacritical languages, because the meaning of sentences can be changed in accordance to diacritics. Writing without diacritics makes the sentences ambiguous; however, there are several reasons make people do not write words with diacritics, such as fast typing, convenience, or texting on unsupported diacritics devices. As a result, these texts are very difficult to process on further natural language processing (NLP) tasks like machine translation, sentiment analysis, or question answering system. Therefore, diacritics restoration is critical for further usage or processing in NLP related tasks. In this study, we propose a method which combines convolutional neural network (CNN) and bidirectional gated recurrent unit (Bi-GRU) to restore diacritics. In addition, we use residual block to resolve vanishing gradient problem of recurrent neural networks. We applied the model for restoring diacritics of Vietnamese language that has the highest ratio of diacritics in words. This approach has character accuracy at 98.63% and word accuracy at 94.77%.

Sentiment analysis has an important role in social media monitoring as it extracts public opinions, emotions, and feelings about certain products or services. There are several publications in building a system to identify opinions from text using rule-based approach, lexicon-based approach, or machine learning. In this paper, we propose and compare several deep learning models to solve sentiment analysis problem of the Internet Movie Database (IMDb) review sentiment dataset. The feature extractor consists of a convolutional layer, followed by a max pooling layer and a batch normalization layer. To solve the vanishing gradient problem, we use a residual connection to concatenate the input values with the extracted features before feeding the output into a recurrent layer. Our best model has an accuracy of 90.02%.

Sounds are ubiquitous in our daily lives, for instance, sounds of vehicles or sounds of conversations between people. Therefore, it is easy to collect all these soundtracks and categorize them into different groups. By doing so, we can use these assets to recognize the scene. Acoustic scene classification allows us to do so by training our machine which can further be installed on devices such as smartphones. This provides people with convenience which improves our lives. Our goal is to maximize our validation rate of our machine learning results and also optimize our usage of hardware. We utilize the dataset from IEEE Detection and Classification of Acoustic Scenes and Events (DCASE) to train our machine. The data of DCASE 2017 contains 15 different kinds of outdoor audio recordings, including beach, bus, restaurant etc. In this work, we use two different types of signal processing techniques which are Log Mel and HPSS (Harmonic-Percussive Sound Separation). Next we modify and reduce the MobileNet structure to train our dataset. We also make use of fine-tuning and late fusion to make our results more accurate and to improve our performances. With the structure aforementioned, we succeed in reaching the validation rate of 75.99% which is approximately the seventh highest performing group of the Detection and Classification of Acoustic Scenes and Events (DCASE) Challenge 2017, with less computational complexity comparing with others having higher accuracy. We deem it a worthy trade-off.

We proposed a novel deep neural network based cover song retrieval method in AAC domain to reduce the computation complexity. The modified discrete cosine transform coefficients from the AAC were extracted and mapped into the 12-dimensional chroma features. Chroma features were further segmented to preserve the melody of music. Each segment of chroma features was trained and learned to reduce its dimension by using an autoencoder, used for deep learning of artificial neural networks. Experiments were conducted on cover80 database, which was provided by Ellis [3]. The results showed that the mean reciprocal rank increased to 0.46. The performance of the proposed method was compared with other systems, including LabROSA'06 (Tempos 120 and 60), and the method of principal component analysis (PCA) from the cover80. Fig. 1 plots the performances of MRR and similarity matching time. The proposed method improved the MRR performance and reduced approximately 80% of the matching time compared with LabROSA'06 (Tempo 120) [1].

SAR is a complex system that integrates two major parts: data collector and image formatter [1-2]. In the phase of data collection, radar transmits electromagnetic waves toward the target and receives the scattered waves. The transmitted signal can be modulated into certain types, commonly linearly frequency modulated with pulse or continuous waveform. The process involves signal transmission from generator, through various types of guided device, to antenna, by which the signal is radiated into free space, and then undergoes propagation. The measured scattered signal been made in bistatic or monostatic configurations is essentially in time-frequency (delay time - Doppler frequency) domain. The role of image formatter is then to map the time-frequency data into spatial domain where the targets are located. The mapping from the data domain to image domain, and eventually, into target or object domain must minimize both geometric and radiometric distortions. Essentially, two models that define the SAR operational process: physical model and system model. This paper concentrates on the physical process of a SAR system from wave scattering to imaging. System simulation based on the stationary (frequency modulation continuous wave) FMCW is developed and implemented for both point target and extended target. To further validate the simulation and thus our physical understanding of the imaging chain, measurements at aniconic chamber with two mental spheres and two dielectric spheres displaced with varying spacing were conducted. Good agreement between the simulated by extend target model and real measured SAR images is obtained.

Large-scale cover song retrieval systems should be able to calculate song-to-song similarity and accommodate differences in timing, key, and tempo. Simple vector distance measure is not adequately powerful to perform cover song recognition, and expensive solutions such as dynamic time warping do not scale to millions of instances, making cover song retrieval inappropriate for commercial-scale applications. In this work, we used the content-based music features of songs as input and transformed them into vectors by using the 2D Fourier transform approach. Furthermore, we explored different machine learning approaches to reinforce the pattern of these vectors. By projecting the songs into a semantic vector space, we can use the efficient nearest neighbor algorithm to compare the similarity of songs and retrieve the most similar songs from the large-scale database. The proposed system is not only efficient enough to perform scalable content-based music retrieval but can also develop the potential of machine learning approaches, making similar music recognition applications faster and more accurate.

In this paper, a fast CU depth decision algorithm based on support vector machine (SVM) is proposed to reduce the computational complexity of HEVC intra coding. It is systematic to develop the criterion of early CU splitting and termination by applying SVM. Appropriate features for training SVM models are extracted from spatial domain and pixel domain. Artificial neural network is used to analyze the impact of each extracted feature on CU size decision, and different weights are assigned to the output of SVMs. The experimental results show that the proposed fast algorithm provides 58.9% encoding time saving at most, and 46.5% time saving on average compared with HM 12.1.

Downsampling video coding, whereby downsampled frames are encoded, provides improved perceptual quality in rate-constrained situations. This method shows considerable advantages over other approaches, particularly in wide-spreading high-definition video formats. This paper provides a comprehensive analysis of downsampling video coding. The study proposes a spatially scalable rate-distortion (RD) model, comprising quantization-distortion and quantization-rate models, and develops an optimal encoding frame size determination framework. The proposed method achieves a gain up to 2.3 dB peak signal-to-noise ratio (PSNR) at 1 Mb/s when compared with conventional full frame size coding. The RD performance is close to the optimal scenario, in which the ideal frame size is obtained by heuristically performing downsampling coding in various allowable sizes.

This paper proposes a fast inter-layer motion estimation algorithm on spatial scalability for scalable video coding extension of H.264/AVC. In the enhancement layer motion estimation, we utilize the relation between two motion vector predictors from the base layer and the enhancement layer respectively to reduce the number of searches. Additionally, we utilize the mode correlations of temporal direction motion estimation to save more encoding time. The simulation results show that the proposed algorithm can save the computation time up to 67.4% compared with JSVM9.12 with less than 0.0476dB video quality degradation.

In the advanced H.264 video coding, the computation complexity is much higher than the previous video coding standards due to the variable block size and multi-reference frame features which are used in the motion compensation process. This paper proposes a hierarchical H.264 fast motion estimation algorithm to decrease the coding complexity in both spatial and temporal domains for encoding high-definition videos. In the spatial domain, we utilize the fast search method with a hierarchical-subsampling structure to decrease the memory access bandwidth of search points. In the temporal domain, we employ the linear motion model to further reduce the search ranges of multiple reference frames. This search algorithm is particularly suitable for being implemented in the parallel-processing architecture with the limited hardware resources. Simulation results show that the proposed algorithm can reduce up to 98.2% computation complexity of full search in JM with less than 0.1 dB video quality degradation.

The allowable computational complexity of video encoding is limited in a power-constrained system. Different video frames are associated with different motions and contexts, and so are associated with different computational complexities if no complexity control is utilized. Variation in computational complexity leads to encoding delay jittering. Typically motion estimation (ME) consumes much more computational complexity than other encoding tools. This work proposes a practical complexity control method based on the complexity analysis of an H.264 video encoder to determine the coding gain of each encoding tool in the video encoder. Experiments performed on a programming optimized source code show that the computational complexity associated with each frame is well controlled below a given limit with very little R-D performance degradation under a reasonable constraint comparing to the unconstrained case.

In this paper, a new multimedia joint fingerprinting and encryption (JFE) scheme embedded into the advanced access content system (AACS) is proposed. Like other security-related systems, there exist some security threats to the proposed framework. To cope with these difficulties, the contributions of this paper include: (i) we apply multimedia encryption at different points to resist some attacks points; and (ii) we propose rewritable fingerprint embedding (RFE) to deal with some multi-point collusion attacks. Experimental results are provided to demonstrate the proposed AACS-compatible JFE method.

The major disadvantage of existing watermarking methods is their limited resistance to extensive geometric attacks. In addition, we have found that the weakness of multiple watermark embedding methods that were initially designed to resist geometric attacks is their inability to withstand the watermark-estimation attacks (WEAs), leading to reduce resistance to geometric attacks. In view of these facts, this paper proposes a robust image watermarking scheme that can withstand geometric distortions and WEAs simultaneously. Our scheme is mainly composed of three components: 1) robust mesh generation and mesh-based watermarking to resist geometric distortions; 2) construction of media hash-based content-dependent watermark to resist WEAs; and 3) a mechanism of false positive-oriented watermark detection, which can be used to determine the existence of a watermark so as to achieve a tradeoff between correct detection and false detection. Furthermore, extensive experimental results obtained using the standard benchmark (i.e., Stirmark) and WEAs, and comparisons with relevant watermarking methods confirm the excellent performance of our method in improving robustness. To our knowledge, such a thorough evaluation has not been reported in the literature before.

Due to the burstiness property of compressed videos, an effective video traffic description scheme is important for efficient network bandwidth management. This paper proposes an elastically deterministic video traffic regulating mechanism (ED-VTR) that models and regulates the video traffic based on constraints of available decoder buffer space. In contrast to traditional methods, ED-VTR generates a transmission schedule that contains a series of monotonically decreasing bandwidth requests. Simulation results show that, using lower initial bandwidth and smaller decoder buffer requirement, ED-VTR can effectively reduce the complexity of bandwidth management and still hold the advantages of traditional traffic smoothing and deterministic traffic modeling methods.

Due to the desired non-invasive property, non-data hiding (called media hashing here) is considered to be an alternative to achieve many applications previously accomplished with watermarking. Recently, media hashing techniques for content identification have been gradually emerging. However, none of them are really resistant against geometrical attacks. In this paper, our aim is to propose a geometry-invariant image hashing scheme, which can be employed for content copy detection and tracing. Our system is mainly composed of three components: (i) robust mesh extraction; (iii) mesh-based robust hash extraction; and (iii) hash matching for similarity measurement. Exhaustive experimental results obtained from benchmark attacks have confirmed the performance of the proposed method.

In this paper, we present a novel energy compaction method, the selective block reordering, which is used with SPIHT (SBR-SPIHT) coding for low rate video coding to enhance the coding efficiency for motion-compensated residuals. The inter-frame coding basically includes three major parts - motion estimation, motion compensation, and motion-compensated residual coding. The motion estimation and overlapped block motion compensation (OBMC) methods of H.263 are used to reduce the temporal redundancy. The motion-compensated residuals are encoded in the wavelet domain. The block-mapping reorganization utilizes the wavelet zerotree relationship that jointly presents the wavelet coefficients from the lowest subband to high frequency subbands at the same spatial location, and allocates each wavelet tree with all descendents to form a wavelet block. The block reordering based on the threshold scan rearranges the significant blocks in the descending order of the energy. Then, the block reordering technique reorders the wavelet sub-blocks recrusively, according to the energy of each sub-block, to yield the maximum energy compaction that allows the SPIHT coding to operate efficiently on the motion-compensated residuals. Simulation results demonstrate that SBR-SPIHT outperforms H.263 by 1.28~0.69 dB on average for various video sequences at very low bit-rates, ranging from 48 to 10 kbps.

The objective of this work is to reconstruct high quality gray-level images from bilevel halftone images. We develop optimal inverse halftoning methods for several commonly used halftone techniques, which include dispersed-dot ordered dither, clustered-dot ordered dither, and error diffusion. At first, the least-mean-square (LMS) adaptive filtering algorithm is applied in the training of inverse halftone filters. The resultant optimal mask shapes are significantly different for various halftone techniques, and these mask shapes are also quite different from the square shape that was frequently used in the literature. In the next step, we further reduce the computational complexity by using lookup tables designed by the minimum mean square error (MMSE) method. The optimal masks obtained from the LMS method are used as the default filter masks. Finally, we propose the hybrid LMS-MMSE inverse halftone algorithm. It normally uses the MMSE table lookup method for its fast speed. When an empty cell is referred, the LMS method is used to reconstruct the gray-level value. Consequently, the hybrid method has the advantages of both excellent reconstructed quality and fast speed. In the experiments, the error diffusion yields the best reconstruction quality among all three halftone techniques.

A precise error tracking scheme for robust transmission of real-time H.263 video is presented. By utilizing a feedback channel, the decoder reports the addresses of corrupted blocks induced by transmission errors back to the encoder. With these negative acknowledgments, the encoder can precisely calculate and track the propagated errors by examining the backward motion dependency for each pixel in the current encoding frame. With this precise tracking, the error-propagation effects can be terminated completely by INTRA refreshing the affected macroblocks. In addition, by utilizing the four-corner tracking approximation and the linear motion model, a fast algorithm is also developed to further reduce the computation and memory requirements. The simulations show that both schemes yield significant video quality improvements in error-prone environments. The advantages of the low memory requirement and the low computation complexity are particularly suitable for real time implementation.

Video sequences compressed by the current video-compression standards-such as MPEG 1/2 and H.261/H.263, which include motion compensation and variable-length coding-are very sensitive to channel disturbances. There exist many error-concealment techniques that can improve the video quality substantially. However, they generally do not prevent or terminate the error propagation. The forced intraupdate technique was proposed for H.245. In this paper, we present an efficient temporal error-propagation prevention method that implements the forced intraupdate concept implicitly. Once an error is detected at the decoder, the starting address of the erroneous macroblocks is sent back to the encoder. The encoder marks the possible damaged area, and the encoding process continues as normal except that the motion estimation will probably not refer to the erroneous part. Thus, the impact of the cell loss is limited to the erroneous slice only, and the damage from the error propagation is greatly reduced. Simulation results of MPEG-2 coding over asynchronous transfer mode (ATM) networks show that the error concealment with feedback can effectively isolate the error and reduce the damage to give satisfactory performance even when the cell-loss rate is higher than 1%. With the delay analysis of ATM networks, we also show that in most cases, the encoder has adequate time to get the feedback information before processing the next I- or P-frames.

IS-96 describes a variable rate code-excited linear prediction (CELP) speech coder in code division multiple access (CDMA) digital cellular environment. In this work we focus on the study of the error protection methods in CDMA cellular environments. We find that although the error protection technique described in IS-96 can improve the speech quality in noisy channels considerably, the error concealment effect in IS-96 can be further improved by a better bit protection pattern without increasing the processing complexity.

Design algorithms and simulation results are presented for vector quantizers for Fourier transformed data. Transforming the data prior to quantization has two potential advantages. First, each sample in the transform domain depends on many samples in the original domain. Thus, even scalar quantization in the transform domain is a form of vector quantization or block source coding in the original waveform domain and the basic coding theorems of information theory show that such block codes can provide better performance than scalar codes, even for memoryless sources. Second, vector quantization of Fourier transformed speech waveforms provides distinctly better subjective quality than ordinary vector quantization of the waveform using codes of comparable complexity. While the system is, of course, more complicated due to the need to take Fourier transforms, its envisioned application is as a coder for the output of FFT chips currently available or under development. The proposed implementation of a Fourier transform vector quantizer (FTVQ) uses a product code structure, providing different codes for different coefficient vectors corresponding to different frequency bands. This is a form of subband coding and yields a simple means of optimizing bit allocations among the subcodes. Two coding structures with corresponding distortion measures are considered: those that quantize vectors of pairs of real and imaginary coefficients and those that quantize separate vectors of magnitude and phase coefficients. Both structures yield good performance for the given complexity in comparison to waveform vector quantizers. For speech coding, a magnitude-phase FTVQ yields better subjective quality than a real-imaginary FTVQ when the rate allocation is properly chosen.