Denoising is the representative task in hyperspectral image processing. This letter proposes to produce a classification-driven hyperspectral image denoiser which is capable of simultaneously reducing noise and preserving semantic-aware detail. The conditional neural adversarial network and multiple loss functions are introduced to produce visually pleasing images by enforcing an additional constraint that the denoised image must be indistinguishable from its corresponding ground truth clean image. Experiments are performed on hyperspectral remote sensing images containing both the simulated hybrid noise and real noise. The results show that the proposed model outperforms many state-of-the-art denoising methods for HSIs.

Abstract Affective computing is an emerging area of research that aims to enable intelligent systems to recognize, feel, infer and interpret human emotions. The widely spread online and off-line music videos are one of the rich sources of human emotion analysis because it integrates the composer’s internal feeling through song lyrics, musical instruments performance and visual expression. In general, the metadata which music video customers to choose a product includes high-level semantics like emotion so that automatic emotion analysis might be necessary. In this research area, however, the lack of a labeled dataset is a major problem. Therefore, we first construct a balanced music video emotion dataset including diversity of territory, language, culture and musical instruments. We test this dataset over four unimodal and four multimodal convolutional neural networks (CNN) of music and video. First, we separately fine-tuned each pre-trained unimodal CNN and test the performance on unseen data. In addition, we train a 1-dimensional CNN-based music emotion classifier with raw waveform input. The comparative analysis of each unimodal classifier over various optimizers is made to find the best model that can be integrate into a multimodal structure. The best unimodal modality is integrated with corresponding music and video network features for multimodal classifier. The multimodal structure integrates whole music video features and makes final classification with the SoftMax classifier by a late feature fusion strategy. All possible multimodal structures are also combined into one predictive model to get the overall prediction. All the proposed multimodal structure uses cross-validation to overcome the data scarcity problem (overfitting) at the decision level. The evaluation results using various metrics show a boost in the performance of the multimodal architectures compared to each unimodal emotion classifier. The predictive model by integration of all multimodal structure achieves 88.56% in accuracy, 0.88 in f1-score, and 0.987 in area under the curve (AUC) score. The result suggests human high-level emotions are automatically well classified in the proposed CNN-based multimodal networks, even though a small amount of labeled data samples is available for training.

This study presents an offline mobile diagnosis system for citrus pests and diseases by compression convolutional neural network. Recently, with the growth of labelled data, the deep neural network incites the revolutionary change with a quantum leap in various fields. Benefiting from the backpropagation method, the proper network structure can automatically extract high‐level representations and find corresponding labels. The authors made use of the advantages of the deep neural network to design an android application, which can be installed in any stand‐alone devices to instantaneously identify the citrus pests and diseases. The proposed diagnosis system has three characteristics: low cost, low latency and high accuracy. These characteristics contribute to make the professional offline prediction for avoiding further economic loss caused by disease spreading. In order to validate the proposed system, the authors conducted thorough evaluations on two data sets, ‘citrus pests and diseases’, CIFAR, which show the superiority of the proposed approach in terms of the accuracy and the number of model parameters.

Deep learning, especially a discriminative model for image denoising, has shown great potential in removing complex spectral-spatial noise in hyperspectral images (HSI). For HSI denoising, it is crucial to extract more context information around each pixel and to predict each pixel according to the surrounding context. Therefore, the effective receptive field plays an important role when performing denoising task. Generally, an HSI denoising model can achieve better performance by reserving the correlation of adjacent spectral bands and extracting more pixel features in the spatial domain. In this paper, 3-D atrous denoising convolution neural network (3DADCNN) is proposed for HSI. The model extracts feature maps along both spatial and spectral dimensions and enlarges the receptive field without significantly increasing the number of network parameters. Simultaneously, the multibranch and multiscale structure is utilized to reduce training difficulty, lessen overfitting risk, and preserve details in texture. The proposed model can be applied to the corrupted image with a mixed type of photon and thermal noise. Experimental results of the quantitative and qualitative evaluation show that 3DADCNN outperforms state-of-the-art HSI denoising methods.

Music mood classification is a crucial component in the field of multimedia database retrieval and computational musicology. There is a constantly growing interest in developing and evaluating music information retrieval (MIR) systems that can provide automated access to the music mood. The proposed method considers the different types of audio features. From each feature's frame, a bin histogram has been calculated to preserve all important information associated with it. The histogram bins of each feature are used to calculate the similarity matrix, and the number of similarity matrices depends on the number of audio features. Therefore, there are 59 similarity matrixes from the corresponding same amount of audio features. The intra and inter similarity matrix are used to calculate the intra-inter similarity ratio. These similarity ratios are sorted in descending order in each feature. Among them, some of the selected similarity ratios are ultimately used as prototypes from each feature and are used for classification by designing the nearest multi-prototype classifier. The Coimbra mood dataset is used to measure the overall performance of the proposed method. We achieved competitive classification accuracies as compared with other existing state-of-the-art music mood classification techniques.

Multimedia database retrieval is growing at a fast rate thereby subsequent increase in the popularity of online retrieval system. The large datasets are major challenges for searching, retrieving, and organizing the music content. Therefore, there is a need of robust automatic music genre classification method for organizing these music data into different classes according to the certain viable information. There are two fundamental components to be considered for genre classification namely audio feature extraction and classifier design. In this paper, diverse audio features set have been proposed to characterize the music contents precisely. The feature sets belong to four different groups, i.e. dynamic, rhythm, spectral, and harmony. From the features, five different statistical parameters are considered as representatives, including up to the 4thorder central moments of each feature, and covariance components. Ultimately, significant numbers of representative attributes are controlled by MRMR algorithm. The algorithm calculates the score level of all feature attributes and orders them. The high score feature attributes are only considered for genre classification. Moreover, we can visualize that which audio features and which of the different statistical parameters derived from them are important for genre classification. Among them, mel frequency cepstral coefficients (MFCCs) have higher scored level than other feature attributes. Furthermore, MRMR does not transform the feature value like as principal component analysis (PCA). Besides these, the comparison has been made based on classification accuracy between two-dimensionality reduction methodologies using support vector machine (SVM). The classification accuracy of MRMR feature reduction set outperforms than PCA. The overall classification is also higher than other existing state-of-the-art of frame base methods.

This paper proposes a novel scheme for evaluating an emotional response to color images. The proposed scheme uses case-based reasoning in which the prototypical color images for each emotion are stored as cases and are compared with the images to be evaluated. In the comparison, the similarities in terms of image descriptors play an important role, and their combination is crucial for the construction of a proper similarity measure. In the training phase of the proposed scheme, the weights that represent the unequal importance of each descriptor is determined in order to obtain a similarity measure that can be used to evaluate and classify a color image with respect to a pair of emotions. Prior to classification, the representative color images are chosen for each emotion by human subjects and are stored as cases. The stored images are compared with an image to be classified using the constructed similarity measure to determine which emotion is appropriate between a pair of emotions. In this study, we used color and texture descriptors recommended by MPEG-7, represented as high-dimensional vectors. In the training, we proposed a method based on the rough approximation and the fuzzy inter- and intra-similarities to determine the weights that represent the unequal importance of the complex MPEG-7 descriptors. Experimental results show a promising performance for the proposed scheme, and better performance could be achieved by including more prototypical images as cases.

This paper proposes a real time system for detecting vehicles that change lanes illegally based on tracking the feature points. The proposed system consists of three stages, which take part of the roles such as feature extraction of corners, registration and tracking the feature points attached to vehicles, and detecting a vehicle that violates legal lane changes. In the stage of feature extraction, we used a fast algorithm that can provide stable corners. The salient points are again selected among the corner points for registration and tracking. NCC(normalized cross correlation) is used to keep tracing the registered feature points. Finally, illegal change-of-lane is detected by the information about the traced corners. As a result of experiment, the proposed system showed excellent performance as amount of 99.09% of correct detection ratio and 0.9% of error. The fast processing could deal with 34.48 frames per second, which is sufficient for real-time processing.