Deep neural networks (DNNs) reveal significant robustness deficiencies due to their susceptibility to being misled by small and imperceptible adversarial examples, thus it is crucial to improve the robustness of DNNs against such harmful perturbations. The current$L_{p}$specification ignores differences in human visual perception when measuring similarity, and most existing image quality assessment (IQA) methods and adversarial example datasets lack subjective scores for evaluation. In this paper, we construct a new database of adversarial examples, called the AED, which contains 35 original images, 1050 adversarial examples, and the corresponding subjective scores of adversarial examples. Then, a novel full-reference IQA model for the quality evaluation of the adversarial examples is proposed by taking into full consideration the hierarchical perception of human visual system (HVS) and the outstanding capabilities of the multi-scale feature extraction network in feature extraction. Specifically, a feature encoding network that uses continuous convolution layers to pre-extract features and expand the receptive field of the image is employed. To simulate the HVS hierarchical perception, the features of different scales are further obtained by designing a multi-scale feature extraction network. The structural similarity scores of the feature maps at different scales are calculated for jointly arriving at the final IQA score of the adversarial examples. Experimental results have demonstrated that our proposed model is closer to the perception of HVS in small imperceptible distortions evaluation of adversarial examples compared with other classical and state-of-the-art models.

Visual security index (VSI) represents a quantitative index for the visual security evaluation of perceptually encrypted images. Recently, the research on visual security of encrypted light field (LF) images faces two challenges. One is that the existing perceptually encrypted image databases are often too small, which is easy to cause overfitting in convolutional neural network (CNN). The other is that existing VSI models did not take a full account the intrinsic characteristics of the LF images and highly relied on handcrafted feature extraction. In this article, we construct a new database of perceptually encrypted LF images, called the PE-SLF, which is 2.6 times as big as the existing largest perceptual encrypted image database. Moreover, a novel visual security index (VSI) model is proposed by taking into full consideration the intrinsic spatial-angular characteristics of the LF images and the outstanding capabilities of CNN in feature extraction. First, we exploit CNN to detect the texture and structure features of encrypted sub-aperture images in the spatial domain. Second, we apply the Gabor filter to detect the Gabor feature over the epi-polar plane images in angular domain. Last, the spatial and angular similarity measurements are subsequently calculated for jointly yielding the final visual security score. Experimental results on the constructed PE-SLF demonstrate that the proposed VSI model is closer to the perception of HVS in visual security evaluation of encrypted LF images compared to other classical and state-of-the-art models.

With the growing availability and popularity of sentiment-rich resources like blogs and online reviews, new opportunities and challenges have emerged regarding the identification, extraction, and organization of sentiments from user-generated documents or sentences. Recently, many studies have exploited lexicon-based methods or supervised learning algorithms to conduct sentiment analysis tasks separately; however, the former approaches ignore contextual information of sentences and the latter ones do not take sentiment information embedded in sentiment words into consideration. To tackle these limitations, we propose a new model named Sentiment Convolutional Neural Network (SentiCNN) to analyze the sentiments of sentences with both contextual and sentiment information of sentiment words, in which, contextual information is captured from word embeddings and sentiment information is identified using existing lexicons. We incorporate a Highway Network into our model to adaptively combine sentiment and contextual information from sentences by strengthening the connection between features of both sentences and their sentiment words. Furthermore, we propose three lexicon-based attention mechanisms (LBAMs) for our SentiCNN model to find the most important indicators of sentiments and make predictions more effectively. Experiments over two well-known datasets indicate that sentiment words, the Highway Network, and LBAMs contribute to sentiment analysis.

Label-specific topics can be widely used for supporting personality psychology, aspectlevel sentiment analysis, and cross-domain sentiment classification.To generate labelspecific topics, several supervised topic models which adopt likelihood-driven objective functions have been proposed.However, it is hard for them to get a precise estimation on both topic discovery and supervised learning.In this study, we propose a supervised topic model based on the Siamese network, which can trade off label-specific word distributions with document-specific label distributions in a uniform framework.Experiments on realworld datasets validate that our model performs competitive in topic discovery quantitatively and qualitatively.Furthermore, the proposed model can effectively predict categorical or real-valued labels for new documents by generating word embeddings from a labelspecific topical space.

The least-squares support vector machine (LS-SVM) is a popular data-driven modeling method and has been successfully applied to a wide range of applications. However, it has some disadvantages, including being ineffective at handling non-Gaussian noise as well as being sensitive to outliers. In this paper, a robust LS-SVM method is proposed and is shown to have more reliable performance when modeling a nonlinear system under conditions where Gaussian or non-Gaussian noise is present. The construction of a new objective function allows for a reduction of the mean of the modeling error as well as the minimization of its variance, and it does not constrain the mean of the modeling error to zero. This differs from the traditional LS-SVM, which uses a worst-case scenario approach in order to minimize the modeling error and constrains the mean of the modeling error to zero. In doing so, the proposed method takes the modeling error distribution information into consideration and is thus less conservative and more robust in regards to random noise. A solving method is then developed in order to determine the optimal parameters for the proposed robust LS-SVM. An additional analysis indicates that the proposed LS-SVM gives a smaller weight to a large-error training sample and a larger weight to a small-error training sample, and is thus more robust than the traditional LS-SVM. The effectiveness of the proposed robust LS-SVM is demonstrated using both artificial and real life cases.

An effective model of the hydraulic actuator is crucial for high-accuracy driving control. However, modeling this driving process is difficult due to strong nonlinearities in the hydraulic system and load as well as unknown influence from sources of noise including sampling, modeling, measurement, and operation errors in the process. In this paper, a probabilistic weighted least square support vector machine (LS-SVM) is proposed to model this kind of processes under noise. The proposed method can increase robustness and accuracy even with outliers or non-Gaussian noise. First, a distribution construction method is developed to extract the probabilistic distribution of the distributed LS-SVM models caused by outliers and/or noise samples. Then, a new objective function is constructed using this distribution information to negate outlier or noise influence. Since this method considers the probabilistic information of outliers and/or noise samples, it has greatly improved the LS-SVM modeling capabilities in a noisy environment. Successful application of the proposed method to artificial cases and practical hydraulic actuators as well as comparison to several common modeling methods further highlight its superiority in modeling the nonlinear processes with noise.

Many industrial processes are complex nonlinear distributed parameter systems (DPSs) with time-varying spatiotemporal dynamics. However, time-varying spatiotemporal dynamics and the nonlinear relationships between spatial points are currently not given much consideration in the existing data-driven modeling methods. Thus, accurately modeling a nonlinear DPS with time-varying spatiotemporal dynamics using these current methods is challenging. Here, we propose a spatiotemporal extreme learning machine (ELM) to accurately model time-varying and nonlinear DPSs. First, we develop the nonlinear spatial activation function to describe the nonlinear relationships between spatial points. As a result, in contrast to the traditional ELM method which is only used to model the temporal dynamics, the spatiotemporal ELM inherently takes the spatial information into consideration. Next, an online time coefficient model is developed, which accounts for the time-varying temporal dynamics of the DPS. After the integration of the spatial activation function with the time coefficient model, this modeling method is able to adapt to real-time spatiotemporal variation. Unlike the existing data-driven DPS modeling approaches, the proposed method has the capability to accurately represent the nonlinear relationships between spatial points and has the adaptive ability for modeling time-varying dynamics. Finally, through application on practical curing experiments, the proposed method can improve the modeling precision for an unknown, time-varying, and nonlinear DPS due to smaller modeling error as compared to the several commonly used DPS modeling methods.

The least-squares support vector machine (LS-SVM) has been successfully used to model nonlinear time dynamics; however, it does not have the capability to handle space information and is, therefore, unable to model the complex nonlinear distributed parameter systems (DPS). Here, we propose a spatiotemporal LS-SVM modeling method for complex nonlinear DPS. The space kernel function is developed to describe the nonlinear correlation between space locations. The time Lagrange multiplier represents the time dynamics. The integration of the space kernel function and the time Lagrange multiplier can reconstruct the nonlinear spatiotemporal dynamics of the DPS. The spatiotemporal LS-SVM method accounts for the time dynamics and the space distribution nature of the DPS, enabling it to adapt well to real-time spatiotemporal variation. The successful application of this spatiotemporal LS-SVM method to a practical curing thermal process and its comparison with several common DPS modeling methods demonstrate its superiority in the modeling of the unknown nonlinear distributed parameter process.

An effective model of the forging process is crucial for the optimal operation and health management of a hydraulic press machine (HPM). Modeling this forging process is difficult, because multiple localized nonlinear solutions and modeling of unknown complex interactions between localized regions are required. In this paper, a novel least squares support vector machine (LS-SVM) method is developed for modeling the forging process. The proposed method integrates the advantages of local LS-SVM modeling and global regularization. Local LS-SVM modeling is performed to capture the local dynamics for each local working region. Global regularization is performed to minimize the global error and improve the global generalization of the local models. These features guarantee continuity and smoothness between the local LS-SVM models and avoid over-fitting of each local LS-SVM model. The algorithm developed here is simple and may easily be added into existing HPM systems. Experiment data from a practical HPM demonstrate the effectiveness of the proposed method.

An effective model of batch forging processes is crucial to ensure the quality conformance control of batch productions. However, obtaining this model has proven difficult due to a variety of the raw forgings produced by manufacturing error, material variation, geometric defects, etc. In this paper, a novel online probabilistic extreme learning machine (ELM) is proposed to model batch forging processes. A probabilistic ELM is first developed to extract the distribution information of the batch forging processes from the data. Due to the highly linear structure of the ELM, the stochastic property of the forging process is easily derived and processed. By using the characteristics of the online ELM, a strategy is then developed to update the distribution model as new forging process data are collected. Finally, case studies on complex batch forging processes demonstrate the effectiveness of the proposed online probabilistic ELM.