Emotional expressions form a key part of user behavior on today's digital platforms. While multimodal emotion recognition techniques are gaining research attention, there is a lack of deeper understanding on how visual and non-visual features can be used to better recognize emotions in certain contexts, but not others. This study analyzes the interplay between the effects of multimodal emotion features derived from facial expressions, tone and text in conjunction with two key contextual factors: i) gender of the speaker, and ii) duration of the emotional episode. Using a large public dataset of 2,176 manually annotated YouTube videos, we found that while multimodal features consistently outperformed bimodal and unimodal features, their performance varied significantly across different emotions, gender and duration contexts. Multimodal features performed particularly better for male speakers in recognizing most emotions. Furthermore, multimodal features performed particularly better for shorter than for longer videos in recognizing neutral and happiness, but not sadness and anger. These findings offer new insights towards the development of more context-aware emotion recognition and empathetic systems.

This paper studies the properties of socially popular news with a focused interest on the emotions conveyed through their headlines. We delve deeply into the notion of emotional salience in news values and extract the emotion intensities features across the valence, joy, anger, fear and sadness dimensions. A novel dataset consisting of 47,611 English news headlines from six publishers that received more than 17 million shares and likes were retrieved using Facebook APIs over ten consecutive months in 2018. In contrast with the conventional knowledge that only high-arousal, negative emotions are associated with viral news, the data revealed that headlines with higher intensities across all five emotion dimensions (including positive, joyful news) are significantly associated with social popularity, though the emotion-popularity correlation patterns differ for different publishers (e.g., daily broadcast vs. politics-slanted publishers). From the predictive experiments, we found that the emotion features had complimentary benefits to existing features, which included strong baselines features and word embedding. The final hybrid model achieved the highest predictive performance (R^2 = .54, tau = .53; F1 = .44, AUC = .85). Using two additional publishers' data, robustness tests further showed the advantage of the proposed model against a state-of-the-art method: The Guardian (tau = .45 vs. .37) and The New York Times (tau = .46 vs. .32).

Mid-level image features have been shown to be helpful to bridge the semantic gap between low-level and high-level image representations. Many existing methods to learn mid-level visual elements consider each mid-level feature individually, and do not take their mutual relationships into account. We follow the intuitive idea that learning discriminative combinations of visual elements can help us deal with ambiguities better, and propose the concept of visual n-grams to effectively represent combinations of visual elements along with their relative spatial configuration and co-occurrence relationships. An overview of our approach is shown in Figure 1. Figure 1 (a) shows the process of learning discriminative visual n-grams based on relative spatial position, orientation and co-occurrence relationships of mid-level image patches. Figure 1 (b) further shows how these visual n-grams are used to finally learn a feature vector representing test and training images.

We present a new method to classify human activities by leveraging on the cues available from depth images alone. Towards this end, we propose a descriptor which couples depth and spatial information of the segmented body to describe a human pose. Unique poses (i.e. codewords) are then identified by a spatial-based clustering step. Given a video sequence of depth images, we segment humans from the depth images and represent these segmented bodies as a sequence of codewords. We exploit unique poses of an activity and the temporal ordering of these poses to learn subsequences of codewords which are strongly discriminative for the activity. Each discriminative subsequence acts as a classifier and we learn a boosted ensemble of discriminative subsequences to assign a confidence score for the activity label of the test sequence. Unlike existing methods which demand accurate tracking of 3D joint locations or couple depth with color image information as recognition cues, our method requires only the segmentation masks from depth images to recognize an activity. Experimental results on the publicly available Human Activity Dataset (which comprises 12 challenging activities) demonstrate the validity of our method, where we attain a precision/recall of 78.1%/75.4% when the person was not seen before in the training set, and 94.6%/93.1% when the person was seen before.

In this study, the authors present a new area‐based stereo matching algorithm that computes dense disparity maps for a real‐time vision system. Although many stereo matching algorithms have been proposed in recent years, correlation‐based algorithms still have an edge because of speed and less memory requirements. The selection of appropriate shape and size of the matching window is a difficult problem for correlation‐based algorithms. In the proposed approach, two correlation windows are used to improve the performance of the algorithm while maintaining its real‐time suitability. The CPU implementation of the proposed algorithm computes more than 10 frame/s. Unlike other area‐based stereo matching algorithms, this method works very well at disparity boundaries as well as in low textured image areas and computes a dense and sharp disparity map. Evaluations on the benchmark Middlebury stereo datasets have been performed to demonstrate the qualitative and quantitative performance of the proposed algorithm.

We present a new example-based method to colorize a gray image. As input, the user needs only to supply a reference color image which is semantically similar to the target image. We extract features from these images at the resolution of superpixels, and exploit these features to guide the colorization process. Our use of a superpixel representation speeds up the colorization process. More importantly, it also empowers the colorizations to exhibit a much higher extent of spatial consistency in the colorization as compared to that using independent pixels. We adopt a fast cascade feature matching scheme to automatically find correspondences between superpixels of the reference and target images. Each correspondence is assigned a confidence based on the feature matching costs computed at different steps in the cascade, and high confidence correspondences are used to assign an initial set of chromatic values to the target superpixels. To further enforce the spatial coherence of these initial color assignments, we develop an image space voting framework which draws evidence from neighboring superpixels to identify and to correct invalid color assignments. Experimental results and user study on a broad range of images demonstrate that our method with a fixed set of parameters yields better colorization results as compared to existing methods.

Colorization of a grayscale photograph often requires considerable effort from the user, either by placing numerous color scribbles over the image to initialize a color propagation algorithm, or by looking for a suitable reference image from which color information can be transferred. Even with this user supplied data, colorized images may appear unnatural as a result of limited user skill or inaccurate transfer of colors. To address these problems, we propose a colorization system that leverages the rich image content on the internet. As input, the user needs only to provide a semantic text label and segmentation cues for major foreground objects in the scene. With this information, images are downloaded from photo sharing websites and filtered to obtain suitable reference images that are reliable for color transfer to the given grayscale photo. Different image colorizations are generated from the various reference images, and a graphical user interface is provided to easily select the desired result. Our experiments and user study demonstrate the greater effectiveness of this system in comparison to previous techniques.

We present a new disparity refinement algorithm that utilize color information of the reference image to generate sharp disparity maps. While existing methods use iterative approaches or require multiple disparity maps, the proposed algorithm uses a single pass approach to reduce errors at depth discontinuities. The experimental results are evaluated on the Middlebury benchmark dataset; show the effectiveness of the proposed algorithm.

This paper aims to demonstrate application of type-2 fuzzy logic systems (FLS) to predict a critical parameter of gas turbine in a power plant viz., the Turbine Inlet Temperature (TIT). Maintaining higher TIT than allowed severely affects the life of the components whereas operating at lower TIT may cause low efficiency and low load. Nonavailability of TIT, which cannot be measured directly, puts great limitations on efficient gas turbine operation. Accurate estimation of this parameter requires significant computing power and the time required places limitations on the conventional modeling methods for use in real time applications. It is also demonstrated here by way of comparison, that a type-2 FLS is more robust in the presence of noise uncertainties than a type-1 conventional FLS for this application. Results are verified through the practical plant data obtained from an 88 MW gas turbine power plant.