Emotions and gestures are essential elements in improving social intelligence and predicting real human action. In recent years, recognition of human visual actions using deep neural networks (DNNs) has gained wide popularity in multimedia and computer vision. However, ambiguous action classes, such as “praying” and “pleading,” are still challenging to classify due to similar visual cues of action. We need to focus on attentive associated features of facial expressions and gestures, including the long-term context of a video for the correct classification of ambiguous actions. This article proposes an attention-aware DNN named human action attention network (HAANet) that can capture long-term temporal context to recognize actions in videos. The visual attention network extracts discriminative features of facial expressions and gestures in the spatial and temporal dimensions. We have further consolidated a class-specific attention pooling mechanism to capture transition in semantic traits over time. The efficacy of HAANet is demonstrated on five benchmark datasets. As per our knowledge, no publicly available dataset exists in the literature, which distinguishes ambiguous human actions by focusing on the visual cues of a human in action. This motivated us to create a new dataset, known as Visual Attention with Long-term Context (VALC), which contains 32 actions with about 101 videos per class and an average length of 30 s. HAANet outperforms UCF101, ActivityNet, and BreakFast-Actions datasets in terms of accuracy.

Text mining in noisy situations, like poor contrast and fainted edges, is one of the challenging areas of research in the domain of social networks and computer vision. Scene text detection is a complicated task as text regionhaving varying span in term of size, orientation, aspect ratio, color, font, and script. Furthermore, the contrast of a scene image varies drastically in noisy situations due to poor illumination and image filtering. This faints the text edges and make the task of detection more challenging. In this article, we bring forward a semantic edge supervised spatial-channel attention network, known as SESANet, for detecting arbitrary-shaped text instances in noisy scene images with faint text edges. Our network learns multiscale (MS) supervised edge semantic, pixel-wise spatial structure information, and interchannel dependencies for precisely localizing the text masks in scene images with poor contrast and illumination. Our network is efficient, precise, and fast in nature. SESANet captures rich, dense, discriminative, and MS semantic information. The experimental results show the success of the proposed network. It shows a superior performance with regard to recall on the publicly available benchmark datasets. A new dataset scene images, named as Edge-fainted Noisy Arbitrary-shaped Scene Text (EFNAST) dataset, having varying noise density, poor contrast, low illumination, and faint edges is created.

With the increasing population, events with large crowds also increased. It often leads to uncontrolled stampede situations, causing several deaths. Deployment of intelligent systems with the quick alert feature may reduce the impact of stampedes. Researchers utilized traditional deep learning models on a centralized server for stampede detection. These models have high time, computational complexity, unaddressed public privacy concerns, and misclassification due to less inter-class variance. We thus propose a low-cost, fast, and intelligent system named StampSys, for accurate stampede detection over large crowds in multi-camera environment. To address complexity and privacy issues, we introduce a novel light-weight multi-modal federated learning setup. We include a novel multi-label fuzzy classifier to improve the global decision. We create a new annotated dataset, entitled CrowdStampede with 6K images. The experimentation results show that our system accurately classifies stampede situations on our dataset.

The ability to recognize human actions in a video is challenging due to the complex nature of video data and the subtlety of human actions. Human activities often get associated with surrounding objects and occur in specific scene contexts. Existing action recognition systems are incapable of separating human actions from representation biases, like co-occurring objects and underlying scene, which often dominate subtle human actions. In this paper, we address the issue of factorization of human actions into the activity performed by the actor, co-occurring objects, and underlying context to mitigate the influence of representation biases when they are irrelevant to the action in consideration. We propose a deep neural network architecture, denoted byFactorNet, for efficient action recognition in videos with long temporal duration. We design an attention mechanism that separates an actor from the associated objects and co-occurring scene followed by capturing long-range temporal context. We perform a comprehensive set of experimentation on six benchmark datasets to show the efficacy of our architecture. To train a model using recent video-based action datasets certainly capture and leverages such bias. The supervised representation may not be competent to new action classes. We therefore design a new dataset, known asFactNet, which consists of activity-object-scene related actions that occur in day-to-day applications. Dataset Link: FactNet.

Intelligent systems, like driver assistance systems, remain within vehicles and help drivers by providing essential information about traffic, blockage of roads, and possible routes for safe driving. The objective of scene text spotting in a driver assistance system is to localize and recognize scene texts, signs of milestones, traffic panels, and road marks in natural scene images. However, text edges get fainted due to adverse weather conditions, like fog, rain, smog, or poor contrast. This makes the task of spotting more challenging. In this paper, we propose an end-to-end trainable deep neural network, known asMet-MLTS, that can address the issue of spotting multi-oriented text instances in scene images captured in adverse meteorological conditions. It localizes words, predicts script class, and performs word spotting for every rotated bounding box. It is a fast multilingual scene text spotter that utilizes hierarchical spatial context, channel-wise inter-dependencies, and semantic edge supervision to localize and recognize words and predict script class in scene images using smartphones. We explore inter-class interference to reduce the misclassification problem. A light-weight recognition module for multilingual character segmentation, word-level recognition, and script identification is incorporated. We demonstrate the efficacy of our spotting network on resource-constraint devices.