Adversarial robustness in the context of Unsupervised Domain Adaptation (UDA) is particularly challenging due to the lack of labels in the target domain. Pseudo labels are often used to make adversarial robust models but compromise robustness and accuracy, falling short of the performance due to noise and inaccuracies in these pseudo labels. The main challenges in achieving robustness and accuracy include ensuring reliable pseudo labels and developing effective training methods that bring alignment between clean and adversarial examples of target data. To address these challenges, we propose a novel training method within the self-training paradigm Consistent Attention Mapping with Self Pseudo Label Refinement (CAM+SPLR). It begins with the pre-training of the UDA model, resulting in a UDA pre-trained model, which is initialized into two separate models: the Anchor model and the TargetNet model. The Anchor model encourages the attention maps of clean images and their adversarial counterparts to be similar, while the TargetNet model simultaneously performs self-training using Adversarial target data and refining the pseudo labels. CAM+SPLR improves both semantically relevant key features and pseudo-labels through a two-step stochastic gradient descent process during training. We conducted extensive experiments on benchmark datasets, including OfficeHome, PACS, and VisDA, demonstrating significant improvements in both robustness and accuracy. Our method achieves an average accuracy improvement of 6% and 8.1% and an average robustness improvement of 10.2% and 4.9%, compared to state-of-the-art methods on the PACS and VisDA datasets.

In the era of social media, accurate and contextually rich captions are essential for enhancing user engagement with video content. Traditional captioning methods, relying solely on visual content, often fall short of capturing audience interest. To address this, advanced captioning solutions are needed to combine visual cues with contextual insights. This article proposes a novel captioning approach that incorporates contextual information, leading to more accurate and engaging captions. By extracting human behavior, object interactions, and social cues, the method generates relevant and informative captions. To further refine the learning process, metalearning is employed to optimize a reward function. The effectiveness of the proposed method is validated on Microsoft video description (MSVD), Microsoft research-video to text (MSR-VTT), and video and text (VATEX) benchmark datasets through ablation studies and detailed experiments. The results demonstrate that contextual video captioning significantly enhances user experience. Behavioral features enrich feature representations, and context learning helps in predicting accurate and informative captions.

Despite the increasing popularity of graph neural networks (GNNs), the security risks associated with their deployment have not been well explored. Existing works follow the standard adversarial attacks to maximize cross-entropy loss within an L-infinity norm bound. We analyze the robustness of GNNs against node injection attacks (NIAs) in black-box settings by allowing new nodes to be injected and attacked. In this work, we propose to design stronger and transferable NIAs. First, we propose margin aware attack (MAA) that uses a maximum margin loss to generate NIAs. We then propose a novel margin and direction aware attack (MDA) that diversifies the initial directions of MAA attack by minimizing the cosine similarity of the injected nodes with respect to their respective random initialization in addition to the maximization of max-margin loss. This makes the NIAs stronger. We further observe that using L2 norm of gradients in the attack step leads to an enhanced diversity amongst the node features, thereby further enhancing the strength of the attack. We incorporate transferability in NIAs by perturbing the surrogate model before generating the attack. An analysis of eigen spectrum density of the hessian of the loss emphasizes that perturbing the weights of the surrogate model improves the transferability. Our experimental results demonstrate that the proposed resilient node injection attack (R-NIA) consistently outperform PGD by margins about 7-15% on both large and small graph datasets. R-NIA is significantly stronger and transferable than existing NIAs on graph robustness benchmarks.

Recent literature has demonstrated that vision transformers (VITs) exhibit superior performance compared to convolutional neural networks (CNNs). The majority of recent research on adversarial robustness, however, has predomi-nantly focused on CNNs. In this work, we bridge this gap by analyzing the effectiveness of existing attacks on VITs. We demonstrate that due to the softmax computations in every attention block in VITs, they are inherently vulnerable to floating point underflow errors. This can lead to a gradient masking effect resulting in suboptimal attack strength of well-known attacks, like PGD, Carlini and Wagner (CW) and GAMA. Motivated by this, we propose Adaptive Attention Scaling (AAS) attack that can automatically find the optimal scaling factors of pre-softmax outputs using gradient-based optimization. We show that the proposed simple strategy can be incorporated with any existing adversarial attacks as well as adversarial training methods and achieved improved performance. On VIT-B16, we demonstrate an improved attack strength of up to 2.2% on CIFAR10 and upto 2.9% on CIFAR100 by incorporating the proposed AAS attack with state-of-the-art single attack methods like GAMA attack. Further, we utilise the proposed AAS attack for every few epochs in existing adversarial training methods, which is termed as Adaptive Attention Scaling Adversarial Training (AAS-AT). On incorporating AAS-AT with existing methods, we outperform them on VITs over 1.3-3.5% on CIFAR10. We observe improved performance on ImageNet-100 as well.

Video captioning has become a broad and interesting research area. Attention-based encoder-decoder methods are extensively used for caption generation. However, these methods mostly utilize the visual attentive feature to highlight the video regions while overlooked the semantic features of the available captions. These semantic features contain significant information that helps to generate highly informative human description-like captions. Therefore, we propose a novel visual and semantic enhanced video captioning network, named as VSVCap, that efficiently utilizes multiple ground truth captions. We aim to generate captions that are visually and semantically enhanced by exploiting both video and text modalities. To achieve this, we propose a fine-grained cross-graph attention mechanism that captures detailed graph embedding correspondence between visual graphs and textual knowledge graphs. We have performed node-level matching and structure-level reasoning between the weighted regional graph and knowledge graph. The proposed network achieves promising results on three benchmark datasets, i.e., YouTube2Text, MSR-VTT, and VATEX. The experimental results show that our network accurately captures all key objects, relationships, and semantically enhanced events of a video to generate human annotation-like captions.

Emotions and gestures are essential elements in improving social intelligence and predicting real human action. In recent years, recognition of human visual actions using deep neural networks (DNNs) has gained wide popularity in multimedia and computer vision. However, ambiguous action classes, such as “praying” and “pleading,” are still challenging to classify due to similar visual cues of action. We need to focus on attentive associated features of facial expressions and gestures, including the long-term context of a video for the correct classification of ambiguous actions. This article proposes an attention-aware DNN named human action attention network (HAANet) that can capture long-term temporal context to recognize actions in videos. The visual attention network extracts discriminative features of facial expressions and gestures in the spatial and temporal dimensions. We have further consolidated a class-specific attention pooling mechanism to capture transition in semantic traits over time. The efficacy of HAANet is demonstrated on five benchmark datasets. As per our knowledge, no publicly available dataset exists in the literature, which distinguishes ambiguous human actions by focusing on the visual cues of a human in action. This motivated us to create a new dataset, known as Visual Attention with Long-term Context (VALC), which contains 32 actions with about 101 videos per class and an average length of 30 s. HAANet outperforms UCF101, ActivityNet, and BreakFast-Actions datasets in terms of accuracy.

Text mining in noisy situations, like poor contrast and fainted edges, is one of the challenging areas of research in the domain of social networks and computer vision. Scene text detection is a complicated task as text regionhaving varying span in term of size, orientation, aspect ratio, color, font, and script. Furthermore, the contrast of a scene image varies drastically in noisy situations due to poor illumination and image filtering. This faints the text edges and make the task of detection more challenging. In this article, we bring forward a semantic edge supervised spatial-channel attention network, known as SESANet, for detecting arbitrary-shaped text instances in noisy scene images with faint text edges. Our network learns multiscale (MS) supervised edge semantic, pixel-wise spatial structure information, and interchannel dependencies for precisely localizing the text masks in scene images with poor contrast and illumination. Our network is efficient, precise, and fast in nature. SESANet captures rich, dense, discriminative, and MS semantic information. The experimental results show the success of the proposed network. It shows a superior performance with regard to recall on the publicly available benchmark datasets. A new dataset scene images, named as Edge-fainted Noisy Arbitrary-shaped Scene Text (EFNAST) dataset, having varying noise density, poor contrast, low illumination, and faint edges is created.

Augmented intelligence is an innovative amplification of artificial intelligence that allows human experts to take over the autonomous decision of machines. It also facilitates human-intelligence-based decisions on the network edge using low-cost and small-sized devices. Augmented intelligence and the Internet of Things collectively create augmented intelligence of things. It logically and effortlessly interrelates human intelligence to articulate smart decisions. Unmanned aerial vehicles find various applications ranging from search operations during disasters to intruders identification; thus, they are suitable for aerial networks, where connections between base stations and servers are extinct. This article presents an approach to enhance the lifetime of unmanned-aerial-vehicles-enabled aerial networks via augmented intelligence. It first considers the available battery power to transform a large-size deep neural network into a lightweight. We next present a knowledge-distillation-based approach, which reduces training time and enhances accuracy. Finally, we evaluate the approach on the existing dataset.

With the increasing population, events with large crowds also increased. It often leads to uncontrolled stampede situations, causing several deaths. Deployment of intelligent systems with the quick alert feature may reduce the impact of stampedes. Researchers utilized traditional deep learning models on a centralized server for stampede detection. These models have high time, computational complexity, unaddressed public privacy concerns, and misclassification due to less inter-class variance. We thus propose a low-cost, fast, and intelligent system named StampSys, for accurate stampede detection over large crowds in multi-camera environment. To address complexity and privacy issues, we introduce a novel light-weight multi-modal federated learning setup. We include a novel multi-label fuzzy classifier to improve the global decision. We create a new annotated dataset, entitled CrowdStampede with 6K images. The experimentation results show that our system accurately classifies stampede situations on our dataset.

Low Power Wide Area Networks (LPWAN) offer a promising wireless communications technology for Internet of Things (IoT) applications. Among various existing LPWAN technologies, Long-Range WAN (LoRaWAN) consumes minimal power and provides virtual channels for communication through spreading factors. However, LoRaWAN suffers from the interference problem among nodes connected to a gateway that uses the same spreading factor. Such interference increases data communication time, thus reducing data freshness and suitability of LoRaWAN for delay-sensitive applications. To minimize the interference problem, an optimal allocation of the spreading factor is requisite for determining the time duration of data transmission. This paper proposes a game-theoretic approach to estimate the time duration of using a spreading factor that ensures on-time data delivery with maximum network utilization. We incorporate the Age of Information (AoI) metric to capture the freshness of information as demanded by the applications. Our proposed approach is validated through simulation experiments, and its applicability is demonstrated for a crop protection system that ensures real-time monitoring and intrusion control of animals in an agricultural field. The simulation and prototype results demonstrate the impact of the number of nodes, AoI metric, and game-theoretic parameters on the performance of the IoT network.

With the availability of low-cost and low-power sensors, it becomes easier to assess river water quality. The existing work on water quality assessment require a large amount of correctly annotated data for training. However, in the real-world scenario, obtaining such annotated data is costly and time consuming. In this work, we propose a sensor-based river water quality assessment system using the deep neural network (DNN). The system first presents a technique to estimate the water quality index (WQI) for labeling the given lab samples. WQI is a vital matrix used to transform large quantities of water data into a single unified number. Next, we present an automatic annotation technique that assigns labels to the sensory data instances using lab data. Finally, the labeled sensory data instances are used to build a DNN classifier that predicts water quality. This work also proposes a noise handling loss function to accommodate noisy labels. We evaluate the performance of the system on the river data set of major Indian rivers. We use four performance metrics during the experiment, including precision, recall, accuracy, and$F1$score. Additionally, the system achieves an accuracy of more than 90%, despite 20% noisy labels. The code is athttps://github.com/sourcecodecselab/river_water_monitoring.

The disease-free growth of a plant is highly influential for both environment and human life, as numerous microorganisms/viruses/fungus may affect the growth and agricultural production of a plant. Early detection and treatment thus becomes necessary and must be treated on time. The existing vision techniques either involve image segmentation or feature classification/regression applied over aerial images. This results in an increase in time and cost consumption due to various challenges, such as generalization ability and learning cost. Therefore, a feature-based disease detection approach with minimal learning time and generalization ability could be fairly befitting such as an extreme learning machine (ELM). In this letter, we demonstrate an algorithm, L1-ELM, after employing Kuan filtering for preprocessing and different feature computations. At the evaluation stage, the experimentation performed over benchmark plant datasets confirms that L1-ELM outperforms all existing one-class classification algorithms, preserving optimal learning and better generalization.

The ability to recognize human actions in a video is challenging due to the complex nature of video data and the subtlety of human actions. Human activities often get associated with surrounding objects and occur in specific scene contexts. Existing action recognition systems are incapable of separating human actions from representation biases, like co-occurring objects and underlying scene, which often dominate subtle human actions. In this paper, we address the issue of factorization of human actions into the activity performed by the actor, co-occurring objects, and underlying context to mitigate the influence of representation biases when they are irrelevant to the action in consideration. We propose a deep neural network architecture, denoted byFactorNet, for efficient action recognition in videos with long temporal duration. We design an attention mechanism that separates an actor from the associated objects and co-occurring scene followed by capturing long-range temporal context. We perform a comprehensive set of experimentation on six benchmark datasets to show the efficacy of our architecture. To train a model using recent video-based action datasets certainly capture and leverages such bias. The supervised representation may not be competent to new action classes. We therefore design a new dataset, known asFactNet, which consists of activity-object-scene related actions that occur in day-to-day applications. Dataset Link: FactNet.

Industrial IoT (IIoT) accomplishes digital manufacturing that incorporates various devices, simulators, and tools with multiple sensors. These sensors provide sufficient data to coordinate and monitor industrial systems. IIoT requires dedicated supporting devices for an application to execute a given task in maximum allowable response time. The requirement of dedicated devices increases system cost. Task containerization is a process of exploiting available resources of the host machines to meet out varying demands of different applications. It avoids the additional cost to buy dedicated IIoT devices while adding new applications. This article proposes an approach to securely process a given IIoT task within an allowable response time. We use the game theory approach to estimate the fractions of the task to be containerized on the machines. Next, the estimated fractions for each machine maximize the system utility. Finally, we illustrate the experimental results to validate the performance of the proposed approach.

Intelligent systems, like driver assistance systems, remain within vehicles and help drivers by providing essential information about traffic, blockage of roads, and possible routes for safe driving. The objective of scene text spotting in a driver assistance system is to localize and recognize scene texts, signs of milestones, traffic panels, and road marks in natural scene images. However, text edges get fainted due to adverse weather conditions, like fog, rain, smog, or poor contrast. This makes the task of spotting more challenging. In this paper, we propose an end-to-end trainable deep neural network, known asMet-MLTS, that can address the issue of spotting multi-oriented text instances in scene images captured in adverse meteorological conditions. It localizes words, predicts script class, and performs word spotting for every rotated bounding box. It is a fast multilingual scene text spotter that utilizes hierarchical spatial context, channel-wise inter-dependencies, and semantic edge supervision to localize and recognize words and predict script class in scene images using smartphones. We explore inter-class interference to reduce the misclassification problem. A light-weight recognition module for multilingual character segmentation, word-level recognition, and script identification is incorporated. We demonstrate the efficacy of our spotting network on resource-constraint devices.

Various wireless local area network technologies are developed for Internet of Things of which Long-Range Wide Area Network is preferred for low power long range communication. In the LoRa network, multiple LoRa nodes can simultaneously communicate with a LoRa gateway which causes the interference problem. This article presents an approach for estimating the association time duration between each LoRa node and the LoRa gateways for transmitting the data of end users to the LoRa gateways with high packet delivery ratio. The approach uses a Beta distribution based reputation model for estimating the association time duration between each LoRa node and LoRa gateways and Bayesian Game strategy which accommodates unknown private information of the LoRa nodes. The approach is validated by simulating the LoRa network using network simulator-3. We also demonstrate an on-campus traffic monitoring system to detect the reckless driving action and estimate the vehicle speed using sensors embedded nodes deployed along both sides of the road.

With the availability of various sensors in the smartphone, identifying a locomotion mode becomes convenient and effortless in recent years. Information about locomotion mode helps to improve journey planning, travel time estimation, and traffic management. Though there exists a significant amount of work towards locomotion mode recognition, the performance of these work is not pertinent and heavily depends on the labeled training instances. As it is impractical to gather a prior information (labeled instances) about all types of locomotion modes, the recognition model should be able to identify a new or unseen locomotion mode without having any corresponding training instance. This paper proposes a sensors based deep learning model to identify a locomotion mode by using labeled training instances. The approach also incorporates a concept of Zero-Shot learning to identify an unseen locomotion mode. The model obtains an attribute matrix based on the fusion of three semantic matrices. It also constructs a feature matrix by extracting the deep learning and hand-crafted features from the training instances. Later, the model builds a classifier by learning a mapping between attribute and feature matrices. Finally, this work evaluates the performance of the approach on collected and existing datasets using accuracy and F1 score.

An important research issue in smart metering is to correctly transfer the smart meter readings from consumers to the operator within the given time period by consuming minimum energy. In this paper, we propose an energy efficient smart metering system using Edge computing in Long Range (LoRa). We assume that all appliances in a house are connected to a smart meter that is affixed with Edge device and LoRa node for processing and transferring the processed smart meter readings, respectively. The energy consumption of the appliances can be represented as an energy multivariate time series. The system first proposes a deep learning based compression-decompression model for reducing the size of the energy time series at the Edge devices. Next, it formulates an optimization problem for finding the suitable compressed energy time series to reduce the energy consumption and delay of the system. Finally, the system presents an algorithm for selecting the suitable spreading factors to transfer the compressed time series to the operator in the given time. Our simulation and prototype results demonstrate the impact of the parameters of the compression model, network, and the number of smart meters and appliances on delay, energy consumption, and accuracy of the system.

Network traffic classification has been an interesting topic of research for many years. It plays a crucial role in many network applications including resource allocation, intrusion detection, and quality of service. Network traffic is essentially a sequence or flow of time-stamped packets that are exchanged between two devices. The traffic flow also contains payload data along with information about packet statistics such as size, inter-arrival time, and direction. As these statistics are obtained from the time-stamped packets, they form a Multivariate Time Series (MTS). Such an MTS needs to be classified as early as possible to identify an Internet application associated with the generated traffic flow. In this paper, we propose an Early traffic Flow Classification (EFC) approach for identifying Internet applications using MTS. The approach estimates application-wise minimum required packets from the training data by employing k-means clustering and Long Short Term Memory model. We also develop a class forwarding method to utilize correlation that exists among different packet statistics. Additionally, we collect a real-world traffic flow dataset to evaluate the effectiveness of the approach. Experimental results show that EFC approach requires only the first 15 packets of the flow to achieve an accuracy of more than 90%.

With the expeditious evolution in technology, recent years have witnessed significant growth in passenger assistance applications in the transportation system. Such applications have varying demands for resources and quality of services. This paper presents a Fog computing based transportation system. The system uses multiple Fog devices to provide assistance to the passengers. The passengers and vehicles work as end-users and the Edge devices in the system, respectively. A passenger generates a task and using the Edge device forwards it to the Fog devices for further processing. Selected Fog devices parallel process the fraction of the task, so that the complete task processes within the given time constraint. We use the gamma function based reputation model of Fog devices, which provides the confidence to complete a given task successfully. We present a Knapsack based task offloading algorithm, which helps to fully utilize the resources of the Fog devices. We also present a competitive game model and near Nash Equilibrium solution for estimating the optimal value of the fraction of the task process at Fog devices. Finally, we develop a prototype and present results to investigate the performance of the propose system.

Coronavirus outbreak is one of the challenging pandemics for the entire human population on Earth. Techniques, such as the isolation of infected people and maintaining social distancing, are the only preventive measures against the pandemic. The actual estimation of the number of infected peoples with limited data is an indeterminate problem faced by data scientists. There are several techniques in the existing literature, including reproduction number and case fatality rate, for predicting the duration of a pandemic and infectious population. This article presents a case study of different techniques for analyzing, modeling, and representing the data associated with a pandemic such as COVID-19. We further propose an algorithm for estimating infection transmission states in a particular area. This work also presents an algorithm for estimating end time of a pandemic from the susceptible infectious and recovered model. Finally, this article presents the empirical and data analysis to study the impact of transmission probability, rate of contact, infectious, and susceptible population on the pandemic spread.

Getting real-time information about the operational behavior of industrial or domestic appliances becomes effortless with the availability of sensors. The sensors generate multiple streams of measurements, called as multivariate time series, corresponding to an operation of the appliance. An appliance can go through various types of faults during its lifetime. Such a fault can be identified by classifying the multivariate time series (MTS), which is generated by the sensors corresponding to this fault. As it is also unfeasible to have prior knowledge about all types of faults, the classification approach should also be able to identify an unseen (unknown) fault using its MTS. In this article, we propose a semantic-information-based early classification approach for MTS. The approach uses a concept of zero-shot learning to classify an unseen fault. This work conducts a case study to evaluate the approach by classifying different faults of a washing machine using sensory data.

Activity classification has been an interesting area of research for many years, to better understand human behavior. Recent advancements in embedded computing systems allowed the emergence of several state-of-art solutions for human activity classification using sensors of a smartphone. The sensors generate temporal sequences of observations for human activity, which is called as Multivariate Time Series (MTS). Current state-of-art solutions for human activity classification suffer from two major limitations: first, the length of testing MTS should be equal to the training MTS and second, the MTS should not have any faulty time series. In real-time applications, it is desirable to classify a human activity using an incomplete MTS as early as possible. In this work, we propose a fault-tolerant early classification of MTS (FECM) approach to address these limitations. FECM builds a set of classification models using MTS training dataset. The approach employs Gaussian Process classifier to estimate minimum required length of time series, which is used to predict a class label of new MTS. Further, FECM uses an Auto Regressive Integrated Moving Average model to identify faulty time series in the new MTS. Finally, we conduct an experiment to evaluate the performance of FECM using accuracy and earliness metrics.

Long Range (LoRa) is a wireless communication technology which enables Internet of Things (IoT) devices to efficiently and robustly communicate over long distances with low power consumption. The LoRa supports six Spreading Factors (SFs) and therefore a limited number of virtual channels are possible at a given time instance. Such limited number of channels restate the simultaneous use of a LoRa Gateway (LG) by the large number of LoRa Nodes (LNs) in a dense LoRa network and create the bottleneck problem at the LG. The bottleneck problem reduces the proper use of the allocated SFs and the utility of the LNs. In this paper, we propose a game theoretic approach that allocates the time duration to the LNs in the network for accessing the SFs. Such time duration maximizes the utility of the LNs in the network. The time duration determined on different SFs are then scheduled to minimize the waiting time of LNs. The proposed approach is validated by simulating the LoRa network using Network Simulator-3. Our simulations show that the proposed approach effectively reduces the waiting time and prolongs the network utility.

Recent years have witnessed unprecedented growth in sensors-based indoor activity recognition. Further, a significant improvement in recognition performance of indoor activities is observed by incorporating Deep Neural Network (DNN) model. In this paper, we propose knowledge distillation based economic and efficient indoor activity recognition approach for low-cost resource constraint devices. Here, we adopt knowledge from teacher and trainee (cumbersome DNN models) for training student (compressed DNN model). Initially, student and trainee both are beginner and trainee helps the student in learning from the teacher. The student, after certain steps, is mature enough for directly learning from the teacher. We introduce an early halting mechanism for simultaneously reducing floating-point operations and training time of the student model.

In the era of the Internet of Things (IoT), the sensor-based devices produce the Multivariate Time Series (MTS). A classification approach helps to predict the class label of an incoming MTS. Due to the large dimension and different sampling rate of the sensors in a given MTS, a classifier takes time to predict the class label. Some IoT applications may require early prediction of the class label where the classifier starts the prediction once the minimum number of data points are collected. In this article, we address the problem of early prediction of the class label of an MTS in IoT. This work considers the sensors with different sampling rate to generate the MTS. Each sensor generates a time series (component) of the MTS. We propose a Divide-and-Conquer–based early classification approach for classifying such MTS. The approach constructs an ensemble classifier using a probabilistic classifier and hierarchical clustering. The ensemble classifier employs a Divide-and-Conquer method to handle the different sampling rate components during the prediction of class label. The experimental results show that our approach significantly outperforms the existing approaches on real-world datasets using various evaluation metrics.

An important issue of research in the transportation system is timely classification of the inside-outside environment of the vehicle using sensors. The sensors generate the multivariate time series data, which requires a classification technique to classify it in real-time. Road surface classification is an example, where multivariate time series data can be used for early identification of the type of road surface. The challenge is to maintain the accuracy of the classification using a minimum number of data points of the multivariate time series. This work proposes an early classification approach for multivariate time series with a desired level of accuracy. It is assumed that the number of samples in the time series are not equal for a given period of time due to different type of sensors. Gaussian Process learning method is used to first estimate the minimum required length of the time series which helps to build an ensemble classifier with a desired level of accuracy. The ensemble classifier is used to predict the class label of an incoming multivariate time series. This work demonstrates a road surface classification system using the built ensemble classifier. Finally, the ensemble classifier is also evaluated on the various existing datasets from other domains. The results demonstrate the significance of early classification approach using accuracy, earliness, and confusion matrix, with the minimum required data points.

Wireless Local Area Networks (WLANs) are one of the most popular networks for the Internet-of-Things (IoT) applications. Among various WLAN technologies, the Long-Range WAN (LoRaWAN) has gained a high demand in recent years because of its low power consumption and long-range communication. However, the Long-Range (LoRa) network suffers from interference problem among LoRa Devices (LDs) that are connected to the LoRa gateway by using the same Spreading Factors (SFs). In this article, we propose a game theory-based approach for estimating the time duration of transmission of data on suitable SFs such that interference problem is reduced and network devices maximize their utilities. We next propose a scheduling algorithm that schedules the allocated time duration on the SFs such that the waiting time of the network can be minimized. We finally use the network simulator-3 for validating the propose work. Various experiments are performed which demonstrate the improvement in the network performance.

Ubiquitous Computing (UbiComp) is a computational paradigm that enhances the use of computing devices by making them available to the user anywhere and anytime. From the energy perspective, it is often very important to compute the entire UbiComp task within a specific deadline with minimum energy. The literature on determining the energy consumption of the system for computing the task does not consider periodic tasks and different sampling rate of the sensors, which eliminates the deadline constraints in the analysis. Since the period of the tasks is not fixed, the estimated delay without considering the fixed period is lower than the actual value. In this paper, we assume that an Edge, Fog, and Cloud layers based UbiComp system computes the periodic task within the specific deadline. We derive the expressions of total delay and energy consumption of the UbiComp system. Using the derived expressions, we estimate fractions of the task that are computed at each layer to reduce the energy consumption such that the task is computed within a specific deadline. Our numerical and prototype results demonstrate the impact of the data size, network topologies, deadline, and characteristics of the sensors on the energy consumption, delay, and accuracy of the system.

Classification of Multivariate Time Series (MTS) data has been an important area of research for many years. In time-critical applications, such as health informatics, fire detection, and disaster forecasting, it is desirable to classify the MTS data as early as possible. This work proposes an early classification approach to classify an incoming MTS. The early classification approach helps to predict the class label of an incoming MTS without waiting for the full length. Different from the existing work, this work considers that sampling rate of the sensors which generated the MTS is different. The performance of the approach is evaluated on a publicly available dataset using accuracy, earliness and energy consumption.

Internet of Things (IoT) finds their applications in many areas that include environmental monitoring, industrial control, traffic congestion control, smart metering, and smart parking. An important issue of research in IoT is to successfully transmit the data to the cloud and yet minimize the energy consumption of the battery-powered IoT devices. Long Range (LoRa) is a long-range wireless communication protocol that competes against other low-power wide-area networks. In this paper, we propose a Stackelberg Game based model for allocation of the appropriate power levels to the LoRa nodes in energy-efficient LoRa network. The utility of the network server, works as a leader player, is to successfully receive the data from the LoRa nodes. The utility of the LoRa nodes, work as followers player, is to reduce the power consumption during transmission of the data to the LoRa gateway. Simulation results evaluate the performance of the proposed game model to validate its effectiveness.

Water is one of the necessary things required for living. Sensor networks play a major role in detecting the pollution level of the river water. Detection of the sudden changes in the pollution level of the river water is a challenging problem because it requires continuous monitoring of the river water. In this paper, we propose a Stackelberg Game based system to detect the sudden changes of the pollution level of the river water. We use Long Range (LoRa) technology for transferring the sensory data to the server. The LoRa uses different Spreading Factors which helps to reduce the communication energy and enhances the lifetime of the system. We demonstrate the utility of the system and study the impact of the sudden changes of the pollution level of the water on the energy consumption of the system.

Scene text detection is an important as well as challenging problem in computer vision. Text information plays an important role in scene understanding, image indexing, and indoor navigation. Deep neural networks are being widely used due to their capability to learn strong text features. However, the state-of-the-art scene text detection methods capture weak text features in the early layers with no scope to improve the captured features. In this paper we propose a novel recurrent architecture to improve the learnings of a feature map at a given time, by using global and local information from the same feature map at the previous time, for detecting occluded texts and long words. We design the recurrent text convolutional layer for seamless integration of recurrent architecture with local-global map. The experimental results on publicly available scene text datasets show the efficiency of our framework. We also create our own scene text dataset.

Scene text detection systems detect texts in natural scene images. Hazy scene text detection is a specific case of scene text detection where detection is done in hazy weather conditions. Haze affects the contrast of the image. In this paper, we reframe the traditional two class hazy scene text detection problem into a four class problem. We develop a deep learning based model that combines features from all layers for accurate and fast text detection from hazy images. In addition, we develop a novel training approach for the four class problem. Merging and patch-NMS are used as post processing steps for fast word detection. We also create a new dataset of hazy scene images and obtain significant improvements on an existing hazy scene text dataset.

Scene text detection has attracted great interest from the computer vision and pattern recognition communities since text information plays an important role in image indexing and scene understanding. Deep neural networks have become popular for the task of scene text detection, especially for their ability to learn strong text features. However, existing deep learning based state-of-the-art scene text detection methods detect texts only from a single feature map which is unable to capture semantic information at all scales. In this paper, we propose a novel deep learning based model that leverages the pyramid structure of feature maps for accurate scene text detection. We also design a deep convolutional neural network model for non-maximum suppression. In addition, we develop a novel loss function and training method for end-to-end training. The experimental results validate that our end-to-end system is simple, fast, and achieves high accuracy on standard datasets, namely, ICDAR 2015 and MSRA-TD500. We also create a dataset for scene text detection.

Recent advances in sensor technology and ubiquitous computing have sustained them as a better alternative for kids activity monitoring system. This paper presents a system that continuously monitors the proper pencil grip and writing activity of kid using accelerometer and pressure sensors. The system creates a labeled dataset for recognizing the correct location of pencil grip, holding direction of the pencil, and writing activity of kids. The system uses a supervised machine learning technique and labeled dataset for classifying whether a kid properly uses a pencil or not. In addition, a prototype system is developed and adequately tested on real-time user data. The prototype uses accelerometer and pressure sensors for extracting the writing activities. The system uses bluetooth low energy for wirelessly transferring the sensed data of writing activities to the parent and teachers. The system is suitable for kids due to its low cost, small size, and low-power consumption.

Animation of videos is a modern form of art. The processing of videos of natural scenes is a great way to produce fast and aesthetically pleasing animations. However, the animation process leads to loss of many details in the produced video. In addition, the process of animating a video is time-consuming, especially in resource constrained devices like smart devices. In this paper, we propose a novel framework to animate videos while preserving their text content. A multiplayer simultaneous extensive-form game based on Markov decision process is used to classify the candidate text regions either as text or non-text. A fast animation technique using optimized glass painting is designed to efficiently animate videos with scene text using smart devices. We have used two public datasets, namely, ICDAR 2013 and Hua's video datasets as well as our own dataset created from videos available from Google, to test our framework. The framework is implemented in different smart devices to show its efficiency.

Recent advances in robotics and low-power embedded systems made three-dimensional (3D) mobile wireless sensor networks (MSNs) an effective solution for monitoring a field of interest (FoI). From a cost perspective, it is often important to ensure the desired coverage ratio for the FoI within a maximum allowable response (MAR) time, by using a minimum number of sensors in MSNs. The literature on determining the minimum number of sensors for the desired coverage ratio assumes that the FoI is unbounded to overcome the border effects. Since the entire sensing sphere of the sensors near the boundary may not be useful for the coverage, the number of sensors estimated without the border effects is lower than the actual value. In this paper, we estimate the minimum number of sensors required to achieve a desired coverage ratio in a given MAR time for a 3D FoI. We term this problem (α; V; T)-coverage problem, where a, V , and T are the desired coverage ratio, average speed of sensors, and MAR time, respectively. We assume straight line mobility model for the sensors and consider the border effects while deriving the expected sensing volume of a sensor useful in coverage. We also consider the restriction of sampling rate of the sensors in this analysis. We discuss the application of our analysis for a non-hyper-rectangle shaped FoI, random walk, and waypoint mobility models, and also the impact of neglecting the border effects. Our numerical and simulation results demonstrate the significance of border effects on the number of sensors and also the relationship between the coverage ratio, MAR time, sampling period, and the sensing range.

A recognition system is very useful to recognize human, object, and animals. An animal recognition system plays an important role in livestock biometrics, that helps in recognition and verification of livestock in case of missed or swapped animals, false insurance claims, and reallocation of animals at slaughter houses. In this research, we propose a fast and cost-effective animal biometrics based cattle recognition system to quickly recognize and verify the false insurance claims of cattle using their primary muzzle point image pattern characteristics. To solve this major problem, users (owner, parentage, or other) have captured the images of cattle using their smart devices. The captured images are transferred to the server of the cattle recognition system using a wireless network or internet technology. The system performs pre-processing on the muzzle point image of cattle to remove and filter the noise, increases the quality, and enhance the contrast. The muzzle point features are extracted and supervised machine learning based multi-classifier pattern recognition techniques are applied for recognizing the cattle. The server has a database of cattle images which are provided by the owners. Finally, One-Shot-Similarity (OSS) matching and distance metric learning based techniques with ensemble of classifiers technique are used for matching the query muzzle image with the stored database.A prototype is also developed for evaluating the efficacy of the proposed system in term of recognition accuracy and end-to-end delay.

Digital watermarking has received much attention in recent years as a promising solution to copyright protection. Video watermarking in compressed domain has gained importance since videos are stored and transmitted in a compressed format. This decreases the overhead to fully decode and re-encode the video for embedding and extraction of the watermark. High Efficiency Video Coding (HEVC/H.265) is the latest and most efficient video compression standard and a successor to H.264 Advanced Video Coding. In this article, we propose a robust watermarking framework for HEVC-encoded video using informed detector. A readable watermark is embedded invisibly in P frames for better perceptual quality. Our framework imposes security and robustness by selecting appropriate blocks using a random key and the spatio-temporal characteristics of the compressed video. A detail analysis of the strengths of different compressed domain features is performed for implementing the watermarking framework. We experimentally demonstrate the utility of the proposed work. The results show that the proposed work effectively limits the increase in video bitrate and degradation in perceptual quality. The proposed framework is robust against re-encoding and image processing attacks.

The current 2G and 3G network infrastructures were not originally designed with multimedia transmission in consideration. Unlike data transmission, multimedia has significantly different Quality of Service requirements. It requires significantly higher bandwidth in comparison with data transmission. Multimedia transmission over handheld devices (such as mobile phones) poses further challenges in maintaining a good Quality of Experience as wireless networks are inherently of low bandwidth and relatively unreliable. Handheld devices are resource constraint devices and therefore huge processing at the handheld devices may introduce delay in the multimedia reception. In this paper, we present an algorithm for bandwidth aggregation for multimedia transmission from a handheld device using the available bandwidth of other handheld devices. We use a distributed approach to play a non-zero sum game among the handheld devices to minimize the end-to-end delay and maximize the throughput. We incorporate a cooperative game to dynamically add or delete other handheld devices based on the requirements of the network. We validate our work and demonstrate its benefits.

One of the challenging issues in video watermarking is robustness against collusion attacks. To resist collusion attack, the same video scene should carry the same watermark, whenever and wherever it appears in the video. Inter-frame correlation is more within a short video neighborhood when no scene change is detected. Motion coherency has recently been recognized as a desirable property for watermarks to resist temporal frame averaging attacks. To the best of our knowledge, motion coherent watermarking in compressed domain has not yet been well explored. In this paper, a compressed domain technique has been proposed to detect motion coherent regions within a short video neighborhood. The time complexity of the proposed method is discussed. Simulation results evaluate the effectiveness of the proposed method.

The security has become an important issue in multimedia applications. The embedding of watermark bits in compressed domain is less computationally expensive as full decoding and re-encoding is not required. The motion coherency is an essential property to resist temporal frame averaging based attacks. The design of motion compensated embedding method in compressed domain is a challenging task. As far we know, no such embedding method is explored yet. In this paper, we propose a motion compensated compressed domain embedding method within a short video neighborhood that gives acceptable visual quality, embedding capacity, and robustness. The simulation results show the effectiveness of the proposed method.