In recent times, plasmonic sensing is widely used for detecting tiny particles (micro / nano-scale) and plays a crucial role in diverse application domains such as sustainability, healthcare etc. In current scenario, numerical simulators are used for predicting the optical response of a plasmonic nanostructure by solving Maxwell’s equations. These simulators are highly expensive and extremely time-consuming. We are proposing an alternative of numerical simulators using DL-based generative model to predict optical response against a geometrical structure of plasmonic sensor with reduced computing time. Our method is based on a variant of variational auto-encoder (VAE) constrained to follow the resonance peak of the optical response during training. Our method is capable of predicting the optical response from a diverse set of plasmonic nano-structures. We have demonstrated the performance using publicly available optical ring resonator an H-shaped nano-structure geometry data.

The growing use of wearable devices requires accurate and compact representations of high dimensional physiological signals. This work presents a UNet inspired autoencoder to represent and reconstruct multiple neuro-physiological signals from single channel data. The architecture comprises single-encoder/dual-branched decoders to obtain self-attention enabled compact embeddings of mixed ExG (EEG /ECG) signals through decaying encoder-decoder skip connections, for improved representation capability. The embeddings are separable into individual ExG components enabling simultaneous reconstruction of high fidelity EEG and ECG sources. The pretrained encoder can be used for a complex downstream task with minimum fine-tuning. Using the proposed method on a large corpus of single-channel mixed ExG generated from overnight Polysomnography (PSG) recordings, we show subject- and class- independent EEG/ECG reconstructions validated by multiple domain-specific metrics, and evaluate the classification performance of the encoded EEG embeddings into five sleep stages as a downstream task.

Deep Neural Networks (DNNs) have significantly enhanced the baseline performance of Human Activity Recognition (HAR) models by learning patterns directly from raw sensor signals. For wearables, on-device inference is crucial for preserving personally identifiable information and ensuring long battery life, particularly in HAR-based health and wellness applications. However, existing DNN models for HAR are often either too large for wearable devices, designed for simple binary activity classification, or rely on manually crafted features. To address these challenges, we propose a scalable solution comprising a tunable tiny attention condenser-based architecture and a Deep Q-Learning-based tuner. Together, these components enable the generation of compact, wearable-friendly DNN models for complex activity recognition tasks. Experimental results demonstrate that the proposed methodology achieves state-of-the-art accuracy on the HAR-Box and UCI-HAR datasets, with model sizes under 1 MB, making it suitable for resource-constrained devices.

Deep learning models for edge deployments must be small, efficient, and robust. Knowledge distillation from large foundation models (FMs) can help, as they capture rich, transferable representations from large, multimodal datasets. However, two key challenges hinder this process: (1) extreme model compression based on a test dataset often leads to brittleness under distribution shifts, and (2) the distribution gap between an FM’s training data and a small model’s target dataset makes standard distillation methods ineffective.We propose a tunable dynamic loss curriculum for knowledge distillation to address these issues. Experiments show that small encoders trained with our approach achieve balanced transfer learning performance across both primary and out-of-distribution tasks. For instance, a tiny GPT-like model effectively transfers to sentiment classification and language modeling. Likewise, a tiny ResNet trained on CIFAR-10 achieves 10% higher accuracy on corrupted CIFAR-10 than state-of-the-art baselines for tiny models. While it trails dedicated robustness training by 4%, our method ensures superior adaptability across diverse datasets and tasks.

In this work we present an experimental demonstration of the comparative study of imaging quality of one of the non-uniform sized (NUS) optically-sparse aperture (OSA) mirror configurations - the Taylor-ln design as proposed in [1] against that of a conventional segmented mirror having sub-apertures of equal size, in laboratory. The perfomance of the proposed NUS OSA mirrors are noted to be equivalent to those of uniform-sized segmented mirrors, using simulations in the ideal scenario of noiseless imaging systems. Therefore, it is of utmost importance to demonstrate the efficiency of such NUS OSA mirror configurations, in the laboratory frame at least, so that real-life applications of these designs can be surmised. Therefore, in this paper we design masks that mimic an uniform sized segmented mirror with each sub-aperture as large as 2 cm and a NUS OSA mirror whose aperture sizes vary as the coefficients of the Taylor expansion series for ln. We consider only the average intensity information of the images captured by these two types of segmented mirrors. Thereafter, simple deconvolution with respective Point Spread Functions (PSFs) results in nearly similar reconstructed images, at least by visual inspection. In case of the uniform sized segmented mirror the Peak Signal-to-Noise-Ratio (PSNR) is 7.8 dB alongwith the Structural Similarity Index Measure (SSIM) being 0.1. The respective values for the NUS system are similar (8.8 dB and 0.24, respectively). This explicitly demonstrates very little degradation of performance even though the fill factor for the NUS design is reduced to 31% of the uniform OSA mirror. However, we emphasize on the fact that this is a very basic experimental setup, demonstrating the efficiency of imaging for such NUS systems and suggest further improvement of the reconstructed image quality with better quality mirrors used for the purpose as well as incorporating advanced reconstruction algorithms coupled with a suitable denoising algorithm. The success of this kind of an experiment is a promising breakthrough to encourage implementation in real-life ultra-lightweight space-based telescopic systems, encompassing a wide plethora of scientific objectives viz., astronomy, natural resources as well as infrastructure monitoring, security and surveillance and even disaster prediction like earthquakes.

Any high spatial resolution space-borne electro-optical sensing system operating in long wavelengths, like Earth-observation facilities operating in the Longwave Infrared are subjected to an inherent design and implementation challenge of deploying large monolithic primary aperture mirrors. To outflank this issue, many present-date missions design and commission lightweight segmented mirrors, mostly with equal sized sub-apertures. To go one step ahead, these sub-apertures could be of non-uniform sizes, in a given proportion, thereby ensuring an even smaller and lighter primary, with a marginal compromise in imaging quality. However, a fragmented mirror is very sensitive to failure of one or more of the sub-apertures, in some random fashion (most likely being hit by space debris or surge of high energy particles) or in a cluster (possibly due to failure of the actuators or control system). In such scenarios, the system suffers loss of the high-frequency components to various extents, so that the edge detection in the reconstruction process (deconvolution) becomes questionable. This is particularly true for the central sub-aperture, rendering it as the most critical component in the partially-filled primary aperture design. In such cases, the imaging system malfunctions as a whole, thereby defeating the purpose of designing of such ultra-lightweight optically-sparse primary aperture mirror with non-uniform sized sub-apertures for satellite-based sensing systems.

Tracking sugar intake has become a trending practice, much like keeping track of calorie intake. However, consumer-grade systems or devices for measuring or detecting sugar levels of unlabeled food items are close to none. In this paper, we discuss how low-cost Multi-Spectral Imaging (MSI) technology can be used to identify the sugar level of grapes in a simple, yet effective manner. Here, grapes are used as a representative for fruit or food. Freshly harvested grapes were used for spectral data collection. Unispectral portable multispectral camera EVK – UNS52000 which has a spectral range of 700-940 nm across 10 spectral bands was used for data collection. Actual sugar content was determined using a handheld refractometer and used for reference. We evaluated the performance of two well-known machine learning-based regression algorithms such as Random Forest (RF) and Light Gradient Boosting Machines (LightGBM). Spectral data derived from images captured in 10 bands were used as features or independent variables and actual sugar measured using a refractometer was used as a reference or the dependent variable in regression models. We tested the model performance using different feature scenarios: a) All 10 spectral bands, b) Top 5 spectral bands and c) Top 3 spectral bands. Results showed that R2 was 0.92 for both RF and LightGBM with negligible variation in RMSE (1.05-1.06 °Brix) when using all 10 bands. There was minimal difference in terms of R2 and RMSE. However, there was a significant difference in model training time with LightGBM being much faster than RF. Moreover, by selecting the top 5 and 3 features, we reduced the R2 to 0.89 and 0.78 respectively in the case of LightGBM. While there is a reduction in R2, selecting the top 5 bands will be useful from an operational perspective in terms of computational power and cost of device development. We aim to design and develop the device using the selected top 5 bands for operational on-the-fly prediction of sugar content in grapes. The proposed approach could be helpful for various stakeholders such as people with Hypoglycemia or Diabetes, grape producers to gauge the maturity of grapes, and industrial use cases for vineyards.

Post-pandemic, wellness and healthcare sector is vocal about balanced food consumption, inclusion of fresh fruits and regular exercise. Freshness and sugar contents of fruits are crucial to understand before their consumption. A normal RGB camera can provide information about the freshness of fruits based on the surface composition of images generated by the camera. Interestingly, the Multi-Spectral Imaging (MSI) provides information that is superior to a standard RGB camera, as it considers the NIR band. In this article, the authors have deliberated how MSI can be used to predict the sugar level and freshness in fruits, particularly grapes using modified EfficientNet-B0 as a Deep Learning (DL) model for the analysis. For predicting sugar level, the average value across the 5-fold cross-validation (cv) achieved by the DL model was RMSE of 2.68 (std 0.33) ◦Bx, MAE of 2.11 (0.34) ◦Bx, MAPE of 12% (2%) using MSI and RMSE of 7.76 (1.79) ◦Bx, MAE of 6.65 (1.69) ◦Bx, MAPE of 41% (9%) using RGB camera images. For freshness prediction, the DL model achieved an average 5-fold cv accuracy of 88.35% (9.71%) using MSI and 82.22% (7.9%) using RGB camera images. Results indicated that MSI can predict both the sugar level and freshness whereas RGB camera images can be used only for predicting the freshness but not the sugar level prediction of the grapes. The findings suggest that MSI offers a valuable and versatile solution for quality assessment in the fruit industry, enabling better dietary choices and healthcare regimes.

Real-time monitoring of ECG signals in wearable and implanted edge devices, such as smart watches, ILRs, pace-makers etc. is essential for early clinical intervention in Cardio-Vascular illnesses. Existing deep learning-based algorithms are not suitable for such battery-powered, low-power, low-memory devices. In this paper, a new peak-based Spiking Hamilton (SH) encoder is proposed and its performance for classifying Atrial Fibrillation (AF) patterns in ECG signal is tested and compared with three existing encoders by deploying reservoir-based SNN and a feed-forward SNN and on five well known ECG dataset. Feed-forward SNN, coupled with proposed SH encoder, achieves best performance while requiring the least amount of computational effort and power. This system running on Neuromorphic Computing (NC) platforms could therefore be a good option for classifying ECG patterns at the wearable edge.

Automatic fashion attribute recognition enables retailers to address an array of applications. Usually, fashion attributes are manually input in the system by retailers, which is a time-consuming and an error prone process. To alleviate this, several existing works use traditional CNN-based backbones to recognize attributes. These backbones generate attribute embeddings that are entangled in the feature space. Existing methods that generate disentangled attribute embedding do not explicitly specify the location of attributes, and often extract features from irrelevant regions. This directly impacts the quality of downstream tasks. To alleviate this problem, we have proposed a novel framework to extract location-aware attribute representation using localization maps created from fashion landmarks. These localization maps highlight regions of interest in an image, aiding localized attribute feature extraction. Moreover, we have proposed a novel fusion module to effectively select important features from the global representation of an image to enhance the local features of the attribute. These attribute embeddings are then used in downstream applications such as attribute recognition, hierarchical taxonomy classification, and retrieval with two large-scale datasets. Using the proposed model, we observe improvement in performance from the state-of-the-art by a significant margin.

Spatiotemporal (ST) climatic signals are used exclusively in the analysis and prediction of weather and climate. These signals are prone to noise due to sensor defects, environmental interference and so on. A novel ST signal denoising method is presented that computes ST signal variability measure at multiple data scales obtained from multivariate variational mode decomposition algorithm. This aids in joint multi-zonal climatic signal denoising directly in multidimensional space$\mathbb {R}^{Z\times N}$where input signal resides, with the usage of interval thresholding applied on multiple data scales in$\mathbb {R}^{Z\times N}$. The performance of the proposed method is assessed and compared against closely related state-of-the-art methods using qualitative and quantitative analysis.

Certain data compression techniques like pruning leads to unstructured sparse Convolution Neural Network (CNN) models without directly leveraging sparsity in optimizing both memory consumption and inference latency of a model having low to medium sparsity. State-of-the-art storage techniques either optimize model size at the cost of execution latency or optimize inference latency at the overhead of the memory consumption of the model. This tradeoff is largely due to the absence of storage selection methodology addressing sparsity sensitivity , arising from varied sparsity and positions of nonzero values called sparsity structure across different sparse layers of a model. However, this issue remains unexplored due to the lack of support to handle sparse data in the current deployment standards for edge devices. This article introduces a data compaction strategy for unstructured sparse data that not only compresses nonzero data but also encodes it, leveraging the memory consumption and latency reduction benefits of both data compression and data encoding techniques . We propose a novel storage representation, named Encoded Partitioned Hybrid Sparse (EPaHS) format, which addresses sparsity sensitivity by customizing data storage based on the sparsity structure of the data. Our data compaction technique and storage solution optimizes the tradeoff between the memory consumption and inference latency of a sparse model without altering the network architecture and affecting its accuracy. Our solution easily extends to higher-dimensional data and outperforms standard storage solutions. It proves to be beneficial to all the valid mode orientations of multi-dimensional data. For an important health and wellness application, a single-lead short-time ECG classification model, EPaHS achieves up to \({\tt 16.18\%}\) reduction in size and \({\tt 15.16\%}\) reduction in latency when compared to its original model of \({\tt 42}\) MB size and \({\tt 26.35}\) sec latency, having \({\tt \approx 59\%}\) sparsity. For a ResNet50 model handling higher-dimensional data, it achieves \({\tt 21.33\%}\) size reduction and \({\tt 53.9\%}\) latency gain against the original model of \({\tt 3265}\) KB size and \({\tt 1.7}\) sec latency, having \({\tt \approx 67\%}\) sparsity.

Identification of user transport mode using mobile phone-based sensors is a key component of Intelligent Transportation System. However, collecting labels/annotations while switching multiple transport modes into different journeys is tedious. Also, transport type identification working across cities and countries is a prime need. This paper proposes a method for generalizable journey mode detection without using any annotations during training exploiting unsupervised representation learning. Our method uses commonalities and diversities across various user’s different journeys, to identify user-specific journey segments from either the same or different city/country. This method is also sensitive to preserve privacy, as it does not use GPS information. We propose a multistage unsupervised learning mechanism to form clusters on the learned latent representation using a choice of best distance measure. We also propose an Invariant Auto-Encoded Compact Sequence, which is a learned compact representation encompassing the common encoded latent feature representation across diverse users and cities. We prove with an exhaustive experimental analysis, that our method, is generalizable across varying users and cities using IMU-Accelerometer sensors. We use real-life publicly available transportation datasets captured from two different cities of different countries -Sussex (United Kingdom) and Bologna (Italy), and also in-house data collected from three Indian cities.

Scientific objectives from earth observation to astronomy require high-resolution observations from space-based platforms. However, designing space telescopes with large primary apertures to achieve high-resolution and high Signal-to-Noise Ratio observations, especially for those operating in longer wavelengths (like Thermal Infrared, TIR), is not feasible due to difficulties in manufacturing, launching, and post-deployment stabilizing. This work proposes three novel lightweight, optically-sparse (also known as partially-filled) mirrors with non-uniform sub-aperture sizes. These designs reduce the mass of the primary mirror and its supporting framework. The crux of these designs is, however, significant suppression of sidelobes in the resulting Point Spread Functions (PSFs). The study includes restored images and image quality indices, demonstrating the effectiveness of such lightweight unequal sub-apertures as replacements for large monolithic mirrors with only a marginal loss in performance.

Automatic object recoloring or swatch generation aims to change color of an object or the entire scene without altering the structural consistency. The challenges exacerbate while dealing with retail items, such as clothing, shoes and accessories due to the presence of complex patterns, folds and shadows, deformation caused by the human model, and small components such as frills, buttons, belts, etc. The challenge associated with this application increases further for multi-colored item and multi-apparel setup. In this work, we aim to address these problems with our novel architecture SwatchNet based on generative adversarial network (GAN). It stems from the proposed apparel components aware feature extraction module to create rich feature embedding, which guides the proposed dual attention u-net to synthesize recolored product image. For seamless information flow, we have also proposed a dual attention module at the bottleneck of encoder and decoder. Finally, we unify a diverse set of recoloring applications using a fixed training and inference pipeline. The experimental results of fashion item recoloring for several test setups using four large-scale datasets demonstrate the effectiveness of our proposed approach.

Emergence of small satellites for earth observation missions has opened up new horizons for space research but at the same time posed newer challenges of limited power and compute resource arising out of the size & weight constraints imposed by these satellites. The currently evolving neuromorphic computing paradigm shows promise in terms of energy efficiency and may possibly be exploited here. In this paper, we try to prove the applicability of neuromorphic computing for on-board data processing in satellites by creating a 2-stage hierarchical cloud cover detection application for multi-spectral earth observation images. We design and train a CNN and convert it into SNN using the CNN2SNN conversion toolkit of Brainchip Akida neuromorphic platform. We achieve 95.46% accuracy while power consumption and latency are at least 35x and 3.4x more efficient respectively in stage-1 (and 230x & 7x in stage-2) compared to the equivalent CNN running on Jetson TX2.

The emerging trend of small satellites for earth observation missions has enabled commercial organisations to exploit the horizon for various business applications related to weather forecasting/monitoring, Land Use Land Cover (LULC) classifications, disaster (such as oil spill or forest fire) monitoring etc. However, the limited power and computational capacity of these small satellites arising out of the size and weight restrictions have posed newer challenges, primarily related to low-power on-board data processing and transmission. One possible approach is to harness the capabilities of the evolving neuromorphic computing paradigm for such low-power computing requirements. One possible application can be lossless compression of high-resolution earth observation images before sending those downstream to ground stations for further analysis. In this paper, we propose a novel method of lossless image compression based on classical Arithmetic Encoding that exploits the low power computing capability of Spiking Neural Networks and neuromorphic platforms. We experimentally prove that our SNN approach achieves compression ratio at-par with state of the art ANN methods with an estimated 2.5× power efficiency and 50% lower latency with a much smaller model - thereby enabling on-board image compression and at the same time, saving on a corresponding amount of energy during transmission.

Recommendation systems improve users' online shopping experience by recommending relevant items from a large pool of items in different categories. Fashion recommendation systems apart from recommending individual fashion items also recommend fashion outfits. In this work, we consider the problem of the outfit compatibility prediction task, an integral part of the fashion outfit recommendation system. A compatibility prediction module determines whether all the items in an outfit are visually compatible with each other and match the user's preferences. Existing approaches can be grouped based on the representation scheme: (i) pair-wise and (ii) set or sequence. Pair-wise representation does not consider the outfit as a whole, and the sequence representation approaches are sensitive to the ordering of the items. Further, these methods do not explicitly capture the visual relationship between the items. We propose a novel method for the personalized outfit-compatible prediction task. The proposed method represents the outfit as a graph and uses a dot-attention graph neural network to capture the visual relationship between items. The graph read-out layer generates the final outfit embedding. A novel approach is proposed to model the user's preference for different styles. The final outfit compatibility score is generated by computing the similarity between outfit embedding and user embedding. Experimental results and ablation study on the Polyvore-U dataset, highlight the effectiveness of the proposed method.

In this paper, we propose StereoGest-SNN which is a robust low power edge compatible ultrasound based gesture detection system. StereoGest-SNN uses easily available off-the-shelf stereo speaker and microphone setup inbuilt in most commercial devices. The stereo setup mimics a$2\times 2$Multiple Input Multiple Output (MIMO) system which provides the requisite diversity to effectively address fading. It also makes use of distinctive Channel Impulse Response (CIR) estimated by imposing sparsity prior for robust gesture detection. A 5-layer CNN has been trained on these distinctive CIRs and the trained model is converted into an equivalent Spiking Neural Network (SNN) via an ANN-to-SNN conversion mechanism. The StereoGest-SNN with converted SNN is found to classify gestures with an accuracy of around 92% (an improvement of over 6% compared to state-of-the-art) and shows$3\times$reduction in number of operations compared to CNN resulting in further energy benefit when run on neuromorphic computation platforms.

In this work, we present a lightweight attention network to perform Time Series Classification on Edge devices. We evaluate the merit of our system on a Human Activity Recognition dataset and show the demonstration with the help of a Wearable device (Smartwatch) with IMU sensors.

Analysis of Electroencephalography (EEG) signals for everyday in-situ applications is hindered by many challenges including artifacts induced from physiological and environmental sources as well as anatomical factors. Recent studies have proposed analytical frameworks for the assessment of scalp EEG quality and identification of artifacts using rule based methods. These methods are typically used in an offline processing manner, employed before signal analysis when abundant data is available. With the advent of wearable devices, it is important to build techniques that work on short term signals giving us the ability of intervention based on the signal quality. Further, ear-EEG is a new modality that requires assessment and calibration with short term signals. To support this new modality we conducted a detailed study of scalp and ear-EEG data from the perspective of different measures as well as time duration. An algorithm is developed to identify the epochs with EOG and EMG artifacts in the ear-EEG using a set of metrics based on the characteristics of EEG. Thresholds of these metrics are determined by training on a dataset containing synchronous ear and scalp EEG with good results. The algorithm obtained an accuracy of 76.7% and 76.84% in classifying artifact EEG for scalp referenced ear-EEG and re-referenced ear-EEG, respectively.

Tracking of inventory and rearrangement of mis-placed items are some of the most labor-intensive tasks in a retail environment. While there have been attempts at using vision-based techniques for these tasks, they mostly use planogram compliance for detection of any anomalies, a technique that has been found lacking in robustness and scalability. Moreover, existing systems rely on human intervention to perform corrective actions after detection. In this paper, we present Co-AD, a Concept-based Anomaly Detection approach using a Vision Transformer (ViT) that is able to flag misplaced objects without using a prior knowledge base such as a planogram. It uses an auto-encoder architecture followed by outlier detection in the latent space. Co-AD has a peak success rate of 89.90% on anomaly detection image sets of retail objects drawn from the RP2K dataset, compared to 80.81% on the best-performing baseline of a standard ViT auto-encoder. To demonstrate its utility, we describe a robotic mobile manipulation pipeline to autonomously correct the anomalies flagged by Co-AD. This work is ultimately aimed towards developing autonomous mobile robot solutions that reduce the need for human intervention in retail store management.

Every human being is unique and behaves differently in any given context. Standard approaches like surveys that are used today to assess human behaviour often generate subjective responses and can be administered either before or after task/activity only. This presents an opportunity to develop solutions that can monitor a human being's cognitive, affective and mental state in their current context continuously and in real time, during activities and in an unobtrusive manner. We have developed a composite system called Mind Indriya that can unobtrusively measure cognitive load (CL), anxiety and visual attention using a combination of frugal sensors like wrist wearable and webcam. The accuracies of each of the individual algorithms have been proven with cognitive load at accuracy 69.5%, anxiety at accuracy of 86 % and the detection of eye blinks as an index of visual attention at$F_{score}$of 0.91. Moreover, our system is person and task independent while state-of-the-art techniques are either task dependent or requires heavy personalization. The proposed system can be used in various real life applications like a) neuro-marketing - to understand how customer behavior towards new products or advertisement, b) online learning - to personalize contents or understand the learning outcome, c) Website/user interface design - to understand the effect of the new design on people's cognitive load, affect and attention span and therefore how effective the design is and so on.

Fashion landmark detection is a fundamental task in several fashion image analysis problems.The associated challenges involving non-rigid structures and variations in style and orientation makes it extremely hard to accurately detect the landmarks.In this paper, we propose Appearance-Context network (ACNet), which encapsulates both global and local contextual information extending the axial attention mechanism.We design axial attention augmented local appearance network and introduce a novel Global-Context aware axial attention module which aggregates the global features attending discriminatory cues across height, width and channel axes.The proposed ACNet architecture outperforms existing methods on two large-scale fashion landmark datasets.

Fusing SAR and multi-spectral images to generate a precise land cover map in a weakly supervised setting is a challenging yet essential problem. The inaccurate, noisy, and inexact ground truth labels pose difficulty training any machine learning models. In this paper, we make a fundamental and pivotal contribution towards improving the ground truth label quality using domain knowledge. We present a simple yet effective mechanism to refine the low-resolution noisy ground truth labels. The proposed approach is trained and tested on a publicly available DFC2020 dataset. Through experiments, we show the effectiveness of our method by training a deep learning model on the refined labels that outperform even the models trained with clean ground truth.

In this paper, we present a novel few-shot cross-sensor domain adaptation technique between SAR and multispectral data for LULC classification. Cross-sensor, such as SAR and multispectral, domain adaptation is a long standing challenge in remote sensing. Due to scarcity of large annotated dataset for every domain, it is desirable to have a method that enables cross-domain training with limited supervisory signal in that domain. We address this problem in this paper with a novel few-shot domain adaptation technique. We leverage large corpus of annotated multispectral dataset to improve performance for SAR based LULC classification. We propose a novel Feature Domain Alignment (FDA) loss function to align higher dimension features between multispectral and SAR domain. We validate our approach in publicly available DFC2020 dataset and achieve 78% overall LULC classification accuracy using only 5% annotated SAR samples.

In recent times, in many industrial domains, the need for in-situ processing of time series data from sensors have grown extensively, to ensure low-latency real-time responses, making embedded edge AI an important area of work. Most current techniques, being computationally intensive, are not suited for edge implementation; and in real world scenarios, connectivity to the cloud for processing sensed data leads to higher latency and often less reliability. Neuromorphic systems, coupled with spiking neural networks (SNN) offer a solution to such problems. This paper explores this evolving paradigm to address the need for low-footprint efficient time series classifiers implementable on edge, and targeted for predictive maintenance scenarios. A reservoir-based SNN architecture is designed and tried for classification of different vibration time series datasets. While the system is found to obtain at par classification accuracy for each of the datasets compared to prior arts, it is also observed to be more efficient in terms of synaptic operations per timestep (13% to 38%) using Gaussian temporal spike encoding scheme compared to Poisson rate encoding. Moreover, the system is found to be robust with respect to learning with reduction in training data (upto 20%).

Scene graph captures rich semantic information of an image by representing objects and their relationships as nodes and edges of a graph. Recent works have demonstrated that scene graph representation improves the performance of various computer vision tasks such as image retrieval, action recognition, visual question answering. Computationally efficient scene graph generation methods are required to leverage scene graphs in various real-world applications (e.g., autonomous driving, robotics). A typical scene graph generation model consists of two modules: (i) object detector and (ii) scene graph classifier. The scene graph classifier module predicts the object category and object-object relationships. The presence of a quadratic number of potential edges poses a major challenge in the scene graph classification task. Detecting the relationship between each object pair using the traditional approach is computationally intensive and non-scalable. To address this issue, we propose a novel module named EdgeNet that directly predicts the set of relevant edges and helps to prune out a significant number of unrelated object pairs, thereby improving the effectiveness and efficiency of the scene graph classifier. The proposed EdgeNet is a generic module and can be plugged into an existing scene graph classifier. Experimental results highlight the effectiveness and efficiency of the proposed approach on the Visual Genome dataset.

In this paper, we propose EchoWrite-SNN, a robust edge compatible air-writing recognition system (used in applications such as AR/VR, HRI etc.) based on principles of SONAR and neuromorphic computing. The bare finger movements in air are captured by a pair of commonly available speaker-microphone pair. A new tracking algorithm based on windowed difference cross-correlation and ESPRIT is employed which shows better tracking accuracy compared to state-of-the-art methods with a median tracking error of only 3.31mm. To classify these air-written shapes, a 5-layer CNN is trained and then converted to a Spiking Neural Network (SNN) using ANN-to-SNN conversion technique to reap the benefits of low power neuromorphic computing on edge. Experimental results show that the converted SNN achieves 92% accuracy (a mere 3% less than the CNN) while showing 4.4 × reduction in number of operations compared to CNN resulting in further energy benefit when run on actual neuromorphic computation platforms.

Intelligent continuous monitoring of an IoT system to identify the operational changes, encompassing both normal and abnormal scenarios, with drift in sensing device is a challenging problem. It demands capability of learning continuously with multiple interventions or shifts, without forgetting past events information. However, forgetting the past learned knowledge, known as catastrophic forgetting, impacts significantly on the performance of continuous monitoring. In this work, we propose a generative neural network based model to handle various operational changes. Here, one objective is to learn continually by capturing past data distributional knowledge, while adapting new data signatures. Other objective, is to handle changes in data distribution like, to identify drifts, as well as, variations in diverse operational conditions of the system. We have experimented using vibration sensor based public real-world bearing data and performed extensive analysis incorporating synthetic drifts. Proposed method outperforms existing benchmark performances with clear separation of drift in learned representation.

Fault diagnosis plays a vital role in ensuring the normal operation of the machine and safe production. In recent years, data-driven techniques have gained a lot of popularity for machine fault diagnosis. But most of these techniques assume the training and test data have the same distribution. However, in most practical application scenarios, domain discrepancy can be observed between the training (source) and test (target) data due to different factors like changes in the operating conditions, different sensor locations, etc. Classical approaches fail to address such domain discrepancy, which leads to poor performance. The problem becomes more challenging when the target is completely unlabeled. To address this scenario, domain adaptation techniques are used to transfer the knowledge learned from the labeled source domain to the unlabeled target domain. Recently, adversarial network based domain adaptation has been extensively explored for fault diagnosis. But the adversarial loss alone does not guarantee the translation of the source to the desired target domain (class consistent). Here, we propose to use cycle-consistency loss employing 1D-CycleGAN for learning the source to target mapping for unsupervised adaptation for bearing fault diagnosis. The proposed method is evaluated for two different scenarios, with the source and target from (i) same machine but different working conditions and (ii) different but related machines. Experimental results show that while the proposed method performs comparable to the best-performing benchmark for the first case, it significantly outperforms all the state-of-the-art methods for the challenging second case.

This work proposes a new approach to deep neural network (DNN) compression. We employ black-box function approximation techniques from signal processing to compress. DNN, in general, can approximate non-smooth and piecewise smooth functions. With only this assumption, we model the function that the DNN has learnt as a piecewise linear function. This is a standard function approximation approach. We compared our approach with two state-of-the-art techniques - spatial singular value decomposition and channel pruning with weight reconstruction; and one of state-of-practice tool - OpenVINO. Two well known 1D DNN models for time series classification - ResNet and InceptionTime were compressed. Results show that our model yields better compression at comparable losses in accuracy on majority of the datasets.

Construction of learning model under computational and energy constraints, particularly in highly limited training time requirement is a critical as well as unique necessity of many practical IoT applications that use time series sensor signal analytics for edge devices. Yet, majority of the state-of-the-art algorithms and solutions attempt to achieve high performance objective (like test accuracy) irrespective of the computational constraints of real-life applications. In this paper, we propose Instant Adaptive Learning that characterizes the intrinsic signal processing properties of time series sensor signals using linear adaptive filtering and derivative spectrum to efficiently construct a low-cost learning model followed by standard classification algorithms. Our empirical studies on a number of time series sensor signals from publicly available time series database (UCR) demonstrate that with slight trade-off in performance, the proposed method achieves very fast learning capability.

Automatic Carnatic Music (CM) transcription is an open problem in need of a standardized descriptive notation. The level of detail needed in a descriptive transcription makes it tedious to obtain ground truth by manual means. In this paper, we propose a novel state-based representation of the pitch curve motivated by CM components called constant-pitch notes and stationary points. We also propose a novel transcription technique that uses the Viterbi algorithm to estimate the states and quantized pitch-values. The proposed technique adheres best to raga-notes compared to the existing critical-points technique and uniform quantization. In a listening test, clips synthesized from the proposed notation were rated significantly better (324 ratings, p <; 0.001) than those from critical-points. Further, speed-halving based on state information best matches the actual, observed CM component-duration ratios without losing raga-characteristics. Thus, the proposed transcription can be corrected manually to obtain ground truth and can enhance learning tools.

In the past two decades, radar-based human sensing has become a topic of intense research. Unlike vision-based techniques which require the use of camera, radars are unobtrusive and privacy preserving in nature. Further, radars are agnostic of the lighting conditions and can be used for through-the-wall imaging thereby making them hugely effective in many situations. Compact, affordable radars have been designed that can be easily integrated with remote monitoring systems. However, the classical machine learning techniques currently used for learning and inferring human actions from radar images are compute intensive, and require large volume of training data, making them unsuitable for deployment on the network edge. In this paper, we propose to use the concepts of neuromorphic computing and Spiking Neural Networks (SNN) to learn human actions from data captured by the radar. To the best our knowledge, this is the first attempt of using SNNs on micro-Doppler data from radars. Our SNN model is capable of learning spatial as well as temporal features from the data and our experiments have resulted in 85% accuracy which is comparable with the classical machine learning approaches that are typically used on similar data. Further, the use of neuromorphic and SNN concepts make our model deployable over evolving neuromorphic edge devices thereby making the entire approach more efficient in terms of data, computation and energy consumption.

Time Series Classification (TSC) is becoming a challenging and important problem to solve specifically due to the advent of sensor-based applications and Internet of Things (IoT). Residual mapping displays easier and evidently near-optimal learning. In this paper, we extend this notion by incorporating fine-grained refining of the residual learning through augmentation of feature space using gamut of signal processing transformations. Our proposed Sig-R2ResNet refines the learning process by introducing newer representation through signal processing primitives without distorting the residual mapping channel, along with an auto-tuning L1regularization. We further adapt the learning rate decay through learning over the trend of validation loss and modify the network parameter update process of Adam optimizer. The proposed method- Sig-R2ResNet can be viewed as an informal game where training experience is augmented through unsupervised signal processing features while the model growth is controlled by a regularization process and smoother learning convergence is achieved by validation loss dependent learning rate. One of the novelties is that the training signal dynamics control the enhancement of representation complexity, regularization and learning rate adaptation. We perform extensive experiments with diverse datasets from publicly available UCR time series database and demonstrate empirical evidences that our method consistently outperforms the existing benchmark results (creating 59.10% new benchmark results) as well as shows significantly better classification outcome than the current baselines and state-of-the-art algorithms like ResNet, BOSS, COTE.

Time Series Classification (TSC) is a challenging problem owing to the practical constraints of lack of availability of training examples and insufficient sample points in the training instances. In order to ensure the construction of a robust trained model (under practical constraints) to address TSC, we propose DyReg-FResNet, which is a dynamically regularized Residual Network (ResNet), amplified by unsupervised feature space training. We generate signal processing, information theoretic and statistical features to augment the representation learning of the ResNet. The unsupervised features are capable of extracting the morphological and structural characteristics of the time series signals, whereas our proposed dynamic regularizer trades off by reducing substantial variance while not perturbing the bias. The regularization factor is a function of the signal dynamics (in effect, regularization factor is different for different training sets). DyReg-FResNet learns through residual mapping to minimize the exploding or vanishing gradient problems, amplified unsupervised features amplify the representation space by introducing lower level representation to guide the learning towards the basins of attraction of minima and dynamic regularizer minimizes the generalization error. We extensively experiment with publicly available UCR time series datasets. DyReg-FResNet demonstrates extremely superior performance by consistently outperforming the existing benchmark results as well as current state-of-the-art algorithms.

A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multi-purpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task. Additionally, some mechanism is required to resolve the ambiguity or missing pieces of information in natural language instruction. Thus, we also develop a dialogue strategy that aims to gather additional information with minimal question-answer iterations and only when it is necessary. This work makes a significant stride towards enabling a human-like task understanding capability in a robot.

Running deep learning algorithms at the edge is a necessity in many industrial use-cases, especially in applications that use robots and drones in disaster recovery, surveillance, oil & gas operations etc. Current state of the art deep learning algorithms are extremely efficient in analysing image, audio, video and other time-series signals. However, their performance degrades considerably on constrained edge devices. In this demo, we show how standard pretrained CNN (Convolutional Neural Network) models can be partitioned for efficient parallel execution between constrained devices and also achieve real-time response.

In this paper we propose a novel process flow of a low-cost, non-invasive screening system for identifying Coronary Artery Disease (CAD) patients using a two-stage classification approach. A statistical rule engine is designed based on patient demography and medical history which is applied at the first stage of the proposed classification system. The misclassification error at this stage is reduced at the second stage based on numerical features extracted from multiple cardiovascular signals. Two sets of features are extracted from phonocardiogram (PCG) and photoplethysmogram (PPG) signals, collected from each subject for creating two independent Support Vector Machine (SVM) classifiers. Outcomes of the two classifiers are fused at the decision level for final prediction at second stage based on absolute distance of the test data-point from its respective SVM hyperplane. Results show that the proposed approach achieves sensitivity of 0.92 and specificity of 0.90 in classifying CAD patients on a hospital dataset of 99 subjects including CAD and non-CAD patients.

In this paper, we present completely automated cardiac anomaly detection for remote screening of cardio-vascular abnormality using Phonocardiogram (PCG) or heart sound signal. Even though PCG contains significant and vital cardiac health information and cardiac abnormality signature, the presence of substantial noise does not guarantee highly effective analysis of cardiac condition. Our proposed method intelligently identifies and eliminates noisy PCG signal and consequently detects pathological abnormality condition. We further present a unified model of hybrid feature selection method. Our feature selection model is diversity optimized and cost-sensitive over conditional likelihood of the training and validation examples that maximizes classification model performance. We employ multi-stage hybrid feature selection process involving first level filter method and second level wrapper method. We achieve 85% detection accuracy by using publicly available MIT-Physionet challenge 2016 datasets consisting of more than 3000 annotated PCG signals.

Feature subset selection and identification of appropriate classification method plays an important role to optimize the predictive performance of supervised machine learning system. Current literature makes isolated attempts to optimize the feature selection and classifier identification. However, feature set has an intrinsic relationship with classification technique and together they form a `model' for classification task. In this paper, we propose AutoModeling that finds optimal learning model and jointly optimize the feature and hypothesis space to maximize performance measure objective function. It is an automated framework of selecting the ensemble model {selected feature subset, selected classifier} from a given superset of features and classifiers learned from given training dataset in a computational efficient manner. We introduce novel relax-greedy search with our proposed patience function as a wrapper feature selection that maximizes the predictive performance and eliminates the classical nesting effect. We perform extensive experimentations on different types of publicly available datasets and AutoModeling demonstrates superior performance over relevant state-of-the-art methods, expert-driven manual methods and deep neural networks.

Heart rate (HR) monitoring is indispensable for several real-world scenarios, especially when acquired in a non-contact manner. It can be accomplished using face videos acquired from ubiquitous cameras in an inexpensive, non-invasive and unobtrusive manner. But the HR monitoring can be erroneous when the video contains facial expressions, out-of-plane movements, change in camera parameters (like focus) and variations in environmental factors (like illumination). The proposed system mitigates these problems for improving the HR monitoring. For this, it defines an adaptive temporal signal selection mechanism which identifies and removes the facial areas affected by facial expressions. Moreover, it introduces a novel post-processing mechanism which perform HR monitoring by utilizing face reconstruction and quality. The post-processing is used when the face video contains facial movements. Experimental results reveal that incorporation of adaptive temporal signal selection and post-processing mechanisms can significantly improve the HR monitoring. It depicts that the Pearson correlation between actual and estimated HR is 0.95 while the average absolute error is 1.63 beats per minute, which indicates that the proposed system provides good HR monitoring.

This paper presents a demo proposal of a standalone smartphone application that can automatically analyse the signal quality of PCG, as it is recorded on a low-cost smartphone-based digital stethoscope. Features, related to the inherent pattern of the autocorrelated signal envelope, have been used for classifying and discarding the noisy portions from a continuous PCG. Our application has been successfully deployed on Nexus 5 and tested on several clean and noisy PCG signals with sensitivity 78.91% and specificity 70.83%.

Remote cardiac health management is an important healthcare application. We have developed Heartmate that enables basic screening of cardiac health using low cost sensors or smartphone-inbuilt sensors without manual intervention. It consists of robust denoising algorithm along with effective anomaly analytics for physiological signals. Heartmate identifies and eliminates signal corruption as well as detects cardiac anomaly condition from physiological cardiac signals like heart sound or phonocardiogram (PCG) and photoplethysmogram (PPG).

Estimating heart rate (HR) accurately using face videos acquired from a low cost camera in contactless manner is of paramount importance for many real-world applications. Such existing systems perform spuriously due to change in camera parameters, respiration, facial expressions and environmental factors. This paper mitigates the issues for accurate HR estimation. The face video consisting of frontal, profile or multiple faces is divided into multiple overlapping fragments to determine HR estimates. The HR estimates are fused using quality-based fusion which aims to minimize illumination and face deformations. Experimental results demonstrate that the proposed system exhibit better performance than the state of the art systems and establishes the efficacy of the quality-based fusion in HR estimation.

In this article, we propose a method to assess the clinical usability of fingertip Photoplethysmogram (PPG) waveform, collected from medical grade oximeter (train data) and smartphone (test data). We introduce a set of novel Signal Quality Indices (SQIs) to represent the noise characteristics of the PPG waveform. The SQIs are presented to a random forest classifier to discriminate between clean and noisy signals. The proposed method was evaluated on datasets annotated by four experts, resulting into a sensitivity and specificity of (92 ± 4.7 %, 95 ± 3 %) and (82.6 ± 4.6 %, 95.4 ± 3.1 %) on train and test data respectively. Further we applied the proposed method on PPG waveform of clinically proven control and disease population of Coronary Artery Disease (CAD), which resulted into (77 %, 77 %) of sensitivity and specificity respectively.

Healthcare data is quite rich and often contains human survival related information. Analyzing healthcare data is of prime importance particularly considering the immense potential of saving human life and improving quality of life. Furthermore, IoT revolution has redefined modern health care systems and management. IoT offers its greatest promise to deliver excellent progress in healthcare domain. In this talk, proactive healthcare analytics specifically for cardiac disease prevention will be discussed. Anomaly detection plays a prominent role in healthcare analytics. In fact, the anomalous events are to be accurately detected with low false negative alarms often under high noise (low SNR) condition. An exemplary case of smartphone based cardiac anomaly detection will be presented.

In this paper we leverage the power of smartphone to enable proactive in-house heart condition monitoring. We introduce Heart-Trend, a nonparametric model to analyze and detect heart abnormality conditions like arrhythmia from photoplethysmogram (PPG) signal. It does on-demand heart status monitoring using smartphones (can also be implemented in PC/ICU monitors) and facilitates timely detection of heart condition deterioration to permit early diagnosis and prevention of fatal heart diseases. Proposed robust anomaly analytics engine accurately detects the morphological trend to find abnormal heart condition in real time through machine learning based trend prediction. PPG signal is frequently corrupted by ambient noise, and motion artifacts, which lead to high amount of false alarms. We introduce precise denoising technique that identifies and eliminates the corrupted segments of clinical signal to minimize its impact on the decision process and analytics. We demonstrate that Heart-Trend ensures high detection capability with lower false alarm rates.

Non-invasive cuff-less Blood Pressure (BP) estimation from Photoplethysmogram (PPG) is a well known challenge in the field of affordable healthcare. This paper presents a set of improvements over an existing method that estimates BP using 2-element Windkessel model from PPG signal. A noisy PPG corpus is collected using fingertip pulse oximeter, from two different locations in India. Exhaustive pre-processing techniques, such as filtering, baseline and topline correction are performed on the noisy PPG signals, followed by the selection of consistent cycles. Subsequently, the most relevant PPG features and demographic features are selected through Maximal Information Coefficient (MIC) score for learning the latent parameters controlling BP. Experimental results reveal that overall error in estimating BP lies within 10% of a commercially available digital BP monitoring device. Also, use of alternative latent parameters that incorporate the variation in cardiac output, shows a better trend following for abnormally low and high BP.

Sensors play a vital role for realizing the vision of connected smart universe. In this paper, we present a novel sensor agnostic model SensIPro to perform robust unsupervised analysis of sensor data to support scalable analytics, a prime need for Internet of things (IoT). In the context of sensor analytics, outliers contain most delicate information. Analysis of anomaly or outlier is mostly dependent on the application domain as well as signal characteristics. Our proposed sensor analytics model SensIPro automates analysis of outliers based on inferring signal characteristics of diverse sensors from different IoT applications like healthcare, smart energy, smart transport. We apply relevant time-series algorithms using statistical analysis, information theoretic measure for sensor data analytics. We measure similarity/dissimilarity of the time series sensor data and correlate with detected outliers. Our algorithm does not require any prior knowledge of sensor data type and metadata. We present results and analysis based on real life heterogeneous sensor data sets. Obtained results further prove efficacy of the proposed mechanism.

In Internet of Things (IoT), numerous and diverse types of sensors generate a plethora of data that needs to be stored and processed with minimum loss of information. This demands efficient compression mechanisms where loss of information is minimized. Hence data generated by diverse sensors with different signal features require optimum balance between compression gain and information loss. This paper presents a unique analysis of contemporary lossy compression algorithms applied on real field sensor data with different sensor dynamics. The aim of the work is to classify the compression algorithms based on the signal characteristics of sensor data and to map them to different sensor data types to ensure efficient compression. The present work is the stepping stone for a future recommender system to choose the preferred compression techniques for the given type of sensor data.

Detection of normal and anomalous events from sensor signal is a key necessity in today's smart world. Here, we propose a novel mechanism to classify normal and anomalous phenomena by using self-learning of signal, i.e., by discovering its pattern. This is the first step in the long drawn out analysis of signals. We demonstrate a prototype of our proposed method by using a real field quasi-periodic photoplethysmogram (PPG) signal with (or without) motion artifacts, which has an immense impact on cardiac health monitoring, stress, blood pressure, and SPO2 measurement. We have achieved more than 90% accuracy to detect anomalous phenomena in the signal.

In this paper, we intend to demonstrate a novel system to measure the high-speed vibration of an anonymous vibrating object using COTS camera and optical strobing. The whole process is unobtrusive and frugal, can be used in machine inspection. The camera used has a frame rate of 30 frames per second (fps), so in conventional fashion, it is incapable to detain significant information about any vibration frequency which is not in the range of Nyquist theory of frequency (within a range of ±15 Hz). We have solved the challenge using optical strobing phenomena for capturing modulo (of division) between object's frequency and strobing frequency using camera. Motion in the video is tracked by conventional image processing technique. Finally, object vibration is calculated from the frequency plot and optical strobing frequency. Under most of the cases, the application estimates vibration frequency values, within a range of ±1.5% of error.

Heart rate variability (HRV) measures the instantaneous change in heart rate and is an important marker for checking physical condition as well as mental stress of a person. In this paper, we propose a methodology to calculate HRV of a person using smart phone audio. Heart sound is captured in the inbuilt microphone of a smart phone, by placing the device on the chest of the person. We propose a process flow to make the phone captured noisy audio signal clean and audible. Furthermore, we propose a novel peak detection algorithm for accurately locating the peaks corresponding to heart sound in the noisy audio signal. The algorithm is also capable of rejecting the noisy peaks present in the captured audio that resembles heart sound pattern. Results show that the proposed methodology yields significant improvement in estimating HRV parameters compared to a clinical pulse-oximeter device, that works on the principle of photoplethysmogram (PPG) technique.

Management of bulk sensor data is one of the challenging problems in the development of Internet of Things (IoT) applications. High volume of sensor data induces for optimal implementation of appropriate sensor data compression technique to deal with the problem of energy-efficient transmission, storage space optimization for tiny sensor devices, and cost-effective sensor analytics. The compression performance to realize significant gain in processing high volume sensor data cannot be attained by conventional lossy compression methods, which are less likely to exploit the intrinsic unique contextual characteristics of sensor data. In this paper, we propose SensCompr, a dynamic lossy compression method specific for sensor datasets and it is easily realizable with standard compression methods. Senscompr leverages robust statistical and information theoretic techniques and does not require specific physical modeling. It is an information-centric approach that exhaustively analyzes the inherent properties of sensor data for extracting the embedded useful information content and accordingly adapts the parameters of compression scheme to maximize compression gain while optimizing information loss. Senscompr is successfully applied to compress large sets of heterogeneous real sensor datasets like ECG, EEG, smart meter, accelerometer. To the best of our knowledge, for the first time 'sensor information content'-centric dynamic compression technique is proposed and implemented particularly for IoT-applications and this method is independent to sensor data types.

Photoplethysmography (PPG) signals, captured using smart phones are generally noisy in nature. Although they have been successfully used to determine heart rate from frequency domain analysis, further indirect markers like blood pressure (BP) require time domain analysis for which the signal needs to be substantially cleaned. In this paper we propose a methodology to clean such noisy PPG signals. Apart from filtering, the proposed approach reduces the baseline drift of PPG signal to near zero. Furthermore it models each cycle of PPG signal as a sum of 2 Gaussian functions which is a novel contribution of the method. We show that, the noise cleaning effect produces better accuracy and consistency in estimating BP, compared to the state of the art method that uses the 2-element Windkessel model on features derived from raw PPG signal, captured from an Android phone.

Sensor nodes are embodiment of IoT systems in microscopic level. As the volume of sensor data increases exponentially, data compression is essential for storage, transmission and in-network processing. The compression performance to realize significant gain in processing high volume sensor data cannot be attained by conventional lossy compression methods. In this paper, we propose ASDC (Adaptive Sensor Data Compression), an adaptive compression scheme that caters various sensor applications and achieve high performance gain. Our approach is to exhaustively analyze the sensor data and adapt the parameters of compression scheme to maximize compression gain while optimizing information loss. We apply robust statistics and information theoretic techniques to establish the adaptivity criteria. We experiment with large sets of heterogeneous sensor datasets to prove the efficacy. Nonlinear lossy compression (Chebyshev) is extensively considered as the standard technique as well as experimental result with frequency domain compression like Discrete Fourier Transform (DFT) is shown as future scope of further improvement.

Smart meter, the important component of smart energy management systems invites intended or unintended, possibly dangerous privacy breaching activities, like in-house activity detection. With the emergence of Non-Intrusive Load Monitoring (NILM), privacy preservation of smart meter data becomes very important for an individual. Emerging solution provides privacy breach minimization by supervised learning through training that incurs higher capex and opex. IoT systems do not consist of human-in-loop. So, involuntary approach of privacy preservation is to be employed. In this paper, we propose a novel solution for addressing the problem of involuntary privacy breaching risk minimization in smart energy management systems. Our proposed solution `Dynamic Privacy Analyzer' scheme is an attempt towards achieving a unique privacy metric that is derived from fundamental principles like robust statistics and information theory. We analyze the performance of our scheme with large set of publicly available real smart meter datasets and evaluate optimality criteria like utility-privacy trade-off. Efficacy of our proposed scheme is demonstrated by minimizing the capability of privacy intruders like NILM. To the best of our knowledge, for the first time the involuntary privacy-aware scheme tailored for IoT system is proposed. Our proposed scheme is generic enough to suit in other IoT applications.

Modern day smart phones are powerful connected sensory and computation nodes for crowd-sensing, urban-sensing and personal-sensing applications. We have developed an Internet of Things (IoT) platform that can seamlessly handle data from the wide variety of sensors available on mobile phones. It can store and run aggregated analysis on the data in real-time. However, mobile phones themselves are a very heterogeneous set of devices. Each phone comes with a different array of sensors with varying sensitivity and control functions. Also, there are multiple development environments and programming languages. A final problem is seamless prototyping of applications offline and then seamless partitioning of the algorithm between phone and the cloud. In this paper we present early design elements of a framework aimed at addressing these issues.

Sensors are one of the primary building blocks of IoT. Owing to close proximity of physical world, sensors often collect sensitive information. Invariably, sensor data has rich information content. Here we propose a novel solution IAS: Information Analytics for Sensors to unlock massive potential of sensor data through information analytics and demonstrate an alerting mechanism based on criticality of sensor information. ECG anomaly detection for healthcare, unusual appliance operation detection from smart energy meter data, bad road condition as well as activity detection from accelerometer data are typical use-case scenarios. We use robust statistical and information theoretic approaches. Our approach is unsupervised and is completely sensor agnostic. This abstract provides overview of design and implementation of our tool IAS along with obtained results tested on publicly available datasets. Last but not the least, IAS validates that outliers contain most delicate information.

Developing analytical applications for IoT based on sensor signal processing tends to be complicated as applications are executed as sequence of steps comprising of multiple alternative algorithms, including suitable feature extraction modules depending on the goal of the application. Experience shows that developers spend considerable time and effort in performing feature extraction and dimensionality reduction. In this paper we propose a framework based on a relevant case study which allows developers to drag and drop algorithms to create a workflow chain, automatically select the most relevant signal features for the particular analytic application using a training data set to generate a model and deploy the model for use. The method reduces the effort and cost of development which is deemed highly important for the analytics industry.

The problem of privacy disclosure hinders large number of ubiquitous applications to collect, disseminate and analyze personal data from providing useful and important services. It is understood that sharing private data has high potential for facilitating innumerable benefits as well as inviting intended or unintended malicious activities leading to severe privacy breach. Such privacy breach attacks mostly capture sensitive or broadly the anomalous events. Fine grained, high resolution smart meter energy consumption data contains sensitive house hold activity signature. In this paper, we propose a tool called `Sensitivity Inspector' that detects sensitivity in smart meter data and inculcates privacy awareness among smart meter users, presuming private events are related to anomalous or unusual activities. Specifically, we analyze the sensitive content of smart meter data through robust unsupervised statistical method considering user activity as a piece-wise, stationary, stochastic process with associated uncertainty. We show the efficacy of our scheme under relevant statistical and information theoretic measures. We implement our algorithm and compare sensitivity detection capability with related supervised learning based approach and relevant privacy breaching attack like Non-Intrusive Load Monitoring (NILM).

Regular ECG check up is a good practice for cardiac patients as well as elderly people. In this paper we propose a low cost methodology to coarsely estimate the range of some important parameters of ECG using Photoplethysmography (PPG). PPG is easy to measure (even with a smart phone) and strongly related to human cardio-vascular system. The proposed methodology extracts a set of time domain features from PPG signal. A statistical analysis is performed to select the most relevant set of PPG features for the ECG parameters. Training model for the ECG parameters are created based on those selected features. Both artificial neural network and support vector machine based supervised learning approach is used for performance comparison. Experimental results, performed on benchmark dataset shows that good accuracy in the estimation of ECG parameters can be achieved in our proposed methodology. Results also show that the overall performance improves in using feature selection technique rather than using all the PPG features for classification.

As part of preventive healthcare, there is a need to regularly monitor blood pressure (BP) of cardiac patients and elderly people. Mobile Healthcare, measuring human vitals like heart rate, Spo2 and blood pressure with smart phones using the Photoplethysmography technique is becoming widely popular. But, for estimating the BP, multiple smart phone sensors or additional hardware is required, which causes uneasiness for patients to use it, individually. In this paper, we present a methodology to estimate the systolic and diastolic BP levels by only using PPG signals captured with smart phones, which adds to the affordability, usability and portability of the system. Initially, a training model (Linear Regression Model or SVM Model) for various known levels of BP is created using a set of PPG features. This model is later used to estimate the BP levels from the features of the newly captured PPG signals. Experiments are performed on benchmark hospital dataset and data captured from smart phones in our lab. Results indicate that by additionally adding information of height, weight and age play a vital role in increasing the accuracy of the estimation of BP levels.

Internet of Things (IoT) has the potential to become a technology revolution with a vision of creating very large scale network, comprising of unprecedented number of connected devices. These devices, often referred to as smart items or intelligent things can be home appliances, healthcare devices, vehicles, buildings, factories and almost anything networked and fitted with sensors, actuators, embedded computers. There has been sustained research work and standardization effort from different IoT perspectives like integration of sensor and RFID devices to the Internet. With the increasing trend of gathering business insights from unstructured data, the high volume of data generated by such devices is also of interest. Cloud based data mining platforms are suitable for analyses of such data and researchers have proposed architectures where personal mobile phones can act as Edge Gateway between the sensor network and cloud analytics platform. It seems that the surge in the volume of data generated by huge number of Smart Items can only be matched if a large percentage of mobile users start sharing the computation capability of their personal devices and work together towards true Participatory Computing in the IoT systems. In this work we try to understand the challenges associated with running computation jobs on the mobile devices using different types of workload often observed in IoT applications. Based on the insights gained from experiments performed by us, we propose a scheme where mobile phones, residential gateways and other edge devices offer free slots to servers in a cloud based data analytics system. Based on the free time slots offered by the mobile phones, if commensurately sized computational jobs can be scheduled, the unpredictability associated with using mobile phones as grid resources can be solved.

In this paper we present a unique approach to make use of CoAP (Constrained Application Protocol) [1] from IETF (Internet Engineering Task Force) in a situation aware mode. The protocol adapts its characteristic for resource optimization depending on the indication inferred from sensed data. We consider a use-case for vehicular telemetry using a constrained in-vehicle sensor gateway which posts the vehicular information (accelerometer, GPS, device-identifier, time). In this use-case bandwidth usage is the main concern for the sensor gateway whereas usage of power which is directly impacted by overall bandwidth consumption is a key factor in case of mobile phone used as sensor gateway. We have reduced communication cost and optimized resource usage in terms of energy and bandwidth which are essential for any constrained sensor gateway by adapting characteristics of CoAP as mentioned above. The improvements are established by analyzing the data captured in real field.

Regular monitoring of certain vital parameters like heart-rate (HR), blood pressure (BP), Electrocardiogram (ECG) are the basic needs for elderly people and patients with chronic diseases residing at home. In this demo, authors would like to demonstrate the possibility of estimating BP levels and certain ECG parameters using the PPG signals captured from smart phones. The work includes mainly three components -- (i) robust PPG signal acquisition, (ii) estimation of BP levels (low, medium, high) from PPG signals and (iii) estimation of PR, RR, QRS and QT intervals of ECG parameters from PPG signals. Initially certain time domain features are extracted from PPG, which are used to create training models for various BP levels and ECG parameters. The approach is tested on two benchmark hospital datasets from (i) University of Queensland and (ii) Capnobase TBME RR dataset and one dataset captured from smart phones. Results indicate that the estimation accuracy is above 75% and sometimes above 95% if the height, weight and age information are considered.

The Connected TV can be described as an Internet enabled TV. In the current paper we have proposed a system for connected TV that mash up the information from internet and RSS feeds related to the breaking news aired over the TV. The proposed system initially localize the text regions from the streamed video in hybrid mode, then recognize them and spot the key words and finally fetch the related information from Internet. Our experimental results show that the localization of the text regions from the video can work with no misses but have some false positives which are taken care by data semantic analysis. The errors of the Optical Character recognition (OCR) module are taken care by using string comparing techniques like longest common subsequence matching and Leveinsthein distance while matching with the RSS feed or internet.

This paper discusses about on-screen keyboard with hierarchical character or symbol organization which are operated by an accompanying remote control to allow navigation with reduced number of key strokes and enhanced user experience for using convergent services like internet browsing, short message service (SMS), Instant Messaging (IM) on devices like TV or Set Top Box obviating the need for separate physical keyboard.