VLDB 2026 Research / reviewers in the wild / expert
Ayan Banerjee 0001
dblp:86/2304-1
· DBLP profile ↗
46ranked-venue papers
12as first author
12since 2021 · last 2026
0000-0001-6529-1644ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 15 · 3 first-author · 5 since 2021Applied, interdisciplinary, general and emerging computing · 11 · 3 first-author · 2 since 2021Graphics, computer vision, multimedia, augmented reality and games · 9 · 4 first-author · 6 since 2021Systems, architecture and hardware · 8 · 2 first-author · 3 since 2021Human-computer interaction and ubiquitous computing · 8 · 2 first-author · 1 since 2021Computer networks · 5 · 2 first-author · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Experience with Single Domain Generalization in Real World Medical Imaging DeploymentsabstractA desirable property of any deployed artificial intelligence is generalization across domains, i.e. data generation distribution under a specific acquisition condition. In medical imagining applications the most coveted property for effective deployment is Single Domain Generalization (SDG), which addresses the challenge of training a model on a single domain to ensure it generalizes well to unseen target domains. In multi-center studies, differences in scanners and imaging protocols introduce domain shifts that exacerbate variability in rare class characteristics. This paper presents our experience on SDG in real life deployment for two exemplary medical imaging case studies on seizure onset zone detection using fMRI data, and stress electrocardiogram based coronary artery detection. Utilizing the commonly used application of diabetic retinopathy, we first demonstrate that state-of-the-art SDG techniques fail to achieve generalized performance across data domains. We then develop a generic expert knowledge integrated deep learning technique DL+EKE and instantiate it for the DR application and show that DL+EKE outperforms SOTA SDG methods on DR. We then deploy instances of DL+EKE technique on the two real world examples of stress ECG and resting state (rs)-fMRI and discuss issues faced with SDG techniques. Ayan Banerjee 0001, Komandoor Srivathsan, Sandeep K. S. Gupta |
AAAI | 1 |
| 2026 | Hardware Software Optimizations for Fast Model Recovery on Reconfigurable Architectures (FPGAs) for Edge and Physical AIabstractModel Recovery (MR) builds physics-guided digital twins from data but runs inefficiently on GPUs due to iterative ODE solvers and memory-bound kernels. We introduce ModEl Recovery IN fpgabased Dynamic Architecture (MERINDA), based on replacing ODE solver with a GRU-based streaming dataflow and co-designs fixedpoint compute and on-chip memory (BRAM tiling, heterogeneous DSP/LUT mapping). On representative MR workloads, MERINDA yields up to 6.3× fewer cycles and 99.3% lower energy than an LTC-based FPGA baseline, enabling real-time edge deployment. Ayan Banerjee 0001, Sandeep K. S. Gupta |
FPGA | 2 |
| 2026 | Human knowledge integrated multi-modal learning for single source domain generalizationabstractGeneralizing image classification across domains remains challenging in critical tasks such as fundus image–based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multi-modal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively. Code and models are available at: https://github.com/IMPACT-Lab-ASU/GenEval. Ayan Banerjee 0001, Kuntal Thakur, Sandeep K. S. Gupta |
WACV | 1 |
| 2026 | Towards Certified Safe Personalization in Learning-Enabled Human-in-the-loop Human-in-the-plant SystemsabstractThis article presents an AI-enabled Personalization Management (AIIM) software for human-in-the-loop, human-in-the-plant learning-enabled systems (LES). AIIM can be integrated with LES software to aid a human user in achieving safe and effective operation under dynamically changing contexts. AIIM consists of: (a) an AI technique to derive model coefficient of a physics-guided surrogate model from operational data shared following privacy norms, and (b) continuous model conformance to identify key changes in LES operational behavior that may jeopardize safety. We demonstrate two capabilities of AIIM, personalization and unknown error detection, through case studies that span a significant breadth of dynamic context change scenarios including (a) involuntary change in user context such as medication-induced glucose metabolism change in automated insulin delivery (AID), (b) actuation failure such as cartridge blockage in AID, (c) latent sensor error in aviation, and (d) unknown coding error in autonomous car software patches. We compare AIIM personalization with human-in-the-loop and self-adaptive model-predictive control design in real-life and simulation settings, to show safe and improved diabetes management. Ayan Banerjee 0001, Aranyak Maity, Imane Lamrani, Sandeep K. S. Gupta |
ACM J. Emerg. Technol. Comput. Syst. | 1 |
| 2026 | MERINDA: Model Recovery in FPGA-Based Dynamic Architecture for Edge and Physical AIabstractUnderstanding the physical laws that govern real-world data is fundamental to achieving safe and explainable operation of mission-critical autonomous systems (MCAS)—a cornerstone of physical AI. Model Recovery (MR) serves as a key mechanism for inferring governing equations directly from data. However, deploying MR within MCAS must contend with stringent latency, computational, and power constraints, making edge-AI acceleration essential. Field-Programmable Gate Arrays (FPGAs) provide an attractive hardware substrate due to their reconfigurability and real-time processing capability. Yet, existing MR techniques often rely on computationally intensive nonlinear optimization or the numerical solution of multiple ordinary differential equations (ODEs) embedded within neural architectures, leading to high compute and memory overheads. To address these challenges, this article introduces MERINDA (Model Recovery in Dynamic Architecture), an FPGA-accelerated MR framework that integrates Gated Recurrent Units (GRUs) with an invertible mapping parameterized as a linear combination of nonlinear basis functions. We theoretically establish that MERINDA is functionally equivalent to Neural Ordinary Differential Equation (NODE)-based MR architectures, while eliminating the need for repeated ODE integration during training and inference. Empirical evaluations on benchmark datasets against Extracting sparse Model from ImpLicit dYnamics (EMILY), Sparse Identification of Nonlinear Dynamics (SINDY), and Physics-Informed Neural Networks with Sparse Regression (PINN+SR) demonstrate that MERINDA achieves comparable accuracy with substantial gains in processing speed, energy efficiency, and DRAM utilization. We further analyze the energy–memory tradeoff in MERINDA, showing that optimizing one resource directly impacts the other under fixed accuracy constraints. Using mixed-integer programming, we derive resource-aware optimal hyperparameters and construct the Pareto front that compares FPGA edge deployment against Mobile GPU (M-GPU) and server-class Graphics Processing Units (GPU) platforms. Our results highlight MERINDA’s energy efficiency, reduced training time, and smaller memory footprint, reinforcing its viability for deployment in resource-constrained autonomous systems. Ayan Banerjee 0001, Sandeep K. S. Gupta |
ACM Trans. Design Autom. Electr. Syst. | 2 |
| 2025 | Model Recovery at the Edge Under Resource Constraints for Physical AIabstractModel Recovery (MR) enables safe, explainable decision-making in mission-critical autonomous systems (MCAS) by learning governing dynamical equations, but its deployment on edge devices is hindered by the iterative nature of neural ordinary differential equations (NODE), which are inefficient on FPGAs. Memory and energy consumption are the main concern of applying MR on edge devices for real-time running MR. We propose MERINDA, a novel FPGA-accelerated MR framework that replaces iterative solvers with a parallelizable neural architecture equivalent to NODEs. MERINDA achieves nearly 11× lower DRAM usage and 2.2× faster runtime compared to mobile GPUs. Experiments reveal an inverse relationship between memory and energy at fixed accuracy, highlighting MERINDA’s suitability for resource-constrained, real-time MCAS. “The implementation and datasets are publicly available at github.com/ImpactLabASU/ECAI2025.” Ayan Banerjee 0001, Sandeep K. S. Gupta |
ECAI | 2 |
| 2024 | Recovering Implicit Physics Model Under Real-World ConstraintsabstractRecovering a physics-driven model, i.e. a governing set of equations of the underlying dynamical systems, from the real-world data has been of recent interest. Most existing methods either operate on simulation data with unrealistically high sampling rates or require explicit measurements of all system variables, which is not amenable in real-world deployments. Moreover, they assume the timestamps of external perturbations to the physical system are known a priori, without uncertainty, implicitly discounting any sensor time-synchronization or human reporting errors. In this paper, we propose a novel liquid time constant neural network (LTC-NN) based architecture to recover underlying model of physical dynamics from real-world data. The automatic differentiation property of LTC-NN nodes overcomes problems associated with low sampling rates, the input dependent time constant in the forward pass of the hidden layer of LTC-NN nodes creates a massive search space of implicit physical dynamics, the physics model solver based data reconstruction loss guides the search for the correct set of implicit dynamics, and the use of the dropout regularization in the dense layer ensures extraction of the sparsest model. Further, to account for the perturbation timing error, we utilize dense layer nodes to search through input shifts that results in the lowest reconstruction loss. Experiments on four benchmark dynamical systems, three with simulation data and one with the real-world data show that the LTC-NN architecture is more accurate in recovering implicit physics model coefficients than the state-of-the-art sparse model recovery approaches. We also introduce four additional case studies (total eight) on real-life medical examples in simulation and with real-world clinical data to show effectiveness of our approach in recovering underlying model in practice. Ayan Banerjee 0001, Sandeep K. S. Gupta |
ECAI | 1 |
| 2024 | Synthesizing Operationally Safe Controllers for Human-in-the-Loop Human-in-the-Plant Hybrid Close Loop Systems
Ayan Banerjee 0001, Imane Lamrani, Sandeep K. S. Gupta |
ICPR (29) | 1 |
| 2024 | Detection of Unknown Errors in Human-Centered Systems
Aranyak Maity, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ICPR (9) | 2 |
| 2023 | High Fidelity Fast Simulation of Human in the Loop Human in the Plant (HIL-HIP) systemsabstractNon-linearities in simulation arise from the time variance in wire- less mobile networks when integrated with human in the loop, human in the plant (HIL-HIP) physical systems under dynamic con- texts, leading to simulation slowdown. Time variance is handled by deriving a series of piece wise linear time invariant simulations (PLIS) in intervals, which are then concatenated in time domain. In this paper, we conduct a formal analysis of the impact of dis- cretizing time-varying components in wireless network-controlled HIL-HIP systems on simulation accuracy and speedup and evaluate trade-offs with reliable guarantees. We develop an accurate simula- tion framework for an artificial pancreas wireless network system that controls blood glucose in Type 1 Diabetes patients with time varying properties such as, physiological changes associated with psychological stress and meal patterns. PLIS approach achieves accurate simulation with > 2.1 times speedup than a non-linear system simulation for the given dataset. Ayan Banerjee 0001, Payal Kamboj, Aranyak Maity, Riya Sudhakar Salian, Sandeep K. S. Gupta |
MSWiM | 1 |
| 2021 | Engendering Trust in Automated Feedback: A Two Step Comparison of Feedbacks in Gesture Based Learning
Sameena Hossain, Azamat Kamzin, Venkata Naga Sai Apurupa Amperayani, Prajwal Paudyal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
AIED (1) | 5 |
| 2021 | Operational Data-Driven Feedback for Safety Evaluation of Agent-Based Cyber-Physical SystemsabstractSafety regulation of safety-critical agent-based cyber-physical systems (CPS) which are manufactured in large scale such as next-gen aircrafts, autonomous driving vehicles, and medical devices is a multifaceted problem. CPS deployments can be presented with new safety-critical scenarios and novel inputs. Hence, operational characteristics of the CPS can be quite different from its safety approved design. This article considers a safety assurance solution where operational data from the sensors and actuators in the field of deployment is fed back to the manufacturing process through the Internet of Things infrastructure to assure and improve operational safety. It considers two cases: 1) model-aware, where the safety assured CPS design is fully specified; 2) modelagnostic, where limited specifications exist. For both the cases, it presents a data science based approach, N-HyMn, that learns a hybrid automaton model of the operational characteristics of the CPS from the input/output (I/O) traces of the observable parameters. For the model-aware case, it investigates the presence of inconsistencies between the learned model and the specifications model provided by the manufacturer, thus facilitating the detection of safety problems that may have been overlooked. For the modelagnostic case, it can detect potential safety failures. We show the usage of N-HyMn on the Medtronic Minimed 670 G system. N-HyMn correctly infers the hybrid automaton specifications of the Minimed 670 G and was able to detect a self-adaptation mechanism that is not declared explicitly in the certification documents of the U Food and Drug Administration. N-HyMn has a computational complexity of O(kn2), where k is the number of samples in the I/O trace, and n is the number of continuous variables. Imane Lamrani, Ayan Banerjee 0001, Sandeep K. S. Gupta |
IEEE Trans. Ind. Informatics | 2 |
| 2020 | Toward Operational Safety Verification of AI-Enabled CPS (Student Abstract)abstractAI-enabled Cyber-physical systems (CPS) such as artificial pancreas (AP) or autonomous cars are using machine learning to make several critical decisions. The system is subject to inputs and scenarios which are not observed during training and the expected outputs are not known. Hence, popular model based verification techniques that characterize behavior of a control system before deployment using predictive models may be inaccurate and often result in incorrect safety analysis results. In addition, regulatory agencies are required to regulate safety-critical AI enabled CPS to ensure their operational safety. However, high complexity of the system result in myriad of safety concerns all of which may not only be comprehensively tested before deployment but also may not even be detected during design and testing phase. In this work, we propose a tool to help regulatory agencies compare the operation of the CPS with the specifications given by the manufacturer to ensure that the operation results conform with the safety assured design of a CPS. Imane Lamrani, Ayan Banerjee 0001, Sandeep K. S. Gupta |
AAAI | 2 |
| 2020 | AI Enabled Tutor for Accessible Training
Ayan Banerjee 0001, Imane Lamrani, Sameena Hossain, Prajwal Paudyal, Sandeep K. S. Gupta |
AIED (1) | 1 |
| 2020 | Concept Embedding through Canonical Forms: A Case Study on Zero-Shot ASL RecognitionabstractIn the recognition problem, a canonical form that expresses the spatio-temporal relation of concepts for a given class can potentially increase accuracy. Concepts are defined as attributes that can be recognized using a soft matching paradigm. We consider the specific case study of American Sign Language (ASL) to show that canonical forms of classes can be used to recognize unseen gestures. There are several advantages of a canonical form of gestures including translation between gestures, gesture-based searching, and automated transcription of gestures into any spoken language. We applied our technique to two independently collected datasets: a) IMPACT Lab dataset: 23 ASL gestures each executed three times from 130 first time ASL learners as training data and b) ASLTEXT dataset: 190 gestures each executed six times on an average. Our technique was able to recognize 19 arbitrarily chosen previously unseen gestures in the IMPACT dataset from seven individuals who are not a part of 130 and 34 unseen gestures from the ASLTEXT dataset without any retraining. Our normalized accuracy on the ASLTEXT dataset is 66% which is 13.6 % higher than the state-of-art technique. Comparison with deep learning techniques revealed that incorporation of concept level knowledge can potentially alleviate under-fitting problems. Azamat Kamzin, Venkata Naga Sai Apurupa Amperayani, Prasanth Sukhapalli, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ICPR | 4 |
| 2019 | Expert Guided Rule Based Prioritization of Scientifically Relevant Images for Downlinking over Limited Bandwidth from Planetary OrbitersabstractInstruments onboard spacecraft acquire large amounts of data which is to be transmitted over a very low bandwidth. Consequently for some missions, the volume of data collected greatly exceeds the volume that can be downlinked before the next orbit. This necessitates the introduction of an intelligent autonomous decision making module that maximizes the return of the most scientifically relevant dataset over the low bandwidth for experts to analyze further. We propose an iterative rule based approach, guided by expert knowledge, to represent scientifically interesting geological landforms with respect to expert selected attributes. The rules are utilized to assign a priority based on how novel a test instance is with respect to its rule. High priority instances from the test set are used to iteratively update the learned rules. We then determine the effectiveness of the proposed approach on images acquired by a Mars orbiter and observe an expert-acceptable prioritization order generated by the rules that can potentially increase the return of scientifically relevant observations. Srija Chakraborty, Subhasish Das, Ayan Banerjee 0001, Sandeep K. S. Gupta, Philip Christensen |
AAAI | 3 |
| 2019 | DAVEE: A Deaf Accessible Virtual Environment for EducationabstractThe post-graduate enrollment statistic for deaf and hard of hearing (DHH) in Science, Technology, Math and Engineering (STEM) fields is very low compared to the hearing population. This drastically reduces DHH representation in the Information Technology (IT) workforce or academic research. DHH students generally use sign language interpreters to understand lecture materials but technically qualified interpreters are rare. These days, traditional in-person classes are being replaced with Massive Open Online Courses (MOOC). MOOCs improve access to materials, but hinder opportunities for collaboration which is vital for the DHH population. In this work, we propose DAVEE, a Virtual Reality (VR) classroom experience that facilitates live interpretation. During live sessions, DHH students can ask questions, receive feedback and have interactions with other students. The lectures and the interpretations can also be recorded for offline viewing. Prajwal Paudyal, Ayan Banerjee 0001, Yijian Hu, Sandeep K. S. Gupta |
Creativity & Cognition | 2 |
| 2019 | A User-adaptive Modeling for Eating Action Identification from Wristband Time SeriesabstractEating activity monitoring using wearable sensors can potentially enable interventions based on eating speed to mitigate the risks of critical healthcare problems such as obesity or diabetes. Eating actions are poly-componential gestures composed of sequential arrangements of three distinct components interspersed with gestures that may be unrelated to eating. This makes it extremely challenging to accurately identify eating actions. The primary reasons for the lack of acceptance of state-of-the-art eating action monitoring techniques include the following: (i) the need to install wearable sensors that are cumbersome to wear or limit the mobility of the user, (ii) the need for manual input from the user, and (iii) poor accuracy in the absence of manual inputs. In this work, we propose a novel methodology, IDEA, that performs accurate eating action identification within eating episodes with an average F1 score of 0.92. This is an improvement of 0.11 for precision and 0.15 for recall for the worst-case users as compared to the state of the art. IDEA uses only a single wristband and provides feedback on eating speed every 2 min without obtaining any manual input from the user. Junghyo Lee, Prajwal Paudyal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ACM Trans. Interact. Intell. Syst. | 3 |
| 2019 | A Comparison of Techniques for Sign Language Alphabet Recognition Using Armband WearablesabstractRecent research has shown that reliable recognition of sign language words and phrases using user-friendly and noninvasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 alphabets for the American Sign Language (ASL)). Detailed analysis of the algorithm used as well as comparisons with other traditional machine-learning algorithms is provided. The system uses an independent multiple-agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-time mobile applications. A thorough explanation and analysis is presented on results obtained on the ASL alphabet corpus for nine people with limited training. An average recognition accuracy of 95.36% is reported and compared with recognition results from other machine-learning techniques. This result is extended by including six additional validation users with data collected under similar settings as the previous dataset. Furthermore, a feature selection schema using a subset of the sensors is proposed and the results are evaluated. The mobile, noninvasive, and real-time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations. A brief discussion of the user interface is provided along with guidelines for best practices. Prajwal Paudyal, Junghyo Lee, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ACM Trans. Interact. Intell. Syst. | 3 |
| 2019 | ContextAiDe: End-to-End Architecture for Mobile Crowd-sensing ApplicationsabstractMobile crowd-sensing (MCS) enables development of context-aware applications by mining relevant information from a large set of devices selected in an ad hoc manner. For example, MCS has been used for real-time monitoring such as Vehicle ad hoc Networks-based traffic updates as well as offline data mining and tagging for future use in applications with location-based services. However, MCS could be potentially used for much more demanding applications such as real-time perpetrator tracking by online mining of images from nearby mobile users. A recent example is tracking the miscreant responsible for the Boston bombing. We present a new design approach for tracking using MCS for such complex processing in real time. Since MCS applications assume an unreliable underlying computational platform, most typically sample size for recruited devices is guided by concerns such as fault tolerance and reliability of information. As the real-time requirements get stricter coupled with increasing complexity of data-mining approaches, the communication and computation overheads can impose a very tight constraint on the sample size of devices needed for realizing real-time operation. This results in trade-off in acquiring context-relevant data and resource usage incurred while the real-time operation requirements get updated dynamically. Such effects have not been properly studied and optimized to enable real-time MCS applications such as perpetrator tracking. In this article, we propose ContextAiDe architecture, a combination of API, middleware, and optimization engine. The key innovation in ContextAiDe is context-optimized recruitment for execution of computation- and communication-heavy MCS applications in edge environment. ContextAiDe uses a notion of two types of contexts, exact (hard constraints), which have to be satisfied, and preferred (soft constraints), which may be satisfied to a certain degree. By adjusting the preferred contexts, ContextAiDe can optimize the operational overheads to enable real-time operation. ContextAiDe provides an API to specify contexts requirements and the code of MCS app, offload execution environment, a middleware that enables context-optimized and a fault-tolerant distributed execution. ContextAiDe evaluation using a real-time perpetrator tracking application shows reduced energy consumption of 37.8%, decrease in data transfer of 24.8%, and 43% less time compared to existing strategy. In spite of a small increase in the minimum distance from the perpetrator, iterations of optimization tracks the perpetrator successfully. Pro-actively learning the context and using stochastic optimization strategy minimizes the performance degradation caused due to uncertainty (<20%) in usage-dependent contexts. Madhurima Pore, Vinaya Chakati, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ACM Trans. Internet Techn. | 3 |
| 2018 | IDEA: Instant Detection of Eating Action using Wrist-Worn Sensors in Absence of User-Specific ModelabstractEating activity monitoring using wearable sensors can potentially enable interventions based on eating speed for critical healthcare problems such as obesity or diabetes. We propose a novel methodology, IDEA that performs accurate eating action identification and provides feedback on eating speed. IDEA uses a single wristband with IMU sensors and functions without any manual intervention from the user. The F1 score for eating action identification was 0.92. Junghyo Lee, Prajwal Paudyal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
UMAP | 3 |
| 2017 | Model Guided Deep Learning Approach Towards Prediction of Physical System BehaviorabstractCyber-physical control systems involve a discrete computational algorithm to control continuous physical systems. Often the control algorithm uses predictive models of the physical system in its decision making process. However, physical system models suffer from several inaccuracies when employed in practice. Mitigating such inaccuracies is often difficult and have to be repeated for different instances of the physical system. In this paper, we propose a model guided deep learning method for extraction of accurate prediction models of physical systems, in presence of artifacts observed in real life deployments. Given an initial potentially suboptimal mathematical prediction model, our model guided deep learning method iteratively improves the model through a data driven training approach. We apply the proposed approach on the closed loop blood glucose control system. Using this proposed approach, we achieve an improvement over predictive Bergman Minimal Model by a factor of around 100. Subhasish Das, Anurag Agrawal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ICMLA | 3 |
| 2017 | FIT-EVE&ADAM: Estimation of Velocity & Energy for Automated Diet Activity MonitoringabstractState-of-the-art techniques for eating activities analysis in dietary monitoring require significant user intervention, which is reported to be one of the major reasons for low adherence. There are limited works using wearables for fine-grained analysis of eating activities in terms of the eating speed, the type of food consumed, and the portion sizes. In this paper, we propose FIT-EVE&ADAM, an armband based diet monitoring system that provides such fine-grained analysis, triggered by a single hand gesture. The system collects the user's gesture using sensors such as electromyogram embedded in the armband device, along with food image data using color and thermal cameras. Finally, a novel feature selection method is applied on the data features to estimate eating speed and caloric intake with high accuracy (0.96 F1 score). Junghyo Lee, Prajwal Paudyal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
ICMLA | 3 |
| 2017 | Geometrical Analysis of Machine Learning Security in Biometric Authentication SystemsabstractFeature extraction and Machine Learning (ML) techniques are required to reduce high variability of biometric data in Biometric Authentication Systems (BAS) toward improving system utilization (acceptance of legitimate subjects). However, reduction in data variability, also decreases the adversary’s effort in manufacturing legitimate biometric data to break the system (security strength). Typically for BAS design, security strength is evaluated through variability analysis on data, regardless of feature extraction and ML, which are essential for accurate evaluation. In this research, we provide a geometrical method to measure the security strength in BAS, which analyzes the effects of feature extraction and ML on the biometric data. Using the proposed method, we evaluate the security strength of five state-of-the-art electroencephalogram-based authentication systems, on data from 106 subjects, and the maximum achievable security strength is 83 bits. Koosha Sadeghi, Ayan Banerjee 0001, Javad Sohankar, Sandeep K. S. Gupta |
ICMLA | 2 |
| 2017 | Performance and Security Strength Trade-Off in Machine Learning Based Biometric Authentication SystemsabstractIn Biometric Authentication Systems (BAS), the variability amongst population biometric data ensures distinctiveness, and helps minimizing false acceptance of non-subject data. However, higher variability implies temporal variations for a given subject, which can potentially reject subject data. Such variations are suppressed using feature extraction and Machine Learning (ML) techniques for improving the performance, but also reduce the adversary’s effort in breaking the system (security strength) using forged data. Typically for BAS design, performance and security strength are evaluated in isolation using experimental analysis. This research provides an analytical approach to evaluate the BAS performance and strength, and their trade-off, by modeling the biometric data, and studying the effect of feature extraction and ML configurations on processing the data. Experimental analysis on 106 subjects’ brain signal validates the analytical methodology results. Koosha Sadeghi, Ayan Banerjee 0001, Javad Sohankar, Sandeep K. S. Gupta |
ICMLA | 2 |
| 2017 | Estimation of dynamic parameters of MODIS NDVI time series nonlinear model using particle filteringabstractNormalized Difference Vegetation Index (NDVI) time series is used to study different land cover dynamics such as change, compare vegetation dynamics between years and analyze intra-annual components. A nonlinear cosine model of the NDVI time series with a constant frequency is used to account for the time-varying nature of the land cover parameters due to seasonality or change. The Extended Kalman Filter (EKF) is used to estimate these parameters, which introduces linearization and negatively impacts the state estimation accuracy. This paper proposes using a Particle Filter (PF) for state estimation to better address nonlinearity in the model. The cosine model is modified to capture frequency variations to account for changes in the vegetation growth cycle caused by abrupt phenomenon such as forest fires. PF obtains better state estimates than EKF, capturing the intra-annual components and time-varying frequency of the model accurately. Srija Chakraborty, Ayan Banerjee 0001, Sandeep K. S. Gupta, Antonia Papandreou-Suppappola, Philip Christensen |
IGARSS | 2 |
| 2017 | DyFAV: Dynamic Feature Selection and Voting for Real-time Recognition of Fingerspelled Alphabet using WearablesabstractRecent research has shown that reliable recognition of sign language words and phrases using user-friendly and non-invasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 letters for ASL). The system uses an independent multiple agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-time mobile applications. The results are demonstrated on the entire ASL alphabet corpus for nine people with limited training and average recognition accuracy of 95.36% is achieved which is better than the state-of-art for armband sensors. The mobile, non-invasive, and real time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations. Prajwal Paudyal, Junghyo Lee, Ayan Banerjee 0001, Sandeep K. S. Gupta |
IUI | 3 |
| 2016 | Optimization of Brain Mobile Interface Applications Using IoTabstractPervasive Brain Mobile Interfaces (BMoI) can be made more accurate and time efficient when knowledge from other sensors and computation power from available devices in the Internet of Things (IoT) infrastructure are utilized. This paper takes the example of Neuro-Movie (nMovie), an interactive movie application that blurs movie scenes based on mental state, to illustrate and analyze optimization opportunities when BMoI is interfaced with IoT. The three way trade-off between accuracy, real-time operation, and energy efficiency can be optimized through usage of physiological responses from IoT sensors and prediction algorithms. Latency and power models of BMoI are developed for thorough analysis of the trade-offs. Experiments on 10 volunteers show that: a) utilizing electrocardiogram responses to psychological stimulus increases the accuracy of mental state recognition by almost 10%, b) predictive models cover computation and communication latencies in the system to satisfy real-time requirements, and c) use of predictive models allows duty cycling of smartphone WiFi that potentially saves upto 71.6% communication energy. Koosha Sadeghi, Ayan Banerjee 0001, Javad Sohankar, Sandeep K. S. Gupta |
HiPC | 2 |
| 2016 | Toward Parametric Security Analysis of Machine Learning Based Cyber Forensic Biometric SystemsabstractMachine learning algorithms are widely used in cyber forensic biometric systems to analyze a subject's truthfulness in an interrogation. An analytical method (rather than experimental) to evaluate the security strength of these systems under potential cyber attacks is essential. In this paper, we formalize a theoretical method for analyzing the immunity of a machine learning based cyber forensic system against evidence tampering attack. We apply our theory on brain signal based forensic systems that use neural networks to classify responses from a subject. Attack simulation is run to validate our theoretical analysis results. Koosha Sadeghi, Ayan Banerjee 0001, Javad Sohankar, Sandeep K. S. Gupta |
ICMLA | 2 |
| 2016 | SCEPTRE: A Pervasive, Non-Invasive, and Programmable Gesture Recognition TechnologyabstractCommunication and collaboration between deaf people and hearing people is hindered by lack of a common language. Although there has been a lot of research in this domain, there is room for work towards a system that is ubiquitous, non-invasive, works in real-time and can be trained interactively by the user. Such a system will be powerful enough to translate gestures performed in real-time, while also being flexible enough to be fully personalized to be used as a platform for gesture based HCI. We propose SCEPTRE which utilizes two non-invasive wrist-worn devices to decipher gesture-based communication. The system uses a multi-tiered template based comparison system for classification on input data from accelerometer, gyroscope and electromyography (EMG) sensors. This work demonstrates that the system is very easily trained using just one to three training instances each for twenty randomly chosen signs from the American Sign Language(ASL) dictionary and also for user-generated custom gestures. The system is able to achieve an accuracy of 97.72% for ASL gestures. Prajwal Paudyal, Ayan Banerjee 0001, Sandeep K. S. Gupta |
IUI | 2 |
| 2016 | MT-Diet: Automated smartphone based diet assessment with infrared imagesabstractIn this paper, we propose MT-Diet, a smartphone-based automated diet monitoring system that interfaces a thermal camera with a smartphone and identifies types of food consumed at the click of a button. The system uses thermal maps of a food plate to increase accuracy of segmentation and extraction of food parts, and combines thermal and visual images to improve accuracy in the detection of cooked food. Test results on 80 different types of cooked food show that MT-Diet can isolate food parts with an accuracy of 97.5% and determine the type of food with an accuracy of 88.93%, which is a significant improvement (nearly 25%) over the state-of-the-art. Junghyo Lee, Ayan Banerjee 0001, Sandeep K. S. Gupta |
PerCom | 2 |
| 2015 | Analysis of Smart Mobile Applications for Healthcare under Dynamic Context ChangesabstractSmart mobile medical computing systems (SMDCSes), e.g., mobile medical applications use context information from the environment to provide useful and often critical healthcare services such as continuous monitoring and control of blood glucose levels by infusion of insulin. Given the unsupervised nature of operation of SMDCSes, context changes that are unaccounted for can cause unprecedented faults leading to violation of requirements such as safety, energy sustainability and reliability. Analysis of SMDCSes for testing requirements violations necessitates consideration of context dependent interactions between the SMDCS software, represented by discrete operating modes and its environment, represented by non-linear partial differential equations over space and time. An intractable number of context change sequence and lack of closed form solutions to differential equations makes the requirements analysis of SMDCSes a challenging task. This paper proposes a novel technique to analyze SMDCSes taking into account the dynamic changes in the context and the constant interaction of the computing systems with the physical environment. To show the usage of the technique, Ayushman pervasive health monitoring system is considered as an example SMDCS. Analytical results show that practices considered healthy for a person such as mobility may not be beneficial when an SMDCS is controlling health. Ayan Banerjee 0001, Sandeep K. S. Gupta |
IEEE Trans. Mob. Comput. | 1 |
| 2014 | Optimal Design for Symbiotic Wearable Wireless SensorsabstractSensors aesthetically embedded in accessoriessuch as jewelry, piercings or contact lenses arebeing proposed recently. These symbiotic wearable wirelesssensors are envisioned to operate on scarce harvestedenergy resources from the human body. In addition tothe hardware and software constraints arising from theform-factor and low energy operations, there are safetyrequirements such as avoidance of physical injury. Thedesign implications of these requirements are non-intuitiveand may involve estimation of human physiological dynamics. The physical impact of a sensor operation canbe controlled by appropriate design of multiple sensorcomponents such as processor, radio, and optimization ofdata algorithm. For example, the risk of thermal injury totissue can be reduced by limiting the sensing frequency, the computation power, and the radio duty cycle of bodyworn sensor. Hence, it is a challenging task to trace backa cause of a physical impact to hardware and softwaredesign decisions in a sensor. This paper proposes a novelnon-linear optimization framework to consider safety andsustainability requirements that depend on the humanphysiology and derive system level design parameters of asensor. We demonstrate our methodology using three casestudies: a) continuously monitoring ECG sensor sustainedby body heat, b) thermally safe network of implantedsensors, and c) infusion pump control algorithm to avoidhypo-glycemia. Priyanka Bagade, Ayan Banerjee 0001, Sandeep K. S. Gupta |
BSN | 2 |
| 2014 | Performance evaluation of multi core systems for high throughput medical applications involving model predictive controlabstractMany medical control devices used in case of critical patients have model predictive controllers (MPC). MPC estimate the drug level in the parts of patients body based on their human physiology model to either alarm the medical authority or change the drug infusion rate. This model prediction has to be completed before the drug infusion rate is changed i.e. every few seconds. Instead of mathematical models like the Pharmacokinetic models more accurate models such as spatio-temporal drug diffusion can be used for improving the prediction and prevention of drug overshoot and undershoot. However, these models require high computation capability of platforms like recent many core GPUs or Intel Xeon Phi (MIC) or IntelCore i7. This work explores thread level and data level parallelism and computation versus communication times of such different model predictive applications used in multiple patient monitoring in hospital data centers exploiting the many core platforms for maximizing the throughput (i.e. patients monitored simultaneously). We also study the energy and performance of these applications to evaluate them for architecture suitability. We show that given a set of MPC applications, mapping on heterogeneous platforms can give performance improvement and energy savings. Madhurima Pore, Ayan Banerjee 0001, Sandeep K. S. Gupta |
HiPC | 2 |
| 2013 | Protect your BSN: No Handshakes, just Namaste!abstractPrivacy of physiological data collected by a network of embedded sensors on human body is an important issue to be considered. Physiological signal-based security is a light weight solution which eliminates the need for security key storage and complex exponentiation computation in sensors. An important concern is whether such security measures are vulnerable to attacks, where the attacker is in close proximity to the BSN and senses physiological signals through processes such as electromagnetic coupling. Recent studies show that when two individuals are in close proximity, the electrocardiogram of one person gets coupled to the electroencephalogram of the other, thus indicating a possibility of proximity-based security attacks. This paper proposes a model-driven approach to proximity-based attack on security using physiological signals and evaluates its feasibility. Results show that a proximity-based attack can be successful even without the exact reconstruction of the physiological data sensed by the attacked BSN. Priyanka Bagade, Ayan Banerjee 0001, Joseph Milazzo, Sandeep K. S. Gupta |
BSN | 2 |
| 2013 | Protect your BSN: No Handshakes, just Namaste!abstractPrivacy of physiological data collected by a network of embedded sensors on human body is an important issue to be considered. Physiological signal-based security is a light weight solution which eliminates the need for security key storage and complex exponentiation computation in sensors. An important concern is whether such security measures are vulnerable to attacks, where the attacker is in close proximity to the BSN and senses physiological signals through processes such as electromagnetic coupling. Recent studies show that when two individuals are in close proximity, the electrocardiogram of one person gets coupled to the electroencephalogram of the other, thus indicating a possibility of proximity-based security attacks. This paper proposes a model-driven approach to proximity-based attack on security using physiological signals and evaluates its feasibility. Results show that a proximity-based attack can be successful even without the exact reconstruction of the physiological data sensed by the attacked BSN. Priyanka Bagade, Ayan Banerjee 0001, Joseph Milazzo, Sandeep K. S. Gupta |
BSN | 2 |
| 2013 | Multi-tier energy buffering management for IDCs with heterogeneous energy storage devicesabstractEnergy buffering, has been proposed to store renewable energy and low cost electricity in Energy Storage Devices (ESDs) and use it judiciously to reduce electricity bill in Internet data centers. Recent research have considered long term variation in electricity price, renewable power and workload and have shown the efficiency of energy buffering in reducing electricity bill. However, these aspects of data centers exhibit both long and short term variation. Further, there is inherent heterogeneity in ESD physical characteristics (e.g., charging and discharging rates). We hypothesize that a multi-tier energy buffering management can leverage the heterogeneity in ESD characteristics and better optimize utilization of renewable energy and low-cost power in presence of both short and long term variabilities in a data center. This paper proposes an analytical study of multi-tier workload and energy buffering management technique that frames each tier as an optimization problem and solves them in an online and proactive way using Receding Horizon Control (RHC). Our study shows that multi-tier energy buffering management increases the utilization of the renewables by upto two times compared to one-tier management. Zahra Abbasi, Madhurima Pore, Ayan Banerjee 0001, Sandeep K. S. Gupta |
HiPC | 3 |
| 2013 | Effects of phase imbalance on data center energy managementabstractPhase imbalance has been considered as a source of inefficiency in the data center that causes energy loss due to line impedance and increases reactive power. Strategies assume high loss due to phase imbalance and propose sophisticated energy management algorithms including phase balance aware workload scheduling algorithm and dynamic power distribution unit assignment to servers. However, such attempts do not utilize an objective measure of the inefficiencies due to phase imbalance to evaluate the significance of their contributions. Excessive imbalance in a three phase load has various undesirable effects. This paper, first objectively characterizes the inefficiencies due to phase imbalance and then provides numerical measures of the losses in realistic data center deployments. Phase imbalanced load in a delta configuration results in reduced power factor, which is undesirable for several reasons. Also, an imbalanced load (both in delta or star configuration), results in higher line currents, leading to higher line loss. However, this increase in loss is a fraction of a percentage of the energy consumed. The paper also discusses effects of work load scheduling on phase imbalance, and how to minimize the same. Sushil Gupta, Ayan Banerjee 0001, Zahra Abbasi, Sandeep K. S. Gupta |
HiPC | 2 |
| 2012 | Health-Dev: Model Based Development Pervasive Health Monitoring SystemsabstractImplementing requirements verified body worn medical sensors and smart phones, acting as base stations, in Body Sensor Networks (BSNs), is of extreme importance for development of reliable pervasive health monitoring systems (PHMS). Models of BSNs have been used to analyze designs with respect to requirements such as energy consumption, lifetime, and network reliability under dynamic context changes due to user mobility. This paper proposes Health-Dev that takes a high level specification of requirements verified BSN design and automatically generates both the sensor and smart phone code. Case studies related to energy efficiency and mobility aware network reliability show whether the resulting implementation satisfies the requirements set forth in the design phase. Ayan Banerjee 0001, Sunit Verma, Priyanka Bagade, Sandeep K. S. Gupta |
BSN | 1 |
| 2012 | Your mobility can be injurious to your health: Analyzing pervasive health monitoring systems under dynamic context changesabstractThe advent of smart phones has enabled health care anywhere and anytime. With pervasive health care, a person can perform necessary day to day tasks while his health is recorded, controlled, and processed continuously using on body sensors and actuators to capture abnormalities, trends, and causes. In such a scenario, seamless operation of the pervasive health management system (PHMS), given dynamic changes in the context induced by mobility of the user is of utmost importance. Such context changes dynamically affect many aspects such as the processing requirements of the health management application, the available energy sources, the interaction between a medical device with the human body. For social acceptability of PHMSes, they have to be tested and verified for their safe, energy sustainable (long term) and reliable operation under such dynamically changing environment. This paper proposes a novel technique to analyze PHMSes under mobility induced dynamic changes in the context and constant interaction of the medical device with the human body. Results show that human mobility induced context changes can cause unsafe conditions such as drug overdose. Ayan Banerjee 0001, Sandeep K. S. Gupta |
PerCom | 1 |
| 2012 | Ensuring Safety, Security, and Sustainability of Mission-Critical Cyber-Physical SystemsabstractCyber-physical systems (CPSs) couple their cyber and physical parts to provide mission-critical services, including automated pervasive health care, smart electricity grid, green cloud computing, and surveillance with unmanned aerial vehicles (UAVs). CPSs can use the information available from the physical environment to provide such ubiquitous, energy-efficient and low-cost functionalities. Their operation needs to ensure three key properties, collectively referred to as S3: 1) safety: avoidance of hazards; 2) security: assurance of integrity, authenticity, and confidentiality of information; and 3) sustainability: maintenance of long-term operation of CPSs using green sources of energy. Ensuring S3 properties in a CPS is a challenging task given the spatio-temporal dynamics of the underlying physical environment. In this paper, the formal underpinnings of recent CPS S3 solutions are aligned together in a theoretical framework for cyber-physical interactions, empowering CPS researchers to systematically design solutions for ensuring safety, security, or sustainability. The general applicability of this framework is demonstrated with various exemplar solutions for S3 in diverse CPS domains. Further, insights are provided on some of the open research problems for ensuring S3 in CPSs. Ayan Banerjee 0001, Krishna K. Venkatasubramanian, Tridib Mukherjee, Sandeep K. S. Gupta |
Proc. IEEE | 1 |
| 2012 | BAND-AiDe: A Tool for Cyber-Physical Oriented Analysis and Design of Body Area Networks and DevicesabstractBody area networks (BANs) are networks of medical devices implanted within or worn on the human body. Analysis and verification of BAN designs require (i) early feedback on the BAN design and (ii) high-confidence evaluation of BANs without requiring any hazardous, intrusive, and costly deployment. Any design of BAN further has to ensure (i) the safety of the human body, that is, limiting any undesirable side-effects (e.g., heat dissipation) of BAN operations (involving sensing, computation, and communication among the devices) on the human body, and (ii) the sustainability of the BAN operations, that is, the continuation of the operations under constrained resources (e.g., limited battery power in the devices) without requiring any redeployments. This article uses the Model Based Engineering (MBE) approach to perform design and analysis of BANs. In this regard, first, an abstract cyber-physical model of BANs, called BAN-CPS, is proposed that captures the undesirable side-effects of the medical devices (cyber) on the human body (physical); second, a design and analysis tool, named BAND-AiDe, is developed that allows specification of BAN-CPS using industry standard Abstract Architecture Description Language (AADL) and enables safety and sustainability analysis of BANs; and third, the applicability of BAND-AiDe is shown through a case study using both single and a network of medical devices for health monitoring applications. Ayan Banerjee 0001, Sailesh Kandula, Tridib Mukherjee, Sandeep K. S. Gupta |
ACM Trans. Embed. Comput. Syst. | 1 |
| 2011 | GeM-REM: Generative Model-Driven Resource Efficient ECG Monitoring in Body Sensor NetworksabstractWith recent advances in smart phones and wearable sensors, Body Sensor Networks (BSNs) have been proposed for use in continuous, remote electrocardiogram (ECG) monitoring. In such systems, sampling the ECG at clinically recommended rates (250 Hz) and wireless transmission of the collected data incurs high energy consumption at the energy-constrained body sensor. The large volume of collected data also makes data storage at the sensor infeasible. Thus, there is a need for reducing the energy consumption and data size at the sensor, while maintaining the ECG quality required for diagnosis. In this paper, we propose GeM-REM, a resource-efficient ECG monitoring method for BSNs. GeM-REM uses a generative ECG model at the base station and its lightweight version at the sensor. The sensor transmits data only when the sensed ECG deviates from model-based values, thus saving transmission energy. Further, the model parameters are continually updated based on the sensed ECG. The proposed approach enables storage of ECG data in terms of model parameters rather than data samples, which reduces the required storage space. Implementation on a sensor platform and evaluation using real ECG data from MIT-BIH dataset shows transmission energy and data storage reduction ratios of 42.1:1 and 37.3:1 respectively, which are better than state of the art ECG data compression schemes. Sidharth Nabar, Ayan Banerjee 0001, Sandeep K. S. Gupta, Radha Poovendran |
BSN | 2 |
| 2010 | Model-driven coordinated management of data centers
Tridib Mukherjee, Ayan Banerjee 0001, Georgios Varsamopoulos, Sandeep K. S. Gupta |
Comput. Networks | 2 |
| 2010 | PSKA: usable and secure key agreement scheme for body area networksabstractA body area network (BAN) is a wireless network of health monitoring sensors designed to deliver personalized healthcare. Securing intersensor communications within BANs is essential for preserving not only the privacy of health data, but also for ensuring safety of healthcare delivery. This paper presents physiological-signal-based key agreement (PSKA), a scheme for enabling secure intersensor communication within a BAN in a usable (plug-n-play, transparent) manner. PSKA allows neighboring nodes in a BAN to agree to a symmetric (shared) cryptographic key, in an authenticated manner, using physiological signals obtained from the subject. No initialization or predeployment is required; simply deploying sensors in a BAN is enough to make them communicate securely. Our analysis, prototyping, and comparison with the frequently used Diffie-Hellman key agreement protocol shows that PSKA is a viable intersensor key agreement protocol for BANs. Krishna K. Venkatasubramanian, Ayan Banerjee 0001, Sandeep K. S. Gupta |
IEEE Trans. Inf. Technol. Biomed. | 2 |
| 2009 | Spatio-temporal thermal-aware job scheduling to minimize energy consumption in virtualized heterogeneous data centers
Tridib Mukherjee, Ayan Banerjee 0001, Georgios Varsamopoulos, Sandeep K. S. Gupta, Sanjay Rungta |
Comput. Networks | 2 |