Autonomous systems and smart-industry deployments increasingly split computation across near-sensor, edge, and cloud resources, where tight energy, latency, and reliability budgets demand runtime adaptivity. In practice, deciding what to compute and transmit at each point is pivotal; yet as multimodal sensor suites (cameras, LiDAR/depth, etc.) proliferate at the edge, most prior approaches either (i) fuse modalities on powerful servers or (ii) apply uni-modal near-sensor filters that ignore cross-modal dependencies, leading to redundant transmissions or missed events. We present Fusion-Sense, a fusion-aware intelligent sensing framework for energy-constrained autonomous edge systems. Lightweight near-sensor classifiers are trained via a three-step procedure: (i) a server-side fusion model learns the downstream task, (ii) filter-out-safe (FoS) labels quantify each modality's necessity relative to the fused decision, and (iii) an edge-side fusion model is compacted by injecting near-sensor predictions as auxiliary signals. The result is a runtime decision layer that jointly reduces compute and communication while scaling linearly with sensor count. On a dual-modality (RGB+Depth/LiDAR) setup with SynDrone, FusionSense sustains task quality at substantially higher data-reduction rates than unimodal filters and delivers large end-to-end gains: up to 33× lower energy at 1% FoI prevalence, 11× at 10%, a 92.3% reduction in quality loss at a fixed 30% data reduction, and roughly 1.5× higher energy savings than the best prior filtering baseline.

We propose an event-centric encoder that extends the CLIP ecosystem to neuromorphic streams, positioning events as a first-class modality in general-purpose multimodal learning. Our approach transfers CLIP’s image–text knowledge directly to the event domain by introducing a lightweight preprocessing pipeline that collapses raw streams into grayscale frames compatible with CLIP’s vision encoder. This simplification preserves CLIP’s scalability and zero-shot capability, while a carefully designed training scheme mitigates catastrophic forgetting through contrastive, consistency, and distributional alignment losses. The resulting encoder achieves competitive performance on object recognition, few-shot learning, and zero-shot anomaly detection, and generalizes to event streams synthesized from videos without additional training. Beyond standalone benchmarks, we demonstrate seamless integration into cross-modal architectures, enabling event–image retrieval and retrieval across sound and depth, thereby broadening CLIP’s ecosystem. While intentionally simple, our representation serves as a transferable starting point, and future extensions with polarity- or temporal-aware encodings could further exploit event-specific characteristics.

What role does a single vision token play inside a multimodal large language model (MLLM)? Despite recent successes, most MLLMs adopt a simple design: a projected visual feature z = P(fx) prepended to the text sequence. Yet it remains unclear whether this vector merely provides context or actively steers generation. We propose a geometric probing framework that analyzes latent–token alignment, intrinsic dimensionality, and perturbation sensitivity. Across four datasets and three representative MLLMs (LLaVA, BLIP-2, Kosmos-2), we find clear operating regimes: BLIP-2 enforces rigid low-rank compression with strong alignment but near-zero sensitivity, LLaVA exhibits flexible high-dimensional mappings with high responsiveness, and Kosmos-2 balances between them. These signatures correlate with downstream behavior—SQA correctness and VQAv2 hallucination severity—showing that reduced or excessive sensitivity predicts unreliable grounding. Our results highlight geometry as a diagnostic lens for vision–language coupling and offer actionable guidance for projection design, alignment objectives, and user-steerable multimodal generation.

Task-oriented object detection is increasingly essential for intelligent sensing applications, enabling AI systems to operate autonomously in complex, real-world environments such as autonomous driving, healthcare, and industrial automation. Conventional models often struggle with generalization, requiring vast datasets to accurately detect objects within diverse contexts. In this work, we introduce iTask, a taskoriented object detection framework that leverages large language models (LLMs) to generalize efficiently from limited samples by generating an abstract knowledge graph. This graph encapsulates essential task attributes, allowing iTask to identify objects based on high-level characteristics rather than extensive data, making it possible to adapt to complex mission requirements with minimal samples. iTask addresses the challenges of high computational cost and resource limitations in vision-language models by offering two configuration models: a distilled, task-specific vision transformer optimized for high accuracy in defined tasks, and a quantized version of the model for broader applicability across multiple tasks. Additionally, we designed a hardware acceleration circuit to support real-time processing, essential for edge devices that require low latency and efficient task execution. Our evaluations show that the task-specific configuration achieves a 15% higher accuracy over the quantized configuration in specific scenarios, while the quantized model provides robust multi-task performance. The hardware-accelerated iTask system achieves a $3.5 x$ speedup and a 40% reduction in energy consumption compared to GPU-based implementations. These results demonstrate that iTask’s dual-configuration approach and situational adaptability offer a scalable solution for task-specific object detection, providing robust and efficient performance in resourceconstrained environments.

Hyperdimensional computing (HDC) enables efficient data encoding and processing in high-dimensional spaces, benefiting machine learning and data analysis. However, under-utilization of these spaces can lead to overfitting and reduced model reliability, especially in data-limited systems-a critical issue in sectors like healthcare that demand robustness and consistent performance. We introduce BoostHD, an approach that applies boosting algorithms to partition the hyperdimensional space into subspaces, creating an ensemble of weak learners. By integrating boosting with HDC, BoostHD enhances performance and reliability beyond existing HDC methods. Our analysis highlights the importance of efficient utilization of hyperdimensional spaces for improved model performance. Experiments on healthcare datasets show that BoostHD outperforms state-of-the-art methods. On the WESAD dataset, it achieved an accuracy of 98.37% ± 0.32%, surpassing Random Forest, XGBoost, and On-lineHD. BoostHD also demonstrated superior inference efficiency and stability, maintaining high accuracy under data imbalance and noise. In person-specific evaluations, it achieved an average accuracy of 96.19%, outperforming other models. By addressing the limitations of both boosting and HDC, BoostHD expands the applicability of HDC in critical domains where reliability and precision are paramount.

Vision Transformers (ViTs) have emerged as the backbone of many segmentation models, consistently achieving state-of-the-art (SOTA) performance. However, their success comes at a significant computational cost. Image token pruning is one of the most effective strategies to address this complexity. However, previous approaches fall short when applied to more complex task-oriented segmentation (TOS), where the class of each image patch is not predefined but dependent on the specific input task. This work introduces the Vision Language Guided Token Pruning (VLTP), a novel token pruning mechanism that can accelerate ViT-based segmentation models, particularly for TOS guided by multi-modal large language model (MLLM). We argue that ViT does not need to process every image token through all of its layers—only the tokens related to reasoning tasks are necessary. We design a new pruning decoder to take both image tokens and vision-language guidance as input to predict the relevance of each image token to the task. Only image tokens with high relevance are passed to deeper layers of the ViT. Experiments show that the VLTP framework reduces the computational costs of ViT by approximately 25% without performance degradation and by around 40% with only a 1% performance drop. The code associated with this study can be found at this URL.

Human pose estimation (HPE) is crucial for various applications. However, deploying HPE algorithms in surveillance contexts raises significant privacy concerns due to the potential leakage of sensitive personal information (SPI) such as facial features, and ethnicity. Existing privacy-enhancing methods often compromise either privacy or performance, or they require costly additional modalities. We propose a novel privacy-enhancing system that generates privacy-enhanced portraits while maintaining high HPE performance. Our key innovations include the reversible recovery of SPI for authorized personnel and the preservation of contextual information. By Jointly optimizing a privacy-enhancing module, a privacy recovery module, and a pose estimator, our system ensures robust privacy protection, efficient SPI recovery, and high-performance HPE. Experimental results demonstrate the system's robust performance in privacy enhancement, SPI recovery, and HPE. The code associated with this study can be found at this URL.

In machine learning (ML), near-sensor AI is transforming edge computing by reducing response times and data transmission, ultimately saving energy and bandwidth. Despite challenges like limited computational resources and the need for transparent decision-making, this approach aims to enhance the intelligence and autonomy of edge devices. Our research presents a novel framework that adds a layer of abstract intelligence to sensors, boosting system efficiency and accuracy through transparent, interpretable sub-symbolic AI. We combine Bayesian algorithms with hyperdimensional computing (HDC), inspired by the human brain's operational efficiency, to deliver an energy-efficient solution matching the accuracy of traditional cloud systems without constant server dependence. This framework uses a binary classifier with Bayesian insights to choose the best data processing location---locally or in the cloud---adapting to data environments. Our method ensures cloud-level performance while significantly reducing energy consumption, improving the sustainability of sensor-based systems. It also enables continual adaptation and learning directly at the sensor level, enriching cloud models with fresh edge insights. Our results have shown to bridge the gap from around 38% quality loss between the standalone near-sensor HDC model and the SOTA cloud-based model to improve the quality loss to only 9% while simultaneously saving 45.34% of energy by not using the cloud. This framework paves the way for more sustainable, efficient, and accurate edge computing in the ML landscape by bridging the gap between simple near-sensor models and their advanced cloud-based counterparts.

Brain-inspired Hyper-dimensional computing (HDC) has recently shown promise as a lightweight machine learning approach. Despite its success, there are limited studies on the robustness of HDC models to adversarial attacks. In this paper, we introduce the first comparative study of the robustness between HDC and deep neural network (DNN) to malicious attacks. We develop a framework that enables HDC models to generate gradient-based adversarial examples using state-of-the-art techniques applied to DNNs. Our evaluation shows that HDC with a proper neural encoding module provides significantly higher robustness to adversarial attacks than existing DNNs. In addition, HDC models have high robustness to adversarial samples generated for DNNs.

Recent strides in deep learning have yielded impres-sive practical applications such as autonomous driving, natural language processing, and graph reasoning. However, the sus-ceptibility of deep learning models to subtle input variations, which stems from device imperfections and non-idealities, or adversarial attacks on edge devices, presents a critical challenge. These vulnerabilities hold dual significance-security concerns in critical applications and insights into human-machine sen-sory alignment. Efforts to enhance model robustness encounter resource constraints in the edge and the black box nature of neural networks, hindering their deployment on edge devices. This paper focuses on algorithmic adaptations inspired by the human brain to address these challenges. Hyper Dimensional Computing (HDC), rooted in neural principles, replicates brain functions while enabling efficient, noise-tolerant computation. HDC leverages high-dimensional vectors to encode information, seamlessly blending learning and memory functions. Its trans-parency empowers practitioners, enhancing both robustness and understanding of deployed models. In this paper, we introduce the first comprehensive study that compares the robustness of HDC to white-box malicious attacks to that of deep neural network (DNN) models and the first HDC gradient-based attack in the literature. We develop a framework that enables HDC models to generate gradient-based adversarial examples using state-of-the-art techniques applied to DNNs. Our evaluation shows that our HDC model provides, on average, 19.9% higher robustness than DNNs to adversarial samples and up to 90% robustness improvement against random noise on the weights of the model compared to the DNN.