Object detection is a critical component in various applications, yet conventional detection architectures are typically developed on RGB or grayscale databases, limiting their effectiveness in challenging real-world scenarios. To address this, this work utilizes broad-spectral databases to enhance detection in low-light and complex environments. We introduce a graph-based representation that captures both in-channel and cross-channel features in broad-spectral images, enabling a more comprehensive scene understanding. A Graph Convolutional Network (GCN)-based aggregation strategy is proposed to integrate information from multiple channels effectively. A data enhancement method is also proposed based on the imbalanced characteristics in the foreground and background regions. Experimental results show that our model achieves superior mean Average Precision (mAP@50) across multiple datasets.

Syllables are compositional units of spoken language that efficiently structure human speech perception and production. However, current neural speech representations lack such structure, resulting in dense token sequences that are costly to process. To bridge this gap, we propose a new model, Sylber, that produces speech representations with clean and robust syllabic structure. Specifically, we propose a self-supervised learning (SSL) framework that bootstraps syllabic embeddings by distilling from its own initial unsupervised syllabic segmentation. This results in a highly structured representation of speech features, offering three key benefits: 1) a fast, linear-time syllable segmentation algorithm, 2) efficient syllabic tokenization with an average of 4.27 tokens per second, and 3) novel phonological units suited for efficient spoken language modeling. Our proposed segmentation method is highly robust and generalizes to out-of-domain data and unseen languages without any tuning. By training token-to-speech generative models, fully intelligible speech can be reconstructed from Sylber tokens with a significantly lower bitrate than baseline SSL tokens. This suggests that our model effectively compresses speech into a compact sequence of tokens with minimal information loss. Lastly, we demonstrate that categorical perception—a linguistic phenomenon in speech perception—emerges naturally in Sylber, making the embedding space more categorical and sparse than previous speech features and thus supporting the high efficiency of our tokenization. Together, we present a novel SSL approach for representing speech as syllables, with significant potential for efficient speech tokenization and spoken language modeling.

Identifying Knowledge Components (KCs) in computer science education improves curriculum design and teaching strategies. We introduce a framework using Large Language Models to identify KCs from programming assignments automatically. Our framework helps educators align assignments with course objectives. GPT-4 identifies relevant KCs well, though there's a low match with expert-generated KCs at the course level. At the problem level, performance is lower, but key KCs are reasonably identified.

We introduce a gaze-tracking-free method to reduce OLED display power consumption in VR with minimal perceptual impact. This technique exploits the time course of chromatic adaptation, the human visual system's ability to maintain stable color perception under changing illumination. To that end, we propose a novel psychophysical paradigm that models how human adaptation state changes with the scene illuminant. We exploit this model to compute an optimal illuminant shift trajectory, controlling the rate and extent of illumination change, to reduce display power under a given perceptual loss budget. Our technique significantly improves the perceptual quality over prior work that applies illumination shifts instantaneously. Our technique can also be combined with prior work on luminance dimming to reduce display power by 31% with no statistical loss of perceptual quality.

Radio Frequency Fingerprints (RFFs) imprinted on the RF signals by imperfections of hardware manufacturing enable device authorization augmentation in the Internet of Things (IoT) communication. However, the identifiable RF fingerprints are exposed to potential adversary attackers that can mimic the identifiable RF fingerprints of the authorized devices and fool the classifier at the receiver to obstruct secure communications in an open environment. This work proposes an attacker based on a Generative Adversarial Network (GAN) with a Variational Autoencoder (VAE) to extract the identifiable RF fingerprints generated from over 220 authorized transmitters. At the receiver, a Convolutional-Neural-Network-(CNN)-based classifier is deployed and shows a multi-class False Positive Rate (FPR) of 97.7% on the signals synthesized by the attacker at 30 dB Signal-to-Noise Ratio (SNR). In dealing with secure communication featuring time-varying RFFs, it’s essential to also monitor the time required for attackers to adapt. Within 60 seconds, results indicate that there’s a 92.9% probability of deceiving the receiver at 30 dB, with the possibility remaining at 87.4%s even when the SNR drops to 5 dB, which shows the attacker’s capability to overcome a wide range of SNR conditions. Furthermore, an evaluation of the distance effect highlights its robustness to device movement.

We propose an assistive technology that helps individuals with Color Vision Deficiencies (CVD) to recognize/name colors. A dichromat’s color perception is a reduced two-dimensional (2D) subset of a normal trichromat’s three dimensional color (3D) perception, leading to confusion when visual stimuli that appear identical to the dichromat are referred to by different color names. Using our proposed system, CVD individuals can interactively induce distinct perceptual changes to originally confusing colors via a computational color space transformation. By combining their original 2D precepts for colors with the discriminative changes, a three dimensional color space is reconstructed, where the dichromat can learn to resolve color name confusions and accurately recognize colors. Our system is implemented as an Augmented Reality (AR) interface on smartphones, where users interactively control the rotation through swipe gestures and observe the induced color shifts in the camera view or in a displayed image. Through psychophysical experiments and a longitudinal user study, we demonstrate that such rotational color shifts have discriminative power (initially confusing colors become distinct under rotation) and exhibit structured perceptual shifts dichromats can learn with modest training. The AR App is also evaluated in two real-world scenarios (building with lego blocks and interpreting artistic works); users all report positive experience in using the App to recognize object colors that they otherwise could not.

Color programmers manipulate lights, materials, and the resulting colors from light-material interactions. Existing libraries for color programming provide only a thin layer of abstraction around matrix operations. Color programs are, thus, vulnerable to bugs arising from mathematically permissible but physically meaningless matrix computations. Correct implementations are difficult to write and optimize. We introduce C ooler S pace to facilitate physically correct and computationally efficient color programming. C ooler S pace raises the level of abstraction of color programming by allowing programmers to focus on describing the logic of color physics. Correctness and efficiency are handled by C ooler S pace . The type system in C ooler S pace assigns physical meaning and dimensions to user-defined objects. The typing rules permit only legal computations informed by color physics and perception. Along with type checking, C ooler S pace also generates performance-optimized programs using equality saturation. C ooler S pace is implemented as a Python library and compiles to ONNX, a common intermediate representation for tensor computations. C ooler S pace not only prevents common errors in color programming, but also does so without run-time overhead: even unoptimized C ooler S pace programs out-perform existing Python-based color programming systems by up to 5.7 times; our optimizations provide up to an additional 1.4 times speed-up.

A reconfigurable power amplifier (PA) is implemented in CMOS 65nm to enable radio identification for secure wireless communication by injecting tunable radio frequency fingerprints (RFFs) into the physical layer. The large ensemble of RFFs is achieved by offsetting the distributions of process variations affecting the PA’s hardware features. The resulting large RFF capacity is exploited to increase resilience to noise and temperature changes by selecting distinct RFFs from the ensemble and reconfiguring the PA to restore nominal RFFs following temperature shifts. The secure PA achieves over 14000 time-varying RFFs while consuming only 22 mW and occupying a core area of$<$0.0951 mm$^{2}$. A reinforcement learning (RL)-based control has been implemented on FPGA for closed-loop reconfiguration of the transmitter to achieve robust and low-overhead security measures that overcome noise and temperature influences in dynamic environments.

One means by which the security of Internet-of-Things (IoT)-enabled devices may be augmented is through radio-frequency fingerprinting-based authentication methods. As variability in CMOS processes increases with technology scaling, the hardware imperfections that form RF fingerprints can be controlled with small reconfigurable elements, enabling the feasibility of RF fingerprinting as a low overhead security measure for device authentication. To achieve rapid RF identification, we present an inherently secure RF power amplifier and a convolutional neural network-based machine learning classifier through an exploration of combinatorial randomness and self-aware detection mechanisms. By selecting different subsets of thinly sliced power amplifier elements, combinations of random process variations are exploited and updated to form a large search space of distinct RF fingerprints and improve fingerprint prominence. The rich features enabled by augmented device primitives are updated in a time-varying manner to strengthen built-in hardware security. Measurement results demonstrate the effectiveness of this approach at generating distinguishable RF fingerprints across a significant number of configurations.

The equipment includes a laptop, Xilinx ZCU102 Dev Board, and a hard-disk drive (HDD) interface module. Fig. 1 illustrates the demonstration setup. All of the devices will be powered by their own power adaptors. The HDD interface module, which is provided by the Data Storage Systems Center (DSSC) at Carnegie Mellon University, can be controlled by the laptop to generate the raw readback signals for the machine- learning (ML) module implemented on the Xilinx ZCU102 to perform data symbol detection. The classified outputs are sent back to the laptop for result analysis and demonstration with a graphical user interface (GUI).

Machine-learning-based readout channels are presented for direct data symbol detection via decision-tree classification with gradient boosting for multiple-actuator data storage systems. The proposed learning module integrates energy-efficient linear classifiers to extract features and structures from raw readback signals. The results demonstrate high detection accuracy, which is robust to inter-symbol interference (ISI) and jitter noise. The low-complexity machine learning module classifies low signal-to-noise ratio raw data with an accuracy rate higher than 95.8% in real-time and consumes only 53 mW.

This work presents an ultra-low-power classifier that can be integrated within energy-constrained bio-sensors to enable rapid analysis for continuous health monitoring. The in-sensor classifier saves significant transmission energy by extracting critical information locally to eliminate the need of transmitting raw data to centralized servers for remote signal processing. The convolutional-neural-network (CNN)-based classifier is built by using reconfigurable delay-locked loops (DLLs) to carry out classification algorithms with time-domain multiply-accumulate (MAC) operations. Pseudo sigmoid activation functions are realized by regenerative comparators that transform weighted timing to probabilities. The presented classifier achieves low-power consumption of 240.34 nW while performing up to 20 k operations per second. The proposed time-domain classifier reduces the energy to 36% of the previous works.

In this paper we present the use of Benign Hardware Trojans (BHT) as a security measure for an embedded system with a software component and a hardware execution environment. Based on delay logic, process variation, and selective transistor aging, the BHT can be incorporated into an embedded system for the software and the hardware components to authenticate each other before functional execution. We will demonstrate an implementation of such a BHT within an embedded system on a Xilinx Spartan-6 FPGA platform. Using the same platform we will also show that the BHT security measurement has a low to modest amount of performance overhead basing on the test results from a variety of synthetic and real world benchmarks.