Deep reinforcement learning (DRL) excels at learning control policies in high-dimensional action spaces, making it crucial for robotic manipulation. However, its real-world application is limited by costly and risky data collection. Offline reinforcement learning (offline RL) addresses this issue by training on precollected datasets but struggles with $Q$ value overestimation in high-dimensional discrete action spaces, where the number of out-of-distribution (OOD) actions rapidly increases, negatively impacting training stability. In this work, we propose the most overestimated $Q$ value regularization (MQR), a novel offline RL algorithm that penalizes the action with the most overestimated $Q$ value, effectively mitigating overestimation in high-dimensional discrete action spaces. By regulating the action most affected by $Q$ value overestimation, rather than applying uniform penalties across the entire action space as in existing methods, MQR further prevents the policy from converging incorrectly. We evaluate MQR on a robotic pushing and grasping task, a challenging high-dimensional discrete action space problem, in both simulated and real-world environments with random, dense, and unknown object arrangements. The results demonstrate that MQR significantly outperforms baseline algorithms, achieving a clearance rate of 96.94% in simulations and 99.04% in real-world dense configurations, while maintaining high action efficiency and stability. These findings highlight MQR's robustness, scalability, and adaptability for robotic manipulation, showcasing its potential for real-world deployment in industrial robotics. The code used in our research is publicly available at https://github.com/Hanyang-Robot/MQR.

Stroke is the fifth leading cause of death in the US. Early recognition and treatment of stroke are essential for a good clinical outcome. It is particularly challenging for Wake-Up Stroke (WUS) to know the time of stroke onset, hence golden time for treatment is easily missed. We propose a Real-tIme StroKe early detection system during Sleep (RISK-Sleep) using wristbands. RISK-Sleep is a solution for early stroke detection tailored for the sleep environment that is cost-effective and practical for daily use. Underneath RISK-Sleep, we define and utilize an abnormal sleep motion model consisting of abnormal intensity and abnormal frequency. The abnormal intensity indicates hemiparesis sleep motion patterns while the abnormal frequency means emergency situations such as full hemiparesis and full paralysis. Based on the model, we seek the best classifier that analyzes the aforementioned two abnormal motion patterns by sliding window in real-time. For performance evaluation, we collect sleep data from 30 healthy people and 14 stroke patients with hemiparesis. Evaluation results show that RISK-Sleep achieves classification accuracy of 96.00% in abnormal intensity with 146-minute window in the KNN classifier with SFS feature selection. In addition, the SVM classifier without feature selection shows classification accuracy of 100% with 108-minute window in abnormal frequency. We expect RISK-Sleep plays a significant role in reducing the incidence of WUS.

In this paper, we propose a spatio-temporal coordination-based media access control (STMAC) protocol for efficiently sharing driving safety information in urban vehicular networks. STMAC exploits a unique spatio-temporal feature characterized from a geometric relation among vehicles to form a line-of-collision graph, which shows the relationship among vehicles that may collide with each other. Based on this graph, we propose a contention-free channel access scheme to exchange safety messages simultaneously by employing directional antenna and transmission power control. Based on an urban road layout, we propose an optimized contention period schedule by considering the arrival rate of vehicles at an intersection in the communication range of a road-side unit to reduce vehicle registration time. Using theoretical analysis and extensive simulations, it is shown that STMAC outperforms legacy MAC protocols especially in a traffic congestion scenario. In the congestion case, STMAC can reduce the average superframe duration by 66.7%, packet end-to-end delay by 68.3%, and packet loss ratio by 88% in comparison with the existing MAC protocol for vehicle-to-infrastructure communication, based on the IEEE 802.11p.

Texting or browsing the web on a smartphone while driving, called distracted driving, significantly increases the risk of car accidents. There have been a number of proposals for the prevention of distracted driving, but none of them has addressed its important challenges completely and effectively. To remedy this deficiency, we present an event-driven solution, called Automatic Identification of Driver's Smartphone (AIDS), which identifies a driver's smartphone by analyzing and fusing the phone's sensory information related to common vehicle-riding activities, such as walking toward the vehicle, standing near the vehicle while opening a vehicle door, entering the vehicle, closing the door, and starting the engine. AIDS extracts features useful for identification of the driver's phone from diverse sensors available in commodity smartphones. It identifies the driver's phone before the vehicle leaves its parked spot, and differentiates seated (front or rear) rows in a vehicle by analyzing the subtle electromagnetic field spikes caused by the starting of the engine. To evaluate the feasibility and adaptability of AIDS, we have conducted extensive experiments: a prototype of AIDS was distributed to 12 participants, both males and females in their 20 and 30s, who have driven seven different vehicles for three days in real-world environments. Our evaluation results show that AIDS identified the driver's phone with an 83.3-93.3 percent true positive rate while achieving a 90.1-91.2 percent true negative rate at a marginal increase of the phone's energy consumption.

Traffic jams often occur without any obvious reasons such as traffic accidents, roadwork, or closed lanes. Under moderate to high traffic density, minor perturbations to traffic flow (e.g., a strong braking motion) are easily amplified into a wave of stop-and-go traffic. This is known as a phantom jam. In this paper, we aim to mitigate phantom jams leveraging the three-phase traffic theory and vehicle-to-vehicle (V2V) communication. More specifically, an efficient phantom jam control protocol is proposed in which a fuzzy inference system is integrated with a V2V-based phantom jam detection algorithm to effectively capture the dynamics of traffic jams. Per-lane speed difference under traffic congestion is taken into account in the protocol design, so that a phantom jam is controlled separately for each lane, improving the performance of the proposed protocol. We implemented the protocol in the Jist/SWAN traffic simulator. Simulations with artificially generated traffic data and real-world traffic data collected from vehicle loop detectors on Interstate 880, California, USA, demonstrate that our approach has by up to 9% and 4.9% smaller average travel times (at penetration rates of 10%) compared with a state-of-the-art approach, respectively.

With the advancement of technology in various domains, many efforts have been made to design advanced classification engines that aid the protection of civilians and their properties in different settings. In this work, we focus on a set of the population which is probably the most vulnerable: children. Specifically, we present ChildSafe , a classification system that exploits ratios of skeletal features extracted from children and adults using a 3D depth camera to classify visual characteristics between the two age groups. Specifically, we combine the ratio information into one bag-of-words feature for each sample, where each word is a histogram of the ratios. ChildSafe analyzes the words that are normalized within and between the two age groups and implements a fuzzy bin-based classification method that represents bin-boundaries using fuzzy sets. We train and evaluate ChildSafe using a large dataset of visual samples collected from 150 elementary school children and 150 adults, ranging in age from 7 to 50. Our results suggest that ChildSafe successfully detects children with a proper classification rate of up to 94%, a false-negative rate as low as 1.82%, and a low false-positive rate of 5.14%. We envision this work as a first step, an effective subsystem for designing child safety applications.

This paper proposes a method for path planning of high-degree of freedom (DOF) articulated robots with adaptive dimensionality. For an efficient path planning in high-dimensional C-space (configuration space), first we describe an adaptive body selection that selects the robot bodies and joints depending on the complexity of path planning. It means that the robot may use necessary DOF to achieve a path planning task. The adaptive body selection method builds the C-space with adaptive dimensionality for a sampling-based path planner. Next, by using the adaptive body selection, the adaptive Rapidly-Exploring Random Tree (RRT) algorithm is introduced, which incrementally grows RRTs in the adaptive dimensional C-space. And we show through several simulation results that the proposed method is more efficient than the original RRT-based path planner, which requires full-dimensional planning.

This study developed the Hanyang Exoskeleton Assistive Robot (HEXAR)-CR50 aimed at improving muscle strength of the wearer while transporting a load. The developed exoskeleton robot HEXAR-CR50 has 7 degrees of freedom (DOF) for one foot, 3-DOF for the hip joints, 1-DOF for the knee joints, and 3-DOF for the ankle joints. Through functional analysis of each human joint, two DOFs were composed of active joints using an electric motor developed in an under-actuated form with heightened efficiency. The rest of the DOFs were composed of passive or quasi-passive joints to imitate human joints. The control of the exoskeleton robot was based on the physical human-robot interaction. In order to verify the performance of the developed HEXAR-CR50, muscle activity was measured using electromyography, vGRF was measured using F-Scan sensor. The experimental results showed that %MVIC was reduced against the external load applied, while GRF had a decrement rate, compared with the external load when the exoskeleton was worn, which verified the performance in accordance with the development objective of load carrying. A muscle strength augment effect from the developed wearable robot was verified.

In this paper, we present an asynchronous, accurate, and real-time distance measurement method using chirp signals, which is capable of accurately locating and tracking, thus forming an ad-hoc network among smartphones in an indoor (e.g., smart meeting room) environment. We then implement our propos

The capability to extract human activity context in a room environment can be used as meaningful feedback for various wireless indoor application systems. Being able to do so with easily installable resource-limited sensing components can even further increase the system's applicability for various purposes. This paper introduces our efforts to design a system consisting of heterogeneous low-cost, resource-limited, wireless sensing platforms for accurately extracting the human activity context from an indoor environment. Specifically, we introduce Resource Limited Sensor-based activity Context Extraction (ReLiSCE), a system consisting of microphone array, passive infra-red (PIR), and illumination sensors that effectively detect the activities that occur in an office (meeting room) environment. The signal processing schemes used in ReLiSCE are designed so that their size and complexity is suitable for the resource limitations that many embedded computing platforms introduce. Using empirical evaluations with a prototype system, we show that despite the simplicity of its data processing schemes, ReLiSCE successfully classifies human activity states in various meeting scenarios. Furthermore, we show that high accuracy is achieved by combining results from heterogeneous sensors. We foresee this paper as a sub-system that interconnects with various application systems for autonomously configuring people's everyday living environments in a more comfortable and energy-efficient manner.

The need to support various digital signal processing (DSP) and classification applications on energy-constrained devices has steadily grown. Such applications often extensively perform matrix multiplications using fixed-point arithmetic while exhibiting tolerance for some computational errors. Hence, improving the energy efficiency of multiplications is critical. In this brief, we propose multiplier architectures that can tradeoff computational accuracy with energy consumption at design time. Compared with a precise multiplier, the proposed multiplier can consume 58% less energy/op with average computational error of $\sim 1$ %. Finally, we demonstrate that such a small computational error does not notably impact the quality of DSP and the accuracy of classification applications.

The model-based implementation is to derive an implementation from a model that has been shown to meet requirements. Even though this approach can be used to guarantee that an implementation satisfies functional requirements that are shown to be correct at the model level, it is still challenging to assure timing requirements at the implementation level. We propose a layered approach in testing timing requirements conformance of implemented systems developed by model-based implementation. In our approach, the abstraction boundary of the implemented system is formally defined using Parnas' four-variables model. Then, the proposed approach tests timing aspects of the interaction between the auto-generated code and the target platform-dependent code based on the four-variables. This approach aims at not only detecting the timing requirement violation, but also at measuring delay-segments that contribute to the timing deviation of the implemented system w.r.t. the model. We show the case study of testing timing requirements of an infusion pump system to illustrate the applicability of the proposed framework.

In this work, we present ChildSafe, a classification system which exploits human skeletal features collected using a 3D depth camera to classify visual characteristics between children and adults. ChildSafe analyzes the histograms of training samples and implements a bin-boundary-based classifier. We train and evaluate ChildSafe using a large dataset of visual samples collected from 150 elementary school children and 43 adults, ranging in the ages of 7 and 50. Our results suggest that ChildSafe successfully detects children with a proper classification rate of up to 97%, a false negative rate of as low as 1.82%, and a low false positive rate of 1.46%. We envision this work as an effective sub-system for designing various child protection applications.

In this work, we address a fundamental problem of distinguishing the driver from passengers using a fusion of embedded sensors (accelerometers, gyroscopes, microphones, and magnetic sensors) in a smart phone. Compared with the state-of-the-art solutions, a key property of our solution is non-intrusiveness, i.e., enabling accurate driver phone detection without relying on any particular situations, events, and dedicated hardware devices. Our system only utilizes naturally arising driver motions, i.e., sitting down sideways, closing the vehicle door, and starting the vehicle, to determine whether the user enters the vehicle from left or right and whether the user is seated in the front or rear seats.

Kintense is a robust, accurate, real-time, and evolving system for detecting aggressive actions such as hitting, kicking, pushing, and throwing from streaming 3D skeleton joint coordinates obtained from Kinect sensors. Kintense uses a combination of: (1) an array of supervised learners to recognize a predefined set of aggressive actions, (2) an unsupervised learner to discover new aggressive actions or refine existing actions, and (3) human feedback to reduce false alarms and to label potential aggressive actions. This paper describes the design and implementation of Kintense and provides empirical evidence that the system is 11% - 16% more accurate and 10% - 54% more robust to changes in distance, body orientation, speed, and person when compared to standard techniques such as dynamic time warping (DTW) and posture based gesture recognizers. We deploy Kintense in two multi-person households and demonstrate how it evolves to discover and learn unseen actions, achieves up to 90% accuracy, runs in real-time, and reduces false alarms with up to 13 times fewer user interactions than a typical system.

Smart home environments require tight coupling between different types of sensors, actuators, and computer algorithms. To address the challenges in building a robust and effective smart home environment, it is essential to develop a systematic approach to handling a large number of components and their interactions. We describe a wholistic, multi-layered, and service oriented approach to smart home environment design that will enable abstractions needed for deploying system applications and analyzing individual modules. This paper presents our approach and modules, sensors, and actuators that are required for intelligent services in smart home environments.

Falls are a significant problem for the elderly living independently in the home. Many falls occur due to household objects left in open spaces. We present KinSpace, a system that uses real-time depth data and human-in-the-loop feedback to adjust its understanding of the open space of an environment. We present results showing the effectiveness of our underlying technical solutions in identifying open spaces and obstacles. The results for both lab testing and a small deployment in an actual home show over 80% accuracy for open space detection and 70% accuracy in obstacle detection even in the presence of many real world issues.

Kintense is a robust, accurate, real-time, and evolving system for detecting aggressive actions such as hitting, kicking, pushing, and throwing from streaming 3D skeleton joint coordinates obtained from Kinect sensors. Kintense uses a combination of: (1) an array of supervised learners to recognize a predefined set of aggressive actions, (2) an unsupervised learner to discover new aggressive actions or refine existing actions, and (3) human feedback to reduce false alarms and to label potential aggressive actions. This abstract provides an overview of the design and implementation of Kintense and provides empirical evidence that Kintense is 11% -- 16% more accurate when compared to standard techniques such as dynamic time warping (DTW) and posture based gesture recognizers.

In an integrated circuit (IC) adopting a power-gating (PG) technique, the virtual supply voltage (VVDD) is susceptible to: 1) negative-bias temperature instability (NBTI) degradation that weakens the PG device over time and 2) temporal temperature variation that affects active leakage current (thus total current) of the IC. The PG device is sized to guarantee a minimum VVDD level over the chip lifetime. Thus, the NBTI degradation and the worst-case total current at high-temperature must be considered for sizing the PG device. This leads to higher VVDD (thus active leakage power) than necessary in early chip lifetime and/or at low temperature, negatively impacting the gate-oxide reliability of transistors. To reduce active leakage power increase and improve the gate-oxide reliability due to these effects, we propose two techniques that adjust the strength of a PG device based on its usage and IC's temperature at runtime. We demonstrate the efficacy of these techniques with an experimental setup using a 32-nm technology model in the presence of within-die spatial process and temperature variations. On an average of 100 die samples, they can reduce dynamic and active leakage power by up to 3.7% and 10% in early chip lifetime. Finally, these techniques also reduce the oxide failure rate by up to 5% across process corners over a period of 7 years.

The increasing power consumption of processors has made power reduction a first-order priority in processor design. Voltage scaling is one of the most powerful power-reduction techniques introduced to date, but is limited to some minimum voltageVDDMIN. BelowVDDMINon-chip SRAM cells cannot all operate reliably due to increased process variability with technology scaling. The use of larger SRAM cells, which are less sensitive to process variability, allows a reduction inVDDMIN. However, since the large-scale memory structures such as last-level caches (LLCs) often determine theVDDMINof processors, these structures cannot afford to use large SRAM cells due to the resulting increase in die area. In this paper we first propose a joint optimization of LLC cell size, the number of redundant cells, and the strength of error-correction coding (ECC) to minimize total SRAM area while meeting yield andVDDMINtargets. The joint use of redundant cells and ECC enables the use of smaller cell sizes while maintaining design targets. Smaller cell sizes more than make up for the extra cells required by redundancy and ECC. In 32-nm technology our joint approach yields a 27% reduction in total SRAM area (including the extra cells) when targeting 90% yield and 600 mVVDDMIN. Second, we demonstrate that the ECC used to repair defective cells can be combined with a simple architectural technique, which can also fix particle-induced soft errors, without increasing ECC strength or processor runtime.

Achieving secure time-synchronization in wireless sensor networks (WSNs) is a challenging, but very important problem that has not yet been addressed effectively. This paper proposes an attack-tolerant time-synchronization protocol (ATSP) in which sensor nodes cooperate to safeguard the time- synchronization service against malicious attacks. ATSP exploits the high temporal correlation existing among adjacent nodes in a WSN to achieve (1) adaptive management of the profile of each sensor's normal behavior, (2) distributed, cooperative detection of falsified clock values advertised by attackers or compromised nodes, and (3) significant improvement of synchronization accuracy and stability by effectively compensating the clock drifts with the calibrated clock. To reduce the risk of losing time-synchronization due to attacks on the reference node, ATSP utilizes distributed, mutual synchronization and confines the impact of attacks to a local area (where attacks took place). Furthermore, by maintaining an accurate profile of sensors' normal synchronization behaviors, ATSP detects various critical attacks while incurring only reasonable communication and computation overheads, making ATSP attack-tolerant and ideal for resource-constrained WSNs.

A novel behavioral detection framework is proposed to detect mobile worms, viruses and Trojans, instead of the signature-based solutions currently available for use in mobile devices. First, we propose an efficient representation of malware behaviors based on a key observation that the logical ordering of an application's actions over time often reveals the malicious intent even when each action alone may appear harmless. Then, we generate a database of malicious behavior signatures by studying more than 25 distinct families of mobile viruses and worms targeting the Symbian OS - the most widely-deployed handset OS - and their variants. Next, we propose a two-stage mapping technique that constructs these signatures at run-time from the monitored system events and API calls in Symbian OS. We discriminate the malicious behavior of malware from the normal behavior of applications by training a classifier based on Support Vector Machines (SVMs). Our evaluation on both simulated and real-world malware samples indicates that behavioral detection can identify current mobile viruses and worms with more than 96% accuracy. We also find that the time and resource overheads of constructing the behavior signatures from low-level API calls are acceptably low for their deployment in mobile devices.

Sensor networks, usually built with a large number of small, low-cost sensor nodes, are characterized by their large-scale and unattended deployment, necessitating “secur ” communications between nearby, as well as remote, sensor nodes for their intended applications and services. Key setup/sharing is crucial to the protection of such applications/services from attacks, but existing (public-key, cluster-based, or pairwise) solutions become too expensive (hence, inefficient) when the underlying applications/services require communications between distant sensor nodes. To remedy this inefficiency, we propose a novel distributed key-sharing scheme, in which each participating sensor node shares unique keys with a small number of other sensor nodes—called distributed key servers (DKSs)—chosen according to their geographic distance and communication direction. Using DKSs, we develop two secure routing protocols: (1) secure geographic forwarding that delivers packets by using a chain of DKS lookups, each secured with its own key and forwarded geographically; and (2) key establishment that creates a secure session between two distant sensor nodes based solely on symmetric-ciphers. These protocols enable low-cost, low-power sensors to provide high-level security at a very low cost.

In sensor networks, secure localization—determining sensors' locations in a hostile, untrusted environment—is a challenging, but very important, problem that has not yet been addressed effectively. This paper presents an attack-tolerant localization protocol, called Verification for Iterative Localization (VeIL), under which sensors cooperatively safeguard the localization service. By exploiting the high spatiotemporal correlation existing between adjacent nodes, VeIL realizes (a) adaptive management of a profile for normal localization behavior, and (b) distributed detection of false locations advertised by attackers by comparing them against the profile of normal behavior. Our analysis and simulation results show that VeIL achieves high-level tolerance to many critical attacks, and is computationally feasible on resource-limited sensors.

Small low-cost sensor devices, each equipped with limited resources, are networked and used for various critical applications, especially those related to homeland security. Making such a sensor network secure is challenging mainly because it usually has to operate in a harsh, sometimes hostile, and unattended environment, where it is subject to capture, reverse-engineering, and manipulation. To address this challenge, we present a program-integrity verification (PIV) protocol that verifies the integrity of the program residing in each sensor device whenever the device joins the network or has experienced a long service blockage. The heart of PIV is the novel randomized hash function tailored to low-cost CPUs, by which the algorithm for hash computation on the program can be randomly generated whenever the program needs to be verified. By realizing this randomized hash function, the PlV protocol 1) prevents manipulation/reverse-engineering/reprogramming of sensors unless the attacker modifies the sensor hardware (e.g., attaching more memory), 2) provides purely software-based protection, and 3) triggers the verification infrequently, thus incurring minimal intrusiveness into normal sensor functions. Our performance evaluation shows that the PIV protocol is computationally efficient and incurs only a small communication overhead, hence making it ideal for use in low-cost sensor networks.

Existing location-based routing protocols are not versatile enough for a large-scale ad hoc environment to simultaneously meet all of the requirements of scalability, bandwidth efficiency, energy efficiency, and quality-of-service routing. To remedy this deficiency, we propose an optimal tradeoff approach that: 1) constructs a hybrid routing protocol by combining well-known location-update schemes (i.e., proactive location updates within nodes' local regions and a distributed location service), and 2) derives its optimal configuration, in terms of location-update thresholds (both distance and time-based), to minimize the overall routing overhead. We also build a route-discovery scheme based on an Internet-like architecture, i.e., first querying the location of a destination, then applying a series of local-region routing until finding a complete route by aggregating the thus-found partial routes. To find the optimal thresholds for the hybrid protocol, we derive the costs associated with both location updates and route discovery as a function of location-update thresholds, assuming a random mobility model and a general distribution for route request arrivals. The problem of minimizing the total cost is then cast into a distributed optimization problem. We first prove that the total cost is a convex function of the thresholds, and then derive the optimal thresholds. Finally, we show, via simulation, that our analysis results indeed capture the real behavior.

Small low-cost sensor devices with limited resources are being used widely to build a self-organizing wireless network for various applications, such as situation monitoring and asset surveillance. Making such a sensor network secure is crucial to their intended applications, yet challenging due to the severe resource constraints in each sensor device. We present a lightweight security protocol (LiSP) that makes a tradeoff between security and resource consumption via efficient rekeying. The heart of the protocol is the novel rekeying mechanism that offers (1) efficient key broadcast without requiring retransmission/ACKs, (2) authentication for each key-disclosure without incurring additional overhead, (3) the ability of detecting/recovering lost keys, (4) seamless key refreshment without disrupting ongoing data encryption/decryption, and (5) robustness to inter-node clock skews. Furthermore, these benefits are preserved in conventional contention-based medium access control protocols that do not support reliable broadcast. Our performance evaluation shows that LiSP reduces resource consumption significantly, while requiring only three hash computations, on average, and a storage space for eight keys.