This paper presents an algorithm to improve state estimation for legged robots. Among existing model-based state estimation methods for legged robots, the contact-aided invariant extended Kalman filter defines the state on a Lie group to preserve invariance, thereby significantly accelerating convergence. It achieves more accurate state estimation by leveraging contact information as measurements for the update step. However, when the model exhibits strong nonlinearity, the estimation accuracy decreases. Such nonlinearities can cause initial errors to accumulate and lead to large drifts over time. To address this issue, we propose compensating for errors by augmenting the Kalman filter with an artificial neural network serving as a nonlinear function approximator. Furthermore, we design this neural network to respect the Lie group structure to ensure invariance, resulting in our proposed Invariant Neural-Augmented Kalman Filter (InNKF). The proposed algorithm offers improved state estimation performance by combining the strengths of model-based and learning-based approaches. Project webpage: https://seokju-lee.github.io/innkf_webpage

This study presents the development of a gait selection strategy for quadrupedal robots capable of responding to external forces. Unlike traditional controllers, our approach directly utilizes information about external forces and introduces a strategy to adapt the robot’s gait patterns accordingly. To simultaneously solve and compare various gait types, we propose a multimodal Model Predictive Control (MPC) framework. This framework dynamically adjusts four distinct gaitstrotting, bounding, pacing, and walking—as well as the contact time with the ground in real-time. The results demonstrate that this approach improves the robot’s ability to withstand disturbances while interacting with external environments, significantly enhancing operational performance and stability.

This paper presents a method for predicting the driving range of an e-mobility on a single charge. In the development of innovative e-mobility solutions, the range attainable on a single charge serves as a critical performance indicator. The driving range is significantly related to the battery system of the e-mobility. Accurate predictions of the driving range can shorten the battery design process and reduce costs in the development of battery systems. A longitudinal dynamics model of the vehicle is introduced to predict the single-charge driving range of the e-mobility. Within the driving cycle, the output exerted by the motor is calculated, as well as the energy consumed by the battery. Prediction of the decrease in the state-of-charge of the battery pack during driving is used for estimating the driving range. The proposed method is validated through MATLAB/Simulink-based simulations and experiments, showing an error rate of around 1% in the driving distance of e-mobility.

In this paper, we propose an adaptive safety distance that is adjusted along the uncertain braking duration of the object vehicle resulting in an improvement of driving comfort. In the real world, the object vehicle decelerates for uncertain duration, while the conventional strategies assume that the object vehicle decelerates until full stop. By imposing a constraint on the ego vehicle to stop simultaneously with the object vehicle, the driving comfort is enhanced by activating full braking (FB) only when it is necessary. To accomplish these, two concepts are proposed; effective collision avoidance (ECA) condition and full braking activation parameter (FBAP). ECA refers to a strategy in which the ego vehicle stops simultaneously with the object vehicle to account for its uncertain deceleration duration. FBAP is suggested to determine when to activate FB by adjusting the duration of partial braking (PB). Virtual experiments are conducted strictly with various scenarios to validate the performance of the proposed FBAP-based safety distance. The test results demonstrate that the suggested FBAP-based safety distance is applicable to a wide range of driving speeds, improving driving comfort and performing collision avoidance compared to conventional methods.

Recently, legged robots have been used for various purposes, such as exploring unknown terrain or interacting with the world. For control and planning legged systems during interactive operations, it is essential to estimate and respond to external forces. However, in legged system, it becomes difficult to estimate forces due to highly dynamic situations. There are several studies that use a force sensor on the foot and end effector, but these approaches have disadvantages in terms of cost and sustainability. Therefore, in this paper, we propose an improved method for estimating external forces without a force sensor. First, each leg force was obtained using the system dynamics of the robot with a disturbance observer. Then, by preintegration, it was tightly coupled with other sensors to estimate the pose and external force simultaneously. Despite the impact and slip, we estimate external forces accurately in standing and walking motions. Moreover, we compared pose estimation performance with VINS-Mono [1], and there is no significant accuracy degradation in spite of highly dynamic force residual.

As autonomous driving research has actively pro-gressed, software for autonomous vehicles and embedded systems such as Apollo and AutoWare have also been developed, providing a complete set of self-driving modules, including perception, localization and mapping, path planning, prediction, decision making, and control. Most of the researchers currently use these software programs, but many researchers have also studied autonomous driving based on the middleware software termed robot operating system (ROS) before such software was released, especially in academia. Accordingly, we intend to develop ROS-based unmanned RC car equipped with autonomous driving sensors such as LiDAR, radar, VIS/IR cameras, GPS, and IMUs that can provide ROS-based datasets to researchers studying self-driving cars and robots using ROS. In addition, unlike conventional datasets, we acquire dataset not only on road but also pedestrian paths that can be used in both vehicles and robots and provides extreme environmental datasets such as snowfall environments. In this sense, the ROS dataset we created will be helpful to researchers studying autonomous vehicles and robots by using ROS.

Visual degradations caused by motion blur, raindrop, rain, snow, illumination, and fog deteriorate image quality and, subsequently, the performance of perception algorithms deployed in outdoor conditions. While degradation-specific image restoration techniques have been extensively studied, such algorithms are domain sensitive and fail in real scenarios where multiple degradations exist simultaneously. This makes a case for blind image restoration and reconstruction algorithms as practically relevant. However, the absence of a dataset diverse enough to encapsulate all variations hinders development for such an algorithm. In this paper, we utilize a synthetic degradation model that recursively applies sets of random degradations to generate naturalistic degradation images of varying complexity, which are used as input. Furthermore, as the degradation intensity can vary across an image, the spatially invariant convolutional filter cannot be applied for all degradations. Hence to enable spatial variance during image restoration and reconstruction, we design a transformer-based architecture to benefit from the long-range dependencies. In addition, to reduce the computational cost of transformers, we propose a multi-branch structure coupled with modifications such as a complimentary feature selection mechanism and the replacement of a feed-forward network with lightweight multiscale convolutions. Finally, to improve restoration and reconstruction, we integrate an auxiliary decoder branch to predict the degradation mask to ensure the underlying network can localize the degradation information. From empirical analysis on 10 datasets covering rain drop removal, deraining, dehazing, image enhancement, and deblurring, we demonstrate the efficacy of the proposed approach while obtaining SoTA performance.

Collision avoidance in emergency situations is a crucial and challenging task in motion planning for autonomous vehicles. Especially in the field of optimization-based planning using nonlinear model predictive control, many efforts to achieve real-time performance are still ongoing. Among various approaches, the iterative linear quadratic regulator (iLQR) is known as an efficient means of nonlinear optimization. Additionally, parallel computing architectures, such as GPUs, are more widely applied in autonomous vehicles. In this paper, we propose 1) a parallel computing framework for iLQR with input constraints considering the characteristics of the problem and 2) a proper environmental formulation that can be covered with single-precision floating-point computation of the GPU. The GPU-accelerated framework was tested on a real-time simulation-in-the-loop system using CarMaker and ROS at a 20 Hz sampling rate on a low-performance mobile computer and was compared against the same framework realized with a CPU.

Semantic segmentation provides scene understanding capability by performing pixel-wise classification of objects within an image. However, the sensitivity of such algorithms towards domain changes requires fine-tuning using an annotated dataset for each novel domain, which is expensive to construct and inefficient. We highlight that irrespective of the training dataset, structural properties of scenes remain the same hence domain sensitivity arises from training methodology. Thus, in this paper, we propose a domain alignment approach wherein multiple synthetic source domains are used to train an underlying segmentation network such that it performs consistently in unknown real target domains. Towards this end, we propose a pixel-wise supervised contrastive learning framework that enforces constraints in latent space resulting in features belonging to the same class being clustered closely and away from different classes. This approach allows for better capturing of global and local semantics while providing domain invariant properties. Our approach can be easily incorporated into prior semantic segmentation approaches without the significant computational overhead. We empirically demonstrate the efficacy of the proposed approach on GTAV → Cityscapes, GTAV+Synthia → Cityscapes, and GTAV+Synthia+Synscapes → Cityscapes scenarios and report state-of-the-art (SoTA) performance without requiring access to images from the target domain.

Increasing usage of traffic cameras provides an opportunity to utilize them for smart city applications. However, the efficacy of such systems is determined by their ability to detect and track objects of interest from diverse viewpoints accurately. This is challenging due to the diverse viewpoints, elevations, and distinct properties of camera sensors. Thus, to ensure robust performance, the training dataset should cover many variations, including viewpoints, illumination changes, and diverse weather conditions. However, constructing such a dataset is expensive in terms of data collection and annotation. This paper proposes an unsupervised domain adaptation approach wherein a synthetic dataset is generated using a simulator and subsequently used to ensure performance consistency of multi-object-tracking (MOT) algorithms across a diverse range of manually annotated natural scenes. Towards this end, we emphasize achieving domain invariant object detection by combining image stylization and class-balancing augmentation. Furthermore, we extend the robust detection algorithm to track detected objects across a large time scale using feature embeddings generated by the detector. Based on qualitative and quantitative results, we demonstrate the viability of such a system that is invariant to illumination, weather, viewpoint, and scene changes while providing a baseline for future research. Codebase and datasets would be made available at https://github.com/pranjay-dev/IS2R.

Presence of haze in images obscures underlying information, which is undesirable in applications requiring accurate environment information. To recover such an image, a dehazing algorithm should localize and recover affected regions while ensuring consistency between recovered and its neighboring regions. However owing to fixed receptive field of convolutional kernels and non uniform haze distribution, assuring consistency between regions is difficult. In this paper, we utilize an encoder-decoder based network architecture to perform the task of dehazing and integrate an spatially aware channel attention mechanism to enhance features of interest beyond the receptive field of traditional conventional kernels. To ensure performance consistency across diverse range of haze densities, we utilize greedy localized data augmentation mechanism. Synthetic datasets are typically used to ensure a large amount of paired training samples, however the methodology to generate such samples introduces a gap between them and real images while accounting for only uniform haze distribution and overlooking more realistic scenario of non-uniform haze distribution resulting in inferior dehazing performance when evaluated on real datasets. Despite this, the abundance of paired samples within synthetic datasets cannot be ignored. Thus to ensure performance consistency across diverse datasets, we train the proposed network within an adversarial prior-guided framework that relies on a generated image along with its low and high frequency components to determine if properties of dehazed images matches those of ground truth. We preform extensive experiments to validate the dehazing and domain invariance performance of proposed framework across diverse domains and report state-of-the-art (SoTA) results. The source code with pretrained models will be available at https://github.com/PS06/DIDH.

Input-output transmission ratio shifting mechanisms provide a variable transmission ratio, which effectively expands a speed-force operating range of actuators. Although it is the most effective solution to increase the performance of robotic systems, its application to compact robotic systems still remains a challenging issue due to its complexity and massive structure. In this paper, we introduce a twisted string actuation-based transmission module for compact robotic systems. The twisted string mechanism provides a simplified transmission design and a compact form factor of the transmission module. An automatic transmission shifting algorithm is proposed for effective and autonomous control strategies. The developed prototype is integrated into a robotic gripper/hand, and its performance is verified with grasping demonstrations.

Vehicle Re-identification (Re-ID) aims to retrieve all instances of query vehicle images present in an image pool. However viewpoint, illumination, and occlusion variations along with subtle differences between two unique images pose a significant challenge towards achieving an effective system. In this paper, we emphasize upon enhancing the performance of visual feature based ReID system by improving feature embedding quality and propose (1) an attention-guided hierarchical feature extractor (HFE) that leverages the structure of a backbone CNN to extract coarse and fine-grained features and (2) to train the proposed network within a hard negative adversarial framework that generates samples exhibiting extreme variations, encouraging the network to extract important distinguishing features across varying scales. To demonstrate the effectiveness of the proposed framework we use VERI-Wild, VRIC and Veri-776 datasets that exhibit extreme intra-class and minute inter-class differences and achieve state-of-the-art (SoTA) performance. Codes related to this paper are publicly available at https://github.com/PS06/VReID.

Volumetric deep learning approach towards stereo matching aggregates a cost volume computed from input left and right images using 3D convolutions. Recent works showed that utilization of extracted image features and a spatially varying cost volume aggregation complements 3D convolutions. However, existing methods with spatially varying operations are complex, cost considerable computation time, and cause memory consumption to increase. In this work, we construct Guided Cost volume Excitation (GCE) and show that simple channel excitation of cost volume guided by image can improve performance considerably. Moreover, we propose a novel method of using top-k selection prior to soft-argmin disparity regression for computing the final disparity estimate. Combining our novel contributions, we present an end-to-end network that we call Correlate-and-Excite (CoEx). Extensive experiments of our model on the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate the effectiveness and efficiency of our model and show that our model outperforms other speed-based algorithms while also being competitive to other state-of-the-art algorithms. Codes will be made available at https://github.com/antabangun/coex.

Anchor boxes act as potential object localization candidates allow single-stage detectors to achieve real-time performance, at the cost of localization accuracy when compared to state-of-the-art two-stage detectors. Therefore, correct selection of the scale and aspect ratio associated with an anchor box is crucial for detector performance. In this work, we propose a novel architecture called DANet for improving the localization performance of single-stage object detectors, while maintaining real-time inference. The proposed network achieves this by predicting (1) the combination of aspect ratio and scale per feature map based on object density and (2) localization confidence per anchor box. We evaluate the proposed network using the benchmark dataset. On the MS COCO dataset, DANet achieves 30.9% AP at 51.8 fps using ResNet-18 and 45.3% AP at 7.4 fps using ResNeXt-101. The code and models will be available at https://github.com/PS06/AnchorNet.

Over the past decade, much research has been done to improve single‐fog images. However, most of these have concentrated on outdoor environments and little has been done for indoor environments. In this study, an effective method of removing fog from images both indoors and outdoors is presented. A new single image enhancement approach is based on mixture of dark channel prior (DCP) and contrast limited adaptive histogram equalisation with discrete wavelet transform (CLAHE‐DWT) algorithms. With the DCP algorithm using modified transmission map, the authors obtained fast processing speed and clean dehazed image without refining process. The CLAHE and DWT methods improved the contrast and sharpness of images. Finally, an enhanced image was produced by fusing the CLAHE and DWT images. To demonstrate the effectiveness of the proposed method, the authors performed objective image quality assessments, and so on. Through a variety of experiments for various indoor and outdoor images with fog, the proposed method was proven to be highly effective.

This paper presents a modified nonlinear pneumatic model for pneumatic force servo systems. The modified model is proposed in order to estimate pressures accurately for both chambers of a pneumatic cylinder by adopting flow coefficient maps, which is different from a conventional model whose flow coefficient is constant. The simulated data from the model is also compared to the experiment result and shows its accuracy compared to a widely used conventional model. The modified model and the experimental procedure are described and verified in this paper as well. Finally, the applicability of the proposed model to model-based controller design is also shown through the experiment of force servo controls based on a force sensor.

In daily life, most hand movements involve the simultaneous activation of multiple fingers. Models generated by semiunsupervised learning in which only individual finger activation data are used in training have recently been suggested for simultaneous and proportional control of prosthetic robot hands. Although training with many datasets should be avoided, simultaneous activation data need to be used, for example, when the model estimation of the simultaneous activation is very poor or highly coupled among the degrees-of-freedoms (DOFs). In this paper, we propose a method for generating a model using any type of activation data (individual, simultaneous, or both) by modifying the nonnegative matrix factorization (NMF) with the Hadamard product (HP). The model provided by this method is called NMF-HP. NMF-HP has two advantages: First, it can use simultaneous activation data for training. Second, NMF-HP decouples coupled DOFs by forcing nonactive DOFs to be zero during the training phase. NMF-HP was tested in two cases (trained with only individual activation data and trained with both individual and simultaneous activation data) in offline and online experiments. In the offline test, NMF-HP outperformed the conventional semiunsupervised models for the simultaneous activation of the fingers. In the online test, NMF-HP was significantly better than NMF in the estimation of finger-motion intentions. This result contrasts with that of a previous study in which performance in the online test revealed a little difference between the models, possibly due to the human-embedded control. Thus, the result of this paper indicates that using simultaneous activation data and reducing the coupling among DOFs may be effective in enhancing the performance of the real-time control of a prosthetic robot hand.

Electrotactile stimulation has been used to provide sensory information of forearm prosthesis to users. Although conventional sensory feedback method, where one electrode expresses sensory information of only one finger, could provide force information of three fingers by using three electrodes, it showed less cognitive accuracy when more than two electrodes were stimulated simultaneously compared to individual stimulation. To improve the cognitive accuracy, we presented a sensory feedback method called two-channel electrotactile stimulation for the thumb, index and middle fingers using only two electrodes. The force information of the index and middle fingers was delivered into two electrodes, respectively, by intermittent stimulation. The information of the thumb was delivered to the user by inserting additional offset pulses onto the channel of the index finger based on an assumption that the force of thumb is proportional to the sum of the forces of the index and middle fingers. The presence of offset pulses in the channel indicates the binary state of thumb if it contacts with an object or not. We conducted two psychophysical experiments where healthy subjects classified the binary states of each finger and identified intensity from two-channel stimuli. The cognitive accuracies for intensity identification were 78.8% and 62.2% for the intermittent stimulation and conventional method, respectively, and accuracy for classifying the fingers was 93.1% for every combination of the three fingers. The results demonstrated that the proposed two-channel electrotactile stimulation could be an attractive method to express the information of fingers of prosthesis with high accuracy.

When designing a lower-limb assisting robot with body-weight support (BWS), it is important to achieve high force fidelity to support body weight at standing phase. Low impedance operation at swing phase is also required not to disturb leg motions for users. The SoftGait is designed to achieve both performances with a simple mechanical structure based on a pneumatic actuator and low cost sensor development. The SoftGait generates large force for body-weight-supporting in squatting mode corresponding to user's body weight. In fact, in walking mode, it enables a user to experience more comfortable walking with support from intrinsic compliance.

In this paper, a dual-mode twisting actuation mechanism with an active clutch is newly presented for a high performance tendon-driven robot (e.g., robot hand). This mechanism is a kind of mechanical automatic power transmission mechanism which provides fast motion and large contraction force by two geared motors. One of the motors is adopted for main actuation and the other is utilized for active clutch. The active clutch consisting of low-power DC motor and gear set allows easy control of the dual-mode twisting actuation and simplifies relaxation process of twisted strings which was a critical problem by a passive brake in previous research [17]. Kinematics of the proposed mechanism is represented and its simulation is performed to verify the performance numerically. By using BLDC & DC motor with 8 W & 0.3 W power, we developed a prototype of the dual-mode twisting actuation with the weight of 45.7 g and the size of 71 mm × 21.5 × 15 mm. Despite of simple structure, the proposed mechanism shows that operation mode-change can be easily managed and the relaxation time was much more reduced than that of the passive brake version.

In this paper, a method to extract the fingertip forces of the index and middle fingers from surface electromyography (sEMG) signals is studied by adopting the known common spatial pattern (CSP) approach. For unsupervised estimation of fingertip forces in real-time, CSP filtering is shown to be a notably effective method compared with known approaches for handling sEMG signals. The results of the proposed method are comparable to those of supervised estimations, such as linear regression and artificial neural network. The efficacy of the proposed method is validated by experiments.

Legged systems are potentially expected to have outstanding mobility. However, they have shown only modest levels of capability to traverse on flat ground. This paper introduces a biped robot inspired by the hind limbs of cats. We present the characteristics of the system and conduct a position analysis based on vector loop equations. Afterwards, the ground clearance and parallel movement of the robotic leg are shown. We present a speed equation in an effort to verify how the major parameters affect the speed. In addition, we explore control strategies for ground speed matching, acceleration, slip, and speed control. A dynamic simulation shows that the biped robot reached 13.31 leg lengths per second (9.3km/h). This biped robot with the speed equation and its control strategies allow us to understand legged locomotion and can show us how to improve the speed.

In this paper, the anthropomorphic robot hand is newly proposed by adopting dual-mode twisting actuation and EM joint locking mechanism. The proposed robot hand consists of five finger modules. Each finger has four links and three joints, and Joint 2 and 3 are coupled by the four-bar linkage mechanism. The dual-mode twisting actuation allows that the robot finger can move fast (up to 356.7 deg/sec) in Mode I and generate a large grasping force(up to 36.5 N) in Mode II. In addition, the workspace of the robot finger module is enlarged by EM joint locking mechanism depending on the locking states. In order to verify the effectiveness of the mechanisms adopted in the robot hand, we theoretically and numerically analyze the performances of the robot finger module such as bending speed, fingertip force, and workspace. Finally, through the developed robot hand, we perform the grasping test for various objects and the grasping performance is experimentally demonstrated.

In this paper, a pneumatic power generator based on the chemical reaction is newly proposed by using a small piston pump for the injection of the fuel. Based on the understanding of chemical reaction property, the piston pump is designed by crank-slider mechanism. The piston pump allows compact size and light weight of the entire power generation system compared to the conventional approaches using blow-down tank. In order to verify the effectiveness of the proposed power generation system, we theoretically and experimentally analyze the performance of the system. In addition, we realize the power generator and applies it to an underactuated robot finger for the feasibility test.

A dual-mode robot finger is proposed to achieve a high-speed motion and large grasping force with a single motor. The robot finger has two actuator modes, which consist of the speed mode and the force mode. Based on the geometric analysis of each mode, the main design parameters of the proposed robot finger are derived, and their effectiveness is verified by simulations. In addition, using experiments with a prototype of a robot finger, the validity of the proposed approach is demonstrated.

In this paper, a distributed-actuation method has been newly proposed based on a simple sliding-actuation mechanism for a finger-type manipulator design. Based on the spatially distributed force on the proposed sliding-actuation mechanism, it has been shown that the distributed actuation provides additional design freedom to optimize the manipulator performance. To verify the effectiveness of the proposed method, we developed a finger-type manipulator, which consists of four links with three joints, and performed experiments. The experimental results show that the fingertip force of the developed manipulator can be effectively increased and easily managed by the distributed-actuation method.