Generalization of synthesizer sound matching to external instrument sounds is highly challenging due to the non-differentiability of sound synthesis process which prohibits the use of out-of-domain sounds for training with synthesis parameter loss. We propose SynthRL, a novel reinforcement learning (RL)-based approach for cross-domain synthesizer sound matching. By incorporating sound similarity into the reward function, SynthRL effectively optimizes synthesis parameters without ground-truth labels, allowing fine-tuning on out-of-domain sounds. Furthermore, we introduce a transformer-based model architecture and reward-based prioritized experience replay to enhance RL training efficiency, considering the unique characteristics of the task. Experimental results demonstrate that SynthRL outperforms state-of-the-art methods on both in-domain and out-of-domain tasks. Further experimental analysis validates the effectiveness of our reward design, showing a strong correlation with human perception of sound similarity.

Reinforcement learning (RL) has achieved considerable success in many fields, but applying it to real-world problems can be costly and risky because it requires a lot of online interaction. Recently, offline RL has shown the possibility of extracting a solution through existing logged data without online interaction. In this work, we propose an offline hierarchical RL method, Guider (Guide to Control), that can efficiently solve long-horizon and sparse-reward tasks from offline data. The high-level policy sequentially generates a subgoal that can guide the agent to arrive at the final goal, and the lower-level policy learns how to reach each given guided subgoal. In the process of learning from offline data, the key is to make the low-level policy reachable to the generated subgoals. We show that high-quality subgoal generation is possible through pre-training a latent subgoal prior model. The well-regulated subgoal generation improves performance while avoiding distributional shifts in offline RL by breaking down long, complex tasks into shorter, easier ones. For evaluations, Guider outperforms prior offline RL methods in long-horizon robot navigation and complex manipulation benchmarks. Our code is available at https://github.com/gckor/Guider.

This paper presents an automatic algorithm for the detection of respiratory events in patients using electrocardiogram (ECG) and respiratory signals. The proposed method was developed using data of polysomnogram (PSG) and those recorded from a patch-type device. In total, data of 1,285 subjects were used for algorithm development and evaluation. The proposed method involved respiratory event detection and apnea-hypopnea index (AHI) estimation. Handcrafted features from the ECG and respiratory signals were applied to machine learning algorithms including linear discriminant analysis, quadratic discriminant analysis, random forest, multi-layer perceptron, and the support vector machine (SVM). High performance was demonstrated when using SVM, where the overall accuracy achieved was 83% and the Cohen's kappa was 0.53 for the minute-by-minute respiratory event detection. The correlation coefficient between the reference AHI obtained using the PSG and estimated AHI as per the proposed method was 0.87. Furthermore, patient classification based on an AHI cutoff of 15 showed an accuracy of 87% and a Cohen's kappa of 0.72. The proposed method increases performance result, as it records the ECG and respiratory signals simultaneously. Overall, it can be used to lower the development cost of commercial software owing to the use of open datasets.

This paper proposes a robust method to screen patients with sleep apnea syndrome (SAS) using a single-lead electrocardiogram (ECG). This method consists of minute-by-minute abnormal breathing detection and apnea-hypopnea index (AHI) estimation. Heartbeat interval and ECG-derived respiration (EDR) are calculated using the single-lead ECG and used to train the models, including ResNet18, ResNet34, and ResNet50. The proposed method, using data from 1232 subjects, was developed with two open datasets and experimental data and evaluated using two additional open datasets and data acquired from an abdomen-attached wearable device (in total, data from 189 subjects). ResNet18 showed the best results, having an average Cohen's kappa coefficient of 0.57, in the abnormal breathing detection. Moreover, SAS patient classification, with 15 as the AHI threshold, yielded an average Cohen's kappa coefficient of 0.71. The results of patient classification were biased toward data from the wearable patch-type device, which may be influenced by different ECG waveforms. The proposed method is tuned with a sample of the data from the device, and the performance result of Cohen's kappa increased from 0.54 to 0.91 for SAS patient classification. Our method, proposed in this paper, achieved equivalent performance results with data recorded using an abdomen-attached wearable device and two open datasets used in previous studies, although the method had not used those data during model training. The proposed method could reduce the development costs of commercial software, as it was developed using open datasets, has robust performance throughout all datasets.