Structure pruning is an effective method to compress and accelerate neural networks. While filter and channel pruning are preferable to other structure pruning methods in terms of realistic acceleration and hardware compatibility, pruning methods with a finer granularity, such as intra-channel pruning, are expected to be capable of yielding more compact and computationally efficient networks. Typical intra-channel pruning methods utilize a static and hand-crafted pruning granularity due to a large search space, which leaves room for improvement in their pruning performance. In this work, we introduce a novel structure pruning method, termed as dynamic structure pruning, to identify optimal pruning granularities for intra-channel pruning. In contrast to existing intra-channel pruning methods, the proposed method automatically optimizes dynamic pruning granularities in each layer while training deep neural networks. To achieve this, we propose a differentiable group learning method designed to efficiently learn a pruning granularity based on gradient-based learning of filter groups. The experimental results show that dynamic structure pruning achieves state-of-the-art pruning performance and better realistic acceleration on a GPU compared with channel pruning. In particular, it reduces the FLOPs of ResNet50 by 71.85% without accuracy degradation on the ImageNet dataset. Our code is available at https://github.com/irishev/DSP.

Prompt tuning has emerged as a successful parameter-efficient alternative to the full finetuning of language models.However, prior works on prompt tuning often utilize long soft prompts of up to 100 tokens to improve performance, overlooking the inefficiency associated with extended inputs.In this paper, we propose a novel prompt tuning method SMoP (Sparse Mixture-of-Prompts) that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced from longer soft prompts.To achieve this, SMoP employs a gating mechanism to train multiple short soft prompts specialized in handling different subsets of the data, providing an alternative to relying on a single long soft prompt to cover the entire data.Experimental results demonstrate that SMoP outperforms baseline methods while reducing training and inference costs.We release our code at https://github.com/jyjohnchoi/SMoP.

We introduce Basic, Tiniest Subword (BTS) units for the Korean language, which are inspired by the invention principle of Hangeul, the Korean writing system.Instead of relying on 51 Korean consonant and vowel letters, we form the letters from BTS units by adding strokes or combining them.To examine the impact of BTS units on Korean language processing, we develop a novel BTSbased word embedding framework that is readily applicable to various models.Our experiments reveal that BTS units significantly improve the performance of Korean word embedding on all intrinsic and extrinsic tasks in our evaluation.In particular, BTS-based word embedding outperforms the state-of-theart Korean word embedding by 11.8% in word analogy.We further investigate the unique advantages provided by BTS units through indepth analysis.Our code is available at https: //github.com/irishev/BTS.

Pre-trained language models have achieved remarkable successes in natural language processing tasks, coming at the cost of increasing model size.To address this issue, knowledge distillation (KD) has been widely applied to compress language models.However, typical KD approaches for language models have overlooked the difficulty of training examples, suffering from incorrect teacher prediction transfer and sub-efficient training.In this paper, we propose a novel KD framework, Tutor-KD, which improves the distillation effectiveness by controlling the difficulty of training examples during pre-training.We introduce a tutor network that generates samples that are easy for the teacher but difficult for the student, with training on a carefully designed policy gradient method.Experimental results show that Tutor-KD significantly and consistently outperforms the state-of-the-art KD methods with variously sized student models on the GLUE benchmark, demonstrating that the tutor can effectively generate training examples for the student 1 .

We develop a hardware and software for EtherCAT slave based on multi-core DSP in order to apply to control applications. Beckhoff's ET1100 ASIC is employed as EtherCAT slave controller; EtherCAT slave stack and control application algorithm are processed by individually assigned DSP core; and data interexchange between the two DSP cores is carried out by using the Inter-Process Communication mechanism. Namely, we intentionally separate the EtherCAT communication stack process from control application process using multi-core DSP, which enables to achieve conducting complicated control application as well as high-speed communication cycle. The high performance of the developed EtherCAT slave are demonstrated by real experiments using EtherCAT network composed of the developed slave and IgH EtherCAT master for Linux.

An electric vehicles which is equipped with the in-wheel motors have an advantages in terms of control. For example, by generating different torques at the left and right wheels, the vehicle stability in cornering can be improved. This is called the direct yaw moment control method. In this paper, adaptive direct yaw moment control based on identification of yaw rate model is proposed for electric vehicles. In practical point of view, the incorrect yaw rate model causes the uncertainty to the reference yaw rate. Because the direct yaw moment is determined for tracking that uncertain reference yaw rate, the lateral motion of the vehicle cannot be desirable one. To solve this problem, the identification of the yaw rate model is considered. Therefore, the identified yaw rate model provides the yaw rate for the direct yaw moment method so that the vehicle follows the desired dynamic model. The simulation results show the possibility of the proposed methods.