Knowledge transfer using convolutional neural networks (CNNs) can help efficiently train a CNN with fewer parameters or maximize the generalization performance under limited supervision. To enable a more efficient transfer of pretrained knowledge under relaxed conditions, we propose a simple yet powerful knowledge transfer methodology without any restrictions regarding the network structure or dataset used, namely self-supervised knowledge transfer (SSKT), via loosely supervised auxiliary tasks. For this, we devise a training methodology that transfers previously learned knowledge to the current training process as an auxiliary task for the target task through self-supervision using a soft label. The SSKT is independent of the network structure and dataset, and is trained differently from existing knowledge transfer methods; hence, it has an advantage in that the prior knowledge acquired from various tasks can be naturally transferred during the training process to the target task. Furthermore, it can improve the generalization performance on most datasets through the proposed knowledge transfer between different problem domains from multiple source networks. SSKT outperforms the other transfer learning methods (KD, DML, and MAXL) through experiments under various knowledge transfer settings. The source code will be made available to the public1.

Algorithms for video action recognition should consider not only spatial information but also temporal relations, which remains challenging. We propose a 3D-CNN-based action recognition model, called the blockwise temporal-spatial path-way network (BTSNet), which can adjust the temporal and spatial receptive fields by multiple pathways. We designed a novel model inspired by an adaptive kernel selection-based model, which is an architecture for effective feature encoding that adaptively chooses spatial receptive fields for image recognition. Expanding this approach to the temporal domain, our model extracts temporal and channel-wise attention and fuses information on various candidate operations. For evaluation, we tested our proposed model on UCF-101, HMDB-51, SVW, and Epic-Kitchen datasets and showed that it generalized well without pretraining. BTSNet also provides interpretable visualization based on spatiotemporal channel-wise attention. We confirm that the blockwise temporal-spatial pathway supports a better representation for 3D convolutional blocks based on this visualization.

When training an anchor-based object detector with a sparsely annotated dataset, the effort required to locate positive examples can cause performance degradation. Because anchor-based object detection models collect positive examples under IoU between anchors and ground-truth bounding boxes, in a sparsely annotated image, some objects that are not annotated can be assigned as negative examples, such as backgrounds. We attempt to solve this problem with two approaches: 1) using an anchor-less object detector and 2) using a single-object tracker for semi-supervised learning-based object detection. The proposed technique performs bidirectional single-object tracking from sparsely annotated bounding boxes as starting points in videos to obtain dense annotations. On applying our method to the EPIC-KITCHENS-55 dataset, we were able to achieve runner-up performance in the Unseen section, while achieving the first place in the Seen section of the EPIC-KITCHENS 2020 object detection challenge under IoU > 0.5 on the EPIC-KITCHENS 2020 object detection challenge.

One of the most important factors in training object recognition networks using convolutional neural networks (CNN) is the provision of annotated data accompanying human judgment. Particularly, in object detection or semantic segmentation, the annotation process requires considerable human effort. In this paper, we propose a semi-supervised learning (SSL)-based training methodology for object detection, which makes use of automatic labeling of un-annotated data by applying a network previously trained from an annotated dataset. Because an inferred label by the trained network is dependent on the learned parameters, it is often meaningless for re-training the network. To transfer a valuable inferred label to the unlabeled data, we propose a re-alignment method based on co-occurrence matrix analysis that takes into account one-hot-vector encoding of the estimated label and the correlation between the objects in the image. We used an MS-COCO detection dataset to verify the performance of the proposed SSL method and deformable neural networks (D-ConvNets) [1] as an object detector for basic training. The performance of the existing state-of-the-art detectors (D-ConvNets, YOLO v2 [2], and single shot multi-box detector (SSD) [3]) can be improved by the proposed SSL method without using the additional model parameter or modifying the network architecture.

Existing maximum-margin support vector machines (SVMs) generate a hyperplane which produces the clearest separation between positive and negative feature vectors. These SVMs are effective when datasets are large. However, when few training samples are available, the hyperplane is easily influenced by outliers that are geometrically located in the opposite class. We propose a modified SVM which weights feature vectors to reflect the local density of support vectors and quantifies classification uncertainty in terms of the local classification capability of each training sample. We derive a primal formulation of an SVM that incorporates those modifications, and implement an RC-margin SVM of the simplest form. We evaluated our model on the recognition of handwritten numerals, and obtained higher recognition rate than a standard maximum-margin SVM, a weighted SVM, or an SVM which accounts for classification uncertainty.

Blocking artifacts, commonly introduced during video encoding, are one of the major causes of reduced perceptual video quality. The trade-off between these artifacts and bitrate can be improved by adaptively selecting frames from a set of video copies encoded at different bitrates, prior to actual video encoding. We propose a new direction of constructing mixed bitrate video based on content-based image analysis on each video frame, which was posed as a problem of pre-analysis for the final video encoding step. The proposed method consists of the following steps: First, we define a simple and fast impact metric in order to identify the blocking artifacts in each frame of multiple videos, encoded at different bitrates. Based on the impact metric, we generate a blocking artifact density functions for the available bitrates, on the whole video. Finally, we define and optimize our objective function from the blocking artifact density functions in order to select a bitrate with minimum perceptual blockiness and file size for each frame. We validated our method throughout multiple types of videos, showing improved visual quality for the same file size based on commonly used quality assessment measures, such as MSU blocking, MSU blurring, SSIM, 3SSIM, and stSSIM. The reduction rates of average file size and average blocking artifact were about 4.9% and 8.3% over maximum bitrate encoding, respectively.

Quantitative analysis of nanostructures from scanning electron microscope (SEM) images requires a clear segmentation of grains and their boundaries. This is not provided by active contour models, which also require user guidance. Our automatic technique creates a rough representation of grain boundaries by adaptive thresholding. It then performs raycasting from a rectangular grid of seed points to ensure that the grain shapes are convex, and selects the best result for each grain. The whole process can be repeated several times to improve the segmentation. We present results for images of titanium foil, which show that our approach compares favorably in terms of speed and segmentation quality with four competing techniques.