Offline learning usually requires a large dataset for training. In this paper, we focus on vision-based robotic manipulation tasks and utilize certain task properties to achieve offline learning with a small dataset. We propose a two-stage agent consisting of a tentative decision stage and a correction stage, where the tentative decision stage determines a tentative action from the original camera image, and the correction stage determines a correction to the tentative action based on the cropped image according to the tentative action. The correction stage utilizes task properties to obtain the cropped image with task-relevant features, enabling efficient correction. In particular, the training of the two stages can be performed individually, which enables a straightforward application of general offline learning algorithms. We conduct experiments by combining the two-stage agent with conventional offline reinforcement learning and imitation learning algorithms. In both cases, we benchmark the proposed method using RLBench and demonstrate that the task performance is significantly improved by the correction stage.

In this paper, we propose a method for reinforcement learning-based laser welding control. Conventional methods apply standard reinforcement learning formulations to welding tasks, but we show that this formulation can minimize bead width or penetration depth errors only when the welding speed is constant. Therefore, conventional methods are suboptimal for training control parameters including the welding speed. The proposed method discounts future rewards with respect to the welding length instead of time steps to solve this issue. This is easily implemented by (1) modifying the discount factor used for$Q$-function updates in existing reinforcement learning algorithms and (2) using an appropriate reward function. Experimental results using simulators show that the proposed method achieves performance that is superior to conventional methods.

Container loading by a picking robot is an important challenge in the logistics industry. When designing such a robotic system, item picking and placing have been planned individually thus far. However, since the condition of picking an item affects the possible candidates for placing, it is preferable to plan picking and placing simultaneously. In this paper, we propose a deep reinforcement learning (DRL) method for simultaneously planning item picking and placing. A technical challenge in the simultaneous planning is its scalability: even for a practical container size, DRL can be computationally intractable due to large action spaces. To overcome the intractability, we adopt a fully convolutional network for policy approximation and determine the action based only on local information. This enables us to produce a shared policy which can be applied to larger action spaces than the one used for training. We experimentally demonstrate that our method can successfully solve the simultaneous planning problem and achieve a higher occupancy rate than conventional methods.

A novel image registration algorithm for super-resolution is proposed. The quality of reconstructed images becomes poor when the arrangement of the registered positions is irregular. However, there are no known ways to acquire the positions arranged regularly without using active cameras. By using a conventional algorithm, the positions registered with subpixel accuracy are often distributed near integer values in the low-resolution grid when the aperture problem occurs, for example, in the case of processing straight edges. Since this phenomenon causes an irregular arrangement, the high-resolution images are reconstructed inefficiently. The proposed method estimates corresponding positions on the shifted image whose image grid is shifted by a desired subpixel displacement. By using the proposed method, the distribution of the corresponding positions is shifted artificially so that the quality of the reconstructed image is improved. The effectiveness of the proposed method is demonstrated in experiments.

This paper describes an object detection framework that learns the discriminative co-occurrence of multiple features. Feature co-occurrences are automatically found by Sequential Forward Selection at each stage of the boosting process. The selected feature co-occurrences are capable of extracting structural similarities of target objects leading to better performance. The proposed method is a generalization of the framework proposed by Viola and Jones, where each weak classifier depends only on a single feature. Experimental results obtained using four object detectors, for finding faces and three different hand gestures, respectively, show that detectors trained with the proposed algorithm yield consistently higher detection rates than those based on their framework while using the same number of features.

In this paper, we propose a new distinctive feature, called joint Haar-like feature, for detecting faces in images. This is based on co-occurrence of multiple Haar-like features. Feature co-occurrence, which captures the structural similarities within the face class, makes it possible to construct an effective classifier. The joint Haar-like feature can be calculated very fast and has robustness against addition of noise and change in illumination. A face detector is learned by stagewise selection of the joint Haar-like features using AdaBoost. A small number of distinctive features achieve both computational efficiency and accuracy. Experimental results with 5, 676 face images and 30,000 nonface images show that our detector yields higher classification performance than Viola and Jones' detector; which uses a single feature for each weak classifier. Given the same number of features, our method reduces the error by 37%. Our detector is 2.6 times as fast as Viola and Jones' detector to achieve the same performance

A new feature selection method for reliable tracking is presented. In this paper, it is assumed that features are tracked by template matching where small regions around the features are defined as templates. The proposed method selects features based on the upper bound of the average template matching error. This selection criterion is directly related to the reliability of tracking and hence, the performance is better than that of other feature detectors. Experimental results are presented to confirm the efficiency of the proposed method.

A cut detection technique using a likelihood ratio test for MPEG compressed video sequences is proposed. Automatic cut detection techniques are essential techniques to analyze video content. The conventional techniques calculate the similarities of two successive frames and determine cut points by comparing the similarities with a given threshold value; the threshold value is determined heuristically. The new technique estimates likelihood functions from video samples and determines cut points by testing the hypothesis that two frames belong to the same shot. This technique makes it possible to calculate the optimum threshold value analytically. The experimental results show that the new technique gives the smallest probability of false negative for fixed probability of false positive in comparison with the conventional techniques.

A new soft-decision maximum-likelihood decoding algorithm is proposed, which generates a set of candidate codewords using hard-decision bounded-distance decoding. By improving the generating method of input vectors for the bounded-distance decoding due to Kaneko et al. (see ibid., vol.40, no.3, p.320-27, 1994), the decoding time complexity is reduced without degradation of the performance. The space complexity is dependent on the bounded-distance decoding.

A new soft decoding algorithm for linear block codes is proposed. The decoding algorithm works with any algebraic decoder and its performance is strictly the same as that of maximum-likelihood-decoding (MLD). Since our decoding algorithm generates sets of different candidate codewords corresponding to the received sequence, its decoding complexity depends on the received sequence. We compare our decoding algorithm with Chase (1972) algorithm 2 and the Tanaka-Kakigahara (1983) algorithm in which a similar method for generating candidate codewords is used. Computer simulation results indicate, for some signal-to-noise ratios (SNR), that our decoding algorithm requires less average complexity than those of the other two algorithms, but the performance of ours is always superior to those of the other two.>