The ability to apply a previously-learned skill (e.g., pushing) to a new task (context or object) is an important requirement for new-age robots. An attempt is made to solve this problem in this paper by proposing a deep meta-imitation learning framework comprising of an attentive-embedding net-work and a control network, capable of learning a new task in an end-to-end manner while requiring only one or a few visual demonstrations. The feature embeddings learnt by incorporating spatial attention is shown to provide higher embedding and control accuracy compared to other state-of-the-art methods such as TecNet [7] and MIL [4]. The interaction between the embedding and the control networks is improved by using multiplicative skip-connections and is shown to overcome the overfitting of the trained model. The superiority of the proposed model is established through rigorous experimentation using a publicly available dataset and a new dataset created using PyBullet [36]. Several ablation studies have been carried out to justify the design choices.

This paper proposes an end-to-end self-supervised feature representation network named Attentive Task-Net or AT-Net for video-based task imitation. The proposed AT-Net incorporates a novel multi-level spatial attention module to highlight spatial features corresponding to the intended task demonstrated by the expert. The neural connections in AT-Net ensure the relevant information in the demonstration is amplified and the irrelevant information is suppressed while learning task-specific feature embeddings. This is achieved by a weighted combination of multiple intermediate feature maps of the input image at different stages of the CNN pipeline. The weights of the combination are given by the compatibility scores, predicted by the attention module for respective feature maps. The AT-Net is trained using a metric learning loss which aims to decrease the distance between the feature representations of concurrent frames from multiple view points and increase the distance between temporally consecutive frames. The AT-Net features are then used to formulate a reinforcement learning problem for task imitation. Through experiments on the publicly available Multi-view pouring dataset, it is demonstrated that the output of the attention module highlights the task-specific objects while suppressing the rest of the background. The efficacy of the proposed method is further validated by qualitative and quantitative comparison with a state-of-the-art technique along with intensive ablation studies. The proposed method is implemented to imitate a pouring task where an RL agent is learned with the AT-Net in Gazebo simulator. Our findings show that the AT-Net achieves 6.5% decrease in alignment error along with a reduction in the number of training iterations by almost 155k over the state-of-the-art while satisfactorily imitating the intended task.

Capsule Network (CapsNet) has motivated researchers to work on it due to its distinct capability of retaining spatial correlations between image features. However, its applicability is still limited because of its intensive computational cost, memory usage and bandwidth requirement. This paper proposes a computationally efficient, lightweight CapsNet which paves its way forward for deployment in constrained edge devices as well as in web based applications. The proposed framework consists of Capsule layers and a deep feature representation layer as an input for capsules. The deep feature representation layer comprises of a series of feature blocks, containing convolution with a 3 × 3 kernel followed by batch normalization and convolution with a 1 × 1 kernel. The deeper or better represented input features help to improve recognition performance even with lesser number of capsules, making the network computationally more efficient. The efficacy of the proposed framework is validated by performing rigorous experimental studies on different datasets, such as CIFAR-10, FMNIST, MNIST and SVHN which include images of object classes as well as text characters. A comparative analysis has also been done with the state-of-the-art technique CapsNet. The comparison with recognition accuracy ensures that, the proposed architecture with deep input features provides more efficient routing between the capsules as compared to CapsNet. The proposed lightweight network has scaled down the number of parameters up to 60% of CapsNet, which is another significant contribution. This is achieved by collaborative effect of deep feature generation module and parametric changes performed in the primary capsule layer.

In this paper, we propose an unsupervised deep learning framework with Bayesian inference for improving the accuracy of per-pixel depth prediction from monocular RGB images. The proposed framework predicts confidence map along with depth and pose information for a given input image. The depth hypotheses from previous frames are propagated forward and fused with the depth hypothesis of the current frame by using Bayesian inference mechanism. The ground truth information required for training the confidence map prediction is constructed using image reconstruction loss thereby obviating the need for explicit ground truth depth information used in supervised methods. The resulting unsupervised framework is shown to outperform the existing state-of-the-art methods for depth prediction on the publicly available KITTI outdoor dataset. The usefulness of the proposed framework is further established by demonstrating a real-world robotic pick-and-place application where the pose of the robot end-effector is computed using the depth predicted from an eye-in-hand monocular camera. The design choices made for the proposed framework is justified through extensive ablation studies.

This paper presents a new GAN-based deep learning framework for estimating absolute scale awaredepth and ego motion from monocular images using a completely unsupervised mode of learning.The proposed architecture uses two separate generators to learn the distribution of depth and posedata for a given input image sequence. The depth and pose data, thus generated, are then evaluated bya patch-based discriminator using the reconstructed image and its corresponding actual image. Thepatch-based GAN (or PatchGAN) is shown to detect high frequency local structural defects in thereconstructed image, thereby improving the accuracy of overall depth and pose estimation. Unlikeconventional GANs, the proposed architecture uses a conditioned version of input and output of thegenerator for training the whole network. The resulting framework is shown to outperform all existing deep networks in this field and beating the current state-of-the-art method by 8.7% in absoluteerror and 5.2% in RMSE metric. To the best of our knowledge, this is first deep network based modelto estimate both depth and pose simultaneously using a conditional patch-based GAN paradigm.The efficacy of the proposed approach is demonstrated through rigorous ablation studies and exhaustive performance comparison on the popular KITTI outdoor driving dataset.

Automatic recognition of text, such as a batch code printed on a box placed on a moving conveyor belt, is still a challenging problem. This paper proposes an end-to-end character recognition technique while addressing the major challenges encountered in a real environment, such as motion blur in the acquired images, slanted or oriented characters, creased batch codes due to wear and tear of boxes, variations in label formats, and variations in printing styles. The major contribution of this work lies in development of three sequential modules: text localization using Connectionist Text Proposal Network(CTPN), character detection and character recognition using a modified version of the capsule network (CapsNet). In contrast to CapsNet, where only a standard single convolution is used, the proposed method uses a series of feature blocks, making it a deep CapsNet which is later proven to generate more comprehensive and better separable feature vectors over its counterpart. The feature generation module is further enhanced by setting a smaller kernel size than CapsNet. The proposed system is validated on a real-world box / packet dataset generated in a retail manufacturing industry. The proposed recognition network architecture is also validated on a standard public dataset (ICDAR 2013). The comparative results are presented with statistical analysis in the experimental results section.

The presented work focuses on automatic recognition of object classes while ensuring near real-time training required for recognizing a new object not seen previously. This is achieved by proposing a two-stage hierarchical deep learning framework which first learns object categories using a Nearest Class Mean (NCM) classifier applied directly to CNN features and then, uses a two-layer artificial neural network to learn the object labels within each category. In order to recognize a new object not seen earlier, the category is identified first and then the second stage neural network is incrementally trained with the features of the new object without forgetting previously learnt labels. The proposed hierarchical framework is shown to provide comparable recognition accuracy with significant reduction in overall computational time in recognizing new objects compared to methods that use end-to-end re-training. The efficacy of the approach is demonstrated through comparison with existing state-of-the-art methods on the publicly available CORe50 dataset.

The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.

This paper presents a deep network based unsupervised visual odometry system for 6-DoF camera pose estimation and finding dense depth map for its monocular view. The proposed network is trained using unlabeled binocular stereo image pairs and is shown to provide superior performance in depth and ego-motion estimation compared to the existing state-of-the-art. This is achieved by introducing a novel objective function and training the network using temporally alligned sequences of monocular images. The objective function is based on the Charbonnier penalty applied to spatial and bi-directional temporal reconstruction losses. The overall novelty of the approach lies in the fact that the proposed deep framework combines a disparity-based depth estimation network with a pose estimation network to obtain absolute scale-aware 6-DoF camera pose and superior depth map. According to our knowledge, such a framework with complete unsupervised end-to-end learning has not been tried so far, making it a novel contribution in the field. The effectiveness of the approach is demonstrated through performance comparison with the state-of-the-art methods on KITTI driving dataset.

This paper presents a novel automatic facial expressions recognition system (AFERS) using the deep network framework. The proposed AFERS consists of four steps: 1) geometric features extraction; 2) regional local binary pattern (LBP) features extraction; 3) fusion of both the features using autoencoders; and 4) classification using Kohonen self-organizing map (SOM)-based classifier. This paper makes three distinct contributions. The proposed deep network consisting of autoencoders and the SOM-based classifier is computationally more efficient and performance wise more accurate. The fusion of geometric features with LBP features using autoencoders provides better representation of facial expression. The SOM-based classifier proposed in this paper has been improved by making use of a soft-threshold logic and a better learning algorithm. The performance of the proposed approach is validated on two widely used databases (DBs): 1) MMI and 2) extended Cohn-Kanade (CK+). An average recognition accuracy of 97.55% in MMI DB and 98.95% in CK+ DB are obtained using the proposed algorithm. The recognition results obtained from fused features are found to be distinctly superior to both recognition using individual features as well as recognition with a direct concatenation of the individual feature vectors. Simulation results validate that the proposed AFERS is more efficient as compared to the existing approaches.

This paper presents an appearance feature based facial expression recognition system using Kohonen Self-Organizing Map (KSOM). Appearance features are extracted using uniform Local binary patterns (LBPs) from equally sub-divided blocks applied over face image. The dimensionality of the LBP feature vector is further reduced using principal component analysis (PCA) to remove the redundant data that leads to unnecessary computation cost. Using our proposed KSOM based classification approach, we train only 59 dimensional LBP features extracted from whole facial region. The classifier is designed to categorize six basic facial expressions (happiness, sadness, disgust, anger, surprise and fear). To validate the performance of the reduced 59 dimensional LBP feature vector, we also train the original data of dimension 944 using the KSOM. The results demonstrates, that with marginal degradation in overall recognition performance, the reduced 59 dimensional data obtains very good classification results. The paper also presents three more comparative studies based on widely used classifiers like; Support vector machine (SVM), Radial basis functions network (RBFN) and Multi-layer perceptron (MLP3). Our KSOM based approach outperforms all other classification methods with average recognition accuracy 69.18%. Whereas, the average recognition rated obtained by SVM, RBFN and MLP3 are 65.78%, 68.09% and 62.73% respectively.

The paper presents a facial expressions recognition system using Bayesian network. We train the network using probabilistic modeling that draws relationship between facial features, action units and finally recognizes six basic emotions. We propose features extraction methods to get geometric feature vector containing angular informations and appearance feature vector containing moments extracted after applying gabor filter over certain facial regions. Both the feature vectors are further used to draw relationships among Action Units (AUs). The angular informations are directly extracted from the facial landmark points. The geometric features extraction approach contains only 22 dimensional angular informations against direct facial landmarks based approach that contains 136 dimensional feature vector. Facial activities are represented by three distinct layers. Bottom level contains landmark measurement data with angular features. Middle level has facial AUs those are coded in facial action coding system (FACS) and the top level, represents emotion node. We also propose a method using k-means clustering to automatically define the states of nodes in anatomical layer that draws relationship among AUs and measurement data. Extended Cohn Kanade Database is being used for our experimental purposes. An average emotion recognition accuracy of 95.7% is achieved using proposed Bayesian network based approach for 22 dimensional angular feature vector. To verify the performance of the proposed approach we apply three different classifiers such as, Support vector machine, Decision tree and Radial basis functions network. The confusion matrices show that the Bayesian network based classification approach outperforms all other applied approaches. The experimental results illustrates the effectiveness of the proposed model.

Machining parameters optimization is very crucial in any machining process. This research focuses on Multi-objective Evolutionary Algorithm based optimization technique, to determine optimal cutting parameters (cutting speed, feed, and depth of cut) in turning operation. Two conflicting objectives (operation time and tool life) with three constraints, which depends on the turning parameters, are optimized using Genetic algorithm (GAs). The Pareto-optimal front of the bi-objective problem is obtained using Non-dominated Sorting Genetic Algorithm (NSGA-II). The extreme and intermediate points of Pareto optimal front is verified using Real coded Genetic Algorithm (RGA) as well as ε-constraint method. The performance of NSGA-II is found to be more effective and efficient as compared to micro-GA. Innovization study carried out to correlate cutting parameters with the aforementioned objective functions. The effect of cutting speed is found more as compared to feed rate and depth of cut.