Neural Network pruning is commonly used to reduce the size of a neural network, reducing the memory footprint, while maintaining an acceptable loss. However, currently the only approach explored for removing a parameter from a neural network is to remove the parameter suddenly, irrespective of the pruning method, be it one-shot, iterative or sparsity-induced. We hypothesize that this sudden removal will cause the loss of the information contained within the removed parameters, information which could be useful when retraining the neural network after pruning. To resolve this, we propose Soft Pruning, a method of slowly decaying parameters out of a neural network. We compare this to one-shot pruning on the vision-based tasks of classification, autoencoding, and latent space dimensionality reduction. In every experiment, Soft Pruning is able to match or outperform one-shot pruning; in classification, Soft Pruning enables pruning to significantly greater extents than one-shot pruning, retaining over 60% accuracy where one-shot pruning becomes equivalent to random guessing. In autoencoding, Soft Pruning is able to achieve up to 17% lower loss after pruning. Finally, applied to latent space dimensionality reduction, Soft Pruning is shown to achieve more than 60% lower loss compared to one-shot pruning.

Abstract This study addresses the problem of hand–eye calibration in robotic systems by developing Continual Learning (CL)-based approaches. Traditionally, robots require explicit models to transfer knowledge from camera observations to their hands or base. However, this poses limitations, as the hand–eye calibration parameters are typically valid only for the current camera configuration. We, therefore, propose a flexible and autonomous hand–eye calibration system that can adapt to changes in camera pose over time. Three CL-based approaches are introduced: the naive CL approach, the reservoir rehearsal approach, and the hybrid approach combining reservoir sampling with new data evaluation. The naive CL approach suffers from catastrophic forgetting, while the reservoir rehearsal approach mitigates this issue by sampling uniformly from past data. The hybrid approach further enhances performance by incorporating reservoir sampling and assessing new data for novelty. Experiments conducted in simulated and real-world environments demonstrate that the CL-based approaches, except for the naive approach, achieve competitive performance compared to traditional batch learning-based methods. This suggests that treating hand–eye calibration as a time sequence problem enables the extension of the learned space without complete retraining. The adaptability of the CL-based approaches facilitates accommodating changes in camera pose, leading to an improved hand–eye calibration system.

Deep Reinforcement Learning (DRL) has gained much attention for solving robotic hand-eye coordination tasks from raw pixel values. Despite promising results, training agents using images is hardware intensive often requiring millions of training steps to converge incurring long training times and increased risk of wear and tear on the robot. To speed up training, images are often cropped and downscaled resulting in a smaller field of view and loss of valuable high-frequency data. In this paper, we propose training the vision system using supervised learning prior to training robotic actuation using Deep Deterministic Policy Gradient (DDPG). The vision system uses a software retina, based on the mammalian retino-cortical transform, to preprocess full-size images to compress image data while preserving the full field of view and high-frequency visual information around the fixation point prior to processing by a Deep Convolutional Neural Network (DCNN) to extract visual state information. Using the vision system to preprocess the environment improves the agent's sample complexity and network update speed leading to significantly faster training with reduced image data loss. Our method is used to train a DRL system to control a real Baxter robot's arm, processing full-size images captured by an in-wrist camera to locate an object on a table and centre the camera over it by actuating the robot arm.

The following topics are dealt with: mobile robots; control system synthesis; learning (artificial intelligence); feature extraction; robot vision; autonomous aerial vehicles; feedback; feedforward neural nets; nonlinear control systems; multi-robot systems.

This paper proposes a single-shot approach for recognising clothing categories from 2.5D features. We propose two visual features, BSP (B-Spline Patch) and TSD (Topology Spatial Distances) for this task. The local BSP features are encoded by LLC (Locality-constrained Linear Coding) and fused with three different global features. Our visual feature is robust to deformable shapes and our approach is able to recognise the category of unknown clothing in unconstrained and random configurations. We integrated the category recognition pipeline with a stereo vision system, clothing instance detection, and dual-arm manipulators to achieve an autonomous sorting system. To verify the performance of our proposed method, we build a high-resolution RGBD clothing dataset of 50 clothing items of 5 categories sampled in random configurations (a total of 2,100 clothing samples). Experimental results show that our approach is able to reach 83.2% accuracy while classifying clothing items which were previously unseen during training. This advances beyond the previous state-of-the-art by 36.2%. Finally, we evaluate the proposed approach in an autonomous robot sorting system, in which the robot recognises a clothing item from an unconstrained pile, grasps it, and sorts it into a box according to its category. Our proposed sorting system achieves reasonable sorting success rates with single-shot perception.

In this paper, we propose a Gaussian Process-based interactive perception approach for recognising highly-wrinkled clothes. We have integrated this recognition method within a clothes sorting pipeline for the pre-washing stage of an autonomous laundering process. Our approach differs from reported clothing manipulation approaches by allowing the robot to update its perception confidence via numerous interactions with the garments. The classifiers predominantly reported in clothing perception (e.g. SVM, Random Forest) studies do not provide true classification probabilities, due to their inherent structure. In contrast, probabilistic classifiers (of which the Gaussian Process is a popular example) are able to provide predictive probabilities. In our approach, we employ a multi-class Gaussian Process classification using the Laplace approximation for posterior inference and optimising hyper-parameters via marginal likelihood maximisation. Our experimental results show that our approach is able to recognise unknown garments from highly-occluded and wrinkled configurations and demonstrates a substantial improvement over non-interactive perception approaches.

We present a visually guided, dual-arm, industrial robot system that is capable of autonomously flattening garments by means of a novel visual perception pipeline that fully interprets high-quality RGB-D images of a clothing scene based on an active stereo robot head. A segmented clothing range map is B-Spline smoothed prior to being parsed by means of shape and topology analysis into ‘wrinkle’ structures. The length, width and height of each wrinkle is used to quantify the topology of each wrinkle and thereby rank wrinkles by size such that a greedy algorithm can identify the largest wrinkle present. A flattening plan optimised for the largest detected wrinkle is formulated based on dual-arm manipulation. We report the validation of our autonomous flattening behaviour and observe that dual-arm flattening requires significantly fewer manipulation iterations than single-arm flattening. Our experimental results also reveal that the flattening process is heavily influenced by the quality of the RGB-D sensor: use of a custom off-the-shelf high-resolution stereo-based sensor system outperformed a commercial low-resolution kinect-like camera in terms of required flattening iterations.

This paper presents insights gained from applying biological vision mechanisms in the context of computer vision algorithm design, implementation and initial evaluation. In this paper a software-based space variant log(z) retina tessellation is used to sample the underlying image, maintaining the high resolution at the central foveal region and an increasingly sparse sampling density at the surrounding peripheral region, in a manner similar to that found in biological vision. This multiresolution sampling strategy differs fundamentaly from conventional image processing where whole field-of-view is processed with equal emphasis. We compare the log(z) retina tessellation sampling quality using simple point based to that of “touching circle” receptive field (RF) sampling. Multi-resolution, space-variant visual information is extracted on a scale-space continuum and SIFT features are then extracted from back-projected retinal response images for matching and recognition.

We present a novel method of image expansion using vector quantisation. The algorithm is inspired by fractal coding and uses a statistical model of the relationship between details at different scales of the image to interpolate detail at one octave above the highest spatial frequency in the original image. Our method aims at overcoming the drawbacks associated with traditional approaches such as pixel interpolation, which smoothes the scaled-up images, or fractal coding, which bears high computational cost and has limited use due to patent restrictions. The proposed method is able to regenerate plausible image detail that was irretrievable when traditional approaches are used. The vector quantisation-based method outperforms conventional approaches in terms of both objective and subjective evaluations. 1

Our paper presents a fully automated computational mechanism for targeting a space-variant retina based on the high-level visual content of a scene. Our retina's receptive fields are organised at a high density in the central foveal region of the retina and at a sparse resolution in the surrounding periphery in a non-uniform, locally pseudo-random tessellation similar to that found in biological vision. Multi-resolution, space-variant visual information is extracted on a scale-space continuum and interest point descriptors are extracted that represent the visual appearance of local regions. We demonstrate the vision system performing simple visual reasoning tasks with the extracted visual descriptors by combining the sparse information from its periphery (which gives it a wide field of view) and the high resolution information from the fovea (useful for accurate reasoning). High-level semantic concepts about content in the scene such as object appearances are formed using the extracted visual evidence, and the system performs saccadic explorations by serially targeting 'interesting' regions in the scene based on the location of high-level visual content and the current task it is trying to achieve.

This paper presents a general framework that aims to address the task of segmenting three-dimensional (3-D) scan data representing the human form into subsets which correspond to functional human body parts. Such a task is challenging due to the articulated and deformable nature of the human body. A salient feature of this framework is that it is able to cope with various body postures and is in addition robust to noise, holes, irregular sampling and rigid transformations. Although whole human body scanners are now capable of routinely capturing the shape of the whole body in machine readable format, they have not yet realized their potential to provide automatic extraction of key body measurements. Automated production of anthropometric databases is a prerequisite to satisfying the needs of certain industrial sectors (e.g., the clothing industry). This implies that in order to extract specific measurements of interest, whole body 3-D scan data must be segmented by machine into subsets corresponding to functional human body parts. However, previously reported attempts at automating the segmentation process suffer from various limitations, such as being restricted to a standard specific posture and being vulnerable to scan data artifacts. Our human body segmentation algorithm advances the state of the art to overcome the above limitations and we present experimental results obtained using both real and synthetic data that confirm the validity, effectiveness, and robustness of our approach.

This paper considers the construction of a biologically inspired front-end for computer vision based on an artificial retina pyramid with a self-organised pseudo-randomly tessellated receptive field tessellation. The organisation of photoreceptors and receptive fields in biological retinae locally resembles a hexagonal mosaic, whereas globally these are organised with a very densely tessellated central foveal region which seamlessly merges into an increasingly sparsely tessellated periphery. In contrast, conventional computer vision approaches use a rectilinear sampling tessellation which samples the whole field of view with uniform density. Scale-space interest points which are suitable for higher level attention and reasoning tasks are efficiently extracted by our vision front-end by performing hierarchical feature extraction on the pseudo-randomly spaced visual information. All operations were conducted on a geometrically irregular foveated representation (data structure for visual information) which is radically different to the uniform rectilinear arrays used in conventional computer vision.

This paper describes recent research concerning the Active Stereo Probe (ASP) stereo vision system that combines animate oculo-motor reflexes with a novel active illumination source. Two methods for accurate surface recovery from stereo at rates approaching real time are described. Results using texture projection combined with a fast sub-pixel matching algorithm are presented. The development of a high accuracy, fast active ranging technique based upon temporal modulation is also presented. A number of novel extensions are introduced to address this technique's traditional shortfalls and place it in a sound theoretical framework. A calibration regime suitable for incorporating photogrammetric techniques into the ASP active vision system during dynamic system operation is described. 1.

This paper describes two preliminary experiments concerned with the construction of a robot head. The initial design and research is aimed at producing a system with two cameras and two microphones on a system capable of operating with the same degrees of freedom and reflex times as its biological counterpart. Whilst the primary goal of the project is develop an anthropomorphic system with the sensory reflex capabilities of a human head, the system will also contain some non-anthropomorphic components. The most obvious of the non-anthropomorphic components is a spatially and temporally programmable light source. Some preliminary results are presented. use this information to direct autonomous behaviour (Brooks, 1989) but these reflexes have not been coupled