Existing skeleton-based human action classification models rely on well-trimmed action-specific skeleton videos for both training and testing, precluding their scalability to real-world applications where untrimmed videos exhibiting concatenated actions are predominant. To overcome this limitation, recently introduced skeleton action segmentation models involve un-trimmed skeleton videos into end-to-end training. The model is optimized to provide frame-wise predictions for any length of testing videos, simultaneously realizing action localization and classification. Yet, achieving such an improvement im-poses frame-wise annotated skeleton videos, which remains time-consuming in practice. This paper features a novel framework for skeleton-based action segmentation trained on short trimmed skeleton videos, but that can run on longer un-trimmed videos. The approach is implemented in three steps: Stitch, Contrast, and Segment. First, Stitch proposes a tem-poral skeleton stitching scheme that treats trimmed skeleton videos as elementary human motions that compose a semantic space and can be sampled to generate multi-action stitched se-quences. Contrast learns contrastive representations from stitched sequences with a novel discrimination pretext task that enables a skeleton encoder to learn meaningful action-temporal contexts to improve action segmentation. Finally, Segment relates the proposed method to action segmentation by learning a segmentation layer while handling particular da-ta availability. Experiments involve a trimmed source dataset and an untrimmed target dataset in an adaptation formulation for real-world skeleton-based human action segmentation to evaluate the effectiveness of the proposed method.

A prevalent family of fully convolutional networks are capable of learning discriminative representations and producing structural prediction in semantic segmentation tasks. However, such supervised learning methods require a large amount of labeled data and show inability of learning cross-domain invariant representations, giving rise to overfitting performance on the source dataset. Domain adaptation, a transfer learning technique that demonstrates strength on aligning feature distributions, can improve the performance of learning methods by providing inter-domain discrepancy alleviation. Recently introduced output-space based adaptation methods provide significant advances on cross-domain semantic segmentation tasks, however, a lack of consideration for intra-domain divergence of domain discrepancy remains prone to over-adaptation results on the target domain. To address the problem, we first leverage prototypical knowledge on the target domain to relax its hard domain label to a continuous domain space, where pixel-wise domain adaptation is developed upon a soft adversarial loss. The development of prototypical knowledge allows to elaborate specific adaptation strategies on under-aligned regions and well-aligned regions of the target domain. Furthermore, aiming to achieve better adaptation performance, we employ a unilateral discriminator to alleviate implicit uncertainty on prototypical knowledge. At last, we theoretically and experimentally demonstrate that the proposed prototypical knowledge oriented adaptation approach provides effective guidance on distribution alignment and alleviation on over-adaptation. The proposed approach shows competitive performance with state-of-the-art methods on two cross-domain segmentation tasks.

In this paper, we propose a biologically-plausible model to explain the emergence of motion tracking behaviour in early development using unsupervised learning. The model's training is biased by a concept called retinal constancy, which measures how similar visual contents are between successive frames. This biasing is similar to a reward in reinforcement learning, but is less explicit, as it modulates the model's learning rate instead of being a learning signal itself. The model is a two-layer deep network. The first layer learns to encode visual motion, and the second layer learns to relate that motion to gaze movements, which it perceives and creates through bi-directional nodes. By randomly generating gaze movements to traverse the local visual space, desirable correlations are developed between visual motion and the appropriate gaze to nullify that motion such that maximal retinal constancy is achieved. Biologically, this is similar to using saccades to look around and learning from moments where a target and the saccade move together such that the image stays the same on the retina, and developing smooth pursuit behaviour to perform this action in the future. Restricted Boltzmann machines are used to implement this model because they can form a deep belief network, perform online learning, and act generatively. These properties all have biological equivalents and coincide with the biological plausibility of using saccades as leverage to learn smooth pursuit. This method is unique because it uses general machine learning algorithms, and their inherent generative properties, to learn from real-world data. It also implements a biological theory, uses motion instead of recognition via local searches, without temporal filtering, and learns in a fully unsupervised manner. Its tracking performance after being trained on real-world images with simulated motion is compared to its tracking performance after being trained on natural video. Results show that this model is able to successfully follow targets in natural video, despite partial occlusions, scale changes, and nonlinear motion.

The paper proposes an automated method for the modeling of objects using multiple discrete levels of detail for virtual reality applications. The method combines classical discrete level of detail approaches with a novel solution for the creation of selectively-densified object meshes. A neural gas network is used to capture regions of interest over a sparse point cloud representing a 3D object. Meshes at different resolutions that preserve these regions are then constructed by adapting a classical simplification algorithm to allow the simplification process to affect only the regions of lower interest. Different interest point detectors are incorporated in a similar manner and compared with the proposed approach. A novel solution based on learning is proposed to select the number of faces for the discrete models of an object at different resolutions.

Simultaneous acquisition of depth and texture information, such as that provided by RGB-D sensors, finds an ever increasing number of applications, including objects modeling, human-machine interfaces, and robot navigation. One of the challenges resulting from the use of densely populated 3D datasets originates from the massive acquisition, management and processing of the data generated. This reality often preempts full usage of the information available for autonomous systems to make educated decisions. Current methods for reducing dataset's dimension remain independent from the content of the model and therefore do not optimize the balance between the richness of the measurements and their compression. This paper presents two computational methods to selectively drive the selection of depth measurements over the most significant regions of a scene, characterized by their 3D features distribution, while capitalizing on the knowledge readily available in previously acquired depth data. One of the methods builds on self-organizing neural networks, namely neural gas, while the second one computes an empirical improvement metric. Both techniques are adapted to automatically establish which subset of depth measurements within a range sensor's field of view contribute most to the representation of the scene, and therefore streamline the depth measurements acquisition process.

This paper discusses the design and implementation of a framework that automatically extracts and monitors the shape deformations of soft objects from a video sequence and maps them with force measurements with the goal of providing the necessary information to the controller of a robotic hand to ensure safe model-based deformable object manipulation. Measurements corresponding to the interaction force at the level of the fingertips and to the position of the fingertips of a three-finger robotic hand are associated with the contours of a deformed object tracked in a series of images using neural-network approaches. The resulting model captures the behavior of the object and is able to predict its behavior for previously unseen interactions without any assumption on the object's material. The availability of such models can contribute to the improvement of a robotic hand controller, therefore allowing more accurate and stable grasp while providing more elaborate manipulation capabilities for deformable objects. Experiments performed for different objects, made of various materials, reveal that the method accurately captures and predicts the object's shape deformation while the object is submitted to external forces applied by the robot fingers. The proposed method is also fast and insensitive to severe contour deformations, as well as to smooth changes in lighting, contrast, and background.

The goal of the work presented in this paper is to develop a novel scheme for the measurement and representation of deformable objects without a priori knowledge on their shape or material. The proposed solution advantageously combines a neural gas network and feedforward neural network architectures to achieve diversified tasks as required for data collection on one side and the modeling of elastic characteristics on the other side. Data is collected for different objects using a joint sensing strategy that combines tactile probing and range imaging. The innovative object models, built as multi-resolution point-clouds associated with ¿tactile patches¿, present certain advantages over classical deformable 3D object models.

In a previous paper by Payeur et al. (1997), probabilistic occupancy modeling has been successfully extended to 3D environments by means of a closed-form approximation of the probability distribution. In this paper, the closed-form approximation is revisited in order to provide more reliable and meaningful models. A merging strategy of local probabilistic occupancy grids originating from each sensor viewpoint is introduced. The merging process takes advantage of the multiresolution characteristics of octrees to minimize the computational complexity and enhance performances. An experimental testbed is used to validate the approach and models computed from real range images are presented.

Probabilistic occupancy grids have proved to be very useful for workspace modeling in 2D environments. Due to the expansion of computational load, this approach was not tractable for mapping a 3D environment in real applications. In this paper, the original occupancy grid scheme is revisited and a generic closed-form function is introduced to avoid numerical computation of probabilities for a range sensor with Gaussian error distribution. Occupancy probabilities are computed and stored in a multiresolution octree for improved performance and compactness. Occupancy models are built in local reference frames and linked to a global reference frame through uncertain spatial relationships that can be updated dynamically. This scheme is used for building a 3D map in a telerobotic maintenance application of electric power lines where perturbations may cause motion of object assembly.