Multimedia content production is nowadays widespread due to technological advances, namely supported by smartphones and social media. Although the massive amount of media content brings new opportunities to the industry, it also obfuscates the relevance of marketing content, meant to maintain and lure new audiences. This leads to an emergent necessity of producing these kinds of contents as quickly and engagingly as possible. Creating these automatically would decrease both the production costs and time, particularly by using static media for the creation of short storytelling animated clips. We propose an innovative approach that uses context and content information to transform a still photo into an appealing context-aware video clip. Thus, our solution presents a contribution to the state-of-the-art in computer vision and multimedia technologies and assists content creators with a value-added service to automatically build rich contextualized multimedia stories from single photographs.

High-speed visual servo systems are used in an increasing number of applications. Yet modeling and optimizing these systems remains a research challenge, largely because these systems consist of tightly-coupled design parameters across multiple domains, including image sensors, vision algorithms, processing systems, mechanical systems, control systems, among others. To overcome such a challenge, this work applies an axiomatic design method to the design of high-speed visual servo systems, such that cross-domain couplings are explicitly modeled and subsequently eliminated when possible. More importantly, methods are proposed to model the sample rate, measurement error, and delay of visual feedback based on design parameters across multiple domains. Lastly, methods to construct a holistic model and to perform cross-domain optimization are proposed. The proposed methods are applied to a representative case study that demonstrates the necessity of cross-domain modeling and optimization, as well as the effectiveness of the proposed methods.

Safe and comfortable path planning in a dynamic urban environment is essential to an autonomous vehicle. This requires the future trajectories of all other road users in the environment of the vehicle. These trajectories are predicted through modeling the motion and behaviour of these road users. In this work we state that for efficient trajectory prediction only motion indicators are not sufficient. Therefore, we propose using a curvilinear coordinate system with curvature as road infrastructure indicators to improve motion modeling and trajectory prediction. With experiments, we show that the curvilinear coordinate system with curvature sufficiently incorporates the road structure. Furthermore, we show that a sequence-to-sequence RNN model is suitable to incorporate road curvature indicators directly into the modeling and prediction.

This paper presents a fully automated system for detecting, classifying, microscopic imaging, and individually identifying multiple parts without ID-marking or tagging. The system is beneficial for product assemblers, who handle multiple types of parts simultaneously. They can ensure traceability quite easily by only placing the parts freely on the system platform. The system captures microscopic images of parts as their "fingerprints," which are matched with pre-registered images in a database to identify an individual part's information such as its serial number. We demonstrate a working prototype and interaction scenario.

This paper presents a new identification system for tiny parts that have no space for applying conventional ID marking or tagging. The system marks the parts with a single dot using ink containing shiny particles. The particles in a single dot naturally form a unique pattern. The parts are then identified by matching microscopic images of this pattern with a database containing images of these dots. In this paper, we develop an automated system to conduct dotting and image capturing for mass-produced parts. Experimental results show that our "Tip-on-a-chip" system can uniquely identify more than ten thousand chip capacitors.

In this paper, a novel active vision strategy is proposed for optimizing the viewpoint of a robot's vision sensor for a given success criterion. The strategy is based on extremum seeking control (ESC), which introduces two main advantages: 1) Our approach is model free: It does not require an explicit objective function or any other task model to calculate the gradient direction for viewpoint optimization. This brings new possibilities for the use of active vision in unstructured environments, since a priori knowledge of the surroundings and the target objects is not required. 2) ESC conducts continuous optimization backed up with mechanisms to escape from local maxima. This enables an efficient execution of an active vision task. We demonstrate our approach with two applications in the object recognition and manipulation fields, where the model-free approach brings various benefits: for object recognition, our framework removes the dependence on offline training data for viewpoint optimization, and provides robustness of the system to occlusions and changing lighting conditions. In object manipulation, the model-free approach allows us to increase the success rate of a grasp synthesis algorithm without the need of an object model; the algorithm only uses continuous measurements of the objective value, i.e., the grasp quality. Our experiments show that continuous viewpoint optimization can efficiently increase the data quality for the underlying algorithm, while maintaining the robustness.

This paper presents a systematic approach to evaluate the tracking performance limits for different sensor modalities (lidar, radar and vision) and for combination of these sensors modalities. The Cramer-Rao lower bound (CRLB) is used to predict the tracking performance limits for state of the art sensors such as the Continental ARS408 radar, Velodyne HDL-64E lidar and a state of the art monocular/stereo camera. The performance is evaluated by computing the theoretical CRLB in urban and highway environments. In both scenarios, the best performance was achieved by a combination of lidar and radar. In the close range, stereo vision improves the longitudinal tracking performance limits. Furthermore, radar is crucial on highways because of the quick longitudinal convergence characteristics.

This paper proposes a quality of service multi-sensor bootstrap filter for automated driving that deals with time-varying or state dependent conditions. In this way, the reliability of the sensor data fusion system is continuously evaluated in order to detect potentially dangerous conditions such as sensor failure or adverse environmental conditions such as rain and fog. Simulations show that the proposed robust multi-sensor bootstrap filter is able to robustly estimate the quality of service of the sensors. Furthermore, the filter outperforms tracking filters that assume a perfect detection profile. In addition, real world experiments in a fog simulator show that the proposed multi-sensor local-bootstrap-LMB filter outperforms all other filters in foggy conditions.

In this paper, we introduce a non-verbal multimodal joint visual attention model for human-robot interaction in household scenarios. Our model combines the bottom-up saliency and depth-based segmentation with the top-down cues such as pointing and gaze to detect the objects of interest according to the user. For generation of the top-down saliency maps, we have introduced novel methods for object saliency, based on the pointing direction as well as the gaze direction. For gaze estimation, a hybrid model has been introduced which automatically selects keypoint-based matching or back-projection based on the textureness of the object model. The combination of different cues ensures reliable object detection and interaction independent of the relative position between the user, robot and objects. Extensive experiments show good detection results in different interaction scenarios as well as in challenging environmental conditions.

This paper presents direct methods for vision-based control for the application of industrial inkjet printing. In this, visual control is designed with a direct coupling between camera measurements and joint motion. Traditional visual servoing commonly has a slow visual update rate and needs an additional local joint controller to guarantee stability. By only using the product as reference and sampling with a high update rate, direct visual measurements are sufficient for controlled positioning. The proposed method is simpler and more reliable than standard motor encoders, despite the tight real-time constraints. This direct visual control method is experimentally verified with a 2D planar motion stage for micrometer positioning. To achieve accurate and fast motion, a balance is found between frame rate and image size. With a frame rate of 1600 fps and an image size of 160 × 100 pixels we show the effectiveness of the approach.

This paper presents a novel mediated reality system designed to support collaboration between crime scene investigators during a first analysis on a crime scene, remotely supported by expert colleagues. Requirements elicited from interviews and interactive sessions with practicing crime scene investigators, provided the design criteria for a head mounted display capable of real-time map-making for spatial collaboration. The resulting system is evaluated in a staged crime scene experiment in which lay investigators collaboratively solve a spatial problem with remote experts. The results show that our novel approach to remote spatial interaction with the physical scene enables investigators to tackle current issues on site in collaboration with experts at a distance.

Extensive research has been conducted in the domain of object tracking. Among the existing tracking methods, most of them mainly focus on using various cues such as color, texture, contour, features, motion as well as depth information to achieve a robust tracking performance. The tracking methods themselves are highly emphasized while properties of the objects to be tracked are usually not exploited enough. In this paper, we first propose a novel adaptive tracking selection mechanism dependent on the properties of the objects. The system will automatically choose the optimal tracking algorithm after examining the textureness of the object. In addition, we propose a robust tracking algorithm for uniform objects based on color information which can cope with real world constraints. In the mean time, we deployed a textured object tracking algorithm which combines the Lucas-Kanade tracker and a model based tracker using the Random Forests classifier. The whole system was tested and the experimental results on a variety of objects show the effectiveness of the adaptive tracking selection mechanism. Moreover, the promising tracking performance shows the robustness of the proposed tracking algorithm. The computation cost of the algorithm is very low, which proves that it can be further used in various real-time robotics applications.

The purpose of this paper is to help engineers and researches to choose among the extremum seeking control (ESC) techniques for robotic applications such as object grasping, active object recognition and viewpoint optimization. These techniques are categorized into five main groups: Sliding mode ESC, neural network ESC, approximation based ESC, perturbation based ESC and adaptive ESC. These groups are explained briefly by stressing their working principles and the effect of the parameters. Then, the techniques are compared with respect to their robustness to noise and system dynamics by simulations. In conclusion, we propose the usage of the approximation based methods when the noise level is negligible. When noise is present, the neural network based optimizers are a better choice thanks to their hysteresis functions. However, if the system has both high noise and dynamic effects, then the perturbation based method is preferable since large motions provide robustness to noise and smooth references generated by the algorithm are less likely to cause instability. An application example is also given on texture density maximization.

A new visual servoing method is proposed which uses position based visual servoing (PBVS) in combination with an additional image based control layer on the target pose to maintain fixation on an object. The proposed method (denoted feed forward PBVS) does not require trajectory generation but instead uses via-points to explore the object. It exploits the advantages of PBVS without the disadvantages of image based visual servoing (IBVS) as occurs in hybrid approaches. The proposed method is experimentally validated with a redundant 7-DOF manipulator. Comparison with existing visual servoing methods (PBVS and one partitioned approach) shows the effectiveness of the method.

In this paper, we present a unifying approach for learning and recognition of objects in unstructured environments through exploration. Taking inspiration from how young infants learn objects, we establish four principles for object learning. First, early object detection is based on an attention mechanism detecting salient parts in the scene. Second, motion of the object allows more accurate object localization. Next, acquiring multiple observations of the object through manipulation allows a more robust representation of the object. And last, object recognition benefits from a multi-modal representation. Using these principles, we developed a unifying method including visual attention, smooth pursuit of the object, and a multi-view and multi-modal object representation. Our results indicate the effectiveness of this approach and the improvement of the system when multiple observations are acquired from active object manipulation.

Grasping unknown objects is a crucial necessity for robots that operate in an unstructured environment. In this paper, we propose a novel grasping algorithm that uses active vision as basis. The algorithm uses the curvature information obtained from the silhouette of the object. By maximizing the curvature value, the pose of the robot is updated and a suitable grasping configuration is achieved. The algorithm has certain advantages over the existing methods: It does not require a 3D model of the object to be extracted, and it does not rely on any knowledge base obtained offline. This leads to a faster and still reliable grasping of the target object in 3D. The performance of the algorithm is examined by simulations and experiments, and successful results are obtained.

Increased product complexity and internal team dynamics pose serious challenges to the quality of collaboration, usually reflected upon long delays, cost overruns and poor design quality during product development. Our focus is to study the factors that drive collaboration, shared understanding and team learning in product development in order to investigate new tools to facilitate this process. The present paper provides the theoretical framework to experiment with novel collaborative tools like Augmented Reality in the product development setting. The core argument is that Augmented Reality technologies act as a catalyst to the communication between the various stakeholders. The main idea behind this work is a dynamic investigation on the nature of collaboration in product development teams, through a socio-cognitive lens. The basic focus of this work is to connect insights from the social sciences to collaborative design and visualization technologies.

In this paper we propose a fast and robust descriptor for multiple view object recognition using a small number of training examples. In order to design a descriptor to be discriminative between many different object appearances, we base it on a combination of invariant color, edge and texture descriptors. We use a color descriptor based on a HSV histogram - as it is robust to size and position of the object -, a gray level cooccurrence matrix as texture descriptor and an edge histogram as shape descriptor. After extraction of feature vectors, we perform normalization on all feature vectors from the training database in order to increase the importance of the most dominant feature components and reduce the less dominant ones. This normalization improves the recognition performance with almost 30% in case of a small number of training objects and in case of noise or occlusion. We tested our descriptor on the Columbia Object Image Library dataset (COIL 100) which presents objects in scaled, translated and rotated versions. Our recognition rate is extremely high: 99% in case of a large number of training objects and 93% for training with only 4 views of the object, or 5% of the database. The descriptor was also tested under various distortions: illumination change, noise corruption and occlusions. It proved to be very robust, with recognition rates decreasing only less then 5%. We compared our results with state of the art methods and we conclude that our descriptor achieves a better performance, both on the regular COIL database and on all distorted variants.

In this paper we present a scene exploration method for the identification of interest regions in unknown indoor environments and the position estimation of the objects located in those regions. Our method consists of two stages: First, we generate a saliency map of the scene based on the spectral residual of three color channels and interest points are detected in this map. Second, we propose and evaluate a method for the clustering of neighboring interest regions, the rejection of outliers and the estimation of the positions of potential objects. Once the location of objects in the scene is known, recognition of objects/object classes can be performed or the locations can be used for grasping the object. The main contribution of this paper lies in a computationally inexpensive method for the localization of multiple salient objects in a scene. The performance obtained on a dataset of indoor scenes shows that our method performs good, is very fast and hence highly suitable for real-world applications, such as mobile robots and surveillance.

Robots controlled by Reinforcement Learning (RL) are still rare. A core challenge to the application of RL to robotic systems is to learn despite the existence of control delay - the delay between measuring a system's state and acting upon it. Control delay is always present in real systems. In this work, we present two novel temporal difference (TD) learning algorithms for problems with control delay. These algorithms improve learning performance by taking the control delay into account. We test our algorithms in a gridworld, where the delay is an integer multiple of the time step, as well as in the simulation of a robotic system, where the delay can have any value. In both tests, our proposed algorithms outperform classical TD learning algorithms, while maintaining low computational complexity.

Real robots demonstrating online Reinforcement Learning (RL) to learn new tasks are hard to find. The specific properties and limitations of real robots have a large impact on their suitability for RL experiments. In this work, we derive the main hardware and software requirements that a RL robot should fulfill, and present our biped robot LEO that was specifically designed to meet these requirements. We verify its aptitude in autonomous walking experiments using a pre-programmed controller. Although there is room for improvement in the design, the robot was able to walk, fall and stand up without human intervention for 8 hours, during which it made over 43; 000 footsteps.

Algorithmic skeletons can be used to write architecture independent programs, shielding application developers from the details of a parallel implementation. In this paper, we present a C-like skeleton implementation language, PEPCI, that uses term rewriting and partial evaluation to specify skeletons for parallel C dialects. By using skeletons to control the iteration of kernel functions, we provide a stream programming language that is better tailored to the user as well as the underlying architecture. Skeleton merging allows us to reduce the overheads usually associated with breaking an application into small kernels. We have implemented an example image processing application on a heterogeneous embedded prototype platform consisting of an SIMD and ILP processor, and show that a significant speedup can be achieved without requiring knowledge of data parallel processing.

SIMD processors are increasingly used in embedded systems for multi-media applications because of their area- and energy-efficiency. Communication between the processing elements (PEs) in an SIMD processor has remained a cause of inefficiency however; the SIMD concept prescribes that all PEs communicate in the same clock cycle. Existing SIMD architectures solve this problem either by multi-hop communication (causing cycle overhead), or by a fully connected communication network (causing area overhead). To solve the communication bottleneck, we propose a reconfigurable SIMD architecture (RC-SIMD) with a set of delay-lines in the instruction bus, distributing the accesses to the communication network over time. We can (re-) configure the size and number of delay-lines, a specific configuration representing a trade-off between the number of clock cycles and the length of a clock period. Reconfiguration time is typically much less than 1% of the execution time of an algorithm, and the extra configuration hardware is less than 2%. Experiments show that our reconfigurable architecture achieves (on average) more than 10% performance improvement over a non-reconfigurable architecture

The paper presents an approach of using algorithmic skeletons for adding data parallelism to an image processing library. The method is used for parallelizing image processing applications composed of low-level image operators on a distributed memory system. In this way, a user who wants to parallelize an image processing application is not involved in the design and the implementation of parallel algorithms, but his only task is how to select for each low-level operator the appropriate skeleton to obtain the parallel version of the application. Example of the multibaseline stereo vision image processing aplication is given for reference.

It is well-known that mixing task and data parallelism to solve large computational applications often yields better speedups compared to either applying pure task parallelism or pure data parallelism. Typically, the applications are modeled in terms of a dependence graph of coarse-grain data-parallel tasks, called a data-parallel task graph. In this paper we present a new compile-time heuristic, named Critical Path Reduction (CPR), for scheduling data-parallel task graphs. Experimental results based on graphs derived from real problems as well as synthetic graphs, show that CPR achieves higher speedup compared to other well-known existing scheduling algorithms, at the expense of some higher cost. These results are also confirmed by performance measurements of two real applications (i.e., complex matrix multiplication and Strassen matrix multiplication) running on a cluster of workstations.

Typical real-time computer vision tasks require huge amount of processing power larger than can be achieved by current state-of-the-art workstations. Parallel processing using heterogeneous SIMD-MIMD systems appears to be the only solution (economically) to obtain sufficient processing power for handling all image processing levels in real-time computer vision applications. In this paper we present the bucket processing technique to bridge the gap between the SIMD and MIMD paradigms, both in the area of programming as well as architecture.

The four main characteristic architectures for low-level image processing are the square processor array (SPA) or mesh, the linear processor array (LPA) or scanning array, the pipeline (PL) and the pyramid (PYR). In this paper a theoretical study is presented which leads from a taxonomy of low-level image processing operations and a theoretical model of a nearest neighbour connected processor to the combinations of 7 possibilities to exchange parallel for sequential solutions when designing an architecture. The implementation of two "standard" image processing routines on the theoretical processor, skeletonization of a binary image and shading correction in a greyvalue image, while making the design decisions in line with the solutions commonly found in the four characteristic architectures, showed that linear processor arrays when properly designed, seem to offer the best speed/efficiency combination even for processing images at video speed.

A real-time pipelined image processing system operating in a time division multiplexing mode to serve up to 16 cameras, was realized to count the mass of a flow of bottles on a conveyor belt. The realized mass counting system proved to be a powerful tool capable of continuously counting bottles with a speed of approximately 500000 bottles a day per measurement point and with an accuracy of less then 0.5%.>

To speed up image processing in the field of robot vision and industrial inspection, a pipeline element was made which is able to perform fast cellular logic operations. This Cellular Logic Processing Element (CLPE) is able to process binary images with a speed of 100 ns per pixel. The processing element is a CMOS VLSI-device which includes a Writable Logic Array for the storage of sets of 3*3 structuring elements which define the cellular logic operations. This paper describes how such CLPEs can be used for building a pipeline for mixed grey value processing and cellular logic processing.>

A product-oriented method for planning the flexible assembly and a reference model for its control system are presented. Product design, process planning, the architecture of a flexible assembly cell and a reference for its control system are presented. The essential difference from existing reference models is that the model presented is tied to the information processing of assembly instead of the assembly process itself.>