Over the last decade, there has been an increasing emphasis on driver-assistance systems for the automotive domain. In this article, we report our work on designing a camera-based surveillance system embedded in a “smart” car door. Such a camera is used to monitor the ambient environment outside the car, for instance, the presence of obstacles such as approaching cars or cyclists who might collide with the car door if opened—and automatically control the car door operations. This is an enhancement to the currently available side-view mirrors that the driver/passenger checks before opening the car door. The focus of this article is on fast and robust image processing algorithms specifically targeting such a smart car door system. The requirement is to quickly detect traffic objects of interest from grayscale images captured by omnidirectional cameras. While known algorithms for object extraction from the image processing literature rely on color information and are sensitive to shadows and illumination changes, our proposed algorithms are highly robust, can operate on grayscale images (color images are not available in our setup), and output results in real time. We present a number of experimental results based on image sequences captured from real-life traffic scenarios to demonstrate the applicability of our algorithm.

The admission control problem is concerned with determining whether a new task may be accepted by a system consisting of a set of running tasks, such that the already admitted and the new task are all schedulable. Clearly, admission control decisions are to be taken on-line, and hence, this constitutes a general problem that arises in many real-time and embedded systems. As a result, there has always been a strong interest in developing efficient admission control algorithms for various setups. In this paper, we propose a novel constant-time admission control test for the Deadline Monotonic (DM) policy, i.e., the time taken by the test does not depend on the number of admitted tasks currently in the system. While it is possible to adapt known utilization bounds from the literature to derive constant-time admission control tests (e.g., the Liu and Layland bound, or the more recent hyperbolic bound), the test we propose is less pessimistic. We illustrate this analytically where possible and through a set of detailed experiments. Apart from the practical relevance of the proposed test in the specific context of DM tasks, the underlying technique is general enough and can possibly be extended to other scheduling policies as well.

An admission control test is responsible for deciding whether a new task may be accepted by a set of running tasks, such that the already admitted and the new task are all schedulable. Admission control decisions have to betaken on-line and, hence, there is a strong interest in developing efficient algorithms for different setups. In this paper, we propose a novel constant-time admission control test for tasks scheduled on identical processors under partitioned Earliest Deadline First (EDF), i.e., once tasks have been assigned to a processor they remain on that processor. In particular, to model demanding real-time systems, we consider the case where relative deadlines may be less than the minimum separation between two consecutive task activations or jobs. The main advantage of the proposed test is that the time it takes is independent of the number of tasks currently admitted in the system. While it is possible to adapt polynomial-time schedulability tests from the literature to design a linear or even constant-time admission control for this setup, the test we propose provides a better accuracy/complexity ratio. We evaluate this test through a set of detailed experiments based on synthetic tasks and a realistic case study consisting of a real-time multimedia server.

Maximizing passengers comfort is an important research topic in the domain of automotive systems engineering. In particular, an automatic adjustment of seat position according to driver height significantly increases the level of comfort during ingress. In this paper, we present a new method to estimate the height of approaching car drivers based on a single omni directional camera integrated with the side-view mirror of a car. Towards this, we propose mathematical descriptions of standard parking scenarios, allowing for an accurate height estimation. First, approaching drivers are extracted from image frames captured by the camera. Second, the scenario and height are initially estimated based on gathered samples of angles to head and foot-points of an approaching driver. An iterative optimization process removes outliers and refines the initially estimated scenario and height. Finally, we present a number of experimental results based on image sequences captured from real-life ingress scenarios.

The collaborative research center SFB453 (www.sfb453.de) aims to realize high-fidelity telepresence and teleaction systems. Telepresence and teleaction systems extend the human workspace to remote locations in order to overcome barriers like distance, scaling, danger or the human skin. Using a human-system interface the human operator controls a remotely located teleoperator. Multi-modal feedback in form of visual, auditory, and haptic data is used to increase the feeling of telepresence. Different application areas including minimally invasive surgery, on-orbit servicing, microassembly as well as tele-manufacturing and tele-maintenance are targeted.

This paper presents two fully polynomial-time sufficient feasibility tests for EDF when considering periodic tasks with arbitrary deadlines and preemptive scheduling on uniprocessors. Both proposed methods are proven, analytically and by means of an extensive experimental comparison, to be more accurate than known polynomial-time feasibility tests. Additionally, we show for a wide interval of practical processor utilization that one of these methods presents almost the same efficiency, in terms of accepted task sets, as the more complex pseudo-polynomial-time exact feasibility tests.

The video shows a rich multi-modal multi-user telepresence system, which was developed within the SFB453 funded by the German Research Foundation (www.sfb453.de). As a complex application scenario, the remote repairing of a broken pipe is presented in this paper. The system basically consists of two operator-teleoperator- pairs. While one of the operators interacts with a stationary human- system-interface, the other operator uses a mobile one. Both systems provide visual, auditory, and haptic feedback and enable to control the motion of head, arms, and grippers as well as the locomotion of the corresponding teleoperator.

When evaluating worst-case execution times of real-time software on an off-the-shelf multiprocessor system, one should consider the architecture of the underlying hardware. This paper evaluates and discusses the impacts of the chipset of a symmetric multi-processing (SMP) architecture on the execution times of software. It will show how one can obtain worst-case execution times of accesses to main memory and to peripheral devices. This paper also introduces parameters to describe the different impacts on these access times. These parameters reflect the worst-case execution times of a set of uninterrupted accesses at different load conditions. Depending on the real-time requirements, accesses to main memory or to peripheral devices can be performed at different configuration states of hardware. At each state, the worst-case access times differ. This paper shows how these access times can be used for the evaluation of worst-case execution times of real-time software.

Describes a vision system for obstacle detection in mobile robot navigation. The system uses an image processing board equipped with an MPEG motion estimation processor that calculates a robust optic-flow-like vector field in real-time. This field is then evaluated by algorithms running in software on the host PC. As the solutions to the general problem of structure and motion from optic flow are too instable for the use in this application, the typical constraints of mobile robotics are exploited, i.e. a reduced set of motion parameters and a known ground plane. Ego-motion can then be reconstructed with robust one dimensional methods. A new criterion for obstacles that copes well with the noise properties of the motion field is introduced. For vectors belonging to obstacles the 3D information is reconstructed allowing not only qualitative detection of obstacles but quantitative path planning.

Predictive displays have proven their suitability to compensate time delays in visual feedback which impair the performance of teleoperation applications. Most current implementations employ simple computer graphics for the prediction of the visual feedback given to the operator who is controlling the remote manipulation. Furthermore, they require a well-known remote environment. The aim of the presented work is to improve the efficacy of the predictive display by increasing the degree of immersion and by enhancing the system's flexibility. Potential applications include maintenance and repair of machines located in remote plant- or office-type environments, which are typically non-static and a priori only partially known. Communication media are high- and medium-bandwidth LANs and WANs inducing round trip delays varying from 0.2 to 2 seconds. Additionally, a sufficient bandwidth cannot be assured during the entire remote operation. The paper describes a predictive display system which achieves photo-realism based on autonomously explored and updated scene models. Model acquisition is performed with a binocular CCD camera system. Photo-realism is obtained by using computer graphics techniques, especially the mapping of original, real textures. Besides the concepts, an early stage of implementation and first experimental results in scene reconstruction and photo-realistic scene prediction are presented.

The presented approach integrates proven techniques and original approaches to one robust and fast 3D model-based recognition system. A speedy recognition is achieved by operating on a stream of filtered 3D sensor features, reconstructed for navigation tasks of the robot, instead of using a separate sensor data processing. Furthermore, simple, inexpensive recognition strategies are applied. Robustness is obtained by integrating complementary recognition strategies: four indexing techniques and two (2D and 3D) matching methods for verification, are completed by a hypothesis promotion, based on feedback of information to the sensor system. All strategies differ in their requirements, reliability, selectivity, and temporal constraints. Hypotheses are integrated using fusion, ruling out, and aging techniques. The approach is evaluated experimentally with varying calibration errors, scene complexity, and sensing conditions.

We present a modular system architecture for simplifying the signal level fusion and sensor configuration of multiple sensors with different sample rates. The fusion process is subdivided into three modules: synchronization, adaptation and fusion. Methods for all three stages are proposed and compared with classical approaches. All sensor sample times, even if they are not periodic, are propagated to the next sample time of the control unit. In the adaptation module we focus on the double integration of biased signals and the determination of the unknown initial conditions. We distinguish four different types of sensors and propose sensor models suitable for off-line simulation that can be automatically changed to interface blocks in case of an online simulation with real sensors. All methods needed for handling our system architecture are embedded in a Matlab toolbox.

Research has focused on developing control strategies that allow to work with imprecisely calibrated or even uncalibrated robot systems. A popular approach is to close the control loop with visual feedback (visual servoing), even though this requires continuous, high rate image processing. In contrast, we developed a strategy that can compensate calibration errors by integrating sparse visual feedback asynchronously. This strategy was motivated by neuroscientific studies which show that human grasping definitely requires only sparse asynchronous visual feedback, even with disturbed visual input. We analyze the performance of our control strategy on a robotic hand-eye system consisting of a 6 d.o.f. manipulator and a pan-tilt head with a stereo camera system. To analyze thoroughly which calibration errors can be compensated by sparse feedback, we furthermore built a complete simulation model of the real hand-eye system, including the vision modules. Taken together, experimental and simulation results show that calibration errors can be compensated with a very moderate amount of (asynchronous) visual feedback. Thus, the proposed control strategy can indeed be seen as a serious alternative to visual servoing for real robotics applications. This shows that robotics can profit from taking a closer look at the results of neuroscience.

In the field of robot motion control, visual servoing has been proposed as the suitable strategy to cope with imprecise models and calibration errors. Remaining problems such as the necessity of a high rate of visual feedback are deemed to be solvable by the development of real-time vision modules. However, human grasping, which still outshines its robotic counterparts especially with respect to robustness and flexibility, definitely requires only sparse, asynchronous visual feedback. We therefore examined current neuroscientific models for the control of human reach-to-grasp movements with the emphasis lying on the visual control strategy used. From this, we developed a control model that unifies the two robotic strategies look-then-move and visual servoing, thereby compensating the problems that each strategy shows when used alone.

This paper proposes an approach for motion planning in indoor environments based on incomplete and uncertain information from a line-based binocular stereo system. The primary goal of the planning process is to plan an optimal path through an unknown or partially known environment, depending on the information gained from exploration and the current mission goal. This paper presents an adaptable motion planner that supports sensor-based map construction, object recognition and navigation in an unknown environment while carrying out a mission. Also presented are some preliminary experimental results that demonstrate the utility of the approach.

Component-based concepts are being increasingly used in software design since they allow high-quality software to be generated within a short time and hence at low costs. By contrast, component technologies haven't yet provided the means of composing software from components without additional programming. The reason for this lies in the combined transfer of control and data flow and the fact that there is no standardized control flow. An architecture model is presented that separates the transfer of the two information flows. The data flow is transferred via an active real-time database (ARTDB) and the control flow reduced to a few, indispensable commands. In this way it is possible to design applications simply by combining and configuring components without any additional code development. Initial positive experience with this architecture was gained from its industrial application. It was possible to demonstrate the suitability for systems with soft deadlines.

In the field of hand-eye coordination, most state-of-the-art systems still require the user to select the grasping points manually. We present a system which autonomously determines 3D grasping points on unknown objects from a pair of greyscale images. The object to be grasped is segmented automatically when put into the scene. Grasping points are searched on the object silhouette; their stability is evaluated by a heuristic algorithm, primarily based on the skeleton of the region. The 3D grasping pose is estimated by triangulation using a simplified geometrical model of the camera system; the corresponding points in the second image are determined via dynamic programming. The whole system has been implemented and validated on the experimental hand-eye system MINERVA.