The integration of Augmented Reality (AR) and Mixed Reality (MR) technologies into everyday environments offers meaningful potential for enhancing safety and accessibility. One critical application is the detection of trip hazards that pose a substantial risk in dynamic and complex spaces. This paper presents a technical feasibility study for real-time trip hazard recognition using Apple Vision Pro as a MR platform. Using its advanced spatial computing capabilities, we implemented a process that uses the depth detection and semantic segmentation features of the MR platform to identify irregularities such as bumps and small obstacles on the floor. Our findings indicate that MR technology-based hazard detection is feasible for detecting small obstacles exceeding 1.5 cm in height. Consequently, the continued advancement of AR/MR technology presents significant potential for fall prevention by effectively identifying trip hazards.

Using Mixed Reality (MR) to plan the motion of robots, for example the path of a drone, is an effective way to optimize their movements in a collaborative working environments. Path planning using MR allows for the incorporation of motion constraints arising from real-world obstacles, virtual objects, and time scheduling considerations.

Robots are increasingly finding their way into home environments, where they can assist with household tasks like vacuuming or surveilling. While the robots can navigate on their own, users might not want them to go everywhere or not in a specific way. For example, users might not want a drone to fly over a table where important letters and the newspaper are stored, even though it is the shortest path to the goal. Therefore, an application is required, that is easy to learn and to apply even for inexperienced users.In this paper, we present a framework that uses a tablet as augmented reality (AR) device to modify a robot’s movement space in 3D. A user can define virtual borders in the real world with the tablet and add them to a map, changing the navigational behavior of the robot. The framework is evaluated by a user study with inexperienced participants that verifies our approach. Further analyses show, that even complex scenarios can be covered with our framework.

In this work, three different algorithms are examined for the evacuation of smart buildings, in particular a Static Evacuation Algorithm (SEA), a Dynamic Evacuation Algorithm (DEA) and a Fire Dynamic Algorithm (FDA). The Static Evacuation Algorithm represents the shortest path out of the building to a safe area. The Dynamic Algorithm calculates the optimal escape route based on the current position and position of a detected fire hazard. The Fire Dynamic escape route predicts how fast the fire will spread and includes this into the calculation of the escape route. The used simulation environment is based on the cross-platform game engine Unity3D and a building model was created using the building plan of the main building at the Campus Minden of University of Applied Sciences Bielefeld, Germany. We found that our proposed FDA performed 31.64% better than the SEA and 23.8% better than the DEA in terms of the hazard area over a minimally longer distance.

In computer science, semantic interoperability is widely researched and used for multiple applications in the Internet of Things. Since semantics involve a lot of data and require high processing capabilities, computers with high performance are used as intermediate devices to represent devices which are constrained in energy, memory, and processing capacity. Knowledge on the explicit setup of a constrained device, e.g. which sensors and actuators are attached for what purpose, is required to register the constrained device at such an intermediate device. Commonly, this knowledge is defined in a file with a semantic description which refers to a specific constrained device, but can not be shipped with the device itself due to the prior described constraints. We propose to optimize semantic descriptions to meet the requirements of constrained devices in terms of energy, memory, and processing capabilities. Our results show that semantic descriptions can be shipped with a constrained device. This enables self-registration and self-configuration when constrained devices are added or removed from networks.

Abstract In a large number of scientific areas, such as immunology, forensics, paleoecology, and archeology, the study of pollen, i.e., palynology, plays an important role: from tracking climate changes, studying allergies, to forensic investigations or honey origin analysis. Since the mid-nineties of the last century, the idea for an automated solution to the problem of pollen identification and classification was formulated and since then, several attempts and proposals have been made and presented, based on different technologies, in particular in the field of Computer Vision. However, as of 2021 microscopic analyses are performed mainly manually by highly trained specialists, although the capabilities of artificial intelligence, especially Deep Neural Networks, are steadily increasing. In this work, we analyzed various state-of-the-art research work concerning pollen detection and classification and compared their methods and results. The problems, such as data accessibility, different methods of Machine Learning, and the intended applicability of the proposed solutions are explored. We also identified crucial issues that require further work and research. Our work will provide a thorough view on the current state of the art, its issues, and possibilities for the future.

Palynology, the study of pollen, is becoming the focus of attention in computer vision in recent years. Various proposed automated classification and segmentation methods have been evaluated on a number of data sets. However, as of 2021 most data sets are sparse; they either contain only a small number of pollen classes, images in total or are imbalanced overall. In this work, we explore the possibility of creating synthetic pollen grain images from less than 2,000 images per pollen class via a Generative Adversarial Network (GAN). For that purpose, we selected two distinct pollen classes from a state of the art pollen data set and evaluated the data set with and without synthetic data on a Convolutional Neural Network (CNN). The enriched data set performed better overall (+1.4%) and specifically for the two pollen classes (+2%). We also drastically reduced the no. of real images and were still able to achieve a score of 60% to 80%. The experiments show, that our synthesized pollen images are visually close to real-life pollen grains and can be used to enrich imbalanced data sets as an addition to traditional data augmentation methods.

Modern building automation systems enable the comprehensive observation and control of intelligent environments. The Internet of Things has reshaped building automation from closed and physically intertwined to more open and accessible systems. The accompanying emergence of multimedia in buildings such as audio and video through stream-based sensors opens up possibilities for profound and feature-rich applications. However, at present, building automation does not benefit from the inherent information contained in streamed multimedia. The goal of this work is to exploit informational value contained in multimedia in order to integrate stream-based sensors into existing building automation systems. This is accomplished with the introduction of an intermediate layer where complex information, such as multimedia, from different origins is untangled into pieces of information. Then, each piece of information is represented with an unambiguous description and can be consumed by microservices. Finally, a fog-based platform, which orchestrates multiple microservices, is used to provide services for users and automation tasks. To proof the usability of our proposed concept, an evaluation is conducted in a residential environment considering multiple use case scenarios involving streamed video and point cloud data. Solid results show that the system fulfills prior defined security, latency, privacy, extensibility, and scalability requirements.

Detecting people in video streams involves privacy concerns since people are often unaware of being recorded. Furthermore, cameras and commodity depth sensors are prone to variances of illumination. This work proposes light detection and ranging as sensing technology for a robust and privacy preserving people detection. A people counter is implemented, which captures a coarse human shape by concatenating multiple two dimensional range scans while people pass the sensor's view. The system works under dynamic illuminated conditions and is running as an embedded node on a single core 700 MHz processing unit. An evaluation with 20 different people crossing an entrance 100 times in each direction was carried out under controlled conditions. Results show a high precision and recall of 100 percent and 99 percent respectively for counting people and determining the walking direction. Privacy is preserved since objects are represented by distance values with a low resolution of 16x16 data points.

The handover of objects is a fundamental task in robot-human interaction. The nature of the handover can be varied by adjusting various parameters in order to achieve a human-like interaction. In this paper the final handover configuration is examined and analyzed. For this purpose, we conducted an experiment in which subjects taught a robot proper and improper end poses for different objects in different situations. We analyze the learned poses to determine what factors proper poses depend on. In addition to the poses, the configurations of the individual joints of the commercial robot are evaluated. Our results demonstrate that proper poses can be grouped into three clusters. These differ mainly in the rotation of the forearm. The height of a proper handover is also determined based on external factors.

Virtual borders are an opportunity to allow users the interactive restriction of their mobile robots' workspaces, e.g. to avoid navigation errors or to exclude certain areas from working. Currently, works in this field have focused on human-robot interaction (HRI) methods to restrict the workspace. However, recent trends towards smart environments and the tremendous progress in semantic scene understanding give new opportunities to enhance the HRI-based methods. Therefore, we propose a novel learning and support system (LSS) to support users during teaching of virtual borders. Our LSS learns from user interactions employing methods from visual scene understanding and supports users through recommendations for interactions. The bidirectional interaction between the user and system is realized using augmented reality. A validation of the approach shows that the LSS robustly recognizes a limited set of typical areas for virtual borders based on previous user interactions (F1 - Score= 91.5%) while preserving the high accuracy of standard HRI-based methods with a median of Mdn= 84.6%. Moreover, this approach allows the reduction of the interaction time to a constant mean value of M = 2 seconds making it independent of the border length. This avoids a linear interaction time of standard HRI-based methods.

Drawing attention to objects and their localization in the environment are essential building blocks for domestic robot applications, e.g. fetch-and-delivery or navigation tasks. For this purpose, human pointing gestures turned out to be a natural and intuitive interaction method to transfer the spatial data of an object from human to robot. Current approaches only use the robot's on-board sensors to perceive gesture-based instructions, which restricts them to the field of view of the robot's camera. The integration of mobile robots into intelligent environments, such as smart homes, opens new possibilities to overcome this limitation by utilizing components of the surrounding environment as additional sensors. We take advantage of these new possibilities and propose a multi-stage object localization system based on human pointing gestures that considers the whole intelligent environment as interaction partner. Our experimental results show that our multi-stage approach successfully refines the position initially proposed by a human pointing gesture by employing a distributed camera network integrated into the environment for object localization.

In times of more and more complex interaction techniques, we point out the powerfulness of colored light as a simple and cheap feedback mechanism. Since it is visible over a distance and does not interfere with other modalities, it is especially interesting for mobile robots. In an online survey, we asked 56 participants to choose the most appropriate colors for scenarios that were presented in the form of videos. In these scenarios a mobile robot accomplished tasks, in some with success, in others it failed because the task is not feasible, in others it stopped because it waited for help. We analyze in what way the color preferences differ between these three categories. The results show a connection between colors and meanings and that it depends on the participants' technical affinity, experience with robots and gender how clear the color preference is for a certain category. Finally, we found out that the participants' favorite color is not related to color preferences.

We address the problem of interactively controlling the workspace of a mobile robot to ensure a human-aware navigation. This is especially of relevance for non-expert users living in human-robot shared spaces, e.g. home environments, since they want to keep the control of their mobile robots, such as vacuum cleaning or companion robots. Therefore, we introduce virtual borders that are respected by a robot while performing its tasks. For this purpose, we employ a RGB-D Google Tango tablet as human-robot interface in combination with an augmented reality application to flexibly define virtual borders. We evaluated our system with 15 non-expert users concerning accuracy, teaching time and correctness and compared the results with other baseline methods based on visual markers and a laser pointer. The experimental results show that our method features an equally high accuracy while reducing the teaching time significantly compared to the baseline methods. This holds for different border lengths, shapes and variations in the teaching process. Finally, we demonstrated the correctness of the approach, i.e. the mobile robot changes its navigational behavior according to the user-defined virtual borders.

Due to the high complexity, the reproducibility of experiments is one of the fundamental problems of robotics research. Although the problem has been identified by the community, the approaches to solve it are limited. The toolkit presented in this paper attempts to provide an approach to address the problem of reproducibility and the associated exchange of experimental artifacts. In order to simplify this process, the toolkit supports both the initial definition of experiments and the subsequent reproduction through a new type of workflow based on dynamic hardware-dependent assignment of services to devices and their automatic initialization. The evaluation of the system shows a high performance compared to a native solution as well as a substantial time saving compared to the manual set-up of an experiment.

As robots are increasingly acting in real-world environments, learning and recognition of objects is a problem. Existing methods for learning visual object models use offline techniques to generate high-quality models or online techniques to dynamically expand the object model library. We present an online learning method that creates visual object models through active object exploration. Our approach enables a robot to use manipulations of an object to learn autonomously visual features from several points of view. The ability to segment background, robot parts and the object in the visual space allows to filter irrelevant feature points. This improves the quality of the object model while decreasing its size. Finally, a human-robot interaction enables a human collaborator to improve the object model. The method is evaluated on a Pepper robot, showing the improvement in performance and accuracy with respect to interactive learning.

The increasing number of robots in home environments leads to an emerging coexistence between humans and robots. Robots undertake common tasks and support the residents in their everyday life. People appreciate the presence of robots in their environment as long as they keep the control over them. One important aspect is the control of a robot's workspace. Therefore, we introduce virtual borders to precisely and flexibly define the workspace of mobile robots. First, we propose a novel framework that allows a person to interactively restrict a mobile robot's workspace. To show the validity of this framework, a concrete implementation based on visual markers is implemented. Afterwards, the mobile robot is capable of performing its tasks while respecting the new virtual borders. The approach is accurate, flexible and less time consuming than explicit robot programming. Hence, even non-experts are able to teach virtual borders to their robots which is especially interesting in domains like vacuuming or service robots in home environments.

This paper proposes a novel activity recognition system that is integrated into a smart home environment. It is characterized by low costs, high energy efficiency and low intrusiveness to increase the acceptance of the users. Activity recognition is performed locally on a single small-sized wearable device incorporating a microprocessor and a tri-axial accelerometer. After investigating on different feature sets and classification algorithms, the final implementation only considers five time domain features and a C4.5 decision tree classifier resulting in an immediate response. This wearable device is successfully integrated into an intelligent environment by Bluetooth Low Energy wireless communication protocol and openHAB as platform-independent integration software. The integration of the system into the home environment allows reactions of the home depending on the activity which enriches the life quality of the residents. Additionally, the system covers fall detection that enables the home to provide a fallen person with urgent support.

SUMMARY: The prediction of significant short functional protein sequences has inherent problems. In predicting phosphorylation sites, problems came from the shortness of phosphorylation sites, the difficulties in maintaining many different predefined models of binding sites, and the difficulties of obtaining highly sensitive predictions and of obtaining predictions with a constant sensitivity and specificity. The algorithm presented in this paper overcomes these problems. The proposed algorithm PHOSITE is based on the case-based sequence analysis. This enables the prediction of phosphorylation sites with constant specificity and sensitivity. Furthermore, this method leads not only to the prediction of phosphorylation sites in general but also predicts the most probable type of kinase involved. AVAILABILITY: The tool PHOSITE implementing the presented method can be evaluated under the website http://www.phosite.com.

In synthetic holography, the emphasis has been on numerical simulation methods for the recording and reconstruction processes, but methods to visualize the calculated results are yet underdeveloped. We show how volume slicing and volume rendering using a threshold for translucency can be applied to synthetic holography. Results of the three-dimensional visualization of the reconstruction are presented, thereby illustrating the usefulness of the threshold operator.

Visualization offers useful tools for understanding large data sets. The visualization techniques in this work, realized in the program GeoVis, depict static as well as dynamic geoscientific data through glyphs, surfaces and animation. The flight over a virtual landscape (where multidimensional data are represented by abstract glyphs) proves to be useful for the quick exploration of coherencies as well as differences in the data compound.