In recent years, 3D scene graphs have become a critical tool in robotics and computer vision for enabling systems to understand both the geometric and semantic aspects of their surroundings. These data structures represent spatial and semantic relationships between objects in a three-dimensional environment, supporting tasks like navigation, object manipulation, and scene understanding. This paper presents a real-time pipeline for 3D scene graph generation that offers flexibility in image segmentation techniques while incorporating room classification that is based on a Random Forest model. Our work enables robots to dynamically update their understanding of complex and large-scale environments in real-time. We evaluate our approach systematically on a dataset and in a real-life experiment. The results demonstrate the capability of running our solution at over 10 Hz on an Nvidia Jetson AGX Orin SoC while also scaling favorably in larger environments. Our proposed room classification approach predicts classes with an average accuracy of 80%.

Abstract Wireless charging of electric vehicles can be achieved by installing a transmitter coil into the ground and a receiver coil at the underbody of a vehicle. In order to charge efficiently, accurate alignment of the charging components must be accomplished, which can be achieved with a camera-based positioning system. Due to an air gap between both charging components, foreign objects can interfere with the charging process and pose potential hazards to the environment. Various foreign object detection systems have been developed with the motivation to increase the safety of wireless charging. In this paper, we propose a foreign object detection technique that utilizes the integrated camera of an embedded positioning system. Due to operation in an outdoor environment, we cannot determine the types of objects that may occur in advance. Accordingly, our approach achieves object-type independence by learning the features of the charging surface, to then classify anomalous regions as foreign objects. To examine the capability of detecting foreign objects, we evaluate our approach by conducting experiments with images depicting known and unknown object types. For the experiments, we use an image dataset recorded by a positioning camera of an operating wireless charging station in an outdoor environment, which we published alongside our research. As a benchmark system, we employ YOLOv8 (Jocher et al. in Ultralytics YOLO, 2023), a state-of-the-art neural network that has been used in various contexts for foreign object detection. While we acknowledge the performance of YOLOv8 for known object types, our approach achieves up to 18% higher precision and 46% higher detection success for unknown objects.

Abstract Pose estimation is an important component of many real-world computer vision systems. Most existing pose estimation algorithms need a large number of point correspondences to accurately determine the pose of an object. Since the number of point correspondences depends on the object’s appearance, lighting and other external conditions, detecting many points may not be feasible. In many real-world applications, the movement of objects is limited, e.g. due to gravity. Hence, detecting objects with only three degrees of freedom is usually sufficient. This allows us to improve the accuracy of pose estimation by changing the underlying equations of the perspective-n-point problem to three variables instead of six. By using the simplified equations, our algorithm is more robust against detection errors with limited point correspondences. In this article, we study three scenarios where such constraints apply. The first one is about parking a vehicle on a specific spot. Here, a stationary camera is detecting the vehicle to assist the driver. The second scenario describes the perspective of a moving camera detecting objects in its environment. This scenario is common for driver assistance systems, autonomous cars or mobile robots. Third, we describe a camera observing objects from a birds-eye view, which occurs in industrial applications. In all three scenarios, observed objects can only move in the ground plane and rotate around the vertical axis. Hence, three degrees of freedom are sufficient to estimate the pose. Experiments with synthetic data and real-world photographs have shown that our algorithm outperforms state-of-the-art pose estimation algorithms. Depending on the scenario, our algorithm is able to achieve 50% better accuracy, while being equally fast.

A truly ubiquitous, planet-wide Internet of Things requires ultra-low-power, long-lasting sensor nodes at its core so that it can be practically utilized in real-world scenarios without prohibitively high maintenance efforts. Recent advances in energy harvesting and low-power electronics have provided a solid foundation for the design of such sensor nodes. However, the issue of reliable two-way communication among such devices is still an active research undertaking due to the high energy footprint of traditional wireless transceivers. Although approaches such as radio duty cycling have proved beneficial for reducing the overall energy consumption of wireless sensor nodes, they come with trade-offs such as increased communication latency and complex protocols.

Queueing systems are used to estimate the attributes of waiting lines, for example the number of people in line or the waiting time. This information helps to reduce the time spent waiting by balancing people amongst multiple lines or aids in the decision making about opening a new line. However, current queueing systems are often mounted at fixed positions, require user participation, or need time consuming manual calibration after each layout change. In this work, we introduce CutiQueue, a flexible and portable queueing system that is battery powered, needs no manual calibration or user participation. CutiQueue is based on passive presence detection using the RSSI of low cost Bluetooth Low Energy transceivers. We analyze the performance of the system in experiments using a prototype implementation to estimate the number of people waiting as well as the length of the line. The experimental evaluation indicates an average accuracy of 97.96% when compared to counting people walking or standing in the waiting line using a laser barrier.

Location-based services have become an important market. At their core, location-based services generally rely on identifying the location of individuals or objects. In outdoor environments, determining the location is typically accomplished by means of GPS. In indoor environments, Bluetooth Low Energy (BLE) based localization has received a lot of attention. Most recently, an increasing number of mobile devices are capable of supporting ANT as an alternative low-power communication protocol. Unlike BLE, however, the suitability of ANT for localization has not been analyzed extensively. We report the initial results of an experimental study to compare ANT and BLE when used for localization. The results suggest that ANT-based localization can clearly outperform BLE, when using fingerprinting as the underlying localization principle.

The wide availability of GPS-enabled devices has fueled the development of location-based applications. Among the most popular are navigation applications that guide users along their way. Due to the limited availability and precision of GPS in typical indoor environments, most navigation applications are limited to outdoor scenarios. To mitigate this, researchers and practitioners have been developing alternative systems to capture the position of a mobile device indoors. Using these systems, they clearly show that it is possible to provide precise location information at a reasonable cost. However, past applications in this domain have often focused exclusively on indoor scenarios.In this paper, we describe an approach to integrate the indoor and outdoor world to provide a seamless hybrid navigation experience. The approach builds upon our past work on indoor localization and integrates with our existing infrastructure for outdoor navigation that has been developed in the GAMBAS European research project. To evaluate the approach, we present data gathered from a real-world deployment as part of the SIMON European research project.

Fingerprinting-based indoor localization systems rely on stable signal distribution characteristics of fixed signal sources for location estimation. However, indoor environments are not static and changes in the environment can lead to displacement of some signal sources, potentially causing a drop in the localization performance. It is therefore necessary to regularly monitor the signal sources and manually recalibrate any whose signal distribution has changed. The effort for calibrating these systems is typically high, especially for large indoor environments. This paper proposes an approach for autonomously detecting the displacement of a signal source using only measurements collected by active users of the system. The proposed approach is demonstrated to reliably detect displaced signal sources as well as multiple simultaneous displacements of up to half of the deployed signals sources. It is further shown that the same measurements can be used to autonomously recalibrate the (WLAN- or Bluetooth-based) indoor localization system, achieving localization performance comparable to manual calibration.

Radio signals can be used to detect the presence of a person (target) in an environment by analysing the fluctuations in the Received Signal Strength Indicator (RSSI).The velocity of the target can be estimated by examining the sequence of disturbances in consecutive radio links over a period of time.This requires knowledge of the deployment of the radio transceivers and the time when the target crosses the Line of Sight (LoS) of each radio link.However, it is not trivial to precisely estimate the exact time of the link crossing due to the broad range of RSSI fluctuations generated as the target approaches the link.In this paper, we evaluate and compare 15 techniques for estimating the velocity of the target and propose enhancements to some of the techniques.In our experiments the techniques perform with an average accuracy in the range between 13.02% and 96.18%, which corresponds to an average error of 0.05m/s for a moving target.

Fingerprinting-based indoor localization systems tend to achieve higher accuracy compared to other approaches such as signal propagation modeling. However, they also tend to have a higher effort/cost for deployment and maintenance. Changes in the configuration of the indoor space like moving of furniture, or defective signal sources can cause the signal characteristics in the environment to change significantly, and thereby render the fingerprint radio map (used for training the system) outdated. This leads to a drop in localization performance of the system over time. In this paper, we propose an approach to using the system infrastructure for periodically detecting changes in the signal characteristics and autonomously recalibrating the fingerprint radio map. We demonstrate that we can reliably detect changes in signal characteristics stemming from the dampening of a signal source (e.g. induced by moving of furniture) and recalibrate the localization system with an accuracy of 83% to 93% of the optimum localization performance achievable through manual system recalibration.

A key challenge for rapidly growing cities of today is to provide effective public transport services to satisfy the increasing demands for urban mobility. Toward this goal, the Internet of Things (IoT) has great potential to overcome existing deficiencies of public transport systems given its ability to embed smart technology into real-life urban contexts. In this paper, we show how this paradigm can be applied to the public transport domain and present the Urban Bus Navigator (UBN), an IoT enabled navigation system for urban bus riders. UBN provides two novel information services for bus users: 1) micro-navigation and 2) crowd-aware route recommendation. Micro-navigation refers to fine-grained contextual guidance of passengers along a bus journey by recognizing boarded bus vehicles and tracking the passenger's journey progress. Crowd-aware route recommendation collects and predicts crowd levels on bus journeys to suggest better and less crowded routes to bus riders. We present the technical system behind the UBN and report on results from an in-the-wild study in Madrid, Spain, that indicates removed barriers for public transport usage and a positive impact on how people feel about bus journeys.

Enabling the automated recognition and sharing of a user's context is a primary motivation for many pervasive computing applications. In the past, a significant amount of research has been focusing on the aspect of effective and efficient recognition. Yet, when context is shared with others, the resulting disclosure of personal information can have undesirable privacy implications. A common solution to this problem is the manual creation of an application-specific privacy policy that defines which information may be shared with whom. However, as the number of applications increases, such a manual approach becomes increasingly cumbersome and over time, it is likely to lead to incomplete or even inconsistent policies. In this paper, we discuss how a privacy policy can be derived automatically by analyzing the user's sharing behaviour when using online collaboration tools. Our approach retrieves shared content and the associated sharing settings, detects context types and automatically derives a privacy policy that reflects the user's past sharing behaviour. To validate our approach, we have implemented it as an extensible software library for the Android platform and we have developed plug-ins for two popular collaboration tools, namely Google Calendar and Facebook.

Intelligent environments rely on context information for providing services to their users. Among various existing platforms for context recognition, smart phones are one of the most widely used. However, despite numerous advantages, smart phones exhibit limited energy resources. To mitigate this there exist approaches for energy efficient context recognition for smart phones but these approaches are usually fine-tuned for specific types of context. As a result their applicability to other context types is limited. In this paper, we present an energy efficient context recognition system for smart phones which provides generalized applicability of four generic energy efficiency techniques which have been described in the literature and applied to location sensing using mobile phones. These four techniques as termed by authors include Suppression, Substitution, Adaptation and Piggybacking. Our system provides their generalized applicability by modelling context recognition applications using a state machine abstraction. Consequently the resulting applications are structured as combinations of states and transitions. To aid rapid prototyping our system is equipped with an off-line development tool which allows the creation and (code) generation of state machines using a graphical editor. We evaluate our system by creating a test application. Using a precise hardware set up, we perform energy measurements to demonstrate the energy savings of these different techniques.

Fingerprinting-based indoor localization involves building a signal strength radio map. This map is usually built manually by a person holding the mapping device, which results in orientation-dependent fingerprints due to signal attenuation by the human body. To offset this distortion, fingerprints are typically collected for multiple orientations, but this requires a high effort for large localization areas. In this paper, we propose an approach to reduce the mapping effort by modeling the WLAN signal attenuation caused by the human body. By applying the model to the captured signal to compensate for the attenuation, it is possible to generate an orientation-independent fingerprint. We demonstrate that our model is location and person independent and its output is comparable with manually created radio maps. By using the model, the WLAN scanning effort can be reduced by 75% to 87.5% (depending on the number of orientations).

Online collaboration tools such as Google+, Face-book or Dropbox have become an important and ubiquitous mediator of many human interactions. In the virtual world, they enable secure interaction by controlling access to shared resources. Yet relying on them to support synchronous direct interactions, such as face-to-face meetings, might be suboptimal as they require reliable online connectivity and even then often introduce delays. A much more efficient way of co-located resource sharing is the use of local communications, such as ad-hoc WiFi. Yet setting up the necessary encryption and authentication mechanisms is often cumbersome. In this paper, we present PIKE, a key exchange protocol that minimizes this configuration effort. PIKE piggybacks the exchange of keys on top of an existing service infrastructure. To support encryption or authentication without Internet connection, PIKE relies on triggers for upcoming personal interactions and exchanges keys before they take place. To evaluate PIKE, we present two example applications and we perform an experimental as well as an analytical analysis of its characteristics. The evaluation indicates that PIKE is broadly applicable, scales well enough to support larger events and provides a level of security that is (at least) comparable to the one provided by the underlying service.

One of the key characteristics of pervasive computing applications is their ability to adapt to their environment. One possible approach to simplify the development of such adaptive applications is to use middleware to automate adaptation decisions. However, in many application scenarios, adaptation decisions can have critical security implications. Thus, when designing middleware to support adaptation it is necessary to take security constraints into account. In previous work, we presented generic role assignment as a middleware abstraction to support adaptation. In contrast to other approaches, role assignment enables adaptation within a single and across multiple smart environments in a flexible, application-overarching manner. In this paper, we extend the concept to support security constraints. Furthermore, we describe the necessary modifications to our existing role assignment system. Finally, we evaluate our approach showing resulting benefits and limitations. In comparison with existing approaches, the proposed approach is more flexible and does not introduce a single point of trust.

Existing context recognition applications for personal mobile devices are usually fine-tuned to recognize the set of characteristics required to support a particular user task. Outside of a laboratory environment, however, users are often involved in multiple tasks at a time which requires the simultaneous execution of several applications. Yet, due to the energy constraints of most personal mobile devices this approach is inherently limited in scale. In this paper, we show how this problem can be avoided by applying a component-based approach to application development and execution. We present a component system that enables developers to build individual applications in isolation without compromising runtime efficiency. When applications are executed simultaneously, the component system applies a novel technique called configuration folding to automatically remove redundancies. Our experimental evaluation of configuration folding shows that it can save up to 48 percent of energy when applied to a music and a speech detection application, thus, amortizing energy costs after a few seconds of execution.

Indoor localization based on signal strength fingerprinting has received significant attention from the community. This method is attractive because it does not require complex hardware beyond a simple radio transmitter. However, its main limitation is the inaccuracy caused by the variability of the signal strength. When applied to the localization of people, the signal variability can be attributed to three main sources: environmental dynamics (movement of people or objects), movement of transceiver (changes in the position and/or orientation of the transceivers) and body effects (distortion of the wireless signal due to body absorption). Our work focuses on the impact of the last two sources and provides two important contributions. First, we present an analysis to quantify the effects of antenna disorientation and transmitter misplacement. For the RFID system used in our work, these effects can decrease the localization accuracy by up to 50%. Motivated by these results, we identify parts of the human body where tags are less affected by unintentional movements. Second, we describe how multiple transmitters can be used to overcome the absorption effects of the human body. Our results indicate that four transmitters provide a reasonable trade-off between accuracy and hardware cost. We validate our findings through an extensive set of measurements gathered in a home environment. Our tests indicate that by following the guidelines proposed in this paper, the localization accuracy can improve from around 20% up to 88%.

A major characteristic of pervasive computing applications is their ability to adapt themselves to changing execution environments and physical contexts. In this article, we analyze different kinds of adaptations and introduce a multidimensional classification for them. On this basis, we propose a novel approach for peer-to-peer-based pervasive computing that provides support for the identified classes and integrates them in a multilevel architecture. We give a comprehensive overview of this architecture and its current realization in the Peer-to-Peer Pervasive Computing (3PC) project, discussing what adaptation is realized on each level, how the levels interact with each other, and how the overall system benefits from the integrated treatment of adaptation.

Due to the fact that distant objects are often less relevant to an application than objects in the proximity, middleware systems for pervasive computing typically exploit locality to improve efficiency. To do this, they configure the environment by introducing logical boundaries that reduce the number of interacting devices. Yet, in cases where applications require the interaction with distant objects, the boundaries become an artificial barrier that must be overcome by supplemental mechanisms. In this paper, we show how this problem can be avoided by using role assignment as a generic mechanism for environment configuration. To do this, we first derive the requirements for configuring a pervasive computing environment. We discuss how these requirements can be met by means of role assignment. To evaluate the approach, we present a prototypical implementation which we use to quantify the resulting overheads. The results indicate that role assignment enables a more flexible definition of boundaries at a low cost.

Pervasive applications have been designed to support users in their everyday life. For this purpose, they are able to interact with their physical environment, their context. They are aware of their context and use this information for configuration decisions. Furthermore, they can actively modify the context to meet their user's needs. This leads to new challenges in multi-user environments as applications which are executed simultaneously share a common context and thus can directly impact each other. In this paper we present a framework to coordinate multiple pervasive applications explicitly considering their context-interactivity. We show how application coordination can be integrated in an existing component-based system exemplified by our system PCOM. We conduct measurements for the extended system and discuss the obtained results.

Energy efficiency in pervasive computing is crucial for devices operated by battery. To provide energy efficiency we created an energy efficient middleware, called SANDMAN. It selects energy-efficient protocol stacks dynamically and switches idle devices in an energy-efficient sleep mode. In this paper we present an overview of the challenges we met when realizing this approach, possible solutions and lessons learned.

The lifetime requirements on wireless sensor networks often require the redundant deployment of sensor nodes with appropriate management mechanisms based on node clustering. Yet, existing clustering approaches do not take the primary task of sensor networks into account: performing relevant measurements. They usually form 'arbitrary' clusters, e.g., using connectivity information, and thus, the resulting measurements are often of only limited use to the applications. This problem can be avoided by considering application-specific semantics. For indoor applications, the notion of a room provides a natural unit of clustering since walls are constructed deliberately to ensure locality. This paper shows that it is feasible to automatically create clusters that reflect boundaries between rooms by analyzing the measurements of inexpensive, broadly available sensors. The paper first analyzes the applicability of statistical clustering methods and based on this analysis, it proposes and evaluates a lightweight approach to determine clusters in real deployments.

Human behavior and housing resist every standardization effort. Many aspects such as different technical equipment, furniture, and usage patterns make our surroundings as individual as ourselves. Thus, the personalization of pervasive applications is a fundamental requirement. To enable the development of custom pervasive applications, we propose a software development process. This process is based on the successful process for modern desktop applications. There, developers create extensible applications and components. Customizers use the resulting artifacts to develop custom applications. Finally, users configure applications to their individual needs by adjusting predefined settings. To adopt this process for pervasive computing, we present a component system for developers, a graphical toolkit for customizers, and self-configuration algorithms to ease the deployment

Adding distributed capabilities to existing programs has come to the forefront of software evolution. As a standard Java distributed technology, applets offer the advantages of being easily deploy able over Web browsers and requiring little to no explicit distributed programming. Yet applets are inflexible: they download remote code and run it only on the client machine. We present appletizing: a semi-automatic approach to transforming a Java GUI application into a client-server application, in which the client runs as a Java applet that communicates with the server through RMI. To enable appletizing, we have expanded the capabilities of J-Orchestra, our automatic partitioning system that takes as input a Java application in bytecode format and transforms it into a distributed application, running across multiple standard JVMs. We discuss the motivation, benefits, and J-Orchestra support for appletizing, and validate our approach via a set of case studies and associated benchmarks.

Applications in the pervasive computing domain are challenged by the dynamism in which their execution environment changes, e.g. due to user mobility. As a result, applications have to adapt to changes regarding their required resources. In this paper we present PCOM, a component system for pervasive computing. PCOM offers application programmers a high-level programming abstraction which captures the dependencies between components using contracts. The resulting application architecture is a tree formed by components and their dependencies. PCOM supports automatic adaptation in cases where the execution environment changes to the better or to the worse. User supplied as well as system provided strategies take users out of the control loop while offering flexible adaptation control.