We have seen rapid deployment of augmented reality devices. However, these devices currently suffer from limited battery life. They require frequent recharge, especially when capturing and streaming information wirelessly. In particular, these devices stream information over the radio spectrum using transceivers that are power-hungry, and wireless communication dominates the energy budget of these devices. We present our ongoing work to design a system called Li-FiAR, which proposes integrating Li-Fi with augmented reality devices. Specifically, we implement a transmitter and receiver and integrate them with an augmented reality device. We argue that Li-Fi receivers may consume less power than their radio counterparts while benefiting from the spatial nature of light propagation. This enables novel application scenarios for augmented reality. We demonstrate the transmission of images and audio through Li-Fi, presented on an augmented reality device in this work.

Wireless embedded systems assist us in collecting data from the physical world, through sensor data analysis, such systems allow us to understand our environment. However, deploying wireless embedded systems and analyzing the collected data remains significantly challenging. This is due to the steep learning curve required to implement custom machine-learning models and other algorithms for data analysis. Furthermore, it is also challenging to program individual embedded devices. The diversity of the available platforms and their capabilities further compounds this problem. In response, we introduce an end-to-end system, called the Otter. It facilitates simple sensor data collection using commodity-embedded platforms. Moreover, it employs a large language model to design a natural language interface for the analysis and extraction of useful information from the sensor data. We present our preliminary work on prototyping this system, applying it to a specific use case of hand gesture detection. Otter represents one of the first systems to leverage the enhanced capabilities of large language models for simplifying wireless embedded system deployments.

An important step in the deployment of wireless embedded systems is the analysis of the sensor data. Traditionally, this requires machine learning models tailored to the application use case. However, this step requires significant expertise from the end user and can be less adaptable to the dynamics of real-world deployments. In recent years, large language models have seen significant developments. These models have been shown to be capable of performing general-purpose tasks. In this work, we explore the hypothesis that large language models can be used to aid in sensor data analysis. Our preliminary findings through real-world experiments show significant promise for two tasks: inferring hand gestures through tracking of light and vibration sensor data. We believe these findings highlight the potential of large language models in sensor data analysis, and thus, it warrants further study.

The rapid growth in wireless embedded systems is threatened by the challenges associated with programming and deploying them. In addition, there is also the complexity inherent in analyzing of the sensor data. Notably, these tasks require high levels of end-user expertise. In this way, an entry barrier is introduced to deploying wireless embedded systems. In this work, we introduce Otter, an end-to-end system designed to simplify these tasks by leveraging the emergent properties of large language models. We demonstrate that Otter allows commodity embedded platforms to capture sensor data, such as light and vibration sensors, which can then be used to identify hand gestures in a near real-time manner. This is all while being prompted using natural language prompts by the end-user. Otter is the first system of its kind and has the potential to facilitate wireless embedded systems proliferation significantly.