Smart homes, medical devices, and education systems, among other emerging cyber-physical systems, hold immense promise for sensing-based user interfaces, especially for using fingers and hand gestures as system input. However, vision approaches compatible with time-consuming image processing adopt low 60 Hz location sampling rate (frame rate) for real-time hand gesture recognition. Furthermore, they are not suitable for low-light environment and long detection range. In this paper, we propose RoFin, which first exploits 6 temporal-spatial 2D rolling fingertips for real-time 3D reconstructing of 20-joint hand pose. RoFin designs active optical labeling for finger identification and enhances inside-frame 3D location tracking via high rolling shutter rate (5-8 kHz). These features enable great potentials for enhanced multi-user HCI, virtual writing for Parkinson suffers, etc. We implement RoFin prototypes with wearable gloves attached with low-power single-colored LED nodes and commercial cameras. The experiment results show that (1) In flexible sensing distances up to 2.5 m, RoFin achieves an average labeling parsing accuracy of 85%, (2) In comparison to vision-based techniques, RoFin improves the tracking grain with 4× more sampled points each frame, (3) RoFin reconstructs a hand pose in real time with 16 mm mean deviation error compared with Leap Motion under flexible distance, and (4) we further investigated real-world applications of RoFin, such as air writing with smoothed trajectories and mobile-based letter/number recognition in our developedXameraapp.

The deployment of satellite constellations for internet access has garnered significant interest in recent years. Current research trends focus on optimizing parameters of uniformly distributed symmetric constellations such as Starlink and Kuiper to better suit evaluation metrics like land coverage or ground station revisit time. These studies largely do not take into account the real distribution of people around the world, nor do they explore the field of Asymmetric Constellations. This work presents a novel optimization algorithm that uniquely designs asymmetric satellite constellations to focus on maximizing population coverage. Through our system evaluation with a fitness function and extensive simulations, our optimized constellations reduce redundancy and enhance efficiency. Results demonstrate that our asymmetrically designed constellations outperform standard uniform constellations in several use cases representing the United States, Europe, and a global scale, highlighting the potential for high-impact satellite deployment.

Self-driving platooning trucks are becoming increasingly common due to the economic gains for companies that utilize them. However, high throughput, long range communication between truck platoons can be very difficult if they are not in areas with pre-built cellular infrastructures. To combat this problem, we propose PlaCoB (Platoon Collaborative Beamforming), which enables multiple platooning trucks to collaboratively beamform to maintain communication over long distances. We conduct extensive simulations under various settings to evaluate PlaCoB’s performance compared to single-truck methods. By leveraging multi-truck platoons, collaborative beamforming, and frequency shifting, (1) PlaCoB can transmit 1.23x more data compared to single vehicle transmission methods, and (2) PlaCoB can transmit data 2.44x further than single vehicle transmission methods. These results demonstrate the effectiveness of our proposed PlaCoB’s approach.

With the proliferation of Internet-of-Things and the advancements in deep learning and Artificial Intelligence, there is an increased need for task-specific mobile devices that can efficiently utilize the wireless spectrum while operating in a wide variety of environments and channel conditions. This work proposes a novel end-to-end Visual-Inertial Odometry architecture that utilizes Semantic Communication-enhanced modulation constellation design for pose estimation of mobile devices. Our proposed model occupies less wireless spectrum bandwidth and does not require complex forward error-correction mechanisms. Yet, it is able to infer the pose information of a remote mobile device with comparable accuracy to a standard wireless Visual-Inertial Odometry system. We propose utilizing a Vision Transformer-based Image-IMU encoder for a network-constrained remote mobile device that transmits wireless encoded data to an Edge receiver. The receiver automatically detects the modulation scheme and decodes the received data for pose estimation. We propose a novel modulation constellation coding design scheme that can transmit data in multiple Modulation and Coding Schemes (MCS) within the same burst, without the need to embed an MCS symbol within the burst. Our proposed receiver can automatically detect the MCS as well. We illustrate how our proposed architecture can transmit encoded Image-IMU data over a wide range of distances and Signal-to-Noise Ratio (SNR) channel conditions and have a decoded pose accuracy comparable to a standard wireless Visual-Inertial Odometry system, with significantly less wireless bandwidth consumption and computational complexity.

Optical Camera Communication (OCC) appears as a promising technology to provide secure and pervasive wireless services with users' daily smart devices. Rolling shutter based modulations can improve the frequency response of the camera. This paper introduces a 2D Rolling Block (2DRB) based OCC modulation to use un-exploited spatial diversity to improve OCC's data rate for real-world applications. 2DRB outperforms traditional 1D strip based modulations. Using our 2DRB prototype with commercial devices, we show a significant data rate enhancement. We also discuss one promising real-world use case: indoor office integrated lighting and communication.

Smart homes, medical devices, and education systems, among other emerging cyber-physical systems, hold immense promise for sensing-based user interfaces, especially for using fingers and hand gestures as system input. However, vision approaches compatible with time-consuming image processing adopt low 60 Hz location sampling rate (frame rate) for real-time hand gesture recognition. Furthermore, they are not suitable for low-light environment and long detection range. In this paper, we propose RoFin, which first exploits 6 temporal-spatial 2D rolling fingertips for real-time 3D reconstructing of 20-joint hand pose. RoFin designs active optical labeling for finger identification and enhances inside-frame 3D location tracking via high rolling shutter rate (5--8 KHz). These features enable great potentials for enhanced multi-user HCI, virtual writing for Parkinson suffers, etc. We implement RoFin prototypes with wearable gloves attached with low-power single-colored LED nodes and commercial cameras. The experiment results show that (1) In flexible sensing distances up to 2.5 m, RoFin achieves an average labeling parsing accuracy of 85%. (2) In comparison to vision-based techniques, RoFin improves the tracking grain with 4× more sampled points each frame. (3) RoFin reconstructs a hand pose in real time with 16 mm mean deviation error compared with Leap Motion under flexible distance.

Drones are gaining more interest thanks to their advantages and great potential for applications. However, present swarming drones’ stand-alone centralized radio frequency control mode from a base station has non-trivial drawbacks such as severe interference, latency caused localization error, etc. Differently, optical camera communication (OCC) is promising as an alternative for integrated communication and sensing for swarming drones. We propose PoseFly, the first 4-in-1 OCC approach for swarming drones. With exploited rolling shutter effect and the already installed camera and LED nodes, PoseFly provides (1) massive drone indication and identification, (2) multilevel on-site localization, (3) quick-link channel among drones, and (4) basic lighting. The design methodology of PoseFly gives a valuable example for the low-cost integrated sensing and communication for swarming drones.

Drones are gaining more interest from the industry and the research community as a result of their many advantages, including low cost, small size, adaptability, and ease of use, as well as their potential applications. However, current control of swarming drones relies on stand-alone modes and centralized radio frequency control from a base station on the ground which is devoid of drone-to-drone communication. This method has drawbacks, including a crowded RF spectrum with mutual interference, high latency, and a lack of on-site drone-to-drone interactions. Because of its high spatial multiplexing capability, Line of Sight (LoS) security capabilities, broader bandwidth, and intuitive vision manner, Optical Camera Communication (OCC) is considered to be a potential alternative for sensing and communication in drone clusters. In this paper, we first utilize the rolling shutter effect in drone sensing and communication and propose PoseFly, a 4-in-1 AI-assisted OCC with drone identification, on-site localization, quick-link communication and lighting. We implement PoseFly prototypes on commercial drones, cameras and LEDs. Our experiments show our PoseFly achieves nearly 100% accuracy for distance estimation (20m), drone identification (12m), angle and speed estimation (4m) and 5 Kbps average quick-link throughput at up to 4 m on current prototypes.

Sensing-based user interfaces hold enormous potential for smart homes, medical equipment, educational systems, AR/VR/MR, etc. However existing hand and body gesture recognition systems are mostly based on frame-level computer vision approaches, which have limitations such as the inability to operate in the environment with low brightness, short detection distance, without the objective identification ability, and coarse-grained tracking when the objectives are in high-speed motion. Therefore, in this paper, we propose to attach active LED elements on objectives and utilize rolling shutter effect to enhance the gesture recognition and achieve the fine-grained motion tracking with objective identification.