As a main approach towards touch-free human-computer interaction,hand gesture recognition(HGR) has long been a research focus for both academia and industry. Meanwhile,visible light communication(VLC) has become increasingly popular with VLC-ready commercial products (e.g., Philips lamps) available on the market. These facts provoke us to ask: can we leverage a VLC-ready lamp to realizeintegrated sensing and communication(ISAC) by conducting both HGR and VLC simultaneously? To this end, we propose ReflexGest as our answer to this question. ReflexGest is implemented upon a table lamp for the sake of practicality; this VLC-ready lamp is equipped with a ring-shaped light-emitting diode (LED) array and a photodiode (PD, for light intensity sensing) originally aiming for up/down-link VLCs. Demanding hand gestures to be performed between the lamp and a table surface, ReflexGest exploits the variation of the reflection and their unique correlation with the corresponding hand gestures to achieve HGR. In particular, ReflexGest first handles the limited sensing ability of the PD by enhancing the LED lamp and thus diversifying the light emission patterns. Moreover, ReflexGest combats the reflection interference from varying table surfaces via an adversarial learning technique to distill only the features relevant to hand gestures. Our extensive evaluations demonstrate that ReflexGest is able to deliver accurate HGR under realistic VLC traffic.

Human-machine interaction (HMI) is much important in factories, and most HMI ways are contact which arise safety and health issues. To avoid such problems, contactless HMI ways such as in-air hand gesture recognition (HGR) via Wi-Fi or radar are widely studied by both academia and industry. However, these RF-based methods are not very appropriate for the industry because of the electromagnetic interference. As for visible light sensing, it is free of electromagnetic radiation and can reuse the existing devices, e.g., lamps on machines, hence utilizing visible light to realize HGR is a good solution for HMI in factories. The current visible-light-enabled HGR (VL-HGR) methods using deep learning algorithms are all supervised, which increases the cost of manual labeling and further hinders the industrial applications of VL-HGR. To this end, we propose SemiGest, a semi-supervised learning (SSL) method for VL-HGR, to facilitate the applications of VL-HGR in industry. The system prototype is built on a table lamp to mimic the lamp on a machine emitting lights at four distinct carrier frequencies, and the lights reflected by hands are collected by a receiver. SemiGest utilizes the variation and correlation of the lights to realize HGR with an SSL algorithm using only a small amount of labeled data and lots of unlabeled data. Furthermore, the SSL algorithm is designed not only for the visible light data but also can be generalized to other time-series data in the industry. To confirm the effectiveness and robustness of SemiGest, we perform various experiments to show the potential for practical implementation in the industry.