Sky Writer is a collaborative communication medium that augments the traditional display of a UAV pilot and allows other stakeholders to communicate their needs and intentions to the pilot. UAV pilots engaging in time-critical missions, such as urban disaster responses, often must allocate most of their cognitive capacity towards flight tasks, making communication and collaboration with other stakeholders difficult or dangerous. Sky Writer addresses the needs of stakeholders while requiring minimal cognitive effort from the UAV pilot. The application presents stakeholders with an interface that provides contextual flight information and a live video stream of the flight. Stakeholders are able to sketch directly on the video stream or use a spotlight indicator that is mirrored across all displays in the system, including the pilot's display. The application can be used in any modern web browser and works with traditional and touch devices. Concept experimentation performed at Disaster City with two pilots indicated that the spotlight feature was particularly useful while the UAV was in motion, and the sketching features were most useful while the UAV was stationary. The system will be tested with professional responders soon to determine its efficacy in a simulated response, and to inform the ongoing design process.

This paper presents a literature review on the use of depth for hand tracking and gesture recognition. The survey examines 37 papers describing depth-based gesture recognition systems in terms of (1) the hand localization and gesture classification methods developed and used, (2) the applications where gesture recognition has been tested, and (3) the effects of the low-cost Kinect and OpenNI software libraries on gesture recognition research. The survey is organized around a novel model of the hand gesture recognition process. In the reviewed literature, 13 methods were found for hand localization and 11 were found for gesture classification. 24 of the papers included real-world applications to test a gesture recognition system, but only 8 application categories were found (and three applications accounted for 18 of the papers). The papers that use the Kinect and the OpenNI libraries for hand tracking tend to focus more on applications than on localization and classification methods, and show that the OpenNI hand tracking method is good enough for the applications tested thus far. However, the limitations of the Kinect and other depth sensors for gesture recognition have yet to be tested in challenging applications and environments.