This paper presents and discusses the integration of human pose estimation techniques into an existing web-based multimodal video annotation tool, applying it to the sports context, where basketball is the first case study. The relevance of video analysis extends across many fields of work (e.g., professional sports, education). In sports, systematic, detailed analysis using videos of players and teams is vital to evaluate many aspects of both training and competition. MotionNotes annotation tool now combines human pose and motion information with existing traditional annotation mechanisms (e.g., text and drawings annotations), allowing users to add further details to their annotation work. The paper reports feedback from a pilot study based on a participatory workshop involving people with relevant competitive experience in basketball. Based on this use case feedback, we conclude with an outlook of future iterations for our video annotation tool.

This paper presents and discusses the introduction of 3D functionalities for an existing web-based multimodal video annotation tool. Over the past years, we have developed a multimodal web video annotation tool that now combines 3D models and 360º content with more traditional annotation types (e.g., text, drawings, images), offering users the possibility of adding extra information in their annotation work. We show how 3D models augment the annotation work and add advantages like viewing or exploring objects in detail and from different angles. The paper reports detailed feedback from a pilot study in form of a workshop with traditional dance experts to whom these new features were presented. We conclude with an outlook of future iterations of the video annotator based on the experts’ feedback.

Creativity and inspiration for problem-solving are critical skills in a group-based learning environment. Communication procedures have seen continuous adjustments over the years, with increased multimedia elements usage like videos to provide superior audience impact. Annotations are a valuable approach for remembering, reflecting, reasoning, and sharing thoughts on the learning process. However, it is hard to control playback flow and add potential notes during video presentations, such as in a classroom context. Teachers often need to move around the classroom to interact with the students, which leads to situations where they are physically far from the computer. Therefore, we developed a multimodal web video annotation tool that combines a voice interaction module with manual annotation capabilities for more intelligent natural interactions towards ubiquitous environments. We observed current video annotation practices and created a new set of principles to guide our research work. Natural language enables users to express their intended actions while interacting with the web video player for annotation purposes. We have developed a customized set of natural language expressions that map the user speech to specific software operations through studying and integrating new artificial intelligence techniques. Finally, the paper presents positive results gathered from a user study conducted to evaluate our solution.