Perceptually accurate colour reproduction is a core requirement of video mapping applications, where objective evaluation of colour rendering chain taking into account human perception becomes greatly beneficial. In this article, we present a workflow for colorimetry evaluation of video mapping software rendering chain, that we implemented in open-source video mapping software Splash, with a set a common metrics for image quality assessment and tools for colour reproduction evaluation. We introduce an accompanying graphical visualization template to help accurate interpretation of the metrics used. We describe different use case examples that we performed with our tool, proving the workflow efficient for simple, understandable and reproducible colorimetry evaluation.

We present LivePose: an open-source (GPL license) tool that democratizes pose detection for multimedia arts and telepresence applications, optimized for and distributed on open edge devices. We designed the architecture of LivePose with a 5-stage pipeline (frame capture, pose estimation, dimension mapping, filtering, output) sharing streams of data flow, distributable on networked nodes. We distribute LivePose and dependencies packages and filesystem images optimized for edge devices (NVIDIA Jetson). We showcase multimedia arts and telepresence applications enabled by LivePose.

Complex data visualization design projects often entail collaboration between people with different visualization-related skills. For example, many teams include both designers who create new visualization designs and developers who implement the resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions, communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more successful and collaborative data visualization design.

Information visualisation is the transformation of abstract data into visual, interactive representations. In this paper we present InfoPhys, a device that enables the direct, tangible manipulation of visualisations. InfoPhys makes use of a force-feedback pointing device to simulate haptic feedback while the user explores visualisations projected on top of the device. We present a use case illustrating the trends in ten years of TEI proceedings and how InfoPhys allows users to feel and manipulate these trends. The technical and software aspects of our prototype are presented, and promising improvements and future work opened by InfoPhys are then discussed.

The recent decade has seen a rapid evolution in the field of digital media. Mobile devices are now being integrated into every aspect of urban life. GPS, sensor technologies and augmented reality have transformed the new generation of mobile devices from a communication and information platform into a navigational tool, fostering new ways of perceiving reality and image building. Touch sensor technology has changed the screen into a joint input and display device. In this paper we present the FlanoGraph , an application for smartphones and tablets designed to take benefit of the changes induced by mobile devices. We first briefly outline the conceptual background, evoking the work of some researchers in the fields of ‘Non Representational Theory’, mobile media, and computational data processing. We then present and describe the FlanoGraph through a set of use cases. Finally, we conclude discussing some techniques necessary for the development of the application.

This paper presents the design process of a desk-set tangible user interface for the navigation and manipulation of media content organized by content-based similarity with off-the-shelf/flea market devices. For intra-media navigation, a refurbished portable vinyl player has its inside mechanics replaced by a webcam monitoring circular gray code analyzed through computer vision for position/speed tracking. For inter-media navigation, a 3D force-feedback controller is mounted in upright position on a truss with cell clamps, repurposed as trackpad. For media recomposition, motorized faders recall the effect presets of the closest/last selected media item.

Recently, various dimensionality reduction approaches have been proposed as alternatives to PCA or LDA. These improved approaches do not rely on a linearity assumption, and are hence capable of discovering more complex embeddings within different regions of the data sets. Despite their success on artificial datasets, it is not straightforward to predict which technique is the most appropriate for a given real dataset. In this paper, we empirically evaluate recent techniques on two real audio use cases: musical instrument loops used in music production and sound effects used in sound editing. ISOMAP and t-SNE are being compared to PCA in a visualization problem, where we end up with a two-dimensional view. Various evaluation measures are used: classification performance, as well as trustworthiness/continuity assessing the preservation of neighborhoods. Although PCA and ISOMAP can yield good continuity performance even locally (samples in the original space remain close-by in the low-dimensional one), they fail to preserve the structure of the data well enough to ensure that distinct subgroups remain separate in the visualization. We show that t-SNE presents the best performance, and can even be beneficial as a pre-processing stage for improving classification when the amount of labeled data is low.

This paper focuses on one aspect of doctoral studies, within the last year of completion, consisting in designing applications for the navigation (by content-based similarity) in audio or video collections: the choice of tangible or free-form interfaces depending on use cases. One goal of this work is to determine which type of gestural interface suits best each chosen use case making use of navigation into media collections composed of audio or video elements, among: classifying sounds for electroacoustic music composition, derushing video, improvising instant music through an installation organizing and synchronizing audio loops. Prototype applications have been developed using the modular Media-Cycle framework for organization of media content by similarity. We conclude preliminarily that tangible interfaces are better-suited for focused expert tasks and free-form interfaces for multiple-user exploratory tasks, while a combination of both can create emergent practices.

This paper presents AudioCycle, a prototype application for browsing through music loop libraries. AudioCycle provides the user with a graphical view where the audio extracts are visualized and organized according to their similarity in terms of musical properties, such as timbre, harmony, and rhythm. The user is able to navigate in this visual representation and listen to individual audio extracts, as well as query the database by providing audio examples. AudioCycle draws from a range of technologies, including audio analysis from music information retrieval research, 3D visualization, spatial auditory rendering, audio time-scaling and pitch modification. The proposed approach extends on previously described music and audio browsers. Possible extension to multimedia libraries is also suggested.