In 2012, the Ecole Polytechnique Fédérale de Lausanne (EPFL) and the University Ca'Foscari launched a program called the Venice Time Machine, whose goal was to develop a large-scale digitisation program to transform Venice's heritage into 'Big Data of the Past'. Millions of register pages and photographs have been scanned at the State Archive in Venice and at the Fondazione Giorgio Cini. These documents were analysed using the deep-learning artificial-intelligence methods developed at EPFL's Digital Humanities Laboratory in order to extract their textual and iconographic content and to make the data accessible via a search engine. The project has now expand to a European scale, including more than 500 institutions and 20 new cities jointly constructing a distributed digital information system mapping the social, cultural and geographical evolution of Europe. The project build upon existing platforms such as Europeana, and accelerate their development. While Europeana drives transformation throughout the cultural heritage sector with innovative standards, infrastructure and networks, Time Machine aims to design and implement advanced new digitisation and artificial intelligence technologies to mine Europe's vast cultural heritage, providing fair and free access to information that will support future scientific and technological developments in Europe.

In recent years there have been multiple successful attempts tackling document processing problems separately by designing task specific hand-tuned strategies. We argue that the diversity of historical document processing tasks prohibits to solve them one at a time and shows a need for designing generic approaches in order to handle the variability of historical series. In this paper, we address multiple tasks simultaneously such as page extraction, baseline extraction, layout analysis or multiple typologies of illustrations and photograph extraction. We propose an open-source implementation of a CNN-based pixel-wise predictor coupled with task dependent post-processing blocks. We show that a single CNN-architecture can be used across tasks with competitive results. Moreover most of the task-specific post-precessing steps can be decomposed in a small number of simple and standard reusable operations, adding to the flexibility of our approach.

The slowing down of Moore's law and the emergence of new technologies puts an increasing pressure on the field of EDA. There is a constant need to improve optimization algorithms. However, finding and implementing such algorithms is a difficult task, especially with the novel logic primitives and potentially unconventional requirements of emerging technologies. In this paper, we cast logic optimization as a deterministic Markov decision process (MDP). We then take advantage of recent advances in deep reinforcement learning to build a system that learns how to navigate this process. Our design has a number of desirable properties. It is autonomous because it learns automatically and does not require human intervention. It generalizes to large functions after training on small examples. Additionally, it intrinsically supports both single- and multi-output functions, without the need to handle special cases. Finally, it is generic because the same algorithm can be used to achieve different optimization objectives, e.g., size and depth.

This paper describes a web-based system for page segmentation and text recognition of historical documents. The system is organised following a pipeline of 4 steps : 1) digitisation, 2) preprocessing, 3) textline extraction, and 4) handwritten text recognition based on hidden Markov models. In this study we used to evaluate the system the “Statuti del Doge Tiepolo”, a 14thcentury manuscript written in Gothic script, digitized and transcribed by an expert paleographer, The paper discusses the initial performances of interface and processing pipeline in this context.

The Venice Time Machine is an international scientific programme launched by the EPFL and the University Ca'Foscari of Venice with the generous support of the Fondation Lombard Odier. It aims at building a multidimensional model of Venice and its evolution covering a period of more than 1000 years. The project ambitions to reconstruct a large open access database that could be used for research and education. Thanks to a parternship with the Archivio di Stato in Venice, kilometers of archives are currently digitized, transcribed and indexed setting the base of the largest database ever created on Venetian documents. The State Archives of Venice contain a massive amount of hand-written documentation in languages evolving from medieval times to the 20th century. An estimated 80 km of shelves are filled with over a thousand years of administrative documents, from birth registrations, death certificates and tax statements, all the way to maps and urban planning designs. These documents are often very delicate and are occasionally in a fragile state of conservation. In complementary to these primary sources, the content of thousands of monographies have been indexed and made searchable.

Studying natural reading and its underlying attention processes requires devices that are able to provide precise measurements of gaze without rendering the reading activity unnatural. In this paper we propose an eye tracking system that can be used to conduct analyses of reading behavior in low constrained experimental settings. The system is designed for dual-camera-based head-mounted eye trackers and allows free head movements and note taking. The system is composed of three different modules. First, a 3D model-based gaze estimation method computes the reader's gaze trajectory. Second, a document image retrieval algorithm is used to recognize document pages and extract annotations. Third, a systematic error correction procedure is used to post-calibrate the system parameters and compensate for spatial drifts. The validation results show that the proposed method is capable of extracting reliable gaze data when reading in low constrained experimental conditions.

We present an eye tracking study to investigate how natural reading behavior and reading comprehension are influenced by in-context annotations. In a lab experiment, three groups of participants were asked to read a text and answer comprehension questions: a control group without taking annotations, a second group reading and taking annotations, and a third group reading a peer-annotated version of the same text. A self-made head-mounted eye tracking system was specifically designed for this experiment, in order to study how learners read and quickly re-read annotated paper texts, in low constrained experimental conditions. In the analysis, we measured the phenomenon of annotation-induced overt attention shifts in reading, and found that: (1) the reader's attention shifts toward a margin annotation more often when the annotation lies in the early peripheral vision, and (2) the number of attention shifts, between two different types of information units, is positively related to comprehension performance in quick re-reading. These results can be translated into potential criteria for knowledge assessment systems.

We present a longitudinal study on the participation regulation effects in the presence of a speech aware interactive table. This study focuses on training meetings of groups of top level managers, whose compositions do not change, in a corporate organization. We show that an effect of balancing participation develops over time. We also report other emerging group-specific features such as interaction patterns and signatures, leadership effects, and behavioral changes between meetings. Finally we collect feedback from the participants and analyze qualitatively the human and social aspects of the participants interaction mediated by the technology.

The historical evolution of human machine interfaces shows a continuous tendency towards more physical interactions with computers. Nevertheless, the mouse and keyboard paradigm is still the dominant one and it is not yet clear whether there is among recent innovative interaction techniques any real challenger to this supremacy. To discuss the future of gesture-based interfaces, I shall build on my own experience in conceiving and launching QB1, probably the first computer delivered with no mouse or keyboard but equipped with a depth-perceiving camera enabling interaction with gestures. The ambition of this talk is to define more precisely how gestures change the way we can interact with computers, discuss how to design robust interfaces adapted to this new medium and review what kind of applications benefit the most from this type of interaction. Through a series of examples, we will see that it is important to consider gestures not as a way of emulating a mouse pointer at a distance or as elements of a vocabulary of commands, but as a new interaction paradigm where the interface components are organized in the user's physical space. This is a shift of reference frame, from a metaphorical virtual space (e.g. the desktop) where the user controls a representation of himself (e.g. the mouse pointer) to a truly user-centered augmented reality interface where the user directly touches and manipulates interface components positioned around his body. To achieve this kind of interactivity, depth-perceiving cameras can be relevantly associated with robotic techniques and machine vision algorithms to create a halo of interactivity that can literally follow the user while he moves in a room. In return, this new kind of intimacy with a computer interface paves the ways for innovative machine learning approaches to context understanding. A computer like QB1 knows more about its user than any other personal computer so far. Gesture-based interaction is not a mean for replacing the mouse with cooler or more intuitive ways of interacting but leads to a fundamentally different approach to the design human-computer interfaces.

Although many augmented tabletop systems have shown the potential and usability of finger-based interactions and paper-based interfaces, they have mainly dealt with each of them separately. In this paper, we introduce a novel method aimed to improve human natural interactions on augmented tabletop systems, which enables multiple users to use both fingertips and physical papers as mediums for interaction. This method uses computer vision techniques to detect multi-fingertips both over and touching the surface in real-time regardless of their orientations. Fingertip and touch positions would then be used in combination with paper tracking to provide a richer set of interaction gestures that the users can perform in collaborative scenarios.

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology.

What kind of motivation drives child language development? This article presents a computational model and a robotic experiment to articulate the hypothesis that children discover communication as a result of exploring and playing with their environment. The considered robotic agent is intrinsically motivated towards situations in which it optimally progresses in learning. To experience optimal learning progress, it must avoid situations already familiar but also situations where nothing can be learned. The robot is placed in an environment in which both communicating and non-communicating objects are present. As a consequence of its intrinsic motivation, the robot explores this environment in an organized manner focussing first on non-communicative activities and then discovering the learning potential of certain types of interactive behavior. In this experiment, the agent ends up being interested by communication through vocal interactions without having a specific drive for communication.

Distributed co-ordination is the result of dynamical processes enabling independent agents to co-ordinate their actions without the need of a central co-ordinator. In the past few years, several computational models have illustrated the role played by such dynamics for self-organizing communication systems. In particular, it has been shown that agents could bootstrap shared convention systems based on simple local adaptation rules. Such models have played a pivotal role for our understanding of emergent language processes. However, only few formal or theoretical results have been published about such systems. Deliberately simple computational models are discussed in this paper in order to make progress in understanding the underlying dynamics responsible for distributed co-ordination and the scaling laws of such systems. In particular, the paper focuses on explaining the convergence speed of those models, a largely under-investigated issue. Conjectures obtained through empirical and qualitative studies of these simple models are compared with results of more complex simulations and discussed in relation to theoretical models formalized using Markov chains, game theory and Polya processes.