Research in dolphin communication and cognition requires detailed inspection of audible dolphin signals. The manual analysis of these signals is cumbersome and time-consuming. We seek to automate parts of the analysis using modern deep learning methods. We propose to learn an autoencoder constructed from convolutional and recurrent layers trained in an unsupervised fashion. The resulting model embeds patterns in audible dolphin communication. In several experiments, we show that the embeddings can be used for clustering as well as signal detection and signal type classification.

We describe the development of a system for recognizing dolphin whistles on the CHAT (cetacean hearing and telemetry) wearable underwater computer system. An Nvidia Jetson TK1 single board computer was installed in the existing chat systems to improve processing power and overall system power efficiency. The inclusion of a GPU allowed the system to recognize in real time dolphin whistles varied in both pitch and time by using a 192khz Fast Fourier Transform for spectral analysis, linear convolution filters for pattern extraction, and dynamic time warping for pattern recognition.

The ACM Recommender Systems Challenge 20171 focused on the problem of job recommendations: given a new job advertisement, the goal was to identify those users who are both (a) interested in getting notified about the job advertisement, and (b) appropriate candidates for the given job. Participating teams had to balance between user interests and requirements for the given job as well as dealing with the cold-start situation. For the first time in the history of the conference, the RecSys challenge offered an online evaluation: teams first had to compete as part of a traditional offline evaluation and the top 25 teams were then invited to evaluate their algorithms in an online setting, where they could submit recommendations to real users. Overall, 262 teams registered for the challenge, 103 teams actively participated and submitted together more than 6100 solutions as part of the offline evaluation. Finally, 18 teams participated and rolled out recommendations to more than 900,000 users on XING2.

The 2016 ACM Recommender Systems Challenge focused on the problem of job recommendations. Given a large dataset from XING that consisted of anonymized user profiles, job postings, and interactions between them, the participating teams had to predict postings that a user will interact with. The challenge ran for four months with 366 registered teams. 119 of those teams actively participated and submitted together 4,232 solutions yielding in an impressive neck-and-neck race that was decided within the last days of the challenge.

In this position paper, we take the experimental approach of putting algorithms aside, and reflect on what recommenders would be for people if they were not tied to technology. By looking at some of the shortcomings that current recommenders have fallen into and discussing their limitations from a human point of view, we ask the question: if freed from all limitations, what should, and what could, RecSys be? We then turn to the idea that life itself is the best recommender system, and that people themselves are the query. By looking at how life brings people in contact with options that suit their needs or match their preferences, we hope to shed further light on what current RecSys could be doing better. Finally, we look at the forms that RecSys could take in the future. By formulating our vision beyond the reach of usual considerations and current limitations, including business models, algorithms, data sets, and evaluation methodologies, we attempt to arrive at fresh conclusions that may inspire the next steps taken by the community of researchers working on RecSys.

Predicting the outcome of an action can help a robot detect failures in advance, and schedule action replanning before an error occurs. We propose using an interactive physics based simulator with the aim of collecting realistic data to be used for learning. We then show how we save and query for specific information from the data more effectively. The data from the simulation is used to learn a failure detection model which is utilized by a real robot performing the same actions. We show that learning from simulation data is realistic enough to be applied on a real robot. The learning algorithm is more simple in design and outperforms the more complex one from our previous work.

The study of dolphin cognition involves intensive research of animal vocalizations. Marine mammalogists commonly study a specific sound type known as the whistle found in dolphin communication. However, one of the main problems arises from noisy underwater environments. Often waves and splash noises will partially distort the whistle making analysis or extraction difficult. Another problem is discovering fundamental units that allow research of the composition of whistles. We propose a method for whistle extraction from noisy underwater recordings using a probabilistic approach. Furthermore, we investigate discovery algorithms for fundamental units using a mixture of hidden Markov models. We evaluate our findings with a marine mammalogist on data collected in the field. Furthermore, we have evidence that our algorithms enable researchers to form hypotheses about the composition of whistles.

In order to manage complex tasks such as cooking, future robots need to be action-aware and posses common sense knowledge. For example flipping a pancake requires a robot to know that a spatula has to be under a pancake in order to succeed. We present a novel approach for the extraction and learning of action and common sense knowledge, and developed a game using a robot-simulator with realistic physics for data acquisition. The game environment is a virtual kitchen, in which a user has to create a pancake by pouring pancake-mix on an oven and flipping it using a spatula. The interaction is done by controlling a virtual robot hand with a 3D input sensor. We incorporate a realistic fluid simulation in order to gather appropriate data of the pouring action. Furthermore, we present a task outcome prediction algorithm for this specific system and show how to learn a failure model for the pouring and flipping action.

Interacting with mobile technology while in-motion has become a daily activity for many of us. Common sense leads one to believe that texting with a mini-qwerty keyboard while mobile can be dangerous since users are distracted and not paying attention to the environment. Previous studies have found that mobility negatively impacts text entry performance for novice participants typing on virtual keyboards on touch screen mobile phones. We investigate the impact of mobility on expert users' ability to quickly and accurately input text on mobile phones equipped with fixed-key mini-qwerty keyboards. In total, 36 participants completed 600 minutes of typing on mini-qwerty keyboards (300 minutes training up to expertise) in three mobility conditions (seated, standing, and walking) generating almost 4,000,000 characters across all conditions. Surprisingly, we found that walking has a significant impact on expert typing speeds but does not significantly impact expert accuracy rates.

False positives are a common problem for interfaces that rely on gesture recognition. Often a gesture can seem fine in development but is found to trigger accidentally during an initial deployment of the interface, restarting development and increasing expense. In this work we introduce MAGIC 2.0, a technique for false positive prediction and prevention that can be used interactively during the interface design process. To ground our research, we implement MAGIC 2.0 as a web service and develop gesture interfaces using sensors on common Android mobile phone platforms. We use iSAX (indexable Symbolic Aggregate approXimation) to enable interactive searching (;1,500,000 sec) of everyday user movements on a standard workstation to determine if a candidate gesture will trigger accidentally during use of an interface. We perform a user-independent study that suggests that the number of matches to this Everyday Gesture Library (EGL) database is indeed predictive of a candidate gesture's suitability. We compare iSAX to hidden Markov models (HMMs) and nearest neighbor with respect to accuracy and speed for the EGL search. Using iSAX on the EGL, we also develop a “garbage” class and show that including this class in recognition reduces errors.

Mobile Music Touch (MMT) helps teach users to play piano melodies while they perform other tasks. MMT is a lightweight, wireless haptic music instruction system consisting of fingerless gloves and a mobile Bluetooth enabled computing device, such as a mobile phone. Passages to be learned are loaded into the mobile phone and are played repeatedly while the user performs other tasks. As each note of the music plays, vibrators on each finger in the gloves activate, indicating which finger is used to play each note. We present two studies on the efficacy of MMT. The first measures 16 subjects' ability to play a passage after using MMT for 30 minutes while performing a reading comprehension test. The MMT system was significantly more effective than a control condition where the passage was played repeatedly but the subjects' fingers were not vibrated. The second study compares the amount of time required for 10 subjects to replay short, randomly generated passages using passive training versus active training. Participants with no piano experience could repeat the passages after passive training while subjects with piano experience often could not.