Model-based reasoning agents are ill-equipped to act in novel situations in which their model of the environment no longer sufficiently represents the world. We propose HYDRA, a framework for designing model-based agents operating in mixed discrete-continuous worlds that can autonomously detect when the environment has evolved from its canonical setup, understand how it has evolved, and adapt the agents' models to perform effectively. HYDRA is based upon PDDL+, a rich modeling language for planning in mixed, discrete-continuous environments. It augments the planning module with visual reasoning, task selection, and action execution modules for closed-loop interaction with complex environments. HYDRA implements a novel meta-reasoning process that enables the agent to monitor its own behavior from a variety of aspects. The process employs a diverse set of computational methods to maintain expectations about the agent's own behavior in an environment. Divergences from those expectations are useful in detecting when the environment has evolved and identifying opportunities to adapt the underlying models. HYDRA builds upon ideas from diagnosis and repair and uses a heuristics-guided search over model changes such that they become competent in novel conditions. The HYDRA framework has been used to implement novelty-aware agents for three diverse domains - CartPole++ (a higher dimension variant of a classic control problem), Science Birds (an IJCAI competition problem), and PogoStick (a specific problem domain in Minecraft). We report empirical observations from these domains to demonstrate the efficacy of various components in the novelty meta-reasoning process.

Numerical reasoning is an essential task for supporting machine learning applications, such as recommendation and information retrieval. The reasoning task aims to compare two items and infer new facts (e.g., is taller than) by leveraging existing relational information and numerical attributes (e.g., the height of an entity) in knowledge graphs. However, most existing methods rely on leveraging attribute encoders or additional loss functions to predict numerical relations. Therefore, the prediction performance is often not robust in cases when attributes are sparsely observed. In this paper, we propose a Relation-AAware attribute representation learning-based Knowledge Graph Embedding method for numerical reasoning tasks, which we call RAKGE. RAKGE incorporates a newly proposed attribute representation learning mechanism, which can leverage the association between relations and their corresponding numerical attributes. In addition, we introduce a robust self-supervised learning method to generate unseen positive and negative examples, thereby making our approach more reliable when numerical attributes are sparsely available. In the evaluation of three real-world datasets, our proposed model outperformed state-of-the-art methods, achieving an improvement of up to 65.1% in Hits@1 and up to 52.6% in MRR compared to the best competitor. Our implementation code is available at https://github.com/learndatalab/RAKGE.

Safety of taking medicine prescribed differently to each patient in hospital relies on the discernment of medical professionals who deals with measuring pill quantity, packaging, and distributing. It is difficult and time consuming to keep track of medication record of each patient. Also, medication safety is prone to be in risk due to the human error. To help patients get accurate medication following their prescription plan with minimizing human labors and mistakes, we developed PillGood, an automated smart pill dispenser system using facial recognition technique. PillGood provides real-time and personalized guidance to take the correct medicine by alarming patients and distributing exact quantity of pills at specific time following each patient's prescription table. The system notify patients through mobile app and speaker when they need to take the medicine, and detect who the patient is through the machine learning based face recognition. Then, based on each patient's prescribing information, the controller distributes pills to each patient. Results show that PillGood enable highly accurate personalized pill dispensation followed by precise face recognition, benefiting both patients and medical professionals. Videos for demonstrating the system can be found on https://youtu.be/Wx7bXxRGjXA

Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes to handling flexible frame rates (e.g., increasing the frame rate of the more dynamic portion of the video as well as handling missing video frames). To resolve the restricted nature of existing video generation models' ability to handle arbitrary timesteps, we propose continuous-time video generation by combining neural ODE (Vid-ODE) with pixel-level video processing techniques. Using ODE-ConvGRU as an encoder, a convolutional version of the recently proposed neural ODE, which enables us to learn continuous-time dynamics, Vid-ODE can learn the spatio-temporal dynamics of input videos of flexible frame rates. The decoder integrates the learned dynamics function to synthesize video frames at any given timesteps, where the pixel-level composition technique is used to maintain the sharpness of individual frames. With extensive experiments on four real-world video datasets, we verify that the proposed Vid-ODE outperforms state-of-the-art approaches under various video generation settings, both within the trained time range (interpolation) and beyond the range (extrapolation). To the best of our knowledge, Vid-ODE is the first work successfully performing continuous-time video generation using real-world videos.

Deep reinforcement learning (DRL) has proven successful for many difficult control problems by learning policies represented by neural networks. However, the complexity of neural network-based policies{—}involving thousands of composed non-linear operators{—}can render them problematic to understand, trust, and deploy. In contrast, simple policies comprising short symbolic expressions can facilitate human understanding, while also being transparent and exhibiting predictable behavior. To this end, we propose deep symbolic policy, a novel approach to directly search the space of symbolic policies. We use an autoregressive recurrent neural network to generate control policies represented by tractable mathematical expressions, employing a risk-seeking policy gradient to maximize performance of the generated policies. To scale to environments with multi-dimensional action spaces, we propose an "anchoring" algorithm that distills pre-trained neural network-based policies into fully symbolic policies, one action dimension at a time. We also introduce two novel methods to improve exploration in DRL-based combinatorial optimization, building on ideas of entropy regularization and distribution initialization. Despite their dramatically reduced complexity, we demonstrate that discovered symbolic policies outperform seven state-of-the-art DRL algorithms in terms of average rank and average normalized episodic reward across eight benchmark environments.

Discovering tractable mathematical expressions that best explain a dataset is a long-standing challenge in artificial intelligence. This problem, known as symbolic regression, is relevant when one seeks to generate new physical knowledge and insights. Since practitioners are primarily interested in knowledge generation, the ability to interact with a symbolic regression algorithm would be highly valuable. Thus, we present an interactive symbolic regression framework that allows users not only to configure runs, but also to control the system during training. The interface provides real-time visualization and diagnostics to help guide the user as they control the algorithm on the fly.

Tracking and predicting extreme events in large-scale spatio-temporal climate data are long standing challenges in climate science. In this paper, we propose Convolutional LSTM (ConvLSTM)-based spatio-temporal models to track and predict hurricane trajectories from large-scale climate data; namely, pixel-level spatio-temporal history of tropical cyclones. To address the tracking problem, we model time-sequential density maps of hurricane trajectories, enabling to capture not only the temporal dynamics but also spatial distribution of the trajectories. Furthermore, we introduce a new trajectory prediction approach as a problem of sequential forecasting from past to future hurricane density map sequences. Extensive experiment on actual 20 years record shows that our ConvLSTM-based tracking model significantly outperforms existing approaches, and that the proposed forecasting model achieves successful mapping from predicted density map to ground truth.

Although Online Social Network (OSN) services offer users a variety of benefits, they also bring new threats and privacy issues to the community. In this paper, we first define the data types in OSN services and the states of shared data with respect to Optimal, Under-shared, Over-shared, and Hybrid states. We also identify the facilitating, detracting, and preventive parameters that are responsible for the state transition of the data. We address that, in a reliable OSN service, a user should be able to set up his or her desired level of information sharing with a certain group of other users. However, it is not always clear to the ordinary users how to decide how much information they should reveal to others. Therefore, we propose an approach for helping OSN users determine their optimum levels of information sharing, taking into consideration the payoffs (potential Reward or Cost) based on the Markov decision process (MDP).