Diffusion-based human animation aims to animate a human character based on a source human image as well as driving signals such as a sequence of poses. Leveraging the generative capacity of diffusion model, existing approaches are able to generate high-fidelity poses, but struggle with significant viewpoint changes, especially in zoom-in/zoom-out scenarios where camera-character distance varies. This limits the applications such as cinematic shot type plan or camera control. We propose a pose-correlated reference selection diffusion network, supporting substantial viewpoint variations in human animation. Our key idea is to enable the network to utilize multiple reference images as input, since significant viewpoint changes often lead to missing appearance details on the human body. To eliminate the computational cost, we first introduce a novel pose correlation module to compute similarities between non-aligned target and source poses, and then propose an adaptive reference selection strategy, utilizing the attention map to identify key regions for animation generation. To train our model, we curated a large dataset from public TED talks featuring varied shots of the same character, helping the model learn synthesis for different perspectives. Our experimental results show that with the same number of reference images, our model performs favorably compared to the current SOTA methods under large viewpoint changes. We further show that the adaptive reference selection is able to choose the most relevant reference regions to generate humans under free viewpoints.

We propose a new system named DOS®(Deployment Operating System for Robots) for reliably deploying any data-driven robots in both production and simulation environments. Compared to existing systems, DOS®features a unique CI/CD (continuous integration and continuous deployment) architecture which allows us to seamlessly integrate agile development and reliable operation in a fully automated fashion. With this CI/CD architecture, this paper mainly introduces three essential components that uniquely differentiate DOS®from existing robotic systems: (i) An environment adapter that provides a systematic and robust approach to handle the deployment complexity in real world environments; (ii) A data replay reservoir that provides a unified data model supporting arbitrary robotic decision models; (iii) An analytical profiler that collects any set of user-defined performance metrics for system optimization. DOS®significantly increases the reliability and maintainability of the deployed robotic systems. To illustrate this point, we compare DOS®with more traditional approaches on deploying a navigational robot in a challenging working environment with many new corner case scenarios. Our results show that DOS®outperforms traditional approach in great magnitudes in terms of deployment time and operational robustness.

Wave data mining and processing are important in ocean prediction. However, wave data often exhibits imbalance, resulting in low accuracy in predicting extreme phenomena. To alleviate this problem, we propose a hybrid feature selection method based on imbalance learning (HFS-IL) to improve accuracy of prediction models. Specifically, we first use a Long Short-Term Memory (LSTM) network to train an imbalance discriminator, which aims to classify input data into common and rare subsets. Secondly, we select the optimal feature subsets by a hybrid feature selection algorithm, which is innovatively designed by combining mutual information and forward selection. To verify the effectiveness of HFS-IL, we process an imbalanced wave dataset from ERA5 by HFS-IL and use the processed data as input for an intelligent prediction model. The experimental results demonstrate that HFS-IL can effectively alleviate the impact of data imbalance and improve the accuracy of prediction, especially at station 51000, the majority of metrics outperform GRU and OSP-FEAN.

We propose EMS®, a cloud-enabled massive computational experiment management system supporting high-throughput computational robotics research. Compared to existing systems, EMS® features a sky-based pipeline orchestrator which allows us to exploit heterogeneous computing environments painlessly (e.g., on-premise clusters, public clouds, edge devices) to optimally deploy large-scale computational jobs (e.g., with more than millions of computational hours) in an integrated fashion. Cornerstoned on this sky-based pipeline orchestrator, this paper introduces three abstraction layers of the EMS® software architecture: (i) Configuration management layer focusing on automatically enumerating experimental configurations; (ii) Dependency management layer focusing on managing the complex task dependencies within each experimental configuration; (iii) Computation management layer focusing on optimally executing the computational tasks using the given computing resource. Such an architectural design greatly increases the scalability and reproducibility of data-driven robotics research leading to much-improved productivity. To demonstrate this point, we compare EMS® with more traditional approaches on an offline reinforcement learning problem for training mobile robots. Our results show that EMS® outperforms more traditional approaches in two magnitudes of orders (in terms of experimental high throughput and cost) with only several lines of code change. We also exploit EMS® to develop mobile robot, robot arm, and bipedal applications, demonstrating its applicability to numerous robot applications.

For surgeries which require general anesthesia, airway management is imperative. Difficult airway, which inhibits proper intubation, can be fatal. As such, pre-operative airway assessments are conducted by clinicians to determine the ease of intubation as well as to identify patients with difficult airway. To improve the process, artificial intelligence (AI) methods can be employed to predict such difficult airway situations so that suitable preparations can be made beforehand. However, due to the need for explainability of AI models required by healthcare regulations, typical black box models which work best with most data-driven AI methods cannot be used. Therefore, in the current work, a machine learning model has been established to predict the specific medical facial landmarks that are currently used by clinicians. These include the eyes, mentum, thyroid notch, suprasternal notch, forehead, tragus and radix. The model is based on convolutional neural network and a practical facial landmark detector concept. Furthermore, k-fold cross-validation sampling and the Adabelief optimizer have been utilized. The model prediction results display accurate prediction of the features, with the testing loss exhibiting good stability and maintaining well below 0.01 throughout. Attributed to that, the current model can lead to meaningful diagnosis of difficult airway during airway assessments.

We exploit Markov games as a framework for collision-free navigation of human-centered robots. Unlike the classical methods which formulate robot navigation as a single-agent Markov decision process with a static environment, our framework of Markov games adopts a multi-agent formulation with one primary agent representing the robot and the remaining auxiliary agents form a dynamic or even competing environment. Such a framework allows us to develop a path-following type adversarial training strategy to learn a robust decentralized collision avoidance policy. Through thorough experiments on both simulated and real-world mobile robots, we show that the learnt policy outperforms the state-of-the-art algorithms in both sample complexity and runtime robustness.