The development of contactless respiration monitoring for infants could enable advances in the early detection and treatment of breathing irregularities, which are associated with neurodevelopmental impairments and conditions like sudden infant death syndrome (SIDS). But while respiration estimation for adults is supported by a robust ecosystem of computer vision algorithms and video datasets, only one small public video dataset with annotated respiration data for infant subjects exists, and there are no reproducible algorithms which are effective for infants. We introduce the annotated infant respiration dataset of 400 videos (AIR-400), contributing 275 new, carefully annotated videos from 10 recruited subjects to the public corpus. We develop the first reproducible pipelines for infant respiration estimation, based on infant-specific region-of-interest detection and spatiotemporal neural processing enhanced by optical flow inputs. We establish, through comprehensive experiments, the first reproducible benchmarks for the state-of-the-art in vision-based infant respiration estimation. We make our dataset, code repository, and trained models available for public use.

Estimating 3D human pose from 2D images in real world contexts remains a challenge, characterized by unique data constraints. Large general datasets of motion-captured 3D adult human poses paired with 2D images exist, but in many application settings, collection of further motion-captured data is impossible, precluding a straightforward fine-tuning approach to adaptation. We present a method for improving 3D pose estimation transfer learning to domains where there are only depth camera images available as supervision. Our heuristic weakly supervised 3D human pose (HW-HuP) estimation method learns partial pose priors from general 3D human pose datasets and employs weak supervision with depth data to guide learning in an optimization and regression cycle. We show that HW-HuP meaningfully improves upon state- of-the-art models in the adult in-bed setting, as well as on large scale public 3D human pose datasets, under comparable supervision conditions. Our model code and data are publicly available at https://github.com/ostadabbas/hw-hup. A significantly expanded version of this paper, with supplementary material, is available as a preprint on arXiv at https://arxiv.org/abs/2105.10996.

Recent advancements in object detection and human activity recognition have shown commendable progress, albeit with a predominant focus on adult-centric applications and datasets. This paper proposes a new vision-based, infant-focused hazard detection framework, CribNet, to assess threats to in-crib safety in the form of blanket occlusions and hazardous toys, as a step towards addressing the broad, critical problem of infant sleep safety. CribNet estimates hazards by considering the proximity and characteristics of detected objects around the infants. To evaluate the framework, we created the first publicly available crib hazard detection (CribHD) dataset, consisting of 1,620 images specific to infant-centric environments. These images present a wide range of real-world challenges, including clutter, occlusion, varied lighting conditions, with and without presence of infants in the images. We show that the framework performs with over 80% mean average precision (mAP) in segmenting toys and blankets and accurately assessing hazards, marking a new advancement in infant safety. CribNet and CribHD lay the foundation for future developments in in-crib hazard detection and infant sleep safety11The code and our data are publicly available at https://github.com/ostadabbas/CribNet.

We apply computer vision pose estimation techniques developed expressly for the data-scarce infant domain to the study of torticollis, a common condition in infants for which early identification and treatment is critical. Specifically, we use a combination of facial landmark and body joint estimation techniques designed for infants to estimate a range of geometric measures pertaining to face and upper body symmetry, drawn from an array of sources in the physical therapy and ophthal-mology research literature in torticollis. We gauge performance with a range of metrics and show that the estimates of most these geometric measures are successful, yielding strong to very strong Spearman's$p$correlation with ground truth values. Furthermore, we show that these estimates, derived from pose estimation neural networks designed for the infant domain, cleanly outperform estimates derived from more widely known networks designed for the adult domain11Code and data available at https://github.com/ostadabbas/Infant-Upper-Body-Postural-Symmetry..

Bilateral postural symmetry plays a key role as a potential risk marker for autism spectrum disorder (ASD) and as a symptom of congenital muscular torticollis (CMT) in infants, but current methods of assessing symmetry require laborious clinical expert assessments. In this paper, we develop a computer vision based infant symmetry assessment system, leveraging 3D human pose estimation for infants. Evaluation and calibration of our system against ground truth assessments is complicated by our findings from a survey of human ratings of angle and symmetry, that such ratings exhibit low inter-rater reliability. To rectify this, we develop a Bayesian estimator of the ground truth derived from a probabilistic graphical model of fallible human raters. We show that the 3D infant pose estimation model can achieve 68% area under the receiver operating characteristic curve performance in predicting the Bayesian aggregate labels, compared to only 61% from a 2D infant pose estimation model and 60% from a 3D adult pose estimation model, highlighting the importance of 3D poses and infant domain knowledge in assessing infant body symmetry. Our survey analysis also suggests that human ratings are susceptible to higher levels of bias and inconsistency, and hence our final 3D pose-based symmetry assessment system is calibrated but not directly supervised by Bayesian aggregate human ratings, yielding higher levels of consistency and lower levels of inter-limb assessment bias1.

We lay the groundwork for research in the algorithmic comprehension of infant faces, in anticipation of applications from healthcare to psychology, especially in the early prediction of developmental disorders. Specifically, we introduce the first-ever dataset of infant faces annotated with facial landmark coordinates and pose attributes, demonstrate the inadequacies of existing facial landmark estimation algorithms in the infant domain, and train new state-of-the-art models that significantly improve upon those algorithms using domain adaptation techniques. We touch on the closely related task of facial detection for infants, and also on a challenging case study of infrared baby monitor images gathered by our lab as part of in-field research into the aforementioned developmental issues1

Understanding and debugging of data structures and algorithms (DSA) is one of the most common tasks in computer science. DSA tests have also become a standard threshold that software developers have to cross to "get the job". One major challenge in comprehending and debugging DSA implementations lies in establishing and maintaining mental models of the quintessentially complex and twisted networks of events that make up their dynamic runtime behavior. Despite the high level of difficulty of this crucial task, general purpose tools to help users understand or reason about DSA implementations still have very limited capabilities. In this work we present Dbux-PDG, a dynamic Program Dependency Graph extension for the Dbux omniscient debugger. It captures data and control flow, as well as data dependencies of a program’s execution for visualization and user interaction. To deal with the immense complexity of non-trivial programs, it offers multiple layers of summarization, that allow the user to explore either the graph as a whole or in parts, one step at a time, as they see fit. We present our findings from applying Dbux-PDG to 94 diverse algorithms and explore its utility in several case studies. All visual results are made available in an online gallery. Dbux-PDG is open source and one-click installable, making it a powerful, easy-to-use tool prototype for DSA comprehension.Video URL: https://youtu.be/dgXj3VoQJZQ

Deep reinforcement learning (RL) algorithms have achieved great success on a wide variety of sequential decision-making tasks. However, many of these algorithms suffer from high sample complexity when learning from scratch using environmental rewards, due to issues such as credit-assignment and high-variance gradients, among others. Transfer learning, in which knowledge gained on a source task is applied to more efficiently learn a different but related target task, is a promising approach to improve the sample complexity in RL. Prior work has considered using pre-trained teacher policies to enhance the learning of the student policy, albeit with the constraint that the teacher and the student MDPs share the state-space or the action-space. In this paper, we propose a new framework for transfer learning where the teacher and the student can have arbitrarily different state- and action-spaces. To handle this mismatch, we produce embeddings which can systematically extract knowledge from the teacher policy and value networks, and blend it into the student networks. To train the embeddings, we use a task-aligned loss and show that the representations could be enriched further by adding a mutual information loss. Using a set of challenging simulated robotic locomotion tasks involving many-legged centipedes, we demonstrate successful transfer learning in situations when the teacher and student have different state- and action-spaces.

International data grids are now being built that support joint management of shared collections. An emerging strategy is to build multiple independent data grids, each managed by the local institution. The data grids are then federated to enable controlled sharing of files. We examine the management issues associated with maintaining federations of production data grids, including management of access controls, coordinated sharing of name spaces, replication of data between data grids, and expansion of the data grid federation.

The integration of grid, data grid, digital library, and preservation technology has resulted in software infrastructure that is uniquely suited to the generation and management of data. Grids provide support for the organization, management, and application of processes. Data grids manage the resulting digital entities. Digital libraries provide support for the management of information associated with the digital entities. Persistent archives provide long-term preservation. We examine the synergies between these data management systems and the future evolution that is required for the generation and management of information.

Data Grids are becoming increasingly important in scientific communities for sharing large data collections and for archiving and disseminating them in a digital library framework. The Storage Resource Broker (SRB) provides transparent virtualized middleware for sharing data across distributed, heterogeneous data resources separated by different administrative and security domains. The MySRB is a Web-based interface to the SRB that provides a user-friendly interface to distributed collections brokered by the SRB. In this paper we briefly describe the use of the SRB infrastructure as tools in the data grid architecture for building distributed data collections, digital libraries, and persistent archives. We also provide details about the MySRB and its functionalities.