Human activity recognition (HAR) has become a key technology for improving the quality of life for individuals with special needs, such as older adults and children with Down Syndrome (CwDS). This study presents a novel application of HAR systems to detect directive behaviors in parent-child interactions, focusing on parents of CwDS during educational activities. Using video data, we developed two models: a Convolutional Neural Network (CNN) and a hybrid CNN-LSTM model, to recognize subtle cues such as physical proximity and verbal interactions. The CNN3D model achieved over 90% accuracy in detecting approach behaviors and around 65% for verbal expressions. The CNN-LSTM model outperformed CNN3D in classifying verbal expressions, achieving over 68% accuracy. These results highlight the potential of deep learning classifiers for analyzing subtle parent-child interactions, offering valuable insights into parent-child dynamics and contributing to the development of assistive tools for studying educational settings.

In this work, we aim at identifying the speaker in interactions between mothers and children with Down syndrome (DS) using audio. We collected audio from a session in which children with DS solved puzzles, and their mothers were by their side. We generated a dataset by manually annotating human speech activity and non-speech. We used machine learning to perform four experiments, including individual and generalized models achieving on average F1-scores of 0.74 (five-class model) and 0.84 (two-class) for individual models and 0.69 (five-class) and 0.82 (two-class) for the generalized models. Our results can be helpful to behavioral researchers, therapists, and those interested in better understanding how mother–child interactions unfold in naturalistic settings.

In this paper, we introduce the Tensor Streaming Processor (TSP) architecture, a functionally-sliced microarchitecture with memory units interleaved with vector and matrix deep learning functional units in order to take advantage of dataflow locality of deep learning operations. The TSP is built based on two key observations: (1) machine learning workloads exhibit abundant data parallelism, which can be readily mapped to tensors in hardware, and (2) a simple and deterministic processor with producer-consumer stream programming model enables precise reasoning and control of hardware components, achieving good performance and power efficiency. The TSP is designed to exploit parallelism inherent in machine-learning workloads including instruction-level, memory concurrency, data and model parallelism, while guaranteeing determinism by eliminating all reactive elements in the hardware (e.g. arbiters, and caches). Early ResNet50 image classification results demonstrate 20.4K processed images per second (IPS) with a batch-size of one— a $4 \times$ improvement compared to other modern GPUs and accelerators [44]. Our first ASIC implementation of the TSP architecture yields a computational density of more than 1 TeraOp/s per square mm of silicon for its $25 \times 29$ mm 14nm chip operating at a nominal clock frequency of 900 MHz. The TSP demonstrates a novel hardware-software approach to achieve fast, yet predictable, performance on machine-learning workloads within a desired power envelope.