Sparse matrix multiplication (SpMM) is a critical kernel used in a wide range of applications, but irregular memory access patterns and memory bandwidth bottleneck as well as load imbalance make the efficient and scalable processing on parallel architectures a significant challenge. Motivated by the memory-bound nature of SpMM computation, we propose a cost-effective SpMM accelerator based on a DRAM processing-in-memory (PIM) approach. Our design introduces a novel dataflow to exploit high bank-level parallelism and reuse both input and output data even for highly sparse matrices. Our proposed architecture, SPIMA, features multiple input buffers for scheduling DRAM access, a output buffer and vector register files working holistically, co-designed to maximize the performance of our novel dataflow. Our experimental results using various sparse matrices demonstrate that our proposed dataflow and architecture are robust in terms of matrix size and sparsity. Compared with the state-of-the-art accelerators implemented on PIM, ASIC, and FPGA, we estimate that our PIM architecture can yield competitive performance with highly sparse matrices.

Learning robotic manipulation skills from vision is a promising approach for developing robotics applications that can generalize broadly to real-world scenarios. As such, many approaches to enable this vision have been explored with fruitful results. Particularly, object-centric representation methods have been shown to provide better inductive biases for skill learning, leading to improved performance and generalization. Nonetheless, we show that object-centric methods can struggle to learn simple manipulation skills in multi-object environments.Thus, we propose DOCIR, an object-centric framework that introduces a disentangled representation for objects of interest, obstacles, and robot embodiment. We show that this approach leads to state-of-the-art performance for learning pick and place skills from visual inputs in multi-object environments and generalizes at test time to changing objects of interest and distractors in the scene. Furthermore, we show its efficacy both in simulation and zero-shot transfer to the real world.

Team members commonly collaborate on visual documents remotely and asynchronously. Particularly, students are frequently restricted to this setting as they often do not share work schedules or physical workspaces. As communication in this setting has delays and limits the main modality to text, members exert more effort to reference document objects and understand others’ intentions. We propose Winder, a Figma plugin that addresses these challenges through linked tapes—multimodal comments of clicks and voice. Bidirectional links between the clicked-on objects and voice recordings facilitate understanding tapes: selecting objects retrieves relevant recordings, and playing recordings highlights related objects. By periodically prompting users to produce tapes, Winder preemptively obtains information to satisfy potential communication needs. Through a five-day study with eight teams of three, we evaluated the system’s impact on teams asynchronously designing graphical user interfaces. Our findings revealed that producing linked tapes could be as lightweight as face-to-face (F2F) interactions while transmitting intentions more precisely than text. Furthermore, with preempted tapes, teammates coordinated tasks and invited members to build on each others’ work.

Validating the kinematic feasibility of a planned robot motion and finding corresponding inverse solutions are time-consuming processes, especially for long-horizon manipulation tasks. Most existing approaches are based on solving iterative gradient-based optimization, so the processes are time-consuming and have a high risk of falling in local minima. In this work, we propose a unified framework to learn a kinematic feasibility model and a one-shot inverse mapping model for a redundant robot manipulator. Once they are trained, the models can compute the kinematic reachability of a target pose and its inverse solutions without iterative process. We validate our approach using a 7-DOF robot arm with an object grasping application.

We address the difficult problem of catching in-flight objects with uneven shapes. This requires the solution of three complex problems: accurate prediction of the trajectory of fastmoving objects, predicting the feasible catching configuration, and planning the arm motion, and all within milliseconds. We follow a programming-by-demonstration approach in order to learn, from throwing examples, models of the object dynamics and arm movement. We propose a new methodology to find a feasible catching configuration in a probabilistic manner. We use the dynamical systems approach to encode motion from several demonstrations. This enables a rapid and reactive adaptation of the arm motion in the presence of sensor uncertainty. We validate the approach in simulation with the iCub humanoid robot and in real-world experiments with the KUKA LWR 4+ (7-degree-of-freedom arm robot) to catch a hammer, a tennis racket, an empty bottle, a partially filled bottle, and a cardboard box.

This work presents a methodology to generate dynamically stable whole-body motions for a humanoid robot, which are converted from human motion capture data. The methodology consists of the kinematic and dynamical mappings for human-likeness and stability, respectively. The kinematic mapping includes the scaling of human foot and Zero Moment Point (ZMP) trajectories considering the geometric differences between a humanoid robot and a human. It also provides the conversion of human upper body motions using the method in. The dynamic mapping modifies the humanoid pelvis motion to ensure the movement stability of humanoid whole-body motions, which are converted from the kinematic mapping. In addition, we propose a simplified human model to obtain a human ZMP trajectory, which is used as a reference ZMP trajectory for the humanoid robot to imitate during the kinematic mapping. A human whole-body dancing motion is converted by the methodology and performed by a humanoid robot with online balancing controllers.

During the communication and interaction with a human using motions or gestures, a humanoid robot needs to not only look like a human but also behavior like a human to avoid confusions in the communication and interaction. Among human-like behaviors, arm motions of the humanoid robot are essential for the communication with people through motions. In this work, a mathematical representation for characterizing human arm motions is first proposed. The human arm motions are characterized by the elbow elevation angle that is determined using the position and orientation of human hands. That representation is mathematically obtained using an approximation tool, response surface method (RSM). Then, a method to generate human-like arm motions in real time using the proposed representation is presented. The proposed method was evaluated to generate human-like arm motions when the humanoid robot was asked to move its arms from a point to another point including the rotation of hand. An example motion was performed using the KIST humanoid robot, MAHRU