Several recently introduced deep learning optimizers utilizing matrix-level preconditioning have shown promising speedups relative to the current dominant optimizer AdamW, particularly in relatively small-scale experiments. However, efforts to validate and replicate their successes have reported mixed results. To better understand the effectiveness of these optimizers at scale, in this work we investigate how to scale preconditioned optimizers via hyperparameter transfer, building on prior works such as $\mu$P. We study how the optimal learning rate and weight decay should scale with model width and depth for a wide range of optimizers, including Shampoo, SOAP, and Muon, accounting for the impact of commonly used techniques such as blocking and grafting. We find that scaling the learning rate according to $\mu$P improves transfer, but can still suffer from significant finite-width deviations that cause drifting optimal learning rates, which we show can be mitigated by blocking and explicit spectral normalization. For compute-optimal scaling, we find scaling independent weight decay as $1/\mathrm{width}$ is nearly optimal across optimizers. Applying these scaling rules, we show Muon, SOAP and Shampoo consistently achieve near $1.4\times$ speedup over AdamW for training Llama-architecture language models of sizes ranging from $190$M to $1.4$B, whereas the speedup vanishes rapidly with scale under incorrect scaling. Based on these results and further ablations, we argue that studying optimal hyperparameter transfer is essential for reliably comparing optimizers at scale given a realistic tuning budget.

When humans work together to complete a joint task, each person builds an internal model of the situation and how it will evolve. Efficient collaboration depends on how these individual models overlap to form a shared mental model among team members; shared models are also important for collaborative processes in human–robot teams. The development and maintenance of an accurate shared mental model requires bidirectional communication of individual intent and the ability to interpret the intent of other team members. To enable effective human–robot collaboration, this article investigates the use of augmented reality (AR) technology and user eye gaze to enable bidirectional communication of intent in a joint action task. We tested this approach through a user study with 37 participants and found that this communication improves task efficiency, trust, as well as task fluency. We conclude that using AR and eye gaze to enable bidirectional communication and support shared mental models is a promising means for improving collaboration between humans and robots.

Dense linear layers are the dominant computational bottleneck in large neural networks, presenting a critical need for more efficient alternatives. Previous efforts to develop alternatives have focused on a small number of hand-crafted structured matrices, and have neglected to investigate whether these structures can surpass dense layers in terms of compute-optimal scaling laws when both the model size and training examples are optimally allocated. In this work, we present a unifying framework that enables searching among all linear operators expressible via an Einstein summation. This framework encompasses many previously proposed structures, such as low-rank, Kronecker, Tensor-Train, and Monarch, along with many novel structures. We develop a taxonomy of all such operators based on their computational and algebraic properties, which provides insights into their scaling laws. Combining these insights with empirical evaluation, we identify a subset of structures that achieve equal or better performance than dense layers as a function of training compute. To further improve their compute efficiency, we develop a natural extension of these performant structures that convert them into a sparse Mixture-of-Experts layer. The resulting layer significantly outperforms dense layers in compute-optimal training efficiency for GPT-2 language models.

This paper provides a status report for RADARSAT-2 system operations and performance now that two years of Routine Phase operations have been completed. Experience from the second year of operations is reviewed. System status, performance, and trends are described along with performance and achievements of the RADARSAT-2 operations functional groups. Normal operations now includes increased attention to orbital collision risk avoidance, and the RADARSAT-2 risk mitigation experience is summarized. In addition to meeting and maintaining specified performance requirements for the Routine Phase, the operations team has implemented a number of improvements to both the system and operations to provide enhanced performance. These include reduced tasking latency, enhanced beam modes, and further improved image quality. The improvements are described as part of the evolution of the mission capabilities along with some further planned enhancements.

Following launch on SOYUZ in December, 2007, the RADARSAT-2 Satellite went through intensive Launch and Early Orbit Phase (LEOP) appendage deployment and attitude configuration activities. The first SAR data, including a remarkable Quad Pole image of Greenland, was acquired on-board on Day 4 of the mission as part of an extensive system commissioning campaign. The initial period of system operations saw introduction of a new high resolution imaging mode, demonstration of rapid tasking capabilities, successful operation through the first annual eclipse season, implementation of a new regulatory framework for control of user access to imagery, and continuous system improvement through tuning and enhancement of performance and to improve operations robustness. This paper provides a summary of the achievements and status of system operations and performance. Comparisons with pre-launch expectations, specifications, and plans are made to draw lessons from the flight experience. Plans for sustaining and enhancing system operations performance are outlined.