We argue that diffusion models' success in modeling complex distributions is, for the most part, coming from their conditioning. This paper investigates the representation used to condition diffusion models from the perspective that ideal representations should improve modeling the data distribution, be easy to generate, and be compositional to allow generalizing outside the training distribution. We introduce Discrete Latent Code (DLC), an image representation derived from Simplicial Embeddings trained with a self-supervised learning objective. DLCs are sequences of discrete tokens, as opposed to the standard continuous image embeddings. They are easy to generate and their compositionality enables sampling of novel images beyond the training distribution. Diffusion models trained with DLCs improve generation fidelity, establishing a new state-of-the-art for unconditional image generation on ImageNet. Additionally, we show that composing DLCs allows the image generator to produce interesting out-of-distribution samples that coherently combine the semantics of images in diverse ways. Finally, we showcase how DLCs can enable text-to-image generation by leveraging large-scale pretrained language models. Using only 9M image-caption pairs, we efficiently finetune a text diffusion model to generate novel DLCs that produces samples outside of the data distribution used to train the image generator.

There are a thousand ways to caption an image. Contrastive Language Pretraining (CLIP) on the other hand, works by mapping an image and its caption to a single vector -- limiting how well CLIP-like models can represent the diverse ways to describe an image. In this work, we introduce Llip, Latent Language Image Pretraining, which models the diversity of captions that could match an image. Llip's vision encoder outputs a set of visual features that are mixed into a final representation by conditioning on information derived from the text. We show that Llip outperforms non-contextualized baselines like CLIP and SigLIP on a variety of tasks even with large-scale encoders. Llip improves zero-shot classification by an average of 2.9% zero-shot classification benchmarks with a ViT-G/14 encoder. Specifically, Llip attains a zero-shot top-1 accuracy of 83.5% on ImageNet outperforming a similarly sized CLIP by 1.4%. We also demonstrate improvement on zero-shot retrieval on MS-COCO by 6.0%. We provide a comprehensive analysis of the components introduced by the method and demonstrate that Llip leads to richer visual representations.

Powerline robotics is slowly becoming key tools for electric utilities. Contrary to drones that are usually limited to inspection tasks, wheeled robots like LineRanger can perform a broader range of applications. In this paper, a suite of mechanical devices is featured, as several new asset management tasks were recently added to LineRanger’s capabilities. While previous applications focused on non-contact inspection (visual, electro-magnetic, etc.), the new tasks at hand involved reaching adjacent conductors to probe line components with micro-Ohmmeter, installing and retrieving custom build sensors for multi-day line monitoring, and assessing aging conductors surface properties, to refine their thermal model and optimize the line capacity during heat waves. All three applications were recently field validated onto LineRanger, and mechanical design insights shall be presented for each module.

Finetuning language models with reinforcement learning (RL), e.g. from human feedback (HF), is a prominent method for alignment. But optimizing against a reward model can improve on reward while degrading performance in other areas, a phenomenon known as reward hacking, alignment tax, or language drift. First, we argue that commonly-used test metrics are insufficient and instead measure how different algorithms tradeoff between reward and drift. The standard method modified the reward with a Kullback-Lieber (KL) penalty between the online and initial model. We propose Elastic Reset, a new algorithm that achieves higher reward with less drift without explicitly modifying the training objective. We periodically reset the online model to an exponentially moving average (EMA) of itself, then reset the EMA model to the initial model. Through the use of an EMA, our model recovers quickly after resets and achieves higher reward with less drift in the same number of steps. We demonstrate that fine-tuning language models with Elastic Reset leads to state-of-the-art performance on a small scale pivot-translation benchmark, outperforms all baselines in a medium-scale RLHF-like IMDB mock sentiment task and leads to a more performant and more aligned technical QA chatbot with LLaMA-7B. Code available https://github.com/mnoukhov/elastic-reset

Compositional generalization, the ability of an agent to generalize to unseen combinations of latent factors, is easy for humans but hard for deep neural networks. A line of research in cognitive science has hypothesized a process, "iterated learning," to help explain how human language developed this ability; the theory rests on simultaneous pressures towards compressibility (when an ignorant agent learns from an informed one) and expressivity (when it uses the representation for downstream tasks). Inspired by this process, we propose to improve the compositional generalization of deep networks by using iterated learning on models with simplicial embeddings, which can approximately discretize representations. This approach is further motivated by an analysis of compositionality based on Kolmogorov complexity. We show that this combination of changes improves compositional generalization over other approaches, demonstrating these improvements both on vision tasks with well-understood latent factors and on real molecular graph prediction tasks where the latent structure is unknown.

This paper presents the design of a multirotor unmanned aerial vehicle (UAV) capable of landing semiautomatically on a power line while carrying a payload. The vehicle then rolls along the line to perform an inspection. Special attention is given to the vehicle's onboard vision system, which consists of a monocular camera and LiDAR used together to compute the pose of the vehicle relative to the power line. Landing assistance is provided to the pilot by a position-based visual controller that aligns and keeps the vehicle centered along the power line. The pilot remains in control of vertical and longitudinal movement during descent. The proposed approach was tested on a full-scale test line and shows promise for future applications of high value to the electric industry such as non-destructive testing of power transmission lines.

This paper presents a new 6-DOF serial manipulator with five revolute joints and one prismatic joint, designed to operate equipment inside Hydro-Que¿bec underground distribution vaults. The manipulator is an upgraded model developed after field testing a 6-DOF wrist-partitioned serial manipulator prototype. The new manipulator architecture is intended to solve space constraint problems in some vaults. This paper covers the geometrical model of the new manipulator, an analytical solution for the inverse kinematic equations, the dynamic model used to compute torques/forces at the actuators, and simulations performed with MATLAB and CATIA. The manipulator is the only robot application in the world designed to operate equipment on underground distribution power lines.

This video summarizes the R&D phases of the first robot prototype designed to operate and maintain equipment in vaults for Hydro-Québec's underground distribution lines, carried out between 2003 and 2009. Detailed segments discuss project's context, proof of concept, simulations, description and design of robotic platform subsystems, prototype testing at IREQ in the lab and in an outdoor vault, field testing in Montreal on a Joslyn switch in a de-energized vault, and the future of the project.