Measuring alignment between language and vision is a fundamental challenge, especially as multimodal data becomes increasingly detailed and complex. Existing methods often rely on collecting human or AI preferences, which can be costly and time-intensive. We propose an alternative approach that leverages cycle consistency as a supervisory signal. Given an image and generated text, we map the text back to image space using a text-to-image model and compute the similarity between the original image and its reconstruction. Analogously, for text-to-image generation, we measure the textual similarity between an input caption and its reconstruction through the cycle. We use the cycle consistency score to rank candidates and construct a preference dataset of 866K comparison pairs. The reward model trained on our dataset, CycleReward, outperforms state-of-the-art alignment metrics on detailed captioning, with superior inference-time scalability when used as a verifier for Best-of-N sampling, while maintaining speed and differentiability. Furthermore, performing DPO and Diffusion DPO using our dataset enhances performance across a wide range of vision-language tasks and text-to-image generation. Our dataset, model, and code are publicly released at https://cyclereward.github.io.

This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, these datasets often have limitations due to the subjects of the drawings belonging to a specific domain, or in the amount of data collected. Although recent work in unsupervised image-to-image translation has shown much progress, the latest methods still struggle to generate compelling line drawings. We observe that line drawings are encodings of scene information and seek to convey 3D shape and semantic meaning. We build these observations into a set of objectives and train an image translation to map photographs into line drawings. We introduce a geometry loss which predicts depth information from the image features of a line drawing, and a semantic loss which matches the CLIP features of a line drawing with its corresponding photograph. Our approach outperforms state-of-the-art un-paired image translation and line drawing generation methods on creating line drawings from arbitrary photographs.

Human speech is often accompanied by hand and arm gestures. We present a method for cross-modal translation from "in-the-wild" monologue speech of a single speaker to their conversational gesture motion. We train on unlabeled videos for which we only have noisy pseudo ground truth from an automatic pose detection system. Our proposed model significantly outperforms baseline methods in a quantitative comparison. To support research toward obtaining a computational understanding of the relationship between gesture and speech, we release a large video dataset of person-specific gestures.

This paper presents a simple method for “do as I do” motion transfer: given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves. We approach this problem as video-to-video translation using pose as an intermediate representation. To transfer the motion, we extract poses from the source subject and apply the learned pose-to-appearance mapping to generate the target subject. We predict two consecutive frames for temporally coherent video results and introduce a separate pipeline for realistic face synthesis. Although our method is quite simple, it produces surprisingly compelling results (see video). This motivates us to also provide a forensics tool for reliable synthetic content detection, which is able to distinguish videos synthesized by our system from real data. In addition, we release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer.

Cloud Radio Access Network (C-RAN) is a trending concept for wireless communication, driven by the increase in mobile data traffic. C-RAN enables the pooling of baseband resources across multiple cell sites. This allows the mobile operator to save both CAPEX and OPEX by statistically multiplexing their baseband resources in response to real-time network demands. The pooling of baseband resources requires a new intelligent mechanism for resource management. In this paper, we propose a pooling system for the LTE MAC layer running on a General Purpose Processor (GPP) platform. The proposed pooling system includes an application-aware task controller that replaces the resource management functionality in the Operating System (OS). In this paper, we provide the experimental results of a pooling prototype running on a single server with an Intel multi-core processor. We show the achieved improvement in CPU utilization and power consumption within the pool. We also provide a detailed design for the application-aware task controller.

A network ethnography methodology was developed to understand and explain the influence of social capital on social network structures. This approach was applied to an inter-organizational technology cooperation group to identify causes of low group participation and engagement and opportunities to increase those factors. A cluster of key actors at the core of the group was found to hold the majority of the social capital, which is viewed as both a current constraint and a future opportunity. The methodology is evaluated with a view continuing its development.