Artificial Intelligence Generated Content (AIGC) models typically have hundreds of billions of parameters, and developers experience prohibitively long pulling time from a central model registry to their local environments. Peer-to-peer (P2P)-enabled model distribution that pulls models from local peers within a cluster instead of the central model registry is emerging as a promising technique to reduce model pulling time. Nonetheless, such model distribution systems have to handle bursty concurrent pulling tasks. This may occupy the network bandwidth of some peers for a long time, thereby making the peers unable to respond to further pulling tasks. We thus aim to design a lightweight and scalable model distribution system to balance the network resource usage of peers, by proposing learning-driven algorithms to accurately predict network status between peers and implementing the design in real production environments. Specifically, we first propose a lightweight network measurement mechanism that combines active delay probing and passive bandwidth inference with low resource overhead. We also propose a learning-driven task scheduling algorithm based on a structural graph representation with a varied-multi-hop attention mechanism, to predict bursty patterns of concurrent pulling tasks. We then design an asynchronous model training and inference method to enable seamless incremental learning based on the dynamic network status data. We finally implement our system design and the learning-driven algorithm in a Cloud Native Computing Foundation (CNCF) projectDragonflythat has already been publicly released since its version$v2.1.0$. Real experiments in the Ant Group’s production environment show that our system reduces the total completion time by at least 10% and increases the average bandwidth utilization of peers by 20%, compared with mainstream systems and algorithms.

In this paper, we present RigAnyFace (RAF), a scalable neural auto-rigging framework for facial meshes of diverse topologies, including those with multiple disconnected components. RAF deforms a static neutral facial mesh into industry-standard FACS poses to form an expressive blendshape rig. Deformations are predicted by a triangulation-agnostic surface learning network augmented with our tailored architecture design to condition on FACS parameters and efficiently process disconnected components. For training, we curated a dataset of facial meshes, with a subset meticulously rigged by professional artists to serve as accurate 3D ground truth for deformation supervision. Due to the high cost of manual rigging, this subset is limited in size, constraining the generalization ability of models trained exclusively on it. To address this, we design a 2D supervision strategy for unlabeled neutral meshes without rigs. This strategy increases data diversity and allows for scaled training, thereby enhancing the generalization ability of models trained on this augmented data. Extensive experiments demonstrate that RAF is able to rig meshes of diverse topologies on not only our artist-crafted assets but also in-the-wild samples, outperforming previous works in accuracy and generalizability. Moreover, our method advances beyond prior work by supporting multiple disconnected components, such as eyeballs, for more detailed expression animation.

We propose a novel method of foveated rendering for virtual reality, targeting head-mounted displays with large fields of view or high pixel densities. Our foveation method removes motion-induced flicker in the periphery by aligning the rendered pixel grid to the virtual scene content during rasterization and upsampling. This method dramatically reduces detectability of motion artifacts in the periphery without complex interpolation or anti-aliasing algorithms.

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form, that is appropriate for the application. The need to design and optimize these pipelines is time-consuming and costly. We explain a method that combines machine learning and image systems simulation that automates the pipeline design. The approach is based on a new way of thinking of the image processing pipeline as a large collection of local linear filters. We illustrate how the method has been used to design pipelines for novel sensor architectures in consumer photography applications.