Classical Federated Learning (FL) encounters significant challenges when deploying large models on power-constrained clients. To tackle this, we propose an asymmetric FL mechanism that enables the aggregation of compact client models into a comprehensive server model. We design the server model as a Mixture-of-Experts (MoE), where each expert has the same architecture as each client model. This uniformity allows for efficient fusion of the most pertinent client models to update each server expert, based on the measured relevance between each client and server expert. To address the Non-IID data issue, we further optimize the server-side MoE architecture by incorporating a main expert that always activates alongside a set of selectively activated routed experts. This configuration ensures a balance between learning general knowledge and specific data distribution. Our Fed-MoE framework is model-agnostic and has demonstrated notable improvements on vision FL tasks with million-scale ResNet backbones, and language tasks with billion-scale BERT and GPT-2 backbones.

This study focuses on multi-human relief modeling using a single photograph. Although previous studies successfully modeled 3D humans from single photographs, they were limited to reconstructing 3D individuals and could not be applied to multi-human scenes with complex inter-body and outer-body occlusions. In this study, we introduce MMRelief, a novel solution that takes a significant step toward high-quality and generalized multi-human relief modeling. MMRelief uses a three-step approach to achieve its objectives. First, it predicts an occlusion-aware depth map based on ZoeDepth [12]. Subsequently, it predicts a detailed normal map using a photo-to-normal network. Finally, MMRelief combines the strengths of both maps and constructs human relief using depth-constrained normal integration. Experimental results demonstrate that MMRelief has achieved state-of-the-art performance in normal human estimation. It can handle different styles of human photos with varying poses and dresses while producing reliefs with accurate body occlusions, reasonable depth ordering, and faithful geometrical details. The project page is at https://github.com/yanqingliu3856/MMRelief.

In this article, we introduce MonoRelief, a novel method that combines the strengths of a depth map and a normal map to achieve high-quality relief recovery from a single image. By constructing a large-scale relief dataset that encompasses a diverse range of relief shapes, materials, and lighting conditions, we enable the training of a robust normal estimation network capable of handling various types of relief images. Furthermore, we leverage the state-of-the-art method, DepthAnything v2 (Yang et al. 2024), to generate depth maps from the input images. By integrating the strengths of both maps, MonoRelief recovers 2.5D reliefs with reasonable depth structures and intricate geometrical details. We validate the effectiveness and robustness of MonoRelief through comprehensive experiments, and showcase its potential in a variety of downstream applications, including Image-to-Relief, Text-to-Relief, Lines-to-Relief and relief reproduction.

In this paper, we present an end-to-end neural solution to model portrait bas-relief from a single photograph, which is cast as a problem of image-to-depth translation. The main challenge is the lack of bas-relief data for network training. To solve this problem, we propose a semi-automatic pipeline to synthesize bas-relief samples. The main idea is to first construct normal maps from photos, and then generate bas-relief samples by reconstructing pixel-wise depths. In total, our synthetic dataset contains 23 k pixel-wise photo/bas-relief pairs. Since the process of bas-relief synthesis requires a certain amount of user interactions, we propose end-to-end solutions with various network architectures, and train them on the synthetic data. We select the one that gave the best results through qualitative and quantitative comparisons. Experiments on numerous portrait photos, comparisons with state-of-the-art methods and evaluations by artists have proven the effectiveness and efficiency of the selected network.

Abstract Different from other types of bas‐reliefs, a flower bas‐relief contains a large number of depth‐discontinuity edges. Most existing line‐based methods reconstruct free‐form surfaces by ignoring the depth‐discontinuities, thus are less efficient in modeling flower bas‐reliefs. This paper presents a neural‐based solution which benefits from the recent advances in CNN. Specially, we use line gradients to encode the depth orderings at leaf edges. Given a line drawing, a heuristic method is first proposed to compute 2D gradients at lines. Line gradients and dense curvatures interpolated from sparse user inputs are then fed into a neural network, which outputs depths and normals of the final bas‐relief. In addition, we introduce an object‐based method to generate flower bas‐reliefs and line drawings for network training. Extensive experiments show that our method is effective in modelling bas‐reliefs with depth‐discontinuity edges. User evaluation also shows that our method is intuitive and accessible to common users.

Abstract In contrast to 3D model that can be freely observed, p ortrait bas‐relief projects slightly from the background and is limited by fixed viewpoint. In this paper, we propose a novel method to reconstruct the underlying 3D shape from a single 2.5D bas‐relief, providing observers wider viewing perspectives. Our target is to make the reconstructed portrait has natural depth ordering and similar appearance to the input. To achieve this, we first use a 3D template face to fit the portrait. Then, we optimize the face shape by normal transfer and Poisson surface reconstruction. The hair and body regions are finally reconstructed and combined with the 3D face. From the resulting 3D shape, one can generate new reliefs with varying poses and thickness, freeing the input one from fixed view. A number of experimental results verify the effectiveness of our method.

We present a novel solution to enable portrait relief modeling from a single image. The main challenges are geometry reconstruction, facial details recovery and depth structure preservation. Previous image-based methods are developed for portrait bas-relief modeling in 2.5D form, but not adequate for 3D-like high relief modeling with undercut features. In this paper, we propose a template-based framework to generate portrait reliefs of various forms. Our method benefits from Shape-from-Shading (SFS). Specifically, we use bi-Laplacian mesh deformation to guide the relief modeling. Given a portrait image, we first use a template face to fit the portrait. We then apply bi-Laplacian mesh deformation to align the facial features. Afterwards, SFS-based reconstruction with a few user interactions is used to optimize the face depth, and create a relief with similar appearance to the input. Both depth structures and geometric details can be well constructed in the final relief. Experiments and comparisons to other methods demonstrate the effectiveness of the proposed method.

Abstract Bas‐relief is designed to provide 3D perception for the viewers under illumination. For the problem of bas‐relief generation from 3D object, most existing methods ignore the influence of illumination on bas‐relief appearance. In this paper, we propose a novel method that adaptively generate bas‐reliefs with respect to illumination conditions. Given a 3D object and its target appearance, our method finds an adaptive surface that preserves the appearance of the input. We validate our approach through a variety of applications. Experimental results indicate that the proposed approach is effective in producing bas‐reliefs with desired appearance under illumination.