Updating diffusion models in an incremental setting would be practical in real-world applications yet computationally challenging. We present a novel learning strategy of Concept Neuron Selection (CNS), a simple yet effective approach to perform personalization in a continual learning scheme. CNS uniquely identifies neurons in diffusion models that are closely related to the target concepts. In order to mitigate catastrophic forgetting problems while preserving zero-shot text-to-image generation ability, CNS finetunes concept neurons in an incremental manner and jointly preserves knowledge learned of previous concepts. Evaluation of real-world datasets demonstrates that CNS achieves state-of-the-art performance with minimal parameter adjustments, outperforming previous methods in both single and multi-concept personalization works. CNS also achieves fusion-free operation, reducing memory storage and processing time for continual personalization.

High energy efficiency and high-capacity embedded SRAMs are essential for data-centric applications. CFET technology offers a scalable solution to reduce the SRAM cell area. However, the optimal routing of signal and power wires in CFET SRAM to achieve better performance remains unresolved. This study proposes a Design Technology Co-Optimization (DTCO) framework for CFET SRAM cells. After generating 3D cell structures through layouts, parasitic resistance, and capacitance are analyzed for 6T and 8T SRAM arrays. Compared to CFET SRAM using single-sided signal routing with backside power delivery network (BS-PDN), CFET SRAM with double-sided signal/power routing, which efficiently leverages the space above and below the CFET SRAM cells, exhibits reduced parasitic capacitance. This reduction significantly improves read delay, write delay, read power, and write power for both 6T and 8T SRAMs, making it a promising option for high-throughput data-centric applications.

The task of image outpainting extends an image beyond its boundaries with semantically plausible content. Recently, Scene Graph Transformer (SGT) introduced a transformer architecture to leverage scene graph guidance for image outpainting. Despite its success, we identified two shortcomings: (a) SGT uses a positional encoding that was originally proposed for 1D signal; (b) SGT uses a scene graph attention layer that propagates information between neighboring nodes which limited the model to learning local graph features. To address these issues, we propose incorporating Laplacian positional encoding and introducing a multiscale scene graph attention into SGT. Extensive results on MS-COCO and Visual Genome show that our proposed approach generates more plausible outpainted images with higher quality.

In this paper, we address the task of semantics-guided image outpainting, which is to complete an image by generating semantically practical content. Different from most existing image outpainting works, we approach the above task by understanding and completing image semantics at the scene graph level. In particular, we propose a novel network of Scene Graph Transformer (SGT), which is designed to take node and edge features as inputs for modeling the associated structural information. To better understand and process graph-based inputs, our SGT uniquely performs feature attention at both node and edge levels. While the former views edges as relationship regularization, the latter observes the co-occurrence of nodes for guiding the attention process. We demonstrate that, given a partial input image with its layout and scene graph, our SGT can be applied for scene graph expansion and its conversion to a complete layout. Following state-of-the-art layout-to-image conversions works, the task of image outpainting can be completed with sufficient and practical semantics introduced. Extensive experiments are conducted on the datasets of MS-COCO and Visual Genome, which quantitatively and qualitatively confirm the effectiveness of our proposed SGT and outpainting frameworks.

A shallow depth-of-field image keeps the subject in focus, and the foreground and background contexts blurred. This effect requires much larger lens apertures than those of smartphone cameras. Conventional methods acquire RGB-D images and blur image regions based on their depth. However, this approach is not suitable for reflective or transparent surfaces, or finely detailed object silhouettes, where the depth value is inaccurate or ambiguous.We present a learning-based method to synthesize the de-focus blur in shallow depth-of-field images from handheld bursts acquired with a single small aperture lens. Our deep learning model directly produces the shallow depth-of-field image, avoiding explicit depth-based blurring. The simulated aperture diameter equals the camera translation during burst acquisition. Our method does not suffer from artifacts due to inaccurate or ambiguous depth estimation, and it is well-suited to portrait photography.

Given a partial image input, image outpainting is to produce the desirable output by recovering or extending the surrounding image regions. While existing image outpainting methods achieve impressive results based on the recent advances of deep learning, they either lack the ability to extend image regions in arbitrary directions or require the filling image margins to be given in advance. To address this challenging task, we propose a unique deep learning framework for robust image outpainting, which consists of a margin prediction network and a teacher-student-based network for producing outpainted images. Our proposed model does not require image filling margins to be known beforehand, while both image appearance and perceptual feature consistencies can be jointly enforced. Our experiments quantitatively and qualitatively verify the effectiveness of our method, which is shown to perform favorably against baseline and state-of-the-art image outpainting works.

Occlusions disrupt the visualization of an object of interest, or target, in a real world scene. Video inpainting removes occlusions from a video stream by cutting out occluders and filling in with a plausible visualization of the object, but the approach is too slow for real-time performance. In this paper, we present a method for realtime occlusion removal in the visualization of a real world scene that is captured with an RGBD stream. Our pipeline segments the current RGBD frame to find the target and the occluders, searches for the best matching disoccluded view of the target in an earlier frame, computes a mapping between the target in the current frame and the target in the best matching frame, inpaints the missing pixels of the target in the current frame by resampling from the earlier frame, and visualizes the disoccluded target in the current frame. We demonstrate our method in the case of a walking human occluded by stationary or walking humans. Our method does not rely on a known 2D or 3D model of the target or of the occluders, and therefore it generalizes to other shapes. Our method runs at an interactive frame rate of 30fps.

Immersive navigation in virtual reality (VR) and augmented reality (AR) leverages physical locomotion through pose tracking of the head-mounted display. While this navigation modality is intuitive, regions of interest in the scene may suffer from occlusion and require significant viewpoint translation. Moreover, limited physical space and user mobility need to be taken into consideration. Some regions of interest may require viewpoints that are physically unreachable without less intuitive methods such as walking in-place or redirected walking. We propose a novel approach for increasing navigation efficiency in VR and AR using multiperspective visualization. Our approach samples occluded regions of interest from additional perspectives, which are integrated seamlessly into the user's perspective. This approach improves navigation efficiency by bringing simultaneously into view multiple regions of interest, allowing the user to explore more while moving less. We have conducted a user study that shows that our method brings significant performance improvement in VR and AR environments, on tasks that include tracking, matching, searching, and ambushing objects of interest.

Occlusions are a severe bottleneck for the visualization of large and complex datasets. Conventional images only show dataset elements to which there is a direct line of sight, which significantly limits the information bandwidth of the visualization. Multiperspective visualization is a powerful approach for alleviating occlusions to show more than what is visible from a single viewpoint. However, constructing and rendering multiperspective visualizations is challenging. We present a framework for designing multiperspective focus+context visualizations with great flexibility by manipulating the underlying camera model. The focus region viewpoint is adapted to alleviate occlusions. The framework supports multiperspective visualization in three scenarios. In a first scenario, the viewpoint is altered independently for individual image regions to avoid occlusions. In a second scenario, conventional input images are connected into a multiperspective image. In a third scenario, one or several data subsets of interest (i.e., targets) are visualized where they would be seen in the absence of occluders, as the user navigates or the targets move. The multiperspective images are rendered at interactive rates, leveraging the camera model's fast projection operation. We demonstrate the framework on terrain, urban, and molecular biology geometric datasets, as well as on volume rendered density datasets.

To date there is no practical means to evaluate the true word error probability (WEP) of a given turbo or LDPC code because typical decoders cannot achieve the performance of ML decoding. In this paper, we propose a viable methodology to establish tight bounds on the ML-decoding WEP for these codes through empirical simulation. Our framework centers on the efficient use of multiple-output decoding induced by receiver-generated side information, or gift. At low WEP regime, perturbed decoding can give tight bounds. In high WEP regime, due to the prohibitive complexity of perturbed decoding, we instead pursue other type of gifts. The effectiveness of various types of gifts is investigated in detail. We observe that the complexity of gift-assisted decoding is dominated by the effort to identify partial gifts that can then be further extended. Using bit values as gifts and an algorithm that maximizes the efficiency of identifying valid partial gifts, the ML bounds of turbo and LDPC codes are evaluated. At low WEP regime, our approach successfully yields the ML performance for these codes. Their WEP are shown to be very far from the sphere packing bound. At higher WEP regime, our results indicate that best-performing message-passing decoders underperform an ML decoder by at least 0.2 dB.

In this paper, we apply a nonlinear filtering algorithm based on SVD and compression-based filtering, to estimate the characteristic modes of observed gene expression data with independently identically distributed (i.i.il.) random variables of Gaussian density of zero mean. The essence of the technique is that when the proposed lossy data compression algorithm, with the allowed loss set equal to the noise strength, is applied to a noisy gene expression data, the loss and the noise tend to cancel. Then we use the estimated noise strength as a threshold to determine the effective characteristic modes for reconstructing the gene expression profiles, followed by fitting of a linear discrete-time dynamical system in which the expression values at a given time point are linear combinations of the values at a previous time point. Furthermore, the time evolution of expression values by using the translation matrix to predict future expression values was investigated. Finally the publicly available data set of yeast from microarray experiments on the synchronized cell cycle (CDC15) is given to exemplify the implementation of the proposed technique.