Reconstructing precise camera poses and floor plan layouts from wide-baseline RGB panoramas is a difficult and unsolved problem. We introduce BADGR, a novel diffusion model that jointly performs reconstruction and bundle adjustment (BA) to refine poses and layouts from a coarse state, using 1D floor boundary predictions from dozens of sparsely captured images. Unlike guided diffusion models, BADGR is conditioned on dense per-column outputs from a single-step Levenberg Marquardt (LM) optimizer and is trained to predict camera and wall positions, while minimizing reprojection errors for view consistency. The objective of layout generation from denoising diffusion process complements BA optimization by providing additional learned layout-structural constraints on top of the co-visible features across images. These constraints help BADGR make plausible guesses about spatial relationships, which constrain the pose graph, such as wall adjacency and collinearity, while also learning to mitigate errors from dense boundary observations using global context. BADGR trains exclusively on 2D floor plans, simplifying data acquisition, enabling robust augmentation, and supporting a variety of input densities. Our experiments validate our method, which significantly outperforms the state-of-the-art pose and floor plan layout reconstruction with different input densities. Visit project website at: https://badgr-diffusion.github.io.

MOTIVATION: Protein-protein interactions play a fundamental role in biological systems. Accurate detection of protein-protein interaction sites (PPIs) remains a challenge. And, the methods of PPIs prediction based on biological experiments are expensive. Recently, a lot of computation-based methods have been developed and made great progress. However, current computational methods only focus on one form of protein, using only protein spatial conformation or primary sequence. And, the protein's natural hierarchical structure is ignored. RESULTS: In this study, we propose a novel network architecture, HSSPPI, through hierarchical and spatial-sequential modeling of protein for PPIs prediction. In this network, we represent protein as a hierarchical graph, in which a node in the protein is a residue (residue-level graph) and a node in the residue is an atom (atom-level graph). Moreover, we design a spatial-sequential block for capturing complex interaction relationships from spatial and sequential forms of protein. We evaluate HSSPPI on public benchmark datasets and the predicting results outperform the comparative models. This indicates the effectiveness of hierarchical protein modeling and also illustrates that HSSPPI has a strong feature extraction ability by considering spatial and sequential information simultaneously. AVAILABILITY AND IMPLEMENTATION: The code of HSSPPI is available at https://github.com/biolushuai/Hierarchical-Spatial-Sequential-Modeling-of-Protein.

The ill-posed nature of monocular 3D geometry (depth map and surface normals) estimation makes it rely mostly on data-driven approaches such as Deep Neural Networks (DNN). However, data acquisition of surface normals, especially the reliable normals, is acknowledged difficult. Commonly, reconstruction of surface normals with high quality is heuristic and time-consuming. Such fact urges methodologies to minimize dependency on ground-truth normals when predicting 3D geometry. In this work, we devise CO-planarity REgularized (CORE) loss functions and Structure-Aware Normal Estimator (SANE). Without involving any knowledge of ground-truth normals, these two designs enable pixel-wise 3D geometry estimation weakly supervised by only ground-truth depth map. For CORE loss functions, the key idea is to exploit locally linear depth-normal orthogonality under spherical coordinates as pixel-level constraints, and utilize our designed Adaptive Polar Regularization (APR) to resolve underlying numerical degeneracies. Meanwhile, SANE easily establishes multi-task learning with CORE loss functions on both depth and surface normal estimation, leading to the whole performance leap. Extensive experiments present the effectiveness of our method on various DNN architectures and data benchmarks. The experimental results demonstrate that our depth estimation achieves the state-of-the-art performance across all metrics on indoor scenes and comparable performance on outdoor scenes. In addition, our surface normal estimation is overall superior.

Proteins usually perform their cellular functions by interacting with other proteins. Accurate identification of protein-protein interaction sites (PPIs) from sequence is import for designing new drugs and developing novel therapeutics. A lot of computational models for PPIs prediction have been developed because experimental methods are slow and expensive. Most models employ a sliding window approach in which local neighbors are concatenated to present a target residue. However, those neighbors are not distinguished by pairwise information between a neighbor and the target. In this study, we propose a novel PPIs prediction model AttCNNPPISP, which combines attention mechanism and convolutional neural networks (CNNs). The attention mechanism dynamically captures the pairwise correlation of each neighbor-target pair within a sliding window, and therefore makes a better understanding of the local environment of target residue. And then, CNNs take the local representation as input to make prediction. Experiments are employed on several public benchmark datasets. Compared with the state-of-the-art models, AttCNNPPISP improves the prediction performance. Also, the experimental results demonstrate that the attention mechanism is effective in terms of constructing comprehensive context information of target residue.

In this paper, we propose a new deep learning-based method for estimating room layout given a pair of 360° panoramas. Our system, called Position-aware Stereo Merging Network or PSMNet, is an end-to-end joint layout-pose estimator. PSMNet consists of a Stereo Pano Pose (SP2) transformer and a novel Cross-Perspective Projection (CP2) layer. The stereo-view SP2 transformer is used to implicitly infer correspondences between views, and can handle noisy poses. The pose-aware CP2layer is designed to render features from the adjacent view to the anchor (reference) view, in order to perform view fusion and estimate the visible layout. Our experiments and analysis validate our method, which significantly outperforms the state-of-the-art layout estimators, especially for large and complex room spaces.

Supervised learning and self-supervised learning address different facets. Supervised learning achieves high accuracy, but it requires numerous expensive labeled data indeed. Correspondingly, self-supervised learning, makes use of abundant unlabeled data to learn, but the performance lags behind that of the supervised counterpart. To overcome the difficulty of acquiring annotated data and contain the high performance in the context of speaker verification, we propose in this work a self-supervised joint learning (SS-JL) framework which complements the supervised main task with self-supervised auxiliary tasks in joint training. These auxiliary tasks help the speaker verification pipeline to generate robust speaker representation that is closely relevant to voiceprints. Our model is trained on English dataset and tested on multilingual datasets, including English, Chinese and Korean datasets, and 13.6%, 12.7% and 13.5% improvement is achieved respectively in terms of equal error rate (EER) compared with the baselines.

Antibodies consisting of variable and constant regions, are a special type of proteins playing a vital role in immune system of the vertebrate. They have the remarkable ability to bind a large range of diverse antigens with extraordinary affinity and specificity. This malleability of binding makes antibodies an important class of biological drugs and biomarkers. In this article, we propose a method to identify which amino acid residues of an antibody directly interact with its associated antigen based on the features from sequence and structure. Our algorithm uses convolution neural networks (CNNs) linked with graph convolution networks (GCNs) to make use of information from both sequential and spatial neighbors to understand more about the local environment of target amino acid residue. Furthermore, we process the antigen partner of an antibody by employing an attention layer. Our method improves on the state-of-the-art methodology.

Protein-protein interactions are of great importance in the life cycles of living cells. Accurate prediction of the proteinprotein interaction site (PPIs) from protein sequence improves our understanding of protein-protein interaction, contributes to the protein-protein docking and is crucial for drug design. However, practical experimental methods are costly and time-consuming so that many sequence-based computational methods have been developed. Most of those methods employ a sliding window approach, which utilize local neighbor information within a window size. However, they don’t distinguish and use the effect of each individual neighboring residue at different position. We propose a novel sequence-based deep learning method consisting of convolutional neural networks (CNNs) and attention mechanism to improve the performance of PPIs prediction. Our attention-based CNNs captures the different effect of each neighboring residue within a sliding window, and therefore making a better understanding of the local environment of target residue. We employ experiments on several public benchmark datasets. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art techniques. The source code can be obtained from https://github.com/biolushuai/attention-based-CNNsfor-PPIs-prediction.

We present Zillow Indoor Dataset (ZInD): A large indoor dataset with 71,474 panoramas from 1,524 real unfurnished homes. ZInD provides annotations of 3D room layouts, 2D and 3D floor plans, panorama location in the floor plan, and locations of windows and doors. The ground truth construction took over 1,500 hours of annotation work. To the best of our knowledge, ZInD is the largest real dataset with layout annotations. A unique property is the room layout data, which follows a real world distribution (cuboid, more general Manhattan, and non-Manhattan layouts) as opposed to the mostly cuboid or Manhattan layouts in current publicly available datasets. Also, the scale and annotations provided are valuable for effective research related to room layout and floor plan analysis. To demonstrate ZInD’s benefits, we benchmark on room layout estimation from single panoramas and multi-view registration.

Secure masking schemes have been proven in theory to be secure countermeasures against side-channel attacks. The security framework proposed by Ishai, Sahai and Wagner, known as the Ishai-Sahai-Wagner scheme, is one of the most acceptable secure models of the existing dth-order masking schemes, where d represents the masking order and plays the role of a security parameter. However, a gap may exist between scheme and design. Several analyses have determined that the glitch has been regarded as the main challenge of masking in hardware designs. A practical method of locating the precise position of leakage points (LPs) in the original hardware design is very rare. Existing research on this glitch mainly focuses on the first-order leakages; however, higher-order analysis can combine several shares to recover the secret key. In this paper, we propose a practical method, sensitive glitch location (SGL) method to locate the less order leakage in hardware design. Specifically, the SGL method can locate any-order of LP in the hardware implementation of dth-order masking schemes. We conducted experiments and verified that the time complexity of SGL on the dth-order masking schemes is O(nm), where m is the number of signals and n is the number of shares in masking scheme. It can therefore be regarded as an efficient tool for the masking designs. In addition, we analyzed the dth-order masking scheme proposed by Rivain and Prouff (2010) along with the SecMult algorithm from the Rivain-Prouff countermeasure, which has been analyzed by our SGL. The experimental results verified that a higher-order leakage may exist in certain hardware designs, even the masking scheme has been proven as a secure countermeasure. To the best of our knowledge, SGL is the first tool that can be used to locate any-order of power/electromagnetic LP in hardware designs. It thus shows the weakness in the original design file of hardware implementations. This property can help designers directly improve the real security of the designs. Moreover, SGL returns the path of the leakages, which can elucidate the original cause and propagation of the weakness.

This paper introduces two acceleration methods for the Monte Carlo (MC) simulation of photons on Graphic Processing Units (GPUs). The GPU-based methods for performing MC method enable the use of the model in more complex vegetation canopy. We describe the computation and how it is mapped onto the many parallel computational units on one NVIDIA GT240M graphic card. For a vegetation canopy consisting of 13020 polygons, the speedup is 16.45× over the code on an Intel T6400 dual-core processor. And this method can be accelerated tremendously by employing better GPU processor, or using more GPUs for simulation.