There have been recent efforts to learn more meaningful representations via fixed length codewords from mesh data, since a mesh serves as a complete model of underlying 3D shape compared to a point cloud. However, the mesh connectivity presents new difficulties when constructing a deep learning pipeline for meshes. Previous mesh unsupervised learning approaches typically assume category-specific templates, e.g., human face/body templates. It restricts the learned latent codes to only be meaningful for objects in a specific category, so the learned latent spaces are unable to be used across different types of objects. In this work, we present WrappingNet, the first mesh autoencoder enabling general mesh unsupervised learning over heterogeneous objects. It introduces a novel base graph in the bottleneck dedicated to representing mesh connectivity, which is shown to facilitate learning a shared latent space representing object shape. The superiority of WrappingNet mesh learning is further demonstrated via improved reconstruction quality and competitive classification compared to point cloud learning, as well as latent interpolation between meshes of different categories. The code is available at https://github.com/InterDigitalInc/WrappingNet.

A deep learning system typically suffers from a lack of reproducibility that is partially rooted in hardware or software implementation details. The irreproducibility leads to skepticism in deep learning technologies and it can hinder them from being deployed in many applications. In this work, the irreproducibility issue is analyzed where deep learning is employed in compression systems while the encoding and decoding may be run on devices from different manufacturers. The decoding process can even crash due to a single bit difference, e.g., in a learning-based entropy coder. For a given deep learning-based module with limited resources for protection, we first suggest that reproducibility can only be assured when the mismatches are bounded. Then a safeguarding mechanism is proposed to tackle the challenges. The proposed method may be applied for different levels of protection either at the reconstruction level or at a selected decoding level. Furthermore, the overhead introduced for the protection can be scaled down accordingly when the error bound is being suppressed. Experiments demonstrate the effectiveness of the proposed approach for learning-based compression systems, e.g., in image compression and point cloud compression.

With the advent of new 3D scanning technologies, point clouds have become a crucial way to depict real and virtual objects/scenes. Point clouds represent the continuous sur-faces of underlying object/scene through a collection (usually millions) of discrete, irregular, and often sparsely distributed 3D samples on the surface of the objects, e.g. LiDAR scans. This nature of point cloud data presents a considerable challenge to not only store but also understand and extract the topology of object(s) from the point cloud data. In this regard, our work presents a point cloud compression procedure that leverages sparse 3D convolutions to extract features at various octree scales for lossless compression of octree representation of point clouds. For hierarchical flow of information between octree levels, our proposed method named SparseContextNet (SCN) also propagates features from a lower resolution scale to higher resolution scale via 3D upsampling convolutions. Our experiments with LiDAR datasets reveal competitive performance of our proposal compared to the state-of-the-art.

Deep neural networks have been recently applied to point cloud compression (PCC). The features extracted via deep neural networks are essential for compression performance. Different from high level tasks such as point cloud classification or segmentation which homogenizes descriptors within same classes, PCC requires low level features discriminative for point-level 3D reconstructions. With this motivation, we first adopt Gaussian distribution to model the shape of feature elements. Then, we propose a deep distribution-aware network (DDA-Net) which manipulates distributions of feature elements on-the-fly to favor the point cloud reconstruction with high fidelity. Moreover, a residual network is integrated to enhance the modification of the Gaussian models. The proposed DDA-Net is incorporated into an end-to-end PCC system. Experimental results show that our DDA-Net significantly improves the compression performance across a wide range of point clouds.

Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. Conventionally, scene flow is estimated from dense/regular RGB video frames. With the development of depth-sensing technologies, precise 3D measurements are available via point clouds which have sparked new research in 3D scene flow. Nevertheless, it remains challenging to extract scene flow from point clouds due to the sparsity and irregularity in typical point cloud sampling patterns. One major issue related to irregular sampling is identified as the randomness during point set abstraction/feature extraction—an elementary process in many flow estimation scenarios. A novel Spatial Abstraction with Attention (SA2) layer is accordingly proposed to alleviate the unstable abstraction problem. Moreover, a Temporal Abstraction with Attention (TA2) layer is proposed to rectify attention in temporal domain, leading to benefits with motions scaled in a larger range. Extensive analysis and experiments verified the motivation and significant performance gains of our method, dubbed as Flow Estimation via Spatial-Temporal Attention (FESTA), when compared to several state-of-the-art benchmarks of scene flow estimation.

We study the problem of reconstructing an object from phaseless measurements in the context of inverse multiple scattering. Our formulation explicitly decouples the variables that represent the unknown object image and the unknown phase, respectively, in the forward model. This enables us to simultaneously optimize over both unknowns with appropriate regularization for each. The resulting optimization problem is nonconvex due to the nonlinear propagation model for multiple scattering and the nonconvex regularization of the phase variables. Nevertheless, we demonstrate experimentally that we can solve the optimization problem using a variation of the fast iterative shrinkage-thresholding algorithm (FISTA)-a convex algorithm, popular for its speed and simplicity-that converges well in our experiments. Numerical results with both simulated and experimentally measured data show that the proposed method outperforms the state-of-the-art phaseless inverse scattering method.

In this paper we develop an optimization-based solution to the problem of distributed radar imaging using antennas with asynchronous clocks. In particular, we consider a distributed radar imaging MIMO system observing a sparse scene under an unknown, but bounded, delay between the transmitter and receiver clocks. Most existing approaches pose the problem as the recovery of a phase shift, leading to non-convex formulations. Instead, inspired by recent work in blind deconvolution, we exploit the realization that synchronization errors in the received data can be modeled as a convolution with an unknown 1-sparse delay signal to be estimated in addition to the image. Thus, we formulate a convex optimization problem that simultaneously recovers all the pair-wise drifts between transmit/receive pairs, as well as the sparse scene being imaged. We verify the validity and performance of our proposed model and recovery method through numerical simulations on synthetic data.