A spatial AI that can perform complex tasks through visual signals and cooperate with humans is highly anticipated. To achieve this, we need a visual SLAM that easily adapts to new scenes without pre-training and generates dense maps for downstream tasks in real-time. None of the previous learning-based and non-learning-based visual SLAMs satisfy all needs due to the intrinsic limitations of their components. In this work, we develop a visual SLAM named Orbeez-SLAM, which successfully collaborates with implicit neural representation and visual odometry to achieve our goals. Moreover, Orbeez-SLAM can work with the monocular camera since it only needs RGB inputs, making it widely applicable to the real world. Results show that our SLAM is up to 800x faster than the strong baseline with superior rendering outcomes. Code link: https://github.com/MarvinChung/Orbeez-SLAM.

Point cloud registration is a crucial problem in computer vision and robotics. Existing methods either rely on matching local geometric features, which are sensitive to the pose differences, or leverage global shapes, which leads to inconsistency when facing distribution variances such as partial overlapping. Combining the advantages of both types of methods, we adopt a coarse-to-fine pipeline that concurrently handles both issues. We first reduce the pose differences between input point clouds by aligning global features; then we match the local features to further refine the inaccurate alignments resulting from distribution variances. As global feature alignment requires the features to preserve the poses of input point clouds and local feature matching expects the features to be invariant to these poses, we propose an SE(3)-equivariant feature extractor to simultaneously generate two types of features. In this feature extractor, representations that preserve the poses are first encoded by our novel SE(3)-equivariant network and then converted into pose-invariant ones by a pose-detaching module. Experiments demonstrate that our proposed method increases the recall rate by 20% compared to state-of-the-art methods when facing both pose differences and distribution variances.

To achieve accurate 3D object detection at a low cost for autonomous driving, many multi-camera methods have been proposed and solved the occlusion problem of monocular approaches. However, due to the lack of accurate estimated depth, existing multi-camera methods often generate multiple bounding boxes along a ray of depth direction for difficult small objects such as pedestrians, resulting in an extremely low recall. Furthermore, directly applying depth prediction modules to existing multi-camera methods, generally composed of large network architectures, cannot meet the real-time requirements of self-driving applications. To address these issues, we propose Cross-view and Depth-guided Transformers for 3D Object Detection, CrossDTR. First, our lightweight depth predictor is designed to produce precise object-wise sparse depth maps and low-dimensional depth embeddings without extra depth datasets during supervision. Second, a cross-view depth-guided transformer is developed to fuse the depth embeddings as well as image features from cameras of different views and generate 3D bounding boxes. Extensive experiments demonstrated that our method hugely surpassed existing multi-camera methods by 10 percent in pedestrian detection and about 3 percent in overall mAP and NDS metrics. Also, computational analyses showed that our method is 5 times faster than prior approaches. Our codes will be made publicly available at https://github.com/sty61010/CrossDTR.

While recent progress has significantly boosted few-shot classification (FSC) performance, few-shot object detection (FSOD) remains challenging for modern learning systems. Existing FSOD systems follow FSC approaches, ignoring critical issues such as spatial variability and uncertain representations, and consequently result in low performance. Observing this, we propose a novelDual-Awareness Attention (DAnA)mechanism that enables networks to adaptively interpret the given support images. DAnA transforms support images intoquery-position-aware(QPA) features, guiding detection networks precisely by assigning customized support information to each local region of the query. In addition, the proposed DAnA component is flexible and adaptable to multiple existing object detection frameworks. By adopting DAnA, conventional object detection networks, Faster R-CNN and RetinaNet, which are not designed explicitly for few-shot learning, reach state-of-the-art performance in FSOD tasks. In comparison with previous methods, our model significantly increases the performance by 47% (+6.9 AP), showing remarkable ability under various evaluation settings.

Ke-Jyun Wang, Yun-Hsuan Liu, Hung-Ting Su, Jen-Wei Wang, Yu-Siang Wang, Winston Hsu, Wen-Chin Chen. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Object counting which aims to calculate the number of total instances of the given class is a classic but crucial task that can be applied to many applications. Most of the prior works only focus on counting certain classes of objects such as people, cars, animals, etc. However, in recent years, there are lots of applications that need to get the count of the unseen class of objects such as a mechanical arm commanded to grab the novel object. In this paper, we present an effective object counting network, Class-agnostic Few-shot Object Counting Network (CFOCNet), that supports counting arbitrary classes of object unseen during training stage. Instead of counting a pre-defined class, our model is able to count instances based on input reference images and reduces the huge cost of data collection, training and parameter tuning for each new object class. Our model utilizes not only the similarity between query image and reference images but self attending the query image to learn the self-repeatedness. Using a two-stream Resnet that matches features in different scales, our network can automatically learn to aggregate different scales of the matching scores. We evaluate our method on the subset of the COCO dataset that contains 80 classes of objects and many diverse scenes. In the experiments, our network outperforms other methods including detection and some previous works by a large margin. To the best of our knowledge, we are the first that mainly focuses on few-shot object counting in the class-agnostic manner.

Removing shadows in document images enhances both the visual quality and readability of digital copies of documents. Most existing shadow removal algorithms for document images use hand-crafted heuristics and are often not robust to documents with different characteristics. This paper proposes the Background Estimation Document Shadow Removal Network (BEDSR-Net), the first deep network specifically designed for document image shadow removal. For taking advantage of specific properties of document images, a background estimation module is designed for extracting the global background color of the document. During the process of estimating the background color, the module also learns information about the spatial distribution of background and non-background pixels. We encode such information into an attention map. With the estimated global background color and attention map, the shadow removal network can better recover the shadow-free image. We also show that the model trained on synthetic images remains effective for real photos, and provide a large set of synthetic shadow images of documents along with their corresponding shadow-free images and shadow masks. Extensive quantitative and qualitative experiments on several benchmarks show that the BEDSR-Net outperforms existing methods in enhancing both the visual quality and readability of document images.

With the popularity of social media nowadays, tons of photos are uploaded everyday. To understand the image content, image classification becomes a very essential technique for plenty of applications (e.g., object detection, image caption generation). Convolutional Neural Network (CNN) has been shown as the state-of-the-art approach for image classification. However, one of the characteristics in social media photos is that they are often applied with photo filters, especially on Instagram. We find that prior works do not aware of this trend in social media photos and fail on filtered images. Thus, we propose a novel CNN architecture that utilizes the power of pairwise constraint by combining Siamese network and the proposed adaptive margin contrastive loss with our discriminative pair sampling method to solve the problem of filter bias. To the best of our knowledge, this is the first work to tackle filter bias on CNN and achieve state-of-the-art performance on a filtered subset of ILSVRC2012.

The Communication and Multimedia Laboratory (CML), National Taiwan University, has been developing an MPEG-4 editor for 5 years since 1998. In 2001, Digimax Production Ltd joined this project and we have addressed ourselves to making the system more complete. We have implemented a visual MPEG-4 scene editor for creating 2D/3D mixed scenes, with the following features: event routing mechanism; visual editing; friendly user interface. With our system, one can easily produce highly interactive MPEG-4 content.

We present an efficient MPEG-4-based interactive player for PDA-like embedded devices in this paper. Our system can receive media from various sources, then decompress and compose them in an object-oriented manner. Embedded devices such as PDA usually have limited computational resources, memory size, power budgets, and multimedia capabilities. To overcome these constraints, two novel mechanisms were introduced, namely adaptive frame rate (AFR) and scene cache graph management (SCGM). Furthermore, a media decoding framework was proposed such that multimedia objects of different formats can be retrieved from different sources and then be decoded. This framework can be extended by add-on components. A semi-pull model was also designed for the synchronization of heterogeneous media objects. Finally, all the key modules were efficiently implemented and optimized. These modules include MPEG-4 video decoder, 2D/3D graphic engine, buffer management, script engine, and streaming service.

We propose a novel system that can real-time capture and compress the full screen of PC into a video clip. It is real-time in that it can capture to 30 frames per second under the resolution of 1600 /spl times/ 1200 with true color. One application of this system is to produce a digital presentation clip for instruction or tutorial. Moreover, as the video clips can be streamed over Internet or intranet, they can be used for remote education or training. We believe this approach is clearer and more efficient than conventional text manual or handbook. As our system only captures the differences of successive snapshots instead of every single screen, it is more efficient and produces more compact clips than other existing systems. In addition, the compression algorithm adopted in our system is also described.

The molecular binding problem, one of the most important problems in structure based drug design, can be formulated as a global energy optimization problem by using molecular mechanics. A novel computational algorithm is proposed to address the molecular binding problem. The algorithm is derived from genetic algorithms (GA) plus simulated annealing (SA) hybrid techniques, namely population based annealing genetic algorithms (PAG). We have applied the algorithm to find binding structures for three drug protein molecular pairs. One of the three drugs is an anti cancer drug methotrexate (MTX) and the other two are analogues of the antibacterial drug trimethoprim. Moreover, we have also studied two other well resolved ligand receptor molecular complex which are obtained from the Protein Data Bank (PDB): Thermolysin-HONH-benzylmalonyl-L-Ala-Gly-p-nitroanilide complex (5tln) and HIV-1 protease-Hydroxyethylene isostere inhibitor complex. Hydroxyethylene isostere inhibitor is one of new potential HIV-1 protease inhibitors synthesized. Through our experiments, all of the binding results not only keep the energy at low levels, but also have a promising binding geometrical structure in terms of number of hydrogen bonds formed.

Molecular binding, important in drug design, explores the accurate binding structures between molecules. This exploration can be formulated as a global optimization problem. However, the problem in molecular binding is that the search space is very large and the computational cost increases tremendously with the growth of the degrees of freedom. In this paper, we utilize a new algorithm called the annealing genetic algorithm to solve the global optimization problem in molecular binding. Using a protein with three anti-cancer drugs in our model, our algorithm can find a binding structure with a complicated energy computation within a couple of hours and the experimental results indicate that the solutions are reasonable.>

The average complexity for searching a record in a sorted file of records that are stored on a tape is analyzed for four search algorithms, namely, sequential search, binary search, Fibonacci search, and a modified version of Fibonacci search. The theoretical results are consistent with the recent simulation results by S. Nishihara and N. Nishino (1987). The results show that sequential search, Fibonacci search, and modified Fibonacci search are all better than binary search on a tape.>

We show how to transform the B-spline curve and surface fitting problems into suffix computations of continued fractions. Then a parallel substitution scheme is introduced to compute the suffix values on a newly proposed mesh-of-unshuffle network. The derived parallel algorithm allows the curve interpolation through n points to be solved in O(logn) time using Θ(n/log n processors and allows the surface interpolation through m × n points to be solved in O(log m log n) time using Θ(mn/(log m log n)) processors. Both interpolation algorithms are cost-optimal for their respective problems. Besides, the surface fitting problem can be even faster solved in O(log m + log n) time if Θ(mn) processors are used in the network.

We present efficient deletion algorithms for three variants of coalesced chaining – late insertion (LICH), early insertion (EICH), and varied insertion (VICH). Our approach is uniform in the sense that each deletion algorithm works simultaneously for all three variants, though the implementation details are of course different. Deletion algorithms for coalesced hashing when there is a cellar have not been studied previously in the literature; these algorithms are useful because coalesced hashing is most efficient when a cellar is utilised. First we present and analyse a deletion algorithm that preserves randomness – in that deleting a record is in some sense like never having inserted it. In particular, the formulas for the average search times after N random insertions intermixed with d random deletions are the same as the formulas for the average search times after N-d random insertions. This answers an open question in the literature. We then present two deletion algorithms that require fewer pointer fields per table slot; the latter one does not relocate records once inserted. These two algorithms do not preserve randomness, but simulations suggest that search times remain good after repeated deletions and insertions.

Direct chaining is a popular and efficient class of hashing algorithms. In this paper we study optimum algorithms among direct chaining methods, under the restrictions that the records in the hash table are not moved after they are inserted, that for each chain the relative ordering of the records in the chain does not change after more insertions, and that only one link field is used per table slot. The varied-insertion coalesced hashing method (VICH), which is proposed and analyzed in [CV84], is conjectured to be optimum among all direct chaining algorithms in this class. We give strong evidence in favor of the conjecture by showing that VICH is optimum under fairly general conditions.

The coalesced hashing method has been shown to be very fast for dynamic information storage and retrieval. This paper analyzes in a uniform way the performance of coalesced hashing and its variants, thus settling some open questions in the literature. In all the variants, the range of the hash function is called the address region , and extra space reserved for storing colliders is called the cellar . We refer to the unmodified method, which was analyzed previously, as late-insertion coalesced hashing. In this paper we analyze late insertion and two new variations called early insertion and varied insertion . When there is no cellar, the early-insertion method is better than late insertion; however, past experience has indicated that it might be worse when there is a cellar. Our analysis confirms that it is worse. The varied-insertion method was introduced as a means of combining the advantages of late insertion and early insertion. This paper shows that varied insertion requires fewer probes per search, on the average, than do the other variants. Each of these three coalesced hashing methods has a parameter that relates the sizes of the address region and the cellar. Techniques in this paper are designed for tuning the parameter in order to achieve optimum search times. We conclude with a list of open problems.

This paper analyzes the early-insertion standard coalesced hashing method (EISCH), which is a variant of the standard coalesced hashing algorithm (SCH) described in [Knu73], [Vit80] and [Vit82b]. The analysis answers the open problem posed in [Vit80]. The number of probes per successful search in full tables is 5% better with EISCH than with SCH.