Monocular 3D object detection (Mono3D) holds noteworthy promise for autonomous driving applications owing to the cost-effectiveness and rich visual context of monocular camera sensors. However, depth ambiguity poses a significant challenge, as it requires extracting precise 3D scene geometry from a single image, resulting in suboptimal performance when transferring knowledge from a LiDARbased teacher model to a camera-based student model. To facilitate effective distillation, we introduce Monocular Teaching Assistant Knowledge Distillation (MonoTAKD), which proposes a camera-based teaching assistant (TA) model to transfer robust 3D visual knowledge to the student model, leveraging the smaller feature representation gap. Additionally, we define 3D spatial cues as residual features that capture the differences between the teacher and the TA models. We then leverage these cues to improve the student model's 3D perception capabilities. Experimental results show that our MonoTAKD achieves state-of-the-art performance on the KITTI3D dataset. Furthermore, we evaluate the performance on nuScenes and KITTI raw datasets to demonstrate the generalization of our model to multi-view 3D and unsupervised data settings. Our code is available at https://github.com/hoiliu-0801/MonoTAKD.

Robust perception is a vital component for ensuring safe autonomous driving. Automotive radar (77 to 81 GHz) offering weather-resilient sensing provides a complementary capability to the vision-or LiDAR-based autonomous driving systems. Raw radio-frequency (RF) radar tensors contain rich spatiotemporal semantics besides 3D location information. Most previous methods take in 3D (Doppler-range-azimuth) RF radar tensors, allowing prediction of an object’s location, heading angle, and size in bird’s-eye-view (BEV). However, they lack the ability to simultaneously infer objects’ size, orientation, and identity in the 3D space. To overcome this limitation, we propose a joint architecture, called CenterRadarNet, designed to facilitate high-resolution representation learning from 4D (Doppler-range-azimuth-elevation) radar data for 3D object detection and re-identification (reID) tasks. Moreover, we build an online tracker utilizing the learned appearance embedding for re-ID. CenterRadarNet achieves the state-of-the-art result on the K-Radar 3D object detection benchmark. In addition, we present the first 3D object-tracking result on the K-Radar dataset. CenterRadarNet shows consistent, robust performance in diverse driving scenarios, emphasizing its wide applicability. Code is available at: https://github.com/Andy-Cheng/CenterRadarNet

Sensor fusion is crucial for an accurate and robust perception system on autonomous vehicles. Most existing datasets and perception solutions focus on fusing cameras and LiDAR. However, the collaboration between camera and radar is significantly under-exploited. Incorporating rich semantic information from the camera and reliable 3D information from the radar can achieve an efficient, cheap, and portable solution for 3D perception tasks. It can also be robust to different lighting or all-weather driving scenarios due to the capability of mmWave radars. In this paper, we introduce the CRUW3D dataset, including 66K synchronized and well-calibrated camera, radar, and LiDAR frames in various driving scenarios. Unlike other large-scale autonomous driving datasets, our radar data is in the format of radio frequency (RF) tensors that contain not only 3D location information but also spatio-temporal semantic information. This kind of radar format can enable machine learning models to generate more reliable object perception results after interacting and fusing the information or features between the camera and radar. We run several camera- and radar-based baseline methods for 3D object detection and multi-object tracking on our dataset. We hope the CRUW3D dataset will foster radar and multi-modal 3D perception research. CRUW3D is available at https://huggingface.co/datasets/uwipl/CRUW3D

In the domain of autonomous driving, the integration of multi-modal perception techniques based on data from diverse sensors has demonstrated substantial progress. Effectively surpassing the capabilities of state-of-the-art single-modality detectors through sensor fusion remains an active challenge. This work leverages the respective advantages of cameras in perspective view and radars in Bird’s Eye View (BEV) to greatly enhance overall detection and tracking performance. Our approach, Camera-Radar Associated Fusion Tracking Booster (CRAFTBooster) represents a pioneering effort to enhance radar-camera fusion in the tracking stage, contributing to improved 3D MOT accuracy. The superior experimental results on K-Radaar dataset, which exhibit 5-6% on IDF1 tracking performance gain, validate the potential of effective sensor fusion in advancing autonomous driving.

This paper presents the algorithm and very-large-scale integration (VLSI) architecture of a high-throughput and highly efficient independent component analysis (ICA) processor for self-interference cancellation (SIC) in in-band full-duplex (IBFD) systems. This is the first VLSI architecture reported in the literature based on the state-of-the-art entropy bound minimization (EBM) approach. A novel ICA algorithm is presented in this paper with momentum gradient descent optimization. Simulation results show that the number of iterations for the proposed algorithm is significantly reduced compared to the conventional ICA algorithms. Furthermore, a novel early-distribution estimation scheme is proposed in the designed ICA processor to compute multiple distribution functions with low latency and low complexity. The processing flow and the efficiency for the hardware utilization are specifically designed so that the processing speed is maximized with minimum employment of hardware components. The proposed ICA processor is designed and implemented based on the application-specific-integrated circuit (ASIC) flow. The post-layout estimations show that compared with the conventional EBM-based scheme, the proposed design improves the throughput and efficiency by 30x. In addition, compared to prior designs shown in the literature, the proposed ICA processor also demonstrates a significant enhancement in terms of throughput and efficiency.

Optical illusions have been brought into recent video games to enhance gaming experiences. However, a large corpus of optical illusions remains unused, while few games incorporate illusions seamlessly. To mitigate the gap, we propose a workflow to guide game designers in applying optical illusions to their video games, i.e., in making more illusion games. In particular, our workflow consists of 5 stages: (1) choosing a game object, (2) searching for a matching illusion, (3) selecting an illusion mechanic, (4) integrating the selected illusion into the game, and (5) optionally revealing the illusion. To facilitate our workflow, we provide a tag database with 163 illusions that are labeled by their in-game visual elements and desired effects. We also provide example editing interfaces of 6 illusions for game designers. We walk through our workflow and showcase 6 resulting illusion games. We implemented these 6 games (with and without illusion) and conducted a 12-participant study to gain a preliminary understanding of how illusions enhance gaming experiences. To evaluate our workflow, we invited 6 game designers and 6 experienced players to follow our workflow and design their own illusion games, where 3 experienced game designers completed 2-week in-depth developments. We report their games, qualitative feedback and discuss reflection on our workflow, database and editing interfaces.

We present Impossible Staircase, a real-walking virtual reality system that allows users to climb an infinite virtual tower. Our set-up consists of an one-level scaffold and a lifter. A user climbs up the scaffold by real walking on a stairway while wearing a head-mounted display, and gets reset to the ground level by a lifter imperceptibly. By repeating this process, the user perceives an illusion of climbing an infinite number of levels. Our system achieves the illusion by (1) controlling the movement of the lifter to generate reverse and imperceptible motion, (2) guiding the user through the scaffold with delay mechanisms to reset the lifter in time, and (3) procedural generating overlapping structures to enlarge perceived height of each level. We built a working system and demonstrated it with a 15-min experience. With the working system, we conducted user studies to gain deeper insights into vertical motion simulation and vertical real walking in virtual reality.

We present Haptic-go-round, a surrounding platform that allows deploying props and devices to provide haptic feedbacks in any direction in virtual reality experiences. The key component of Haptic-go-round is a motorized turntable that rotates the correct haptic device to the right direction at the right time to match what users are about to touch. We implemented a working platform including plug-and-play prop cartridges and a software interface that allow experience designers to agilely add their haptic components and use the platform for their applications. We conducted technical experiments and two user studies on Haptic-go-round to evaluate its performance. We report the results and discuss our insights and limitations.

SUMMARY: MetaABC is a metagenomic platform that integrates several binning tools coupled with methods for removing artifacts, analyzing unassigned reads and controlling sampling biases. It allows users to arrive at a better interpretation via series of distinct combinations of analysis tools. After execution, MetaABC provides outputs in various visual formats such as tables, pie and bar charts as well as clustering result diagrams. AVAILABILITY: MetaABC source code and documentation are available at http://bits2.iis.sinica.edu.tw/MetaABC/ CONTACT: [email protected]; [email protected] SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.