Most conventional swept volumetric displays do not allow for direct physical interaction because their rigid, high-speed sweep screens make touch dangerous or impractical. However, several existing approaches enable direct interaction with volumetric content through the use of devices such as re-imaging plates and flexible screens. A major challenge in implementing direct interaction systems lies in the high demand for latencies, but it is, in general, difficult to achieve low latencies with swept volumetric displays because their refresh rates are limited by the physical sweep periods.

Abstract In an era defined by the relentless influx of data from diverse sources, the ability to harness and extract valuable insights from streaming data has become paramount. The rapidly evolving realm of online learning techniques is tailored specifically for the unique challenges posed by streaming data. As the digital world continues to generate vast torrents of real-time data, understanding and effectively utilizing online learning approaches are pivotal for staying ahead in various domains. One of the primary goals of online learning is to continuously update the model with the most recent data trends while maintaining and improving the accuracy of previous trends. Based on the various types of feedback, online learning tasks can be divided into three categories: learning with full feedback, learning with limited feedback, and learning without feedback. This survey aims to identify and analyze the key challenges associated with online learning with full feedback, including concept drift, catastrophic forgetting, skewed learning, and network adaptation, while the other existing reviews mainly focus on a single challenge or two without considering other scenarios. This article also discusses the application and ethical implications of online learning. The results of this survey provide valuable insights for researchers and instructional designers seeking to create effective online learning experiences that incorporate full feedback while addressing the associated challenges. In the end, some conclusions, remarks, and future directions for the research community are provided based on the findings of this review.

We introduced the disturbance degree of ground and proposed an evaluation method to measure the mobile performance of a crawler on soft ground during direction switching. First, we developed a planar omnidirectional crawler, which had a configuration with two left and right unit crawlers for performing turning motion, as the target for evaluation. Second, by utilizing the proposed disturbance degree of ground evaluation method, we investigated how the turning and translational motions of the crawler mechanism affected soft ground by measuring the flow of sand on a horizontal surface. It was quantitatively shown that translational motion switched the travel direction with lesser disturbance to the road surface compared to turning motion. We confirmed that ground disturbance could be evaluated during direction switching using the proposed method.

Recent progress of high-speed projectors using DMD (Digital Micromirror Device) has enabled low-latency motion adaptability of displayed images, which is a key challenge in achieving projection-based dynamic interaction systems. This paper presents evaluation of different approaches in achieving fast motion adaptability with DMD projectors through a subjective image evaluation experiment and a discrimination experiment. The results suggest that the approach proposed by the authors, which updates the image position for every binary frame instead of for every video frame, applied to 60-fps video input offers perceptual image quality comparable with the quality offered by 500-fps projection.

This paper presents a fast projection mapping method for moving image content projected onto a markerless planar surface using a low-latency Digital Micromirror Device (DMD) projector. By adopting a closed-loop alignment approach, in which not only the surface texture but also the projected image is tracked by a camera, the proposed method is free from a calibration or position adjustment between the camera and projector. We designed fiducial patterns to be inserted into a fast flapping sequence of binary frames of the DMD projector, which allows the simultaneous tracking of the surface texture and a fiducial geometry separate from a single image captured by the camera. The proposed method implemented on a CPU runs at 400 fps and enables arbitrary video contents to be "stuck" onto a variety of textured surfaces.

This paper describes a mobile pedestrian navigation system that provides users with clues that help understanding spatial relationship between mobile camera views and a 2D map. The proposed method draws on the map upright billboards that correspond to the basal planes of past and current viewing frustums of the camera. The user can take photographs of arbitrary landmarks on the way to build billboards with photographs corresponding to them on the map. Subjective evaluation by eight participants showed that the proposed method offers improved experiences over navigation using a standard 2D map.

This paper presents high speed and high accuracy visual servoing system. The algorithm has three major improvements, which can be implemented in practical applications; Firstly high-accuracy pose estimation by using stereo cameras, secondly real time implementation issues with non-real-time image processing platform and thirdly a consideration for industrial position controller. To resolve the issues, position-based visual servoing (PBVS) is adopted and appearance model based virtual visual servoing (VVS) is applied for pose estimation. VVS approach does not compute the stereo matching but directly compares the OpenGL rendered image and camera image for each camera; estimate the position/orientation using VVS independently for each camera; and provides a theoretically optimal compromise among those estimates. To enhance estimation accuracy, a hybrid method of stereo trigonometry for position estimation and weighted least squares for orientation estimation is proposed to combine the information from the stereo cameras. Operation speed is increased by using graphic processing unit (GPU) acceleration and an on-line trajectory generator which can accommodate the variable cycle of the image processing and the fixed cycle of a common robot controller. Finally, some experimental results illustrate the effectiveness of the proposed framework.

When using a camera as a position sensor, the measurement is limited in bandwidth, mainly due to the blur effects. The knowledge of an accurate model of the camera is then necessary to reconstruct the trajectory from the measurements given by the camera. This paper deals with the reconstruction of the continuous-time trajectory from the discrete-time measurements provided by the camera and shows the improvement obtained by using an accurate camera model. In the proposed methodology, a Kalman filter is used for the data fusion between the model and the measurement. The tuning and implementation of the filter are discussed in the specific context of the camera measurement. The system is evaluated in the context of a biomedical application: the reconstruction of the movement of a beating-heart.

We describe sensorless visualizing techniques of 3D range information of object surfaces, in which the range information is projected directly onto the real-world surface by projectors. The proposed system consists of two projectors that merely project still images. Two techniques, namely, contour map projection based on the moire method and pseudo-color map projection based on complementary colors, are shown to be used simultaneously.

We propose a new approach for augmented reality, in which real-world scene images are augmented with video fragments manipulated in the time domain. The proposed system aims to display slow-motion video sequences of moving objects instantly without accumulated time lag so that a user can recognize and observe high-speed motion on the spot. Images from a high-speed camera are analyzed to detect regions with important visual features, which are overlaid on a normal-speed video sequence.

The problem considered here involves the design and application of a recursive algorithm to extract and predict the position of an object in a 3D environment from one feature correspondence from a monocular image sequence. Translational model involves an object moving in a parabolic path using projectile physics. A state-space model is constructed incorporating kinematic states, and recursive techniques are used to estimate the state vector as a function of time. The measured data are the noisy image plane coordinates of object match taken from image in the sequence. Image plane noise levels are allowed and investigated. The problem is formulated as a tracking problem, which can use an arbitrary large number of images in a sequence. The recursive estimation is done using Recursive Least Squares (RLS). Results on both synthetic and real imagery illustrate the performance of the estimator.

Vision can be used as a sensor for measuring the position of a visual marker. When the displacement of the marker over the camera exposure time is significant, the obtained position measurement is an image of the trajectory over the exposure time. This paper deals with dynamical models providing a prediction of the measurement from the continuous-time trajectory. The first contribution of the paper consists in the development of models for the global shutter and rolling shutter modes with variable exposure time. The second contribution consists in a methodology for the validation of these models. This methodology is used for validating the model of a camera with global shutter mode with full exposure time.

This paper proposes a method for identifying an object which contains an accelerometer out of many moving objects in the view of a stationary camera using motion data obtained by the camera and the accelerometer. The camera and the accelerometer are assumed to be connected with a network but not synchronized. In order to evaluate the similarity of the motion data despite the unknown time lag between the accelerometer and the camera, NCC (normalized cross-correlation) of the signals is computed and its peak is tracked. Since the coordinate system of the accelerometer is unknown, NCC is computed for the norms of the acceleration vectors, which were compensated for the gravitational acceleration component, obtained by the camera and the accelerometer. The experimental results show that the proposed method successfully identified the person wearing the accelerometer out of three walking people. It is also shown that the hand holding the accelerometer was successfully identified out of three moving hands even though the directions of the accelerometer coordinate axes varied temporally due to the free motion of the hand.

A 1,000-jps camera-projector system in which projected patterns are adoptively controlled according to image processing results is described. Adaptive structured light projection enables fast and efficient 3-D information acquisition. The prototype system is applied to the tracking of the nearest point of an object, and experimental results show that the system successfully tracked an apex of a fast-moving large! object.

This paper describes a new reconfigurable processor architecture specialized for high frame rate visual processing. This architecture employs a 2-D mesh processing element (PE) array in which the PEs can be configured to operate as SIMD arrays or operation-pipeline trees depending on image processing algorithms so that maximum on-chip memory consumption is reduced. To achieve high on-chip memory utilization, the architecture features that the instruction register in each PE is mapped in its local memory space and that the ALU network and the local memory network can be configured independently. Simulation results show that the proposed architecture effectively utilizes both of the SIMD and operation pipeline modes.

This paper proposes a high frame rate 3-D shape measurement system based on structured light projection using temporally-coded binary patterns. This system consists of a DMD (digital micromirror device) projector and a high-speed camera, both of which operate at over 6,000 fps. Introducing an image processing algorithm of detecting and tracking the projected light patterns enables frame-by-frame 3-D image measurement. Experimental results show that the proposed algorithms worked effectively, and the system achieved 3-D imaging at 3,350 fps.

A sensor selection method for distributed real-time sensing is proposed where the effect of communication delays is taken into account. First, a multi-sensor fusion algorithm that allows communication delays is proposed. This method takes advantage of the information form of Kalman filtering in order to maintain scalability with the number of sensors. Based on this algorithm, an algorithm is developed that calculates mutual information between an observation and the state to be estimated, which provides a criterion for sensor selection. Simulated results of a target-tracking task show its effectiveness.

A real-time visual processing system using a general-purpose vision chip, an image sensor in which photo detectors and processing elements are integrated, is described. In order to control the vision chip and process its output at high speed, a novel architecture called SPARSIS, in which the control process of the vision chip is pipelined and integrated with a RISC type integer pipeline, was developed. This architecture can guarantee real-tune operation with high temporal resolution, and even makes possible software-controlled A/D conversion. Sample algorithms demonstrating its fine-grained real timeliness, and experimental results with the implemented system, are also described.