Accurate indoor drone four degrees-of-freedom (4DoF) estimation, including both 3D position and heading, is essential for safe flight in GNSS-denied environments, opening up opportunities for fully autonomous IoT applications such as infrastructure inspection and surveillance. Approaches using a built-in camera are attractive due to their low cost and ease of deployment, however, existing methods face key limitations. Visual-Inertial Odometry suffers from drift and often fails in textureless environments, while Visible Light Positioning requires a direct line of sight to ceiling LEDs, which is challenging for drones with a limited field of view. We propose ReFDrone, a system that simultaneously estimates 3D position and heading from a single image using a downward-facing rolling-shutter (RS) landing camera and floor-reflected light from modulated ceiling LEDs. Our core insight is that RS distortion produces spectral lobes in the frequency domain whose orientation directly encodes the angle of arrival (AoA) of the LED. Leveraging this property, ReFDrone extracts AoA from a single RS image, enabling estimation of 2D position and heading from just two LEDs, and derives altitude using a virtual-view image with a distance model. We evaluated ReFDrone under 1296 conditions in 2.5 m x 2.5 m testbed, achieving a 90th percentile 3D position error of 0.335 m and a heading error of 8.101 degrees, demonstrating its accuracy and practicality.

Beamforming with speaker arrays allows for directional audio communication, selectively delivering information to users in specific locations. However, this approach typically requires a specialized array hardware. We propose an adaptive acoustic spot communication method targeting moving receivers that utilizes only two commercial off-the-shelf (COTS) speakers. A key feature of our method is its ability to achieve high-performance by predicting the receiver’s position. We conducted real-world experiments with a moving microphone to evaluate the effectiveness of the proposed method. The results showed a demodulation success rate of 94.7% for the adaptive spot communication, excluding the region directly in front of the speakers, demonstrating the feasibility of adaptive spot communication for a moving target.

Smartphone positioning utilizing a single indoor speaker and its reflections offers the advantage of low deployment cost. However, unlike the direct wave, reflected waves are not always stably detectable, even under Line-of-Sight (LOS) conditions, leading to inconsistent positioning performance. Additionally, associating each detected reflection with its corresponding virtual speaker poses a challenge. This study proposes a method to improve positioning performance by tracking the smartphone based on hypotheses about the identity of each detected reflection. Evaluations were conducted in a real-world environment along both rectangular and circular paths, using only the smartphone’s bottom built-in microphone. The results demonstrated 90th percentile errors of 0.62 m and 0.77 m for the rectangular and circular paths, respectively. This accuracy, approximately equivalent to a step length, suggests potential for navigation applications. These results represent error reductions of 55% and 46%, respectively, compared to conventional approaches.

This study investigates an indoor 3D positioning method for a microphone using a single speaker. The proposed method estimates the distance, azimuth, and elevation of the microphone from the speaker and calculates the 3D position of the microphone. Multiple short signals with different frequency bands were transmitted sequentially at sufficient intervals to avoid the influence of indoor reflected waves. We extracted these signals at the receiver side, and their amplitude spectra were computed. The azimuth and elevation of the microphone were estimated by matching the obtained amplitude spectrum with the reference data previously measured at each azimuth and elevation. The distance was estimated from amplitude attenuation. The 3D positioning performance was evaluated at ten points in a practical environment. The results show that the 90-th percentile value of the error was 0.583 m.

This paper proposes a tracking system that integrates visible-light positioning and pedestrian dead reckoning using a smartphone’s built-in ambient light sensor and inertial sensors. By correcting the accumulated errors in the position estimation via ambient light sensing, the proposed system can achieve impressive tracking performance. The challenges of this approach and techniques to alleviate the various problems are described. Experiments demonstrate that the proposed system can achieve an average tracking error of 0.89 m. Issues to be investigated further are discussed, which will enable the system to be deployed in places such as museums and shopping malls.

In this paper, we propose an unauthorized photography detection technique by using a frequency spectrum obtained from a photo under light-emitting diode illuminations emitting multi-carrier modulated signals. We utilize characteristics of a rolling-shutter camera as a frequency filter whose function depends on its shutter opening ratio. By using one test photo image, the proposed method can judge whether it was taken with an authorized or unauthorized camera. When the value of an authorized shutter opening ratio is updated periodically or randomly on an authorized camera only, it is possible to use a frequency spectrum of a photo taken by the camera as its own one-time fingerprint. Technical details of the proposed method are described and its characteristics are established through controlled and laboratory-setting experiments. The experimental results confirm that the proposed method can achieve an f1-score of 99.1% and an area under the receiver operating characteristic curve of 0.9997. Causes of correct and wrong detection results are investigated and the robustness of the proposed method is discussed using compressed and resized photo images. Issues related to limitations and deployment for real-world applications are described.

Positioning with a small number of anchors is an important issue in the study of acoustic indoor positioning systems (AIPS). We present PT-Sync, a novel approach to time-synchronize between a commercial off-the-shelf (COTS) speaker and a mobile device by leveraging acoustic sensing and reflected signals from the floor. PT-Sync enables ranging with one speaker and 2-D positioning with two speakers without any additional hardware. Our proposed time synchronization method requires height information from the floor in the calculation process, which we estimate by active acoustic sensing. Using a unique averaging technique and IMU enables robust height estimation. PT-Sync can be used in a variety of indoor environments and can be time-synchronized with already installed speakers. The results of the evaluation experiment confirmed that a synchronization error of 0.16 ms was achieved even at a distance of 6 m from the speaker, and that synchronization on the order of microseconds could be achieved. Furthermore, pedestrian tracking experiments confirmed that positioning of less than 38.9 cm can be achieved at the 90th percentile.

In this paper, we propose a method for the 2D positioning of a smartphone by receiving sinusoidally modulated light signals emitted by ceiling-mounted LEDs. An ambient light sensor (ALS) mounted on the smartphone measures the reflected illumination from the floor. The advantage of this combination is that it consumes very little power, does not require the implementation of additional devices, and removes constraints on the placement of the LEDs caused by the field of view. To the best of our knowledge, this method of using the ALS on a smartphone with reflected light from the floor has not been proposed before. We conducted several experiments to investigate the effectiveness and limitations of our method for visible-light positioning. One experiment with different LED frequencies and floor materials showed that, for multiple LEDs emitting modulated light at around 60 Hz and highly reflective floor materials, positioning is possible with a 90th-percentile error of 35.5 cm. Other experiments involved different types of ambient light and different LED arrangements. Based on the results of these experiments, we can show how to improve the positioning performance. We also discuss future developments in real environments.

In this paper, we describe NL-Beep, which is a novel ranging method between multiple smartphones using acoustic sensing in non-line-of-sight (NLOS) environments. We need to know the distance between people to maintain social distancing because of the COVID-19 pandemic. Acoustic-based ranging methods that use a smartphone built-in speaker and microphone represent one approach to high-accuracy ranging methods. These methods can be used in line-of-sight (LOS) environments where there are no obstacles between the smartphones. However, in daily life, there are usually many obstacles, including pedestrians, and NLOS environments are the norm rather than the exception. Therefore, we propose the NL-Beep system for NLOS environments. To measure the distance between two smartphones without using a direct signal, NL-Beep efficiently uses a reflected signal from the ceiling in an indoor environment. It can also detect LOS and NLOS environments based on the profile of the received signal, and adapt the method used to estimate the distance between the smartphones. In our experiments, we used two smartphones in a room and estimated the distance between them for several placement configurations. In our results, we obtained 90th-percentile errors of less than 11.97 cm for the distances between the smartphones. In this paper, we also mention the effectiveness and limitations of NL-Beep.

In this paper, we propose a 2D localization method using a single speaker and a microphone. Our proposed method estimates the direction of the microphone based on the knowledge that the spectral amplitude of a speaker in each direction is different. Additionally, the distance between the speaker and microphone is estimated based on the received amplitude. The 2D location of the microphone is calculated from the estimated direction and distance values. We conducted real-environment experiments for two different types of speakers and confirmed that the 90-percentile localization error was 0.265 m. We also discuss the systematic errors in the distance estimation that occurred in the experiment, revealing that these errors were caused by the characteristics of the speakers.

This paper proposes a spot-communication method that can direct the spot to the user location adaptively. In the proposed method, the first transmitted signals are used to estimate user location, which is then reflected in the second transmitted signals; therefore, the spot directed to the user can be generated instantaneously. As the signals consist of mutually orthogonal sinusoidal waves, multiple spots can be generated simultaneously, and the shape of the spots can be controlled by overlapping them. Through real environmental experiments using four speakers, we confirmed that our proposed method can generate adaptive spots. Moreover, in the case that appropriate spots were not generated, comparisons were carried out with respect to the theoretical values to determine the cause.

The following topics are dealt with: indoor radio; indoor navigation; wireless LAN; smart phones; learning (artificial intelligence); pedestrians; Global Positioning System; radionavigation; mobile computing; Kalman filters.

This paper describes a high-speed optical camera communication (OCC) technique using an LED and a rolling-shutter camera. In the proposed technique, the symbols being transmitted are encoded as time delays of optimally modulated signals derived theoretically. A receiver decodes the symbols by using intensities obtained from four consecutive line sensors of a camera. Experiments using a camera having performance similar to that of a general-purpose camera show that the proposed technique can achieve 0.833 ~ 1.17 bits per line sensing and that symbol transmission is possible with a longer exposure time setting; this is difficult to achieve using existing on-off keying OCC techniques.

In this paper, we describe a novel position-recognition method that uses passive acoustic signals from two previously installed speakers (passive acoustic sensing) and active acoustic signals from a smartphone's loudspeakers (active acoustic sensing). In passive acoustic sensing, a locus of positions for the smartphone can be calculated from the measured time difference of arrival from the two installed speakers. In active acoustic sensing, a chirp signal is transmitted from the speakers of the smartphone, and the distance to the side wall is measured from the propagation time of arrival at its microphone. We can obtain the smartphone position from our proposed model equations by combining these two results. In our experiments, we installed speakers at intervals of 10 m along a corridor and estimated the smartphone position at several places. From these results, we obtained 90th percentile errors of less than 0.224 m for 2-D positioning. We found that multipaths from the side wall were causing the positioning error in passive acoustic sensing, and the variance of the positioning error using the top microphone which was omnidirectional was smaller than the bottom one. When we introduced a weighting based on the result of the active acoustic sensing and the difference in the performance between microphones, the 90th percentile errors were reduced to less than 0.134 m.

This paper describes an indoor positioning technique using a video camera that captures LED light reflected by the floor. Indoor positioning for mobile devices can be very useful. In particular, localization techniques using LEDs and cameras, so-called visible light positioning, are known to be effective and have high accuracy. However, existing methods have the constraint that they must capture the light source directly. This requires a high-performance processor and a high-resolution image. However, light sources cannot always be detected directly (loss of signal: LOS). Our proposal aims to solve these problems by estimating the position of a camera that does not face the light directly but monitors light reflected by the floor. Specifically, individual LED ceiling lights emit sinusoidal waves modulated with different frequencies, and the camera captures the overlapped light from the LEDs reflected by the floor then demodulates the signal. The camera need not seek the ceiling lights directly from an image, unlike existing methods. The position can be estimated using any part of the image because usually the ceiling light is reflected by the whole floor. Experimental results show that the proposal requires less than 1/100 as many pixels for localization as existing methods and the position can be estimated within 0.4 m at the 90th percentile in a 2.5 m square room. We show that the cause of errors is mainly the difference between the LED diffusion model and the actual light diffusion, the occlusion and the noise and movement of the camera. Overcoming these problems remains as our future work.

In this paper, we propose a selective visible light communication method using a single light source and multiple video cameras by using their differences in reception sensitivity between carrier waves composed of OFDM signals. In our evaluation experiments, the transmitted signals were composed of 50th, 75th, 78th, and 80th order carrier waves encoding symbols modulated by 4, 8, and 16 PSK. The exposure time ratio for each camera was set to different values so that symbols encoded in one of the carrier waves could be decoded successfully, but erroneously in the other waves. The selectivity of individual carrier waves as theoretically derived was confirmed through the experimental results.

We describe a time-of-arrival-(ToA-) based localization system for smartphones. In this system, the transmitter emits modulated light-emitting diode (LED) light and sound waves, then the smartphone catches them. The smartphone measures the time of flight of sound waves and localizes its position using multilateration. The LED light is used for visible light communication, which also conveys the time reference of the sound emission. Using the time reference, we can synchronize between the transmitter and the receiver, then the ToA-based localization can be available. The precision of time synchronization is the key factor for localization based on ToA. Hence, we have proposed the SyncSync method using a modulated LED light and a smartphone video camera, which enables ToA localization by measuring the time of flight of sound waves. This method gives better results than those of time-difference-of-arrival localization. However, we had to use a dedicated light synchronization device for our method. Visible light communication (VLC) is becoming a popular application of smartphones. If VLC demodulation could be used for time synchronization in acoustic localization, VLC and indoor localization would be integrated into a single application. In this paper, we examined the feasibility of VLC time synchronization for localization. Then, ToA-based localization was performed using a smartphone application. The standard deviation of the 3D localization was around 100 mm in a dark room, which is sufficiently precise for practical applications.

The paper describes a rapid and accurate time-synchronization technique for smartphones using their built-in cameras and its preliminary evaluations for application development.

This paper describes a rapid and flicker-free visible light communication technique that uses an off-the-shelf video camera. Transfer rates at least 53% faster than existing methods were achieved in experiments. Features of the proposed method are discussed via theoretical analysis.

This paper describes a spot-controllable data-transfer method. The proposed method generates a beam-shaped spot using two commercial off-the-shelf speakers. In our method, a symbol consists of a pair of sinusoidal waves having different angular frequencies. The width and direction of a beam-shaped spot are controlled by the angular-frequency difference between the sinusoidal waves and the transmission-time difference between the two speakers. Multiple spots can be generated by transmitting multiple pairs of sinusoidal waves based on the principle of orthogonal frequency-division multiplexing. By over-lapping multiple beam-shaped spots, the locations and sizes of the areas enabled to receive data are controllable. Experiments using four speakers and computer simulation show that the proposed method can generate controllable spots. An analysis of the errors in a real indoor environment indicate that they are caused by multipath signals, radiation damping of transmitted signals, and the incident/output angle characteristics of the microphone and speakers.

In this paper we describe SyncSync, a novel time-of-arrival (ToA) localization method for smartphones. ToA measurements generally show better precision than time-difference-of-arrival measurements, but ToA systems require a synchronization mechanism between anchor and mobile nodes. For this synchronization, we employ modulated light with an acoustic signal for the time-of-flight distance measurement. These are detected by the smartphone's video camera and microphone. The time resolution in consumer video cameras is typically only a few tenths of a second, but by utilizing a CMOS image sensor's rolling shutter effect we obtain synchronization resolutions of a few microseconds, sufficient for precise acoustic ToA measurement. Experiments confirm operation of the system with localization errors within 10 mm.

This paper presents a rapid and accurate time-synchronization technique for acoustic localization. Two LED arrays for modulated illumination and an off-the-shelf camera are used. An equation that specifies the time difference between them accurately and precisely is proposed. Experiments in real environments show that the proposed technique using rectangular signals can achieve time synchronization to within 29.4 μs (5.94 μs standard deviation) and acoustic ranging to within 9.9 mm (2.01 mm standard deviation) for a measurement period of 0.1 s. Because of the smaller standard deviation in comparison with the error, it is indicated that there exist systematic errors to be removable. This technique will enable the implementation of an acoustic 3D localization system for mobile devices such as tablet PCs and smartphones based on time-of-arrival trilateration, which is more accurate than positioning taking a time-difference-of-arrival approach. Theoretical analyses for modulated illumination considering a camera exposure time clarify how an optimally modulated signal is designed for achieving rapid and accurate time synchronization.

We describe a novel time-synchronization technique for mobile smart nodes. We capture modulated LED light with a video camera, which is usually built into a smart node. The CMOS image sensor of a video camera does not take a snapshot at a certain time. Instead, the sensor captures data on a line-byline basis and the sensor output consists of lines taken at slightly different times. Therefore, we can extract time information from the image. This can be used for time synchronization for time-of-arrival (ToA) measurements. In this paper, we describe the fundamentals and an experiment using light-synchronized acoustic ToA measurements with mobile smart nodes. The acknowledged precision of the obtained time information is equivalent to 5.8 mm with an airborne sound wave.

We describe an indoor localization technique for smart phones. Our new method, called the Frequency Division Multiplexing Phase Accordance Method (FDM-PAM), uses a beat called a sync pattern composed of a pair of sinusoidal waves with slightly different frequencies, which is similar to our original ultrasound ranging technique called the Phase Accordance Method (PAM). By generating multiple sync patterns with different central frequencies and transmitting them from different speakers, FDM-PAM conducts time-difference-of-arrival (TDOA) multilateration for localizing smart phones. In the current implementation of FDM-PAM, the 2D indoor position of a smart phone can be estimated. Three sync patterns are generated by using two out of six sinusoidal waves with frequencies ranging from 14.75 kHz to 17.25 kHz. The transmission of the sync pattern from the speakers lasts 4 ms. Through experiments, we have confirmed that FDM-PAM achieves accuracy of around 10 cm using only a short burst transmission, which indicates that the localization technique is sufficiently rapid and accurate.

We propose a novel technique for 3D localization that integrates a single camera and ultrasound. We use the Extended Phase Accordance Method and the ultrasound to measure accurately the distance to a moving target and we use the camera to identify the target's 2D position on the image plane. A prototype system consists of a transmitter unit mounting one ultrasound transmitter and three infrared LEDs around it, and a receiver unit with one inexpensive camera and one ultrasound receiver. We implemented these units in a lightweight and compact way (receiver unit size: 55 mm × 44 mm), to make the system robust against the no-line-of-sight problems that frequently occur in trilateration or multicamera-based systems. Experimental results show that the RMSEs of the proposed system are 1.20 mm and 1.66 mm for static and mobile (velocity: 1.0 m/s) targets, respectively. These indicate that the performance of the system is comparable with that of high-end systems.

In this paper, we propose an innovative motion-capture system using ultrasonic communications. Compared with existing commercial motion-capture systems that use optical or magnetic sensing, the proposed system can provide a cost-effective solution for industrial and entertainment applications. To design and implement the system, a distance-estimation method called the Extended Phase Accordance Method (EPAM), which can measure the distance to a moving object with a standard deviation of less than 1 mm, was devised. To improve the capture rate of the proposed system, the EPAM algorithm was implemented in a field-programmable gate array (FPGA). The current version of the system conducts motion capture using five markers attached to a user. It can work at around 10 frames per second (fps), with an error of less than 55 mm and a standard deviation of 42 mm. This demonstrates a moderate level of accuracy, which will be useful for several applications.

We describe a fast and accurate indoor localization technique using the multiple signal classification (MUSIC) algorithm. This algorithm is known as a high-resolution method for estimating directions of arrival (DOAs) or propagation delays. A critical problem in using the MUSIC algorithm for localization is its computational complexity. Therefore, we devised a novel algorithm called Time Space additional Temporal MUSIC (TSaT-MUSIC), which can rapidly and simultaneously identify DOAs and delays of multicarrier ultrasonic waves from transmitters. Computer simulations have proved that the computation time of the proposed algorithm is almost constant in spite of increasing numbers of incoming waves, and is faster than that of existing methods based on the MUSIC algorithm. Experiments in real environments showed that the standard deviation of position estimations in 3D space is less than 10 mm, which is satisfactory for indoor localization.

An ultrasonic localization system is described in the paper. To the best of our knowledge, this is the first system that can simultaneously identify not only the 3D position, but also the velocity of a moving object. The proposed system uses an original and innovative method called Extended Phase Accordance Method (EPAM) that can precisely identify the distance between an ultrasonic microphone and a moving transmitter by rapidly estimating the frequency shift of the transmitted signal. One remarkable feature of the proposed system is the use of a single compact receiver unit, which will reduce deployment labor and costs. Experiments demonstrate that the proposed system shows the 3D position and velocity estimation with sufficient accuracy.

Too narrow, the productivity-oriented vision guiding ubiquitous computing should be replaced or enriched with humanistic aspects. We discuss the role of Maslow's hierarchy of needs in the creation and adoption of smart spaces, robots and wearable computers worldwide to provide elements for alternative visions of ubiquity. We show that current ubiquitous systems are stratified at the lowest levels of the hierarchy. Based on interviews, questionnaires and experiments, we highlight a positive correlation between the hierarchy of needs and the general public's perception and possible adoption of services. Finally, we discuss implications of these results, and notably the importance of creating humanistic frameworks, services and environments.

This correspondence describes a system called ePro for supporting face-to-face group activities. ePro connects a sensor-embedded board that utilizes radio frequency identification (RFID) technology for object recognition and a computer simulation, and is currently used to discuss urban planning and environmental problems. Users collaboratively construct a town by placing pieces such as houses on the board. The computer simulation program automatically recognizes the arrangement of pieces on the board. It then shows environmental changes of the town through visualized simulation results. The goal of ePro is to initiate group discussions: The physical board supports the participation of each user by allowing them to manipulate physical objects, and the computer simulation gives users feedback on their manipulation, which triggers their further actions. A comparative user study of ePro (with and without the sensor-embedded board) was conducted. Through the study, ePro has proved its high usability and effectiveness in activating discussions in face-to-face group activities

In this paper, a system called Caretta that integrates personal and shared spaces to support face-to-face collaboration is described. We use PDAs and a multiple-input sensing board for personal and shared spaces, respectively. Users of Caretta can discuss and negotiate with each other in the shared space by manipulating physical objects, while they individually examine their ideas in their own personal spaces. Caretta allows users to participate in group activities interchangeably and seamlessly using both these spaces. Caretta is applicable to various collaborative tasks. In this paper, it supports users in urban planning tasks. User studies of Caretta demonstrated that it allowed users to collaborate in a flexible fashion: users could work individually in their personal spaces at their own pace, cooperatively work together in the shared space, and smoothly transition between both of the spaces.

The authors have so far developed a system that supports collaborative learning and evaluated it in school education. The system integrates physical and virtual worlds to teach urban planning and environmental problems by game playing by the pupils themselves. Intensive experiments with the system have proven that it can support the externalization of learners' thinking, active participation, interactions among them, and discussions in a learning situation. The paper addresses the effects of collaborative learning, and discusses how pupils discover the way of thinking of other pupils during game play.

Many systems that support group activities utilize computer and network technologies so that people in different places or at different times can communicate with each other. In this paper, we propose a system called ePro for supporting face-to-face collaboration by integrating physical and virtual worlds. In order to enhance interactions among group members and raise their engagement, ePro connects a sensor-embedded board and a computer simulation. In our current implementation, ePro is used to discuss urban planning and environmental problems. Group members collaboratively construct a town by placing a piece, such as a house, on a sensor-embedded board (physical world). The computer simulation program automatically recognizes the arrangement of pieces on the board. It then visualizes environmental changes to the town on the board through simulations (virtual world). The visualization shown to the group members amplifies the interaction among them and gives them feedback for re-designing the town on the board. In order to evaluate the effects of supporting group activities by combining physical and virtual worlds, preliminary experiments with ePro were carried out.