In this paper, we introduce a novel simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted non-orthogonal multiple access (NOMA) model designed for non-line-of-sight (NLoS) scenarios. To ensure energy self-sustainability, an unmanned aerial vehicle (UAV) is introduced for wireless energy transfer. In the proposed model, ground users (GUs) situated in communication-obstructed environments are supported by STAR-RIS to connect with the base station (BS). Energy harvested from the UAV is utilized to enable prolonged communication with 360° full spatial coverage. An optimization problem is formulated to maximize the system’s sum-rate and is decomposed into three subproblems: phase-shift optimization, power allocation, and time allocation. These subproblems are solved using semidefinite relaxation (SDR), Dinkelbach’s method, and game theory, respectively. A joint resource allocation algorithm based on the block coordinate descent (BCD) method is then proposed. Simulation results show that the proposed UAV-assisted STAR-RIS-NOMA scheme, combined with the BCD algorithm, achieves a 43.64% improvement in system capacity compared to existing approaches.

Exploiting the correlation between multimodal data to generate tactile data has become a preferred approach to enhance tactile rendering fidelity. Nevertheless, existing studies have often overlooked the temporal dynamics of force tactile data. To fill this gap in the literature, this paper introduces a joint visual-audio approach to generate a temporal tactile data (VA2T) algorithm, focusing on the temporal and long-term dependencies of force tactile data. VA2T uses a feature extraction network to extract audio and image features and then uses an attention mechanism and decoder to fuse these features. The tactile reconstructor generates temporal friction and a normal force, with dilated causal convolution securing the temporal dependencies in the force tactile data. Simulation experiments on the LMT dataset demonstrate that compared with the transformer and audio-visual-aided haptic signal reconstruction (AVHR) algorithms, the VA2T algorithm reduces the RMSE for generated friction by 29.44% and 32.37%, respectively, and for normal forces by 23.30% and 35.43%, respectively. In addition, we developed a haptic rendering approach that combines electrovibration and mechanical vibration to render the generated friction and normal force. The subjective experimental results showed that the rendering fidelity of the data generated using the VA2T method was significantly higher than that of the data generated using the transformer and AVHR methods.

Haptic feedback plays a crucial role in virtual reality (VR) interaction, helping to improve the precision of user operation and enhancing the immersion of the user experience. Instrumental haptic feedback in virtual environments is primarily realized using grounded force or vibration feedback devices. However, improvements are required in terms of the active space and feedback realism. We propose a lightweight and flexible haptic feedback glove that can haptically render objects in VR environments via kinesthetic and vibration feedback, thereby enabling users to enjoy a rich virtual piano-playing experience. The kinesthetic feedback of the glove relies on a cable-pulling mechanism that rotates the mechanism and pulls the two cables connected to it, thereby changing the amount of force generated to simulate the hardness or softness of the object. Vibration feedback is provided by small vibration motors embedded in the bottom of the fingertips of the glove. We designed a piano-playing scenario in the virtual environment and conducted user tests. The evaluation metrics were clarity, realism, enjoyment, and satisfaction. A total of 14 subjects participated in the test, and the results showed that our proposed glove scored significantly higher on the four evaluation metrics than the no-feedback and vibration feedback methods. Our proposed glove significantly enhances the user experience when interacting with virtual objects.

High-fidelity tactile rendering offers significant potential for improving the richness and immersion of touchscreen interactions. This study focuses on a quantitative description of tactile rendering fidelity using a custom-designed hybrid electrovibration and mechanical vibration (HEM) device. An electrovibration and mechanical vibration (EMV) algorithm that renders 3D gratings with different physical heights was proposed and shown to achieve 81% accuracy in shape recognition. Models of tactile rendering fidelity were established based on the evaluation of the height discrimination threshold, and the psychophysical-physical relationships between the discrimination and reference heights were well described by a modification of Weber’s law, with correlation coefficients higher than 0.9. The physiological-physical relationship between the pulse firing rate and the physical stimulation voltage was modeled using the Izhikevich spiking model with a logarithmic relationship.

Tactile teleoperation allows users to physically interact with the remote environment, and revolutionizes the set of new applications like remote surgical operation. Thus, it is sensitive to the reliability and end-to-end (E2E) latency including the sojourn latency, the transmission latency, and the computation latency. For reducing E2E latency and increasing reliability, the grant-free transmission with$K$-repetition and the mobile edge computing (MEC) are usually utilized. In this paper, we study the upper bounds of the tail-distribution function of the E2E latency in tactile teleoperation. First, we analyze the success probability of transmission, and formulate the statistical model of service process with consideration of the grant-free access, the$K$-repetition and the pilot collision. Then, based on martingales theory, the upper bounds of the tail-distribution of E2E latency are derived with joint consideration of the computation latency of MEC. Finally, the minimum E2E latency bound violation probability is obtained by allocating the latency budgets of uplink latency and MEC computation latency. Simulation results show that our theoretical analysis is accurate and the allocation is feasible.

In virtual reality applications, translation, orientation, and scaling are the three basic properties of 3D model manipulation. The 6-DOF tracking device can be used to directly manipulate the former two properties. However, the scaling property cannot be directly mapped with a hand-held tracking device because it is physically impossible. In this study, we first compare various schemes for mapping the translation and orientation of a handheld tracking device to the 3D model’s scaling property. Next, we propose a scaling manipulation framework that combines the advantages of different schemes. This enables accurate and natural wide-range scaling with the orthogonal combination design of the translation and orientation data of the tracking device. The efficiency and accuracy of the proposed framework were examined in a user study, which verified that it improved the scaling error of wide-range scaling by over 55% without a significant decrease in time performance. Finally, two scaling scenarios for the application of this framework were presented.

Quality of experience (QoE) has been widely recognized as the primary metric to evaluate user experience in multimedia applications. However, the QoE assessment of tactile virtual environments is still highly dependent on subjective measures. Inspired by the fact that physiological signals can characterize the user's emotional state, we propose a QoE measurement method for virtual reality (VR) with vibrotactile feedback based on frontal lobe power asymmetry (FLPA). The subjective score of vibrotactile experience in VR is used as the ground truth of QoE. The selection of QoE measurement indicators consists of two steps. First, the relationship between FLPA phenomenon and scores of QoE is preliminarily established by statistical methods and Spearman Correlation Coefficient. Then, the most important FLPA feature is selected by random forest, which is the best indicator for QoE measurement. The brain neural images show that vibrotactile feedback in VR can evoke FLPA phenomenon. Correlation analysis shows that there is a significant correlation between subjective scores of QoE and FLPA features. The classification results show that the selected best FLPA feature can be used as a physiological indicator to measure and predict QoE. We achieve mutual interpretation of EEG-based physiological measurements and subjective cognitive outcomes of QoE.

A six degrees of freedom (6-DOF) controller is a commonly used input device in three-dimensional user interface (3DUI) applications. However, for the fundamental task of target selection in 3D space, the selection accuracy decreases owing to the Heisenberg effect during the manipulation of the 6-DOF controller. Based on the pointing action of a 6-DOF device, we establish the mathematical model for raycasting and analyze the possibility of using a rotation gesture to select a target. This study proposes a target selection method using the axial rotation of the user's wrist, which reduces the negative impact of a discrete input for triggering the selection on accuracy. The detection model can identify the start time of the user's rotation action. The custom designed control display gain (CD gain) function can maintain the ray's stability during the rotation gesture. This method was verified using a 6-DOF pen and Vive controller in three experiments. The results show that the accuracy of the proposed rotation gesture-based raycasting target selection method is superior to that of the traditional button-pressing method, and it can be integrated into existing tracking systems for further application.

The interpretation of tactile stimuli empowers humans to identify substances, distinguish materials, and engage in tactile communication. For stimulus design in human-computer interaction, objective similarity measures improve efficiency and save costs. Inspired by the fact that biological systems are robust in recognizing multimedia stimuli, we propose a neuromorphic method for similarity measurement. The method is divided into two steps. First, tactile information is translated into biological representations by mimicking a low-threshold mechanoreceptor through a physiological neuronal model. Then, three measures are nominated to assess the similarity of neural spike trains from the following perspectives: interval spike counting, temporal matching, and vector space embedding. Regression analysis showed that the linearity of these measures was significant, indicating that the filtering ability of the physiological neuron model is robust. One of the measures is selected for comparison with the signal-to-noise ratio, structural similarity, and hybrid metric. The results suggest that the correlation between the predictions of our method and the subjective evaluation is stable, above 0.9 for each experimental stimulus. We achieve a mutual interpretation between quantitative measures of vibrotactile similarity and subjective cognitive outcomes. Furthermore, the feasibility of this method in material classification has been substantiated through an exploratory experiment.

Adequate-data collection could enhance the realism of surface texture haptic online-rendering or offline-playback. A parallel challenge is how to reduce communication delays and improve storage space utilization. Based on the similarity of the short-term amplitude spectrumtrend, this paper proposes a frequency-domain compression method. A compression framework is designed, firstly to map the amplitude spectrum into a trend similarity grayscale image, compress it with the stillpicture-compression method, and then to adaptively encode the maximum amplitude and part of the initial phase of each time-window, achieving the final compression. The comparison between the original signal and the recovered signal shows that when the time-frequency similarity is 90%, the average compression ratio of our method is 9.85% in the case of a single interact point. The subjective score for the similarity reached an excellent level, with an average score of 87.85. Our method can be used for offline compression of vibrotactile data. For the case of multi-interact points in space, the trend similarity grayscale image can be reused, and the compression ratio is further reduced.

Abstract At present, the tactile perception of 3D geometric bumps (such as sinusoidal bumps, Gaussian bumps, triangular bumps, etc.) on touchscreens is mainly realized by mapping the local gradients of rendered virtual surfaces to lateral electrostatic friction, while maintaining the constant normal feedback force. The latest study has shown that the recognition rate of 3D visual objects with electrovibration is lower by 27$\%$ than that using force-feedback devices. Based on the custom-designed tactile display coupling with electrovibration and mechanical vibration stimuli, this paper proposes a novel tactile rendering algorithm of 3D geometric bumps, which simultaneously generates the lateral and the normal perceptual dimensions. Specifically, a mapping relationship with the electrostatic friction proportional to the gradient of 3D geometric bumps is firstly established. Then, resorting to the angle between the lateral friction force and the normal feedback force, a rendering model of the normal feedback force using mechanical vibration is further determined. Compared to the previous works with electrovibration, objective evaluations with 12 participants showed that the novel version significantly improved recognition rates of 3D bumps on touchscreens.

Abstract Tactile feedback added to touchscreens provides users with a high-quality interactive experience. The effect of tactile feedback on typical interaction gestures requires to be evaluated. With a custom-designed electrostatic tactile feedback device, we explore the effects of tactile feedback on zoom-in/out gestures and determine the issues satisfied by the relationship between completion time (CT) and index of difficulty (ID). Specifically, we compare the effect of electrostatic tactile feedback on the efficiency and accuracy of zoom-in/out gestures in three conditions, that is, no tactile feedback, linearly increasing tactile feedback force over operation process, and tactile feedback only in a target area. Then, we study the relationship between CT and ID with tactile feedback added to the target area. Results of experimental data from 12 participants show that tactile feedback added only to a target area can significantly increase operational efficiency and accuracy of zoom-in/out gestures. Furthermore, the relationship between CT and ID agrees well with Fitts’ law, and the correlation coefficient is larger than 0.9.

The application of electrostatic tactile feedback to a touchscreen can provide a real sense of immersion to enhance the user interaction experience. With a custom-designed electrostatic tactile feedback device, we explore the effect of electrostatic tactile feedback on performing rotation gestures. Specifically, we design three-part experiments that include no tactile feedback, tactile feedback in the target area and tactile feedback increased linearly in the process. The effect of electrostatic tactile feedback on performing rotation gestures is then examined. The results with 12 participants indicate that adding tactile feedback to the process or the target area significantly enhances the efficiency and accuracy of rotation operations. In addition, the relationship between the index of difficulty (ID) and rotation time (RT) is studied by applying electrostatic tactile feedback in the process and the target area. The results indicate that the relations between ID and RT under the above two conditions satisfy Fitts’ Law with correlation coefficients >0.96.

Mobile CrowdSensing (MCS) is a promising paradigm that recruits mobile users to cooperatively perform various sensing tasks. When assigning tasks to users, most existing works only consider the fairness of users, i.e., the user's processing ability, with the goal of minimizing the assignment cost. However, in this paper, we argue that it is necessary to not only give full use of all the users' ability to process the tasks (e.g., not exceeding the maximum capacity of each user while also not letting any user idle too long), but also satisfy the assignment frequency of all corresponding tasks (e.g., how many times each task should be assigned within the whole system time) to ensure a long-term, double-fair and stable participatory sensing system. Hence, to solve the task assignment problem which aims to reasonably assign tasks to users with limited task processing ability while ensuring the assignment frequency, we first model the two fairness constraints simultaneously by converting them to user processing queues and task virtual queues, respectively. Then, we propose a Fair Task Assignment Strategy (FTAS) utilizing Lyapunov optimization and we provide the proof of the optimality for the proposed assignment strategy to ensure that there is an upper bound to the total assignment cost and queue backlog. Finally, extensive simulations have been conducted over three real-life mobility traces: Changchun/taxi, Epfl/mobility, and Feeder. The simulation results prove that the proposed strategy can achieve a trade-off between the objective of minimizing the cost and the fairness of tasks and users compared with other baseline approaches.

Abstract Electrostatic tactile feedback has an important role in improving touchscreen operations and user experience. In this paper, panning performance under different forces and different waveforms were explored using subjective and objective experiments. The effects of electrostatic tactile feedback under one linear force and three non-linear force profiles with the fixed waveform were studied. The results indicated that panning accuracy, efficiency and user experience under exponential force was superior to those under linear, quadratic or logarithmic forces. Second, the effects of four different waveforms under exponential force were studied. Results indicated that panning performance was better with square wave, compared to those with sinusoidal, triangular or sawtooth waves. The relationship between panning time and index of panning difficulties was also explored and found to be in agreement with the Steering law and Meyer’s law, with correlations of 0.9395 and 0.9397, respectively. The results suggest ways for designing touchscreens with tactile feedback that is useful in both research and end-user communities.

This paper focuses on the problem of sequential prediction for imbalanced data streams. A novel hybrid algorithm called Weighted OS-ELM and Dynamic Generative Adversarial Nets (GAN-WOSELM) is presented for handling this issue. In the data reconstruction stage, GAN is utilised for generating lifelik e minority class samples to equilibrate the data distribution. Then a principal component score threshold helps judge unusual data. In the model update stage, the new constructive ELM is applied to forecast time-varying data chunk. After concerning fitting accuracy and data change, the analytical relationship between the new weight and the shifting imbalance ratio is determined. Therefore, the GAN-WOSELM can update weight quantificationally, and it avoids interactive parameter optimization. According to the suitable weight for the arriving chunk, the proposed method is able to perceive the changeable data distribution and do the adaptation by itself, thus building a reliable model with a low fitting deviation. Numerical experiments are conducted on four different kinds of UCI datasets. The results demonstrate that the proposed algorithm not only has better generalisation performance but also provides higher numerical stability.

Background Ultrasonic tactile representation utilizes focused ultrasound to create tactile sensations on the bare skin of a user’s hand that is not in contact with a device. This study is a preliminary investigation on whether different ultrasonic haptic rendering methods have an impact on the perceptual threshold. Methods This study conducted experiments with the adaptive step method to obtain participants’ perceptual thresholds. We examine (1) whether different parts on the palm of the hand have different perceptual thresholds; (2) whether the perceptual threshold is different when the ultrasonic focus point is stationary and when it moves in different trajectories; (3) whether different moving speeds of the ultrasonic focus point have an influence on the perceptual threshold; and (4) whether the addition of a DC offset to the modulating wave has an impact on the perceptual threshold. Results The results show that the center of the palm is more sensitive to ultrasonic haptics than the fingertip; compared with a fast-moving focus point, the palm is more sensitive to a stationary and slow-moving focus point. When the modulating wave has a DC offset, the palm is sensitive to a much smaller modulation amplitude . Conclusion For the future ultrasonic tactile representation systems, dynamic adjustment of intensity is required to compensate the difference in perceptual thresholds under different rendering methods to achieve more realistic ultrasonic haptics.

This paper presents the EV2-Pen which leverages electrovibration technology and vibration technology in pen interaction. Electrovibration technology can produce multisensory feedback when the pen is in motion (sliding/moving on the screen), and vibration technology can provide vibrative feedback when the pen is stationary (pointing/resting on the screen). We conducted an experiment to investigate user performance with the EV2-Pen. The results indicated that the EV2-Pen outperformed the EV-Pen [18, 19] in pointing-steering tasks. Finally, we discuss the characteristics of the EV2-Pen, and explore some possible applications and scenarios.

This paper presents an Electrovibration Pen (EV-Pen) which incorporates electrovibration technology into pen interactions. The EV-Pen has two unique characteristics: precise interaction and pen-on-paper feeling. We conducted four experiments for evaluating the EV-Pen. Experiment 1 was to determine preliminary characteristics of the EV-Pen. Experiments 2 and 3 were to evaluate precise interaction task performance through a steering task and a tracing task. We compared user performance between the EV-Pen, a mechanical-vibration pen, a normal pen without feedback and electrovibration-based finger interaction. Experimental results indicated that the EV-Pen outperformed the other devices in precise task interactions. Experiment 4 tested pen-on-paper feeling in drawing and handwriting tasks, and it was observed that the EV-Pen significantly enhances user experience. Based on the experimental results, we discuss implications and potential benefits for the design of the EV-Pen.

In this study, a new mixed near‐field and far‐field sources localisation algorithm based on sparse signal recovery is addressed. In this scheme, two special cumulant vectors are constructed successively, the first one is used to obtain the azimuth estimations of all the incoming signals, and the second one is used to distinguish the mixed sources as well as estimate the range related to the near‐field sources. The reweighted ℓ 1 ‐norm minimisation with one iteration is utilised for sparse signal recovery. In the recovery process, the authors propose to select the regularisation parameter by a special case of 2‐fold cross‐validation. The simulation results demonstrate the effectiveness and the efficiency of the proposed algorithm.

In this letter, we present a covariance difference algorithm to cope with the mixed far-field and near-field sources localization problem. By exploiting the eigenstructure differences between the far-field covariance matrix and the near-field one, the spatial differencing technique can be adopted to classify the signals types. Based on the symmetric property of the uniform linear array geometry, a near-field estimator without any spectral search or parameter-pairing is performed. Compared to the previous works, the resultant algorithm can realize a more reasonable classification of the signals types, as well as provide the improved estimation accuracy. Computer simulations are carried out to evaluate the performance of the proposed algorithm.

In this study, the authors propose an efficient third‐order cyclic moment‐based estimation of signal parameters via rotational invariance techniques algorithm to cope with the mixed far‐field and near‐field cyclostationary sources localisation problem. The key point is that the auto‐pairing parameters of near‐field bearing and range is realised by estimating the unique non‐singular matrix between the signal subspace and the matrix of direction vectors. This algorithm firstly constructs two special third‐order cyclic moment matrices, in which the rotational factor is the function of the direction‐of‐arrival (DOA) and range of mixed far‐field and near‐field sources. By implementing the singular value decomposition and the high‐resolution multiple signal classification spectral search, the authors estimate the azimuth DOAs for all the incoming sources. Then, the authors perform the estimation of the unique non‐singular matrix between the signal subspace and the matrix of direction vectors. Finally, the range of near‐field sources is determined from the obtained rotational factor. The proposed algorithm is computationally more efficient than the previous works, and it avoids pairing parameters. Computer simulations are carried out to demonstrate the performance of the proposed algorithm.

In this letter, a novel mixed sources localization method based on sparse signal reconstruction is presented, which can efficiently estimate direction-of-arrival (DOA) and range parameters of near-field and far-field sources. By constructing the cumulant domain data of array which is only related to DOA parameters of mixed sources, we obtain DOA estimation of all sources using the weightedl1-norm minimization. And then, a mixed overcomplete matrix on the basis of DOA estimation is introduced in the sparse signal representation framework to estimate range parameters and distinguish far-field sources from mixed sources. Compared with the two-stage MUSIC algorithm, the proposed method can provide improved accuracy and resolve closely spaced sources. The simulation results show the effectiveness of our method.

This paper presents un approach for beamforming based on third-order cyclic statistics. The approach has fairly well accuracy. Further, a new scheme joint beamforming and space-time block coding is suggested that multiple antenna arrays transmit at base station and multi-antenna elements receive at mobile terminal. This scheme improves the receiving error performance of mobile terminal and increases the capacity of the entire wireless communication system. The computer simulation validates that the aforementioned scheme is superior and effective.