While gesture typing is widely adopted on touchscreen keyboards, its support for low vision users is limited. We have designed and implemented two keyboard prototypes, layout-magnified and key-magnified keyboards, to enable gesture typing for people with low vision. Both keyboards facilitate uninterrupted access to all keys while the screen magnifier is active, allowing people with low vision to input text with one continuous stroke. Furthermore, we have created a kinematics-based decoding algorithm to accommodate the typing behavior of people with low vision. This algorithm can decode the gesture input even if the gesture trace deviates from a pre-defined word template, and the starting position of the gesture is far from the starting letter of the target word. Our user study showed that the key-magnified keyboard achieved 5.28 words per minute, 27.5% faster than a conventional gesture typing keyboard with voice feedback.

Writing text with eye gaze only is an appealing hands-free text entry method. However, existing gaze-based text entry methods introduce eye fatigue and are slow in typing speed because they often require users to dwell on letters of a word, or mark the starting and ending positions of a gaze path with extra operations for entering a word. In this paper, we propose GlanceWriter, a text entry method that allows users to enter text by glancing over keys one by one without any need to dwell on any keys or specify the starting and ending positions of a gaze path when typing a word. To achieve so, GlanceWriter probabilistically determines the letters to be typed based on the dynamics of gaze movements and gaze locations. Our user studies demonstrate that GlanceWriter significantly improves the text entry performance over EyeSwipe, a dwell-free input method using “reverse crossing” to identify the starting and ending keys. GlanceWriter also outperforms the dwell-free gaze input method of Tobii’s Communicator 5, a commercial eye gaze-based communication system. Overall, GlanceWriter achieves dwell-free and crossing-free text entry by probabilistically decoding gaze paths, offering a promising gaze-based text entry method.

Although menu selection has been extensively studied in HCI, most existing studies have focused on sighted users, leaving blind users’ menu selection under-studied. In this paper, we propose a computational model that can simulate blind users’ menu selection performance and strategies, including the way they use techniques like swiping, gliding, and direct touch. We assume that selection behavior emerges as an adaptation to the user’s memory of item positions based on experience and feedback from the screen reader. A key aspect of our model is a model of long-term memory, predicting how a user recalls and forgets item position based on previous menu selections. We compare simulation results predicted by our model against data obtained in an empirical study with ten blind users. The model correctly simulated the effect of the menu length and menu arrangement on selection time, the action composition, and the menu selection strategy of the users.

Suggesting multiple target candidates based on touch input is a possible option for high-accuracy target selection on small touchscreen devices. But it can become overwhelming if suggestions are triggered too often. To address this, we propose SATS, a Suggestion-based Accurate Target Selection method, where target selection is formulated as a sequential decision problem. The objective is to maximize the utility: the negative time cost for the entire target selection procedure. The SATS decision process is dictated by a policy generated using reinforcement learning. It automatically decides when to provide suggestions and when to directly select the target. Our user studies show that SATS reduced error rate and selection time over Shift [51], a magnification-based method, and MUCS, a suggestion-based alternative that optimizes the utility for the current selection. SATS also significantly reduced error rate over BayesianCommand [58], which directly selects targets based on posteriors, with only a minor increase in selection time.

Text correction on mobile devices usually requires precise and repetitive manual control. In this paper, we present EyeSayCorrect, an eye gaze and voice based hands-free text correction method for mobile devices. To correct text with EyeSayCorrect, the user first utilizes the gaze location on the screen to select a word, then speaks the new phrase. EyeSayCorrect would then infer the user’s correction intention based on the inputs and the text context. We used a Bayesian approach for determining the selected word given an eye-gaze trajectory. Given each sampling point in an eye-gaze trajectory, the posterior probability of selecting a word is calculated and accumulated. The target word would be selected when its accumulated interest is larger than a threshold. The misspelt words have higher priors. Our user studies showed that using priors for misspelt words reduced the task completion time up to 23.79% and the text selection time up to 40.35%, and EyeSayCorrect is a feasible hands-free text correction method on mobile devices.

Back-of-device interaction is a promising approach to interacting on smartphones. In this paper, we create a back-of-device command and text input technique called BackSwipe, which allows a user to hold a smartphone with one hand, and use the index finger of the same hand to draw a word-gesture anywhere at the back of the smartphone to enter commands and text. To support BackSwipe, we propose a back-of-device word-gesture decoding algorithm which infers the keyboard location from back-of-device gestures, and adjusts the keyboard size to suit the gesture scales; the inferred keyboard is then fed back into the system for decoding. Our user study shows BackSwipe is feasible and a promising input method, especially for command input in the one-hand holding posture: users can enter commands at an average accuracy of 92% with a speed of 5.32 seconds/command. The text entry performance varies across users. The average speed is 9.58 WPM with some users at 18.83 WPM; the average word error rate is 11.04% with some users at 2.85%. Overall, BackSwipe complements the extant smartphone interaction by leveraging the back of the device as a gestural input surface.

Selecting targets accurately and quickly with eye-gaze input remains an open research question. In this paper, we introduce BayesGaze, a Bayesian approach of determining the selected target given an eye-gaze trajectory. This approach views each sampling point in an eye-gaze trajectory as a signal for selecting a target. It then uses the Bayes' theorem to calculate the posterior probability of selecting a target given a sampling point, and accumulates the posterior probabilities weighted by sampling interval to determine the selected target. The selection results are fed back to update the prior distribution of targets, which is modeled by a categorical distribution. Our investigation shows that BayesGaze improves target selection accuracy and speed over a dwell-based selection method, and the Center of Gravity Mapping (CM) method. Our research shows that both accumulating posterior and incorporating the prior are effective in improving the performance of eye-gaze based target selection.

Recent works have designed systems containing tiny devices to communicate with harvested ambient energy, such as the ambient backscatter and renewable sensor networks. These systems often encounter the heterogeneity and randomness of ambient energy. Meanwhile, the energy storage unit, such as the battery or capacitor, has the inherent property of imperfect charge efficiency λ (λ ≤ 1), which is usually low when the power of the ambient energy is weak or variable. These features bring new challenges in using the harvested energy efficiently. This article calls it the stochastic duty cycling problem and studies it under three cases—offline, online, and correlated stochastic duty cycling—to maximize utilization efficiency. We design an offline algorithm 1 for the offline case with optimal performance. An approximation algorithm with the ratio 1 − e −γ is designed for the online case. By adding initial negotiation among devices, we present a correlated algorithm and prove its approximation ratio theoretically. Experiment evaluation on our real energy harvesting platform shows that the offline algorithm performs over the other two algorithms. The correlated algorithm may not perform over the online one under the impacts of the three metrics: heterogeneity, charge efficiency, and energy harvesting probability.

Public Bike Systems (PBSs) offer convenient and green travel service and become popular around the world. In many cities, the local governments build thousands of fixed stations for PBS to alleviate the city traffic jam and solve the last-mile problem. However, the increasing use of PBSs leads to new congestion problems in the form that users have, such as no bike to rent or no dock to return the bike. Further, users wish to receive assistance on deciding how to select bike trips with minimal time cost while taking congestion into account. Meanwhile, crowdsourcing attracted increasing attention in recent years. This paper applies it to help users share information and select bike trips before the bikes or docks are occupied. An interesting and important problem is how to help users select bike trips so that the time consumed on the trips can be minimized. We model the problem as a Bike Trip Selection (BTS) game which is shown to be equivalent to the symmetric network congestion game. This equivalence allows us to design a BTS algorithm by which the users can find at least one Nash Equilibria (NE) distributively. Furthermore, this paper evaluates the algorithm based on real datasets collected from the PBS of Hangzhou City in China. We also design a BTS system including an Android APP and a server to conduct the experiment for the distributed BTS algorithm in practice.

In Bike-Sharing System (BSS), great efforts have been devoted to performing resources prediction, redistribution and trip planning to alleviate the unbalance of resources and inconvenience of bike utilization caused by the explosion of users. However, there is few work in trip planning noticing that the complete trip composes of three segments: from user's start point to a start station, from the start station to a target station and from the target station to user's terminal point. To study the case, this paper addresses a static trip planning problem in BSS by considering system-wide conflicts so as to achieve higher service quality of the system. The problem is formulated as the well-known weighted k-set packing problem. We design two algorithms, a Greedy Trip Planning algorithm (GTP) and a Humble Trip Planning algorithm (HTP), for the problem. For comparison, we design a Random Trip Planning algorithm (RTP) as a benchmark. Extensive simulation results show that GTP and HTP outperform RTP and reveal the impact of different factors on our algorithms.

A common feature among many crowdsourcing applications is to decompose the huge or complex tasks into some small sub-ones, which require some users with different skills to implement. The kind of tasks are composite and called Composite Tasks (CTs) , which are said to be completed and return reward only after all of their sub-tasks are finished successfully. Meanwhile, users may have various capabilities to implement diverse sub-tasks (STs) with corresponding cost so users are heterogeneous. In this paper, we study the problem of how users choose the STs to maximize their payoff (reward minus cost) when there are multiple such CTs. This payoff maximization problem with multiple CTs and heterogeneous users turns out to be NP-completed. We then propose a Local Composite Task Selection (LCTS) algorithm to help the users choose their subtask strategies. Its convergence and complexity are analyzed theoretically. For comparison, we design a centralized Composite Task Selection (CTS) algorithm and a Low Cost and Random sub-task selection (LCR) algorithm as benchmarks. Numerical results suggest that the LCTS algorithm achieves a similar payoff and task completion ratio to the CTS when the number of users is large. The performance of LCTS is highly over LCR on both of the payoff and the task completion ratio. The results also illustrate the quick convergence of the LCTS algorithm.

Energy-harvested wireless sensor networks may operate perpetually with extra energy supply from natural energy such as solar energy. Nevertheless, harvested energy is often too limited to support perpetual network operation with full duty cycle. To achieve perpetual network operation and process the data with high importance, measured by value of information (VoI), sensor nodes have to operate under partial duty cycle and to improve the efficiency of harvested energy. A challenging problem is how to deal with the stochastic feature of natural energy and variable data VoI. We consider the energy consumption during the energy storage and the diversity of the data process including sampling, transmitting, and receiving, which consume different power levels. The problem is then mapped as a budget-dynamic multiarm bandit problem by treating harvested energy as budget and the data process as arm pulling. This paper proposes an opportunistic duty cycling (ODC) scheme to improve the energy efficiency while satisfying perpetual network operation. ODC chooses some proper opportunities to store harvested energy or to spend it on the data process based on historical information of energy harvesting and VoI of the processed data. With this scheme, each sensor node only needs to estimate ambient natural energy in short term so as to reduce computation cost and storage capacity for the historical information. It then can adjust its own duty cycle distributively with its local historical information. This paper conducts extensive theoretical analysis for the performance of our scheme ODC on the regret, which is the difference between the optimal scheme and ours. Our experimental results also manifest the promising performance of ODC.

In recent years, there have been several kinds of energy harvesting networks containing some tiny devices, such as ambient backscatter [1], ring [2] and renewable sensor networks [3]. During the energy harvesting, the networks suffer from the energy heterogeneity, dynamics and prediction hardness because the access to natural resources is often spatiotemporal different and timely changing among the devices. Meanwhile, the charging efficiency is quite low especially when the power of the harvested energy is weak. It results in the energy waste to store the harvested energy indirectly. These features bring challenging and interesting issues on efficient allocation of the harvested energy. This paper studies the stochastic duty cycling by considering these features with the objective characterized by maximizing the common active time. We consider two cases: offline and online stochastic duty cycling. For the offline case, we design an optimal solution: offline duty cycling algorithm. For the online case, we design an online duty cycling algorithm, which achieves the approximation ratio with at least equation where γ is the probability able to harvest energy. We also evaluate our algorithms with the experiment on a real energy harvesting network. The experiment results show that the performance of the online algorithm can be very close to the offline algorithm.

To support the sustainable operation of wireless sensor networks using limited energy, duty cycling is a promising solution. However, it is a challenge to guarantee each node communicating with its neighbors under duty cycle when the network is asynchronous. The challenge becomes bigger when nodes’ duty cycles are required to be adjusted separately according to their demands to save energy and achieve high channel utilization. Existing low power listening- and contention-based protocols are not energy-efficient and cannot ensure high channel utility. Additionally, synchronization-based media access control (MAC) protocols suffer from extra energy consumption and low synchronization precision. This paper proposes a localized and on-demand (LOD) duty cycling scheme based on a specifically designed semi-quorum system. LOD can adjust duty cycle of each node adaptively according to its demand so as to avoid channel contention, consequently achieving high channel utilization. This allows the fairness for channel access within asynchronous sensor networks. Extensive experiments are conducted on a real test-bed of 100 TelosB nodes to evaluate the performance of LOD. As compared with B-MAC, LOD substantially reduces contention for channel access and the energy consumption, thus improving the network throughput significantly.