We present a mixed-methods study of viewers on their practices and motivations around watching mukbang — video streams of people eating large quantities of food. Viewers' experiences provide insight on future technologies for multisensorial video streams and technology-supported commensality (eating with others). We surveyed 104 viewers and interviewed 15 of them about their attitudes and reflections on their mukbang viewing habits, their physiological aspects of watching someone eat, and their perceived social relationship with mukbangers. Based on our findings, we propose design implications for remote commensality, and for synchronized multisensorial video streaming content.

This paper describes the design of a group-based project that combines both the hardware and software disciplines of our Computer Science (CS) and Computer Engineering (CE) undergraduates. The goals for the multi-disciplinary design project (MDP) are to provide our 3rdyear undergraduates with the opportunities to apply the theoretical knowledge acquired from their early years to solving a practical problem, foster a continuous improvement mindset, and experience the benefits and challenges of working collaboratively in a large team. This requires the formulation of a suitably complex problem that can integrate the skill sets from both the CE and CS programmes. It also requires suitable assessment strategies that can evaluate the students at the individual level, at the group level and at the inter-group level. This paper shares our pedagogical considerations in designing the project module, our experience gleaned from conducting the module over the last five years and some results of a recent online survey conducted to gather students' feedback on their experience of MDP.

MuSeeCol is a collaborative music creation system that allows two users to jointly and playfully compose melodic or percussive music. Collaborative music is rendered when two face-to-face users interact with their respective side of a see-through multi-touch sensitive acrylic panel. We present user study findings from nine pairs of individuals who have used the MuSeeCol system collaboratively and summarize some general observations of what types of interaction designs make for effective face-to-face collaboration.

DIY cardboard-based VR headset that enfolds a commodity smartphone is a popular means of achieving low cost immersive stereoscopic viewing experiences. However, existing applications for these devices have limited user-input capability, allowing mainly look-at control by head tilting. CardboardSense is a set of novel tap-gesture-based inputs that take advantage of the motion sensing capabilities already available in most smartphones. The key premise is that finger contacts with the cardboard surface produce incidental but detectable vibrations that can be reliably differentiated from typical user head motion. Moreover, tapping on different surface locations produces differentiable temporal signatures in the sensor signals. We present methods to extract and analyze these temporal patterns, and experimentally determine a set of tapping gestures that can be accurately recognized. Experimental results show promising potential for CardboardSense to extend the interactivity of future cardboard-based VR headset applications.

A novel Mine-Ours-Yours (MOY) interaction design framework is proposed for designing collaborative play activities in environments that combine both private and shared interactive spaces. A collaborative game designed on a system that integrates multiple mobile devices with an interactive tabletop was presented to demonstrate the implementation of the proposed MOY framework. Observations from field trials involving two groups of children were used to summarize the collaborative behaviors that are likely to be observed under the different interaction design configurations.

Manipulatives are tangible objects designed to support learning through exploratory arrangement and manipulation. The i-Cube is a cube-shaped digital manipulative that provides unique 3-D spatial awareness of the facets and orientation of neighboring i-Cubes. This paper discusses the considerations adopted in its design and the advantages of the proposed design to that of other cube-based tangible user interfaces. The i-Cubes are then employed in the design of two applications. MusiCube Arranger is a tangible music composition and layering system and Spelling Cube is an interactive system for learning spelling. These applications are used to illustrate how the unique features of the i-Cube can be exploited to implement novel tangible interactions such as free-form 3D stacking, interactive control through block orientation change and context-aware feedback.

Commercial 3D scene acquisition systems such as the Microsoft Kinect sensor can reduce the cost barrier of realizing mid-air interaction. However, since it can only sense hand position but not hand orientation robustly, current mid-air interaction methods for 3D virtual object manipulation often require contextual and mode switching to perform translation, rotation, and scaling, thus preventing natural continuous gestural interactions. A novel handle bar metaphor is proposed as an effective visual control metaphor between the user's hand gestures and the corresponding virtual object manipulation operations. It mimics a familiar situation of handling objects that are skewered with a bimanual handle bar. The use of relative 3D motion of the two hands to design the mid-air interaction allows us to provide precise controllability despite the Kinect sensor's low image resolution. A comprehensive repertoire of 3D manipulation operations is proposed to manipulate single objects, perform fast constrained rotation, and pack/align multiple objects along a line. Three user studies were devised to demonstrate the efficacy and intuitiveness of the proposed interaction techniques on different virtual manipulation scenarios.

In this work, we propose a connectionist memory structure for spatio-temporal sequence learning and recognition inspired by the Long-Term Memory structure of human cortex. Besides symbolic data, our framework is able to continuously process real-valued multi-dimensional data stream. This capability is made possible by addressing three critical problems in spatio-temporal learning, namely error tolerance, significance of sequence's elements and memory forgetting mechanism. We demonstrate the potential of the framework with a synthetic example and a real world example, namely the task of hand-sign language interpretation with the Australian Sign Language dataset.

This paper proposes a neural network structure for spatio-temporal learning and recognition inspired by the long-term memory (LTM) model of the human cortex. Our structure is able to process real-valued and multidimensional sequences. This capability is attained by addressing three critical problems in sequential learning, namely the error tolerance, the significance of sequence elements and memory forgetting. We demonstrate the potential of the framework with a series of synthetic simulations and the Australian sign language (ASL) dataset. Results show that our LTM model is robust to different types of distortions. Second, our LTM model outperforms other sequential processing models in a classification task for the ASL dataset.

While very large collaborative surfaces are already being widely employed to facilitate concurrent interactions with multiple users, they involve no personalization in the touch interactions. Augmenting them to identify the touch interactions with multiple smart-phones can enable interesting co-located communal applications with context-based personalized interactions and information exchange amongst users' portable devices and the shared wall display. This paper proposes a novel matching technique, called tilt correlation, which employs the built-in tilt sensor to identify smart-phones that make concurrent two-point contacts on a common multi-touch wall display. Experimental investigations suggest that the resultant error rate is relatively low; in addition, we also propose a quantitative measure, called the Bourne Identity Index to allow application designers to determine the reliability of each device identification.

Visual exploration of volume data often requires the user to manipulate the orientation and position of a slicing plane in order to observe, annotate or measure its internal structures. Such operations, with its many degrees of freedom in 3D space, map poorly into interaction modalities afforded by mouse-keyboard interfaces or flat multi-touch displays alone. We addressed this problem using a what-you-see-is-what-you-feel (WYSIWYF) approach, which integrates the natural user interface of a multi-touch wall display with the untethered physical dexterity provided by a handheld device with multi-touch and 3D-tilt sensing capabilities. A slicing plane can be directly and intuitively manipulated at any desired position within the displayed volume data using a commonly available mobile device such as the iPod touch. 2D image slices can be transferred wirelessly to this small touch screen device, where a novel fast fat finger annotation technique (F3AT) is proposed to perform accurate and speedy contour drawings. Our user studies support the efficacy of our proposed visual exploration and annotation interaction designs.

Enabling efficient exploration of large-scale virtual environments such as those simulating astrophysical environments is highly challenging. Astrophysical virtual worlds span exceptionally large spatial scales occupied mostly by empty space, and this makes it difficult for the user to comprehend the spatial context during exploratory navigation. Public exhibits, where novice users have little experience using complicated virtual navigation interfaces, pose additional challenges.

The exploratory visualization of 3D data sets or multiple axis motion control of a monitoring camera mounted on a six-axis robot often require the user to control the viewing orientation and position of a camera view within a 3D space. This paper describes the design of an inertial motion sensing system, which uses only accelerometers to provide interactive 3D control of both orientation and position in such systems. We describe the interaction design strategies for exploratory visualization of 3D data using the proposed input device, which is mounted beneath a portable display. Experiments on the use of the proposed device suggest that with some training, the users are able to improve the speed with which they can navigate the orientation and position of a virtual camera to a desired target view in the 3D visual space.

In this paper, we present a novel biologically-inspired spatio-temporal sequence learning architecture of visual place cells to leverage autonomous navigation. The construction of the place cells originates from the well-known architecture of Hubel and Wiesel to develop simple to complex features in ventral stream of the human brain. To characterize the contribution of each feature towards scene localization, we propose a novel significance analysis based on the activation profiles of features throughout the spatio-temporal domain. The K-iteration Fast Learning Neural Network (KFLANN) is then used as a Short-Term Memory (STM) mechanism to construct our sequence elements. Subsequently, each sequence is built and stored as a Long-Term Memory (LTM) cell via a one-shot learning mechanism. We also propose a novel algorithm for sequence recognition based on the LTM organization. The efficiency and efficacy of the architecture are evaluated with the vision dataset from ImageCLEF 2010 Competition.

This paper presents a novel human-computer interface for projector/camera-based applications that uses a deformable interaction surface. We discuss its design and implementation within the context of a radically different approach for controlling home appliances by pressing virtual buttons that are projected on soft deformable surfaces such as a sofa pillow. Effective real-time computer vision algorithms for implementing pointing and selection action detection for such an interface are discussed. Experimental results highlight the parameters and factors that have significant effect on the overall performance of such an interface.

We present a part-based shape descriptor that incorporates both the description of the general shape form of each subpart and its geometric relationship with other connected parts. Associated with each descriptor is a saliency measure that weighs each part’s visual significance. By incorporating this saliency measure into the shape matching process, we able to discriminate between shape forms as well as take boundary texture into consideration when computing shape similarity. This paper also describes a multi-resolution pyramidal framework for generating the required gradient vector field and vector field disparity map from which the shape descriptors, in the form of gradient vector field histograms, are derived. Experimental results involving silhouettes images are presented to demonstrate the various characteristics of the proposed shape descriptor, which includes its invariance to similarity transform and its ability to match composite shapes containing boundary noise and texture, limb articulation and occlusion. 1

A least-square infinite series approximation (L-SISA) technique is proposed for modeling amplitude modulated (AM) sinusoidal components of naturally occurring signals, such as those produced by traditional musical instruments. Each AM sinusoid is iteratively extracted using an analysis-by-synthesis technique and the problem of parameter estimation is linearised for least-square approximation through a systematic search in the frequency vector space. Some timbre analysis results obtained using the AM sinusoidal model are presented.

Camera self-calibration techniques make it possible to compute three-dimensional measurements of an observed scene from video sequences without prior knowledge of the camera intrinsic parameters. However, most previous techniques make the overly constraint assumption that camera intrinsic parameters remain constant throughout the image sequence. The self-calibration method described in this article generalises the camera model by adding into the intrinsic parameter set a variable focal length, whose change is approximated by a cubic B-spline. The modified intrinsic parameters are then calculated with adapted Kruppa equations, together with an initialization procedure that utilizes the genetic algorithm. Results obtained using synthetic image sequences are presented. It shows the algorithm's ability to trade the changing focal length (zooming) even under noisy conditions.

This paper proposes a new spatial-temporal MRF model for the detection of missing data (also referred to as blotches) in image sequences. The blotches in noise-corrupted image sequences exhibit a temporal discontinuity characteristic which is commonly used for the detection of blotches. However, the badly motion compensated pixels will also appear as temporal discontinuities, thus making it difficult to distinguish the true blotches from the poorly motion compensated regions. The proposed MRF model addresses the problem of incorrect detection. It is found that the degree of false-alarm in the detection of the blotches in image sequences can be reduced by using a moving-edge detector in the MRF model to identify the blotch-edges from the moving edges.

Most image restoration algorithms will blur edges of moving objects in the vicinity of occluded and uncovered image regions. These artifacts are visually annoying. This paper proposes a new bi-directional 3D auto-regressive model which is able to suppress mixed noise processes and recover lost signals in both the covered and uncovered regions in an image sequence. The performance of this algorithm is shown to be superior to the existing 3D AR model.

This paper aims to develop effective parallel processing techniques for 3-dimensional (3D) multi-level median filtering (3DMMF) with motion compensation on video sequences. Due to the nature of unbalanced load of the filtering algorithm and the objective for the parallel approach, two methods of dynamic load balancing have been proposed and compared. They are sender-initiated-load-balancing (SILB) and receiver-initiated-load-balancing (RILB) algorithms. We propose a SILB algorithm which utilises the spatial-temporal characteristics of the processed sequences for load prediction to achieve dynamic load balancing. Both theoretical analysis and experimental results on the IBM SP2 computing surface have been presented in this paper.