Color smudge operations from digital painting software enable users to create natural shading effects in high-fidelity paintings by interactively mixing colors. To precisely control results in traditional painting software, users tend to organize flat-filled color regions in multiple layers and smudge them to generate different color gradients. However, the requirement to carefully deal with regions makes the smudging process time-consuming and laborious, especially for non-professional users. This motivates us to investigate how to infer user-desired smudging effects when users smudge over regions in a single layer. To investigate improving color smudge performance, we first conduct a formative study. Following the findings of this study, we design SmartSmudge, a novel smudge tool that offers users dynamical smudge brushes and real-time region selection for easily generating natural and efficient shading effects. We demonstrate the efficiency and effectiveness of the proposed tool via a user study and quantitative analysis.

3D sketching in virtual reality (VR) enables users to create 3D virtual objects intuitively and immersively. However, previous studies showed that mid-air drawing may lead to inaccurate sketches. To address this issue, we propose to use one hand as a canvas proxy and the index finger of the other hand as a 3D pen. To this end, we first perform a formative study to compare two-handed interaction with tablet-pen interaction for VR sketching. Based on the findings of this study, we design HandPainter, a VR sketching system which focuses on the direct use of two hands for 3D sketching without requesting any tablet, pen, or VR controller. Our implementation is based on a pair of VR gloves, which provide hand tracking and gesture capture. We devise a set of intuitive gestures to control various functionalities required during 3D sketching, such as canvas panning and drawing positioning. We show the effectiveness of HandPainter by presenting a number of sketching results and discussing the outcomes of a user study-based comparison with mid-air drawing and tablet-based sketching tools.

Sensory conflict theory and postural instability theory were often tested individually to explain cybersickness in VR systems, but they were seldom systematically compared. An earlier study evaluated them on a large screen using 2D videos. This study evaluated sensory conflict and postural instability on the discomfort in VR. Virtual visual locomotion were shown on an head-mounted display. A motion platform vibrated in low-frequency while the participant stood on top. Each factor was manipulated alone or in combination. Results showed that the visual motion only condition led to the highest miserable score, higher than the physical vibration only condition. This suggested that consistent with previous literature, sensory conflict may be a major contributing factor of cybersickness.

Sensory conflict theory attempts to provide the framework of cyber-sickness in virtual reality (VR) systems by the mismatch between visual and vestibular senses. This study examined whether coupling motion sensations to the visual stimulus in a VR setting could reduce the discomfort. A motion-coupled VR system was used. Motion platform provides motion that supplements visual stimulus from the head-mounted display. Participants experience programmed visual and motion yaw rotations while viewing a virtual apartment. Three conditions were tested on how motion and visual stimuli synchronise which each other: purely visual, motion synchronised with visual, and visually-levelled frame of reference. Results showed that providing matching visual-motion stimuli decreased the miserable score (MISC) of cybersickness and increased the joyfulness score (JOSC) of their subjective feeling.

The perceived distance estimation in an immersive virtual reality system is generally underestimated to the actual distance. Approaches had been found to provide users with better dimensional perception. One method used in head-mounted displays is to interact by walking with visual feedback, but it is not suitable for a CAVE-like system, like imseCAVE, with confined spaces for walking. A verbal corrective feedback mechanism is proposed. The result shows that estimation accuracy generally improves after eight feedback trials although some estimations become overestimated. One possible explanation is the need of more verbal feedback trials. Further research on top-down approach for improvement in depth perception is suggested.

Surgical robotic systems with miniaturized flexible Tendon-driven Serpentine Manipulators (TSM) have enjoyed increasing popularities among surgeons and researchers for their advantages of working in constrained and torturous human lumen such as oral cavity and upper GI tract. However, they suffer from sufficient nonlinearities and model uncertainties due to friction, tension varying, tendon slacking, etc. Model based control is insufficient to overcome such uncertainties and automate challenging surgical related tasks. The objective of this work is to automate certain clinical tasks to alleviate surgeon fatigue and promote task efficiency in kinematics free and sensor free circumstances. We present a data-driven approach based on Learning from Demonstration (LfD), which utilizes statistical machine learning models to encode system underlying dynamics and generalize smooth motor trajectories by direct actuation space learning. Motion segmentation is enabled with soft margin Support Vector Machine (soft-SVM) in complicated tasks. We also make attempts to retrieve task-specific properties by Locally Weighted Regression (LWR). We evaluated the approach on two surgical related tasks: compliant insertion and simplified Endoscopic Submucosal Dissection (ESD). The flexible TSM successfully reproduced both tasks and demonstrated superior trajectory performance. A video is available at: https://youtu.be/rLQo6xKtyMI.

Wireless Sensor Networks (WSNs) provide an effective means to perform data acquisition in remote areas. However, limitations exist that prohibit their widespread use. In this paper, an object tracking algorithm based on Artificial Immune Systems (AIS) is proposed. Based on the immune network theory of AIS, the activities of wireless sensor nodes are stimulated by target in-coming objects but suppressed by other wireless sensor nodes based on a dynamic changing environment. When a sensor node is being suppressed, the sensor node will go to a low-power state momentarily, otherwise, it will be actively estimating the location of the target objects. In doing so, the energy efficiency of the overall network will be optimized through the dynamic stimulation and suppression of sensor nodes that is mediated by the immunity-based algorithm. A number of experiments are conducted to verify the algorithm in terms of the degree of accuracy in TARGET TRACKING and the energy efficiency of the entire sensor network.

Research in Artificial Immune Systems (AIS) for optimization has attracted attention in recent years. Exploration and adoption of the inspired immune theories in clonal selection, immune network, negative selection, and danger signaling is becoming a popular basis for algorithm design for solving optimization problems, especially on multi-objective optimizations. Novel algorithms are design, benchmarked and applied to real life applications. This paper aims to review and outlook on the latest development of AIS-based algorithms in the recent decade. An analysis of the AIS applications is also discussed.

This paper presents a novel hybrid framework for cooperative conflict resolution in distributed system (DS) optimization problems that combines the Lagrangian decomposition (LD) method and the mechanisms offered by artificial immune systems. LD provides a means to derive simpler sub-problems by dropping complicated constraints, yet the challenge is to determine effective algorithms for updating the corresponding multipliers. After full decomposition of a single optimization problem derived from a DS into a number of sub-problems that are to be solved by individual intelligent agents, this paper introduces a distributed cooperative search framework (DCSF) whereby intelligent agents are designed for solving those sub-problems. The development of DCSF is inspired by the human immune system where algorithms of suppression and stimulation are formulated to compute the infeasibility (violation of relaxation constraints) and the affinity of candidate solutions individually. In this paper, DCSF is implemented on a distributed platform supported by Matlab to solve a generalized assignment problem (GAP) so as to demonstrate its behavior and performance in the realm of distributed combinatorial optimization.

The human immune system is a complex system of cells, molecules, tissues, and diverse organs that can provide us with primary defense against pathogenic organisms. These components are highly interactive and execute the immune response in a coordinated and specific manner. This paper presents a formal mathematical model of an artificial immune system (AIS)-based control framework. The framework aims to provide an integrated solution to control and coordinate complex distributed systems with a large number of autonomous agents such as automated warehouses, distribution centers, and automated material-handling systems. The control framework consists of a set of AIS agents working in response to the changing environment and the occurrence of tasks. The AIS agents manipulate their capabilities to derive appropriate responses to tackle different problems. A methodology describing the response-manipulation algorithm of the AIS agents and their ability to generate new capabilities is discussed in this paper. Through response manipulation and knowledge building, a self-organized and fully distributed system with agents that are able to adapt and accommodate in a dynamic environment via distributed decision making and interagent communication is achieved.

For MSR robots to successfully configure from one configuration into another, the control system must be able to visualize the current structure of the robot, which cannot be done without appropriate information about each module's docking status. Although the type of information required to visualize the structure of a MSR robot differs with the physical design of the modules, there are essential information that are commonly required, such as docking orientation and identity of neighboring modules. This paper presents a novel multi-port and multi-orientation modular robot, and a representation method that can uniquely represent the geometric structure of a group of connected modules and to analyze the number of "reconfigurable DOF" within the structure. The proposed method uses labeled planar graphs and incidence matrices to describe the docking status of the modules within the structure, which helps to effectively encode the data in computer understandable expressions. In addition to the work in configuration analysis, an innovative mechanism for detecting the orientation of each docking port is also presented.

Distributed control paradigm offers robustness, scalability, and simplicity to the control and organization of module based systems. MSR (modular self-reconfigurable) robot is a class of robot that best demonstrate the effectiveness of distributed systems, as all modules in the robot are individuals that perform their own actuation and computation; the behavior of the complete robot is a collective behavior of all independent modules. In this paper, a general control framework, named general suppression framework, is proposed and a distributed control system based on the framework is presented. The control system is designed to control a set of MSR robots configured into a planar manipulator arm. All modules in the manipulator arm contain their own processing and actuation units, which allow them to evaluate and react to the environment independently. The modules can perform passive communication with their immediate neighbors and can exhibit aggressive or tolerant behavior based on the environment change to generate emergent group behaviors. A simulation program is developed to demonstrate the effectiveness of the distributed system in controlling the module based planar manipulator arm.

The mapping between the Cartesian space and joint space of robot manipulators has long been a difficult task for redundant robots. Two main methods are used in the classical approach. One is by using direct kinematic inversion in the position regime; the other is to use Jacobian Transformation in the velocity regime. However, for a redundant robot, a non-squared Jacobian matrix is resulted when mapping between the two spaces. This results in using appropriate optimization algorithms to compute along with the Jacobian matrix. Taking the second approach, the Jacobian matrix for a redundant robot will be non-square. One approach to obtain a solution is to use pseudo inverse, this approach is however computational intensive. This paper presents a pragmatic approach by which a joint of a 7-DOF whole arm manipulator (WAM) is initially fixed to facilitate the computation of the squared Jacobian matrix. Based on this approach, appropriate optimization strategies that are outlined in the paper, can then be applied to determine the optimal value of the 'fixed' joint in real time. Experiments are performed to verify the viability of this approach, and the results established that a robust and flexible Cartesian trajectory planning framework can be achieved for general redundant manipulators.

Recent applications of robots for industrial automation have shown significant improvement in manufacturing processes in terms of reducing labor participation, enhancing flexibility, efficiency and quality of the products. However, most applications are limited to point-to-point and noninteractive operations in which the availability of a highly structured setup is a prerequisite. This prompts the vast emergency of researches on intelligent robotics that are aimed to improve the adaptability, flexibility and dexterity so as to enhance the intelligence of industrial robots. This paper investigates the designs of intelligent robotic systems and discusses the proposed criteria required to achieve an intelligent robotic system. A proposed conceptual framework for robotic assembly is then presented that contains two main parts, namely, a robotic state recognizer and a control strategy generator. In addition to these two components, the integration of compliant motion control into the framework will be described. An example of using the proposed framework to develop a robotic assembly system is given.