The paper introduces ”Environment Question Answering (EnvQA),” an advanced task derived from Embodied Question Answering (EQA), aiming to enhance the practicality of robotic systems in real-world scenarios. Unlike EQA, which involves repeated exploration of familiar environments for answering questions, EnvQA integrates spatial memory and user feedback to address these limitations. The EnvQA system is designed to autonomously navigate, answer queries, and store environmental information using a cognition-inspired cognitive map with intermediate features of Vision-language models (VLMs). Additionally, the paper presents a new zero-shot image-image matching method, ViTPR, which utilizes large-scale vision transformers to achieve state-of-the-art performance in Visual Place Recognition (VPR) on the highly challenging Nordland dataset. Experimental results also demonstrate the effectiveness of global localization and question-answering in both simulated and physical environments. In summary, we successfully use VLMs to enhance the robot’s ability to understand and associate human questions with the environment.

As the impact of chronic mental disorders increases, multimodal sentiment analysis (MSA) has emerged to improve diagnosis and treatment. In this paper, our approach leverages disentangled representation learning to address modality heterogeneity with self-supervised learning as a guidance. The self-supervised learning is proposed to generate pseudo unimodal labels and guide modality-specific representation learning, preventing the acquisition of meaningless features. Additionally, we also propose a text-centric fusion to effectively mitigate the impacts of noise and redundant information and fuse the acquired disentangled representations into a comprehensive multimodal representation. We evaluate our model on three publicly available benchmark datasets for multimodal sentiment analysis and a privately collected dataset focusing on schizophrenia counseling. The experimental results demonstrate state-of-the-art performance across various metrics on the benchmark datasets, surpassing related works. Furthermore, our learning algorithm shows promising performance in real-world applications, outperforming our previous work and achieving significant progress in schizophrenia assessment.

This paper introduces a dual-modal early cognitive impairment detection system based on autobiographical memory (AM) tests, and our approach is to automatically extract pre-defined acoustic features and self-designed embeddings to enhance linguistic representation of the spontaneous speech data. By integrating dual-modal data, we effectively enrich the features that aid in model learning, especially addressing the subtle symptoms exhibited by individuals with mild cognitive impairment (MCI), an intermediate stage between healthy individuals and those with Alzheimer's disease (AD). To account for spontaneous speech's unstructured and implicit nature, two additional embeddings, namely, speaker embedding and conversation embedding, are introduced to augment the information available for model learning, thus enriching the feature set for improving the model accuracy. The proposed dual-modal approach is tested on a self-collected Chinese spontaneous speech dataset due to the limited unstructured speech open-access dataset for MCI detection. The system's effectiveness is evaluated through a series of experiments, including ablation studies, to determine the impact of each module on overall performance. The proposed system achieved an average accuracy of 78% in detecting MCI, demonstrating its comparative effectiveness. Enhancements in our system are achieved by integrating a directional encoder tailored to capture temporal information across sequential visits. This addition leads to a 3% increase in detection accuracy within a subset of participants who have undergone multiple AM test sessions. Implementing such a longitudinal approach in analyzing unstructured speech data for MCI detection taps into a relatively underexplored area of research, offering new insights.

Dementia affects 50 million people worldwide, underscoring the urgent need for effective interventions to enhance their well-being. While reminiscence intervention shows promise, its implementation is hindered by limited human resources, making machine-aided systems a viable automated solution for seamless photo-reminiscence sessions. In this paper, we introduce an empathetic response generation system specifically designed to enhance a question-only photo-reminiscence chatbot, with a focus on improving emotional context understanding and enhancing conversation engagement. We leverage Transformers to encode dialogue history, infer emotional states from user responses, and extract named entities. By combining template-based utterances with a retrieval chatbot, our system generates relevant and empathetic responses to user replies. Our system’s effectiveness is validated through human evaluations using a Likert-like scale to assess engagement levels. The results demonstrate that our approach surpasses both the question-only system and other models from existing works, including retrieval and generated models. This highlights our system’s potential to enhance interactions and engagement, advancing technology-driven interventions for dementia that improve well-being and quality of life.

Parkinson's disease (PD) is the cause of the gradual decline of nerve cells that control movement disorder disease, which is most common among the elderly in the US after Alzheimer's disease. There are several studies on detecting Parkinson's disease from speech using machine learning techniques; however, most of them focus on classifying healthy patients (HC) against Parkinson's disease (PD). This paper focuses on developing a screening system that could detect healthy (HC) vs. mild Parkinson's (MP) vs. severe Parkinson's (SP) from spontaneous speech. Four acoustic feature sets were compared for the screening system. Our proposed AeXGB was also compared with six different classifying approaches. The result has shown that extracting the phonation features and using AeXGB could achieve an accuracy of 92% for classifying HC vs. MP vs. SP, which outperforms the traditional machine learning approach for three class classifications of Parkinson's severity level from spontaneous speech.

As older adults' memory and cognitive ability deteriorate, designing a cognitive robot system to find the desired objects for users becomes more critical. Cognitive abilities, such as detecting and memorizing the environment and human activities are crucial in implementing effective human–robot interaction and navigation. In addition, robots must possess language understanding capabilities to comprehend human speech and respond promptly. This research aims to develop a mobile robot system for home care that incorporates human activity inference and cognitive memory to reason about the target object's location and navigate to find it. The method comprises three modules: 1) an object-goal navigation module for mapping the environment, detecting surrounding objects, and navigating to find the target object, 2) a cognitive memory module for recognizing human activity and storing encoded information, and 3) an interaction module to interact with humans and infer the target object's position. By leveraging Big Data, human cues, and a commonsense knowledge graph, the system can efficiently and robustly search for target objects. The effectiveness of the system is validated through both simulated and real-world scenarios.

Many LiDAR-based methods for detecting large objects, single-class object detection, or under easy situations were claimed to perform well. However, due to their failure to exploit image semantics, their performance in detecting small targets or under challenging conditions does not exceed that of fusion-based approaches. In order to elevate the detection performance in a complex environment, this paper proposes a multi-modal and multi-class 3D object detection network, named Voxel-Pixel Fusion Network (VoPiFNet). Within this network, we design a key novel component called the Voxel-Pixel Fusion Layer, which takes advantage of the geometric relation of a voxel-pixel pair and effectively fuses voxel features and pixel features with the cross-modal attention mechanism. Moreover, after considering the characteristics of the voxel-pixel pair, we design four parameters to guide and enhance this fusion effect. This proposed layer can be integrated with voxel-based 3D LiDAR detectors and 2D image detectors. Finally, the proposed method is evaluated on the public KITTI benchmark dataset for multi-class 3D object detection at different levels. Extensive experiments show that our method outperforms the state-of-the-art methods in detecting challenging pedestrian category and achieve promising performance in overall 3D mean average precision (mAP).

As eye-tracking technology becomes increasingly prevalent in augmented reality (AR), new opportunities arise for collaborative applications. In this paper, we propose a novel approach to improve collaborative interaction through object-based user intent detection using gaze data. Our system uses reinforcement learning (RL) to dynamically adapt the user interface based on the context of the collaborative task. The system visualizes the user’s intent on a shared environment, allowing for improved collaborative awareness between users. We evaluate our approach in a user study scenario focused on visual search tasks. The results demonstrate that our system significantly improves task completion times and reduces cognitive load for users. Additionally, subjective feedback suggests that users are more aware of each other’s activity, further highlighting the benefits of our approach. We encourage conducting future user studies to assess the suitability of our approach for additional collaborative tasks.

Being able to recognize emotional intensity is a desirable feature for a facial emotional recognition (FER) system. However, the development of such a feature is hindered by the paucity of intensity-labeled data for model training. To ameliorate the situation, the present study proposes using face morphing as a way of data augmentation to synthesize faces that express different degrees of a designated emotion. Such an approach has been successfully validated on humans and machines. Specifically, humans indeed perceived different levels of intensified emotions in these parametrically synthesized faces, and FER systems based on neural networks indeed showed improved sensitivities to intensities of different emotions when additionally trained on the synthesized faces. Overall, the proposed data augmentation method is not only simple and effective but also useful for building FER systems that recognize facial expressions of mixed emotions.

In this article, we propose a new 3-D maneuver controller for a class of nonlinear multiagent systems (MASs) with nonholonomic constraint and saturated control. The system is designed under a distributed communication topology and the controller is more flexible and efficient for general formation maneuver tasks. The saturation design generates control inputs within pregiven bounds, which makes the system more applicable in practice. Moreover, based on the nonholonomic model, the proposed control also considers the heading angles of the agents. Thus, the maneuver controller can achieve a more natural tracking movement where the heading of the formation will align to the direction of the reference trajectory during the tracking motion. Several simulation examples are given to validate our results and demonstrate the competence for various maneuver tasks of MASs.

In healthcare facilities, answering the questions from the patients and their companions about the health problems is regarded as an essential task. With the current shortage of medical personnel resources and an increase in the patient-to-clinician ratio, staff in the medical field have consequently devoted less time to answering questions for each patient. However, studies have shown that correct healthcare information can positively improve patients' knowledge, attitudes, and behaviors. Therefore, delivering correct healthcare knowledge through a question-answering system is crucial. In this article, we develop an interactive healthcare question-answering system that uses attention-based models to answer healthcare-related questions. Attention-based transformer models are utilized to efficiently encode semantic meanings and extract the medical entities inside the user query individually. These two features are integrated through our designed fusion module to match against the pre-collected healthcare knowledge set, so that our system will finally give the most accurate response to the user in real-time. To improve the interactivity, we further introduce a recommendation module and an online web search module to provide potential questions and out-of-scope answers. Experimental results for question-answer retrieval show that the proposed method has the ability to retrieve the correct answer from the FAQ pairs in the healthcare domain. Thus, we believe that this application can bring more benefits to human beings.

Traditional text classification requires thousands of annotated data or an additional Neural Machine Translation (NMT) system, which are expensive to obtain in real applications. This paper presents a Contrast-Enhanced Semi-supervised Text Classification (CEST) framework under label-limited settings without incorporating any NMT systems. We propose a certainty-driven sample selection method and a contrast-enhanced similarity graph to utilize data more efficiently in self-training, alleviating the annotation-starving problem. The graph imposes a smoothness constraint on the unlabeled data to improve the coherence and the accuracy of pseudo-labels. Moreover, CEST formulates the training as a “learning from noisy labels” problem and performs the optimization accordingly. A salient feature of this formulation is the explicit suppression of the severe error propagation problem in conventional semi-supervised learning. With solely 30 labeled data per class for both training and validation dataset, CEST outperforms the previous state-of-the-art algorithms by 2.11% accuracy and only falls within the 3.04% accuracy range of fully-supervised pre-training language model fine-tuning on thousands of labeled data.

Alzheimer’s disease (AD) and other types of dementia have become a public health priority worldwide. To lessen the burden of AD diagnosis, an automatic screening system that can be deployed in large-scale and cost-efficient screening methods will be needed. This paper presents a speech assessment system for cognitive impairment detection, detecting whether elders have AD or suffer from mild cognitive impairment (MCI) based on their audio recordings taken from neuropsychological tests. The audio waveform first is transformed to Mel-spectrogram and done the downsampling. With the combination of Transformer and convolutional neural network (CNN) architecture, we can do the feature extraction and get a better representation for the classifier. We conducted experiments on 120 subjects with a balanced distribution of ordinary aging, MCI, and AD patients to validate our study. Our experiments achieve an accuracy of 91% and 79% for classifying groups of AD and MCI from ordinary aging people, respectively.

As the elderly population grows globally, health-care systems face a burden from the rise in Alzheimer’s patients due to an increase in demand for early diagnosis. Therefore, more people have started focusing on developing systems helping doctors diagnose Alzheimer’s, such as cognitive impairment detection systems. This paper presents a contrast-enhanced automatic cognitive impairment screening system combining paused-encoder based on the automatic transcription. We use the pre-trained automatic speech recognition model and adapt it to generate transcripts of the elderly’s speech. The pattern of pauses in speech is a commonly-studied acoustic feature since it can provide additional information besides the semantic information for the model prediction. The back-translation with contrastive learning is used to improve the encoded model further. The model also fine-tunes with the pause-encoded transcriptions to detect the cognitive impairment. Our result shows excellent performance with an accuracy of 81% in detecting Alzheimer’s disease. Also, the accuracy is acceptable on a more challenging task of detecting mild cognitive impairment, the middle stage between healthy and Alzheimer’s. In addition to the outperforming performance, our system is fully automatic and can be used easily.

Schizophrenia is a mental disorder that will progressively change a person's mental state and cause serious social problems. Symptoms of schizophrenia are highly correlated to emotional status, especially depression. We are thus motivated to design a mental status detection system for schizophrenia patients in order to provide an assessment tool for mental health professionals. Our system consists of two phases, including model learning and status detection. For the learning phase, we propose a multi-task learning framework to infer the patient's mental state, including emotion and depression severity. Unlike previous studies inferring emotional status mainly by facial analysis, in the learning phase, we adopted a Cross-Modality Graph Convolutional Network (CMGCN) to effectively integrate visual features from different modalities, including the face and context. We also designed task-aware objective functions to realize better model convergence for multi-task learning, i.e., emotion recognition and depression estimation. Further, we followed the correlation between depression and emotion to design the Emotion Passer module, to transfer the prior knowledge on emotion to the depression model. For the detection phase, we drew on characteristics of schizophrenia to detect the mental status. In the experiments, we performed a series of experiments on several benchmark datasets, and the results show that the proposed learning framework boosts state-of-the-art (SOTA) methods significantly. In addition, we take a trial on schizophrenia patients, and our system can achieve 69.52 in mAP in a real situation.

The pain intensity level is one of the properties to know the patients’ status. In this work, we focus on estimating each individual’s self-reported pain metric called Visual Analogue Scale (VAS), which is considered the standard gold measurement in the triage system. The VAS pain score is highly subjective, and its range may vary significantly among different patients. To tackle these issues, we designed an end-to-end training deep learning model for the automatic measurement of VAS based on video facial recognition. We proposed a novel loss method named Distance Ordering. By using Distance Ordering, we can extract the features with ordinal meaning according to the ordinal relationship of pain intensity levels. Experimental results on the UNBC-McMaster Pain Archive Database benchmark show that the model we designed outperforms the other previous works and achieves the state-of-the-art performance with Mean Square Error (MSE), Mean Absolute Error (MAE), Intra-class correlation (ICC), and Pearson coefficient correlation (PCC). Also, the ablation studies demonstrate that our approach can improve the VAS estimation.

Overcrowding in the Emergency Department (ED) has become one of the most severe healthcare issues worldwide. A number of researchers have reported that many countries, including Taiwan, have a significant and noticeable increase in ED visits. This phenomenon, overcrowding in the ED, has caused several adverse effects not only on patients but also on the healthcare delivery process, quality of care, and care efficiency. However, the currently adopted five-level triage system, Taiwan Triage and Acuity Scale (TTAS), is insufficient to distinguish patients’ conditions into different priorities. Therefore, a system that could help to triage a patients’ conditions accurately is demanded. While most of the existing studies have only utilized retrospective data and used traditional machine learning as the approaches, which might not satisfy for clinical decisions and real-world situations, this paper proposed an interpretable novel triage prediction system for predicting hospitalization based on prospectively collected data in the emergency department, including vital signs and chief complaints. The performance of our proposed model is evaluated on our own collected data in National Taiwan University Hospital (NTUH), with 73 samples needing hospital admission and 73 samples discharged. Ten times of 10-fold cross-validation were done to evaluate the performance of our proposed model, a mean area under the receiver operating characteristic curve (AUROC) and mean accuracy are achieved at 0.836 and 0.805, respectively.

Advanced Driver Assistance Systems (ADAS) consists of two basic functions. One is the object detection for preventing vehicles from hitting pedestrians or other obstacles. The other is image segmentation for recognizing drivable areas and guiding the vehicle forward. For the latter, unlike those traditional image segmentation methods, image semantic segmentation based on deep learning architecture can handle the irregularly shaped road areas better, guiding a vehicle to drive in a more complex environment. With the popularity of Convolution Neural Networks (CNNs) in recent year, the traditional hand-crafted features methods have shown to be outperformed. However, deep CNN models are difficult to implement on vehicle application because the severe cost of time for complex processing. Although some proposed methods, such as Efficient neural network (Enet), achieved higher speed by removing some layers, it also led to the decrease of segmentation accuracy. In this research work, we propose a novel semantic segmentation network, Edgenet, which contains a class-aware edge loss module and a channel-wise attention mechanism, aiming to improve the accuracy with no harm to inference speed. We evaluate Edgenet on Cityscapes dataset, which is the most challenging and authoritative on-road semantic segmentation dataset. The results show that our proposed method can achieve over 70% mean IOU on Cityscapes test set and run at over 30 FPS in a single GTX Titan X (Maxwell) GPU.

There are limitations of conventional active-assistive control for upper limb rehabilitation exoskeleton robot, such as 1). prior time-dependent trajectories are generally required, 2). task-based rehabilitation exercise involving multi-joint motion is hard to implement, and 3). assistive mechanism normally is so inflexible that the resulting exercise performed by the subjects becomes inefficient. In this paper, we propose a novel velocity field based active-assistive control system to address these issues. First, we design a Kalman filter based interactive torque observer to obtain subjects' active intention of motion. Next, a joint-position-dependent velocity field which can be automatically generated via the task motion pattern is proposed to provide the time-independent assistance to the subjects. We further propose a novel integration method that combines the active and assistive motions based on the performance and the involvement of subjects to guide them to perform the task more voluntarily and precisely. The experiment results show that both the execution time and the subjects' torque exertion are reduced while performing both given single joint tasks and task-oriented multi-joint tasks as compared with the related work in the literature. To sum up, the proposed system not only can efficiently retain subjects' active intention but also can assist them to accomplish the rehabilitation task more precisely.

Human social interactions are laden with behavioral preferences that stem from hidden social network representations. In this study, we applied an artificial neural network with machine theory of mind (ToMnet+) to learn and predict social preferences based on implicit information from the way agents and social targets interact behaviorally. Our findings have implications for machine applications that seek to infer hidden information structures solely from third-person observation of behaviors. We consider that social machines with such an ability would have an enhanced potential for more naturalistic human-machine interactions.

Emergency Department (ED) crowding has become an issue of delayed patient treatment and even a public healthcare problem around the world. According to recent research studies of many countries, the increasing number of patients in the emergency department which has led to unprecedented crowding and delays in care. For that reason, triage into five-level Emergency Severity Index (ESI) has become a major method for improving medical priorities in ED. Although the ESI mitigates the process of ED treatment, so far it still heavily relies on the nurse's subjective judgment and is easy to triage most patients to ESI level 3 in current practice. Therefore, a system that can help the doctors to accurately triage a patient's condition is imperative. In this work, we propose a system based on the patients' ED electronic health record to predict hospitalizations after assigned procedures in ED are completed. While most of the related studies have employed traditional machine learning for triage-related classification and highly relied on a feature selection process, our proposed system used data-to-image transform to produce the input and a convolutional neural network as a classifier. For validation, the data from an open dataset (National Hospital Ambulatory Medical Care Survey) is used which includes 118,602 patient visits of United States EDs from 2012 to 2016 survey years. To sum up, the resulting AUROC and accuracy achieve 0.86 and 0.77, respectively, in our work.

Because of the worldwide aging population, more and more elders suffer from dementia problem. Nowadays, it is an inconvenient and time-consuming process for medical doctors to diagnose elders who live independently with possible dementia because the process imposes a large quantity of diagnostic questions from a checklist that needs to be answered by elders themselves or their caregivers either directly or after a long-term observation. In order to help doctors to make this diagnostic process easier, this article proposes a supporting system that can quickly estimate the likelihood for an elder of having dementia based on 2 to 4 hours monitoring of a behavioral test done by the elder. During the test, the elder only needs to perform certain activities selected from the so-called instrumental activities of daily living (IADL) in a smart home environment, and their movement trajectories will be extracted from motion sensors deployed in the smart home environment and be analyzed to find a potential correlation with the indoor wandering patterns. A machine learning algorithm is selected to carry out the classification task, namely, into dementia and nondementia groups, based on our proposed features of the aforementioned wandering patterns. Two data sets are employed for performance evaluation, where the first one is 232 elders including seven dementia, whereas the second one is collected by ourselves from a senior center, which is 30 elders including nine dementia. It turns out that the average precision and recall for the first data set are both up to 98.3% with area under the ROC curve (AUC-ROC) being 0.846, and those for the second data set are 89.9% and 90.0% with AUC-ROC being 0.921. Note to Practitioners-We proposed a supporting system which can classify the elders as either dementia or nondementia with high accuracy. The trajectories of the elders will be extracted from motion sensors that deployed in the smart home environment. The indoor wandering patterns according to repetitive movements are analyzed and classified using the machine learning technique. The proposed system used ambient sensors instead of wearable sensors or cameras to let the elders feel more comfortable when they are being monitored. In addition, the proposed system only required a short period of time to screen the elders and easier for medical doctors to diagnose the elders without wasting time for asking the large quantity of diagnostic questions from a checklist that needs to be answered by the elders themselves or their caregivers.

The synchronization of the movement between exoskeleton robot and human arm is crucial for Robot-assisted training (RAT) in upper limb rehabilitation. In this paper, we propose a deep learning based motion prediction model which is applied to our recently developed 8 degrees-of-freedom (DoFs) upper limb rehabilitation exoskeleton, named NTUH-II. The human arm dynamics and surface electromyography (sEMG) can be first measured by two wireless sensors and used as input of deep learning model to predict user's motion. Then, the prediction can be used as desired motion trajectory of the exoskeleton. As a result, the robot arm can follow the movement on either side of the user's arm in real-time. Various experiments have been conducted to verify the performance of the proposed motion prediction model, and the results show that the proposed motion prediction implementation can reduce the mean absolute error and the average delay time of movement between human arm and robot arm.

In this paper, we propose a novel system architecture called multi-layer environmental affordance map for social and service companion robots. Based on this architecture, robots can organize the perception and inference information efficiently and generate social friendly navigation strategies. In other words, robots are able to strengthen their perception and inference abilities to interact with domestic environment and users under our efficient framework. The main feature of this architecture is that the relations between layers can be viewed as affordances to improve the accuracy and the robustness of the detection and inference. The results show that our architecture achieves robust indoor localization, scene localization, human event detection and socially friendly navigation in real time under limited computational resource.

Since deep-learning-based method has been widely-used and is capable of generating generic model, most existing methods about action recognition use either two-stream structure, considering spatial and temporal features separately, or C3D, costing lots of prices in memory and time. We aim to design a robust system to extract spatiotemporal features with aggregation mechanism to integrate local features in temporal order. In light of this, we propose ResFlow to estimate optical flow and predict action recognition simultaneously. Leveraging the characteristic of optical flow estimation, we extract spatiotemporal feature via an autoencoder. Via a novel Sequentially Pooling Mechanism which literally pool global spatiotemporal feature sequentially, we extract spatiotemporal feature at each time and aggregate these local features into global feature. This design use only RGB images as input with temporal information encoded, pre-trained by optical flow, and sequentially aggregate spatiotemporal features in high efficiency. We evaluate our ability of estimating optical flow on FlyingChairs dataset and show the promising results of action recognition on UCF-101 dataset through a series of experiments.

In this work, we created an end-to-end autonomous robotic platform to give emotional support to children in long-term, multi-session interactions. Using a mood estimation algorithm based on visual cues of the user's behaviors through their facial expressions and body posture, a multidimensional model predicts a qualitative measure of the subject's affective state. Using a novel Interactive Reinforcement Learning algorithm, the robot is able to learn over several sessions the social profile of the user, adjusting its behavior to match their preferences. Although the robot is completely autonomous, a third party can optionally provide feedback to the robot through an additional UI to accelerate its learning of the user's preferences. To validate the proposed methodology, we evaluated the impact of the robot on elementary school aged children in a long-term, multi-session interaction setting. Our findings show that using this methodology, the robot is able to learn the social profile of the users over a number of sessions, either with or without external feedback as well as maintain the user in a positive mood, as shown by the consistently positive rewards received by the robot using our proposed learning algorithm.

The following topics are dealt with: learning (artificial intelligence); medical signal processing; neurophysiology; electroencephalography; mobile robots; feature extraction; brain-computer interfaces; neural nets; convolutional neural nets; production engineering computing.

Alzheimer's disease and the other type of dementia have become one of the most serious global issues and the fifth leading cause of death worldwide nowadays. Therefore, early detection of the disease is crucial in order to improve the quality of life of the patients and to decrease the burden of their caregiver and clinicians. Mild cognitive impairment (MCI) is a prodromal stage of progressing to Alzheimer's disease which should be focus on. In this paper, we have proposed a screening system based on the Rey-osterrieth Complex Figure, a neuropsychological test, that can automatically assist the clinicians to detect whether the subject is MCI or not. A data-driven deep learning approach is implemented in this work. Convolution autoencoder is designed initially to extract features from the input image. The features learned by the encoder are then used for further training the classifier. In order to validate the performance of our work, 59 MCI subjects and 59 healthy controls are recruited under the approval of institutional review board from the National Taiwan University Hospital. The performance of our proposed model is evaluated by using 10-fold cross-validation and it is repeated five times. As a result, a mean area under the receiver operating characteristic curve score of 0.851 and 0.810 of accuracy are achieved.

Atomic force microscope (AFM) is capable of constructing accurate 3-D surface profile at a nanometer resolution. This paper demonstrates the amplitude-detection mode atomic force microscopy (AM-AFM) with long-range of the modified cycloid trajectory. For this purpose, the proposed system contains three axis scanner including the hybrid xy-scanning subsystem and the z-measuring subsystem. Besides, the internal model principle-based neural network complementary sliding mode control (IMP-based NNCSMC) approach of designing controller is implemented for the xy-piezoelectric scanner to overcome some uncertainties, i.e. hysteresis and cross-coupling effect. On the other hand, the neural network complementary sliding mode control (NNCSMC) scheme is employed on controller design along the z-piezoelectric scanner to precisely trace the topography change based on amplitude feedback signals. Due to incorporating the piezoelectric leg-based long traveling range nano-positioning stage (LTRPS) in xy-plane, an accurate AFM imaging can be obtained with the modified cycloid trajectory for long-range scanning.

Alzheimer's disease has become one of the biggest challenges in the healthcare system worldwide. Researches have shown that Alzheimer's disease is the sixth leading cause of death in the United States and even the fifth leading cause among people aged 65 and older. Moreover, the number of patients is escalating rapidly in recent years, which also increases the burden on the healthcare system. Therefore, a screening system that can help the doctor to diagnose Alzheimer's disease is demanded. In this paper, we proposed a screening system based on the transcripts of speeches spoken by subjects undertaking a neuropsychology test. While most of the related studies have utilized extracted syntactic and semantic features and relied on a feature selection process, the proposed system used word vectors as the representation of a spoken speech, and Recurrent Neural Network together with attention mechanism as the classifier. Using ten times 10-fold cross validation on an open dataset with 242 speeches samples spoken by healthy controls and 257 samples spoken by subjects with Alzheimer's disease, a mean accuracy of 0.835 is achieved in our work, which is outperforming the current state-of-the-art while requiring less effort.

In this paper, we propose joint-oriented features for skeleton-based action recognition, which aims to decrease the influence of ambiguous joints in the skeleton sequences. When doing skeleton-based action recognition, we noticed that if the skeleton data contains noisy joints, the result would be influenced by the noise. Since the selections of disambiguous joints might be impossible, it would be a hard task for humans to distinguish whether the joint in the frame is correct. To deal with this situation, we propose joint-oriented features to train joint-oriented models. If some joints are noise in the frame, the corresponding joint-oriented models would not perform well on the case. As we could not preFigure the noise in the data, we apply ensemble modeling with our joint-oriented models to let the disambiguous joints correct the ambiguous ones. To demonstrate the effectiveness of our proposed method, we conducted the experiments on three benchmark skeleton-based action datasets, including the large-scale challenging NTU-RGBD, and our approach achieves competitive performance over the state-of-the-art.

Traditional robotic walkers have primarily focused on safety and navigation. In this paper, we challenge the previous work on walkers by implementing a deep learning module developed with the goal of using a robot to provide mobility assistance to the elders. Through the data collected from multiple sensors, we are capable of leveraging the maneuverability of robotic walkers under different scenarios and gait requirements. This capability is achieved by CNN (Convolutional Neural Networks) and LSTM (Long Short-Term Memory) architectures. Thus, the system can provide personalized assistance to the elders performing the locomotion activities indoors accurately. Furthermore, the robot learns the optimal behavior based on the interactions with the environment in a supervised learning approach. To validate our system, we evaluated the system with some users who provided qualitative comments about the comfort degree of using the robot.

In smart grids, demand-side management (DSM) has become an important topic since it can reduce the total electricity cost by smart control and rescheduling of loads, meanwhile, reduce the peak-to-average ratio (PAR) under real-time pricing policy. On the other hand, with the growth of computation ability in recent years and the huge amount of data collected in home area network (HAN), machine learning skills such as reinforcement learning can be well applied into the DSM problem. However, it is hard to determine an optimal energy management strategy since the uncertainty of user behavior and the electricity consumption. In the proposed work, a real-time multi-agent deep reinforcement learning based approach has been proposed to solve the DSM problem in HAN and additionally considers the user behavior to avoid disturbing user comfort, meanwhile, adaptively learns the appliance usage preference and updates the system after each day. The simulation results reveal that the proposed DSM system has improved the energy efficiency in a smart home that not only reduces the electricity cost and peak value but also the PAR value.

We propose a new navigation method for an omnidirectional mobile robot to maneuver in a complex and populated environment. In an indoor populated environment, for example, robot has to react to such circumstances and achieve both physical and psychological navigational safety. From social researches, human motion is a hybrid system composed of holonomic and non-holonomic movements. Based on this concept, we develop a socially-aware navigation method for an omnidirectional mobile robot to achieve natural, humanlike movement. By using laser range finder and camera as sensors, robot detects geometric features and human behavior information. From the geometric features robot can construct an environment model and extract the information about distances and directions of the obstacles. Whereas with the heading and orientation information from camera, robot can model individual humans successfully. While acquiring these models as context, a suitable navigation behavior would be executed. To interact with surroundings, we develop the extended Social Force Model(ESFM) to describe the interactive force, referred to as social force, with human and environment. As a whole, the contributions of our work are twofold one being that we propose a dynamic grouping model based on human behavior using learning-based method and another being that we develop an extended Social Force Model for the system based on which a successful navigation strategy can be realized.

Throughout the course of life, there is time when we live independently in our house without anyone to look after each other. In order to support these people and ensure their safety with limited medical resources and human labors, it is important to constantly monitor one's activity of daily living (ADL). Therefore, we propose an activity recognition (AR) system for people living independently in smart homes to achieve the concept of “aging in place.” The AR model adopted by the proposed system is powerful to recognize meaningful ADL by integrating heterogeneous data from both ambient and on-body sensors. Moreover, this proposed system adopts a nonparametric approach, which requires much fewer efforts from humans. The average AR precision and recall rates of this proposed system are up to 98.7% and 99.0%, which indicates its feasibility of deployment in a real-life home environment for monitoring users' ADL with promising performance and thus helps realize “aging in place.” Note to Practitioners-With the advance of Internet of Things technologies, it is convenient to collect data about ADL in a smart home environment via ambient sensors and on-body sensors. This proposed system integrates data from these two heterogeneous sensors and discovers potential activities automatically without user labeling or parameter setting. By reducing the above-mentioned user efforts, it is more suitable for users and thus greatly helps realize the concept of “aging in place.”

For a service robot, it is not adequate to let its navigational movement be based only on a single metric, such as minimum distance path. In the environment where the robot and humans are coexisting, the robot should always perform social navigation whenever it is moving. However, to perform social navigation, the robot needs to follow certain “social norms” of the environment. Recently, deep reinforcement learning (DRL) technique is popularly applied to the robotics field; yet, it is rarely used to solve the mentioned social navigation problem, generally deemed as a high dimension complex problem. In this paper, we propose the composite reinforcement learning (CRL) framework under which the robot learns appropriate social navigation with sensor input and reward update based on human feedback. For learning the aspect of human robot interaction (HRI), we provide a method to facilitate the training of DRL in real environment by incorporating prior knowledge to the system. It turns out that our CRL system not only can incrementally learn how to set its velocity and to perform HRI but also keep collecting human feedback to synchronize the reward functions to the current social norms. The experiments show that the proposed CRL system can safely learn how to navigate in the environment and show that our system is able to perform HRI for social navigation.

Recently, as the rise of deep reinforcement learning, it not only can help the robot to convert the complicated environment scene to motor control command directly but also can accomplish the navigation task properly. In this paper, we propose a novel structure, where the objective is to achieve navigation in large-scale indoor complex environment without pre-constructed map. Generally, it requires good understanding of such indoor environment to make complex spatial perception possible, especially when the indoor space consists of many walls and doors which might block the view of robot leading to complex navigation path. By the proposed distributed deep reinforcement learning in different local regions, our method can achieve indoor visual navigation in the aforementioned large-scale environment without extra map information and human instruction. In the experiments, we validate our proposed method by conducting highly promising navigation tasks both in simulation and real environments.

An approach of robust localization for mobile robot working in indoor is proposed in this paper. A novel method for laser SLAM and visual SLAM fusion is introduced to provide robust localization. This architecture can be applied to a situation where any two kinds of laser-based SLAM and monocular camera-based SLAM can be fused together instead of being limited to single specific SLAM algorithm. While laser-based SLAM and monocular camera-based SLAM have their own strengths and drawbacks, the integration of these two kinds of SLAM algorithm can then promote the algorithmic effectiveness. Instead of using feature matching methods to achieve fusion procedure, trajectories matching is proposed with an attempt to achieve the generalization over all different kinds of SLAM algorithms, since localization is a natural function associated with any SLAM algorithm. It turns out that the hereby proposed approach is very lightweight during the run time, and the calculation can run in real-time without unnecessary computation waste. The experimental results show the localization error in terms of the real distance can be less than 5%. Furthermore, through the experiment the proposed system can be shown able to improve the localization when the sensors are not very powerful.

Alzheimer disease and other dementias have become the 7th cause of death worldwide. Still lacking a cure, an early detection of the disease in order to provide the best intervention is crucial. To develop an assessment system for the general public, speech analysis is the optimal solution since it reflects the speaker's cognitive skills abundantly and data collection is relatively inexpensive. While most of the related studies extracted statistics-based features and relied on a feature selection process, we have proposed a novel Feature Sequence representation and utilized a recurrent neural network to perform classification in this paper. To validate our work, an experiment has been conducted with 150 speech samples, and the score in terms of the area under the receiver operating characteristic curve is as high as 0.954, potentially outperforming the current state-of-the-art method.

Demand-side management (DSM) is a very important topic in recent years thanks to the growth of Electric Vehicle(EV) and the renewable energy nowadays. DSM ability in a residential area has been improved a lot under the enforcement of real-time pricing(RTP) mechanism. In this work, we aim to integrate the residential user and commercial user into one DSM system. By integrating them, we can utilize the energy more efficiency by sharing the surplus energy in residential area to the commercial area. On the other hand, optimization is always a crucially important process to be adopted in a DSM system, and particle swarm optimization (PSO) turns out to be a popular optimization method for solving DSM problem. In this work, we enhance the PSO by adding two popular concepts in deep learning and call it improved particle swarm optimization(IPSO). The simulation results reveal that the proposed DSM system has improved the energy efficiency in community, and at the same time, the energy costs paid by both residential and commercial users will be reduced.

Facial expression recognition(FER) is a challenging task even for human since individuals have their own way to express their feelings with different intensity. In order to extract commonality of facial expressions from different individuals, personality effect of individual needs to be minimized as much as possible. In this paper, we present a temporal-contrastive appearance network (TCAN) that utilizes the temporal feature to remove the personality effect. The high level feature is extracted from a video consisting of a sequence of frames through a proposed by convolutional neural networks (CNN). In order to let our CNN framework be able to extract similar features from adjacent frames, special loss function is introduced. Moreover, the neutral and peak expression frames are identified through comparison of distances among frames. Then, facial expressions can be classified by the so-called contrastive representation between neutral and peak expressions. We conducted our experiment in the most widely used databases (CK+ and Oulu-CASIA) for facial expression recognition. The experiment results show that the proposed method outperforms those from the state-of-the-art methods.

3D hand pose estimation is a hot research topic in recent years. It's been widely used in many advanced applications for virtual reality and human-computer interaction, since it provides a natural interface for communication between human and cyberspace. Despite the fast development of this field, it is still a difficult task due to the various challenges. In this paper, we aim to build a 3D hand pose estimation system which can correctly detect human hands and accurately estimate its pose using depth images. To guarantee the robustness of our system, we design a hand model called spherical part model (SPM), and train a deep convolutional neural network using this model. Moreover, to reduce the influence of human's omissions, we use a data-driven approach to integrate them together. Our network can more accurately estimate hand pose based on prior knowledge of human hand. To demonstrate the superiority of our method, a complete experiment is conducted on two public and one self-built datasets. The results show that our system can detect human hands with average precision at almost 90% and the average error distance of the pose estimation is about 10 millimeters, and is better than the other state of the art works.

This paper proposes a new method for human posture recognition from top-view depth maps on small training datasets. There are two strategies developed to leverage the capability of convolution neural network (CNN) in mining the fundamental and generic features for recognition. First, the early layers of CNN should serve the function to extract feature without specific representation. By applying the concept of transfer learning, the first few layers from the pre-learned VGG model can be used directly without further fine-tuning. To alleviate the computational loading and to increase the accuracy of our partially transferred model, a cross-layer inheriting feature fusion (CLIFF) is proposed by using the information from the early layer in fully connected layer without further processing. The experimental result shows that combination of partial transferred model and CLIFF can provide better performance than VGG16 [1] model with re-trained FC layer and other hand-crafted features like RBPs [2].

As robots participate in human's daily activities more and more frequently, mobility performance has become one of the main factors determining how robots will share an environment with humans harmoniously in the near future. Among several different kinds of mobile platforms, using omnidirectional configurations is gradually becoming a trend in the robotics community; however, few researchers have addressed the impact of omnidirectional mobility from the perspective of human-robot interaction (HRI). In this paper, we have proposed a socializing model for the robot while participating in an interaction with a group of human peers to achieve its socially optimal position. From a theoretic perspective, we first identify the most prominent features required for social acceptance of robots interacting with multiple humans, backing our arguments with relevant sociological theory. To validate our results, we have conducted experiments where human participants were invited to interact with a robot, which can be constrained to perform either holonomic or nonholonomic motions only. Then, through an observer survey, we testify the appropriateness of utilizing omnidirectional mobility and verify the promotion of social acceptance using the aforementioned features, which is a goal that both the HRI and robotics communities aim to achieve.

In an ageing society, we expect that a robotic caregiver is able to persuade the elderly to perform a healthier behavior. In this work, pragmatic argument is adopted to make the elderly realize that a choice beneficial for health is really worthwhile, such as eating suitable fruits. Based on this concept, an adaptive recommendation dialogue system through pragmatic argumentation is proposed. There are three objectives in this system. First, a knowledge base for pragmatic argument construction is built, which concerns not only the effect of a decision but also the reason for the effect. Secondly, the robot is endowed with the ability to do recommendation that adapts to different states of the elder, and the recommendation is determined based on the integration of both the robot's and the elder's preference for different perspectives so that the robot knows how to reach a compromise with the elder. Lastly, through learning about the elder's preference for perspectives in conversation, the robot will try to select such a perspective to construct arguments that the elder can be more easily convinced to accept its recommendation. We invited 21 volunteers to interact with the robot. The experimental result has proved that the recommendation system has potential to affect the decision making of the elderly and help him/her pursue a healthier life.

In smart grids, demand-side management (DSM) is one of the important function for both customers and utility, since it can reduce the total electricity cost of each customer, meanwhile, alleviate the aggregate peak-to-average ratio (PAR) subjected to real-time pricing (RTP) policy. On the other hand, while bidirectional charging/discharging Plug-in Electric vehicles (PEV) become more general, the capability of storing electrical energy for load shifting may take smart grid to a next level. This works aims at integrating PEV into DSM system, which considers renewable energy and energy trading as well. As it comes to community, we design a fairness strategy to share PEV's battery with neighbors to reduce the total electric cost and peak to average ratio (PAR). The simulation results show that the proposed DSM system not only meets the requirement of the PEV and reduce the cost for each household but also creates a win-win situation through the energy trading among homes based on consideration of both fairness and privacy protection in a residential community.

Because of the worldwide aging population, more and more elders suffer from dementia. Nowadays, it is inconvenient and time-consuming for doctors to diagnose whether elders who live independently have dementia because lots of diagnostic questions on a checklist must be asked first, and part of them even require a long-term observation. In order to help doctors and make this diagnostic process easier, we proposed a supporting system that can quickly screen the elders and estimate the likelihood of them having dementia based on a behavioral test in 2 to 4 hours. During the behavioral test, the elders only need to perform some activities selected from so-called Instrumental Activities of Daily Living (IADL) in a smart home environment, and a machine learning algorithm is adopted to carry out the classification based on our proposed features extracted from motion sensors deployed in the smart home environment. Our system supports the classification of two classes, Dementia and Non-Dementia, and its average precision and recall are both up to 98.3%. Besides, the value of Area Under the ROC Curve (AUC) is 0.851.

For a visual servo system, there usually exists the problem of time-delay likely caused by long image processing and data transmission. The visual servo system of our robot is subject to two main limitations stemming from the specific commercial mobile manipulator, of which one is the large time-delay due to image transmission, whereas the other is failure to directly command each joint velocity as the visual servo's control input. In this paper, we propose a novel visual servo system to compensate the large time-delay, and we use integral of velocity which incorporating the image estimation and prediction with the kinematic model so as to achieve indirect velocity control with position based control mode joint. Our proposed visual servo system controls the neck joints of robot and keeps the object within the field of view of the camera during the approaching phase. We evaluate the proposed approach by experiment on a real wheeled humanoid robot and find the resulting performance quite promising, showing that our method is feasible and practical.

Most previous work in household energy conservation has focused on rule-based home automation to achieve energy savings, with relatively few researchers focusing on context-aware technologies. As a result, user comfort is often disregarded and few solutions handle decision conflicts caused by multiple activities undertaken by multiple users. The main contribution of this work is twofold. First, a comprehensive human-centric and context-aware comfort index is proposed to evaluate how users feel under particular environmental conditions with regard to thermal, illumination, and appliance-usage preferences. Second, the energy savings is formulated into an optimization problem to minimize the total energy consumption, even under multiple user comfort constraints. Short-term evaluation in our simulated home environment resulted in energy savings of at least 28.98%. Long-term evaluation using a home simulator resulted in energy savings of 33.7%. Most importantly, the energy savings in both situations was achieved under multiple user comfort constraints, representing a truly human-centric living environment.

We propose a novel framework which integrates human hand detection and pose estimation into one single pipeline. Unlike most of previous works which only focus on the pose estimation part subject to some strong assumptions or relying on a weak detector to detect human hands, we employ a deep learning architecture to complete both aforementioned tasks. By letting three different neural networks share the convolutional layers, this deeply learning architecture can efficiently and accurately detect human hands and compute their hand pose configuration. Moreover, we propose a new energy function to optimize the predicted result of convolutional neural network. To validate the proposed framework, experiments have been conducted and the results show that our approach is highly reliable and suitable for real-world applications.

In this paper, we present an efficient framework for human activity recognition in daily environment. We use depth information mainly for privacy protection, and then focus on the motion analysis of informative body parts, since most activities are much associated with these particular parts, e.g., head and hands in upper body. Based on the idea, we propose two novel features with intuitive physical meaning, which are Histogram of Located Displacements (HOLD) and Local Depth Motion Maps (L-DMM) based Gabor representation. They can capture discriminative posture and motion cues from skeletal joints and depth data respectively. Combing the advantages of joint and depth features as well as emphasizing the reliable parts can enhance the robustness of classification ability. The experimental results show that our proposed feature representation is very discriminative for the task of daily activity recognition and outperforms several state-of-the-art methods.

Many neurological or orthopedic disorders may cause motor impairments of shoulder. Patients require intensive training in order to recover from those impairments. However, the intensive training will lead to growing demand for strenuous work. Robot-aided therapy is able to alleviate the therapist's laborious burden and to provide more information about patient's condition during the therapy. Generally, the requirement of active therapy arises during the later stage of recovery. To realize the active therapy, a useful active control of the robot and effective recovery assessment for the active robot-aided therapy are essential. In this paper, a new active control method and an active therapy protocol with assessment indexes are designed. In this pilot research, several experiments on healthy subjects are designed to verify the proposed method with its active control performance and the effectiveness of the assessment indexes.

In this paper, we propose an indoor action detection system which can automatically keep the log of users' activities of daily life since each activity generally consists of a number of actions. The hardware setting here adopts top-view depth cameras which makes our system less privacy sensitive and less annoying to the users, too. We regard the series of images of an action as a set of key-poses in images of the interested user which are arranged in a certain temporal order and use the latent SVM framework to jointly learn the appearance of the key-poses and the temporal locations of the key-poses. In this work, two kinds of features are proposed. The first is the histogram of depth difference value which can encode the shape of the human poses. The second is the location-signified feature which can capture the spatial relations among the person, floor, and other static objects. Moreover, we find that some incorrect detection results of certain type of action are usually associated with another certain type of action. Therefore, we design an algorithm that tries to automatically discover the action pairs which are the most difficult to be differentiable, and suppress the incorrect detection outcomes. To validate our system, experiments have been conducted, and the experimental results have shown effectiveness and robustness of our proposed method.

In this paper, we propose a sensory cues guided robotic walker for improving gaits of Parkinson Disease (PD) patients. A completely non-intrusive, 3D real-time leg pose tracking and gait analysis are proposed by using a depth camera mounted on the rear of the robotic walker. It has been studied that the sensory cues can serve as effective stimuli to the PD patients for gait improvement. In our work, the sensory cues including visual and auditory cues are incorporated into the robotic walker. More specifically, both sensory cues are gait-adaptive, of which the visual cue in particular is projected onto the ground by a projector installed on the walker, in order to stimulate patients walking gait more easily. Since the adjustable cues can improve patients gaits and reduce their uncomfortableness simultaneously, the hereby developed robotic walker serves as a gait rehabilitation mechanism. To demonstrate the performance of the developed walker, several real experiments have been conducted. First, the accuracy of the proposed 3D leg pose tracking is verified by a standard motion capture system. Next, seven participants (4PD patients and 3 healthy elders) are recruited to test the system three days for verifying the effectiveness of participants gait improvement. The experimental results confirm the potential of the walker serving as a rehabilitation device for PD patients. Note to Practitioners-We proposed a completely non-intrusive, relatively inexpensive and real-time gait analyzer integrated with sensory cues guided rehabilitation system on an active robotic walker. Our goal is to provide a smart robotic walker with safety, reliability, and with rehabilitation function for elders whoever suffered from chronic disease or health problem. In this work, a reliable 3D leg pose tracking is proposed. Then, a gait analysis for acquiring spatio-temporal gait parameters such as stride length and gait velocity is proposed based on the tracking to analyze the walking gait of the patients. On the other hand, improving Parkinson disease patients gaits by using sensory cues which are known to have remarkable effects for Parkinson disease patients is in many research. Therefore, we concentrate on the sensory cues include visual cue and a rhythmic auditory cue cooperating with the robotic walker which aimed to stimulate and provide walkingassistance in their ambulation. For the sensory cues, an adaptive gait mechanism is proposed to improve patients gait based on their personal gait pattern as well as to reduce uncomfortableness of them by doing the rehabilitation. The experimental results confirm the potential of the walker serving as a rehabilitation device for Parkinson disease patients. We hope the new invention of this kind assistive robotic walker will become more popular and may provide more living aid facilities to elders.

In this paper, we propose a gait tracking and analysis method using a depth image sensor installed on robotic walker. Nowadays, robotic walker not only can assist elders who have suffered deteriorating mobility but also help to provide rehabilitating function to people who have crippled walking ability. This approach is meant to be sufficiently accurate, non-intrusive, and low-cost. The goal of this research is to enable the robotic walker to become more active in terms of walker control and comfortable user experiences thorough gait analysis. In experiment, the accuracy of the proposed 3D leg pose tracking method was evaluated by a motion capture system and the result is quite promising. After actual trials, the proposed method has the potential to be used in a safer and more reliable application such as those assisting elders and specific group of patient subjects.

Object search is a fundamental ability for a service robot to provide higher level services. We focus on object search in an environment with limited free space to place objects and constrained viewpoints to observe the environment, such as a shelf or a cupboard. We propose an object search planner based on A* search algorithm with tree node sampling. The proposed approach also combines visual and arm manipulation search. In other words, the robot searches occluded target object by either repositioning one of the accessible object with its arm or moving its platform to view the environment from a different pose. We evaluate the proposed approach with experiment performed by real robot in the scenario which objects may occlude or block access to one another.

The high development of medicine causes the world's population aging quickly. To resolve the problem with limited medical resources, constant monitoring of elders' activity of daily living is important. We propose an activity recognition system for smart home, so elders can live alone and their children can monitor their parents' living activity to achieve the concept of "Aging in Place". The living activity monitoring model is powerful to recognize meaningful activities by using both ambient and wearable sensors. It's feasible to deploy in the real living environment be-cause it's a non-parametric learning model. Elders need less effort to label activity in training part, and the model may have chance to find some special activities that the elders did not consider in the past. We demonstrate the living activity monitoring model is feasible to be deployed in a living home with high accuracy performance of the activity recognition result.

In this paper, a novel feature for activity recognition from vertical top-view depth image sequences is firstly proposed. Most of previous works are focusing mainly on the side-view depth or color image sequences, which unfortunately may encounter occlusion problems. Therefore, top-view camera setting is adopted in our research. Based on the idea of computed tomography (CT) from medical imaging, the depth images are segmented to different layer along the transverse plane. The representative body points which are found from the centroids of the regions on each slice. And those points will be a meaningful descriptor for the activity posture. Dynamic time warping algorithm is also applied to address the different sequence length problem. Finally, a SVM classifier is trained to classify our activities. To verify our performance, a new Top-View 3D Daily Activity Dataset is constructed. In our experiments, a challenging cross-subject test is conducted, and the performance of our representative body points is demonstrated. The result shows that the accuracy can achieve up to 97%, which is promising while being compared with those from the state-of-the-art methods in the literature.

With the popularity of Internet of Things, lots of resource constrained devices equipped with sensors and actuators are pervasively deployed to compose a smart environment, and Big Data are obtainable for a system to do further analytics thus to achieve human-centric purposes. One such human-centric system is a smart home which analyze Big Data to recognize contexts and their corresponding preferences for service configuration thus to provide context-aware services. However, since these Big Data are generated in real-time with huge amount, analytics based on conventional supervised way is not desirable due to the requirement of human efforts. In addition, there are usually multiple inhabitants with multiple combination of contexts in a home environment, and it is difficult to fully collect all these possible context combination as well as their corresponding preferences in advance. Therefore, this paper proposes an unsupervised nonparametric analytics method with a framework for human-centric smart homes to automatically discover contexts and their corresponding service configurations, and the models resulting from the proposed analytics can also be used to determine the preference for a context combination unseen before.

Pedestrian detection is an important issue in the field of intelligent transportation systems. As a pedestrian is not an apparent object at nighttime, it brings about critical difficulties in effectively detecting a pedestrian for a driving assistant vision system. While using an infrared projector to enhance the illumination contrast, objects in a nighttime environment might reflect the infrared projected by the emitted spotlight. In some cases, however, the clothes on a pedestrian might absorb most of the infrared, thus causing the pedestrian to be partially invisible. To deal with this problem, a nighttime part-based pedestrian detection method is proposed. It divides a pedestrian into parts for a moving vehicle with a camera and a near-infrared lighting projector. Due to a high computation load, selecting effective parts becomes imperative. By analyzing the spatial relationship between every pair of parts, the confidence of the detected parts can be enhanced even when some parts are occluded. At the last stage of this system, the pedestrian detection result is refined by a block-based segmentation method. The system is verified by experiments, and the appealing results are demonstrated.

Due to the proliferation of smart mobile devices and the developments in wireless communication, mobile ad hoc networks (MANETs) are gaining more and more attention in recent years. Routing in MANETs is a challenge, especially when the network contains a large number of nodes. The clustering technique is a popular method to organize the nodes in MANETs. It divides the network into several clusters and assigns a cluster head to each cluster for intra- and inter-cluster communication. Clustering is NP-hard and needs to consider multiple objectives. In this paper we propose a Pareto-based ant colony optimization (ACO) algorithm to deal with this multiobjective optimization problem. A new encoding scheme is proposed to reduce the size of search space, and a new decoding scheme is proposed to generate high-quality solutions effectively. Experimental results show that our approach is better than several benchmark approaches.

Frozen shoulder is a functional disorder related to shoulder muscles. Among many treatment strategies, rehabilitation exercise is one of the most common and viable therapy. In this research, a new 8 degrees of freedom (DOFs) rehabilitation robot arm named NTUH-II has been developed. The robot arm is able to provide most of upper limb rehabilitation motions in passive, active, and assistive modes. From the nature of muscle stretching, the exponential torque-angle relationship curve can be found, and thus it is possible to model and evaluate the condition of the patient's motion quality. The use of two parameters, stiffness and control authority, is proposed in this work. Based on these two parameters, an adjustment method of dynamic torque gain is developed and implemented on NTUH-II. Various experiments have been conducted, and appealing performance has been observed, which validates the method proposed in this paper.

This paper proposes a real-time indoor surveillance system which installs multiple depth cameras from vertical top-view to track humans. This system leads to a novel framework to solve the traditional challenge of surveillance through tracking of multiple persons, such as severe occlusion, similar appearance, illumination changes, and outline deformation. To cover the entire space of indoor surveillance scene, the image stitching based on the cameras' spatial relation is also utilized. The background subtraction of the stitched top-view image can then be performed to extract the foreground objects in the cluttered environment. The detection scheme including the graph-based segmentation, the head hemiellipsoid model, and the geodesic distance map are cascaded to detect humans. Moreover, the shape feature based on diffusion distance is designed to verify the human tracking hypotheses within particle filter. The experimental results demonstrate the real-time performance and robustness in comparison with several state-of-the-art detection and tracking algorithms.

For service robots to be able to enter a multi-human office environment, it is important to find a group of human users' social patterns and then to provide a proper service to them in time. Usually, human users' social patterns are represented in terms of nonverbal social signals. In this paper, a new integrated approach on recognizing multi-human social signals is proposed. Specifically, the nonverbal social signals are detected by a laser range finder and a RGB-D camera and are processed to find the multi-human (spatial) social patterns. Those recognized patterns are then applied to human-to-human, human-to-robot or multi-human-to-robot interactive formation. Experimental results shows that our robot successfully recognizes the aforementioned users' social patterns followed by appropriate services.

Understanding book‐locating behavior in libraries is important and leads to more effective services that support patrons throughout the book‐locating process. This study adopted a design‐based approach to incorporate robotic assistance in investigating the book‐locating behaviors of child patrons, and developed a service robot for child patrons in library settings. We describe the iterative cycles and process to develop a robot to assist with locating resources in libraries. Stakeholders, including child patrons and librarians, were consulted about their needs, preferences, and performance in locating library resources with robotic assistance. Their needs were analyzed and incorporated into the design of the library robot to provide comprehensive support. The results of the study suggest that the library robot was effective as a mobile and humanoid service agent for providing motivation and knowledgeable guidance to help child patrons in the initially complicated sequence of locating resources.

Human activity is among the critical information for a context-aware energy saving system since knowing what activities are undertaken is important for judging if energy is well spent. Most of the prior works on energy saving do not make the best of context-awareness especially in a multiuser environment to assist the energy saving system. In addition, they often ignore whether appliances are operating implicitly or explicitly related to the context. These factors may compromise the practicality and acceptability of most of the currently available energy saving systems, thus failing to meet real user needs. Therefore, we propose Energy-Responsive Aggregate Context (ERAC) to model multi-resident activities and their associated energy consumption. Based on the relationship, implicit or explicit, between a given appliance and its associated context, an energy saving system and its users can better determine whether the power consumed by the appliance is wasted. Our experimental results demonstrate the effectiveness of the proposed approach.

We develop a human awareness Decision Network model for robot performance on decision making. To accomplish more natural and intelligent human robot interaction (HRI), a robot should not only be able to infer the user's intention through recognizing the actions, but also to perform appropriate decisions and to learn from the user's feedback. In traditional approaches, user intention inference and feedback learning are dealt with separately. In this paper, we propose an integrated strategy of human-oriented perception, user modeling and user sensitivity in a social environment. The robot can analyze a user's feedback to adjust its decisions as the user expects through the strategy. The experimental results show the effectiveness of the proposed approach that enables autonomous adaptation of robot's decision to the user desires. Also, we demonstrate a satisfactory performance in terms of successful inference of human intentions, as well as adequacy of the decisions made by the robot for meeting user expectation.

In this paper, we propose a Bayesian conditional probability with latent-structure model for context-aware activities of daily living (ADL) recognition. The proposed ADL recognition system takes RGBD sensor (Microsoft Kinect) as the input device. In ADL recognition, the object interacted with human is a sort of important context as well as human action. To better understand the activity, we model the interacted object and the human action together. As far as we known, many related works failed to take into account the relation between the context information and human action features, instead, most of them only consider the human action features, causing ambiguity in classifying the activities with similar human actions. In this paper, the context information and human action features are taken into consideration, concurrently, so that the performance of recognition can be greatly improved from previous works as has been demonstrated in our experimental results.

In this paper, we propose an efficient part-based approach for action recognition. The main concept is to recognize human actions by less occluded parts without using a large set of part filters. Therefore, our approach is robust to occlusion and cost-effective. We extract spatiotemporal features from RGB-D videos, and assign a part-label to each feature. Then, for each part, a recognition score is computed for each action class by pyramid-structural bag of words (BoW-Pyramid) representation. The final result is determined by weighted sum of these scores and contextual information, which is based on the ratio of features between every pair of parts. Several contributions have been made in this work. First, the proposed part-based method is robust to occlusion and operates on-line. Second, our BoW-Pyramid representation can distinguish actions with reversed temporal orders. Third, recognition accuracy is increased by incorporating contextual information. The provided experimental results have verified effectiveness of our method and demonstrated high promise of surpassing performance of the state-of-the-art works.

In this paper, the authors proposed a local motion planner with robot controller of the mobile manipulator to grasp the object in dynamic environments without any prior knowledge of environments. Our local motion planner is based on the concept of potential field and composed of the attractive and repulsive vectors. Then, the local motion planner decides the potential vector according to the attractive and repulsive vectors in various situations. Also, an approach to deal with the drawback of local minima is contained. The attractive and repulsive vectors are generated by the distances between the target, obstacles and the manipulator. For robot control, we take the end-effector as the control point and apply the potential vector with joint-level control, and moreover evaluate the mobility and the kinematic constraints of the robot to modify the joint velocities. The experiment platform is a wheeled mobile robot with a 5-DOF manipulator using Softkinetic DS325 which is a close range RGB-D camera as our sensor. Through several experiments, the results show that our framework is fast enough and valid to grasp the object in dynamic environments.

To robustly detect people in a video sequence is hard due to various challenges. One of the most successful discriminative features for finding people goes to the Histograms of Oriented Gradients (HOG). Although the major contour information is encoded in the HOG feature well, the background clutter disturbs the gradient information. Thus, an extension of HOG, called histograms of oriented gradient of granules (HOGG), is proposed. Instead of collecting gradient information at each pixel, the histograms of gradients in small regions are computed. HOGG with different granularity can describe the contour while ignoring the noisy edges. Moreover, the clutter background problem can be solved by encoding extra region information. With the help of the integral image technique, the evaluation of HOGG can be efficient. The final HOG+HOGG classifier obtains 92% detection rate at 10-4false positive per window in the experiments.

In this paper, a vision based adaptive controller is proposed for the obstacle avoidance of a micro aerial vehicle (MAV) by using the optical flow information. In order to employ the optical flow for indicating the distance between the MAV and surrounding, the multi-thread processing algorithm is proposed to reliably obtain the optical flow information uniformly diffused in the whole image frame. The MAV system is treated as a switched system while manipulating the different modes during the obstacle avoidance task. Based on the desired flying direction of obstacle avoidance obtained by the optical flow estimation, we design an adaptive controller such that the desired trajectory can be tracked under different switching modes. The simulations present the tracking response of the adaptive controller of a switched system, and the experiments of the overall system validate the collision-avoidance performance of the MAV.

Identifying on-going activities for the provision of services that are capable of matching the needs of users poses a number of daunting challenges. Most existing approaches to activity recognition require training offline activity models before being applied to the identification of activities in real time. However, the dynamic nature of actual living environments can make previously learned activity models irrelevant. This study addressed the problem of learning and recognizing daily activities in a dynamic smart-home environment, using a novel approach referred to as hybrid user-assisted incremental model adaptation. This approach involves reconfiguring previously learned activity models within a dynamic environment, while pursuing maximum efficiency by using assistance from users as well as the system to annotate new training data. Experiments that are conducted in a fully equipped smart-home lab demonstrate the efficacy of the proposed approach.

The use of message-oriented middleware (MOM) in pervasive systems has increased noticeably because of its flexible and failure-tolerant nature. Meanwhile, decentralized service management protocols such as UPnP are believed to be more suitable for administrating applications in small-scale pervasive environments such as smart homes. However, administering MOM-based pervasive systems by UPnP often suffers from network flood problems due to the replications of too many unnecessary messages. This paper presents several traffic reduction schemes, namely, decomposing the multicast traffic, service-based node searching, heartbeat by decomposing the multicast traffic, and on-demand heartbeat, which reduce the replications of unnecessary messages in MOM-based UPnP networks. The analytical predictions agree well with the simulated and experimental results, which show that the message counts of presence and leave announcements, node searching, and heartbeat can be greatly reduced.

In this paper, we propose a novel approach based on particle swarm optimization (PSO) for solving the minimum energy broadcast (MEB) problem, which has been proven to be NP-complete. Wireless sensor networks (WSNs) have attracted large intention in recent years due to its powerful ability. One crucial issue in WSN is energy saving because of the limited battery resource. The MEB problem is one of the important scenarios in WSN, where a node needs to broadcast packets to all other nodes in the network. The objective is to minimize power consumption of all nodes in the network. Here we take advantage of fast and guided convergence characteristics of PSO to solve the MEB problem. For applying PSO to the MEB problem, we use the power degree to define the particle position. We go a step further to analyze one well-known local search mechanism: r-shrink and propose an improved version. The experimental results show that the proposed approach is able to compete and even outperform state-of-the-art works.

Hyper-heuristics solve problems by manipulating low-level domain-specific heuristics. The aim is to raise the level of generality of the algorithm to solve problems in different domains. In this paper we propose a hyper-heuristic based on Variable Neighborhood Search (VNS), which consists of two main steps: shaking and local search. Shaking disturbs solutions, and then local search seeks for the local optima. In our algorithm, we propose a mechanism to adjust the computational budget of local search periodically based on the search status. We also use a dynamically-sized population to store good solutions during the search process. Performance of the proposed algorithm is compared with four benchmark algorithms by four kinds of problems, Max-SAT, bin packing, flow shop scheduling, and personnel scheduling. Our algorithm finds the best solutions for around 90% of the tested instances.

In this paper, we propose an online scene reconstruction algorithm with monocular camera since there are many advantages on modeling and visualization of an environment with physical scene reconstruction instead of resorting to sparse 3D points. The goal of this algorithm is to simultaneously track the camera position and map the 3D environment, which is close to the spirit of visual SLAM. There're plenty of visual SLAM algorithms in the current literature which can provide a high accuracy performance, but many of them rely on stereo cameras. It's true that we'll face many more challenges to accomplish this task with monocular camera. However, the advantages of cheaper and easier deployable hardware setting have made monocular approach more attractive. Specifically, we apply a maximum a posteriori Bayesian approach with optimization technique to simultaneously track the camera and build a dense point cloud. We also propose a feature expansion method to expand the density of points, and then online reconstruct the scene with a delayed approach. Furthermore, we utilize the reconstructed model to accomplish visual localization task without extracting the features. Finally, a number of experiments have been conducted to validate our proposed approach, and promising performance can be observed.

This research presents a 3-D human arms tracking method with a monocular camera. In our previous work, multiple clues have been integrated by the multiple importance sampling particle filter to track the arms with arbitrary motion on the images. Due to the lack of depth information when using a monocular camera, an online sequential pose estimation based on the structure-from-motion is proposed here to provide the 3D arm posture hypotheses of multiple importance sampling particle filter. In our 3D sequential pose estimating algorithm, the structure of each arm parts is priory assumed to be a volumetric model with the points uniformly distributed on its surface. The 3D motion of arms can be recovered more reliably by visual tracking of the points on the known structure of each arm. In addition, while the objects in view appear larger when they are closer to the camera, the size effect from perspective projection is considered to disambiguate the arm posture. The posture hypotheses from the multiple importance sampling are finally verified by 2D visual features and the 3D posture information augmented with the size effect. The robustness and efficiency of the proposed algorithm have been validated in the experiments.

In this paper, the second order sliding mode control strategy is applied on a 6-dof parallel manipulator, so-called Stewart platform. The advantages such as robustness and design simplicity of sliding mode control are the main reason that it was often used on robotic systems. However, the chattering phenomenon of conventional sliding mode control may cause unstability due to the practical implementations of the switching control. As a result of discontinuous control signal, the chattering phenomenon can be eliminated via designing a higher order control strategy. Once the switching signal is designed in higher order manifold, the real input will be smoothed because of the integration. The paper represents a robustness control method without chattering for Stewart platform which is a multi-input nonlinear system. The control performance of the second-order approach was compared to conventional sliding mode control in the simulation results.

This paper presents a head and hands tracking method with a monocular camera for human machine interaction (HMI). The targets are tracked independently when they are far from each other, however, they are merged with dependent likelihood measurements in higher dimension while they are likely to interrupt each other. When tracking one target in the independent situation, other targets are masked to decrease the disturbances of skin color on the tracked one. Multiple clues, including the combination of the locally discriminative color weighted image and the back-projection image of the reference color model, the motion history image and the gradient orientation feature, are employed to verify the hypotheses originated from the particle filter. On the other hand, when the head and hands are closing or even overlapping, the multiple importance sampling (MIS) particle filter generates the tracking hypotheses of merged targets by the skin blob mask and the depth order estimation. These merged hypotheses are then evaluated by the visual cues of occluded face template, hand shape orientation and motion continuity. The experimental results present the real-time efficiency and the robustness in comparison with the OpenNI tracker which has been released recently for the Kinect sensor.

This paper introduces a context-aware assisted active robotic walker for Parkinson's disease (PD) patients. Most of PD patients suffer from not only loss of balance but also abnormal gaits. These symptoms tend to make PD patients fall down more easily and result in low quality of life. We use Hidden Markov Model (HMM) to analyze the gait of PD patients, and then use our walker to help patients adjust their gait to become normal while applying auditory cues when abnormal gaits are recognized. To prevent user from leaning forward before falling down, the walker locks the motors when sudden forward pushing by the user is detected. Moreover, the walker can record the statics of gait from the user, making the therapists monitor the rehabilitation process relatively easier. Finally, the road conditions in front of the walker will be automatically analyzed, making user able to adjust his/her walking pace dynamically. To our best knowledge, the hereby proposed active robotic walker should be the first system which can provide walking aid to PD patients. In our experiments, the feasibility and performance of this system are evaluated by PD patients at two actual senior care units.

Service robot has received enormous attention with rapid development of advanced technology in recent years, and it is endowed with the capabilities of performing human-robot interaction (HRI). We construct a sensor fusion based system to integrate the information from both sensors by using a data association approach - Covariance Intersection (CI). It will be used to increase the robustness and reliability of HRI in the real world environment. In this paper, we propose a Behavior System for analyzing human features and classifying the behavior by the crucial information from sensor fusion system. The system is used to infer the human behavioral intentions, and also allow the robot to perform more natural and intelligent interaction. We apply a spatial model based on proxemics rules to our robot, and design a behavioral intention inference strategy. Furthermore, the robot will make the corresponding reaction in accordance with the identified behavioral intention.

In this paper, we present a boosting approach by combining the pose estimation and the upper body tracking to on-line recognize human actions. Instead of using a predefined pose to initialize the human skeleton, we construct a key poses database with depth HOG features as searching indexes. When user enters the camera view, we automatically search the database to get the initial skeleton. Then we use the particle filter to track human upper body parts. At the same time, we feed the tracking joints into the hidden Markov models to on-line spot and recognize the performed action. In order to rectify tracking errors, we apply the action recognition results and reuse our key poses database to reinforce the tracking process. Our contributions of the proposed approach are three-fold. First, our method can recognize human poses and actions at the same time. Second, with the key poses database and action recognition results as the feedback, the tracking process becomes more efficient and accurate. Third, we propose a spotting method based on the gradient of HMM probabilities, which thus enables our method to achieve on-line spotting and recognition. Experimental results demonstrate the effectiveness of the proposed approach.

Energy overuse has caused many environmental and economic issues, so energy saving for household is challenging and important for a smart home. For home energy saving based on context-awareness, human activity is critical information since knowing what activities are undertaken is important for judging if energy consumed by appliances is well spent by users. Such contextual information is an important clue for providing an energy saving service. However, most of the prior works on home energy saving often ignore those appliances which are operating indirectly or implicitly related to the context. These factors may compromise the practicality and acceptability of most of the currently available energy saving systems, thus failing to meet real user needs. Therefore, we propose utilizing an Energy-Prone Context to model a context and its associated energy consumption. In addition, we also propose a systematic method to determine energy-saving services based on the Energy-Prone Contexts. Our experimental results demonstrate the effectiveness of the proposed approach.

Human activity is among the critical information for a context-aware smart home since knowing what activities are undertaken is important for providing appropriate services. Most of the prior works primarily focus on recognizing individual activity, thus requiring high cost to track people and performs not well when there are multiple users, which is common in a real home environment. Therefore, we propose hierarchical generalized context inference to infer multi-user contexts. By treating a multi-user context as a generalized context caused by an aggregated entity, our approach generalizes these multi-user contexts with different information granularity, and then dynamically infers and aggregates these generalized contexts. Based on the inference results of generalized contexts, a context-aware smart home can provide appropriate services as much as possible. Our experimental results demonstrate the effectiveness of the proposed approach.

RGB-D sensor has gained its popularity in the study of object recognition for its low cost as well as its capability to provide synchronized RGB and depth images. Thus, researchers have proposed new methods to extract features from RGB-D data. On the other hand, learning-based feature representation is a promising approach for 2D image classification. By exploiting sparsity in 2D image signals, we can learn image representation instead of using hand-crafted local descriptors like SIFT or HoG. This framework inspired us to learn features from RGB-D data. Our work focuses on two goals. First, we propose a novel Hierarchical Sparse Shape Descriptor (HSSD) to form learning-based representation for 3D shapes. To achieve this, we analyze several 3D feature extraction techniques and propose a unified view of them. Then, we learn hierarchical shape representation with sparse coding, max pooling and local grouping. Second, we investigate whether RGB and depth information should be fused at lower level or higher level. Experimental results show that, first, our HSSD algorithm can learn shape dictionary and provide shape cues in addition to the 2D cues. Using the proposed HSSD algorithm achieves 84% accuracy on a household RGB-D object dataset and outperforms a widely used VFH shape feature by 13%. Second, fusing RGB-D information at lower level does not improve recognition performance.

Changing lanes while having no information about the blind spot area can be dangerous. We propose a vision-based vehicle detection system for a lane changing assistance system to monitor the potential sedan vehicle in the blind-spot area. To serve our purpose, we select adequate features, which are directly obtained from vehicle images, to detect possible vehicles in the blind-spot area. This is challenging due to the significant change in the view angle of a vehicle along with its location throughout the blind-spot area. To cope with this problem, we propose a method to combine two kinds of part-based features that are related to the characteristics of the vehicle, and we build multiple models based on different viewpoints of a vehicle. The location information of each feature is incorporated to help construct the detector and estimate the reasonable position of the presence of the vehicle. The experiments show that our system is reliable in detecting various sedan vehicles in the blind-spot area.

The current smart home is a ubiquitous computing environment consisting of multiple autonomous spaces, and its advantage is that a service interacting with home users can be set with different configurations in space, hardware, software, and quality. As well as being smart technologically speaking, a smart home should also never forget to retain the “home nature” when it is serving its users. In this paper, we first analyze the relationship among services, spaces, and users, and then we propose a framework as well as a corresponding algorithm to model their interaction relationship. Later, we also realize the human-system interaction framework to implement a smart home system and develop “pervasive applications” to demonstrate how to utilize our framework to fulfill the human-centric interaction requirement of a smart home. Finally, our preliminary evaluations show that our proposed work can enhance the performance of the human-system interaction in a smart home environment.

Recognizing Activities of Daily Living (ADL) plays an important role in healthcare. However, it is often impractical and sometimes impossible for a person to collect those useful data manually, not to mention constant long-term data maintenance and analysis. To address the above-mentioned challenges, we propose an architecture, in which many health-care applications and services can easily build upon, for collective long-term ADL pattern analysis that leverages several prominent advantages inherent in cloud computing. The core of the proposed infrastructure includes a module to perform MapReduce-assisted Bayesian activity recognition based on all collected ADL data. Better yet, the resultant data analysis can be delivered as a service from a service station which serves as a readily accessible interface to 3rdparty service providers and end-users. For the evaluation of the proposed architecture, a simulation of persuasive health engagement is presented and discussed as one potential application.

Composing services in a pervasive environment usually involves user-in-the-loop adaption which is absent in most of the traditional enterprise service composition mechanisms. In such environment, the criteria for selecting and ranking services are usually specified by users, which tend to be vague and subjective. The criteria can be contradictory and the activated services can interfere with one another. This paper addresses these issues by defining a unifiable and negotiable expression language called the Preference Expression that is capable of specifying both enumerative/numeric as well as mandatory/negotiable user preferences. A set of unification rules for possible conflicting preferences is also derived. Experimental results show that the proposed approach is able to achieve high composition precision and maintains reasonable success rate at the same time.

In real-world applications, the optimization problems usually include some conflicting objectives and subject to many constraints. Much research has been done in the fields of multiobjective optimization and constrained optimization, but little focused on both topics simultaneously. In this study we present a hybrid constraint handling mechanism, which combines the ε-comparison method and penalty method. Unlike original s-comparison method, we set an individual ε-value to each constraint and control it by the amount of violation. The penalty method deals with the region where constraint violation exceeds the ε-value and guides the search toward the ε-feasible region. The proposed algorithm is based on a well-known multiobjective evolutionary algorithm, NSGA-II, and introduces the operators in differential evolution (DE). A modified DE strategy, DE/better-to-best_feasible/l, is applied. The better individual is selected by tournament selection, and the best individual is selected from an archive. Performance of the proposed algorithm is compared with NSGA-II and an improved version with a self-adaptive fitness function. The proposed algorithm shows competitive results on sixteen public constrained multiobjective optimization problem instances.

Energy saving has become an important issue in recent years due to the problems relating to energy shortage and global warming. However, most prior works in a domestic environment often ignore users' perception of the deployed technologies, let alone users' high-level contexts such as on-going activities and preferences. Without the high-level contexts, an ES system may fail to provide sufficient clues for a user to determine the most desirable ES strategy. In addition, the prior works are more technology-oriented and assume that our real-life environment stayed fixed once the system has been established. This causes their energy-saving (ES) strategies to be less human-centric and less adaptive to users' context changes. Moreover, good ES strategies are often established after the collection of long-term data and thorough analysis such that efficient ES models can be well trained, but such models are often not reusable or not sharable among different users or even communities. This indirectly leads to extra "en-ergy" waste in setting up a new energy-efficient environment. To remedy these drawbacks, in this paper we propose a cloud-enabled adaptive activity-aware energy-saving system which not only can facilitate more human-centric and context-aware ES strategies in a dynamic environment, but also can share the promising ES models that embed these desirable ES strategies among different users to facilitate community-based energy-saving.

In this paper, we develop a multi-human detection system with a team of robots basically in an indoor environment. To start with, we propose a hybrid approach to resolve the problem of human leg detection using Laser Range Finder (LRF) for each robot, that returns not only “true” or “false” type of answer but also a probability. Specifically, the set of measurement data obtained from the laser range finder mounted on a robot is further decomposed into several sectors using an appropriate segmentation technique. Then, we apply a probabilistic model to compare these sectors with leg patterns to check if any of them belongs to the set of human leg patterns or not. Next, we examine the promising leg sectors with a modified Inscribe Angle Variance (IAV) method in order to confirm if these sectors are from human leg's arc feature or not. Moreover, we also use motion detector to check if these objects move or not as an enhancement of the detection. For the entire multi-human detection system, each robot of the team delivers the detected human information to our central control computer through the Inter-Process Communication (IPC). With prior map information of the residing environment and supposing each robot in the team has a localization module, we can then map these results of human detection from every robot into their global coordinates after process of data association. But in order to reduce the computational complexity while doing the data association among these robots in a team, we introduce a set of appropriate rules. Finally, we apply a particle filter based tracking algorithm to keep accurate track of people being detected and to improve the robustness of the detection outcome. This work has been evaluated through several experiments with a number of mobile robots and humans in an indoor environment, and promising performance has been observed.

This paper proposes a navigation algorithm that considers the states of humans and robots in order to achieve harmonious coexistence between them. A robot navigation in the presence of humans and other robots is rarely considered in the field of robotics. When navigating through a space filled with humans and robots with different functions, a robot should not only pay attention to obstacle avoidance and goal seeking, it should also take into account whether it interferes with other people or robots. To deal with this problem, we propose several harmonious rules, which guarantee a safe and smooth navigation in a human-robot environment. Based on these rules, a practical navigation method-human-centered sensitive navigation (HCSN)-is proposed. HCSN considers the fact that both humans and robots have sensitive zones, depending on their security regions or on a human's psychological state. We model these zones as various sensitive fields with priorities, whereby robots tend to yield socially acceptable movements.

Reliability is one of the key challenges of pervasive systems. Numerous message-oriented architectures and service discovery protocols have been proposed to support service management in pervasive systems. Nevertheless, few researches have been done to improve the reliability of pervasive systems. This paper attempts to propose a reliable service management framework by formally defining a message-oriented service application model and protocols that facilitate autonomous composition, failure detection, and recovery of services. Proposed approaches are realized by constructing a developer's toolkit that enables rapid-prototyping of services. We evaluate the proposed approach by first proving the reliability property and then conducting experiments on recovery rate and performance. The results show that the recovery rate can be greatly improved by the proposed approach. Furthermore, the services developed by using the proposed approach are capable of integrating heterogeneous software/hardware, and can be deployed in dissimilar sites with little efforts.

This paper presents a tracking-based surveillance system that is capable of tracking multiple moving objects, with almost real-time response, through the effective cooperation of multiple pan-tilt cameras. To construct this surveillance system, the distributed camera agent, which tracks multiple moving objects independently, is first developed. The particle filter is extended with target depth estimate to track multiple targets that may overlap with one another. A strategy to select the suboptimal camera action is then proposed for a camera mounted on a pan-tilt platform that has been assigned to track multiple targets within its limited field of view simultaneously. This strategy is based on the mutual information and the Monte Carlo method to maintain coverage of the tracked targets. Finally, for a surveillance system with a small number of active cameras to effectively monitor a wide space, this system is aimed to maximize the number of targets to be tracked. We further propose a hierarchical camera selection and task assignment strategy, known as the online position strategy, to integrate all of the distributed camera agents. The overall performance of the multicamera surveillance system has been verified with computer simulations and extensive experiments.

Every home has its own unique considerations for location-aware applications. This makes a flexible architecture very crucial for efficiently integrating various tracking devices/models for adapting to real human needs. Here, we propose a reciprocal and extensible architecture to flexibly add/remove tracking sensors/models for tracking multiple targets in a smart home. Regarding tracking devices, we employ sensors from two different categories, those with seamless sensors and those with seamful ones. This allows us to take human-centric needs into consideration and to facilitate reciprocal and cooperative interaction among sensors from the two categories. Such reciprocal cooperation aims to increase the accuracy of location estimates and to compensate for the limitations of each sensor or a tracking algorithm, which allows us to track multiple targets simultaneously in a more reliable way. Moreover, the approach demonstrated in this paper can serve as a guideline to help users customize sensor arrangements to fulfill their requirements. Our experimental results, which comprise three tracking scenarios using a load sensory floor as the seamless sensor and RF identifications (RFIDs) as seamful sensors, demonstrate the effectiveness of the proposed architecture.

Classification rule mining, addressed a lot in machine learning and statistics communities, is an important task to extract knowledge from data. Most existing approaches do not particularly deal with data instances matched by more than one rule, which results in restricted performance. We present a two-phase multiobjective evolutionary algorithm which first aims at searching decent rules and then takes the rule interaction into account to produce the final rule sets. The algorithm incorporates the concept of Pareto dominance to deal with trade-off relations in both phases. Through computational experiments, the proposed algorithm shows competitive to the state-of-the-art. We also study the effect of a niching mechanism.

This paper addresses a multiobjective scheduling problem in the permutation flow shop. The objectives are to minimize makespan and total flow time. The proposed approach is based on the framework of memetic algorithm, which is known as a hybrid of genetic algorithm and local search. The local search procedure is an iterative process repeating neighbor generation, neighbor evaluation, and neighbor selection. We take a problem-specific heuristic for neighbor generation and propose several strategies for neighbor evaluation and neighbor selection. Archive injection (adding non-dominated solutions to the population) is another issue under investigation. We examine the effects of the proposed strategies through experiments using forty widely used problem instances with different scales. We also evaluate the proposed approach by comparing it with other twenty-six ones in terms of three performance metrics. Our approach outperforms all benchmarks and updates a large portion of the sets of best known non-dominated solutions for large-scale instances.

This paper proposes a constructive control approach, dynamic state feedback formation control, for achieving the realization of the Multi-Robot Formation System (MRFS) with respect to the problem of dilation of a formation shape and stabilization issue in nonholonomic system simultaneously. Combining with theoretical analysis, the proposed control approach has been successfully to deal with the following design issues: one is to elude the switch control of the nonholonomic MRFS for preventing the divergence of the MRFS; another one is to stabilize the MRFS which is allowed to change the interconnection structure dynamically.

This paper presents an upper body tracking algorithm with a single monocular camera. In order to be suitable for human robot interaction, the designed method should be free to work on the moving camera platform and also can achieve real-time performance. The dimension of human posture model is extremely high, and we hereby focus on the visual extraction of head and arms. A hierarchical structure model is proposed to solve the tracking problem by particle filter with partitioned sampling in the order of head, upper arm and the forearm. The hand position, straight edge of arm and temporal information are combined by the multiple importance sampling particle filter to efficiently estimate the irregular gesture of arms on image frames. The visual clues of the motion, appearance and shape to human face and arms are to verify the various hypotheses from the multiple importance sampling schemes. To validate the effectiveness of the proposed tracking approach, extensive experiments have been performed, of which the results appear to be quite promising.

Multi-resident activity recognition is among a key enabler in many context-aware applications in a smart home. However, most of prior researches ignore the potential interactions among residents in order to simplify problem complexity. On the other hand, multiple-resident activities are usually recognized using cameras or wearable sensors. However, due to human-centric concerns, it is more preferable to avoid using obtrusive sensors. In this paper, we propose dynamic Bayesian networks which extend coupled hidden Markov models (CHMMs) by adding some vertices to model both individual and cooperative activities. In order to improve performance of the model, we categorize sensor observations based on data association and some domain knowledge to model multiple-resident activity patterns. We then validate the performance using a multi-resident dataset from WSU (Washington State University), which only includes non-obtrusive sensors. The experimental result shows that our model performs better than other baseline classifiers.

This paper proposes a navigation algorithm that considers the states of humans and other robots in order to achieve harmonious coexistence between robots and humans. When navigating through humans and robots with different functions, a robot should not only pay attention to obstacle avoidance and goal seeking, it should also take care of whether it interferes with other people or robots. To deal with this problem, we propose several harmonious rules, which guarantee a safe and smooth navigation in multi-human and multi-robot (MHMR) environment. Based on those rules, a practical navigation method-human- centered sensitive navigation (HCSN)-is proposed. HCSN considers the fact that both humans and robots have sensitive zones depending on their security regions or on psychological feeling of people. We model these zones as various sensitive fields with priorities, whereby robots tend to yield socially acceptable movements.

The objective of this study is to design a robot system to assist the rehabilitation of patients so that they can afterwards do various daily activities. It is difficult to determine the desirable posture of a 9-DOFs exoskeleton manipulator in such a system and each joint control design as well. In this paper, we resolve the difficulties by mapping the kinematics of a human arm to that of the manipulator so that we can avoid going through the ill-postured configurations while searching for the desired solutions, and then reach the desired rehabilitation motion as precisely as possible. In addition, this study combines electromyography (EMG) and force sensor to detect the patient's motion on his/her volition, so that the rehab-robot can support the human's upper limb appropriately to fulfill the intended motion. For validation of our rehab-robot design, experiments are conducted and promising results are obtained.

It is desirable to know a resident's on-going activities before a robot or a smart system can provide attentive services to meet real human needs. This work addresses the problem of learning and recognizing human daily activities in a dynamic environment. Most currently available approaches learn offline activity models and recognize activities of interest on a real time basis. However, the activity models become outdated when human behaviors or device deployment have changed. It is a tedious and error-prone job to recollect data for retraining the activity models. In such a case, it is important to adapt the learnt activity models to the changes without much human supervision. In this work, we present a self-reconfigurable approach for activity recognition which reconfigures previously learnt activity models and infers multiple activities under a dynamic environment meanwhile pursuing minimal human efforts in relabeling training data by utilizing active-learning assistance.

Tracking the pose of human face has long been an important research topic which has many important applications, and it is particularly challenging with a monocular camera because the depth information is lost due to the perspective projection. This work adopts particle filter with partitioned sampling to decompose the state space of face pose tracking into two subspaces for increasing the sampling efficiency, thus achieving satisfactory performance with fewer particles. The parameters in the first subspace describe the target on image plane, and the parameter in the second subspace is used for the estimate of the face pose in yaw angle direction. For the evaluation of each hypothesis in the second subspace, a statistical learning algorithm called relevance vector machine (RVM) is used to map a face containing image to the pose of the face. The training of RVM is tailored to each detected frontal face, and it takes less than half second, which is suitable for a real-time application. The learning based regression model also presents the insensitive ability to expression variation and unmodeled degree of freedom. The experimental results verify that the combination of particle filter and RVM can efficiently reduce the processing time and add robustness to the performance of the system, thus making this algorithm applicable to human-machine interface with low-cost webcams.

This research presents an upper body tracking method with a monocular camera. The human model is defined in a high dimensional state space. We hereby propose a hierarchical structure model to solve the tracking problem by SIR (Sampling Importance Resampling) particle filter with partitioned sampling. The image spatial and temporal information is used to track the human body and estimate the human posture. When doing the human-machine interaction, a static monocular camera may not get plenty of information from 2D images, so we must move the camera platform to a better position for acquiring more enriched image information. The proposed upper body tracking technique will then adjust to estimating the human posture during the camera moving. To validate the effectiveness of the proposed tracking approach, extensive experiments have been performed, of which the result appear to be quite promising.

Reliable recognition of activities from cluttered sensory data is challenging and important for a smart home to enable various activity-aware applications. In addition, understanding a user's preferences and then providing corresponding services is substantial in a smart home environment. Traditionally, activity recognition and preference learning were dealt with separately. In this work, we aim to develop a hybrid system which is the first trial to model the relationship between an activity model and a preference model so that the resultant hybrid model enables a preference model to assist in recovering performance of activity recognition in a dynamic environment. More specifically, on-going activity which a user performs in this work is regarded as high level contexts to assist in building a user's preference model. Based on the learned preference model, the smart home system provides more appropriate services to a user so that the hybrid system can better interact with the user and, more importantly, gain his/her feedback. The feedback is used to detect if there is any change in human behavior or sensor deployment such that the system can adjust the preference model and the activity model in response to the change. Finally, the experimental results confirm the effectiveness of the proposed approach.

In this paper, we present a novel data structure representing the environment with occupancy grid cells while each grid map is associated with a set of line features extracted from laser scan points. Due to the fact that line segments are principal elements of artificial environments, they provide considerable geometric information about the environment which can be used for enhancing the accuracy of localization. Orthogonal characteristic of line features is the key issue to guarantee the consistency of the SLAM algorithm by allowing us to deal with lines that are parallel or perpendicular to each other. This behavior allows us to sample robot poses more correctly. As a result, the proposed algorithm can close bigger loops with the same number of particles. Experimental results are carried out using SICK LMS-100 laser scanner which has a maximum range of 20 m and Pioneer 3DX mobile robot mapping an indoor environment with the size of 40 m × 47 m.

This paper presents a robust location-aware activity recognition approach for establishing ambient intelligence applications in a smart home. With observations from a variety of multimodal and unobtrusive wireless sensors seamlessly integrated into ambient-intelligence compliant objects (AICOs), the approach infers a single resident's interleaved activities by utilizing a generalized and enhanced Bayesian Network fusion engine with inputs from a set of the most informative features. These features are collected by ranking their usefulness in estimating activities of interest. Additionally, each feature reckons its corresponding reliability to control its contribution in cases of possible device failure, therefore making the system more tolerant to inevitable device failure or interference commonly encountered in a wireless sensor network, and thus improving overall robustness. This work is part of an interdisciplinary Attentive Home pilot project with the goal of fulfilling real human needs by utilizing context-aware attentive services. We have also created a novel application called ldquoActivity Maprdquo to graphically display ambient-intelligence-related contextual information gathered from both humans and the environment in a more convenient and user-accessible way. All experiments were conducted in an instrumented living lab and their results demonstrate the effectiveness of the system.

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. In order to save computation time and be robust to noise effects, a region detection algorithm incorporating edge information is first proposed to identify the regions of interest, within which the spatial relationships are represented by a region adjacency graph. Next, we consider the motion of the foreground objects and, hence, utilize the temporal coherence property in the regions detected. Thus, the foreground segmentation problem is formulated as follows. Given two consecutive image frames and the segmentation result priorly obtained, we simultaneously estimate the motion vector field and the foreground segmentation mask in a mutually supporting manner by maximizing the conditional joint probability density function of these two elements. To represent the conditional joint probability density function in a compact form, a Bayesian network is adopted, which is derived to model the interdependency of these two elements. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.

Current trends suggest that the smart home systems should be adaptive and robust. Many service-oriented architectures and discovery protocols for the smart home environments have been proposed to support on-line re-composition of home services in reaction to changing environments. Nevertheless, few researches have been done to improve the robustness of smart home systems. This paper attempts to propose a service management protocol that supports fast self-healing and self-organizing of home services by enhancing SSDP/UPnP with two protocol extensions and a lightweight heartbeat mechanism. Experiment results show that for average scale of services (less than 10 service components), the proposed protocol is capable of activating a service and recovering a failed service within 1200 ms and 2000 ms, respectively.

Weaving computing technologies into a living environment without interfering with natural interactions is nontrivial. In this paper, we have proposed utilizing ambient-intelligence compliant object (AICO) to facilitate context-aware service provision in a smart home; furthermore, a hybrid weaving strategy, called Hide and Not Easy to Seek, is proposed for designing the weaving layer of each AICO and for popularizing ubiquitous computing. Seven weaving guidelines which combine seamless and seamful designs are learned from our actual instrumentation of a living lab and continuous cooperation with specialists from various domains. By following the proposed guidelines, our expectation is that people will be able to use the weaved technologies more naturally to accomplish everyday tasks.

The multiobjective flowshop problem with makespan and total flow time as objectives is addressed. A genetic local search algorithm is proposed with the ability to allocate the computational resources through the dynamic population size and local search intensity. The proposed method is compared with existing algorithms for flowshop scheduling with a public benchmark problem set. The experimental results show that the proposed method is capable of discovering solutions with better quality and diversity. The proposed method yields the best known nondominated solutions for the commonly studied permutation flowshop benchmarks, and the set of best known solutions is useful for the evaluation of performance of future studies.

In this paper we propose a method to recognize the arm motions performing within a short time, which are called “gesture strokes”, for instant interaction. We combine two modalities, computer vision and linear accelerometer, to obtain robust recognition results. The arm motion is first detected by the accelerometer, and a time window is created for this motion. Both modalities individually estimate the probability mass distribution of the gesture stroke classes from the information gathered inside this window. The estimation results of these two modalities are then combined by the dynamic model combination which is a log-linear combination with different weights for all probability masses. The set of weight exponents are learned by the Nelder-Mead method that minimizes the empirical error rate of classifying all training samples. The experiments show that these two modalities compensate for each other and the combination framework improves the recognition correct rate.

We introduce an Augmented Histograms of Oriented Gradients (AHOG) feature for human detection from a nonstatic camera. We increase the discriminating power of original Histograms of Oriented Gradients (HOG) feature by adding human shape properties, such as contour distances, symmetry, and gradient density. Based on the biological structure of human shape, we impose the symmetry property on HOG features by computing the similarity between itself and its’ symmetric pair to weight HOG features. After that, the capability of describing human features is much better than the original one, especially when the humans are moving across. We also augment the gradient density into features to mitigate the influences caused by repetitive backgrounds. In the experiments, our method demonstrates most reliable performance at any view of targets.

A general control framework for a formation system composed of a team of nonholonomic wheeled mobile robots (WMRs) considering the formation dynamics under static connection structure but subject to dynamic connection stability is proposed in this paper. Hence, a rigorous formation theory is obtained essentially based on the differential structure of the formation system. With this result, a case design using a Lyapunov based control complying with the proposed formation theory is provided to demonstrate the capability and feasibility of the proposed framework. Finally, the wander-mode simulation with three WMRs is conducted while the formation configuration may be changing on-line. It is believed that the present results can be further extended to many applications involving the nonlinear multi-agent system.

We present several new methods to promote the ship simulator applied to both physically and graphically real-time dynamic rendered ocean. The simulation system is divided into three subsystems, the ships dynamics system, the hydrodynamics system, and two ways coupling between them. We apply hydrodynamic and mechanics to simulate the ocean surface and ship model respectively. In this paper, wave decay is taken into consideration waves propagating velocity due to sea floor terrain, wave generated by object floating on the ocean and reflection of wave due to conflict with the ship. On the other hand, the ship would be affected not only by buoyancy but also viscosity, current, wave and damping. Finally we integrate this algorithm to a 6 DOF platform and construct a virtual ships driving simulator.

In the control issue of the 6-DOF moving platform, positions and some rotation angles of the platform can not be measured with accelerometers and tilt sensors. Without full state feedback, the inverse kinematics must be applied into the control scheme to convert the desired platform position and orientation to the leg lengths. The actual control goal becomes to control individual leg lengths. To achieve the output feedback control, the forward kinematics solution is solved by using a nonlinear observer designed to estimate the system states including 3-axes translations and rotations. The conventional forward kinematics solutions using Newton-Raphson and other elimination-based methods have too much computational burden or are too complex to take out in the real time applications. In this paper, the nonlinear observer and a sliding mode controller are used to control the six states of the platform directly. The stability of whole system is verified to ensure the control errors would converge.

The identical parallel batch machine scheduling problem is addressed in this paper. Incompatible job families and dynamic job arrivals are considered, and the objective is to minimize total weighted tardiness. A memetic algorithm is proposed to assign the batches to machines and to determine their processing sequences. The proposed approach is shown to outperform an existing approach in terms of solution quality and computational efficiency through comprehensive experiments.

This paper presents a nearly real-time surveillance system to track multiple moving objects by controlling multiple pan-tilt camera platforms. In order to describe the relationship between the targets and camera in this surveillance system, the input/output hidden Markov model (HMM) is applied here in the well-defined spherical camera coordinate. For the less number of cameras to effectively monitor a wide surveillance space, the overall cameras have to closely cooperate. We propose a hierarchical camera selection and task distribution strategy, and the action decision of each camera platform is according to its assigned role. Furthermore, an optimal camera action selection strategy is presented for one camera which is assigned to track multi-target within its limited field of view. The maximization of mutual information for the action design is evaluated by the Monte Carlo method. The overall performance has been validated in the experiments of real-time surveillance.

Service Discovery is an interesting challenge in highly dynamic Pervasive Environments such as smart living spaces. In addition to adaptation to varied environments, autonomous failure recovery and activation of services are two fundamental issues of a Service Discovery protocol in order to achieve high service availability. In [1], we have introduced an enhanced version of UPnP's Simple Service Discovery Protocol (SSDP), which aims to fulfill these two fundamental issues. However, our experiences in the previous work show that the failure recovery performance is not desirable due to the multicast mechanism used by SSDP. In this paper, we propose a supporting data structure, Mapped Eviction SND Tree, composed of a set of specialized mapped-trees to speed up the failure recovery process. The system with this structure is aware of existing service nodes and need not to perform the discovery procedure again when failure is detected. Hence, the time for restoring the failed service is minimized. Experiment results show that the proposed approach helps reducing the failure recovery time up to 90 percent in average. Issues concerning maintenance and collaboration between the sets of internal data structures are also discussed in this paper.

In this paper, we solve two problems in a 300-mm wafer fabrication facility (fab). Firstly, for the lot scheduling problem, we propose a multi-objective genetic programming based rule generator (MOGPRG) to evolve useful dispatching rules, which can provide near-optimal lot schedules concerning multiple objectives. Secondly, the overhead hoist transports (OHT) routing problem is considered. As the modern automated material handling system (AMHS) is capable of doing tool-to-tool direct delivery, the congestion of OHTs may happen more often than the past. To deal with the traffic congestion in AMHS, a dynamic routing method is proposed to find the near-shortest and less-congested path for the OHT to travel along. It can reduce the traffic congestion and achieve fast lot delivery by adapting to the dynamic traffic environment. The proposed MOGPRG is integrated with the dynamic routing method to improve two fab performance metrics: mean cycle time and tardy rate. Experimental results show the effectiveness of the proposed MOGPRG and dynamic routing method.

In this paper, we propose a method which uses vision-based gesture recognition to control character animation. Each animation sequence has a corresponding gesture to be recognized, and we focus on upper-body motions and use one camera to capture images. Human gestures are modeled by a learned graph model whose nodes are key frames of these gestures. The animation sequences are pre-processed to generate a motion graph, and the mapping between the gesture model and the animation motion graph is created. At run time, the recognized node sequence in the gesture model will guide the animation to traverse the animation motion graph. Our method avoids complex process of completely reconstructing the human motion and still holds the advantages such as being intuitive, quickly responsive and versatile. The proposed method can be applied to control avatar actions in a large virtual environment. Our experiments show that the segmented gesture recognition can robustly control the animation with quick response even there are ambiguities in the initial poses of some gestures.

Home automation systems should evolve to the next phase in which they are aware of contexts, because providing services based on contexts will upgrade the service quality, thus making people more comfortable and home living more convenient. In particular, location is a piece of important and useful context information for seeking appropriate services, as well as providing them to people living in a home. So far, there have been several studies focused on tracking inhabitants in smart home environments. However, these approaches are often intrusive or require inhabitants to wear some sort of devices, which may make the inhabitants uncomfortable or even inconvenient. This problem could devastate the ultimate goal, which is to provide convenient services, and hence cause such approaches somewhat controversial. In this study, we utilize a number of load sensors together to construct a sensory floor on which exerted pressure can be detected and cover its surface with wooden flooring as in a normal home environment. Although the wooden flooring provides a flat surface for inhabitants to walk on, it also causes clutter in the sensor readings, which lead to difficulty in clearly identifying the location(s) of pressure source(s). Thus, we apply probabilistic data association technique and LeZi-Update approach to analyze the cluttered pressure readings collected by the load sensors so as to determine the positions of the inhabitants, as well as to track their movements. With our nonintrusive approach, there is no need for inhabitants to wear any devices, and this also solves the cross-walking problem in the home environment. Note to Practitioners-This system aims for detecting the location of multiple inhabitants in the home environment. We adopt the sensory floor as our tracking sensor, which consists of many blocks, each with a load cell to collect the human body weight. These blocks are covered with conventional wooden flooring to provide a flat surface on which inhabitants can walk freely. After collecting data, we apply mathematical techniques to analyze them, thus determining inhabitants' locations and tracking their movements. The limitation is that the system cannot differentiate different persons if they have close weight readings, which is the natural limitation of the load cell.

The virtual conduction system (VCS) allows a user to conduct a photo-realistic pseudo orchestra. The VCS includes four modules: (1) gesture recognition module, (2) audio rendering module, (3) video rendering module, and (4) multi-resolution display module. With gesture recognition and tempo adjustment, the user not only can change the playback rate of an audio and video recording, but also can control the volume from different portion of an orchestra in real time. With the video rendering module and multi-resolution display module, the VCS provides a new visual experience with an interactive multi-resolution wall-size display.

Negotiations have been a widespread research topic in politics, economics, and management for decades. Recently, with the rapid growth of on-line bargains, automatic negotiations have become more and more important. Although many automatic negotiation strategies have been presented, most of them are focused on simple negotiations composed of independent multiple issues. These strategies can not be applied to realistic complicated negotiations made up of dependent multiple issues. Therefore, we propose a mechanism named joint genetic algorithm (JGA) to deal with e-negotiations of dependent multiple issues. In JGA, a joint search strategy is applied to find the satisfactory contract accepted by both parties, by means of the genetic algorithm to predict and learn opponent's preference. Experimental results show that JGA can facilitate to make a deal efficiently under different circumstances of conflict scenarios.

This paper presents a real-time surveillance system to track multiple moving objects by controlling a pan-tilt camera platform. In order to describe the relationship between the targets and camera in this surveillance system, the input/output hidden Markov model (HMM) is applied here in the well-defined spherical camera coordinate. Since the targets are hard to be distinguished with one single camera when they are close to each other, we extend the particle filter for multitarget tracking with depth level estimate to track interacting targets. The targets overlapping each other still can be tracked in the images captured by single camera. Furthermore, an optimal camera action selection strategy is proposed to track multitarget within its limited field of view. The maximization of mutual information for the action design is formalized and implemented by the Monte Carlo method. The overall performance has been validated in the experiments of real-time tracking.

This paper presents a new approach to automatic segmentation of foreground objects from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the classification problem as a graph labeling over a region adjacency graph based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, a temporal coherence is also maintained by modeling it as the prior energy. On the other hand, color distributions of two neighboring regions are taken into consideration to impose spatial coherence. Then, the a priori energy of MRFs takes both spatial and temporal coherence into account to maintain the continuity of our segmentation. Finally, a labeling is obtained by maximizing the a posteriori energy of the MRFs. Under such formulation, we integrate two different kinds of techniques in an elegant way to make the foreground detection more accurate. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.

The architecture of a conventional smart home is usually server-centric and thus causes many problems. Mobile devices and dynamic services affect a dynamically changing environment, which can result in very difficult interaction. In addition, how to provide services efficiently and appropriately is always an important issue for a smart home. To solve the problems caused by traditional architectures, to deal with the dynamic environment, and to provide appropriate services, we propose a service-oriented architecture (SOA) for smart-home environments, based on Open Services Gateway Initiative (OSGi) and mobile-agent (MA) technology. This architecture is a peer-to-peer (P2P) model based on multiple OSGi platforms, in which service-oriented mechanisms are used for system components to interact with one another, and MA technology is applied to augment the interaction mechanisms

A job shop scheduling problem with total tardiness and the maximum tardiness as objectives is addressed. We solve it by a rule-coded genetic algorithm. Characteristics of three existing fitness assignment mechanisms are identified and then combined through the proposed cyclic fitness assignment mechanism. Experiments are conducted on a public benchmark problem set, and the results show that the proposed algorithm outperforms the existing ones.

In this paper, a novel design and implementation of a hybrid system of the multi-agent is proposed. First of all, a hybrid system is established to characterize different behaviours of an agent. From the well designed feedback linearization control law, the autonomous agent which is composed by wheeled mobile manipulator (WMM) switches well-depicted behaviours such as obstacle avoid, target approach, cruise, grasp etc. from the feedback signals. Next, a two-level control architecture is devised for the WMM system, where the first level mainly handles sensor feedback subject to sensor fusion, whereas the second level is responsible for providing a well performed supervisory control that can manipulate all available surrounding resources to successfully accomplish the aforementioned mission. An actual WMM is developed, named The Treasure Hunter (TTH), which is able to hunt treasures in an unknown environment whereby the effectiveness of the proposed approach is satisfactorily demonstrated

This paper addresses the job shop scheduling problem with the due date-based objectives including the tardy rate, mean tardiness, and the maximum tardiness. The focused approach is dispatching rules. Sixteen dispatching rules are selected from the literature and used as the benchmarks. Their features and design concepts are also discussed. Then a dispatching rule is proposed with the goal as achieving good and balanced performance when more than one objective is concerned at the same time. The experimental results verified its superiority, especially on the tardy rate and mean tardiness

In this paper, the problem of human pose estimation is formulated as a multi-objective optimization problem so as to fuse multiple cues more properly, which is in contrast with the hypothesis that the cues are mutually independent so that their consolidation can be solely through the product of their individual likelihood distributions. An evolutionary algorithm for optimizing the defined objectives optimization is applied to evolve a set of non-dominated alternative solutions, known as the Pareto-optimal set. For convergence improvement of the evolutionary algorithm, the DD-MCMC method is used to generate a set of good initial solutions. Evaluating solutions by using relative dominant relation rather than quantitative absolute difference in value makes the solution exploration not dominated by the poor cue and result in more effective solutions for further decision making. Experimental results to the images obtained from different scene are provided to demonstrate the effectiveness and efficiency of our proposed framework

This paper proposes a robust visual tracking framework to track circle-like objects in cluttered environment. Instead of using the resulting positions after circle detection and tracking in the image domain, we directly perform the visual tracking task in the parameter space which describes the measurement features. The visual tracking technique is combined with targets generation closely. We utilize the probabilistic data association filter (PDAF) to filter the detected measurements with noise and disturbance. The likelihood ratio through the Hough transform is employed to modify the evaluation of the association probability and make the estimate more reliable. Furthermore, the joint probabilistic data association filter (JPDAF) is used to deal with the multiple circle-like objects tracking. The likelihood variation of each target is introduced with JPDAF as a basis of the predictions for different targets. The overall performance has been verified in several challenging experiments

This paper aims to track multiple people in a multicamera system for cluttered environment which can be divided into two important parts: one is tracking multiple people in a single camera environment, and the other is tracking multiple people in a multi-camera environment. In a single camera environment, we apply the motion detector and the ellipse algorithm to detect a new person intruding the surveillance area. Then, we utilize the template matching and the ellipse matching to track the person. To prevent tracking failure when people cross over each other, we include the hereby proposed joint visual probabilistic data association filter (JVPDAF) to track multiple people successfully. In a multi-camera environment, the major problem is to determine whether the new person intruding into some surveillance area of a camera is an identified person by some other camera or not. To resolve the aforementioned problem, we propose an approach called consistence labeling. After such labeling process, we track this person by the JVPDA algorithm. Finally, effectiveness of this tracking algorithm is validated via extensive experiments

This paper addresses the problem of distributed motion control of highly structured formations of nonholonomic multi-agent robotic systems. For concentrating on the multi-agent formation problem, a local stable feedback control laws based on relative distance and angular and bearing measurements are used. There are topological obstructions to globally stable system operation for such laws, but by appropriately switching among them, it is possible to stably control a rich class of formation motions. In this paper, the modeling and stability analysis is obtained to the proposed nonholonomic multi-agent formation systems. Beyond the modeling, a leader's control configuration is obtained from the cost function of the MFCS with local control of followers. Hence, we start with the fundamental definitions of a noholonomic WMR for looking for the system properties. After extending to the problem of multi-agent formation, an innovative analysis of the platform is provided for the further study on multi-agent cooperation/coordination problem.

To solve the forward kinematics is one of the major problems for the 6-dof parallel manipulators. The complexity of formulation and computational burden make the conventional methods hard to be implemented in practice. In this paper, the tetrahedron configurations are applied to solve the forward kinematics of the 6-6 Stewart platform. This method turns the forward kinematics to tetrahedron geometry with the advantages that it avoids to solve numerical iteration, and the solution is unique. This approach can be used in real time application for its simplicity. An experiment designed to compare the actual orientation of the moving platform and the forward kinematics solution is presented in this paper. It is found through the experiment that the proposed method can achieve real time application.

Smart home which understands user's preference and provides right services at right time is the current trend. In this paper, we aim at developing a system which can achieve this objective by using the Bayesian network to model user's preference. Instead of assuming the structure of Bayesian network is invariant, our system interacts with user appropriately to obtain some useful information and we use the semi-supervised learning with these information to both learn and adjust the Bayesian network for modeling the user's preference in a more accurate manner. We can use preference model to provide adequate service in home environment. A simulation and a real home environment are constructed based on the proposed method, and the experiments also show the usefulness.

This paper addresses the lot scheduling problem in the semiconductor wafer fabrication facilities. We provide a simulation study to examine the performance of sixteen existing dispatching rules on the tardy rate, mean tardiness, and the maximum tardiness. A public and representative test bed, the MIMAC (measurement and improvement of manufacturing capacities) test bed is used. The best rules with respect to each objective are identified through the experiments, and some findings are provided to be guidelines for designing new dispatching rules.

In this paper, we design adaptive and robust controller for the nonlinear MIMO heating, ventilating, and air conditioning (HVAC) system, respectively. The designed adaptive controller maintains good tracking performance for temperature and humidity ratio regardless of the slowly time-varying load changes, which are regarded as parametric uncertainties in thermal space. The robust controller considering non-parametric uncertainties is designed so as to remedy the drawback of feedback linearization technique. The obtained control system shows robustness to the non-constant thermal loads and mismatched uncertainties, respectively. Some simulation results are provided to illustrate the satisfactory control performance of the adaptive controller.

In this paper, we propose a new approach "HVB senseless maneuver optimal washout filter" which is based on human vestibular system, senseless maneuver and motion platform limitation for designing washout filter such that a cost function constraining the pilot sensation error (between simulator and vehicle) is minimized. This approach can curtail over strong feelings of pilot reception and increase efficiency of platform workspace for task running. Finally, the experimental results confirm the effectiveness of our algorithm hereby proposed. Moreover, the results show that a better performance can be attained

In this paper, we present an approach to infer the arm pose in real-time for human-computer interaction. The approach is divided into example-based matching and local optimization. We build a database using synthesized disparity maps as examples and use them to compare with the input disparity map. The pose of the best match is used as the initial condition for local optimization. We implement the local optimization by applying physical forces to attract a dynamic arm model. The arm model contains prismatic joints that can alleviate the problem from imprecision model size. The example-based matching can cover a large range of motion and provide a good initial condition which prevents arm model from being trapped into local maxima, and the local optimization provides a better estimation of pose that compensates for the insufficient pose resolution of examples.

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. A Bayesian network is presented to model the interactions among three variables, such as foreground segmentation mask, motion segmentation field, and motion vector held. Given two consecutive images, the conditional joint probability density of the three variables is maximized iteratively to simultaneously achieve foreground segmentation and motion segmentation in a mutually beneficial manner. The solution to the optimization problems are obtained by using iterative conditional mode (ICM) and graph cut algorithm. Incorporating motion information with background subtraction technique makes the segmentation perform in a more semantic level and obtain more accurate results. Experimental results for two video sequences are provided to demonstrate the effectiveness of the proposed approach.

In visual tracking, Multi-target tracking (MTT) systems encounter the problem that unavoidably moving targets may overlap each other and the measurement process of each target becomes dependent, so we construct a tracking system with considering joint image likelihood to track recognize targets, even homogeneous ones. Also, in order to enhance the tracking performance, we extend the sequential importance sampling (SIS) particle filter with the separated importance functions for tracking each target and detection at the same time. The overall performance is validated in the experiments and shows the robustness with near real-time tracking.

We propose a physically-based modeling approach to generate effect of aerodynamics. We take the impulse-based method that allows us to treat, articulation, contact, collision in a unified manner. We use the concept of dynamic pressure which is the pressure related to the relative wind velocity, and is frequently adopted in flight simulation and wing design. Moreover, we calculate the aerodynamics according to their shapes, simply in a unified and physically-based approach without tuning too many parameters. A freezing algorithm is also proposed for speed up the simulation. This approach is designed for the use in interactive systems such as VR motion simulator, computer games, and general purpose interactive environments. We demonstrate our approach in an interactive simulation environment with a helicopter levitating with its propeller to show the effect of lift and drag. We also demonstrate the Magnus effect with a ball demo.

Several studies have focused on tracking inhabitants in a smart environment. However, these approaches have often required inhabitants to wear devices, or they have needed lengthy pre-calibration before tracking could be engaged. Such approaches are often intrusive, thus making inhabitants uncomfortable when the ultimate purpose is to provide convenient services. Thus, the approaches are somewhat controversial. In this study, we constructed an environment consisting of load sensors, with wooden flooring covering the surfaces as in a normal home environment. The wooden flooring provided a flat surface for inhabitants to walk on but caused clutter in the load sensor. Thus, we applied Probabilistic Data Association and LeZi-Update to analyze the cluttered pressure phenomenon collected by the load sensors and to determine the inhabitants' locations and track their movements. With our non-intrusive approach, there is no need for inhabitants to wear any devices, and there are no complicated pre-settings, unlike other approaches.

Here we propose an extensible multiple target tracking architecture which utilizes RFID (Radio Frequency Identification) technology and cooperates with currently existing tracking sensors. Our proposed approach demands every individual sensor to predefine a confidence factor in advance, which allows us to track multiple targets simultaneously in a more accurate and power-efficient way. We have previously shown that a load sensory floor can non-intrusively keep track of residents' locations based on both their regular movement patterns and distinguishable features such as their weights; however, it generally becomes infeasible to simultaneously differentiate two or more residents with irregular movement patterns or similar weight measurements. With the assistance of active RFID tags, our proposed system can cope with these limitations and improve upon the tracking results of previous systems by fusing RFID-based correction signals with those from currently available sensors. Each sensor has to reckon its corresponding confidence factor which is later used in conjunction with the RFID signals to dynamically discriminate erroneous outputs from correct ones, meanwhile mitigating the power consumption issues inherent in RFID systems. Our experimental results, which comprise three scenarios having distinct irregular movement patterns, demonstrate the effectiveness of the architecture.

The rapid advancement in computer technology enables home automation system to provide a variety of convenient and novel services to people. Generally speaking, locating residents' positions in home environment is a key issue for service provision. In this paper, we propose a human localization system for our Smart Home. The human localization system uses the Condensation algorithm to locate residents' positions via multi-camera and sensory floor approaches. The Condensation algorithm is a kind of Bayesian filters and has ability to handle multi-target tracking. By integrating information from multiple sensors, we can overcome the static occlusion in the environment and make the localization system more robust. We also describe architecture of the proposed and discuss its performance.

In this paper, we propose a modeling tool named Queueing Colored Petri nets (QCPN) for performance evaluation and scheduling for wafer fabrication. The main idea of this tool is to combine colored timed Petri nets with the queueing systems, and it aims to make simulation over the model more efficient. Due to the wide acceptance of priority rules in the wafer manufacturing industry, we also proposed a mechanism to realize priority rules in the QCPN models. Since it is known that no single rule can dominate in any circumstance, we proposed a genetic algorithm (GA) to search for the optimal combination of a number of priority rules based on the status and performance measures of the fab. Our approach can be considered as taking the advantage of the lot execution sequence generated by priority rules to guide the search. This approach can reduce the solution space and help us find the good solution more quickly. In addition, the QCPN-based GA scheduler can greatly reduce the computation time so that this GA scheduler can meet the need for a rapidly changing environment. Note to Practitioners-Performance evaluation and scheduling are two functions required by fab managers and engineers. This paper proposed a tool which consists of a simulator and a scheduler. By connecting to the Manufacturing Execution System (MES) and providing the scheduling rules, we can see how the fab runs virtually with the simulator. General information such as throughput and average cycle time and specific information like lot activity history can be obtained. This can be used for decision making, delivery prediction, bottleneck seeking, and testing of newly developed heurisitcs. The implementation cost is only on data communication between the MES and the simulator and the incorporation of rule modules. The scheduler, which takes the simulator as the performance evaluation module, can generate the suitable scheduling rule based on the current fab status, preference of performance criteria, and rule candidates. There is almost no extra cost after the simulator is connected to the MES. The scheduler can be easily made faster by common parallelization techniques.

Visual tracking in cluttered environments is attractive and challenging. This paper establishes a probabilistic framework, called the visual probabilistic data-association filter (VPDAF), to deal with this problem. The algorithm is based on the probabilistic data-association method for estimating a true target from a cluster of measurements. There are two other key concepts which are involved in VPDAF. First, the sensor data are visual, similar to the target in the image space, which is a crucial property that should not be ignored in target estimation. Second, the traditional probabilistic data-association filter for the underlying application is vulnerable to stationary disturbances in image space, mainly due to some annoying background scenes which are rather similar to the target. Intuitively, such persistent noises should be separated out and cleared away from the continuous measurement data for seeking successful target detection. The proposed VPDAF framework, which incorporates template matching, can achieve the goal of reliable realtime visual tracking. To demonstrate the superiority of the system performance, extensive yet challenging experiments have been conducted.

In this paper, we propose a supervisory controller for the discrete model of a heating, ventilating and air conditioning (HVAC) system and then design the nonlinear controller for the heating and cooling subsystems within. The HVAC system includes the discrete event dynamics and nonlinear continuous dynamics, both are very complex and challenging. Here we discuss the whole system and systematically design the controller and the interface between continuous and discrete controller.

To solve job shop scheduling problems, the priority rule is one of the most popular approach. It has the appeal because of simplicity, efficiency and effectiveness. However, the paradigm conventionally used to apply priority rules has a certain flaw. In this paper, we first point out this flaw and then propose a paradigm to remove it. A rule is also developed to exploit the potential of the new paradigm. The performance of the proposed approach is verified by several simulation experiments. The experimental results are quite satisfactory.

This paper presents a new approach to automatic segmentation of the foreground objects from the sequence of images by integrating techniques of background subtraction and motion-based segmentation. At first, a background model is built to represent information of both color and motion of the background scene. Based on temporal and spatial information, an initial partition of each image is obtained. Next, we formulate the classification problem as a graph labeling over a region adjacency graph (RAG) based on Markov random fields (MRFs) statistical framework. The Bhattacharyya distance for estimating the similarity between color and motion distributions of the background model and the currently obtained regions are used to model the likelihood energies. The object tracking strategy for finding the correspondence between region at different time instant is used to maintain the temporal coherence of the segmentation. For spatial coherence, the length of the common boundaries of two regions is taken into consideration for classification. Both spatial and temporal coherence are incorporated into the prior energy to maintain the continuity of the segmentation. Finally, a labeling is obtained by maximizing a posterior probability of the MRFs. Under such formulation, we integrate two different kinds of framework in an elegant way to make the foreground detection become more accurate. Experimental results for two image sequences including the hall monitoring and our e-home demo site are provided to demonstrate the effectiveness of the proposed approach.

This paper presents the development of a visual tracking and visual servo system for a reconnoitering air vehicle. The air vehicle is mounted with a camera platform underneath. It is proposed to perform an aerial reconnoitering mission for continuously viewing an interesting target. The vision system is composed of a hybrid visual tracking system and an image based fuzzy logic visual servo system. The designed hybrid visual tracking system can track arbitrary-shaped moving object. Also, the followed visual servo system is to control the air vehicle hovering the target autonomously. The overall system is simulated and experimented in a virtual reality environment to validate the research work.

The objective of this research is to develop a vision-based driver assistance system to enhance the driver's safety in the nighttime. The proposed system performs both lane detection and vehicle recognition. In lane detection, three features including lane markers, brightness, slenderness and proximity are applied to detect the positions of lane markers in the image. On the other hand, vehicle recognition is achieved by using an evident feature which is extracted through three four steps: taillight standing-out process, adaptive thresholding, centroid detection, and taillight pairing algorithm. Besides, an automatic method is also provided to calculate the tilt and the pan of the camera by using the position of vanishing point which is detected in the image by applying Canny edge detection, Hough transform, major straight line extraction and vanishing point estimation. Experimental results for thousands of images are provided to demonstrate the effectiveness of the proposed approach in the nighttime. The lane detection rate is nearly 99%, and the vehicle recognition rate is about 91%. Furthermore, our system can process the image in almost real time.

This paper addresses the FMS scheduling problem. The objective concerned here is maximizing the meet-due-date rate. The authors propose two rules for job sequencing and job dispatching, two common subtasks in solving this problem. These two rules are designed based on the critical ratio values of jobs. We also propose a mechanism to obtain better performance than the stand-alone scheduling process via genetic algorithms. With the nature of design of the proposed job sequencing rule, the genetic algorithm is designed not only to improve the schedule quality but also to save computation time. All the proposed rules and idea are carefully examined through several different scenarios.

In this paper, we use a graphical and mathematical modeling tool colored-timed Petri nets (CTPN) to model the testing flow in the wafer probe center. With this CTPN model, we can simulate the production processes, and keep track of the equipment status and the lot conditions efficiently and precisely. In the dispatching phase, we present the lot-based and the equipment-based selection schemes. Each of these two schemes has its own advantages, but also some drawbacks. Therefore, we propose a new approach pair generation mechanism and adaptive lot/equipment matching strategy, which can promise a dispatching strategy that can be more optimal in the sense that both lot-based and equipment-based viewpoints are taken into account simultaneously. In this paper, we further adopt an efficient algorithm auction algorithm to help us to find out the optimal solution to the internally generated lot/equipment matching problem. Besides, some adaptive factors are also applied. Lastly in the scheduling phase, we apply the genetic algorithm (GA) based approach to obtain a near-optimal solution to our scheduling problem. From our experiment results, the developed CTPN based genetic algorithm yields a more efficient solution than several other schedulers.

The objectives of this research are to develop a driving assistance system that can locate the positions of the lane boundaries and detect the existence of the front-vehicle. By providing warning mechanism, the system can protect drivers from dangerousness. In lane recognition, Gaussian filter, peak-finding procedure, and line-segment grouping procedure are used to detect land markers successfully and effectively. On the other hand, vehicle detection is achieved by using three features, such as underneath, vertical edge, and symmetry property. The proposed system is shown to work well under various conditions on the roadway. The vehicle detection rate is higher than 97%. Besides, the computation cost is inexpensive and the system's response is almost real time. Thus, the results of the present research work can improve traffic safety for on-road driving.

This work presents a region-based approach for background modeling to economize the use of space. In order to be immune to noise and changes resulting from illumination or motion, the partial directed Hausdorff distance is adopted while subtracting the foreground objects from the scene robustly. Instead of determining the threshold values manually, we use an adaptive method to automatically choose the threshold values. Finally, experimental results of applying our approach on a sequence of an indoor scene are provided to demonstrate the effectiveness of the proposed method.

The architecture of the conventional home automation system is usually centralized and thus causes many problems, e.g. lots of network traffic, heavy computation load, and poor fault tolerance. Another problem with the architecture is that there exists a variety of different home control networks incompatible with one another, which makes it difficult to integrate them in order to provide high level management functions in a home automation system. Besides the problems mentioned above, the most serious problem with a home automation system is its dynamically changing environment. Such frequently changing situation of the peripherals disallows straightforward configuration and maintenance of the home automation system. In this paper, we propose a mobile agent based integrated control architecture for home automation system, which is able to solve the problem mentioned above. This architecture is a distributed one, reducing the network traffic and computation load by delegating the management function to each control node. To integrate various kinds of home control network, we use computer network as the backbone and apply the concept of gateway to facilitate different networks to communicate with one another. Finally, we take advantages of the characteristics of mobile agent to cope with the problem of the dynamic environment and to enhance the fault tolerance mechanism.

In this paper, we develop a globally adaptive decentralized control scheme of time-varying robot manipulators for trajectory tracking control. Since the proposed adaptive control law is in a decentralized manner, only low-cost hardware is required for implementation. Furthermore, even of the time-varying parameters of the robot manipulator change arbitrarily fast, both the position and velocity tracking errors of the manipulators will converge to zero after employment of the proposed adaptive control law. For practical implementation, the adaptive law can be combined with a leakage term so that there is no chattering in the motion of the manipulator but at the price a residual set of the tracking errors as mentioned above, of which the size can be however made smaller by use of some larger design parameters. Finally, in order to illustrate the performance of the proposed scheme, simulation results are also provided, which turn out to be quite satisfactory.

In this paper, we propose architecture to simulate wafer probe. We use a modeling tool named CTPN (colored-timed Petri nets) to model all testing flow. With CTPN model, we can predict the delivery date of any specific product under some scheduling policies efficiently and precisely. In the scheduling phase, we combine two popular methods to contract high-quality schedules. One is to select machines for lots and the other is to select lots for machines. In each method, we use the GA-based approach to search for the optimal combination of a number of heuristic rules. This CTPN-based GA scheduler and helps us to find the good solution so as to meet the requirements in the complicated environment.

Autonomous driving system can assist to prevent the traffic accidents caused by the negligence of the driver. Obstacle detection and warning mechanism plays an important role in this research field. In this paper, we adopt the computer vision technology because of its large detecting range and abundant information when compared to other kinds of sensors. The proposed system is suitable for both the simplified environment such as freeway and the urban environment with complex background. Thus, the result here improves traffic safety not only for drivers, but also for all pedestrians on the road.

To provide good transportation services for passengers in modern buildings, a good elevator group control system (EGCS) is inevitably necessary. The viewpoint of designing the EGCS is very important. The passenger-based viewpoint proposed provides a new way to think about this system. The capacity constraint following consideration for the passengers is utilized to make the performance better. Details of elevator dynamics are modeled to meet the requirements. A traffic database is constructed in order to rebuild the system environment containing information of passengers so that it can be made as close to the real environment as possible. The rescheduling ability is achieved by a new mechanism-HCPM refinement, which is a priority maker for hall calls. The advantages of our EGCS are shown through extensive simulation results. We make a comparison between our EGCS and the previous one, which shows that our results are quite satisfactory and superior.

We propose a modeling tool named QCPN (queueing colored Petri net). The main idea of this tool is to combine the original CTPN (colored timed Petri net) with the queueing systems. With the QCPN model, we can predict the delivery date of any specific product under some scheduling policies efficiently and rather precisely. In the scheduling phase, we use the GA based approach to search for the optimal combination of a number of heuristic rules. This QCPN based GA scheduler can greatly reduce the computation time so as to meet the need for a rapidly changing environment.

In recent years, electronic appliances can be monitored and controlled by embedded microprocessors and be displayed on terminals, but they are still in lack of integration. Since the present home automation (HA) system is not equipped with efficient integration mechanism, it cannot fully manifest the worth of these developments. In order to achieve this goal of integration, many appliance manufacturers focus on the development of intelligent (or information) appliances to be integrated into a complete HA system for monitoring and controlling. Due to the advent of advanced computer and wideband network, the personal computer-based environment seems to be a very suitable platform for system integration. The personal computers can be linked by the network and are capable of powerful computation and easy display. We can take advantage of such abilities to develop an integration system.

Applying computer technology to vehicle driving has been studied for many years. In this research field, obstacle detection plays an important role in assisting drivers with warning mechanism when some dangerous situations may happen. In this paper, we propose a fast method for detecting and tracking bikes, pedestrians, and vehicles in front of a moving vehicle. In order to detect bikes and pedestrians efficiently, we apply a simplified fast stereo vision method to estimate their approximate positions. On the other hand, we apply the so-called sign pattern technique to estimate the vehicle positions. After that, different methods are used to classify and confirm different kinds of obstacles for adapting their heterogeneity.

In the past years, buildings are designed to be more and more powerful and intelligent mechanism, but it is still far from being perfect since building systems are lack of a uniform specification to establish standards for various appliances and communication protocol, and are also lack of a robust kernel to integrate all sub-systems in the buildings. For the former problem, we expect that the major appliance producers will eventually set up the standard for the appliances and communication protocol. For the latter problem, this paper proposes a complete system architecture with integrated control kernel to construct an intelligent building system rapidly and efficiently.

We can decide the operation order with unknown potential order requirements using the scheduling architecture proposed in the paper. First, the branch-and-bound search based on a Markov chain method is proposed. The Markov chain gets the service rate records and arrival rate records from the manufacturing execution system. We can get the possible beginning times of operations for each job via the Markov chain. The information of the possible beginning time can help us to approximate the solution space. Thus, by the information of the possible beginning times of operations, a branch-and-bound search scheduler can be used to find a sub-optimal scheduling.

Applying computer technology to vehicle driving has been studied for many years. Most of the studies focused on autonomous vehicle driving in a simplified environment like freeways, or life independent systems like GPS. A general case where kind of unexpectedly fatal conditions may occur which we are driving in an urban area, however, has not been considered. We proposed a system to satisfy the basic criteria for such a general driving assistance. We detect the obstacles on the ground in front of the vehicle we are driving, and then classify them into three predefined categories: pedestrians, vehicles and others. In the proposed system, we exploit a simplified stereovision system to detect the obstacles instantly. After that, to search and track pedestrians and vehicles, different methods are used for adapting to their heterogeneity. For the sake of implementation, we also propose a method to decide the maximal speed of driving to keep such kind of systems working.

Wafer fabrication is one of the most competitive manufacturing business in the world. In order to survive in such a strongly competitive environment, finding an effective schedule which can result in higher machine utilization and throughput rate, shorter cycle time, and lower WIP (work-in-process) inventory becomes a major task. Besides that, in order to help customers to make ordering decisions as well as to let the manager control the processing conditions of the fab, we need to predict some performance measures efficiently. We propose a modeling tool called queueing-Petri net (Q-PN) which combines the characteristics of queueing theory and Petri nets. It can be used to model various details of the manufacturing systems as well as to evaluate its performance very efficiently. Then, a general Q-PN model is presented to simulate the semiconductor manufacturing system. Based on this model, we propose a genetic algorithm (GA) based scheduler and an analysis-based predictor. In the GA scheduler, the chromosome represents a combination of scheduling policies, including lot release policies, machine selection rules, dispatch rules and batch rules. So, when the GA finishes its optimization process, an optimal scheduling policy is produced. As for the predictor, because it inherits the analytical property of queueing theory from the Q-PN model, we can use it to predict those performance measures efficiently such as the exact due date of some particular lot.

A real-time visual tracking system based on our proposed motion estimation algorithm is developed. The proposed motion estimation algorithm is used to predict the location of the target and then generate a control input so as to keep the target stationary in the center of image. The work differs from previous ones in that it is able to decouple the estimation of motion from the estimation of structure. The major contribution of this work is that simple, none computation intensive, correspondence-free, and numerically stable 3D motion estimation algorithms are developed. The robust target detection method in simple environment and a time reduction of the sum of squared difference method in a complex environment are minor contributions. The visual tracking system can achieve at a rate of 30 Hz. The robustness of the visual tracking system is validated by a number of experiments.

The probabilistic data association (PDA) filter is considered for the tracking of a single target in an environment of randomly distributed clutters. Significant performance degradation occurs when the measurements originate from biased and nearly stationary clutters rather than a non-stationary nor non-biased one. We propose a modified PDA filter to achieve a successful tracking in such an environment. Simulation results demonstrate the feasibility of the proposed approach.

A genetic algorithm (GA) embedded search strategy over a colored timed Petri net (CTPN) for wafer fabrication is proposed. Through the CTPN model, all possible behaviors of the wafer manufacturing systems, such as WIP status and machine status, can be completely tracked down by the reachability graph of the net. The chromosome representation of the search nodes in GA is constructed directly from the CTPN model, recording information about the appropriate scheduling policy for each workstation in the fabrication. A better chromosome found by GA is received by the CTPN based schedule builder, and a near-optimal schedule is then generated.

A significant amount of risk is involved in the wafer fabrication due to huge investment costs, long production cycle time, and short production life cycle. In this paper, a genetic algorithm (GA) embedded search strategy over a hybrid color-timed Petri-net (HCTPN) for wafer fabrication is proposed. Through the HCTPN model, all possible behaviors of the wafer manufacturing systems such as WIP status and machine status can be completely tracked down by the reachability graph of the net. The chromosome representation of the search nodes in GA is constructed directly from the HCTPN model, recording the information about the appropriate scheduling policy for each workstation in the fabrication. A better chromosome found by GA is received by the HCTPN based schedule builder, and then a near-optimal schedule is generated.

The intelligent building/home system (IBS) enhances the human life style. It makes our life more comfortable, efficient, and safe. With increasing use of computer, communication network, and building automation protocol, it will be possible to implement the IBS in every building soon. The critical problem to the IBS that can be a popular one is how to construct the IBS quickly and efficiently. We propose a systematic method and model to construct an IBS and a control kernel to integrate it. After that, the IBS will be easy to construct and it will be a flexible and scalable system.

A hybrid model of a multiple elevator system is proposed, consisting of a color-timed transition Petri net (CTTPN) model and a set of control rules implemented via the so-called control places in the CTTPN model. The Petri net model is a highly modular structure, whose constituent modules can be classified into four types: call management module, loading/unloading module, basic movement module, and direction reversing module. The whole complete model is a combination of the copies of the above four modules. Since the firing sequences of the CTTPN equate the evolution of the modeled system, they can be regarded as a schedule. A dynamic scheduling with learning ability is proposed to obtain the desirable schedule. A new concept of control places is also introduced in the proposed model so as to make the modeling more precise and to reduce the reachability graph more efficiently. To show the feasibility of the proposed method, an emulator in elevator control kernel and elevator scheduler kernel were constructed for demonstration.

The overwhelming complexity of the deburring process and imprecise knowledge about robot manipulators leads to a certain control problem. In the paper, a new design of hybrid position/force control of robot manipulators via an adaptive fuzzy control approach is proposed to solve these problems. The control architecture consists of an outer-loop command generator which can automatically determine the robot motion profile to yield the desired chamfering force and an inner-loop adaptive fuzzy hybrid position/force controller which can achieve the desired chamfer depth compliantly as well as the aforementioned command in real time. The proposed adaptive fuzzy controller using B-spline type membership functions can compensate the uncertainties in a much smoother and locally weighted manner and consequently guarantee global stability of closed-loop systems. To demonstrate the effectiveness of the developed work, it is applied to the control of an industrial robot arm for deburring tasks.

The major control problems for robot deburring mainly arise from the uncertainty of the robot manipulators and complex deburring process. In this paper, a new design of hybrid force/position control of robot manipulators via adaptive fuzzy approach is proposed to solve these problems. The control architecture consists of an outer-loop command generator which can automatically determine the desired robot motion profile and an inner-loop adaptive fuzzy hybrid force/position controller which can achieve the command in real time. To demonstrate the effectiveness of the present work, the approach proposed is applied to the control of a five degree-of-freedom articulated robot manipulator for deburring tasks.

This paper proposes a new generation of evaluation tool for online design and scheduling, wherein the planned control policy designed by engineers can be online evaluated. Performance can be evaluated by a new prediction rule that differs from existent state-independent or steady state model such as a queuing network model. In addition to that, a new optimal control rule or schedule can then be derived from the originally planned ones and the feedback from the manufacturing engineers. Simulation and prediction methods are applied together to give an optimum in the time slot of snapshot. Such evaluation tool can also be used to look for the possible conflicts in the potential solution and suggest some necessary steps to avoid those conflicts.

Nowadays, the market place is continuously changing and unpredictable, and hence an efficient prototyping environment is crucial. We propose a virtual factory wherein an efficient prototyping testbed is provided. This paper is one among very few that tries to represent the virtual factory in an analytic form so that many existing mathematical analysis can be applied. New pseudo resources can be added to form a new virtual environment, and control policy designed by engineers will be evaluated before being issued. An example of utilizing the prototyping testbed is given. The proposed testbed is compared with a traditional testbed and the results validate the intelligence and efficiency of the present prototyping testbed.

This paper proposes a multi-agent based dynamic scheduling approach for a flexible assembly system. We first introduce a flexible control system developed by Intelligent Robotics and Automation Laboratory in National Taiwan University. Based on that control system, the agents can communicate with each other conveniently. A generic agent architecture is proposed to model the pieces of equipment in the flexible assembly system. With a distributed architecture, the agents make their scheduling decisions using their local rule base. The agents acquire the resources following the distributed resource allocation protocol. The scheduling complexity is reduced to meet the real-time response requirement in the applications for flexible automated production. The present work is applied to the experimental robotized flexible assembly system in the above laboratory.

A hybrid model of a multiple elevator system is proposed, consisting of a timed place Petri net (TPPN) model and a set of control rules implemented via the so-called control places in the TPPN model. The Petri net model is a highly modulized structure, whose constituent modules can be classified into four types: basic movement module, loading/unloading module, direction reversing module, and call management module. The whole complete model is a combination of the copies of the above four modules. Since the firing sequences of the TPPN equate the evolution of the modeled system, they can be regarded as a schedule. A dynamic scheduling strategy is proposed to obtain the desirable schedule. A new concept of control places is also introduced in the proposed model so as to make the modeling more precise and to reduce the reachability graph more efficiently. To show the feasibility of the proposed method, an emulator of the elevator system is constructed for demonstration.

Proposes a systematic method of creation and application of a virtual factory. Basic definition of a virtual factory is first given, followed by its foundation principles and architectural overview, and then a systematic method based on object technologies is proposed to create the virtual factory. Specially, for a real manufacturing environment, a corresponding virtual factory can be created via the polymorphism parameterized universal virtual factory with several interactions. The virtual factory can be linked to the real factory and enable simulation-based control by means of the switching architecture we proposed. Its reliable predictions can not only prove the production scenarios but also improve the decision making process of acquisition managers in the applications for flexible automated production. The present work is being applied to the experimental robotized flexible assembly system developed by Intelligent Robotics and Automation Laboratory in National Taiwan University.

We propose a method of controlling a deburring flexible manipulators. Maintaining a constant contact normal force and constant tangential cutting force is required. The dynamics of both the deburring process and the flexible manipulator are investigated in detail, and the latter is derived using Lagrangian method with an assume-mode approach. To facilitate the controller design, a singular perturbation technique is then utilized to separate the system into a slow- and fast-subsystem. For solving the deburring problem, a new hybrid force/position controller is proposed for the slow subsystem; it is implemented by adaptive control strategy, whereas a dynamic feedback controller is developed for the fast subsystem. It is shown that both the position tracking error and the force error converge to zero as time approach infinity. Finally, the computer simulations and experiments of a two-link flexible manipulator confirm the effectiveness of the proposed adaptive controller.

An intelligent automated robotic assembly system consists of several subsystems capable of providing dynamic interactions with the environment in order to accomplish a task properly. These subsystems perform various functions like data gathering, decision making, and task execution. Although a great deal of work has been done on individual subsystems, more attention must be given to the way how these subsystems are integrated so as to achieve the high efficiency of automated production. We propose a cooperative multi-agent model of a shop floor control system architecture of robotic assembly automation and extend this model to all automated production system. Based on this model, we develop a control kernel named TOFAK (task oriented flexible automation kernel) to support users to easily implement any shop floor control system. The by-product is to allow system designers to easily expand an existing system or to integrate several automation systems which are all controlled by TOFAK.

How to perform control and achieve stability of robotic manipulators with joint flexibility forms a problem of profound practical and theoretical interest. This paper is to investigate and to solve this problem without strict assumption on the joint stiffness. Here, an adaptive control scheme of a flexible-joint manipulator, which takes into account its full nonlinear dynamics, is presented. Without the knowledge of the system model, the developed control lams, requiring only the position and velocity information of the actuators and links, is capable of driving the link tracking errors asymptotically to zero, while maintaining the uniform boundedness of all signals in the closed-loop system. To demonstrate the effectiveness of the proposed control law, an example of a two-link flexible-joint manipulator is constructed and a number of computer simulations are performed which show quite satisfactory results.

The goal of the automated deburring can be achieved by maintaining a constant force on the grinding tool in the direction normal to the constraint surface while following the positional trajectory in the direction tangential to the surface. In this paper, the dynamics of both the deburring process and the flexible manipulator are investigated in detail, and a singular perturbation technique is then utilized to separate the system into a slow subsystem and a fast subsystem, whereby an adaptive hybrid position/force controller is derived for the slow subsystem whereas a dynamic feedback controller is developed for the fast subsystem. It is shown that the motional tracking error and the force regulation error converge to a small residual set. Finally, computer simulations and experiments of a 2-link flexible manipulator confirm the effectiveness of the proposed controller.

In this paper, a genetic algorithm (GA) embedded dynamic search strategy over a Petri net model provides a new scheduling method for a flexible manufacturing system (FMS). The chromosome representation of the search nodes is constructed directly from the Petri net model of an FMS, recording the information about all conflict resolutions, such as resource assignments and orders for resource allocation. The GA operators may enforce some change to the chromosome information in the next generation. A Petri net based schedule builder receives a chromosome and an initial marking as input, and then produces a near-optimal schedule. Due to the NP-complete nature of the scheduling problem of an FMS, we also propose a dynamic FMS scheduler incorporating the proposed GA embedded search scheme, which generates successive partial schedules, instead of generating a full schedule for all raw parts, as the production evolves.

This paper proposes a rule-based dynamic scheduler for a flexible assembly system. We first introduce a flexible control system developed by the Intelligent Robot and Automation Laboratory in National Taiwan University. Based on that control system, the relationship between the control system and the scheduler is characterized. With focus on realization, hardware limitations such as long computation time and excessive memory-space usage are relaxed by incorporating several heuristic measures. The present work is applied to the simulated robotized flexible assembly system in our laboratory.

In this paper, firstly we discuss the architecture of a flexible automated production system under which all components are well-modularized. Then, a solution, EMFAK (event-driven multi-tasking flexible automation kernel) is proposed to speed up the construction of system control. Finally, a flexible assembly system using EMFAK is described to show how to apply it to a real flexible automated production system.

As has been realized, the flexible-link manipulators have attracted more and more attention from robot control theorists and/or robot users because of its various potential advantages. But since the control degree-of-freedom is much less than that of the system, many control strategies which succeed in the conventional rigid robot control cannot be directly used in the flexible robot control problems. In this paper, a nonlinear control scheme has been proposed as a solution to these control problems. In particular, to cope with the existing model uncertainty, an adaptive version of this nonlinear control law has been proposed. Stability proof of the overall closed-loop system is then given via Lyapunov analysis. In addition, extensive experimental results are also provided to demonstrate the effectiveness of the proposed controller.

This paper proposes an adaptive fuzzy hybrid force/position control scheme, which can force the end effector of robot manipulators to follow the contour of an object in lack of knowledge of the exact geometric shape. The control objective is to perform hybrid force/position control regardless of the existence of the manipulator dynamics. The control algorithm proposed can adaptively update the position trajectory command as well as fuzzy control rules, and consequently, guarantee the global stability and drive the tracking errors to a neighborhood of zero. The present work is applied to the control of a five degree-of-freedom (DOF) articulated robot manipulator. Simulation results show that the proposed control architecture is featured in fast algorithmic convergence.

In this paper, a hybrid model of a multiple elevator system is proposed, consisting of a timed place Petri net (TPPN) model and a set of control rules implemented via the so-called control places in the TPPN model. The Petri net model is a highly modulized structure, whose constituent modules can be classified into three types: basic movement module, loading/unloading module, and direction reversing module. The whole complete model is a combination of the copies of the above three modules. Since the firing sequences of the TPPN equate the evolution of the modeled system, they can be regarded as a schedule. A heuristic search algorithm, A* search, is thus used on the reachability graph of the TPPN model to obtain the desirable schedule. To show the feasibility of the proposed method, an emulator of the elevator system is constructed for demonstration. The experiment result for a 15-floor building with four elevators is shown to be satisfactory.

The major problems of hybrid force/position control arise from uncertainty of the robot manipulators and unknown parameters of the task environment. In this paper, a new design method of the hybrid force/position control of the robot manipulators is proposed to solve these problems. The control objective is to track the desired force and position trajectories simultaneously regardless of the unknown parameters of the task environment and the existence of the manipulator dynamics, represented as a fuzzy rule-base. The algorithm embedded in the proposed architecture can automatically update the fuzzy rules and, consequently, guarantee the global stability and drive the tracking errors to a neighborhood of zero. The present work is applied to the control of a five degree-of-freedom (DOF) articulated robot manipulator. Simulation results show that the proposed control architecture is featured in fast convergence.

In this paper, we propose the scheduling methodology for a multirobot assembly cell, which is then integrated into a simulation environment. Since modeling a multirobot assembly system is difficult and tedious, a systematic method is proposed to transfer the AND/OR product assembly graph and the domain knowledge to assembly rules. Given this rule-base knowledge, an inference engine first generates all possible subsequent assembly tasks, and then a search algorithm finds the optimal ones. These promising tasks thus become the operation commands assigned to the assembly system. A simulation environment of a two-robot assembly cell is built and an experiment is performed using the proposed scheduling strategy. Satisfactory performance has been demonstrated.

In this paper, we tackle the problem of nonlinear adaptive hybrid control of constrained robots with flexible links. According to the physical properties of a flexible manipulator, a two time-scale approach, namely singular perturbation approach, is further utilized for thorough analysis and general controller design. It is shown that asymptotic motion tracking can be effectively achieved, whereas the force regulation errors can be made arbitrarily small. For demonstration of the controller performance, experiments of a two-link flexible manipulator were performed for the proposed controller and satisfactory results observed.

Because the control DOF (Degree of Freedom) is much less than the motion DOF when a flexible manipulator is commanded to track a desired trajectory, many control strategies that succeed in conventional rigid-robot control cannot be directly applied to solve the flexible robot control problem. In this work, an adaptive variable structure scheme has been proposed to solve such a problem. The full nonlinear dynamics of the whole system are all taken into account for the control design. To alleviate the chattering phenomenon commonly seen in variable structure type of control, a saturation type adaptive scheme has also been proposed. For verification of the effectiveness of the proposed controller, a two-link flexible manipulator is built up and the promise of the controller is experimentally demonstrated.

This paper presents an adaptive robust fuzzy control architecture for robot manipulators. The control objective is to adaptively compensate for the unknown nonlinearity of robot manipulators which is represented as a fuzzy rule-base consisting of a collection of if-then rules. The algorithm embedded in the proposed architecture can automatically update fuzzy rules and, consequently it is guaranteed to be globally stable and to drive the tracking errors to a neighborhood of zero. Focusing on realization, hardware limitations such as traditional long computation time and excessive memory-space usage are also relaxed by incorporating heuristic concepts, which reveals the flexible feature of this architecture. The present work is applied to the control of a five degree-of-freedom (DOF) articulated robot manipulator. Experiment results show that the proposed control architecture features fast convergence.

Flexible manufacturing systems (FMSs) have received considerable attention and evolve to be one of the fastest growing industrial fields in the last decade. In these systems, much higher efficiency of manufacturing can be achieved (owning to their intrinsic flexibility) provided a good scheduling policy is adopted. In this paper, we propose a dynamic scheduler with a hierarchical structure to cope with the unavoidable disturbing events in such dynamic systems like an FMS. In particular, based on our earlier work (1991), we handle the rescheduling problem as well as the deadlock avoidance problem. The merit of this work is its completeness in considering all possible components in an FMS, including AGV transportation system.>

In this paper, a timed place Petri-net (TPPN) model for flexible manufacturing systems with the components of machines, limited buffers, robots and the material handling systems, automated guided vehicles (AGV's) is constructed. Since a firing sequence of the TPPN from the initial marking to the final marking can be seen as a schedule of the modeled FMS, by using an A* based search algorithm, namely, Limited-Expansion A algorithm, a near-optimal schedule of the part processing can be obtained using reasonable computing time and memory requirement. For a large volume of parts, the authors also propose an adaptive scheduling approach to generate a near-optimal schedule in an economical computing time. In order to show the effectiveness of the proposed method, a prototype FMS in the Automation Lab of the Department of Mechanical Engineering, National Taiwan University, is used as a target system for implementation.>

This paper presents an adaptive robust fuzzy control architecture for robot manipulators motion. The control objective is to adaptively compensate for the unknown nonlinearity of robot manipulators, which is represented as a fuzzy rule-base consisting of a collection of if-then rules. The algorithm embedded in the proposed architecture can automatically update fuzzy rules and, consequently, it is guaranteed to be globally stable and to drive the tracking errors to a neighborhood of zero. Focused on realization, hardware limitations such as traditional long computation line and excessive memory-space usage are also related by incorporating heuristic concepts, which reveals the flexible feature of this architecture. The present work is applied to the control of a five degree-of-freedom (DOF) articulated robot manipulator. Simulation results show that the proposed control architecture is featured in fast convergence.>

This work focuses on the control of flexible manipulators. Due to the complicated dynamics of a flexible manipulator, which does not satisfy the so-called "matching condition" many control strategies which succeed in conventional rigid robot control cannot be directly used in flexible robot control problems. In this work, a generalized computed torque control scheme has been proposed to solve this kind of control problem. Furthermore, to cope with the model uncertainty, an adaptive control scheme has also been proposed. Simulation and experimental results demonstrate the effectiveness of the proposed controller.>

The problem of adaptive coordinated control of multiple robot arms transporting an object is addressed. A stable adaptive control scheme for both trajectory tracking and internal force control is presented. Detailed analyses on tracking properties of the object position and velocity and the internal forces exerted on the object are given. It is shown that this control scheme achieves satisfactory tracking performance without using the measurement of contact forces and their derivatives. It can be shown that this scheme can be realized by decentralized implementation to reduce the computational burden. Moreover, some efficient adaptive control strategies can be incorporated to reduce the computational complexity.>

A modular approach is presented for constructing Petri-net models for a class of flexible manufacturing systems (FMSs) composed of a transportation vehicle and several functional groups of entities such as machines and buffers. The resulting model preserves the geometric characteristics of the transportation subsystem as well as the flexibility of alternative routes for material flow in an FMS. By separating the machine-dependent part from the whole system, the final model in a modular structure is adaptable to various task flow requirements. In addition, the methodology can deal conveniently with a reconfiguration of the transportation layout.>

The authors address the problem of adaptive hybrid controller design for constrained robots, with consideration of computational efficiency. Two efficient control schemes based on Lagrange-Euler and Newton-Euler dynamics formulations are presented. Detailed analyses of tracking properties of joint positions, velocities, and constrained forces are derived for both the Lagrange-Euler approach and the Newton-Euler approach. Although control laws in these two approaches are developed independently, a close connection between them is indicated, which suggests a possible bridge over different general adaptive approaches based on the two dynamics formulations.>

An adaptive force control, scheme is presented that enables a single-link mechanism with joint flexibility to track a desired force trajectory. A dynamic model of the link system is derived, based on which a two-stage controller is constructed. It is shown that although all the system parameters including environment stiffness are unknown except for some of their bounds, all signals inside the closed-loop system remain uniformly bounded. Moreover, the force tracking error is driven to zero asymptotically. Simulation examples are provided to demonstrate the effectiveness of the proposed controller.>

An efficient method of solving problems of classification and selection in an FMS, using a minimum spanning tree, is proposed. Computer simulation examples which show a satisfactory result are provided. The total computational time spent is economical. The application of the method to these classes of problems is promising.>

A Hopfield network has been proposed as a novel approach to achieve memory associativity and to solve combinatorial optimization problems. The authors presently relate optimization problems to problems of memory association of a Hopfield network and show the limitations of the network when it is used to solve NP-complete problems from the viewpoints of the aliasing effect among pattern sets and the information capacity embedded in such a network. A simplest Hopfield network for solving the race traffic problem is constructed to manifest the similarity between memory association and optimization problem resolution as well as to discuss the stability of convergence in synchronous and asynchronous operation modes. By transforming the traveling salesman problem to a memory association problem, it is shown that the use of a Hopfield network for solving NP-complete problems is, in fact, overloaded

The authors present an adaptive control scheme for trajectory tracking of robot motion. A dynamic model of a manipulator with flexible joints that is particularly useful for derivation of the control law is adopted. Based on this model, a two-stage controller consisting of a feedback loop and a parameter adaption loop is established. It is shown that without precise knowledge of the parameters of the manipulator and its joint flexibilities the tracking error converges to zero asymptotically. This implies robustness of the control scheme to the variation of payload or parameters as long as their rate of change is moderate. The convergence can be sped up by increasing the suitable controller gains.>

An approach that can dynamically solve the planning and scheduling problem in a flexible manufacturing system (FMS) is presented. This problem is formulated as the determination of an optimal routing assignment of p automated guided vehicles among m workstations in order to accomplish N tasks in an FMS. A useful task representation called workgraph is introduced to facilitate the latter computation; then the A* search algorithm, the minimax criterion, and source heuristic rules are used to solve this routing assignment problem dynamically. The approach obtains a near-optimal solution in moderate computation time, and, in addition, solves some dynamic situations so as to make the FMS more flexible.>

The problem of navigating a mobile robot around barriers in an unexplored terrain is studied, where all the obstacles within the terrain are not necessarily of polygonal shape or convex. A model map is used to memorize the configuration of the environment observed so far and it is updated while the robot is being navigated. With safety' as a important factor to the solution of the problem, an algorithm which tends to find a center-line path among obstacles is proposed. The case where the sensor has only limited effective range is also considered. Detailed proof is provided of the collision-free and goal-convergent properties of the algorithm.>

A system with a two-level structure (high and low level) for dynamically solving the problem of planning and scheduling in an FMS (flexible manufacturing system) is presented. The problem is first formulated as the determination of an optimal routing assignment of p automated guided vehicles (AGVs) among m workstations to accomplish N tasks, facing several possible dynamical situations, e.g. change of due date or breakdown of some workstation(s). A hierarchical system is then built to solve this optimization problem in a dynamical manner. The low-level structure aims to solve the AGV routing problem among workstations given a set of AND/OR graphs which represent tasks to be processed. On the other hand, the high-level structure, using a rule-based system, provides necessary data for low-level use and simultaneously determines principles concerning how to respond to the occurrence of some unexpected events. It is shown that a near-optimal solution can be derived with moderate computation time that allows operation in an FMS to be more flexible.>