To reach the goals of low carbon emissions and a high quality of life (QOL) in Thailand, the Japan Science and Technology Agency (JST) and the Japan International Cooperation Agency (JICA) have undertaken the technical cooperation project known as the Smart Transportation Strategy for Thailand 4.0. As part of this project, an approach for estimating a QOL value from an image in a driving scene is proposed in this paper. The proposed method is based on a deep neural network. It assumes, as in the previous approach, that the QOL of a driving scene depends on the type and number of objects in the scene. Therefore, the proposed approach first applies a semantic segmentation method to an input image, and the QOL value is estimated by the deep neural network based on the MetaFormer. The loss function and optimizer are changed from the previous method to increase the accuracy of the proposed method. Smaller images than those used in the previous approach are used for faster processing. These approaches made the proposed method process more accurate and faster than the previous approach. The effectiveness of the proposed method was demonstrated by some experiments.

Colonoscopy video acquisition has been tremendously increased for retrospective analysis, comprehensive inspection, and detection of polyps to diagnose colorectal cancer (CRC). However, extracting meaningful clinical information from colonoscopy videos requires an enormous amount of reviewing time, which burdens the surgeons considerably. To reduce the manual efforts, we propose a first end-to-end automated multi-stage deep learning framework to extract an adequate number of clinically significant frames, i.e., keyframes from colonoscopy videos. The proposed framework comprises multiple stages that employ different deep learning models to select keyframes, which are high-quality, non-redundant polyp frames capturing multi-views of polyps. In one of the stages of our framework, we also propose a novel multi-scale attention-based model, YcOLOn, for polyp localization, which generates ROI and prediction scores crucial for obtaining keyframes. We further designed a GUI application to navigate through different stages. Extensive evaluation in real-world scenarios involving patient-wise and cross-dataset validations shows the efficacy of the proposed approach. The framework removes 96.3% and 94.02% frames, reduces detection processing time by 38.28% and 59.99%, and increases mAP by 2% and 5% on the SUN database and the CVC-VideoClinicDB, respectively. The source code is available at https://github.com/Vanshali/KeyframeExtractionNote to Practitioners—The widespread acceptance of colonoscopy procedures as a gold standard for CRC screening is constrained by the massive amount of data recorded during the process that needs to be manually reviewed. Such manual procedures are burdensome and induce human diagnostic errors. This article suggests an automated framework to extract keyframes (important frames) from colonoscopy videos that can efficiently represent the clinically relevant information captured in the video streams. This is achieved by the automated removal of uninformative and highly correlated frames, which do not add to clinical findings. The approach ensures diversity among keyframes and provides clinicians with a multi-view of polyps for easy resection. In addition, the proposed multi-scale attention-based model improves the polyp localization performance, which further helps in refining the keyframe selection process. The comprehensive experimental results corroborate that discarding insignificant frames can enhance polyp detection and localization performance and reduce computational requirements. The study estimates 30% to 60% time saving for clinicians during video screening. In clinical practices, the proposed automated framework and our designed GUI would enable surgeons to visualize the essential data better with minimal manual interventions and assist in precise polyp resection.

International audience

One of JST and JICA's technical cooperation projects, ”Smart Transportation Strategy to Realize Thailand 4.0,” aims to reduce traffic congestion to simultaneously achieve low carbon emissions and improve citizens’ quality of Life (QoL). To achieve this goal, it is necessary to develop an AI-based system that allows users to select a combination of transportation modes that meets the various needs of people of all ages and genders, and to realize this goal, evaluation of QoL from images is required. The project suggests how to improve QoL when people moves from some point to another point and some application software to recommend the time scheduling about how to optimize the QoL in a day. It recommends the highest QoL route when we have several path candidate of roads and further some application is necessary to show how QoL dynamically changes in the route. That is scenario of the QoL project. In the previous research, a method for QoL estimation from images was proposed based on semantic segmentation and MLP (Multi Layer Perceptron) by constructing a unique questionnaire form and collecting data in a car view scene. However, the previous method estimates QoL without considering differences in the attributes of the questionnaire data, so the constructed model does not satisfy the attributes of all people. In the proposed method, multiple subjects are asked to subjectively evaluate QoL in 5 levels for the same driving image data set, and the subjects are grouped by clustering based on the evaluation results. The effectiveness of the proposed method was demonstrated by comparing the MAE (mean absolute error) of the model constructed by the proposed method with that constructed by other subjects in a 5-step cross-validation.

Medical doctors identify benign or malignant colon polyps by their size and shape. Since this identification is related to whether or not resection surgery is necessary, a technology for estimating absolute size and shape from endoscopic images is required. In previous research, the method for recovering the size and shape of polyps using a blood vessel region as a reference object has been proposed. This paper proposes a method to recover the size and shape of polyps by using a cylindrical endoscopic hood as a reference object. The experimental results confirm that the proposed method can estimate reflectance factor without using blood vessel information.

Hyperspectral image classification provides land cover information for environment and urban management. However, one of the major challenges of hyperspectral image classification is the small amount of labeled data, which makes the training model have low accuracy and poor robustness. Therefore, this paper proposes a cross-scene classification model based on feature learning (CSCFL). Firstly, the model reduces the differences between different scenes based on unsupervised domain adaptation technology. Secondly, the depth separable convolution is introduced to improve the ability of the model to capture fine spatial features. Finally, the spherical tree is introduced for spatial segmentation to improve the search efficiency of weighted KNN (WKNN) classifier and save computing cost. Experimental results on Pavia and Indiana datasets show that the proposed algorithm has higher classification accuracy and lower computational cost.

Hyperspectral imagery (HSI) and light detection and ranging (LiDAR) remote sensing technologies are important ways to obtain surface information. Combining the characteristics and advantages of hyperspectral images and LiDAR DSM data, we propose a classification method for co-classification of hyperspectral images and LiDAR data in this paper. The model uses a dual-branch HSI image and LiDAR data classification method. First, the features of shallow convolution and deep convolution are merged in the spatial feature extraction process of HSI, which combines to focus on more global information. Then the space and spectrum are combined, considering the characteristics of different layers, the network has the characteristics of multiscale. Finally, the LiDAR branch introduces dilated convolution to obtain a larger receptive field and reduce information loss. The classification experimental results on Houston and Trento dataset show the superiority of our proposed method in the collaborative classification of hyperspectral images and LiDAR data.

This paper treats an image translation using Multimodal Unsupervised Image-to-Image Translation (MUNIT) from the white light source image to the Narrow Band Imaging (NBI) of the endoscope images and proposes a method to improve the detection performance by the Single Shot Multibox Detector (SSD) which trains dataset of white light source image by adding the generated NBI-like images. The proposed approach makes it possible to generate an NBI-like image by keeping the existing polyp, inner wall and specific features of coloring and brightness of the original endoscope image. It is shown that increasing the number of data of the generated images achieves the better performance. The performance of the proposed approach was evaluated using the actual endoscope images which include the polyps of the various shapes. Recall of 80.62% and precision of 93.47% were obtained as a result through the evaluation of computer experiments.

The presence or absence of cancer metastasis in the lymph-nodes using an AI-based approach has been important these days in the medical field of gastroenterological surgery. The medical field needs the introduction of machine learning to help the knowledge and skill of surgery of medical doctors who are taking operations by checking CT scans or MRI images for cancer disease. Recent machine learning based researches on lymph-node are mainly either detection of the location of lymph-nodes or classification of cancer metastasis. This paper proposes a method to perform both tasks at the same time with a single U-Net model. Obtained results in the experiments show that the classification accuracy is further improved compared with the previous approach while keeping the detection ratio as almost the same level as that of the related paper.

In recent years, opportunities for diagnosis using endoscopy aiming a less invasive treatment are increasing following the disease rate of colorectal cancer. Computer-aided diagnosis has been developed based on deep learning methodology, it aiming to improve the accuracy of diagnosis and support immature medical doctors. To satisfy the learning dataset, this paper proposes a data augmentation methodology where automatic image generation of polyp images using Pix2Pix and depth map obtained from the original image. The problem of lack of the learning dataset of polyp images can be solved by the proposed approach and the effectiveness of the generated data was confirmed by the quantitative evaluation with the improved performance of SSD (Single Shot Multibox Detector) in the experiments.

This paper proposes a framework to automatically construct an attribute dataset with images and tags on the Social Network Service (SNS). In the field of general object recognition, attributes that represent characteristics and categories of objects (e.g., ‘cat’ has ‘white’ and ‘cute’ as its attributes) showed high effectiveness. In the existing study, the attributes have been prepared by researchers, and the costs of the annotation and the subjectivity of the annotator should be the issues to construct attribute dataset. In this paper, we verified the feasibility that tags on the SNS could be assumed the attributes. Based on the study, the proposed method automatically collects images and their corresponding attributes from the SNS; the attributes satisfy a)understandable for humans, b) understandable for computer, and c) available on multiple categories. The experiments confirmed the effectiveness of the attribute datasets constructed by the proposed method in the generic object recognition; each dataset from Instagram and Flicker showed 71.6% and 79.3% accuracy, respectively. It was suggested that the attribute datasets that the proposed method automatically constructed showed almost the same effectiveness as the human-created dataset.

It is important to estimate the size and shape of the polyp from endoscope images in the medical diagnosis. Ref.[1] was proposed as a method to estimate the absolute size of the polyp, where movement of endoscope in depth direction was estimated from two endoscope images. Ref.[1] introduced template matching using blood vessel to determine the correspondence between two images. However, it is not appropriate for actual endoscope environment since actual motion includes not only the camera transition but also the camera rotation. Further object motion is non-rigid motion and it is difficult to determine correct correspondence points between two images. So, this paper proposes a new approach to extract a blood vessel structure based on the blood vessel region using a graph structure, where the graph is matched between two images. Here, a matching method of blood vessels is described, where camera rotation and non-rigid motion of the target object are under consideration. Effectiveness of the proposed approach is confirmed in the experiments.

Young patients with neurological pathologies develop walking inadequacies. These inadequacies are due to a strong contraction of the muscle at three levels: pelvis, knee, and foot. Patients can have surgery only once adulated. Before that, physicians, after observing the patient walking and detecting the zone of disorder, inject toxin in one of the three levels in order to relax the muscle. The detection of trouble areas is done by extracting walking cycles and walking phases, then calculating angles formed between the different joints at key moments in the walking phases. With the naked eye, it is difficult for the physician to detect each phase in the walking cycle of patients and to extract angles of inclination. As a solution, the physician takes frontal and sagittal videos of patients and analyzes them offline. The objective of this work is to extract and classify automatically the walking phases of a walking cycle to help physicians in their task. This process is made automatic thanks to transfer learning of VGG16 network. Extensive experiments compare the different Network settings in order to study the integration of the model in clinical routines.

As a method to judge the benign or malignant polyp from endoscope images, some methods have been proposed using an ultra-high magnification endoscope. The ultra-high magnification endoscope enables the diagnosis at the cell level. However, it tends to spend many times for diagnosis and requires specific expensive devices. There are three types of images that are taken for diagnosis by the regular endoscope: white light, dye, and narrowband image (NBI) in general. This paper proposes a benign/malignant polyp classification method using these images taken by the regular endoscope. Each image features derived from endoscope images are extracted by adapting a pre-trained CNN to each domain. Finally, polyps are classified using extracted features. Experiments confirmed that the proposed method enabled the classification of benign or malignant colorectal polyps with over 90% accuracy.

This paper proposes a method for the automatic construction of a dataset with annotation data for object detection. The accuracy of the object detection method depends on the dataset in general. The dataset for object detection needs many images with annotation data. Obtaining image data by manual operation takes a lot of costs. It also costs much that annotation data are made by manual annotation software. This paper tries to solve these problems to construct the image dataset for the object detection automatically. The proposed method uses a method to collect the image data automatically among images on the Web using Web image mining. A new method for making annotation data is also proposed. The Mask R-CNN is used for automatical annotation. The proposed approach constructs a dataset automatically without almost manual operation. It is confirmed that the proposed approach performs the automatic construction of the dataset with high accuracy.

In this paper, the Generative Adversarial Network (GAN) is applied to LiDAR data classification. Generative Adversarial Network usually includes a generating network and a discriminant network. In GAN, a convolutional neural network (CNN) is designed to distinguish inputs. Another CNN is used to generate so-called false inputs. Combining with the actual training samples, the discriminant CNN is fine-tuned to improve the final classification performance. The proposed classifier is implemented on real data sets. The results show that the accuracy of the proposed network is higher than that of the classification method based on CNN, which shows that the features extracted by the network have better discrimination and stronger competitiveness.

Light detection and ranging (LiDAR) data is a very important type of data used for terrain classification. The traditional convolutional neural network (CNN) has insufficient effective transmission of features and gradients in feature classification and can only set a fixed input size by experience. In this paper, spatial transformation network (STN) and densely connected convolutional network (DenseNet) are combined to form STN-DenseNet, which makes the input data adaptively deform according to the network needs, making full use of all information from the front layers of the network. Thus the transmission of features and gradients are more effective. The proposed framework performs experiments on two LiDAR-DSM datasets (i.e. Bayview Park and Recology datasets). The results show that, comparing with the traditional deep convolution model, STN-DenseNet can improve the classification accuracy of LiDAR-DSM data.

Nuclei segmentation is an important pre-processing step for any vision based cytopathological diagnostic system which extracts information from nuclei to perform tasks such as cancer detection. A cell nuclei segmentation pipeline should be robust, accurate and fast. We propose a deep learning based model, Contour-Aware Residual W-Net (WRC-Net), which consists of double U-Net, [5] or W-Net. The first U-Net learns to predict nuclei boundaries and the second generates the segmentation map. Our model can accurately segment a 128x128 dimensional image in less than 0.05s. Our model can learn from a very limited training data with as low as a single training image. We tested our model on real HE (Hematoxylin and Eosin) stained cell images and it showed better overall performance against previous state-of-the-art nuclei segmentation methods.

Shape from Shading and Photometric Stereo are famous approaches to recover the 3D shape from image(s). These approaches can obtain 3D shape from observed gray scale image(s) but it is necessary to estimate reflectance parameters for objects when some specular reflectance components are observed. Specular reflectance is difficult to be handled as diffuse reflectance in general. It is expected to remove specular reflectance component without assuming any kind of reflectance function (reflectance model). This paper proposes a method to remove specular reflection components using 4 observed images taken under 4 different light source directions using relighting approach of diffused reflectance based on image linearization. Results are demonstrated via computer simulation and experiments.

This paper proposes a method for augmenting learning data of road damage dataset considering the influence of the augmented data on classification accuracy. Data augmentation is a very important task in the field of machine learning because more learning data causes increasing the accuracy of classification accuracy in general. The quality of the augmented data influences the accuracy of the classification. Effective data augmentation method for increasing classification accuracy is needed. The proposed method generates learning data by selecting effective data augmentation methods depending on the class of road damage. The method uses You Only Look Once v3 (YOLOv3) for detection and classification of road damage in an image. It is tuned by data adding the data augmented by the proposed method to the road damage dataset presented to the public. The experimental results show that the proposed method can increase the accuracy efficiently and effectively. The proposed selection of data augmentation methods improves remarkably mean Average Precision (mAP) which is one of the accuracy indices.

This paper proposes a new defect classification method of electronic board using Dense SIFT and CNN which can represent the effective features to the gray scale image. Proposed method does not use any reference image and effective keypoints are detected using Dense SIFT on the defect candidate region. Removing the feature points except defect region and Bag of Features are used to represent the histogram features. Dense SIFT and SVM are used to judge defect or not. CNN is further introduced to classify true or pseudo defect. Classification accuracy was evaluated and effectiveness of the proposed method is shown.

This paper proposes a new approach to archive the robust tracking of moving objects under moving camera environment where the similar moving objects cross each other. Tracking with moving camera sometimes fails to track the object with similar color objects or similar background. Proposed approach is a particle filter based approach. It introduces the likelihood calculated by probabilistic background model which is constructed using dense optical flow and fast density estimation. Proposed approach introduces SVM (Support Vector Machine)1 to judge the scene where it is difficult to construct the probabilistic background model with non-uniform optical flow. This SVM uses the degree histogram of optical flow. Usefulness of proposed approach is evaluated in the experiments using actual video and the performance is compared with recent tracking approaches by quantitative evaluations.

Cytology is one of the decisive factors for the early detection of cancer. It is necessary to have a certain number of years of experience to be able to screen tumor cells for cytodiagnosis. However, this diagnosis has poor objectivity because there are so many parts that the judge’s experience and skill is responsible for. In this paper, we present a new method to extract cell nuclei from HE stained images generally used cell staining for Creation of digitized objective indicators in cytology. Our method extracts cell nuclei using combining the input image and the image previously prepared by SVM. Features used in SVM are automatically generated from CNN. Results are demonstrated by experiments using the real images and its ground truths.

Obtaining polyp size and shape is important for precise diagnosis. The 3-D shape reconstruction technology of computer vision is tried to be used in the medical diagnosis for this reason. Some approaches based on Shape from Shading are proposed for polyp shape recovery from endoscope image. These approaches need some parameters such as surface reflectance coefficient parameter C. It is important to obtain those parameters for accurate polyp shape recovery. This paper proposed a method to extract corresponding blood vessel regions from two endoscope images based on blood vessel structure analysis. Parameters for polyp shape recovery are calculated from geometric constraint equations of the pair of extracted regions.

This paper proposes an approach for a tracking method robust to the intersection with objects with appearances similar to a target object. The proposed method targets image sequences taken by a moving camera and is based on the particle filter. Tracking methods using color information tend to track mistakenly a background region or an object with color similar to the target object. The method constructs the probabilistic background model by the histogram of the optical flow and defines the likelihood function so that the likelihood in the region of the target object may become large. This causes increasing the accuracy of tracking. The probabilistic background model is made by the density forests. It can infer a probabilistic density fast. Results are demonstrated by experiments using the real videos of outdoor scenes.

As a method to recover 3D shape under point light source and perspective projection, a method to recover the depth distribution has been proposed using optimization with both photometric and geometrical constraints which represents the relation between an interesting point and neighboring points under the assumption of Lambertian reflectance. This method assumes one light source at the same positions of viewing point and point light source although actual endoscope has two light sources. This paper proposes a new approach using a photometric constraint equation considering two light sources. The procedures are as follows. First, obtain depth distributions by optimizing photometric constraint under two light sources. Next, obtain the surface normal vector from depth using numerical difference at each point. Then the mapping between the obtained normal vector and true normal vector is learned by Radial Basis Function Neural Network (NN) for a Lambertian sphere and generalized to another target image. Finally, optimize the depth using photometric constraint to recover the final 3D shape. The validity of this method is confirmed in comparison with the previous methods via computer simulation and experiments using actual endoscope images.

An endoscope is a medical instrument that acquires images inside the human body. In this paper, a new 3-D reconstruction approach is proposed to estimate the size and shape of the polyp under conditions of both point light source illumination and perspective projection. Previous approaches could not know the size of polyp without assuming reflectance parameters as known constant. Even if it was possible to estimate the absolute size of polyp, it was assumed that the parameter of camera movement ∆Z is treated as a known along the depth direction. Here two images are used with a medical suture which is known size object to solve this problem and the proposed approach shows the parameter of camera movement can be estimated with robust accuracy with correspondence between two images taken via slight movement of Z. Experiments with endoscope images are demonstrated to evaluate the validity of proposed approach.

In this paper, we propose a new method for object tracking robust to the intersection with other objects with similar appearance and to the great rotation of the camera. The method uses 3D information of the feature points and the camera position by the image-based localization method. The movement information of the camera is used by the prediction process and the calculation process of the likelihood. Furthermore, the method extracts foreground objects by the homography transformation. The result is used by the likelihood function and by the process judging whether the target object is occluded by a background object or not. The proposed method can track the target object robustly when the camera rotates greatly and when the target object is occluded by the background object or the other object. Results are demonstrated by experiments using real videos.

Computer vision methods need to deal with shadows explicitly because shadows often have a negative effect on the results computed. A new shadow detection method is proposed. The proposed method is a shadow model based method. A new feature for detecting shadows is introduced. The feature is obtained by L*a*b* components, Peripheral Increment Sign Correlation and Normalized Vector Distance. These features are robust to illumination changes. Shadows can be treated as local illumination changes. Using these features results in removing shadow effects, in part. The histogram is generated by the three features and is treated as the feature for detecting shadows. The SVM is used for the classifier. The SVM is trained in advance by shadow data and the trained SVM is used for detecting shadows. The proposed method can extract shadows with the accuracy similar to the previous approach in shorter time. Results are demonstrated by experiments using the real videos.

This paper describes the effectiveness comparison of noise removal method using visual, semantic and the both features. To automatically generate image dataset from Web, noise image removal should be conducted. Visual and semantic features are available to detect noise images. However, which type of features, and how to combine the two types of feature are unraveled. In this paper, six types of noise image detection method are prepared: the method using visual feature, the method using semantic feature, two methods using both features in parallel and two methods using both features in serial. Through the comparison experiments, it was confirmed that the method that used both visual and semantic features in parallel focusing on noise images: the method showed 77.5% F-measure values. The image dataset with the method would be applied into image recognition in our future.

In the field of computer vision, “generic object recognition” is one of the most important topics. Generic object recognition needs three types research: feature extraction, pattern recognition, and database preparation. This paper targets at database preparation, and proposes a method to automatically generate hierarchic image database. The proposed method considers both object semantic and visual features in images. In the proposed method, semantic is covered by ontology framework, and visual similarity is covered by images clustering based on Gaussian Mixture Model. The image databases generated by the proposed method covered over 4,800 concepts (where 152 concepts have more than 100 images) and its structure was hierarchic. Through the subjective evaluation experiments, whether images in the database were correctly mapped or not was examined. The results of the evaluation experiments showed over 84% precision in average. It is suggested that the generated image database was sufficiently practicable as learning database for generic object recognition.

Previous approaches have proposed to recover the poly shape but it is desired that absolute size of polyp can be obtained as a medical endoscope system. The VBW (Vogel-Breuss-Weickert) model is proposed as a method to recover 3-D shape under point light source illumination and perspective projection. However, the VBW model recovers relative, not absolute, shape. Here, shape modification is introduced to recover the exact shape. Modification is applied to the output of the VBW model. First, a local brightest point is used to estimate the reflectance parameter from two images obtained with movement of the endoscope camera in depth. After the reflectance parameter is estimated, a sphere image is generated and used for Radial Basis Function Neural Network (RBF-NN) learning. The NN implements the shape modification. NN input is the gradient parameters produced by the VBW model for the generated sphere. NN output is the true gradient parameters for the true values of the generated sphere. Depth can then be recovered using the modified gradient parameters. It was confirmed that NN gives better performance than the linear regression via computer simulation and real experiment.

Tracking methods based on the particle filter uses frequently the appearance information of the target object to calculate the likelihood. The method using it often fails in tracking when the target object intersects with objects similar to the target object. We propose a new approach for tracking an object in a video sequence taken by a moving camera. The proposed method is based on the particle filter. During tracking the target object, the method detects a similar object near the target object by the Mean-Shift tracker. After detecting the object, the size of it is recalculated and the similar object is tracked by the same way with the target object. The positions of the similar objects are used for calculating the likelihood and for judging situation under which the target object exists. These prevent the method from tracking the other object mistakenly. Results are demonstrated by experiments using real video sequences.

An endoscope is a medical instrument that acquires images inside the human body. This paper proposes a new approach for the automatic detection of polyp regions in an endoscope image using a Hessian Filter and machine learning approaches. The approach improves performance of automatic detection of polyp detection with higher accuracy. The approach uses HOG feature as a local feature since the polyp and non-polyp region often have similar color information. The approach also uses Real Adaboost and Random Forests as classifiers which works effciently even when the dimension of feature vector becomes large. It is suggested that Hessian filter can contribute to reducing the computational time in comparison with the case when only HOG features are used to detect the polyp region. K-means++ is introduced to integrate the detection results in the classification. It is shown that polyp detection with high accuracy is performed in the computer experiments with endoscope images.

A new approach to recover 3-D shape from a Scanning Electron Microscope (SEM) image is described. With an ideal SEM image, 3-D shape can be recovered using the Fast Marching Method (FMM) applied to the Eikonal equation. However, when the light source direction is oblique, the correct shape cannot be obtained by the usual one-pass FMM. The new approach modifies the intensities in the original SEM image using an additional SEM image of a sphere and Neural Network (NN) training. Image modification is a two degree-of-freedom (DOF) rotation. No assumption is made about the specific functional form for intensity in an SEM image. The correct 3-D shape can be obtained using the FMM and NN learning, without iteration. The approach is demonstrated through computer simulation and validated through real experiment.

Endoscopy is a very powerful technology to examine the intestinal tract and to detect the presence of any possible abnormalities like polyps, the main cause of cancer. This paper presents an edge based method for polyp detection in endoscopic video images. It utilizes discrete singular convolution (DSC) algorithm for edge detection/segmentation scheme, then by using conic fitting techniques (ellipse and hyperbola) potential candidates are determined. These candidates are first rotated so as to make major axis in the x-axis direction, and then classified as polyp or non-polyp by SVM classifier which is trained separately for ellipse and hyperbola with HOG features.

While web-based learning has been given the opportunity of education to many people, it suffers many problems of its own, such as online teaching methods and materials. One of the most difficult of these problems is the general inability of web-based learning systems to measure (and maintain) levels of comprehension among web-based learners. Although webcam-based measurement of blinking frequency can give us some indication of a subject's concentration level, it does not work so much as an indicator of the level of comprehension has been found. We have shown that pupil diameter is valid as an indicator of the level of comprehension, but from the initial experiments, we showed that the measurement is very difficult. In this study, we investigated the improvement of measurement accuracy and shortening of the time of pupil diameter calculation, polarizing filter showed play an important role.

This paper proposes a new approach to improve the classification accuracy of true defect and pseudo defect of electronic circuit board. The proposed approach introduces the defect detection which corresponds to the color image and concept of random sampling using multiple SVMs. The approach first detects the defect candidate region with high accuracy based on the difference between test image and reference image, then extracts the features to recognize true or pseudo defect. Data and features for multiple subsets based on random sampling and feature selection is applied to find the effective combination of features. Selected combination of features are used for the recognition by each SVM and weighted voting process is applied to determine the final discrimination. Computer experiments were demonstrated and the usefulness of the proposed approach is evaluated with the accuracy of defect classification.

Computer vision methods need to deal with shadows explicitly because shadows often have a negative effect on the results computed. A new shadow detection method is proposed. The new method constructs three shadow models. Three features robust to illumination changes are used to construct the models. The method uses color information, Peripheral Increment Sign Correlation image and edge information. Each of these features removes shadow effects, in part. The overall method can construct an effective shadow model by using all of the features. The result is improved further by region based analysis and by online update of the shadow model. The proposed method extracts shadows accurately. Results are demonstrated by experiments using the real videos of outdoor scenes.

While web-based learning has opened educational opportunities to more people in more situations, it has also introduced a number of problems related to independent, online interaction with learning material. One of the most challenging of these problems is the general inability of web-based learning systems to measure (and maintain) levels of comprehension among web-based learners. Although webcam-based measurement of blinking frequency can give us some indication of a subject's concentration level, it has proven to be an unreliable indicator of a subject's level of comprehension. To supplement such measurement, we herein propose a system that measures proportional changes in pupil diameter, as an improved indicator of comprehension level. Initial experimental suggest that the system can reliably distinguish between subjects who succeeded and failed in solving a mathematical problem.

Not only geometric information but also optical information is needed to reproduce ruins using three-dimensional realistic computer graphics as they were when those were founded. In order to give a model a sense of reality, it is common to carry out the texture mapping of the photographed image. However such information can not be acquired from either weathered or partially destroyed ruins. While there are various conventional techniques for image restoration, which can overcome in the case of small missing and cracks, it is difficult to restore such a heavy damaged mural painting well when there is no information from the periphery. In this paper, we propose an image reproduction of a heavy damaged mural painting using a texture information extracted from another mural painting which has actually been restored by conservators and a traced drawing which the specialist guessed and drew. The restored image was used same pigment inks. Based on texture information from the restored image and a segmented traced drawing, we produce a restored image by applying the texture extension to each segment.

An endoscope is a medical instrument that acquires images inside the human body. This paper proposes a new approach to recover 3-D shape under conditions of both point light source illumination and perspective projection. Previous approaches recovered shape based on a Fast Marching Method under conditions of parallel light source and orthographic projection. The new approach uses optimization based on constraints provided by geometry and by a suitable image irradiance equation. Experiments with endoscope images are demonstrated to confirm that the recovered shape is improved.

Tracking methods based on the particle filter uses frequently the appearance information of the target object to calculate the likelihood. The method using it often fails in tracking when the target object intersects with other objects with similar appearances. We propose a new approach for tracking objects with similar patterns in a video sequence taken by a moving camera. The proposed method based on the particle filter is robust to the intersection with other objects. Two state transition functions are defined for robust tracking. The method changes the function depending on the situation. In addition, the likelihood is calculated by using four factors which are the information of the color, the velocity, the distance between the objects and the values calculated by the probability background model. The method detects objects which are similar to the target object and which exist around the target object. This prevents the method from tracking other object mistakenly. Results are demonstrated by experiments using real video sequences.

In Web study, a learner studies alone toward a personal computer. Therefore, the learner cannot see the appearance of the other learners, and cannot receive the appropriate advice from teachers. Therefore, it is difficult to maintain and to improve the incentive to learn. To give a learner a sense of accomplishment is one of the methods to maintain the incentive to learn. For this, a Web learning system should give the learner a moderately difficult problem (problem let him feel that he can do it if he does his best a little more). To do this, the system should automatically measure the learner’s understanding level accurately. So far, there are a couple of methods to grasp learner’s level, e.g. Item response theory and SP score table analysis, etc. However, a lot of data are necessary for the analysis in such methods. And, it is difficult to deal with a new problem. In psychology, it is said that there is a blink to follow to the change in feelings besides a physiological blink. In this paper, we thought that the feelings change can be known by the frequency change of the blink, and tried to measure the frequency of the blink in real time. Furthermore, it is tried to clarify the relation to the learner’s understanding level by comparing the result of the problem.

Detecting shadows is needed for object detection methods because shadows often have a harmful effect on the result. Shadow detection methods based on shadow models are proposed. The shadow model should be updated to detect shadows which are not included in the learning data. When it is updated, outlier should be removed from them. In this paper, a data selection method for removing outlier from the data is proposed. The proposed method selects data by information of object regions and Normalized Distance. Using only the selected data results in obtaining better shadow regions because the better shadow model can be constructed. Results are demonstrated by the experiments using the real videos.

Data glove is widely used interface device which measures hand posture (finger joint angles) and inputs it into a computer in virtual reality field. But it dose not spread throughout home because of expensive interface. On the other hand, researches to recognize a hand posture from photo/video images are performed, which are a kind of hand posture measurement system and called Vision Based Data Glove (VBDG) in recent years. In this paper, we propose a new VBDG system. It estimates hand motion using detected fingertip positions with monocular camera and inverse kinematics. However it cannot estimate hand motion when a fingertip is undetectable with self-occlusion. So our system estimates hidden finger motion, then estimates hand motion. Our experimental results show that the proposed method can estimate it with sufficient accuracy in real-time. Since camera base system is inexpensive, it fits personal use.

An endoscope is a medical instrument that acquires images inside the human body. An endoscope carries its own light source. Classic shape-from-shading can be used to recover the 3-D shape of objects in view. Recent implementations have used the Fast Marching Method (FMM). Previous FMM approaches recover 3-D shape under assumptions of parallel light source illumination and orthographic projection. This paper extends the FMM approach to recover the 3-D shape under more realistic conditions of endoscopy, namely nearby point light source illumination and perspective projection. The new approach is demonstrated through experiment and is seen to improve performance.

This paper proposes a GPU based algorithm for extracting moving objects in real time. The whole process of the proposed approach is handled on GPU. GPU is used for acceleration and the proposed approach increases processing speed dramatically. The method uses a* component and b* component of CIELAB color space without extracting shadow areas as moving objects. It is robust to intensity changes because an estimated background image is generated and moving objects are extracted using background subtraction of the estimated background image and the observed image. The proposed method reduces the times for transferring calculation results from GPU into CPU and the opposite transfer. Reducing the transfer times contributes to speeding up of the proposed method. Results are demonstrated with experiments on real data.

Shape-from-shading methods recover 3-D shape from intensity images. Often, Lambertian reflectance is assumed. The Lambertian assumption is attractive because it simplifies the analysis. Alternatively, non-Lambertian reflectance, including specularity, is accommodated in methods that measure reflectance empirically either using a separate calibration object or the target object itself, in self-calibration. Here, a new fast marching method (FMM) is described to recover 3-D shape. It assumes a single, directional light source aligned with the viewing direction. Non-Lambertian reflectance is handled via self-calibration based on controlled rotation of the target object. Experiments using both synthetic and real data are demonstrated.

Describes a method to realize the interactive manipulation of a virtual liquid using virtual vessels which are expressed by a general convex-shape polyhedron. We propose a liquid manipulation model which has some functions to treat the relation between the volume of liquid in a vessel and the height level of the liquid surface in it while it is being tilted. For a general-shape vessel, a lookup table is implemented to calculate the above functions. Our system with this proposed model makes it possible to catch the liquid using the virtual vessel, to hold the liquid in it, then to spill the liquid by tilting it. Also, the system realizes a manipulation method to skim the liquid from another liquid vessel.

Describes a method to recover the sign of the local Gaussian curvature at each point on the visible surface of a 3-D object. Multiple (p>3) shaded images are acquired under different conditions of illumination. The required information is extracted from a 2-D subspace obtained by applying principal components analysis (PCA) to the p-dimensional space of normalized irradiance measurements. The number of dimensions is reduced from p to 2 by considering only the first two principal components. The sign of the Gaussian curvature is recovered based on the relative orientation of measurements obtained on a local five point test pattern to those in the 2-D subspace, called the eigen plane. The method does assume generic diffuse reflectance. The method recovers the sign of Gaussian curvature without assumptions about the light source directions or about the specific functional form of the diffuse surface reflectance. Multiple (p>3) light sources minimize the effect of shadows by allowing a larger area of visible surface to be analyzed. Results are demonstrated by experiments on synthetic and real data. The results are more accurate and more robust compared to previous approaches.

Article Exploiting heterogeneous parallelism in the presence of communication delays Share on Authors: Dingchao Li Educational Center for Information Processing, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, Japan Educational Center for Information Processing, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, JapanView Profile , Yuji Iwahori Educational Center for Information Processing, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, Japan Educational Center for Information Processing, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, JapanView Profile , Naohiro Ishii Department of Intelligence and Computer Science, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, Japan Department of Intelligence and Computer Science, Nagoya Institute of Technology, Gokisocho Showaku, Nagoya, 466, JapanView Profile Authors Info & Claims ICS '98: Proceedings of the 12th international conference on SupercomputingJuly 1998 Pages 157–164https://doi.org/10.1145/277830.277862Online:13 July 1998Publication History 0citation846DownloadsMetricsTotal Citations0Total Downloads846Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This paper addresses the following scheduling problem: given a precedence graph with communication costs and a machine architecture with different types of processors, construct a schedule that runs on the given architecture at the minimum possible execution time. The main contributions are: Firstly, we present a static scheduling algorithm that keeps processors idle for future important tasks and fills idle time slots incurred due to interprocessor communication. Secondly, to evaluate the effectiveness of the algorithm, we develop a lower bound on the length of a optimal schedule as a yardstick. Experiments show that this new approach produces better schedules and takes much less compile time.

A photometric method called point source illuminating stereo is proposed for determining the 3D shape of an object from multiple shading images under point light source illumination. When the surface is a perfect diffuser with uniform reflectance, an algorithm for the determination of 3D shape with positions is developed by using the method of least squares and relying on the principle of the monocular vision and the inverse square law for illuminance. In the proposed method, the number of the necessary images is four for the general surface, and can be reduced to three for the continuous surface.>