Self-supervised learning technology has been applied to calculate depth and ego-motion from monocular videos, achieving remarkable performance in various real-world scenarios. Unfortunately, challenges such as specular reflections and soft tissue deformations in endoscopic scenes greatly undermine the performance of these methods, inevitably compromising the accuracy of depth and ego-motion estimation. To address these two problems, we introduce a novel strategy based on image distance transform for robust self-supervised learning for monocular depth estimation, effectively handling specular reflections in endoscopic scenes. Furthermore, we propose a soft tissue deformation constraint based on biomechanical principles, which mitigates the adverse effects of deformed region pixels, ultimately enhancing the model's depth estimation precision. Additionally, our method employs a lightweight architecture ensuring a reduced number of model parameters and faster inference time. Extensive experiments are conducted on both public datasets (SCARED, SERV-CT) and our own datasets to validate the effectiveness of our method. Compared with other SOTA methods, our approach demonstrates comparable accuracy and robustness while ensuring faster inference time. On the SCARED dataset, our approach attains an RMSE of 4.96 mm with only 2.25M model parameters for depth estimation. Especially, experiment results on SERV-CT dataset and our own datasets further demonstrate the model's generalization ability and potential clinical value in computer-assisted surgical navigation.

Complex-Valued Neural Networks are a rising field in research and applications. However, the lack of pretrained complex-valued models complicates their training process, since users can not benefit from the advantages of having large pretrained models publicly available. We present three methods to utilize readily available pretrained real-valued models to initialize complex-valued models. We show that this consistently outperforms training the Complex-Valued Neural Networks from scratch. Additionally, we show that in some scenarios Complex-Valued Neural Networks can outperform their real-valued counterparts, confirming their suitability as an alternative for various applications.

In this work we propose CVKAN, a complex-valued Kolmogorov-Arnold Network (KAN), to join the intrinsic interpretability of KANs and the advantages of Complex-Valued Neural Networks (CVNNs). We show how to transfer a KAN and the necessary associated mechanisms into the complex domain. To confirm that CVKAN meets expectations we conduct experiments on symbolic complex-valued function fitting and physically meaningful formulae as well as on a more realistic dataset from knot theory. Our proposed CVKAN is more stable and performs on par or better than real-valued KANs while requiring less parameters and a shallower network architecture, making it more explainable.

Electroencephalography (EEG)-based applications in Brain-Computer Interfaces (BCIs), neurological disease diagnosis, rehabilitation, and other areas rely on the utilization of extensive data for model development. Nevertheless, this raises concerns regarding storage and privacy, since model development needs a significant amount of data, and EEG sharing discloses sensitive information such as identity and health. To address this challenging problem, we provide the paradigm of EEG condensation, aiming to generate a synthetic sample set that is highly information-concentrated yet not visually similar. Correspondingly, we propose a novel dataset condensation framework where the knowledge of the original EEG dataset is condensed into diffusion models, named EEGCiD. Specifically, EEGCiD first utilizes a deterministic denoising diffusion implicit model (DDIM) to store the information of the original dataset and optimizes the condensation latent codes z to obtain the EEG condensation dataset. Further, to enhance the modeling of EEG knowledge in DDIM, we design a transformer architecture incorporating the spatial and temporal self-attention block (STSA) to replace the traditional U-Net backbone. In the condensation phase, EEGCiD randomly initializes a subset of samples from the original dataset to obtain the condensation latent codes z through the forward process in DDIM. Then, it optimizes z by matching the feature distributions in multiple EEG decoding models between the synthetic samples and the original dataset. Extensive experiments across three EEG datasets demonstrate that the condensation dataset from the proposed model not only achieves superior classification performance with limited sample sizes, but also effectively prevents membership inference attacks (MIA). Note to Practitioners—This paper aims to investigate a novel EEG generation paradigm that extracts representative synthetic samples from large-scale datasets. Existing studies in EEG generation primarily concentrate on generating real-like signals, and some work claims that the generated EEG can serve as a substitute for the original dataset to achieve privacy preservation. In the EEGCiD framework, the deterministic DDIM is pre-trained with the original dataset to store the knowledge. Besides, an ensemble feature matching strategy is proposed to condense the information from the original dataset into a small latent code set. Experiments on three datasets demonstrate that EEGCiD addresses two fundamental challenges: 1) obtaining superior classification performance within a small dataset (limited storage capacity); 2) avoiding potential privacy issues during EEG sharing and transmission.

Maxillary Sinus Lifting is a crucial surgical procedure for addressing insufficient alveolar bone mass andsevere resorption in dental implant therapy. To accurately analyze the geometry changesof the bone graft (BG) in the maxillary sinus (MS), it is essential to perform quantitative analysis. However, automated BG segmentation remains a major challenge due to the complex local appearance, including blurred boundaries, lesion interference, implant and artifact interference, and BG exceeding the MS. Currently, there are few tools available that can efficiently and accurately segment BG from cone beam computed tomography (CBCT) image. In this paper, we propose a distance-constrained attention network guided by prior anatomical knowledge for the automatic segmentation of BG. First, a guidance strategy of preoperative prior anatomical knowledge is added to a deep neural network (DNN), which improves its ability to identify the dividing line between the MS and BG. Next, a coordinate attention gate is proposed, which utilizes the synergy of channel and position attention to highlight salient features from the skip connections. Additionally, the geodesic distance constraint is introduced into the DNN to form multi-task predictions, which reduces the deviation of the segmentation result. In the test experiment, the proposed DNN achieved a Dice similarity coefficient of 85.48 6.38%, an average surface distance error is 0.57 0.34mm, and a 95% Hausdorff distance of 2.64 2.09mm, which is superior to the comparison networks. It markedly improves the segmentation accuracy and efficiency of BG and has potential applications in analyzing its volume change and absorption rate in the future.

Productive Failure is a variant of problem-based learning in which the order of the instruction and problem-solving phase is reversed. The effectiveness of Productive Failure with respect to conceptual knowledge has been demonstrated through a number of studies. The majority of these studies, however, took place in secondary Mathematics classrooms, whereas other studies resulting in less or no support of such an effectiveness were contextualized in other disciplines, including Computer Science, or in tertiary education. This has raised the question of which conditions support or hamper the use of Productive Failure. To deepen our understanding of such conditions, we designed and executed a Productive Failure intervention for a Pattern Recognition course, thus shifting the intervention context into a tertiary setting while maintaining proximity to Mathematics. In an experimental study, we compared the problem-solving progression of students in a Productive Failure setting with the progression of students in a traditional Direct Instruction setting. For this, we analyzed patterns of discourse arising among the participants as well as the longer-term retention of the concepts addressed. The results of our qualitative analysis suggest that, even in a short intervention, Productive Failure can be used to elicit a distinct pattern of progressing though the problem-solving process. At the same time, our study confirmed previous findings that the mode of instruction does not affect exam performance with respect to the specific topics addressed in the intervention. We discuss limitations of the study setting and possible implications for designing future research studies and teaching interventions.

We study the application of large language models to zero-shot and few-shot classification of tabular data. We prompt the large language model with a serialization of the tabular data to a natural-language string, together with a short description of the classification problem. In the few-shot setting, we fine-tune the large language model using some labeled examples. We evaluate several serialization methods including templates, table-to-text models, and large language models. Despite its simplicity, we find that this technique outperforms prior deep-learning-based tabular classification methods on several benchmark datasets. In most cases, even zero-shot classification obtains non-trivial performance, illustrating the method’s ability to exploit prior knowledge encoded in large language models. Unlike many deep learning methods for tabular datasets, this approach is also competitive with strong traditional baselines like gradient-boosted trees, especially in the very-few-shot setting.

Most deep learning pipelines are built on real-valued operations to deal with real-valued inputs such as images, speech or music signals. However, a lot of applications naturally make use of complex-valued signals or images, such as MRI or remote sensing. Additionally the Fourier transform of signals is complex-valued and has numerous applications. We aim to make deep learning directly applicable to these complex-valued signals without using projections into ℝ2. Thus we add to the recent developments of complex-valued neural networks by presenting building blocks to transfer the transformer architecture to the complex domain. We present multiple versions of a complex-valued Scaled Dot-Product Attention mechanism as well as a complex-valued layer normalization. We test on a classification and a sequence generation task on the MusicNet dataset and show improved robustness to overfitting while maintaining on-par performance when compared to the real-valued transformer architecture.

The security of credit card fraud detection (CCFD) models based on machine learning is important but rarely considered in the existing research. To this end, we propose a black-box attack-based security evaluation framework for CCFD models. Under this framework, the semisupervised learning technique and transfer-based black-box attack are combined to construct two versions of a semisupervised transfer black-box attack algorithm. Moreover, we introduce a new nonlinear optimization model to generate the adversarial examples against CCFD models and a security evaluation index to quantitatively evaluate the security of them. Computing experiments on two real data sets demonstrate that, facing the adversarial examples generated by the proposed attack algorithms, all six supervised models considered largely lose their ability to identify the fraudulent transactions, whereas the two unsupervised models are less affected. This indicates that the CCFD models based on supervised machine learning may possess substantial security risks. In addition, the evaluation results for the security of the models generate important managerial implications that help banks reasonably evaluate and enhance the model security. History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Learning. Funding: This work was supported in part by the National Natural Science Foundation of China [Grants 72171160 and 71988101], Key Program of National Natural Science Foundation of China and Quebec Research Foundation (NSFC-FRQ) Joint Project [Grant 7191101304], Key Program of NSFC-FRQSC Joint Project [Grant 72061127002], Excellent Youth Foundation of Sichuan Province [Grant 2020JDJQ0021], and National Leading Talent Cultivation Project of Sichuan University [Grant SKSYL2021-03]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1297 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0076 ) at ( http://dx.doi.org/10.5281/zenodo.7631457 ).

Computing a consensus object from a set of given objects is a core problem in machine learning and pattern recognition. One popular approach is to formulate it as an optimization problem using the generalized median. Previous methods like the Prototype and Distance-Preserving Embedding methods transform objects into a vector space, solve the generalized median problem in this space, and inversely transform back into the original space. Both of these methods have been successfully applied to a wide range of object domains, where the generalized median problem has inherent high computational complexity (typically NP-hard) and therefore approximate solutions are required. Previously, explicit embedding methods were used in the computation, which often do not reflect the spatial relationship between objects exactly. In this work we introduce a kernel-based generalized median framework that is applicable to both positive definite and indefinite kernels. This framework computes the relationship between objects and its generalized median in kernel space, without the need of an explicit embedding. We show that the spatial relationship between objects is more accurately represented in kernel space than in an explicit vector space using easy-to-compute kernels, and demonstrate superior performance of generalized median computation on datasets of three different domains. A software toolbox resulting from our work is made publicly available to encourage other researchers to explore the generalized median computation and applications.

The random walker method for image segmentation is a popular tool for semi-automatic image segmentation, especially in the biomedical field. However, its linear asymptotic run time and memory requirements make application to 3D datasets of increasing sizes impractical. We propose a hierarchical framework that, to the best of our knowledge, is the first attempt to overcome these restrictions for the random walker algorithm and achieves sublinear run time and constant memory complexity. The goal of this framework is- rather than improving the segmentation quality compared to the baseline method- to make interactive segmentation on out-of-core datasets possible. The method is evaluated quantitatively on synthetic data and the CT-ORG dataset where the expected improvements in algorithm run time while maintaining high segmentation quality are confirmed. The incremental (i.e., interaction update) run time is demonstrated to be in seconds on a standard PC even for volumes of hundreds of gigabytes in size. In a small case study the applicability to large real world from current biomedical research is demonstrated. An implementation of the presented method is publicly available in version 5.2 of the widely used volume rendering and processing software Voreen (https://www.uni-muenster.de/Voreen/).

BACKGROUND: Recent advances in 3D imaging technologies provide novel insights to researchers and reveal finer and more detail of examined specimen, especially in the biomedical domain, but also impose huge challenges regarding scalability for automated analysis algorithms due to rapidly increasing dataset sizes. In particular, existing research towards automated vessel network analysis does not always consider memory requirements of proposed algorithms and often generates a large number of spurious branches for structures consisting of many voxels. Additionally, very often these algorithms have further restrictions such as the limitation to tree topologies or relying on the properties of specific image modalities. RESULTS: We propose a scalable iterative pipeline (in terms of computational cost, required main memory and robustness) that extracts an annotated abstract graph representation from the foreground segmentation of vessel networks of arbitrary topology and vessel shape. The novel iterative refinement process is controlled by a single, dimensionless, a-priori determinable parameter. CONCLUSIONS: We are able to, for the first time, analyze the topology of volumes of roughly 1 TB on commodity hardware, using the proposed pipeline. We demonstrate improved robustness in terms of surface noise, vessel shape deviation and anisotropic resolution compared to the state of the art. An implementation of the presented pipeline is publicly available in version 5.1 of the volume rendering and processing engine Voreen.

Drosophila melanogaster is an important model organism for research in neuro- and behavioral biology. Automated studies of their locomotion are crucial to link sensory input and neural processing to motor output which has led to numerous vision-based tracking systems. However, most of these approaches share the inability to segment the contours of colliding animals causing identity losses, appearing and disappearing animals, and the absence of posture and motion related measurements during the time of the collision. We present a novel collision resolution algorithm enabling an accurate contour segmentation of multiple touching Drosophila larvae. Our algorithm utilizes an adapted active shape model (ASM) to learn a low dimensional posture space which is fitted to random-walker generated pre-segmentations. We evaluate our collision resolution algorithm using three publicly available datasets and compare it with the current state-of-the-art methods. In addition, we introduce a refined dataset enabling a segmentation evaluation on the level of pixel accuracy. The results demonstrate that our approach outperforms the state-of-the-art approaches in both accuracy and computational time. We will incorporate this algorithm into our widely used tracking program to improve the statistical strength of the behavioral quantification and allow marker-free studies of interacting Drosophila larvae.

Learning a prototype from a set of given objects is a core problem in machine learning and pattern recognition. A commonly used approach for consensus learning is to formulate it as an optimization problem in terms of generalized median computation. Recently, a prototype-embedding approach has been proposed to transform the objects into a vector space, compute the geometric median, and then inversely transform back into the original space. This vector space embedding approach has been successfully applied in several domains, where the generalized median problem has inherent high-computational complexity (typically NP-hard) and thus approximate solutions are required. Generally, it can be expected that the embedding should be done in a distance-preserving manner. However, the previous work based on the prototype-embedding approach did not take this embedding aspect into account. In this paper, we discuss the drawbacks of the current prototype-embedding approach and present an extensive empirical study that provides strong evidence of significantly improved quality of generalized median computation using distance-preserving embedding (DPE) methods. We also give concrete suggestions about suitable DPE methods. Moreover, we show that this framework can be used to effectively compute other consensus objects like the closest string. Finally, a MATLAB toolbox resulting from this paper is made publically available in order to encourage other researchers to explore the embedding-based consensus computation.

Document image binarization algorithms are usually evaluated by a pixelwise comparison. Such metrics can be misleading and do not assess the overall structure of the text in the image, thus they do not measure the character recognition capability of the binarized image. In this paper we propose the use of metrics, based on skeleton comparisons, to evaluate structural consistency of the strokes that better correspond to character readability in the binarized image. This approach divides the skeleton of two binary images to be compared (e.g. binarization result and ground truth) in small segments and measures curve similarity between those segments. We conducted experiments with manually generated data with small distortions in the image, which greatly affect common pixelwise metrics but do not hinder the readability of the text. We also binarized images of well-known document binarization datasets using classical and state-of-the-art algorithms such as Otsu's and deep learning methods. On both manually generated and real data it could be demonstrated, amongst others, that the skeletal similarity metrics are more consistent than the pixelwise comparison regarding small character distortions and better capture the readability of the binarized image. Skeletal similarity metrics can be used to complement the pixelwise comparison towards multifaceted performance evaluation for document binarization.

The traditional supervised classification algorithms tend to focus on uncovering the relationship between sample attributes and the class labels; they seldom consider the potential structural characteristics of the sample space, often leading to unsatisfactory classification results. To improve the performance of classification models, many scholars have sought to construct hybrid models by combining both supervised and unsupervised learning. Although the existing hybrid models have shown significant potential in industrial applications, our experiments indicate that some shortcomings remain. With the aim of overcoming such shortcomings of the existing hybrid models, this article proposes a hybrid classification framework based on clustering (HCFC). First, it applies a clustering algorithm to partition the training samples intoKclusters. It then constructs a clustering-based attribute selection measure—namely, the hybrid information gain ratio, based upon which it then trains a C4.5 decision tree. Depending on the differences in the clustering algorithms used, this article constructs two different versions of the HCFC (HCFC-K and HCFC-D) and tests them on eight benchmark datasets in the healthcare and disease diagnosis industries and on 15 datasets from other fields. The results indicate that both versions of the HCFC achieve a comparable or even better classification performance than the other three hybrid and six single models considered. In addition, the HCFC-D has a stronger ability to resist class noise compared with the HCFC-K.

Recently, applications of complex-valued neural networks (CVNNs) to real-valued classification problems have attracted significant attention. However, most existing CVNNs are black-box models with poor explanation performance. This study extends the real-valued group method of data handling (RGMDH)-type neural network to the complex field and constructs a circular complex-valued group method of data handling (C-CGMDH)-type neural network, which is a white-box model. First, a complex least squares method is proposed for parameter estimation. Second, a new complex-valued symmetric regularity criterion is constructed with a logarithmic function to represent explicitly the magnitude and phase of the actual and predicted complex output to evaluate and select the middle candidate models. Furthermore, the property of this new complex-valued external criterion is proven to be similar to that of the real external criterion. Before training this model, a circular transformation is used to transform the real-valued input features to the complex field. Twenty-five real-valued classification data sets from the UCI Machine Learning Repository are used to conduct the experiments. The results show that both RGMDH and C-CGMDH models can select the most important features from the complete feature space through a self-organizing modeling process. Compared with RGMDH, the C-CGMDH model converges faster and selects fewer features. Furthermore, its classification performance is statistically significantly better than the benchmark complex-valued and real-valued models. Regarding time complexity, the C-CGMDH model is comparable with other models in dealing with the data sets that have few features. Finally, we demonstrate that the GMDH-type neural network can be interpretable.

Drosophila melanogaster is an important model organism for ongoing research in neuro- and behavioral biology. Especially the locomotion analysis has become an integral part of such studies and thus elaborated automated tracking systems have been proposed in the past. However, most of these approaches share the inability to precisely segment the contours of colliding animals leading to the absence of model and motion-related features during collisions. Here, we translate the task of tracking and resolving colliding animals into a filtering problem solvable by Markov Chain Monte Carlo methods and elaborate an adequate larva model. By comparing our method with state-of-the-art approaches, we demonstrate that our algorithm produces significantly better results in a fraction of time and facilitates the analysis of animal behavior during interaction in more detail.

Computer-Aided Diagnosis (CAD) represents a relevant instrument to automatically classify between patients with and without Alzheimer's Disease (AD) using several actual imaging techniques. This study analyzes the optimization of volumes of interest (VOIs) to extract three-dimensional (3D) textures from Magnetic Resonance Image (MRI) in order to diagnose AD, Mild Cognitive Impairment converter (MCIc), Mild Cognitive Impairment nonconverter (MCInc) and Normal subjects. A relevant feature of the proposed approach is the use of 3D features instead of traditional two-dimensional (2D) features, by using 3D discrete wavelet transform (3D-DWT) approach for performing feature extraction from T-1 weighted MRI. Due to the high number of coefficients when applying 3D-DWT to each of the VOIs, a feature selection algorithm based on mutual information is used, as is the minimum Redundancy Maximum Relevance (mRMR) algorithm. Region optimization has been performed in order to discover the most relevant regions (VOIs) in the brain with the use of Multi-Objective Genetic Algorithms, being one of the objectives to be optimize the accuracy of the system. The error index of the system is computed by the confusion matrix obtained by the multi-class support vector machine (SVM) classifier. Principal Component Analysis (PCA) is used with the purpose of reducing the number of features to the classifier. The cohort of subjects used in the study consisted of 296 different patients. A first group of 206 patients was used to optimize VOI selection and another group of 90 independent subjects (that did not belong to the first group) was used to test the solutions yielded by the genetic algorithm. The proposed methodology obtains excellent results in multi-class classification achieving accuracies of 94.4% and also extracting significant information on the location of the most relevant points of the brain. This suggests that the proposed method could aid in the research of other neurodegenerative diseases, improving the accuracy of the diagnosis and finding the most relevant regions of the brain associated with them.

Imaging and analyzing the locomotion behavior of small animals such as Drosophila larvae or C. elegans worms has become an integral subject of biological research. In the past we have introduced FIM, a novel imaging system feasible to extract high contrast images. This system in combination with the associated tracking software FIMTrack is already used by many groups all over the world. However, so far there has not been an in-depth discussion of the technical aspects. Here we elaborate on the implementation details of FIMTrack and give an in-depth explanation of the used algorithms. Among others, the software offers several tracking strategies to cover a wide range of different model organisms, locomotion types, and camera properties. Furthermore, the software facilitates stimuli-based analysis in combination with built-in manual tracking and correction functionalities. All features are integrated in an easy-to-use graphical user interface. To demonstrate the potential of FIMTrack we provide an evaluation of its accuracy using manually labeled data. The source code is available under the GNU GPLv3 at https://github.com/i-git/FIMTrack and pre-compiled binaries for Windows and Mac are available at http://fim.uni-muenster.de.

The closest string problem is a core problem in computational biology with applications in other fields like coding theory. Many algorithms exist to solve this problem, but due to its inherent high computational complexity (typically NP-hard), it can only be solved efficiently by restricting the search space to a specific range of parameters. Often, the run-time of these algorithms is exponential in the maximum distance between strings, restricting these solutions to very small distances. Recently, a prototype embedding method has been proposed to solve the similar generalized median problem for arbitrary objects. In this approach, objects are transformed into vector space using prototype embedding. The problem is solved in vector space and afterwards inversely transformed back into original space. This method has been successfully applied to generalized median computation in several domains where the computational complexity is inherently high. In this work, we apply prototype embedding to the closest string problem. We show that different embedding methods can result in a very good and fast approximation of the closest string, independent of the maximum distance and other parameters.

Cancer is one of the widest spread diseases in human society. Therefore, the need has grown to monitor, evaluate, and predict its development. Cancer registries address this problem by collecting data on cancer cases, striving for high quality, accuracy, and completeness. One of the basic challenges in this context is the linkage of data from multiple sources. In order to link new cancer records with existing ones, the cancer registries typically use an algorithm referred to as record linkage. Although the algorithm has automated a significant amount of the linking process, there still is a certain percentage of records that cannot be linked automatically. This study addresses the problem of reducing the need of manually matching records with machine learning methods. The particular challenge is caused by pseudonymization of the data. The main contribution is thus finding ways to encode the-pseudonymized-data, i.e., feature extraction so that it can be interpreted by a classifier. Three classifiers (neural network, support vector machines, decision tree) manage to achieve at least 93% classification rate on a dataset of 73 000 cancer records extracted from the inventory of a cancer registry. In addition, ensemble techniques boost the performance further to over 95%. We present an in-depth discussion of the experimental results from a perspective of applying the classification-based record linkage in real practice. Two scenarios of translating to practice will be outlined with a potential of reducing the human workload by an order of magnitude of hundreds of hours.

Clustering analysis is a powerful tool in customer segmentation. Although various algorithms have been proposed, the determination of the optimal number of clusters remains to be a difficult issue. In this paper, a clustering method based on consistency criterion is proposed to address this issue. The main characteristic of the new approach is that it requires little prior information and can find the optimal number of clusters automatically. Extensive comparisons are done over 22 real-world datasets from different domains, in which four well-known clustering algorithms in combination with six clustering indices are used as the benchmark methods. The results demonstrate the superiority of our method in appropriately determining the number of clusters. An application of the new approach in customer segmentation of credit card users is also illustrated.

Disocclusion filling is a critical problem in depth- based view synthesis. Exposed regions in the target view that correspond to occluded areas in the reference view have to be filled in a meaningful way. Current approaches aim to do this in a plausible way, mostly inspired by image inpainting techniques . However, disocclusion filling is a video-based problem which exhibits more information than just the current frame. By utilizing texture found in temporally adjacent frames, we propose to fill disocclusions in a faithful way, i.e., using texture that a real camera would observe in place of the virtual camera. Only if faithful information is not available we fall back to plausible filling. Our approach is designed for single view video-plus-depth where neighboring camera views are not available for disocclusion filling. In contrast to previous approaches , our method uses superpixels instead of square patches as filling entities to reduce the amount of artifacts introduced into the filling region. Despite its importance , faithfulness has not obtained the due attention yet. Our experiments show that situations are common where a simple plausible filling does not lead to satisfying filling results. Thus, it is important to stress faithful disocclusion filling. Our current work is an attempt in this direction.

We present a new super pixel-based spatio-temporal disocclusion filling algorithm for view synthesis in Depth Image Based Rendering. When rendering new views from a single video-plus-depth sequence, background regions get exposed in the virtual view that are occluded in the video stream. To fill these regions we propose to use texture information that is found both in the current and in temporally neighboring frames. Instead of using a patch-based search for texture to fill the holes, we propose a new super pixel-based filling method that exploits the property of super pixels being more meaningful sub-entities of an image than simple square patches. We present experiments with both real and synthetic sequences to evaluate our proposed method.

The relatively poor spatial resolution is one of the limitations for accurate disease diagnosis in Positron Emission Tomography (PET) imaging. This paper presents an effective super-resolution (SR) approach to gated cardiac PET based on a mass-preserving motion estimation, which enables to deal with the intensity modulation caused by cardiac motion and partial volume effects. A performance evaluation is conducted to demonstrate the performance of the proposed algorithm, in particular the benefit of mass-preserving based motion estimation for SR computation in gated cardiac PET.

In this paper, we address the disocclusion problem that occurs during view synthesis in depth image-based rendering (DIBR). We propose a method that can recover faithful texture information for disoccluded areas. In contrast to common disocclusion filling methods, which usually work frame-by-frame, our algorithm can take information from temporally neighboring frames into account. This way, we are able to reconstruct a faithful filling for the disocclusion regions and not just an approximate or plausible one. Our method avoids artifacts that occur with common approaches and can additionally reduce compression artifacts at object boundaries.

The region-based segmentation paradigm is a well known technique for image segmentation. In the first part of this work the robustness of region-based algorithms is studied. It is shown that within a small parameter range, which leads to good segmentation results in the majority of cases, bad segmentation results may occur. In fact, such local instability is a problem of region-based methods and reasons for its occurrence are discussed. In the second part of the work, an ensemble solution for this problem based on the median concept is proposed. Two variants, set median and generalized median, are presented and experimentally compared. Extensive experimental results demonstrate the potential of the proposed median approach for solving the instability problem.

Respiratory and cardiac motion leads to image degradation in positron emission tomography (PET) studies of the human heart. In this paper we present a novel approach to motion correction based on dual gating and mass-preserving hyperelastic image registration. Thereby, we account for intensity modulations caused by the highly nonrigid cardiac motion. This leads to accurate and realistic motion estimates which are quantitatively validated on software phantom data and carried over to clinically relevant data using a hardware phantom. For patient data, the proposed method is first evaluated in a high statistic (20 min scans) dual gating study of 21 patients. It is shown that the proposed approach properly corrects PET images for dual-cardiac as well as respiratory-motion. In a second study the list mode data of the same patients is cropped to a scan time reasonable for clinical practice (3 min). This low statistic study not only shows the clinical applicability of our method but also demonstrates its robustness against noise obtained by hyperelastic regularization.

A mobile computer vision system is presented that helps visually impaired pedestrians cross roads. The system detects pedestrian lights in the environment and gives feedback about the current phase of the crucial light. For this purpose the live video stream of a mobile phone is analyzed in four steps: localization, classification, video analysis, and time-based verification. In particular, the temporal analysis allows us to alleviate the inherent problems such as occlusions (by vehicles), falsified colors, and others, and to further increase the decision certainty over a period of time. Due to the limited resources of mobile devices very efficient and precise algorithms have to be developed to ensure the reliability and the interactivity of the system. A prototype system was implemented on a Nokia N95 mobile phone and tested in real environment. It was trained to detect German traffic lights. For the prototype training and testing, we generated image and video databases including manually specified ground truth meta-data. These databases described in this paper are publicly available for the research community. Quantitative performance analysis is provided to demonstrate the reliability and interactivity of the prototype system.

In this paper we present the adaptation of a random walker algorithm for combination of image segmentations to work with clustering problems. In order to achieve it, we pre-process the ensemble of clusterings to generate its graph representation. We show experimentally that a very small neighborhood will produce similar results if compared with larger choices. This fact alone improves the computational time needed to produce the final consensual clustering. We also present an experimental comparison between our results against other graph based and well known combination clustering methods in order to assess the quality of this approach.

BACKGROUND: Visualization of DNA microarray data in two or three dimensional spaces is an important exploratory analysis step in order to detect quality issues or to generate new hypotheses. Principal Component Analysis (PCA) is a widely used linear method to define the mapping between the high-dimensional data and its low-dimensional representation. During the last decade, many new nonlinear methods for dimension reduction have been proposed, but it is still unclear how well these methods capture the underlying structure of microarray gene expression data. In this study, we assessed the performance of the PCA approach and of six nonlinear dimension reduction methods, namely Kernel PCA, Locally Linear Embedding, Isomap, Diffusion Maps, Laplacian Eigenmaps and Maximum Variance Unfolding, in terms of visualization of microarray data. RESULTS: A systematic benchmark, consisting of Support Vector Machine classification, cluster validation and noise evaluations was applied to ten microarray and several simulated datasets. Significant differences between PCA and most of the nonlinear methods were observed in two and three dimensional target spaces. With an increasing number of dimensions and an increasing number of differentially expressed genes, all methods showed similar performance. PCA and Diffusion Maps responded less sensitive to noise than the other nonlinear methods. CONCLUSIONS: Locally Linear Embedding and Isomap showed a superior performance on all datasets. In very low-dimensional representations and with few differentially expressed genes, these two methods preserve more of the underlying structure of the data than PCA, and thus are favorable alternatives for the visualization of microarray data.

In this work we present a novel method to model instance-level constraints within a clustering algorithm. Thereby, both similarity and dissimilarity constraints can be used coevally. The proposed extension is based on a distance transformation by shortest path computations in a constraint graph. With a new technique cannot-links are consistently supported and the dissimilarity is extended to their neighbourhoods. We quantitatively compare the results achieved by our COPGB-K-Means algorithm with the state-of-the-art algorithms on standard databases and show that qualitatively good results and a fast realisation are not mutually exclusive.

In this work we present an approach to generating depth image sequences for standard videos, which satisfy a proposed motion model. We take advantage that the background in a video scene is relatively fix in contrast to moving objects in the foreground. By robust methods the camera motion is eliminated automatically to extract a background mosaic. By means of this mosaic the moving objects are extracted and the depth is assigned by the user. We apply the DIBR approach on the depth information to render 3D videos. The results demonstrate the practicability of our approach and highlight the advantages of the proposed motion model against previous depth tracking methods.

In this paper we present an algorithm for the recognition of 1D barcodes using camera phones, which is highly robust regarding the the typical image distortions. We have created a database of barcode images, which covers typical distortions, such as inhomogeneous illumination, reflections, or blurriness due to camera movement. We present results from experiments with over 1,000 images from this database using a Matlab implementation of our algorithm, as well as experiments on the go, where a Symbian C++ implementation running on a camera phone is used to recognize barcodes in daily life situations. The proposed algorithm shows a close to 100% accuracy in real life situations and yields a very good resolution dependent performance on our database, ranging from 90.5% (640 × 480) up to 99.2% (2592 × 1944). The database is freely available for other researchers.

In this paper we propose an algorithm for combining multiple image segmentations to achieve a final improved segmentation. In contrast to previous works we consider the most general class of segmentation combination, i.e. each input segmentation has an arbitrary number of regions. Our approach is based on a random walker segmentation algorithm which is able to provide high-quality segmentation starting from manually specified seeds. We automatically generate such seeds from an input segmentation ensemble. An information-theoretic optimality criterion is proposed to automatically determine the final number of regions. The experimental results on 300 images with manual ground truth segmentation clearly show the effectiveness of our combination approach.

Ventilator weaning is the process of discontinuing mechanical ventilation from patients with respiratory failure. Ventilator support should be withdrawn as soon as possible when it is no longer necessary in order to reduce the likelihood of known nosocomial complications and costs. Previous investigation indicated that clinicians were often wrong when predicting weaning outcome. The motivation of this study is that although successful ventilator weaning of ICU patients has been widely studied, indicators for accurate prediction are still under investigation. The goal of this study is to find a prediction model for successful ventilator weaning using variables such physiological variables, clinical syndromes, demographic variables, and other useful information. The data obtained from 231 patients who had been supported by mechanical ventilator for longer than 21 days within the period from Nov. 2002 to Dec. 2005 were studied retrospectively. Among them, 188 patients were recruited from the period within Nov. 2002 to Dec. 2004 and the other 43 patients from Jan. 2004 to Dec. 2005. All the patients were clinically stable before being considered to undergo a weaning trial. Twenty-seven variables in total were collected with only 6 variables reaching significant level (p<0.05) were used for support vector machine (SVM) classification after statistical analysis. The results show that the constructed model is valuable in assisting clinical doctors to decide if a patient is ready to wean from the ventilator with the sensitivity, specificity, and accuracy as high as 94.74%, 95.83%, and 95.35%, respectively. Further prospective bed side test is needed to verify the efficacy of the model.

We propose a novel dual dynamic programming (DDP) technique for detecting intimal and adventitial layers of the common carotid artery of the B-mode sonographic images. This method embeds the anatomic knowledge into its structure so that the robustness against the speckles is increased. Moreover, it inherits the property of getting the optimal solution as the traditional dynamic programming (TDP). Our experimental study shows that the DDP technique achieves a detection performance comparable to manual tracing achieved by physicians. The results demonstrate that it has the potential to perform qualitatively better than applying TDP twice in intimal and adventitial layer detection on sonographic B-mode images.

The problem of motion is well known in positron emission tomography (PET) studies. The PET images are formed over an elongated period of time. As the patients cannot hold breath during the PET acquisition, spatial blurring and motion artifacts are the natural result. These may lead to wrong quantification of the radioactive uptake. We present a solution to this problem by respiratory-gating the PET data and correcting the PET images for motion with optical flow algorithms. The algorithm is based on the combined local and global optical flow algorithm with modifications to allow for discontinuity preservation across organ boundaries and for application to 3-D volume sets. The superiority of the algorithm over previous work is demonstrated on software phantom and real patient data.

The lower the resolution of a given text is, the more dif- ficult it becomes to segment it into single characters. The resolution of screen-rendered text can be very low. This pa- per focuses on smoothed screen-rendered text of very low resolution with typical x-heights of 4 to 7 pixels which is much lower than in other low resolution OCR situations. We propose a recognition-based segmentation algorithm which makes use of oversegmentation by dynamic programming, candidate rating by single character classifiers and a graph based search algorithm for an optimal cut sequence. The al- gorithm is described in detail and experimental results are presented which show the performance on example screen- shot images taken from the public Screen-Word database.

The recognition of screen-rendered text is a novel task. It is performed e.g. by translation tools which allow users to click on any text on the screen and give a translation. Also some commercial OCR programs start to address the problem of reading screenshots. Optical character recognition on screen-shot images can be very challenging due to very small and smoothed fonts. In order to build and compare recognition approaches for screen-rendered text, the availability of standard databases is a fundamental prerequisite. In this paper two freely available databases are presented, one that consists of annotated screenshot images of 28080 single characters and another holding 400 words extracted from documents plus 2 400 generated isolated words. Both databases include meta-information such as x-height, font type, style and rendering conditions. At the example of a developed recognition system, it is shown how these databases can serve for training, testing and optimization.

This paper addresses challenges connected to the integration of small-size cameras into everyday objects and their impact on human computer interaction (HCI) paradigms. Along the lines of Weiser's vision of ubiquitous computing more and more everyday objects are equipped with sophisticated sensors, including accelerometers, temperature and pressure sensors as well as cameras. Many off-the-shelf mobile phones are equipped with a mega-pixel camera today. In the future it is likely that other objects of daily live use will come with integrated cameras as well, such as cups, books, or doors. In this paper we will briefly review some state-of-the-art HCI-related projects relying on embedded cameras and highlight related challenges in the context of vision algorithms and technologies.

The fundamental assumption of 3D videos using depth-image-based rendering is the full availability of range images at video rate. In this work we alleviate this hard demand and assume that only limited resources of range images are available, i.e. corresponding range images exist for some, but not all, color images of the monoscopic video stream. We propose to synthesize the missing range images between two consecutive range images. Experiments on real videos have demonstrated very encouraging results. Especially, one 3D video was generated from a 2D video without any sensory 3D data available at all. In a quality evaluation using an autostereoscopic 3D display the test viewers have attested similar 3D video quality for our synthesis technique and rendering based on depth ground truth

Autostereoscopic displays are an emerging technology which provide 3D viewing experiences without the need of glasses or other encumbering viewing aids. In this paper we explore their potential in ophthalmologic studies. While the vast majority of applications of autostereoscopic displays in other fields is merely based on producing 3D viewing effects, we can distinguish between several classes of ophthalmologic tasks in which autostereoscopic displays play a very different role. Three concrete applications are described. With the steady improvements in autostereoscopic displays we expect to develop qualitatively new ophthalmologic tests in future

Motion is a source of degradation in positron emission tomography (PET)/computed tomography (CT) images. As the PET images represent the sum of information over the whole respiratory cycle, attenuation correction with the help of CT images may lead to false staging or quantification of the radioactive uptake especially in the case of small tumors. We present an approach avoiding these difficulties by respiratory-gating the PET data and correcting it for motion with optical flow algorithms. The resulting dataset contains all the PET information and minimal motion and, thus, allows more accurate attenuation correction and quantification.

In this paper, we propose a general framework of adaptive local thresholding based on a verification-based multithreshold probing scheme. Object hypotheses are generated by binarization using hypothetic thresholds and accepted/rejected by a verification procedure. The application-dependent verification procedure can be designed to fully utilize all relevant informations about the objects of interest. In this sense, our approach is regarded as knowledge-guided adaptive thresholding, in contrast to most algorithms known from the literature. We apply our general framework to detect vessels in retinal images. An experimental evaluation demonstrates superior performance over global thresholding and a vessel detection method recently reported in the literature. Due to its simplicity and general nature, our novel approach is expected to be applicable to a variety of other applications.

In this paper, we describe a systematic approach to the lipreading of whole sentences. A vocabulary of elementary words is considered. Based on the vocabulary, we define a grammar that generates a set of legal sentences. Our lipreading approach is based on a combination of the grammar with hidden Markov models (HMMs). Two different experiments were conducted. In the first experiment a set of e-mail commands is considered, while the set of sentences in the second experiment is given by all English integer numbers up to one million. Both experiments showed promising results, regarding the difficulty of the considered task.

In object prototype learning and similar tasks, median computation is an important technique for capturing the essential information of a given set of patterns. We extend the median concept to the domain of graphs. In terms of graph distance, we introduce the novel concepts of set median and generalized median of a set of graphs. We study properties of both types of median graphs. For the more complex task of computing generalized median graphs, a genetic search algorithm is developed. Experiments conducted on randomly generated graphs demonstrate the advantage of generalized median graphs compared to set median graphs and the ability of our genetic algorithm to find approximate generalized median graphs in reasonable time. Application examples with both synthetic and nonsynthetic data are shown to illustrate the practical usefulness of the concept of median graphs.

A range image segmentation contest was organized in conjunction with ICPR'2000. The goal is to continue the effort of experimentally evaluating range image segmentation algorithms initiated by Hoover et al. (1996) and Powell et al. (1998). This paper summarizes the results of the contest.

Acoustic and visual signals carry complementary information and a combination of both information sources therefore possesses the potential of increasing the performance of speech recognition, particularly in noisy environments. In this paper we consider such a combination. Earlier works on the combination of visual and acoustic classifiers for speech recognition typically deal with small vocabularies and use simple combination rules such as majority vote and Borda count. The large number of spoken sentences, however, necessitates a conceptually new approach to classifier combination which explores the syntactic structural of a sentence. In this paper we present such a structure combination strategy and show results for the task of e-mail command recognition.

The potential of edge-based complete image segmentation into regions has not gained the due attention in the literature thus far. The present paper attempts to explore this potential by proposing an adaptive grouping algorithm to solve the contour closure problem that is the key to a successful edge-based complete image segmentation. The effectiveness of the proposed algorithm is extensively tested in the range image domain and compared to several region-based segmentation methods within a rigorous comparison framework. On three range image databases of varying quality acquired by different range scanners, it is shown that the proposed approach is able to achieve very appealing performance with respect to both segmentation quality and computation time.

Describes an algorithm to estimate the skew angle of document images. It utilizes the nearest-neighbor clustering paradigm. In contrast to earlier approaches, the local clustering process is focused on a subset of plausible neighbors. The proposed skew detection algorithm is potentially usable for any feature points that reveal the dominant orientation of document images in their entirety. Experimental results using connected components and pass codes as features are presented to show the general usefulness of the proposed algorithm.

This work focuses on creating a framework for objectively evaluating the performance of range image segmentation algorithms. The algorithms are evaluated in terms of correct segmentation, over- and under-segmentation, missed and noise regions. A set of images with ground truth was created for this work. The images were captured using a structured light scanner. Images used in the evaluation contain planar, spherical, cylindrical, toroidal and conical surface patches. The different surface patches in each image were manually identified to establish ground truth for performance evaluation. Two segmentation algorithms from the literature are compared.

In this paper we introduce six measures for detecting the presence of glasses. We also investigate combinations of these measures. Results are given for two facial image sets.

In this paper we describe a contour closure algorithm to derive a complete region segmentation of range images from an edge detection. The effectiveness of our approach is demonstrated by good results on real range images.

In this paper we present a robust approach to facial profile recognition. The high robustness results from a localization method that automatically locates the facial profile from the full contour of a person's head, and a facial profile matching method that is upgraded by a procedure for tuning facial profile normalization parameters. A model preselection method is introduced to exclude a large part of the model database from the actual matching. The facial profile recognition system has been implemented and achieved good results.

In this paper we present a novel algorithm for very fast segmentation of range images into both planar and curved surface patches. In contrast to other known segmentation methods our approach makes use of high-level features (curve segments) as segmentation primitives instead of individual pixels. This way the amount of data can be significantly reduced and a very fast segmentation algorithm is obtained. The proposed algorithm has been tested on a large number of real range images and demonstrated good results. With an optimized implementation our method has the potential to operate in quasi real-time (a few range images per second).

A methodology for evaluating range image segmentation algorithms is proposed. This methodology involves (1) a common set of 40 laser range finder images and 40 structured light scanner images that have manually specified ground truth and (2) a set of defined performance metrics for instances of correctly segmented, missed, and noise regions, over- and under-segmentation, and accuracy of the recovered geometry. A tool is used to objectively compare a machine generated segmentation against the specified ground truth. Four research groups have contributed to evaluate their own algorithm for segmenting a range image into planar patches.

We propose a scale-invariant polyhedral object recognition algorithm that is based on the pose clustering paradigm using fragmentary edge segments. Two novel feature-focus techniques are introduced to reduce the computational complexity for matching a scene with n edge fragments and a model with m edges from Q(m/sup 2/n/sup 2/) to O(mn) without loss of matching quality. In addition, we suggest a mixed data structure that requires only a three-dimensional accumulation array. The proposed recognition method has been successfully tested on real range data.

A model-based system for the recognition of 3-D overlapping convex objects with planar and curved surfaces from the needle map of a scene is presented. The recognition is based on tree search and EGI (extended Gaussian image) matching. A set of constraints is proposed to effectively limit search space, and several heuristics are introduced to enhance the tree search and EGI matching. Moreover, all information needed for the recognition method is automatically generated from CAD models. >