It is widely known that adversarial examples cause misclassification in classifiers trained by deep learning. In spite of numerous previous researches, the phenomenon has not been well understood and clearly explained, hence there is no effective countermeasures available now. In this research, instead of exploring on neural networks and their training algorithms, we focus on the training data of the classifier, as submanifolds embedded in Euclidean space. We show that occurrence of adversarial examples is due to the embedding structure of the data manifold in the data space, specifically existence of so-called co-dimension of the data manifold. In particular, they lie in the orthogonal complementary subspaces of the tangent spaces of the data manifold. The theory is then verified on adversarial examples by visualization in 3D space and analysis in high dimensional space.

We describe a framework which can be used to investigate the geometrical structure of datasets in high-dimensional spaces. Understanding these geometrical properties is essential in machine learning in general. An application which has received much attention recently is the investigation of adversarial examples which can be easily identified by humans but which are misleading neural networks. We will argue that these examples can be understood (and consequently avoided) by an investigation of the dimension and geometry of the classification hypersurface of the underlying neural network as a manifold using the tools introduced in this paper.We argue that a key concept in the analysis is the dimension of a manifold. Many machine learning methods use dimension reduction techniques to understand or classify the input data. We point out different definitions of a dimension of a manifold. We use tools from the theory of topological manifolds to introduce local, or intrinsic and global dimensions. The local dimension describes the local similarity between the dataset and a Euclidean space. The global dimension is the lowest dimension of a Euclidean space into which the dataset can be embedded. For a data space with a Riemannian structure the isometric embedding dimension is the lowest dimensional space into which the dataset can be embedded under the condition that the metric tensor is preserved between the two spaces.We introduce a framework for dimension estimation and topological manifold learning based on the measure ratio method to estimate the dimensions and structure of the data manifold. As an illustration we use images of handwritten digits and points of a Klein-bottle embedded in a five-dimensional space. We compare the results obtained by the measure ratio method with the well-known local-principal component analysis estimation.

Dimension reduction is one of the most important issues in machine learning and computational intelligence. Typical data sets are point clouds in a high dimensional space with a hidden structure to be found in low dimensional submanifolds. Finding this intrinsic manifold structure is very important in the understanding of the data and for reducing computational complexity. In this paper, we propose a novel approach for dimension estimation of topological manifolds based on measures of simplices. We also investigate the effects of resolution changes for dimension estimation in the framework of Morse theory. The result is a method that can be used for data located in simplical complexes of varying dimensions and with no continuous or differentiable structure. The proposed method is applied to images of handwritten digits with known deforming dimensions, data with a nontrivial topology and noisy data. We compare the estimates with results obtain by local PCA.

We use methods from Riemann geometry to investigate transformations between the color spaces of color-normal and color-weak observers. The two main applications are the simulation of the perception of a color weak observer for a color-normal observer, and the compensation of color images in a way that a color-weak observer has approximately the same perception as a color-normal observer. The metrics in the color spaces of interest are characterized with the help of ellipsoids defined by the just-noticeable-differences between the colors which are measured with the help of color-matching experiments. The constructed mappings are the isometries of Riemann spaces that preserve the perceived color differences for both observers. Among the two approaches to build such an isometry, we introduce normal coordinates in Riemann spaces as a tool to construct a global color-weak compensation map. Compared with the previously used methods, this method is free from approximation errors due to local linearizations, and it avoids the problem of shifting locations of the origin of the local coordinate system. We analyze the variations of the Riemann metrics for different observers obtained from new color-matching experiments and describe three variations of the basic method. The performance of the methods is evaluated with the help of semantic differential tests.

We introduce normal coordinates in Riemann spaces as a tool to construct color-weak compensation methods. We use them to compute color stimuli for a color weak observers that result in the same color perception as the original image presented to a color normal observer in the sense that perceived color-differences are identical for both. The compensation is obtained through a color-difference-preserving map, i.e. an isometry between the 3D color spaces of a color-normal and any given color-weak observer. This approach uses discrimination threshold data and is free from approximation errors due to local linearization. The performance is evaluated with the help of semantic differential (SD) tests.

We investigate several topics related to manifold-techniques for signal processing. On the most general level we consider manifolds with a Riemannian Geometry. These manifolds are characterized by their inner products on the tangent spaces. We describe the connection between the symmetric positive-definite matrices defining these inner products and the Cartan and the Iwasawa decomposition of the general linear matrix groups. This decomposition gives rise to the decomposition of the inner product matrices into diagonal matrices and orthonormal and into diagonal and upper triangular matrices. Next we describe the estimation of the inner product matrices from measured data as an optimization process on the homogeneous space of upper triangular matrices. We show that the decomposition leads to simple forms of partial derivatives that are commonly used in optimization algorithms. Using the group theoretical parametrization ensures also that all intermediate estimates of the inner product matrix are symmetric and positive definite. Finally we apply the method to a problem from psychophysics where the color perception properties of an observer are characterized with the help of color matching experiments. We will show that measurements from color weak observers require the enforcement of the positive-definiteness of the matrix with the help of the manifold optimization technique.

This paper extends a successful fibre-bundle surface model using l-parameter groups of a linear Lie algebra as fibres. The new model uses a fibre-bundle of l-parameter groups of a Hamilton Lie algebra, therefore is rich in descriptive power to represent bounded shapes as closed surfaces. A surface represented by this model is uniquely determined by a finite number of invariants. The complete invariant set of the model under action of Euclidean motions is obtained. Conditions for the surfaces to be bounded or closed are also given. Another feature is that the surfaces can be synthesized by elementary functions therefore free of numerical integration errors. This model can be used in recognition-synthesis-based coding of 3D images, image retrieving and copyright protection as well.

In this paper, a method of copyright monitoring or tracing using invariants of contents is proposed. We first define certain feature parameters of color images which are invariant under arbitrary bicontinuous or smooth transforms, called topological or differentially topological invariants. To extract the topological invariants robustly, we introduce scale-space of invariant features. By selecting topologically stable features with respect to scale transform, noise-robust invariants of images are obtained. These invariants can be applied to copyright tracing or monitoring and protection due to their robustness against various deformation attacks.

A new fibre bundle surface model is proposed for 3D object modeling. This fibre bundle model, consisting of local direct product of a base curve and a fibre curve, is able to represent arbitrary surfaces. In particular, the fibre bundle of 1-parameter groups, i.e. with fibres as 1-parameter groups are efficient in both synthesis and recognition. Indeed, the 1-parameter groups can be uniquely determined by finite, e.g. six invariants of their Lie algebras. Besides, the surfaces can be quickly generated by elementary functions without numerical integration errors. This model is also expected to be useful in object-based 3D image coding.

This paper presents a novel invariant representation of spatial topology of 3D objects which may have arbitrary genera and the number of connected components or separating edges. An object is represented by topological invariants in different hierarchies. We apply this representation to object recognition for 2D image inputs and 3D volume images respectively. For fast implementation of this algorithm, we use the cubic pyramid for spatial resolution transformation and fast skeletonisation.

Representation models used for 3D object recognition are desired because their invariants can be easily extracted under Euclidean and affine transformations, the shapes can be easily synthesized from these invariants and they have enough representation ability. For currently used 3D models such as generalized cylinders or super-quadratics, it is usually difficult to find the invariant features, especially to find the complete set of invariants in order to uniquely determine and reproduce the objects. The Lie algebra models which have been proposed by these authors to represent 3D objects are known to have above features. This paper presents algorithms to extract invariants of the linear Lie algebra model from 3D objects. In particular, we apply the Hough transformation to find accurate estimates of the normal vectors. Moreover, we show an efficient segmentation algorithm for 3D objects using the invariants of the linear Lie algebra. These algorithms can be applied effectively in 3D object recognition, synthesis and image coding.

In this paper we show a differential geometric model for polychromatic perception, which is based on a new neural network model assuming only a basic physiological mechanism: the lateral inhibition between different colors in visual pathway. This neural network model explains the interaction between colors as "complementary-color inhibition" in the opponent color space, or a color is subjected to attraction to or repulsion from the complementaries of other colors. Thus we are able to realize successfully the simultaneous contrast and assimilation illusions using the neural network. This model can be generalized to a new theory of color space bundle for polychromatic perception.

This paper discusses 3D free object synthesis using models of linear and Hamiltonian Lie algebra, which can represent a wide class of non-algebra and compact shapes and recover the objects uniquely using their invariants with respect to Euclidean or affine transformations. A curvature analysis of the surfaces represented by these Lie algebra models is shown, based on which an efficient algorithm of shape generation is proposed. Then the description power of these Lie algebra object models is evaluated by comparing them with the B-splines model.

This paper proposes a new simplex model for 3D objects: a cubical singular simplex model instead of the traditional triangularization simplex model. An extended Kohonen mapping is then presented as an efficient learning rule of the model. Based on this model, one can derive a pyramid structure for multi-resolution hierarchy, which is not possible for triangularization simplex. Besides, a fast algorithm is shown to calculate the homology groups of the objects as topological invariants from the proposed model.

The output error method (OEM) IIR adaptive filters (ADF) can not guarantee their stability and global convergence. We present a sufficient condition for the stability of time-varying IIR filters, and prove that the IIR ADF restricted by the condition can approximate arbitrary unknown systems, provided that the degree of ADF is high enough. Then, we propose a new adaptive method, the "Rolling-Stone scheme" and apply it to the constrained OEM IIR ADF. It is proved that the new OEM IIR ADF is guaranteed to be stable and globally convergent to a global minima.>

This paper presents rigorous conditions, without the independence assumption on the input signal, for the first and second moment convergence of the gradient adaptive algorithms including FIR and IIR (equation error method) LMS and IV algorithms.>

Discrete linear time-varying (TV) systems have been playing an important role in many real-time signal processing and control systems. The stability criteria of these TV systems require information of the impulse response sequence of the state transition matrices in an infinite time interval which is often difficult to obtain in practice. This paper reports some results on this problem. We first presents sufficient conditions for exponential, l/sup p/ and l/sup p/-BIBO stabilities. A detailed analysis is then given on this condition and the distribution domain of the instantaneous poles meeting the condition.>

A novel IIR adaptive gradient instrumental variable echo canceler (GIVE) is presented. Its features include adaptive controllability during double-talk periods in acoustic conference systems; guarantee of global convergence; low computational cost (the same order as the IIR LMS algorithm of the equation error method); and flexible structures (parallel or series-parallel structures). We also show a convergence analysis for gradient adaptive algorithms including GIVE. Based on this analysis, the optimum stepsize for GIVE and three suboptimum algorithms are proposed to accelerate convergence and reduce misadjustment. In addition, a simple method that guarantees the stability of IIR filters and a configuration of GIVE applicable to closed loop systems are presented. These proposals are extensively studied by computer simulations.>

A noninteractive ID-based key sharing scheme based on a method of iterative elimination of additive random numbers is proposed. With this method, a common key is generated step by step, leaving no opportunity for conspiracy attacks. The integrity of the scheme is discussed, and a method for improving computational efficiency is proposed.>

A three-port echo canceler (EC) configuration is proposed which observes the signal of the near-end side on a two-wire circuit in addition to the four-wire circuit signals. Embedding these signals on hybrid ports into a three-dimensional autoregressive process, echo path and innovations of near- and far-end speeches can be estimated through a three-channel lattice filter. The new configuration is then able to track echo path time variance, even during double talk (DT), and requires no changeover at either the beginning or end of DT, thus eliminating the need for DT detection. Two echo synthesizers utilizing inverse lattice and the echo path estimate possess guaranteed stability without the need for testing.>

IIR (infinite-impulse-response) echo cancellers (EC) and howling cancellers (HC) have to freeze adaptation during 'double talk' (DT). However, a fixed series-parallel structure induces signal distortion. Current methods, using a fixed parallel structure or equalizer, need a stability test which costs O(N/sup 2/) multiplications per sample period. A configuration of IIR EC and HC, consisting of a maximum-entropy two-channel lattice filter and its inverse filter, is proposed. It can cancel echoes and during DT eliminate the signal distortion by the fixed inverse lattice. Its stability is automatically guaranteed without the necessity of tests. The computational cost is O(N).>