The differential geometry of images (one of numerous applications but certainly an important one) involves singly isotropic space, rather than Euclidean space, the reason being that image intensity is not commensurate with the dimensions of the image plane. The Procrustes root mean square nonuniformity measure in such spaces immediately leads to a principled definition of curvedness, shape index, and orientation of local second-order structure. However, it is categorically different from the conventional curvature-based measures. One obtains a natural Euclidean metric of shape space that is readily extended to the cubic and quartic orders of approximation. In this setting, it is simple to derive the marginal probability densities for curvedness, shape index, and orientation for isotropic and anisotropic Gaussian random fields. For slight anisotropies, the marginals are much closer to those empirically found in natural images than the conventional formalisms suggest. The main articulation fits the edge and ridge structure imposed by the linear and quadric orders. The cubic and quartic structures contribute in a natural manner to these edge and ridge structures, whereas their higher-order saddle structures contribute little to the variance and can generally be ignored in applications.

Color pickers are widely used in all kinds of display applications. They vary greatly in their utility, depending on user expertise. We focus on nonprofessional, occasional users. Such users may spend from a few seconds up to a few minutes to select a color. Yet, typically they reach final accuracy within the initial 20s. Additional effort leads to random walks in the neighborhood of the target. We explore the efficaciousness of five generic color pickers, analyzing the results in terms of generic user interface properties. There is a major dichotomy between three-slider interfaces, and those that offer some form of 2D selectivity. The accuracy in rgb coordinates is about one-tenth to one-twentieth of the full scale (often 0--255 in r , g , and b ), whereas a little over 100 hues are resolved. The most efficient color picker, which is presently rarely used in popular applications, is much more efficient than the worst one. We speculate that this derives from a closer match to the user’s internal representation of color space. The study results in explicit recommendations for the implementation of user-friendly and efficient color tools.

``Pictorial space” is the mental structure that appears to be the scaffold for the visual awareness when looking “into” (as opposed to “at'') a picture. Its structure differs from the “visual space” that is the scaffold for the visual awareness when looking into the scene in front of the observer. The structure of pictorial space has been probed empirically and explored theoretically. Here we propose a framework that allows one to handle cases that have been encountered empirically, but thus far have not been explored in a formal, geometrical setting. The framework allows one to handle many idiosyncrasies of human visual observers, as well as to characterize the (frequent) individual differences in a principled manner. This opens the door to a principled formalism of the structure (e.g., quality) of the pictorial spaces evoked by various methods of presentation, as required for applications.

We investigate the estimation of illuminance flow using Histograms of Oriented Gradient features (HOGs). In a regression setting, we found for both ridge regression and support vector machines, that the optimal solution shows close resemblance to the gradient based structure tensor (also known as the second moment matrix). Theoretical results are presented showing in detail how the structure tensor and the HOGs are connected. This relation will benefit computer vision tasks such as affine invariant texture/object matching using HOGs. Several properties of HOGs are presented, among others, how many bins are required for a directionality measure, and how to estimate HOGs through spatial averaging that requires no binning.

This special issue is devoted to understanding human movement by walking in real and virtual environments, investigating activities including obstacle avoidance, estimation of travel distance, visuomotor calibration of walking, gender differences in path integration, heading assessment in low vision and visual speed-matching on a feedback-controlled walking machine (treadmill).Whether implicitly or explicitly, every study in the issue takes a comparative approach, relating the activity in the virtual world to its counterpart in the real world, and vice versa, as without cross-validating one realm with the other it is arguably impossible to get it right.Necessarily, this is also a multidisciplinary area of research and thus, appropriately, the background of the contributors to this issue range from computer science to psychology, engineering to neuroscience, optics to ophthalmology and others.Indeed, there is as much of a goal for the issue to understand multimodal processing in human locomotion as there is to further the technology of virtual reality through its understanding, two aspects that embody the cross-disciplinary ethos of ACM TAP.The capability for a user to move around in the virtual, as in the real, world has long been a key requirement and challenge for virtual reality technologists and scientists.Full motion for navigation in virtual environments (VEs) using joy-stick, mouse or keyboard, can be effective but does not mimic important aspects of human action nor, it seems, do all graphic environments convey navigational cues equivalently.What can we learn from human locomotor movement and behavior in the real world that could yield a better interface in virtual reality (VR), and how can we use VR to improve our understanding of the former?Obstacle avoidance is no less than a requirement to our survival while moving, and thus to faithfully include it in VR requires a comparison to be made between its characteristics in VR and the real world.In Fink et al. in this issue, small but reliable differences are found in locomotor paths, as subjects' avoided

Specularities on surfaces with tangential hairs or grooves are readily observable in nature. Examples of such phenomena are the arched or looped highlights observed on horses and on human heads and the linear or curved specularities observed on parts of industrial machinery that have tangential grooves. We investigate the geometry of curvilinear specularities on surfaces of different curvature with tangential hairs or grooves of different orientation, under controlled lighting and viewing conditions. First the nature of these specularities is investigated qualitatively. Then specularities on parametric surfaces and hair or groove orientations are calculated for some specific cares. Explicit calculations of specularities on some special surfaces, cylinders, cones, and spheres, are verified by photographs of the reflections. Aspects of the work are applicable to computer graphics and can be utilized for the image interpretation of surface specularities.

In this work, we investigate the visual appearance of real-world surfaces and the dependence of appearance on the geometry of imaging conditions. We discuss a new texture representation called the BTF (bidirectional texture function) which captures the variation in texture with illumination and viewing direction. We present a BTF database with image textures from over 60 different samples, each observed with over 200 different combinations of viewing and illumination directions. We describe the methods involved in collecting the database as well as the importqance and uniqueness of this database for computer graphics. A related quantity to the BTF is the familiar BRDF (bidirectional reflectance distribution function). The measurement methods involved in the BTF database are conducive to simultaneous measurement of the BRDF. Accordingly, we also present a BRDF database with reflectance measurements for over 60 different samples, each observed with over 200 different combinations of viewing and illumination directions. Both of these unique databases are publicly available and have important implications for computer graphics.

In this work, we investigate the visual appearance of real-world surfaces and the dependence of appearance on imaging conditions. We present a BRDF (bidirectional reflectance distribution function) database with reflectance measurements for over 60 different samples, each observed with over 200 different combinations of viewing and source directions. We fit the BRDF measurements to two recent models to obtain a BRDF parameter database. These BRDF parameters can be directly used for both image analysis and image synthesis. Finally, we present a BTF (bidirectional texture function) database with image textures from over 60 different samples, each observed with over 200 different combinations of viewing and source directions. Each of these unique databases has important implications for a variety of vision algorithms and each is made publicly available.

Motion of an observer relative to objects in a scene provides information about the structure of the scene. Changing patterns of shading due to motion relative to the light source provide information about surface structure, albedos, and light sources. One can stratify this photometric information into affine, unitary, and metric structure, much like the stratification of structure from motion. For Lambertian surfaces, if either motion or photometry give us more than affine structure, the two cues can be combined to yield full metric information. Edge constraints plus unitary photometry also give us full metric photometry. Affine structure alone contains much of the quantitative structure information, allowing us to judge such things as the ordinal relationships between the albedos.

REALISE was designed to extract from sequences of images, acquired with a moving camera, the information necessary for determining the 3D (CAD-like) structure of a real-life scene together with information about the radiometric signatures of surfaces bounding the extracted 3D objects (e.g. reflectance behaviour). The retrieved information is then integrated in a virtual reality (VR) software environment. The R&D work is been performed principally in the following areas of computer vision and computer graphics: structure from motion, recovery of geometries, recovery of photometric and texture information, highly realistic rendering on the basis of empirically-based reflectance models, and the design and development of improved rendering processes together with a new VR system. Beside this innovative R&D work another key aspect of REALISE is to have computer vision & computer graphics cooperate to produce realistic 3D data efficiently.

We have constructed model textures with simple known structure. Control of the local reflectance properties and surface topography, facilitates the study of texture. We focus on the bi-angular dependence of the texture. We identify geometrical phenomena such as masking and shadowing as well as interreflections, and their effects to the spatial distribution of reflected light. Texture is studied through direct observations of the actual images and with the help of gray level image histograms measuring the proportion of gray tone in every image, and bidirectional reflection distribution function (BRDF) measurements describing the directionally dependent ratio of radiance to irradiance.

Surfaces play an important role in visual perception. They are perceived as «(perceptual) reliefs», that are surfaces in 2 + 1D perceptual space, that is the product space of the 2D visual field and the 1D «depth dimension». It is in many respects irrelevant whether the observer views a true 3D scene or a flat (2D) picture of a scene. In both cases, the percepts are reliefs in 2 + 1D perceptual space. In the latter case, one speaks of «pictorial relief». We discuss how perceptual reliefs can be measured and which aspects of these reliefs are especially robust against day-to-day intraobserver variations, changes of viewing conditions and interobserver differences. It turns out that only aspects of the partial depth order (based on depth precedence in infinitesimal regions) are stable. Thus, features of the relief are invariants of general «relief preserving transformations» that may actually scramble depth values at different locations. This is evident from the fact that human observers can only judge depth precedence with some degree of certainty for points that are on a single slope. We discuss the formal structure of these relief invariants. Important ones are the Morse critical points and the ridges and courses of the relief

Presents a method of multi-resolution image analysis that gives the most concise set of local orthogonally invariant features of 2-dimensional input images. Solving the equivalence problem corresponding to a local jet and the group of orthogonal transformations of the cartesian coordinate frame the authors find a complete and irreducible set of local algebraic invariants that may describe any local orthogonally invariant feature of the jet.>

A method that treats linear neighborhood operators within a unified framework that enables linear combinations, concatenations, resolution changes, or rotations of operators to be treated in a canonical manner is presented. Various families of operators with special kinds of symmetries (such as translation, rotation, magnification) are explicitly constructed in 1-D, 2-D, and 3-D. A concept of 'order' is defined, and finite orthonormal bases of functions closely connected with the operators of various orders are constructed. Linear transformations between the various representations are considered. The method is based on two fundamental assumptions: a decrease of resolution should not introduce spurious detail, and the local operators should be self-similar under changes of resolution. These assumptions merely sum up the even more general need for homogeneity isotropy, scale invariance, and separability of independent dimensions of front-end processing in the absence of a priori information.>

A technique that is functionally equivalent to the oriented smoothing concept and reduces numerical complexity and computational costs by eliminating the smoothness requirement from the iteration process is introduced. Local affine transformations are applied to propagate uniquely computed flow vectors into homogeneous regions and along edges in a single step. The window within which the local affine transformation is performed can adapt to the local structure of the intensity pattern in accordance with the oriented smoothness concept as formulated by H. Nagel (1987).>

A hitherto unnoticed singularity of scale space occurs only at isolated points in scale space. Thus it does not generically occur for single images, but it can occur occasionally in members of time sequences (say). It occurs at those critical points of the image at which the Laplacean of the illuminance vanishes (a nongeneric condition). The structure of scale space in an infinitesimal neighborhood of such a singularity is explored. The effect of the singularity of mappings between copies of an image at different levels of resolution is evaluated and checked with a numerical calculation.>

In contradistinction with technical pattern recognizers, humans are very apt at the detection of regions of coherent movements in changing images. They detect coherent patches in spatiotemporal white noise at threshold signal-to-noise ratios of one hundredth! Possible physiological mechanisms that sustain this ability are discussed in terms of simple mechanistic models, and the results of psychophysical experiments are presented. These results are compatible with two different mechanistic interpretations. The main result is that the human movement detectors are completely unlike the speedometer in an automobile: they are tuned, and a whole ensemble of mechanisms, tuned to different velocities, reside at any location in the visual field. Thus you may easily see two velocity vectors simultaneously at a given place. Segregation occurs when different detectors are stimulated at each side of a border. The spatiotemporal parameters that characterize these units are found: these limit the resolution in time and space, whereas the sensitivity depends on the number of units that participate in a detection. This number may range between a few (perhaps one) to a thousand or more. Apparently, resolution can be traded against noise immunity. It is argued that technical systems, developed on a similar basis, might be useful as preprocessors of sequences of images in order to detect features of interest (coherent regions) and to suggest a first rough image segmentation.

A dynamic model based on electrophysiological findings is presented for information processing in the visual system. The visual system behaves as an optimal encoder both of information perturbed by Poisson noise at low luminances and of noise-distorted images at suprathreshold level. The basic elements of the model, are: (1) a first layer of square-root scalers mainly performing noise reduction, where each scaler consists of two leaky integrators and a comparator; (2) a layer containing a light detector consisting of one leaky integrator and a comparator, and an increment/decrement detector consisting of two leaky integrators and a comparator. The results of simulations of threshold behavior are given. The authors show that all generally known psychophysical facts can be described with this model. When spatial interaction between neighboring basic elements is introduced, the effects of these interactions spread over a large area, thus changing properties of the total network. So far, this extensive effect has only been proved with phenomenological models. Possible applications of this model in image processing are proposed.