This paper introduces a new approach to sound source localization using head-related transfer function (HRTF) characteristics, which enable precise full-sphere localization from raw data. While previous research focused primarily on using extensive microphone arrays in the frontal plane, this arrangement often encountered limitations in accuracy and robustness when dealing with smaller microphone arrays. Our model proposes using both time and frequency domain for sound source localization while utilizing Deep Learning (DL) approach. The performance of our proposed model, surpasses the current state-of-the-art results. Specifically, it boasts an average angular error of 0.24° and an average Euclidean distance of 0.01 meters, while the known stateof-the-art gives average angular error of 19.07° and average Euclidean distance of 1.08 meters. This level of accuracy is of paramount importance for a wide range of applications, including robotics, virtual reality, and aiding individuals with cochlear implants (CI).

The field of implicit neural representation has made significant progress. Models such as neural radiance fields (NeRF) [12], which uses relatively small neural networks, can represent high-quality scenes and achieve state-of-the-art results for novel view synthesis. Training these types of networks, however, is still computationally expensive and the model struggles with real life 360° scenes. In this work, we propose the depth distribution neural radiance field (DDNeRF), a new method that significantly increases sampling efficiency along rays during training, while achieving superior results for a given sampling budget. DDNeRF achieves this performance by learning a more accurate representation of the density distribution along rays. More specifically, the proposed framework trains a coarse model to predict the internal distribution of the transparency of an input volume along each ray. This estimated distribution then guides the sampling procedure of the fine model. Our method allows using fewer samples during training while achieving better output quality with the same computational resources.

One of the fascinating properties of deep learning is the ability of the network to reveal the underlying factors characterizing elements in datasets of different types. Autoencoders represent an effective approach for computing these factors. Autoencoders have been studied in the context of enabling interpolation between data points by decoding convex combinations of latent vectors. However, this interpolation often leads to artifacts or produces unrealistic results during reconstruction. We argue that these incongruities are due to the structure of the latent space and to the fact that such naively interpolated latent vectors deviate from the data manifold. In this paper, we propose a regularization technique that shapes the latent representation to follow a manifold that is consistent with the training images and that forces the manifold to be smooth and locally convex. This regularization not only enables faithful interpolation between data points, as we show herein but can also be used as a general regularization technique to avoid overfitting or to produce new samples for data augmentation.

The weight decay regularization term is widely used during training to constrain expressivity, avoid overfitting, and improve generalization. Historically, this concept was borrowed from the SVM maximum margin principle and extended to multiclass deep networks. Carefully inspecting this principle reveals that it is not optimal for multi-class classification in general, and in particular when using deep neural networks. In this paper, we explain why this commonly used principle is not optimal and propose a new regularization scheme, called Pairwise Margin Maximization (PMM), which measures the minimal amount of displacement an instance should take until its predicted classification is switched. In deep neural networks, PMM can be implemented in the vector space before the network’s output layer, i.e., in the deep feature space, where we add an additional normalization term to avoid convergence to a trivial solution. We demonstrate empirically a substantial improvement when training a deep neural network with PMM compared to the standard regularization terms.

Wavelet denoising is a classical and effective approach for reducing noise in images and signals. Suggested in 1994, this approach is carried out by rectifying the coefficients of a noisy image, in the transform domain, using a set of shrinkage functions (SFs). A plethora of papers deals with the optimal shape of the SFs and the transform used. For example, it is widely known that applying SFs in a redundant basis improves the results. However, it is barely known that the shape of the SFs should be changed when the transform used is redundant. In this paper, we introduce a complete picture of the interrelations between the transform used, the optimal shrinkage functions, and the domains in which they are optimized. We suggest three schemes for optimizing the SFs and provide bounds of the remaining noise, in each scheme, with respect to the other alternatives. In particular, we show that for subband optimization, where each SF is optimized independently for a particular band, optimizing the SFs in the spatial domain is always better than or equal to optimizing the SFs in the transform domain. Furthermore, for redundant bases, we provide the expected denoising gain that can be achieved, relative to the unitary basis, as a function of the redundancy rate.

We present RENE --- a novel encoding scheme for short ranges on Ternary content addressable memory (TCAM), which, unlike previous solutions, does not impose row expansion, and uses bits proportionally to the maximal range length. We provide theoretical analysis to show that our encoding is the closest to the lower bound of number of bits used. In addition, we show several applications of our technique in the field of packet classification, and also, how the same technique could be used to efficiently solve other hard problems such as the nearest-neighbor search problem and its variants. We show that using TCAM, one could solve such problems in much higher rates than previously suggested solutions, and outperform known lower bounds in traditional memory models. We show by experiments that the translation process of RENE on switch hardware induces only a negligible 2.5% latency overhead. Our nearest neighbor implementation on a TCAM device provides search rates that are up to four orders of magnitude higher than previous best prior-art solutions.

Similarity search, and specifically the nearest-neighbor search (NN) problem is widely used in many fields of computer science such as machine learning, computer vision and databases. However, in many settings such searches are known to suffer from the notorious curse of dimensionality, where running time grows exponentially with d. This causes severe performance degradation when working in high-dimensional spaces. Approximate techniques such as locality-sensitive hashing [2] improve the performance of the search, but are still computationally intensive.

We present a linear-time subspace clustering approach that combines sparse representations and bipartite graph modeling. The signals are modeled as drawn from a union of low-dimensional subspaces, and each signal is represented by a sparse combination of basis elements, termed atoms, which form the columns of a dictionary matrix. The sparse representation coefficients are arranged in a sparse affinity matrix, which defines a bipartite graph of two disjoint sets: 1) atoms and 2) signals. Subspace clustering is obtained by applying low-complexity spectral bipartite graph clustering that exploits the small number of atoms for complexity reduction. The complexity of the proposed approach is linear in the number of signals, thus it can rapidly cluster very large data collections. Performance evaluation of face clustering and temporal video segmentation demonstrates comparable clustering accuracies to state-of-the-art at a significantly lower computational load.

A fast pattern matching scheme termed matching by tone mapping (MTM) is introduced which allows matching under nonlinear tone mappings. We show that, when tone mapping is approximated by a piecewise constant/linear function, a fast computational scheme is possible requiring computational time similar to the fast implementation of normalized cross correlation (NCC). In fact, the MTM measure can be viewed as a generalization of the NCC for nonlinear mappings and actually reduces to NCC when mappings are restricted to be linear. We empirically show that the MTM is highly discriminative and robust to noise with comparable performance capability to that of the well performing mutual information, but on par with NCC in terms of computation time.

The graph Laplacian operator, which originated in spectral graph theory, is commonly used for learning applications such as spectral clustering and embedding. In this paper we explore the Laplacian distance, a distance function related to the graph Laplacian, and use it for visual search. We show that previous techniques such as Matching by Tone Mapping (MTM) are particular cases of the Laplacian distance. Generalizing the Laplacian distance results in distance measures which are tolerant to various visual distortions. A novel algorithm based on linear decomposition makes it possible to compute these generalized distances efficiently. The proposed approach is demonstrated for tone mapping invariant, outlier robust and multimodal template matching.

Video analysis often requires mapping of activity or object locations from image coordinates to ground plane coordinates. This process is termed Plane Rectification. In this paper we propose a geometric method to find plane rectification using the plane's vanishing line and the vertical vanishing point. Unlike common methods that provide sophisticated algebraic solutions and non-linear optimizations, the proposed approach is intuitive and simple to implement while providing a geometric explanation and interpretation of the plane rectification. We show that the proposed approach provides stable and accurate solutions also in the presence of noise.

Histogram Matching (HM) is a common technique for finding a monotonic map between two histograms. However, HM cannot deal with cases where a single mapping is sought between two sets of histograms. This paper presents a novel technique that finds such a mapping in an optimal manner under various histograms distance measures.

We present a probabilistic subspace clustering approach that is capable of rapidly clustering very large signal collections. Each signal is represented by a sparse combination of basis elements (atoms), which form the columns of a dictionary matrix. The set of sparse representations is utilized to derive the co-occurrences matrix of atoms and signals, which is modeled as emerging from a mixture model. The components of the mixture model are obtained via a non-negative matrix factorization (NNMF) of the co-occurrences matrix, and the subspace of each signal is estimated according to a maximum-likelihood (ML) criterion. Performance evaluation demonstrate comparable clustering accuracies to state-of-the-art at a fraction of the computational load.

In this paper, we propose a novel video foreground detection method that exploits the statistics of 3D spacetime patches. 3D space-time patches are characterized by means of the subspace they span. As the complexity of real-time systems prohibits performing this modeling directly on the raw pixel data, we propose a novel framework in which spatiotemporal data is sequentially reduced in two stages. The first stage reduces the data using a cascade of linear projections of 3D space-time patches onto a small set of 3D Walsh-Hadamard (WH) basis functions known for its energy compaction of natural images and videos. This stage is efficiently implemented using the Gray-Code filtering scheme [2] requiring only 2 operations per projection. In the second stage, the data is further reduced by applying PCA directly to the WH coefficients exploiting the local statistics in an adaptive manner. Unlike common techniques, this spatiotemporal adaptive projection exploits window appearance as well as its dynamic characteristics. Tests show that the proposed method outperforms recent foreground detection methods and is suitable for real-time implementation on streaming video.

We propose a fast pattern matching scheme termed Matching by Tone Mapping (MTM) which allows matching under non-linear tone mappings. We show that, when tone mapping is approximated by a piecewise constant function, a fast computational scheme is possible requiring computational time similar to the fast implementation of Normalized Cross Correlation (NCC). In fact, the MTM measure can be viewed as a generalization of the NCC for non-linear mappings and actually reduces to NCC when mappings are restricted to be linear. The MTM is shown to be invariant to non-linear tone mappings, and is empirically shown to be highly discriminative and robust to noise.

We present a novel weighted approach for shrinkage functions learning in image denoising. The proposed approach optimizes the shape of the shrinkage functions and maximizes denoising performance by emphasizing the contribution of sparse overcomplete representation components. In contrast to previous work, we apply the weights in the overcomplete domain and formulate the restored image as a weighted combination of the post-shrinkage overcomplete representations. We further utilize this formulation in an offline Least Squares learning stage of the shrinkage functions, thus adapting their shape to the weighting process. The denoised image is reconstructed with the learned weighted shrinkage functions. Computer simulations demonstrate superior shrinkage-based denoising performance.

Polynomial Texture Maps (PTM)[SIGGRAPH 2001] form an alternative method for apprehending surface colour and albedo that extends a simple model of image formation from the Lambertian variant of Photometric Stereo (PST) to more general reflectances. Here we consider solving such a model in a robust version, not to date attempted for PTM. But the main upshot of utilizing robust regression is in the identification of both shadows and specularities automatically, without the need for any thresholds, in a tripar-tite set of weights for pixels that are labelled as matte, shadow, or specularity. Original images are captured using a hemispherical set of lights, and pixel values across the light-ing directions are then labelled as inliers, or outliers of two types. A per-pixel robust regression on luminance is carried out using Least Median of Squares, and automatically-identified outlier pixels are labelled as shadows if they are darker than matte and corre-spondingly, specular outliers are too bright. Inlier identification generates correct values for chromaticity and for surface albedo and thus matte luminance and colour. Then a robust version of PST, using only PTM inliers, improves estimates of normal vectors and albedo recovered. With specular pixel values over the lights in hand we model specu-larity using a radial basis function (RBF) regression, and non-specular pixel departures from matte using a second RBF set. Then for a new lighting direction, we can readily interpolate both specular content as well as shadows. 1

Many applications in computer vision require comparisons between two images of the same scene. Comparison applications usually assume that corresponding regions in the two images have similar colors. However, this assumption is not always true. One way to deal with this problem is to apply a color mapping to one of the images. In this paper we address the challenge of computing color mappings between pairs of images acquired under different acquisition conditions, and possibly by different cameras. For images taken from different viewpoints, our proposed method overcomes the lack of pixel correspondence. For images taken under different illumination, we show that no single color mapping exists, and we address and solve a new problem of computing a minimal set of piecewise color mappings. When both viewpoint and illumination vary, our method can only handle planar regions of the scene. In this case, the scene planar regions are simultaneously co-segmented in the two images, and piecewise color mappings for these regions are calculated. We demonstrate applications of the proposed method for each of these cases.

This paper suggests a discriminative approach for wavelet denoising where a set of mapping functions (MFs) are applied to the transform coefficients in an attempt to produce a noise free image. As opposed to the descriptive approaches, modeling image or noise priors is not required here and the MFs are learned directly from an ensemble of example images using least-squares fitting. The suggested scheme generates a novel set of MFs that are essentially different from the traditional soft/hard thresholding in the over-complete case. These MFs are demonstrated to obtain comparable performance to the state-of-the-art denoising approaches. Additionally, this framework enables a seamless customization of the shrinkage operation to a new set of restoration problems that were not addressed previously with shrinkage techniques, such as deblurring, JPEG artifact removal, and various types of additive noise that are not necessarily Gaussian white noise.

In this paper, we introduce a family of filter kernels--the Gray-Code Kernels (GCK) and demonstrate their use in image analysis. Filtering an image with a sequence of Gray-Code Kernels is highly efficient and requires only two operations per pixel for each filter kernel, independent of the size or dimension of the kernel. We show that the family of kernels is large and includes the Walsh-Hadamard kernels, among others. The GCK can be used to approximate any desired kernel and, as such forms, a complete representation. The efficiency of computation using a sequence of GCK filters can be exploited for various real-time applications, such as, pattern detection, feature extraction, texture analysis, texture synthesis, and more.

Shift-invariant linear algorithms can be described completely by the algorithm's response to an impulse input. The so called impulse response can be used as filter kernels when the algorithm is equivalently implemented using convolution. This, however, is not true when the system is block-shift invariant, i.e. when invariance is only at repetitive locations. This paper describes a generalization of the impulse response for block shift-invariant systems. The proposed technique, takes any computer program which implements a linear block-shift-invariant algorithm, and produces its equivalent filter kernels. These kernels can then be applied efficiently by using convolution. This scheme can assist in finding the filter kernels of linear operations applied directly to mosaic images acquired by digital cameras. For example, using this approach any algorithmic description for the demosaicing problem, which is linear and block shift-invariant, can be translated into actual filter kernels that can be applied efficiently.

A novel approach to pattern matching is presented in which time complexity is reduced by two orders of magnitude compared to traditional approaches. The suggested approach uses an efficient projection scheme which bounds the distance between a pattern and an image window using very few operations on average. The projection framework is combined with a rejection scheme which allows rapid rejection of image windows that are distant from the pattern. Experiments show that the approach is effective even under very noisy conditions. The approach described here can also be used in classification schemes where the projection values serve as input features that are informative and fast to extract.

A novel approach to pattern matching is presented, which reduces time complexity by two orders of magnitude compared to traditional approaches. The suggested approach uses an efficient projection scheme which bounds the distance between a pattern and an image window using very few operations. The projection framework is combined with a rejection scheme which allows rapid rejection of image windows that are distant from the pattern. Experiments show that the approach is effective even under very noisy conditions. The approach described here can also be used in classification schemes where the projection values serve as input features that are informative and fast to extract.

Motion estimation, motion detection and tracking invariably require finding a particular pattern in a set or sequence of images. The task involves finding appearances of a given pattern in an image under various transformations and at various locations. This process is of very high time complexity since a search must be implemented both in the transformation domain and in the spatial domain. Contributing to this complexity is the chosen distance metric that measures the similarity between patterns. The Euclidean distance, for example, may change drastically when a small transformation is applied to the pattern. Applying a different metric distance might be advantageous, though at the expense of loosing the norm structure of the Euclidean space. In this work we present a new method for fast search in the transformation domain, which can also be applied in metric spaces. The method is based on recursive decomposition of the transformation domain, and a rejection scheme, which enables the process to quickly reject as irrelevant large percentages of this decomposition.

In this paper we introduce the idea of using computer vision techniques for improving and enhancing watermarking capabilities. Specifically, we incorporate geometric hashing techniques into the watermarking methodology. Geometric hashing was developed to detect objects in a visual scene under a class of geometric transformations. The technique is incorporated into watermarking to detect watermarks encoded under transformations. This allows randomization of the watermark code without the need of maintaining the random generator seed. In turn, this randomization increases robustness under attacks such as collusion (determining the watermark from multiple watermarked examples). Depending on the embedding domain, robustness of the watermark under geometric attacks can be achieved.

This paper addresses the problem of recovering a super-resolved image from a set of warped blurred and decimated versions thereof. Several algorithms have already been proposed for the solution of this general problem. In this paper, we concentrate on a special case where the warps are pure translations, the blur is space invariant and the same for all the images, and the noise is white. We exploit previous results to develop a new highly efficient super-resolution reconstruction algorithm for this case, which separates the treatment into de-blurring and measurements fusion. The fusion part is shown to be a very simple non-iterative algorithm, preserving the optimality of the entire reconstruction process, in the maximum-likelihood sense. Simulations demonstrate the capabilities of the proposed algorithm.

Gradient based approaches for motion estimation (optical-flow) estimate the motion of an image sequence based on local changes in the image intensities. In order to best evaluate local changes in the intensities, specific filters are applied to the image sequence. These filters are typically composed of spatio-temporal derivatives. The design of these filters plays an important role in the estimation accuracy. This paper proposes a method for the design of these filters in an optimal manner. Unlike previous approaches that design optimal derivative filters in some sense, the proposed technique defines the optimality directly with respect to the motion estimation goal. The suggested approach takes into account prior knowledge on the motion distribution, the image characteristics, and the allocated filter length. Simulations demonstrate the advantage of the new design approach.

An efficient method of computing the least-squares optimal basis functions to steer any function locally is presented. The method combines the Lie group-theoretic and the singular value decomposition approaches. Its efficiency is demonstrated with the design of basis functions to steer a Gabor function under the four-parameter linear transformation group.

A new cascade basis reduction method of computing the optimal least-squares set of basis functions to steer a given function is presented. The method combines the Lie group-theoretic and the singular value decomposition approaches such that their respective strengths complement each other. Since the Lie group-theoretic approach is used, the set of basis and steering functions computed can be expressed in analytic form. Because the singular value decomposition method is used, this set of basis and steering functions is optimal in the least-squares sense. Most importantly, the computational complexity in designing basis functions for transformation groups with large numbers of parameters is significantly reduced. The efficiency of the cascade basis reduction method is demonstrated by designing a set of basis functions to steer a Gabor function under the four-parameter linear transformation group.

We present a computational, group-theoretic approach to steerable functions. The approach is group-theoretic in that the treatment involves continuous transformation groups for which elementary Lie group theory may be applied. The approach is computational in that the theory is constructive and leads directly to a procedural implementation. For functions that are steerable with n finite number of basis functions under a k-parameter group, the procedure is efficient and is guaranteed to return the minimum number of basis functions. If the function is not steerable, a numerical implementation of the procedure could also be used to compute basis functions that approximately steer the function over a range of transformation parameters. Examples of both applications are demonstrated.

Steerable functions find application in numerous problems in image processing, computer vision and computer graphics. As such, it is important to develop the appropriate mathematical tools to analyze them. In this paper, we introduce the mathematics of Lie group theory in the context of steerable functions and present a canonical decomposition of these functions under any transformation group. The theory presented in this paper can be applied and extended in various ways.

A method for fusing and integrating different 2D and 3D measurements for pose estimation is proposed. The 2D measured data is viewed as 3D data with infinite uncertainty in particular directions. The method is implemented using Kalman filtering. It is robust and easily parallelizable.>

This paper presents a method for interpretation of modeled objects that is general enough to cover articulated and other types of constrained models. The flexibility between components of the model are expressed as spatial constraints which are fused into the pose estimation during the interpretation process. The constraint fusion assists in obtaining the correct interpretation and in reducing the search of possible correspondences. The proposed method can handle any constraint (including inequalities) between any number of different components of the model. The framework is based on Kalman filtering.>

Nonmetric multidimensional scaling (MDS) is a family of algorithms that allow one to derive a quantitative representation of data from a set of qualitative measurements which must satisfy certain simple constraints. As a tool for vision, MDS combines the advantages of both qualitative and classical approaches, by relying, on the one hand, on an ordinal-scale input representation, and by supporting, on the other hand, the extraction of metric information. The present paper illustrates an application of MDS to the recovery of depth from the rank order of binocular disparity differences for a set of points. Our results indicate that multidimensional scaling constitutes a promising approach to the integration of biological and computational insights into the problem of depth perception.

Point displacement constraints constitute an attractive technique for interactive design of smooth curves, surfaces, and volumes. The user defines an arbitrary number of “control points” on the object and specifies their desired spatial location, while the system computes the object's degrees of freedom so that the constraints are satisfied. A constraint-based interface gives a feeling of direct manipulation of the object. In this article we introduce soft constraints , constraints which do not have to be met exactly. The softness of each constraint serves as a nonisotropic, local shape parameter enabling the user to explore the space of objects conforming to the constraints. Additionally, there is a global shape parameter which determines the amount of similarity of the designed object to a rest shape, or equivalently, the rigidity of the rest shape. We present an algorithm termed probabilistic point constraints (PPC) for implementing soft constraints. The PPC algorithm views constraints as stochastic measurements of the state of a static system. The softness of a constraint is derived from the covariance of the “measurement.” The resulting system of probabilistic equations is solved using the Kalman filter , a powerful estimation tool in the theory of stochastic systems. We also describe a user interface using direct-manipulation devices for specifying and visualizing covariances in 2D and 3D. The algorithm is suitable for any object represented as a parametric blend of control points, including most spline representations. The covariance of a constraint provides a continuous transition from exact interpolation to controlled approximation of the constraint. The algorithm involves only linear operations and allows real-time interactive direct manipulation of curves and surfaces on current workstations.

A general and flexible method for fusing and integrating different 2D and 3D measurements for pose estimation is proposed. The 2D measured data are viewed as 3D data with infinite uncertainty in a particular direction. This representation unifies the two categories of the absolute orientation problem into a single problem that varies only in the uncertainty values associated with the measurements. With this paradigm a uniform mathematical formulation of the problem is obtained, and different kinds of measurements that can be fused to obtain a better solution. The method, which is implemented using Kalman filtering, is robust and easily parallelizable.>

The authors study the notion of chirality for two-dimensional binary shapes, and introduce measures to test whether a shape is symmetric, and if not whether it is left-handed or right-handed. The measures are based on boundary analysis, and perform well even when digital images of left-handed shapes differ from the mirror images of right-handed shapes. Such situations may occur due to natural variations and digitization errors. The measures can also successfully treat partially occluded shapes, and provide indications on the change of chirality as resolution changes.>