Novel view synthesis methods, such as Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS), rely on Structure-from-Motion (SfM) pipelines like COLMAP for camera parameter estimation. However, these pipelines are prone to errors due to factors like doppelgangers and perspective distortion. While edge devices (e.g., mobile phones) capture images with embedded GPS and IMU data, this meta-data is often noisy. We introduce MetaReg, a robust pipeline that improves camera parameter estimation by leveraging noisy GPS metadata. MetaReg enhances COLMAP with pre-and post-processing steps: the preprocessing stage estimates image overlap using metadata to reduce unnecessary image matching, while the post-processing stage aligns estimated camera coordinates with the world coordinate system using noisy GPS priors. Experiments on challenging datasets demonstrate that MetaReg significantly improves camera parameter estimation, enhancing robustness and accuracy.

Recent advances in machine learning and computer vision have made it simple to manipulate a variety of media, including satellite images. Most of the commercially available satellite images go through the process of orthorectification to remove potential distortions due to terrain variations. This orthorectification process typically involves the use of rational polynomial coefficients (RPC) that geometrically remap the pixels in the original image to the rectified image. This paper proposes the first method to verify the authenticity of RPC metadata in an orthorectified satellite image. The steps include calculating the Residual Discrete Fourier Transform (DFT) pattern from the image using a linear predictor based residual spectral analysis and comparing with Expected Residual DFT pattern that is obtained using the RPC metadata associated with the image. If the metadata associated with orthorectified image is correct, then the Residual-DFT pattern (which represents image data) and the Expected-Residual-DFT pattern (which represents metadata) should be similar. We use SSIM (Structural Similarity Index Metric) to quantify the similarity and thereby verify if the data has been tampered or not. Detailed experimental results demonstrate that our method achieves over 97% accuracy in the majority of binary tampering detection tests.

Video activity analysis systems are often trained on large datasets. Activities and events in the real world do not occur in isolation, instead, they occur as interactions between related objects. This work introduces a novel method that jointly exploits relational information between pairs of objects and temporal dynamics of each object. The proposed method effectively leverages a new simple architecture that is flexible and easily trained to detect relational activities and events using small datasets (hundreds of samples). The solution is constructed and tested using synthetic videos of car-collision events. The annotated datasets in this work will be made available online to the research community. Experimental results demonstrate the efficacy of the network to perform complex activity analysis.

This tutorial provides an introduction to the development of fast matrix algorithms based on the notions of displacement and various low-rank structures.

We describe a system for content-based retrieval from large surveillance video archives, using behavior, action and appearance of objects. Objects are detected, tracked, and classified into broad categories. Their behavior and appearance are characterized by action detectors and descriptors, which are indexed in an archive. Queries can be posed as video exemplars, and the results can be refined through relevance feedback. The contributions of our system include the fusion of behavior and action detectors with appearance for matching; the improvement of query results through interactive query refinement (IQR), which learns a discriminative classifier online based on user feedback; and reasonable performance on low resolution, poor quality video. The system operates on video from ground cameras and aerial platforms, both RGB and IR. Performance is evaluated on publicly-available surveillance datasets, showing that subtle actions can be detected under difficult conditions, with reasonable improvement from IQR.

In this paper we introduce novel regularization techniques for level set segmentation that target specifically the problem of multiphase segmentation. When the multiphase model is used to obtain a partitioning of the image in more than two regions, a new set of issues arise with respect to the single phase case in terms of regularization strategies. For example, if smoothing or shrinking each contour individually could be a good model in the single phase case, this is not necessarily true in the multiphase scenario. In this paper, we address these issues designing enhanced length and area regularization terms, whose minimization yields evolution equations in which each level set function involved in the multiphase segmentation can “sense” the presence of the other level set functions and evolve accordingly. In other words, the coupling of the level set function, which before was limited to the data term (i.e. the proper segmentation driving force), is extended in a mathematically principled way to the regularization terms as well. The resulting regularization technique is more suitable to eliminate spurious regions and other kind of artifacts. An extensive experimental evaluation supports the model we introduce in this paper, showing improved segmentation performance with respect to traditional regularization techniques.

We present practical approaches for steganography that can provide improved security by closely matching the second-order statistics of the host rather than just the marginal distribution. The methods are based on the framework of statistical restoration, wherein a fraction of the host symbols available for hiding is actually used to restore the statistics; thus reducing the rate, but providing security against steganalysis. We establish correspondence between steganography and the earth-mover's distance (EMD), a popular distance metric used in computer vision applications. The EMD framework can be used to define the optimum flow (modifications) of the host symbols for compensation. This formulation is used for image steganography by restoring the second-order statistics of the blockwise discrete cosine transform (DCT) coefficients. Some practical limitations of this approach (such as computational complexity and difficulty in dealing with overlapping coefficient pairs) are noted, and a new method is proposed that alleviates these deficiencies by identifying the coefficients to modify based on a local compensation criterion. Experimental results on several thousand natural images demonstrate the utility of the presented methods.

In this paper, we present a framework for the design of steganographic schemes that can provide provable security by achieving zero Kullback-Leibler divergence between the cover and the stego signal distributions, while hiding at high rates. The approach is to reserve a number of host symbols for statistical restoration: host statistics perturbed by data embedding are restored by suitably modifying the symbols from the reserved set. A dynamic embedding approach is proposed, which avoids hiding in low probability regions of the host distribution. The framework is applied to design practical schemes for image steganography, which are evaluated using supervised learning on a set of about 1000 natural images. For the presented JPEG steganography scheme, it is seen that the detector is indeed reduced to random guessing.

In steganography (the hiding of data into innocuous covers for secret communication) it is difficult to estimate how much data can be hidden while still remaining undetectable. To measure the inherent detectability of steganography, Cachin suggested the ϵ-secure measure, where ϵ is the Kullback Leibler (K-L) divergence between the cover distribution and the distribution after hiding. At zero divergence, an optimal statistical detector can do no better than guessing; the data is undetectable. The hider's key question then is, what hiding rate can be used while maintaining zero divergence? Though work has been done on the theoretical capacity of steganography, it is often difficult to use these results in practice. We therefore examine the limits of a practical scheme known to allow embedding with zero-divergence. This scheme is independent of the embedding algorithm and therefore can be generically applied to find an achievable secure hiding rate for arbitrary cover distributions.

We consider a wired network of analog sensors in one dimension. Each sensor measures some physical quantity and "communicates" with its adjacent neighbors. The base station can access the network by communicating with the sensors at the boundaries. We show that by making measurements at the boundary (i.e. from the first sensor) we can determine the values of the unknown field values along the entire network stably. The method is based on building an analogy with classical one dimensional inverse scattering (Helmholtz equation) for which the Chen-Rokhlin algorithm provides a highly surprising stable method for recovering the scattering parameters. Simulation results validating the scheme for thousands of sensors are presented.

Print-scan resilient data hiding finds important applications in document security and image copyright protection. This paper proposes methods to hide information into images that achieve robustness against printing and scanning with blind decoding. The selective embedding in low frequencies scheme hides information in the magnitude of selected low-frequency discrete Fourier transform coefficients. The differential quantization index modulation scheme embeds information in the phase spectrum of images by quantizing the difference in phase of adjacent frequency locations. A significant contribution of this paper is analytical and experimental modeling of the print-scan process, which forms the basis of the proposed embedding schemes. A novel approach for estimating the rotation undergone by the image during the scanning process is also proposed, which specifically exploits the knowledge of the digital halftoning scheme employed by the printer. Using the proposed methods, several hundred information bits can be embedded into images with perfect recovery against the print-scan operation. Moreover, the hidden images also survive several other attacks, such as Gaussian or median filtering, scaling or aspect ratio change, heavy JPEG compression, and rows and/or columns removal

The difficult task of steganalysis, or the detection of the presence of hidden data, can be greatly aided by exploiting the correlations inherent in typical host or cover signals. In particular, several effective image steganalysis techniques are based on the strong interpixel dependencies exhibited by natural images. Thus, existing theoretical benchmarks based on independent and identically distributed (i.i.d.) models for the cover data underestimate attainable steganalysis performance and, hence, overestimate the security of the steganography technique used for hiding the data. In this paper, we investigate detection-theoretic performance benchmarks for steganalysis when the cover data are modeled as a Markov chain. The main application explored here is steganalysis of data hidden in images. While the Markov chain model does not completely capture the spatial dependencies, it provides an analytically tractable framework whose predictions are consistent with the performance of practical steganalysis algorithms that account for spatial dependencies. Numerical results are provided for image steganalysis of spread-spectrum and perturbed quantization data hiding.

We investigate data hiding techniques that attempt to defeat steganalysis by restoring the statistics of the composite image to resemble that of the cover. The approach is to reserve a number of host symbols for statistical restoration: host statistics perturbed by data embedding are restored by suitably modifying the symbols from the reserved set. While statistical restoration has broad applicability to a variety of hiding methods, we illustrate our ideas here for quantization index modulation (QIM) based hiding. We propose a method for significantly reducing the detectability of QIM, while preserving its robustness to attacks. We next use the framework of statistical restoration to develop a method to combat steganalysis techniques which detect block-DCT embedding by evaluating the increase in blockiness of the image due to hiding. Numerical results demonstrating the efficacy of these techniques are provided.

This paper proposes a method to hide information into images that achieves robustness against printing and scanning with blind decoding. A significant contribution of this paper is a technique to estimate and undo rotation. The method is based on the fact that laser printers use an ordered digital halftoning algorithm for printing. Using the proposed hiding method, several hundred information bits can be embedded into 512/spl times/512 images with perfect recovery against the print-scan operation. Moreover, the hidden images also survive other attacks such as Gaussian or median filtering, scaling or aspect ratio change, heavy JPEG compression, and rows and/or columns removal.

Quantization index modulation (QIM) techniques have been gaining popularity in the data hiding community because of their robustness and information-theoretic optimally against a large class of attacks. In this paper, we consider detecting the presence of QIM hidden data, which is an important consideration when data hiding is used for covert communication, or steganography. For a given host distribution, we are able to quantify detectability compactly in terms of a parameter related to the robustness of the hiding scheme to attacks. Using detection theory we show that QIM quickly transitions from easily detectable to virtually undetectable as this parameter varies. We also obtain performance benchmarks for QIM hiding in images, indicating that a scheme designed to be robust to say a moderate degree of JPEG compression, should be easily detectable. While practical application of detection theory to images is difficult because of statistical variations across images, we employ supervised learning to show that standard QIM schemes for images are indeed quite easily detectable. However, it remains an open issue as to whether it is possible to devise QIM variants that are less vulnerable to steganalysis.

Information-theoretic analyses for data hiding prescribe embedding the hidden data in the choice of quantizer for the host data. In this paper, we propose practical realizations of this prescription for data hiding in images, with a view to hiding large volumes of data with low perceptual degradation. The hidden data can be recovered reliably under attacks, such as compression and limited amounts of image tampering and image resizing. The three main findings are as follows. 1) In order to limit perceivable distortion while hiding large amounts of data, hiding schemes must use image-adaptive criteria in addition to statistical criteria based on information theory. 2) The use of local criteria to choose where to hide data can potentially cause desynchronization of the encoder and decoder. This synchronization problem is solved by the use of powerful, but simple-to-implement, erasures and errors correcting codes, which also provide robustness against a variety of attacks. 3) For simplicity, scalar quantization-based hiding is employed, even though information-theoretic guidelines prescribe vector quantization-based methods. However, an information-theoretic analysis for an idealized model is provided to show that scalar quantization-based hiding incurs approximately only a 2-dB penalty in terms of resilience to attack.

We consider the problem of hiding images in images. In addition to the usual design constraints such as imperceptible host degradation and robustness in presence of variety of attacks, we impose the condition that the quality of the recovered signature image should be better if the attack is milder. We present a simple hybrid analog-digital hiding technique for this purpose. The signature image is compressed efficiently (using JPEG) into a sequence of bits, which is hidden using a previously proposed digital hiding scheme. The residual error between the original and compressed signature image is then hidden using an analog hiding scheme. The results show (perceptual as well as mean-square error) improvement as the attack becomes milder.

In this paper we consider a hypothesis testing approach for detection of hiding in the least significant bit (LSB). This steganalysis problem is a composite hypothesis testing problem. We state a regularity condition on the image histogram, which reduces this problem to a simple hypothesis testing problem. We then develop a number of simple practical tests based on the estimation of the optimal log likelihood ratio statistic. We show that our tests significantly outperform Stegdetect, a popular hypothesis test available in the literature. Our approach also leads to good estimates of the hiding rate.

Information-theoretic analyses for data hiding prescribe embedding the hidden data in the choice of quantizer for the host data. In this paper, we consider a suboptimal implementation of this prescription, with a view to hiding high volumes of data in images with low perceptual degradation. Our two main findings are as follows: (a) In order to limit perceptual distortion while hiding large amounts of data, the hiding scheme must use perceptual criteria in addition to information-theoretic guidelines. (b) By focusing on “benign” JPEG compression attacks, we are able to attain very high volumes of embedded data, comparable to information-theoretic capacity estimates for the more malicious Additive White Gaussian Noise (AWGN) attack channel, using relatively simple embedding techniques.

This paper presents an algorithm for constructing object representations suitable for recognition. The system automatically selects a representative subset of the views of the object while constructing the eigenspace basis. These views are actively located for object identification and pose determination. All processing is performed on-line. The camera is actively positioned during both representation and recognition. When tested with 240 views for each of seven objects, the system achieves 100% accurate object recognition and pose determination. These results are shown to degrade gracefully as conditions deteriorate.

This paper deals with the important problem of parameter estimation in the presence of bounded data uncertainties. Its recent closed-form solution leads to more meaningful results than alternative methods (e.g., total least-squares and robust estimation), when a priori bounds about the uncertainties are available. The derivation requires the computation of the SVD of the data matrix and the determination of the unique positive root of a nonlinear equation. This paper establishes the existence of a fundamental contraction mapping and uses this observation to propose an approximate recursive algorithm that avoids the need for explicit SVDs and for the solution of the nonlinear equation. Simulation results are included to demonstrate the good performance of the recursive scheme.

We formulate and solve a new parameter estimation problem in the presence of bounded data uncertainties. The new method is suitable when a priori bounds on the uncertain data are available; its solution guarantees that the effect of the uncertainties will never be unnecessarily overestimated beyond what is reasonably assumed by the a priori bounds.