Solving partial differential equations (PDEs) requires precise numerical iterations that impose significant demands on computational resources and memory capacities, which can be addressed by adopting computing-in-memory (CIM) architecture to reduce the latency and power consumption during data transmission. Among PDEs, nonlinear PDEs present heightened complexities in both analytical investigations and numerical simulations as the presence of nonlinear terms introduces intricate dynamics and mathematical intricacies. The high-precision requirements of PDE solvers, particularly for nonlinear PDE solvers, pose challenges in constructing CIM PDE solvers. In this work, a flash-based high-precision PDE solver has been demonstrated to solve the intractable nonlinear partial differential equation. It’s based on 55nm NOR flash technology with well-optimized Program/Erase (PE) schemes. Utilizing the proposed optimization scheme, the PE endurance can be largely enhanced up to$10^{10}$cycles, which is a record high with suppressed cell degradation and robust reliabilities. Then, applying the Fourier neural operator (FNO) to the optimized flash-based high-precision CIM (32-bit) in the hardware system, a series of nonlinear PDEs can be solved with ~2TOPS/W high energy efficiency, which is$\sim 110\times $higher than CPU. Our optimization strategies make it feasible to use flash-based CIM for high-precision computing with frequent weight updating and the demonstrated PDE solver provides an energy-saving solution to implement general-purpose computation tasks.

We propose an image reconstruction approach for highly structured image based on entropy optimization, extending the idea of sparsity with minimal number of non-zero elements to the concept of minimum entropy and simple structure. Minimum entropy signals have unique characteristics of low uncertainty, simple structure, and high concentration, and are usually sufficiently sparse. However, not all low-entropy signals can be linearly sparsified. Compared to a sparse signal in L0or L1norm having one dominant mode centered around 0, a highly structured signal with minimum entropy tends to be concentrated in one or more dominant modes, not necessarily around 0. We study histogram entropy for finite discrete signal and vector entropy for continuous real or complex signal, and propose two algorithms to seek the optimal signal with minimum entropy or low theoretic dimension, subject to the reconstruction constraint. Experiments are carried out on 1-D finite discrete signal, 2-D halftone, industrial and medical images to demonstrate the efficacy.

We propose binary component analysis (BCA) for compact image representation and apply it to fractional dimension reduction and Binaryface representation. BCA is similar to the widely used principal component analysis (PCA), but with the restriction of the base vectors taking binary values of +1 and -1, instead of real values. In a finite set of all binary vectors, the projection of the correlated data onto the binary components (BC) have the largest possible variances. BCA leads to fractional dimension reduction, where binary components successively reduce data variance. The top few BC's capture the largest variances, and each additional BC further reduces the variance by a fraction of the amount reduced in one dimension by PCA. BCA is applied to compact face representation as Binaryface and used for face classification.

Modern photo editing software enables increasingly realistic image manipulations, through splicing, region copy and paste, and content-aware fill. One potential flaw in manipulations is generating realistic object reflections, such as in bodies of water or glass surfaces. Image reflections involve complicated interactions between lighting, surface materials, and geometry, and they can be very hard to fake. Any inconsistency between the directly observed scene and its corresponding reflection is a strong indication of content manipulation. We propose a novel, physical-level image forensic approach to verify the integrity of reflections through identifying mismatches between objects and their reflections. The proposed algorithm consists of reflection-invariant feature detection and matching, geometric transform estimation, and robust change detection between a scene region and its warped reflection region. This approach complements the large body of digital-level forensic methods, with the advantage of being robust to digital-level image transformations such as compression, blurring, and added noise. Experimental results on authentic and manipulated images demonstrate its efficacy.

Tattoos have been increasingly used as a discriminative soft biometric for people identification, such as criminal and victim identification in forensics investigation and law enforcement. However, automatic detection of tattoo images and accurate localization of the regions of interest are challenged by the large variations in artistic composition, color, shape, texture, location on the body, local geometric shape (e.g. neck and finger), imaging conditions, and image quality. In this paper, we train a tattoo detector from the Tatt-C and PASCAL VOC 2007 image datasets using region-based deep learning. The detector can effectively determine if an image contains tattoos and the locations of tattoo regions. We carry out a comprehensive evaluation of our tattoo image classification and detection localization. The detector improves upon the state-of-the-art algorithms in the Tatt-C challenge, achieving a better detection error trade-off curve. It yields low confidence scores on randomly sampled non-tattoo images from 397 scene categories in the MIT-SUN dataset. In addition, the same detector is also validated on the NTU Tattoo Image Dataset with 10000 images.

We describe a system for content-based retrieval from large surveillance video archives, using behavior, action and appearance of objects. Objects are detected, tracked, and classified into broad categories. Their behavior and appearance are characterized by action detectors and descriptors, which are indexed in an archive. Queries can be posed as video exemplars, and the results can be refined through relevance feedback. The contributions of our system include the fusion of behavior and action detectors with appearance for matching; the improvement of query results through interactive query refinement (IQR), which learns a discriminative classifier online based on user feedback; and reasonable performance on low resolution, poor quality video. The system operates on video from ground cameras and aerial platforms, both RGB and IR. Performance is evaluated on publicly-available surveillance datasets, showing that subtle actions can be detected under difficult conditions, with reasonable improvement from IQR.

In this paper, an active power filter (APF) based on a modified one-cycle control is proposed. This control method can be applied in a three-level three-phase four-wire system. By adding an additional current sensor to detect the fundamental component of the output current, the proposed APF is capable of compensating harmonic components independently from the VAR compensation so that the capacity of APF can be greatly reduced. The viability of the proposed scheme is verified by performing experimental validation.

MOTIVATION: Document similarity metrics such as PubMed's 'Find related articles' feature, which have been primarily used to identify studies with similar topics, can now also be used to detect duplicated or potentially plagiarized papers within literature reference databases. However, the CPU-intensive nature of document comparison has limited MEDLINE text similarity studies to the comparison of abstracts, which constitute only a small fraction of a publication's total text. Extending searches to include text archived by online search engines would drastically increase comparison ability. For large-scale studies, submitting short phrases encased in direct quotes to search engines for exact matches would be optimal for both individual queries and programmatic interfaces. We have derived a method of analyzing statistically improbable phrases (SIPs) for assistance in identifying duplicate content. RESULTS: When applied to MEDLINE citations, this method substantially improves upon previous algorithms in the detection of duplication citations, yielding a precision and recall of 78.9% (versus 50.3% for eTBLAST) and 99.6% (versus 99.8% for eTBLAST), respectively. AVAILABILITY: Similar citations identified by this work are freely accessible in the Déjà vu database, under the SIP discovery method category at http://dejavu.vbi.vt.edu/dejavu/.

In this paper, we study disjoint information as a metric for image comparison and its applications in image matching, alignment, and video tracking. Disjoint information is the joint entropy of random variables excluding the mutual information. This measure of statistical dependence and information redundancy satisfies more rigorous metric conditions than mutual information. For image comparison, compound disjoint information is derived from the marginal densities of the image distributions. By using marginal densities other than color histograms, it can overcome the difficulties (such as a lack of spatial information) inherent in histogram-based mutual information methods and enrich the vocabulary of image description. Disjoint information is not sensitive to illumination and appearance changes, and it is particularly suited for multimodal applications.

This paper studies video halftoning that renders a digital video sequence onto display devices, which have limited intensity resolutions and color palettes, by trading the spatiotemporal resolution for enhanced intensity/color resolution. This trade is needed when a continuous tone video is not necessary or not practical for video display, transmission, and storage. In particular, the quantization error of a pixel is diffused to its spatiotemporal neighbors by separable one-dimensional temporal and two-dimensional spatial error diffusions. Motion-adaptive gain control is employed to enhance the temporal consistency of the visual patterns by minimizing the flickering artifacts. Experimental results of halftone and colortone videos are demonstrated and evaluated with various halftoning techniques.

The paper studies video halftoning that renders a digital video sequence onto display devices, which have limited intensity resolutions and color palettes, by trading the spatiotemporal resolution for enhanced intensity/color resolution. Such a trade is needed when a continuous tone video is not necessary or not practical for video display, transmission, and storage. In particular, the quantization error of a pixel is diffused to its spatiotemporal neighbors by separable 1D temporal and 2D spatial error diffusions. Motion-adaptive gain control is employed to enhance the temporal consistency of the visual patterns by minimizing the flickering artifacts. Experimental results of halftone and colortone videos are demonstrated and evaluated with various halftoning techniques.

Many classification tasks can be carried out by casting a domain-specific problem to general graph representation (with objects to be organized as graph nodes and pairwise similarities as graph edges) followed by a graph partition. In this paper, an adaptation scheme is proposed to integrate multiple graphs from various cues to a single graph, such that the distance between the ideal transition probability matrix to the one derived from cue integration is minimized. Four different distance measures, i.e., the Frobenius norm, the Kullback-Leibler directed divergence, the Jeffrey divergence and the cross entropy, are investigated to minimize the discrepancy. It is shown that the minimization leads to a closed-form nonlinear optimization that can be solved by the Levenberg-Marguardt method. Domain and task-specific knowledge is explored to facilitate the generic pattern classification task. Experimental results are demonstrated for image content description by multiple cue integration.

With the advance of digital and information technology, there is an urgent need to make image taking and sharing easy, fun, and cost and time efficient. In this paper, we present a multimedia system to compose visual and audio information into high resolution multimedia slide show, a video compact disc (VCD) compliant bitstream readable by electronic devices such as video CD/DVD players, for TV-centered multimedia rendering, sharing and entertainment. By using the technologies of multimedia composition, multimedia coding, and the sophisticated bit allocation scheme, we can achieve the same spatial resolution of DVD (without full motion). Special efforts are devoted to the multimedia experience, image quality improvement, and automation in the system.

The traditional approach of handling motion tracking and structure from motion (SFM) independently in successive steps exhibits inherent limitations in terms of achievable precision and incorporation of prior geometric constraints about the scene. This paper proposes a projections onto convex sets (POCS) framework for iterative refinement of the measurement matrix in the well-known factorization method to incorporate multiple geometric constraints about the scene, thereby improving the accuracy of both 2D feature point tracking and 3D structure estimates. Regularities in the scene, such as points on line and plane and parallel lines and planes, that can be interactively identified and marked at each POCS iteration, enforce rank and parallelism constraints on appropriately defined local measurement matrices, one for each constraint. The POCS framework allows for the integration of the information in each of these local measurement matrices into a single measurement matrix that is "closest" to the initial observed measurement matrix in Frobenius norm, which is then factored in the usual manner. Experimental results demonstrate that the proposed interactive POCS framework consistently improves both 2D correspondences and 3D shape/motion estimates and similar results cannot be achieved by enforcing these constraints as either post or preprocessing.

A variable-coefficient fixed-length coding scheme is proposed for wavelet-based image transmission over noisy channels. Multiple subband coefficients are grouped into an extended source symbol and coded as a fixed-length code word such that the extended source symbols have almost the same probabilities. Part of the codebook is fixed based on the observation of the coefficient spatial distribution patterns in each subband to alleviate the transmission of the codebook. The remaining code positions within the fixed-length codebook can be utilized to combat channel errors by carefully arranging the code positions or be filled with other frequently appearing coefficient sequences to achieve higher compression ratio.

We propose a new parametric motion model based on the so-called "trifocal tensor" representation, which captures rigid 3D motion of static scenes with a depth of field. The estimation of the trifocal tensor requires solution of a set of linear equations given at least seven point correspondences across three frames. The proposed parametric representation, called the trilinear model, is superior to other forms such as translational, affine, perspective, and bilinear models, because it can implicitly encode the depth of the scene and 3D motion of the scene/camera under perspective projection unlike others. A video object can thus be represented by its first VOP, a set of trifocal tensors and the corresponding prediction residues. Motion estimation and compensation based on the new parametric model are incorporated into the MPEG-4 Video Verification Model to compare its efficacy for object-based video compression with the state-of-the art motion compensation methods. Experimental results are provided to demonstrate the performance of the trilinear model for object-based video compression.

We propose a trifocal motion model, which captures rigid scene/camera motion and 3-D scene structure under perspective projection, to register multiple images without camera calibration. Three images/frames are processed at a time to register the current frame F/sub i/ with a common reference frame R by using a common correspondence frame C involving only linear operations. The registration method requires estimation of 27 trifocal tensor parameters between every tripled of frames (R, C, F/sub i/), and a dense correspondence field between the frames R and C. The proposed image registration method can be employed in mosaic synthesis, superresolution, and video standard conversion. The superiority of the proposed trifocal image registration over affine (6 parameters) and perspective (8 parameters) registration is demonstrated by experimental results.

Following an overview of two-dimensional (2-D) parametric motion models commonly used in video manipulation and compression, we introduce trifocal transfer, which is an image-based scene representation used in computer vision, as a motion compensation method that uses three frames at a time to implicitly capture camera/scene motion and scene depth. Trifocal transfer requires a trifocal tensor that is computed by matching image features across three views and a dense correspondence between two of the three views. We propose approximating the dense correspondence between two of the three views by a parametric model in order to apply the trifocal transfer for object-based video compression and background mosaic generation. Backward, forward, and bidirectional motion compensation methods based on trifocal transfer are presented. The performance of the proposed motion compensation approaches using the trifocal model has been compared with various other compensation methods, such as dense motion, block motion, and global affine transform on several video sequences. Finally, video compression and mosaic synthesis based on the trifocal motion model are implemented within the MPEG-4 Video Verification Model (VM), and the results are compared with those of the standard MPEG-4 video VM. Experimental results show that the trifocal motion model is superior to block and affine models when there is depth variation and camera translation.