We propose Few-Class Arena (FCA), as a unified benchmark with focus on testing efficient image classification models for few classes. A wide variety of benchmark datasets with many classes (80-1000) have been created to assist Computer Vision architectural evolution. An increasing number of vision models are evaluated with these many-class datasets. However, real-world applications often involve substantially fewer classes of interest (2-10). This gap between many and few classes makes it difficult to predict performance of the few-class applications using models trained on the available many-class datasets. To date, little has been offered to evaluate models in this Few-Class Regime. We conduct a systematic evaluation of the ResNet family trained on ImageNet subsets from 2 to 1000 classes, and test a wide spectrum of Convolutional Neural Networks and Transformer architectures over ten datasets by using our newly proposed FCA tool. Furthermore, to aid an up-front assessment of dataset difficulty and a more efficient selection of models, we incorporate a difficulty measure as a function of class similarity. FCA offers a new tool for efficient machine learning in the Few-Class Regime, with goals ranging from a new efficient class similarity proposal, to lightweight model architecture design, to a new scaling law. FCA is user-friendly and can be easily extended to new models and datasets, facilitating future research work. Our benchmark is available at https://github.com/bryanbocao/fca.

We design a video analytics system to measure gait over time and detect trend and outliers in the data. The purpose is for health monitoring, the thesis being that trend especially can lead to early detection of declining health and be used to prevent accidents such as falls in the elderly. We use the OpenPose deep learning tool for recognizing the back and neck angle features of walking people, and measure speed as well. Trend and outlier statistics are calculated upon time series of these features. A challenge in this work is lack of testing data of decaying gait. We first designed experiments to measure consistency of the system on a healthy population, then analytically altered this real data to simulate gait decay. Results on about 4000 gait samples of 50 people over 3 months showed good separation of healthy gait subjects from those with trend or outliers, and furthermore the trend measurement was able to detect subtle decay in gait not easily discerned by the human eye.

There are many video analysis applications in which the flow of people or traffic over time is of more interest than tracking the discrete objects contained. Motion pattern maps, loosely called heat maps, can show accumulated activity over time via temporal filtering. In this paper, we examine three parameters that control this filtering with respect to: event resolution, time-merging of events, and dynamic background. Design of these parameters is detailed and their effects are shown on simulated videos. Design considerations for real video events are discussed and shown by example.

For IoT networks, signals are captured by edge sensors, system-wide decisions are made at a central host, and processing may be performed at either or both sides of the network. Decision on edge or cloud placement of methods depends in part on processing and bandwidth efficiencies. Video analytics offers a good application to investigate this because signals from edge cameras are large both spatially and temporally, and processing usually involves a sequence of methods. For public camera analytics where processing and bandwidth need only be expended for active periods, we find by experiment a data range for crowd, traffic, and building scenes. For a typical sequence of video analytics methods applied to surveillance, we find upper bounds for processing and bandwidth, and experimental measurements for real - much sparser - video. Explicit description of algorithmic requirements and knowledge of experimentally determined loads gives information to balance methods between edge and cloud.

Spatiotemporal activity maps have been used to visualize where activity occurs over time, and are often displayed as pseudo-color heat maps. Our multi-dimensional activity map includes the following motion features: density, direction, bi-direction, velocity, and dwell. The primary contribution of this paper is to describe a set of mappings that will transform the activity maps captured from the cameras of different perspectives to the ones from a single orthographic perspective. The purpose of this is to be able to view and compare multiple activity maps from different camera views over a wide area with consistently comparable data. A second contribution is that the most mappings are based upon statistically learned camera perspectives to minimize manual camera calibration. We demonstrate mapping results with multiple video data sets and describe applications for visualization and wide-area spatial probability estimation.

Spatio-temporal activity maps have been used to visualize where activity occurs over time, as captured by a camera, and have typically been displayed as heat maps. We extend this work in two ways. 1) Multiple maps from many cameras in a network are geometrically translated to create a unified, wide-area map whose viewing angle and relative feature values are consistent to all cameras. 2) Activity features are extended beyond density, to include direction, bi-direction, and dwell time. Experimental results are shown to illustrate the types of transformations and corrections needed for unified multi-camera display, and their accuracy.

Public camera feeds are increasingly being opened to use by multiple authorities (e.g., police, fire, traffic) as well as to the public. Because of the difficulty and insecurity of sharing cryptographic keys, these data are available in the clear. However, authorities must have a mechanism to assure trust in the video, that is, to authenticate it. While lossless video is straightforward to authenticate by cryptographic means, lossy video as may result from UDP, wireless, or transcoded transmissions, is more difficult to authenticate. We describe a method that combines a concise and efficiently computed video fingerprint with public key cryptography. Essential components of our approach are: the procedure to combine inexact video fingerprint with exact digital signature to enable lossy authentication, and matching for misaligned video via a major incline approach. Experimental results relate video fingerprint length to authentication accuracy and latency (time to authentication).

For videoconferencing, there are network bandwidth and screen real-estate constraints that limit the number of user channels. We propose an intermediate transmission mode that transmits only at "events", where these are detected by both audio and video changes from the short-term signal average. Our objective in this paper is to determine latency until the audio portion of a single telepresence channel stabilizes. It is this stable signal from which we detect events. We describe a recursive filter approach for feature determination and experiments on the Switchboard telephone call database. Results show latency to stable signal of up to 10 seconds. Although events can be detected much more quickly (<;;1 sec) if the signal is already stable, this latency time must be considered at the start of a conversation or at changes in the short-term average.

The guest editors provide an overview of the articles selected for this special issue. The issue's goal is to document the current state-of-the-art, acknowledge the latest breakthroughs achieved by scientists working in the area of biometric recognition, and identify future promising research areas. It is thought the selection of papers discussed should give readers a good idea of where researchers have been focusing, both on long- studied problems still needing more work and on newer challenges. A fundamental of the field of biometrics is an ever-increasing need for better recognition and stronger security. But, as public and commercial biometric deployments increase in number, there is also more need to understand privacy issues and to provide greater ease-of-use. The volume and quality of papers in this special issue indicate that much progress has been made in many aspects of the biometrics field and that there are challenging and promising future directions still to follow.

For decades, the password has been the standard means for user authentication on computers. However, as users are required to remember more, longer, and changing passwords, it is evident that a more convenient and secure solution to user authentication is necessary. This paper examines passwords, security tokens, and biometrics-which we collectively call authenticators-and compares these authenticators and their combinations. We examine their effectiveness against several attacks and suitability for particular security specifications such as compromise detection and nonrepudiation. Examples of authenticator combinations and protocols are described to show tradeoffs and solutions that meet chosen, practical requirements. The paper endeavors to offer a comprehensive picture of user authentication solutions for the purposes of evaluating options for use and identifying deficiencies requiring further research.

We present an approach to authenticating photo-ID documents that relies on pattern recognition and public-key cryptography and has security advantages over physical mechanisms that currently safeguard cards, such as optical laminates and holograms. The pattern-recognition component of this approach is based on a photo signature that is a concise representation of the photo image on the document. This photo signature is stored in a database for remote authentication or in encrypted form on the card for stand-alone authentication. Verification of the ID document entails scanning both the photo image and a machine-readable form of the text information, determining the photo signature, and comparing this information against the reference. In this paper, we describe the method and present results of testing a large database of images for photo-signature match in the presence of noise.

This correspondence deals with the problem of designing planar shapes for subpixel image registration. Basic theoretical considerations are shown to lead to a lower bound on location accuracy. Optimal registration marks achieving this bound are discussed. These optimal designs, however, require very high printing or etching resolution and are inherently very sensitive to variations in the image sampling model (like scaling of grid size and rotation). More robust, optimal and suboptimal "topology-preserving" registration marks are then introduced and analyzed.

Personal identification documents such as ID-cards are being used increasingly for such activities as cashless transactions, entry into restricted areas, and personal verification for medical and government services. With this trend, there is also the increasing trend toward counterfeiting these documents for illegal gain. We present an approach to authenticating photo-ID documents that relies on pattern recognition and cryptography, and has cost and security advantages over physical mechanisms such as holograms that are currently used. The approach is based on a photo-signature that is a concise representation of the photo-image on the document. This photo-signature is stored in a database for remote verification or in encrypted form on the card for self-verification. Verification of the ID document entails scanning both the photo-image and a machine-readable form of the text information, determining the photo-signature, and comparing this information against the reference. In this paper, we describe the method, and present results of testing a large database of images for photo-signature match in the presence of noise.

The precision by which a region is located or measured on the image plane is limited by the sampling density. In this paper, the worst-case precision errors are determined for calculating the average image location of an edge, line, and straight-edged region. For each case, it is shown how the worst-case error can be minimized as a function of the geometric parameters. These results can be used to determine the worst case error by which the location of a known shape is measured. Another application is to design shapes for use in registration, such as fiducial marks used in electronic assembly. The main conclusion of this paper is that, to achieve better precision, measurement of a straight-edged region should be made at an angle askew to the sampling axis (not 0, 45, or 90 degrees) and this should be at a certain length that is a function of this skew angle.

Modern computer networks make it possible to distribute documents quickly and economically by electronic means rather than by conventional paper means. However, the widespread adoption of electronic distribution of copyrighted material is currently impeded by the ease of unauthorized copying and dissemination. In this paper we propose techniques that discourage unauthorized distribution by embedding each document with a unique codeword. Our encoding techniques are indiscernible by readers, yet enable us to identify the sanctioned recipient of a document by examination of a recovered document. We propose three coding methods, describe one in detail, and present experimental results showing that our identification techniques are highly reliable, even after documents have been photocopied.>

A major impediment to the widespread adoption of services for electronic distribution of copyrighted material is the ease with which illicit copies can be made and disseminated. In this paper, we describe feature coding techniques to mark document images with codes that are indiscernible by readers but can be decoded by document feature analysis techniques. These codes can be used to trace the source of errant documents. We propose three coding methods: line-shift coding, word-shift coding, and character coding. We describe how the coding features are found, inserted, and decoded. Finally, we show preliminary experimental results indicating robustness of the line-shift coding method where decoding can be performed even from images of up to ten generations of photocopying.

Thresholding methods are applied here to document images and their experimental results compared. In one set of tests, different thresholding methods are used to binarize document images, then optical character recognition (OCR) is performed on the resulting text and the recognition results are compared. In the other set of tests, multi-thresholding is performed on document images-to obtain three or more levels for images with more than binary levels-and the results are compared. Four thresholding methods are compared in the experiments: a discriminant analysis method, a maximum entropy method, a moment-preserving method, and a connectivity-preserving method. A method using a minimum-error criterion is also commented upon. The moment-preserving and connectivity-preserving methods are found to yield the best OCR results from the binarized images, and the connectivity-preserving method yields the fewest binarization and multi-thresholding failures.

Modern computer networks make it possible to distribute documents quickly and economically by electronic means rather than by conventional paper means. However, the widespread adoption of electronic distribution of copyrighted material is currently impeded by the ease of illicit copying and dissemination. The authors propose techniques that discourage illicit distribution by embedding each document with a unique codeword. The encoding techniques are indiscernible by readers, yet enable one to identify the sanctioned recipient of a document by examination of a recovered document. The authors propose three coding methods, describe one in detail, and present experimental results showing that their identification techniques are highly reliable, even after documents have been photocopied.>

The AT&T Bell Laboratories Library Network is striving to fulfill its vision of providing all AT&T professional employees with an “electronic window” to meet their information needs. People must be able to access information quickly and easily without the fear of information overload. Taking these needs and requirements into account, the Library and computer researchers combined their interests and areas of expertise to create The RightPages™ Service. This service brings the library to users' desktop work-stations by incorporating ASCII text, bitmap images, and an alerting service, in a system that is comfortable and efficient to use. © 1993 John Wiley & Sons, Inc.

Page layout analysis is a document processing technique used to determine the format of a page. This paper describes the document spectrum (or docstrum), which is a method for structural page layout analysis based on bottom-up, nearest-neighbor clustering of page components. The method yields an accurate measure of skew, within-line, and between-line spacings and locates text lines and text blocks. It is advantageous over many other methods in three main ways: independence from skew angle, independence from different text spacings, and the ability to process local regions of different text orientations within the same image. Results of the method shown for several different page formats and for randomly oriented subpages on the same image illustrate the versatility of the method. We also discuss the differences, advantages, and disadvantages of the docstrum with respect to other lay-out methods.>

Describes some of the document processing techniques used in the RightPages electronic library system. Since the system deals with scanned images of document pages, these techniques are critical to the use and appearance of the system. The author describes three techniques: (1) for noise reduction from binary document pages to improve page appearance and subsequent optical character recognition and compression; (2) for subsampling the text image to fit on the computer screen white maintaining readability; and (3) a document layout analysis technique to determine text blocks.>

An examination of the effects of spatial sampling and image noise on the precision with which the centroids of different geometric shapes can be determined is presented. The concentric ring fiducial-a bull's-eye pattern-is identified as having desirable qualities of high location precision and rotational invariance. The performance of the concentric fiducial, as a function of diameter, number of rings. and ring spacing, has been tested, and these results are shown.>

A common approach to finding features in digitized lines is to estimate the curvature along the lines and determine the features from the curvature plot. The authors compare two approaches to curvature estimation by analyzing performance with respect to signal-to-noise ratio and signal localization of corner features. One approach, curvature estimation by difference of slopes, is analyzed to determine the spacing between slope estimates which yields optimum signal-to-noise ratio. The other approach, Gaussian smoothing of the second line derivative, is compared with the difference of slopes method and found to yield poorer signal localization for low signal-to-noise ratio. Besides analytical comparisons, the methods are tested and compared for digitized lines containing chosen corner angles and random noise. These empirical comparisons corroborate the analytical results.>

Proposes an application-specific IC architecture for implementing a contour line filter for a real-time fingerprint recognition system. The architecture demonstrates the fact that a library-based set of data-path modules which are steered by powerful, synchronized controllers can result in real-time performance on a single chip, even for a complex image processing application. Moreover, the external frame memories required are minimized in the design by working in-place and by appropriate scheduling at the cost of little extra hardware and few additional evaluation cycles.>

The primitives chain code (PCC) is described as an extension of the Freeman chain code, with each PCC codeword representing either a portion of a zero- to three-pixel chain or an endline, junction, or break feature. A simple context-free syntax is described which enables PCC to maintain inherent two-dimensional branching of the line structures. The compression efficiency of PCC is attributed primarily to use of this syntax. Examples are given showing comparisons of coding efficiency for different coding schemes. Advantage is taken of PCC's compact and feature-based code for aid in image analysis. PCC is described as the lowest level of the thin line code (TLC). TLC is a hierarchy of data structures for use in image analysis of line images. As such, PCC is the link between the image space and feature space. Examples are described of applications such as noise reduction and feature recognition performed on PCC and other TLC levels, and it is shown how analysis is facilitated by the knowledge of context and features.>

A procedure for filter design is described for enhancing fingerprint images. Four steps of this procedure are described: user specification of appropriate image features, determination of local ridge orientations throughout the image, smoothing of this orientation image, and pixel-by-pixel image enhancement by application of oriented, matched filter masks. The contribution of this work is to quantify and justify the functional relationships between image features and filter parameters so that the design process can be easily modified for different conditions of noise and scale. Application of the filter shows good ridge separation, continuity, and background noise reduction.>

In [1], the converging squares algorithm was introduced as a method designed to effectively and efficiently locate peaks in data of two dimensions or higher. In this correspondence, the performance of the algorithm on a signal in noise is examined, and some extensions of the algorithm-beyond peak-picking-are introduced. The minimum-area enclosing square is one extension, which locates an image region in a uniform background, and finds the smallest square which entirely encloses it. The maximum-difference enclosing square is another extension by which a global feature of the image is found which separates it into a foreground square region and background region, based on the maximum statistical difference between the two. Some applications of these extensions are shown, including object location, tracking of a moving object, and adaptive binarization.

This paper describes a method for segmentation of convex shaped image regions. The wedge filter technique first employs the converging squares algorithm [1] to locate a region of interest. Then a region oriented boundary estimation technique, called the wedge filter, is applied. This wedge filter entails angular filtering and subsampling, and boundary interpolation. The technique is more capable of segmenting noncircular shapes than some earlier methods based on the Hough transform. In addition, unlike many edge-based segmentation schemes, this method is relatively tolerant to edge gaps and to blurred or thick edges. This technique is tested on a number of synthesized images over a range of convex shapes, for different algorithm parameters, and under various conditions of region size and image noise. In addition, the technique has been applied to segmentation of liver cell nuclei in light microscope images of human liver tissue.

The converging squares algorithm is a method for locating peaks in sampled data of two dimensions or higher. There are two primary advantages of this algorithm over conventional methods. First, it is robust with respect to noise and data type. There are no empirical parameters to permit adjustment of the process, so results are completely objective. Second, the method is computationally efficient. The inherent structure of the algorithm is that of a resolution pyramid. This enhances computational efficiency as well as contributing to the quality of noise immunity of the method. The algorithm is detailed for two-dimensional data, and is described for three-dimensional data. Quantitative comparisons of computation are made with two conventional peak picking methods. Applications to biomedical image analysis, and for industrial inspection tasks are discussed.

The converging squares algorithm is a method for locating peaks in sampled data of 2 dimensions or higher. There are two primary advantages of this algorithm over other conventional methods. First, it is robust with respect to noise and data type. There are no empirical parameters to allow adjustment of the process, so results are completely objective. Secondly, the method is computationally efficient. The inherent structure of the algorithm is that of a resolution pyramid. This enhances computational efficiency as well as contributing to the quality of noise immunity of the method. The algorithm is detailed for 2-dimensional data. Quantitative comparisons of computation are made with two conventional peak picking methods.