Abstract Recently, Digital Twins solutions have attracted a growing interest as a fundamental paradigm for managing data driven processes on smart cities. They are complex modelling that should include 3D interactive representations of buildings and infrastructures, integrated with a wide range of data for Smart City cyber-physical ecosystem monitoring and controlling. This paper presents a framework for modelling, generating and distributing Digital Twin representations with 3D models from a various set of data, as well as its integration into the open-source Smart City framework, where many kinds of real time and historical data are available. The proposed solution offers a method for creating integrated data rendering of 3D city entities coupled with Smart City data (e.g., IoT Devices with time-series and historical data, heatmaps, geometries and shapes related to traffic flows, bus routes/stops, cycling paths). The solution for generating 3D representation is based on a number of computer vision and machine learning solutions, thus shortening the activities of passing from raw data (i.e., Lidar, shapes, patterns, etc.) to 3D representations. Implementation has been enforced into the quite widespread open-source Snap4City Smart City platform and has been validated by using hundreds of buildings in Florence city central area, Italy, plus hundreds of thousands of data as points of interest, IoT Devices, traffic flows, dynamic heatmaps, etc.

The field of 3D city modelling has attracted a growing interest for representing the city digital twin, providing interactive visualizations of building infrastructures integrated with a wide range of data typically produced in a Smart City environment. This paper presents a method for producing a 3D city model with photorealistic rooftop textures extracted from aerial images, as well as the integration of the 3D city model into an open-source Smart City framework. The proposed solution provides a smart visualization of 3D city entities integrated with a large variety of Smart City data (coming, for instance, from IoT Devices which generate time-series data, heatmaps, geometries and shapes related to traffic flows, bus routes, cycling paths etc.). The proposed method for rooftop detection and alignment follows a deep learning approach based on U-Net architecture, and it has been validated against a manually created ground-truth of 50 buildings scattered uniformly on the covered area. The solution is implemented in the open-source Snap4City Smart City platform. DEMO: The solution will be demonstrated and the readers and conference attendees will be capable to interact and work with the tools by using the links provided.

In the context of Smart Cities digital transformation, the field of 3D city modelling has attracted a growing interest for representing the city digital twin. This paper presents a method for producing a 3D city model with photorealistic rooftop textures extracted from aerial images, as well as the integration of the 3D city model into an open-source Smart City framework. The proposed solution provides a smart visualization of 3D city entities integrated with a large variety of Smart City data (coming, for instance, from IoT Devices which generate time-series data, heatmaps, geometries and shapes related to traffic flows, bus routes, cycling paths etc.). The proposed method for rooftop detection and alignment follows a deep learning approach based on U-Net architecture, and it has been validated against a manually created ground-truth of 50 buildings scattered uniformly on the covered area. The solution is implemented in the open-source Snap4City Smart City platform.

This study introduces an extension of the sGLOH2 local image descriptor inspired by RootSIFT ‘square rooting’ as a way to indirectly alter the matching distance used to compare the descriptor vectors. The extended descriptor, named RootsGLOH2, achieved the best results in terms of matching accuracy and robustness among the latest state‐of‐the‐art non‐deep descriptors in recent evaluation contests dealing with both planar and non‐planar scenes. RootsGLOH2 also achieves a matching accuracy very close to that obtained by the best deep descriptors to date. Beside confirming that ‘square rooting’ has beneficial effects on sGLOH2 as it happens on SIFT, experimental evidence shows that classical norm‐based distances, such as the Euclidean and Manhattan distances, only provide suboptimal solutions to the problem of local image descriptor matching. This suggests matching distance design as a topic to investigate further in the near future.

This paper proposes a novel approach for registering the PRNU pattern between different camera acquisition modes by relying on the imaged scene content. First, images are aligned by establishing correspondences between local descriptors: The result can then optionally be refined by maximizing the PRNU correlation. Comparative evaluations show that this approach outperforms those based on brute-force and particle swarm optimization in terms of reliability, accuracy and speed. The proposed scene-based approach for PRNU pattern alignment is suitable for video source identification in multimedia forensics applications.

sGLOH (shifting GLOH) is a histogram-based keypoint descriptor that can be associated to multiple quantized rotations of the keypoint patch without any recomputation. This property can be exploited to define the best distance between two descriptor vectors, thus avoiding computing the dominant orientation. In addition, sGLOH can reject incongruous correspondences by adding a global constraint on the rotations either as an a priori knowledge or based on the data. This paper thoroughly reconsiders sGLOH and improves it in terms of robustness, speed and descriptor dimension. The revised sGLOH embeds more quantized rotations, thus yielding more correct matches. A novel fast matching scheme is also designed, which significantly reduces both computation time and memory usage. In addition, a new binarization technique based on comparisons inside each descriptor histogram is defined, yielding a more compact, faster, yet robust alternative. Results on an exhaustive comparative experimental evaluation show that the revised sGLOH descriptor incorporating the above ideas and combining them according to task requirements, improves in most cases the state of the art in both image matching and object recognition.

This paper introduces a new compositional framework for classifying color correction methods according to their two main computational units. The framework was used to dissect fifteen among the best color correction algorithms and the computational units so derived, with the addition of four new units specifically designed for this work, were then reassembled in a combinatorial way to originate about one hundred distinct color correction methods, most of which never considered before. The above color correction methods were tested on three different existing datasets, including both real and artificial color transformations, plus a novel dataset of real image pairs categorized according to the kind of color alterations induced by specific acquisition setups. Differently from previous evaluations, special emphasis was given to effectiveness in real world applications, such as image mosaicing and stitching, where robustness with respect to strong image misalignments and light scattering effects is required. Experimental evidence is provided for the first time in terms of the most recent perceptual image quality metrics, which are known to be the closest to human judgment. Comparative results show that combinations of the new computational units are the most effective for real stitching scenarios, regardless of the specific source of color alteration. On the other hand, in the case of accurate image alignment and artificial color alterations, the best performing methods either use one of the new computational units, or are made up of fresh combinations of existing units.

This paper addresses the problem of classifying human actions in a video sequence. A representation eigenspace approach based on the PCA algorithm is used to train the classifier according to an incremental learning scheme based on a “one action, one eigenspace” approach. Before dimensionality reduction, a high dimensional description of each frame of the video sequence is constructed, based on foreground blob analysis. Classification is performed by matching incrementally the reduced representation of the test image sequence against each of the learned ones, and accumulating matching scores according to a probabilistic framework, until a decision is obtained. Experimental results with real video sequences are presented and discussed.

A single-camera iris-tracking and remapping approach based on passive computer vision is presented. Tracking is aimed at obtaining accurate and robust measurements of the iris/pupil position. To this purpose, a robust method for ellipse fitting is used, employing search constraints so as to achieve better performance with respect to the standard RANSAC algorithm. Tracking also embeds an iris localization algorithm (working as a bootstrap multiple-hypotheses generation step), and a blink detector that can detect voluntary eye blinks in human-computer interaction applications. On-screen remapping incorporates a head-tracking method capable of compensating for small user-head movements. The approach operates in real time under different light conditions and in the presence of distractors. An extensive set of experiments is presented and discussed. In particular, an evaluation method for the choice of layout of both hardware components and calibration points is described. Experiments also investigate the importance of providing a visual feedback to the user, and the benefits gained from performing head compensation, especially during image-to-screen map calibration.

We describe a low cost system for metric 3D scanning from uncalibrated images based on rotational kinematic constraints. The system is composed by a turntable, an offthe- shelf camera and a laser stripe illuminator. System operation is based on the construction of the virtual image of a surface of revolution (SOR), from which two imaged SOR cross-sections are obtained in an automatic way, and internal camera calibration is performed by exploiting the same object being scanned. Shape acquisition is finally obtained by laser profile rectification and collation. Experiments with real data are shown, providing an insight into both camera calibration and shape reconstruction performance. System accuracy appears to be adequate for desktop applications.

Image analysis and computer vision can be effectively employed to recover the three-dimensional structure of imaged objects, together with their surface properties. In this paper, we address the problem of metric reconstruction and texture acquisition from a single uncalibrated view of a surface of revolution (SOR). Geometric constraints induced in the image by the symmetry properties of the SOR structure are exploited to perform self-calibration of a natural camera, 3D metric reconstruction, and texture acquisition. By exploiting the analogy with the geometry of single axis motion, we demonstrate that the imaged apparent contour and the visible segments of two imaged cross sections in a single SOR view provide enough information for these tasks. Original contributions of the paper are: single view self-calibration and reconstruction based on planar rectification, previously developed for planar surfaces, has been extended to deal also with the SOR class of curved surfaces; self-calibration is obtained by estimating both camera focal length (one parameter) and principal point (two parameters) from three independent linear constraints for the SOR fixed entities; the invariant-based description of the SOR scaling function has been extended from affine to perspective projection. The solution proposed exploits both the geometric and topological properties of the transformation that relates the apparent contour to the SOR scaling function. Therefore, with this method, a metric localization of the SOR occluded parts can be made, so as to cope with them correctly. For the reconstruction of textured SORs, texture acquisition is performed without requiring the estimation of external camera calibration parameters, but only using internal camera parameters obtained from self-calibration.

We present a novel approach to obtain a mosaic image for the surface texture content of a surface of revolution (SOR) from a collection of uncalibrated views. The SOR scene constraint is used to calibrate each view and align the corresponding pictorial content into a global representation. Metric surface properties are extracted from each view by exploiting special properties of the imaged SOR geometry expressed in terms of homologies. Image alignment is achieved by projecting imaged surface elements onto a reference plane, and then registering them according to a translational motion model. This work extends previous research on calibrated scenes of right circular cylinders to the more general case of uncalibrated SOR scenes. Experimental results with images taken from the web demonstrate the effectiveness and the general applicability of the approach. 1 Introduction and Related Work Image mosaicing consists in merging collections of images having a partially overlapping content. The process can be decomposed into three main steps. First, the transformations

Broadcasters are demonstrating interest in systems that ease the process of annotation the huge amount of live and archived video materials. Exploitation of such assets is considered a key method for the improvement of production quality, and sport videos are one of the most marketable assets. In particular, in Europe, soccer is one of the most relevant sport types. This paper deals with detection and recognition of soccer highlights, using an approach based on temporal logic models.

We present a nonintrusive system based on computer vision for human-computer interaction in three-dimensional (3-D) environments controlled by hand pointing gestures. Users are allowed to walk around in a room and manipulate information displayed on its walls by using their own hands as pointing devices. Once captured and tracked in real-time using stereo vision, hand pointing gestures are remapped onto the current point of interest, thus reproducing in an advanced interaction scenario the "drag and click" behavior of traditional mice. The system, called PointAt (patent pending), enjoys a careful modeling of both user and optical subsystem, and visual algorithms for self-calibration and adaptation to both user peculiarities and environmental changes. The concluding sections provide an insight into system characteristics, performance, and relevance for real applications.

We describe a geometric approach for reconstructing 3D textured graphical models of surfaces of revolution (SOR) from a single uncalibrated view. Metric reconstruction of 3D shape is complemented with the extraction of flattened 2D texture, so as to support visual retrieval from 2D/3D cues and to generate realistic 3D visualization models. The approach developed is quite simple, yet accurate and robust; its applications range from the preservation, analysis and classification of cultural heritage, to advanced graphics and multimedia.

This paper describes a system for advanced man-machine interaction based on computer vision technology allowing users to manipulate information displayed on large wall panels by using their own hands as pointing devices. The system gets its input from a pair of color video cameras (placed so as to have the user in view); a personal computer performs image analysis and updates interaction parameters. Such parameters, which reflect current user status, are then transformed into graphic interface commands and output to the screen through a beamer. Graphic interface operation is the same as with standard computer mice; the main points of innovation concern naturality of interaction, low intrusiveness and a priori training, and low equipment cost. An experimental version of the system is currently being employed to provide museum visitors with advanced interactive services.

Broadcasters are demonstrating interest in building digital archives of their assets for reuse of archive materials for TV programs, on-line availability, and archiving. This requires tools for video indexing and retrieval by content exploiting high-level video information such as that contained in super-imposed text captions. In this paper we present a method to automatically detect and localize captions in digital video using temporal and spatial local properties of salient points in video frames. Results of experiments on both high-resolutionDV sequences and standard VHS videos are presented and discussed. 1.

This paper addresses the problem of estimating time to collision from focal motion field measurements in the case of unconstrained relative rigid motion and surface orientation. It is first observed that, as long as time to collision is regarded as a scaled depth, the above problem does not admit a solution unless a narrow camera field of view is assumed. By a careful generalization of the time to collision concept, it is then expounded how to compute novel solutions which hold however wide the field of view. The formulation, which reduces to known literature approaches in the narrow field of view case, extends the applicability range of time to collision based techniques in areas such as mobile robotics and visual surveillance. The experimental validation of the main theoretical results includes a comparison of narrow- and wide-field of view time to collision approaches using both dense and sparse motion estimates.

We investigate the exploitation of linear models of camera-object interaction for an efficient modeling and control of image-based visual servoing systems. The approach includes a method for coping with those representation ambiguities typical of linear interaction models which may affect both planning and control. The implementation of an eye-in-hand servoing system based on affine camera models and using image contours as relevant visual features is described and discussed. The system, including an image planner, a two-dimensional/three-dimensional (2-D/3-D) controller, and a visual analysis module, allows an intuitive specification and execution of relative positioning tasks w.r.t. still or moving rigid objects. Results of real-time experiments with a robotic platform featuring a PUMA manipulator provide a further insight into characteristics and performance of affine visual servoing systems.

We present an approach for the design and control of both reflexive and purposive visual tasks. The approach is based on the bidimensional appearance of the objects in the environment and explicitly takes into account independent object motions. A linear model of camera-object interaction is embedded in the control scheme, which dramatically simplifies visual analysis and control by reducing the size of visual representation. We describe the implementation of three visual tasks of increasing complexity, obtained with the proposed scheme and based on the active contour analysis and polynomial planning of image contour transformations. Both simulations and real-time experiments with a robotic eye-in-hand configuration are discussed, validating the approach in terms of robustness and applicability to visual navigation, active exploration and perception, and human-robot interaction.

Videos convey information through several planes of communication, encompassing what is represented in the images how the images are linked together and how the subject is imaged. This feature is stressed in commercials where colors, editing effects, rhythms, and object motion are exploited to influence human purchasing habits. In this paper, based on researches in the marketing field, a link is formalized between low level features of a commercial video and feelings that the video would inspire in the observer. This link is used to define high level indices capturing the main semantics of the video. These indices are embedded in a video retrieval system to support access to a database of video based on their semantics.

In this paper, a framework for vision-based relative positioning called affine visual servoing (AVS) is presented, in which the changes in shape of image contours are used in order to control the positioning process. A feedforward control strategy complements the feedback loop, thus enhancing both speed and the overall performance of the system. The full linearization of the visual servoing problem results in a dramatical reduction of the computational burden for both the control and vision subsystems. Good simulation results with stable behavior show that the approach may be successfully applied for the design of complex positioning tasks, such as navigating by means of natural landmarks, or mimicking human gestures.

This paper describes preliminary work on a non-intrusive gaze-driven interface for man-machine interaction based on vision techniques. The current computer screen location pointed at by the user is evaluated using a linear model of image-to-screen mapping, which can be easily auto-calibrated at run-time on the basis of the current relative position of camera and user and camera settings. A simple active deformable model of the eye is defined, which is used both to track user's movements and estimate the current position of the user's pupil in the image in a decoupled fashion. Experiments show that the proposed approach is fast and accurate enough for the design of low-cost man-machine interfaces, with applications ranging from the assistance to the disabled people to multimedia systems.

A two-layers architecture for optical flow computation is presented, which uses neural nets in the lower layer and a special relaxation system in the upper layer. The high degree of parallelism of the architecture makes it particularly suitable for real-time applications.>