This is the overview paper for the Deep Video Understanding (DVU) Grand Challenge. In recent years, a growing trend towards working on understanding videos (in particular movies) to a deeper level started to motivate researchers working in multimedia and computer vision to present new approaches and datasets to tackle this problem. This is a challenging research area which aims to develop a deep understanding of the relations which exist between different individuals and entities in movies using all available modalities such as video, audio, text and metadata. The aim of this grand challenge is to foster innovative research in this new direction and to provide benchmarking evaluations to advance technologies in the deep video understanding community.

3D models of humans are commonly used within computer graphics and vision, and so the ability to distinguish between body shapes is an important shape retrieval problem. We extend our recent paper which provided a benchmark for testing non-rigid 3D shape retrieval algorithms on 3D human models. This benchmark provided a far stricter challenge than previous shape benchmarks. We have added 145 new models for use as a separate training set, in order to standardise the training data used and provide a fairer comparison. We have also included experiments with the FAUST dataset of human scans. All participants of the previous benchmark study have taken part in the new tests reported here, many providing updated results using the new data. In addition, further participants have also taken part, and we provide extra analysis of the retrieval results. A total of 25 different shape retrieval methods are compared.

In this paper, we present a Support Vector Machine (SVM) based pixel classifier for a semi-automated segmentation algorithm to detect neuronal membrane structures in stacks of electron microscopy images of brain tissue samples. This algorithm uses high-dimensional feature spaces extracted from center-surrounded patches, and some distinct edge sensitive features for each pixel in the image, and a training dataset for the segmentation of neuronal membrane structures and background. Some threshold conditions are later applied to remove small regions, which are below a certain threshold criteria, and morphological operations, such as the filling of the detected objects, are done to get compactness in the objects. The performance of the segmentation method is calculated on the unseen data by using three distinct error measures: pixel error, wrapping error, and rand error, and also a pixel by pixel accuracy measure with their respective ground-truth. The trained SVM classifier achieves the best precision level in these three distinct errors at 0.23, 0.016 and 0.15, respectively; while the best accuracy using pixel by pixel measure reaches 77% on the given dataset. The results presented here are one step further towards exploring possible ways to solve these hard problems, such as segmentation in medical image analysis. In the future, we plan to extend it as a 3D segmentation approach for 3D datasets to not only retain the topological structures in the dataset but also for the ease of further analysis.

This paper presents a novel convexity measurement for 3D meshes. The new convexity measure is calculated by minimizing the ratio of the summed area of valid regions in a mesh's six views, which are projected on faces of the bounding box whose edges are parallel to the coordinate axes, to the sum of three orthogonal projected areas of the mesh. The complete definition, theoretical analysis, and a computing algorithm of our convexity measure are explicitly described. This paper also proposes a new 3D shape descriptor CD (i.e., Convexity Distribution) based on the distribution of above-mentioned ratios, which are computed by randomly rotating the mesh around its center, to better describe the object's convexity-related properties compared to existing convexity measurements. Our experiments not only show that the proposed convexity measure corresponds well with human intuition, but also demonstrate the effectiveness of the new convexity measure and the new shape descriptor by significantly improving the performance of other methods in the application of 3D shape retrieval.

Matching non-rigid shapes is a challenging research field in content-based 3D object retrieval. In this paper, we present an image-based method to effectively address this problem. Multidimensional Scaling (MDS) and Principal Component Analysis (PCA) are first applied to each object to calculate its canonical form, which is afterward represented by 66 depth-buffer images captured on the vertices of an unit geodesic sphere. Then, each image is described as a word histogram obtained by the vector quantization of the image's salient local features. Finally, a multi-view shape matching scheme is carried out to measure the dissimilarity between two models. Experimental results on the McGill Articulated Shape Benchmark database demonstrate that, our method obtains better retrieval performance compared to the state-of-the-art.

This paper presents a novel 3D shape retrieval method, which uses Bag-of-Features and an efficient multi-view shape matching scheme. In our approach, a properly normalized object is first described by a set of depth-buffer views captured on the surrounding vertices of a given unit geodesic sphere. We then represent each view as a word histogram generated by the vector quantization of the view's salient local features. The dissimilarity between two 3D models is measured by the minimum distance of their all (24) possible matching pairs. This paper also investigates several critical issues including the influence of the number of views, codebook, training data, and distance function. Experiments on four commonly-used benchmarks demonstrate that: 1) Our approach obtains superior performance in searching for rigid models. 2) The local feature and global feature based methods are somehow complementary. Moreover, a linear combination of them significantly outperforms the state-of-the-art in terms of retrieval accuracy.

This paper investigates the capabilities of the Bag-of-Words (BW) method in the 3D shape retrieval field. The contributions of this paper are: 1) the 3D shape retrieval task is categorized from different points of view: specific vs. generic, partial-to-global (PG) vs. global-to-global (GG) retrieval, and articulated vs. non-articulated; 2) The spatial information, which is represented as concentric spheres, is integrated into the framework to improve the discriminative capability; 3) the analysis of the experimental results on Purdue Engineering Benchmark (PEB) reveals that some properties of the BW approach make it perform better on the PG task than the GG task. 4) The BW approach is evaluated on non-articulated database PEB and articulated database McGill Shape Benchmark (MSB) and compared to other methods.

Fourier Mellin Transform (FMT) has been used effectively in previous work for 2D image analysis, reconstruction and retrieval. In this paper, we perform 3D shape retrieval based on FMT on the Purdue Shape Benchmark. The whole procedure includes three steps: 1) generate silhouettes along the six principle directions for each 3D model; 2) compute a collection of FMT coefficients for all the silhouettes, which are translation, scale, and rotation invariant; and 3) compute a match measure between the query coefficients collection and those in the 3D shape repositories. The main contribution of this paper is the novel approach to extract the 3D signatures by Fourier Mellin Transform. Our experimental results validate the effectiveness of our approach.

In this paper we propose a novel algorithm for 3D shape searching based on the visual similarity by cutting the object into sections. This method rectifies some of the shortcomings of the visual similarity based methods, so that it can better account for concave areas of an object and parts of the object not visible because of occlusion. As the first step, silhouettes of the 3D object are generated by partitioning the object into number of parts with cutting planes perpendicular to the view direction. Then Zernike moments are applied on the silhouettes to generate shape descriptors. The distance measure is based on minimizing the distance among all the combinations of shape descriptors and then these distances are used for similarity based searching. We have performed experiments on the Princeton shape benchmark and the Purdue CAD/CAM database, and have achieved results comparable to some of the best algorithms in the 3D shape searching literature.