The Automated Domain-Understanding and Collaborative Agency (ADUCA) system allows for the acquisition of conceptual models of both objects and actions. With exposure to a small number of training exemplars, the ADUCA system constructs a canonical model based on semantically meaningful attributes and relationships that are common to the training data. At test time, the ADUCA system is first tasked with describing an image or video in generic semantic terms which constitutes a bottom up description of the input. The ADUCA system then attempts to determine which if any of its canonical class models is most consistent with the bottom up description. When prompted, the ADUCA system can provide a grounded explanation for its decision as well as answer questions such as why a different conclusion was not drawn. When a mistake has been identified, the ADUCA system attempts to reason over why the mistake was made as well as propose how best to modify its canonical model so as to avoid making similar mistakes in the future. This allows for a form of continuous learning.

This paper presents the outcomes of the PETS2021 challenge held in conjunction with AVSS2021 and sponsored by the EU FOLDOUT project. The challenge comprises the publication of a novel video surveillance dataset on through-foliage detection, the defined challenges addressing person detection and tracking in fragmented occlusion scenarios, and quantitative and qualitative performance evaluation of challenge results submitted by six worldwide participants. The results show that while several detection and tracking methods achieve overall good results, through-foliage detection and tracking remains a challenging task for surveillance systems especially as it serves as the input to behaviour (threat) recognition.

Person Re-ID is a challenging problem and is gaining more attention due to demands in security, intelligent system and other applications. Most person Re-ID works are vision-based, such as image, video, or broadly speaking, face recognition-based techniques. Recently, several multi-modal person Re-ID datasets were released, including RGB+IR, RGB+text, RGB+WiFi, which shows the potential of the multi-modal sensor-based person Re-ID approach. However, there are several common issues in public datasets, such as short time duration, lack of appearance change, and limited activities, resulting in un-robust models. For example, vision-based Re-ID models are sensitive to appearance change. In this work, a person Re-ID testbed with multi-modal sensors is created, allowing the collection of sensing modalities including RGB, IR, depth, WiFi, radar, and audio. This novel dataset will cover normal daily office activities with large time span over multi-seasons. Initial analytic results are obtained for evaluating different person Re-ID models, based on small datasets collected in this testbed.

Visual curve completion, an early visual process that completes the occluded parts between observed boundary fragments (a.k.a. inducers), is a major problem in perceptual organization and a critical step toward higher level visual tasks in both biological and machine vision. Most computational contributions to solving this problem suggest desired perceptual properties that the completed contour should satisfy in the image plane, and then seek the mathematical curves that provide them. Alternatively, few studies (including by the authors) have suggested to frame the problem not in the image plane but rather in the unit tangent bundleR (2) × S(1), the space that abstracts the primary visual cortex, where curve completion allegedly occurs. Combining both schools, here we propose and develop a biologically plausible theory of elastica in the tangent bundle that provides not only perceptually superior completion results but also a rigorous computational prediction that inducer curvatures greatly affects the shape of the completed curve, as indeed indicated by human perception.

We propose a theory for cortical representation and computation of visually completed curves that are generated by the visual system to fill in missing visual information (e.g., due to occlusions). Recent computational theories and physiological evidence suggest that although such curves do not correspond to explicit image evidence along their length, their construction emerges from corresponding activation patterns of orientation-selective cells in the primary visual cortex. Previous theoretical work modeled these patterns as least energetic 3D curves in the mathematical continuous space R2 × S1, which abstracts the mammalian striate cortex. Here we discuss the biological plausibility of this theory and present a neural architecture that implements it with locally connected parallel networks. Part of this contribution is also a first attempt to bridge the physiological literature on curve completion with the shape problem and a shape theory. We present completion simulations of our model in natural and synthetic scenes and discuss various observations and predictions that emerge from this theory in the context of curve completion.

Visual curve completion is a fundamental perceptual mechanism that completes the missing parts (e.g., due to occlusion) between observed contour fragments. Previous research into the shape of completed curves has generally followed an "axiomatic" approach, where desired perceptual/geometrical properties are first defined as axioms, followed by mathematical investigation into curves that satisfy them. However, determining psychophysically such desired properties is difficult and researchers still debate what they should be in the first place. Instead, here we exploit the observation that curve completion is an early visual process to formalize the problem in the unit tangent bundle R(2) × S(1), which abstracts the primary visual cortex (V1) and facilitates exploration of basic principles from which perceptual properties are later derived rather than imposed. Exploring here the elementary principle of least action in V1, we show how the problem becomes one of finding minimum-length admissible curves in R(2) × S(1). We formalize the problem in variational terms, we analyze it theoretically, and we formulate practical algorithms for the reconstruction of these completed curves. We then explore their induced visual properties vis-à-vis popular perceptual axioms and show how our theory predicts many perceptual properties reported in the corresponding perceptual literature. Finally, we demonstrate a variety of curve completions and report comparisons to psychophysical data and other completion models.

The phenomenon of visual curve completion, where the visual system completes the missing part (e.g., due to occlusion) between two contour fragments, is a major problem in perceptual organization research. Previous computational approaches for the shape of the completed curve typically follow formal descriptions of desired, image-based perceptual properties (e.g, minimum total curvature, roundedness, etc.). Unfortunately, however, it is difficult to determine such desired properties psychophysically and indeed there is no consensus in the literature for what they should be. Instead, in this paper we suggest to exploit the fact that curve completion occurs in early vision in order to formalize the problem in a space that explicitly abstracts the primary visual cortex. We first argue that a suitable abstraction is the unit tangent bundle R2× S1and then we show that a basic principle of “minimum energy consumption” in this space, namely a minimum length completion, entails desired perceptual properties for the completion in the image plane. We present formal theoretical analysis and numerical solution methods, we show results on natural images and their advantage over existing popular approaches, and we discuss how our theory explains recent findings from the perceptual literature using basic principles only.