Abstract Reflective surfaces are notoriously difficult to detect and reconstruct, as they reflect light from surrounding objects. Thus, the quality of 3D reconstructions and the performance of downstream computer vision tasks, such as navigation, depth prediction, or object detection are severely impaired by the prevalence of mirrors in indoor environments. This paper proposes a novel reflection-aware method for 3D mirror segmentation and pose estimation, based on the reinterpretation of captured RGB-D data. It is a top-down approach, where information consensus, unsupervised learning, and ray casting are employed, formulating mirror pose estimation as an optimization problem. Experimental results show that the proposed approach significantly outperforms the existing benchmarks for RGB and RGB-D-based 3D mirror segmentation and pose estimation.

Multi-agent consensus problems can often be seen as a sequence of autonomous and independent local choices between a finite set of decision options, with each local choice undertaken simultaneously, and with a shared goal of achieving a global consensus state. Being able to estimate probabilities for the different outcomes and to predict how long it takes for a consensus to be formed, if ever, are core issues for such protocols. Little attention has been given to protocols in which agents can remember past or outdated states. In this paper, we propose a framework to study what we call `memory consensus protocol'. We show that the employment of memory allows such processes to always converge, as well as, in some scenarios, such as cycles, converge faster. We provide a theoretical analysis of the probability of each option eventually winning such processes based on the initial opinions expressed by agents. Further, we perform experiments to investigate network topologies in which agents benefit from memory on the expected time needed for consensus.