Model compression is essential in the deployment of large Computer Vision models on embedded devices. However, static optimization techniques (e.g. pruning, quantization, etc.) neglect the fact that different inputs have different complexities, thus requiring different amounts of computations. Dynamic Neural Networks allow conditioning the number of computations to the specific input. The current literature on the topic is very extensive and fragmented. We present a comprehensive survey that synthesizes and unifies existing Dynamic Neural Networks research in the context of Computer Vision. Additionally, we provide a logical taxonomy based on which component of the network is adaptive: the output, the computation graph or the input. Furthermore, we argue that Dynamic Neural Networks are particularly beneficial in the context of Sensor Fusion for better adaptivity, noise reduction and information prioritization. We present preliminary works in this direction. We complement this survey with a curated repository listing all the surveyed papers, each with a brief summary of the solution and the code base when available: https://github.com/DTU-PAS/awesome-dynn-for-cv .

Even if the depth maps captured by RGB-D sensors deployed in real environments are often characterized by large areas missing valid depth measurements, the vast majority of depth completion methods still assumes depth values covering all areas of the scene. To address this limitation, we introduce SteeredMarigold, a training-free, zero-shot depth completion method capable of producing metric dense depth, even for largely incomplete depth maps. SteeredMarigold achieves this by using the available sparse depth points as conditions to steer a denoising diffusion probabilistic model. Our method outperforms relevant top-performing methods on the NYUv2 dataset, in tests where no depth was provided for a large area, achieving state-of-art performance and exhibiting remarkable robustness against depth map incompleteness. Our source code is publicly available at https://steeredmarigold.github.io.

We explore the use of uncertainty estimation in the maritime domain, showing the efficacy on toy datasets (CIFAR10) and proving it on an in-house dataset, SHIPS. We present a method joining the intra-class uncertainty achieved using Monte Carlo Dropout, with recent discoveries in the field of outlier detection, to gain more holistic uncertainty measures. We explore the relationship between the introduced uncertainty measures and examine how well they work on CIFAR10 and in a real-life setting. Our work improves the FPR95 by 8% compared to the current highest-performing work when the models are trained without out-of-distribution data. We increase the performance by 77% compared to a vanilla implementation of the Wide ResNet. We release the SHIPS dataset and show the effectiveness of our method by improving the FPR95 by 44.2 % with respect to the baseline. Our approach is model agnostic, easy to implement, and often does not require model retraining.

Weed detection is a challenging case within object detection as the weed targets do not generally strike out from the background in terms of color. This paper investigates how the density of structural features can be used to assist the training process of a Deep-Learning-based object detector. SIFT keypoint density is used to create overlay masks to augment images, emphasizing low-density areas—typically corresponding to weed plants. Our method is shown to improve detection $$mAP_{.5:.05:.95}$$ on the YOLOR-CSP detector by up to 0.0215.

We present a lightweight encoder-decoder architecture for monocular depth estimation, specifically designed for embedded platforms. Our main contribution is the Guided Upsampling Block (GUB) for building the decoder of our model. Motivated by the concept of guided image filtering, GUB relies on the image to guide the decoder on upsampling the feature representation and the depth map reconstruction, achieving high resolution results with fine-grained details. Based on multiple GUBs, our model outperforms the related methods on the NYU Depth V2 dataset in terms of accuracy while delivering up to 35.1 fps on the NVIDIA Jetson Nano and up to 144.5 fps on the NVIDIA Xavier NX. Similarly, on the KITTI dataset, inference is possible with up to 23.7 fps on the Jetson Nano and 102.9 fps on the Xavier NX. Our code and models are made publicly available14https://github.com/mic-rud/GuidedDecoding.

In near coast navigation, buoys and beacons convey essential information about dangers. At night-time, selected buoys send out individual blink-sequences that are marked in sea charts. International regulations require that navigation officer on watch makes visual confirmation of objects and their type in order to navigate safely. With rapid developments of highly automated vessels, this duty needs be carried out by algorithms that detect and locate objects without human intervention. At night-time, this requires algorithms that decode blink sequences and are able to classify from this information. The paper investigates this problem and suggests an algorithm that solves the problem. Convolutional Neural Networks (CNN) with Gated Recurrent Units (GRU) are developed for classification. A dedicated architecture is suggested that includes both temporal and color decoding to obtain unique precision. We demonstrate how networks are trained on synthetically generated data, and the paper shows, on real-world data, how the suggested approach yields 100.0% accurate results on 44 real-world recordings while being robust to inaccuracy in actual blink sequences. Comparison with baseline signal processing and with a recent state-of-the-art 3D CNN model shows that the new blink-sequence classifier outperforms alternative algorithms. A showcase of the results of this work is available in this video:https://youtu.be/KEi8qNnKV2w.

Marine vessels are subject to high wear and tear due to the conditions they operate in. To reduce risk of failure during operation, vessels are inspected periodically every five years. These inspections are prone to high subjectiveness that makes them hard to reproduce for the shipping owners. The purpose of this paper is to present a regressor to a Faster R-CNN network that can help alleviate some of the subjective assessment currently performed by human surveyors by estimating the severity of a corroded area, autonomously using drones. A feature pyramid backbone is shared between the Faster R-CNN and the added regression head. The goal of the regressor is to introduce a more objective assessment of the vessel that gives a consistent output for a consistent input. The system is evaluated on a real dataset, acquired in ballast tanks and the experimental results indicate that our deep learning approach can be used to detect and quantify corroded areas during the inspection process of marine vessels.

Autonomous robotic weeding in grasslands requires robust weed segmentation. Deep learning models can provide solutions to this problem, but they need to be trained on large amounts of images, which in the case of grasslands are notoriously difficult to obtain and manually annotate. In this work we introduce Few-leaf Learning, a concept that facilitates the training of accurate weed segmentation models and can lead to easier generation of weed segmentation datasets with minimal human annotation effort. Our approach builds upon the fact that each plant species within the same field has relatively uniform visual characteristics due to similar environmental influences. Thus, we can train a field-and-day-specific weed segmentation model on synthetic training data stemming from just a handful of annotated weed leaves. We demonstrate the efficacy of our approach for different fields and for two common grassland weeds: Rumex obtusifolius (broad-leaved dock) and Cirsium vulgare (spear thistle). Our code is publicly available at https://github.com/RGring/WeedAnnotator.

Inverse dynamics models are applied to a plethora of robot control tasks such as computed-torque control, which are essential for trajectory execution. The analytical derivation of such dynamics models for robotic manipulators can be challenging and depends on their physical characteristics. This paper proposes a machine learning approach for modeling inverse dynamics and provides information about its implementation on a physical robotic system. The proposed algorithm can perform online multi-target learning, thus allowing efficient implementations on real robots. Our approach has been tested both offline, on datasets captured from three different robotic systems and online, on a physical system. The proposed algorithm exhibits state-of-the-art performance in terms of generalization ability and convergence. Furthermore, it has been implemented within ROS for controlling a Baxter robot. Evaluation results show that its performance is comparable to the built-in inverse dynamics model of the robot.

Many robot learning algorithms depend on a model of the robot's forward dynamics for simulating potential trajectories and ultimately learning a required task. In this paper, we present a data-driven reservoir computing approach and apply it for learning forward dynamics models. Our proposed machine learning algorithm exploits the concepts of dynamic reservoir, self-organized learning and Bayesian inference. We have evaluated our approach on datasets gathered from two industrial robotic manipulators and compared it on both step-by-step and multi-step trajectory prediction scenarios with state-of-the-art algorithms. The evaluation considers the algorithms' convergence and prediction performance on joint and operational space for varying prediction horizons, as well as computational time. Results show that the proposed algorithm performs better than the state-of-the-art, converges fast and can achieve accurate predictions over longer horizons, which makes it a reliable, data-efficient approach for learning forward models.

Cognitive robots, able to adapt their actions based on sensory information and the management of uncertainty, have begun to find their way into manufacturing settings. However, the full potential of these robots has not been fully exploited, largely due to the lack of vertical integration with existing IT infrastructures, such as the manufacturing execution system (MES), as part of a large-scale cyber-physical entity. This paper reports on considerations and findings from the research project STAMINA that is developing such a cognitive cyber-physical system and applying it to a concrete and well-known use case from the automotive industry. Our approach allows manufacturing tasks to be performed without human intervention, even if the available description of the environment-the world model-suffers from large uncertainties. Thus, the robot becomes an integral part of the MES, resulting in a highly flexible overall system.

In certain cases analytical derivation of physics-based models of robots is difficult or even impossible. A potential workaround is the approximation of robot models from sensor data-streams employing machine learning approaches. In this paper, the inverse dynamics models are learned by employing a novel real-time deep learning algorithm. The algorithm exploits the methods of self-organized learning, reservoir computing and Bayesian inference. It is evaluated and compared to other state of the art algorithms in terms of generalization ability, convergence and adaptability using five datasets gathered from four robots. Results show that the proposed algorithm can adapt to real-time changes of the inverse dynamics model significantly better than the other state of the art algorithms.

We address the problem of object segmentation in image sequences where no a-priori knowledge of objects is assumed. We take advantage of robots' ability to move, gathering multiple images of the scene. Our approach starts by extracting edges, uses a polar domain representation and performs integration over time based on a simple dilation operation. The proposed system can be used for providing reliable initial segmentation of unknown objects in scenes of varying complexity, allowing for recognition, categorization or physical interaction with the objects. The experimental evaluation on both self-captured and a publicly available dataset shows the efficiency and stability of the proposed method.

A simple low voltage circuit topology able to generate any positive real number power-law function is presented. The proposed circuit exploits BJTs and is based on piecewise linear approximation of the nonlinear function to be generated. An in-depth mathematical analysis is deployed. The instances of a squarer, a cube-law, a square rooting and cube rooting circuit are thoroughly examined through simulation. The obtained results verify the theoretical calculations

Polysilicon thin-film technology has become of great interest due to the demand for large area electronic devices. Display applications, memories and optical copier are among the fields where polysilicon thin-film transistors (poly-Si TFTs) are most commonly used. However the design of analogue blocks, by using Poly-Si TFTs, with constant specifications is very difficult because of the large variation of the threshold voltage of the poly-Si TFT across the wafer and the kink effect makes. In this paper we propose a circuit that can sense the voltage difference between two TFTs. This voltage can be used in order to design an improved current mirror with cancellation of threshold voltage variation