Abstract Input images are the main source of information for vision‐based algorithms. The presence of raindrops in input images degrades their quality and, consequently, reduces the quality of the target vision‐based algorithm that consumes them. Many image restoration algorithms were proposed in the literature to remove rain presence in images to improve the input image quality. These algorithms, however, cannot remove all the raindrop presence and sometimes introduce undesirable side‐effects, such as the blurring rain‐occluded sections of the image and incorrectly de‐raining areas in the image that are clear. It is hypothesized that a comparable performance improvement can be achieved by decreasing the sensitivity of vision‐based algorithms to noisy input images, rather than denoising these images, through the process of de‐raining. To test this hypothesis, the performance of state‐of‐the‐art object detection and semantic segmentation models was evaluated, with de‐rained image datasets used as input, and compared it to that performance of the same models, retrained with rained image sets. Results showed that the performance of the retrained models was better than that of the baseline detector with de‐rained images used as input.

Panoptic Segmentation aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level. It combines the separate tasks of semantic segmentation (pixel level classification) and instance segmentation to build a single unified scene understanding task. Typically, panoptic segmentation is derived by combining semantic and instance segmentation tasks that are learned separately or jointly (multi-task networks). In general, instance segmentation networks are built by adding a foreground mask estimation layer on top of object detectors or using instance clustering methods that assign a pixel to an instance center. In this work, we present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours (boundaries of things). Instance contours along with semantic segmentation yield a boundary aware semantic segmentation of things. Connected component labeling on these results produces instance segmentation. We merge semantic and instance segmentation results to output panoptic segmentation. We evaluate our proposed method on the CityScapes dataset to demonstrate qualitative and quantitative performances along with several ablation studies. Our overview video can be accessed from https://youtu.be/wBtcxRhG3e0.

Body-worn devices have significant potential to improve the health and well-being of many individuals. In this work, wearable inertial sensors are used in order to track and discriminate shoulder motion gestures, without using visual markers or other approaches that constrain the system to a laboratory environment. The device, consisting of a set of orthogonal accelerometers, gyroscopes, and magnetic field sensors, is attached to the person's upper arm to help prevent shoulder over-use injuries in strenuous work and in athletics. The sensor suite is used to track the orientation of the arm as a function of time. We present a detection and classification approach that can be used to evaluate the number of times certain motion gestures occur.