We describe a novel algorithm for an open-domain trending hashtag recommendation task using zero-shot hashtag prediction in an online learning paradigm. Our method utilizes joint representation learning of latent embeddings for features extracted from long-form videos and semantic embeddings of hashtags trending on social platforms. In particular, we apply graph convolutional networks to a link prediction task using videos and hashtags as nodes in a heterogeneous graph. Comparing it to the existing models for closely related tasks, we demonstrate state-of-the-art results in trending hashtag recommendations for videos. The architecture is designed to be modular in a plug-and-play fashion to enable quick and easy incorporation of the latest advances in natural language understanding and image and video processing, with a practical view to its implementation in a real-time, online setting.

Photorealistic rendering of rain streaks with lighting and viewpoint effects is a challenging problem. Raindrops undergo rapid shape distortions as they fall, a phenomenon referred to as oscillations. Due to these oscillations, the reflection of light by, and the refraction of light through, a falling raindrop produce complex brightness patterns within a single motion-blurred rain streak captured by a camera or observed by a human. The brightness pattern of a rain streak typically includes speckles, multiple smeared highlights and curved brightness contours. In this work, we propose a new model for rain streak appearance that captures the complex interactions between the lighting direction, the viewing direction and the oscillating shape of the drop. Our model builds upon a raindrop oscillation model that has been developed in atmospheric sciences. We have measured rain streak appearances under a wide range of lighting and viewing conditions and empirically determined the oscillation parameters that are dominant in raindrops. Using these parameters, we have rendered thousands of rain streaks to create a database that captures the variations in streak appearance with respect to lighting and viewing directions. We have developed an efficient image-based rendering algorithm that uses our streak database to add rain to a single image or a captured video with moving objects and sources. The rendering algorithm is very simple to use as it only requires a coarse depth map of the scene and the locations and properties of the light sources. We have rendered rain in a wide range of scenarios and the results show that our physically-based rain streak model greatly enhances the visual realism of rendered rain.

Rain produces sharp intensity fluctuations in images and videos, which degrade the performance of outdoor vision systems. These intensity fluctuations depend on various factors, such as the camera parameters, the properties of rain, and the brightness of the scene. We show that the properties of rain - its small drop size, high velocity, and low density - make its visibility strongly dependent on camera parameters such as exposure time and depth of field. We show that these parameters can be selected so as to reduce or even remove the effects of rain without altering the appearance of the scene. Conversely, the parameters of a camera can also be set to enhance the visual effects of rain. This can be used to develop an inexpensive and portable camera-based rain gauge that provides instantaneous rain rate measurements. The proposed methods serve to make vision algorithms more robust to rain without any necessity for post-processing. In addition, they can be used to control the visual effects of rain during the filming of movies