Future video prediction provides valuable information that helps a computer machine understand the surrounding environment and make critical decisions in real-time. However, long-term video prediction remains a challenging problem due to the complicated spatiotemporal dynamics in a video. In this paper, we propose a dynamic motion estimation and evolution (DMEE) network model to generate unseen future videos from the observed videos in the past. Our primary contribution is to use trained kernels in convolutional neural network (CNN) and long short-term memory (LSTM) architectures, adapted to each time step and sample position, to efficiently manage spatiotemporal dynamics. DMEE uses the motion estimation (ME) and motion update (MU) kernels to predict the future video using an end-to-end prediction-update process. In the prediction, the ME kernel estimates the temporal changes. In the update step, the MU kernel combines the estimates with the previously generated frames as reference frames using a weighted average. The kernels are not only used for a current frame, but also are evolved to generate successive frames to enable temporally specific filtering. We perform qualitative performance analysis and quantitative performance analysis based on the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and video classification score developed for examining the visual quality of the generated video. It is demonstrated with experiments that our algorithm provides better qualitative and quantitative performance superior to the current state-of-the-art algorithms. Our source codes are available inhttps://github.com/Nayoung-Kim-ICP/Video-Generation.

In this paper, we propose a novel long-term video generation algorithm, motivated by the recent developments of unsupervised deep learning techniques. The proposed technique learns two ingredients of internal video representation, i.e., video textures and motions to reproduce realistic pixels in the future video frames. To this aim, the proposed technique uses two encoders comprising convolutional neural networks (CNN) to extract spatiotemporal features from the original video frame and a residual video frame, respectively. The use of the residual frame facilitates the learning with fewer parameters as there are high spatiotemporal correlations in a video. Moreover, the residual frames are efficiently used for evolving pixel differences in the future frame. In a decoder, the future frame is generated by transforming the combination of two feature vectors into the original video size. Experimental results demonstrate that the proposed technique provides more robust and accurate results of long-term video generation than conventional techniques.

In this paper, a residual differential pulse code modulation (RDPCM) coding technique using a weighted linear combination of neighboring residual samples is proposed to provide coding efficiency in the screen content video coding. The RDPCM performs the sample-based prediction of residue to reduce spatial redundancies. The proposed method uses the$l_1$optimization in the weight derivation by considering the statistical characteristics of graphical components in videos in an intracoding. Specifically we use the least absolute shrinkage and selection operator to derive the weights because the solution is accurate in high variance residue. Furthermore, we enhance parallelism in a line processing by restricting the support to the row-wise prediction to above samples or the column-wise prediction to the left samples. The proposed method uses an explicit RDPCM scheme, so a coding mode determined by rate-distortion optimization is transmitted to a decoder. For coding the overhead, we develop a context design in CABAC based on correlation between an intraprediction direction and an RDPCM prediction mode. It is demonstrated with the experimental results that the proposed method provides a significant coding gain over the state-of-the-art reference codec for screen content video coding.