Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in real applications. To address these issues, this paper breaks down the text-based video editing task into two stages. First, we leverage an pre-trained text-to-image diffusion model to simultaneously edit few keyframes in an zero-shot way. Second, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the edited keyframes, using the structural guidance from intermediate frames. Experimental results suggest that our MaskINT achieves comparable performance with diffusion-based methodologies, while significantly improve the inference time. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

Deep dictionary learning seeks multiple dictionaries at different image scales to capture complementary coherent characteristics. We propose a method for learning a hierarchy of synthesis dictionaries with an image classification goal. The dictionaries and classification parameters are trained by a classification objective, and the sparse features are extracted by reducing a reconstruction loss in each layer. The reconstruction objectives in some sense regularize the classification problem and inject source signal information in the extracted features. The performance of the proposed hierarchical method increases by adding more layers, which consequently makes this model easier to tune and adapt. The proposed algorithm furthermore, shows remarkably lower fooling rate in presence of adversarial perturbation. The validation of the proposed approach is based on its classification performance using four benchmark datasets and is compared to a CNN of similar size.

Hierarchical dictionary learning seeks multiple dictionaries at different image scales to capture complementary coherent characteristics. We propose a method to learn a hierarchy of two overcomplete synthesis dictionaries with an image classification goal. The classification objective in some sense regularizes the joint optimization of the hierarchical dictionaries and injects refinement feedback. The validation of the proposed approach is based on its classification performance using two well-known data sets.

In this paper, we are interested in modeling the diffusion of information in a multilayer network of agents using a thermodynamic diffusion approach. The state of each agent is viewed as a topic mixture, to describe his/her resources, and represented by a distribution over multiple topics. We observe and learn diffusion-related thermodynamical patterns in the training data set, and we use the estimated diffusion structure to predict the future states of the agents. With a priori knowledge of a fraction of the state of all agents, the problem is shown to turn into a Kalman predictor problem that refines the predicted system states using the estimation error of the agents' states. A real world Twitter data set is then used to evaluate and validate our information diffusion model.