In this paper, we introduce a novel framework for creating multimodal interactive digital twin characters, from dialogue videos of TV shows. Specifically, these digital twin characters are capable of responding to user inputs with harmonious textual, vocal, and visual content. They not only replicate the external characteristics, such as appearance and tone, but also capture internal attributes, including personality and habitual behaviors. To support this ambitious task, we collect the Multimodal Character-Centric Conversation Dataset, named MCCCD, which includes character-specific and high-quality multimodal dialogue data with detailed annotations, featuring 6.8 k utterances and 4.6 hours of audio/video per character. Notably, the MCCCD dataset is approximately ten times larger than existing datasets in terms of per-character data volume, facilitating the detailed modeling of complex character-centric traits. Further, we propose a baseline framework to create digital twin characters, consists of dialogue generation through large language models, voice generation via speech synthesis models, and visual representation with 3D talking head models. Experimental results demonstrate that our approach significantly outperforms existing methods in generating consistent and character-specific responses, setting a new benchmark for digital character creation. Our collected dataset and proposed baseline have paved the way for the creation of highly interactive and natural digital avatars, opening the door to extensive and practical applications of digital humans.

As a method of estimating the parameters of statistical models in statistics, maximum likelihood (ML) method has gained much attention and been popularly applied to a variety of fields. In the subjects of information, control, as well as system science and engineering, it is usually exploited to es timate the parameters of diverse linear dynamic systems. The tough task of estimating ML parameters of diverse linear dynamic systems can be attributed to a constrained optimization problem. In an effort to solve the constrained optimization problem effectively, we first use the ML method to address the statistical modelling of ML parameter estimation of general linear dynamic systems. Then we present a novel heuristic particle swarm search algorithm called sliding mode controlling particle swarm optimization (SMCPSO) algorithm. In SMCPSO, we introduce a sliding mode controller (SMC) into standard particle swarm optimization (SPSO). The SMC is intentionally placed between the particle position and the global best particle position so as to improve the particles' position information. It ameliorates the exploration and exploitation search abilities of the particles. Thereafter, SMCPSO is exploited to solve the problem of the ML parameter estimation of a given linear dynamic system together with SPSO, recursive ML generalized least squares (RMLGLS) method, and recursive ML least squares (RMLLS) algorithm. The simulation results demonstrate that SMCPSO is an effective approach and superior to other three approaches in estimating the ML parameters of the linear dynamic system.

In this paper, we introduce direct back propagation (BP) neural dynamic programming (NDP) into particle swarm optimisation (PSO). Thus, a direct BP NDP inspired PSO algorithm, which we call NDPSO, is proposed. In NDPSO, since direct BP NDP belongs to the class of heuristic dynamic programming algorithms based on model-based adaptive critic designs and often serves as an online learning control paradigm, critic BP neural network is trained to optimise a total reward-to-go objective, namely to balance Bellman's equation, while action BP neural network is used to train the inertia weight, cognitive, and social coefficients so that the critic BP network output can approach an ultimate reward-to-go objective of success. With the collective aid of action-critic BP neural networks, inertia weight, cognitive, and social coefficients become more adaptive. Besides, the NDPSO's mutation mechanism also has greatly improved the dynamic performance of the standard PSO. Empirical experiments are conducted on both unimodal and multimodal benchmark functions. The experimental results demonstrate NDPSO's effectiveness and superiority to many other PSO variants on solving most multimodal problems.