Motion customization plays a pivotal role in video generation by preserving the original appearance and context while adhering to specific motion patterns. In contrast, video generation techniques often lack coherence and realism due to difficulties in capturing and transferring motion patterns. Building upon the Video Motion Customization (VMC) framework, we proposed a few-shot learning approach using our unified Multi-Head Temporal Attention (MHTA) module for motion customization in text-to-video diffusion models. This significantly reduces computational requirements while maintaining and improving motion quality. Our model provides a streamlined mechanism for motion distillation while maintaining separate self-, cross-, and temporal attention. Moreover, the temporal attention layer is adapted through a simplified mechanism with efficient Q/K/V projections, while maintaining fixed spatial self- and cross-attention. The model distills a ground-truth motion vector from consecutive frames to align the predicted and ground-truth motion. Our proposed MHTA model outperforms the baseline in video generation using motion customization while being significantly more resource-efficient. Moreover, our approach can easily be applied to generate conditional prompt-based videos in the gaming industry.

The metaverse presents an emerging creative expression and collaboration frontier where generative artificial intelligence (GenAI) can play a pivotal role with its ability to generate multimodal content from simple prompts. These prompts allow the metaverse to interact with GenAI, where context information, instructions, input data, or even output indications constituting the prompt can come from within the metaverse. However, their integration poses challenges regarding interoperability, lack of standards, scalability, and maintaining a high-quality user experience. This article explores how GenAI can productively assist in enhancing creativity within the contexts of the metaverse and unlock new opportunities. We provide a technical, in-depth overview of the different generative models for image, video, audio, and 3D content within the metaverse environments. We also explore the bottlenecks, opportunities, and innovative applications of GenAI from the perspectives of end users, developers, service providers, and AI researchers. This survey commences by highlighting the potential of GenAI for enhancing the metaverse experience through dynamic content generation to populate massive virtual worlds. Subsequently, we shed light on the ongoing research practices and trends in multimodal content generation, enhancing realism and creativity and alleviating bottlenecks related to standardization, computational cost, privacy, and safety. Last, we share insights into promising research directions toward the integration of GenAI with the metaverse for creative enhancement, improved immersion, and innovative interactive applications.

Domain knowledge exists in various forms, including text, ontologies, graphs, images, audio, and videos. In plant disease detection, most works solely utilize images with disease labels, neglecting textual descriptions of visual disease symptoms used by human experts for diagnosis. These text descriptions and sample images aid expert identification of visual symptoms. We propose a novel method that leverages text descriptions and image data by modeling domain-specific knowledge about visual symptoms in leaf images as separate feature channels. Each channel corresponds to specific features whose absence or presence in the image influences model predictions. We introduce a channel attention-guided fusion module for weighting each channel based on the input and corresponding output. The combined feature channels are transformed into a standardized 3-channel input format, which can then be processed by any pre-trained convolutional neural network (CNN) as input for feature extraction and subsequent classification. Furthermore, intermediate activations of the channel attention layer combined with the weights from the fusion layer make model predictions explainable. Experimental results on three publicly available datasets of apple and cucumber leaf diseases demonstrate improvements of up to 5% utilizing various state-of-the-art CNN architectures, indicating the efficacy of incorporating textual disease descriptions using the proposed approach.

Segmentation of skin lesions from dermatological images is critical in diagnosing and treating skin cancer. Despite this, the diversity of lesion shapes, sizes, and textures against a similar-toned skin backdrop makes these images challenging to analyze. Current segmentation methods are often less precise in delineating boundaries and more susceptible to interference from background noise. To address this issue, we introduce an end-to-end framework called DeepSkinFormer (DSF) for skin lesion segmentation using the Skin Edge Enhancement Module (SEEM) to enhance boundaries for efficient detection. We evaluate the proposed model on standard benchmarks, HAM10000, ISIC2017, and PH2 datasets. Our model outperforms existing methods and achieves stateof-the-art results using the Dice and mean Intersection Over Union (mIOU) scores. Furthermore, we conduct an ablation study to confirm the significant contributions of DSFspecialized modules to their effectiveness.

Camouflage Object Detection (COD) involves the challenge of isolating a target object from a visually similar background, presenting a formidable challenge for learning algorithms. Drawing inspiration from state-of-the-art (SOTA) Focal Modulation Networks, our objective is to proficiently modulate the foreground and background components, thereby capturing the distinct features of each. We introduce a Feature Split and Modulation (FSM) module to attain this goal. This module efficiently separates the object from the background by utilizing foreground and background modulators guided by a supervisory mask. For enhanced feature refinement, we propose a Context Refinement Module (CRM), which considers features acquired from FSM across various spatial scales, leading to comprehensive enrichment and highly accurate prediction maps. Through extensive experimentation, we showcase the superiority of CamoFocus over recent SOTA COD methods. Our evaluations encompass diverse benchmark datasets, including CAMO, COD10K, CHAMELEON, and NC4K. The findings underscore the potential and significance of the proposed CamoFocus model and establish its efficacy in addressing the critical challenges of camouflage object detection.

Deploying object detectors in embedded devices such as unmanned aerial vehicles (UAVs) comes with many challenges. This is due to both the UAV itself having low embedded resources in terms of computation and memory, and also due to the nature of the captured visual data with the variations in objects' scale, orientation, density, viewpoint, distribution, shape, context and others. It is crucial for the object detector to be robust with high accuracy, real-time with fast inference and light-weight to be applicable. Inspired by YOLO architecture, we propose a novel single-stage detection architecture. Our contributions are, first, feature fusion spatial pyramid pooling (FFSPP) block that applies attention-based feature fusion across both time and space utilizing the information of subsequent frames and scales in an efficient manner. Secondly, we introduce a multi-dilated attention-based cross-stage partial connection (MDACSP) block that helps in increasing the receptive field and producing per-channel modulation weights after aggregating the feature maps across their spatial domain. Third, scaled feature fusion head (SFFH) fuses both the FFSPP block features and the connected MDACSP block features specific for this head. For a more robust result across different scenarios, we perform cross-ensembling with three of the top UAV/traffic surveillance datasets: UAVDT, UA-DETRAC and VisDrone. Our ablation study shows how every contribution improves over the baseline. Our approach yielded the state-of-the-art results in all the aforementioned datasets achieving 89.3% mAP, 93.5% mAP, and 42.9% mAP respectively. Testing the model performance on NVIDIA Jetson Xavier NX board shows a desirable balance between the inference time and the memory cost. We also show qualitatively the model robustness and efficiency across the diverse complex scenarios of these datasets. We hope this work facilitates the advancement of the UAV-based perception in such crucial industrial applications.

In this work, a novel online model-free controller for an underactuated dirigible is developed based on reinforcement learning and optimal control theory. A reinforcement learning structure is used while overcoming the dependence of the value function on future values by introducing a neural network that is adapted using input-output data. The suboptimal critic neural network is structured such that optimality is guaranteed over the interval from which the data is valid. The system performance is validated using a highly realistic physics engine, Gazebo, with the robot operating system (ROS) interface and the results are compared to the performance of a model-based controller specifically designed to control the airship model. It is emphasized that the proposed formulation does not leverage any knowledge of vehicle dynamics and thus is considered a vehicle agnostic control strategy.

The flock-guidance problem enjoys a challenging structure where multiple optimization objectives are solved simultaneously. This usually necessitates different control approaches to tackle various objectives, such as guidance, collision avoidance, and cohesion. The guidance schemes, in particular, have long suffered from complex tracking-error dynamics. Furthermore, techniques that are based on linear feedback strategies obtained at equilibrium conditions either may not hold or degrade when applied to uncertain dynamic environments. Pre-tuned fuzzy inference architectures lack robustness under such unmodeled conditions. This work introduces an adaptive distributed technique for the autonomous control of flock systems. Its relatively flexible structure is based on online fuzzy reinforcement learning schemes which simultaneously target a number of objectives; namely, following a leader, avoiding collision, and reaching a flock velocity consensus. In addition to its resilience in the face of dynamic disturbances, the algorithm does not require more than the agent position as a feedback signal. The effectiveness of the proposed method is validated with two simulation scenarios and benchmarked against a similar technique from the literature.

Underactuated systems like sea vessels have degrees of motion that are insufficiently matched by a set of independent actuation forces. In addition, the underlying trajectory-tracking control problems grow in complexity in order to decide the optimal rudder and thrust control signals. This enforces several difficult-to-solve constraints that are associated with the error dynamical equations using classical optimal tracking and adaptive control approaches. An online machine learning mechanism based on integral reinforcement learning is proposed to find a solution for a class of nonlinear tracking problems with partial prior knowledge of the system dynamics. The actuation forces are decided using innovative forms of temporal difference equations relevant to the vessel's surge and angular velocities. The solution is implemented using an online value iteration process which is realized by employing means of the adaptive critics and gradient descent approaches. The adaptive learning mechanism exhibited well-functioning and interactive features in react to different desired reference-tracking scenarios.

Robotic path planning must avoid obstacles or singularities, and to check for such constraints periodically is computation and time intensive. This paper introduces Constraint-free Discretized Manifolds for robotic Path planning (CDMP) to formulate a constraint free space in the configuration and work spaces. Application based necessities such as obstacle and path visualization, or computational ease guides the choice of working in either space. The chosen constraint free manifold is then meshed using DistMesh, for use with path planning algorithms, which in this paper is A*. The merits offered by this solution are two fold- first, the formulated constraint free manifold is guaranteed to be singularity free irrespective of the start and end locations; second, the path chosen on this manifold will be the shortest path by virtue of using A*.

A data-driven computational heuristic is proposed to control MIMO systems without prior knowledge of their dynamics. The heuristic is illustrated on a two-input two-output balance system. It integrates a self-adjusting nonlinear threshold accepting heuristic with a neural network to compromise between the desired transient and steady state characteristics of the system while optimizing a dynamic cost function. The heuristic decides on the control gains of multiple interacting PID control loops. The neural network is trained upon optimizing a weighted-derivative like objective cost function. The performance of the developed mechanism is compared with another controller that employs a combined PID-Riccati approach. One of the salient features of the proposed control schemes is that they do not require prior knowledge of the system dynamics. However, they depend on a known region of stability for the control gains to be used as a search space by the optimization algorithm. The control mechanism is validated using different optimization criteria which address different design requirements.

Approximate dynamic programming platforms are employed to solve dynamic graphical games, where the agents interact among each other using communication graphs in order to achieve synchronization. Although the action dependent dual heuristic dynamic programming schemes provide fast solution platforms for several control problems, their capabilities degrade for systems with unknown or uncertain dynamical models. An online model-free adaptive learning solution based on action dependent dual heuristic dynamic programming is proposed to solve the dynamic graphical games. It employs distributed actor-critic neural networks to approximate the optimal value function and the associated model-free control strategy for each agent. This is done using a policy iteration process where it does not employ any extensive computational effort, as traditionally observed. The duality between the model-free coupled Bellman optimality equation and the underlying coupled Riccati equation is highlighted. This is followed by a graph simulation scenario to test the usefulness of the proposed policy iteration process.

A typical firing doctrine is the Shoot-Look-Shoot tactic. In this tactic, the defence launches a salvo of interceptors against the targets (Shoot), assesses the outcomes of the engagements (Shoot-Look), and launches another salvo (Shoot-Look-Shoot) if time and the inventory of interceptors permit. In the open literature, it is often assumed that the targets are identical. This is not always true as targets come in with different ranges, speeds, sizes, cross sections etc. In this paper, we consider two types of targets. Each type of target has a different number engagement opportunities due to their ranges and speeds. Through the use of dynamic programming, a genetic algorithm, and a recursive generating function, we determine the probability of raid annihilation (the probability of neutralizing all of the targets) for two different Shoot-Look-Shoot (SLS) tactics. The first SLS tactic is based on variable size salvos and maximizes the probability of raid annihilation (PRA) for heterogeneous targets. The second SLS tactic is based on fixed-size salvos and is robust as it is independent of the number and types of targets. Theoretical results are validated through some computer simulations.

Using two measurements with simultaneous speech and noise presentation, Hagerman and Olofsson have suggested a time-domain method to estimate the speech and noise signals at the output of a hearing device. The method, which uses a simple phase-inversion scheme, has gained popularity in hearing-aid research, although receiving only limited validation. In this work, we present an evaluation of this signal-separation method using simulated measurements with different hearing aids and listening conditions. Estimates of the speech and noise spectra from the phase-inversion method are compared to those obtained using the coherence function. New measures of speech and noise distortion are proposed as tools to evaluate the phase-inversion method. Additionally, we analyze the intelligibility predictions computed from the recovered spectral estimates, while accounting for the proposed speech distortion measure. Under additive-noise conditions, the phase-inversion method provides ideal signal separation without suffering any biases at low signal-to-noise ratios. For conditions involving automatic gain control, compressive output limiting, and peak clipping, the intelligibility predictions based on the phase-inversion method are found to agree with relevant findings from the literature.

In this paper, we present a comparative evaluation of two speech and noise estimation methods commonly used with nonlinear hearing devices: the coherence function used for spectral estimation, and a noise phase-inversion scheme used to perform signal separation. The speech and noise spectra estimated at the output with both methods are compared for normal-hearing subjects and for hearing-impaired subjects using linear and nonlinear hearing aid processing. The spectra are found to be similar except for very low SNR conditions. However, Speech Intelligibility Index (ANSI S3.5-1997 [1]) estimates are relatively unaffected by differences in the spectra obtained with each method.

Child obesity is one of the major challenges facing modern societies, especially in developed countries. Exergaming tools are considered as effective means to reduce obesity among kids because they require the children to exert physical strength while playing the games. However, most of the existing exergaming tools focus more on the physical well-being of its users and almost neglect the mental aspect. In this paper, we present an exergaming system that combines both aspects by promoting not only entertainment, but also learning through physical activity. The system consists of a set of footpads that allow the user to interact with video games enriched with multimedia and aimed at enhancing the math knowledge of children. Our study shows that the system have created an atmosphere of fun among the children and engaged them in learning.

Arm paresis is a very common disability among post-stroke survivors. It is characterized by the inability of a person to perform some specific movements in the arm. A Long term Rehabilitation process plays a key role in the recovery of this kind of disabilities, but such treatment might not be easily accessible to people living away from the cities where most of the rehabilitation centers are located. In this paper, we present our interactive rehabilitation system called "E-Glove" that is aimed to help patients with wrist impairments to perform some daily exercises in a joyful and interactive manner. A 2D golf game that could be played with the glove was developed for this purpose.

Recently, there has been a high demand for developing tools that promote education through learning. We introduce our edutainment tool called Magic Stick that helps children learn about new objects by providing their names associated by visual representations regarding these objects. Children's parents or teachers can pick the entities they would like their children to learn about by simply attaching RFID tags to these entities. Afterwards, they can customize the type of information and visualizations related to these entities through the use of a friendly GUI designed for this purpose. In our study with young children, we found that the Magic Stick created an entertaining atmosphere among children and greatly engaged them in learning.

Body sensor networks (BSN) can capture physical phenomena from a human body, contextual information from the environment and high level events of a person. Associating contextual information and events with the captured raw sensory data can serve as a crucial input for many applications such as e- Health. For example, to accurately and timely monitor an elderly person with several physical disabilities while he is at home or outdoors, the context and event information along with raw sensory data needs to be reached to an e-Health service provider to assist in taking time critical decision. Such process includes receiving the sensory data, analyzing it to trigger necessary services such as sending an alert message to the family physician, hospital, emergency service, his immediate caregiver, family members, friends and so on. A BSN also allows members of one's community of interest, referred to as a social network, to query real-time sensory, contextual and event data. Combining the social network with BSN is envisioned to enhance the current state of the art in e-Health applications. In this paper, we propose a framework, called SenseFace, that can dynamically pass sensory data from one's BSN to his/her social network and vice versa. Finally, we illustrate the design and implementation of the framework.

An ant colony-based routing protocol is presented in this paper that is specifically designed to route many-to-one sensory data in a multi-hop Wireless Sensor Network (WSN). Because a many-to-one routing paradigm generates lots of traffic in a multi-hop WSN resulting in greater energy wastage, higher end-to-end delay and packet loss, the proposed routing protocol also comes with a lightweight congestion control mechanism, which is capable of handling both event-based and periodic upstream sensory data flow to the base station. The proposed protocol works in two-phases. During the first phase, the protocol uses ant-based intelligence to find and enforce the shortest path and in the second phase, when the actual many-to-one sensory data transmission takes place, the protocol combines the knowledge gained during the first phase with the congestion control mechanism to avoid packet loss and traffic while routing the sensory data. When compared with the related algorithms, the proposed algorithm shows promising results.

In this paper, we propose a VLSI (very large scale integrated) implementation of a hybrid neural fuzzy control scheme on a Xilinx Virtex2 Pro 2VP30 field programmable gate array (FPGA) for a flexible-joint robot manipulators with uncertain dynamics. The control strategy is based on a feedforward artificial neural network that approximates the manipulator's inverse dynamics. An adaptive feedback fuzzy sliding mode controller is used to compensate for residual errors. A systolic top-down hardware design methodology takes full advantage of the neural networks' inherent parallelism that allows the controller to operate at high frequencies. Furthermore, a pipeline strategy was used to speed-up the feedback fuzzy controller's inference process. Numerical simulations and the synthesis of the results highlight the effectiveness of the proposed controller in compensating for the nonlinear unknown manipulator's dynamics.

Mobile robot navigation using an analog signal strength of a Radio Frequency IDentification (RFID) device is a promising alternative of different types of robot navigation methods in the state of the art. Skilled navigation in mobile robotics usually requires solving two problems: the knowledge of the position of the robot, and a motion control strategy. Moreover, when no prior knowledge of the environment is available, the problem becomes even more challenging, since the robot has to build a map of its surroundings as it moves. These three problems ought to be solved in conjunction, since they are inter dependent. The objective of this manuscript is to introduce an innovative cost-effective mobile robot navigation strategy using the features of a specific analog signal from an RFID system. The proposed navigation method is easily implementable on virtually any indoor robotic system with a low computational complexity. Numerical computer simulations are carried out to illustrate the performance of the proposed scheme.

This paper tackles the issue of bandwidth allocation in asynchronous transfer mode (ATM) networks using recently developed tools of computational intelligence. The efficient bandwidth allocation technique implies effective resources utilization of the network. The fluid flow model has been used effectively among other conventional techniques to estimate the bandwidth for a set of connections. However, such methods have been proven to be inefficient at times in coping with varying and conflicting bandwidth requirements of the different services in ATM networks. This inefficiency is due to the computational complexity of the model. To overcome this difficulty, many approximation-based solutions, such as the fluid flow approximation technique, were introduced. Although such solutions are simple, in terms of computational complexity, they nevertheless suffer from potential inaccuracies in estimating the required bandwidth. Soft computing-based bandwidth controllers, such as neural networks- and neurofuzzy-based controllers, have been shown to effectively solve an indeterminate nonlinear input-output (I-O) relations by learning from examples. Applying these techniques to the bandwidth allocation problem in ATM network yields a flexible control mechanism that offers a fundamental tradeoff for the accuracy-simplicity dilemma.

We examine in this article the complex problem of simultaneous position and internal force control in multiple cooperative manipulator systems. This is done in the presence of unwanted parametric and modeling uncertainties as well as external disturbances. A decentralized adaptive hybrid intelligent control scheme is proposed here. The controller makes use of a multi-input multi-output fuzzy logic engine and a systematic online adaptation mechanism. Unlike conventional adaptive controllers, the proposed one does not require a precise model of the system's dynamics. The performance of the proposed controller is compared to that of a well known conventional adaptive controller.

We present soft computing-based results pertaining to the hierarchical tuning process of PID controllers located within the control loop of a class of nonlinear systems. The results are compared with PID controllers implemented either in a stand alone scheme or as a part of conventional gain scheduling structure. This work is motivated by the increasing need in the industry to design highly reliable and efficient controllers for dealing with regulation and tracking capabilities of complex processes characterized by nonlinearities and possibly time varying parameters. The soft computing-based controllers proposed are hybrid in nature in that they integrate within a well-defined hierarchical structure the benefits of hard algorithmic controllers with those having supervisory capabilities. The controllers proposed also have the distinct features of learning and auto-tuning without the need for tedious and computationally extensive online systems identification schemes.

This paper presents a survey about different types of fuzzy information measures. A number of schemes have been proposed to combine the fuzzy set theory and its application to the entropy concept as a fuzzy information measurements. The entropy concept, as a relative degree of randomness, has been utilized to measure the fuzziness in a fuzzy set or system. However, a major difference exists between the classical Shannon entropy and the fuzzy entropy. In fact while the later deals with vagueness and ambiguous uncertainties, the former tackles probabilistic uncertainties (randomness).

This paper presents the application of soft computing-based techniques to the bandwidth allocation (BA) problem in ATM networks. Efficient bandwidth allocation technique implies effective resources utilization. The fluid flow model has been known to be among the most accurate conventional methods to estimate the bandwidth of a set of connections. However, and due to the computational complexity, such methods have been proven to be inefficient in coping with varying and conflicting bandwidth requirements in ATM networks. To overcome this difficulty, many approximation-based solutions were introduced. Although such solutions are not simple, they nevertheless suffer from possible inaccuracy in estimating the required bandwidth. Soft computing-based bandwidth controllers, such as neural networks and neurofuzzy based controllers, have the capability to solve indeterminate non-linear input-output relations by learning from examples. Applying these techniques to the bandwidth allocation problem in ATM network yields a flexible control mechanism that offers a fundamental trade-off for the accuracy-simplicity dilemma.

We discuss here the implementation aspects of recently developed tools of computational intelligence for tackling the issue of joint trajectory generation of a class of multi-joint cooperative robotic systems. This is closely related to the inverse kinematics problem which usually represents a heavy computational burden on the processing power of any complex robotic structure. High nonlinearities, heavy coupling between the degrees of freedom, and time variant configuration of the robot structure heavily contribute to these difficulties. Soft computing techniques have surged in recent years as effective computational tools for emulating the human capabilities when dealing with complex systems. Some of them are used here to synthesize approaches capable of substantially improving solving the inverse kinematics problem for a class of robotic systems and help generating the joint trajectories in a faster way.

A robust control design for high performance joint trajectory tracking of a flexible lightweight manipulator system is proposed. The design is based on a combined controller-observer scheme involving the sliding manifold approach and the optimal interpolation technique. This controller provides the designer with an enhanced joint tracking performance when the system is subject to parametric variations due to structural disturbances caused by link flexibility and load uncertainties. The parametric variations are handled by sliding control and the estimation of the nonlinearly excited elastic dynamics by an optimal interpolator of the structure's dynamic responses. The design procedure is progressive, i.e., we start with a basic controller and then modify it in order to improve the performance. Closed loop simulations with the various designed controllers are used to validate the analytical results and to help choosing the most suitable one.