Multiagent reinforcement learning (MARL) has been extensively applied to coordination optimization for its task distribution and scalability. The goal of the MARL algorithms for coordination optimization is to learn the optimal joint strategy that maximizes the expected cumulative reward of all agents. Some cooperative MARL algorithms exhibit exciting characteristics in empirical studies. However, the majority of the convergence results are confined to repeated games. Moreover, few MARL algorithms consider adaptation to the switched environments such as the alternation between peak hours and off-peak hours of urban traffic flow or an obstacle suddenly appearing on the planned route for the automated guided vehicle. To this end, we propose a cooperative MARL algorithm known as adaptive individual Q-learning (A-IQL). Each agent updates the Q-function of its own action with period T to adapt to the switched environments. Convergence analysis shows that the optimal joint strategy can be obtained in stochastic games with deterministic state transitions occurring in chronological order. The influence of period T on convergence is studied through a fictitious stochastic game. The efficacy of the A-IQL algorithm is validated through two switched environments-the distributed sensor network (DSN) task and the target transportation task.

We propose InNeRF360, an automatic system that accu-rately removes text-specified objectsfrom 360° Neural Radi-ance Fields (NeRF). The challenge is to effectively remove objects while inpainting perceptually consistent content for the missing regions, which is particularly demanding for existing NeRF models due to their implicit volumetric rep-resentation. Moreover, unbounded scenes are more prone to floater artifacts in the inpainted region than frontal-facing scenes, as the change of object appearance and background across views is more sensitive to inaccurate segmentations and inconsistent inpainting. With a trained NeRF and a text description, our method efficiently removes specified ob-jects and inpaints visually consistent content without arti-facts. We apply depth-space warping to enforce consistency across multiview text-encoded segmentations, and then re-fine the inpainted NeRF model using perceptual priors and 3D diffusion-based geometric priors to ensure visual plau-sibility. Through extensive experiments in segmentation and inpainting on 360° and frontal-facing NeRFs, we show that our approach is effective and enhances NeRF's ed-itability. Project page: https://ivr1.github.io/InNeRF360/.

We propose NEMTO, the first end-to-end neural rendering pipeline to model 3D transparent objects with complex geometry and unknown indices of refraction. Commonly used appearance modeling such as the Disney BSDF model cannot accurately address this challenging problem due to the complex light paths bending through refractions and the strong dependency of surface appearance on illumination. With 2D images of the transparent object as input, our method is capable of high-quality novel view and relighting synthesis. We leverage implicit Signed Distance Functions (SDF) to model the object geometry and propose a refraction-aware ray bending network to model the effects of light refraction within the object. Our ray bending network is more tolerant to geometric inaccuracies than traditional physically-based methods for rendering transparent objects. We provide extensive evaluations on both synthetic and real-world datasets to demonstrate our high-quality synthesis and the applicability of our method.

Image enhancement is an essential and longstanding task in computer vision, in which the 3D lookup table (3DLUT) is widely used due to its powerful mapping capability and high time efficiency. However, the 3DLUT generally requires a large parameter amount since it caches the mapping results for all colors of the entire discrete color space with a 3D array. As a result, standard 3DLUT-based enhancement methods suffer from heavy memory footprints, limiting their practical applications. Based on the analyses of the inherent low grid utilization rate of 3DLUT, we propose HashLUT, an efficient hash form of the standard 3DLUT, and further build a lightweight real-time image enhancement network that adaptively learns HashLUTs and handles hash collisions end-to-end. Experiments on two benchmarks demonstrate that our model achieves comparable enhancement performance to the state-of-the-art methods with significantly fewer parameters. Source codes are available at https://github.com/Xian-Bei.

In the traditional gradient descent (T-GD) algorithm, the convergence rate is strongly depend on the condition number of the information matrix: a larger condition number leads to a poor optimal convergence factor infimum$\mu _{\text{op}}$, which sets a convergence rate ceiling. That is, once the information matrix is fixed, the convergence factor of the T-GD algorithm reaches at most the infimum$\mu _{\text{op}}$. This article studies a varying infimum gradient descent algorithm, which can move down the infimum by using different order iterative preconditioning methods, as follows: first, for infinite iterative algorithm, the infimum becomes smaller and smaller with the increased iteration numbers; second, for finite iterative algorithm, the infimum is equal to zero, and the parameter estimates can be obtained in only one iteration; third, construct an adaptive interval between zero and$\mu _{\text{op}}$, which can establish a link between the least squares and T-GD algorithms. Based on the varying infimum gradient descent algorithm, researchers can adaptively choose preconditioning matrices for different kinds of models on a case by case basis. The convergence analysis and simulation examples show effectiveness of the proposed algorithms.

Object interaction recommendation from Internet of Things (IoT) is a crucial basis for IoT related applications. While many efforts are devoted to suggesting object for interaction, the majority of models rigidly infer relationships from human social network, overlook the neighbor information in their own object social network and the correlation of multiple heterogeneous features, and ignore multi-scale structure of the network. To tackle the above challenges, this work focuses on object social network, formulates object interaction recommendation as multi-modals object ranking, and proposes Multi-Modal Attention-based Hierarchical Graph Neural Network (MM-AHGNN), that describes object with multiple knowledge of actions and pairwise interaction feature, encodes heterogeneous actions with multi-modal encoder, integrates neighbor information and fuses correlative multi-modal feature by intra-modal hybrid-attention graph convolution and inter-modal transformer encoder, and employs multi-modal multi-scale encoder to integrate multi-level information, for suggesting object interaction more flexibly. With extensive experiments on real-world datasets, we prove that MMAHGNN achieves better recommendation results (improve 3-4% HR@3 and 4-5% NDCG@3) than the most advanced baseline. To our knowledge, our MM-AHGNN is the first research in GNN design for object interaction recommendation. Source codes are available at: https://github.com/gaosaroma/MM-AHGNN.

Gradient-based method has been extensively used in today's multiagent reinforcement learning (MARL). In a gradient-based MARL algorithm, each agent updates its parameterized strategy in the direction of the gradient of some performance index. However, studies on the convergence of the existing gradient-based MARL algorithms for identical interest games are quite few. In this article, we propose a policy gradient potential (PGP) algorithm that takes PGP as the source of information for guiding the strategy update, as opposed to the gradient itself, to learn the optimal joint strategy that has a maximal global reward. Since the payoff matrix and the joint strategy are often unavailable to the learning agents in reality, we consider the probability of obtaining the maximal reward as the performance index. Theoretical analysis of the PGP algorithm on the continuous model involving an identical interest repeated game shows that if the component action of every optimal joint action is unique, the critical points corresponding to all optimal joint actions are asymptotically stable. The PGP algorithm is experimentally studied and compared against other MARL algorithms on two commonly used collaborative tasks-the robots leaving a room task and the distributed sensor network task, as well as a real-world minefield navigation problem where only local state and local reward information are available. The results show that the PGP algorithm outperforms the other algorithms in terms of the cumulative reward and the number of time steps used in an episode.

Multiagent reinforcement learning (MARL) has been extensively used in many applications for its tractable implementation and task distribution. Learning automata, which can be classified under MARL in the category of independent learner, are used to obtain the optimal joint action or some type of equilibrium. Learning automata have the following advantages. First, learning automata do not require any agent to observe the action of any other agent. Second, learning automata are simple in structure and easy to be implemented. Learning automata have been applied to function optimization, image processing, data clustering, recommender systems, and wireless sensor networks. However, a few learning automata-based algorithms have been proposed for optimization of cooperative repeated games and stochastic games. We propose an algorithm known as learning automata for optimization of cooperative agents (LA-OCA). To make learning automata applicable to cooperative tasks, we transform the environment to a P-model by introducing an indicator variable whose value is one when the maximal reward is obtained and is zero otherwise. Theoretical analysis shows that all the strict optimal joint actions are stable critical points of the model of LA-OCA in cooperative repeated games with an arbitrary finite number of players and actions. Simulation results show that LA-OCA obtains the pure optimal joint strategy with a success rate of 100% in all of the three cooperative tasks and outperforms the other algorithms in terms of learning speed.

This article concerns a novel auxiliary-model-based expectation maximization (EM) estimation method for Hammerstein systems with data loss by extending the EM method to estimate models with multiple parameter vectors. The novel EM method relaxes the requirements on an autoregression model with one parameter vector, interactively maximizes the expectation over multiple parameter vectors in a more general model, and uses the output of an auxiliary model to substitute the missing outputs in the information vector in iteration processes. A numerical simulation is employed to demonstrate the effectiveness of the proposed novel EM method.

In the camera equipment manufacturing industry, the exposure calibration is one of the basic steps for manufacturers to consider before launching their products to the market. To this end, a method that can objectively and automatically assess the exposure levels of images taken by the camera is highly desired. However, few studies have been conducted in this area. In this paper, we attempt to solve this issue to some extent and our contributions are twofold. Firstly, in order to facilitate the study of image exposure assessment, an Image Exposure Database$(IE_{ps}D)$is established. In this database, there are 15, 582 images with various exposure levels, and for each image there is an associated subjective exposure score which could reflect its perceptual exposure level. Secondly, we propose a novel highly accurate DCNN-based model, namely$IE_{ps}M$(Image Exposure Metric), to predict the exposure level of a given image.

In this paper, we propose a multiagent reinforcement learning algorithm dealing with fully cooperative tasks. The algorithm is called frequency of the maximum reward Q-learning (FMRQ). FMRQ aims to achieve one of the optimal Nash equilibria so as to optimize the performance index in multiagent systems. The frequency of obtaining the highest global immediate reward instead of immediate reward is used as the reinforcement signal. With FMRQ each agent does not need the observation of the other agents' actions and only shares its state and reward at each step. We validate FMRQ through case studies of repeated games: four cases of two-player two-action and one case of three-player two-action. It is demonstrated that FMRQ can converge to one of the optimal Nash equilibria in these cases. Moreover, comparison experiments on tasks with multiple states and finite steps are conducted. One is box-pushing and the other one is distributed sensor network problem. Experimental results show that the proposed algorithm outperforms others with higher performance.

This study examines the electromyogram (EMG)-torque relation for chronic stroke survivors using a novel EMG complexity representation. Ten stroke subjects performed a series of submaximal isometric elbow flexion tasks using their affected and contralateral arms, respectively, while a 20-channel linear electrode array was used to record surface EMG from the biceps brachii muscles. The sample entropy (SampEn) of surface EMG signals was calculated with both global and local tolerance schemes. A regression analysis was performed between SampEn of each channel's surface EMG and elbow flexion torque. It was found that a linear regression can be used to well describe the relation between surface EMG SampEn and the torque. Each channel's root mean square (RMS) amplitude of surface EMG signal in the different torque level was computed to determine the channel with the highest EMG amplitude. The slope of the regression (observed from the channel with the highest EMG amplitude) was smaller on the impaired side than on the nonimpaired side in 8 of the 10 subjects, regardless of the tolerance scheme (global or local) and the range of torques (full or matched range) used for comparison. The surface EMG signals from the channels above the estimated muscle innervation zones demonstrated significantly lower levels of complexity compared with other channels between innervation zones and muscle tendons. The study provides a novel point of view of the EMG-torque relation in the complexity domain, and reveals its alterations post stroke, which are associated with complex neural and muscular changes post stroke. The slope difference between channels with regard to innervation zones also confirms the relevance of electrode position in surface EMG analysis.

This letter focuses on identification problems of a Hammerstein-Wiener system with an output error linear element embedded between two static nonlinear elements. A hierarchical least squares algorithm is presented for the Hammerstein-Wiener system by using the auxiliary model identification idea and the hierarchical identification principle. The major contributions of the present study are that the identification model is formulated by using the auxiliary model identification idea (the estimate of the unknown internal variable is replaced with the output of an auxiliary model) and that the bilinear parameter vectors in the identification model are estimated by using the hierarchical identification principle. The proposed hierarchical identification approach is computationally more efficient than the existing over-parametrization method.