Asynchronous action coordination presents a pervasive challenge in Multi-Agent Systems (MAS), which can be represented as a Stackelberg game (SG). However, the scalability of existing Multi-Agent Reinforcement Learning (MARL) methods based on SG is severely restricted by network architectures or environmental settings. To address this issue, we propose the Stackelberg Decision Transformer (STEER). It efficiently manages decision-making processes by incorporating the hierarchical decision structure of SG, the modeling capability of autoregressive sequence models, and the exploratory learning methodology of MARL. Our approach exhibits broad applicability across diverse task types and environmental configurations in MAS. Experimental results demonstrate both the convergence of our method towards Stackelberg equilibrium strategies and its superiority over strong baselines in complex scenarios.

In multi-agent reinforcement learning (MARL), self-interested agents attempt to establish equilibrium and achieve coordination depending on game structure. However, existing MARL approaches are mostly bound by the simultaneous actions of all agents in the Markov game (MG) framework, and few works consider the formation of equilibrium strategies via asynchronous action coordination. In view of the advantages of Stackelberg equilibrium (SE) over Nash equilibrium, we construct a spatio-temporal sequential decision-making structure derived from the MG and propose an N-level policy model based on a conditional hypernetwork shared by all agents. This approach allows for asymmetric training with symmetric execution, with each agent responding optimally conditioned on the decisions made by superior agents. Agents can learn heterogeneous SE policies while still maintaining parameter sharing, which leads to reduced cost for learning and storage and enhanced scalability as the number of agents increases. Experiments demonstrate that our method effectively converges to the SE policies in repeated matrix game scenarios, and performs admirably in immensely complex settings including cooperative tasks and mixed tasks.

This work considers the problem of deep reinforcement learning (RL) with long time dependencies and sparse rewards, as are found in many hard exploration games. A graph-based representation is proposed to allow an agent to perform self-navigation for environmental exploration. The graph representation not only effectively models the environment structure, but also efficiently traces the agent state changes and the corresponding actions. By encouraging the agent to earn a new influence-based curiosity reward for new game observations, the whole exploration task is divided into sub-tasks, which are effectively solved using a unified deep RL model. Experimental evaluations on hard exploration Atari Games demonstrate the effectiveness of the proposed method. The source code and learned models will be released to facilitate further studies on this problem.

This paper proposes a high throughput architecture for AES encryption/decryption targeting on the recent FPGAs with 6-input LUTs. Unlike previous works which share multiplicative inverse logics to realize SubBytes and InvSubBytes, the proposed architecture directly employs the look-up-table based Sbox for both SubBytes and InvSubBytes. Efficient reordering and merging techniques are applied to achieve a highly integrated encryption/decryption datapath with reduced area and delay. By sharing Sbox instead of inversion, the encryption datapath remains simple with unchanged MixColumns. For decryption, the linear operations including two inverse Affine functions and InvMixColumns between SubBytes are merged into a new InvMixColumns (NIMC-I) transformation. The NIMC-I is further optimized to reduce resources and share logics with MixColumns. Through loop-unrolling and fair pipelining, the proposed 3-stage subpipelined design achieves 68.82 Gbps on XC7VX330T using 3930 slices and the 5-stage deep-pipelined one achieves 76.19 Gbps on XC6VLX240T using 4426 slices, which outperform previous equivalent designs in terms of throughput per area.

Inversion over GF(2m) is crucial for cryptographic applications such as elliptic curve cryptography. The commonly used Itoh-Tsujii algorithm (ITA) computes the inversion by an entirely sequential process consisting of multiplications and squarings. In this paper, we first propose a modified ITA algorithm (MITA) for inversion with polynomial basis (PB). The MITA reduces the required clock cycles of ITA inversion by enabling the parallel computation between part of multiplications and squarings. Furthermore, we generalize the MITA to inversion with arbitrary addition chains. Several criteria are proposed to find the optimal addition chains (OACs) leading to the fastest inverters with given hardware resources. Implemented on Xilinx Virtex-4 FPGA, the proposed inversion architecture with a digit-serial multiplier achieves averagely 61% faster speed with 69% less resources than previous designs with normal basis. Using a fully combinational multiplier, the OAC inverters outperform existing PB-based designs by at least 60.9%, 35.1%, 94.9% for m = 163, 233, 283 respectively in terms of area-time product.

This paper proposes an efficient pipelined architecture of elliptic curve scalar multiplication (ECSM) over GF(2m). The architecture uses a bit-parallel finite field (FF) multiplier accumulator (MAC) based on the Karatsuba-Ofman algorithm. The Montgomery ladder algorithm is modified for better sharing of execution paths. The data path in the architecture is well designed, so that the critical path contains few extra logic primitives apart from the FF MAC. In order to find the optimal number of pipeline stages, scheduling schemes with different pipeline stages are proposed and the ideal placement of pipeline registers is thoroughly analyzed. We implement ECSM over the five binary fields recommended by the National Institute of Standard and Technology on Xilinx Virtex-4 and Virtex-5 field-programmable gate arrays. The three-stage pipelined architecture is shown to have the best performance, which achieves a scalar multiplication over GF(2163) in 6.1 μs using 7354 Slices on Virtex-4. Using Virtex-5, the scalar multiplication for m = 163, 233, 283, 409, and 571 can be achieved in 4.6, 7.9, 10.9, 19.4, and 36.5 μs, respectively, which are faster than previous results.