With the stagnation of Moore’s Law scaling, efficient hardware architectures employing compute-in-memory paradigms have become increasingly crucial to sustain AI innovations. This motivates the development of high-throughput architectures with balanced energy-latency profiles through machine learning algorithms. However, for human-in-the-loop optimization methods, human labor is involved in most of the iterations, whereas for automated methods, either expert domain knowledge is required in the design or the search space is relatively small. To address these challenges, we propose ArchERL, an evolutionary reinforcement learning (ERL) framework that represents the first application of ERL to general hardware architecture design. Specifically, ArchERL tightly couples population-based evolutionary algorithm for global exploration with an actor-critic reinforcement learning module for prior-light policy refinement, and ArchERL employs periodic weight synchronization and gradient feedback between the two modules to achieve efficient collaborative search and rapid convergence. To evaluate the proposed method, extensive experiments are conducted in multiple simulated hardware environments, including the DRAM controller and DNN mapping. The results demonstrate that ArchERL achieves leading performance and outperforms widely used baselines in both efficiency and effectiveness.

This article focuses on maximizing the anti-jamming sum throughput in a time division multiple access (TDMA)-based reconfigurable intelligent surfaces (RIS)-assisted wireless powered Internet of Things (WP-IoT) network. In this setup, multiple IoT devices harvest energy from wireless energy stations (WES) and then utilize the collected energy to upload their own data to an information receiver (IR). The network also includes a jammer that sends jamming signals to the IR, and a RIS is deployed to mitigate this jamming effect and enhance the sum throughput. This study addresses both an upper bound design and a robust design with fractional nonlinear energy harvesting model. The primary optimization goal is to maximize the anti-jamming sum throughput, with the constraints of RIS phase shifts and time scheduling. For both designs, closed-form expressions for time scheduling are derived using the Lagrangian duality and Karush-Kuhn-Tucker (KKT) conditions. The quadratic transformation (QT) technique is used to handle fractional functions within the optimization. Furthermore, the phase shifts are optimized iteratively using the element-wise block coordinate descent (EBCD) and Riemannian manifold optimization (RMO) algorithms. Simulation results are presented to validate the effectiveness of the proposed approaches.

In this paper, we present an open-source pure-Python library called PyPop7 for black-box optimization (BBO). As population-based methods (e.g., evolutionary algorithms, swarm intelligence, and pattern search) become increasingly popular for BBO, the design goal of PyPop7 is to provide a unified API and elegant implementations for them, particularly in challenging high-dimensional scenarios. Since these population-based methods easily suffer from the notorious curse of dimensionality owing to random sampling as one of core operations for most of them, recently various improvements and enhancements have been proposed to alleviate this issue more or less mainly via exploiting possible problem structures: such as, decomposition of search distribution or space, low-memory approximation, low-rank metric learning, variance reduction, ensemble of random subspaces, model self-adaptation, and fitness smoothing. These novel sampling strategies could better exploit different problem structures in high-dimensional search space and therefore they often result in faster rates of convergence and/or better qualities of solution for large-scale BBO. Now PyPop7 has covered many of these important advances on a set of well-established BBO algorithm families and also provided an open-access interface to adding the latest or missed black-box optimizers for further functionality extensions. Its well-designed source code (under GPL-3.0 license) and full-fledged online documents (under CC-BY 4.0 license) have been freely available at https://github.com/Evolutionary-Intelligence/pypop and https://pypop.readthedocs.io, respectively.

Master-standby database deployment is a commonly adopted database architecture in modern production environments, thanks to its fault tolerance and high availability. However, despite the architecture's widespread application in various online services, relatively few research efforts have been made to improve its overall query performance. When a sequence of queries arrive, existing methods of scheduling them across master and standby servers still rely on rules or heuristics, which may overlook some potential optimization directions such as buffer utilization. If we can efficiently reuse the database buffers resident in memory through intelligent query scheduling, the average response time of user queries can be significantly reduced as opposed to reading data from disk. To address this issue, in this paper, we introduce a new buffer-aware query scheduling system named Laser. The system integrates a lightweight learned model that can directly map a query to the data blocks it accesses. Then, based on the predictions of the queries, we develop adaptive query scheduling algorithms to perform query allocation as well as query rearrangements, aiming to maximize the overall buffer hit rate while also maintaining load balance. The proposed system requires no pre-training, and can adjust to unseen workloads on the fly through constant model updates and query re-allocation. In our experiments, we observe a reduction of ~80% in query completion time compared to other traditional heuristic-based methods, with relatively low extra overhead added to the critical path of query execution.

Intelligent reflecting surface (IRS) has emerged as a promising technique to control wireless propagation wireless environment for improving the communication performance cost-effectively and extending the wireless signal coverage of access point (AP). In order to reduce the path-loss of the cascaded user-IRS-AP channels, the IRS-integrated AP architecture has been proposed to deploy the antenna array of the AP and the IRSs within the same antenna radome. To reduce the pilot overhead for estimating all IRS-involved channels, in this paper, we propose a novel codebook-based IRS reflection design for the IRS-integrated AP to enhance the coverage performance in a given area. In particular, the codebook consisting of a small number of codewords is designed offline by employing an efficient sector division strategy based on the azimuth angle. To ensure the performance of each sector, we optimize its corresponding codeword for IRS reflection pattern to maximize the sector-min-average-effective-channel-power (SMAECP) by applying the alternating optimization (AO) and semidefinite relaxation (SDR) methods. With the designed codebook, the AP performs the IRS reflection training by sequentially applying all codewords and selects the one achieving the best communication performance for data transmission. Numerical results show that our proposed codebook design can enhance the average channel power of the whole coverage area, as compared to the system without IRS. Moreover, the proposed codebook-based IRS reflection design is compared with several benchmark schemes, which achieves significant performance gain in both single-user and multi-user transmissions.

Intelligent reflecting surface (IRS) has emerged as a cost-efficient technique to improve the wireless network’s capacity and performance. Existing works on IRS have mainly considered IRS being deployed in the environment to dynamically control the wireless channels between the base station (BS) and its served users in favor of their communications. In contrast, we propose in this paper a new integrated IRS-BS architecture by deploying IRSs inside the BS’s antenna radome to directly reconfigure the signal radiation to/from the BS’s antennas. In other words, the IRSs can be considered as auxiliary passive arrays with real-time reconfigurability equipped at the BS to enhance its communication performance cost-effectively. Since the distance between the integrated IRSs and BS’s antenna array is practically small (in the order of several to tens of wavelengths), the path loss among them is significantly reduced as compared to conventional IRS deployed much farther away from the BS, while the real-time control of the IRS’s reflection by the BS becomes easier to implement. However, the resultant near-field channel model also becomes drastically different from its far-field counterpart for conventional far-away IRSs in the literature. Thus, we propose an element-wise channel model for IRS to characterize the channel vector between each single-antenna user and the antenna array of the BS, which includes the direct (without any IRS’s reflection) as well as the single and double IRS-reflection channel components. Based on this channel model, we formulate a problem to optimize the reflection coefficients of all IRS reflecting elements for maximizing the uplink sum-rate of the users. By considering two typical cases with/without perfect channel state information (CSI) at the BS, the formulated problem is solved efficiently by adopting the successive refinement method and iterative random phase algorithm (IRPA), respectively. Numerical results validate the substantial capacity gain of the integrated IRS-BS architecture over the conventional multi-antenna BS without integrated IRS. Moreover, the proposed algorithms significantly outperform other benchmark schemes in terms of sum-rate, and the IRPA without CSI can approach the performance upper bound with perfect CSI as the training overhead increases.

Intelligent reflecting surface (IRS) has emerged as a promising technique to enhance wireless communication performance cost-effectively. The existing literature has mainly considered IRS being deployed near user terminals to improve their performance. However, this approach may incur a high cost if IRSs need to be densely deployed in the network to cater to random user locations. To avoid such high deployment cost, in this paper we consider a new IRS aided wireless network architecture, where IRSs are deployed in the vicinity of each base station (BS) to assist in its communications with distributed users regardless of their locations. Besides significantly enhancing IRSs’ signal coverage, this scheme helps reduce the IRS-associated channel estimation overhead as compared to conventional user-side IRSs, by exploiting the nearly static BS-IRS channels over short distance. For this scheme, we propose a new two-stage transmission protocol to achieve IRS channel estimation and reflection optimization for uplink data transmission efficiently. In addition, we propose effective methods for solving the user-IRS association problem based on long-term/statistical channel knowledge and the selected user-IRS-BS cascaded channel estimation problem. Finally, all IRSs’ passive reflections are jointly optimized with the BS’s multi-antenna receive combining to maximize the minimum achievable rate among all users for data transmission. Numerical results show that the proposed co-site-IRS empowered BS scheme can achieve significant performance gains over the conventional BS without co-site IRS and existing schemes for IRS channel estimation and reflection optimization, thus enabling an appealing low-cost and high-performance BS design for future wireless networks.

This paper considers an unmanned aerial vehicle (UAV)-enabled uplink non-orthogonal multiple-access (NOMA) system, where multiple users on the ground send independent messages to a UAV via NOMA transmission. We aim to design the UAV's dynamic maneuver in real time for maximizing the sum-rate throughput of all ground users over a finite time horizon. Different from conventional offline designs considering static user locations under deterministic or stochastic channel models, we consider a more challenging scenario with mobile users and segmented channel models, where the UAV only causally knows the users' (moving) locations and channel state information (CSI). Under this setup, we first propose a new approach for UAV dynamic maneuver design based on reinforcement learning (RL) via Q-learning. Next, in order to further speed up the convergence and increase the throughput, we present an enhanced RL-based approach by additionally exploiting expert knowledge of well-established wireless channel models to initialize the Q-table values. Numerical results show that our proposed RL-based and enhanced RL-based approaches significantly improve the sum-rate throughput, and the enhanced RL-based approach considerably speeds up the learning process owing to the proposed Q-table initialization.

This paper investigates a new scenario of spectrum sharing between unmanned aerial vehicle (UAV) and terrestrial wireless communication, in which a cognitive/secondary UAV transmitter communicates with a ground secondary receiver (SR), in the presence of a number of primary terrestrial communication links that operate over the same frequency band. We exploit the UAV’s mobility in three-dimensional (3D) space to improve its cognitive communication performance while controlling the co-channel interference at the primary receivers (PRs), such that the received interference power at each PR is below a prescribed threshold termed as interference temperature (IT). First, we consider the quasi-stationary UAV scenario, where the UAV is placed at a static location during each communication period of interest. In this case, we jointly optimize the UAV’s 3D placement and power control to maximize the SR’s achievable rate, subject to the UAV’s altitude and transmit power constraints, as well as a set of IT constraints at the PRs to protect their communications. Second, we consider the mobile UAV scenario, in which the UAV is dispatched to fly from an initial location to a final location within a given task period. We propose an efficient algorithm to maximize the SR’s average achievable rate over this period by jointly optimizing the UAV’s 3D trajectory and power control, subject to the additional constraints on UAV’s maximum flying speed and initial/final locations. Finally, numerical results are provided to evaluate the performance of the proposed designs for different scenarios, as compared to various benchmark schemes. It is shown that in the quasi-stationary scenario the UAV should be placed at its minimum altitude while in the mobile scenario the UAV should adjust its altitude along with horizontal trajectory, so as to maximize the SR’s achievable rate in both scenarios.