Three-dimensional (3D) spot beamfocusing (SBF), in contrast to conventional angular-domain beamforming, concentrates radiating power within a very small volume in both radial and angular domains in the near-field zone. Recently the implementation of channel-state-information (CSI)-independent machine learning (ML)-based approaches have been developed for effective SBF using extremely large-scale programmable metasurface (ELPMs). These methods involve dividing the ELPMs into subarrays and independently training them with Deep Reinforcement Learning to jointly focus the beam at the desired focal point (DFP). This paper explores near-field SBF using ELPMs, addressing challenges associated with lengthy training times resulting from independent training of subarrays. To achieve a faster CSI-independent solution, inspired by the correlation between the beamfocusing matrices of the subarrays, we leverage transfer learning techniques. First, we introduce a novel similarity criterion based on the phase distribution image (PDI) of subarray apertures. Then we devise a subarray policy propagation scheme that transfers the knowledge from trained to untrained subarrays. We further enhance learning by introducing quasi-liquid layers as a revised version of the adaptive policy reuse technique. We show through simulations that the proposed scheme improves the training speed about 5 times. Furthermore, for dynamic DFP management, we devised a DFP policy blending process, which augments the convergence rate up to 8-fold.

We propose a novel approach to smart spot-beamforming (SBF) in the Fresnel zone leveraging extremely large-scale programmable metasurfaces (ELPMs). A smart SBF scheme aims to adaptively concentrate the aperture's radiating power exactly at a desired focal point (DFP) in the 3D space utilizing some Machine Learning (ML) method. This offers numerous advantages for next-generation networks including ultra-high-speed wireless communication, location-based multiple access (LDMA), efficient wireless power transfer (WPT), interference mitigation, and improved information security. SBF necessitates ELPMs with precise channel state information (CSI) for all ELPM elements. However, obtaining exact CSI for ELPMs is not feasible in all environments; we alleviate this by developing a novel CSI-independent ML scheme based on the TD3 deep-reinforcement-learning (DRL) method. While the proposed ML-based scheme is well-suited for relatively small-size arrays, the computational complexity is unaffordable for ELPMs. To overcome this limitation, we introduce a modular highly scalable structure composed of multiple sub-arrays, each equipped with a TD3-DRL optimizer. This setup enables collaborative optimization of the radiated power at the DFP, significantly reducing computational complexity while enhancing learning speed. The proposed structure's benefits in terms of 3D spot-like power distribution, convergence rate, and scalability are validated through simulation results.