Automated surveillance cameras equipped with intelligent person detection systems are believed to have reached the maturity required for deployment in Intelligent Transport Systems, Intelligent Plants, and so on. However, recent studies have revealed that Deep Learning Neural Networks (DNN), on which mainstream person detection models are built, are vulnerable to adversarial attacks. Several methods have been proposed to generate adversarial patches that can evade person detectors. Nevertheless, these methods have limitations, as these adversarial patches are either restricted to being presented without any occlusion and placed in the center of the person, or they are too large in size and standing-out in pattern to be easily ignored by human eyes. Therefore, the adversarial patches in previous works did not consider both robustness and stealthiness when human posture changes and the patches are not in the center of person and partially occluded. In this paper, we propose AdvOcl that leverages the learned image manifold of the diffusion model to generate patterns that resemble one kind of the typical textures of daily clothes, such as common floral styles. Moreover, AdvOcl improved the adaptability and adversarial effectiveness by supporting changes in posture and partially occlusion during walking or running with warping and alignment module modeling deformation of clothes. Through extensive quantitative experiments, the results demonstrate the effectiveness of the proposed approach in generating more adversarially effective and naturalistic patterns in occluded scenarios compared to other state-of-the-art patch generation methods.