Panoramic roaming with HMDs provides an immersive VR experience and the realism of user’s navigation in panoramic virtual environments (VEs) can be further enhanced by real-walking locomotion technology. However, the mismatched viewpoint height & fluctuation with the user’s visual senses may lead to perceptual conflicts during real-walking interaction. We conducted a series of user studies to investigate how viewpoint height & fluctuation affect the walking perception of users by real-walking interaction to navigate panoramic VEs and constructed a virtual camera motion model of viewpoint fluctuation and eyetracking compensation for walking behavior in panoramic virtual roaming. It was found that mismatched viewpoint fluctuation triggered generalized discomfort in participants and that participants preferred virtual viewpoint heights below eye level when roaming in panoramic VEs by real-walking. Considering these findings, we proposed viewpoint adaptive adjustment method based on visual manipulation and conducted a preliminary evaluation, which showed that the proposed method can effectively mitigate the conflict between the visual system and other sensory systems. It made that panoramic VEs constructed at a certain fixed height are rendered universal for users with different eyelevel heights and viewpoint fluctuation is matched to the undulation of their center of gravity during real-walking interaction.