Laparoscope-holding robots significantly enhance the stability and precision of visualization in minimally invasive surgeries. Most existing robots of this kind depend on visual servo systems and struggle with efficient, rapid adjustments in the field-of-view (FOV), especially when identifying organs and needles outside the FOV. This paper presents a laparoscope-holding robot system capable of employing both vision-driven and force-driven mechanisms for continuous and large-scale FOV adjustments, respectively. The system features an integrated tactile handle, enabling the reception of human-robot interaction forces during surgical navigation. We propose a hybrid control method that leverages both force and vision inputs for laparoscopic FOV adjustments. This approach integrates a virtual wrench, generated from visual information, and an interaction wrench, obtained from the tactile handle, into the robot's dynamic model, which complies with remote center of motion constraints. The interaction wrench's gain is adjusted with the gripping force on the integrated tactile handle, ensuring that unintended movements caused by accidental contacts are prevented, thus safeguarding operational safety. The proposed method eliminates the need to switch control modes, enabling simultaneous visual tracking and tactile interaction guidance. Experimental results demonstrate that the proposed method not only allows for FOV adjustments with surgical instrument guiding but also adapts well to large-scale FOV adjustment tasks.