Surgical approaches tailored to an individual patient's anatomy and pathology have become standard in neurosurgery. Precise preoperative planning of these procedures, however, is necessary to achieve an optimal therapeutic effect. Therefore, multiple radiological imaging modalities are used prior to surgery to delineate the patient's anatomy, neurological function, and metabolic processes. Developing a three-dimensional perception of the surgical approach, however, is traditionally still done by mentally fusing multiple modalities. Concurrent 3D visualization of these datasets can, therefore, improve the planning process significantly. In this paper we introduce an application for planning of individual neurosurgical approaches with high-quality interactive multimodal volume rendering. The application consists of three main modules which allow to (1) plan the optimal skin incision and opening of the skull tailored to the underlying pathology; (2) visualize superficial brain anatomy, function and metabolism; and (3) plan the patient-specific approach for surgery of deep-seated lesions. The visualization is based on direct multi-volume raycasting on graphics hardware, where multiple volumes from different modalities can be displayed concurrently at interactive frame rates. Graphics memory limitations are avoided by performing raycasting on bricked volumes. For preprocessing tasks such as registration or segmentation, the visualization modules are integrated into a larger framework, thus supporting the entire workflow of preoperative planning.

Virtual endoscopy has proven to be a very powerful tool in endoscopic surgery. However, most virtual endoscopy systems are restricted to rendering isosurfaces or require segmentation in order to visualize additional objects behind occluding tissue. This paper presents a system for real-time perspective direct volume and isosurface rendering, which allows to simultaneously visualize both the interesting tissue and everything that is behind. Large volume data can be viewed seamlessly from inside or outside the volume without any pre-computation or segmentation. Our system uses a novel ray-casting pipeline for GPUs that has been optimized for the requirements of virtual endoscopy and also allows easy incorporation of auxiliary geometry, e.g., for displaying parts of the endoscopic device, pointers, or grid lines for orientation purposes. We present three main applications of this system and the underlying ray-casting algorithm. Although our ray-casting approach is of general applicability, we have specifically applied it to virtual colonoscopy, virtual angioscopy, and virtual pituitary surgery.

Endoscopy has recently been introduced to endonasal transsphenoidal pituitary surgery as a minimally invasive procedure for the removal of various kinds of pituitary tumors. To reduce morbidity and mortality with this new technique, the surgeon must be well-trained and well-prepared. Virtual endoscopy can be beneficial as a tool for training, preoperative planning, and intraoperative support. This paper introduces STEPS, a virtual endoscopy system designed to aid surgeons in getting acquainted with the endoscopic view of the anatomy, the handling of instruments, the transsphenoidal approach, and challenges associated with the procedure. STEPS also assists experienced surgeons in planning a real endoscopic intervention by getting familiar with the individual patient anatomy, identifying landmarks, planning the approach, and deciding upon the ideal target position of the actual surgical activity. The application provides interactive visualization, navigation, and perception aids and the possibility of simulating the procedure, including haptic feedback and simulation of surgical instruments.

Endonasal transsphenoidal pituitary surgery is a minimally invasive endoscopic procedure, applied to remove various kinds of pituitary tumors. To reduce the risk associated with this treatment, the surgeon must be skilled and well-prepared. Virtual endoscopy can be beneficial as a tool for training, preoperative planning and intraoperative support. This work introduces STEPS, a virtual endoscopy system designed to aid surgeons in getting acquainted with the endoscopic view, the handling of instruments, the transsphenoidal approach and challenges associated with the procedure. STEPS also assists experienced surgeons in planning a real endoscopic intervention by getting familiar with the individual patient anatomy, identifying landmarks, planning the approach and deciding upon the ideal target position of the actual surgical activity. Besides interactive visualization using two different first-hit ray casting techniques, the application provides navigation and perception aids and the possibility to simulate the procedure, including haptic feedback and simulation of surgical instruments.