AbstractThree‐dimensional display of medical image data has under gone rapid development since its introduction. This article explores the current state‐of‐the‐art of 3‐dimensional imaging applied to surgery planning and discusses the various image acquisition sources: computed tomography (CT), cine CT, magnetic resonance imaging, and ultrasound. Since increased physician involvement is the goal of the surgery planning software described, the future directions discussed emphasize the inclusion of “expert“ knowledge and ways to facilitate realistic interaction for surgery simulation.