A method is presented to (semi-)automatically generate 3D models of urban buildings from aerial images. The process starts by selecting (with a mouse click) one or more roofs in the image(s). The program then automatically delineates the roof structures in the images and extracts straight line segments which are matched across different views and reconstructed in 3D. These 3D line segments are then grouped into coplanar sets and combined into polygonal shapes. Each polygon hypothesis is verified with respect to the 3D reconstruction and the original image data; and, if necessary, corrected accordingly. Subsequently, the polygons are combined into a polyhedral roof model. The emphasis here is on extracting the correct topology of the roof structure. Metric accuracy is obtained by back-projecting the model into the images and minimizing the total reprojection error. Finally, the building model is completed by adding vertical walls to the reconstructed roof. The approach is tested on a state-of-the-art dataset of aerial images of residential areas in Brussels.