A tool was developed for structured and detailed analysis of video data from user tests of interactive systems. It makes use of a table format for representing an interaction at multiple levels of abstraction. Interactions are segmented based on threshold times for pauses between actions. Usability problems are found using a list of observable indications for the occurrence of problems. The tool was evaluated by having two analysts apply it to three data sets from user tests on two different products. The segmentation technique proved to yield meaningful segments that helped in understanding the interaction. The interaction table was explicit enough to discuss in detail what had caused the differences in the analysts' lists of usability problems. The results suggested that the majority of differences were caused by unavoidable differences in interpretations of subjects' behaviour and that only minor improvements should be expected by refining the tool.