We address the problem of motion flow estimation for a scene with multiple moving objects, observed from a possibly moving camera. We take as input a (possibly sparse) noisy velocity field, as obtained from local matching, produce a set of motion boundaries, and identify pixels with different velocities in overlapping layers. For a fixed observer, these overlapping layers capture occlusion information. For a moving observer, further processing is required to segment independent objects and infer structure. Unlike previous approaches, which generate layers by iteratively fitting data to a set of predefined parameters, we instead find boundaries first, then infer regions and address occlusion overlap relationships. All computational steps use a common framework of tensors to represent velocity information, together with saliency (confidence), and uncertainty. Communication between sites is performed by convolution-like tensor voting. The scheme is non-iterative, and the only free parameter is the scale, related to neighborhood size. We illustrate the approach with results obtained from synthetic sequences and from real images. The quantitative results compare favorably with those of other methods, especially in the presence of occlusion.