Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Andy Lippman

dblp:60/2616 · also Andrew Lippman · DBLP profile ↗
← Back
54ranked-venue papers
3as first author
3since 2021 · last 2026
0000-0003-2541-2266ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 28 · 3 first-authorArtificial intelligence and machine learning · 12Computer networks · 7Human-computer interaction and ubiquitous computing · 5 · 1 first-author · 1 since 2021Applied, interdisciplinary, general and emerging computing · 5 · 2 since 2021Databases, data management, data science and information retrieval · 3

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Human-computer interaction and pervasive computing
6 papers
Human-AI interaction · 36% Learning and educational technologies · 36% Collaborative and social computing · 14%
Computer networks
5 papers
Physical-layer communications · 56% Internet of things and sensor networks · 15% Transport protocols and congestion control · 15%
Artificial intelligence
9 papers
Language models and text generation · 53% Video understanding and tracking · 16% Face, body and person analysis · 10%
Computer graphics and multimedia
10 papers
Computational photography and imaging · 40% Multimedia analysis and retrieval · 35% Image and video processing · 15%

Topics — the 30 heaviest of 58, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Computational photography and imaging › camera model
rolling shutter
0.112012
VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter · ISMAR 2012
Accessibility and assistive technology
image description
0.112012
VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter · ISMAR 2012
Ubiquitous computing and smart environments
incentive mechanism
0.112012
Barter: mechanism design for a market incented wisdom exchange · CSCW 2012
Collaborative and social computing › knowledge management
knowledge sharing
0.112012
Barter: mechanism design for a market incented wisdom exchange · CSCW 2012
Collaborative and social computing
online communities
0.112012
Barter: mechanism design for a market incented wisdom exchange · CSCW 2012
Collaborative and social computing › user-generated content
location sharing
0.112010
'Follow me': a web-based, location-sharing architecture for large, indoor environments · WWW 2010
Physical-layer communications › beamforming › adaptive beamforming
blind beamforming
0.112010
Simple, zero-feedback, distributed beamforming with unsynchronized carriers · IEEE J. Sel. Areas Commun. 2010
Physical-layer communications › beamforming
distributed beamforming
0.112010
Simple, zero-feedback, distributed beamforming with unsynchronized carriers · IEEE J. Sel. Areas Commun. 2010
Internet of things and sensor networks
wireless sensor network
0.112010
Simple, zero-feedback, distributed beamforming with unsynchronized carriers · IEEE J. Sel. Areas Commun. 2010
Physical-layer communications › cooperative communication
cooperative diversity
0.112006
A simple Cooperative diversity method based on network path selection · IEEE J. Sel. Areas Commun. 2006
Physical-layer communications › cooperative communication
distributed space-time coding
0.112006
A simple Cooperative diversity method based on network path selection · IEEE J. Sel. Areas Commun. 2006
Wireless networking
relay selection
0.112006
A simple Cooperative diversity method based on network path selection · IEEE J. Sel. Areas Commun. 2006
Computer vision › Face, body and person analysis
face recognition
0.112005
A multiresolution manifold distance for invariant image similarity · IEEE Trans. Multim. 2005
Image and video processing
image similarity
0.112005
A multiresolution manifold distance for invariant image similarity · IEEE Trans. Multim. 2005
Multimedia analysis and retrieval
shot boundary detection
0.122000
Statistical models of video structure for content analysis and characterization · IEEE Trans. Image Process. 2000
Bayesian Video Shot Segmentation · NIPS 2000
Computer vision › Video understanding and tracking
motion segmentation
0.022001
Empirical Bayesian Motion Segmentation · IEEE Trans. Pattern Anal. Mach. Intell. 2001
Empirical Bayesian EM-based Motion Segmentation · CVPR 1997
Interaction techniques and input
large display interaction
0.012012
VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter · ISMAR 2012
Transport protocols and congestion control › transport protocols
multicast transport
0.012002
A Multicast Transport Protocol for Cooperative Media Distribution · ICNP 2002
Transport protocols and congestion control › real-time communication
real-time streaming
0.012002
A Multicast Transport Protocol for Cooperative Media Distribution · ICNP 2002
Transport protocols and congestion control › congestion control fairness
TCP-friendly congestion control
0.012002
A Multicast Transport Protocol for Cooperative Media Distribution · ICNP 2002
Physical-layer communications › synchronization
carrier synchronization
0.012010
Simple, zero-feedback, distributed beamforming with unsynchronized carriers · IEEE J. Sel. Areas Commun. 2010
Privacy and data protection
location privacy
0.012010
'Follow me': a web-based, location-sharing architecture for large, indoor environments · WWW 2010
Information retrieval › image retrieval
content-based image retrieval
0.012000
A Probabilistic Architecture for Content-Based Image Retrieval · CVPR 2000
Information retrieval › retrieval models
probabilistic retrieval model
0.012000
A Probabilistic Architecture for Content-Based Image Retrieval · CVPR 2000
Multimedia analysis and retrieval
video content analysis
0.012000
Statistical models of video structure for content analysis and characterization · IEEE Trans. Image Process. 2000
Multimedia analysis and retrieval › video content analysis
video structure analysis
0.012000
Bayesian Video Shot Segmentation · NIPS 2000
Information retrieval
interactive information retrieval
0.011999
Learning from User Feedback in Image Retrieval Systems · NIPS 1999
Information retrieval
relevance feedback
0.011999
Learning from User Feedback in Image Retrieval Systems · NIPS 1999
Machine learning › Probabilistic and Bayesian machine learning › structured models › latent variable model › mixture model
hierarchical mixture model
0.011998
Learning Mixture Hierarchies · NIPS 1998
Machine learning › Representation and self-supervised learning › hierarchical representation
hierarchical representation learning
0.011998
Learning Mixture Hierarchies · NIPS 1998

Methods — techniques the papers use, named apart from their topics

longitudinal study · 2.0decentralized web architecture · 0.3bluetooth detection · 0.3rolling-shutter decoding · 0.3mechanism design · 0.3information economics · 0.3binary coding · 0.3tangent distance · 0.1signal alignment · 0.1robust estimation · 0.1multiresolution framework · 0.1statistical model · 0.1information-theoretic analysis · 0.1bayesian formulation · 0.1empirical bayes · 0.0user modeling · 0.0relevance feedback · 0.0expectation-maximization · 0.0
YearPublicationVenuePosition
2026 Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills
abstract
Given the growing prevalence of fake information, including increasingly realistic AI-generated news, there is an urgent need to train people to better evaluate and detect misinformation. While interactions with AI have been shown to durably reduce people’s beliefs in false information, it is unclear whether these interactions also teach people the skills to discern false information themselves. We conducted a month-long study where 67 participants classified news headline-image pairs as real or fake, discussed their assessments with an AI system, followed by an unassisted evaluation of unseen news items to measure accuracy before, during, and after AI assistance. While AI assistance produced immediate improvements during AI-assisted sessions (+21% average), participants’ unassisted performance on new items declined significantly by 15.3% in week 4 compared to week 0. These results indicate that while AI may help immediately, it may ultimately degrade long-term misinformation detection abilities.
Anku Rani, Valdemar Danry, Paul Pu Liang, Andy Lippman, Pattie Maes
CHI4
2024 Form Forge: Latent Space Exploration of Architectural Forms via Explicit Latent Variable Manipulation
Kevin Dunnell, Andy Lippman
ICCC2
2023 Latent Lab: Large Language Models for Knowledge Exploration
Kevin Dunnell, Trudy Painter, Andrew Stoddard, Andy Lippman
ICCC4
2016 8K Time into Space: Web-Based Interactive Storyboard with Playback for Hundreds of Videos
abstract
8K Time into Space uses the extreme size and resolution of emerging modern large-format displays to present a video sequence as an array of shorter "snippets" so that the entire event can be seen at once. A TV program or movie is subdivided into short, repetitive sequences that each represents successive segments of the longer program. This "storyboard" is then arrayed across a large, touch-sensitive display to present the entire movie at once. A viewer can indicate any part of the presentation and the display will enlarge the sequences near that point in time. Or, the movie can occupy the entire screen at a point selected from the array. The 8K, 33-million pixel screen stresses the number of live video windows that graphics controllers and HTML5 can support, so we developed a novel scheme that treats each segment as a sprite, thus allowing interaction and 15fps (frames per second) playback with stability that corresponds to human perception. This paper describes the video playback architecture. Moreover, we applied this system to an actual TV program to confirm the availability.
Hisayuki Ohmata, Andy Lippman
ISM2
2013 CoCam: A collaborative content sharing framework based on opportunistic P2P networking
abstract
In this paper, we propose CoCam, a framework for mobile phones that enables uncoordinated real-time image and video collaboration between different users sharing the same context, in the same physical location. CoCam addresses the complexity and difficulty introduced when users wish to collaborate, create, and share media contents at the same time. CoCam is based on a middleware that creates a self-organizing ad-hoc network within the context of a shared event in a proximal physical space (a common scene). This middleware automatically handles the context detection, as well as the network configuration and peer discovery. It also enables real-time content sharing while reducing the burden of complicated settings and configurations by the users. As a result of operation tests and user study with the prototype implementation, we have verified that CoCam is feasible and has the potential to enrich the users' experience when sharing contents in an event scenario.
Eyal Toledano, Dan Sawada, Andy Lippman, Henry Holtzman, Federico Casalegno
CCNC3
2013 CoCam: Real-time photo sharing based on opportunistic P2P networking
abstract
We describe a mobile application for sharing user-authored photo content in realtime called CoCam. CoCam is a collaborative content sharing framework based on opportunistic P2P proximal networking. CoCam users who are located in the same physical space can automatically share the photos they create as well as receive photos from other users around them. Since CoCam is based on an opportunistic P2P network middleware, users are not required to know each other in advance. It is also not necessary for them to agree on the same service provider nor coordinate the network configuration, infrastructure and security settings. This middleware automatically discovers other peers and handles the organization of ad-hoc network connections. With CoCam, we demonstrate that users are able to share and enjoy shared photos and video streams without the effort of manual setup and cost associated with the 3G/4G network.
Eyal Toledano, Dan Sawada, Andy Lippman, Henry Holtzman, Federico Casalegno
CCNC3
2012 Barter: mechanism design for a market incented wisdom exchange
abstract
Information markets benefit the communities they serve by facilitating electronic distributed exchange and enhancing knowledge sharing, innovation, and productivity. This research explores innovative market mechanisms to build incentives while encouraging pro-social behavior. A key advantage of this study is a direct appeal to theories of information economics and macro policies to market design. We built and deployed a web-based software platform called Barter at several universities. Preliminary analysis of user data helps test information market effectiveness and illustrate effects of various market interventions. We present our design framework, demonstrate why such an architecture provides sustainable incentives, and list key findings learned in the process of system deployment.
Dawei Shen, Marshall W. van Alstyne, Andy Lippman, Hind Benbya
CSCW3
2012 VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter
abstract
We show a new visible tagging solution for active displays which allows a rolling-shutter camera to detect active tags from a relatively large distance in a robust manner. Current planar markers are visually obtrusive for the human viewer. In order for them to be read from afar and embed more information, they must be shown larger thus occupying valuable physical space on the design. We present a new active visual tag which utilizes all dimensions of color, time and space while remaining unobtrusive to the human eye and decodable using a 15fps rolling-shutter camera. The design exploits the flicker fusion-frequency threshold of the human visual system, which due to the effect of metamerism, can not resolve metamer pairs alternating beyond 120Hz. Yet, concurrently, it is decodable using a 15fps rolling-shutter camera due to the effective line-scan speed of 15×400 lines per second. We show an off-the-shelf rolling-shutter camera can resolve the metamers flickering on a television from a distance over 4 meters. We use intelligent binary coding to encode digital positioning and show potential applications such as large screen interaction. We analyze the use of codes for locking and tracking encoded targets. We also analyze the constraints and performance of the sampling system, and discuss several plausible application scenarios.
Grace Woo, Andy Lippman, Ramesh Raskar
ISMAR2
2011 The Glass Infrastructure: Using Common Sense to Create a Dynamic, Place-Based Social Information System
abstract
Most organizations have a wealth of knowledge about themselves available online, but little for a visitor to interact with on-site. At the MIT Media Lab, we have designed and deployed a novel intelligent signage system, the Glass Infrastructure (GI) that enables small groups of users to physically interact with this data and to discover the latent connections between people, projects, and ideas. The displays are built on an adaptive, unsupervised model of the organization developed using dimensionality reduction and common sense knowledge which automatically classifies and organizes the information.The GI is currently in daily use at the lab. We discuss the AI models development, the integration of AI into an HCI interface, and the use of the GI during the labs peak visitor periods. We show that the GI is used repeatedly by lab visitors and provides a window into the workings of the organization.
Catherine Havasi, Richard Borovoy, Boris Kizelshteyn, Polychronis Ypodimatopoulos, Jon Ferguson, Henry Holtzman, Andy Lippman, Dan Schultz, Matthew Blackshaw, Greg T. Elliott, Chaki Ng
IAAI7
2011 Zero-Feedback, Collaborative Beamforming for Emergency Radio: Asymptotic Analysis
Aggelos Bletsas, Andy Lippman, John N. Sahalos
Mob. Networks Appl.2
2010 'Follow me': a web-based, location-sharing architecture for large, indoor environments
abstract
We leverage the ubiquity of bluetooth-enabled devices and propose a decentralized, web-based architecture that allows users to share their location by following each other in the style of Twitter. We demonstrate a prototype that operates in a large building which generates a dataset of detected bluetooth devices at a rate of ~30 new devices per day, including the respective location where they were last detected. Users then query the dataset using their unique bluetooth ID and share their current location with their followers by means of unique URIs that they control. Our separation between producers (the building) and consumers (the users) of bluetooth device location data allows us to create socially-aware applications that respect user's privacy while limiting the software necessary to run on mobile devices to just a web browser.
Polychronis Ypodimatopoulos, Andy Lippman
WWW2
2010 Simple, zero-feedback, distributed beamforming with unsynchronized carriers
abstract
This work studies zero-feedback distributed beamforming; we are motivated by scenarios where the links between destination and all distributed transmitters are weak, so that no reliable communication in the form of pilot signals or feedback messages can be assumed. Furthermore, we make the problem even more challenging by assuming no specialized software/hardware for distributed carrier synchronization; we are motivated by ultra-low complexity transceivers. It is found that zero-feedback (i.e. blind), constructive, distributed signal alignment at the destination is possible; the proposed scheme exploits lack of carrier synchronization among M distributed transmitters and provides beamforming gains. Possible applications include reachback communication in low-cost sensor networks with simple (i.e. conventional, no carrier frequency/phase adjustment capability) radio transceivers.
Aggelos Bletsas, Andy Lippman, John N. Sahalos
IEEE J. Sel. Areas Commun.2
2009 Connected Consumption: The Hidden Networks of Consumption
abstract
In this paper, we present the Connected Consumption Network (CCN) that allows a community of consumers to collaboratively sense the market from a mobile device, enabling more informed financial decisions in geo-local context. The mobile application allows one to log one's wish list and itemized list of transactions to form a social network around the list of interests. Individuals can share this data to inform and guide others in a timely, personal and contextual manner when they are shopping for a product or seeking a service. It can also help people connect opportunistically in a local area to make group purchases, to pick up an item for a friend, and to perform reverse auctions. We present the design, architecture and concept prototype. We simulate a social network with three months of existing credit/debit card transaction data in various geographical areas to analyze the mutual information and recommendations that can be shared among networked consumers.
Kwan Hong Lee, Dawei Shen, Andy Lippman, David P. Reed 0001, Hans D. Schumacher
CCNC3
2008 Demonstrating SnapN'Share: Content Sharing over a Socially Informed, Hyper-Local, Face-to-Face Networking Platform
abstract
In this demonstration we present SnapN'Share, the first sample application of a new platform implementing a wireless, peer-to-peer information system that bypasses the need for any centralized servers, coordination, or administration. SnapN'share runs on wireless pocket devices as well as stationary computers, and enables users to generate and seamlessly share content with different groups and communities that they belong to, as they come into close vicinity of their peers. At CCNC'08 we demonstrate the capabilities and features of the system, as well as its advantages when compared to traditional central-infrastructure based platforms.
Nadav Aharony, Andy Lippman, David P. Reed 0001
CCNC2
2008 Interference-Aware Minimum Energy Path Routing in Multi-Hop Wireless Ad Hoc Networks
abstract
Interference and energy-restriction are two of the most important technical challenges in the design of networking protocols in wireless ad hoc networks. The interference is an inherent problem for wireless communication and it is not easy to control. On the other hand, the lifetime of an ad hoc network totally relies on the power consumption of the network nodes, each of which is normally equipped with a limited power supply such as batteries. In this paper, we develop algorithms for interference-aware minimum energy path (MEP) routing in wireless ad hoc networks. The empirical results show that the interference-aware MEP tends to avoid congested core in the network and leads to better load balance and overall power savings, compared with its noninterference-aware counterparts.
Fulu Li, Andy Lippman
CCNC2
2008 Presence Information in Mobile Mesh Networks
abstract
Given a mobile mesh network, we attempt to address the question of which nodes are accessible at any point in time through some path from a given node. From the viewpoint of some node in the mesh network, we claim that it is impossible to deterministically infer which other nodes are still accessible through some path from the current node. By formulating the arrival of presence updates from various nodes in the network as a Poisson process, we introduce a probabilistic presence mechanism that offers user-defined levels of assurance on whether a remote node is still accessible and provide a distance metric for each node, based on the corresponding arrival rate of presence updates.
Polychronis Ypodimatopoulos, David P. Reed 0001, Andy Lippman, Michail Bletsas
CCNC3
2008 Analysis on Probabilistic View Coverage for Image Sensing - A Geometric Approach
abstract
In this paper we study the probabilistic view coverage problem for image sensing in wireless sensor networks. The view coverage of an image sensor network determines the quality of the surveillance services that an image sensor network can provide. In this paper, we present an indepth analysis on probabilistic view coverage in an image sensor network, where omnidirectional image sensors are randomly dropped to a given field and the locations of the image sensors may not be immediately known. We intend to answer the following question: if we randomly drop a given number of image sensors into a targeted field, what is the probability that a given area of interest can be effectively imaged and view-covered. The key to our analytical approach is to cast the probabilistic view coverage problem in wireless image sensor networks as a geometric one and then use the geometric techniques to find the solution. The analysis in this paper provides probabilistic assurance of the view coverage that one can expect for random dropping of omnidirectional image sensors into a given field.
Fulu Li, Ramesh Raskar, Andy Lippman
IPCCC3
2008 Cocktail Party on the Mobile
abstract
We present the architecture, user interface and prototype implementation of fluid voice, a proximity based mobile group communication system for opportunistic social exchanges. It is designed for coordinating people in outdoor settings, spreading news during emergency responses and supporting impromptu social exchanges by exploiting the inherent broadcast nature of wireless and mobility of people. It supports diverse media: live audio, asynchronous audio, text messages, audio messages, audio polls and wish lists. In this paper, we focus on the audio support and the design for intermittent connectivity characteristics of mobile systems with limited range. Users can transition from live to asynchronous audio communication in a fluid manner depending on the wireless environment. When every node is able to hear each other, the receiver mixes all sources and provides the user with a "push to listen" interface to "tune in" to who they want to listen to. When the wireless channel is congested, it transitions to a "push to talk" interface (without taking over the whole channel) to reduce collisions while allowing people to barge in. When nodes cannot hear each other, Fluid Voice caches content for future delivery using a dynamic multi-stream mixer with time shifting (Tivo) functionality. The system was prototyped on the Nokia N810's for real world testing and we discuss its technical and social limitations.
Kwan Hong Lee, Andy Lippman, Thiago Santos
ISM2
2006 Energy-efficient cooperative routing in multi-hop wireless ad hoc networks
abstract
We study the routing problem for multi-hop wireless ad hoc networks based on cooperative transmission. We prove that the minimum energy cooperative path (MECP) routing problem, i.e., using cooperative radio transmission to find the best route with the minimum energy cost from a source node to a destination node, is NP-complete. We thus propose a cooperative shortest path (CSP) algorithm that uses the Dijkstra's algorithm as the basic building block and reflects the cooperative transmission properties in the relaxation procedure. Simulation results show that with more nodes added in the network, our approach achieves more energy saving compared to traditional non-cooperative shortest path algorithms. Another interesting observation is that the proposed algorithm achieves better fairness among different nodes with denser networks. Implementation issues are also discussed
Fulu Li, Kui Wu 0001, Andy Lippman
IPCCC3
2006 Minimum Energy Cooperative Path Routing in Wireless Networks: An Integer Programming Formulation
abstract
We study the problem of minimum energy cooperative path (MECP) routing in wireless networks. The joint problem of cooperative routing in wireless networks that combines route selection and the transmit diversity was recently presented in A. Khandani et al. (2003). Several heuristic algorithms were also proposed in A. Khandani et al. (2003) to approximate the minimum energy cooperative path, while it has not been possible to judge the quality of the solutions with respect to the optimal. In this paper, we present an integer programming formulation of the MECP problem towards an optimal solution for a network where the locations of the nodes are known a priori and fixed
Fulu Li, Andy Lippman, Kui Wu 0001
VTC Spring2
2006 Cooperative diversity with opportunistic relaying
abstract
In this paper, we present single-selection-opportunistic-relaying with decode-and-forward (DaF) and amplify-and-forward (AaF) protocols under an aggregate power constraint. We show that opportunistic DaF relaying is equivalent to the outage bound of the optimal DaF strategy using all potential relays. We further show that opportunistic AaF relaying is outage-optimal with single-relay selection and significantly outperforms an AaF strategy with multiple-relay (MR) transmissions, in the presence of limited channel knowledge. These findings reveal that cooperative diversity benefits (under an aggregate power constraint) are useful even when cooperative relays choose not to transmit but rather choose to cooperatively listen; they act as passive relays and give priority to the transmission of a single opportunistic relay
Aggelos Bletsas, Hyundong Shin, Moe Z. Win, Andy Lippman
WCNC4
2006 A simple Cooperative diversity method based on network path selection
abstract
Cooperative diversity has been recently proposed as a way to form virtual antenna arrays that provide dramatic gains in slow fading wireless environments. However, most of the proposed solutions require distributed space-time coding algorithms, the careful design of which is left for future investigation if there is more than one cooperative relay. We propose a novel scheme that alleviates these problems and provides diversity gains on the order of the number of relays in the network. Our scheme first selects the best relay from a set of M available relays and then uses this "best" relay for cooperation between the source and the destination. We develop and analyze a distributed method to select the best relay that requires no topology information and is based on local measurements of the instantaneous channel conditions. This method also requires no explicit communication among the relays. The success (or failure) to select the best available path depends on the statistics of the wireless channel, and a methodology to evaluate performance for any kind of wireless channel statistics, is provided. Information theoretic analysis of outage probability shows that our scheme achieves the same diversity-multiplexing tradeoff as achieved by more complex protocols, where coordination and distributed space-time coding for M relay nodes is required, such as those proposed by Laneman and Wornell (2003). The simplicity of the technique allows for immediate implementation in existing radio hardware and its adoption could provide for improved flexibility, reliability, and efficiency in future 4G wireless systems.
Aggelos Bletsas, Ashish Khisti, David P. Reed 0001, Andy Lippman
IEEE J. Sel. Areas Commun.4
2005 A multiresolution manifold distance for invariant image similarity
abstract
Accounting for spatial image transformations is a requirement for multimedia problems such as video classification and retrieval, face/object recognition or the creation of image mosaics from video sequences. We analyze a transformation invariant metric recently proposed in the machine learning literature to measure the distance between image manifolds - the tangent distance (TD) - and show that it is closely related to alignment techniques from the motion analysis literature. Exposing these relationships results in benefits for the two domains. On one hand, it allows leveraging on the knowledge acquired in the alignment literature to build better classifiers. On the other, it provides a new interpretation of alignment techniques as one component of a decomposition that has interesting properties for the classification of video. In particular, we embed the TD into a multiresolution framework that makes it significantly less prone to local minima. The new metric - multiresolution tangent distance (MRTD) - can be easily combined with robust estimation procedures, and exhibits significantly higher invariance to image transformations than the TD and the Euclidean distance (ED). For classification, this translates into significant improvements in face recognition accuracy. For video characterization, it leads to a decomposition of image dissimilarity into "differences due to camera motion" plus "differences due to scene activity" that is useful for classification. Experimental results on a movie database indicate that the distance could be used as a basis for the extraction of semantic primitives such as action and romance.
Nuno Vasconcelos, Andy Lippman
IEEE Trans. Multim.2
2002 A Multicast Transport Protocol for Cooperative Media Distribution
abstract
This paper describes the main ideas of DRMTP (Distributed Real-time Multicast Transport Protocol), an adaptive application-level protocol which allows cooperative multicast of real-time streams. Our protocol is designed for peer-to-peer operation, where multiple sources residing on end-user machines may be present in the network, and is based on a dynamic stream aggregation and congestion control scheme. The protocol solves the problem of network asymmetry that plagues residential broadband networks and allows for fast fail-over and adaptation to departure of source nodes, mitigating the reliability and upstream capacity problems of end-user nodes. Coupled with traffic localization, stream patching, and TCP-friendly congestion control, we design a protocol that enables scalable soft real-time media distribution in a completely decentralised, serverless fashion.
Dimitris Vyzovitis, Andy Lippman
ICNP2
2001 Empirical Bayesian Motion Segmentation
abstract
We introduce an empirical Bayesian procedure for the simultaneous segmentation of an observed motion field and estimation of the hyperparameters of a Markov random field prior. The new approach exhibits the Bayesian appeal of incorporating prior beliefs, but requires only a qualitative description of the prior, avoiding the requirement for a quantitative specification of its parameters. This eliminates the need for trial-and-error strategies for the determination of these parameters and leads to better segmentations.
Nuno Vasconcelos, Andy Lippman
IEEE Trans. Pattern Anal. Mach. Intell.2
2000 A Probabilistic Architecture for Content-Based Image Retrieval
abstract
The design of an effective architecture for content-based retrieval from visual libraries requires careful consideration of the interplay between feature selection, feature representation, and similarity metric. We present a solution where all the modules strive to optimize the same performance criteria: the probability of retrieval error. This solution consists of a Bayesian retrieval criteria (shown to generalize the most prevalent similarity metrics in current use) and an embedded mixture representation over a multiresolution feature space (shown to provide a good trade-off between retrieval accuracy, invariance, perceptual relevance of similarity, and complexity). The new representation extends standard models (histogram and Gaussian) by providing simultaneous support for high-dimensional features and multi-modal densities and performs well on color texture, and generic image databases.
Nuno Vasconcelos, Andy Lippman
CVPR2
2000 Learning Over Multiple Temporal Scales in Image Databases
Nuno Vasconcelos, Andy Lippman
ECCV (1)2
2000 Distributional Clustering for Efficient Content-Based Retrieval of Images and Video
abstract
We present an approach to clustering images for efficient retrieval using relative entropy. We start with the assumption that visual features are represented by probability densities and develop clustering algorithms for probability densities (for example, normalized histograms are crude approximations of probability densities). These clustering algorithms are then used for efficient retrieval of images and video.
Giridharan Iyengar, Andy Lippman
ICIP2
2000 A Unifying View of Image Similarity
abstract
We study solutions to the problem of evaluating image similarity in the context of content-based image retrieval (CBIR). Retrieval is formulated as a classification problem, where the goal is to minimize probability of retrieval error. It is shown that this formulation establishes a common ground for comparing similarity functions, exposes assumptions hidden behind in most commonly used ones, enables a critical analysis of their relative merits, and determines the retrieval scenarios for which each may be most suited. We conclude that most of the current similarity functions are sub-optimal special cases of the Bayesian criteria that results from explicit minimization of error probability.
Nuno Vasconcelos, Andy Lippman
ICPR2
2000 Bayesian Video Shot Segmentation
abstract
Prior knowledge about video structure can be used both as a means to improve the peiformance of content analysis and to extract features that allow semantic classification. We introduce statistical models for two important components of this structure, shot duration and activity, and demonstrate the usefulness of these models by introducing a Bayesian formulation for the shot segmentation problem. The new formulations is shown to extend standard thresholding methods in an adaptive and intuitive way, leading to improved segmentation accuracy.
Nuno Vasconcelos, Andy Lippman
NIPS2
2000 Statistical models of video structure for content analysis and characterization
abstract
Content structure plays an important role in the understanding of video. In this paper, we argue that knowledge about structure can be used both as a means to improve the performance of content analysis and to extract features that convey semantic information about the content. We introduce statistical models for two important components of this structure, shot duration and activity, and demonstrate the usefulness of these models with two practical applications. First, we develop a Bayesian formulation for the shot segmentation problem that is shown to extend the standard thresholding model in an adaptive and intuitive way, leading to improved segmentation accuracy. Second, by applying the transformation into the shot duration/activity feature space to a database of movie clips, we also illustrate how the Bayesian model captures semantic properties of the content. We suggest ways in which these properties can be used as a basis for intuitive content-based access to movie libraries.
Nuno Vasconcelos, Andy Lippman
IEEE Trans. Image Process.2
1999 Learning from User Feedback in Image Retrieval Systems
Nuno Vasconcelos, Andy Lippman
NIPS2
1998 A Spatiotemporal Motion Model for Video Summarization
abstract
The compact description of a video sequence through a single image map and a dominant motion has applications in several domains, including video browsing and retrieval, compression, mosaicing, and visual summarization. Building such a representation requires the capability to register all the frames with respect to the dominant object in the scene, a task which has been, in the past, addressed through temporally localized motion estimates. In this paper, we show how the lack of temporal consistency associated with such estimates can undermine the validity of the dominant motion assumption, leading to oscillation between different scene interpretations and poor registration. To avoid this oscillation, we augment the motion model with a generic temporal constraint which increases the robustness against competing interpretations, leading to more meaningful content summarization.
Nuno Vasconcelos, Andy Lippman
CVPR2
1998 A Bayesian Framework for Semantic Content Characterization
abstract
Current systems for content filtering, browsing, and retrieval rely on low-level image descriptors which are unintuitive for most users. In this paper, we propose an alternative framework that exploits the structured nature of most content sources to achieve semantic content characterization, and lead to much more meaningful user interaction. Computationally, this framework is based on the principles of Bayesian inference and can be implemented efficiently with Bayesian networks. As an illustration of its potential we apply it to the domain of movie databases.
Nuno Vasconcelos, Andy Lippman
CVPR2
1998 A Bayesian Framework for Content-Based Indexing and Retrieval
abstract
Summary form only given. One of the important requirements for practical retrieval systems is the ability to jointly address the issues of indexing and compression. By formulating query by example as a problem of Bayesian inference and establishing a link between probability density estimation and vector quantization, we have previously introduced a representation that leads to very efficient procedures for indexing and retrieval directly in the compressed domain without compromise of the coding efficiency. In this paper, we build on the potential of the Bayesian formulation to support sophisticated inference, to incorporate this representation in a very flexible indexing and retrieval framework that (1) leads to intuitive retrieval procedures, (2) can integrate different content modalities to eliminate some of the strongest limitations of the query by example paradigm, and (3) supports statistical learning of all the model parameters and can, therefore, be trained automatically.
Nuno Vasconcelos, Andy Lippman
Data Compression Conference2
1998 Bayesian Modeling of Video Editing and Structure: Semantic Features for Video Summarization and Browsing
Nuno Vasconcelos, Andy Lippman
ICIP (3)2
1998 Learning Mixture Hierarchies
Nuno Vasconcelos, Andy Lippman
NIPS2
1997 Empirical Bayesian EM-based Motion Segmentation
abstract
A recent trend in motion-based segmentation has been to rely on statistical procedures derived from expectation-maximization (EM) principles. EM-based approaches have various advantages for segmentation, such as proceeding by taking non-greedy soft decisions regarding the assignment of pixels to regions, or allowing the use of sophisticated priors capable of imposing spatial coherence on the segmentation. A practical difficulty with such priors is, however the determination of appropriate values for their parameters. The authors exploit the fact that the EM framework is itself suited for empirical Bayesian data analysis to develop an algorithm that finds the estimates of the prior parameters which best explain the observed data. Such an approach maintains the Bayesian appeal of incorporating prior beliefs, but requires only a qualitative description of the prior avoiding the requirement of a quantitative specification of its parameters. This eliminates the need for trial-and-error strategies for parameter determination and leads to better segmentation with fewer iterations.
Nuno Vasconcelos, Andy Lippman
CVPR2
1997 Library-based Coding: a Representation for Efficient Video Compression and Retrieval
abstract
The ubiquity of networking and computational capacity associated with the new communications media unveil a universe of new requirements for image representation. Among such requirements is the ability of the representation used for coding to support higher-level tasks such as content-based retrieval. We explore the relationships between probabilistic modeling and data compression to introduce a representation-library-based coding-which, by enabling retrieval in the compressed domain, satisfies this requirement. Because it contains an embedded probabilistic description of the source, this new representation allows the construction of good inference models without compromise of compression efficiency, leads to very efficient procedures for query and retrieval, and provides a framework for higher level tasks such as the analysis and classification of video shots.
Nuno Vasconcelos, Andy Lippman
Data Compression Conference2
1997 Towards Semantically Meaningful Feature Spaces for the Characterization of Video Content
abstract
Efficient procedures for browsing, filtering, sorting or retrieving pictorial content require accurate content characterization. Of particular interest are representations based on semantically meaningful feature spaces, capable of capturing properties such as violence, sex or profanity. In this work we report on a first step towards this goal, the design of a stochastic model for video editing which provides a transformation from the image space to a low-dimensional feature space where categorization by degree of action can be easily accomplished.
Nuno Vasconcelos, Andy Lippman
ICIP (1)2
1997 Content-Based Pre-Indexed Video
abstract
The viability of large distributed image databases is strongly dependent on the development of new image representations capable of providing support for extended functionality, directly in the compressed domain. We have previously introduced one such representation (library-based coding) which we now augment with statistical pre-indexing schemes, automatically built at the time of encoding, that provides several layers of content description allowing efficient content-based retrieval and summarization.
Nuno Vasconcelos, Andy Lippman
ICIP (2)2
1997 Multiresolution Tangent Distance for Affine-invariant Classification
Nuno Vasconcelos, Andy Lippman
NIPS2
1996 VideoBook: an experiment in characterization of video
abstract
We present a framework for content based query and retrieval of information from large video databases. This framework enables content based retrieval of video sequences by characterizing the sequences using motion, texture and colorimetry cues. This characterization is biologically inspired and results in a compact parameter space where every segment of video is represented by an 8 dimensional vector. Searching and retrieval is done in real-time with high accuracy in this parameter space. The present version of the VideoBook has 165 video sequences, each 15 seconds long at 30 frames a second representing storage of 65 Giga bytes. The VideoBook is able to search and retrieve video sequences with 92% accuracy in real-time. Experiments thus demonstrate that the characterization is capable of extracting higher level structure from raw pixel values.
Giridharan Iyengar, Andy Lippman
ICIP (3)2
1996 Frame-free video
abstract
Current digital video representations emphasize compression efficiency, lacking some of the flexibility required for interactive manipulation of digital bitstreams. We present a video representation which can encompass both space and time, providing a temporally coherent description of video sequences. The video sequence is segmented into its component objects, and the trajectory of each object throughout the sequence is described parametrically, according to a spatiotemporal motion model. Since the motion model is a continuous function of time, the video representation becomes frame-rate independent and the temporal resolution a user-definable parameter. i.e. the traditional sequence of frames, with the temporal structure hardcoded into the bitstream at the time of production, is replaced by a collection of scene snapshots assembled on the fly by the decoder. This enables random access and temporal scalability, the major building blocks for interactivity.
Nuno Vasconcelos, Andy Lippman
ICIP (3)2
1996 Video semantics
Andy Lippman, Philippe Aigrain, Arcot Desai Narasimhalu, Alex Pentland
Signal Process. Image Commun.1
1996 Video on demand
Andy Lippman, Richard Nicol, Masahiro Shibata
Signal Process. Image Commun.1
1995 Spatio-temporal segmentation based on motion and static segmentation
abstract
The problem of segmenting an image sequence in terms of regions characterized by a coherent motion is among the most challenging in image sequence analysis. This paper proposes a new technique which sequentially refines the segmentation and the motion estimation by combining static segmentation and motion information. The motion is robustly computed by a global estimation which remove the camera motion, followed by a local estimation using a matching technique and a robust estimator. Simulation results show the efficiency of the proposed technique.
Frédéric Dufaux, Fabrice Moscheni, Andy Lippman
ICIP3
1995 Coding for content: enhanced resolution from coding
abstract
This paper describes a new video coding algorithm [Kermode 1994] that extends the traditional hybrid transform coding techniques used in MPEG in two ways. First, pictures are analyzed and segmented into a number of regions roughly corresponding to the background and any moving foreground objects. These regions are then encoded independently from one another thereby incorporating degree of useful content into the encoded representation beyond the pictures themselves. Second, the new algorithm also stores the background regions from several frames which can then be used to construct an artificial image at a higher resolution than that of the original background. Furthermore, both of these features are implemented at no overall cost in compression efficiency. In fact the new algorithm still outperforms MPEG, even for particularly degenerate sequences where the underlying assumption used for segmentation is broken.
Roger G. Kermode, Andy Lippman
ICIP (3)2
1995 Spatiotemporal model-based optic flow estimation
abstract
We introduce a spatiotemporal model-based algorithm capable of providing estimates of optic flow which are coherent along a set of video frames. The algorithm is based on a spatiotemporal motion model that consists of a quadratic constraint in time and an affine constraint in space. Optic flow is computed through a delayed-decision process that incorporates knowledge about both image correlation along time, and the goodness of fit to the underlying motion-model. The temporal coherence and parametric nature of the recovered optic flow can facilitate interactive access to the video stream and improve the efficiency of tasks such as video compression, interpolation or classification.
Nuno Vasconcelos, Andy Lippman
ICIP2
1995 Digital television: a perspective
abstract
Commercial television was standardized originally mainly for entertainment, sports, and news, using over-the-air broadcasts. While a few upward compatible changes have been added over the years, the standard has remained essentially unchanged. Moreover, no other standard has had so many products and services built around it and withstood such a period of rapid technological change. Digital television promises to disturb the current equilibrium in consumer television and in the broader media industries. It is forcing a convergence with the simultaneous expansion of personal computing into the home and the commercialization of research networks. Digitizing television in a cost-effective manner and integrating it with computers, telecommunication networks, and consumer products will produce a large array of new products and services. The paper describes digital television, factors affecting it, its potential, and some major issues in its evolution.>
Arun Netravali, Andy Lippman
Proc. IEEE2
1994 Library-based image coding
abstract
We explore a video coding algorithm that uses long term memory in addition to motion compensated prediction to build a model from video frames for coding. The long term memory, modeled on the cache used in operating systems and searched by a vector quantization formalism, holds recurring visual elements such as cyclically revealed information and actors that persist though several images. This video "library" exploits persistence to gain coding efficiency. Further, the existence of such a library can facilitate interactive and content-based operations on the video sequence such as browsing, searching by content, categorizing and enhancing the pictorial quality of the images.>
Nuno Vasconcelos, Andy Lippman
ICASSP (5)2
1994 Re-Codable Video
abstract
This paper describes a two-stage digital video encoding technique that is compliant with emerging international standards and is optimized for multiple uses of the material. Applications include production, post-processing, archival storage, and distribution through diverse channels. The first phase of the encoding is computationally intensive and the second phase is bandwidth conserving; all motion analysis is done in phase one. The intermediate format resulting from phase one uses less storage than the raw data but more than a fully compressed image sequence. It is designed to serve as the source for distribution through diverse channels: transcoding from the intermediate representation to various compressed data streams is both simple and more efficient than single-pass encoding.>
Manuela Pereira, Andy Lippman
ICIP (2)2
1983 Artists interfacing with technology: Basic concepts of digital creation
abstract
Article Artists interfacing with technology: Basic concepts of digital creation Share on Authors: Larry Cuba independent artist independent artistView Profile , Darcy Gerbarg New York University New York UniversityView Profile , Andrew Lippman MIT MITView Profile , Dan Sandin University of Illinois - Chicago University of Illinois - ChicagoView Profile , Chairman: Frank Dietrich West Coast University West Coast UniversityView Profile Authors Info & Claims SIGGRAPH '83: Proceedings of the 10th annual conference on Computer graphics and interactive techniquesJuly 1983 Pages 311–313https://doi.org/10.1145/800059.801162Published:01 July 1983 0citation503DownloadsMetricsTotal Citations0Total Downloads503Last 12 Months5Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Frank Dietrich, Larry Cuba, Darcy Gerbarg, Andy Lippman, Dan Sandin
SIGGRAPH4
1980 Movie-maps: An application of the optical videodisc to computer graphics
abstract
An interactive, dynamic map has been built using videodisc technology to engage the user in a simulated “drive” through an unfamiliar space. The driver, or map reader, is presented with either sparsely sampled sequences of images taken by single frame cameras that replicate actual imagery from a space, or with computer synthesized replicas of those images. The reader may control the speed, route, angle of view and mode of presentation of this information and may thus tour the area. In addition, he may access spatially stored ancillary data stored in the buildings or in locales in the environment. This basic map is being enhanced to provide topographic views, and to incorporate optical and electronic image processing to provide a more responsive, visually complete representation of an environment.
Andy Lippman
SIGGRAPH1