Underwater mapping is important for many studies such as underwater cable/pipe platform placement and monitoring, bridge piers placement, dam construction, geological and geophysical studies. The position and orientation information of the sea surface vehicle is measured from the Global Positioning System (GPS) and Inertial Measurement Unit (IMU) sensors are placed on the vehicle, and the height from the seafloor is measured from a single beam sonar. External disturbance effects such as waves and wind cause oscillations in the surface vehicle. In bathymetric measurements, which are of great importance in mapping, errors due to oscillations occur. In underwater mapping, the orientation effect of the sea surface vehicle should be known in order to minimize these errors. In the study, to minimize these error sources, data from 3 different IMUs integrated into the sea surface vehicle are fused with sensor fusion algorithms such as the Integrated Navigation System (INS) and Support Vector Machine (SVM). In this study, a machine learning-based SVM integrated navigation system with minimum error in optimum positioning of the surface vehicle under external disturbance effects is proposed. With the INS, the performance of the machine learning-based SVM sensor fusion algorithm is analyzed comparatively.

This panel paper presents motivations for discussing mobile media search and contains statements from the panelists who are industry research leaders in this field.

Mobile applications are increasingly becoming part of our lives and our work. Moreover, visual capture capability on mobile devices is increasingly being used for linking the real world and the digital world. In this demonstration paper we present an application for accessing and browsing Web sites by pointing a mobile phone at a text book or a printout of a Web site, taking a picture of a small region, performing content recognition and automatically displaying a Web site that is linked to the content.

Recently, many exciting media search applications have been introduced to take advantage of smart phones' audiovisual capture capabilities and their being always on and connected. These applications address a real pain point for most mobile users and allow them to search with minimal text entry, if any. Is the mobile platform an ideal fit for media search? Are audio and visual signal processing technologies sufficiently accurate to support most mobile search applications? What are the killer applications of mobile media search? Earlier in 2009 at ICASSP, a panel on this topic stirred up great interest and enthusiasm while leaving many questions untouched due to the limited time.

The popularity of camera phones enables many exciting multimedia applications. In this paper, we present a novel technology and several applications that allow users to interact with paper documents, books, and magazines. This interaction is in the form of reading and writing electronic information, such as images, web urls, video, and audio, to the paper medium by pointing a camera phone at a patch of text on a document. Our application does not require any special markings, barcodes, or watermarks on the paper document. Instead, we propose a document recognition algorithm that automatically determines the location of a patch of text in a large collection of document images given a small document image. This is very challenging because the majority of phone cameras lack autofocus and macro capabilities and they produce low quality images and video. We developed a novel algorithm, Brick Wall Coding (BWC), that performs image-based document recognition using the mobile phone video frames. Given a document patch image, BWC utilizes the layout, i.e. relative locations, of word boxes in order to determine the original file, page, and the location on the page. BWC runs real-time (4 frames per second) on a Treo 700w smartphone with a 312 MHz processor and 64MB RAM. Using our method we can recognize blurry document patch frames that contain as little as 4-5 lines of text and a video resolution as low as 176x144. We performed experiments by indexing 4397 document pages and querying this database with 533 document patches. Besides describing the basic algorithm, this paper also describes several applications that are enabled by mobile phone-paper interaction, such as inserting electronic annotations to paper, using paper as a tangible interface to collect and communicate multimedia data, and collaborative homework.

HotPaper enables users to electronically interact with paper documents by pointing their camera phones at a document. Electronic interaction is in the form of reading and/or writing images, audio and video clips, urls, notes, etc to the paper. HotPaper does not require any machine readable code, such as a barcode or watermark, to be present on the document. It utilizes our Brick Wall Coding document recognition algorithm [1] that uses a small document patch image to retrieve electronic versions of documents without performing OCR. The HotPaper demonstration runs on a Treo 700w 312 MHz mobile phone and performs document recognition at 4 frames per second using 176x144 video input.

Small displays on mobile handheld devices, such as personal digital assistants (PDAs) and cellular phones, are the bottlenecks for usability of most content browsing applications. Generally, conventional content such as documents and Web pages need to be modified for effective presentation on mobile devices. This paper proposes a novel visualization for documents, called multimedia thumbnails, which consists of text and image content converted into playable multimedia clips. A multimedia thumbnail utilizes visual and audio channels of small portable devices as well as both spatial and time dimensions to communicate text and image information of a single document. The proposed algorithm for generating multimedia thumbnails includes 1) a semantic document analysis step, where salient content from a source document is extracted; 2) an optimization step, where a subset of this extracted content is selected based on time, display, and application constraints; and 3) a composition step, where the selected visual and audible document content is combined into a multimedia thumbnail. Scalability of MMNails that allows generation of multimedia clips of various lengths is also described. A user study is presented that evaluates the effectiveness of the proposed multimedia thumbnail visualization.

An algorithm is presented that automatically generates ground-truthed video from a symbolic description for an object and a specification for the movement of a handheld video camera around that object. This provides a method to generate large amounts of training and test data for the development of computer vision algorithms. We describe an implementation of this technique for an imaging application in which a cell phone video camera is moved over a paper document. Experimental results demonstrate the similarity of images captured by the real camera to images generated by the proposed technique.

In this paper, we describe a method for storing an entire video, animation sequence, or any other media type, on paper. The method is based on printing a key frame from a video on paper along with a barcode that encodes the motion information and other auxiliary information in MPEG-4 format. Unlike other video barcode systems in the prior art, a barcode in our system does not contain a link to the video clip; instead it contains motion information. A client device applies the motion information to an image of the video key frame to obtain full motion video. Therefore, the paper document is a self contained representation of a video clip and access to a server is not required. We modified an MPEG-4 [1] encoder and decoder to implement the video flip-book encoder and decoder. Experiments show that it is possible to encode several seconds of video on paper using our method. This is sufficient to create small animations for some printed materials such as video greeting cards and children's books.

It is difficult to view multipage, high resolution documents on devices with small displays. As a solution, we introduce a multimedia thumbnail representation, which can be seen as a multimedia clip that provides an automated guided tour through a document. Multimedia thumbnails are automatically generated by taking a document image as input and first performing visual and audible information analysis on the document to determine salient document elements. Next, the time and information attributes for each document element are computed by taking into account the display and application constraints. An optimization routine, given a time constraint, selects elements to be included in the multimedia thumbnail. Last, the selected elements are synthesized into animated images and audio to create the final multimedia representation

As small portable devices are becoming standard personal equipments, there is a great need for the adaptation of information content to small displays. Currently, no good solutions exist for viewing formatted documents, such as pdf documents, on these devices. Adapting content of web pages to small displays is usually achieved by complete redesign of a page or automatically reflowing text for small displays. Such techniques may not be applicable to documents whose format needs to be preserved. To address this problem, we propose a new document representation called Multimedia Thumbnail. Multimedia Thumbnail uses the visual and audio channels of small portable devices to communicate document information in form of a multimedia clip, which can be seen as a movie trailer for a document. Generation of such a clip includes a document analysis step, where salient document information is extracted, an optimization step, where the document information to be included in the thumbnail is determined based on display and time constraints, and a synthesis step, where visual and audible information are formed into a playable Multimedia Thumbnail. We also present user study results that evaluate an initial system design and point to further modification on analysis, optimization, and user interface components.

Browsing multi-page high-resolution documents on devices with small displays and limited navigational capabilities is very difficult. In this paper we demonstrate Multimedia Thumbnails, which are automatically computed guided-tours of documents. Multimedia Thumbnails utilize temporal dimension and the audio channel of the portable device in order to compensate for the limited resolution of the display area.

Over the past few years, with the rapid adoption of broadband communication and advances in multimedia content capture and delivery, Web-based meetings and lectures, also referred to as e-meeting and e-lecture, have become popular among businesses and academic institutions because of their cost savings and capabilities in providing self-paced education and convenient content access and retrieval. In fact, the technological achievements in capture, analysis, access, and delivery of e-meeting and e-lecture media have already resulted in several working systems that are currently of regular usage. This paper gives an overview of existing work as well as state-of-the-art in these two research areas which are bound to affect the way we teach, learn, and collaborate.

A novel system is described that significantly enhances the usefulness of handwritten notes taken during a presentation by creating a multimedia document that includes scanned images of handouts, personal notes, and links to a multimedia recording of the presentation. Notes are linked to the e-presentation media with automatic content analysis without any special notes capture device. Layout segmentation and template matching automatically detects the presence of presentation handouts during scanning. Presentation-level and slide-level linking of handouts to e-media use text and image features from slides. Experimental results show 95% accuracy in linking of the scanned handouts to the e-presentation media.

Overcoming the semantic-feature gap and adapting to context are two main challenges in content-based retrieval. The problem is even more complicated for unstructured videos such as automated recordings of meetings. To address this problem, we propose a model-based approach to meeting retrieval with user controlled weighting for dynamic similarity comparison. Each video is represented by an HMM, and the similarity between videos is determined by comparing the corresponding models. Users can control the relative importance of temporal and static features by adjusting a weighting parameter in a way similar to content-based image retrieval. Experimental results demonstrate the feasibility and versatility of this approach.

The Office Blogger (OBlog) is an experimental prototype of a multimedia appliance that allows an office worker to easily record events, conversations, meetings, pictures and documents, and helps create blog entries from that data. OBlog employs a novel image classification algorithm and automatic post processing to make the information about images more accessible to users. The OBlog system uses an älways-on" design that makes it convenient to use and it's packaged as a stand-alone system to remove any computational burden from a user's PC. Results show that the image classification algorithm achieves over 90% recall on five image categories commonly encountered by office workers.

The increasing availability of object-based video content requires new technologies for automatically extracting and matching of the low level features of arbitrarily shaped video. This paper proposes methods for shape retrieval of arbitrarily shaped video objects. Our methods take into account not only the still shape features but also the shape deformations that may occur in an object's lifespan. We compute the shape similarity of video objects by comparing the similarity of their representative temporal instances. We also describe motion of a video object via describing the deformations in an object's shape. Experimental results show that our proposed methods offer very good retrieval performance and match closely with the human ranking.

A multimedia note taking system called MinuteAid is described that lets users of laptops include multimedia objects in notes they write during a presentation. The multimedia data is provided by devices that capture slide images, whiteboard images, audio and video clips. The MinuteAid system includes a software interface between these devices and a commonly available word processing application (Microsoft Word) and lets the user either insert copies of multimedia objects while they're captured or references to multimedia objects that are automatically retrieved and inserted in the document later. We describe the architecture, implementation and user experience with this system.

The components of a key frame selection algorithm for a paper-based multimedia browsing interface called Video Paper are described. Analysis of video image frames is combined with the results of processing the closed caption to select key frames that are printed on a paper document together with the closed caption. Bar codes positioned near the key frames allow a user to play the video from the corresponding times. This paper describes several component techniques that are being investigated for key frame selection in the Video Paper system, including face detection and text recognition. The Video Paper system implementation is also discussed.

Background subtraction is an essential processing component for many video applications. However, its development has largely been application driven and done in an ad hoc manner. In this paper, we provide a Bayesian formulation of background segmentation based on Gaussian mixture models. We show that the problem consists of two density estimation problems, one application independent and one application dependent, and a set of intuitive and theoretically optimal solutions can be derived for both. The proposed framework was tested on meeting and traffic videos and compared favorably to other well-known algorithms.

Recorded meetings are useful only if people can find, access, and browse them easily. Key-frames and video skims are useful representations that can enable quick previewing of the content without actually watching a meeting recording from beginning to end. This paper proposes a new method for creating meeting video skims based on audio and visual activity analysis together with text analysis. Audio activity analysis is performed by analyzing sound directions-indicating different speakers-and audio amplitude. Detection of important visual events in a meeting is achieved by analyzing the localized luminance variations in consideration with the omni-directional property of the video captured by our meeting recording system. Text analysis is based on the term frequency-inverse document frequency measure. The resulting video skims better capture the important meeting content compared to the skims obtained by uniform sampling.

An algorithm is presented that automatically matches images of presentation slides to the symbolic source file (e.g., PowerPoint™ or Acrobat™) from which they were generated. The images are captured either by tapping the video output from a laptop connected to a projector or by taking a picture of what's displayed on the screen in a conference room. The matching algorithm extracts features from the image data, including OCR output, edges, projection profiles, and layout and determines the symbolic file that contains the most similar collection of features. This algorithm enables several unique applications for enhancing a meeting in real-time and accessing the audio and video that were recorded while a presentation was being given. These applications include the simultaneous translation of presentation slides during a meeting, linking video clips inside a PowerPoint file that show how each slide was described by the presenter, and retrieving presentation recordings using digital camera images as queries.

Video Paper is a prototype system for multimedia browsing, analysis, and replay. Key frames extracted from a video recording are printed on paper together with bar codes that allow for random access and replay. A transcript for the audio track can also be shown so that users can read what was said, thus making the document a stand-alone representation for the contents of the multimedia recording. The Video Paper system has been used for several applications, including the analysis of recorded meetings, broadcast news, oral histories and personal recordings. This demonstration will show how the Video Paper system was applied to these domains and the various replay systems that were developed, including a self-contained portable implementation on a PDA and a fixed implementation on desktop PC.

The design and implementation of a portable meeting recorder is presented. Composed of an omni-directional video camera with four-channel audio capture, the system saves a view of all the activity in a meeting and the directions from which people spoke. Subsequent analysis computes metadata that includes video activity analysis of the compressed data stream and audio processing that helps locate events that occurred during the meeting. Automatic calculation of the room in which the meeting occurred allows for efficient navigation of a collection of recorded meetings. A user interface is populated from the metadata description to allow for simple browsing and location of significant events.

The increasing availability of object-based video content requires new technologies for automatically extracting and matching of the low level features of arbitrarily shaped video. in this paper, we propose methods for the efficient retrieval of video object shapes. Our methods take into account not only the still shape features but also the shape deformations that may occur in the lifespan of video objects. We define a new shape similarity measure that is based on the shape similarity of the representative temporal instances of video objects. We also propose shape deformation features that are based on the variances of the still shape features. The proposed visual features can be derived directly from the MPEG-4 compressed domain or computed from the shape masks of the video objects in the spatial domain. Our experiments show that our proposed methods offer very good retrieval results.

Motion features play an important role in video retrieval. We propose three new motion features that describe the local motion of arbitrarily shaped video objects. The proposed descriptors are based on the angular and circular area variances of the video object, the variances of the angular radial transform coefficients of the object's area, and the variances of the Fourier coefficients of the object's boundary. These descriptors can be derived directly from the MPEG-4 compressed domain or can be computed using the binary shape masks of the video objects in the spatial domain. Our experiments show that the ranking obtained by querying with our proposed features closely matches the human ranking.

Summarization of video content is necessary in order to reduce the large amount of data involved in video retrieval. In a frame-based digital video retrieval framework, this is achieved by representing the content of a video sequence using key frames. Similarly, in an object-based framework, such as the one suggested by the MPEG-4 standard, video object planes can be used for summarization of video object content. We propose a method for key video object plane selection using the shape information in the MPEG-4 compressed domain. Two popular shape distance measures, the Hamming and Hausdorff distance measures, are employed to measure the similarities between the approximated shapes of the video objects. The corresponding algorithms, with different implementation complexity and computation tradeoffs, select key video object planes that represent efficiently the salient content of the video objects.

We propose an error concealment method for shape information in the MPEG-4 coded video sequences. A maximum a posteriori estimator, which employs an adaptive Markov random field, is used to restore the missing shape information. The proposed concealment method successfully reconstructs the missing shape data, with good computation-performance tradeoffs.

We propose an algorithm to partition MPEG-4 video objects into temporal segments with uniform activity levels. These segments can be used for efficient summarization of video objects and/or retrieval of video objects by their motion. The proposed algorithm is based on the shape and texture coding modes of inter coded video object planes. Because the employed filtering techniques are simple and full decoding of the MPEG-4 bitstream is not required, the algorithm is computationally efficient.

This paper presents efficient coding and mapping algorithms that lead to a significant speed improvement in low bit rate H.263/H263+ video encoding while maintaining high video-reproduction quality. First, by exploiting the statistical properties of low resolution and slowly varying video sequences, we reduce significantly the computation times of the most computationally intensive components of video coding, particularly the discrete cosine transform, the inverse discrete cosine transform, quantization, and motion estimation. We also map some of the single instruction multiple data (SIMD)-oriented functions onto Intel's MMX architecture. The developed algorithms are implemented using our public-domain H.263/H.263+ encoder/decoder software. Using the above algorithms, our H.263/H.263+ baseline video-encoder implementation can encode more than 15 fps in QCIF resolution on a Pentium MMX 200-MHz computer.

Object-based video representation, such as the one suggested by the MPEG-4 standard, offers a framework that is better suited for object-based video indexing and retrieval. In such a framework, the concept of a "key frame" is replaced by that of a "key video object plane". In this paper, we propose a method for key video object plane selection using the shape information in the MPEG-4 compressed domain. The shape of the video object (VO) is approximated using the shape coding modes of I, P, and B video object planes (VOPs) without decoding the shape information in the MPEG-4 bit stream. Two popular shape distance measures, the Hamming and Hausdorff distance measures, are modified to measure the similarities between the approximated shapes of the video objects. Although they feature different computational and implementation complexity tradeoffs, the corresponding algorithms achieve essentially the same performance levels in selecting key video object planes that represent efficiently the salient content of the video objects.

We propose a new method for error concealment of shape information in MPEG-4 video bit streams that are transmitted over error prone channels. The proposed method employs a MAP estimator with a Markov random field (MRF) as the image a priori model. The MRF is designed for binary shape information and its parameters are adapted based on the information of neighboring blocks. Our experimental results show that the proposed concealment method restores missing shape blocks with high accuracy. Compared to the median filtering method, our method restores 20% more missing shape data, with a much greater subjective improvement. The proposed algorithm requires a relatively small number of integer multiplications and additions and simple logic operations, making it suitable for real-time implementations.

The emerging ITU-T H.263+ low bit-rate video coding standard is version 2 of the draft international standard ITU-T H.263. In this paper, we discuss this emerging video coding standard and present compression performance results based on our public domain implementation of H.263+.

We discuss the ITU-T H.263+ (or H.263 Version 2) low-bit-rate video coding standard. We first describe, briefly, the H.263 standard including its optional modes. We then address the 12 new negotiable modes of H.263+. Next, we present experimental results for these modes, based on our public-domain implementation (http:ilspmg.ece.ubc.ca). Tradeoffs among compression performance, complexity, and memory requirements for the H.263+ optional modes are discussed. Finally, results for mode combinations are presented.