The Arabic script allows the replacement of certain character sequences by more compact forms called ligatures. Such ligatures lack a systematic analysis despite their importance in Arabic text recognition research. In this paper, we present analysis of ligatures and compile a comprehensive list of Arabic ligatures. Then, we perform recognition experiments on a benchmark database to show the impact of ligatures annotation in Arabic databases. Finally, we propose several guidelines for the design of representative and compact Arabic databases from the aspect of ligatures.

In this paper, we present a character-level ground-truth for an Arabic dataset of 8,640 character-shapes along with an evaluation metric that is adapted from image segmentation. These tools together constitute a benchmark for character-segmentation. We demonstrate the use of this benchmark on some segmentation algorithms and report their quantitative results in bits. Our approach can be extended to other scripts.

In this paper, we report our comprehensive Arabic offline Handwritten Text database (KHATT) after completion of the collection of 1000 handwritten forms written by 1000 writers from different countries. It is composed of an image database containing images of the written text at 200, 300, and 600 dpi resolutions, a manually verified ground truth database that contains meta-data describing the written text at the page, paragraph, and line levels. A formal verification procedure is implemented to align the handwritten text with its ground truth at the form, paragraph and line levels. Tools to extract paragraphs from pages and segment paragraphs into lines are developed. Preliminary experiments on Arabic handwritten text recognition are conducted using sample data from the database and the results are reported. The database will be made freely available to researchers world-wide for research in various handwritten-related problems such as text recognition, writer identification and verification, etc.

Handwritten Arabic manuscripts have been highly underutilized owing to the fact that, to our knowledge, there is no system or interface in existence, utilizing which user can browse or search the desired information. Given vast content of these manuscripts, it is important to develop indexing systems that support content-based retrieval from historical manuscripts. In this paper, we propose a "Digital library framework for Arabic Manuscripts". A prototype system has been implemented which supports preprocessing of document images. To enhance their quality, feature extraction of the user identified words for similarity matching purposes is carried out. Utilizing relevance feedback from the user, the system indexes the manuscript pages for later efficient retrieval. The prototype system has been tested and encouraging results were achieved.

Prosody has been widely used in many speech-related applications including speaker and word recognition, emotion and accent identification, topic and sentence segmentation, and text-to-speech applications. An important application we investigate is that of identifying question sentences in Arabic monologue lectures. Languages other than Arabic have received a lot of attention in this regard. We approach this problem by first segmenting the sentences from the continuous speech using intensity and duration features. Prosodic features are, then, extracted from each sentence. These features are used as input to decision trees to classify each sentence into either question or non question sentence. Our results suggest that questions are cued by more than one type of prosodic features in natural Arabic speech. We used C4.5 decision trees for classification and achieved 75.7% accuracy. Feature specific analysis further reveals that energy and fundamental frequency features are mainly responsible for discriminating between questions and non-question sentences.

With the rapid proliferation of multimedia applications that require video data management, it is becoming more desirable to provide proper video data indexing techniques capable of representing the rich semantics in video data. In real-time applications, the need for efficient query processing is another reason for the use of such techniques. We present models that use the object motion information in order to characterize the events to allow subsequent retrieval. Algorithms for different spatiotemporal search cases in terms of spatial and temporal translation and scale invariance have been developed using various signal and image processing techniques. We have developed a prototype video search engine, PICTURESQUE (pictorial information and content transformation unified retrieval engine for spatiotemporal queries) to verify the proposed methods. Development of such technology will enable true multimedia search engines that will enable indexing and searching of the digital video data based on its true content.

Multimedia databases have been the subject of extensive research for the last several years. In particular, semantic modeling of video data spurred tremendous interest and produced various formalisms for content-based retrieval of video data. In this paper, we propose the use of Hierarchical Petri-nets (HPN's) for multi-level representation of video data with each level consisting of an augmented Petrinet structure. We show how such representation is able to capture video data semantics, from very low level semantics such as scene change and object movements to higher level semantics involving high-level textual description of events. A Petri-net based method for query processing of multi-object spatio-temporal video-data queries is presented.

In this paper, we present the current state of the art in semantic data modeling of multimedia data. Semantic conceptualization can be performed at several levels of information granularity, leading to multilevel indexing and searching mechanisms. Various models at different levels of granularity are compared. At the finest level of granularity, multimedia data can be indexed based on image contents, such as identification of objects and faces. At a coarser level of granularity, indexing of multimedia data can be focused on events and episodes, which are higher level abstractions. In light of the above, we also examine modeling and indexing techniques of multimedia documents.