BACKGROUND: Honey bees are the principal commercial pollinators. Along with other arthropods, they are increasingly under threat from anthropogenic factors such as the incursion of invasive honey bee subspecies, pathogens and parasites. Better tools are needed to identify bee subspecies. Genomic data for economic and ecologically important organisms is increasing, but in its basic form its practical application to address ecological problems is limited. RESULTS: We introduce HBeeID a means to identify honey bees. The tool utilizes a knowledge-based network and diagnostic SNPs identified by discriminant analysis of principle components and hierarchical agglomerative clustering. Tests of HBeeID showed that it identifies African, Americas-Africanized, Asian, and European honey bees with a high degree of certainty even when samples lack the full 272 SNPs of HBeeID. Its prediction capacity decreases with highly admixed samples. CONCLUSION: HBeeID is a high-resolution genomic, SNP based tool, that can be used to identify honey bees and screen species that are invasive. Its flexible design allows for future improvements via sample data additions from other localities.

Human mobility nowcasting is a fundamental research problem for intelligent transportation planning, disaster responses and management, etc. In particular, human mobility under big disasters such as hurricanes and pandemics deviates from its daily routine to a large extent, which makes the task more challenging. Existing works mainly focus on traffic or crowd flow prediction in normal situations. To tackle this problem, in this study, disaster-related Twitter data is incorporated as a covariate to understand the public awareness and attention about the disaster events and thus perceive their impacts on the human mobility. Accordingly, we propose a Meta-knowledge-Memorizable Spatio-Temporal Network (MemeSTN), which leverages memory network and meta-learning to fuse social media and human mobility data. Extensive experiments over three real-world disasters including Japan 2019 typhoon season, Japan 2020 COVID-19 pandemic, and US 2019 hurricane season were conducted to illustrate the effectiveness of our proposed solution. Compared to the state-of-the-art spatio-temporal deep models and multivariate-time-series deep models, our model can achieve superior performance for nowcasting human mobility in disaster situations at both country level and state level.

Abstract A six‐month‐long Atlantic hurricane season impacts Florida residents every year and can result in devastating consequences, including loss of life, property damage, and business interruptions. Hurricane risk assessment and loss prediction are critical to various uses such as determining homeowner insurance premiums, regulating these premiums, conducting scenario analysis, conducting stress tests for companies, disaster management, and evaluating the benefits of disaster mitigation techniques. This article describes the Florida Public Hurricane Loss Model (FPHLM): a large‐scale catastrophe model with massive databases and analytics tools for business and government decision‐making. We will discuss the design and implementation of each component in FPHLM and explain the tools and techniques utilized to tackle challenges in data availability, data analytics, and the interface between the data, analytical techniques, and computing. Results are shown to validate the software system's effectiveness and reliability and illustrate some of the system's use cases.

Pictures or videos captured from a low-altitude aircraft or an unmanned aerial vehicle are a fast and cost-effective way to survey the affected scene for the quick and precise assessment of a catastrophic event’s impacts and damages. Using advanced techniques, such as deep learning, it is now possible to automate the description of disaster scenes and identify features in captured images or recorded videos to gain situational awareness. However, building a large-scale, high-quality dataset with annotated disaster-related features for supervised model training is time-consuming and costly. In this article, we propose a weakly supervised approach to train a deep neural network on low-altitude imagery with highly imbalanced and noisy crowd-sourced labels. We further make use of the rich spatiotemporal data obtained from the pictures and its sequence information to enhance the model’s performance during training via label propagation. Our approach achieves the highest score among all the submitted runs in the TRECVID2020 Disaster Scene Description and Indexing (DSDI) Challenge, indicating its superior capabilities in retrieving disaster-related video clips compared to other proposed methods.

Nowadays, the world faces extreme climate changes, resulting in an increase of natural disaster events and their severities. In these conditions, the necessity of disaster information management systems has become more imperative. Specifically, in this paper, the problem of flood event detection from images with real-world conditions is addressed. That is, the images may be taken in several conditions, including day, night, blurry, clear, foggy, rainy, different lighting conditions, etc. All these abnormal scenarios significantly reduce the performance of the learning algorithms. In addition, many existing image classification methods use datasets that usually include high-resolution images without considering real-world noise. In this paper, we propose a new image classification framework based on adversarial data augmentation and deep learning algorithms to address the aforementioned problems. We validate the performance of the flood event detection framework on a real-world noisy visual dataset collected from social networks.

The four papers in this special section examine the use of multimedia Big Data for leveraging big events such as natural disasters. Leveraging technological advances and using Big Data approaches for information management and processing in extreme events have attracted much research attention. Multimedia is increasingly becoming the most important and valuable source for insight and information on extreme events. It covers from everyone’s experiences to everything happening in the world. There will be lots of multimedia big data in extreme events—surveillance video, disaster images, social media, voice and video, to name a few, only if their volumes grow to the extent that the traditional multimedia processing and analysis systems may not handle effectively. As such, multimedia big data in extreme events are spurring on tremendous amounts of research and development of related technologies and applications. The papers in this issue is intended to provide a premier forum for researchers working on multimedia big data for extreme events to present their recent research results. It also provides an important opportunity for multidisciplinary studies connecting multimedia computing to disaster information management.

Multimedia concept detection is a challenging topic due to the well-known class imbalance issue, where the data instances are distributed unevenly across different classes. This problem becomes even more prominent when the minority class that contains an extremely small proportion of the data represents the concept of interest as has occurred in many real-world applications such as frauds in banking transactions and goal events in soccer videos. Traditional data mining approaches often have difficulty handling largely skewed data distributions. To address this issue, in this paper, an importance-factor (IF)-based multiple correspondence analysis (MCA) framework is proposed to deal with the imbalanced datasets. Specifically, a hierarchical information gain analysis method, which is inspired by the decision tree algorithm, is presented for critical feature selection and IF assignment. Then, the derived IF is incorporated with the MCA algorithm for effective concept detection and retrieval. The comparison results in video concept detection using the disaster dataset and the soccer dataset demonstrate the effectiveness of the proposed framework.

Deep learning has led to many breakthroughs in machine perception and data mining. Although there are many substantial advances of deep learning in the applications of image recognition and natural language processing, very few work has been done in video analysis and semantic event detection. Very deep inception and residual networks have yielded promising results in the 2014 and 2015 ILSVRC challenges, respectively. Now the question is whether these architectures are applicable to and computationally reasonable in a variety of multimedia datasets. To answer this question, an efficient and lightweight deep convolutional network is proposed in this paper. This network is carefully designed to decrease the depth and width of the state-of-the-art networks while maintaining the high-performance. The proposed deep network includes the traditional convolutional architecture in conjunction with residual connections and very light inception modules. Experimental results demonstrate that the proposed network not only accelerates the training procedure, but also improves the performance in different multimedia classification tasks.

In disaster management, people are interested in the development and the evolution of the disasters. If they intend to track the information of the disaster, they will be overwhelmed by the large number of disaster-related documents, microblogs, and news, etc. To support disaster management and minimize the loss during the disaster, it is necessary to efficiently and effectively collect, deliver, summarize, and analyze the disaster information, letting people in affected area quickly gain an overview of the disaster situation and improve their situational awareness.

Numerous deep learning architectures have been designed for a variety of tasks in the past few years. However, it is almost impossible for one model to work well for all kinds of scenarios and datasets. Therefore, we present an ensemble deep learning framework in this paper, which not only decreases the information loss and over-fitting problems caused by single models, but also overcomes the imbalanced data issue in multimedia big data. First, a suite of deep learning algorithms are utilized for deep feature selection. Thereafter, an enhanced ensemble algorithm is developed based on the performance of each single Support Vector Machine classifier on each deep feature set. We evaluate our proposed ensemble deep learning framework on a large and highly imbalanced video dataset containing natural disaster events. Experimental results demonstrate the effectiveness of the proposed framework for semantic event detection, and show how it outperforms several state-of-the-art deep learning architectures, as well as handcrafted features integrated with ensemble and non-ensemble algorithms.

Classification of imbalanced data is an important research problem as lots of real-world data sets have skewed class distributions in which the majority of data instances (examples) belong to one class and far fewer instances belong to others. While in many applications, the minority instances actually represent the concept of interest (e.g., fraud in banking operations, abnormal cell in medical data, etc.), a classifier induced from an imbalanced data set is more likely to be biased towards the majority class and show very poor classification accuracy on the minority class. Despite extensive research efforts, imbalanced data classification remains one of the most challenging problems in data mining and machine learning, especially for multimedia data. To tackle this challenge, in this paper, we propose an extended deep learning approach to achieve promising performance in classifying skewed multimedia data sets. Specifically, we investigate the integration of bootstrapping methods and a state-of-the-art deep learning approach, Convolutional Neural Networks (CNNs), with extensive empirical studies. Considering the fact that deep learning approaches such as CNNs are usually computationally expensive, we propose to feed low-level features to CNNs and prove its feasibility in achieving promising performance while saving a lot of training time. The experimental results show the effectiveness of our framework in classifying severely imbalanced data in the TRECVID data set.

The snake algorithm has been proposed to solve many remote sensing and computer vision problems such as object segmentation, surface reconstruction, and object tracking. This paper introduces a framework for 3-D building model construction from LIDAR data based on the snake algorithm. It consists of nonterrain object identification, building and tree separation, building topology extraction, and adjustment by the snake algorithm. The challenging task in applying the snake algorithm to building topology adjustment is to find the global minima of energy functions derived for 2-D building topology. The traditional snake algorithm uses dynamic programming for computing the global minima of energy functions which is limited to snake problems with 1-D topology (i.e., a contour) and cannot handle problems with 2-D topology. In this paper, we have extended the dynamic programming method to address the snake problems with a 2-D planar topology using a novel graph reduction technique. Given a planar snake, a set of reduction operations is defined and used to simplify the graph of the planar snake into a set of isolated vertices while retaining the minimal energy of the graph. Another challenging task for 3-D building model reconstruction is how to enforce different kinds of geometric constraints during building topology refinement. This framework proposed two energy functions, deviation and direction energy functions, to enforce multiple geometric constraints on 2-D topology refinement naturally and efficiently. To examine the effectiveness of the framework, the framework has been applied on different data sets to construct 3-D building models from airborne LIDAR data. The results demonstrate that the proposed snake algorithm successfully found the global optima in polynomial time for all of the building topologies and generated satisfactory 3-D models for most of the buildings in the study areas.

The goal of this special issue is to provide a premier forum for researchers to present their recent research results on multimedia big data. It follows the recent success event—the First IEEE International Conference on Multimedia Big Data (BigMM 2015) that took place at the Chinese National Convention Center in Beijing, China, from April 20–22, 2015 It also provides an important opportunity for multidisciplinary work connecting big data to multimedia computing.

Smart TVs have realized the convergence of TV, Internet , and PC technologies, but still do not provide a seamless content interaction for TV-enabled shopping. To purchase interesting items displayed in a TV show, consumers must resort to a store or the Web, which is an inconvenient way of purchasing products. The fundamental challenge in realizing such a use case consists of understanding the multimedia content being streamed. Such a challenge can be realized by utilizing object detection to facilitate content understanding though it has to be executed as a computationally bound process so that consumers are provided with a responsive and exciting user interface. To this end, we propose a computational- and temporal-aware multimedia abstraction framework that facilitates the efficient execution of object detection tasks. Given computational and temporal rate constraints, the proposed framework selects the optimal video frames that best represent the video content and allows the execution of the object detection task as a computationally bound process. In this sense, the framework is computationally scalable as it can adapt to the given constraints and generate optimal abstraction results accordingly. Additionally, the framework utilizes “object views” as the basis for the frame selection process, which depict salient information and are represented as regions of interest (ROI). In general, an ROI can be a whole frame or a region that discards background information. Experimental results demonstrate the computational scalability of the proposed framework and the benefits of using the regions of interest as the basis of the abstraction process.

The Florida Public Hurricane Loss Model (FPHLM) is a large scale, multidisciplinary project developed to assist the state of Florida with the regulation of residential insurance premiums as they relate to insured losses caused by hurricane winds. The modeling services provided to clients using the F

Sequential data is prevalent in many scientific and commercial applications such as bioinformatics, system security and networking. Similarity search has been widely studied for symbolic and time series data in which each data object is a symbol or numeric value. Textual event sequences are sequences of events, where each object is a message describing an event. For example, system logs are typical textual event sequences and each event is a textual message recording internal system operations, statuses, configuration modifications or execution errors. Similar segments of an event sequence reveals similar system behaviors in the past which are helpful for system administrators to diagnose system problems. Existing search indexing for textual data only focus on unordered data. Substring matching methods are able to efficiently find matched segments over a sequence, however, their sequences are single values rather than texts. In this paper, we propose a method, suffix matrix, for efficiently searching similar segments over textual event sequences. It provides an integration of two disparate techniques: locality-sensitive hashing and suffix arrays. This method also supports the k-dissimilar segment search. A k-dissimilar segment is a segment that has at most k dissimilar events to the query sequence. By using random sequence mask proposed in this paper, this method can have a high probability to reach all k-dissimilar segments without increasing much search cost. We conduct experiments on real system log data and the experimental results show that our proposed method outperforms alternative methods using existing techniques.

The booming multimedia technology is incurring a thriving multi-media data propagation. As multimedia data have become more essential, taking over a major potion of the content processed by many applications, it is important to leverage data mining methods to associate the low-level features extract

In this paper, we propose a Correlation based Feature Analysis (CFA) and Multi-Modality Fusion (CFA-MMF) framework for multimedia semantic concept retrieval. The CFA method is able to reduce the feature space and capture the correlation between features, separating the feature set into different feature groups, called Hidden Coherent Feature Groups (HCFGs), based on Maximum Spanning Tree (MaxST) algorithm. A correlation matrix is built upon feature pair correlations, and then a MaxST is constructed based on the correlation matrix. By performing a graph cut procedure on the MaxST, a set of feature groups are obtained, where the intra-group correlation is maximized and the inter-group correlation is minimized. Finally, one classifier is trained for each of the feature groups, and the generated scores from different classifiers are fused for the final retrieval. The proposed framework is effective because it reduces the dimensionality of the feature space. The experimental results on the NUSWIDE-Lite data set demonstrate the effectiveness of the proposed CFA-MMF framework.

This demo paper presents a system based on PostgreSQL and the AH-Tree that supports Content-Based Image Retrieval (CBIR) through similarity queries. The AH-Tree is a balanced, tree-based index structure that utilizes high-level semantic information to address the well-known problems of semantic gap and user perception subjectivity. The proposed system implements the AH-Tree inside PostgreSQL's kernel by internally modifying PostgreSQL's GiST access mechanism and thus provides a DBMS with a viable and efficient content-based multimedia retrieval functionality.

In this paper we propose a novel framework for object retrieval based on automatic foreground object extraction and multi-layer information integration. Specifically, user interested objects are firstly detected from unconstrained videos via a multimodal cues method, then an automatic object extraction algorithm based on Grab Cut is applied to separate foreground object from background. The object-level information is enhanced during the feature extraction layer by assigning different weights to foreground and background pixels respectively, and the spatial color and texture information is integrated during the similarity calculation layer. Experimental results on both benchmark data set and real-world data set demonstrate the effectiveness of the proposed framework.

With the rapid development of next generation sequencing technology, the amount of biological sequence data of the cancer genome increases exponentially, which calls for efficient and effective algorithms that may identify patterns hidden underneath the raw data that may distinguish cancer Achilles' heels. From a signal processing point of view, biological units of information, including DNA and protein sequences, have been viewed as one-dimensional signals. Therefore, researchers have been applying signal processing techniques to mine the potentially significant patterns within these sequences. More specifically, in recent years, wavelet transforms have become an important mathematical analysis tool, with a wide and ever increasing range of applications. The versatility of wavelet analytic techniques has forged new interdisciplinary bounds by offering common solutions to apparently diverse problems and providing a new unifying perspective on problems of cancer genome research. In this paper, we provide a survey of how wavelet analysis has been applied to cancer bioinformatics questions. Specifically, we discuss several approaches of representing the biological sequence data numerically and methods of using wavelet analysis on the numerical sequences.

Techniques to efficiently discover, collect, organize, search, and disseminate real-time disaster information have become national priorities for efficient crisis management and disaster recovery tasks. We have developed techniques to facilitate information sharing and collaboration between both private and public sector participants for major disaster recovery planning and management. We have designed and implemented two parallel systems: a web-based prototype of a Business Continuity Information Network system and an All-Hazard Disaster Situation Browser system that run on mobile devices. Data mining and information retrieval techniques help impacted communities better understand the current disaster situation and how the community is recovering. Specifically, information extraction integrates the input data from different sources; report summarization techniques generate brief reviews from a large collection of reports at different granularities; probabilistic models support dynamically generating query forms and information dashboard based on user feedback; and community generation and user recommendation techniques are adapted to help users identify potential contacts for report sharing and community organization. User studies with more than 200 participants from EOC personnel and companies demonstrate that our systems are very useful to gain insights about the disaster situation and for making decisions.

In this work, we have designed and developed a Multimedia-Aided Disaster Information Integration System (MADIS), which utilizes advanced data mining techniques to analyze situation reports and pictures as well as text captured in the field and automatically link the reports directly to relevant mult

In this paper, a disaster image filtering and summarization (DIFS) framework is proposed based on multi-layered affinity propagation. The proposed framework is able to automatically identify and summarize latent semantic themes (scenes) in a disaster topic and filter junk images at the same time. Specifically, the images belonging to a disaster topic are first clustered into different groups based on visual descriptors using affinity propagation (AP). Then the typical instances within each cluster are collected to perform the second-layer clustering for identifying final positive clusters by utilizing both visual and textual similarities concurrently. At both layers, the proposed curve fitting function is applied to select appropriate preference values for the AP algorithm. The experimental results on the real world Flickr data set demonstrate the effectiveness of the proposed framework.

Data management support to enable effective and efficient information sharing in collaborative environments is critical, especially in semantics based search and retrieval. In this paper, a novel spectrum perturbation-based subspace classification is proposed to mine semantics and other useful i

This paper presents the AH+-tree, a balanced, tree-based index structure that efficiently supports Content-Based Image Retrieval (CBIR) through similarity queries. The proposed index structure addresses the problems of semantic gap and user subjectivity by considering the high-level semantics of multimedia data during the retrieval process. The AH+-tree provides the same functionality as the Affinity-Hybrid Tree (AH-Tree) but utilizes the high-level semantics in a novel way to eliminate the I/O overhead incurred by the AH-Tree due to the process of affinity propagation, which requires a complete traversal of the tree. The novel structure of the tree is explained, and detailed range and nearest neighbor algorithms are implemented and analyzed. Extensive discussions and experiments demonstrate the superior efficiency of the AH+-tree over the AH-Tree and the M-tree. Results show the AH+-tree significantly reduces I/O cost during similarity searches. The I/O efficiency of the AH+-tree and its ability to incorporate high-level semantics from different machine learning mechanisms make the AH+-tree a promising index access method for large multimedia databases.

It is a great challenge to detect an object that is overlapped or occluded by other objects in images. For moving objects in a video sequence, their movements can bring extra spatio-temporal information of successive frames, which helps object detection, especially for occluded objects. This paper proposes a moving object detection approach for occluded objects in a video sequence with the assist of the SPCPE (Simultaneous Partition and Class Parameter Estimation) unsupervised video segmentation method. Based on the preliminary foreground estimation result from SPCPE and object detection information from the previous frame, an n-steps search (NSS) method is utilized to identify the location of the moving objects, followed by a size-adjustment method that adjusts the bounding boxes of the objects. Several experimental results show that our proposed approach achieves good detection performance under object occlusion situations in serial frames of a video sequence.

We present a novel visual analytics system and multimedia enabled mobile application that allows emergency management (EM) personnel access to timely and relevant disaster situation information. The system is able to semantically integrate text-based emergency management disaster situation reports with related disaster imagery taken in the field by EM responders and community residents. In addition, through an intuitive and seamless Apple iPad application, users are able to interact with the system in diverse places and conditions and thus provide a more effective response. The system is demonstrated via its iPad application which aims at providing relevant and actionable information.

The improvement of Crisis Management and Disaster Recovery techniques are national priorities in the wake of man-made and nature inflicted calamities of the last decade. Our prior work has demonstrated that the efficiency of sharing and managing information plays an important role in business recovery efforts after disaster event. With the proliferation of smart phones and wireless tablets, professionals who have an operational responsibility in disaster situations are relying on such devices to maintain communication. Further, with the rise of social media, technology savvy consumers are also using these devices extensively for situational updates. In this paper, we address several critical tasks which can facilitate information sharing and collaboration between both private and public sector participants for major disaster recovery planning and management. We design and implement an All-Hazard Disaster Situation Browser (ADSB) system that runs on Apple's mobile operating system (iOS) and iPhone and iPad mobile devices. Our proposed techniques create a collaborative solution on a mobile platform using advanced data mining and information retrieval techniques for disaster preparedness and recovery that helps impacted communities better understand the current disaster situation and how the community is recovering. Specifically, hierarchical summarization techniques are used to generate brief reviews from a large collection of reports at different granularities; probabilistic models are proposed to dynamically generate query forms based on user's feedback; and recommendation techniques are adapted to help users identify potential contacts for report sharing and community organization. Furthermore, the developed techniques are designed to be all-hazard capable so that they can be used in earthquake, terrorism, or other unanticipated disaster situations.

The fast development of microscopy imaging techniques nowadays promotes the generation of a large amount of data. These data are very crucial not only for theoretical biomedical research but also for clinical usage. In order to decrease the inter-intra observer variability and save the human effort on labeling and classifying these images, a lot of research efforts have been devoted to the development of algorithms for biomedical images. Among such efforts, histology image classification is one of the most important areas due to its broad applications in pathological diagnosis such as cancer diagnosis. To improve classification accuracy, most of the previous work focuses on extracting more features and building algorithms for a specific task. This paper proposes a framework based on the novel and robust Collateral Representative Subspace Projection Modeling (C-RSPM) supervised classification model for general histology image classification. In the proposed framework, a cell image is first divided into 25 blocks to reduce the spatial complexity of computation, and one C-RSPM model is built on each block set which contains blocks in the same location from different images. For each testing image, our proposed framework first classifies each of its blocks using the C-RSPM classification model built for that block set, and then applies a multimodal late fusion algorithm with a weighted majority voting strategy to decide the final class label of the whole image. Experimenting using three-fold cross validation with three benchmark histology data sets shows that the proposed framework outperforms other well-known classifiers in the comparison and gives better results than the highest accuracy reported previously.

Crisis Management and Disaster Recovery have gained immense importance in the wake of recent man and nature inflicted calamities. A critical problem in a crisis situation is how to efficiently discover, collect, organize, search and disseminate real-time disaster information. In this paper, we address several key problems which inhibit better information sharing and collaboration between both private and public sector participants for disaster management and recovery. We design and implement a web based prototype implementation of a Business Continuity Information Network (BCIN) system utilizing the latest advances in data mining technologies to create a user-friendly, Internet-based, information-rich service and acting as a vital part of a company's business continuity process. Specifically, information extraction is used to integrate the input data from different sources; the content recommendation engine and the report summarization module provide users personalized and brief views of the disaster information; the community generation module develops spatial clustering techniques to help users build dynamic community in disasters. Currently, BCIN has been exercised at Miami-Dade County Emergency Management.

Associative classification (AC) has been studied in the areas of content-based multimedia retrieval and semantic concept detection due to its high accuracy. The traditional AC algorithm discovers the association rules with the frequency count (minimum support) and ranking threshold (minimum confidence) while restricted to the concepts (class labels). In this paper, we propose a novel framework with a new associative classification algorithm which generates the classification rules based on the correlation between different feature-value pairs and the concept classes by using multiple correspondence analysis (MCA). Experimenting with the high-level features and benchmark data sets from TRECVID, our proposed algorithm achieves promising performance and outperforms three well-known classifiers which are commonly used for performance comparison in the TRECVID community.

This paper presents a 3D traffic animation system that provides features of vehicle interaction and storm surge response to emulate realistic scenarios in a hurricane affected area. The proposed system constructs road objects based on a series of line segments. Vehicles on the roads are animated by keeping them coherent with the direction of the road segment and adjusting their speeds with respect to one another. The vehicle speed is controlled based upon a collision-circumventing policy and they automatically respond to flood water caused by a storm surge. The system can automatically plant lamp posts along the roads, which also respond to surge flooding. An implementation conducted with the 3D scenes on a couple of south Florida's surge-susceptible areas demonstrates the system's excellence in animating real traffic scenes and impacts of storm surge on coastal regions.

With the rapid increase in the amount of multimedia data, the researches on semantic information retrieval are facing a very challenging problem - the number of positive data instances with the target concept/object/event compared with the number of negative data instances without the target concept/object/event is much smaller, which is also called the data imbalance issue. Therefore, one of the popular topics in multimedia information processing and retrieval is data pruning, a technique that can automatically identify and prune the data instances from the training data set so that the pruned data set is able to enhance the performance of model learning, classification, and concept detection. In this paper, a novel data pruning framework which gives each transaction a weight based on multiple correspondence analysis (MCA) is proposed. These transaction weights are used as the measure for pruning the training data set. Meanwhile, the testing data set could be weighted and pruned as well so that the computational cost is reduced not only when building the model but also when applying the classifiers. Experimenting with 18 high-level concepts and the benchmark (both balanced and imbalanced) data sets from TRECVID, our proposed framework achieves promising results to enhance the concept detection performance of three well-known classifiers commonly used for concept detection.

Hospitals and medical centers produce an enormous amount of digital medical images every day, especially in the form of image sequences, which requires considerable storage space. One solution could be the application of lossless compression. Among available methods, JPEG-LS has excellent coding performance. However, it only compresses a single picture with intracoding and does not utilize the interframe correlation among pictures. Therefore, this paper proposes a method that combines the JPEG-LS and an interframe coding with motion vectors to enhance the compression performance of using JPEG-LS alone. Since the interframe correlation between two adjacent images in a medical image sequence is usually not as high as that in a general video image sequence, the interframe coding is activated only when the interframe correlation is high enough. With six capsule endoscope image sequences under test, the proposed method achieves average compression gains of 13.3% and 26.3% over the methods of using JPEG-LS and JPEG2000 alone, respectively. Similarly, for an MRI image sequence, coding gains of 77.5% and 86.5% are correspondingly obtained.

This paper proposes a human-centered interactive framework for automatically mining and retrieving semantic events in videos. After preprocessing, the object trajectories and event models are fed into the core components of the framework for learning and retrieval. As trajectories are spatiotemporal in nature, the learning component is designed to analyze time series data. The human feedback to the retrieval results provides progressive guidance for the retrieval component in the framework. The retrieval results are in the form of video sequences instead of contained trajectories for user convenience. Thus, the trajectories are not directly labeled by the feedback as required by the training algorithm. A mapping between semantic video retrieval and multiple instance learning (MIL) is established in order to solve this problem. The effectiveness of the algorithm is demonstrated by experiments on real-life transportation surveillance videos.

The overall aim of this special issue is to bring together the industrial applications or review a genre that: 1) provide a summary of research that advances information reuse and integration , and 2) serve as a comprehensive collection of some of the current state-of-the-art technologies within the context of information reuse and integration. Five papers were selected for this issue.

In this paper, we propose a tree-based multidimensional structure, GeM-Tree, which indexes both images and videos within a single general framework utilizing Earth Moverpsilas Distance. It can support different content-based image and video retrieval approaches, and can accommodate applications where the cross-similarity between images and videos need to be considered during content-based retrievals. Furthermore, it is flexible enough to index different video classification units and can maintain the hierarchical relationship between them. In addition, it uses a construct called hierarchical Markov model mediator to introduce high-level semantic relationships among images and different levels of video units. The experimental results indicate that GeM-Tree is a promising generalized index structure for multimedia data with low computational overhead, is flexible enough to support different retrieval approaches and generates query results with high relevance.

Semantic concept detection has emerged as an intriguing topic in multimedia research recently. The ability to interpret high-level semantics from low-level features has been the long desired goal of many researchers. In this paper, we propose a novel framework that utilizes the ability of multiple correspondence analysis (MCA) to explore the correlation between different items (feature-value pairs) and classes (concepts) to bridge the gap between the extracted low-level features and high-level semantic concepts. Using the concepts and benchmark data identified and provided by the TRECVID project, we have shown that our proposed framework demonstrates promising results and performs better than the decision tree (DT),support vector machine (SVM), and naive Bayesian (NB) classifiers that are commonly applied to the TRECVID datasets.

In this paper, a Role-based Access Control (RBAC) model is applied and extended to a multimedia version called Multi-Role Based Access Control (MRBAC), which can fully support the comprehensive and multi-level security control requirements of the distributed multimedia applications. The object-oriented concept is adopted in MRBAC to perform the hybrid role hierarchy management and security roles and rules administration. In summary, MRBAC can: 1) support the multi-level security protection for multimedia data; 2) provide access control by checking both the time constrains and IP addresses; and 3) decentralize the administration functions to make the access control management more efficient.

In this paper, a subspace-based multimedia data mining framework is proposed for video semantic analysis, specifically video event/concept detection, by addressing two basic issues, i.e.,semantic gapandrare event/concept detection. The proposed framework achieves full automation via multimodal content analysis and intelligent integration of distance-based and rule-based data mining techniques. The content analysis process facilitates the comprehensive video analysis by extracting low-level and middle-level features from audio/visual channels. The integrated data mining techniques effectively address these two basic issues by alleviating the class imbalance issue along the process and by reconstructing and refining the feature dimension automatically. The promising experimental performance on goal/corner event detection and sports/commercials/building concepts extraction from soccer videos and TRECVID news collections demonstrates the effectiveness of the proposed framework. Furthermore, its unique domain-free characteristic indicates the great potential of extending the proposed multimedia data mining framework to a wide range of different application domains.

Energy-minimizing active contour models (snakes) have been proposed for solving many computer vision problems such as object segmentation, surface reconstruction, and object tracking. Dynamic programming which allows natural enforcement of constraints is an effective method for computing the global minima of energy functions. However, this method is only limited to snake problems with one dimensional (ID) topology (i.e., a contour) and cannot handle problems with two-dimensional (2D) topology. In this paper, we have extended the dynamic programming method to address the snake problems with 2D topology using a novel graph reduction algorithm. Given a 2D snake with first order energy terms, a set of reduction operations are defined and used to simplify the graph of the 2D snake into one single vertex while retaining the minimal energy of the snake. The proposed algorithm has a polynomial-time complexity bound and the optimality of the solution for a reducible 2D snake is guaranteed. However, not all types of 2D snakes can be reduced into one single vertex using the proposed algorithm. The reduction of general planar snakes is an NP-complete problem. The proposed method has been applied to optimize 2D building topology extracted from airborne LIDAR data to examine the effectiveness of the algorithm. The results demonstrate that the proposed approach successfully found the global optima for over 98% of building topology in a polynomial time.

Digital audio and video have recently taken a center stage in the communication world, which highlights the importance of digital media information management and indexing. It is of great interest for the multimedia research community to find methods and solutions that could help bridge the semantic gap that exists between the low-level features extracted from the audio or video data and the actual semantics of the data. In this paper, we propose a novel framework that works towards reducing this semantic gap. The proposed framework uses the a priori algorithm and association rule mining to find frequent itemsets in the feature data set and generate classification rules to classify video shots to different concepts (semantics). We also introduce a novel pre-filtering architecture which reduces the high positive to negative instances ratio in the classifier training step. This helps reduce the amount of misclassification errors. Our proposed framework shows promising results in classifying multiple concepts.

In this paper, the rare event detection issue in video event detection is addressed through the proposed data mining framework which can be generalized to be domain independent. The fully automatic process via the combination of distance-based and rule-based data mining techniques can greatly reduce the number of negative (non-event) instances and the feature dimension to facilitate the final event detection, without pruning away any positive (event) testing instance along the process. The effectiveness and efficiency of the proposed framework are demonstrated over the goal event detection application based on a large collection of soccer videos with different styles.

This paper addresses the learning of search-control knowledge expressed in semi-structured natural language. That knowledge is mined from semi-structured databases and applied to search the same. As is the case with most self-referential systems, this process necessarily addresses the issues of randomization, representation, machine learning, and evolutionary programming. Machine learning becomes pervasive at all levels of representation. That is, the randomization of knowledge necessarily includes that of its representation. Application to the search and retrieval of multimedia databases is suggested.

Recently, network security has become an extremely vital issue that beckons the development of accurate and efficient solutions capable of effectively defending our network systems and the valuable information journeying through them. In this article, a distributed multiagent intrusion detection system (IDS) architecture is proposed, which attempts to provide an accurate and lightweight solution to network intrusion detection by tackling issues associated with the design of a distributed multiagent system, such as poor system scalability and the requirements of excessive processing power and memory storage. The proposed IDS architecture consists of (i) the Host layer with lightweight host agents that perform anomaly detection in network connections to their respective hosts, and (ii) the Classification layer whose main functions are to perform misuse detection for the host agents, detect distributed attacks, and disseminate network security status information to the whole network. The intrusion detection task is achieved through the employment of the lightweight Adaptive Sub-Eigenspace Modeling (ASEM)-based anomaly and misuse detection schemes. Promising experimental results indicate that ASEM-based schemes outperform the KNN and LOF algorithms, with high detection rates and low false alarm rates in the anomaly detection task, and outperform several well-known supervised classification methods such as C4.5 Decision Tree, SVM, NN, KNN, Logistic, and Decision Table (DT) in the misuse detection task. To assess the performance in a real-world scenario, the Relative Assumption Model, feature extraction techniques, and common network attack generation tools are employed to generate normal and anomalous traffic in a private LAN testbed. Furthermore, the scalability performance of the proposed IDS architecture is investigated through the simulation of the proposed agent communication scheme, and satisfactory linear relationships for both degradation of system response time and agent communication generated network traffic overhead are achieved.

In this paper, a novel supervised classification approach called collateral representative subspace projection modeling (C-RSPM) is presented. C-RSPM facilitates schemes for collateral class modeling, class-ambiguity solving, and classification, resulting a multi-class supervised classifier with high detection rate and various operational benefits including low training and classification times and low processing power and memory requirements. In addition, C-RSPM is capable of adaptively selecting nonconsecutive principal dimensions from the statistical information of the training data set to achieve an accurate modeling of a representative subspace. Experimental results have shown that the proposed C-RSPM approach outperforms other supervised classification methods such as SIMCA, C4.5 decision tree, decision table (DT), nearest neighbor (NN), KNN, support vector machine (SVM), I-NN best warping window DTW, I-NN DTW with no warping window, and the well-known classifier boosting method AdaBoost with SVM

The development of effective classification techniques, particularly unsupervised classification, is important for real-world applications since information about the training data before classification is relatively unknown. In this paper, a novel unsupervised classification algorithm is proposed to meet the increasing demand in the domain of network intrusion detection. Our proposed UNPCC (unsupervised principal component classifier) algorithm is a multiclass unsupervised classifier with absolutely no requirements for any a priori class related data information (e.g., the number of classes and the maximum number of instances belonging to each class), and an inherently natural supervised classification scheme, both which present high detection rates and several operational advantages (e.g., lower training time, lower classification time, lower processing power requirement, and lower memory requirement). Experiments have been conducted with the KDD Cup 99 data and network traffic data simulated from our private network testbed, and the promising results demonstrate that our UNPCC algorithm outperforms several well-known supervised and unsupervised classification algorithms

A novel indexing and access method, called affinity hybrid tree (AH-tree), is proposed to organize large image data sets efficiently and to support popular image access mechanisms like content-based image retrieval (CBIR) by embedding the high-level semantic image-relationship in the access mechanism as it is. AH-tree combines space-based and distance-based indexing techniques to form a hybrid structure which is efficient in terms of computational overhead and fairly accurate in producing query results close to human perception. Algorithms for similarity (range and k-nearest neighbor) queries are implemented. Results from elaborate experiments are reported which depict a low computational overhead in terms of the number of I/O and distance computations and a high relevance of query results. The proposed index structure solves the existing problems of introducing high-level image relationships in a retrieval mechanism without going through the pain of translating the content-similarity measurement into feature-level equivalence and yet maintaining an efficient structure to organize the large sets of images

Event detection is of great importance in high-level semantic indexing and selective browsing of video clips. However, the use of low-level visual-audio feature descriptors alone generally fails to yield satisfactory results in event identification due to the semantic gap issue. In this paper, we propose an advanced approach for exciting event detection in soccer video with the aid of multi-level descriptors and classification algorithm. Specifically, a set of algorithms are developed for efficient extraction of meaningful mid-level descriptors to bridge the semantic gap and to facilitate the comprehensive video content analysis. The data classification algorithm is then performed upon the combination of multimodal mid-level descriptors and low-level feature descriptors for event detection. The effectiveness and efficiency of the proposed framework are demonstrated over a large collection of soccer video data with different styles produced by different broadcasters.

This paper presents a framework that applies a series of algorithms to automatically extract building footprints from airborne light detection and ranging (LIDAR) measurements. In the proposed framework, the ground and nonground LIDAR measurements are first separated using a progressive morphological filter. Then, building measurements are identified from nonground measurements using a region-growing algorithm based on the plane-fitting technique. Finally, raw footprints for segmented building measurements are derived by connecting boundary points, and the raw footprints are further simplified and adjusted to remove noise caused by irregularly spaced LIDAR measurements. Data sets from urbanized areas including large institutional, commercial, and small residential buildings were employed to test the proposed framework. A quantitative analysis showed that the total of omission and commission errors for extracted footprints for both institutional and residential areas was about 12%. The results demonstrated that the proposed framework identified building footprints well

A rapid increase in the amount of image data and the inefficiency of traditional text-based image retrieval systems have served to make content-based image retrieval an active research field. It is crucial to effectively discover users' concept patterns through an acquired understanding of the subjective role played by humans in the retrieval process for such systems. A learning and retrieval framework is used to achieve this. It seamlessly incorporates multiple instance learning for relevant feedback to discover users concept patterns-especially in the region of greatest user interest. It also maps the local feature vector of that region to the high-level concept pattern. This underlying mapping can be progressively discovered through feedback and learning. The user guides the retrieval systems learning process using his/her focus of attention. Retrieval performance is tested to establish the feasibility and effectiveness of the proposed learning and retrieval framework

The focal goal of our research is to develop a general framework which can automatically analyze the sports video, detect the sports events, and finally offer an efficient and user-friendly system for sports video retrieval. In our earlier work, a novel multimedia data mining technique was proposed for automatic soccer event extraction by adopting multimodal feature analysis. Until now, this framework has been performed on the detection of goal and corner kick events and the results are quite impressive. Correspondingly, in this work, the detected video events are modeled and effectively stored in the database. A temporal query model is designed to satisfy the comprehensive temporal query requirements, and the corresponding graphical query language is developed. The advanced characteristics make our model particularly well suited for searching events in a large scale video database.

In this paper, we propose a multi-buffer scheduling scheme for streaming video systems. A transmission rate is obtained via a rate control algorithm, which optimally utilizes the network bandwidth and client buffer resources. The server side maintains multiple buffers for packets of different importance levels. It schedules the transmission of each packet based on the source buffer size and playback deadline to reduce the end-to-end distortion. The performance of proposed scheme is evaluated in terms of peak signal-to-noise ratio (PSNR) in the simulations, and the simulation results demonstrate the improvement of the average PSNR values compared with two other scheduling schemes.

This paper describes an implementation of vertex weighting for real-time animation of 3D trees as a result of wind force and direction. This method improves over similar techniques because it incorporates individual branch animation on 3D models, whereas previous works perform animations on the model as a whole or use planar billboards to visualize the tree. Our tree animation is used in an interactive system that visualizes the effects caused by a hurricane’s impact on a virtual city. The system uses models created with 3D Studio Max and the animations are implemented in OpenGL and the nVidia Cg shader language.

Multiple Instance Learning (MIL) is a special kind of supervised learning problem that has been studied actively in recent years. In this paper, we propose an approach based on One-Class Support Vector Machine (SVM) to solve MIL problem in the region-based Content Based Image Retrieval (CBIR). Relevance Feedback technique is incorporated to provide progressive guidance to the learning process. Performance is evaluated and the effectiveness of our retrieval algorithm has been shown through comparative studies.

Relevance feedback (RF) is a widely used technique in incorporating user's knowledge with the learning process for content-based image retrieval (CBIR). As a supervised learning technique, it has been shown to significantly increase the retrieval accuracy. However, as a CBIR system continues to receive user queries and user feedbacks, the information of user preferences across query sessions are often lost at the end of search, thus requiring the feedback process to be restarted for each new query. A few works targeting long-term learning have been done in general CBIR domain to alleviate this problem. However, none of them address the needs and long-term similarity learning techniques for region-based image retrieval. This paper proposes a latent semantic indexing (LSI) based method to utilize users' relevance feedback information. The proposed region-based image retrieval system is constructed on a multiple instance learning (MIL) framework with one-class support vector machine (SVM) as its core. Experiments show that the proposed method can better utilize users' feedbacks of previous sessions, thus improving the performance of the learning algorithm (one-class SVM).

In this paper, a neural network based framework for semantic event detection in soccer videos is proposed. The framework provides a robust solution for soccer goal event detection by combining the strength of multimodal analysis and the ability of neural network ensembles to reduce the generalization error. Due to the rareness of the goal events, the bootstrapped sampling method on the training set is utilized to enhance the recall of goal event detection. Then a group of component networks are trained using all the available training data. The precision of the detection is greatly improved via the following two steps. First, a pre-filtering step is employed on the test set to reduce the noisy and inconsistent data, and then an advanced weighting scheme is proposed to intelligently traverse and combine the component network predictions by taking into consideration the prediction performance of each network. A set of experiments are designed to compare the performance of different bootstrapped sampling schemes, to present the strength of the proposed weighting scheme in event detection, and to demonstrate the effectiveness of our framework for soccer goal event detection.

It follows from the nonreducibility of the theorization problem that an arbitrary proof cannot be valid on an absolute scale. Thus, in order for an arbitrary proof to be generative, it must be self-referential; but then, it must also be heuristic if not incomplete as a consequence. By relaxing the validity requirement, heuristic (i.e., relative) proof techniques are enabled. We show that heuristics are search randomizations in space-time. It is shown how one can develop heuristics, which are randomizations of knowledge. Even more intriguing, it is shown that heuristic proof is to formal proof what fuzzy logic is to formal logic. Simply put, the paper argues for the need to relax the notion of formal proof if AI is to advance.

We propose a new multimedia data mining framework for the extraction of soccer goal events in soccer videos by using combined multimodal analysis and decision tree logic. The extracted events can be used to index the soccer videos. We first adopt an advanced video shot detection method to produce shot boundaries and some important visual features. Then, the visual/audio features are extracted for each shot at different granularities. This rich multimodal feature set is filtered by a pre-filtering step to clean the noise as well as to reduce the irrelevant data. A decision tree model is built upon the cleaned data set and is used to classify the goal shots. Finally, the experimental results demonstrate the effectiveness of our framework for soccer goal extraction.

As the development of the Internet continues, congestion control has become a big issue to the computer network society. Most congestion control schemes fall into two categories, end-to-end and hop-by-hop schemes. We propose a novel hop-by-hop algorithm that originates from a classical traffic control algorithm. The experimental results show that our proposed algorithm can achieve short delays and quick responses to the congestion situations and cause no packet loss. It can also minimize the bandwidth requirement and achieve a very high buffer usage level for nodes along the transmission path.

We present a new framework for streaming video over Internet in this paper. The sending rate is dynamically adjusted to obtain a maximal utilization of the client buffer and minimal bandwidth allocation. To guarantee a more reliable and better quality of delivery of the video frames, retransmission and selective drop of different frame packets are integrated into our framework. Under severe network congestion, an adaptive playback schedule can provide relative high video quality at a low frame rate. Comparisons are made with the most current streaming approaches using the H.26L video coder to evaluate the performance of the framework. Simulations results show that PSNR is increased in our approach, which provides a better quality of the decoded frames, and the quality of the decoded frames also changes more smoothly.

A novel active queue management scheme is proposed to employ in the routers that deal with both the best-effort traffic flows and multimedia traffic flows. Most of the available active queue management schemes consider only TCP flows, which results in unexpected congestion when dealing with multimedia flows. The queue size and packet receiving rate are the common parameters to calculate the marking probability in the queue. The use of round trip time (RTT) in the process of marking probability calculation has advantages over other approaches. We discuss the applicability of our approach when the framework in our earlier paper is used for rate adaptation in multimedia flows. With the use of RTT in packet marking probability calculations, we can assure rate reduction in both best-effort and multimedia flows before a queue gets exhausted. The most influential feature of the proposed approach is that it can work with the existing systems. The soundness of the proposed approach is inferred from the simulation results.

The paper proposes a method to discover effectively users' concept patterns when multiple objects of interest (e.g., foreground and background objects) are involved in content-based image retrieval. The proposed method incorporates multiple instance learning into the user relevance feedback in a seamless way to discover where the user's objects/regions of most interest are and how to map the local features of that(those) region(s) to the user's high-level concepts. A three-layer neural network is used to model the underlying mapping progressively through the feedback and learning procedure.

In this paper, a framework named SMARXO is proposed to address the security issues in multimedia applications by adopting RBAC (Role-Based Access Control), XML, and Object-Relational Databases. Compared with the other existing security models or projects, SMARXO can deal with more intricate situations. First, the image object-level security and video scene/shot-level security can be easily achieved. Second, the temporal constrains and IP address restrictions are modeled for the access control purpose. Finally, XML queries can be performed such that the administrators can proficiently retrieve useful information from the security roles and policies.

In this paper, we propose a unified framework, called Markov Model Mediator (MMM), to facilitate image database clustering and to improve the query performance. The structure of the MMM framework consists of two hierarchical levels: local MMMs and integrated MMMs, which model the affinity relations among the images within a single image database and within a set of image databases, respectively, via an effective data mining process. The effectiveness and efficiency of the MMM framework for database clustering and image retrieval are demonstrated over a set of image databases which contain various numbers of images with different dimensions and concept categories.

Abstract As an environmental phenomenon, hurricanes cause significant property damage and loss of life in coastal areas almost every year. Research concerning hurricanes and their aftermath is gaining more and more attention nowadays. This paper presents our work in designing and building a Web‐based distributed software system that can be used for the statistical analysis and projection of hurricane occurrences. Firstly, our system is a large‐scale system and can handle the huge amount of hurricane data and intensive computations in hurricane data analysis and projection. Secondly, it is a distributed system, which allows multiple users at different locations to access the system simultaneously and to share and exchange the data and data model. Thirdly, our system is a database‐centered system where the Oracle database is employed to store and manage the large amount of hurricane data, the hurricane model and the projection results. Finally, a three‐tier architecture has been adopted to make our system robust and resistant to the potential change in the lifetime of the system. This paper focuses on the three‐tier system architecture, describing the design and implementation of the components at each layer. Copyright © 2004 John Wiley & Sons, Ltd.

The number of information sources and the volumes of data in these information sources have greatly increased, which may be attributed to the ever-increasing complexity of real-world applications. The enormous amount of information available in the information sources in a distributed information-providing environment has created a need to provide users with tools to effectively and efficiently navigate and retrieve information. Queries in such an environment often access information from multiple information sources. This may be attributed to navigational characteristics. Clusters provide a structure for organizing the large number of information sources for efficient browsing, searching, and retrieval. This paper presents a stochastically-based clustering mechanism, called the Markov model mediator (MMM), to group the information sources into a set of useful clusters. Each information source cluster groups those information sources that show similarities in their data access behavior. Information sources within the same cluster are expected to be able to provide most of the required information among themselves for user queries that are closely related with respect to a particular application. This can significantly improve system response time, query performance, and result in an overall improvement in decision support. Empirical studies on real databases are performed and the results demonstrate that our proposed mechanism leads to a better set of clusters in comparison with other clustering methods. This serves to illustrate the effectiveness of our proposed MMM mechanism.

In this paper, a powerful open Multiple Instance Learning (MIL) framework is proposed. Such an open framework is powerful since different sub-methods can be plugged into the framework to generate different specific Multiple Instance Learning algorithms. In our proposed framework, the Multiple Instance Learning problem is first converted to an unconstrained optimization problem by the Minimum Square Error (MSE) criterion, and then the framework can be constructed with an open form of hypothesis and gradient search method. The proposed Multiple Instance Learning framework is applied to the drug activity problems in bioinformatics applications. Specifically, experiments are conducted on the Musk-I dataset to predict the binding activity of drug molecules. In the experiments, an algorithm with the exponential hypothesis model and the Quasi-Newton method is embedded into our proposed framework. We compare our proposed framework with other existing algorithms and the experimental results show that our proposed framework yields a good accuracy of classification, which demonstrates the feasibility and effectiveness of our framework.

This paper describes an approach to develop a high performance animation environment for storm surge. The system provides the capability to simulate the storm surge effects in the physical world by (1) modeling a region using the airborne light detection and ranging (LIDAR) data, USGS orthophotos, RLG road data and photos; (2) animating the storm impact by using the features of this model; and (3) providing the capability for users to explore the animation environment. We present our system by modeling the dataset collected from Ft. Lauderdale, a region in South Florida, USA.

This paper presents a content-based image retrieval (CBIR) system that incorporates real-valued multiple instance learning (MIL) into the user relevance feedback (RF) to learn the user's subjective visual concepts, especially where the user's most interested region and how to map the local feature vector of that region to the high-level concept pattern of the user. RF provides a way to obtain the subjectivity of the user's high-level visual concepts, and MIL enables the automatic learning of the user's high-level concepts. The user interacts with the CBIR system by relevance feedback in a way that the extent to which the image samples retrieved by the system are relevant to the user's intention is labeled. The system in turn applies the MIL method to find user's most interested image region from the feedback. A multilayer neural network that is trained progressively through the feedback and learning procedure is used to map the low-level image features to the high-level concepts.

In this paper, we propose a rate control mechanism for multimedia multicast streaming in the Internet. A single and optimal transmission rate that is adaptive to the network congestion, buffer occupancies, and the playback requirements of the member clients in the multicast group is allocated. We optimize the multicasting delivery in the Internet by improving the buffer occupancy of all clients for on-time presentation while using the minimal transmission rate. Scalable playback is used for the clients with low bandwidth capabilities in a heterogeneous environment. Simulation results show that the buffer capacity can be efficiently improved and fairness is ensured among the member clients.

In this paper, we propose a per-class queue management and adaptive packet drop mechanism in the routers for Internet congestion control. We model active queue management as an optimization problem and our proposed mechanism provides congestion control and fairness for different types of traffic flows. An optimal packet drop rate is obtained to maintain a relatively small queue occupancy, which provides a less queue delay delivery of packets. Simulations are conducted to compare our approach with the fixed packet drop rate approach. Moreover, the queue occupancy and the packet drop rates obtained are both upper bounded, which is meaningful for providing the class-based guaranteed delay services for real-time multimedia applications.

In this demonstration, we present an image retrieval system to support multimedia authoring and presentation. An affinity-based mechanism, Markov Model Mediator (MMM), is used as the search engine for the system, which utilizes both the low-level image features and the learned high-level concepts via user access patterns and access frequencies. This system is one of the major components of MediaManager, a distributed multimedia management system developed by us. Both retrieval and learning facilities are supported in this system. The retrieval system also provides input information to the Multimedia Augmented Transition Network (MATN) environment for multimedia authoring and presentation.

This paper addresses the problem of mapping natural language to its semantics. It presupposes that the input is in random (compressed) form and proceeds to detail a methodology for extracting the semantics from that normal form. The idea is to enumerate contextual cues and learn to associate those cues with meaning. The process is inherently fuzzy and for this reason is also inherently adaptive in nature. It is shown that the influence of context on meaning grows exponentially with the length of a word sequence. This suggests that rule-based randomization plays a key role in rendering a field-effect natural language semantic mapping tractable. An example of rule-based randomization for semantic normalization is as follows. Suppose that two commands to a robot are deemed to be equivalent; namely, "Grasp and pick up the glass" and "Hold the cup and raise it". Their mutual normalization might then be, "Grab container. Lift container." Clearly, the randomization process can be effected by rules. Also, the normalized syntax makes the result of any semantic mapping process-such as detailed herein-more efficient. A natural language front-end is described, which is designed to reduce the impedance mismatch between the human and the machine. Most significantly, the effective translation of natural language semantics is shown to critically depend on an accelerated capability for learning.

Recent advances in airborne light detection and ranging (LIDAR) technology allow rapid and inexpensive measurements of topography over large areas. This technology is becoming a primary method for generating high-resolution digital terrain models (DTMs) that are essential to numerous applications such as flood modeling and landslide prediction. Airborne LIDAR systems usually return a three-dimensional cloud of point measurements from reflective objects scanned by the laser beneath the flight path. In order to generate a DTM, measurements from nonground features such as buildings, vehicles, and vegetation have to be classified and removed. In this paper, a progressive morphological filter was developed to detect nonground LIDAR measurements. By gradually increasing the window size of the filter and using elevation difference thresholds, the measurements of vehicles, vegetation, and buildings are removed, while ground data are preserved. Datasets from mountainous and flat urbanized areas were selected to test the progressive morphological filter. The results show that the filter can remove most of the nonground points effectively.

One key technology of intelligent transportation systems is the use of advanced sensor systems for on-line surveillance to gather detailed information on traffic conditions. Traffic video analysis can provide a wide range of useful information to traffic planners. In this context, the object-level indexing of video data can enable vehicle classification, traffic flow analysis, incident detection and analysis at intersections, vehicle tracking for traffic operations, and update of design warrants. In this paper, a learning-based automatic framework is proposed to support the multimedia data indexing and querying of spatio-temporal relationships of vehicle objects in a traffic video sequence. The spatio-temporal relationships of vehicle objects are captured via the proposed unsupervised image/video segmentation method and object tracking algorithm, and modeled using a multimedia augmented transition network model and multimedia input strings. An efficient and effective background learning and subtraction technique is employed to eliminate the complex background details in the traffic video frames. It substantially enhances the efficiency of the segmentation process and the accuracy of the segmentation results to enable more accurate video indexing and annotation. The paper uses four real-life traffic video sequences from several road intersections under different weather conditions in the study experiments. The results show that the proposed framework is effective in automating data collection and access for complex traffic situations.

In this paper, a method of automatically classifying web documents into a set of categories using the fuzzy association concept is proposed. Using the same word or vocabulary to describe different entities creates ambiguity, especially in the web environment where the user population is large. To solve this problem, fuzzy association is used to capture the relationships among different index terms or keywords in the documents, i.e., each pair of words has an associated value to distinguish itself from the others. Therefore, the ambiguity in word usage is avoided. Experiments using data sets collected from two web portals: Yahoo! and Open Directory Project are conducted. We compare our approach to the vector space model with the cosine coefficient. The results show that our approach yields higher accuracy compared to the vector space model.

An effective content-based visual image retrieval system is presented. This system consists of two main components: visual content extraction and indexing, and query engine. Each image in the image database is represented by its visual features: color and spatial information. The system uses a color label histogram with only thirteen bins to extract the color information from an image in the image database. A unique unsupervised segmentation algorithm combined with the wavelet technique generates the spatial feature of an image automatically. The resulting feature vectors are relatively low in dimensions compared to those in other systems. The query engine employs a color filter and a spatial filter to dramatically reduce the search range. As a result, queue processing is speeded up. The experimental results demonstrate that our system is capable of retrieving images that belong to the same category.

Automatic video scene change detection is a challenging task. Using audio or visual information alone often cannot provide a satisfactory solution. However, how to combine audio and visual information efficiently still remains a difficult issue since there are various cases in their relationship due to the versatility of videos. We present an effective scene change detection method that adopts the joint evaluation of the audio and visual features. First, video information is used to find the shot boundaries. Second, the audio features for each video shot can be extracted. Lastly, an audio-video combination schema is proposed to detect the video scene boundaries.

Recently, efficient network resource management and quality-of-service (QoS) guarantee become more and more important for multimedia applications and services, especially when considering network delays. An optimal bandwidth allocation scheme is introduced. It achieves maximal utilization of the client buffer and minimal allocation of bandwidth for each client. The proposed scheme allocates bandwidth to multiple clients requesting services from a server by adjusting the transmission rates based on the client buffer occupancy, the playback requirements of the individual client and the network delays. Simulations for the single client and multiple client scenarios are conducted under different network congestion levels. The simulation results show that our approach performs better in comparison with the fixed rate allocation approach and the rate by playback requirement approach, since it avoids underflows and overflows efficiently and provides QoS for more clients with limited available bandwidth in the network.

Wavelet neural networks (WNN) have recently attracted great interest, because of their advantages over radial basis function networks (RBFN) as they are universal approximators but achieve faster convergence and are capable of dealing with the so-called "curse of dimensionality". In addition, WNN are generalized RBFN. However, the generalization performance of WNN trained by least-squares approach deteriorates when outliers are present. In this paper, we propose a robust wavelet neural network based on the theory of robust regression for dealing with outliers in the framework of function approximation. By adaptively adjusting the number of training data involved during training, the efficiency loss in the presence of Gaussian noise is accommodated. Simulation results are demonstrated to validate the generalization ability and efficiency of the proposed network.

Due to the entry of Taiwan into WTO and the recently liberalized Petroleum Management Law, the oil market in Taiwan is liberalized and thus is becoming more competitive. However, the space limitation and the residents' increasing awareness of environmental protection issues in the island make international vendors unavoidably have to rent tanks from domestic oil companies. In order to help the leaseholder maximize revenue by increasing the availability of tanks, an efficient maintenance scheduling is needed. This paper introduces a tabu-based genetic algorithm (TGA) and its implementation for solving a real-world maintenance scheduling problem of oil storage tanks. TGA incorporates a tabu list to prevent inbreeding and utilizes an aspiration criterion to supply moderate selection pressure so that the selection efficiency is improved, and the population diversity is maintained. The experimental results validate that TGA outperform GA in terms of solution quality and convergence efficiency. Keywords: Tabu-based genetic algorithm, maintenance scheduling, tabu search, genetic algorithm.

Due to the explosive growth of available information on the World Wide Web (WWW), users have suffered from the information overload. To alleviate this problem, there is a need for an intelligent tool to help the users screening and filtering for interesting and useful information. In this paper, a method of automatically identifying topics for Web documents via a classification technique is proposed. Topic identification can be applied as a filtering tool for recommender systems to prune down the number of documents to within some particular topics. We adopt the fuzzy association concept as a machine learning technique to classify the documents into some predefined categories or topics. Our approach is compared to the vector space model with the cosine coefficient using the data sets collected from three different Web portals: Yahoo!, Open Directory Project and Excite. The results show that our approach yields higher classification accuracy compared to the vector space model.

In this paper a novel unsupervised segmentation framework for texture image queries is presented. The proposed framework consists of an unsupervised segmentation method for texture images, and a multi-filter query strategy. By applying the unsupervised segmentation method on each texture image, a set of texture feature parameters for that texture image can be extracted automatically. Based upon these parameters, an effective multi-filter query strategy which allows the users to issue texture-based image queries is developed The test results of the proposed framework on 318 texture images obtained from the MIT VisTex and Brodatz database are presented to show its effectiveness.

In order to manage the growing amount of video information efficiently, a video scene change detection method is necessary. Many advanced video applications such as video on demand (VOD) and digital library also require the scene change detection to organize the video content. In this paper, we present an effective scene change detection method using an unsupervised segmentation algorithm and the technique of object tracking based on the results of the segmentation. Our results have shown that this method can perform not only accurate scene change detection, but also obtain object level information of the video frames, which is very useful for video content indexing and analysis. 1.

Efficient utilization of network resources is essential in the provision of quality multimedia services. Different approaches have been proposed to shape the multimedia streams as a transmission schedule with smoothed traffic burst. In this paper, we concentrate on the problem of efficient use of bandwidth and client buffer. For this purpose, an optimal transmission schedule is proposed to provide the minimal allocation of the bandwidth and maximal utilization of the client buffer. Simulation results show that the shaping results obtained can dynamically adjust the transmission rate according to the buffer packet sizes and playback rates at the client to avoid the loss of packets, and at the same time achieve the minimal bandwidth allocation and maximal utilization of the client buffer. 1.

In today's fast-growing information age, multimedia data is becoming more and more common in daily applications, however this data is nearly useless if there is no computer-aided browsing, searching, and retrieving mechanism to obtain the desired contents. In this demonstration, we present a multimedia query and presentation management system for multimedia databases. It uses an unsupervised segmentation method for image and video feature extraction. A presentation design interface systems is provided that allows query results to be used to create multimedia presentations.

In this paper, an affinity-based approach that provides good similarity measures for Web document clustering to discover user access behavior on the World Wide Web (WWW) is proposed. The proposed approach generates the similarity measures for groups of Web documents by considering the user access patterns. Any clustering algorithm using better similarity measures should yield better clusters for discovering user access behavior. By utilizing the discovered user access behavior, for example, the companies can precisely target their potential customers and convince them to purchase their products or services in electronic commerce. An experiment on a real data set is conducted and the experimental result shows that the proposed approach yields a better performance than the cosine coefficient and the Euclidean distance method under the partitioning around medoid (PAM) method.

As more information sources become available in multimedia systems, the development of abstract semantic models for video, audio, text, and image data is becoming very important. An abstract semantic model has two requirements: it should be rich enough to provide a friendly interface of multimedia presentation synchronization schedules to the users and it should be a good programming data structure for implementation in order to control multimedia playback. An abstract semantic model based on an augmented transition network (ATN) is presented. The inputs for ATNs are modeled by multimedia input strings. Multimedia input strings provide an efficient means for iconic indexing of the temporal/spatial relations of media streams and semantic objects. An ATN and its subnetworks are used to represent the appearing sequence of media streams and semantic objects. The arc label is a substring of a multimedia input string. In this design, a presentation is driven by a multimedia input string. Each subnetwork has its own multimedia input string. Database queries relative to text, image, and video can be answered via substring matching at subnetworks. Multimedia browsing allows users the flexibility to select any part of the presentation they prefer to see. This means that the ATN and its subnetworks can be included in multimedia database systems which are controlled by a database management system (DBMS). User interactions and loops are also provided in an ATN. Therefore, ATNs provide three major capabilities: multimedia presentations, temporal/spatial multimedia database searching, and multimedia browsing.

The World Wide Web (WWW) has become one of the fastest growing applications on the Internet today. More and more information sources have linked online through WWW, but finding information on the WWW is also a great challenge. For most of the users, the information retrieved is not well organized and the access time is considered high on the WWW currently. Therefore, there is a need to develop a good mechanism to organize and manage the tremendous size and various kinds of information to facilitate the functionality of a search engine for information retrieval on the WWW. In response to such a demand, we propose a Markov Model Mediator (MMM) mechanism which employs the affinity-based data mining techniques to organize and manage the information sources so that the most relevant documents are clustered together to achieve higher recall and precision values for information retrieval on the WWW. 1 Introduction Since its introduction in the early 1990s, the World Wide Web (WWW) has becom...

Summary form only given. We present optimizations that improve the compression and computational efficiency of Lempel-Ziv (LZ77) and Huffman algorithms. The compression performance of the LZ77 algorithm can be improved by using an optimized longest match parsing strategy. Another factor that can be considered is the size of the reference to the matching string, which can vary due to the secondary Huffman compression. We present an efficient parsing algorithm that considers both factors while minimizing the computational requirements. Our second optimization technique optimizes static Huffman encoding by efficiently dividing the input into blocks of varying size with uneven character frequency distributions.

S.C. Chen et al. (1999) proposed a multimedia augmented transition network (ATN) model, together with its multimedia input strings, to model and structure video data. This multimedia ATN model was based on an ATN model that had been used within the artificial intelligence (AI) arena for natural-language understanding systems, and its inputs were modeled by multimedia input strings. The temporal and spatial relations of semantic objects were captured by an unsupervised video segmentation method called the SPCPE (simultaneous partitioning and class parameter estimation) algorithm, and they were modeled by the multimedia input strings. However, the segmentation method used was not able to identify objects that are overlapped together within video frames. The identification of overlapped objects is a great challenge. For this purpose, a backtrack-chain-update-split algorithm is developed in this paper that identifies the split segment (object) and uses this information in the current frame to update the previous frames in a backtrack-chain manner. The proposed split algorithm provides more accurate temporal and spatial information of the semantic objects for video indexing.

This paper presents a presentation semantic model that is based on the augmented transition network (ATN) for asynchronous distance learning system called Java-based Integrated Asynchronous Distance Learning (JIADL) system. The JIADL system can support diverse asynchronous distance learning services. Unlike most related work in the literature, we integrate RealPlayer and Java technology so that the superiority of both models can be complemented. A course sample is illustrated to validate the effectiveness of the paradigm proposed. How to use the proposed multimedia ATN model to model the diverse requirements of a distance learning multimedia presentation is also discussed. The multimedia ATN model is powerful in modeling the asynchronization for distance learning multimedia presentations. Furthermore, in addition to supporting asynchronous distance learning, our system can be applied to a wide range of potential value-added applications.

A distributed database is a collection of data sources distributed across many computers. It is a single logical and centrally managed database stored in multiple physical locations. In order to access data locally and manage data globally, a good distributed database system (DDBS) that can supply reliable and up-to-date information is critical. A good DDBS should have functionalities such as providing users with timely and flexible access to information, providing DBA (database administrator) personnel with tools to analyze the data in a meaningful manner, and allowing the personnel to control the safety and integrity of the data. However, query processing in such a distributed database system is complicated. For example, some important issues are how to keep data up-to-date when it is physically dispersed, and how to propagate information to the whole DDBS when an update occurs. In response to these issues, we propose an expert query system that uses a Bayesian network as its framework to propagate information and to keep data up-to-date for a DDBS. The goal of this query system is to assist the DBA personnel in maintaining the data dynamically and to assist the users in accessing reliable and timely information.

Due to the complexity of real-world applications, the number of databases and the volumes of data in databases have increased tremendously. With the explosive growth in the amount and complexity of data, how to effectively organize the databases and utilize the huge amount of data becomes important. For this purpose, a probabilistic network that organizes a network of databases and manages the data in the databases is proposed. Each database is represented as a node in the probabilistic network and the affinity relations of the databases are embedded in the proposed Markov model mediator (MMM) mechanism. Probabilistic reasoning is used to formulate and derive the probability distributions for an MMM. Once the probability distributions of each MMM are generated, a stochastic process is conducted to calculate the similarity measures for pairs of databases. The similarity measures are transformed into the branch probabilities of the probabilistic network. Then, the data in the database can be managed and utilized to allow user queries for database searching and information retrieval. An example is included to illustrate how to model each database into an MMM and how to organize the network of databases into a probabilistic network.

Association rule mining has recently attracted strong attention and proven to be a highly successful technique for extracting useful information from very large databases. In this paper, we explore a generalized affinity-based association mining which discovers quasi-equivalent media objects in a distributed information-providing environment consisting of a network of heterogeneous databases which could be relational databases, hierarchical databases, object-oriented databases, multimedia databases, etc. Online databases, consisting of millions of media objects, have been used in business management, government administration, scientific and engineering data management, and many other applications owing to the recent advances in high-speed communication networks and large-capacity storage devices. Because of the navigational characteristic, queries in such an information-providing environment tend to traverse equivalent media objects residing in different databases for the related data records. As the number of databases increases, query processing efficiency depends heavily on the capability to discover the equivalence relationships of the media objects from the network of databases. Theoretical terms along with an empirical study of real databases are presented.

In an interactive multimedia information system, users should have the flexibility to browse and choose various scenarios they want to see. This means that two-way communications should be captured by the conceptual model. Digital video has gained increasing popularity in many multimedia applications. Instead of sequential access to the video contents, the structuring and modeling of video data so that users can quickly and easily browse and retrieve interesting materials has become an important issue in designing multimedia information systems. An abstract semantic model called the augmented transition network (ATN), which can model video data and user interactions, is proposed in this paper. An ATN and its subnetworks can model video data based on different granularities, such as scenes, shots and key frames. Multimedia input strings are used as inputs for ATNs. The details of how to use multimedia input strings to model video data are also discussed. Key frame selection is based on the temporal and spatial relations of semantic objects in each shot. These relations are captured from our proposed unsupervised video segmentation method, which considers the problem of partitioning each frame as a joint estimation of the partition and class parameter variables. Unlike existing semantic models, which only model multimedia presentation, multimedia database searching or browsing, ATNs together with multimedia input strings can model these three in one framework.