Current text-to-image (T2I) models often fail to account for diverse human experiences, leading to misaligned systems. We advocate for pluralism in AI alignment, where an AI understands and is steerable towards diverse, and often conflicting, human values. Our work provides three core contributions to achieve this in T2I models. First, we introduce a novel dataset for Diverse Intersectional Visual Evaluation (DIVE) -- the first multimodal dataset for pluralistic alignment. It enables deep alignment to diverse safety perspectives through a large pool of demographically intersectional human raters who provided extensive feedback across 1000 prompts, with high replication, capturing nuanced safety perceptions. Second, we empirically confirm demographics as a crucial proxy for diverse viewpoints in this domain, revealing significant, context-dependent differences in harm perception that diverge from conventional evaluations. Finally, we discuss implications for building aligned T2I models, including efficient data collection strategies, LLM judgment capabilities, and model steerability towards diverse perspectives. This research offers foundational tools for more equitable and aligned T2I systems.Content Warning: The paper includes sensitive content that may be harmful.

Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo, Aida Mostafazadeh Davani, Alicia Parrish, Alex Taylor, Mark Diaz, Ding Wang, Gregory Serapio-García. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Machine learning approaches often require training and evaluation datasets with a clear separation between positive and negative examples. This requirement overly simplifies the natural subjectivity present in many tasks, and obscures the inherent diversity in human perceptions and opinions about many content items. Preserving the variance in content and diversity in human perceptions in datasets is often quite expensive and laborious. This is especially troubling when building safety datasets for conversational AI systems, as safety is socio-culturally situated in this context. To demonstrate this crucial aspect of conversational AI safety, and to facilitate in-depth model performance analyses, we introduce the DICES (Diversity In Conversational AI Evaluation for Safety) dataset that contains fine-grained demographics information about raters, high replication of ratings per item to ensure statistical power for analyses, and encodes rater votes as distributions across different demographics to allow for in-depth explorations of different aggregation strategies. The DICES dataset enables the observation and measurement of variance, ambiguity, and diversity in the context of safety for conversational AI. We further describe a set of metrics that show how rater diversity influences safety perception across different geographic regions, ethnicity groups, age groups, and genders. The goal of the DICES dataset is to be used as a shared resource and benchmark that respects diverse perspectives during safety evaluation of conversational AI systems.

Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems. Neglecting the fundamental importance of data has given rise to inaccuracy, bias, and fragility in real-world applications, and research is hindered by saturation across existing dataset benchmarks. In response, we present DataPerf, a community-led benchmark suite for evaluating ML datasets and data-centric algorithms. We aim to foster innovation in data-centric AI through competition, comparability, and reproducibility. We enable the ML community to iterate on datasets, instead of just architectures, and we provide an open, online platform with multiple rounds of challenges to support this iterative development. The first iteration of DataPerf contains five benchmarks covering a wide spectrum of data-centric techniques, tasks, and modalities in vision, speech, acquisition, debugging, and diffusion prompting, and we support hosting new contributed benchmarks from the community. The benchmarks, online evaluation platform, and baseline implementations are open source, and the MLCommons Association will maintain DataPerf to ensure long-term benefits to academia and industry.

Abstract Large neural models have brought a new challenge to natural language generation (NLG): It has become imperative to ensure the safety and reliability of the output of models that generate freely. To this end, we present an evaluation framework, Attributable to Identified Sources (AIS), stipulating that NLG output pertaining to the external world is to be verified against an independent, provided source. We define AIS and a two-stage annotation pipeline for allowing annotators to evaluate model output according to annotation guidelines. We successfully validate this approach on generation datasets spanning three tasks (two conversational QA datasets, a summarization dataset, and a table-to-text dataset). We provide full annotation guidelines in the appendices and publicly release the annotated data at https://github.com/google-research-datasets/AIS.

Ka Wong, Praveen Paritosh, Lora Aroyo. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

AI models are increasingly applied in high-stakes domains like health and conservation. Data quality carries an elevated significance in high-stakes AI due to its heightened downstream impact, impacting predictions like cancer detection, wildlife poaching, and loan allocations. Paradoxically, data is the most under-valued and de-glamorised aspect of AI. In this paper, we report on data practices in high-stakes AI, from interviews with 53 AI practitioners in India, East and West African countries, and USA. We define, identify, and present empirical evidence on Data Cascades—compounding events causing negative, downstream effects from data issues—triggered by conventional AI/ML practices that undervalue data quality. Data cascades are pervasive (92% prevalence), invisible, delayed, but often avoidable. We discuss HCI opportunities in designing and incentivizing data excellence as a first-class citizen of AI, resulting in safer and more robust systems for all.

Successful knowledge graphs (KGs) solved the historical knowledge acquisition bottleneck by supplanting an expert focus with a simple, crowd-friendly one: KG nodes represent popular people, places, organizations, etc., and the graph arcs represent common sense relations like affiliations, locations, etc. Techniques for more general, categorical, KG curation do not seem to have made the same transition: the KG research community is still largely focused on methods that belie the common-sense characteristics of successful KGs. In this paper, we propose a simple approach to acquiring and reasoning with class-level attributes from the crowd that represent broad common sense associations between categories. We pick a very real industrial-scale data set and problem: how to augment an existing knowledge graph of places and products with associations between them indicating the availability of the products at those places, which would enable a KG to provide answers to questions like, "Where can I buy milk nearby?" This problem has several practical challenges, not least of which is that only 30% of physical stores (i.e. brick & mortar stores) have a website, and fewer list their product inventory, leaving a large acquisition gap to be filled by methods other than information extraction (IE). Based on a KG-inspired intuition that a lot of the class-level pairs are part of people's general common sense, e.g. everyone knows grocery stores sell milk and don't sell asphalt, we acquired a mixture of instance- and class- level pairs (e.g. , , resp.) from a novel 3-tier crowdsourcing method, and demonstrate the scalability advantages of the class-level approach. Our results show that crowdsourced class-level knowledge can provide rapid scaling of knowledge acquisition in this and similar domains, as well as long-term value in the KG.

Video summaries or highlights are a compelling alternative for exploring and contextualizing unprecedented amounts of video material. However, the summarization process is commonly automatic, non-transparent and potentially biased towards particular aspects depicted in the original video. Therefore, our aim is to help users like archivists or collection managers to quickly understand which summaries are the most representative for an original video. In this paper, we present empirical results on the utility of different types of visual explanations to achieve transparency for end users on how representative video summaries are, with respect to the original video. We consider four types of video summary explanations, which use in different ways the concepts extracted from the original video subtitles and the video stream, and their prominence. The explanations are generated to meet target user preferences and express different dimensions of transparency: concept prominence, semantic coverage, distance and quantity of coverage. In two user studies we evaluate the utility of the visual explanations for achieving transparency for end users. Our results show that explanations representing all of the dimensions have the highest utility for transparency, and consequently, for understanding the representativeness of video summaries.

Event detection is still a difficult task due to the complexity and the ambiguity of such entities. On the one hand, we observe a low inter-annotator agreement among experts when annotating events, disregarding the multitude of existing annotation guidelines and their numerous revisions. On the other hand, event extraction systems have a lower measured performance in terms of F1-score compared to other types of entities such as people or locations. In this paper we study the consistency and completeness of expert-annotated datasets for events and time expressions. We propose a data-agnostic validation methodology of such datasets in terms of consistency and completeness. Furthermore, we combine the power of crowds and machines to correct and extend expert-annotated datasets of events. We show the benefit of using crowd-annotated events to train and evaluate a state-of-the-art event extraction system. Our results show that the crowd-annotated events increase the performance of the system by at least 5.3%.

Information Retrieval systems rely on large test collections to measure their effectiveness in retrieving relevant documents. While the demand is high, the task of creating such test collections is laborious due to the large amounts of data that need to be annotated, and due to the intrinsic subjectivity of the task itself. In this paper we study the topical relevance from a user perspective by addressing the problems of subjectivity and ambiguity. We compare our approach and results with the established TREC annotation guidelines and results. The comparison is based on a series of crowdsourcing pilots experimenting with variables, such as relevance scale, document granularity, annotation template and the number of workers. Our results show correlation between relevance assessment accuracy and smaller document granularity, i.e., aggregation of relevance on paragraph level results in a better relevance accuracy, compared to assessment done at the level of the full document. As expected, our results also show that collecting binary relevance judgments results in a higher accuracy compared to the ternary scale used in the TREC annotation guidelines. Finally, the crowdsourced annotation tasks provided a more accurate document relevance ranking than a single assessor relevance label. This work resulted is a reliable test collection around the TREC Common Core track.

This paper reports on a reimplementation of a system on detecting implicit positive meaning from negated statements. In the original regression experiment, different positive interpretations per negation are scored according to their likelihood. We convert the scores to classes and report our results on both the regression and classification tasks. We show that a baseline taking the mean score or most frequent class is hard to beat because of class imbalance in the dataset. Our error analysis indicates that an approach that takes the information structure into account (i.e. which information is new or contrastive) may be promising, which requires looking beyond the syntactic and semantic characteristics of negated statements.

This paper presents a data assessment tool, InfoQ, developed within the context of the QuPiD2 project (Quality and Perspectives in Deep Data). QuPiD2 investigates methods and tools for computational support to capture, model and assess the diversity in quality of online information and the multitude of perspectives (i.e. beliefs, opinions, and world views) being reflected in this online information. It explores the perception factors and linguistic phenomena reflecting a certain perspective or influencing the perceived quality of text, and develop tools to automatically detect perspectives and assess quality. The quality assessment performed by InfoQ provides a multi-perspective view along multiple quality dimensions (precision, trustworthiness, accuracy, neutrality, readability, relevance with respect to a given topic). In order to address the intrinsic subjectivity of quality assessment, InfoQ is based on a symbiotic pipeline that brings together humans and machines to gather and train information assessments and the factors that impact them. InfoQ machine learning models (multi -label regression and Support Vector Machines) are trained on quality assessments provided by experts and crowds and rely on automatic extraction of document features, such as NLP features, provenance features, Web-based technical features (e.g., latency, i.e., Website speed test), and crowdsourced features (e.g., Web of Trust trustworthiness scores) to identify correlations between document features and human assessments, and allow for the assessment of any Web document.

FrameNet is a computational linguistics resource composed of semantic frames, high-level concepts that represent the meanings of words. In this paper, we present an approach to gather frame disambiguation annotations in sentences using a crowdsourcing approach with multiple workers per sentence to capture inter-annotator disagreement. We perform an experiment over a set of 433 sentences annotated with frames from the FrameNet corpus, and show that the aggregated crowd annotations achieve an F1 score greater than 0.67 as compared to expert linguists. We highlight cases where the crowd annotation was correct even though the expert is in disagreement, arguing for the need to have multiple annotators per sentence. Most importantly, we examine cases in which crowd workers could not agree, and demonstrate that these cases exhibit ambiguity, either in the sentence, frame, or the task itself, and argue that collapsing such cases to a single, discrete truth value (i.e. correct or incorrect) is inappropriate, creating arbitrary targets for machine learning.

Cognitive computing systems require human labeled data for evaluation and often for training. The standard practice used in gathering this data minimizes disagreement between annotators, and we have found this results in data that fails to account for the ambiguity inherent in language. We have proposed the CrowdTruth method for collecting ground truth through crowdsourcing, which reconsiders the role of people in machine learning based on the observation that disagreement between annotators provides a useful signal for phenomena such as ambiguity in the text. We report on using this method to build an annotated data set for medical relation extraction for the cause and treat relations, and how this data performed in a supervised training experiment. We demonstrate that by modeling ambiguity, labeled data gathered from crowd workers can (1) reach the level of quality of domain experts for this task while reducing the cost, and (2) provide better training data at scale than distant supervision. We further propose and validate new weighted measures for precision, recall, and F-measure, which account for ambiguity in both human and machine performance on this task.

In this paper, we propose a model to operationalise serendipity in content-based recommender systems. The model, called SIRUP, is inspired by the Silvia's curiosity theory, based on the fundamental theory of Berlyne, aims at (1) measuring the novelty of an item with respect to the user profile, and (2) assessing whether the user is able to manage such level of novelty (coping potential). The novelty of items is calculated with cosine similarities between items, using Linked Open Data paths. The coping potential of users is estimated by measuring the diversity of the items in the user profile. We deployed and evaluated the SIRUP model in a use case with TV recommender using BBC programs dataset. Results show that the SIRUP model allows us to identify serendipitous recommendations, and, at the same time, to have 71% precision.

In collaborative Web-based platforms, user reputation scores are generally computed according to two orthogonal perspectives: (a) helpfulness-based reputation (HBR) scores and (b) centrality-based reputation (CBR) scores. In HBR approaches, the most reputable users are those who post the most helpful reviews according to the opinion of the members of their community. In CBR approaches, a “who-trusts-whom” network—known as a trust network —is available and the most reputable users occupy the most central position in the trust network, according to some definition of centrality. The identification of users featuring large HBR scores is one of the most important research issue in the field of Social Networks, and it is a critical success factor of many Web-based platforms like e-marketplaces, product review Web sites, and question-and-answering systems. Unfortunately, user reviews/ratings are often sparse, and this makes the calculation of HBR scores inaccurate. In contrast, CBR scores are relatively easy to calculate provided that the topology of the trust network is known. In this article, we investigate if CBR scores are effective to predict HBR ones, and, to perform our study, we used real-life datasets extracted from CIAO and Epinions (two product review Web sites) and Wikipedia and applied five popular centrality measures—Degree Centrality, Closeness Centrality, Betweenness Centrality, PageRank and Eigenvector Centrality—to calculate CBR scores. Our analysis provides a positive answer to our research question: CBR scores allow for predicting HBR ones and Eigenvector Centrality was found to be the most important predictor. Our findings prove that we can leverage trust relationships to spot those users producing the most helpful reviews for the whole community.

A viewpoint is a triple consisting of an entity, a topic related to this entity and sentiment towards this topic. In time-aware multi-viewpoint summarization one monitors viewpoints for a running topic and selects a small set of informative documents. In this paper, we focus on time-aware multi-viewpoint summarization of multilingual social text streams. Viewpoint drift, ambiguous entities and multilingual text make this a challenging task. Our approach includes three core ingredients: dynamic viewpoint modeling, cross-language viewpoint alignment, and, finally, multi-viewpoint summarization. Specifically, we propose a dynamic latent factor model to explicitly characterize a set of viewpoints through which entities, topics and sentiment labels during a time interval are derived jointly; we connect viewpoints in different languages by using an entity-based semantic similarity measure; and we employ an update viewpoint summarization strategy to generate a time-aware summary to reflect viewpoints. Experiments conducted on a real-world dataset demonstrate the effectiveness of our proposed method for time-aware multi-viewpoint summarization of multilingual social text streams.

Crowdsourcing has proved to be a feasible way of harnessing human computation for solving complex problems. However, crowdsourcing frequently faces various challenges: data handling, task reusability, and platform selection. Domain scientists rely on eScientists to find solutions for these challenges. CrowdTruth is a framework that builds on existing crowdsourcing platforms and provides an enhanced way to manage crowdsourcing tasks across platforms, offering solutions to commonly faced challenges. Provenance modeling proves means for documenting and examining scientific workflows. CrowdTruth keeps a provenance trace of the data flow through the framework, thus allowing to trace how data was transformed and by whom to reach its final state. In this way, eScientists have a tool to determine the impact that crowdsourcing has on enhancing their data.

Since 2007, the PATCH workshop series (https://patchworkshopseries.wordpress.com/) have been gathering successfully researchers and professionals from various countries and institutions to discuss the topics of digital access to cultural heritage and specifically the personalization aspects in this process. Due to this rich history, the reach of the PATCH workshop in various research communities is extensive.

Recent years witnessed an explosion in the number and variety of data crowdsourcing initiatives. From OpenStreetMap to Amazon Mechanical Turk, developers and practitioners have been striving to create user interfaces able to effectively and efficiently support the creation, exploration, and analysis of crowdsourced information.

In this work we present an in-depth analysis of the user behaviors on different Social Sharing systems. We consider three popular platforms, Flickr, Delicious and StumbleUpon, and, by combining techniques from social network analysis with techniques from semantic analysis, we characterize the tagging behavior as well as the tendency to create friendship relationships of the users of these platforms. The aim of our investigation is to see if (and how) the features and goals of a given Social Sharing system reflect on the behavior of its users and, moreover, if there exists a correlation between the social and tagging behavior of the users. We report our findings in terms of the characteristics of user profiles according to three different dimensions: (i) intensity of user activities, (ii) tag-based characteristics of user profiles, and (iii) semantic characteristics of user profiles.

With the rapidly increasing popularity of social sharing sites, the traditional manual indexing techniques are no longer feasible to cope with the growing amount of multimedia content. Emerging folksonomies of user tags through crowdsourcing provide a potential for the collaborative annotation of various types of online multimedia resources. However, the shortcomings of folksonomies still present researchers with challenges to effectively use the collected user tags in professional or public collections. Examples of such challenges are determining how to tackle the quality of tags, to understand tags' meaning and relevance to the resource material, and to define quality parameters of the final (targeted) annotations of multimedia resources. This work addresses such challenges in a concrete use case -- the crowdsourcing video annotation game called Waisda?. This game is used to collect user tags for videos from the Dutch National Audiovisual Archive 'Sound and Vision'. In this paper we explore the interactive aspects of a post-game crowdsourcing tool called 'Tag Gardening' for curating user tags. We tackle the challenges of bringing out quality and extracting meaning from the user tags in order to finally achieve satisfactory video annotations.

The primary goal of this workshop is to gather researchers and practitioners from different fields, e.g., multimedia retrieval, user interaction, arts and heritage curation, interface design and user modeling, in order to showcase novel applications and discuss opportunities that grow from the connections between users and multimedia systems in the cultural heritage domain.

This workshop focuses on the specific challenges for personalization in the cultural heritage setting from the point of view of user interaction and visitor experience. It investigates how the user interface - the contact point of visitors and systems - can become more intelligent by means of personalization. Overall, the workshop aims at attracting presentations of novel ideas for addressing these challenges and the current state of the art in this field.

Recently, various crowdsourcing initiatives showed that targeted efforts of \nuser communities result in massive amounts of tags. For example, the \nNetherlands Institute for Sound and Vision collected a large number of tags \nwith the video labeling game \\emph{Waisda?}. To successfully utilize these \ntags, a better understanding of their characteristics is required. \nThe goal of this paper is twofold: (i) to investigate the vocabulary that \nusers employ when describing videos and compare it to the vocabularies used by \nprofessionals; and (ii) to establish which aspects of the video are typically \ndescribed and what type of tags are used for this. We report on an analysis of \nthe tags collected with \\emph{Waisda?}. With respect to the first goal, we \ncompared the the tags with a typical domain thesaurus used by professionals, \nas well as with a more general vocabulary. With respect to the second goal, we \ncompare the tags to the video subtitles to determine how many tags are derived \nfrom the audio signal. In addition, we perform a qualitative study in which a \ntag sample is interpreted in terms of an existing annotation classification \nframework. The results suggest that the tags complement the metadata provided \nby professional cataloguers, the tags describe both the audio and the visual \naspects of the video, and the users primarily describe objects in the video \nusing general descriptions.

We present a Linked Data analysis method which relies on knowledge patterns for constructing a logical architecture of the knowledge in a dataset. This can then be exploited to compare heterogeneous datasets, enhance interoperability between them and make implicit knowledge emerge.

Within cultural heritage collections, objects are often grounded in a particular historical setting. This setting can currently not be made explicit, as structured descriptions of events are either missing or not marked up explicitly. This paper reports a study on automatic extraction of an historical event thesaurus from unstructured texts. We show how this preliminary thesaurus accommodates event- and object-driven search and browsing of two cultural heritage collections.

Recent innovations in the Social and Semantic Web fields have resulted in large amounts of data created, published and consumed by users of the Web. This vast amount of data exists in a variety of formats, from the traditional ones such as text, image, video to the more recent additions such as streams of status information from Twitter and Facebook. The ability to easily integrate such vast amounts of data raises significant and exciting research challenges, not least of which how to provide effective access to and navigation across heterogeneous data sources on different platforms (e.g. computers, mobile devices, set-top boxes). Building on the success of the VISSW2009 workshop, the IUI2010 workshop on Visual Interfaces to the Social and Semantic Web aims to bring together researchers and practitioners from different fields to discuss the latest research results and challenges in designing, implementing, and evaluating intelligent interfaces supporting access, navigation and publishing of different types of contents on the Social and Semantic Web. This paper outlines the context of the workshop and provides an overview of the research to be presented at the event.

No abstract available.

In this paper we present our iFanzy demonstrator, which aims at offering users television content in a personalized and context-sensitive way. With iFanzy we integrate different volatile content sources like video-on-demand, broadcast and Web information on-the-fly to disperse it on different clients on different platforms like set-top boxes and Web interfaces. Via semantic Web techniques, TV content and background data from various heterogeneous sources is integrated into a transparent RDF/OWL knowledge graph, which allows the user to navigate and browse the vast content sets nowadays available. Via illustrative examples we explain how the main integrated data set is built and how we exploit it to provide iFanzypsilas main functionality, like semantic search of the available content and execution of context sensitive recommendations.

The increasing availability of (digital) cultural heritage artefacts offers great potential for increased access to art content, but also necessitates tools to help users deal with such abundance of information. User-adaptive art recommender systems aim to present their users with art content tailored to their interests. These systems try to adapt to the user based on feedback from the user on which artworks he or she finds interesting. Users need to be able to depend on the system to competently adapt to their feedback and find the artworks that are most interesting to them. This paper investigates the influence of transparency on user trust in and acceptance of content-based recommender systems. A between-subject experiment ( N = 60) evaluated interaction with three versions of a content-based art recommender in the cultural heritage domain. This recommender system provides users with artworks that are of interest to them, based on their ratings of other artworks. Version 1 was not transparent, version 2 explained to the user why a recommendation had been made and version 3 showed a rating of how certain the system was that a recommendation would be of interest to the user. Results show that explaining to the user why a recommendation was made increased acceptance of the recommendations. Trust in the system itself was not improved by transparency. Showing how certain the system was of a recommendation did not influence trust and acceptance. A number of guidelines for design of recommender systems in the cultural heritage domain have been derived from the study’s results.

The move.me prototype illustrates a scenario for social interaction in which users can manipulate audio-visual sources presented on various screens through an interaction with a sensor-enhanced pillow. The technology developed for move.me uses the surface of a pillow as a tactile interface. We describe the underlying concepts of move.me and its motivations. We present a case study of the environment as the context of evaluating aspects of our approach and conclude with plans for future work.

This paper describes the move.me interaction prototype developed in conjunction with V2_lab in Rotterdam. move.me proposes a scenario for social interaction and the notion of social intimacy. Interaction with sensory--enhanced, soft, pliable, tactile, throw-able cushions afford new approaches to pleasure, movement and play. A somatics approach to touch and kinaesthesia provides an underlying design framework. The technology developed for move.me uses the surface of the cushion as an intelligent tactile interface. Making use of a movement analysis system called Laban Effort-Shape, we have developed a model that provides a high-level interpretation of varying qualities of touch and motion trajectory. We describe the notion of social intimacy, and how we model it through techniques in somatics and performance practice. We describe the underlying concepts of move.me and its motivations. We illustrate the structural layers of interaction and related technical detail. Finally, we discuss the related body of work in the context of evaluating our approach and conclude with plans for future work.

Currently, there is an increasing effort to provide various personalized services on museum web sites. This paper presents an approach for determining user interests in a museum collection with the help of an interactive dialog. It uses a semantically annotated collection of the Rijksmuseum Amsterdam to elicit specific user's interests in artists, periods, genres and themes and uses these values to recommend relevant artefacts and related concepts from the museum collection. In the presented prototype, we show how constructing a user profile and applying recommender strategies in this way enable dynamical generation personalized museum tours for different users.

In order to efficiently manage information on the Web it becomes crucial to provide personalization not only within single systems, but across various Web applications. Interoperability of user models, on both syntactic and semantic level, emerges as a vital issue in order to achieve seamless personalization across different user-adaptive systems. In this paper we discuss the architecture of a broker-based discovery service for user models (BD-SUM), which allows applications to discover and invoke semantically described user models, in a multi-application context.

The goal of the CHIP (Cultural Heritage Information Personalization) project is to provide personalized access to combined cultural heritage content. The driving case is given by the Rijksmuseum content presented on the museum web site and visitor guides. The CHIP project aims to extend and integrate existing technologies for semantic browsing and search (e.g. Topia1[1], Noadster[2], OntoAIMS), ontology-based user modeling (e.g. OWL-OLM2[3]), adaptation strategies (e.g. AHA! 3[4]) and presentation of structured information (e.g. Hera4[5]) in order to achieve its goals. We use as a starting point the Topia demonstrator, which provides search and browsing interface to 1250 semantically described Rijksmuseum artifacts. We extend further this research by involving semantic-aware adaptation strategies and semantic based reasoning user model elicitation to allow for effective personalization of the content navigation and presentation on various devices (e. g. web application, PDA, etc.). In this we integrate artifacts and background information that span over several collections and sources. Adaptive clustering mechanisms show the relations between art objects and links to background information, based on their metadata. Further, hypermedia formats support multi-branched story lines connecting art objects in “ precooked” or automatically generated ways.

Authoring of concept-based Web courseware is more complicated than the 'standard' courseware authoring and thus needs more adaptive and intelligent support. In order to provide such support the authoring system itself needs to have understanding about the process of courseware authoring, to be able to decompose it in steps and procedures and reason over it; to provide hints and recommendations to the author, and perform various (semi-) automatic actions. Our goal is to develop a knowledge-based system for concept-based courseware authoring, where the knowledge representation is realized with the help of Courseware Authoring Tasks Ontology (CATO).

The goal of this paper is to explore issues concerned with the design of concept/ontology-based WBES and to outline the state-of-the-art in the development of ontology-aware systems that facilitate the reuse, sharing, and exchange of knowledge and components with respect to both their creating and authoring. We hope that it will contribute to defining the path towards a new generation authoring strategies, which will meet the complex requirements of concept-based WBES authoring.