ABSTRACT Most graph summarization algorithms are tailored to a specific graph summary model and were designed for one‐time computations only, that is, batch‐based computations. We developed a universal approach for parallel graph summarization and three algorithms to compute graph summaries—a batch‐based algorithm for static graphs, an incremental algorithm for evolving graphs, and a hash‐based algorithm that scales to large graphs and large schema structures, that is, using paths of length up to to define vertex equivalence. Experimenting with benchmark and real‐world datasets, we observe that the incremental algorithm almost always runs faster than batch computation, even when 50% of the graph changes, and even when using fewer cores; however, it only uses 8% more memory (). Furthermore, we show that the hash‐based algorithm can compute 10‐hop equivalent subgraphs on graphs with over 10 M edges within seconds, on graphs of 100 + M edges within a few minutes, and on graphs of 1 + B edges in less than an hour. We analyse the complexity of our algorithms in detail and prove that the incremental algorithm is correct. Overall, we show with these three algorithms that our parallel approach for graph summarisation is versatile and opens the path for various applications that require summaries of large‐scale graphs.

Graph homophily has been considered an essential property for message-passing neural networks (MPNN) in node classification. Recent findings suggest that performance is more closely tied to the consistency of neighborhood class distributions. We demonstrate that the MPNN performance depends on the number of components of the overall neighborhood distribution within a class. By breaking down the classes into their neighborhood distribution components, we increase measures of neighborhood distribution informativeness but do not observe an improvement in MPNN performance. We propose a Gumbel-Softmax-based rewiring method that reduces deviations in neighborhood distributions. Our results show that our new method enhances neighborhood informativeness, handles long-range dependencies, mitigates oversquashing, and increases the classification performance of the MPNN. The code is available at https://github.com/Bobowner/Gumbel-Softmax-MPNN and the supplementary material is available at http://arxiv.org/abs/2508.17531.

We introduce HYDRA, a simple yet effective multi-head encoder-only architecture for hierarchical text classification that treats each level in the hierarchy as a separate classification task with its own label space.State-of-the-art approaches rely on complex components like graph encoders, label semantics, and autoregressive decoders.We demonstrate that such complexity is often unnecessary.Through parameter sharing and level-specific parameterization, HYDRA enables flat models to incorporate hierarchical awareness without architectural complexity.Experiments on four benchmarks (NYT, RCV1-V2, BGC, and WOS) demonstrate that HYDRA always increases the performance over flat models and matches or exceeds the performance of complex state-ofthe-art methods.

Low isotropy in an embedding space impairs performance on tasks involving semantic inference.Our study investigates the impact of isotropy on semantic code search performance and explores post-processing techniques to mitigate this issue.We analyze various code language models, examine isotropy in their embedding spaces, and its influence on search effectiveness.We propose a modified ZCA whitening technique to control isotropy levels in embeddings.Our results demonstrate that Soft-ZCA whitening improves the performance of pre-trained code language models and can complement contrastive fine-tuning.

Shallow node embeddings like node2vec (N2V) can be used for nodes without features or to supplement existing features with structure-based information. Embedding methods like N2V are limited in their application on new nodes, which restricts them to the transductive setting where the entire graph, including the test nodes, is available during training. We propose inductive node2vec (iN2V), which combines a post-hoc procedure to compute embeddings for nodes unseen during training and modifications to the original N2V training procedure to prepare the embeddings for this post-hoc procedure. We conduct experiments on several benchmark datasets and demonstrate that iN2V is an effective approach to bringing transductive embeddings to an inductive setting. Using iN2V embeddings improves node classification by 1 point on average, with up to 6 points of improvement depending on the dataset and the number of unseen nodes. Our iN2V is a plug-in approach to create new or enrich existing embeddings. It can also be combined with other embedding methods, making it a versatile approach for inductive node representation learning. Code to reproduce the results is available at https://github.com/Foisunt/iN2V.

Despite the ample availability of graph data, obtaining vertex labels is a tedious and expensive task. Therefore, it is desirable to learn from a few labeled vertices only. Existing few-shot learners assume a class oracle, which provides labeled vertices for a desired class. However, such an oracle is not available in a real-world setting, i. e., when drawing a vertex for labeling it is unknown to which class the vertex belongs. Few-Shot learners are often combined with prototypical networks, while classical semi-supervised vertex classification uses discriminative models, e. g., Graph Convolutional Networks (GCNs). In this paper, we train our models by iteratively prompting a human annotator with vertices to annotate. We perform three experiments where we continually relax our assumptions. First, we assume a class oracle, i. e., the human annotator is provided with an equal number of vertices to label for each class. We denote this as "Balanced Sampling". In the subsequent experiment, "Unbalanced Sampling," we replace the class oracle with k-medoids clustering and draw vertices to label from the clusters. In the last experiment, the "Unknown Number of Classes," we no longer assumed we knew the number and distribution of classes. Our results show that prototypical models outperform discriminative models in all experiments when fewer than 20 samples per class are available. While dropping the assumption of the class oracle for the "Unbalanced Sampling" experiment reduces the performance of the GCN by 9%, the prototypical network loses only 1% on average. For the "Unknown Number of Classes" experiment, the average performance for both models decreased further by 1%.Source code: https://github.com/Ximsa/2023-felix-ma

Recent approaches in hierarchical text classification (HTC) rely on the capabilities of a pre-trained transformer model and exploit the label semantics and a graph encoder for the label hierarchy. In this paper, we introduce an effective hierarchical text classifier RADAr (A Transformer-based Autoregressive Decoder Architecture) that is based only on an off-the-shelf RoBERTa transformer to process the input and a custom autoregressive decoder with two decoder layers for generating the classification output. Thus, unlike existing approaches for HTC, the encoder of RADAr has no explicit encoding of the label hierarchy and the decoder solely relies on the samples’ label sequences observed during training. We demonstrate on three benchmark datasets that RADAr achieves results competitive to the state of the art with less training and inference time. Our model consistently performs better when organizing the label sequences from children to parents versus the inverse, as done in existing HTC approaches. Our experiments show that neither the label semantics nor an explicit graph encoder for the hierarchy is needed. This has strong practical implications for HTC as the architecture has fewer requirements and provides a speed-up by a factor of 2 at inference time. Moreover, training a separate decoder from scratch in conjunction with fine-tuning the encoder allows future researchers and practitioners to exchange the encoder part as new models arise. The source code is available at https://github.com/yousef-younes/RADAr.

HyperAggregation is a hypernetwork-based aggregation function for Graph Neural Networks. It uses a hyper-network to dynamically generate weights in the size of the current neighborhood, which are then used to aggregate this neighborhood. This aggregation with the generated weights is done like an MLP-Mixer channel mixing over variable-sized vertex neighborhoods. We demonstrate HyperAggregation in two models, GraphHyperMixer is a model based on MLP-Mixer while GraphHyperConv is derived from a GCN but with a hypernetwork-based aggregation function. We perform experiments on diverse benchmark datasets for the vertex classification, graph classification, and graph regression tasks. The results show that HyperAggregation can be effectively used for homophilic and heterophilic datasets in both inductive and transductive settings. GraphHyperConv performs better than GraphHyperMixer and is especially strong in the transductive setting. On the heterophilic dataset Roman-Empire it reaches a new state of the art. On the graph-level tasks our models perform in line with similarly sized models. Ablation studies investigate the robustness against various hyperparameter choices. The implementation of HyperAggregation as well code to reproduce all experiments is available under https://github.com/Foisunt/HyperAggregation.

We study the problem of lifelong graph learning in an open-world scenario, where a model needs to deal with new tasks and potentially unknown classes. We utilize Out-of-Distribution (OOD) detection methods to recognize new classes and adapt existing non-graph OOD detection methods to graph data. Crucially, we suggest performing new class detection by combining OOD detection methods with information aggregated from the graph neighborhood. Most OOD detection methods avoid determining a crisp threshold for deciding whether a vertex is OOD. To tackle this problem, we propose a Weakly-supervised Relevance Feedback (Open-WRF) method, which decreases the sensitivity to thresholds in OOD detection. We evaluate our approach on six benchmark datasets. Our results show that the proposed neighborhood aggregation method for OOD scores outperforms existing methods independent of the underlying graph neural network. Furthermore, we demonstrate that our Open-WRF method is more robust to threshold selection and analyze the influence of graph neighborhood on OOD detection. The aggregation and threshold methods are compatible with arbitrary graph neural networks and OOD detection methods, making our approach versatile and applicable to many real-world applications. The source code is available at https://github.com/Bobowner/Open-World-LGL.

Graph neural networks have triggered a resurgence of graph-based text classification methods, defining today's state of the art.We show that a wide multi-layer perceptron (MLP) using a Bag-of-Words (BoW) outperforms the recent graph-based models TextGCN and Hete-GCN in an inductive text classification setting and is comparable with HyperGAT.Moreover, we fine-tune a sequence-based BERT and a lightweight DistilBERT model, which both outperform all state-of-the-art models.These results question the importance of synthetic graphs used in modern text classifiers.In terms of efficiency, DistilBERT is still twice as large as our BoW-based wide MLP, while graph-based models like TextGCN require setting up an O(N 2 ) graph, where N is the vocabulary plus corpus size.Finally, since Transformers need to compute O(L 2 ) attention weights with sequence length L, the MLP models show higher training and inference speeds on datasets with long sequences. Model Synthetic Graph Position-Aware Arbitrary Length InductiveBag-of-Words No No Yes Yes Graph: TextGCN Yes No Yes No Graph:

The goal of graph summarization is to represent large graphs in a structured and compact way. A graph summary based on equivalence classes preserves predefined features of each vertex within a k-hop neighborhood, such as the vertex and edge labels. Based on these neighborhood characteristics, the vertex is assigned to an equivalence class. The calculation of the assigned equivalence class must be a permutation invariant operation on the predefined features. This is typically achieved by sorting on the feature values, which is computationally expensive, and subsequently hashing the result. Graph Neural Networks (GNNs) fulfill the permutation invariance requirement. We formulate the problem of graph summarization as a subgraph classification task on the root vertex of the k-hop neighborhood. We adapt different GNN architectures, both based on the popular message-passing protocol and alternative approaches, to perform the structural graph summarization task. We compare different GNNs with a standard multi-layer perceptron (MLP) and Bloom filter as a non-neural method. We consider four popular graph summary models on a large web graph. This resembles challenging multi-class vertex classification tasks with the numbers of classes ranging from 576 to hundreds of thousands. Our results show that the performance of GNNs are close to each other. In three out of four experiments, the non-message-passing Graph-MLP model outperforms the other GNNs. The performance of the standard MLP is extraordinarily good, especially in the presence of many classes. Finally, the Bloom filter outperforms all neural architectures by a large margin, except for the dataset with the fewest number (576) of classes. This is an interesting result, since it sheds light on how well and in which contexts GNNs are suited for graph summarization. Furthermore, it demonstrates the need for considering strong non-neural baselines for standard GNN tasks such as vertex classification.

Large pretrained language models (PreLMs) are rev-olutionizing natural language processing across all benchmarks. However, their sheer size is prohibitive for small laboratories or for deployment on mobile devices. Approaches like pruning and distillation reduce the model size but typically retain the same model architecture. In contrast, we explore distilling PreLMs into a different, more efficient architecture, Continual Multiplication of Words (CMOW), which embeds each word as a matrix and uses matrix multiplication to encode sequences. We extend the CMOW architecture and its CMOW/CBOW-Hybrid variant with a bidirectional component for more expressive power, per-token representations for a general (task-agnostic) distillation during pretraining, and a two-sequence encoding scheme that facilitates downstream tasks on sentence pairs, such as sentence similarity and natural language inference. Our matrix-based bidirectional CMOW/CBOW-Hybrid model is competitive to DistilBERT on question similarity and recognizing textual entailment, but uses only half of the number of parameters and is three times faster in terms of inference speed. We match or exceed the scores of ELMo for all tasks of the GLUE benchmark except for the sentiment analysis task SST-2 and the linguistic acceptability task CoLA. However, compared to previous cross-architecture distillation approaches, we demonstrate a doubling of the scores on detecting linguistic acceptability. This shows that matrix-based embeddings can be used to distill large PreLM into competitive models and motivates further research in this direction.

Abstract We address the problem of recommending relevant items to a user in order to “complete” a partial set of already-known items. We consider the two scenarios of citation and subject label recommendation, which resemble different semantics of item co-occurrence: relatedness for co-citations and diversity for subject labels. We assess the influence of the completeness of an already known partial item set on the recommender’s performance. We also investigate data sparsity by imposing a pruning threshold on minimum item occurrence and the influence of using additional metadata. As models, we focus on different autoencoders, which are particularly suited for reconstructing missing items in a set. We extend autoencoders to exploit a multi-modal input of text and structured data. Our experiments on six real-world datasets show that supplying the partial item set as input is usually helpful when item co-occurrence resembles relatedness, while metadata are effective when co-occurrence implies diversity. The simple item co-occurrence model is a strong baseline for citation recommendation but can provide good results also for subject labels. Autoencoders have the capability to exploit additional metadata besides the partial item set as input, and achieve comparable or better performance. For the subject label recommendation task, the title is the most important attribute. Adding more input modalities sometimes even harms the results. In conclusion, it is crucial to consider the semantics of the item co-occurrence for the choice of an appropriate model and carefully decide which metadata to exploit.

Traditional approaches for data anonymization consider relational data and textual data independently. We propose rx-anon, an anonymization approach for heterogeneous semi-structured documents composed of relational and textual attributes. We map sensitive terms extracted from the text to the structured data. This allows us to use concepts like k-anonymity to generate a joint, privacy-preserved version of the heterogeneous data input. We introduce the concept of redundant sensitive information to consistently anonymize the heterogeneous data. To control the influence of anonymization over unstructured textual data versus structured data attributes, we introduce a modified, parameterized Mondrian algorithm. We evaluate our approach with two real-world datasets using a Normalized Certainty Penalty score, adapted to the problem of jointly anonymizing relational and textual data. The results show that our approach is capable of reducing information loss by using the tuning parameter to control the Mondrian partitioning while guaranteeing k-anonymity. As rx-anon is a framework approach, it can be reused and extended by other anonymization algorithms, privacy models, and textual similarity metrics.

A common writing style for statistical results are the recommendations of the American Psychology Association (APA). In practice, writing styles vary as reports are not 100% following APA-style or parameters are not reported despite being mandatory. In addition, the statistics are not reported in isolation but in context of experiment conditions investigated and the general experiment topic. We address these challenges by proposing a flexible pipeline STEREO based on wrapper induction and unsupervised aspect detection to extract experiment statistics, conditions, and topics. Thus, in contrast to existing rule-based tools like statcheck with a pre-defined set of rules, we learn rules via induction. It required only 0.25% of the CORD-19 corpus (about 500 documents) to learn statistics extraction rules that cover 95% of the sentences in CORD-19. The statistic extraction has 100% precision on APA-conform statistics, which is identical with statcheck. In addition, STEREO can extract non-APA writing styles with precision, which statcheck does not support. Extracting non-APA conform statistics is important as they make more than 99% of all 113k extracted statistics. We could extract in 46% the correct conditions from APA-conform reports (30% for non-APA). The best model for topic extraction achieves a precision of 75% on statistics reported in APA style (73% for non-APA conform).

Modern multi-document summarization (MDS) methods are based on transformer architectures. They generate state of the art summaries, but lack explainability. To overcome this, we analyze the attention weights of a graph-based MDS such as GraphSum. We compare GraphSum’s performance utilizing different textual units, i. e., sentences versus paragraphs, on two MDS benchmark datasets, namely WikiSum and MultiNews. Our experiments show that paragraph-level representations provide the best summarization performance. Subsequently, we focus on analyzing the paragraph-level attention weights of GraphSum’s multi-heads and decoding layers. Furthermore, we examine source origin information via text similarity over a ROUGE-based reference metric. We observe a high correlation between the attention weights and our reference metric, especially on the later decoding layers. Finally, we investigate if the generated summaries follow a pattern of positional bias. Our results show that there is a high correlation between the position of paragraphs in the input documents and the content provided for the generated summary.

Graph neural networks (GNNs) have emerged as the standard method for numerous tasks on graph-structured data such as node classification. However, real-world graphs are often evolving over time and even new classes may arise. We model these challenges as an instance of lifelong learning, in which a learner faces a sequence of tasks and may take over knowledge acquired in past tasks. Such knowledge may be stored explicitly as historic data or implicitly within model parameters. In this work, we systematically analyze the influence of implicit and explicit knowledge. Therefore, we present an incremental training method for lifelong learning on graphs and introduce a new measure based on k-neighborhood time differences to address variances in the historic data. We apply our training method to five representative GNN architectures and evaluate them on three new lifelong node classification datasets. Our results show that no more than 50% of the GNN's receptive field is necessary to retain at least 95% accuracy compared to training over the complete history of the graph data. Furthermore, our experiments confirm that implicit knowledge becomes more important when fewer explicit knowledge is available.

Deep-learning models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) have been successfully used for process-mining tasks. They have achieved better performance for different predictive tasks than traditional approaches. We extend the existing body of research by testing four different variants of Graph Neural Networks (GNN) and a fully connected Multi-layer Perceptron (MLP) with dropout for the tasks of predicting the nature and timestamp of the next process activity. In contrast to existing studies, we evaluate our models' performance at different stages of a process, determined by quartiles of the number of events and normalized quarters of the case duration. This provides new insights into the performance of a prediction model, as they behave differently at different stages of a business-process. Interestingly, our experiments show that the simple MLP often outperforms more sophisticated deep-learning models in both prediction tasks. We argue that care needs to be taken when applying automated process-prediction techniques at different stages of a process. We further argue that researchers should reflect their results with strong baselines methods like MLPs.

Graph summarization is the task of finding condensed representations of graphs such that a chosen set of (structural) subgraph features in the graph summary are equivalent to the input graph. Existing graph summarization algorithms are tailored to specific graph summary models, only support one-time batch computation, are designed and implemented for a specific task, or evaluated using static graphs. Our novel, incremental, parallel algorithm addresses all these shortcomings. We support various structural graph summary models defined in our formal language FLUID. All graph summaries defined with FLUID can be updated in time O(Δ · dk), where Δ is the number of additions, deletions, and modifications to the input graph, d is its maximum degree, and k is the maximum distance in the subgraphs considered. We empirically evaluate the performance of our algorithm on benchmark and real-world datasets. Our experiments show that, for commonly used summary models and datasets, the incremental summarization algorithm almost always outperforms their batch counterpart, even when about $50%$ of the graph database changes. The source code and the experimental results are openly available for reproducibility and extensibility.

Text extraction from scientific figures has been addressed in the past by different unsupervised approaches due to the limited amount of training data. Motivated by the recent advances in Deep Learning, we propose a two-step neural-network-based pipeline to localize and extract text using Fully Convolutional Networks. We improve the localization of the text bounding boxes by applying a novel combination of a Residual Network with the Region Proposal Network based on Faster R-CNN. The predicted bounding boxes are further pre-processed and used as input to the of-the-shelf optical character recognition engine Tesseract 4.0. We evaluate our improved text localization method on five different datasets of scientific figures and compare it with the best unsupervised pipeline. Since only limited training data is available, we further experiment with different data augmentation techniques for increasing the size of the training datasets and demonstrate their positive impact. We use Average Precision and F1 measure to assess the text localization results. In addition, we apply Gestalt Pattern Matching and Levenshtein Distance for evaluating the quality of the recognized text. Our extensive experiments show that our new pipeline based on neural networks outperforms the best unsupervised approach by a large margin of 19-20%.

We present multi-modal adversarial autoencoders for recommendation and evaluate them on two different tasks: citation recommendation and subject label recommendation. We analyze the effects of adversarial regularization, sparsity, and different input modalities. By conducting 408 experiments, we show that adversarial regularization consistently improves the performance of autoencoders for recommendation. We demonstrate, however, that the two tasks differ in the semantics of item co-occurrence in the sense that item co-occurrence resembles relatedness in case of citations, yet implies diversity in case of subject labels. Our results reveal that supplying the partial item set as input is only helpful, when item co-occurrence resembles relatedness. When facing a new recommendation task it is therefore crucial to consider the semantics of item co-occurrence for the choice of an appropriate model.

Bar charts are widely used to visualize core results of experiments in research papers or display statistics in news, media, and other reports. However, visualizations like bar charts are mostly manually designed, static presentations of data without the option of adaption to a user's needs. But so far, it is unknown whether interactivity improves the understanding of charts. In this work, we compare static with dynamic bar charts, which offer an interactive stacking option. We assess the efficiency, effectiveness, and satisfaction when answering questions regarding the content of a bar chart. An eye-tracker is used to measure the efficiency. We have conducted a between group experiment with 38 participants. While one group had to solve the aggregation tasks using stackable, i.e., interactive bar charts, the other group was limited to static visualizations. Even though new interactive features require familiarization, we found that the stacking feature significantly helps completing the tasks with respect to efficiency, effectiveness, and satisfaction for bar charts of varying complexity.

We conduct the first systematic comparison of automated semantic annotation based on either the full-text or only on the title metadata of documents. Apart from the prominent text classification baselines kNN and SVM, we also compare recent techniques of Learning to Rank and neural networks and revisit the traditional methods logistic regression, Rocchio, and Naive Bayes. Across three of our four datasets, the performance of the classifications using only titles reaches over 90% of the quality compared to the performance when using the full-text.

Nowadays, most recommender systems exploit user-provided ratings to infer their preferences. However, the growing popularity of social and e-commerce websites has encouraged users to also share comments and opinions through textual reviews. In this paper, we introduce a new recommendation approach which exploits the semantic annotation of user reviews to extract useful and non-trivial information about the items to recommend. It also relies on the knowledge freely available in the Web of Data, notably in DBpedia and Wikidata, to discover other resources connected with the annotated entities. We evaluated our approach in three domains, using both DBpedia and Wikidata. The results showed that our solution provides a better ranking than another recommendation method based on the Web of Data, while it improves in novelty with respect to traditional techniques based on ratings.

Educational and Knowledge Technologies (EdTech), especially in connection to multimedia content and the vision of mobile and personalized learning, is a hot topic in both academia and the business start-ups ecosystem. The driver and enabler of this is on the one side the development and widespread availability of multimedia materials and MOOCs, which represent multimedia content produced specifically for supporting e-learning; and, on the other side, the ever increasing availability of all sorts on information on the Internet and in social media channels (e. g. lectures, research papers, user-generated videos, news items), which, despite not directly targeting e-learning, can prove to be valuable complements to the more targeted learning materials. Although the availability of such content is not a problem these days, finding the right content and associating different relevant pieces of multimedia so as to enable a comprehensive learning experience on a chosen subject is by no means a trivial task. This workshop provides research in areas related to multimedia-based educational and knowledge technologies and particularly on the use of multimedia search and retrieval, analysis and understanding, browsing, summarization, recommendation, and visualization technologies on multimedia content available in specialized learning platforms, the Web, mobile devices and/or social networks for supporting personalized and adaptive e-learning and training.

Many Linked Open Data applications require fresh copies of RDF data at their local repositories. Since RDF documents constantly change and those changes are not automatically propagated to the LOD applications, it is important to regularly visit the RDF documents to refresh the local copies and keep them up-to-date. For this purpose, crawling strategies determine which RDF documents should be preferentially fetched. Traditional crawling strategies rely only on how an RDF document has been modified in the past. In contrast, we predict on the triple level whether a change will occur in the future. We use the weekly snapshots of the DyLDO dataset as well as the monthly snapshots of the Wikidata dataset. First, we conduct an in-depth analysis of the life span of triples in RDF documents. Through the analysis, we identify which triples are stable and which are ephemeral. We introduce different features based on the triples and apply a simple but effective linear regression model. Second, we propose a novel crawling strategy based on the linear regression model. We conduct two experimental setups where we vary the amount of available bandwidth as well as iteratively observe the quality of the local copies over time. The results demonstrate that the novel crawling strategy outperforms the state of the art in both setups.

Existing research on analyzing information graphics assume to have a perfect text detection and extraction available. However, text extraction from information graphics is far from solved. To fill this gap, we propose a novel processing pipeline for multi-oriented text extraction from infographics. The pipeline applies a combination of data mining and computer vision techniques to identify text elements, cluster them into text lines, compute their orientation, and uses a state-of-the-art open source OCR engine to perform the text recognition. We evaluate our method on 121 infographics extracted from an open access corpus of scientific publications. The results show that our approach is effective and significantly outperforms a state-of-the-art baseline.

We introduce a framework for automated semantic document annotation that is composed of four processes, namely concept extraction, concept activation, annotation selection, and evaluation. The framework is used to implement and compare different annotation strategies motivated by the literature. For concept extraction, we apply entity detection with semantic hierarchical knowledge bases, Tri-gram, RAKE, and LDA. For concept activation, we compare a set of statistical, hierarchy-based, and graph-based methods. For selecting annotations, we compare top-k as well as kNN. In total, we define 43 different strategies including novel combinations like using graph-based activation with kNN. We have evaluated the strategies using three different datasets of varying size from three scientific disciplines (economics, politics, and computer science) that contain 100, 000 manually labeled documents in total. We obtain the best results on all three datasets by our novel combination of entity detection with graph-based activation (e.g., HITS and Degree) and kNN. For the economic and political science datasets, the best F-measure is .39 and .28, respectively. For the computer science dataset, the maximum F-measure of .33 can be reached. The experiments are the by far largest on scholarly content annotation, which typically are up to a few hundred documents per dataset only.

We present initial results of finding temporal patterns of entity dynamics on the Linked Open Data (LOD) cloud. For the analysis, we use the dataset of the three-year observation of the Dynamic Linked Data Observatory. Using k-means++ clustering with Euclidean distance, we reveal the temporal patterns of entity dynamics. In addition, we conduct the first investigation of periodicity in entity dynamics on the LOD cloud. While a large portion of entities are static, a certain number of entities have a temporal pattern with substantial changes. We observe different periodicity with respect to temporal patterns of entity dynamics. Knowing about the temporal patterns and their periodicity is important for applications that are depending on fresh data caches and indices of the distributed LOD cloud. They can concentrate in crawling and refreshing those parts of the LOD cloud, which are a) known to have changes in the past and b) currently have their highest periodical change rate.

We present State Hopper, an interactive multimedia application that uses Linked Open Data to generate a map-based educational game. A prototype of the system using Europe as map was implemented in a student project. Goal of the game is to move from country to country by learning facts about the countries like population, GDP, size, and others. Each movement from one country to another country on the map generates new insights and allows the player to capture new knowledge about the countries.

HuEvent'15 is a continuation of previous year's successful workshop on events in multimedia. It focuses on the human-centered aspects of understanding events from multimedia content. This includes the notion of objects and their relation to events. The workshop brings together researchers from the different areas in multimedia and beyond that are interested in understanding the concept of events.

We present an approach and mobile application for the interactive exploration and search of geo-located social media entities from different, distributed data providers on the web. When querying the providers, the returned results typically have some overlap. In addition, one has no guarantee that the providers reply within a given time interval. Thus, in order to provide users with geo-located entities in their vicinity in a timely manner, we need to take the asynchronous nature of the data providers' replies into account. Our novel on-the-fly entity resolution engine starts the entity resolution once it retrieves the first responses. It incrementally extends the entity resolution model when more responses arrive. Entities are propagated to the client once the resolution engine has processed them for the first time. Resolution results produced at a later point in time are sent as updates to the client and improve earlier, incomplete results. Our experiments show a matching precision of 95% and scalability of the on-the-fly entity resolution w.r.t. the number of resources being simultaneously processed.

Manually selecting subsets of photos from large collections in order to present them to friends or colleagues or to print them as photo books can be a tedious task. Today, fully automatic approaches are at hand for supporting users. They make use of pixel information extracted from the images, analyze contextual information such as capture time and focal aperture, or use both to determine a proper subset of photos. However, these approaches miss the most important factor in the photo selection process: the user. The goal of our approach is to consider individual interests. By recording and analyzing gaze information from the user's viewing photo collections, we obtain information on user's interests and use this information in the creation of personal photo selections. In a controlled experiment with 33 participants, we show that the selections can be significantly improved over a baseline approach by up to 22% when taking individual viewing behavior into account. We also obtained significantly better results for photos taken at an event participants were involved in compared with photos from another event.

Labeled image regions provide very valuable information that can be used in different settings such as image search. The manual creation of region labels is a tedious task. Fully automatic approaches lack understanding the image content sufficiently due to the huge variety of depicted objects. Our approach benefits from the expected spread of eye tracking hardware and uses gaze information obtained from users performing image search tasks to automatically label image regions. This allows to exploit the human capabilities regarding the visual perception of image content while performing daily routine tasks. In an experiment with 23 participants, we show that it is possible to assign search terms to photo regions by means of gaze analysis with an average precision of 0.56 and an average F-measure of 0.38 over 361 photos. The participants performed different search tasks while their gaze was recorded. The results of the experiment show that the gaze-based approach performs significantly better than a baseline approach based on saliency maps.

This workshop focuses on the human-centered aspects of understanding events from multimedia content. This includes the notion of objects and their relation to events. The workshop brings together researchers from the different areas in multimedia and beyond that are interested in understanding the concept of events.

Commercial apps for nearby search on mobile phones such as Qype, AroundMe, Foursquare, or Wikitude have gained huge popularity among smartphone users. Understanding the way how people use and interact with such applications is fundamental for improving the functionality and the user interface design. In our two-step field study, we developed and evaluated mobEx, a mobile app for faceted exploration of social media data on Android phones. mobEx unifies the data sources of related commercial applications in the market by retrieving information from various providers. The goal of our study was to find out, if the subjects understood the metaphor of a time-wheel as novel user interface feature for finding and exploring places and events and how they use it. In addition, mobEx offers a grid-based navigation menu and a list-based navigation menu for exploring the data. Here, we were interested in gaining some qualitative insights about which type of navigation approach the users prefer when they can choose between them. We have collected qualitative user feedback via questionnaires. We also conducted a quantitative user study, where we evaluated user-generated logging data over a period of three weeks with a group of 18 participants. Our results show that the time-wheel can serve as an intuitive way to explore timedependent resources such as events. In addition, it seems that the grid-based navigation approach is the preferable choice when exploring large spaces of faceted data.

Providing an intuitive and user friendly interface is a challenging task. This challenge is even harder when it comes to mobile phones as such devices have limited interaction possibilities and smaller display size. The present work will develop a study about semiotic profiles as a way to improve techniques for designing graphical interfaces using large social media data spaces such as GeoNames, Eventful and DBpedia in the context of mobile phones screens.

The Linked Open Data (LOD) cloud provides a vast amount of heterogeneous data, distributed over numerous data sources. This makes it difficult to find those data sources in the cloud which are relevant for a given information need. Existing search engines for the Semantic Web focus on instance-oriented information needs, i. e., searching for specific RDF instances or literals and exploring the search results. However, they do not address the question of finding linked data sources relevant to a schema-oriented information need, i. e., queries based on triple patterns relating to a specific combination of RDF types and/or properties. In this paper, we present the semantic search system LODatio leveraging a schema-level index for finding sources of Linked Data relevant to a schema-oriented information need. Beyond its capability to retrieve relevant data sources, LODatio actively supports the user in his schema-oriented search tasks. To this end, it provides ranked result lists of relevant data sources together with example snippets and an estimation of the result set size. Furthermore, LODatio provides support for novel features in semantic search such as recommending alternative queries in order to refine or broaden the result set.

When publishing graph data on the web such as vocabularies using RDF(S) or OWL, one has only limited means to verify the authenticity and integrity of the graph data. Today's approaches require a high signature overhead and do not support iterative signing of graph data. This paper describes a first step towards a framework for signing arbitrary graph data provided in RDF(S), Named Graphs, or OWL. Our framework supports signing graph data at different levels of granularity: minimum self-contained graphs (MSG), sets of MSGs, and entire graphs. It supports iteratively signing graph data, e. g., when different parties provide different parts of a common graph, and allows for signing multiple graphs. Both can be done with a constant, low overhead for the resulting signature statements, even when iteratively signing.

Higher-level semantics for multimedia content is essential to answer questions like ``Give me all presentations of German Physicists of the 20th century''. The tutorial provides an introduction and overview to such semantics and the developments in multimedia metadata. It introduces current advancements for describing media on the web using Linked Open Data and other more expressive semantic technologies. The application of such technologies will be shown at concrete examples.

The automated selection of satisfying subsets from large collections of photos is a central challenge in multimedia research. Objective criteria like the depiction of persons or the photo quality are met by existing approaches. But it is difficult to know the users' personal interest, which plays an important role in the selection process. The expected spread of devices with eye tracking support in the near future allows us to measure this interest in a new way. In an experiment with 12 participants, we derive the most interesting photos of a collection for every person from gaze information recorded during the free viewing of the photos. We can show that the eye tracking information delivers valuable information about the users' preferences by comparing the results to a manual selection. The selection based on gaze information significantly outperforms baseline approaches and improves the results by up to 17%. For photo sets of personal interest this improvement is even up to 23%.

We compare two approaches for exploring large, hierarchical data spaces on mobile devices using facets. While the first approach arranges the facets in a 3x3 grid, the second approach makes use of a scrollable list of facets for exploring the data. As concrete scenario, we use category hierarchies that are dynamically obtained from different, distributed social media sources. We have conducted a between-group experiment of the two approaches with 64 subjects (41 male, 23 female) executing the same set of tasks of typical mobile users' information needs. The results show that subjects using the grid-based approach require significantly more time and more clicks for completing the tasks. However, regarding the subjects' satisfaction there is no significant difference between the two approaches. Thus, if efficiency is not the primary objective, the grid-based approach might be of interest as it provides a navigation element that allows the users to see the exact position in the data space. This might be a very useful feature in scenarios where knowing the exact position is quite crucial such as browsing classification schemas in digital libraries or other formal taxonomies.

This paper presents an overview of the Joint Workshop on Modeling and Representing Events (JMRE), which is held as part of ACM Multimedia 2011. JMRE is concerned with the understanding of events from multimedia, and with using events in order to better organize and consume multimedia.

No abstract available.

We propose a new approach for mobile visualization and interaction of temporal information by integrating support for time with today's most prevalent visualization of spatial information the map. Our approach allows for an easy and precise selection of the time that is of interest and provides immediate feedback to the users when interacting with it. It has been developed in an evolutionary process gaining formative feedback from end users.

The semantics of rich multimedia presentations in the web such as SMIL, SVG, and Flash cannot or only to a very limited extend be understood by search engines today. This hampers the retrieval of such presentations and makes their archival and management a difficult task. Existing metadata models and metadata standards are either conceptually too narrow, focus on a specific media type only, cannot be used and combined together, or are not practically applicable for the semantic description of rich multimedia presentations.

Semantic desktop environments aim at improving the effectiveness and efficiency of users carrying out daily tasks within their personal information management (PIM) infrastructure. They support the user by transferring and exploiting the explicit semantics of data items across different PIM applications. Whether such an approach does indeed reach its aim of facilitating users' life and--if so--to which extent, however, remains an open question. In this paper we address this question with the first summative evaluation of a semantic desktop. We have developed a test environment to evaluate two semantic PIM applications against standard PIM tools. As result, we have found significant efficiency and satisfaction improvements for typical PIM tasks.

The lack of a formal model of events hinders interoperability in distributed event-based systems. In this paper, we present a formal model of events, called Event-Model-F. The model is based on the foundational ontology DOLCE+DnS Ultralight (DUL) and provides comprehensive support to represent time and space, objects and persons, as well as mereological, causal, and correlative relationships between events. In addition, the Event-Model-F provides a flexible means for event composition, modeling event causality and event correlation, and representing different interpretations of the same event. The Event-Model-F is developed following the pattern-oriented approach of DUL, is modularized in different ontologies, and can be easily extended by domain specific ontologies.

No abstract available.

Photos are often a means to remember personal events, and the creation of photo albums is the attempt to preserve our memories in a nice book. For a long time people have been creating such photo albums on the basis of prints from analog photos arranged in an album book with scissors and glue and annotated with comments and captions - a tedious task which in these days is getting support by authoring tools and digitally mastered photo books. Relying on the content of others such as printed travel guides, news papers, leaflets, but also friends and family the personal content often has been enriched, enhanced, and completed. This is the starting point of our work: with digital photography and the increasing amount of content-based and contextual metadata of personal photos we can now use this metadata to actually support the targeted and semi-automatic inclusion of interesting, related information from content of others, e.g., from Web 2.0 communities, and offer and add it at the right spot in the personal album. In this paper, we show how photo album creation can benefit from leveraging information learned from many users in regard of the album's content, structure, and semantics.

In recent years, many highly sophisticated multimedia authoring tools have been developed. Up to today, these system's integration of the targeted user context, however, is limited. With our Context-aware Smart Multimedia Authoring Tool (xSMART) we developed a semi-automatic authoring tool that integrates the targeted user context into the different authoring steps and exploits this context to guide the author through the content authoring process. The design of xSMART allows that it can be extended and customized to the requirements of a specific domain by domain-specific wizards. These wizards realize the user interface that meets best the domain-specific requirements and effectively supports the domain experts in creating their content targeted at a specific user context.

In this paper, we present the Media Gallery, a MHP-based interactive multimedia application on digital TV. This application allows customers to view and order their digital photos and to order physical prints and fun products from these digital photos directly from TV. The Media Gallery opens a new distribution channel and market opportunity for the photo finisher and a platform to comfortably view and order their digital images directly on their TV.

Users of multimedia applications today are equipped with a variety of different (mobile) devices that each come with different operating systems, memory and CPU capabilities, network connections, and also different software such as multimedia players. To be able to efficiently deliver appealing multimedia presentations to all the users, we need to overcome the last mile in multimedia presentation delivery and meet the different requirements at the end user’s site. Consequently, one needs to provide multi-channel multimedia presentation generation such that all different users can get and use it in their individual device configuration. With our approach, we aim at developing an abstract multimedia content model that embeds the central characteristics of today’s multimedia presentation formats: the definition of the temporal and spatial layout as well as the interaction possibilities. The abstract model allows to be easily transformed to different multimedia presentation formats on different devices and by this serve different output channels. We present our abstract multimedia model and transformation process that allows to compose and to generate suitable multimedia presentations for each different channel such as a SMIL 2.0 presentation created for rendering on a PC or a SVG Tiny presentation designed for a mobile device. With our implementation, application developers can efficiently realize on-the-fly multichannel generation of multimedia content.

Mobile applications such as mobile tourist guides that provide tourists with location-based information today mostly aim to adapt the multimedia content to the different end user devices. More and more, these applications also exploit positioning information like GPS to guide the user on the trip. What is still lacking, however, is a personalization of the content to the interests and preferences of the individual tourist and the characteristics of the used end device. However, such a personalization increases the application's complexity since every individual alternatives have to be considered and implemented. To provide substantial support for the development of personalized (mobile) multimedia applications, we developed a domain independent software framework for an efficient and cost-effective development of personalized mobile multimedia applications. We illustrate the framework in the specific domain of personalized mobile tourist information.