In order to minimize the impact of influenza on public health, accurate early forecasting is essential. Various deep-learning-based models have been proposed to predict future influenza occurrences by capturing temporal/regional patterns from past occurrence time-series data. However, the prediction performance of these unimodal approaches is limited because they extract knowledge only from collected data, and users cannot input contextual information and domain knowledge to them. Recently, large language models (LLMs) have demonstrated the potential to improve prediction accuracy by linking contextual text information to time-series predictions. In this paper, we propose MIFlu, a multimodal influenza forecasting scheme that can fuse contextual text information to time-series influenza occurrences using two LLMs. It first extracts text embeddings from the user's text prompts that contain contextual information using a text-embedding LLM. Then, MIFlu fuses the text embeddings and time-series embeddings and uses the fused embeddings to predict future occurrences using a forecasting LLM. In extensive experiments using public national/regional influenza datasets, MIFlu outperforms other predictive models, improving prediction performance by up to 26.2% compared to state-of-the-art models. We also analyze the effect of various textual input embedders, hyperparameters, and the amount of training data on forecasting accuracy.

Early forecasting of influenza is an important task for public health to reduce losses due to influenza. Various deep learning-based models for multi-regional influenza forecasting have been proposed to forecast future influenza occurrences in multiple regions. While they only use historical data for forecasting, temporal and regional patterns need to be jointly considered for better accuracy. Basic deep learning models such as recurrent neural networks and graph neural networks have limited ability to model both patterns together. A more recent approach uses an attention mechanism or its variant, self-attention. Although these mechanisms can model regional interrelationships, in state-of-the-art models, they consider accumulated regional interrelationships based on attention values that are calculated only once for all of the input data. This limitation makes it difficult to effectively model the regional interrelationships that change dynamically during that period. Therefore, in this article, we propose a recurrent self-attention network (RESEAT) for various multi-regional forecasting tasks such as influenza and electrical load forecasting. The model can learn regional interrelationships over the entire period of the input data using self-attention, and it recurrently connects the attention weights using message passing. We demonstrate through extensive experiments that the proposed model outperforms other state-of-the-art forecasting models in terms of the forecasting accuracy for influenza and COVID-19. We also describe how to visualize regional interrelationships and analyze the sensitivity of hyperparameters to forecasting accuracy.

Due to the spread of various information and communication technologies, a huge amount of images are produced and shared for diverse purposes. Several de-identification techniques for photos, such as pixelation, blur, and mask, are routinely used in light of recent worries about the growing number of privacy leakages. However, due to the low image quality and loss of many facial features, these de-identified images are not suitable for use in applications such as training models that require a lot of high-quality data. Therefore, in this paper, we propose a new face de-identification method focusing only on facial regions essential for personal identification. By generating facial landmarks differently from the original person using masking and generative adversarial networks-based inpainting, our method can perform de-identification efficiently. To demonstrate the performance of our proposed scheme, we conducted quantitative and qualitative evaluations using an open dataset. We show that our proposed scheme outperforms other de-identification methods.

Daily peak load forecasting (DPLF) is critical in smart grid applications for security analysis, unit commitment, and scheduling of outages and fuel supplies. Although excellent single machine learning methods using tree-based ensemble learning or deep learning have shown satisfactory performance for DPLF, there is still room for improvement. This study proposes a hybrid tree-based ensemble learning model, called HYTREM, for robust DPLF. We first collected two commercial buildings’ energy consumption data from publicly available datasets. We then performed data preprocessing, such as input variable configuration, for the HYTREM modeling. We divided both datasets into training and test sets and generated the prediction values of several tree-based ensemble learning models, such as gradient boosting machine, extreme gradient boosting, Cubist, and random forest (RF), for each set as novel input variables. We reconstructed datasets using the Boruta algorithm to select all the relevant features and built an online RF model trained on these datasets using time-series cross-validation for day-ahead DPLF. The experimental results showed that the HYTREM performed a better performance than tree-based ensemble and deep learning methods in building-level DPLF in terms of the mean absolute percentage error and normalized root mean square error.

Photo cartoonization aims to convert photos of real-world scenes into cartoon-style images. Recently, generative adversarial network (GAN)-based methods for photo cartoonization have been proposed to generate pleasable cartoonized images. However, as these methods can transfer only learned cartoon styles to photos, they are limited in general-purpose applications where unlearned styles are often required. To address this limitation, an arbitrary style transfer (AST) method that transfers arbitrary artistic style into content images can be used. However, conventional AST methods do not perform satisfactorily in cartoonization for two reasons. First, they cannot capture the unique characteristics of cartoons that differ from common artistic styles. Second, they suffer from content leaks in which the semantic structure of the content is distorted. In this paper, to solve these problems, we propose a novel arbitrary-style photo cartoonization method, Cartoon-Flow. More specifically, we construct a new hybrid GAN with an invertible neural flow generator to effectively preserve content information. In addition, we introduce two new losses for cartoonization: (1) edge-promoting smooth loss to learn the unique characteristics of cartoons with smooth surfaces and clear edges, and (2) line loss to mimic the line drawing of cartoons. Extensive experiments demonstrate that the proposed method outperforms previous methods both quantitatively and qualitatively.

Early prediction of influenza plays an important role in minimizing the damage caused, as it provides the resources and time needed to formulate preventive measures. Compared to traditional mechanistic approach, deep/machine learning-based models have demonstrated excellent forecasting performance by efficiently handling various data such as weather and internet data. However, due to the limited availability and reliability of such data, many forecasting models use only historical occurrence data and formulate the influenza forecasting as a multivariate time-series task. Recently, attention mechanisms have been exploited to deal with this issue by selecting valuable data in the input data and giving them high weights. Particularly, self-attention has shown its potential in various forecasting tasks by utilizing the predictive relationship between objects from the input data describing target objects. Hence, in this study, we propose a forecasting model based on self-attention for regional influenza forecasting, called SAIFlu-Net. The model exploits a long short-term memory network for extracting time-series patterns of each region and the self-attention mechanism to find the similarities between the occurrence patterns. To evaluate its performance, we conducted extensive experiments with existing forecasting models using weekly regional influenza datasets. The results show that the proposed model outperforms other models in terms of root mean square error and Pearson correlation coefficient.

ABSTRACT In this paper, we propose a new security log management scheme for smart surveillance in a multi‐camera environment. Basically, our security log consist of descriptions for various behavior properties of moving objects, such as motion type, time, and speed in a merged camera view. To generate such security log, we first analyze the input video frame from each surveillance camera and construct a motion vector of interest points in the frame. By analyzing the motion vector, we recognize moving objects and trace their local behavior in the video. On the basis of this analysis, we can calculate various global behavior features of the objects in the merged camera view, which can be acquired by stitching together the frames from multiple camera inputs. Such global behavior features are captured into security logs, which can be used to smartly carry out various surveillance operations such as retrieving objects whose behavior is similar to a query behavior or whose behavior shows predefined abnormality. Because our scheme treats all the objects in the frame independently, it can handle multiple objects simultaneously. We implemented a prototype system and performed various experiments to demonstrate that our scheme can achieve a reasonable performance. Copyright © 2013 John Wiley & Sons, Ltd.

With the wide-spread interest in various healthcare services, there has been increasing demand for quantitative and objective criteria on which health condition of subjects can be effectively evaluated. Human organs show different symptoms depending on the type of health problem. Among human organs, skin has drawn much attention since it is the outermost part and hence easy to investigate. Skin condition is known to be analyzed and estimated using various features such as wrinkle and elasticity. In this paper, we propose an automatic skin aging evaluation scheme. More specifically, we first collect wrinkle-related features from subjects of different ages and dermatologists evaluate their aging level for the ground truth. And then, we train and compare non-linear, multi-class SVM (Support Vector Machine) using these datasets and ground truth for classification. To evaluate the effectiveness of our proposed scheme, we performed experiments using the SVM. We report some of the result.

Dominant features for content-based image retrieval usually consist of high-dimensional values. So far, many researches have been done to index such values for fast retrieval. Still, many existing indexing schemes are suffering from performance degradation due to the curse of dimensionality problem. As an alternative, heuristic algorithms have been proposed to calculate the result with `high probability' at the cost of accuracy. In this paper, we propose a new hash tree-based indexing structure called tertiary hash tree for indexing high-dimensional feature values. Tertiary hash tree provides several advantages compared to the traditional extendible hash structure in terms of resource usage and search performance. Through extensive experiments, we show that our proposed index structure achieves outstanding performance.

A typical music clip consists of one or more segments with different moods and such mood information could be a crucial clue for determining the similarity between music clips. One representative mood has been selected for music clip for retrieval, recommendation or classification purposes, which often gives unsatisfactory result. In this paper, the authors propose a new music retrieval and recommendation scheme based on the mood sequence of music clips. The authors first divide each music clip into segments through beat structure analysis, then, apply the k-medoids clustering algorithm for grouping all the segments into clusters with similar features. By assigning a unique mood symbol for each cluster, one can transform each music clip into a musical mood sequence. For music retrieval, the authors use the Smith-Waterman (SW) algorithm to measure the similarity between mood sequences. However, for music recommendation, user preferences are retrieved from a recent music playlist or user interaction through the interface, which generates a music recommendation list based on the mood sequence similarity. The authors demonstrate that the proposed scheme achieves excellent performance in terms of retrieval accuracy and user satisfaction in music recommendation.

Music evokes various human emotions or creates music moods through low level musical features. In fact, typical music consists of one or more moods and this can be used as an important factor for determining the similarity between music. In this paper, we propose a new music retrieval scheme based on the mood change pattern. For this, we first divide music clips into segments based on low level musical features. Then, we apply K-means clustering algorithm for grouping them into clusters with similar features. By assigning a unique mood symbol for each group, each music clip can be represented into a sequence of mood symbols. Then, we estimate the similarity of music based on the similarity of their musical mood sequence using the longest common subsequence (LCS) algorithm. To evaluate the performance of our scheme, we carried out various experiments and measured the user evaluation. We report some of the results.

In this paper, we discuss the use of knowledge for analyzing and retrieving music contents semantically. First, we present Context-based Music Recommendation (COMUS) ontology to reason desired user emotion state from context and user preference information in the ontology. COMUS is a music dedicated ontology in OWL constructed by incorporating domain specific classes for music recommendation into the Music Ontology, which include situation, mood and musical features. More specifically, we describe the ontologies of mood and situation in music using low-level features like pitch or duration and musical factors like tempo or rhythm. Our proposed ontology defines generic as well as domain-specific concepts whose detection is important for the analysis and description of music in a specific domain. As a novelty, our ontology can express detailed and complicated relations among the music, moods and situations, enabling users to find appropriate music for the music retrieval and recommendation application. We present some of the experiments we performed as a case-study for music recommendation.

To date, common acoustic features such as MPEG-7 and Fourier/wavelet transform-based features have been frequently used for environmental sound classification. However, these transforms have difficulty dealing with specific properties of environmental sounds, due to their limited scopes. In this paper, we investigate three types of transforms as yet untried for this purpose, and show that they are more effective than traditional features. This result is mainly due to the fact that they have functionalities that were not easily treatable with traditional transforms. Experimental results show that the combination of these features with traditional features can achieve 86.09% of the maximum accuracy in environmental sound classification, compared to 74.35% of the maximum accuracy when confined to traditional features.

With the rapid deployment of information technology and the availability of cheap yet high performance image capturing devices, new types of healthcare services such as self-diagnosis and treatment have become possible. Skin is the outer layer of the human body and has long attracted a great deal of attention, since its appearance conveys useful information on the health condition of the subject. In this paper, we propose a skin age estimation scheme based on its wrinkle features such as length, width and depth, which represents the physical condition of skin statistically and quantitatively. We collected wrinkle features and personal data from various subjects, including age and gender, and constructed the ground truth in consultation with dermatologists. For the estimation, we used a non-linear, multi-class SVM (support vector machine). Via extensive experiments on our prototype system, we show that our scheme achieves a reasonable accuracy.

With the advent of the ubiquitous era, context-based music recommendation has become one of rapidly emerging applications. Context-based music recommendation requires multidisciplinary efforts including low level feature extraction, music mood classification and human emotion prediction. Especially, in this paper, we focus on the implementation issues of context-based mood classification and music recommendation. For mood classification, we reformulate it into a regression problem based on support vector regression (SVR). Through the use of the SVR-based mood classifier, we achieved 87.8% accuracy. For music recommendation, we reason about the user's mood and situation using both collaborative filtering and ontology technology. We implement a prototype music recommendation system based on this scheme and report some of the results that we obtained.

Query reformulation has been suggested as an effective way to improve retrieval efficiency in text information retrieval and one of the well-known techniques for query reformulation is user relevance feedback. Recently, there has been an increased interest in the query reformulation using relevance feedback with evolutionary techniques such as genetic algorithm for multimedia information retrieval. However, these techniques have still not been exploited widely in the field of music retrieval. In this paper, we propose a novel music retrieval scheme that is based on user relevance feedback with genetic algorithm and evolutionary method with neural network. The former is for reformulating a user query and the latter is for reducing the population size by learning neural network. We implemented a prototype music retrieval system called MUSEMBLE based on this scheme. Experimental results showed that our proposed scheme achieves a good performance.

Accurate voice humming transcription and efficient indexing schemes are essential for a large-scale humming-based music retrieval system. Although many researches have been done to develop such schemes, their performances are not still satisfactory. In our previous works, we proposed (i) a new voice query transcription scheme [4], (ii) a popularity-adaptive indexing structure called FAI [6] for fast retrieval, and (iii) a semi-supervised relevance feedback and query reformulation scheme based on a genetic algorithm [7] in order to improve retrieval efficiency. In this demonstration, we extend our efforts to a mobile environment and develop a prototype mobile music retrieval system called M-MUSICS. Our focus in this implementation includes versatile user interface for easy querying and browsing on a typical mobile device such as PDA phone and satisfactory performance in a wireless mobile environment. We report some of the results.

This demonstration presents a content-based leaf image retrieval system that supports wired/wireless access. For example, if we want to know about a plant that we encounter in a mountain or field, we might look it up in an illustrated book. But, it will take a long time to search due to the lack of appropriate indexing or search clues and huge amounts of similar plants. In order to solve this problem, we developed a content-based leaf image retrieval system called mCLOVER that supports both wired and wireless access and includes a set of novel features for easy querying and efficient retrieval.

Proliferation of audio databases on the WWW (World Wide Web) necessitates an audio retrieval system to find certain audio content within the audio corpus. Many papers have presented concepts, methodologies and systems to offer users ways to retrieve melody from collections of music contents. In this paper, we present a new index scheme to retrieve music based on the accumulated data of previous user queries to music. We first describe the current status of existing music information retrieval systems, and then present the design and implementation of our prototype system. We report the results obtained from an empirical evaluation of our approach.

A distributed video-on-demand (VoD) system is one where a collection of video data is located at dispersed sites across a computer network. In a single site environment, a local video server retrieves video data from its local storage device. However, in distributed VoD systems, when a customer requests a movie from the local server, the server may need to interact with other servers located across the network. In this paper, we present different types of presentation plans that a local server can construct in order to satisfy a customer request. Informally speaking, a presentation plan is a temporally synchronized sequence of steps that the local server must perform in order to present the requested movie to the customer. This involves obtaining commitments from other video servers, obtaining commitments from the network service provider, as well as making commitments of local resources, while keeping within the limitations of available bandwidth, available buffer, and customer data consumption rates. Furthermore, in order to evaluate the quality of a presentation plan, we introduce two measures of optimality for presentation plans: minimizing wait time for a customer and minimizing access bandwidth which, informally speaking, specifies how much network/disk bandwidth is used. We develop algorithms to compute three different optimal presentation plans that work at a block level, or at a segment level, or with a hybrid mix of the two, and compare their performance through simulation experiments. We have also mathematically proven effects of increased buffer or bandwidth and data replications for presentation plans which had previously been verified experimentally in the literature.

Considers a distributed video server environment where video movies need not be stored entirely in one server. Blocks of a video movie are be distributed and replicated over multiple video servers. Customers are served by one video server. This video server, termed the originating server, might have to interact with other servers for downloading missing blocks of the requested movie. We present three types of presentation plans that an originating server can possibly construct for satisfying a customer's request. A presentation plan can be considered as a detailed (temporally synchronized) sequence of steps carried out by the originating server for presenting the requested movie to the customer. The creation of presentation plans involves obtaining commitments from other video servers and the network service provider, as well as making local resource commitments, within the limitations of available bandwidth, available buffer and customer consumption rates. For evaluating the goodness of a presentation plan, we introduce two measures of optimality for presentation plans: minimizing the waiting time for a customer and minimizing the access bandwidth. We present algorithms for computing optimal presentation plans and compare their performance experimentally. We have also mathematically proved certain results for the presentation plans.