In the paper, we propose a two-stage face-inpainting approach based on conditional generative adversarial networks. In the first stage, a deep-learning model is trained for predicting face landmarks. It also dynamically adjusts the penalty value of the loss function based on the view-degree of a face to improve the ability of predicting high view-degree faces. In the second stage, masked face images and their corresponding face landmarks are concatenated to form the condition of training a conditional Generative Adversarial Network (GAN) for inpainting the masked face. If an input masked image is a nearly-frontal face, an additional procedure for face symmetry processing will be performed before the image is input into the inpainting model. The experimental results show that the proposed training method in the first stage can effectively enhance the robustness of the face-landmark prediction model and reduce the impact of data imbalance, thereby improving the effect of later face inpainting. They also show that the proposed face-inpainting model in the second stage can better maintain the geometric structures and symmetric outlooks of inpainted faces than previous ones.

Music has always been a popular media because it can relax our pressure of life. However, the music appealing to an individual could shift under his/her different emotions. For example, the preferred music in a sad mode is very possibly different from that in a happy manner. Therefore, effectively representing the human sense hidden in music can link the user emotion to music. To aim at this issue, Music Retrieval Information (MIR) were proposed for recognizing musical emotion. In the past, although some studies have been made on music emotion recognition, their effectiveness is not satisfactory. A potential reason is that the audio features extracted are not robust enough to discriminate the diversity between music and emotion. Hence, in this paper, we propose an effective music recognition method, which fuses Deep Learning (DL) and Support Vector Machine (SVM). The major difference between the proposed method and traditional audio-based studies is that the proposed method aggregates the partial recognition results of music to achieve the better recognition precision. The experimental results on a real dataset of CAL500 show that the proposed method performs better than some other audio-based music emotion labeling methods.

In recent years, people are used to listening to music because the music can effectively relax our tight life. Hence, how to retrieve the preferred music from a large amount of music data has been an attractive topic for many years. Traditionally, music retrieval contains two main types, namely text-based music retrieval and content-based music retrieval. However, these traditional music retrieval types ignore a human sense: emotion. That is, the preferred music might be different in different emotions. In fact, the emotion is highly related to the environment and it can be represented by brain actions. Therefore, in this paper, we propose a creative approach that performs a ubiquitous music search by content comparisons of brains and music. The major intent of this paper is to provide affective music retrieval in different contexts. Without any query, the context-brain triggers the music search and the context-related music will be retrieved by computing brain similarities and music similarities. The proposed approach was materialized and evaluated by a number of volunteers. The evaluation results reveal that, the proposed affective music retrieval can obtain high satisfactions for the invited testing users.

In recent years, how to discover the valuable knowledge from a huge amount of data has been a hot topic. Data mining is one of the solutions for this topic. Actually, data mining has been studied for a long time, including a lot of paradigms. Among these paradigms, High-Utility Itemset Mining attracts much research attention because it can find the itemsets different from traditional frequent itemsets. Although these related works have been shown to be efficient, it still cannot mine the really rare itemsets infrequent by only one minimum utility support. In addition, an efficient mining algorithm relies on an important factor “search strategy”. For these concerns, in this paper, an efficient high-utility itemset mining algorithm with multiple minimum utility support and prefix-search strategy is proposed to effectively mine the really valuable itemsets. For effectiveness, the rare but infrequent itemsets can be discovered by the individually specified utility supports. For efficiency, the aimed itemsets can be mined without the level-wise searching by a prefix-search way. The experimental results show the proposed algorithm performs better than the compared one on the synthetic and real datasets.

Recently, music recommender systems have been proposed to help users obtain the interested music. Traditional recommender systems making attempts to discover users' musical preferences by ratings always suffer from problems of rating diversity, rating sparsity and lack of ratings. These problems re sult in unsatisfactory recommendation results. To deal with traditional problems, in this paper, we propose a novel music recommender system, namely Multi-modal Music Recommender system (MMR), which integrates social and collaborative information to predict users' preferences. In this work, the playcounts are transformed into collaborative information to cope with problem of lack of rating information, while item tags and artist tags are employed as social information to cope with problems of rating diversity and rating sparsity. Through optimizing the integrated social-and-collaborative information, the users' preferences can be inferred more accurately and efficiently. The experimental results reveal that, three problems can be alleviated significantly and our proposed method outperforms other state-of-the-art recommender systems in terms of RMSE (Root Mean Square Error) and NDCG (Normalized Discount Cumulative Gain).

In this paper, to increase the diversity of stock portfolios, the diverse group stock portfolio mining algorithm is proposed based on the grouping genetic algorithm. Each chromosome is represented by grouping part, stock part and stock portfolio part. The fitness function that consists of portfolio satisfaction, group balance and diversity factor is designed to evaluate quality of chromosomes. The diversity factor is used to make the numbers of stock categories in groups as similar as possible. The genetic operations are then executed on population to generate offspring for finding a near-optimal group stock portfolio. Finally, experiments on a real financial data were made to show the effectiveness of the proposed approach.

In the past, several algorithms were developed to mine fuzzy frequent itemsets (FFIs) in which each item is represented at most one linguistic term based on maximum scalar cardinality. In real-life situations, multiple fuzzy linguistic terms instead of the single one can, however, produce more useful and meaningful fuzzy association rules. The Apriori-based algorithm was developed to mine multiple fuzzy frequent itemsets (MFFIs), which requires to generate the amounts of candidates and determine them in a level-wise way. In this paper, a fuzzy-list-based (FL)-Miner algorithm is developed to mine the complete set of MFFIs without candidate generation. Two efficient pruning strategies are also developed to reduce the search space, thus speeding up the mining process to directly discover the MFFIs. Experiments are conducted to show the performance of the proposed approaches compared to the state-of-the-art level-wise algorithm in terms of execution time and memory usage.

Mining of high-utility itemsets in transactional databases is emerging topic in recent years since it can be used to reveal more information for decision making, which has been widely used in many real-life applications. For the traditional high-utility itemset mining (HUIM), only the utility values of the itemsets are considered without timestamps or periodic constraints. In this paper, we present a new short periodic high-utility itemset mining (SPHUIM) to mine the set of short periodic high-utility itemsets (SPHUIs) by considering both the period and utility measures. A baseline two-phase SPHUI-TP algorithm is first presented to mine SPHUIs in level-wise manner. To reduce the search space of SPHUI-TP algorithm, two pruning strategies are also developed to speed up the mining performance of the SPHUI-TP algorithm. Substantial experiments both on real-life and synthetic datasets showed the efficiency and effectiveness of the designed approaches.

Mining association rules from large databases is a predominant research problem in the data mining community. Although in the past two decades we have witnessed substantial achievements on developing efficient algorithms, the classical model of mining association rules with high frequency is not feasible for some real applications. For example, in the area of medical data mining, useful or valuable patterns, such as the associations that disclose the relationship between drugs and ADRs (Adverse Drug Reactions) usually occur rarely, i.e., less than 0.1%. This leads to at least two drawbacks when applying contemporary efficient algorithms to these applications: (1) most discovered frequent patterns are uninteresting or useless, and (2) most of the efforts spent on pattern discovery are wasted. As such, even the fastest algorithm will become dramatically inefficient in discovering such infrequent but promising rules. In light of these observations, we propose a new model of association rule mining, called range supported association rules, which extends the strict support threshold to an interval. Problems of adopting Apriori-like algorithms for this new model are discussed and a promising algorithm is also proposed.

Over the past few years, the recommender system has been proposed as a critical role to help users choose the preferred product from a massive amount of data. For music recommendation, most recent recommender systems made attempts to associate music with the user's preferences primarily based on user ratings. However, this kind of recommendation mechanism encounters the problem called rating diversity that makes the prediction results unreliable. To cope with this problem, in this paper, we propose a novel music recommendation approach that utilizes social media tags instead of ratings to calculate the similarity between music pieces. Through the proposed tag-based similarity, the user preferences hidden in tags can be inferred effectively. The empirical evaluations on real social media datasets reveal that our proposed approach using social tags outperforms the existing ones using only ratings in terms of predicting the user's preferences to music.

This paper proposes a novel audiovisual presentation system, called PhotoSense, to enrich photo navigation experience by associating emotionally harmonic music with a given photo collection. Different from many conventional photo visualization systems which predominantly focus on the visual elements for presentation, we explore both visual and aural perspectives which can enhance the browsing experience from each other. This is achieved by building an emotion space shared by visual and aural domains, and a set of emotion classifiers which can associate each visual and aural element with this space. Furthermore, we design a sequence matching algorithm to associate a set of music with a photo collection by maximizing similarity in the emotion space. Photo-Sense represents one of the first mash-up applications which build a natural connection between the ever increasing personal photo collections on the Web and music-sharing sites. Experiments show that PhotoSense provides better browsing experience for photo collections.

Nowadays, content-based image retrieval (CBIR) is the mainstay of image retrieval systems. To be more profitable, relevance feedback techniques were incorporated into CBIR such that more precise results can be obtained by taking user's feedbacks into account. However, existing relevance feedback-based CBIR methods usually request a number of iterative feedbacks to produce refined search results, especially in a large-scale image database. This is impractical and inefficient in real applications. In this paper, we propose a novel method, Navigation-Pattern-based Relevance Feedback (NPRF), to achieve the high efficiency and effectiveness of CBIR in coping with the large-scale image data. In terms of efficiency, the iterations of feedback are reduced substantially by using the navigation patterns discovered from the user query log. In terms of effectiveness, our proposed search algorithm NPRFSearch makes use of the discovered navigation patterns and three kinds of query refinement strategies, Query Point Movement (QPM), Query Reweighting (QR), and Query Expansion (QEX), to converge the search space toward the user's intention effectively. By using NPRF method, high quality of image retrieval on RF can be achieved in a small number of feedbacks. The experimental results reveal that NPRF outperforms other existing methods significantly in terms of precision, coverage, and number of feedbacks.

Image annotation based on visual features has been a difficult problem due to the diverse associations that exist between visual features and human concepts. In this paper, we propose a novel approach called Annotation by Image-to-Concept Distribution Model (AICDM) for image annotation by discovering the associations between visual features and human concepts from image-to-concept distribution. Through the proposed image-to-concept distribution model, visual features and concepts can be bridged to achieve high-quality image annotation. In this paper, we propose to use “visual features”, “models”, and “visual genes” which represent analogous functions to the biological chromosome, DNA, and gene. Based on the proposed models using entropy, tf-idf, rules, and SVM, the goal of high-quality image annotation can be achieved effectively. Our empirical evaluation results reveal that the AICDM method can effectively alleviate the problem of visual-to-concept diversity and achieve better annotation results than many existing state-of-the-art approaches in terms of precision and recall.

Mining generalized association rules with fuzzy taxonomic structures has been recognized as a important extension of generalized associations mining problem. To date most work on this problem, however, required the taxonomies to be static, ignoring the fact that the taxonomies of items cannot necessarily be kept unchanged. For instance, some items may be reclassified from one hierarchy tree to another for more suitable classification, abandoned from the taxonomies if they will no longer be produced, or added into the taxonomies as new items. Additionally, the membership degrees expressing the fuzzy classification may also need to be adjusted. Under these circumstances, effectively updating the discovered generalized association rules is a crucial task. In this paper, we examine this problem and propose two novel algorithms, called FDiffET and FDiff_ET2, to update the discovered frequent generalized itemsets.

Up to the present, the contemporary studies are not really successful in image annotation due to some critical problems like diverse regularities between visual features and human concepts. Such diverse regularities make it hard to annotate the image semantics correctly. In this paper, we propose a novel approach called AICDM (Annotation by Image-Concept Distribution Model) for image annotation by discovering the associations between visual features and human concepts from image-concept distribution. Through the proposed image-concept distribution model, the uncertain regularities between visual features and human concepts can be clarified for achieving high-quality image annotation. The empirical evaluation results also reveal that our proposed AICDM method can effectively alleviate the uncertain regularity problem and bring out better annotation results than other existing approaches in terms of precision and recall.

Traditional image retrieval based on visual-based matching is not effective in multimedia applications. Consequently, the modeling of high-level human sense for image retrieval has been a challenging issue over the past few years. In fact, the concepts hidden in the images play key roles in semantic image retrieval. In this paper, we propose a novel method named Intelligent Concept-Oriented Search (ICOS) that can capture the high-level concepts in images by utilizing data mining and query decomposition techniques. The contributions of the proposed method lie in that we provide: 1) effective annotation for conceptual objects, 2) association mining for conceptual objects, 3) visual ranking for conceptual objects and 4) intelligent search method for enhancing high-level concept image retrieval. Through experimental evaluations, ICOS is shown to be very effective and efficient in capturing the implicit high-level concepts for image retrieval.

To support effective multimedia information retrieval, video annotation has become an important topic in video content analysis. Existing video annotation methods put the focus on either the analysis of low-level features or simple semantic concepts, and they cannot reduce the gap between low-level features and high-level concepts. In this paper, we propose an innovative method for semantic video annotation through integrated mining of visual features, speech features, and frequent semantic patterns existing in the video. The proposed method mainly consists of two main phases: 1) Construction of four kinds of predictive annotation models, namely speech-association, visual-association, visual-sequential, and statistical models from annotated videos. 2) Fusion of these models for annotating un-annotated videos automatically. The main advantage of the proposed method lies in that all visual features, speech features, and semantic patterns are considered simultaneously. Moreover, the utilization of high-level rules can effectively complement the insufficiency of statistics-based methods in dealing with complex and broad keyword identification in video annotation. Through empirical evaluation on NIST TRECVID video datasets, the proposed approach is shown to enhance the performance of annotation substantially in terms of precision, recall, and F-measure.