This study aims to clarify how functional network hubs change during hierarchical visual processing in the human brain through the estimation of brain states from features extracted using a convolutional neural network (CNN), a hierarchical model of image processing. We used representational similarity analysis for brain states predicted through encoding models based on feature representations at each layer of the CNN, and applied the PageRank algorithm to matrices converted from the generated representational dissimilarity matrices to capture the hub characteristics of brain region-related systems. This succeeded in capturing changes in the hubness of interregional brain coordination during hierarchical information processing in the human cerebral cortex in visual processing. Specifically, we found that the hubness of the occipital visual cortex increased in the early phase of visual processing, and that the hubness of the prefrontal cortex and temporal lobe increased in the late phase of visual processing. From the above, we found that our proposed method allows us to capture hierarchical changes in the hubness of interregional coordination.

A content recommendation system based on human brain activity has become a reality. However, the cost of collecting the information from people is problematic. This article proposes a scheme that resolves the tradeoff between the inference performance from a profile model to a brain model and the cost of collecting profile information. In the proposed scheme, a machine learning model infers the brain model from the profile model and a feature selection method is applied to reduce the cost, i.e., the number of questionnaire items, of collecting profile information. Since only the top questionnaire items with the highest importance scores are used, we can maintain the inference performance as high as possible while limiting the number of questionnaire items. We demonstrate the effectiveness of the proposed scheme with a performance evaluation using an experimentally obtained brain model and a profile model created from real profile information. The results over different experimental parameters, video lengths, and feature selection methods demonstrate that the proposed scheme successfully identifies the top questionnaire items that contribute most significantly to the inference of brain models.

The quantitative modeling of semantic representations in the brain plays a key role in understanding the neural basis of semantic processing. Previous studies have demonstrated that word vectors, which were originally developed for use in the field of natural language processing, provide a powerful tool for such quantitative modeling. However, whether semantic representations in the brain revealed by the word vector-based models actually capture our perception of semantic information remains unclear, as there has been no study explicitly examining the behavioral correlates of the modeled brain semantic representations. To address this issue, we compared the semantic structure of nouns and adjectives in the brain estimated from word vector-based brain models with that evaluated from human behavior. The brain models were constructed using voxelwise modeling to predict the functional magnetic resonance imaging (fMRI) response to natural movies from semantic contents in each movie scene through a word vector space. The semantic dissimilarity of brain word representations was then evaluated using the brain models. Meanwhile, data on human behavior reflecting the perception of semantic dissimilarity between words were collected in psychological experiments. We found a significant correlation between brain model- and behavior-derived semantic dissimilarities of words. This finding suggests that semantic representations in the brain modeled via word vectors appropriately capture our perception of word meanings.

The human brain can effectively learn a new task from a small number of samples, which indicates that the brain can transfer its prior knowledge to solve tasks in different domains. This function is analogous to transfer learning (TL) in the field of machine learning. TL uses a well-trained feature space in a specific task domain to improve performance in new tasks with insufficient training data. TL with rich feature representations, such as features of convolutional neural networks (CNNs), shows high generalization ability across different task domains. However, such TL is still insufficient in making machine learning attain generalization ability comparable to that of the human brain. To examine if the internal representation of the brain could be used to achieve more efficient TL, we introduce a method for TL mediated by human brains. Our method transforms feature representations of audiovisual inputs in CNNs into those in activation patterns of individual brains via their association learned ahead using measured brain responses. Then, to estimate labels reflecting human cognition and behavior induced by the audiovisual inputs, the transformed representations are used for TL. We demonstrate that our brain-mediated TL (BTL) shows higher performance in the label estimation than the standard TL. In addition, we illustrate that the estimations mediated by different brains vary from brain to brain, and the variability reflects the individual variability in perception. Thus, our BTL provides a framework to improve the generalization ability of machine-learning feature representations and enable machine learning to estimate human-like cognition and behavior, including individual variability.

This work introduces the spectrum sensing-based deep learning approach to overcome the wireless interference in the communication-based train control application. The fourlevel- SNR classification problem in this application is defined. Recently, several works applied the end-to-end learning approach using convolutional neural networks for spectrum sensing. However, the present work points out that the over-fitting problem easily occurs if only the limited frequency selective fading conditions of a data set are considered for the training process in the end-to-end learning approach on the multiple-SNR classification. This over-fitting problem cannot be solved simply by adding more frequency selective fading conditions into the training data set because there are many possible conditions in real communication transmission. This paper then proposes a new learning network, including a new input feature that has a strong relationship with the multiple-SNR classification problem. The evaluation results suggest that the proposed approach can solve such an over-fitting problem.

Quantitative modeling of human brain activity based on language representations has been actively studied in systems neuroscience. However, previous studies examined word-level representation, and little is known about whether we could recover structured sentences from brain activity. This study attempts to generate natural language descriptions of semantic contents from human brain activity evoked by visual stimuli. To effectively use a small amount of available brain activity data, our proposed method employs a pre-trained image-captioning network model using a deep learning framework. To apply brain activity to the image-captioning network, we train regression models that learn the relationship between brain activity and deep-layer image features. The results demonstrate that the proposed model can decode brain activity and generate descriptions using natural language sentences. We also conducted several experiments with data from different subsets of brain regions known to process visual stimuli. The results suggest that semantic information for sentence generations is widespread across the entire cortex.

In this study, we investigate whether sparse coding helps explain the semantic representation in human cerebral cortex. We show this by using sparse coding to model semantic representation in the cerebral cortex. We propose three methods for estimating semantic representation from brain activity data. For estimating a new semantic representation, in the first method, we use only a semantic representation dictionary obtained via sparse coding. The semantic representation estimated using this method is more similar to the actual semantic representation of the cerebral cortex than that estimated without sparse coding. In the second method, we use only a brain activity dictionary obtained via sparse coding. The semantic representation estimated using this method is also better than that estimated without sparse coding. In addition, in the third method, we estimate semantic representation by applying sparse coding to both semantic representation and brain activity data. The semantic representation estimated by using this third method is better than that estimated by the first or second methods. Through the above three experiments, we have confirmed that sparse coding helps explain the semantic representation in human cerebral cortex.