As large language models (LLMs) advance in their linguistic capacity, understanding how they capture aspects of language competence remains a significant challenge. This study therefore employs psycholinguistic paradigms, which are well-suited for probing deeper cognitive aspects of language processing, to explore neuron-level representations in language model across three tasks: sound-shape association, sound-gender association, and implicit causality. Our findings indicate that while GPT-2-XL struggles with the sound-shape task, it demonstrates human-like abilities in both sound-gender association and implicit causality. Targeted neuron ablation and activation manipulation reveal a crucial relationship: When GPT-2-XL displays a linguistic ability, specific neurons correspond to that competence; conversely, the absence of such an ability indicates a lack of specialized neurons. This study is the first to utilize psycholinguistic experiments to investigate deep language competence at the neuron level, providing a new level of granularity in model interpretability and insights into the internal mechanisms driving language ability in the transformer-based LLM.

New medical treatment development requires multiple phases of clinical trials. Recent literature indicates that the design of the trial protocols significantly contributes to trial performance. We investigated Clinical Trial Outcome Prediction (CTOP) using trial design documents to predict trial phase transitions automatically. We propose CTP-LLM, the first Large Language Model (LLM) based model for CTOP. We also introduce the PhaseTransition (PT) Dataset; which labels trials based on their progression through the regulatory process and serves as a benchmark for CTOP evaluation. Our fine-tuned GPT-3.5-based model (CTP-LLM) predicts clinical trial phase transition by analyzing the trial’s original protocol texts without requiring human-selected features. CTP-LLM achieves a 67% accuracy rate in predicting trial phase transitions across all phases and a 75% accuracy rate specifically in predicting the transition from Phase III to final approval. Our experimental performance highlights the potential of LLM-powered applications in forecasting clinical trial outcomes and assessing trial design.

We can visually discriminate and recognize a wide range of materials. Meanwhile, we use language to describe what we see and communicate relevant information about the materials. Here, we investigate the relationship between visual judgment and language expression to understand how visual features relate to semantic representations in human cognition. We use deep generative models to generate images of realistic materials. Interpolating between the generative models enables us to systematically create material appearances in both well-defined and ambiguous categories. Using these stimuli, we compared the representations of materials from two behavioral tasks: visual material similarity judgments and free-form verbal descriptions. Our findings reveal a moderate but significant correlation between vision and language on a categorical level. However, analyzing the representations with an unsupervised alignment method, we discover structural differences that arise at the image-to-image level, especially among ambiguous materials morphed between known categories. Moreover, visual judgments exhibit more individual differences compared to verbal descriptions. Our results show that while verbal descriptions capture material qualities on the coarse level, they may not fully convey the visual nuances of material appearances. Analyzing the image representation of materials obtained from various pre-trained deep neural networks, we find that similarity structures in human visual judgments align more closely with those of the vision-language models than purely vision-based models. Our work illustrates the need to consider the vision-language relationship in building a comprehensive model for material perception. Moreover, we propose a novel framework for evaluating the alignment and misalignment between representations from different modalities, leveraging information from human behaviors and computational models.

Few-shot text classification has extensive application where the sample collection is expensive or complicated. When the penalty for classification errors is high, such as early threat event detection with scarce data, we expect to know "whether we should trust the classification results or reexamine them.'' This paper investigates the Uncertainty Estimation for Few-shot Text Classification (UEFTC), an unexplored research area. Given limited samples, a UEFTC model predicts an uncertainty score for a classification result, which is the likelihood that the classification result is false. However, many traditional uncertainty estimation models in text classification are unsuitable for implementing a UEFTC model. These models require numerous training samples, whereas the few-shot setting in UEFTC only provides a few or just one support sample for each class in an episode. We propose Contrastive Learning from Uncertainty Relations (CLUR) to address UEFTC. CLUR can be trained with only one support sample for each class with the help of pseudo uncertainty scores. Unlike previous works that manually set the pseudo uncertainty scores, CLUR self-adaptively learns them using our proposed uncertainty relations. Specifically, we explore four model structures in CLUR to investigate the performance of three common-used contrastive learning components in UEFTC and find that two of the components are effective. Experiment results prove that CLUR outperforms six baselines on four datasets, including an improvement of 4.52% AUPR on an RCV1 dataset in a 5-way 1-shot setting. Our code and data split for UEFTC are in https://github.com/he159ok/CLUR_UncertaintyEst_FewShot_TextCls.

Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Importantly, without supervision, human-understandable scene attributes, including the object's shape, material, and body color, spontaneously emerge in the model's layer-wise latent space in a scale-specific manner. By embedding an image into the learned latent space, we can manipulate specific layers' latent code to modify the appearance of the object in the image. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object's shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object's shape or body color. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts.

Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao, ChangTien Lu. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

We use force-feedback device and a game engine to measure the effects of material appearance on the perception of mass of virtual objects. We discover that the perceived mass is mainly determined by the ground-truth mass output by the force-feedback device. Different from the classic Material Weight Illusion (MWI), however, heavy-looking objects (e.g. steel) are consistently rated heavier than light-looking ones (e.g. fabric) with the same ground-truth mass. Analysis of the initial accelerated velocity of the movement trajectories of the virtual probe shows greater acceleration for materials with heavier rated mass. This effect is diminished when the participants lift the object for the second time, meaning that the influence of visual appearance disappears in the movement trajectories once it is calibrated by the force-feedback. We also show how the material categories are affected by both the visual appearance and the weight of the object. We conclude that visual appearance has a significant interaction with haptic force-feedback on the perception of mass and also affects the kinematics of how participants manipulate the object.

Deformable objects such as cloth exhibit their mechanical properties (e.g. stiffness) through shape deformation over time under external forces. Mechanical properties are important because they tell us the affordance of the object and helps us predict what type of action can be done upon it. Previous research shows that motion statistics can be used to develop computer vision algorithms to estimate mechanical properties of cloth under an unknown wind force. It is unclear what motion cues human use to estimate mechanical properties. Estimating mechanical properties is difficult because both the intrinsic properties of the fabric and the external force contribute to the apparent motion of the fabric. However, in order to achieve invariant material perception, the visual system needs to discount the effects of external force. In this paper, we investigate whether humans have an invariant representation of mechanical properties of fabrics under varying external forces in dynamic scenes. Then we study what visual cues allow humans to achieve this perceptual constancy. The stimuli are animated videos containing a hanging fabric moving under oscillating wind. We vary both intrinsic mechanical properties such as mass and stiffness of the cloth as well as the strength of the wind force. We discuss our results in the context of optical flow statistics. This advances the current understanding of the role of motion in perception of material properties in dynamic scenes.

In information systems (IS) literature, there is ongoing debate as to whether the field has become fragmented and lost its identity in response to the rapid changes of the field. The paper contributes to this discussion by providing quantitative measurement of the fragmentation or cohesiveness level of the field. A co-word analysis approach aiding in visualization of the intellectual map of IS is applied through application of clustering analysis, network maps, strategic diagram techniques, and graph theory for a collection of 47,467 keywords from 9551 articles, published in 10 major IS journals and the proceedings of two leading IS conferences over a span of 20 years, 1993 through 2012. The study identified the popular, core, and bridging topics of IS research for the periods 1993–2002 and 2003–2012. Its results show that research topics and subfields underwent substantial change between those two periods and the field became more concrete and cohesive, increasing in density. Findings from this study suggest that the evolution of the research topics and themes in the IS field should be seen as part of the natural metabolism of the field, rather than a process of fragmentation or disintegration.

This study employs hierarchical cluster analysis, strategic diagrams and network analysis to map and visualize the intellectual landscape of the CHI conference on Human Computer Interaction through the use of co-word analysis. The study quantifies and describes the thematic evolution of the field based on a total of 3152 CHI articles and their associated 16035 keywords published between 1994 and 2013. The analysis is conducted for two time periods (1994-2003, 2004-2013) and a comparison between them highlights the underlying trends in our community. More significantly, this study identifies the evolution of major themes in the discipline, and highlights individual topics as popular, core, or backbone research topics within HCI.

Passively estimating the intrinsic material properties of deformable objects moving in a natural environment is essential for scene understanding. We present a framework to automatically analyze videos of fabrics moving under various unknown wind forces, and recover two key material properties of the fabric: stiffness and area weight. We extend features previously developed to compactly represent static image textures to describe video textures, such as fabric motion. A discriminatively trained regression model is then used to predict the physical properties of fabric from these features. The success of our model is demonstrated on a new, publicly available database of fabric videos with corresponding measured ground truth material properties. We show that our predictions are well correlated with ground truth measurements of stiffness and density for the fabrics. Our contributions include: (a) a database that can be used for training and testing algorithms for passively predicting fabric properties from video, (b) an algorithm for predicting the material properties of fabric from a video, and (c) a perceptual study of humans' ability to estimate the material properties of fabric from videos and images.

Multiple scattering contributes critically to the characteristic translucent appearance of food, liquids, skin, and crystals; but little is known about how it is perceived by human observers. This article explores the perception of translucency by studying the image effects of variations in one factor of multiple scattering: the phase function. We consider an expanded space of phase functions created by linear combinations of Henyey-Greenstein and von Mises-Fisher lobes, and we study this physical parameter space using computational data analysis and psychophysics. Our study identifies a two-dimensional embedding of the physical scattering parameters in a perceptually meaningful appearance space. Through our analysis of this space, we find uniform parameterizations of its two axes by analytical expressions of moments of the phase function, and provide an intuitive characterization of the visual effects that can be achieved at different parts of it. We show that our expansion of the space of phase functions enlarges the range of achievable translucent appearance compared to traditional single-parameter phase function models. Our findings highlight the important role phase function can have in controlling translucent appearance, and provide tools for manipulating its effect in material design applications.