Developing a multilayered structure of adjective words that explains semantic relationships between human perceptual and affective responses to stimuli is instrumental in the design of affective aspects of products. However, the determination of multilayered structure is demanding and, thus far, it has been conducted by experienced developers in a trial-and-error manner. This study developed a method to systematically establish such structures through common tasks for sensory evaluation where products are rated along adjective labels. This method gradually expands the model from a simple two-layered to complex multilayered structures until it is accepted by structural equation modeling. The lower and higher layers of the initial two-layered model are composed of sensory and affective adjectives, respectively. The parts with weak fit indices of the higher layer are then remodeled, resulting in a multilayered affective structure. To validate the method, we built adjective structures based on responses to touching plastic plates. The method resulted in three- and four-layered structures that were quantitatively and semantically valid.