From words to sentences and back: characterizing context-dependent meaning representations in the brain

Abstract

How do people understand concepts such as olive oil, baby oil, lamp oil,or oil paint? Embodied approaches to knowledge representation suggest that words are represented as a set of features that are the basic components of meaning. In particular, Binder et al. (2009) grounded this idea by mapping semantic features (attributes) to different brain systems in their Concept Attribute Representations (CAR) theory. Their fMRIexperiments demonstrated that when humans listen or read sentences,they use different brain systems to simulate seeing the scenes and performing the actions that are described. An intriguing challenge to this theoryis that concepts are dynamic, i.e.,word meaning depends on context. This dissertation addresses this challenge through the Context-dEpendent meaning REpresentations in the BRAin (CEREBRA) neural network model. Based on changes in the fMRI patterns, CEREBRA quantifies how word meanings change in the context of a sentence. CEREBRA was evaluated through three different computational experiments and through behavioral analysis. The experiments demonstrated that words in different contexts have different representations, that the changes observed in the concept attributes encode unique conceptual combinations, and that the new representations are more similar to the other words in the sentence than to the original representations. The behavioral analysis confirmed that the changes produced by CEREBRA are actionable knowledge that can be used to predict human responses. Together, these experiments constitute a comprehensive evaluation of CEREBRA’s context-based representations as well as the soundness of CAR theory. Thus, CEREBRA is a useful tool for understanding how semantic knowledge is represented in the brain, and for providinga human-like context-based representations for NLP applications