Konstantina Margiotoudi and Friedemann Pulvermüller
Many studies have reported the matching of meaningless pseudowords to abstract shapes 1,2, a phenomenon know as sound symbolism3. However, few hypotheses tried to explain the mechanism behind this human-specific phenomenon4. One prominent hypothesis is the syneasthetic articulatory account5, where the authors explain that
the sharp edges of a shape mimic the sharp phonemic inflections and the sharp movement trajectory of the tongue on the palate when for example uttering the pseudoword “kiki”. However, looking at the kinematic trajectories of the tongue while uttering ‘sharp’ or ‘round’ sounding phonemes, we see that these trajectories do not resemble any abstract round or sharp visual shapes. Beyond our articulators, other body parts –such as our hands– are visible to the individuals and could provide a mechanistic explanation for sound symbolic processing. While performing a hand movement we learn via associative learning the auditory, visual and motor products of our hand actions. These sensorimotor representations of our hand actions, which are carried by brain’s action-perception networks6, could plausible explain why we map ‘round’ and ‘sharp’ sound symbolic pseudowords to round and sharp shapes.
Hence, we hypothesize that the audiovisual products of our actions provide a mechanistic explanation of sound symbolic mappings. Here we tested 24 healthy participants on a two-alternative forced choice task in two conditions. In the sound symbolic condition, participants had to match a ‘sharp’ or a ‘round’ pseudoword to a curved or edgy shape. In the action sound-shape condition, participants had to match ‘sharp’ or ‘round’ action sounds to curved or edgy action shapes. The action sounds and shapes were recorded, while the experimenter drew with a pen curved or edgy shapes on a piece of paper. Analysis on the acoustic properties of both ‘sharp’ and ‘round’ action sounds and pseudowords, revealed similarities between the two categories across conditions.
Moreover, the visual products of the actions resembled the edgy or curved shapes of the first condition. Participants performed significantly above chance on both mappings (i.e., sound symbolic and action sound-shape mappings). Notably, we found a significant positive correlation on the performance of the participants in the two conditions (see Fig. 1) . Good action sound-shape mappers were also good sound symbolic mappers. We suggest that the knowledge of our hand actions is the link between mapping speech sounds to figurative information, due to the physical similarities we observed between the audiovisual sound symbolic stimuli and the audiovisual products of our actions. This binding has special preconditions found at a neuronal level, with auditory-visual and motor systems linked in the brain via the branches of the arcuate fasciculus 7. To conclude, we propose that the mechanism behind sound symbolism is grounded in the audiovisual products of our hand actions. Figure 1. Bivariate scatterplot with regression line and correlation coefficient of Spearmean correlation between sound symbolic and action sound-shape performance.
between sound symbolic and action sound-shape performance.
References
1) Lockwood, G., & Dingemanse, M. (2015). Iconicity in the lab: A review of behavioral, developmental, and neuroimaging research into sound-symbolism. Frontiers in psychology, 6, 1246.
2) Fort, M., Lammertink, I., Peperkamp, S., Guevara-Rukoz, A., Fikkert, P., & Tsuji, S. (2018). Symbouki: a meta-analysis on the emergence of sound symbolism in early language acquisition. Developmental science, 21(5), e12659.
3) Köhler, W. (1929). Gestalt psychology. New York, USA: Liveright 4) Margiotoudi, K., Allritz, M., Bohn, M., & Pulvermüller, F. (2019). Sound symbolic congruency detection in humans but not in great apes. Scientific reports, 9(1), 1-12.
5) Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia–a window into perception, thought and language. Journal of consciousness studies, 8(12), 3-34.
6) Pulvermüller, F. (2018). Neural reuse of action perception circuits for language, concepts and communication. Progress in neurobiology, 160, 1-44.
7) Rilling, J. K., Glasser, M. F., Preuss, T. M., Ma, X., Zhao, T., Hu, X., & Behrens, T. E. (2008). The evolution of the arcuate fasciculus revealed with comparative DTI. Nature neuroscience, 11(4), 426.