Cross-modal correspondences in sine wave: Speech versus non-speech modes
- 22 Downloads
The present study aimed to investigate whether or not the so-called “bouba-kiki” effect is mediated by speech-specific representations. Sine-wave versions of naturally produced pseudowords were used as auditory stimuli in an implicit association task (IAT) and an explicit cross-modal matching (CMM) task to examine cross-modal shape-sound correspondences. A group of participants trained to hear the sine-wave stimuli as speech was compared to a group that heard them as non-speech sounds. Sound-shape correspondence effects were observed in both groups and tasks, indicating that speech-specific processing is not fundamental to the “bouba-kiki” phenomenon. Effects were similar across groups in the IAT, while in the CMM task the speech-mode group showed a stronger effect compared with the non-speech group. This indicates that, while both tasks reflect auditory-visual associations, only the CMM task is additionally sensitive to associations involving speech-specific representations.
KeywordsCross-modal correspondences Sound symbolism Bouba-kiki effect Sine-wave speech
Both authors were supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) - Brazil. The authors also gratefully acknowledge Professors Barry C. Smith, Sarah Garfinkel, Vincent Hayward, and André J. Abath, as well as anonymous reviewers for thoughtful comments and suggestions.
Compliance with ethical standards
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Open practices statement
The data used in this research were made available to reviewers and to journal editors.
- Adeli, M., Rouat, J., & Molotchnikoff, S. (2014). Audiovisual correspondence between musical timbre and visual shapes. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00352
- Adler, D. (2019). vioplot: Violin Plot (Version 0.3.0). Retrieved from https://CRAN.R-project.org/package=vioplot
- Baart, M., Stekelenburg, J. J., & Vroomen, J. (2014). Electrophysiological evidence for speech-specific audiovisual integration. Neuropsychologia, 53, 115-121. https://doi.org/10.1016/j.neuropsychologia.2013.11.011 Google Scholar
- Boersma, P., & Weenink, D. (2013). Praat: Doing phonetics by computer (Version 5.3. 41)[Software].Google Scholar
- Bremner, A., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K. and Spence, C. (2013). Bouba and Kiki in Namibia? A remote culture makes similar shape–sound matches, but different shape–taste matches to Westerners. Cognition 126, 165–172. https://doi.org/10.1016/j.cognition.2012.09.007 Google Scholar
- Darwin, C. (2003). Sine-wave speech produced automatically using a script for the PRAAT program. http://www.lifesci.sussex.ac.uk/home/Chris_Darwin/SWS/ (Last viewed July, 2017)
- Fort, M., Weiß, A., Matin, A., & Peperkamp, S. (2013). Looking for the bouba-kiki effect in prelexical infants. In S. Ouni, F. Berthommier, & A. Jesse (Eds.), Proceedings of the 12th international conference on auditory-visual speech processing (pp. 71–76). Annecy France: Inria.Google Scholar
- Godoy, M., Duarte, A. C. V., Silva, F. L. F., Albano, G. F., Souza, R. J. P., & Silva, Y. U. A. P. M. (2018). The replication of takete-maluma effect in Brazilian Portuguese. Revista do GELNE, 20(1), 87–100. https://doi.org/10.21680/1517-7874.2018v20n1ID13331 Google Scholar
- Kohler, W. (1947). Gestalt psychology (1929). New York, NY: Liveright.Google Scholar
- Liew, K., Lindborg, P., Rodrigues, R., & Styles, S. J. (2018). Cross-modal perception of noise-in-music: Audiences generate spiky shapes in response to auditory roughness in a novel electroacoustic concert setting. Frontiers in Psychology, 9. https://doi.org/10.3389/fpsyg.2018.00178
- Lockwood, G., & Dingemanse, M. (2015). Iconicity in the lab: A review of behavioral, developmental, and neuroimaging research into sound-symbolism. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01246
- McCormick, K., Kim, J., List, S., & Nygaard, L. C. (2015). Sound to Meaning Mappings in the Bouba-Kiki Effect. In Noelle, D. C., Dale, R., Warlaumont, A. S., Yoshimi, J., Matlock, T., Jennings, C. D., & Maglio, P. P. (Eds.). Proceedings of the 37th annual meeting of the cognitive science society. Austin, TX: Cognitive Science Society.Google Scholar
- Monahan, P. J. (2018). Phonological knowledge and speech comprehension. Annual Review of Linguistics, 4(1), 21–47. https://doi.org/10.1146/annurev-linguistics-011817-045537 Google Scholar
- Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia--A window into perception, thought and language. Journal of Consciousness Studies, 8(12), 3-34.Google Scholar
- Remez, R. E., Rubin, P. E., & Pisoni, D. B. (1983). Coding of the speech spectrum in three time-varying sinusoids. Annals of the New York Academy of Sciences, 405(1), 485–489. https://doi.org/10.1111/j.1749-6632.1983.tb31663.x Google Scholar
- Shang, N., & Styles, S. J. (2017). Is a high tone pointy? Speakers of different languages match Mandarin Chinese tones to visual shapes differently. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.02139
- Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New York, NY: Springer-Verlag.Google Scholar
- Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011). The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 143-146). ACM.Google Scholar