Results: Japanese Words Quiz

A few weeks ago, I invited you to take part in an online survey for which I provided very little background. Your job was to determine which of two English words was the appropriate translation of a Japanese word. Since most people in the Western world have very little knowledge of the Japanese language (aside from "konnichiwa" and "sayonara") you were probably just guessing. But, is there anything systematic about people's guessing in an experiment like this?

What you didn't know while taking the quiz was that the appropriate translation was not always provided as one of the answers. Sometimes the Japanese word was paired with its actual translation and an unrelated item. Other times, it was paired with the opposite of its actual translation and an unrelated item (so, the Japanese word for "bright" was paired with "dark"). A recent experiment in press at the journal Cognition used a similar design. Nygaard, Cook, and Namy (in press) taught English-speaking participants the translations of a series of Japanese words that were presented auditorily, but unbeknownst to the participants, they were either learning the direct translation of the word, it's antonym, or an unrelated translation.

Nygaard et al. found that participants learned the items more efficiently (evidenced by accuracy and response time in a recognition task) if they were learning the true meaning or its antonym than if they were learning an unrelated translation.

from Nygaard et al. (in press)

Languages are thought to be abstract and arbitrary, with no systematicity between word sounds and word meanings (aside from onomatopoetic words). The authors argue that this experiment suggests a different story, that there is some sort of underlying sound to meaning correspondence that is systematic across language systems, as none of the participants in this study had even a rudimentary knowledge of Japanese. The antonyms were thought to benefit learning since they were from the same semantic category as the paired Japanese item.

I find this conclusion highly suspect. The authors point out a possible confound that I think is at the heart of the effect being seen here. During the learning phase of the experiment, the Japanese tokens were presented auditorily, paired with a written English translation on the computer screen. The spoken Japanese items were, of course, provided by a native Japanese speaker, meaning that the speaker knew the actual meanings of the words they were saying. Spoken language is a much richer source of information than the printed word. From a speaker's voice, we can ascertain things like their gender and age as well as gain information that can aid in the understanding of the words they're saying. In fact, the authors cite research demonstrating that listeners can intuit word meanings based only on voice inflections (or prosody).

In the "Japanese Words Quiz," I wanted to determine whether people could guess the meanings of the Japanese words without any sort of auditory input. Granted, this is a much different experiment from Nygaard et al. and I didn't have a fully-counterbalanced design, but if there is any systematic phonetic (or orthographic) to semantic relationship that readers can pick up on across languages, then it would be quite possible that they could guess the meanings above chance levels.

I had planned to carry out a full statistical analysis of this data, but looking at the raw numbers tells me that further analysis is not necessary. The results were almost perfectly at chance, with an overall accuracy of 51%. Accuracy on matched trials (where the actual translation was one of the possible choices) was 52%, and accuracy on antonym trials was 50% exactly. That's not a promising outcome.

I'm thinking of following up on this experiment with a formal replication using spoken word tokens generated by someone who doesn't speak Japanese but who has training in Japanese pronunciation...Someone like a vocal performance major. Thus, they would be unequipped to provide prosody that would be informative to the true meaning of the word. Could be fun! Stay tuned.

References:

Nygaard, L. C., Cook, A. E., & Namy, L. L. (in press). Sound to meaning correspondences facilitate word learning. Cognition. (link)