Researchers Use Brain Scans to Translate Thoughts Into Words

If a picture is worth 1,000 words, how many words make up a thought?

By Rebecca Boyle

Word Associations Color-coded figures illustrate the probability of words within a Wikipedia article about the object actually being associated with the object. The more red a word is, the more likely a person is to associate it, in this case, with "cow." On the other hand, bright blue suggests a strong correlation with "carrot." Black and grey "neutral" words had no specific association or were not considered at all.Francisco Pereira/via Princeton University

A new study that matches words with brain activity patterns could help neuroscientists understand how people think about abstract, complex concepts, researchers say. It lends aphysiological definition to the concept of higher thinking, using functional magnetic resonance imaging and a computer program that condensed 3,500 Wikipedia articles.

Researchers at Princeton University looked at fMRI scans to identify brain regions that were activated when participants thought about certain objects, like a carrot or a house. Then the team generated a list of topics that were also associated with those words. They looked at the same fMRI scans to determine the brain activity that was shared by the words within each topic, as aPrinceton news release puts it. For instance, thoughts about the idea of “furniture” shared similar patterns with words like “table,” “desk” and “chair.”

Once they could tell the fMRI activity that would be sparked by a particular topic, the researchers were able to look at the brain activity alone and extrapolate what the person was thinking about. If a scientist studying brain scans spotted the neural patterns corresponding to “chair,” he could tell that the person was thinking about furniture.

“Whatever subject matter is on someone's mind — not just topics or concepts, but also emotions, plans or socially oriented thoughts — is ultimately reflected in the pattern of activity across all areas of his or her brain," said the team’s senior researcher, Matthew Botvinick, an associate professor in Princeton's Department of Psychology and in the Princeton Neuroscience Institute.

The researchers started out with brain scans from a 2008 word association study, in which participants were shown a picture and a word of five objects in 12 categories. The participants had been asked to visualize the object for three seconds, and the fMRI recorded their neural activity. Then the Princeton researchers came up with a list of their own topics with which to characterize this fMRI data. They used a computer program to condense 3,500 Wikipedia articles about objects — like an airplane, heroin, birds and manual transmission. The program came up with 40 topics to which these things could relate — i.e. aviation, drugs, animals or machinery. (Their full paper is available online for those interested in the specific methods.)

They arranged the fMRI scans by subject matter, and were ultimately able to tell the general topic on a person’s mind. It was harder to pick out an individual object, however, the Princeton news office explains. The eventual goal is to translate brain activity patterns into the correct words to fully describe thoughts, the researchers say.

This could have applications for helping people with disabilities, for whom brain scans might be able to elucidate their thinking more effectively than pictures. The research appears in the journalFrontiers in Human Neuroscience.