Ying Yanga, Jing Wanga, Cyntia Bailerb, Vladimir
Cherkasskya, Marcel Adam Justa, Commonality
of neural representations of sentences across languages: Predicting brain
activation during Portuguese sentence comprehension using an English-based
model of brain function. NeuroImage Available online 19 October 2016
Highlights
•
An English-based model predicted the fMRI patterns for
Portuguese sentences.
•
A cross-language model predicted both monolinguals’ and
bilinguals’ activation.
•
Sentences from two languages were decodable from the same
set of 38 brain locations.
Abstract
The aim of the study was to test the cross-language
generative capability of a model that predicts neural activation patterns
evoked by sentence reading, based on a semantic characterization of the
sentence. In a previous study on English monolingual speakers (Wang et al.,
submitted), a computational model performed a mapping from a set of 42
concept-level semantic features (Neurally Plausible Semantic Features, NPSFs)
as well as 6 thematic role markers to neural activation patterns (assessed with
fMRI), to predict activation levels in a network of brain locations. The model
used two types of information gained from the English-based fMRI data to
predict the activation for individual sentences in Portuguese. First, it used
the mapping weights from NPSFs to voxel activation levels derived from the
model for English reading. Second, the brain locations for which the activation
levels were predicted were derived from a factor analysis of the brain
activation patterns during English reading. These meta-language locations were
defined by the clusters of voxels with high loadings on each of the four main
dimensions (factors), namely people, places, actions and feelings, underlying
the neural representations of the stimulus sentences.
This cross-language model succeeded in predicting the brain
activation patterns associated with the reading of 60 individual Portuguese
sentences that were entirely new to the model, attaining accuracies reliably
above chance level. The prediction accuracy was not affected by whether the
Portuguese speaker was monolingual or Portuguese-English bilingual. The model's
confusion errors indicated an accurate capture of the events or states
described in the sentence at a conceptual level. Overall, the cross-language
predictive capability of the model demonstrates the neural commonality between
speakers of different languages in the representations of everyday events and
states, and provides an initial characterization of the common meta-language
neural basis.
________________________________________________________________________
When the brain “reads” or decodes a sentence in English or
Portuguese, its neural activation patterns are the same, researchers report.
Published in NeuroImage, the study is the first to show that
different languages have similar neural signatures for describing events and
scenes. By using a machine-learning algorithm, the research team was able to
understand the relationship between sentence meaning and brain activation
patterns in English and then recognize sentence meaning based on activation
patterns in Portuguese.
The findings can be used to improve machine translation,
brain decoding across languages, and, potentially, second language instruction.
“This tells us that, for the most part, the language we
happen to learn to speak does not change the organization of the brain,” says
Marcel Just, professor of psychology at Carnegie Mellon University.
“Semantic information is represented in the same place in
the brain and the same pattern of intensities for everyone. Knowing this means
that brain to brain or brain to computer interfaces can probably be the same
for speakers of all languages,” Just says…
With 67 percent accuracy, the model predicted which
sentences were read in Portuguese. The resulting brain images showed that the
activation patterns for the 60 sentences were in the same brain locations and
at similar intensity levels for both English and Portuguese sentences.
Bilingual babies know when the rules don’t apply
Additionally, the results revealed the activation patterns
could be grouped into four semantic categories, depending on the sentence’s
focus: people, places, actions, and feelings. The groupings were very similar
across languages, reinforcing the organization of information in the brain is
the same regardless of the language in which it is expressed.
“The cross-language prediction model captured the conceptual
gist of the described event or state in the sentences, rather than depending on
particular language idiosyncrasies. It demonstrated a meta-language prediction
capability from neural signals across people, languages, and bilingual status,”
says Ying Yang, a postdoctoral associate in psychology and first author of the
study.
http://www.futurity.org/brain-language-portuguese-1290222-2/
No comments:
Post a Comment