Tom M. Mitchell, Svetlana V. Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L. Malave, Robert A. Mason, Marcel Adam Just
Predicting Human Brain Activity Associated with the Meanings of Nouns

• authors present a computational model, which predicts the functional magnetic resonance imaging (fMRI) of neural activation associated with words for which no fMRI data are available
• fMRI prediction for a word w is a two-step process:
  1. compute a vector of semantic features over a huge corpus
     • 25 features are defined in terms of co-occurrence of w with forms of 25 manually selected sensory-motor verbs
  2. predict neural fMRI activation as a weighted sum of semantic features
     • weights for every voxel (3D pixel) and feature are estimated using multiple regression
• fMRI data
  • they created 60 representative fMRI images
    • word - picture combination from 12 semantic categories
    • they measured brain activation of 9 participants after being exposed to all 60 word - picture combinations
• evaluation
  • they carried out “leave-two-out” cross validation with all 60 word-fMRI instances
    • 58 of them served as a training data
    • for two of them fMRI images were predicted and compared with observed images. On the basis of cosine similarity measure a matching was determined. If the predicted image for the first word matches with its corresponding observed image, one positive point is scored - aggregated over all folds, it forms an accuracy measure

• authors selected 25 sensory-motor verbs as a basis for their co-occurence features. But they did not sufficiently explain what led them to pick exactly these ones.

Written by Michal Novák