Abstract
In the proposed work machine learning algorithm is applied on Functional Magnetic Resonance Imaging (fMRI) data to analyze the human brain activity and then predicting the word that the subject was thinking. The algorithm that can learn to identify and track the cognitive processes and gives rise to predict the word from observed fMRI data is developed. The major problem here is that we have limited data in very high dimensional feature space. Thereby, making the model susceptible to overfit the data. Also, the data is highly noisy through most of the dimensions, leaving only a few features that are discriminative. Due to high noise domain shift problem is very likely to occur. Most of the previous approach focused only on feature selection and learning the embedding space. Here our main objective is to learn the robust embedding space and handling the domain shift problem [11] in an efficient way. Unlike the previous approach instead of learning the dictionary that projects the visual space to the word embedding space, we are using the joint dictionary learning approach based on the matrix factorization. Our experiment shows that the proposed approach based on the joint dictionary learning and domain adaptation method has the significant advantage over the previous approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Roland, J.L., Hacker, C.D., Breshears, J.D., Gaona, C.M., Hogan, R.E., Burton, H., Corbetta, M., Leuthardt, E.C.: Brain mapping in a patient with congenital blindness-a case for multimodal approaches. Front. Hum. Neurosci. 7, 431 (2013)
Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.-M., Malave, V.L., Mason, R.A., Just, M.A.: Predicting human brain activity associated with the meanings of nouns. Am. Assoc. Adv. Sci. 320(5880), 1191–1195 (2008)
Palatucci, M., Pomerleau, D., Hinton, G.E., Mitchell, T.M.: Zero-shot learning with semantic output codes. In: Advances in Neural Information Processing Systems, pp. 1410–1418 (2009)
Bruni, E., Tran, G.B., Baroni, M.: Distributional semantics from text and images. In: Proceedings of the GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics, pp. 22–32 (2011)
Silberer, C., Lapata, M.: Grounded models of semantic representation. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 2452–2460 (2012)
Murphy, B., Talukdar, P.P., Mitchell, T.: Learning effective and interpretable semantic models using non-negative sparse embedding. In: Association for Computational Linguistics (2012)
Murphy, B., Talukdar, P., Mitchell, T.: Selecting corpus-semantic models for neuro-linguistic decoding. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, pp. 114–123 (2012)
Silberer, C., Ferrari, V., Lapata, M.: Models of semantic representation with visual attributes. In: PACL, vol. 1, pp. 572–582 (2013)
Anderson, A.J., Bruni, E., Bordignon, U., Poesio, M., Baroni, M.: Of words, eyes and brains: correlating image-based distributional semantic models with neural representations of concepts. In: EMNLP, pp. 1960–1970 (2013)
Fyshe, A., Talukdar, P.P., Murphy, B., Mitchell, T.M.: Interpretable semantic vectors from a joint model of brain-and text-based meaning. In: Proceedings of the Conference on Association for Computational Linguistics, Meeting, p. 489 (2014)
Kodirov, E., Xiang, T., Fu, Z., Gong, S.: Unsupervised domain adaptation for zero-shot learning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1423–1433 (2015)
Zhang, Y., Yang, Q.: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 (2017)
Mairal, J.: SPAMS: A SPArse Modeling Software, v2. 3 (2012)
http://www.cs.cmu.edu/afs/cs/project/theo-73/www/science2008/data.html
Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. In: Chemometrics and Intelligent Laboratory Systems, pp. 37–52 (1987)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Mishra, A. (2018). Predicting Word from Brain Activity Using Joint Sparse Embedding with Domain Adaptation. In: Rameshan, R., Arora, C., Dutta Roy, S. (eds) Computer Vision, Pattern Recognition, Image Processing, and Graphics. NCVPRIPG 2017. Communications in Computer and Information Science, vol 841. Springer, Singapore. https://doi.org/10.1007/978-981-13-0020-2_48
Download citation
DOI: https://doi.org/10.1007/978-981-13-0020-2_48
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-0019-6
Online ISBN: 978-981-13-0020-2
eBook Packages: Computer ScienceComputer Science (R0)