Learning Speech Through Touch

Our sensory organs are exquisitely designed for bringing sensory information about our world to our brain. However, it is our brain’s ability to process this information that allow us to actually perceive the world. The goal of sensory substitution is to use one sensory system to provide information to the brain that is usually delivered via another sense. For example, braille conveys information about visual words through touch. In this project, we investigated the neural bases of learning to associate vibrotactile stimuli as spoken words. We use fMRI and representational similarity analysis (RSA) to test under what connections trained vibrotactile stimuli engage auditory word representations in the brain. Results can provide further evidence about interactions between sensory systems and principles of multisensory learning.

Re-analyzing open-science datasets for speech processing

This project is part of a broad project to test hypotheses about multimodal speech perception using a large open-science dataset (MOUS – “Mother of all unification studies”, Schoffelen et al., 2019). We are exploring how advanced neuroimaging analysis methods, including multivariate pattern analysis and functional connectivity analyses, can be used to investigate novel questions beyond those envisioned in the original study design.

On the Specificity of Descending Intracortical Connections

Descending connections are believed to be involved in sensory, motor and intellectual processes including selective attention, object recognition, disambiguation, mental imagery, preparation for movement, visuo-motor coordination, visual surround suppression, efference copy and corollary discharge, and a variety of Gestalt phenomena.

What are the specificities of descending connections, and how do they arise? Answers will be complicated by the multiplicity of targets of descending connections onto pyramidal cells and inhibitory interneurons in superficial and deep layers of the cortex and by the multiplicity of sources of descending connections from both superficial and deep pyramidal cells in higher cortical areas. This project involves a review of the specificities of descending intracortical connections and how those specificities are acquired.

Investigating the Neural Bases of Internal Models During Speech Production

Language is a uniquely sophisticated ability of the human brain that is critical for communication with the outside world. Spoken language processing can be broken down into two major components: speech perception and speech production. Both components have traditionally been studied separately, although it is obvious that they are intimately intertwined. Speech perception deals with the process of extracting meaning from heard speech (“sound-to-meaning”), whereas speech production orchestrates the activity of large sets of muscles to generate meaningful words and phrases (“meaning-to-articulation”). The neural bases of these processes, in particular the brain’s ability to create complex sound sequences for speech production, are still poorly understood. Recent major models share an overall “two-stream” architecture, with one processing pathway mediating speech perception, and another pathway participating in speech production. However, these models still have major disagreements regarding which brain regions are involved and how the two systems (perception and production) interact, and these disagreements are barriers to developing improved treatments of disorders involving speech perception and production deficits.

We use multivariate imaging techniques, specifically functional magnetic resonance (fMRI) representational similarity analysis (RSA), fMRI-RSA, and electroencephalography representational similarity analysis (EEG-RSA), to test novel, model-based hypotheses regarding the spatial location and temporal dynamics of neural representations for speech production and speech perception.

If you're interested in learning more, shoot Plamen an email at pn243 at Georgetown dot edu!

Next-Generation Machine Learning

There is growing realization that current deep learning algorithms, their remarkable successes notwithstanding, suffer from fundamental flaws. For instance, backpropagation-based deep learning, in contrast to human learning, requires large amounts of labeled data, and is ill-suited for incremental learning. In addition, as already pointed out more than 30 years ago by Francis Crick, backprop is implausible as a model of learning in the brain. Together with our collaborators at Lawrence Livermore National Labs, we are analyzing EEG data obtained while human participants are learning novel tasks to understand how supervised learning is accomplished in the brain's deep processing hierarchies.