Mind reading by computer

The ultimate inverse problem

July 1, 2017 — March 2, 2021

life
machine learning
mind
neuron
photon choreography
statistics

Suspiciously similar content

Figure 1

A placeholder.

I’d like to know how good the results are getting in this area, and how general across people/technologies etc. How close are we to the point that someone can put an arbitrary individual in some kind of tomography machine and say what they are thinking without pre-training or priming?

1 Base level: brain imaging

The instruments we have are blunt. Consider, could a neuroscientist even understand a microprocessor? () What hope is there of brains?

TODO: discuss the infamously limp state of fMRI inference, problem of multiple testing in correlated fields etc.

TBC.

2 Advanced: brain decoding

Assuming you can get information out of your instruments, can you decode something meaningful. Marcel Just et al do a lot of this. It for sure leads to fun press releases, e.g. CMU Scientists Harness “Mind Reading” Technology to Decode Complex Thoughts but I need time to see details to understand how much progress they are making towards the science-fiction version()

Researchers watch video images people are seeing decoded from their fMRI brain scans in near-real-time. If you want to have a crack at this yourself, you might check out Katja Seeliger’s mind reading datasets.

More intrusively, in rats… Real-time readouts of location memory:

by recording the electrical activity of groups of neurons in key areas of the brain they could read a rat’s thoughts of where it was, both after it actually ran the maze and also later when it would dream of running the maze in its sleep

3 References

Boettiger. 2015. “An Introduction to Docker for Reproducible Research, with Examples from the R Environment.” ACM SIGOPS Operating Systems Review.
Cox, and Rogers. 2021. “Finding Distributed Needles in Neural Haystacks.” Journal of Neuroscience.
Davidson, Kloosterman, and Wilson. 2009. “Hippocampal Replay of Extended Experience.” Neuron.
Davis III, Spapé, and Ruotsalo. 2021. “Collaborative Filtering with Preferences Inferred from Brain Signals.” In Proceedings of the Web Conference 2021. WWW ’21.
Jonas, and Kording. 2017. “Could a Neuroscientist Understand a Microprocessor?” PLOS Computational Biology.
Le, Ambrogioni, Seeliger, et al. 2021. “Brain2Pix: Fully Convolutional Naturalistic Video Reconstruction from Brain Activity.” bioRxiv.
Miyawaki, Uchida, Yamashita, et al. 2008. “Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders.” Neuron.
Nishimoto, Vu, Naselaris, et al. 2011. “Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies.” Current Biology.
Shen, Horikawa, Majima, et al. 2017. “Deep Image Reconstruction from Human Brain Activity.” bioRxiv.
Wang, Cherkassky, and Just. 2017. “Predicting the Brain Activation Pattern Associated with the Propositional Content of a Sentence: Modeling Neural Representations of Events and States: Modeling Neural Representations of Events and States.” Human Brain Mapping.