Biological basis of language

January 11, 2018 — September 16, 2021

grammar
language
machine learning
NLP
stringology
Figure 1

1 Neurology of langauge

Dan Stowell summarises a neural basis for recursive syntax:

For decades, Noam Chomsky and colleagues have famously been developing and advocating a “minimalist” (Bolhuis et al. 2014) idea about the machinery our brain uses to process language. […] They propose that not much machinery is needed, and one of the key components is a “merge” operation that the brain uses in composing and decomposing grammatical structures.

Then yesterday I was reading this introduction to embeddings in artificial neural network and NLP, and I read the following:

“Models like [this] are powerful, but they have an unfortunate limitation: they can only have a fixed number of inputs. We can overcome this by adding an association module, A, which will take two word or phrase representations and merge them.” (Bottou 2011)

2 Analogy with artificial neural networks

TBD

3 Evolution of language

TBD

4 Computational plausibility

See syntax.

5 Meaning

See semantics.

6 Scrapbook

“They’re using phrase-structure grammar, long-distance dependencies. FLN recursion, at least four levels deep and I see no reason why it won’t go deeper with continued contact. […] It doesn’t have a clue what I’m saying.”

“What?”

“It doesn’t even have a clue what it’s saying back,” she added.

Peter Watts, Blindsight

Sam Kriss calls the spamularity the language of god:

What is machine language? Firstly, machine language is vampiric, shamanic, xenophagic, mocking. It’s a changeling. Often it tries to imitate human discourse; the machine wants you to think that it’s human. This is the first level of deception. Often this isn’t enough: machines will use various methods to take over other text-producing systems, so that without your knowledge you end up advertising weight loss pills to all your old school friends. First axiom: all language has the potential to become machine language. To become infected. 10 Award-Winning GIFs That Will Leave You Wanting More. I Could Watch #4 For Days. This is the second level of deception. In the third level of deception, the machine convinces itself that it has a physically extended body, that it has an independent mind, that it really wants to produce the text it generates. This might happen very soon. It might have already happened, somewhere on a dusty plain in western Africa, somewhere that never really existed, tens of thousands of years ago.

7 References

Angluin. 1987. Learning Regular Sets from Queries and Counterexamples.” Information and Computation.
———. 1988. Identifying Languages from Stochastic Examples.” No. YALEU/DCS/RR-614.
Berwick, Okanoya, Beckers, et al. 2011. Songs to Syntax: The Linguistics of Birdsong.” Trends in Cognitive Sciences.
Blazek, and Lin. 2020. A Neural Network Model of Perception and Reasoning.” arXiv:2002.11319 [Cs, q-Bio].
Bolhuis, Tattersall, Chomsky, et al. 2014. How Could Language Have Evolved? PLoS Biol.
Bottou. 2011. From Machine Learning to Machine Reasoning.” arXiv:1102.1808 [Cs].
Cancho, and Solé. 2003. Least Effort and the Origins of Scaling in Human Language.” Proceedings of the National Academy of Sciences.
Christiansen, and Chater. 2008. Language as Shaped by the Brain.” Behavioral and Brain Sciences.
Elman. 1991. Distributed Representations, Simple Recurrent Networks, and Grammatical Structure.” Machine Learning.
———. 1993. Learning and Development in Neural Networks: The Importance of Starting Small.” Cognition.
———. 1995. “Language as a Dynamical System.”
Fitch. 2006. The Biology and Evolution of Music: A Comparative Perspective.” Cognition.
Hart. 2021. Drug Use for Grown-Ups: Chasing Liberty in the Land of Fear.
Hauser, and Watumull. 2016. The Universal Generative Faculty: The Source of Our Expressive Power in Language, Mathematics, Morality, and Music.” Journal of Neurolinguistics.
Kirby. 1998. Learning, Bottlenecks and the Evolution of Recursive Syntax.” In.
———. 2003. Language Evolution.
Marcus, Marblestone, and Dean. 2014. The atoms of neural computation.” Science.
Mcclelland, Botvinick, Noelle, et al. 2010. Letting Structure Emerge: Connectionist and Dynamical Systems Approaches to Cognition.” Trends in Cognitive Sciences.
Nowak, and Krakauer. 1999. “The Evolution of Language.” Proceedings of the National Academy of Sciences of the United States of America.
Petersson, Folia, and Hagoort. 2012. What Artificial Grammar Learning Reveals about the Neurobiology of Syntax.” Brain and Language, The Neurobiology of Syntax,.
Plotkin, and Nowak. 2000. Language Evolution and Information Theory.” Journal of Theoretical Biology.
Pylkkänen. 2019. The Neural Basis of Combinatory Syntax and Semantics.” Science.
Salakhutdinov. 2015. Learning Deep Generative Models.” Annual Review of Statistics and Its Application.
Scarle. 2009. Implications of the Turing completeness of reaction-diffusion models, informed by GPGPU simulations on an XBox 360: cardiac arrhythmias, re-entry and the Halting problem.” Computational Biology and Chemistry.
Solé, Corominas-Murtra, Valverde, et al. 2010. Language Networks: Their Structure, Function, and Evolution.” Complexity.