Hardware for neural networks
Neuromorphic computing, non-von-Neumann architectures, and other ways to compute for AI
2023-11-16 — 2024-03-25
Wherein neural computation is surveyed through hardware modalities, and optical processing using randomized linear algebra and direct feedback alignment is noted as a concrete alternative to GPU backpropagation.
Placeholder, for thinking about the implementation and theory of computation as it has been perturbed by our increasing dependence upon neural models for computing.
1 GPU design
A whole field I will make no attempt to catch up on right now.
2 Quantum devices
TBD
3 Optical devices
I am slightly familiar with Igor Carron’s work on optical processing using randomized linear algebra. (Brossollet et al. 2021; Cavaillès et al. 2022) do train NNs using direct feedback alignment.