Hardware for neural networks
Neuromorphic computing, non-von-Neumann architectures, and other ways to compute for AI
2023-11-16 — 2024-03-25
Wherein Neural Computation Is Surveyed Through Hardware Modalities, and Optical Processing Using Randomized Linear Algebra and Direct Feedback Alignment Is Noted as a Concrete Alternative to GPU Backpropagation.
Placeholder, for thinking about the implementation and theory of computation as it has been perturbed by our increasing dependence upon neural models for computing.
1 GPU design
A whole field I will make no attempt to catch up on right now.
2 Quantum devices
TBD
3 Optical devices
I am slightly familiar with Igor Carron’s work on optical processing using randomized linear algebra. (Brossollet et al. 2021; Cavaillès et al. 2022) seem to train NNs using direct feedback alignment.