Notes for browser-based machine-learning, for projects like synestizer or other javascript AI nerdery.
Examples
- Gaussian Belief Propagation
- gaussianBP/gaussianBP.github.io
- GANLab is a JS-backed generative adversarial network exploration tool, published as a paper
- webcam transfer learning is an interesting essay in building a good UI for these things. It learns to use your webcam live.
- diffcam is a webcam delta heatmap for motion detection.
Infrastructure, tooling
First see the fundamental tools in javascript mathematics. Higher-level examples live here.
WebNN
A unifying API?
The WebNN API is a specification for constructing and executing computational graphs of neural networks. It provides web applications with the ability to create, compile, and run machine learning networks on the web browsers. The WebNN API may be implemented in web browsers using the available native operating system machine learning APIs for the best performance and reliability of results.
TVM
- apache/tvm: Open deep learning compiler stack for cpu, gpu and specialized accelerators
- Apache TVM
- Apache TVM Unity: a vision for the ML software & hardware ecosystem in 2022
Apache TVM is a compiler stack for deep learning systems. It is designed to close the gap between the productivity-focused deep learning frameworks, and the performance- and efficiency-focused hardware backends. TVM works with deep learning frameworks to provide end to end compilation to different backends.
Impressive example: mlc-ai/web-stable-diffusion: Bringing stable diffusion models to web browsers. Everything runs inside the browser with no server support.
ONNX
microsoft/onnxruntime allows us to run neural nets in the browser:
ONNX Runtime Web is a Javascript library for running ONNX models on browsers and on Node.js.
ONNX Runtime Web has adopted WebAssembly and WebGL technologies for providing an optimized ONNX model inference runtime for both CPUs and GPUs.[…]
With ONNX Runtime Web, web developers can score models directly on browsers with various benefits including reducing server-client communication and protecting user privacy, as well as offering install-free and cross-platform in-browser ML experience.
Tensorflow.js
A less generic runtime, in that it supports primarily tensorflow. NB it seems to support the superior jax.
Approximate scope is demonstrated in
TBD
Incoming
Have not updated the subsequent ones for quite some time and I suspect many of them are defunct.
Teachable machine is a web-based tool that makes creating machine learning models fast, easy, and accessible to everyone.
The models you make with Teachable Machine are real Tensorflow.js models that work anywhere javascript runs, so they play nice with tools like Glitch, P5.js, Node.js & more.
Plus, export to different formats to use your models elsewhere, like Coral, Arduino & more.
deeplearn.js, which can load tensorflow checkpoints Update: It’s now called tensorflow.js: “A WebGL accelerated, browser based JavaScript library for training and deploying ML models.” More precisely, “TensorFlow.js, an ecosystem of JavaScript tools for machine learning, is the successor to deeplearn.js which is now called TensorFlow.js Core.” Includes lots of sample code for integration with the browser etc.
- subgraphs.js is a visual IDE for developing computational graphs, particularly designed for deep neural networks. Subgraphs is built with tensorflow.js, node, and react, and serves on Google Cloud.
keras-js runs pre-trained keras deep learning models in the browser
Run Keras models (trained using Tensorflow backend) in your browser, with GPU support. Models are created directly from the Keras JSON-format configuration file, using weights serialized directly from the corresponding HDF5 file.
Inspiration is drawn from a number of deep learning / neural network libraries for JavaScript and the browser, including Tensorflow Playground, ConvNetJS, synaptic, brain, CaffeJS, MXNetJS. However, the focus of this library is on inference only.
Tensor operations are extended on top of the ndarray library. GPU support is powered by WebGL through weblas.
hardmaru presents an introduction to running sophisticated neural networks in the browser, targeted at artists
This looks genuinely amazing in terms of functionality and even includes native support for worker threads and concurrency. However… it is lacking modern web wrappings such as npm packaging etc, so is not convenient to use from e.g. webpack.
Twin to lalolib, a linear algebra library.
Javascript inference and training: convnetjs
- plus bonus interview
- less active, sister project for recurrent networks: recurrentjs
synapticjs is a full-featured javascript training, inference and visualisation system for neural network, with good documentation. Great learning resource, with plausible examples. Actively maintained.
This library includes a few built-in architectures like multilayer perceptrons, multilayer long-short term memory networks (LSTM), liquid state machines or Hopfield networks, and a trainer capable of training any given network, which includes built-in training tasks/tests like solving an XOR, completing a Distracted Sequence Recall task or an Embedded Reber Grammar test, so you can easily test and compare the performance of different architectures.
javascript inference only, neocortex.js in the browser. Civilised, but untouched since February 2016.
brainjs is unmaintained now but looked like a simple javascript neural network library.
mindjs is a simple one where you can see the moving parts.
WebPPL is a successor to Church designed as a teaching language for probabilistic reasoning in the browser. Has hip features.
- Runs on the command line with node.js or in the browser.
- Supports modular and re-usable code using packages built on top of the npm package system, and interoperates with existing Javascript packages in the npm ecosystem.
- Includes a large and expanding library of primitive distributions.
- Implements a variety of inference algorithms, including exact inference via enumeration, rejection sampling, Sequential Monte Carlo, Markov Chain Monte Carlo, Hamiltonian Monte Carlo, and inference-as-optimization (e.g. variational inference).
- Provides inference as a first-class operator in the language, allowing for nested inference (‘inference about inference’).
- Supports optimizable models with neural network components using adnn.
No comments yet. Why not leave one?