Gesture recognition

I want to recognise gestures made with generic interface devices for artistic purposes, in realtime. Is that so much to ask?

Related: synestizer, time warping, functional data analysis, controller mapping.

To Use

  • Reverse engineer Face the music.

  • Gesture variation following has particular algorithms optimised for realtime music an video control using AFAICT particle filter. This is a different approach to the other ones, which use off-the-shelf algorithms for the purpose, which leads to some difficulties. (source is c++, puredata and maxmsp interfaces available)

  • GRT: The Gesture Recognition Toolkit other software for gesture recognition; lower level than wekinator (default API is raw C++), more powerful algorithsm, although a less beguiling demo video. Now also includes a GUI and puredata opensoundcontrol interfaces in addition to the original C++ API.

  • Eyesweb: An inscrutably under-explained GUI(?) for integrating UI stuff somehow or other.

  • Wekinator: Software for using machine learning to build real-time interactive systems. (Which is to say, a workflow optimised for ad-hoc, slippery, artsy applications of cold, hard, calculating machine learning techniques.)

  • Beautifully simple “graffiti” letter recogniser (NN-search on normalised characters, neat hack. Why you should always start from the simplest thing.) (via Chr15m)

  • how the kinext recognises (spoiler: random forests)

BTW, you can also roll your own with any machine learning library; It’s not clear how much you need all the fancy time-warping tricks.

Likely bottlenecks are constructing a training data set and getting the damn thing to work in real time. I should make some notes on that theme.

Apropos that, Museplayer can record opensoundcontrol data.