Recently I was wandering around with my phone in my hand, as one does, and got to wondering: what if I could use the camera feed and other sensors on the phone to create a procedurally-generated soundtrack?
And just to make things more interesting: I wanted to do it entirely in HTML5 + JavaScript with no backend server at all. In other words, something you could use without an internet connection, provided you’d already downloaded the page.
A few late night tinkering sessions later, and I had a working, somewhat well tuned prototype of my new web app: resonate_glitch.
As I’ve explained in the README:
Generative audio in HTML5. Designed to be run on phones.
resonate_glitch takes in various inputs: the camera feed, mic input, and accelerometer motion data on your phone. Running these parameters through an algorithm, it produces synthesized electronic “music” with a glitchy camera view.
The aim of the project is to create a synthetic synchronicity. Does the music have any meaning? Are you able to affect it in any way? It’s up to the user to experiment and find out.
Note: There is no backend server, everything runs in your web browser. So while the webapp can’t be used for spying, it may cause your phone to overheat.
Have a moment to make some messy, glitchy music? Great! It’s free and open source, try it yourself:
Source code is available in this GitHub repo.
