CV++ Tendrils, interactive AV ~ project @ Max Cooper

Tendrils, interactive AV ~ project @ Max Cooper

An experiment in nature-evoking art, physics, graphics, and interaction. Adapted into an interactive music video collaboration on Max Cooper’s Emergence project, featured on Experiments with Google and its Chrome Experiments Collection front page. Developed it further into a standalone visuals version for myriad live shows and events with audience interactions, and published Open-Source for reuse and learning. The nature-evoking visuals are an emergent physical system of interacting particles – complex organic forms emerge spontaneously, from simple interactions at the individual scale; in a fluid-like advection, particles recursively affect and affected by the field of motion. Executed the process via low-level stack.gl, WebGL 1.0, GLSL, JavaScript; custom-built partly as a learning exercise – creating custom General-Processing GPU methods, GPGPU; keyframe-based animation system, and keyboard-driven timeline editing; audio-response triggers, based on audio waveform and Fast-Fourier Transform, FFT, reacting to thresholds across time-windows in a calculus-based interpretation.

2018.01–2022.10 [0 1 2 3 4 5 6 7 8 9 A B C D E F G H] ++

[toggle any arrows for details ~ double-tap media for full-screen ~ try any broken links on **archive.org]**

At the core of the effect, each particle leaves a trace of its motion in the environment, which in turn affects the motion of any particles passing through. This simple interaction at the individual scale gives rise to the complex organic forms at the collective scale, without explicit control or guidance; an emergent process.
Exploring intuitive natural interactions, viewers’ intuitive natural motions directly influence the particles’ motions, via optical-flow interpretation of the camera input; offering lighter and more flexible interactions than more explicit feature-tracking approaches. Thomas DieWald, Adam Ferris, and Andrew Benson show some nice examples of this technique, which I later adapted for glsl-optical-flow.
Audio-response is interpreted via audio and microphone input, via AudioContext APIs, so music and ambient sounds trigger visual reactions; this uses a GLSL calculus-based method on the raw audio data signals, triggering responses to changes surpassing chosen thresholds across any of multiple tiers of processed signals.
After the initial prototype came the involved marathon collaboration process with Max on the interactive music video, alongside my full-time role - exploring creative possibilities together; interpreting between music, art, and tech perspectives and feedback; and developing these ever-evolving ideas into a publishable work, which gained much visibility on Google’s showcase.
In addition to the music-video version for the track Trust, the standalone visuals version was presented further in myriad live and online events and spaces - from a short film intro, to many varied gigs, to exhibitions, to bigger stage shows, to websites.