blank canvas
An interactive audiovisual installation. A custom iPad app turns a participant's drawing into continuous AI imagery via a stable-diffusion model, while the act of drawing drives procedural music across a 4-speaker array.
01visuals
05design artifacts
An interactive audiovisual installation that turns a participant's drawing into a generated world. Strokes on a custom iPad app feed a stable-diffusion model that renders continuous imagery, while the act of drawing drives procedural music surrounding the participant in a four-speaker array.
approach
The goal was a generative playground where people create multimodal representations of their imaginative input through minimalist illustration, gesture, and text. The framing is deliberate: AI that augments the creative process rather than automating it — celebrating post-human creativity instead of the fearful, sometimes nihilistic posture that has saturated the discourse around "creative" AI tools.
process
Signal flow runs entirely on a local network — Apple Pencil strokes and text prompts leave the iPad over OSC, with the screen cast over NDI:
- **iPad → ** custom Swift drawing app (OSCKit) sends pencil data + text prompts.
- **Visual system → ** TouchDesigner driving a stable-diffusion model → projector.
- **Audio system → ** a VCV Rack patch generating procedural audio → Motu interface → four-speaker array.
Running the diffusion model locally was the central constraint: ~4 fps on a laptop, lifted to a consistent 10 fps once an Nvidia GPU was in the loop.
collaboration
- Kyle Smith — model implementation on the laptop, portable local network for OSC and NDI, spatialization, and hardware for quadraphonic sound and local generation.
- Ishaan Jagyasi — TouchDesigner patch integrating the stable-diffusion model; VCV Rack procedural-audio patch.
- Rafael Collado — custom Swift iPad app; pencil/prompt routing via OSCKit.
credits
- Context: MUSI 6201 (Integrated Multimedia), Georgia Tech, Fall 2025. Instructor: Dr. Jeremy Muller.
- Tools: Swift / OSCKit, TouchDesigner, stable diffusion, VCV Rack, OSC, NDI, quadraphonic array.
- Shown at the Goat Farm.