Making music together, with AI: a crowd-machine collaboration

By April 2, 2023 No Comments

Final board screenshot

The final result of the public board, closed after seven days. 

In this project, I conducted an ‘interactive symphony’, an experiment using machine learning to generate an original piece of music. To start, I created a Miro board that was made public for a week. Here, anyone could add, edit, or remove whatever they wanted. The resulting board was a chaotic mess of elements created by a random group of people.

I then used a timelapse video of the board as a prompt for a video-to-music machine learning model called ‘Melobytes’. The AI generated an unusual piece of music – one that raises questions about ownership in the age of generative AI. Here’s the final result:



This output was a collective effort that involved many people. But suppose I wanted to sell this piece of work. Who has the rights? There are many groups that could argue for their stake:

  • the companies who created the instruments used, like the Melobytes AI model, or even Miro
  • the people who contributed to the visual prompt
  • the one who pushed the button, initiating the music generation
  • the unknown artists whose work was included (without consent or credit) in the datasets used to train the AI

The ethics of generative AI are murky. As a creative tool, its possibilities are endlessly inspiring. But once monetisation comes in, things get tricky. I’ll need to review this once we know more about the Getty vs. Stable Diffusion lawsuit, but until then: I think it’s pretty cool.

This project was first shared at Interaction 23 in Zürich: