Piece by Piece

When you are training a machine learning model the first question you ask yourself is, “What data should I use?” While preparing the album PROTO I thought it would be more interesting to see if I could train an artificial intelligence (AI) on only my data and the data of people close to me. The construction of that album was built around this premise of consenting data. I started a vocal ensemble and would feed those recordings to the machine and also held large training ceremonies where we invited the audience to train our AI. As a result, we now have unique AI models constructed from the aggregate, consenting input of thousands of voices. I felt this was a beautiful and harmonious way to approach the subject of AI, which can so easily be viewed as inevitably extractive and entitled.

So much of the work we have been doing since then has been guided by this idea of experimenting with a digital abstraction of a training subject. PROTO presented an AI as an ensemble member, trained on the ensemble itself. Holly+ began as an experiment in decentralizing control of my voice and allowing others to make music with me, or as me, with a system in place to share profits when profits are made. At Helsinki Festival last year, we held an hour-long performance where Sam Fuller-Smith performed with my voice, accompanied by Ryan Norris and Philomela Choir. I have also created a number of visual works exploring Contrastive Language-Image Pre-Training (CLIP) embedding, which is another kind of digital twin that exists of me based on aggregate things that people post online about me. We also helped start an organization building consent tools for AI training called Spawning!

I am still working it all out, but once you dive deep into the subject of AI there is so much unexplored territory. It has helped me to orient myself by experimenting with the very personal aspects of it: my voice, my identities. Otherwise it can be quite overwhelming. I think the way we make and experience art is going to change forever, and also some things are evergreen. I have no desire to enter a room and listen to an ensemble of machines singing. I prefer to think of ways these fascinating tools can augment me and the things I love.

Luz: How do you see the role of sampling evolving in the future of music-making, and what do you think the implications of this evolution will be for artists and audiences alike?

Holly: It will soon be very common to assume the identity of others, just as common as using a sample. I call this identity play, a pun about IP. So long as we establish etiquette and consent, I think this could be very cool and expressive. I think people will construct very different sonic and visual identities for themselves from pieces of other people and things. On PLATFORM and PROTO I experimented with trying to mutate my voice in ways like this, where for a short while in a song my voice would resemble a bird on the high end and a garbage truck on the low end. I think this kind of chimeric modeling will become very powerful and expressive and is perhaps more interesting than attempting to recreate an individual performer. Navigating multi-dimensional models is something we have only just begun to scratch the surface of in music-making and performance.

These new tools will almost certainly change consumption habits and audience interactions with music and art. It is hard to do more than speculate as to how, but the ability for anyone to spawn infinite media is a capacity we have never had before. I am optimistic that when we are surrounded with infinite derivative media, we may begin to seek out unique artistic practices that are not as easy to emulate and automate as content.

Luz: It’s been almost two years since launching Holly+. What are some benefits and challenges you’ve seen after implementing a decentralized autonomous organization (DAO) model to govern the usage and licensing of the Holly+ voice model?

Holly: The benefits initially were that it would have been prohibitively expensive to experiment with this arrangement otherwise. Smart contracts, revenue splits, etc. would all have to be manual and messy, and we don’t have the capacity to run something like that. The DAO allows for more fluid membership, so even though I have a veto vote over usage of my own voice, the DAO can self-organize if it likes. Many of the DAO members are people I have never met, but we have at least a shared interest in stewarding my voice. Time will tell if that infrastructure works best for this challenge, but it seemed worthwhile to experiment, and I was really happy with the first experiment.

Luz: Can you give us a background on communal voice ownership and how you think it could change the way we think about the relationship between individual artists and their creative output? What implications could this have for the music industry as a whole?

Holly: Voices consist of a combination of social influence, physiology, and technique. We learn to speak from mimicking others, we pick up accents and learn affects for when we sing. Even though there are protections around personality rights, you cannot copyright a voice, and so it feels inevitable that the future of our voices will be communally determined to some extent. Voices are both private and public.

Our icons are the same. I am different from the public perception of me. I made some portraits exploring the “Holly Herndon” embedding in Open AI’s CLIP, which is basically an aggregate interpretation of who I am trained on the internet, and because most pictures of me on the internet are promo shots, my digital twin in that environment looks like a promo shot. It is different from who I am in private. It knows aspects of me, but it does not know me any more than a Wikipedia page knows me.

Juggling the weird tensions between a public and private persona is not new, but I think it will get very strange in an AI augmented internet, and as that internet interacts more with physical reality. I think it makes sense to lean into the idea that our public personas have a life of their own, and maybe experiment with ways of governing them. When you interact with many pop stars or influencers on Instagram, you are actually interacting with a team of people who curate their public persona. Those teams are pretty centralized, and mysterious.

With Holly+, we were trying to think of ways to decentralize my identity, and one of the ideas that made sense was to give a small piece of ownership in the voice to a lot of people, who would then be able to vote on “official” uses of the voice. I have a veto that I may never use, but this approach makes sense. If my public persona is a product of public perceptions of me, why not make governance of that persona more public?

Maybe then Holly+ over time, the voice, its visual representation, will start to deviate from my private self in interesting ways. Maybe people will find Holly+ more interesting or entertaining than Holly Herndon. I find the potential of it all fascinating.