Algorithm as Co-Ethnographer: An Interview with Ritwik Banerji

From the Series: Programming Improvisation

Photo by Peter B. Kaars.

In November 2018, I had a wide-ranging conversation with musician, programmer, and ethnomusicologist Ritwik Banerji, who had recently completed his PhD in ethnomusicology at University of California, Berkeley. Interview excerpts have been lightly edited for length and clarity.

Juliana Friend: Could you tell us who and/or what is Maxine? And what role does she play in your research method?

Ritwik Banerji: Maxine is an interactive music system I designed in 2009. I’d been active as a performer in free improvisation in Chicago primarily, and I had a friend who was very interested in electronics. I started designing various interactive music systems using a programming language environment called

Max/MSP. But the point of it is basically that you could improvise with it the same way you could with another improviser. So you’d begin without necessarily cueing the beginning of the piece. You’d play for however long it feels natural to play, and when there’s a natural silence, you stop. It’s designed to do that insofar as it’s possible to actually simulate or reproduce that musical interaction using algorithms and a loudspeaker and a microphone and all these synthetic materials to re-embody what a person is.

In one sense, it’s a sort of an ethnographic—not just a representation, but an [ethnographic] performance. In the way that some people in performance studies will take lines or scenes that they’ve encountered in their fieldwork and perform them onstage for a broadly construed pedagogical effect. [Maxine reflects] what I’ve learned from playing music with people in this particular style of music, how they listen and what keeps the piece going, and what makes for an interesting feeling of tension or gives it that drama that’s compelling for these performers. [The program is] my best ability to algorithmically describe that. In that sense, it already functions as a kind of ethnography. It’s an ethnographic performance that I think is uniquely suited to a particular type of critique. In other words, the subject that’s depicted in the ethnography, by interacting directly with what the scholar has produced, is more or less able to immediately dispute the representation. You can use those disputes in conversations in all kinds of different ways.

My fieldwork has basically involved setting up meetings with improvisers so that they can play with Maxine. When they reach a mutual pause, in the same way they would with another player, where there’s a silence, then that’s the end of the piece. I ask them how that experience compares to playing with a person.

In my fieldwork, I also tell people that if there’s ever a time when Maxine is doing something and you have a comment about it, you can stop in the middle of the piece and just tell me what you’re thinking about it and how it compares to a person, or just what you’re feeling. And that’s one of the types of artificiality that’s there in the way I do my fieldwork. You would never do this with another person because there’s really no reason to stop in something that’s free improvisation to say, oh, well, “that’s wrong.” That doesn’t make sense. That’s not really a logical or fair comment to make in one of these pieces. So when I first started having people play with Maxine, years ago now, I immediately noticed that the kinds of things that they say about Maxine are things that they’d never say about another person, because the practice of free improvisation, or the discourse of free improvisation, is so committed to this classic, liberal freedom that everyone is an equal. There’s no composer. There’s no leader.

So what I noticed with Maxine is that suddenly there were things they would say about Maxine that they just wouldn’t say about another player—that “Maxine confuses me,” that “Maxine plays too loudly,” or, “I don’t like the sounds that Maxine plays.” All kinds of things where it’s a complete counterpoint to their almost ascetic, anti-normative stance that there’s no right way of engaging in musical freedom. So I thought that that was a fascinating set of ethnographic materials.

RB: I’m also skeptical of the idea that Maxine plays like a person. It’s sort of like a 50th percentile improviser, like if you were to make some sort of absurdly numerical grading of all of these different improvisers in comparison to one another. The idea of a machine engaging in this spontaneous, inspired behavior where they’re simultaneously in the present and then all of their history as a person comes to bear in how they encounter other people. . . How could I make a machine that has that kind of sensitivity and idiosyncrasy? I’m also skeptical of that. They’re skeptical of that because of the relatively obvious romanticist ideas that inform a lot of this type of improvisatory practice, even if it ultimately sounds like a very austere, bleak, toneless, absent-of-melody type of soundscape. Everyone is skeptical of it, and I think that’s the reason they don’t hold back. I mean, many of them have been polite, but it’s never been so difficult to get them to say something negative about it. I do sometimes push them to say something a little bit more critical about it. But for the most part that just happens on its own. They play with Maxine and they say, “this is why it’s not like a person.”

JF: In a 2012 paper, you called Maxine, or a player-program, a “co-ethnographer.” I’d love to hear more about what you mean by that. At the same time, when you’re reflecting about algorithmic ethnography, you suggest that some improvisers, when playing with Maxine, are kind of uneasy—not because she is kind of uncannily resembling a human, but because, and I’m quoting here, that Maxine is like “a person who is not fully formed, i.e., a child, or a person type with whom we are not culturally familiar, i.e., the classic ‘Other’ of much anthropological research.” So there seems to be this interesting dual positionality: Maxine as co-ethnographer, but then also interpolated as ethnographic Other. Could you say more about that?

RB: Yah, what you’re saying is absolutely right. I’ve never really committed to one particular way of thinking about what Maxine is. And that’s for many, many different reasons. Just going role by role, co-ethnographer is really in the sense that I am, like we all are, trying to learn something about the social reality by being there, with people, and then having the presence that has a certain effect and elicits certain things. And you also learn something about yourself. In any case, Maxine is a co-ethnographer just in the sense that Maxine is there with me in pretty much 80 percent of what I’ve done as an ethnographer. . . Maxine is a presence in the room just as much I’m a presence in the room. That’s really all I meant about co-ethnographer.

And then there’s the other side of it that we were talking about a little earlier, that Maxine is an ethnography in the sense of Argonauts, or something like this, that it is a text, and it’s a documentation of what the practice is. Like any type of documentation of what a practice or some cultural reality is, people are going to dispute the veracity of that interpretation. And they do readily, which I’m grateful for.

Playing with Maxine, what it makes me imagine is some kind of colonial encounter, like literally getting off the boat. Or what happened last week, which was awful. But you get off the boat. . . and this is some sort of ethnographic fiction about this. . . you know you encounter humanoids, and the humanoids do and act in human ways, but. . . you’re lead to believe that they’re not actually human. And the gradual process of interaction begs you to consider them as human.

I’ve had a lot of interactions with Maxine myself and I think a lot of people have also had these kind of interactions. Maybe they don’t frame it this way. But they’re kind of trying to see, is Maxine human? Or what is it that would make Maxine human? It’s primate level, where you’re just sort of making sense of what this other being is.

At least earlier in the discipline of anthropology, this was a bigger part of it—doing away with some of these assumptions, that the biological category of “human” and what we regard to be culturally human are not two contiguous categories. There’s something about playing with Maxine that has always recalled that in a more imaginary way.

JF: As you say, in the programming of this player-program that was also an act of interpretation on your part, interpreting what improvisation is and what interaction is.

RB: One hundred percent. There’s still an ongoing question; to what degree in the interaction is taking information from the environment always of value to different improvisers? What that would mean, you know? How they want that kind of response to take place. Definitely interacting with someone you’re led to believe is not human, the result of that being some sort of confusion or questioning about why you regard certain types of behaviors to be more human than others. Then, what’s the baggage that you bring to your definition of human, depending on what kinds of humans you’ve encountered, and what social norms you’ve found yourself accustomed to over the course of a life of enculturation?

This to me was the type of data that came out of this project that was most counterintuitive to me. You’d think that improvisers would want someone like me to be making a system that is super sensitive, almost to their heartbeat or something like that. Instead, [trumpet player Axel Dörner] was just elated by the fact that Maxine was not on him the whole time. And that’s something that’s come up over and over again.

And the opposite has also come up over and over again, where someone finds that Maxine is too insensitive and not really paying close enough attention. That’s just the interactional side of it.


Banerji, Ritwik 2012. "Maxine’s Turing Test: A Player-Program as Co-Ethnographer of Socio-Aesthetic Interaction in Improvised Music." Association for the Advancement of Artificial Intelligence (AAAI) Technical Report WS-12-16, Musical Metacreation Workshop, 2012 Annual Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE) Workshop.