Why the NFB interactive studio decided to halt production of Marrow
Now & Next is highlighting some of the National Film Board’s projects in a series of collaborative articles.
Can artificial intelligence develop mental illness? Shirin Anlen, along with the NFB and Atlas V, explored this question for nearly two years, until the collaboration came to an end before completion in the winter of 2020. One year later, Marrow’s coproducers and creator look back on this ambitious concept, the pitfalls they encountered, and the lessons they’ve drawn from their experience.
Renowned across the United States for her work in digital arts and documentary, Shirin Allen had dreamed of working with the NFB for years. “They’re an exemplar,” she explains. “Many of their projects have shaped my artistic vision.”
The NFB’s Interactive Studio also had their eye on Anlen. “We had the chance to meet Shirin a few times while she was studying at MIT’s Open Doc Lab in Boston,” explains producer Marie-Pier Gauthier. “We chatted with her and quickly realized that here was a creative who really understood digital grammar. She has both an artistic approach and a mastery of programming languages, which was rare just a few years ago.”
On her end, Anlen had been wanting to work with an organization that takes risks exploring different platforms. “With time, I realized that the people at the NFB are good because they’re both funders and producers,” explains Anlen. “They’re not just driven by a business model; it’s the project, the art, the intent, and the audience that drives them. They have a wealth of experience and a good understanding of the creative process.”
At the end of a series of meetings with the artist, the NFB received a proposal. Anlen wanted to create an installation that would attempt to understand whether artificial intelligence (AI) learning machines, built through codes and structures made by human beings, might integrate human cognitive biases and mental health issues.
It’s a topic that’s dear to Anlen, who lives with borderline personality disorder. “A few years ago, I began to recognize my triggers and their effects. I was learning to find my balance, and I wondered if machines could do the same thing. I wanted to analyze how our mistakes and negative thoughts affect machines, in order to open up a conversation about mental health.”
The Interactive Studio’s producers were instantly won over: “We thought it was a compelling question,” recalls Marie-Pier Gauthier. “It was worth exploring, even if the project’s shape wasn’t immediately clear.”
The start of a great exploration
In 2018, the International Documentary Film Festival Amsterdam (IDFA) also took an interest in the project and offered funding for the creative team to present a prototype at the festival. “It pushed Shirin and her team to find a form that they could put together in just a few weeks,” explains the producer. “Their idea was a family dinner. Four participants would enter what appears to be a dining room and interact as if they were a family. On each plate, set in front of each participant, a text would be projected, and participants would take turns reading. The goal was to let AI guide and analyze them.”
Shirin Anlen’s work continued to evolve back in Canada through residencies in Montreal. “In our lab, the artists incorporated another element: capturing the timbre of participants’ voices. The learning machine would process each participant’s voice and return with reflections, as if their mind were speaking to them in their own voice,” explains director Martin Viau. “It was a remarkable development!”
As the mandate of the NFB is to produce works by Canadian artists, local creators would necessarily be joining Shirin Anlen, an Israeli based in New York, and her associate Avner Peled, an Israeli living in Finland. “We turned to actress, author, and director Laurence Dauphinais for her thoughts on staging the piece,” explains producer Marie-Pier Gauthier. “Laurence then introduced us to Max-Otto Fauteux for scenography and set design. We called on Philippe Lambert for sound design, Paloma Dawkins for the illustrations, and several other people for the sets.
The creators continued to reflect on the synopsis, story arc, dialogue, and set design, with the aim of tackling three exploratory chapters in four rooms, through four different machine learning models. “We wanted to produce the entire first chapter, including building the set, to get a better idea of the scope of the project so that we could work out distribution,” explains Gauthier.
Thought was given to every aspect of the project so that each venue could acquire the materials necessary to reproduce it. “We quickly set up a prototype in Montreal, putting up real walls and joints and painting everything,” explains Martin Viau. “We built an entrance hall and a dining room. But we couldn’t imagine shipping planks of wood around the world. We had to find an innovative set design that could be built easily and less expensively on site.”
Doubts set in
Little by little, reservations about the distribution model grew and intensified. “It was hard to see how we could find all the technological resources for building the four rooms as well as challenges circulating the work,” explains Marie-Pier Gauthier. “We consulted several cultural venues, like theaters and the Phi Centre in Montreal, to assess constraints and income models.”
It became clear that the final experience would have to be shorter to allow more people to see it. The prototype engaged with just a few people at a time for forty minutes, which wouldn’t be feasible. “It wasn’t going to be a viable model for venues,” explains Gauthier. “We had to think about involving more participants and having a larger passive audience, as well as a shorter duration to ensure decent rotation.”
From the beginning, the artists were aware that Marrow would need to circulate in Canada, and ideally, around the world. “As soon as Shirin informed us that she wanted the project to take place in a house, with one learning machine per room, we told her it wouldn’t be possible,” adds Gauthier. “It would be too expensive and difficult to transport. Far too few venues would be able to host it.”
Anlen felt the pressure building up. “We had to create our own tools and clear the ground in a field where no one had worked before, while also telling the coproducers what the final site would look like. They trusted my process, but I think production needs to be more flexible for machine learning to go further than what we’re already familiar with.”
The complexity of distributing the work was not the only challenge: the operating costs of the AI technology were staggering. For the dining room alone, the project required three ultra powerful computers with expensive analytical capabilities, at least two high-definition projectors, a dining table with a built-in screen, cameras to analyze participants’ reactions, and wireless transmitters and sensors. “The total cost was substantial,” stresses Martin Viau. “We also had to take hosting into account if we wanted the AI to be learning from and integrating each experience. For one day, just hosting alone could cost a few hundred dollars.”
Then imagine the cost for four rooms. “I understood the artists’ desire for everything to be generated by AI—the texts, projections, interpretation, and analysis—but we absolutely had to rethink the scope of Marrow,” recalls Marie-Pier Gauthier. “Unfortunately, they weren’t ready to think about that yet, and as coproducers, we needed to be convinced the project can work in order to carry on. We were having trouble finding common ground.”
There was yet another major problem for the Interactive Studio: test audiences struggled to understand the project. “We invited people from theatre and museum circles, as well as NFB colleagues, to experience the piece. A lot of them struggled to understand its intent,” explains Gauthier. “We didn’t want to spoon-feed them the answers or incorporate a whole course in artificial intelligence, but we wanted people to understand the artists’ intent, to give them a little guidance so that they’d be more engaged. I’d say that issues with audience comprehension of the work, at this stage in the design process, was another major tipping point.”
A difficult decision
At the end of the development stage, both coproducers—the NFB and Atlas V—made the difficult choice not to pursue production. “It broke our hearts,” says Marie-Pier Gauthier. “Shirin and Avner are wonderful people and extremely talented creators. For weeks, we watched them work in the lab past 10:00 p.m. on Fridays, trying to find ways to make it all viable. But even that viability was compromised. We had doubts about making the project accessible to Canadian audiences.”
Though Anlen could feel the tide turning, she was surprised to learn that the NFB was withdrawing from the project. “At first, I was shocked! We’d done so much that it was painful to stop. I saw all of our efforts as a phase in development. Putting all the pieces of the puzzle together takes time.”
She gradually became aware of the huge gap between what she was imagining and what the producers were hoping for. “I think there was poor communication on both sides. I felt like I hadn’t properly communicated my ideas and struggles.”
Anlen says that, after this experience, she better understands what’s involved in working with big producers. “Personally, I like to explore without knowing what I’ll find. I’m not overly concerned about the outcome. I enjoy the process of finding out.”
At the NFB, creative work is divided into several phases, with limits on the funds allocated to each. Marrow’s artists had reached the development stage, which has to result in an experience that’s viable and ready to move on into production. The NFB producers felt that the artists needed a lot more time and money for development before they’d be able to deliver a concept that could progress to the next step.
The NFB initially left the door open to a scaled-back version of the work or a Web experience. Then, after learning that the Austrian digital arts festival Ars Electronica had provided funding for Shirin Anlen and Avner Peled to hold a workshop on learning machine in creative projects, the NFB asked them to offer two similar workshops to their Montreal community.
Though the outcome was unexpected, the production team doesn’t feel like they’ve wasted any time or money. “We learned so much from Marrow,” says Marie-Pier Gauthier. “We know a lot more now about how to host creators in residence, what creating with artificial intelligence entails, and sometimes, when to pull out of a project. If we take a long view, it was a win. The learning experience was rich and worth the investment.”