Will algorithm-powered dubbing be a boon or a bust for the industry?
The long tail phenomenon has changed the rules of the marketing game for niche series like Squid Game. Their spontaneous global and unexpected success has seriously ramped up demand for making “foreign” content accessible to viewers around the world.
More than ever, the film industry, broadcasters, and their viewers have a shared interest in seeing titles made available in multiple languages as quickly and at the highest quality possible. International hits such as the Spanish La casa de papel (The House of Paper) series and the South Korean film Parasite, not to mention Canadian films like C.R.A.Z.Y. by the late Jean-Marc Vallée, have all benefited hugely from today’s dynamic and ever-expanding dubbing industry.
And that’s just for starters. Business Wire magazine predicts that the dubbing market will continue to grow from US$2.4 billion in 2019 to US$3.6 billion in 2027, an increase of 50% in less than a decade. Maybe it’s because “No one likes subtitles,” as Kevin Drum recently wrote in Mother Jones magazine, provoking the ire of social media experts on the subject… even if one must admit that the quality and the speed of the dubbing post-production process is impressive by any standard.
No matter how you look at it, dubbing and subtitling are playing an increasingly important role in our increasingly global market for films and series.
Dubbing on the double
The Hollywood Reporter made the point just ten years ago that there were only a few large non-English speaking markets where viewers could actually enjoy films and series in their native language. France, Italy, Spain, and Germany had become acknowledged experts in the art of dubbing, with their own domestic actors and studios with all the necessary resources.
The only option for smaller markets was to make do with subtitles.
It’s difficult to estimate the exact price producers pay for dubbing, but a number of sources mention costs of up to $150,000 or more for a single feature film. For any major production looking to reach viewers in 40 countries, this represents a multi-million-dollar investment, a high-risk bet at best, especially for producers from small markets like Canada.
Fortunately, exciting new players have tossed their hats into the dubbing ring lately with solutions that not only automate the process and eliminate any unnecessary duplication, but also measurably improve the quality by using different layers of technology that accurately match the tone of the original by using a synthesized double of the actor in the target language.
Drilling down on dubbing
Tweaking algorithms in translation is nothing new, of course. Google Translate continues to make steady progress in quality. Accuracy, as measured by the BLEU (Bilingual Evaluation Understudy) metric has been improving by about one percentage point every year for the past ten years.
What this means is that Google Translate now exceeds the 50 benchmark of “very high quality, adequate, and fluent translations” for translating from English and into English for a number of major languages.
At the 60 benchmark, the translation will be considered better than actual human translation. This level is likely to be reached in the next few years.
As Srishti Mukherjee wrote in the January edition of Analytics India Magazine:
“Previously, synthetic voices were just words glued together to produce a clunky, robotic effect. Nowadays, voice developers no longer need to provide the exact pacing, pronunciation, or intonation of the generated speech themselves. Instead, they simply fill hours of audio into an algorithm that deciphers those patterns for itself. These voices can change in style and emotion, take pauses and breathe in the correct places, and learn the tone and patterns of the speech of an actor.”
Israeli-based Deepdub raised US$20 million in February to further develop its algorithm for preserving a virtually identical voice in different languages. From a relatively small sample, you can use the original actor’s voice, instead of a new voice selected by a local dubbing service.
One of the major challenges in dubbing has been getting the facial expressions and mouth movements right so they correspond to the sounds and the words being used. No longer a problem. Startups like British-based Synthesia can now adjust the actor’s facial expressions and mouth movements to mimic the local language with stunningly realistic results.
Radical advances like the above not only drastically reduce the time required for dubbing, but also make the process significantly more affordable when it comes to making films and series available in multiple languages.
The deep, dark downside of algorithm dubbing
While these technologies promise to make foreign films and series more accessible and enjoyable, the interest and investment made in them promotes the same technological breakthroughs as those used by the creators of deepfakes.
By making art and content more accessible, the new technologies can undermine consumer trust in their authenticity and intent. Algorithmic dubbing certainly might require a stringent degree of oversight where art and political action committees meet.
It’s also likely from a microeconomic perspective that the dubbing industry will get taken over by a small number of technology startups where the winner takes all, and that could be extremely damaging to ecosystems where local dubbing companies play a major role.
The switch to algorithms could also mean the end of a profession for the actors who brought the original actors to life with their deep-seated local sensitivity to meaning and expertise.
While algorithm-powered dubbing is a major opportunity for creators everywhere, it also poses a real threat to the integrity of content and to the survival of the ecosystems that support it, especially in smaller markets.
