TThe Australian Open wants to use AI to generate simulated lip movements of Novak Djokovic and his fellow tennis pros for multilingual marketing and synchronize them with computer-produced voices that already replicate the players’ tone and pitch.
The attempt to use artificial intelligence was described as “scary” by a tennis commentator during the tournament’s official “demo day” on Friday.
The showcase highlighted Tennis Australia’s (TA) efforts to grow audience and revenue by pursuing technical innovation and investing in start-ups.
TA head of innovation Machar Reid says that while players have not consented to having their voices imitated, they will be consulted as the technology begins to improve their image.
“Hopefully you create a win-win situation where the content is easier for a fan to consume, but in return the brand or the player themselves also gets into more media and into more markets in a way that is relevant to them “It also helps to expand your own profile,” says Reid.
Reid played a video of Djokovic, already live on YouTube, that sounds like the world No. 1 is speaking, but is in Spanish.
“The priority is that it makes sense, the second priority is that it sounds like him,” Reid says. “Third priority is getting the lips to actually sync with what’s coming out of his mouth, which will happen over time thanks to AI.”
Tennis commentator and event host Nick McCarvel was stunned by the demonstration. “That scares me,” he says.
The Djokovic demo is one example (the Serb has previously spoken Spanish in press conferences), and the Australian Open’s YouTube channel already has several videos using the technology, including videos that mimic Djokovic’s distinctive voices Coco Gauff And Daniil Medvedev.
These “dubs” are created by UAE-based Camb.AI, part of the Australian Open 2024 Startup Program, and will be delivered during the tournament in under 12 hours.
The TA team has taken steps to minimize the risk that the translated words it attributes to players could cause controversy.
“Players obviously care a lot about their image and the way they are portrayed,” says Xavier Muhlebach, Head of Original Content at TA.
“Right now we’re actually also providing English subtitles that actually relate exactly to what they’re saying in our transcripts, so there’s no risk of misinterpretation if it crosses that language boundary.”
Reid says the team plans to consult with players as the project progresses.
after newsletter advertising
“The focus there was particularly on translating audio. In the future, people will ask, ‘Okay, from an image perspective, how do you improve fidelity?'” Reid says.
“I think it’s at such an intersection that the consultation process begins, in and around ‘Now we’re changing the image of someone’, which can be sensitive again, so engage in that.”
TA chief executive and Australian Open director Craig Tiley defended the tournament’s financial health two weeks ago after it received tens of millions of dollars in Victorian government subsidies over the last three years.
TA – operator of the Australian Open – has an internal innovation program and works with startups to potentially invest in them through its venture capital division.
The company has already invested in Israeli AI video analytics company Minute.ly and Californian ball tracking startup SwingVision.
Reid says the push into technology has helped make the Australian Open the best tournament possible for fans and players.
“Technology is a critical part of what we do from a fan perspective, and we essentially want to try to improve and accelerate it in a way that benefits the player and the fan,” Reid says.
“So in that sense they are symbiotic. Hopefully the rising tide lifts all boats and everyone benefits.”
