Positive Applications for Deepfake Technology

The video industry is in the midst of a technological revolution, as the exploration and application of artificial intelligence, machine learning, and deep learning radically expands the possibilities for business practices. Deepfake is a technology gaining particular momentum in the modern video industry.

Emerging technologies, in particular artificial intelligence (AI), provide many exciting advancements in our industry. However, while the positive potential of these capabilities is being realized with incredible gains, those with malicious intent are developing methods for using these technologies in criminal or otherwise immoral ways. Deepfake technology is one of those exciting advancements that is certainly not an exception to this rule.

Voice Manipulation Technology

Deepfake — a term derived from the combination of “deep learning” and “fake” — uses advanced technologies to alter videos in monumentally new ways, effectively making people appear to say or do things that did not happen. Deepfakes entered the mainstream consciousness due to both the amazing technology application and concerns about its misuse.

An excellent example of the positive usage of deepfakes is a video created to launch a petition to end malaria created by the Malaria Must Die campaign. The video features legendary soccer player David Beckham appearing to speak in nine different languages. Actually, the voice changes — even to female — yet Beckham’s mouth seems to be perfectly in sync with the words. With the help of AI, deepfake technology was used to manipulate his facial movements, creating the visual illusion that he was saying the words in each language.

Beckham’s video illustrates the potential for applying deepfake technology in dubbing films. Manipulating an actor’s facial movements can make it appear they are saying words they did not say. This concept can be taken further by using AI to manipulate actors’ voices so that people who watch dubbed film versions will still hear the original actor’s voice.

Voice manipulation technologies already exist. For example, Lyrebird AI uses AI to clone any voice by just using a one-minute voice recording. Pair it with deepfake technology, and you have the ultimate dubbing tool.

Democratizing VFX Technology

Deepfake technology is full of new possibilities in filmmaking, including affordable video production and crafting interactive images of late artists and celebrities. This not only allows audiences to engage with their favorites long after their passing but also streamlines time and labor in the film industry.

Traditional video production often involves constructing physical sets or scouting locations, incurring expenses for materials, labor, and permits, but with the use of deepfake technology for visual effect (VFX), filmmakers can create virtual sets and backgrounds, eliminating the need for physical construction and reducing location expenses. This technology also allows for greater flexibility in setting scenes, enabling filmmakers to change backgrounds easily and adapt to different narrative requirements without the logistical challenges of on-location shoots.

Deepfake technology also allows filmmakers to reimagine scenes with actors who, for whatever reason, can no longer participate in the traditional processes — or there is a need to recreate their characters at a younger age. For example, the surprise appearance of Luke Skywalker in The Mandalorian season two finale swept the Star Wars fandom off their feet. The digital recreation of a youthful Mark Hamill also faced swift criticism from viewers, showing the challenges in leveraging VFX technology to deliver its benefits effectively.

Creative Screen Production

Creating convincing deepfakes is a tough task, but despite potential harm, collaborations between technology and creative professionals are actively seeking solutions to identify and counteract deceptive videos.

The atrocious use of deepfakes, primarily in non-consensual pornographic content, paints a disturbing picture. However, this dark side shows the pressing need for robust defenses. The Content Authenticity Initiative (CAI), a collaborative effort among tech companies, aims to provide a reliable way to assess media authenticity. Truepic and Qualcomm’s metadata embedding system represents a promising step towards verifying the origin and history of images.

Yet, the response to deepfakes is not solely defensive. Researchers are exploring positive applications, particularly in healthcare and education. MIT Media Lab’s exploration of using deepfakes to aid individuals with aphantasia demonstrates a potentially transformative use of the technology. Meanwhile, creative minds are employing deepfakes to resurrect historical figures or enhance cultural experiences, demonstrating the dual nature of this evolving tool.

The emergence of a network of film producers, researchers, and AI technologists in the UK signals a proactive approach to harnessing the positive potential of deepfakes in creative screen production. From resurrecting animators for documentaries to entertaining, educational content, these endeavors showcase the diverse applications that can enrich our cultural and educational landscape.

Deepfake Presenters or News Reporters

Creating deepfake presenters or news reporters has various benefits, particularly enhancing customer engagement and personalizing messaging. Deepfake technologies offer the opportunity to provide viewers with more personalized experiences based on their preferences.

For instance, Reuters, an international news agency, has harnessed AI-generated deepfake presenters to deliver sports news summaries. This virtual news reporter is not just a pre-programmed entity but can provide personalized news broadcasts tailored to individual preferences. The ability to seamlessly dub these reports into different languages further enhances the viewer experience, providing a smoother and more accessible way to consume news content. Hence, this application allows for a dynamic and interactive news delivery, where viewers can receive content that aligns with their specific interests.

Altering Video Transcripts

The dark side of deepfakes garnered public attention when videos of celebrities appeared online where they said and did things they never did. Another example that received widespread attention is an altered video of President Barack Obama that was circulated by researchers at the University of Washington to illustrate the potential harm that could result from using deepfakes with ill intent.

However, while these concerns are valid, deepfake technology carries the potential for several positive applications in the broadcasting and entertainment world. Studios can save massive amounts of money and time by utilizing the capabilities of these tech advancements to edit and change video transcripts after they have been filmed without the need for a reshoot.

Currently, film studios have to jump through hoops to reshoot a scene. Suppose a necessary alteration has been made to the dialogue in the scripts after the scene has been shot; the film studio will need to assign a budget for reshoots and book all the actors involved in the scene and the location. And this is not all that is involved with reshoots. Imagine how many people will be implicated and how much cost will that require. Meanwhile, when deepfake technology is developed enough not to demand a significant amount of money, the costs will be significantly lowered. In this case, only visual effects artists and actors will be paid. The latter depends on the laws that the Screen Actors Guild (SAG) will decide on. No crew or location booking costs are involved.

Educational Application

In 1963, President John F. Kennedy was on his way to deliver a speech in Dallas when he was assassinated. The beloved politician did not get to bring those words to the world that day, but thanks to modern technologies and innovative techniques, we can hear them now. To bring this idea to fruition, the team at CereProc analyzed the recordings of 831 JFK speeches to “build his voice” by separating the audio into 116,177 phonetic units. The incredibly challenging task was made even more difficult because the recordings were made on various types of equipment at different times. CereProc used AI to successfully create a very realistic audio speech derived entirely from data.

This proves the vast possibilities for using deepfake technology for educational purposes, such as making it possible to create new videos of historical figures telling their own story to bring important tales to life. Last year, the Illinois Holocaust Museum and Education Center had a showcase where the holographic images of 15 Holocaust survivors were shown on rotation. Visitors had the chance to ask their questions to the holographs of survivors. A sphere of cameras recorded the interviews with the survivors; each interview took 5 days to shoot.

With deepfake technology, the same can be done on a bigger scale. Historical figures can be brought back to life, and more interactive historical classes can be created for schools. This practice already exists, and deepfakes can take it to the next level.

Final Thoughts

While the potential misuse of deepfake technology raises valid concerns, it is important to recognize its transformative impact on the broadcasting and entertainment industries. From personalized news broadcasts to virtual fitting rooms, deepfakes offer a spectrum of positive and exciting possibilities. The ability to resurrect historical figures, enhance educational experiences, and engage viewers on a personalized level showcases the creative potential these tools bring to storytelling and content creation.

To delve deeper into how DataArt can assist media and video companies in navigating and leveraging emerging technologies, visit our Video Expertise page.

Author: Max Kalmykov
Vice President of Media and Entertainment Practice at DataArt

Originally published on https://www.dataart.com/blog.