AI in Filmmaking — Issues, Possibilities, and the Need for Better Data Management

It’s no secret that AI could revolutionize filmmaking and how the entire media and entertainment industry operates. Right now, many studios are understandably apprehensive about it, and the topic is still highly contentious. As Gaming and M&E Evangelist for the Digital Creation suite at Perforce Software, I’ve talked to many studios who have outright banned it, for fear of the legal ramifications of using AI that was trained on potentially copyrighted works. 

Still, some studios are cautiously experimenting with it. As the world adjusts to this breakthrough technology, we are bound to see its use across M&E pick up gradually. 

Let’s look at the current climate around AI in film and media and discuss some of the possibilities it offers. Then, we will dive into the technical challenges of implementing it — including the need to assess your data management infrastructure.

Current Conversations Around Using AI in Film, TV, and Other Media

The use of AI in film and media is obviously still a contentious topic, just as it is across industries. Let’s look at some of the ethical and legal considerations of using this technology, then go over some of the potential benefits. 

Ethical Considerations

The media and entertainment industry is still reeling from the history-making strikes in Hollywood last year. While agreements between the Writers and Actors Guilds and the Alliance of Motion Picture and Television Producers (AMPTP) included protections against AI, there are many other roles across the industry that could be impacted by its adoption. 

Some industry vets have spoken out about the far-reaching effects it could have on the industry — like Tyler Perry, who halted an $800 million expansion of his studio because of gen AI. Perry expressed concern that AI will “touch every corner of our industry,” affecting the jobs of everyone from actors, to sound technicians, to editors, and more.

Legal Considerations

Whether or not studios even have the right to use AI-generated art in their projects is still a bit murky, legally-speaking. As an example, in a copyright lawsuit filed by visual artists last year against Stability AI, Midjourney, and other AI image generation companies, the court dismissed many of the artists’ claims (including copyright infringement), though a claim for direct infringement against Stability AI was allowed to proceed. 

For now, it seems that generative AI has advanced too quickly for the law to catch up, which is why some studios (like tabletop games developer Paizo) have banned the use of any off-the-shelf gen AI tools for art. 

The Obvious Benefits for Studios

Among the studio leaders I’ve been in conversation with, the consensus seems to be that AI is here to stay, so the best way to proceed is cautiously, and to focus on using it to improve manual processes. By automating time-consuming processes like mockups, storyboarding, and editing, AI has the potential to save studios a lot of time, and to reduce a lot of the tedium in their current processes.

The Real Benefit of Using AI in Film and Media Creation: Eliminating Tedium 

I can’t imagine a world where people are happy to watch a movie they know is totally or even mostly generated by AI. We’ll always need human beings to moderate its input and output. For one, there’s evidence that AI’s output breaks down if it is continually fed AI-generated content. Original, unique input from people is necessary to prevent it from turning out nonsense.

There’s an ineffable quality to real art made by real artists.

The real power of AI lies in its ability to automate the tedium out of creative work.

Examples of How AI Can Be Used in Film and Media Production

Here are just some of the ways AI can help automate repetitive or dull, time-consuming tasks in filmmaking and media creation:

Preliminary Mockups or Storyboards

Gen AI can help give rough shape to a character, object, or scene, or even generate a rough storyboard. These can serve as a starting point so an artist doesn’t have to create something completely from scratch.

Recreating Real Spaces and Objects

Manually creating a digital replica of a real-world space is painstaking. With neural radiant fields (NeRFs), a technology that uses AI and machine learning, you can recreate 3D environments with stunning photorealism from 2D images. 

Generating Realistic Background Characters

Designing hundreds or thousands of unique background characters with unique faces would be incredibly time-consuming. AI can automatically generate countless characters to populate your scenes.

Aging and De-Aging

AI has already saved actors like Tyler Perry and Tom Hanks hundreds of hours in the makeup chair. Gen AI has been used in films (like the upcoming film “Here”) to age or de-age them in post-production.

Editing and Post-Production 

AI can streamline the editing and post-production processes by automating tasks such as color grading, sound mixing, and removing objects from scenes. AI-powered editing software can analyze footage and make recommendations.

We all remember the infamous Starbucks to-go cup in Game of Thrones. If they had been able to use AI in editing then, it’s possible it would have caught it before the episode was shipped.

File Management

AI can help rename files, automatically apply textures to thousands of files, auto-tag files for quick bulk uploads, apply descriptive metadata, and create more accurate transcripts and captions. AI can also make it faster and easier to find existing assets in a digital asset manager. 

The Technical Challenges of Using AI in Filmmaking

If your team is curious about using AI to speed up certain processes or help automate rote work, here are the technical challenges you should keep in mind before beginning to implement it.

Quality Control 

Humans need to closely review and revise everything generative AI creates. Just take the outrage over the AI-generated skeleton in “Late Night With the Devil” for example. People want the art they see to be made by other people, not computers. If it was clearly created by a computer with minimal editing and without care in how it will be received by your audience, your reputation will tank. 

Data Management

As more studios adopt modern tech and methods like game engines and virtual production, they’re already having to manage more data than ever before. It’s a term we’re calling the 10x environment. AI will only introduce exponentially more data into the picture, and teams will need to have scalable and powerful infrastructure in place in order to manage it.

Storage and Computational Limitations

It takes a tremendously large dataset to train AI, so studios who opt to train AI on their own intellectual property will need storage for millions of files, as well as very powerful graphicly (GPU) intense server hardware. 

To Use Gen AI, Film & Media Studios Need a Modern Tech Stack

AI is poised to permanently change the way the media & entertainment industry creates art. Some studios are bound to feel pressure to adopt it sooner than later, for fear of falling behind the competition and being overlooked in favor of studios who can create better work, faster, thanks to AI. If your studio is looking into implementing it, for any reason, it is imperative to pause and ask yourself: Can our tech stack handle the massive amount of data involved in using AI?

In my role at Perforce, I’ve worked with M&E studios of all sizes, and I can tell you it’s quite possible the answer to that question is “no.” Many studios, no matter how big or small, are working with very outdated development pipelines — pipelines that are already buckling under the weight of modern technologies like real-time 3D engines, neural networks, and distributed rendering.

Here are some signs your studio’s development pipeline is outdated and will not be capable of handling the influx of data involved in using AI:

·      Your artists have to save their art to their local machines.

·      Your team exchanges large art files via email or file sharing services.

·      You frequently run into file conflicts and storage issues.

·      You track file versions manually or with complicated custom scripts.

·      Collaborating on the development of digital assets is usually slow and cumbersome.

If that sounds like your team, it’s time to update your data management infrastructure. Generative AI is going to be next to useless to you if your team doesn’t have an efficient system to manage all of its output. And it’s even more important to have a powerful and scalable data management system if you plan on training AI on your own IP, since training AI models requires a massive dataset.

If upgrading your tools sounds like a costly, time-consuming undertaking that is not in budget: that’s understandable. Many studios are still recovering from the turmoil of last year, and even in a good year, it’s difficult to allocate funds for a project like this. But this is not a can you want to kick down the road forever. Now is the time to assess the weak points in your infrastructure and start planning how you will solve them.

And, as a tip, there are industry experts who can take on this work for you and set you up with a future-proofed pipeline. Learn more about solutions from Perforce + ICVR, and book a free consultation. >>

Scalable, Secure Version Control for Generative AI Data Management

If you’re developing art for movies, television, or other media using modern technology like game engines, then you need a version control system. For a film or VFX studio, “version control” might mean a file-path based network storage solution, but to software developers and the kinds of teams who have been using game engines for decades, it’s a specific kind of software. You can learn all about version control here, but one thing to note is if you’re using a game engine, you need version control software because game engines need consistent file names and paths. Managing these files manually is a nightmare.

So, a version control system is essential for working with game engines or real-time 3D engines, but depending on the volume of assets you’re working with, a robust system is a necessity. If you’re creating complex 3D environments for film, TV, virtual reality, or other media, you will want something more scalable, secure, and performant than Git or legacy tools like Clearcase or Subversion.

Perforce Helix Core is the version control platform trusted by 19/20 top AAA game development companies. They rely on it not only for its seamless integration with leading game engines, like Unreal Engine, but for its ability to handle petabytes of data. More importantly, though, it lets teams more easily collaborate on the development of art and 3D assets by preventing file conflicts and giving the entire team one secure, central location to store and track every iteration of every file. As game engine tech spreads to creators across industries, top VFX studios have adopted Helix Core to streamline and accelerate production. For example, DNEG uses it for multi-vendor collaboration on Unreal Engine projects.

Learn More About Helix Core, and Get it Free for 5 Users

Learn more about Helix Core, or try it out yourself — it’s free for up to five users.

GET HELIX CORE FREE