What is a Virtual Set? Overview of the Tech + Best Live Virtual Set Software

With live virtual set software, you can film from seemingly anywhere and quickly jump between locations without leaving the room. In this blog, we cover the basics of virtual set software, including green screen virtual sets.  

Read along or jump ahead to the section that interests you the most.

➡️ FREE VIRTUAL PRODUCTION COURSES

What Is a Virtual Set?

A virtual set, AKA a virtual studio set, is an environment in a studio that allows teams to manipulate a digital background. This allows filmmakers and broadcasters to alter things like the location, weather, and time of day using software. Teams can place on-set actors anywhere!

What’s the Difference Between a Set and Virtual Set? 

The main difference between a virtual set and a traditional set is the ability to manipulate the background. Teams can use a variety of methods — green screen, LED walls — to transform the scenery. Because virtual sets are created and displayed digitally, teams can film at multiple locations in one day, on a single stage.

What is the Advantage of a Virtual Set?

There are a lot of benefits when it comes to using a virtual set. You can:

• Switch to different locations without needing to travel on location.
• Control the weather and time of day, allowing more flexibility in shooting schedules. 
• Create environments and objects that don’t exist in real life.
• Allow the director, camera operators, and talent to see the virtual environments in real-time.
• Make last-minute changes to camera angles, background scenes, and lighting while filming, giving teams greater flexibility.

How The Batman Used LED Wall Virtual Sets

Warner Bros. Pictures’ The Batman used virtual sets with large LED walls/screens to plunge viewers into the streets of Gotham City. The backgrounds were created digitally prior to filming, and then game engines were used to render them in real-time on LED screens behind the props and actors.

Unlike green screen virtual sets, LED walls can offer more accurate lighting and reflections while also allowing the entire cast and crew to see the environment around them. Although they cost more up front, teams can spend less time in post manipulating scenes. 

Video game technology — like Unreal Engine and Unity — allows artists to create photorealistic 3D backgrounds (like the ones used in The Batman). Combined with motion tracking, the game engine can update the background images to perfectly match the movements of the camera. From the camera’s perspective, it looks as if they are actually filming inside a real 3D space.

Unlike being on a traditional set, teams can adjust lighting and reflections inside the game engine. These techniques are unavailable with green screens. For actors, LED walls allow them to physically see the final images that will end up on television and movie screens. 

Green Screen Virtual Sets

Green screens can also be used to create a virtual set with virtual production techniques. Rather than displaying images on LED walls in the background, the image from the camera is processed in real-time by the game engine. It removes the green and replaces it with the virtual set. Then it is broadcasted back to the camera operator, director, and producers while filming.

There are certain disadvantages of green screens, such as not providing realistic reflections and lighting from the background and not allowing the actors to see the environment. But, there are certain areas where green screen virtual sets shine.

Why Do Studios Use Green Screens Virtual Sets?

Because green screen virtual sets don’t rely on the resolution of the LED panels, the camera is able to focus clearly on objects and people in the background. It can record in the virtual “distance” while filming without concerns about moiré or aliasing that you would have on an LED screen. Green screen virtual sets also have a lower infrastructure cost. Building a green screen stage is much cheaper and can be more easily changed than an LED stage.

Whether you are using a green screen or an LED wall, real-time software is key to allowing creatives to view the environment while filming, instead of waiting until post-production adds in the backgrounds. When using real-time software, version control ensures you are always using the correct versions of the virtual sets while filming.

How to Build Your Virtual Set

At Perforce U College of Virtual Production, you can learn how to use the live virtual set software top studios are using on set.

✨START LEARNING✨

How Do Virtual Sets Work? 

Virtual sets work by bringing together digital assets created beforehand with live camera tracking. Teams can use a variety of software and hardware components, depending on the nature of the final project. 

How Do You Create a Set Using Live Virtual Set Software?

The process for creating a virtual set depends on your needs and the scale of your project. Whether you use a green screen or LED wall, though, the first step is using digital content creation (DCC) tools to build a digital environment. Much like creating a game asset, making assets for a virtual set involves using 3D modeling programs like Maya, Blender, and zBrush and art programs like Photoshop and Substance Painter to create props and set pieces for the virtual environment. You might then bring those together into a game engine like Unreal (more on that below) to arrange and finalize the scene.    
    
Once your virtual environment is built, you will set up the virtual camera within the game engine to sync with your physical camera, using a camera tracking system. This will ensure the perspective of the camera is coordinated with that of the environment displayed, so that the scene looks realistic.

Let’s dive into the game development technology reshaping how studios build virtual sets.

Unreal Engine

A commonly used game engine for virtual sets is Epic's Unreal Engine. It was used for Ford v Ferrari, Westworld, The Mandalorian and more to create next-level images. Virtual sets can be created in Unreal Engine during previs and adjusted in real-time over the course of production, allowing teams to tweak scenes and virtual production images, even while on set. 

The 3D assets can also be reused in post-production for full CG scenes, or in marketing images to promote the final film. Being able to share virtual set files helps unite cross-functional teams and teams spread around the world. 

➡️ LEARN HOW TO USE UNREAL ENGINE

Unity 

Unity has been used by game development studios for years. Recently, they are breaking into the virtual production space with the purchase of Weta Digital. This game engine offers similar capabilities as Unreal Engine and is behind CG animation for the Jungle Book and Blade Runner 2045. 

➡️ LEARN HOW TO USE UNITY

Power Behind Game Engines

Both Unreal and Unity game engines allow teams to create new worlds which can be dropped into a virtual set. But…

• How can teams manage the numerous large assets and multiple iterations associated with virtual set production?
• How can teams deliver files and feedback to dispersed contributors in real-time?

The solution: version control. It can solve collaboration challenges, providing a foundation for digital asset management. That way creatives can focus on innovation, not their tools.

Version Control = Foundation for Digital Asset Management

A version control system creates a single source of truth for digital assets. It allows team members to work together without stepping on each other’s toes or losing work, while ensuring each artist has the most up-to-date files.

Small studios may use Google Drive or even a shared hard drive to store and track digital assets to be used on set. But these options can be slow, unsecure, and don’t provide the ability to restore previous versions or avoid accidentally deleting or overwriting files. 

Why Helix Core Is the Best Live Virtual Set Software 

For top virtual production studios, the solution is Perforce Helix Core. It seamlessly tracks and manages every change to every file, and it allows you to manage your team’s changes to files, ensuring more effective collaboration. With Helix Core as a secure foundation for all of your virtual set assets, your team can make quick changes to creative and take on more technically complex challenges.

The tight integrations between Helix Core and game engines have made it the standard for game development (95% of AAA studios use it). Plus, Perforce is part of the Trusted Partner Network (TPN), making it the choice for studios who need to protect clients’ intellectual property. With Helix Core, teams can work quickly. The high-performing system even makes remote teams feel like they are all working together in the same room.

Why Use Virtual Production? Learn From 4 Studios

Learn 4 key benefits of version control, according to 4 innovative virtual production studios: Dimension Studio, Silver Spoon, Final Pixel, and HaZ Film.

GET THE EBOOK

How ETC Created Virtual Sets with Helix Core

When Entertainment Technology Center (ETC) started working on their short film Ripple Effect, they wanted to build a virtual set and leverage an LED wall. They brought together ICVR, Halon Entertainment, and other studios to create the visuals in Unreal Engine.

To unite contributors, they versioned everything in Helix Core. From the start of the project, they were able to iterate quickly and constantly improve their work. Storing all the content in Helix Core allowed them to leverage a library of assets for different scenes.

“Perforce Helix Core has been a foundational tool. It helps create a single source for all our film's digital assets. We can version our critical IP and digital files within an infrastructure that supports and integrates with leading tools like Unreal Engine, 5th Kind, ftrack, and more.”— Erik Weaver, Director, Adaptive Production & Special Projects at ETC

Creatives and producers didn’t need to guess where assets were located. They were able to quickly find and update their 3D files from anywhere. Being able to efficiently use time on set kept overall costs down. Everyone was on the same page throughout previs, production, and even into post.

Manage Your Virtual Set Digital Assets 

As you embark on this new frontier of filmmaking, you need the best virtual set software to support your dreams. If you need to manage:

• Remote siloed contributors
• Lots of large files
• A range of digital assets
• Numerous iterations
• Security for valuable IP

then you need Helix Core. It gives your team a single source of truth through the entire production process.

Binary files, audio files, video files, code, and more can all be stored and securely versioned inside a Helix Core depot. Your teams get lightning-fast performance thanks to Perforce Federated Architecture. You can easily deploy Helix Core in the cloud, and it offers integrations with the tools digital creators are already using — Unreal, Unity, 3ds Max, Maya, and more.

Best Virtual Set Software — Free

See for yourself why so many virtual production studios choose Helix Core. Get started using Helix Core free for up to five users.

➡️ BEST VIRTUAL SET SOFTWARE FREE