Virtual Production: Are You Game?
Virtual productions using video game technology to combine pre-production, production, and post-production on the set have been, for the most part, the private reserve of big-budget films and series. Fortunately, there are now tools you can use to get familiar with the concept or to use it for free on a smaller scale in case you ever find yourself in a tight spot, like say in a pandemic.
Using gaming engines on the set
Virtual productions allow actors to be filmed and inserted live into virtual 3D sets instead of having to create special effects in post-production. In series like The Mandalorian or Westworld, scenes are displayed on huge LED walls in the background. This makes filming easier, improves lighting, and lets you see the final result of a scene during the actual shoot.
Video gaming engines are what make the technology possible. By synchronizing the physical camera to a virtual camera in the gaming engine, images on the screen adapt in real time to the shots chosen by the director during preview or during actual shooting.
Two of the leading engines on the market, Unreal Engine (Epic Games) and Unity, can be used free of charge for small virtual productions. Anyone wishing to get a handle on the concept will find Unreal Engine the easier choice since Epic Games offers a number of in-depth guides and tutorials on the subject.
Epic’s Virtual Production Primer is a free 15-hour course designed to teach film and TV professionals how to use the software. Epic Games has also published a guide to virtual production, explaining how the technology affects the various jobs on a film set, from cinematographers to stagehands. The company offers a downloadable version of the software as well so you can quickly test your results.
Using real online tools to create virtual sets
Creating a virtual set from scratch is a colossal task. Luckily, it’s easy to find pre-designed sets online. Not to mention 3D objects that can be assembled, photorealistic textures, virtual characters...and much more.
A good example is the Downtown West Modular Pack you can pick up in Epic’s UE Marketplace. The pack includes hundreds of urban environment objects that can be used to recreate any typical American downtown. There’s also a full range of high-resolution environments for recreating forest trails, futuristic cities, seaside settings, and just about everything in between.
There are many 3D artists who also allow their works to be re-used for as little as ten bucks a pop. Some offer complete sets, like a full reproduction of the White House press room, while others post collections of objects on specific themes, like chairs or public benches.
With Quixel’s Megascans service (starting at $20 a month), you can access the same photorealistic textures (including sand, mud, and bricks) that were used in films such as The Lion King, Jumanji Welcome to the Jungle, and Pacific Rim Uprising.
Finding the right hardware
More and more professional studios are now offering all the equipment you need for virtual production filming, including MELS in Montreal.
Anyone wishing to experiment with the technology or to produce an independent production on a tight budget can do so by limiting the technology they use. While an LED wall is often associated with virtual productions, it’s possible to get close to the same quality using a green screen. You do lose some advantages (like context setting for actors and lighting) but the cost of setting up an in-house studio is significantly less. (An LED wall can easily cost hundreds of thousands of dollars, depending on the size and the technology.) If the concept suits the purpose, a simple LED TV can also fit in a conventional setting to show the landscape behind a window, for example.
There are some things you can’t get by without, like a computer with a graphics card powerful enough to render the 3D environment in real time. Professional cards like the NVIDIA RTX A6000 are preferred by studios, but some powerful consumer cards like the NVIDIA GeForce RTX 3090 will do the job with a green screen (certainly for 1080p, and also for 4K in certain cases, depending on the complexity of the 3D environment).
A system for tracking movements of the physical cameras and linking them to the virtual cameras in the 3D environment is also a must-have. While there are professional solutions designed for virtual production and motion capture like OptiTrack, you can also get good results with Vive Tracker sensors, which are compatible with HTC Vive virtual reality systems. Depending on the camera, you might also need a video capture card like Blackmagicdesign’s DeckLink.
For anyone interested in putting together an in-house virtual production studio, cinematographer Matt Workman’s YouTube Cinematography Database channel is a good place to start. The channel regularly posts videos on the different types of equipment currently available. If you’d like to do some experimenting in a professional environment, get in touch with the Screen Industries Research and Training Centre at Sheridan College in Toronto, Ontario. The centre offers a studio for research and development projects, as well as online courses in virtual production.
So, are you game or not?
Dig deeper
Find out more about virtual production by listening to the following episodes from our podcasts Now & Next and Futur et Médias:
"Virtual production techniques for film and TV: sky is the limit" - Now & Next podcast
"Under COVID-19 restrictions, pre production and tech come under the spotlight" - Futur et Médias podcast (in French with English transcript available)
