Filming with an Inner & Outer Frustum

February 28, 2025 14:32

Overview

When operating a virtual set in a larger volume, what your camera sees defines the bounds (or frustum) of the rendered environment on the wall. This portion is often at the highest quality or as-needed for the shot. However, we still would like our actors to look as though they are truly in the scene. What if there's a sun illuminating them from beyond the frame? Or a car is driving through a tunnel, and we wanted to capture the windshield reflections while filming from the front? This is where the outer frustum comes in, and is a lower quality render of the environment that covers the rest of the volume that isn't actively in the camera. This serves two purposes: to save on computational power (allowing for it to be re-allocated to higher priority renders) while still providing supplemental lighting and reflections from all the directions the volume covers. The frustum that the camera sees is called the inner frustum.

This workflow will go over setting up your Unreal Engine scene for an inner and outer workflow, as well as covering how to configure the front Quasar LED light fixtures.

Inner & Outer Frustum on the LED Wall

Unreal Engine Configuration

This is assuming that you have already created your environment. We will not be going over any environment creation tools/processes.
Your scene will need two cameras: one for the inner frustum and the other for the outer frustum. The scene we're working off of in this example already has a camera called BP_Camera, which will serve as the inner frustum camera. For the sake of clarity, we'll rename BP_Camera to Inner_Camera.
Select Inner_Camera, duplicate ([CTRL] + [D]) and rename the second camera to Outer_Camera.
We now have our two cameras/RenderStream channels. From here, the project is technically all set to be transferred to the RXII Render servers through the Project_Dropoff folder on the EML-XR-NAS. However, you will want to check to make sure that the Renderstream metadata file is updated with the correct channel names that correspond to your Unreal Engine cameras.

disguise Designer Configuration

Create a RenderStream layer on your disguise Timeline.
1. Right click on the timeline, name your new layer.
2. Select RenderStream.
A new properties window should open on the top-left of the interface for the RenderStream layer.
Right-click on Workload.
For Asset, choose your Unreal Engine project. In this example, it will be XR_Example3.
For Cluster Pool, we will need to use multiple Render Servers. As such, we will choose the a+b mapping.
Since we are using a multi-render workflow, we will need to designate a render machine to be the content source machine or the reference machine from which all other machines will sync/copy the project from in the event of a change.
We can do this by right-clicking on the Asset field.
Click on the Content Source Machine. You will see a list of available Render Machines. We'll select RXII-A in this case.
From here, we go back to the Workload configuration window. Let's have the Default Assigner set to Default. We'll set the Assigner on a per channel basis.
Next, proceed to Channel Mappings. Create two new Channel Mappings.
Replace the second channel to be Outer_Camera.
Let's go down each channel, and across for each of their fields. Let's start with Inner_Camera.
Inner_Camera should be set as follows: emerson_xr target (backplate) for Mapping, inner frustum full for Assigner.
Outer_Camera should be set as follows: emerson_outer target (backplate) for Mapping, outer frustum for Assigner.
Make sure the Inner_Camera channel is at the bottom, otherwise it will be covered by the Outer_Camera/outer frustum feed.
Finally, check to make sure the correct scene is selected for the RenderStream workload. We'll be rendering Example_Map.
From here, press Start on the bottom of the Cluster Workload window, or alternatively right-click on the RenderStream layer on the timeline and Start workload.
Our scene should now be on the wall, though the virtual Unreal Engine camera may not be in the correct position/orientation. We can fix this using a custom Scene Origin layer.

Creating the Scene Origin Layer

On the timeline, right-click and create an Open Layer.
Next, we will need to access each mappings Scene Origin in order to add them to our Scene Origin/Open layer. This will allow us to quickly change the origin(s) as needed.
Ctrl + Left Click on the current RenderStream layer and go to the Cluster Workload window (right-click on Workload). We'll be going through each camera/channel's Mapping.
Right-click on the Mapping for Outer_Camera: emerson_outer target (backplate).
With the settings window for the emerson_outer target (backplate) mapping open, and the Scene Origin layer open (Ctrl + Click that layer on the timeline if it was closed), Alt + Left Click from the top of the Scene Origin layer and drag into the Scene Origin field under the emerson_outer target (backplate) properties. You should see a white arrow extending from the Scene Origin layer. Let go of the left-mouse button once you are hovering over the Scene origin field in the properties window of emerson_outer target (backplate).
That Scene Origin should now be added to the Scene Origin layer. Do the same thing for the Mapping for Inner_Camera: emerson_xr target (backplate).
From here, we should have two Origins in our Scene Origin layer.
Let's change them both to be the same origin, we'll make and configure a new one: example_origin.
Make sure your RenderStream layer is running, and see if the current virtual camera position looks correct. You should see the inner frustum rendered at a higher quality, while the outer frustum rendered at a lower (25%) quality.

Environmental Lighting on the Quasars

At this point, we have managed to configure the inner and outer frustum to be rendered on the wall; however, you may find that the outer frustum in this configuration does very little to provide supplemental environmental lighting on our subject(s). We only have a flat wall, and not an entire curved & surrounding volume.
That said, we do have the Quasar LED light fixtures, as they can provide supplemental front lighting.
Let's configure our RenderStream workload to utilize these lights.
For now, stop the workload.
Next, bring up the Cluster Workload properties window for our RenderStream layer.
Change the Mapping for the Outer_Camera to emerson_outer target (quasar backplate)
Next, change the Assigner for the Outer_Camera to emerson_outer target (quasar backplate).
Start the workload.
Once the workload is running, pay attention to which render server is currently rendering the Outer_Camera channel under Pool Usage. In this case, it is RXII-A.
Next, on the top navigation bar of the Disguise Designer interface, go to Devices and right-click on that button.
Click on Video Input Patch Editor.
Click on Switch to Quick Edit.
Go down to videoin_8.mov and click its entry/value for Input, currently Unmapped.
We will need to find and select the currently active video stream being rendered by RXII-A (the input stream is titled RXII-A-56486 (13294467740054570097) Compressed in this example). Please note that every time you restart the RenderStream workload, you will need to reselect the video stream like this as a new one will be generated.
Next, create a Video Layer in the same section of the timeline as your RenderStream and Scene Origin layer.
Make sure the Video Layer's Mapping is set to quasar both.
Next, set the Video content to Videoin_8.mov or video input 8.
And now, your Outer Frustum stream is being actively mapped onto the quasars. For reference, we have a virtual camera in our template disguise project (called outer_camera-rear) that is parented to the tracked camera, but pointed back and upwards in order to provide a real-time feed of that portion of the virtual environment, and have it displayed on the quasars. If you move the tracked camera, it should update both the wall and the quasar light feeds.
You may find the quasars too bright initially, so adjust it by playing around with the intensity of the active video layer. You can find the intensity/brightness value next to the lightbulb icon of a given layer.
From here, refer to the Atomos HDR monitor or the camera's monitor as reference, and adjust your scene/lighting accordingly!