This workflow allows you to not only render backplate/LED wall content behind an actor, but also for live-compositing virtual elements from Unreal Engine as a frontplate, over the live camera footage & backplate virtual set extension content.
Unreal Engine Configuration
In your Unreal Engine level, you will need two Cine Camera Actors. One for the Frontplate camera and the other for the Backplate camera. In this example, we will have two cameras called XR_BPcam and XRFP_cam.
Both will need a Render Stream Channel Definition component; however, their Default Visibility will differ. The backplate camera will have assets be Visible by default, while the frontplate will be set to Hidden, as to be selective of what’s composited over our live actors/camera feed.
From there, you will choose which assets in your scene will be forced to be visible for the Frontplate and which assets will be Hidden for the backplate in order to prevent you seeing duplicates of the same asset between the foreground and background, trailing after one-another.
With your front plate camera selected (XR_FPCam, in this example), and under its components select the RenderStreamChannelDefinition, then scroll down to Visibility. Here you will see two lists: Force Visible, and Force Hidden. As their names imply, this is where you select and add assets to either to force assets/scene objects to be rendered/hidden.
In either list, you can click the plus icon to Add Element, which allows you to add an asset from your scene.
This will add another index to the respective list. Next to a given entry, you can click the eyedropper tool and select an asset from your scene, to make it visible/hidden.
As mentioned, your backplate camera (XR_BPcam, in this example) should render/have everything be visible by default. As such, you will be selecting which elements will be hidden, as they will be brought “forward” into the frontplate to be composited over our camera feed.
Meanwhile, our frontplate camera (XR_FPcam, in this example) will have everything hidden, by default, otherwise, our entire scene will be rendered over our camera feed. We want to be selective about what’s rendered/composited over the camera feed. As such, we select assets that we want to be visible (which will likely be the very elements we’ve hidden on the backplate camera).
That said, it is recommended that for your frontplate camera, go into the RenderStream Channel Definition component, then under the Details panel, scroll down to Scene Capture > Advanced > General Show Flags and disable Atmosphere and Fog, or they will render over your camera feed and backplate even if everything is hidden, by default.
From here, save your projects and transfer your project over onto the RXII servers. You will need to ensure that your project is copied onto both render servers. You can find a reference on transferring your Unreal Engine project to the disguise Render servers here.
Disguise RenderStream Configuration
Potential Issue: I can't see my Unreal Engine cameras as Channels!
In the event that you are setting up your channel mappings for your Render Workload in disguise (d3), and you do not see your Unreal Engine cameras (whether a specific one, or both), go to your project's root directory and find the RenderStream metadata JSON file (it will usually be called rs_<yourprojectname>.json).
Open the file, and examine the section/field called Channels. With the SubwayTrain scene, we should have two cameras called XR_BPcam and XR_FPcam. Depending on your scene and how many cameras/channels you need, your cameras should be accounted for in the Channels list. Ensure they are named exactly as they are in your Unreal Engine scene.
In the event they are named incorrectly or one is missing, either rename the channels to match the names in the Unreal level or, if they're missing, add a comma after the last channel entry and type in the name of the other channel, in quotations, after a new line. In the event you need to add several cameras, each one (aside from the last channel) will need to be proceeded by a comma and newline.
Configuring in Disguise Designer
Similar to how you’ve configured a backplate-only workload in disguise Designer, you will need to configure your RenderStream layer to render this scene on two separate servers, one dedicated to the backplate and the other to the frontplate/AR elements.
Go to your timeline and to a RenderStream layer by either selecting/left-clicking an existing one or creating a new one (right-click on the timeline, name the layer, select 'RenderStream').
From there, look at the newly opened RenderStream properties window that’s opened up at the top-left of the interface, and right-click Workload to go further into its properties. This is where we will configure this RenderStream workload for backplate & frontplate multi-server rendering. We will go over the configuration process.
Starting with a fresh RenderStream configuration. Left-click on the RenderStream layer on your timeline. This should open a properties window for it on the top left of your interface. Right-click on Workload.
First, select your project under the Assets field.
Then, for Cluster Pool, we will need both machines. As such, select a+b as this includes both RXII-A and RXII-B for this workload.
\
When you select a multi-render server setting, this will cause your selected project to be highlighted in red with an error: No content source set, content syncing is not possible!. When using a multi-render workflow, we need to designate a render machine to be the content source machine or the reference machine from which all other machines will sync/copy the project from in the event of a change.
We can do this by right-clicking on the Asset field.
In the Asset's properties window, left-click the Content source machine field.
And select the render machine you would like to be the reference. In this case, we will select RXII-A.
From here, we should be set to return to the Cluster Workload window and continue down to Default Assigner.
Set the Default assigner to Default, as we will be setting this on a per mapping/camera basis.
Look under the Channel mappings section, click on New Channel Mapping.
This will be populated by one of your Unreal Engine cameras. It may initially be mistakenly assigned to both RXII servers.
For now click on New channel mapping below.
Select your other camera, the frontplate camera (XR_FPcam) in this case.
Now one server should be assigned per camera/channel mapping.
Let’s make sure each channel is mapped to the correct plate. The backplate camera should be assigned to emerson_xr_target (backplate). As you may have guessed, the frontplate camera ought to be corrected to emerson_xr_target (frontplate).
Finally, let’s correct the Assigners for each channel. The backplate camera should be set to Inner Frustum Full.
The frontplate camera channel should be assigned as AR.
Make sure that the front-most channel (the frontplate in this case) is on the bottom of the Channel mappings section, otherwise it will be covered by other active channel feeds. You can left-click and drag to change the ordering of the Channel mappings.
Your workload should be configured and set to launch. Feel free to press Start.
Allow the scene to start, compile, and begin being output to the LED wall and through the XR camera feed. Have someone step into the camera and see if the front plate elements are compositing correctly. Enjoy!