Authoring a Virtual Environment in Unreal Engine for the XR Studio

February 09, 2025 17:22

Overview

This is a basic overview of navigating Unreal Engine in terms of controls and interface elements, as well as going over creating/sculpting terrain, importing assets, and painting foliage/assets along your scene. This is not meant to be a comprehensive guide on environment design. Currently, you will need Unreal Engine 5.5.1 which should be pre-installed throughout the EML. This example will work off of a custom XR Basic Template which has the disguise plugin and project settings pre-configured for loading into the XR Studio/disguise Designer.

Unreal Project Setup

Launch Unreal Engine.
The project launcher should open. Assuming you are working on an EML workstation, you should see a custom category on the left titled EML XR Studio.
Under that category, you should see a template titled XR Studio Template. Select the template, name the project, then create.
You will open to a mostly empty level with a floor and a chair asset.
You are able to look around the viewport or your 3D scene by holding the [Right Mouse Button] and moving around using the [W], [A], [S], and [D] keys while continuing to hold the [Right Mouse Button].
You are also able to increase/decrease your flying speed by holding the [Right Mouse Button] and scrolling the mouse wheel.
Whether in the 3D viewport or using the Outliner, select the Floor and Chair (SM_Chair) and delete them from the scene. You can either press the [Delete] key on your keyboard, or right-click on them in the Outliner then go to Edit > Delete.

Creating & Sculpting your Landscape

Switch to Landscape Mode by either selecting from the mode dropdown on the top-left of your Editor or by pressing [SHIFT] + [2] on your keyboard.
Assuming you do not currently have a landscape object in your level, you will be prompted to create one. There are a number of properties you are able to configure such as the size/number of sections of your landscape, the active landscape material, the landscape object's location, etc. Set these as needed. For this example, we will use the following settings:
Material: M_Ground_Grass
Location: 0,0,0
It is recommended to have your landscape at the origin's level. If you can, build your scene around the origin point (0,0,0) as that is where your camera perspective will be on the wall, when you are bringing your scene into the XR Studio/onto the wall. If you do not, it is all right however. You are able to overwrite the Unreal Engine camera location when you are operating the XR Studio when using the disguise Designer software later on.
Rotation: Keep as is
Scale: Keep as is
Section Size: 127x127 Quads
Sections Per Component: Keep as is
Number of Components: 3x3
Overall Resolution: Keep as is
Press Create.
From here, you are able to click and drag in the viewport over your landscape to raise/sculpt your landscape. Doing so while pressing [SHIFT] will allow you to reverse your current tool. You're able to adjust your tools' mode and settings on the left side bar. The bracket keys can quickly allow you to change the active brush size.
Once finished, Save (Ctrl + S) and go back to Selection Mode. Now let's move onto importing assets from the Fab marketplace.

Importing 3D Assets using the Fab Marketplace

Go to Window > Get Content > Fab. Please note, you will only see this option if the Fab plugin is installed to the current version of Unreal Engine.
This will launch the Fab marketplace window. Click on the profile icon on the top-right, and login using either your personal Epic Games account if you have one, or create a new one by logging in via Google using your Emerson email (or using another personal address).
Once logged in, feel free to search up assets, as needed. For instance, let's search for Quixel, who are the creators/maintainers of the Megascans Library. To see creator profiles, switch to the Creators tab at the top.
Click on the Creator's profile, and filter/browse through their available assets.
In this example, we will look through Quixel's Nature and Plants section.
Let's select Norway Maple, then Add to Project.
Depending on the size of the assets, it may take some time for them to download into your project. You can see the progress of the download on the bottom right.
In this example, we will also download Field Poppy, Grass Clumps, and Nordic Forest Ledge Rock Medium. Some available assets allow you to set what quality level you can download them as (this can range from low quality to raw quality).
Return to the editor, and Save All (Ctrl + Shift + S).
You can find your Fab-imported Megascans assets by navigating to the Fab directory in your Content Drawer (Ctrl + Spacebar), then Megascans. You will find separate folders for 3D models and Plants. Our Norway Maple tree pack was downloaded into the NorwayMaple folder directly located in the project's Content folder.
For example, by navigating into the Nordic Forest Ledge directory, I can see the imported material, textures, and static mesh. You are able to click and drag the static mesh and place it directly into the scene. Press [F] to focus the viewport camera onto the placed mesh. Please note, in this example, the rock's material had to be recreated since the original was not imported successfully from Fab.
While it is possible to add trees and foliage one-by-one, there is a tool that can assist with painting assets all over the environment.

Foliage Painting & Asset Scattering

Go into Foliage Mode (Shift + 3).
This will open sidebar containing (from top to bottom) the foliage painting tools, tool properties, and foliage palette. Let's add the foliage that we've imported from the Fab marketplace.
Open the Content Drawer (CTRL + Spacebar). Starting with the Field Poppy, let's navigate to where we can find their Static Mesh Foliage objects, which can be found under Content\Fab\Megascans\Plants\Field_Poppy_vmcobd0ja\High\FoliageTypes.
Click on the first of the Static Mesh Foliage objects (green outline), then [SHIFT] click the last one. Make sure all of them are selected. Afterwards, click and drag these objects into the bottom of Foliage mode sidebar or the Foliage palette.
Your Foliage Palette should now contain the imported assets that you have just added. You are able to select one or several assets to be painted into your scene by hovering over their icon in the palette and toggling the checkbox on the top-left. You can also select several (or all using [SHIFT] + [A]) and toggling several at once.
From here, at the top of the Foliage settings sidebar, you can adjust your brush settings, as well as filtering over what surfaces/assets can the 'foliage' be painted over. Furthermore, there are a variety of other tools aside from the default Paint mode.
Select your desired foliage/assets and set your brush settings accordingly, then similarly to using the Landscape sculpting tools, click and drag along the landscape to paint/scatter your selected assets.

A Note on Trees

Depending on the asset pack that you've imported, in the case of Quixel Megascans and their tree/forest packs, they may be in a slightly different location. They'll be in their own folder in the project's Contents folder.
To find the Static Mesh Foliage objects for the trees (if they've been imported from Quixel), under the root folder for the imported asset, go to Foliage> Simple Wind. Similarly, these can be dragged into the Foliage palette.
Alternatively, rather than tracking each individual set of Static Mesh Foliage objects, go to your project's root directory, then click the dropdown arrow beside the Content Drawer's search bar, and filter by Foliage > Static Mesh Foliage.
Now, as a note, when painting using Trees, it is recommended that you set your brush's paint density significantly lower as they will be clumped too closely together. A paint density setting that may have worked well for grasses and flora, may be far too dense for trees (at least, in the case of Quixel's packs). Start from a Paint Density of around 0.002, then adjust the density and brush size accordingly.

Lighting & Atmosphere

While in the Editor viewport, hold [CTRL] + [L] and drag your mouse around to adjust your scene's default Directional Light, which controls the direction of the Sky atmosphere's Sun.
If you need additional lights, you are able to add more by going to the Add Content dropdown (Cube with the green plus) > Lights.
If you are looking to, for example, have your scene look more overcast then consider looking at the Sky Atmosphere and Exponential Height Fog.
1. With Sky Atmosphere selected, look under the Details section on the right-side bar and scroll down to Atmosphere - Mie. Let's set Mie Scattering and Mie Absorption Scale to 2.0.
Select the Exponential Height Fog, scroll to the Exponential Height Fog Component. Let's set the Fog Density to 0.15,and Fog Height Falloff to 0.1. Then under Second Fog Data, set Fog Density to 0.25.
Now your scene should look mildly cloudier and foggier.
There are many options and methods to craft and customize your virtual environment through a combination of assets, landscaping & layout, and volumes. What we currently don't cover though may be of further relevance later is Post-Processing as well as Optimization.
Save your level [Ctrl] + [S] and entire project [Ctrl] + [Shift] + [S].

Transferring your Project into the XR Studio

Potential Issue: Misnamed RenderStream Metadata file

Heads up: Go into your Unreal Engine project's file directory. In the root directory, containing the .uproject file, there should also be a JSON file that may be named rs_tp_xrstudiobasic. If you see this file, and not a file titled rs_<your project name>, rename it in all lowercase to match your project's title. For example, if I have a project titled Example_Project, that RenderStream metadata JSON file should be named rs_example_project. If this file is not named accordingly, it may make it difficult to select your scene and map your Unreal Engine cameras as channels in Disguise Designer later on.

Transferring from Workstation to XR Studio

Go to the EML-XR-NAS using the shortcut or using the address '\\eml-xr-nas.emerson.edu'.
For Unreal Engine and/or any projects requiring real-time rendering, they will need to be copied directly into the Project Drop-off folder on the NAS. This will sync projects into the RenderStream Projects directory on the RXII Render servers (both A and B).
Copy your Unreal Project directory into Project_Dropoff.
Remote onto the Director server, whether using the KVM on the XR Operator's station or Parsec on the student workstations.

Configuring the RenderStream workload in Disguise Designer

We will be in the EML_General disguise (d3) project, using the track titled example_track.
On the timeline below, click on the tick marks where you would like your layer to start. The blinking white indicator shows your current position on the timeline. Right-click and a prompt will open up, at the top of which you can name your new layer.
We will name this layer RS - Example Project. Press OK or hit [ENTER].
You will be prompted to select a layer type. Under Content, click on RenderStream. The RenderStream layer will be created on the timeline. This layer type is primarily what you will use to configure real-time rendered content, , such as Unreal Engine projects, for the LED wall and within the camera.
From here, the new RenderStream layer is created on the timeline. Left-click on the new layer, then locate its properties panel on the top left of the interface.
On the RenderStream layer properties, right-click on Workload. This will open the Workload properties window and is where you configure what scene/content will be rendering on the RXII render servers.
Left-click on Asset, this is where you will select your Unreal Engine project. You will see a list populated by all available projects contained on the render servers' RenderStream Projects folder. Let's search and select xr_example for now.
Next, click on Cluster Pool, this is where we choose which server(s) will be responsible for rendering our chosen project. For now, we only need one server so 'a only' or 'b only' should do well for now.
Next, set the Default Assigner to default, as this will decide what each channel for our Unreal Camera will be initially set to. Assigners define how content is distributed/handled by each of the nodes (especially in cases where we use multiple) such as setting up tiling, render prioritization, or proxy streams. We will set these on a per-channel basis anyways. Speaking of which...
The next step is to define our Channel Mappings, this will usually correspond to our Unreal Engine camera in the scene, which is pre-configured with a RenderStream Channel Definition component. The Add New Channel Mapping should now be green. Click on it, and it will automatically add a new row for your channel mapping.
Now, since we've created the project from the XR Studio Basic template, using the initial Example_Map level, there is already a Cine Camera Actor (or a virtual camera, in the Unreal Engine scene) with a RenderStream Channel Definition component. This component allows us to see and map the camera (or each of several cameras) as a channel in disguise, for it to appear on the wall, respond to the camera tracking, and- later on- to live-composite virtual elements.
You should see a new row for our BP_Camera (BP standing for 'Backplate') with the two ensuing columns that we'll need to setup: Mapping and Assigner.
To adjust each column's value, left click and a new window will appear allowing you to select from a range of available options. Let's first look at our Mapping which is set to emerson_outer target (backplate). Mapping controls the way output is treated/behaves when mapped on a screen/projectable surface, whether it's mapped directly to the surface or if it follows the camera tracking/interacts spatially in 3d space.
In this case, the preset in the current disguise project, emerson_outer target (backplate) will be used later for an Inner/Outer Frustum workflow. Frustum refers to the bounds of what the camera currently sees, where the scene is typically rendered at the highest quality. In this case, we only need to render the Inner Frustum, and since we are not compositing anything we will need the environment rendered as simply the backplate. Since we care about the camera's position and directionality, we will be using a Spatial Map. Left-Click on emerson_outer target (backplate) in the Mapping column. THis will open a new Mapping properties window. While still looking under the Spatial tab (switch to it on the top of the Mapping window if needed), then set the active Mapping to emerson_xr target (backplate).
Back in the Cluster Workload window, for the BP_Camera channel look at the next column: Assigner. In our case, it is currently set to Default Assigner. Similar to how we switched the Mapping, left-click on Default Assigner under the Assigner column.
This will open a new Cluster Assigner window, providing a list of pre-created options. We are not compositing, as such AR will not be correct. Our scene currently does not have an Outer Frustum, we are primarily working with the Inner Frustum. Finally, we are not splitting the inner frustum. Which leaves us with one option: inner frustum full, which will be the usual option we go with for configuring our scene/virtual set as a backplate. Select 'inner frustum full'.
Your Cluster Workload properties should now resemble the image below. Once confirmed, let's also ensure we are rendering the correct scene. Going back to the RenderStream properties window, left-click on Scene and select your desired level. In our case, it will be Example_Map.
Once you have chosen the correct scene, under Cluster Workload at the bottom of the window, click 'Start'.
Alternatively, from here on, you are also able to right-click on the RenderStream layer on the timeline in order to quickly start/stop the workload as well.
The first time you render your Unreal Engine level, it may need to compile its shaders. Depending on the computational/graphical intensity of your scene and its assets/components, this initial run may take some time.
Unless you have constructed your scene around the origin point (0,0,0), you may find that the initial position of the virtual camera may be incorrect. No worries, as we are able to address and fix that. We will go over how to create a custom Open Layer to modify where the camera's Scene Origin is located in the next section.

Scene Origin Offset: Positioning the Virtual camera

Right-click on the timeline and create a new layer. Here we will name it Scene Origin - Example. OK.
We will create an Open layer.
This will create for us an empty-yet-customizable layer, where we can add custom attributes and parameters. In our case, we will add the ability to control the our virtual camera's position, that is controlling what portion of the Unreal Engine scene that we are seeing.
We'll need to open multiple layer editors for this. In order to do so, hold [CTRL] and left click on the layers on the timeline you'd like to have open simultaneously. We will want to open the RenderStream layer, and bring the Cluster Workload properties window where we've setup our channel mapping in the previous section.
Looking in the Cluster Workload window, under Channel Mappings, right-click on 'emerson_xr target (backplate)'. A further properties window, specifically for that mapping, will open.
Take note of the Scene Origin attribute in this window. Now, we should still have our new Open Layer (currently titled 'Scene Origin - Example') open on the top left of the disguise Designer interface.
Hold [ALT] then Left-Click and drag from the Scene Origin - Example properties window and to the Scene Origin parameter on the emerson_xr target (backplate) properties window.
From here, that Scene Origin attribute should now be added to the Open Layer, available to conveniently configure as we need.
Left-click on whatever value is set for the backplate's Scene origin. A window bearing a list of available Scene Origins will open. We will create a new one. In the text field next to New SceneOrigin, type in the name of your new Scene Origin. Here, we will create one titled example_origin. Press OK.
By using different parameters such as Rotation and Engine Offset, you are able to adjust the orientation of the project's virtual camera and therefore the portion/perspective of the virtual set actively rendered in the camera's frustrum. If you intend to have several shots of different perspectives within the same Unreal Engine environment, you may find yourself needing to preconfigure several of these Scene Origins.