[DRAFT] World Labs to XR Studio

Overview

This is a drafted workflow combining Multi-User Editing, which allows active editing of Unreal Engine scenes while they're rendering in the XR Studio, as well as importing Gaussian Splats generated by World Labs Marble. This is currently using a pre-configured RenderStream layer and an Unreal Engine project as a test/demo of this workflow. Please be aware you may encounter bugs/issues.

Setting up: Real-time Editing in the XR Studio: This is for bringing up the Multi-User Editing configuration.
Importing from World Labs to Unreal Engine: A quick overview of downloading/importing a World Labs Marbled-generated Splat into Unreal Engine, using the Akiya 3D Gaussian plugin.
 

Setting up: Real-time Editing in the XR Studio

  1. On the Director VX2+ Media Server: We have a test RenderStream layer for multi-user editing titled RS: World Labs Marble Test. This can be found under the demo track and prod_demo track. 

  2. Navigate to one of them, left-click on the layer to bring up its properties.
  3. Right-click on Workload. In the Workload properties window, go to the bottom and click on Sync. Make sure it completes successfully.


  4. Start this workload.
  5. Switch to the RXII-B Render Server: Go to Renderstream Projects > MU_Marble, and open the .uproject file. Wait until Unreal finishes launching.


  6. You will notice this project is attempting to automatically join a Multi-User Session called XR_Studio_Edit. Disregard this, as we have not yet started the Multi-User Session Server.
  7. Once the project has fully loaded, navigate to the top bar of Unreal's interface, and look for a blue 2-person icon for the Multi-User Editing window. Click on the three dots.
  8. Launch the Multi-User Server.
  9. You should see a brief confirmation on the bottom-right corner.
  10. This window allows you to view the ongoing sessions and connected clients for the Multi-User Server, and should open upon launch of the server. If you do not see it, it is likely minimized on the task bar. Closing it will end the Multi-User server. When the server is first launched, you will not immediately join the available session. Either join through the auto-join notification on the bottom right of the interface, or double-click on the XR_Studio_Edit session, then Continue on the lower-right notification window.

  11. The joining notification from earlier should now be able to find the server and the available XR_Studio_Edit session. It will warn regarding local changes and ask if you would like to continue. Click Continue.
  12. Now open the Multi-User Session Browser...
  13. We should now see both our current machine and RXII-A's RenderStream connected to the current Multi-User Session.
  14. Now go to the Content Drawer and (re)open the NewMap level, to ensure we are on the synced version of the Level.
  15. Test and confirm syncing by selecting the currently active Splat/object in the level, deleting, then Saving. After a moment, the active scene should update in the XR Studio.



  16. Let's proceed to importing a Splat generated by World Labs Marble.

Importing from World Labs to Unreal Engine

  1. This guide assumes you have access to downloading generated splats from World Labs Marble as .ply. As of this article, downloading splats requires a paid plan.
  2. On your computer of choice, log into World Labs Marble, and navigate their Explore menu (or your profile/Studio). Find your chosen splat/environment, click on Download on the bottom bar.
  3. Check your settings.
  4. Make sure the Splat file format is set to PLY.
  5. Download as a Splat.
  6. Confirm the splat has successfully downloaded as a PLY.
  7. Open the EML-XR-NAS by entering \\EML-XR-NAS.emerson.edu on File Explorer's address bar, navigating to it from Network, or using a Desktop shortcut.

  8.  Navigate to Project Drop-off.
  9. Find a folder titled Marble_Splats.

  10. Paste your downloaded splat into this folder.
  11. Now, remote onto RXII-B.
  12. There, with the open MU_Marble Unreal project, click on the red & white circle icon on the top toolbar from Akiya's 3D Gaussian plugin.
  13. Navigate to the EML-XR-NAS > Project_DropOff > Marble Splats (or the directory you've pasted the splat .ply file) and select your splat's .PLY.
  14. Once the import has finished, this should generate a new directory in your Unreal project under Content > ThreeDGaussians, named after your imported PLY file.
  15. Navigate into that directory. There you will see several folders and a Blueprint Class object.
  16. Click and drag the Blueprint Class into your level.
  17. You will notice that the splat is not centered on the origin, and is upside-down.
  18. Select the splat, and set the Location coordinates to 0,0,0. Then set the Pitch rotation (Green) to 180 to flip the splat right-side-up. Save (CTRL+S). You should be seeing the changes reflected in the XR Studio.

  19. Click on Revision Control on the bottom-right of Unreal's interface.
  20. Persist Session Changes.
  21. Persist.
  22. Adjust and edit the imported splat as needed. Updates should reflect on the wall when saved.

     

Troubleshooting: Multi-User Editing not Updating

RenderStream Workload stopped/crashed on RXII-A or Unreal has closed on RXII-B

This fix/workaround is for the event where either your RenderStream layer has stopped/crashed and needs to be restarted, or if the MU_Marble Unreal project closed/crashed on RXII-B.

  1. If the RenderStream workload needs to be restarted: On the Director (or whichever machine is actively serving as Director), examine the RenderStream layer and attempt to (re)start the workload.
  2. Once running, go to the RenderStream layer properties window, look under level and click on NewMap. You should see a dropdown of available levels, which will only include NewMap. Select that, and confirm that the XR Studio and RXII-B are properly syncing again.
  3. If the project has closed/crashed on RXII-B: follow steps 5-13 under Setting up: Real-time Editing in the XR Studio to ensure RXII-B is running the server.

Configuring from Scratch: Multi-User Editor

TBD

Resources

  1. disguise.one | Multi-User Editing

Articles in this section

See more