360° Video & Ambisonic Audio Workflow Guide (GoPro MAX & Zoom H3-VR)

Overview

This article will cover the workflow for recording 360 video footage together with ambisonic audio, using the GoPro Max and the H3-VR 360º audio recorder/ambisonic microphone. We will go over steps in putting together a recording setup, capturing and exporting 360 video/ambisonic audio, importing for editing, orienting your audio with your video, and finalizing for publishing.

Setup

You will need:

• 256GB Micro SD Card x2
• microSD Adapter x1-2
• GoPro Max 360 Camera x1
  • Rubber Lens Cap x2 
  • USB Type-C to USB Type-A Cable for Charger x1
  • USB Type-C to USB Type-A Cable for Camera x1
  • Dual Battery Charger x1
  • 5V AC Adapter for Battery Charger x1
  • Tripod Adapter x1
  • Rechargeable Battery x1-3
  • Protective Lenses x2
  • Mounting Buckle + Thumbscrew x1
  • Monopod Counterweight x1
  • Monopod Legs x1
  • Monopod Body x1
• Zoom H3-VR Ambisonic Recorder x1
  • Shure SRH840 Headphone (or your choice of monitor headphones) x1
  • Foam Windscreen x1
  • Dual Mounting Bracket x1
  • AA Battery x2

Note: If authorized, you may be able to reserve this from the EDC.

Tripod & Device Power/Storage

First you will need to set up your monopod, with the counterweight twisted on the top. Twist the bottom ring in order to flatten out the monopod legs out along the niches below. Afterwards, attach the monopod body (the extendable pole) atop the counterweight.

Next, for powering both devices, you will need a charged GoPro Max battery and two AA batteries. Please make sure you have two microSD cards as both devices require them for storing your recorded footage and audio.

The GoPro Max has a battery compartment on the right side. It has a small lever that you flip up to red, in order to unlock and open the flap.

There you will see an open slot for a rechargeable GoPro Max battery, and a slot on the right side for the microSD. When putting in the rechargeable battery, the notched side should be at the top. Ensure both are seated, before closing and locking the battery compartment.

The Zoom H3-VR's battery compartment is on the bottom. Pushing the latch in, pull the battery compartment cover away. Seat the two AA batteries. Between the two batteries, is a small niche with a microSD symbol nearby. That's where you'll insert the microSD. Once done, place the battery cover back on.

Power on both devices to ensure they work and are at an appropriate charge level. Double-check there's available recording time/storage on both. Format them as needed (instructions included under Recording).

Mounting

1. You will need to twist the dual mounting bracket onto your tripod (or monopod). You will find it as part of the Zoom H3-VR kit.
2. The dual mounting bracket can be used to attach two separate devices. This is where the GoPro Max and the Zoom H3-VR ambisonic recorder will both be attached to.
3. On the bottom, twist and tighten the Zoom H3-VR recorder.
4. Take out the GoPro Max from its case, and bag. Take off the caps from both of the lenses. In their stead, pop on the lens protectors. You will find them in the same bag the GoPro Max is stored in, inside of their own little sewn-in pockets.
5. On the top of the dual mounting bracket, twist on the Tripod adapter for the GoPro Max. Remove the thumbscrew by twisting and pulling it out. On your GoPro Max, look at the bottom of the camera, you will see two flaps with holes flattened against the camera. Pull both of them down, and make sure the holes are aligned. Then, please the flaps in between the gaps on the Tripod adapter. Put the thumbscrew back in and twist until tightened and the camera is locked in place.
   
6. Ensure that the screen on your GoPro Max and on your Zoom H3-VR are aligned on the same side. The non-screen sides of the devices is considered to be the front. This is to ensure your spherically-mapped video and audio are aligned when recording.

Recording

1. Format the SD cards on the GoPro and Zoom H3-VR respectively, if they are freshly borrowed and you have not already formatted them.
   • GoPro Max: Swipe down Preferences Reset Format SD Card
   • Zoom H3-VR: Menu System SD Card Format Execute
2. Before you begin recording, ensure the GoPro Max is in 360 mode, otherwise it will record a regular non-360/flat video based on the currently active lens. You can switch modes by clicking the bottom left corner button. In the bottom center, you should see "360 Video" in bold white letters, rather than "Video".
3. For the Zoom H3-VR, ensure your recording is encoded as AmbiX format: Menu Rec Settings Rec Mode.
4. Once both devices are powered on, confirmed to be charged, and have available storage, begin recording the ambisonic audio first. You will find the record button (red dot symbol) on the bottom of the screen on the Zoom H3-VR. However, because the microphone is currently upside down, this button will now be instead at the top. Once the Zoom H3-VR is recording, you will see a red light switch on and be visible from both sides.
5. Next, begin recording 360 footage on the GoPro Max. You will see a button with a red circle up on top of the camera. Click it, then after a short beep, you will be recording 360 video. You will see a red light blinking on both sides of the GoPro while recording. 
6. Congratulations, you are now recording both 360 video and ambisonic audio!

Exporting

Once you have finished capturing your 360 video and ambisonic audio, you will need to get them off of your devices and onto a computer for your editing/post-production workflow.

GoPro Max

You are able to export footage off of your GoPro by using a USB-C to USB 3.0 cable, or by inserting the microSD card via an SD adapter.

The 360 footage on the GoPro Max is stored in a .360 file format. You are able to view this footage using the GoPro Player application. However, to work with this footage, you need to convert it into a format importable into an editing program, such as Premiere.

Do note, if you are exporting this on a Windows machine, then you may need the HEVC video extension from the Microsoft Store. For macOS users, this will not apply.

Make sure you have GoPro Player installed, and then open your 360 footage.  Here you can scrub through and look around your footage.

Additionally, you can create a shorter edit of your footage using the Trim Tool at the bottom of the screen. For example, you may want to remove shots of you starting or ending the recording. Clicking that button will toggle you into Trim Mode. On your timeline, similar to the Trim button, you will see a blue scissor icon. Scrub to where you'd like to have your clip start, then click the blue scissor icon on the timeline. This will place down a blue vertical marker. Next scrub to the end of your desired clip, and do the same.

Confirm your clip looks right to you, and click the checkmark below. You will be able to either export the trim or create a new .360 file.

When exporting your trim or from File Export, you will be brought to the following settings window:

You have sections for defining your export's resolution, codec, and other options. We'll export at the highest resolution at 5.6K. As for using an uncompressed codec, if you're on Windows you will likely export your footage as Cineform. If you're on Mac then it will be as ProRes.

Decide whether you would like your footage's rotation and level to be locked, and enable/disable World Lock/Horizon level accordingly. Do note that this will disable the GoPro's built-in 360 audio options. This may not affect you if you recorded that audio separately via an ambisonic microphone. That said, if you recorded 360 video and ambisonic audio separately, enabling World Lock/Horizon Level is not recommended as this will solely affect your video, and may make aligning your audio difficult, depending on how much/what kind of movement was in your shot.

Wait until the exporting process is complete, and confirm your file is as you expect it.

Zoom H3-VR

Take out the microSD from your Zoom H3-VR, place it into an SD adapter, then insert it into your computer. You will find your audio files saved as a .WAV. Take it off of your SD card and place it under the same directory as your saved 360 footage. Match and label accordingly, to ensure your footage and audio are paired and organized.

Importing for Editing

Premiere

Open Premiere and create a new project. Create a new sequence under File New Sequence [Ctrl+N]. In the New Sequence window, you will find a collection of presets. For our purposes, please look under the VR folder. There you will find several resolutions, some of which comes included with "Ambisonics". Choose the preset befitting your project and ensure it also has "Ambisonics" in the name. It is recommended that you wear headphones for the upcoming editing process.

Additionally, do pay attention to the following note in the Preset Description on the right.

"NOTE: Within Premiere Pro’s Timeline Preferences, ensure that the “Multichannel Mono Media” default audio track option is set to “Adaptive”."

Keep this in mind for the following step, let us first finish setting up the VR sequence before returning to this.

For further customization regarding the Sequence Preset, look through the Settings, Tracks, and VR Video tabs. 

In the Settings tab, check the sequence resolution and video preview codec to ensure those match that of your footage. In the case of the GoPro Max, the sequence resolution should be 5376x2688 pixels if you exported at 5.6K. If you did not export it at 5.6K resolution, please refer to GoPro's Video Settings and Resolutions page, and adjust your sequence settings accordingly. If you have exported your GoPro Max footage on Windows, you will be using GoPro CineForm, otherwise it will be Apple ProRes 422 if exported on Mac. 

Under Tracks, make sure your Mix is set to "Multichannel", with 4 Channels, and that your tracks are set to Track Type "Adaptive".

Under VR Video, make sure your Projection is set to "Equirectangular" and your Layout is set to "Monoscopic".

Once everything is confirmed to be set as intended, press OK.

After creating the sequence: Now returning to the Preset Description regarding setting up your Multichannel Mono Media correctly.
This can be done by navigating to the program dropdowns at the top, click Edit Preferences Timeline. Under "Default Audio Tracks" you will find the option for Multichannel Mono Media. Ensure that is set to "Adaptive".

Drag your exported 360 footage and your ambisonic audio .WAV file, and bring it into your project's asset browser. Afterwards, drag your footage into a video track, and your ambisonic recording into an audio track. Ambisonic audio is recorded in four channels. Due to how we've set this sequence up earlier, your ambisonic audio should take up only one track.

Assuming your ambisonic audio and 360 video footage have different start and end times, you will first need to line them up. Select both your respective video and audio track, right click, and select "Synchronize". In the "Synchronize Clips" window, select "Audio", then Track Channel 1. After a brief moment, your ambisonic audio should be lined up with the audio from the 360 video recorder (GoPro Max).

Setting up the 360 view for your video preview

To preview your 360 footage as an actual 3D environment, click the wrench icon on the right, underneath your Sequence player.

Under VR Video, click Enable. You'll now be able to look around your 360 video. You may notice it's rather small, to change this click the wrench icon VR Video Settings.

Set your Monitor View Horizontal to 180 degrees. To further open up your view, go to VR Video again and disable "Show Controls". You will be able to click and drag your view around.

This will require you to use headphones, therefore please ensure you are wearing them when working with audio whether for this or in general. To properly hear your ambisonic audio, on the top left section, go to your "Audio Track Mixer", click the small arrow on the left side. This will reveal your audio Effects and Sends. 

You will see four gray columns appear, on their right side are a number of dropdown arrows. At the top of the rightmost column, click the arrow Special Binauralizer - Ambisonics. You will now hear the audio as though it's in your ears; however, this is purely to preview the audio within Premiere. This should be disabled before the exporting process.

Next, to ensure your audio is orientated in a way that matches your 360 footage, in the bottom left section, under Effects go to Audio Effects Special Panner - Ambisonics, and drag it onto your ambisonic audio track.

At the top left section, under "Effects Controls", you will see your options for modifying the properties and effects of your ambisonic audio track. Under "Panner - Ambisonics" you will see three important properties "Pan", "Tilt", and "Roll". They are the three axes by which you may rotate your ambisonic audio, if you imagine it as an invisible auditory sphere around you. Clicking on the dropdown arrow beside any of these three will reveal a slider, allowing you to slide between -180 degrees on the left and 180 degrees on the right, rotating the ambisonic audio along those axes.

Using your 360 footage as reference, rotate and orient your ambisonic audio until it matches in alignment with what you see. Whether someone is speaking or creating a noise, ensure that you hear it from the right source/direction within your video.

Finalizing & Publication

Once you have successfully oriented your audio and video, and finished editing, go back to the Audio Track Mixer where we have enabled the Binauralizer - Ambisonics, click the arrow next to it and set it to none, getting rid of it entirely.

Go to Export Media once you are ready. Check the settings that are personally pertinent to your project. Otherwise, it is essential to take note of a couple of items within the Video and Audio sections.

Under Video, ensure that the toggle Video is VR is toggled ON. From there, ensure your Frame Layout and Field of View settings are correct. These will likely match the set VR sequence settings.

Under Audio, ensure that you have changed Channels from Stereo to 4.0, which will cause the Audio is Ambisonics toggle to be unlocked. Ensure that is toggled to ON.

BEFORE YOU FULLY FINALIZE THESE SETTINGS, please ensure you review the next section. Once you have confirmed your export settings, either Export directly from Premiere or send to Media Encoder as needed. 

Publishing and Viewing

As long as your video/audio have been marked as VR/Ambisonic respectively, your video should be ready to upload and be automatically formatted correctly. This is assuming that your platform of choice supports 360 video along with Ambisonic audio. As examples, you ought to review the guidelines for sites such as YouTube and Vimeo, and export/adjust accordingly.

• YouTube: Upload 180- or 360-degree videos
• Vimeo: Upload 360 video
• Facebook: How do I upload a 360 video on Facebook?

Once you have reviewed, exported and uploaded your video, wait until it is processed. For YouTube, you will be able to watch your 360 video using the YouTube VR app on your respective headset (i.e. Meta Quest 2, Mobile Headset using the YouTube mobile app). With the ambisonic audio component, you are recommended to have headphones on as well.

With that, this has been an overall summary and guide of the 360 Video + Ambisonic Audio workflow from Recording to Editing and to Exporting/Publishing. For any questions, please reach out to postproduction@emerson.edu and/or the EML staff. Best of luck!