One of the most time consuming and frustrating tasks we encountered during our first 360 3D video productions was finding the optimal encoding settings for each of the currently available VR headsets. Each platform supports different resolutions, frame rates, codecs, and bitrates. This article explains the settings we started with, what we learned from analyzing some of the legends in the field (like Chris Milk and Felix & Paul), and finally we’ll share a simple yet powerful free tool we built to help you encode your VR video content with the best possible settings. Let’s go!

Our ignorant phase

We at Purple Pill VR film in stereoscopic 3D on all sides, which ultimately results in one video file containing 2 panoramas, one for your left eye and one for your right eye, stacked on top of each other. This file, a .mov or .avi, has an exotic 1:1 aspect ratio and a resolution of 4096×4096 pixels at 60 frames per second. Not a single VR headset is currently able to play this back smoothly, so we scaled the resolution down until we had smooth playback, which was around 2048×2048@60.

We knew that the playback resolution wasn’t great, and some viewers, even though they loved the VR experience, commented on this fact as well. We told them that our camera filmed in a much higher resolution, but that we’re only able to display 25% of the pixels that we shot due to hardware limitations of the display devices. We also told them how you lose a lot of resolution since you spread the pixels out over the surface of a sphere, and then on top of that look at it through two big lenses which magnify each pixel of your phone screen (in case of Gear VR).

All reasonably valid points, but still we felt we needed to do something about this. We started to scour the dark corners of the internet to find out who had the magic answer, but there were many conflicting opinions. Therefore we decided to analyze what some of the legendary VR video producers were doing to solve this problem…

 

Studying Chris Milk

One of the biggest VR filmmakers of this moment is Chris Milk from VRSE. We downloaded the VRSE app for both Cardboard and Gear VR for “study purposes”. When we opened their videos in Mediainfo, we noticed several things:

Chris Milk Media Info

 

Chris Milk, like most of us lesser mortals, is using the h.264 MP4 codec. However, the key is in the profile and level he uses. We encountered these settings often in our encoding tools, but usually didn’t pay a whole lot of attention to them. Now we know better..

The Baseline profile is primarily used for decoders with limited computing power, like mobile applications. Google also recommends the Baseline profile for h.264 video playback on Android. This makes it the go to profile for mobile VR, like Google Cardboard and Gear VR.

The level describes the max resolution and bitrate acceptable. Each device supports videos up to a certain h.264 level. For example, iPhones only support videos up to level 3.1 (1920×1080@30 is the absolute max for iPhone 5.. so pretty appalling for VR video playback), while most Android phones will play level 4.2 videos just fine.

The resolution of 3840×2160 is the official Ultra HD (UHD) standard for 4k videos, also referred to as 2160p. This resolution has a 16:9 aspect ratio, not the 1:1 aspect ratio you usually start with when shooting stereoscopic VR content. This means that the video resolution is compressed in the vertical direction. So in other words, these videos have more pixel density on the horizontal axis than on the vertical axis, and so everything in the scene looks short and fat. The following diagram might make things more clear:

 

Best 360 Video Resolutions

 

The funny thing is that when you display this stretched/compressed video on the inside of a sphere for playback in a VR headset, everything is pushed back to the correct proportions again! And by using a resolution of 3840×2160 instead of 2048×2048 (what we initially used), Chris Milk has a whopping 97% more pixels to display! A massive improvement!

Amazingly, he even uses this UHD resolution for the VRSE Cardboard app, something which we thought would not be playable! However, my 2 year old Nexus 5 phone is more than able to smoothly play these 4k videos, resulting in an amazing experience, even when using a low-end cardboard viewer. The downside of this resolution is that there is a limit at 30 frames per second. Higher frame rates simply won’t play on the current (mobile) VR devices at these resolutions.

 

Funny sidenote

Chris Milk’s video The Displaced, a powerful story about refugees, is also used in the New York Times VR app for Cardboard. But while VRSE plays this video at 3840×2160 (stereoscopic), the NYT VR app plays this same video at a mere 1920×1080 (monoscopic).

 

Felix & Paul & h.265

So, we just saw that using Chris Milk’s encoding settings can greatly increase your final video quality. Next to VRSE, the legendary Felix & Paul studio has been at the forefront of VR filmmaking for a long time, and a friend of ours from the Dutch production house Scopic mentioned that their Inside Impact video, featuring Bill Clinton, was one the sharpest looking VR video he had seen so far. So what makes this video so much crisper than other VR content out there? Well, a number of ingenious things..

 

Felix & Paul Mediainfo

 

You may notice the rather strange 3840x1536 resolution, which would imply a serious reduction in pixels compared to the 3840x2160 pixels of Chris Milk. However, the trick is revealed once we study a frame from the video:

 

Felix & Paul The Displaced Missing Desk

 

In the screenshot you notice that you’re unable to see the bottom of Clinton’s desk. However, when viewing the video in the Felix & Paul Gear VR app, you can see the bottom of his desk. What kind of voodoo is this?! Well, Felix & Paul realized that only a limited number of pixels can be displayed as video on the current line of (mobile) VR devices, and they also realized that 95% of the action is happening at eye-height, not above or below you (this obviously varies with each piece of content). So their solution was brilliant: display only a certain band of pixels as video, and show the top and bottom parts as still images.

 

Felix & Paul trick

 

This masterful trick allowed them to effectively achieve 3840×3840 pixels in resolution, of which only 1536 pixels in the vertical dimension are displayed as video, while the rest of the pixels are still images. This reduced amount of video pixels also allowed them to push their production to 60 frames per second!

While the resulting viewing experience is unparalleled, there are some serious limitations to this trick. First, it requires a crapton of extra work and technical expertise to create a custom video player app which plays hi-res 360 videos and switches between top/bottom images in sync. Also, if you film something from a drone for example, where the bottom of the video does move, using still images there is no option. Nevertheless, a very creative way to push this new medium to its limits! But Felix & Paul didn’t stop there..

While the video is 9.21 minutes long, super high resolution, as well as 60 frames per second, the file size is “only” 825MB. Mediainfo teaches us that this is due to an, at first sight, rather low 12.3Mbps bitrate. Don’t let this bitrate fool you though, because instead of using the common h.264 codec, they encoded their files with the more powerful h.265/HEVC codec, which essentially gives you the same video quality at half the bitrate.

So their 12.3Mbps h.265 video is comparable in quality to a 24.6Mbps h.264 video, resulting in half the file size! An additional benefit of h.265 is that it supports higher resolutions, bitrates, and frames per second than h.264, making it an ideal candidate for future VR video players. The only major downside is that there are hardly any devices capable of playing back h.265 videos, but fortunately the Samsung Gear VR phones are! (up to 3840×2160@30 for now)

 

“Optimal” encoding settings

After studying how the big guys do it, we decided that the following encoding settings would be optimal for now:

 

Platform Codec Resolution FPS Avg. bitrate
Gear VR h.265 3840×2160 30 10 – 20Mbps
Cardboard Android h.264 (Baseline, level 4.2) 3840×2160 30 20 – 30Mbps
Cardboard iOS h.264 (Baseline, level 3.1) 1920×1080 30 10 – 14Mbps
Oculus Rift h.265 / h.264 4096×4096 60 40 – 60Mbps

 

So, we use h.265 for Gear VR to get an optimal balance between video quality and file size. If you would wish to play your videos at 60fps, Samsung recommends 2048×2048.

For Cardboard Android we can use Chris Milk’s settings for the highest possible resolution. However, if you are concerned about file size, you feel that the Cardboard experience is sub-optimal anyways, or if you want maximum compatibility with older smartphones, you can always reduce the resolution to 2k or even 1080p, like the New York Times does.

As mentioned before, iOS has put hardware restrictions on the maximum video size you can play on their devices, which is set to 1080p at 30 fps for iPhone 5 and 1080p at 60fps for iPhone 6. Because of this terribly low resolution and because developing for iOS is a torturous process, we are not a massive fan of VR for iPhone. However, many clients request it, so that’s why we included it in the list.

For Oculus it is hard to determine what settings to use, because everyone’s computer will be different. However, based on Oculus’ minimum system requirements, it should in theory be possible to play high res, high frame rate h.265 videos smoothly, provided that the correct codecs are installed on the system and a powerful 360 video player is used. For now though, we use h.264 and UHD resolution for maximum compatibility since even our Mac Pro has difficulty playing h.265 files.

 

Monoscopic

For monoscopic videos (360 2D) the resolutions are slightly different. For example, the Gear VR can play 4096×2048@30 in h.264. Using Chris Milk’s settings, you should also be able to play this resolution for Cardboard on most newer Android phones. However, on Gear VR you could also opt for h.265, in which case the maximum resolution we managed to play smoothly was 3840×1920@30. This was on a Note 4 though. If you want 60fps, Samsung recommends 2880×1440. We did not test this resolution in h.265 yet.

 

Encoding tools

None of these resolutions might turn out to be optimal, but for us these work very well for now. But what tools do you need to encode your videos to these resolutions and codecs? We started out with MPEG Streamclip, then tried Compressor in combination with Final Cut Pro, and finally Handbrake. While they all sort of worked, none of them was able to do everything we needed and we kept having to re-render our files because of a missed setting. So we ended up at the most reliable, most powerful, yet most user-unfriendly encoding tool ever created: FFmpeg!

This open-source command line tool has full support for h.265 and is used to power most online video encoding services. It is great, except for the fact that learning the right commands and typing them over and over again is more frustrating than mowing the lawn with a nail clipper. That is why we decided to write a little Python script to make our life a bit easier.

We now simply copy our hi-res, uncompressed files to an input folder, run the program, select which VR platforms we would like to render (Gear VR, Oculus etc.), press “Start encoding” and then sit back to see our output folder fill up with perfectly rendered files ready for playback in the respective VR headsets!

VRencoder - Free 360 Video Encoding Tool

Jealous? Don’t be! Because we decided to make this tool available for free, nada, niente! Just fill in your name and email address in the form below and the download links for both Mac & Windows will magically appear in your inbox 🙂

 

This tool works, but there is plenty of room for improvement. If you are a programmer interested in tweaking it to your own needs, feel free to tweak the source code, which is also part of the download package.

Let us know in the comments if you have suggestions for improvements!

 

NEW! Create VR video apps in minutes with Headjack

66 thoughts on “The Best Encoding Settings For Your 4k 360 3D VR Videos + FREE Encoding Tool

  1. Hi guys, Brilliant wotk – really ! ! we are a VR start up in Germany and we are also into livestreaming 360° content for a German TV show.
    We are building our own 360° rigs and your tool seems to be very promising if not to say mega amazing. We would love to get access. Many thanks in davance , really looking forward to hearing from you.

  2. Wow just wow.
    I will inform you about my experience with this encoder.
    thanks guys. I am new in this vr and 360 tehnology but learning very fast.

  3. Nice blog post. Just a heads up, you can go all the way up to 4096×2160 or 2160×4096 at 60fps. I have this playing on the Note 5 or S6 in H265.

    So with stereo you can stack two images which are roughly 1:4 or .5:1 pixel aspect ratio depending if you want to loose resolution on your equirectangular image in the horizontal or vertical.

    The hardware mpeg decoder seems to have hard coded dimensions which it work with. You can use most combinations of the following dimensions in the horizontal or vertical:
    4096
    3840
    2160
    2048
    1920
    1080

    The only catch is you cannot have a square texture, which is a major bummer, so no 4096×4096. Also I couldn’t get 4096×3840 to work either.

    1. Hi Deke,

      Thanks for the tips! However, at those resolutions it wouldn’t play at 60fps on our Note 4’s. What bitrate are you using?

      Nick

      1. Yes, I could only get 30fps on the Note 4 with h264. Only the Note 5/S6 could playback h265 at 60fps. The bitrate was 15.9Mbps using the ffmpeg settings:
        -c:v libx265 -preset slow -x265-params crf=20

        My next goal is to try out cube maps and see if the quality is any better. If I make a 2×3 six pack into a 4096×2160 area, each 90 degree field of view of the cube will be 1365x 1080. This should give more resolution even for the pixels in the main FOV and remove pinching at the poles.

  4. Hi there, thank you for this awesome post, i have filled my email address, but did not receive the download links for VrenCode, could you please send me the download links ?^^

  5. I can’t thank you enough for sharing your knowledge and experience. I’m just starting to teach these juicy new communication methods at my school, it’s an exciting time to be in visual storytelling!

    1. Hi Georgy,

      The encoding does not include the YouTube metadata yet, so you’ll still have to use their little tool. You could in theory alter the source code of the encoder to also automatically change the metadata.

  6. Great article! I was wondering how Felix & Paul make their stuff so crisp! Really appreciate your sharing your knowledge with the community.

    Also, trying to download the encoder but never got the emails (not in my spam either). Would love to try it out!

    1. Thanks Irad! There is something up with our auto-email system.. I’ll check it out! I just re-send the download links to VRencoder to you 🙂

  7. Wow, this tool is amazing, I can’t thank you enough! Do you by any chance know or have plans to release the presets for 180 degree video encoding? I can’t find that information anywhere, for either SBS or TB.

    1. Thanks Gerard, glad you like it! 🙂 I do not plan to release a 180 degree encoding tool, but I guess how it works is similar to 360, except half your image is black.

  8. Thankyou, I wait to get the link in high anticipation! Ive spent the last two days testing different settings and have become very frustrated. I just started reading about how difficult it is encoding moving water (My 360 video im trying to encode perfectly is from a drone moving over water) and stumbled across this page. Thank the lord 🙂 – How long does link usually take? Thanks again for your work here!

    1. Hi Marcus,

      Yeah I changed several settings on our email server, so hopefully the email with the download links will now be sent properly. For now I sent you the links manually, enjoy!

      Nick

  9. Hi, Nick

    Thanks for useful information and free software. Can you also send me the link for vrencode, I tried with automated email but no luck. Thanks

    1. Should be in your inbox as we speak 😉 Contacted both our autoresponder and SMTP host to fix this annoying bug. They switched us to a new server (said it could have something to do with a bad IP address), so let’s pray everyone will receive the emails now.

  10. Nice stuff, thanks for sharing!
    But is it just me, or the Youtube settings are a bit too heavy? I get a 700MB file from a 20 seconds clip…

    1. Hi,

      YouTube asks for high bitrate uploads and then encodes them to lower bitrates for playback on YouTube. For 4k videos, YouTube recommends 65Mbps bitrates.

      Nick

  11. Hi Nick, thanks in advance to share all your knowledge, really appreciate, thats the way to build a better VR world!
    Te VRencoder 0.9-OSX doesn’t work for me, I can choose input & output folders (items are in grey) and choose Gear VR for example, hit Start, and nothing happens, any progress bar, notification?.
    My mac is Yosemite 10.0.5
    Thanks

    1. Hi Pablo,

      Yeah I’m not a pro Python programmer, so I did not manage to add a progress bar to it :-p We simply use Activity Manager on Mac to check how much CPU is being used by FFmpeg (the encoder). As long as FFmpeg is active, it means the VRencoder is still running. Not ideal, I know, but it should work!

      Nick

  12. Hi Nick,

    tnx for the great article and information provided.

    I have also filled in the auto email form but did not receive an email containing the download link. Could you please sent it manually?

    Kind regards!

  13. Hi there, thank you for this post, i have filled my email address, could you send download link for me , thanks!

  14. Thanks so much guys, very helpful. Do you plan to deploy any tools that would help with Felix and Paul’s static image technique, or do you have any recommendations for tools that can mimic this? Cheers

  15. Thank you for the script. Its been very useful.

    I have one question. I would like to animate moving objects in a static camera. Can I go this with a cube 6×1? Will the GearVR understand a cube 6×1 movie and format it correctly? I know this works with a still image.

    Thank you

  16. Thanks for your work and information – keep up the great work.
    Unfortunately the magical trick with the inbox-send-them-an-email-with-download-direction didn’t work for me 🙂

    kind regards from germany
    Norbert

  17. Hi guys!
    Great work 🙂 I also dug inside Felix & Paul video to find out how they do it – a real treasure!

    Did you try encoding an 8k video? I know it’s not yet possible to play smoothly on Gear VR, but hey, we want to get as future proof as we can with our works and google isn’t helping a lot.

    Right now the mp4 container isn’t even letting me do 4096×4096.

    On the plus side, I can play 4096×2160 30fps videos (stereo sbs) with over 100mbps bitrate 😀 It’s really necessary when we deal with a lot of tiny details we do 😉

  18. Thanks a Millions for this priceless article !
    Next time i go to Amsterdam i must offer you a drink guys !

  19. Thank you for this brilliant tool and explanation! Just wondering… If output is to the Vive, would I assume the same settings as Oculus? Thank you

  20. Question about Exporting your uncompressed file with Media Encoder:

    When you create the timeline (in AE) with 3840×2160 30 fps and you stretch the video for TOP/BOTTOM view. What pixel aspect ratio do you configure you export settings?

    Also, if you could provide full details about your Export Settings for Stereo and Mono using Adobe Media Encoder prior to using VRencoder that would be AWESOME 🙂
    Thank you so much, this blog post is very useful.

    1. From Media Encoder we export in the highest possible resolution, 4096×4096 (1:1 aspect ratio) @ 60fps in CineForm, AVI or ProRes.

  21. This is an absolute jewel! Thanks so much for making it and sharing with the community.

    As a filmmaker with just basic understanding of programming, I have a quick question: is there a way to tweak/modify some of the encoding settings the VR Encoder pulls and if so, where are they stored and how to edit them?

    Alternatively, is there a way to see the actual command sent to FFMPEG? I can see the encoder’s responses / prompts but can’t figure out what is the command that goes in. It would help me greatly as I am trying to test YouTube’s new 5K and 8K VR options…

  22. Hi Nick,

    Thank you very much for this so helpful post!

    For my part, I created a video from a sequence of images (from a digital simulation) for the GearVR.

    I use FFMPEG directly, to create my videos (because your software, VRencoder, takes videos in input only, it seems, but looks very powerful for video). However, my videos are not very smooth (jerks appear every 2 or 3 seconds), even when I reduced significantly the resolution or the bitrate.

    Have you ever encountered this problem? What is your line of code for the GearVR?

    Here, my line of classic code:
    ffmpeg -framerate 30 -i Animation_%04d.png -vcodec libx265 -pix_fmt yuv420p -vb 15M Animation.mp4

    Thank you,

    Jcharles

    1. Hi Jcharles,

      Thanks for your comment! Here is my line of code for Gear VR:

      ffmpeg -i input.mov -vf “scale=3840×2160,setdar=16:9” -r 30 -c:v libx265 -pix_fmt yuv420p -vb 15M -preset fast -c:a aac -strict experimental -b:a 128k output.mp4

      Nick

  23. Hi Nick,

    Thank you for this post, it helps me a lot in my current project!

    However, one thing confuses me. According to the Wikipedia article, the maximum resolution is 2048×1080 at level 4.2. How can Chris encode videos at 3840×2160 then?

    All the best,

    greg

    1. Yeah when we run this on Zencoder we also get a warning. It still encodes though, but it is not according to the exact standards.

Leave a Reply

Your email address will not be published. Required fields are marked *