60fps is Everywhere… Except Online

Stroll through the video camera department of your local Best Buy and you’ll be increasingly hard-pressed to find HD camcorders that don’t record in 60 frames per second (fps). The same holds true with other video recording devices, including medical imaging and industrial capture devices, action cams like the GoPro, and mobile devices like the iPhone 6 and Samsung Galaxy Note 4.

The devices we use to watch video in our homes and offices support 60fps as well. HDTVs, laptop and desktop monitors, tablets, and smartphones generally support playback at up to 60 frames per second.

Video games, most notably those built for the Xbox One and Playstation 4, are increasingly developed for 60fps playback.

And in recent years, major media companies like ESPN, ABC, and FOX began broadcasting their channels in 720p/60 (1280×720 resolution at 60fps).

It would seem that the entire digital video ecosystem has upgraded to 60 frames per second, were it not for one notable holdout: online video. Since the earliest days of internet video — in 2004 with VloggerCon and 2005 with the first video posted to YouTube — videos streamed across the internet and our corporate networks have been capped at a measly 30 frames per second.

It’s time for online video to catch up 

In 2015, that decade-long trend is being bucked. A few months ago, YouTube announced limited support for 60fps video (viewable only in Google’s Chrome browser). And last week, Panopto announced that our video platform is the first to support 60fps in all web browsers and on mobile devices. In this post, we look at why 60fps provides a better playback experience, some use cases for 60fps video in business and universities, the reasons that online video has been unnaturally throttled at 30fps, and how Panopto implemented 60fps in our video platform. We’ve also got a handful of 60fps video samples to show you how good online streaming can look.

What is 60fps, and What Makes it a Better Experience?

Watching video is a learned suspension of disbelief.

When we watch video, we’re not actually watching smooth motion. We’re simply watching a rapid sequence of still images that tricks our minds into believing that there’s motion. In that sense, video is little more than a highly sophisticated digital flipbook.

An early concept of YouTube 

The faster you flip through a flipbook, the smoother the motion appears. Similarly, the higher the frame rate of video (measured in frames per second, or “fps”), the smoother the perceived motion. And the smoother the perceived motion, the more lifelike the video appears.

That is arguably the purpose of video — to represent the reality of what was recorded with the highest fidelity possible. So as we increase the frame rate of video, its quality looks better because it more closely approximates the smooth motion that our eyes naturally see in the real world.

60 frames per second, however, is a unique benchmark for two reasons.

First, it’s the highest frame rate where most people can perceive differences in the smoothness of playback. At frame rates beyond 60fps, you reach diminishing returns of human perception. Very few people can tell the difference between 60fps and 120fps playback, and so frame rates beyond 60fps are generally only desirable for videos meant to be viewed in slow-motion.

Second, even if most people could tell the difference between 60 frames per second and higher frame rates, we’re limited today by the screens on our desktops, laptops, and mobile devices. CRT and LCD monitors refresh at 60Hz. In other words, they update the on-screen image 60 times per second. So when you stream video at 60fps, you’re perfectly matching the maximum refresh rate of your screen.

For an example of how video becomes more smooth and lifelike as frame rate increases, the video below was shot at three different frame rates. The first part is recorded at 15fps, the second part is captured at 30fps, and the final part is recorded at 60fps.

An example of 15, 30, and 60fps video streaming 

Not Just for Sports and Video Games

High frame rate (HFR) video is best known for its use in consumer-facing media — specifically video games and sports broadcasts. Both of these genres have benefitted from 60fps because they’re motion-intensive and accentuate the benefits of smooth video playback.

HFR video doesn’t only benefit the consumer segment, however. Because high frame rates make videos look more lifelike, they have the potential to improve the quality of nearly every type of media recorded for businesses and universities as well.

Consider industrial video that captures processes on a manufacturing floor. Industrial cameras, like those offered by Point Grey, have supported HFR video capture for years. Now, with HFR supported natively in video platforms, managers will be able to watch recorded or live streamed video in greater detail than was ever before possible.

Industrial machinery captured at 60 frames per second 

Medical imaging also benefits from HFR video, where biological events can occur in 1/60th of a second. If your video platform doesn’t capture at 60fps, an important event could be missed between frames.

Animations of all kinds are visibly improved with high-frame rate video playback. Medical animations can be seen with greater precision and CAD simulations provide more lifelike rendering of models. In the example below, Blausen Medical shows the potential of HFR video for broadcast-quality medical animations.

60fps medical animation provided by Blausen Medical 

And as the first example above shows, even a mundane “talking head” video looks noticeably better at 60 frames per second. For everything from keynote presentations, executive communications, and live-streamed events to architectural videos, university lectures and science labs, 60fps delivers smoother, sharper video for businesses and academic institutions that feels more like an HDTV broadcast than the “good enough” online feeds we’ve become accustomed to in the past ten years.

But if 30fps is perceived to be “good enough”, do businesses and universities really need 60fps?

We looked at this question from a different angle: Does our technology stack have the ability to support 60fps, and if we implement it, will it benefit our customers?

The truth is that today’s video capture, processing, and delivery technology is ready for high frame rates. And because 60fps doesn’t increase the cost of storing and streaming media (video file size is a product of bitrate and duration), we believe that anyone in any business or university should have the option to dial up the knob from choppy 15fps video up to buttery-smooth 60fps.

But Why Has Online Video Been Capped at 30fps For So Long?

Even with the recent advances in network bandwidth and CPU processing power, achieving 60fps isn’t trivial.

To support HFR for live or on-demand online streaming, you still need a string of capable technology that begins at the point of capture and ends at the point of playback:

• A good camera sensor. Your video capture device needs to have the light-gathering capability and timing to sample video at 60 frames per second. Although this is possible with all professional video cameras as well as most consumer camcorders and DSLR cameras, webcams from vendors like Logitech and Microsoft still can’t achieve 60fps.
• A fast pipe from the camera to the computer. Even if you’ve got a camera sensor capable of HFR, the connection to the computer can be a bottleneck. Specifically, your camera needs to be able to relay information to your computer at 60 frames per second, and because of this, USB 2.0 doesn’t support high frame rate video. With transfer speeds of 480Mbps, the USB 2.0 pipe isn’t big enough to send 60 uncompressed frames of video every second to your computer. By contrast, USB 3.0 — with speeds of up to 5Gbps — makes it possible.
• Efficient memory management and compression. Once these frames reach your computer, they’re sent to the video platform’s encoding pipeline where they’re loaded into memory, processed, and then flushed from memory. The process of allocating memory, compressing frames, and deallocating memory has to be incredibly efficient to prevent frames from being dropped. And if you’re planning to encode multiple bitrates for adaptive bitrate streaming, then for each frame of video, the encoding pipeline needs to process it not once, but three or four times before moving along to process the next frame.
• Efficient playback. Finally, HFR video streamed across the wire needs to be unpacked on the client and displayed on-screen, typically in a web browser. At 60 frames per second, the client technology — whether it’s Flash or HTML5 — needs to work incredibly productively in order to grab frames off the wire and display them on screen at the same rate that the screen itself is being refreshed.

In short, online streaming has been capped at 30fps for over a decade because achieving 60fps is hard. It’s hard for hardware manufacturers and it’s hard for video software providers. But as network speeds only get faster and processors only get more powerful, the 60fps standard for online video is inevitable. In fact, HFR online video was being tested as early as two years ago, when Blender Foundation and Janus Kristensen released what’s become an iconic 60fps animation, Big Buck Bunny.

Big Buck Bunny – An early example of what’s possible with 60fps online streaming 

60fps in Panopto’s Online Video Platform

With the latest update to Panopto’s video platform, we’ve implemented support for HFR video recording, live streaming, and uploading. This means that, from a Windows desktop or laptop, Panopto customers can now capture video at up to 60 frames per second from any camcorder, document camera, medical imaging device, industrial camera, or any other media capture source that supports high-frame rate capture. It also means that pre-recorded videos, such as those captured on your iPhone or on professional video cameras in the field, can be uploaded into Panopto’s video content management system where they’ll be encoded for 60fps playback on any desktop or laptop.

In addition, a new 1080p60 quality setting for video podcasts makes it possible to stream video in 60fps to mobile devices as well.

Record, live stream, and upload 60fps videos into Panopto 

Our video platform achieves high-frame rate streaming through a combination of:

• Careful performance tuning of our encoding stack to support multi-bitrate 60fps with minimal frame drop.
• Deep awareness and effective negotiation of upstream device capabilities, which is critical to ensuring that video cameras and other media sources deliver high-quality HFR video to your Panopto capture PC.
• Optimization of our video player code, which wraps Flash- and HTML5-based video controls in JavaScript.

As our customers begin capturing presentations and lectures in 60fps, and as YouTube and other video platforms roll out support for the new benchmark, the use of video as a medium for sharing knowledge and communicating will only accelerate. We’re excited to be a part of this evolution in online video, and look forward to seeing what you capture in HFR.

In the meantime, here are a few more sample 60fps videos currently streaming from Panopto Cloud.

An overview of NASA’s MMS mission 

Our own Jesse Vernon showing how to play Blackberry Blossom on the banjo 

…and for our fellow Star Wars fans, here’s the 60fps trailer for “The Force Awakens.”