Normal

medium_screen-shot-2021-01-26-at-11-48-47-am.jpg

small_screen-shot-2021-01-26-at-11-48-47-am.jpg

thumbnail_screen-shot-2021-01-26-at-11-48-47-am.jpg

screen-shot-2021-01-26-at-11-48-47-am.jpg

Erikka Innes

Video trends

## The benefits of a cloud video platform
The cloud is collections of servers in data centers that run software, maintain databases and serve all kinds of content to all the corners of the world. As we make it easier to create and share video by adding technologies like fast 5G networks and cheaper camera equipment, more and more people are creating video content. To handle the demand, most video streaming platforms use the cloud. Netflix famously started the move to the cloud after a corrupted database prevented them from mailing anyone their DVDs for 3 days! YouTube uses the cloud, and so does api.video. There are many benefits to providing video via the cloud. 

## 1. You can use a network that can handle video streaming and heavy traffic
A major benefit to using a cloud video platform is that it's up to someone else to handle the technical details of video streaming and coping with heavy traffic. You can just add video to your company portal, or build your own viewing interface or whatever you like.

## 2. Lower cost
It's less money for your business or project if you don't have to store your video content, manage the servers with the content, or hire an IT team. Instead, you pay for a video service, like api.video, and everything is stored and managed for you. 

## 3. Video focused content delivery network
If you choose a cloud video platform like api.video, you get to use our private content delivery network. Because api.video handles only video content, the network is optimized to provide the best experience when working with video. Many companies that work with video have their own networks for their content, including Netflix - their network is called Netflix Open Connect. 

## 4. Continuity
When you store content yourself, you have to have a plan for what happens if part of your system goes down. If you use a cloud service for your video, this is handled for you. By storing it in the cloud, it can be replicated and maintained in multiple locations if need be, so that your video is always there when you need it. 

## 5. Mobility
If your video content is stored in the cloud, it's easier to access. You can stream content at home or a coffee shop, anywhere there's internet, you'll be able to get to your videos and interact with them. 

## api.video and the cloud
At api.video, all our services are in the cloud. We use our own Content Delivery Network (CDN) to ensure fast, reliable uploading, transcoding, and delivery. 

large_top-5-benefits-of-a-cloud-video-platform-1.jpg

medium_top-5-benefits-of-a-cloud-video-platform-1.jpg

small_top-5-benefits-of-a-cloud-video-platform-1.jpg

thumbnail_top-5-benefits-of-a-cloud-video-platform-1.jpg

top-5-benefits-of-a-cloud-video-platform-1.jpg

small_yvanmichel.png

thumbnail_yvanmichel.png

yvanmichel.png

Yvan Michel

All companies in the video streaming sector have the same goal-- guarantee the best watching experience with efficient cost control for ingest, transcoding, and delivery. It is not an easy task and requires a lot of considerations on the Product and Engineering sides.

Measuring the Quality of Experience (QoE) is one of the methods widely used to assist this process. In this blog post, we will go through the QoE principle and functioning by relying on common use cases and metrics.

## Principle

As soon as someone uses a website or an app, stakeholders need feedback in order to learn about the strengths and weaknesses of what they've built. The measurements performed for the QoE purpose are based on this same principle and are seamless for the end-user.

The QoE consists of collecting and sending users subjective data to better understand how the product is used to make the right decisions. In this case, knowing more about how videos are consumed by viewers through a first-party player.

It helps to get more insights about questions like "What will be the effects of reducing the 1080p video bitrate on the end-user engagement?" and "Is it a good approach to preload the video on the start-screen to boost its loading time?"

NOTE: QoE data must comply with local privacy policies like [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation) or [CCPA](https://en.wikipedia.org/wiki/California_Consumer_Privacy_Act). No data that breaks these laws or disrespects user privacy shall be stored without the user's explicit consent.

## Functioning

The player performs periodic measures related to the video playback but also computes metrics from its own runtime context like the [FCP](https://web.dev/fcp/), its loading time, and more. It then forwards all this info to a dedicated backend for later processing and analysis.

large_qos-and-qoe-workflow.jpg

medium_qos-and-qoe-workflow.jpg

small_qos-and-qoe-workflow.jpg

thumbnail_qos-and-qoe-workflow.jpg

qos-and-qoe-workflow.jpg

The objective is to gather enough relevant data about the "Where", the "Who" and the "How". The "Where" and the "Who" are quite important but are more "context" oriented. The "How" really refers to the QoE.

### The "Where"

Locate where videos are watched to know the distribution of the audience:

- The `continent` → *Is North America, the main continent in terms of audience?*
- The `country` → *How many videos are watched in France?*

GeoIP Databases allow retrieving the `continent` and the `country` based on the user IP.

### The "Who"

Identify the profile of devices used to watch the videos:

- The `device_type` → *Are CTVs representing a significant part of the audience in Asia?*
- The `platform` and the `browser` family or version → *When was the latest version of macOS released? What is the part of IE11 to clean legacy code?*

Even if it is not 100% trustable because it tends to be deprecated and it can be overridden, the `User-Agent` HTTP header exposes this info and provides, at least, good hints. There are some alternatives like [UA-CH](https://web.dev/user-agent-client-hints/).

For a given visit, having a `user_session` with the number of `views` helps to confirm that after consecutive video playbacks, the experience remains stable and great.

### The "How"

Understand the video consumption behaviours:

- The `rebuffering_time` → *Why is the engagement so low in Asia?*
- The `watch_time` → *What is the best moment to trigger an ad-break?*
- The `abr_max_rendition` → *Is the 720p more played when increasing the default player dimensions?*
- The `playback_error` → *Why are macroblocks appearing on Safari/desktop?*

There are a lot of factors that can influence video playback. Rebuffering is part of them but is the consequence of underlying problems: slow delivery, low user bandwidth, etc.

> A viewer who experienced a rebuffer delay that equals or exceeds 1% of the video duration played 5.02% less of the video in comparison to a similar viewer who experienced no rebuffering.
> 
> Source from [Akamai](https://people.cs.umass.edu/~ramesh/Site/HOME_files/imc208-krishnan.pdf)

But what is rebuffering? A `rebuffering` describes the moment when the video playback stopped because of an insufficient amount of downloaded video bytes *(buffering)* without any user interactions. Seeking or switching manually to another video rendition is not considered to be `rebuffering`.

In other words, `rebuffering` is an unexpected buffering lasting *N* milliseconds causing the video playback to stop.

The `watch_time` is a duration representing the time a user has watched a video. It can be associated with the `rebuffering_time` to compute the `rebuffering_precentage` during a `user_session`.

ABR (Adaptive Bitrate) algorithms choose the best video renditions according to user network conditions. The `abr_max_rendition` describes the highest video rendition reached when ABR is enabled.

Finally, the `playback_error` indicates which errors have occurred during the video playback to explain sudden disruptions or degradations within the network, transmuxing, or decoding.

## Conclusion

We've had a sneak peek at how QoE can be useful to drive operational decisions. Combined with the [Quality of Service](https://en.wikipedia.org/wiki/Quality_of_service) *(QoS)*, this data constitutes a powerful set of valuable information to continuously ensure that video playback is optimal.

At [api.video](https://api.video), we are committed to providing the best watching experience for all, no matter the location or the device. In another blog post, we will go through other side usages based on QoE data especially for observability to detect proactively issues and assess impact in case of outages. Stay tuned!

## Resources

- [https://www.akamai.com/blog/performance/enhancing-video-streaming-quality-for-exoplayer-part-1-quality-of-user-experience-metrics](https://www.akamai.com/blog/performance/enhancing-video-streaming-quality-for-exoplayer-part-1-quality-of-user-experience-metrics)

The benefits of measuring the video playback QoE

Benefits of measuring the video playback QoE

large_qoe-1.jpg

medium_qoe-1.jpg

small_qoe-1.jpg

thumbnail_qoe-1.jpg

qoe-1.jpg

doug.jpg

twitter-brands.svg

Doug Twitter

Doug Sillars

## Responsive media on the web (ok, mostly responsive video)


As the web has evolved, building sites with media rich content has gotten more and more complex. We (as developers) are expected to build sites that are media-rich and engaging - but at the same time load quickly and work on many different platforms. This is a very hard balancing act that has been the focus of the web performance community...well since the beginning of the web.

In this post, we’ll focus on how media can be efficiently delivered on the web - we’ll focus mostly on video, but will use images as examples, as there has been a lot of work to make images fast for all devices and screens.


## Images

Images are a logical place to start when considering responsive media - I mean you could just consider them a single frame video!

When it comes to image delivery on the web, there are a lot of HTML native approaches to deliver images as small as possible - and properly sized for the device. Let’s use some examples with the ```<picture>``` tag to walk through how you can properly serve images responsively. Here’s the example from the MDN page:

```
<picture>
    <source srcset="/media/cc0-images/surfer-240-200.jpg"
            media="(min-width: 800px)">
    <img src="/media/cc0-images/painted-hand-298-332.jpg" alt="" />
</picture>
```
	
There are 2 images in the picture tag. The ```<img src>``` is the fallback for browsers that do not support the picture tag, and is the “image of last resort” in case the ```<source>``` images do not fit the criteria of the browser or space on the screen.

The ```<source>``` attribute describes alternative images that can be displayed. In the example above - there is a different image for screens where the screen width is over 800 pixels.  The idea is that a “big image” can be delivered to big screens, and a “small image” for mobile or smaller screens. We can use the picture tag with even more complex methods:  

```
<picture>
    <source media="(min-width: 1200px)" src="hero-large.webp 1200w" type="image/webp">
    <source media="(min-width: 1200px)" src="hero-large.jpg 1200w" type="image/jpg">
    <source media="(min-width: 800px)" src="hero-medium.webp 800w" type="image/webp">
    <source media="(min-width: 800px)" src="hero-medium.jpg 800w" type="image/jpg">
    <source src="hero-small.webp" type="image/webp">
    <source src="hero-small.jpg" type="image/jpg">
</picture>
```
	
In this example, we have 3 different versions of the same image - ‘hero-large’, ‘hero-medium’ and ‘hero-small’.  They are offered in 2 different formats: webp and jpg. While this is can lead to a great (and fast) experience for your customers, Iit does mean 6 different versions of each image must be created.

When the browser encounters the picture tag - it iterates through the options, and will use the first version of the image that meets all the criteria. If the browser supports webp, and the screen is over 1200 pixels wide, the first image will be used. If the screen is under 800px, and the browser does not support webp, the ‘hero-small.jpg’ will be used.

This is incredible - your website now supports 3 different screen sizes and 2 different image formats.  Of course, this means either the developers or the content management system must create 6 versions of each image that is uploaded - an extra development step, and potentially a lot more storage usage.

	
## Video

The video tag works in similar ways to the image tag when it comes to formats:

```
<video autoplay muted controls>
	<source src=”hero.webm” type=”video/webm”>
	<source src=”hero.mp4” type = “video/mp4”>
</video>
```
	
If the browser can handle webm videos - this format will be used, otherwise it will fallback to the mp4 (98.1% support in browsers today - and the last 1.9% don't support video at all).

In a soon to be published article, HTTP Archive data finds that using multiple source files is not used widely - less than 10% of video tags use this feature (on 7.6M webpages tested).  Even worse, of those that use multiple source files ~75% lead with the mp4, negating the presence of the other versions of the file (since the browser uses the first format it knows how to use, and all browsers know how to use a mp4.) This implies that only 2.5% of the video tags are correctly utilizing the source attribute to deliver different versions of the same video.

### No media queries

There are no media queries for video.  There *once were* media queries in the video specification, but they were removed, and support for media queries left Chrome in version 34 (In November 2021 Chrome is version 95).  So, the pattern we used to make images responsive will not work for video.


### Why were they removed?

When it comes to images, generating 3-4 different types of image can be tricky - which formats?  Which sizes? What is the process to generate - is it automated, or do you need to build a specialized upload process?  If your site has hundreds of images - you may now have thousands of images to store and deliver.

Video suffers from the same issues.  But videos - being generally more than one frame long - make the problem more immediate.  [Re-encoding](https://api.video/what-is/video-encoding/) a video takes much longer than reformatting an image - so the time to generate the files can be on the order of minutes (as opposed to a second or 2 for images). 

If you are using a CMS - it must know how to take an uploaded video, and turn it into several different sizes - with a couple of different formats.  If you are not using a CMS, guess what - it falls on you to learn how to re-encode the different sizes and formats for each video.

### Video doesn't work the same way

When you add an image to a webpage - it stays where it is.  It’s static.  If you download a huge image on a poor connection - it takes longer, but the page will still load.

Video is not static.  But since the ```<video>``` tag is markup, it cannot change what is rendered on the page.  

### Let’s consider a couple of use cases:

You have a ```<video>``` embedded on your page, and it renders at 600px wide.  Your page uses logic to choose and download a video source file that is optimized for a 600px space (saving a lot of data over the 'desktop' 1920x1080 video.)  

1. What happens when the user presses the “fullscreen” button?  
2. What if your customers have a chromecast, and they cast it to their 60” TV?  

The video sent to full screen/TV will be 600px wide, and will look pixelated. Even with media queries - there is no way to switch the video input without stopping the first download, and starting a new download.  If you are 15 minutes into the mp4  - the browser would have to have logic to figure out where that time is in the video.  Needless to say, this is not a good user experience.

3. What happens when you start a video, but then you move to an area with lower bandwidth?  

The video download will not be able to keep up with the video playback, and the video will stall.



## Responsive video: JavaScript player

Rather than create a myriad of different sizes and file types in the ```<video>``` tag, the solution is adaptive bitrate streaming (ABS), supported by a JavaScript player sitting on top of the ```<video>``` tag. We’ve described how [ABS streaming](https://api.video/what-is/adaptive-bitrate-streaming) [works](https://api.video/blog/video-trends/responsive-video-delivery/) in  several posts. Esentially several different sizes of the video are created, and the video is delivered in short segments or chunks.  

The JavaScript player sits atop the video tag, and works as a controller for the video - monitoring both the screen size (and changes in screen size) and the network conditions.  When a change occurs that could affect the video playback, the player is able to make changes midstream to the video.  Since each chunk defines a timestamp in the video - switching midstream between different video qualities is a simple exercise for the player.

With ABS, jumping from 600p wide to full screen, or casting to a TV *just works*.  When the netwrork conditions deterioriate or improve, the player can adapt the playback quality to optimzie the experience for the viewer.

It is totally understandable to be hesitant in adding a new JavaScript framework to your page - we are so bloated in JS today (that's an entirely different topic) - but for a few hundred KB, the JS player enhances the video playback experience on all devices.  With a video that is specifically sized for mobile devices - the savings in video delivery is very likely to exceed the cost of the JS incurred on video setup.


## Conclusion

The video tag is amazing, and will continue to be the backbone of video on the web.  To make the video tag more responsive to various screen sizes, there are 2 possible approaches - reading media queries, or using a JavaScript controller to power adaptive bitrate video.

The media query approach solves one problem - but raises several more that impact the user experience.  The JavaScript player adds additional page weight, but delivers the video responsively, in a network-aware way.  We at api.video believe that adaptive bitrate video *is* the future of video, and the added weight of the JavaScript player to the video tag is a small tradeoff in exchange for providing stellar video experiences on the web.





Responsive media: the right approach for images vs video 

thumbnail_diseno-web-responsive-design.jpeg

diseno-web-responsive-design.jpeg

Responsive media: the right approach for images is not the right approach for video

small_anthony.png

thumbnail_anthony.png

anthony.png

Anthony Dantard

Customers are "divinely discontent" and their expectations are constantly on the rise. Thanks to services like YouTube & Netflix, we all expect blazingly fast load times and smooth video playback for streaming on-demand video. What might be less obvious is that, whether we're aware of it or not, that expectation is slowly transferring to real-time video communication and live streaming applications as well.

At api.video, we're obsessed with delivering the best developer and viewer experience for our customers. So it's no surprise that we spend a lot of time thinking about and keeping up-to-date on video protocol development. Each video protocol has its strengths and weaknesses. Each is better suited for different applications. 

In this post, we outline four of the major low-latency protocols, talk about the pros and cons of each, and give our commentary on where we think the future of low latency streaming is going. We'll talk about the following protocols in depth in this post:

- WebRTC
- LL-HLS
- LL-DASH
- HESP

## WebRTC
The WebRTC protocol supports both audio and video streams for real-time bi-directional communication. It can be used for ingestion and distribution with an end-to-end latency between 300ms - 600ms depending on network quality and the distance between users. The protocol really took off in 2011 when Google bought Global IP Solutions, the company that originally developed the protocol, and gave the project significantly more human and financial resources. 

Ten years later, WebRTC has become the standard for real-time video communication on the web. The components that WebRTC is based on are accessible via a JavaScript API maintained by the W3C and the IETF, allowing users to live stream directly to a web browser without installation of any third-party tool.

In theory, WebRTC allows the establishment of a video conference system between 2 browsers by connecting them to each other peer-to-peer. However, in practice the architecture of computer networks (internet, corporate, and domestic) can greatly complicate matters.

large_webrtcprotocolstack.png

medium_webrtcprotocolstack.png

small_webrtcprotocolstack.png

thumbnail_webrtcprotocolstack.png

webrtcprotocolstack.png

### Finding your way through the internet labyrinth

In some cases, we are not directly connected to the internet and we are behind several network layers such as a NAT, a firewall, or a proxy. To get around these problems, WebRTC allows you to use the ICE (Interactive Connectivity Establishment) protocol. It allows you to find the most direct path to make two machines communicate and cross the different network layers. For this, a STUN / TURN server is necessary in order to obtain the external address of a user and relay the traffic if a direct connection is not possible.

The WebRTC protocol requires, in addition to its own multimedia server infrastructure, a STUN / TURN server infrastructure to operate communications in a production environment with real massive use.

## To further confuse the issue

The protocol specification requires that all data exchanged between peers must be encrypted (video, audio, data application). Thus, the protocol embeds several security protocols to fill the gaps of the UDP protocol when it's used. 

WebRTC uses SDP (Session Description Protocol) to expose some properties of the peer-to-peer connection like the types of media to be exchanged and their associated codecs, the network transports, and some bandwidth information.

Respectively DTLS (Datagram Transport Layer Security) is involved to generate Self-Signed certificates, for the negotiation of the secrets used for encrypting media data, and for secure transport of application data between peers. SRTP (Secure Real-Time Transport) is used for the transport of video and audio streams. The last security protocol involved is SCTP (Stream Control Transport Protocol), which is used for the exchange of application data.

### The challenges of WebRTC scalability

Several network architectures and topologies can be implemented to operate communications with the WebRTC protocol, each responding to a specific use case, with advantages and drawbacks specific to each topology. We will not dig into topology details in this article, but be aware that SFU (Selective Forwarding Unit) is perhaps the most popular architecture in modern WebRTC applications today.

The main challenge with this protocol is the scalability on a planetary scale that is cost effective. We are not saying that WebRTC can't be scaled worldwide, it was true a few years ago at its early stage but it's not true anymore thanks to the great improvements and work bring by some great companies to improve topology. 

As WebRTC is a mashup of several protocols that each have to be learned and mastered by engineers rendering it very complex. It's difficult for most companies to implement it in their infrastructure at a global scale in a cost effective way. That's why only a few companies have succeeded at making it fully scalable today. They do so by mixing SFU and MCU (MultiPoint Conferencing Unit) topologies, then putting it on the edge.

## Low Latency HTTP ABR streaming protocols

In contrast to WebRTC, which is a peer to peer protocol that, in theory does not require a central server to establish communication between two machines, HTTP-based streaming protocols require the use of a server and communication over standard HTTP. They are based on the very foundations of the web.

### HLS/LL-HLS

The HTTP Live Streaming (HLS) protocol can be used to deliver live and on-demand content streams to global-scale audiences and was introduced by Apple in 2009 with scalability in mind rather than latency. While the average latency faced by users is around 15s, the HLS latency can reach between 30s and 1 minute. Even with performant infrastructure and optimized packaging and player configuration, one can expect latencies of around 6s, which is just too high for real-time live video use cases. However it's still the most popular ABR streaming protocol. Its success is mainly due to its great compatibility with a wide range of devices and browsers and supports advanced features like closed captions, ads, and content protection with AES encryption or DRM.  

Although the protocol can also be used to ingest video, it's normally only used for distribution and then paired with the RTMP protocol for ingestion. Below is a typical live streaming architecture involving both the RTMP and HLS protocol.

large_rtmphlslivestreamingworkflow.png

medium_rtmphlslivestreamingworkflow.png

small_rtmphlslivestreamingworkflow.png

thumbnail_rtmphlslivestreamingworkflow.png

rtmphlslivestreamingworkflow.png

Without digging into the details of how HLS works (see our blog [articles](https://api.video/blog/video-trends/what-is-hls-video-streaming-and-how-does-it-work/) to learn more on this topic), below is a simple schema describing how playlists and media segmentation gives HLS the ability to do adaptive bitrate streaming (ABS).

Diagram of an HLS playlist and segmentation

large_hlsplaylistsmediasegmentation.png

medium_hlsplaylistsmediasegmentation.png

small_hlsplaylistsmediasegmentation.png

thumbnail_hlsplaylistsmediasegmentation.png

hlsplaylistsmediasegmentation.png

So how did HLS evolve to support lower latencies? In the beginning, HLS only worked with MPEG-TS container formats associated with the H.264/AVC video codec. In 2016, support for fragmented MP4 (FMP4) was added and this led to support for the CMAF format and compatibility with DASH distribution. One year later, HLS added support for H.265/HEVC (FMP4 only), which then allowed for a significant reduction in bandwidth usage.

Thus, Low-Latency HLS, an extension of the base HLS protocol, was finalized in April 2020 and enables low-latency video and live streaming by utilizing partial segments and on-the-fly transfer of those segments while maintaining HLS's inherent scalability. On the first draft of this extension, the protocol required the support of "HTTP/2 Push" which made it difficult to implement. Unsurprisingly, as major CDN vendors chose not to support this feature, killing the ability to scale LL-HLS, as none of the CDN players could cache the contents. Fortunately, Apple listened to community and industry concerns and came back with a new revision of the protocol extension, that removed the need for "HTTP/2 Push," but kept the need of relying on the "HTTP/2" protocol. They merged the spec into HLS making it fully backward compatible with the HLS specification and that also means that all features covered by HLS can be retrieved in LL-HLS. 

LL-HLS works practically the same way as HLS, but makes some important changes to reduce the latency in the packaging process. Below are the most important updates to the specification which make low-latency possible while preserving scalability and ability to do ABR streaming:

- **Partial Segments**: A segment is divided into Partial Segments or "Parts" of a few milliseconds which are referenced in the Media Playlist. These have a very short cache lifetime in contrast to the parent segment which can have a longer lifespan on the CDN. Once a complete segment is generated, its associated Partial Segments can be removed from the media playlist to reduce the bandwidth footprint.
- **Preload hints**: The media playlist can have a “Preload Hints” tag, which allows the player to know that new partial segments are coming so that the player can start to request them and the server will respond to this request as soon as the data is available.
- **Block Playlist Reload**: This feature avoids media playlist polling between the player and the server by adding query parameters to the request which inform the server that the player wants the response only when the playlist includes a new segment or partial segment.
- **Playlist Delta Updates**: Thanks to the new "EXT-X-SKIP" tag, a player can request just the updated part of the Media Playlist and save the transfer cost of the data that it already has.
- **Rendition Reports**: The "EXT-X-RENDITION-REPORT" tag added for each rendition line indexed in the Primary Manifest (aka: Master Manifest) provides some useful information for the Player to operate ABR such as the last sequence number and the last partial segment available for each rendition.

large_ll-hls-workflow-and-media-segmentation.png

medium_ll-hls-workflow-and-media-segmentation.png

small_ll-hls-workflow-and-media-segmentation.png

thumbnail_ll-hls-workflow-and-media-segmentation.png

ll-hls-workflow-and-media-segmentation.png

With current public cloud providers you can expect to reach an end-to-end latency in the range of 2s with appropriate Group of Pictures , partial fragments, and reasonable buffer size. While this is still much slower than what WebRTC can achieve, it involves significantly less complexity and is easier to implement in your existing live streaming architecture. You can also push the protocol to its limits by changing some settings and achieve a latency of around 1s, but this comes at the expense of stability and the quality of the playback experience.

### DASH/LL-DASH

DASH stands for Dynamic Adaptive Streaming over HTTP and was born in April 2012 by the work of the Moving Picture Experts Group (MPEG) in order to create an industry-standard alternative to the HLS protocol owned by Apple. It works similar to HLS in that it's based on the preparation of content at different quality levels and divides each video quality into several chunks indexed in manifest files to enable ABR streaming. 

It also supports content protection with encryption through Common Encryption standard (CENC) which makes it compatible with all common DRM systems.

The main difference between LL-HLS is that LL-DASH is completely agnostic of the codecs used for encoding video and audio streams. Unfortunately this codec freedom comes with the cost that LL-DASH is not compatible with Apple devices relying on iOS, including Apple TV.

In 2017, LL-DASH provided the necessary changes to the standardized protocol to decrease latency to 2s. Behind the scenes, it relied on the Common Media Application Format (CMAF). CMAF offers the ability to play with some useful HTTP features that can be used to decrease latency a lot. The two features that allow Low Latency for DASH with CMAF are "Chunked Encoding" and "Chunked Transfer Encoding" — both part of the HTTP/1.1 spec (prohibited in HTTP/2) that respectively allow you to break a segment into smaller chunks of a few milliseconds that can be sent to the distribution layer once encoded and this is where the "Chunked Transfer Encoding" feature take the relay to allow to distribute encoded chunks on the fly. However to operate this kind of delivery process, the whole distribution stack has to support the chunked transfer encoding feature from the origin right through to the CDN.

Diagram of LL-HLS workflow and media segmentation

large_dashll-dash2.png

medium_dashll-dash2.png

small_dashll-dash2.png

thumbnail_dashll-dash2.png

dashll-dash2.png

### HESP

The High Efficiency Stream Protocol (HESP) is another ABR HTTP based streaming protocol and is the most recent one of all. It was created by THEOPlayer through the HESP Alliance, whose main mission is to work on the standardization of the protocol and its promotion.

HESP has been developed with the aim of solving several of the main constraints of other HTTP streaming protocols. It's goals are:

- To reach ultra low latency(<500ms) while preserving scalability (i.e. it can still work with major CDN vendors)
- To reduce the bandwidth required during the streaming
- To reduce zapping times (zapping is the time it takes to switch between streams, think of it as changing the channels on cable television)

These three performance metrics have a direct impact on the user experience of the audience of a live stream. Given the extremely low latency, it can also be a viable alternative to WebRTC. The main drawback of HESP is that it's a proprietary commercial protocol and–for commercial use–it's quite expensive. Let's dig a bit into a technical overview of how it works.

The main difference with HESP in comparison to other low-latency protocols is that it relies on two streams rather than one. Before we go into the details of how these streams allow us to reach sub-second of latency, let's talk a bit about the different types of frames that are required when streaming video:

In video compression there are several types of frames:

- IDR-Frame (aka: Key Frame)
- I-Frame
- P-Frame
- B-Frame

Let's start with the I-Frame, you will better understand the others. An I-Frame contains the whole image and is coded without reference to any other frame except itself.

The key frame (or IDR-Frame) is a special I-Frame where no frame after the IDR frame can reference any frame before it. That being said, all IDR-Frames are I-Frames but not the other way around. The key frame is used by any player to be able to start a playback.

The P-Frame holds only the changes in the image from the previous one.

The B-Frame holds the changes between the current frame and both the preceding and following frames.

Now that you know the role of the different kinds of frames composing a video, we can go back to the streams that compose the HESP protocol:

- The first stream is called the "Initialization Stream" and contains only key frames.
- The second one is called the "Continuation Stream" and is like a normal encoded stream, meaning that it can contain all types of frames depending on the encoding parameters and defined profiles for maximum performance or maximum compatibility.

The Initialization Stream is used only for the start of the playback or when you slide on the video timeline to change the position of the playback. Because this stream contains only key frames, the decoder behind the player is able to instantly decode the frame and so start(or restart) the playback of the live event. Once the first frame is pulled from the first stream and decoded, the player switches automatically to the second stream to continue video playback. This is because key frames are complete images, so they have a high cost in terms of bandwidth. By switching to the second stream, the player falls back to a regular live streaming bandwidth footprint and profits from the CDN capability for scaling the audience and reducing the load to the origin.

As for DASH/LL-DASH, HESP works also with CMAF-CTE for packaging and distribution. It inherits from all the features such as encryption, DRM support, subtitles and captions, advertisements, and more.

large_hesp-workflow.png

medium_hesp-workflow.png

small_hesp-workflow.png

thumbnail_hesp-workflow.png

hesp-workflow.png

This protocol sounds revolutionary on paper, right? It does respond to many issues, it's true. But as with other protocols–and everything else in life–nothing is perfect and therefore it brings its own set of drawbacks and limitations.

The first drawback is that, as the protocol works with two streams which are encoded in sync, the encoding and storage cost is higher than the other HTTP based streaming protocols.

The second drawback is that even if it relies on CMAF-CTE packaging and distribution, the packager has to be updated to handle the two streams instead of one in the encoding and distribution steps.

Finally, as with the packager, the player must also be updated in order to be able to play with the HESP protocol and handle all of its subtleties.

And finally, last but not least, you have to pay royalties for any commercial use which inherently limits its adoption.

### So which protocol is the best?

The quick answer is none. What a helpful answer for making a choice, right?

The more complete answer is that it will depend heavily on the use case that you want to cover and the time and resources (both financial and human) you have at your disposal.

If your needs is to offer to your customers and audience the ability to live stream with a reasonable amount of latency (6s-15s), while remaining cost effective , we recommend you use HLS and/or DASH protocols as they can scale easily to millions of viewers. You can find many open-source libraries and streaming platforms that have implemented them very well. Our preferred choice, is HLS, as it has the largest footprint in terms of adoption and support across browsers and devices. It can be played virtually anywhere. (If you want to test it, you can try our live streaming service for free just by creating an account [here](https://dashboard.api.video/register)).

In the case that latency is critical for your business, you should look at low latency or even ultra low latency protocols. If you only needs mono-directional live streaming capability with a latency in the range of ~2s (e.g. for sport events, concerts, e-learning with students listening a teacher and can ask questions in a chat) and you don't have a big budget, you should look at LL-HLS and/or LL-DASH protocols.

For the remaining use cases where more than 1s of latency is not acceptable, you don't have many options left. It's either going to be WebRTC or HESP.

If you're a non-profit organization but need to serve a large audience or just don't want to have to build super complex infrastructure and don't have a lot of cash in your pocket, take a look at the HESP protocol.

In case you need bi-directional video communication, WebRTC has proven its ability to do it well, but if building an in-house infrastructure isn't part of your core business, you'll probably want to rely on a provider who already successfully built such an infrastructure in a scalable fashion. Be aware that the likelihood of having to pay a high cost is quite high. Since HESP Alliance's royalty fees are client-based and offer different packages to meet the fees, if you have a reasonable audience this can also be a good option for limited budget.

Finally, if money is not a concern, our choice would be HESP because it involves much less work compared to an implementation of WebRTC and is compatible with a wide range of devices and of browsers.

At [api.video](https://api.video) we deeply believe that HTTP-based Low or Ultra Low Latency streaming protocols will win this battle in the end. The reasons are quite simple, these protocols involve much less work to be implemented in a legacy infrastructure, profit from more support across devices and browsers, can scale more easily than WebRTC, and–as they don't have prohibitive licensing fees–they can be adopted by a larger number of companies. 

This is why we're building end-to-end Edge video infrastructure based on HLS. We're confident that, in the end, HLS will be the single streaming protocol that will meet all video streaming use cases in the near future.

The future of [ultra] low-latency video streaming

Ultra low latency for live streaming video

large_add-a-subheading.png

medium_add-a-subheading.png

small_add-a-subheading.png

thumbnail_add-a-subheading.png

add-a-subheading.png

Top 7 causes of streaming video buffering issues

Nobody likes waiting for buffering. In [one survey](https://www.streamingmedia.com/Articles/News/Online-Video-News/Streaming-Video-Services-Are-Getting-Worse-63-See-Buffering-122408.aspx), 63% of streaming viewers went so far as to say buffering issues were one of the most serious problems they encountered when trying to watch video. Slow buffering can take you out of the moment in an important part of a TV show, film, or game. Sometimes it can end your interest in watching something altogether if the buffering keeps happening with long delays. The way buffering is supposed to work, is your device streams ahead of what you’re watching so that there’s always more video ready for you to see. If everything goes according to plan, you’ll seamlessly be able to watch your video content. But when it doesn’t go to plan, what’s the cause? The source of your buffering problem can come from a few different places.

The easiest way to start diagnosing your buffering problem is to check your internet speed. 
If your internet is slow, you’re going to have buffering problems. There’s a number of reasons your internet could be slow, so it’s best to start with a test. You can choose from any of these free online speed tests: [Internet Speed Test Sites](https://www.lifewire.com/internet-speed-test-sites-2626177). Should you determine that your problem is internet speed, you can then start looking at what’s slowing you down. 

## 1. You’re using wifi

Wifi is great, but sometimes it’s a lot slower than Ethernet. [Ethernet can transmit data at up to 10 Gbps per second, or higher, depending on the service you buy and your equipment. Wifi on the other hand, maxes out at around 6.9 Gbps or less.](https://blog.rottenwifi.com/wifi-vs-ethernet-speed/) If you’re having trouble with buffering, you might consider connecting your playback device using an Ethernet cable if it’s possible. This is a secure, fast solution that might solve your buffering problem. If you can’t set up your device to work with an Ethernet cable (for example if you’re using your mobile phone, or sometimes a device like a Roku or Fire stick), we’ll go over additional ways you can solve buffering issues.

## 2. Too many devices on your system or too many applications running

If you connect the maximum number of devices to your system, or you have too many devices trying to do something at once, this can slow down your network. Then you’ll get buffering problems on what you’re watching. A simple solution is to turn off some of the devices while you’re watching TV or a movie. If you’re not watching on your TV, but using a computer instead, then you might try closing applications and windows. The less your computer has to process, the more it can focus on delivering you a seamless video experience. 

## 3. Your internet plan

Some internet plans don’t provide speeds that are suitable for seamless video viewing or gaming. Check to see what you have, and then use the speed test we talked about earlier to make sure you’re getting what you were promised. If everything seems under the speed offered and you’ve diagnosed everything else, you can consider a different provider. You can also upgrade your plan if you are getting what you were promised, but the internet speed isn’t fast enough for your needs.

## 4. Old technology 

Some people love updating their computers and gadgets and routers all the time. If you’re not one of these people, you might have old computers, TVs, or an old router. Old computers and TVs are less efficient, but the real show stopper here would most likely be an old router. Old routers can cause interference with Wifi connectivity and slow down your internet speeds. So, if you can, try to keep your router up-to-date. It also helps to keep your devices up-to-date. If you can’t, then sometimes you can just update your graphics card and that will help. 

## 5. Change video resolution

If you don’t mind a lower quality video, you can try changing the resolution for your content. Choosing a lower level of quality will make it easier for the video content to be delivered as part of a seamless viewing experience. If you are using a small screen, it’s difficult to tell the difference between 720p and 1080p, so you won’t be missing much. As a general rule of thumb, the smaller your viewing screen, the lower the quality can be. Try different resolutions to see what works for your eyes and your system. 

## 6. Check your VPN 

If you added a VPN to your system, this slows performance because your data must be encrypted then decrypted at a remote server. Something you can do, is test your VPN to figure out if it’s impacting video delivery on your devices. 

## 7. Check your browser settings

If you are viewing from a browser on a computer, the problem could be your web browser. Clearing the cache, removing cookies and browsing history can speed everything up and improve your video viewing experience. 

## api.video and buffering

api.video helps avoid buffering issues by providing blazing fast transcoding and delivery via HLS. HLS uses adaptive bitrate, meaning your customers get the best quality video at the best speed for their devices. api.video also helps you and your customers avoid buffering by using its own, video-focused Content Delivery Network (CDN) and edge servers so that content can be delivered to users as quickly as possible from somewhere local to them. If you’re looking to add video to your website, platform, or application, api.video offers a complete, flexible end-to-end solution for VOD, live streaming, transcoding, hosting, delivery, and usage analytics. Sign up for an [api.video trial account](https://api.video) to try it out. 

large_top-7-causes-of-streaming-video-buffering-issues-2.jpg

medium_top-7-causes-of-streaming-video-buffering-issues-2.jpg

small_top-7-causes-of-streaming-video-buffering-issues-2.jpg

thumbnail_top-7-causes-of-streaming-video-buffering-issues-2.jpg

top-7-causes-of-streaming-video-buffering-issues-2.jpg

## The history of live streaming

Live internet streaming didn’t start all that long ago - in fact it debuted only ten years after the official birthday of the Internet, which is touted as January 1st, 1983. Since then, live streaming technology has developed a lot. It’s hard for us to imagine now, but when the Internet first started there was no music or video. Let’s go back in time and see how far this technology has come! 

1993: The first live streaming event on the internet is created by a garage band
On June 24th, 1993, the band Severe Tire Damage was playing a gig at Xerox PARC. While this was going on, scientists were working on new technology called MBone. MBone was an experimental network for carrying IP multicast traffic. (IP multicast is used to stream media to multiple viewers.) It just so happened that the Chief Technologist at Xerox PARC, Mark Weiser was the drummer for Severe Tire Damage. They decided to test the network’s capabilities by broadcasting their band playing. This first stream was everything you’d imagine — super slow frame rate, very pixellated and it involved a bunch of nerds jamming together. See for yourself: https://player.vimeo.com/video/56349011

Additionally, in order to stream the content, they used half the bandwidth of THE ENTIRE INTERNET. 

## 1994: The first virtual concert live streams on the internet
In this situation, Severe Tire Damage reappeared to open for The Rolling Stones. Surprise! Because the internet was still small, and there was one Mbone network, the band connected about twenty minutes before The Rolling Stones concert started, and described themselves as being part of the first ever virtual concert, because they were popping in to play from California and then The Rolling Stones concert was going to happen in Texas. The news media was watching, and got a big enough kick out of their surprise performance that some articles spent more time on Severe Tire Damage than they did describing the rest of the concert (which was probably epic). When asked to comment on their extra act, The Rolling Stones told The New York Times “the surprise opening act by Severe Tire Damage was a good reminder of the democratic nature of the Internet.” (Reference: [https://query.nytimes.com/gst/fullpage.html?res=9A04EEDE1F31F931A15752C1A962958260](https://query.nytimes.com/gst/fullpage.html?res=9A04EEDE1F31F931A15752C1A962958260) 

## 1995: The first sporting event live streams on the internet
The internet company RealNetworks created a media player that could handle live streaming - RealPlayer. To show off RealPlayer’s capabilities, the company broadcasted a baseball game between the New York Yankees and the Seattle Mariners. The demo was interesting, but live streaming didn’t catch on.

## 1997: RealVideo launches
RealVideo was a program that let you commercialize live streaming. Things weren’t quite in place for this to catch on, but it remains an important milestone in live streaming. 

## 1999: The first presidential webcast streams on the internet
Bill Clinton appeared at George Washington University in Washington DC to speak in a webcast titled ‘Third Way Politics in the Information Age.’ Viewers online could submit questions and have them answered.

## 2008: YouTube hosts a live event and streams it from San Francisco and Tokyo at the same time
YouTube’s live event, named YouTube Live streamed from San Francisco and Tokyo. It included performances from artists like Katy Perry, Smosh, MythBusters and Bo Burnham. They continued offering occasional live streams, but their broadcasts were a bit slow so this didn’t take off for YouTube. It did put live streaming on the map though. 

## 2011: Twitch launches a live streaming platform for video games
Originally justin.tv, which was founded by Justin Kan. He broadcast his life 24/7 and popularized the idea of live or "life" streaming. Later he rebranded his company as Twitch and opened it up to many users. Twitch capitalized on YouTube’s inability to capture the live streaming market. They focused on live streams of gamers and within two years they had 45 million viewers each month. Their success led to other social media platforms rushing to add live streaming capabilities. 

## 2015: The live streaming app Periscope launches
Periscope was an app bought by Twitter before launch. It allowed users to live stream immediately. When it became obvious through Periscope that live streaming was popular and could be lucrative, other social media platforms followed suit. Facebook Live appeared in 2016. Today popular social media apps have great live streaming - YouTube and Instagram included. 

Today there are a number of trends in live streaming that are pushing the value of live streaming even further. People can easily give broadcasts talking about their company or work. There’s live shopping, where products are modeled and talked about by influencers live and viewers can purchase while watching. 

## 2020: api.video and the history of live streaming
api.video wants to help developers build the incredible video products they dream of. That’s why we offer an end-to-end solution that lets you build live streaming features into your website or mobile app with ease.

## Resources

1. The Not-So-Ancient History of Live Streaming - https://switchboard.live/blog/live-streaming-history/
2. The History of Live Streaming - https://espressolive.com/blog/the-history-of-live-streaming/
3. The Fascinating History of Live Streaming - https://restream.io/blog/history-of-live-streaming/

 The history of live streaming 

small_severe-tire-damage.png

thumbnail_severe-tire-damage.png

severe-tire-damage.png

The history of live streaming (and how a bunch of nerds opened for the Rolling Stones)

If you're new to video content creation, at some point you'll ask yourself whether you should host your videos on a site like YouTube or your own website. YouTube is incredible for content creators and for those looking to build fanbases for their content. For a business that wants to make a conversion to a buying customer, YouTube may not be the best spot for that business's video content. Instead, for many businesses the best place for video is their own website. And by that I mean hosting video content yourself, either entirely or by using a flexible cloud video/video streaming provider. But rather than take my word for it, consider these aspects of handling video content for yourself to make the best decision: 

* Business goals and requirements
* Reach 
* Engagement 
* Conversion 
* Analytics 

## Choosing business goals and requirements
The first step in choosing the right video strategy for your business is going over your goals and requirements. For example, if you prioritize minimal effort, YouTube might be great. However if you prioritize conversions and you're willing to put in a little effort for video SEO and other optimizations, then hosting video on your website might be better. 

Make a list of everything that's a requirement or goal to start with. And as you go over the rest of this article, you can refer back to and refine your list. If anything in your list conflicts (for example choosing minimal effort and hosting content on your own site) you can reprioritize or remove certain items. By the end of this article you should have a great idea of what option works best for you.

## Reach
When deciding whether to post videos on YouTube, or host them yourself on your site, the first thing to consider is reach. In marketing, reach is how many people see your content. It doesn't necessarily mean they engage with the content, but it means it's at least put in front of them. A big selling point for YouTube is that when you add your video there, you get the benefit of YouTube's algorithms like related videos, circulating popular videos, automatic video SEO and indexing of your content and features like playlists (see how to create your own playlist feature here: [Playlist](https://watch.a.video/playlist/)), YouTube search and more. This has the potential for huge reach.

It sounds great. The problem is about 500 hours of video are uploaded to YouTube every minute. The odds that content for your business is going to be popular right off the bat and benefit from some of the YouTube algorithms geared for popularity are not strong. While you might put in minimal effort tweaking video SEO for reach, you will put in maximum effort figuring out how to: 
* Get subscribers
* Engage relevant audience
* Buy google ads to advertise your content 
* Drive people to your YouTube channel 

Another issue is that you are spending time driving people to your YouTube channel. If you're selling something, you probably want people to go to your website and buy what you're selling. So you're sending traffic to YouTube to watch a video to convince them to go to your website. 

You can cut out a step by placing the video on your own website. It can be tricky to host your own video content, so you can host, stream and deliver content from a cloud video provider. By placing the content on your own site, you now benefit from your own site's existing reach, the SEO work you do for your site and when you advertise you are always driving traffic to something on your site instead of elsewhere. You can also do your own video SEO to get your content indexed. The end result is a centralized spot where people go to learn about you. The downside is if you wanted monetization of content handled for you, it wouldn't be. You have to handle that yourself. 

## Engagement 
Engagement is about creating an interactive experience with your content. It's coming up with ways to get them to watch more of the videos you create, check out more of your products and become more loyal and attached to your product by encouraging them to interact with you. Interactions can be all different kinds of things, from clicking through different pages of content to agreeing to join a newsletter to signing up for a free trial of a product.

YouTube is designed to maximize audience engagement, but specifically focused on watching videos. You can subscribe to content, explore new content, review your watch history, create or watch playlists made by others and YouTube is constantly trying to show you content it think you will like based on your other viewing habits. If all you need is people watching video content, this is perfect. The issue is, YouTube has its own goals for engagement, and they might not align with yours.

On YouTube, their goal is to keep viewers on YouTube, watching as many videos as possible so they can show you as many ads as possible. Your content is not a priority. If you know how to create material that is valued by YouTube because of the kind of ads they can place on your content, then you can do very well there. You won't be able to easily add other kinds of engagement though. Everything will be focused on interacting with video content.

If instead, you decide to set up content on your own site, the immediate plus is you can make your content the priority. At the end of a video, you can always recommend your own content for further viewing. You can also control how it may be viewed and by who (check out our article about creating self-guided content [Build a self guided training course with api.video](https://api.video/blog/tutorials/build-a-self-guided-training-course-with-api-video/). And to start with, only your content will be available. Depending on how elaborate you want to get with the video content you offer, you can create your own algorithms for what videos to show the viewer next based on their session (see our article about [Dynamic Metadata](https://api.video/blog/tutorials/dynamic-metadata/)). You can choose to keep your content ad free and create your own calls to action. Maybe you ask the user to watch something and then sign up for your product right on the site. Or you ask for their email so you can send them a newsletter. You have many more options than you do on YouTube. 

## Conversion
There is no doubt that the best place for conversion starts with your website where you're selling your product or offering your trial. Extra steps, like bringing a person from YouTube to your site, increase the chance that a potential customer will lose interest or find something else to do online. 

YouTube, as we discussed earlier, is great about coming up with new content for viewers to watch. If you just interested a potential customer in your product, but YouTube distracts them into watching a cool video where they then forget to check out your website, you've lost a conversion. There aren't tons of obvious ways to get conversions out of a YouTube video. You can possibly embed links, but these take you away from YouTube, and frequently people would prefer to stay on the site watching content. 

Another issue is that your YouTube video is using the excellent, automatic video SEO provided by YouTube. This is great, but YouTube gets the traffic from that. And then, because it's not a great place for conversions you lose out.

It's much better to put the content on your own website, spend the extra time to set up the video SEO, and then manage the traffic from that. 

## Analytics
In order to improve the effectiveness of your marketing, analytics are a must. YouTube offers detailed analytics and a plus is they're easily accessible through a GUI if you aren't technically inclined. One potential issue is you may have to do some extra work to figure out what traffic comes from your YouTube content to your site. 

You can get the same degree of detail with analytics from cloud video providers as well. Places like Vimeo and api.video (yes us!) offer detailed analytics. At api.video, you can get all kinds of details about your videos, like location (here's a fun post about creating a [Choropleth map](https://api.video/blog/tutorials/use-observable-to-create-a-choropleth-map-of-your-viewers/)), device type, where the viewer came from to start the session, what they did with a video during a session (for example, did they drop off part way through, where was that?) and more. 

large_choropleth.png

medium_choropleth.png

small_choropleth.png

thumbnail_choropleth.png

choropleth.png

On a flexible video platform like api.video, because you have control of everything, you can use analytics to decide whether to localize content, when and what to show viewers, and more. 

## Conclusion 
YouTube is an incredibly interesting and creative platform. It's perfect for content creators making a name for themselves, developing a visual aspect of their brand that doesn't require conversions, or anyone who wants to spend time mastering how to create successful video content on YouTue. 

Your own website is the best platform for video content if you are interested in driving traffic to your site for conversions. Nothing beats having everything in one place or pointing people to one place to learn more. You should consider a cloud video platform that will transcode, host and deliver your content but leaves the rest up to you. The video SEO is a bit more effort, but it is well worth it. If you want to give api.video a try, register here: [https://my.api.video/register](https://my.api.video/register)

Should I host my videos on YouTube or my own website? 

small_youtube.png

thumbnail_youtube.png

youtube.png

Live stream status

[FFMPEG](https://api.video/what-is/ffmpeg) allows you to stream and [live stream](https://api.video/live-streaming/) video, but sometimes what the output means can get confusing. 

We are going to go over every part of the information in the output, so if something seems obvious, skip ahead to the part you wanted to learn about.

Here's our sample. This sample shows a session where a live stream is started using my Mac and the Mac's iSight camera. I started it using the [livestream.a.video](https://livestream.a.video) demo through a Chrome browser. There might be some differences based on what computer you're on, so keep that in mind. 

## Hello from mediarecorder-to-rtmp server!

We're using an RTMP connection, which is based on TCP. So when I connect with my request to create a live stream, my client gets a response: 

"SERVER:Hello from mediarecorder-to-rtmp server!" 

I'm asked to provide my stream key, which is going to be what I will use to broadcast. Because of how the demo is set up, that's already been sent, so now the server tells me it's received the information about where I want to broadcast my live stream to: 

"SERVER:Please set rtmp destination before start streaming.
SERVER:rtmp destination set to:rtmp://broadcast.api.video/s/30087931-229e-42cf-b5f9-e91bcc1f7332"

The last part tells me what version of FFMPEG we're using and provides a link to static builds of FFmpeg for Linux kernels 3.2.0 and up. It also gives me copyright information - wow we've had FFMPEG for 22 years!

## Configuration details 

The next part of the FFMPEG output shows us the configuration details for the live stream. 

In this section, we're told that FFMPEG was built with gcc 8. GCC stands for GNU Compiler Collection and it's the standard compiler for Linux. It's associated with the Linux distribution Debian (Debian GNU/Linux). The Debian Linux distribution is super popular and while it's less user friendly than distributions like Ubuntu, it is great for people who like free and open source software. 


## FFMPEG library versions
This section of output lists the main libraries in FFMPEG and the versions you have installed. 


## FFMPEG input info 

This section shows you details about your video live stream. 

Here's the first line: 

"FFMPEG:Input #0, matroska, webm, from 'pipe:':

This explains how video is being input. We can see here that the multimedia container matroska and the open video format webm is being used to capture video.

"Metadata:
  encoder         : Chrome
 Duration: N/A, start: 0.000000, bitrate: N/A"
 
 Next, there's more information about the streams being input: 
 
 "Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
  Stream #0:1(eng): Video: h264 (Constrained Baseline), yuv420p(progressive),   
	240x240, SAR 1:1 DAR 1:1, 1k tbr, 1k tbn (default)"

### The audio stream
The audio stream is using the opus codec with a sample rate of 48000 Hz, which is standard for digital video files. Mono means the audio is monophonic, meaning all sounds are mixed together in a single channel. And fltp is Planar Floating point format. It's a way of retrieving the sample format for your audio. Sample format information is used to determine how many bits will be in each audio sample. The higher the number of bits per sample in a file, the more data that will be stored in the sample. You can learn more about sample format here: What is Sample Format?

### The video stream
The video stream is using the h264 codec with a constrained baseline. The term constrained baseline describes the video stream profile. A profile determines the methods that an encoder can use to generate streams and also describe the methods a decoder is expected to perform in order to decode the stream later. The term 'yuv420p' describes the type of video signal you're using. 

YUV is a type of video signal where colors are split into three channels, one for luminance (brightness) which includes green, and two for chrominance (color) which include blue and red in separate channels. 420 describes the chroma subsampling used on the video. Normally, you'd notate this as 4:2:0. It's telling you that the color channels have half the resolution of the luminance channel. If you want to learn a bit more about this way of representing color, see the api.video definition for YCbCr. The term progressive means that the video is in a progressive format, which means that the TBR value (coming up in our discussion) is talking about the frame rate rather than the field rate. There's confusion about both these terms. For the purpose of this walkthrough, let's keep things simple. A progressive video format will use frame rate to describe frames per second. An interlaced video format will use field rate. It takes two fields to show you one image. 

240x240 describes the pixels in each frame of video. SAR stands for Sample Aspect Ratio, and it's showing you the ratio between the width and the height of each individual pixel - 1:1. DAR stands for Display Aspect Ratio and that's the ratio between the width and height of a full image in your video. In this case it's also 1:1. 

In the final line, it closes out with "1k tbr, 1k tbn (default)." TBR represents the frame rate baseline. This is guessed by FFMPEG when it looks at the video stream. At the start of a video stream, TBR is low. In our stream, it's 1 at the very start. Sometimes if you have a video format that's interlaced, the TBR will be twice as high as expected, because there are two fields used to create an image. TBN represents the time base in AVStream that comes from the container. TBC is the time base for the codec for the stream.

## Stream mapping 
Next in our information from FFMPEG we have a section about stream mapping that looks like this:

Stream #0:1 is our video stream. This means that the input stream is from the first file '0' and it is the second stream in that file '1.' h264 (native) means that the detected codec of the input stream is H.264 and that FFMPEG is going to use the built-in decoder to decode the stream. It's being mapped to h264 (libx264) meaning that the output stream is going to be H.264 and FFMPEG will use the libx264 encoder to generate the stream. 

Stream #0:0 is our audio stream. This means that the input stream is from the first file '0 and its the first stream in that file '0.' opus (native) means that the detected audio codec of the input stream is opus and as before, native means that FFMPEG is going to use its built-in decoder for opus to decode the stream. aac (native) means the output audio will be in aac, and FFMPEG will use the built in encoder for aac to generate the stream. 

## Configuration details for libx264
We're mapping to libx264, so now we receive some details about how libx264 is configured for our stream. 

You can see the Video Buffering Verifier confirms that there's a buffer size set, but the max bit rate it can handle isn't specified, so it's ignored. Then the SAR is 1:1, as it was to start with. We can see the CPU capabilities available for sending our live stream and that we'll be using the constrained baseline for H.264. 

Next there's the line: 

"FFMPEG:264 - core 164 r3065 ae03d92 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html"

This tells us what version of the H.264 codec FFMPEG uses. It's followed by options that are configured for the codec. 

**cabac=0** 
Cabac stands for context-adaptive binary arithmetic coding. It's a type of entropy encoding, which is a lossless data compression technique. H.264 is lossy, so cabac is used at the end to do a final compression that is lossless. You should not mistake its inclusion in H.264's compression steps as meaning all the steps are lossless. Also, explaining this algorithm in detail would take some time. If you want to learn more about cabac, look here: Context=adaptive binary arithmetic coding

**ref=1**
This indicates which frame will be the first reference frame. Here, it's the first frame.

**deblock=0:0:0**
This is the deblock filter for H.264. It creates pixelation to help improve the appearance of your video. It's useful when macroblocking occurs, which would be a huge block of pixels that look unnatural. Deblocking blurs these blocks to make them look normal again.

**analyse=0:0** 
This is part of deblocking. It helps determine the size of each block you will apply deblocking to. 

**me=dia**
This is saying that the motion estimation method is diamond or dia. Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. Diamond search technique for block motion estimation is based on diamond search patterns. The process of block matching motion estimation is to find out a candidate block, within a search area in the previous frame, which is most similar to the current block in the present frame. So you're saying here use this technique for motion estimation.

**subme=0** 
This is for the sub-pixel motion estimation method. It's important for video coding inter-frame prediction and has high computation complexity. It's set to 0 meaning there will be zero sub-pixel motion estimation done. 

**psy=1 psy_rd=1.00:0.00**
This is also to do with motion-compensated DCT, which is a lossy block compression transform coding technique. If set to 1 it means use psychovisual optimizations. It means optimizing quantization tables based on the human visual system and humans see. It's a complex topic that can't be fully explained here. psy_rd is about setting how strong the psychovisual optimization will be.

**mixed_ref=0** 
This allows you to use one reference per partition as opposed to per macroblock. A macroblock is a large block of pixels and a partition is a smaller, multiple variable-sized prediction block. 

**me_range=16**
Again about motion estimation, but this time about the range for a motion vector. In video compression, a motion vector is the key element in the motion estimation process. It is used to represent a macroblock in a picture based on the position of this macroblock in another picture, called the reference picture. This parameter lets you set the range.

**chroma_me=1** 
Typically when doing motion estimation, only luminance is examined. You can have chroma considered as well by setting chroma_me to be 1.

**trellis=0** 
This lets you set the rate-distortion optimal quantization. This is a method used to improve video quality in video compression. It optimizes the amount of video quality lost per chunk of data by measuring how far an optimization will deviate from the source material and the cost of that deviation. This can be used to decide what choices are useful vs. resulting in the loss of too much data. It's a slow process, so sometimes it's better not to use it, which is what's being done here by setting trellis=0.

**8x8dct=0** 
With this parameter you can enable adaptive spatial transform. This is a way of altering an image or video frame through rotation, flips, scaling and deformation. For our case, where it's for video, it's about ways to manipulate the block of data so it appears less distorted and of higher quality.

**cqm=0** 
Allows the use of a custom quantization matrix to weight frequencies differently in the quantization process. The presets quant matrices are "jvt" and "flat". Recommended only if you know what you're doing.

**deadzone=21,11** 
When trellis isn't activated, deadzone parameters determine how many DCT coefficients are rounded up or down. Rounding up results in higher quality and more detail retention, but costs more bits--so rounding is a balance between quality and bit cost. Lowering these settings will result in more coefficients being rounded up, and raising the settings will result in more coefficients being rounded down. Recommended: keep them at the defaults. This is part of the settings for rate-distortion optimal quantization. This is again fairly complex to explain. If you want to learn more about it conceptually, check out this page: Quantization

**fast_pskip=1** 
By default, x264 will skip macroblocks in P-frames that don't appear to have changed enough between two frames to justify encoding the difference. This considerably speeds up encoding. However, for a slight quality boost, P-skip can be disabled. In this case, the full analysis will be done on all P-blocks, and the only skips in the output stream will be the blocks whose motion vectors happen to match that of the skip vector and motion vectors happen to match that of the skip vector and which have no residual. The speed cost of enabling no-fast-pskip is relatively high, especially with many reference frames. There is a similar B-skip internal to x264, which is why B-frames generally encode much faster than P-frames, but it cannot be disabled on the commandline.

**chroma_qp_offset=0** 
This stands for chroma quantization parameter offset. It is the quantization parameter difference between chroma and luma. 

**threads=3**
FFMPEG is multi-threaded by default. How many threads it will use depends on the codec. By default for libx264 it is cores x1.5 for frame threads and cores x 1 for slice threads. 

**lookahead_threads=3** 
A lookahead thread looks at incoming frames before they are transcoded. Number of lookahead threads can impact performance speed.

**sliced_threads=1** 
Sliced threading decodes multiple parts of a frame at the same time.

**slices=3**
If the codec you're using supports multi-threading, you can decode more than one part of a single frame at once by working on multiple slices. This is only available if the video was encoded using slices.

**nr=0** 
Confusingly, this stands for noise-sensitivity. You can use it with static-thresh to change the threshold on blocks that are examined in an image. If they fall below the specified threshold the encoder skips them.

**decimate=1** 
This allows you to drop duplicated frames at regular intervals. 

**interlaced=0** 
This is a simple interlacing filter from progressive contents. This interleaves upper (or lower) lines from odd frames with lower (or upper) lines from even frames, halving the frame rate and preserving image height.

**bluray_compat=0** 
Configure the encoder to be compatible with the bluray standard. It is a shorthand for setting "bluray-compat=1 force-cfr=1".

**constrained_intra=0** 
You can enable constrained intra prediction, which is required for the base layer of SVC encodes. This is an intra prediction technique where the video encoder limits the use of neighboring blocks as reference blocks in the intra prediction process. If you're using constrained intra prediction, you can configure video to not use neighboring blocks as reference blocks if the neighboring blocks were coded using inter prediction. By not using inter predicted blocks as reference blocks for intra prediction, the video encoder may create an encoded video bitstream that is more error resilient.
 
**bframes=0** 
You can set the maximum number of concurrent B-frames that x264 can use. Without B-frames, a typical x264 stream has frame types like so: IPPPPP...PI. With --bframes 2, up to two consecutive P-frames can be replaced with B-frames, like: IBPBBPBPPPB...PI. B-frames are similar to P-frames, except they can use motion prediction from future frames as well. This can lead to significantly better efficiency in terms of compression ratio. 

**weightp=0** 
Use this to enable use of explicit weighted prediction to improve compression in P-frames. Higher modes are slower.

**keyint=30** 
This sets the maximum interval between IDR-frames (aka keyframes) in x264's output. You can specify "infinite" to never insert non-scenecut IDR-frames. IDR-frames are 'delimiters' in the stream - no frame can reference data from the other side of the IDR-frame. As well as this, IDR-frames are also I-frames, so they don't reference data from any other frame. This means they can be used as seek points in a video.

**keyint_min=16** 
This sets the minimum length between IDR-frames. Very small keyint ranges can cause "incorrect" IDR-frame placement (for example, a strobing scene). This option limits the minimum length in frames after each IDR-frame before another can be placed.

**scenecut=0** 
This sets the threshold for I/IDR frame placement (read: scene change detection). x264 calculates a metric for every frame to estimate how different it is from the previous frame. If the value is lower than scenecut, a 'scenecut' is detected. An I-frame is placed if it has been less than keyint_min frames since the last IDR-frame, otherwise an IDR-frame is placed. Higher values of scenecut increase the number of scenecuts detected.

**intra_refresh=0** 
You can disable IDR-frames, instead H.264 uses intra coding for each macroblock in the frame every keyint_min frames. Blocks are refreshed in a horizontally scrolling column - the 'refresh wave'. This benefits low-latency streaming by making it possible to achieve more constant frame sizes than is possible with standard IDR-frames. It also increases the resilience of the video stream to packet loss. This option does reduce compression efficiency, hence only use it when needed.

**rc=crf**
rc stands for Rate Control. You can adjust the file data rate in several ways, the option shown here is CRF, or Constant Rate Factor. CRF is an encoding mode that adjusts the file data rate up or down to achieve a selected quality level rather than a specific data rate. CRF values range from 0 to 51, with lower numbers delivering higher quality scores.

**mbtree=0** 
This stands for macroblock tree rate control. Macroblock tree rate control improves compression by tracking how long different parts of an image last across frames. For parts of an image that stay the same, you don't need to continually save that data, only make a note that it continues to be there so you can use that as a reference to reconstruct that part of the frame later. 

**crf=25.0** 
This is related to the rc=crf command. CRF is an encoding mode that adjusts the file data rate up or down to achieve a selected quality level rather than a specific data rate. CRF values range from 0 to 51, with lower numbers delivering higher quality scores.

**qcomp=0.60** 
The default setting is 0.60. qcomp stands for quantizer curve compression factor. A setting of 0 will give you a constant bitrate while a setting of 1 will give you a constant quantizer. This can be combined with mbtree, but the higher your qcomp setting, the weaker the effect of the mbtree setting.

**qpmin=0** 
The default setting for this is 0 and raising this minimum is discouraged because it can negatively impact the quality of background areas in your frame. 
qpmin defines the minimum quantizer that H.264 will ever use. The lower the quantizer, the closer the output is to the input. 

**qpmax=51** 
The default setting for qpmax is 51. The default of 51 is the highest quantizer available for use in the H.264 spec, and is extremely low quality. This default disables qpmax. You can set this as low as the 30-40 range if you want to cap the minimum quality x264 can output, but adjusting it is not recommended.

**qpstep=4**
This allows you to set the maximum change in the quantizer between two frames. 

**ip_ratio=1.40** 
The default setting is 1.40. You can change the average decrease in quantizer for B-frames vs. P-frames, with higher values decreasing B-frame quality.

**aq=0**
aq stands for Adaptive Quantization. This helps distribute bits between macroblocks in your video. If you don't use it, it can underallocate bits to sections that lack detail. A setting of 0, as with other FFMPEG settings, means you will not use it at all. 

## Output to api.video
Now that you've got all the configuration settings for your streams and you've mapped them, you'll want to see some details about what will be output to your destination. 

Here you can see the destination, shown by the stream key and then a summary of details about your video and audio streams. The encoders being used are listed as well. We don't need to walk through everything listed here because we covered it in the earlier sections when we mapped the video and audio streams. FPS is frames per second. 

Now we can take a look at the lines that show us details about each chunk of data being streamed. Here's one of the lines: 

"FFMPEG:frame=65 fps= 55 q=12.0 size=29kB time=00:00:05.06 bitrate=  47.1kbits/s speed=4.26x"

The line is a snapshot where everything that is happening at that moment is reported to the terminal. It says that in this moment, you're streaming frame 65. The frames per second is 55, and the quality (q) is 12. The lower the number for quality, the better the video. The higher the number, the worse. The size of the chunk of data being streamed is also listed in kB with size. The time is the time from when the live stream started, and the bitrate is how many kbits per second yu're streaming. speed is how fast everything is streaming. 

## Conclusion
While this guide doesn't tell you everything about all concepts (for example quantization, which is too complicated to include in this post) I hope that this walkthrough helped explain what things are a bit more.
To help you further, you can go on [our Community](https://community.api.video/) or [start building](https://dashboard.api.video/register) now.



What do my video output live stream details from ffmpeg mean ? 

What do my video output live stream details from ffmpeg mean?

large_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

medium_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

small_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

thumbnail_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

Software video codecs are very diverse. Each one is carefully optimized depending on what the codec designers hope to achieve. For example codecs used when capturing video will probably try to retain as much data as possible. However a codec that's going to be used to compress video for conferencing might decrease video quality and increase transmission speed to lower latency. While they may have diverse use cases, one thing software codecs have in common is the design principles applied to their creation. There are five principles applied to every software codec. 

## Principle 1: Maximize allocation of processing resources based on your codec use case 
As I just mentioned, different codecs may hope to achieve different levels of efficiency across their elements based on what they'll be used for. For software codecs, processing resources and available transmission bit rate are what limit the design. Each design is about considering trade-offs, if you put more resources into one area of functionality in a codec, it will become more efficient, but then less resources are available for other areas of functionality. There are four main areas a codec will allocate resources to:

* **Encoded frame rate** - You'd refer to the use case for your codec, say it's video conferencing, and then try to maximize allocation of resources to efficiently achieve the standard 12-15 frames per second that's expected for desktop video conferencing. If you have a unique, efficient way of increasing quality, maybe you would be able to increase the frames per second. At a minimum though, you'd want to achieve the standard.
* **Frame size** - Frame size is what it sounds like, how many pixels wide and tall each frame of your video is. Depending on what level of quality or detail you want to offer with your codec, you'd want to maximize frame size as much as possible. Your use case for the codec would dictate how you choose to maximize frame size. 
* **Peak coded bit rate** - The coded bit rate is how many bits are used per second to represent video. While higher coded bit rates mean the video is less compressed, they can be useful if you want to be able to store or transfer video at a high quality. High bit rate is hard on processors, so it's a decision that must be made carefully. Again, depending on your use case you might want to maximize the peak coded bit rate, or focus your resources on other areas that are more important for your codec. 
* **Video quality** - Video quality is tied to bit rate, so depending on use case, strategies will vary. Quality and computational complexity have a complicated relationship, but within any codec there are many places where you can trade off quality for decreased computational complexity. 

## Principle 2: Minimize latency whenever possible
No matter the codec, the less the delay, the better. Delays are less important for some codecs, but to be avoided at all costs for other codecs - like those used for video conferencing. 

## Principle 3: Minimize compiled code and storage requirements
Not all systems using software video codecs will have a lot of available memory, so codec designs take this into account. Whenever possible, they cut down on the amount of code that needs to be compiled and the amount of storage needed for the video codec to function.

## Principle 4: Make your platform code robust and portable
The more places a codec can easily be used, the greater the chance it will be adopted everywhere. Easy-to-use means the code's been well tested so the codec isn't buggy, and the codec is easy to use on many different platforms. 

## Principle 5: Provide a flexible API
Make your codec developer friendly by providing an easy-to-use, well documented API. 

## Resources 

1. Chapter 13 of Video Codec Design - http://lad.dsc.ufcg.edu.br/mpeg/VCD/Ch13.pdf 
2. Codec - https://en.wikipedia.org/wiki/Codec 
3. A Comprehensive Guide to Video Codec Design - https://www.semiconductorstore.com/blog/2014/A-Comprehensive-Guide-to-Video-Codec-Design/866/
4. Thumbnail by Photo by [Diego PH](https://unsplash.com/@jdiegoph?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on Unsplash
  

The five design principles applied to software video codec

medium_diego-ph-fiq0tet6llw-unsplash.jpg

small_diego-ph-fiq0tet6llw-unsplash.jpg

thumbnail_diego-ph-fiq0tet6llw-unsplash.jpg

diego-ph-fiq0tet6llw-unsplash.jpg

The five design principles applied to every software video codec

Pretty much all digital display devices today are based on some type of RGB (Red, Green, Blue) color model. RGB is the most efficient way to represent most colors that humans can see (there are a few colors it doesn't produce very well, but that's another discussion). Cameras, cellphones, televisions, computer screens and digital video all display color using RGB color models and RGB color spaces. You may be surprised to hear this, after all, as a kid didn't you get taught the RYB (Red, Yellow, Blue) color model? 

While you probably got taught the RYB color model, RYB isn't used for a couple of reasons. First, it's not as effective at creating all the colors humans see. You can get more colors out of RGB. This is actually why we don't use other random color models. You could, for example, have a color model that's purple, blue and pink. You won't be able to make very many colors with it compared to RYB or RGB, but you can still do this if you wanted to. The second reason we don't use RYB is because the human eye is more sensitive to green. You are most sensitive to green, then to red, and a tiny bit to blue. 

Because these are the colors the human eye notices most, the RGB color model is used most often to create different color spaces. A color space is a combination of a color model and a way of interpreting or mapping those colors to what you'll see. There are many RGB-based color spaces. one common way of interpreting the RGB color model is to include an extra channel - this style of color space is called RGBa. It's one channel for each color - red, blue and green, then an extra channel that describes the transparency of each pixel in each channel. 

RGB based color spaces that stick with an RGB representation create very high quality representations of the images and video. They're often used to generate computer graphics and capture video. There's just one little problem - RGB is not good for image processing or storage. As I mentioned a bit earlier, the human eye sees green best. This is partially because the human eye is most sensitive to luma, or how bright something is. Green is mixed in with brightness in human vision. Unfortunately, with RGB, brightness is mixed in to all the color channels, and there isn't an easy way to separate it out. So most of the time, you'll find an RGB colorspace applied to still images or very high end computer animation. For video, another, more compressable solution is needed. 

## Y'CbCr and YUV
Sometimes Y'CbCr and YUV are used interchangeably, which isn't accurate. YUV is for analog TV, and Y'CbCr is for digital video, but they're both using the same concept to achieve color compression. For this color space, based on the RGB color model, the way the human eye perceives chroma (color) and luma (brightness) is capitalized on. Because you see luma better than color, the channels for color information are split out like this: 
* Cb - blue minus luminance
* Cr - red minus luminance
* Y' - luminance 

If you asked where the green is, you're not alone, that's a popular question. Because you see green as part of how you perceive luma, it's included in the Y' channel. Luminance is also accounted for in the other channels where you're subtracting it out. The benefit of this way of interpreting the RGB color model, is you can gather what your eye sees best into the luminance channel, leaving room to compress color in the other two channels. This technique can be used to make it easier to transmit video and to store video. The color compression is called chroma subsampling.

## Chroma Subsampling in Y'CbCr and YUV
Because you can separate out luminance, in the remaining channels for red and blue you can use chroma subsampling to to compress the data. Chroma subsampling uses the notation x:y:z. x tells you how many luma pixels you're going to take in a sample. So if you had 2:y:z, you're sampling two pixels at a time. 4 is most popular, so you're more likely to see 4:y:z. 

y tells you how many chroma pixels to sample out of the first line of the data, and then every alternating line thereafter. So say you have 4:4:z. This would mean you're taking a luma sample of 4 pixels, so you go to the frame of your video (or your image) and in the first line, you're also taking a chroma sample of 4 pixels. So you didn't throw any data away. 

z tells you how many chroma pixels to sample out of the line of pixels after y. So let's say it's 4 again. You would sample 4 chroma pixels. The complete notation would be 4:4:4. 

Now for the rest of the image, you would sample in blocks of 4 luma by 2 rows, with y being the first row, and z being the second, until everything was subsampled. 4:4:4 is rare, because its saving all the data, this is most often used for internal processing and otherwise not much. Sometimes this is called YUV-9 sampling, but that's when it's used with analog video

4:2:2 you're chucking half the color data per line. This might sound like a lot, but this is the subsampling method used for professional video authoring.  It's also frequently used to capture video. 

4:2:0 is used in many delivery codecs for broadcast, DVD/Blu-ray, PAL DV, HDV, AVCHD and the web. You may have seen it as YV12.

4:1:1 is used by NTSC DV25 formats like DV. This is a great format if you are capturing using interlaced video. 

## Resources 

1. Why is Web Color RGB and Not RYB? - https://medium.com/@austincoding/why-is-web-color-rgb-and-not-ryb-47f4aea0f557

2. Color space - https://en.wikipedia.org/wiki/Color_space

3. RGBa Pixel Transparency - http://www.georeference.org/doc/rgba_pixel_transparency.htm#:~:text=RGBa%20images%20can%20have%20a,also%20known%20as%20alpha%20transparency.&text=They%20are%20simply%20transparent%20because,been%20set%20to%20high%20amounts.

4. Thumbnail image by - Photo by [Michael Maasen](https://unsplash.com/@mgmaasen?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/s/photos/rgb?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)
  

Why is RGB the predominant color model for videos ?

small_michael-maasen-akygyymfqo-unsplash.jpg

thumbnail_michael-maasen-akygyymfqo-unsplash.jpg

michael-maasen-akygyymfqo-unsplash.jpg

Why is RGB the predominant color model for computers, TV and video?

codec

video create

small_cedric.jpeg

thumbnail_cedric.jpeg

cedric.jpeg

Cédric Montet

Product updates

At api.video, our goal is for video to be easy and affordable for anyone to build, be it individuals or organizations. We understand that achieving this goal means reducing costs at some stage of the video streaming process. That’s why, we now offer adaptive bitrate encoding for your videos at zero cost. 

&nbsp;

Yes, you heard me right. 

&nbsp;

**Encoding videos will be absolutely free for all video qualities (1080p or 4K) at [api.video](https://api.video/) - no hidden calculations, footer notes or tricks.**

(Check out this [video](https://www.youtube.com/watch?v=ouosgLbW7Ts) for more details) 


## A step towards making video streaming affordable

Video encoding is crucial as it makes it possible for videos to be shared and played smoothly on the internet. It is why we are easily able to consume various types of video media, whether it’s a YouTube video, our favorite shows on Netflix, or even catching up with friends across the globe on Zoom. Video encoding basically makes the world go around in regards to anything related to video and video content.

&nbsp;

We at api.video recognize that. From [increasing the transcoding and playback speed ](https://api.video/blog/product-updates/2021-in-review/)to building [specialized hardware like ASICs](https://api.video/blog/video-trends/the-comeback-of-specialized-hardware/) cards to increase our transcoding capacities, we've been working to make video streaming easier for you every day. And after dedicating ample time to engaging with our customers and learning about their concerns, free encoding seemed to be the next step in this direction. 

&nbsp;

So, we worked tirelessly behind the scenes to give you what no one else will - free encoding for all video qualities, be it 360p or 4K. 

## How does free video encoding benefit you as a customer

The benefit is straightforward: you save a LOT of money!

&nbsp;

- With free video encoding, you will have one less thing to worry about. As you know, you pay for three things for streaming videos — encoding, hosting, and delivery – out of which encoding is the most expensive, especially if you have a lot of videos. Free video encoding will allow you to focus your attention and resources to what is most important.

&nbsp;

- As the video needs of your app users grow, you might need to scale up your encoding capacity. Free video encoding services will give you the power to scale without any additional costs, meaning your users can handle more videos without increasing your expenses.

&nbsp;

- You can [migrate from your previous provider to api.video](https://api.video/compare/azure-media-services/) without paying an ‘entry tax’ to us. With free video encoding, you no longer need to spend money on encoding videos with a new provider when you have already paid for encoding with an older one.

&nbsp;

Now with all the saved cost, you can allocate your resources and money to a place where it is actually important – building your product and delighting your users. 

## How to get started

To use the free video encoding benefit, getting started is easy. 

&nbsp;

- If you’re a new user, [sign-up](https://dashboard.api.video/register) for a free sandbox account and start uploading your videos
- If you are looking to migrate from another vendor, [import](https://import.api.video/) your videos from your current vendor
- If you have any questions, [talk to us](https://meetings.hubspot.com/diego398/cc-talk-to-us?__hstc=54467703.754940701eda85e3c99a902d8ab0b988.1687266643206.1694417789132.1694427271479.149&__hssc=54467703.3.1694427271479&__hsfp=3654297906) by booking a slot in our calendar

&nbsp;

To know more about our[ features](https://api.video/features/), [pricing](https://api.video/pricing/) or ask any other questions, [talk to our team ](https://meetings.hubspot.com/diego398/cc-talk-to-us?__hstc=54467703.754940701eda85e3c99a902d8ab0b988.1687266643206.1694162373815.1694167498585.144&__hssc=54467703.1.1694167498585&__hsfp=3654297906)and we will help you out. 

&nbsp;

Or, you can straightaway [sign-up](https://dashboard.api.video/register) for a free sandbox account and check our product out for yourself. 


Huge pricing update: Introducing video encoding at no cost

large_Blog_Zero Cost (2).png

medium_Blog_Zero Cost (2).png

small_Blog_Zero Cost (2).png

thumbnail_Blog_Zero Cost (2).png

Blog_Zero Cost (2).png

api.video team

Modern technology has been helpful for an increasingly fast and simple method of consuming various types of video media, whether it’s a YouTube video, our favorite shows on Netflix, or even catching up with friends across the globe on Zoom. Despite the relative ease of doing anything related to video, a few complex processes take place behind the scenes, away from public view. One such process is video encoding.

Video encoding basically makes the world go around in regards to anything related to video and video content. Thanks to the technologies behind the video encoding process, it has become much easier for people and companies to share video content on the internet. But what exactly is video encoding, and what happens during the process? We’ll take a closer look at this and much more in this handy guide to video encoding.

## What is video encoding?

In a nutshell, video encoding occurs when you take raw video footage, either from a digital camera or an analog film camera, and digitally compress it to make it a more manageable file size. Video encoding happens daily in both professional and personal capacities, and many people don’t realize it.

But, without encoding, services like Netflix and YouTube would not exist – or, at least, not in the same way that it does today. Without it, we couldn’t send videos of our beloved pets to our friends and family as easily as we can today.

## Why is video encoding used?

There are a variety of reasons why video encoding is used, but all factors point to the single function of video encoding – **compression**.

This is done to ensure optimal performance and compatibility with a selected suite of software and hardware, be it for DVD/Blu-ray playback, mobile viewing, online video streaming, or general video editing. A file's video and audio information undergo transformation and compression during the encoding process, all in accordance with the requirements of the selected encoding standard.

On a professional level, the ability to compress raw video into more manageable sizes makes it easier to transmit video content such as training, promotion, presentation, and so on over the internet. 

On a personal level, video encoding makes it much easier to send videos taken from your phone directly to friends and family. Most modern-day personal devices have built-in video encoding capabilities to allow for this to happen without a second thought.

To put things in perspective, uncompressed or unencoded video can have a data rate that’s as low as 3,000 Mbps and as high as 12,000 Mbps, depending on the length and quality of the video recorded. 

**Without encoding, it would take up to days for anyone to not just transmit video, but to be able to consume it, too.**

## Differences between video encoding and video transcoding

You need to be familiar with two more [essential terms related to video encoding](https://api.video/what-is/) – transcoding and codecs.

Transcoding relates to the conversion from one digital encoding format to another. It takes the already encoded video file, decompresses it, and then alters it before re-encoding or compressing it into a different codec. Codecs are what allow encoding to occur in the first place. They are software or hardware that gives us the ability to compress video to make them more easily stored and delivered tightly.

While you may hear “encoding” and “transcoding” used interchangeably, the two terms actually mean different things. Putting it succinctly, encoding is the initial process of compressing a video from a source – such as a camera – into a codec, while transcoding takes an already encoded video file to convert it into a different codec.

## Examples of video codecs

In order to understand video encoding, you must first be familiar with two interrelated terms: [video codecs and video formats](https://api.video/blog/product-updates/every-video-format-codec-and-container-explained/). These also tend to be used interchangeably, but much like the terms encoding and transcoding, they refer to two different things.

Video formats are the type of files in which the compressed video/audio is stored. Also known as a “container”, you can tell what a video format is by looking at the file’s extension — the three to four characters that sit after a period in a file name. Here are some of the most common video formats used today:

* MP4
* FLV
* MOV
* MKV
* LXF
* MXF
* AVI
* WebM
* QT

On the other hand, **the codec is short for “Coder / Decoder” (compression / decompression)** and relates to the video compression algorithms that allow encoding to happen. Over the years, codecs have gone through multiple iterations as it coincides with the increasing sophistication and rise in the quality of filmmaking. 

If you’re a content creator, you might be aware of some of the video codecs that exist today:

- **H.264 – also known as AVC, this is the most widely-used video codec available on every platform and mobile device.**
- H.265 – also known as HEVC, stands for High-Efficiency Video Coding, and it’s slowly overtaking H.264 as the standard. This codec can compress video much more efficiently than its predecessor.
- **VP9 – is an open-source, royalty-free codec developed by Google and is considered the direct competitor of H.265. The video compression codec VP9 was created to reduce file sizes, making streaming videos simpler.**
- **AV1 – is another royalty-free, open-source codec released in 2019 by the non-profit Alliance for Open Media. It was developed to become the successor to Google’s VP9 and contend with HEVC more directly.**

## How do different technologies handle video encoding

As you may have suspected, there are a majority of ways that different technologies handle the video encoding process. Let’s take a look at the most common technologies that use video and how video encoding works on each of them:

### Video editing software

When it comes to video content creation, a majority of people will use one of two video editing software — Adobe Premiere Pro or Apple’s Final Cut Pro. Both of these video editing software handle video encoding in very similar ways: when exporting the finished video.

Depending on which device the video content is recorded on, it will already be encoded and formatted according to the settings of the device. For those using iPhones recording 1080p video at 25 frames per second (FPS), for example, the iPhone will automatically encode the video using H.264 and contain it in a .mov file format. The video editing software will “open up” the container to get to the encoded, compressed video file within so the editing process can begin.

After editing the footage is complete, it will need to be exported, which typically concerns video encoding. The edited raw media taken from a recording device such as a camera will be encoded via the codec of the person’s choosing before being put into a .mp4 container once they’ve hit “export”. As we’ve mentioned before, the most common codec used in video content creation is H.264 for HD videos, with 4K videos using H.265. 

### Websites

Different websites utilize different techniques to handle video encoding – with a majority of the major sites like YouTube, Netflix, and so on using their own technologies to handle the process. A majority of social media websites allow you to upload video files directly onto the site to share, although most of them require the files to be saved in specific containers. 

These websites will take your video file and transcode it before using their own systems and serves to re-encode it to match multiple display resolutions. This process will happen regardless of your video file's original codec and settings. Different websites have different processes, but companies that don’t have the capacity to build their own video encoding systems can use a third-party application programming interface (or API) to handle this process. These APIs act as [dedicated video encoder](https://api.video/encoding/) that allows companies to add on-demand videos to their websites more efficiently.

Using [api.video](http://api.video) as an example, all of the videos submitted to the platform will be transformed into H.264 codec in an MP4 container. This may be done using transmuxing and/or transcoding.

We expect the video bitrate to be lower than 16 Mbps for up to 1080p resolution and 50 Mbps for up to 4K resolution. We accept any bitrates, however, we will scale them down to the limits mentioned because anything above is usually challenging for most viewers’ connection to watch.

### API

The ability to generate, store, and distribute video is made possible via a video API. [api.video](http://api.video)’s solution can be tailored to suit any specific need. 

The backend process of video on demand (VOD) is exceptionally complex, even if you’re a software programmer specializing in video. It is necessary to encode, package, and typically transcode the data before sending it to any of several possible destinations. All of these things, and more, may be done with minimal fuss using api.video.

Our API basically keeps the complexities of streaming media hidden from view, so you can rapidly develop apps with as much control as you need. This saves time and effort so developers can concentrate on making the product.

## Conclusion

As you can see through this video encoding guide, a lot of work goes into making the video you watch and the video you share possible. Without video encoding compressing raw video footage, it could take days to upload and transmit video content over the internet and perhaps even longer to be able to download and enjoy them. 

The best part about encoding at api.video is that is offered at ZERO cost. You will only be charged for delivery and hosting of your videos, and the encoding will be free. Now integrate your videos in minutes with api.video or [contact us](https://api.video/contact) for more information.

A Guide to Video Encoding & Video Codecs

large_poster-3.png

medium_poster-3.png

small_poster-3.png

thumbnail_poster-3.png

poster-3.png

Encoding and transcoding are often used interchangeably but they have slightly different definitions. **Encoding is the initial process of converting raw video files into a specific digital format or codec.** It involves compressing and packaging the video data, often to reduce file size while maintaining acceptable quality. Encoding prepares the video for storage, transmission, or playback.

&nbsp;

**Transcoding, on the other hand, involves converting a video from one digital format or codec to another.** This is typically done to optimize the video for different devices or platforms, adapt to varying network conditions, or ensure compatibility with specific playback requirements. Transcoding occurs after encoding and often involves changing resolution, bitrates, or formats without altering the content itself.

## What goes on behind encoding a video?

It's easy to say that encoding is when you go from an _uncompressed source_ to a _compressed source_, but you may ask, what exactly do compressed and uncompressed sources mean?

&nbsp;

Uncompressed sources are video files that contain data that comes directly from a camera sensor vs. encoded video. In the digital space, it's SDI (Serial Digital Interface) or HDMI (High Definition Multimedia Interface). Both of them are popular standards. You'll find HDMI in places like your GoPro camera, or a mini camcorder. These days, you might also find HDMI in professional settings, but SDI is more common due to its reliability. Another example of an uncompressed source would be something analog, for instance, VHS tapes.

&nbsp;

A compressed sources, on the other hand, are video files that have undergone compression during the encoding process. Compression involves reducing the file size by removing unnecessary information or encoding the data in a more efficient format. This process results in a smaller file size, making it more manageable for storage, transmission, and playback.


## What are the types of transcoding methods?

There are three types of transcoding, depending on what you are trying to do to get a video to suit a particular browser, player, or platform:

* **Lossless-to-lossless** - This type of transcoding maintains the quality of your video across formats but allows you to take advantage of better hardware or compression algorithms. 
* **Lossless-to-lossy** - This type of transcoding will decrease your video in quality, but it will benefit you in ways such as – you will get a smaller and/or faster playing file, or a file that is compatible with the requirements of a particular platform, player, or browser.
* **Lossy-to-lossless** - Sometimes people think this means they are regaining quality. This isn't the case in this type of transcoding. If you transcode from a lossy format to a lossless format, all that it means is that you will not lose additional quality in the conversion process. You will not regain data you already lost through compression.

## Codecs for encoding and transcoding

For both encoding and transcoding, you will need to make use of codecs. A codec's job is to compress and decompress video files. It's right there in the name -  **co**mpress **dec**ompress or encode decode. So basically, codec is an algorithm that takes the video and shrinks it or changes it into a format that makes it compatible with the requirements of wherever you want the video to play. There are many codecs because there are many ways to compress a video or livestream for playback.



## What is transmuxing?

Now that you know the difference between encoding and transcoding, let's throw one more term in there - transmuxing. Transmuxing is when you change the container for a video instead of the format. It's a way of repackaging a video to make it compatible somewhere without changing what's in the file. 

&nbsp;

This process is faster than encoding or transcoding and takes less computing power because you only change the container, not the file. It's great when you need to quickly make something available, but the downside is that it doesn't handle things like making multiple video resolutions available. If you choose a container type that's incompatible with your video file, you may also cause compatibility issues, resulting in not being able to play the video.

## In closing

* **Encoding** - Encoding happens when you compress a raw data file from a video camera sensor or an analog source like a VHS tape. 
* **Transcoding** - Transcoding happens when you take an already compressed video file and change it to another format.
* **Transmuxing** - Transmuxing happens when you change the container of a video file but leave the video file untouched. 

We hope that explains the difference between encoding and transcoding clearly. 

_If you are looking for an affordable video API solution that does not burn a hole in your pocket for transcoding and encoding videos, check out our offerings on our [homepage](https://api.video/) or [sign-up for a free account here](https://dashboard.api.video/register)!_

Encoding and transcoding: what's the difference? 

Encoding and Transcoding - What's the Difference?

large_encoding-and-transcoding-whats-the-difference.jpg

medium_encoding-and-transcoding-whats-the-difference.jpg

small_encoding-and-transcoding-whats-the-difference.jpg

thumbnail_encoding-and-transcoding-whats-the-difference.jpg

encoding-and-transcoding-whats-the-difference.jpg

Once I upload my video, how quickly will it be ready to be viewed?

Our reply to this question has always been “it's really fast.”  And it is really fast. But, the dev team gave me access to the backend database, and pointed me to a few tables that might be helpful (thanks to Thomas Persohn for your patient help!)

Thus armed, I waded into a sea of data to better understand how fast our videos are encoded at api.video

## Encoding

When a video is uploaded to api.video, it is re-encoded to a HLS stream, with several different video dimensions and bitrates.  This allows api.video to deliver properly sized video to all users - no matter the device or network connection. (Read our post on [Responsive video delivery](https://api.video/blog/video-trends/responsive-video-delivery/) to learn more.)

> Summary:  All videos are encoded at 240p, 360p, 480p, 720p, 1080p and 2160p. However, the videos are only encoded up to the initial size of the video (so if the original video is 720p, there will not be 1080p or 2160p versions available). 

**FYI:** For our customers, encoding is absolutely free – we offer adaptive bitrate encoding at zero cost. 

How long does this step take? If a customer uploads a video to our service - they want to know how efficient api.video creates the content for viewing.

### Data in the study

We are constantly making adjustments to our infrastructure to speed the encoding step of our video (and that is one reason this post is dated 2H 2020 - we expect to be faster in early 2021). The last major adjustment was made in mid October, so the data presented in this study from every video uploaded to the api.video service between 15 October and 2 December 2020.

### Parameters that affect encoding

There are 2 major parameters that affect the speed of encoding a new video stream: the output dimensions and the length (duration) of the video.  

#### Dimensions

The output video dimensions have a huge impact on video creation - a 3840 × 2160 (or 4K or 2160p) video has millions more pixels than a 320x240 (240p) video, so it takes a lot longer to create.  In general, the videos exit the encoding process from smallest to largest.

#### Duration

If a video is 30 frames per second, and is 60 seconds long, there are 1800 frames to be created.  If a 30 frame per second video is an hour long 3600s), there are 108,000 frames to be created, so the longer the video - the more time for creation.

## Analysis

To better understand the encoding time, I divided the data by the size of the output video size, and looked at 2 lengths of video  - those under 5 minutes, and videos that are 45-75 minutes long (we are often asked how long a 1 hour video takes to encode).

I calculated encoding time for each bucket of video using percentiles.  

> Quick review - percentiles break the data into 100 evenly sized segments, so the 25th percentile is when 25% of the videos have been encoded. The median is the 50th percentile, and is when 50% of videos have been encoded.  We report here to the 95th percentile for video encoding.

### Short videos - under 5 minutes long

Many of the videos uploaded to api.video are short - under 5 minutes long.  These videos are generally ready to play in a few seconds:



The above chart has a lot of data, the 25, 50, 75, 90 and 95th percentile encoding times for all 6 variants of the video that api.video creates.

The median video is ready to play (at 240p) in 5.6 seconds, but a 720p version will be ready in 15s after encoding begins, and the 4K version will be ready in 37 seconds.

95% of all videos (under 5 minutes) will be playable in 32.6s, and will have 1080p versions ready in just over 2 minutes (140s).

### Long videos - 45 - 75 minutes long

Our most frequently asked question on encoding is "how long for a 1 hour video to be encoded and ready?"  Here are the numbers:


Observant viewers will see there are no 4K videos in this report. At this time, there are very few 4K streams that reach the 1 hour timeframe, and the data was quite noisy.  Hopefully, more of our 'long uploads' will add 4K support, so we can add this metric in our next update.

The median hour long video (red line) will be ready to play (at 240p) in just over 9 minutes (555s), and a 720p version will be ready about 18 minutes after encoding begins (1099s).

95% of hour long videos will have a 240p version ready in 33 minutes, and 720p in 54 (3230s).  the 1080p version will be available for 95% of videos in 108 minutes,

## Conclusion

Short videos will be ready to play - even at high resolution - in under a minute: the median 4K encoding for a short video is 37s. 

Hour long videos take longer to encode, and the median 720p video will be ready in 18 minutes.  95% of 720p videos will be ready to play in 54 minutes.

This is really fast! You can have responsive streaming video ready for your customers in just a few minutes with a simple upload!

That said, our team is working hard in the background to improve our infrastructure to make our encoding speeds even faster! 

We're really excited to have hard numbers to share with you on our encoding speeds, and we cannot wait to re-benchmark this data in 2021 to see how much faster your videos will be encoded as we improve our service.

Do you have questions about api.video or video encoding? Reach out on our [community page](https://community.api.video).


How fast does api.video encode video? 2H 2020 data

medium_gif.jpg

small_gif.jpg

thumbnail_gif.jpg

gif.jpg

Huge pricing update! Introducing video encoding at zero cost

Featured article

# api.video blog

Blog

Follow the api.video Blog to explore tutorials, video use cases, and learn about on-demand and live-streaming video updates.

Video on demand

Live streaming

Analytics

See all features

Online learning

Short-form videos

Marketplace & E-commerce

Video messaging and recording

Quick start guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Help center

Blog

Huge pricing update! Introducing video encoding at zero cost