Normal

thumbnail_anne-sophie.jpg

anne-sophie.jpg

twitter-brands.svg

Twitter (Anne-Sophie)

Anne-Sophie Poupard

large_api-video-blog-no-code.png

medium_api-video-blog-no-code.png

small_api-video-blog-no-code.png

thumbnail_api-video-blog-no-code.png

api-video-blog-no-code.png

No-code

With video files becoming larger and larger, WeTransfer a simple way to transfer them via the internet. 

The service evolved and now it requires extra steps to get started (accept terms, select a video, fill the form, create an account, validate the account, and finally send your video).

We've developed a dead simple tool to upload your videos directly into your local folder! As easy as drag and drop!

small_apivideouploader.png

thumbnail_apivideouploader.png

apivideouploader.png

## How does it work?

1. [Download the app](https://github.com/apivideo/api.video-desktop-synchronizer/releases/latest/) (from GitHub). 
2. When you launch the app for the first time, the Settings window appears. Here, you'll have to enter your api.video API key, and the folder you want to watch:

small_apivideo-local-uploader.png

thumbnail_apivideo-local-uploader.png

apivideo-local-uploader.png

3. Drag and drop videos to the folder
4. And that's it! You'll get sharing links to your transcoded videos

The tool is available for Windows and MacOS clients!


## How is it built?

The application has been developed using the TypeScript language and the Electron framework.
Electron makes the creation of cross-operating system applications possible with only a few lines of code.
All the interaction of the app with the api.video API are made through our [nodejs api client](https://github.com/apivideo/nodejs-api-client). The client handles all the parts that could have been boring to deal with by hand, like the authentication or the upload of big files by chunks.


You can use it to automatically to send your Zoom recordings into your local file too!
Feel free to test it and give us your feedback on [our community forum](https://community.api.video/)!

A handy desktop app for uploading and sharing your videos

A handy desktop app for syncing local folders with api.video

large_sync-app.jpg

medium_sync-app.jpg

small_sync-app.jpg

thumbnail_sync-app.jpg

sync-app.jpg

A handy desktop app for uploading and sharing your videos in a few clicks

small_cedric.webp

thumbnail_cedric.webp

cedric.webp

Cédric Montet

large_api-video-blog-product-updates.png

medium_api-video-blog-product-updates.png

small_api-video-blog-product-updates.png

thumbnail_api-video-blog-product-updates.png

api-video-blog-product-updates.png

Product updates

As the year comes to an end, we wanted to take a moment to look back at everything the team released to help you build your video products (and to share a surprise)!

Let's start with our core principle: *make videos available worldwide in seconds*.
For years, it has been highly complicated to add video features to apps and platforms because of the various services you had to aggregate (ingest, transcode, host, deliver, analyze). 
Our goal is to bring you a dead-simple API to implement the entire video workflow in minutes and provide an exceptional experience to your end-users, whether they are creators or your audience. 
In 2021 we've welcomed users from **122 countries** (astonishing!) and have ingested **110%** more video minutes than in the previous year. 

## Providing the greatest video performances for your app

* All videos you send to us have never been transcoded faster. Our recent [benchmark](https://api.video/platforms-benchmark/) reveals that a 5-minute video is available for playback in 6 seconds with api.video. In contrast, other services take up to 60 seconds.

large_benchmark.png

medium_benchmark.png

small_benchmark.png

thumbnail_benchmark.png

benchmark.png

* Once the video is uploaded, your audiences expect an exceptional playback with no buffer or latency. We have been delighted to discover the [PlayerX benchmark](https://dev.playerx.io/compare/). Considering all B2B and B2C video platforms, api.video provides the greatest viewer experience on the Market!

**The best part of that? This is nothing compared what we'll bring you in 2022!**

## New features 

We've introduced a lot of new features, most of them inspired by your feedback 🙏
* After releasing live streaming in 2020, we added [private live streaming](https://api.video/blog/product-updates/new-beta-feature-private-livestreams/) to help your users broadcast their video feed securely into your app.
* [Progressive upload](https://api.video/blog/tutorials/progressive-uploads-with-the-api-video-python-client/) was released earlier this month to let you build recording features that your users can launch right from their browser or record a video directly from a video stream.
* Thanks to the [webhooks](https://api.video/blog/endpoints/webhooks/), you are now notified when on-demand and live videos are ready to be played.
* Clipping became available a few days ago. This new feature allows you to edit your videos right from the service without downloading them first.

## Reduce your workload and make api.video available in your CMS
To help you launch your product fast, we've added many new ways to implement [api.video](https://api.video) in your stack:

* 8 (yes...8!) [new clients](https://docs.api.video/) were published on our GitHub account: PHP, Python, Go, Java, NodeJs, C#, Android, and iOS
* If you are using a CMS to build your portal, you can now install our official [plugins for WordPress](https://github.com/apivideo/api.video-wordpress-plugin), [Contentful](https://docs.api.video/sdks/nocode/contentful), [Bubble.io](https://docs.api.video/sdks/nocode/bubbleio-plugin), and Moodle to allow your users to upload and share videos right from their backend.
* To enable customization of the video experience, we've also released standalone plugins for [analytics](https://docs.api.video/sdks/player/apivideo-videojs-analytics), the video uploader modules for [Flutter](https://docs.api.video/sdks/vod/apivideo-flutter-uploader), [Android](https://docs.api.video/sdks/vod/apivideo-android-uploader) & [iOS](https://docs.api.video/sdks/vod/apivideo-swift-uploader), and the live streaming modules for [ReactNative](https://docs.api.video/sdks/livestream/react-native-livestream-component), [Android](https://docs.api.video/sdks/livestream/apivideo-android-livestream-module), Flutter, and [iOS](https://docs.api.video/sdks/livestream/swift-livestream-library).

## New branding
You may have noticed our [new logo](https://api.video/blog/product-updates/new-logo-same-mission-connecting-people-through-unique-video-experences/) last summer, but there is more. 
Check out our [fully redesigned dashboard](https://dashboard.api.video/overview) that lets you easily switch between your environments, create, view, and edit videos. Plus, you can share the on-demand and live stream videos you're managing and access your KPIs and metrics! The new dashboard also lets you switch from one plan to another at any time as your needs evolve.

We hope you enjoyed reading through some of the major releases we've had in 2021 and that you've discovered some new features that you'll start using today. We are building [api.video](https://api.video) for you and with you: your ideas and feedback are essential to us. Let us know which new features you'd love to see us add in 2022 either in chat or by posting in our [community forum](https://community.api.video). 

On behalf of the whole api.video team we would like to thank our amazing community for all the feedback and support over this past year. We wish you all a wonderful festive period and a very happy New Year! ✨

large_apivideoteam.png

medium_apivideoteam.png

small_apivideoteam.png

thumbnail_apivideoteam.png

apivideoteam.png

## One more thing!

We couldn't end this review without a gift 🙂

You might have videos on your computer that you want to upload during your implementation, or you might need to iterate and quickly share different versions of your content without having to open & close WeTransfer again and again.

Do you remember the first time you used Dropbox and how magical it was to sync a local folder with them?

We're thrilled to announce the Windows and macOS clients!

- [Download the app](https://github.com/apivideo/api.video-desktop-synchronizer/releases/tag/v0.1.2), pick a local folder to sync, and add your API Key.
- Drag and drop videos to the folder.
- Voilà! The sharing links to your transcoded videos are ready to be use

2021 in review: video transcoding, playback and new features

2021-in-review-4.gif

2021 in review: lightning-fast video transcoding & playback and many new product features!

Video upload

medium_screen-shot-2021-01-26-at-11-48-47-am.jpg

small_screen-shot-2021-01-26-at-11-48-47-am.jpg

thumbnail_screen-shot-2021-01-26-at-11-48-47-am.jpg

screen-shot-2021-01-26-at-11-48-47-am.jpg

Erikka Innes

large_api-video-blog-tutorials.png

medium_api-video-blog-tutorials.png

small_api-video-blog-tutorials.png

thumbnail_api-video-blog-tutorials.png

api-video-blog-tutorials.png

Tutorials

Python

Today we'll walk through how to use the Python client to do a progressive upload for a video. Progressive uploads are great for large videos. We recommend choosing this type of upload if your video is 200MB or greater.

## Prerequisites

* Sign up for an api.video account - [https://dashboard.api.video/register](https://dashboard.api.video/register) 
* Set up a virtual environment for your project (optional)
* Install the api.video Python client - `pip install api.video`
* Get a video file that's 200MB or greater (you can use a smaller one to see how it works, but it should be at least around 50MB) 

## Code sample 
Here's our code sample for today: 

In this sample we do the following: 

1. Set up the API client.
2. Create a container for our video with a payload containing our video metadata. 
3. Retrieve the video ID from the new container we made.
4. Create a session to handle our progressive video upload using the video ID we retrieved.
5. Set the chunk size we want to use. This value will be used to create chunks of this size. It's set to a smaller number so you can demo with a smaller video. If you want to upload something bigger, you might want to use a bigger number for each chunk.
6. Set up a method that handles breaking the video into chunks of data.
7. Set up a progressive video uploader. Of note - you have to take each chunk of data and save it as a clearly numbered part. For example chunk_file.part.1, chunk_file.part.2. End the files with part.* so that the api.video server can figure out the chronological order for each chunk of video. Also of note - you must signal that you're uploading the last chunk of video with the session.uploadLastPart() method. Otherwise, the api.video servers cannot determine if you're still uploading or not, so your upload will fail. You also have to open your file after saving it, using 'rb' or read binary. 

## Conclusion
If you want to try this upload without the client, see the [article about how to do a progressive upload with just Python](https://api.video/blog/tutorials/progressively-upload-large-video-files-without-compromising-on-speed/).

Happy coding!

Progressive uploads with the api.video Python client

Video list

Today we'll learn how to grab video data from api.video, add it to a database, then use that information to create an infinite wall of videos that you can keep scrolling through.

large_infinitevideos.jpg

medium_infinitevideos.jpg

small_infinitevideos.jpg

thumbnail_infinitevideos.jpg

infinitevideos.jpg

For this project, we're going to also use Laravel 8. 

## Prerequisites

* api.video account - [Register here](https://dashboard.api.video/register)
* [Laravel 8](https://laravel.com/docs/8.x/installation) installed
* Project repository - [https://github.com/apivideo/api.video-php-client/tree/main/examples/laravel/infinite_scroll](https://github.com/apivideo/api.video-php-client/tree/main/examples/laravel/infinite_scroll)
* api.video [PHP client](https://github.com/apivideo/api.video-php-client) 
* Symfony HTTP client - `composer require symfony/http-client` 
* composer - [https://getcomposer.org/](https://getcomposer.org/)

## What you'll be doing
You have three tasks for this project: 
1. Retrieve your data from api.video
2. Add your data to your database
3. Set up your page to show the videos in an infinite scroll 

## Retrieve your data
You need to collect some data from api.video about the videos you're going to use. To keep it simple, for this walkthrough I'm just retrieving videos tagged 'holiday' and keeping the title, video ID, and MP4 link for each one. You can retain any information you might want to use or display, the most important thing is the MP4 link. 

For this part of the project, I didn't use Laravel 8, I just set up a php file with my code and used composer. 

After you run the code, you'll have a list of all videos from your account that you tagged **holiday** in a .csv file. Save the file, we'll use it for the next part of our project.

## Set up your project
Now it's time to set up our Laravel 8 project. We'll need this before we can do anything with our database. These instructions are for Macs.

1. Make a new project by opening a terminal in the place where you want to store your project by typing `laravel new holiday`. 

2. Install the api.video PHP client. `composer require api-video/php-api-client`

3. Install the Symfony HTTP client. `composer require symfony/http-client`

4. Go into your project and find the .env file. For the section that starts **DB_** delete every **DB_** item except for **DB_CONNECTION**. Set **DB_CONNECTION** to sqlite. `DB_CONNECTION=sqlite`

5. Create a database table for the .csv file we made. `php artisan make:migration create_holidays_table`

6. Go into your project in the database folder and open the migrations folder. There should be a .php file with a timestamp at the start followed by the name of the table you just created in the terminal. Open this file and paste in this code:

7. Migrate your new table. `php artisan:migrate` 

8. Now we're going to create two models. The first is for handling infinite scroll. `php artisan make:model Model/InfiniteScroll` 

9. After you've made this model, go find it in your project and paste in this code: 



10. The second model we're going to make is for setting up our Holidays table contents. `php artisan make:model Model/Holiday`

11. Navigate to your new Holiday model and paste in this code:



12. Let's create a seeder that we can use to populate our new table using our .csv file. `php artisan make:seeder HolidaySeeder` 

13. Take your .csv file and place in the data folder for your database. 

14. Go to the seeders folder and open your HolidaySeeder.php file. Paste in this code:

15. Now we can run our seeder! `php artisan db:seed --class=HolidaySeeder` This should populate your database with the information from your .csv file. We're ready to set up our page to display our video collection. 

## Display videos
In this section we'll set up routes, a controller, and the page that will display videos whose links are retrieved from our database. 

1. We're going to add a new route, but before we do that, navigate to RouteServiceProvider.php. Scroll down to the line that reads `protected $namespace = 'App\\Http\\Controllers';` - it's probably commented out. Remove the commenting so it is active again. 

2. Navigate to routes\web.php. Add a new route `
Route::get('/load-more-data', 'VideoController@InfiniteScroll' );`

3. Make a new controller. `php artisan make:controller VideoController` 

4. Navigate to Controllers and open your VideoController file. Paste in this code:

5. Now let's set up our blade file. Navigate to views and craete a new blade file called **holidayView.blade.php**. Open the new file and paste this in: 

6. If you don't have a **data.blade.php** file, create one in the same folder as your other blade files. Open **data.blade.php** and paste in this code: 

## Run your project

Open a terminal in your project and type `php artisan serve`. You should be able to see a list of scrolling videos! 

If you have any questions about this tutorial, or would like to share what you've built, please join our [community forum](https://community.api.video/). 

Create an infinite scrolling wall of videos with Laravel 8

Create an infinite scrolling wall of holiday videos with Laravel 8

large_create-an-infinite-scrolling-wall-of-holiday-videos-with-laravel-8.jpg

medium_create-an-infinite-scrolling-wall-of-holiday-videos-with-laravel-8.jpg

small_create-an-infinite-scrolling-wall-of-holiday-videos-with-laravel-8.jpg

thumbnail_create-an-infinite-scrolling-wall-of-holiday-videos-with-laravel-8.jpg

create-an-infinite-scrolling-wall-of-holiday-videos-with-laravel-8.jpg

[](https://)If you work with video, you know that when you have a gigantic video file, you must break it into chunks to send it to the server. We at api.video already offer a way for you to break up your video into chunks and send it to us, but today we are introducing a new way of uploading large files: [Progressive Upload](https://docs.api.video/vod/progressive-upload).

Instead of spending time worrying about getting the bytes right for the header, or relying on our API clients to handle it for you (still a great choice if you have a client in use) you can simplify the header for a chunk to be about what part of your video you're sending. Here's an example of what I mean:

Old header: 
`Content-Range: bytes 0-1000000/200000000`

Here you can see the Content-Range header shows the bytes being sent from the video and then the total number of bytes. But there's a couple of issues with this. First, it's tricky setting up the header correctly (which makes the clients great to use since they handle this for you) and second, what if you want to upload a video where you don't know its size? It'd also be great if you could upload video chunks concurrently and have that handled for you. Progressive Upload does this, and the new header looks like this: 

New header:
`Content-Range: part 1/*`

Every video chunk just receives a part number. The only video chunk that can be smaller than 5MB is the last chunk. Everything else must be larger. And with this explained, we're ready to prepare and try a code sample. 

## Prerequisites
For this project, you'll need:

* (Optional) Familiarity with other ways to upload (it might make this tutorial easier, but this is written so you don't need the other tutorials)
* An api.video account, you can sign up [here](https://dashboard.api.video/register)
* Python or a Python virtual environment to work with (recommended)
* Requests library for Python installed 

## Progressive Upload code sample 
The code sample we're going to walk through looks like this:

In the code sample, you're doing the following things: 

1. Retrieving a token using your API key. 

2. Setting up a video container to upload to. You upload in two parts - first you create a container with metadata about your video. Then you retrieve the videoID for your video container, and reference it so you can upload your chunks to the right container. You can only upload once to a container, so make sure you have everything set up correctly. Otherwise you'll have to start over with a new container. 

3. We set a variable **CHUNK_SIZE** to represent the size of each chunk of video we'll send. You can make this bigger if you like to speed up the process. 

4. We write a little function **read_in_chunks** that breaks a chunk of data off our video file to send, then remembers the spot it was in so when we come back to the function again, it gives us the next chunk of data. This is accomplished by using **yield** in the function.

5. In the **upload** function, we get ready to loop through our video file, sending a chunk of data each time with the appropriately labeled header. For Content-Range we just increase the part number by 1 each time, and leave it set as unknown, like: `part 1/*` If you know your video size ahead of time, you can change the asterisk to the final part number. This code sample assumes you don't know the size. What you do per chunk is check if it's smaller than 5 MB, since this will signify we've reached the final piece of the video. For this last piece of video, we change the total number of parts to match the part we're on. This signals to api.video that we aren't going to send any more chunks and that the upload is complete. 

And we're done!
To start building right now, check our [docs](https://docs.api.video/reference/api/Videos#upload-a-video) and [sign up for a free account](https://dashboard.api.video/register).

Upload large video files without compromising on speed

large_progressive-upload.png

medium_progressive-upload.png

small_progressive-upload.png

thumbnail_progressive-upload.png

progressive-upload.png

Progressively upload large video files without compromising on speed

live - create

doug.jpg

Doug Twitter

Doug Sillars

large_api-video-blog-video-trends.png

medium_api-video-blog-video-trends.png

small_api-video-blog-video-trends.png

thumbnail_api-video-blog-video-trends.png

api-video-blog-video-trends.png

Video trends

Frequently asked question: How many live streams can I use simultaneously with api.video? 
Answer: How many cameras do you have?

That sounds like a snarky response (and perhaps it is *a little bit* snarky). Some platforms - like YouTube Live or Twitch - allow just one live stream per account at a time. But, with api.video there is no such limitation - as long as you have cameras (and available bandwidth), you can stream to your heart’s content from every single one of them.

## Why would I stream more than once?

There’s no reason to limit your viewers to just one view - stream from every angle (or stream from every angle that you have a camera for!).  Or if your client needs video from a number of locations at once - api.video can support you.  Here are some example use cases:


Are you running a multi track conference?  Stream each room from the same api.video account. Add a hallway cam to show the social interactions.  
Do you support more than one client?  Don’t drive yourself crazy scheduling to fill the one slot - let everyone have their own live stream(s).
Sporting event - multiple angles.  Does the coach want a stream from the opposite side?  From behind the goal/end-zone? From the top of the stadium?

## Create your live streams


The first step in having multiple live streams is to create your live streams.  You can create as many streams as you’d like. 

Creating [live streams](https://api.video/blog/tutorials/live-stream-tutorial/) is easy with our [API](https://docs.api.video/reference/api/Live-Streams#create-live-stream).  

Once you’ve created your live stream you (or your clients/users) can re-use each livestream over and over (like a Space-X rocket). 

* Stop the stream after each conference talk, and restart before the next. If you have ```record:true``` set for the live stream, you’ll get VOD versions of each video saved into your account.  Each recording will have the livestream name, and the time (in GMT). 
* Assign live stream(s) to your clients, and let them use them to their heart’s content. No more work is needed on your part.
* Name each livestream for easy recovery of the videos:  “50 yard line”  “Client X” “Conference Fizz 2021: Buzz track”




## A wildly absurd demo 


As an example of using live concurrent live streams, I hooked up all the cameras I could find (and that were charged up) to my laptop/wifi. In the example below, I was able to run 5 concurrent live streams - showing the capabilities of api.video, but also pushing my computer (and my internet connection) to their limits:



medium_257523051420257536349489364058826396261704n.jpg

small_257523051420257536349489364058826396261704n.jpg

thumbnail_257523051420257536349489364058826396261704n.jpg

257523051420257536349489364058826396261704n.jpg

* [https://Livestream.a.video](https://Livestream.a.video) - using an USB webcam (mounted on monitor)
[https://Record.a.video](https://Livestream.a.video) - using Mac webcam
[OBS](https://api.video/blog/no-code/start-and-record-a-live-stream-with-obs/) - using Iriun and a phone - this app connects your phone to a computer via Wi-Fi (on right of photo on a stand)
[Android livestream](https://api.video/blog/tutorials/livestream-from-your-android-phone/) - phone using Larix Broadcaster to stream (leaning on laptop)
[Pi livestream](https://api.video/blog/tutorials/video-streaming-with-a-raspberry-pi/) - my raspberry Pi camera (pointed out the window)





This was a fun demo to try out, but it did push my laptop to the limit. What it does show is that with a few computers & cameras and an api.video account, it is really easy to simultaneously live stream as many views, angles, rooms as you desire.



How many live streams can I run at once?

large_concurrent-livestreams.jpg

medium_concurrent-livestreams.jpg

small_concurrent-livestreams.jpg

thumbnail_concurrent-livestreams.jpg

concurrent-livestreams.jpg

## The benefits of a cloud video platform
The cloud is collections of servers in data centers that run software, maintain databases and serve all kinds of content to all the corners of the world. As we make it easier to create and share video by adding technologies like fast 5G networks and cheaper camera equipment, more and more people are creating video content. To handle the demand, most video streaming platforms use the cloud. Netflix famously started the move to the cloud after a corrupted database prevented them from mailing anyone their DVDs for 3 days! YouTube uses the cloud, and so does api.video. There are many benefits to providing video via the cloud. 

## 1. You can use a network that can handle video streaming and heavy traffic
A major benefit to using a cloud video platform is that it's up to someone else to handle the technical details of video streaming and coping with heavy traffic. You can just add video to your company portal, or build your own viewing interface or whatever you like.

## 2. Lower cost
It's less money for your business or project if you don't have to store your video content, manage the servers with the content, or hire an IT team. Instead, you pay for a video service, like api.video, and everything is stored and managed for you. 

## 3. Video focused content delivery network
If you choose a cloud video platform like api.video, you get to use our private content delivery network. Because api.video handles only video content, the network is optimized to provide the best experience when working with video. Many companies that work with video have their own networks for their content, including Netflix - their network is called Netflix Open Connect. 

## 4. Continuity
When you store content yourself, you have to have a plan for what happens if part of your system goes down. If you use a cloud service for your video, this is handled for you. By storing it in the cloud, it can be replicated and maintained in multiple locations if need be, so that your video is always there when you need it. 

## 5. Mobility
If your video content is stored in the cloud, it's easier to access. You can stream content at home or a coffee shop, anywhere there's internet, you'll be able to get to your videos and interact with them. 

## api.video and the cloud
At api.video, all our services are in the cloud. We use our own Content Delivery Network (CDN) to ensure fast, reliable uploading, transcoding, and delivery. 

large_top-5-benefits-of-a-cloud-video-platform-1.jpg

medium_top-5-benefits-of-a-cloud-video-platform-1.jpg

small_top-5-benefits-of-a-cloud-video-platform-1.jpg

thumbnail_top-5-benefits-of-a-cloud-video-platform-1.jpg

top-5-benefits-of-a-cloud-video-platform-1.jpg

small_yvanmichel.png

thumbnail_yvanmichel.png

yvanmichel.png

Yvan Michel

All companies in the video streaming sector have the same goal-- guarantee the best watching experience with efficient cost control for ingest, transcoding, and delivery. It is not an easy task and requires a lot of considerations on the Product and Engineering sides.

Measuring the Quality of Experience (QoE) is one of the methods widely used to assist this process. In this blog post, we will go through the QoE principle and functioning by relying on common use cases and metrics.

## Principle

As soon as someone uses a website or an app, stakeholders need feedback in order to learn about the strengths and weaknesses of what they've built. The measurements performed for the QoE purpose are based on this same principle and are seamless for the end-user.

The QoE consists of collecting and sending users subjective data to better understand how the product is used to make the right decisions. In this case, knowing more about how videos are consumed by viewers through a first-party player.

It helps to get more insights about questions like "What will be the effects of reducing the 1080p video bitrate on the end-user engagement?" and "Is it a good approach to preload the video on the start-screen to boost its loading time?"

NOTE: QoE data must comply with local privacy policies like [GDPR](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation) or [CCPA](https://en.wikipedia.org/wiki/California_Consumer_Privacy_Act). No data that breaks these laws or disrespects user privacy shall be stored without the user's explicit consent.

## Functioning

The player performs periodic measures related to the video playback but also computes metrics from its own runtime context like the [FCP](https://web.dev/fcp/), its loading time, and more. It then forwards all this info to a dedicated backend for later processing and analysis.

large_qos-and-qoe-workflow.jpg

medium_qos-and-qoe-workflow.jpg

small_qos-and-qoe-workflow.jpg

thumbnail_qos-and-qoe-workflow.jpg

qos-and-qoe-workflow.jpg

The objective is to gather enough relevant data about the "Where", the "Who" and the "How". The "Where" and the "Who" are quite important but are more "context" oriented. The "How" really refers to the QoE.

### The "Where"

Locate where videos are watched to know the distribution of the audience:

- The `continent` → *Is North America, the main continent in terms of audience?*
- The `country` → *How many videos are watched in France?*

GeoIP Databases allow retrieving the `continent` and the `country` based on the user IP.

### The "Who"

Identify the profile of devices used to watch the videos:

- The `device_type` → *Are CTVs representing a significant part of the audience in Asia?*
- The `platform` and the `browser` family or version → *When was the latest version of macOS released? What is the part of IE11 to clean legacy code?*

Even if it is not 100% trustable because it tends to be deprecated and it can be overridden, the `User-Agent` HTTP header exposes this info and provides, at least, good hints. There are some alternatives like [UA-CH](https://web.dev/user-agent-client-hints/).

For a given visit, having a `user_session` with the number of `views` helps to confirm that after consecutive video playbacks, the experience remains stable and great.

### The "How"

Understand the video consumption behaviours:

- The `rebuffering_time` → *Why is the engagement so low in Asia?*
- The `watch_time` → *What is the best moment to trigger an ad-break?*
- The `abr_max_rendition` → *Is the 720p more played when increasing the default player dimensions?*
- The `playback_error` → *Why are macroblocks appearing on Safari/desktop?*

There are a lot of factors that can influence video playback. Rebuffering is part of them but is the consequence of underlying problems: slow delivery, low user bandwidth, etc.

> A viewer who experienced a rebuffer delay that equals or exceeds 1% of the video duration played 5.02% less of the video in comparison to a similar viewer who experienced no rebuffering.
> 
> Source from [Akamai](https://people.cs.umass.edu/~ramesh/Site/HOME_files/imc208-krishnan.pdf)

But what is rebuffering? A `rebuffering` describes the moment when the video playback stopped because of an insufficient amount of downloaded video bytes *(buffering)* without any user interactions. Seeking or switching manually to another video rendition is not considered to be `rebuffering`.

In other words, `rebuffering` is an unexpected buffering lasting *N* milliseconds causing the video playback to stop.

The `watch_time` is a duration representing the time a user has watched a video. It can be associated with the `rebuffering_time` to compute the `rebuffering_precentage` during a `user_session`.

ABR (Adaptive Bitrate) algorithms choose the best video renditions according to user network conditions. The `abr_max_rendition` describes the highest video rendition reached when ABR is enabled.

Finally, the `playback_error` indicates which errors have occurred during the video playback to explain sudden disruptions or degradations within the network, transmuxing, or decoding.

## Conclusion

We've had a sneak peek at how QoE can be useful to drive operational decisions. Combined with the [Quality of Service](https://en.wikipedia.org/wiki/Quality_of_service) *(QoS)*, this data constitutes a powerful set of valuable information to continuously ensure that video playback is optimal.

At [api.video](https://api.video), we are committed to providing the best watching experience for all, no matter the location or the device. In another blog post, we will go through other side usages based on QoE data especially for observability to detect proactively issues and assess impact in case of outages. Stay tuned!

## Resources

- [https://www.akamai.com/blog/performance/enhancing-video-streaming-quality-for-exoplayer-part-1-quality-of-user-experience-metrics](https://www.akamai.com/blog/performance/enhancing-video-streaming-quality-for-exoplayer-part-1-quality-of-user-experience-metrics)

The benefits of measuring the video playback QoE

Benefits of measuring the video playback QoE

large_qoe-1.jpg

medium_qoe-1.jpg

small_qoe-1.jpg

thumbnail_qoe-1.jpg

qoe-1.jpg

## Responsive media on the web (ok, mostly responsive video)


As the web has evolved, building sites with media rich content has gotten more and more complex. We (as developers) are expected to build sites that are media-rich and engaging - but at the same time load quickly and work on many different platforms. This is a very hard balancing act that has been the focus of the web performance community...well since the beginning of the web.

In this post, we’ll focus on how media can be efficiently delivered on the web - we’ll focus mostly on video, but will use images as examples, as there has been a lot of work to make images fast for all devices and screens.


## Images

Images are a logical place to start when considering responsive media - I mean you could just consider them a single frame video!

When it comes to image delivery on the web, there are a lot of HTML native approaches to deliver images as small as possible - and properly sized for the device. Let’s use some examples with the ```<picture>``` tag to walk through how you can properly serve images responsively. Here’s the example from the MDN page:

```
<picture>
    <source srcset="/media/cc0-images/surfer-240-200.jpg"
            media="(min-width: 800px)">
    <img src="/media/cc0-images/painted-hand-298-332.jpg" alt="" />
</picture>
```
	
There are 2 images in the picture tag. The ```<img src>``` is the fallback for browsers that do not support the picture tag, and is the “image of last resort” in case the ```<source>``` images do not fit the criteria of the browser or space on the screen.

The ```<source>``` attribute describes alternative images that can be displayed. In the example above - there is a different image for screens where the screen width is over 800 pixels.  The idea is that a “big image” can be delivered to big screens, and a “small image” for mobile or smaller screens. We can use the picture tag with even more complex methods:  

```
<picture>
    <source media="(min-width: 1200px)" src="hero-large.webp 1200w" type="image/webp">
    <source media="(min-width: 1200px)" src="hero-large.jpg 1200w" type="image/jpg">
    <source media="(min-width: 800px)" src="hero-medium.webp 800w" type="image/webp">
    <source media="(min-width: 800px)" src="hero-medium.jpg 800w" type="image/jpg">
    <source src="hero-small.webp" type="image/webp">
    <source src="hero-small.jpg" type="image/jpg">
</picture>
```
	
In this example, we have 3 different versions of the same image - ‘hero-large’, ‘hero-medium’ and ‘hero-small’.  They are offered in 2 different formats: webp and jpg. While this is can lead to a great (and fast) experience for your customers, Iit does mean 6 different versions of each image must be created.

When the browser encounters the picture tag - it iterates through the options, and will use the first version of the image that meets all the criteria. If the browser supports webp, and the screen is over 1200 pixels wide, the first image will be used. If the screen is under 800px, and the browser does not support webp, the ‘hero-small.jpg’ will be used.

This is incredible - your website now supports 3 different screen sizes and 2 different image formats.  Of course, this means either the developers or the content management system must create 6 versions of each image that is uploaded - an extra development step, and potentially a lot more storage usage.

	
## Video

The video tag works in similar ways to the image tag when it comes to formats:

```
<video autoplay muted controls>
	<source src=”hero.webm” type=”video/webm”>
	<source src=”hero.mp4” type = “video/mp4”>
</video>
```
	
If the browser can handle webm videos - this format will be used, otherwise it will fallback to the mp4 (98.1% support in browsers today - and the last 1.9% don't support video at all).

In a soon to be published article, HTTP Archive data finds that using multiple source files is not used widely - less than 10% of video tags use this feature (on 7.6M webpages tested).  Even worse, of those that use multiple source files ~75% lead with the mp4, negating the presence of the other versions of the file (since the browser uses the first format it knows how to use, and all browsers know how to use a mp4.) This implies that only 2.5% of the video tags are correctly utilizing the source attribute to deliver different versions of the same video.

### No media queries

There are no media queries for video.  There *once were* media queries in the video specification, but they were removed, and support for media queries left Chrome in version 34 (In November 2021 Chrome is version 95).  So, the pattern we used to make images responsive will not work for video.


### Why were they removed?

When it comes to images, generating 3-4 different types of image can be tricky - which formats?  Which sizes? What is the process to generate - is it automated, or do you need to build a specialized upload process?  If your site has hundreds of images - you may now have thousands of images to store and deliver.

Video suffers from the same issues.  But videos - being generally more than one frame long - make the problem more immediate.  [Re-encoding](https://api.video/what-is/video-encoding/) a video takes much longer than reformatting an image - so the time to generate the files can be on the order of minutes (as opposed to a second or 2 for images). 

If you are using a CMS - it must know how to take an uploaded video, and turn it into several different sizes - with a couple of different formats.  If you are not using a CMS, guess what - it falls on you to learn how to re-encode the different sizes and formats for each video.

### Video doesn't work the same way

When you add an image to a webpage - it stays where it is.  It’s static.  If you download a huge image on a poor connection - it takes longer, but the page will still load.

Video is not static.  But since the ```<video>``` tag is markup, it cannot change what is rendered on the page.  

### Let’s consider a couple of use cases:

You have a ```<video>``` embedded on your page, and it renders at 600px wide.  Your page uses logic to choose and download a video source file that is optimized for a 600px space (saving a lot of data over the 'desktop' 1920x1080 video.)  

1. What happens when the user presses the “fullscreen” button?  
2. What if your customers have a chromecast, and they cast it to their 60” TV?  

The video sent to full screen/TV will be 600px wide, and will look pixelated. Even with media queries - there is no way to switch the video input without stopping the first download, and starting a new download.  If you are 15 minutes into the mp4  - the browser would have to have logic to figure out where that time is in the video.  Needless to say, this is not a good user experience.

3. What happens when you start a video, but then you move to an area with lower bandwidth?  

The video download will not be able to keep up with the video playback, and the video will stall.



## Responsive video: JavaScript player

Rather than create a myriad of different sizes and file types in the ```<video>``` tag, the solution is adaptive bitrate streaming (ABS), supported by a JavaScript player sitting on top of the ```<video>``` tag. We’ve described how [ABS streaming](https://api.video/what-is/adaptive-bitrate-streaming/) [works](https://api.video/blog/video-trends/responsive-video-delivery/) in  several posts. Esentially several different sizes of the video are created, and the video is delivered in short segments or chunks.  

The JavaScript player sits atop the video tag, and works as a controller for the video - monitoring both the screen size (and changes in screen size) and the network conditions.  When a change occurs that could affect the video playback, the player is able to make changes midstream to the video.  Since each chunk defines a timestamp in the video - switching midstream between different video qualities is a simple exercise for the player.

With ABS, jumping from 600p wide to full screen, or casting to a TV *just works*.  When the netwrork conditions deterioriate or improve, the player can adapt the playback quality to optimzie the experience for the viewer.

It is totally understandable to be hesitant in adding a new JavaScript framework to your page - we are so bloated in JS today (that's an entirely different topic) - but for a few hundred KB, the JS player enhances the video playback experience on all devices.  With a video that is specifically sized for mobile devices - the savings in video delivery is very likely to exceed the cost of the JS incurred on video setup.


## Conclusion

The video tag is amazing, and will continue to be the backbone of video on the web.  To make the video tag more responsive to various screen sizes, there are 2 possible approaches - reading media queries, or using a JavaScript controller to power adaptive bitrate video.

The media query approach solves one problem - but raises several more that impact the user experience.  The JavaScript player adds additional page weight, but delivers the video responsively, in a network-aware way.  We at api.video believe that adaptive bitrate video *is* the future of video, and the added weight of the JavaScript player to the video tag is a small tradeoff in exchange for providing stellar video experiences on the web.





Responsive Media: the right approach for images vs video 

Responsive media: The right approach for images is not the right approach for video

small_ResponsiveMedia.png

thumbnail_ResponsiveMedia.png

ResponsiveMedia.png

small_anthony.png

thumbnail_anthony.png

anthony.png

Anthony Dantard

Customers are "divinely discontent" and their expectations are constantly on the rise. Thanks to services like YouTube & Netflix, we all expect blazingly fast load times and smooth video playback for streaming on-demand video. What might be less obvious is that, whether we're aware of it or not, that expectation is slowly transferring to real-time video communication and live streaming applications as well.

At api.video, we're obsessed with delivering the best developer and viewer experience for our customers. So it's no surprise that we spend a lot of time thinking about and keeping up-to-date on video protocol development. Each video protocol has its strengths and weaknesses. Each is better suited for different applications. 

In this post, we outline four of the major low-latency protocols, talk about the pros and cons of each, and give our commentary on where we think the future of low latency streaming is going. We'll talk about the following protocols in depth in this post:

- WebRTC
- LL-HLS
- LL-DASH
- HESP

## WebRTC
The WebRTC protocol supports both audio and video streams for real-time bi-directional communication. It can be used for ingestion and distribution with an end-to-end latency between 300ms - 600ms depending on network quality and the distance between users. The protocol really took off in 2011 when Google bought Global IP Solutions, the company that originally developed the protocol, and gave the project significantly more human and financial resources. 

Ten years later, WebRTC has become the standard for real-time video communication on the web. The components that WebRTC is based on are accessible via a JavaScript API maintained by the W3C and the IETF, allowing users to live stream directly to a web browser without installation of any third-party tool.

In theory, WebRTC allows the establishment of a video conference system between 2 browsers by connecting them to each other peer-to-peer. However, in practice the architecture of computer networks (internet, corporate, and domestic) can greatly complicate matters.

large_webrtcprotocolstack.png

medium_webrtcprotocolstack.png

small_webrtcprotocolstack.png

thumbnail_webrtcprotocolstack.png

webrtcprotocolstack.png

### Finding your way through the internet labyrinth

In some cases, we are not directly connected to the internet and we are behind several network layers such as a NAT, a firewall, or a proxy. To get around these problems, WebRTC allows you to use the ICE (Interactive Connectivity Establishment) protocol. It allows you to find the most direct path to make two machines communicate and cross the different network layers. For this, a STUN / TURN server is necessary in order to obtain the external address of a user and relay the traffic if a direct connection is not possible.

The WebRTC protocol requires, in addition to its own multimedia server infrastructure, a STUN / TURN server infrastructure to operate communications in a production environment with real massive use.

## To further confuse the issue

The protocol specification requires that all data exchanged between peers must be encrypted (video, audio, data application). Thus, the protocol embeds several security protocols to fill the gaps of the UDP protocol when it's used. 

WebRTC uses SDP (Session Description Protocol) to expose some properties of the peer-to-peer connection like the types of media to be exchanged and their associated codecs, the network transports, and some bandwidth information.

Respectively DTLS (Datagram Transport Layer Security) is involved to generate Self-Signed certificates, for the negotiation of the secrets used for encrypting media data, and for secure transport of application data between peers. SRTP (Secure Real-Time Transport) is used for the transport of video and audio streams. The last security protocol involved is SCTP (Stream Control Transport Protocol), which is used for the exchange of application data.

### The challenges of WebRTC scalability

Several network architectures and topologies can be implemented to operate communications with the WebRTC protocol, each responding to a specific use case, with advantages and drawbacks specific to each topology. We will not dig into topology details in this article, but be aware that SFU (Selective Forwarding Unit) is perhaps the most popular architecture in modern WebRTC applications today.

The main challenge with this protocol is the scalability on a planetary scale that is cost effective. We are not saying that WebRTC can't be scaled worldwide, it was true a few years ago at its early stage but it's not true anymore thanks to the great improvements and work bring by some great companies to improve topology. 

As WebRTC is a mashup of several protocols that each have to be learned and mastered by engineers rendering it very complex. It's difficult for most companies to implement it in their infrastructure at a global scale in a cost effective way. That's why only a few companies have succeeded at making it fully scalable today. They do so by mixing SFU and MCU (MultiPoint Conferencing Unit) topologies, then putting it on the edge.

## Low Latency HTTP ABR streaming protocols

In contrast to WebRTC, which is a peer to peer protocol that, in theory does not require a central server to establish communication between two machines, HTTP-based streaming protocols require the use of a server and communication over standard HTTP. They are based on the very foundations of the web.

### HLS/LL-HLS

The HTTP Live Streaming (HLS) protocol can be used to deliver live and on-demand content streams to global-scale audiences and was introduced by Apple in 2009 with scalability in mind rather than latency. While the average latency faced by users is around 15s, the HLS latency can reach between 30s and 1 minute. Even with performant infrastructure and optimized packaging and player configuration, one can expect latencies of around 6s, which is just too high for real-time live video use cases. However it's still the most popular ABR streaming protocol. Its success is mainly due to its great compatibility with a wide range of devices and browsers and supports advanced features like closed captions, ads, and content protection with AES encryption or DRM.  

Although the protocol can also be used to ingest video, it's normally only used for distribution and then paired with the RTMP protocol for ingestion. Below is a typical live streaming architecture involving both the RTMP and HLS protocol.

large_rtmphlslivestreamingworkflow.png

medium_rtmphlslivestreamingworkflow.png

small_rtmphlslivestreamingworkflow.png

thumbnail_rtmphlslivestreamingworkflow.png

rtmphlslivestreamingworkflow.png

Without digging into the details of how HLS works (see our blog [articles](https://api.video/blog/video-trends/what-is-hls-video-streaming-and-how-does-it-work/) to learn more on this topic), below is a simple schema describing how playlists and media segmentation gives HLS the ability to do adaptive bitrate streaming (ABS).

Diagram of an HLS playlist and segmentation

large_hlsplaylistsmediasegmentation.png

medium_hlsplaylistsmediasegmentation.png

small_hlsplaylistsmediasegmentation.png

thumbnail_hlsplaylistsmediasegmentation.png

hlsplaylistsmediasegmentation.png

So how did HLS evolve to support lower latencies? In the beginning, HLS only worked with MPEG-TS container formats associated with the H.264/AVC video codec. In 2016, support for fragmented MP4 (FMP4) was added and this led to support for the CMAF format and compatibility with DASH distribution. One year later, HLS added support for H.265/HEVC (FMP4 only), which then allowed for a significant reduction in bandwidth usage.

Thus, Low-Latency HLS, an extension of the base HLS protocol, was finalized in April 2020 and enables low-latency video and live streaming by utilizing partial segments and on-the-fly transfer of those segments while maintaining HLS's inherent scalability. On the first draft of this extension, the protocol required the support of "HTTP/2 Push" which made it difficult to implement. Unsurprisingly, as major CDN vendors chose not to support this feature, killing the ability to scale LL-HLS, as none of the CDN players could cache the contents. Fortunately, Apple listened to community and industry concerns and came back with a new revision of the protocol extension, that removed the need for "HTTP/2 Push," but kept the need of relying on the "HTTP/2" protocol. They merged the spec into HLS making it fully backward compatible with the HLS specification and that also means that all features covered by HLS can be retrieved in LL-HLS. 

LL-HLS works practically the same way as HLS, but makes some important changes to reduce the latency in the packaging process. Below are the most important updates to the specification which make low-latency possible while preserving scalability and ability to do ABR streaming:

- **Partial Segments**: A segment is divided into Partial Segments or "Parts" of a few milliseconds which are referenced in the Media Playlist. These have a very short cache lifetime in contrast to the parent segment which can have a longer lifespan on the CDN. Once a complete segment is generated, its associated Partial Segments can be removed from the media playlist to reduce the bandwidth footprint.
- **Preload hints**: The media playlist can have a “Preload Hints” tag, which allows the player to know that new partial segments are coming so that the player can start to request them and the server will respond to this request as soon as the data is available.
- **Block Playlist Reload**: This feature avoids media playlist polling between the player and the server by adding query parameters to the request which inform the server that the player wants the response only when the playlist includes a new segment or partial segment.
- **Playlist Delta Updates**: Thanks to the new "EXT-X-SKIP" tag, a player can request just the updated part of the Media Playlist and save the transfer cost of the data that it already has.
- **Rendition Reports**: The "EXT-X-RENDITION-REPORT" tag added for each rendition line indexed in the Primary Manifest (aka: Master Manifest) provides some useful information for the Player to operate ABR such as the last sequence number and the last partial segment available for each rendition.

large_ll-hls-workflow-and-media-segmentation.png

medium_ll-hls-workflow-and-media-segmentation.png

small_ll-hls-workflow-and-media-segmentation.png

thumbnail_ll-hls-workflow-and-media-segmentation.png

ll-hls-workflow-and-media-segmentation.png

With current public cloud providers you can expect to reach an end-to-end latency in the range of 2s with appropriate Group of Pictures , partial fragments, and reasonable buffer size. While this is still much slower than what WebRTC can achieve, it involves significantly less complexity and is easier to implement in your existing live streaming architecture. You can also push the protocol to its limits by changing some settings and achieve a latency of around 1s, but this comes at the expense of stability and the quality of the playback experience.

### DASH/LL-DASH

DASH stands for Dynamic Adaptive Streaming over HTTP and was born in April 2012 by the work of the Moving Picture Experts Group (MPEG) in order to create an industry-standard alternative to the HLS protocol owned by Apple. It works similar to HLS in that it's based on the preparation of content at different quality levels and divides each video quality into several chunks indexed in manifest files to enable ABR streaming. 

It also supports content protection with encryption through Common Encryption standard (CENC) which makes it compatible with all common DRM systems.

The main difference between LL-HLS is that LL-DASH is completely agnostic of the codecs used for encoding video and audio streams. Unfortunately this codec freedom comes with the cost that LL-DASH is not compatible with Apple devices relying on iOS, including Apple TV.

In 2017, LL-DASH provided the necessary changes to the standardized protocol to decrease latency to 2s. Behind the scenes, it relied on the Common Media Application Format (CMAF). CMAF offers the ability to play with some useful HTTP features that can be used to decrease latency a lot. The two features that allow Low Latency for DASH with CMAF are "Chunked Encoding" and "Chunked Transfer Encoding" — both part of the HTTP/1.1 spec (prohibited in HTTP/2) that respectively allow you to break a segment into smaller chunks of a few milliseconds that can be sent to the distribution layer once encoded and this is where the "Chunked Transfer Encoding" feature take the relay to allow to distribute encoded chunks on the fly. However to operate this kind of delivery process, the whole distribution stack has to support the chunked transfer encoding feature from the origin right through to the CDN.

Diagram of LL-HLS workflow and media segmentation

large_dashll-dash2.png

medium_dashll-dash2.png

small_dashll-dash2.png

thumbnail_dashll-dash2.png

dashll-dash2.png

### HESP

The High Efficiency Stream Protocol (HESP) is another ABR HTTP based streaming protocol and is the most recent one of all. It was created by THEOPlayer through the HESP Alliance, whose main mission is to work on the standardization of the protocol and its promotion.

HESP has been developed with the aim of solving several of the main constraints of other HTTP streaming protocols. It's goals are:

- To reach ultra low latency(<500ms) while preserving scalability (i.e. it can still work with major CDN vendors)
- To reduce the bandwidth required during the streaming
- To reduce zapping times (zapping is the time it takes to switch between streams, think of it as changing the channels on cable television)

These three performance metrics have a direct impact on the user experience of the audience of a live stream. Given the extremely low latency, it can also be a viable alternative to WebRTC. The main drawback of HESP is that it's a proprietary commercial protocol and–for commercial use–it's quite expensive. Let's dig a bit into a technical overview of how it works.

The main difference with HESP in comparison to other low-latency protocols is that it relies on two streams rather than one. Before we go into the details of how these streams allow us to reach sub-second of latency, let's talk a bit about the different types of frames that are required when streaming video:

In video compression there are several types of frames:

- IDR-Frame (aka: Key Frame)
- I-Frame
- P-Frame
- B-Frame

Let's start with the I-Frame, you will better understand the others. An I-Frame contains the whole image and is coded without reference to any other frame except itself.

The key frame (or IDR-Frame) is a special I-Frame where no frame after the IDR frame can reference any frame before it. That being said, all IDR-Frames are I-Frames but not the other way around. The key frame is used by any player to be able to start a playback.

The P-Frame holds only the changes in the image from the previous one.

The B-Frame holds the changes between the current frame and both the preceding and following frames.

Now that you know the role of the different kinds of frames composing a video, we can go back to the streams that compose the HESP protocol:

- The first stream is called the "Initialization Stream" and contains only key frames.
- The second one is called the "Continuation Stream" and is like a normal encoded stream, meaning that it can contain all types of frames depending on the encoding parameters and defined profiles for maximum performance or maximum compatibility.

The Initialization Stream is used only for the start of the playback or when you slide on the video timeline to change the position of the playback. Because this stream contains only key frames, the decoder behind the player is able to instantly decode the frame and so start(or restart) the playback of the live event. Once the first frame is pulled from the first stream and decoded, the player switches automatically to the second stream to continue video playback. This is because key frames are complete images, so they have a high cost in terms of bandwidth. By switching to the second stream, the player falls back to a regular live streaming bandwidth footprint and profits from the CDN capability for scaling the audience and reducing the load to the origin.

As for DASH/LL-DASH, HESP works also with CMAF-CTE for packaging and distribution. It inherits from all the features such as encryption, DRM support, subtitles and captions, advertisements, and more.

Video on demand

Live streaming

Analytics

Video infrastructure

Video player

See all features

Online learning

Short-form videos

Marketplace & E-commerce

Video messaging and recording

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Help center

Blog

Huge pricing update! Introducing video encoding at zero cost