Live stream status

Normal

large_logo_avatar.webp

medium_logo_avatar.webp

small_logo_avatar.webp

thumbnail_logo_avatar.webp

logo_avatar.webp

Erikka Innes

large_api-video-blog-video-trends.png

medium_api-video-blog-video-trends.png

small_api-video-blog-video-trends.png

thumbnail_api-video-blog-video-trends.png

api-video-blog-video-trends.png

Video trends

[FFMPEG](https://api.video/what-is/ffmpeg) allows you to stream and [live stream](https://api.video/live-streaming/) video, but sometimes what the output means can get confusing. 

We are going to go over every part of the information in the output, so if something seems obvious, skip ahead to the part you wanted to learn about.

Here's our sample. This sample shows a session where a live stream is started using my Mac and the Mac's iSight camera. I started it using the [livestream.a.video](https://livestream.a.video) demo through a Chrome browser. There might be some differences based on what computer you're on, so keep that in mind. 

## Hello from mediarecorder-to-rtmp server!

We're using an RTMP connection, which is based on TCP. So when I connect with my request to create a live stream, my client gets a response: 

"SERVER:Hello from mediarecorder-to-rtmp server!" 

I'm asked to provide my stream key, which is going to be what I will use to broadcast. Because of how the demo is set up, that's already been sent, so now the server tells me it's received the information about where I want to broadcast my live stream to: 

"SERVER:Please set rtmp destination before start streaming.
SERVER:rtmp destination set to:rtmp://broadcast.api.video/s/30087931-229e-42cf-b5f9-e91bcc1f7332"

The last part tells me what version of FFMPEG we're using and provides a link to static builds of FFmpeg for Linux kernels 3.2.0 and up. It also gives me copyright information - wow we've had FFMPEG for 22 years!

## Configuration details 

The next part of the FFMPEG output shows us the configuration details for the live stream. 

In this section, we're told that FFMPEG was built with gcc 8. GCC stands for GNU Compiler Collection and it's the standard compiler for Linux. It's associated with the Linux distribution Debian (Debian GNU/Linux). The Debian Linux distribution is super popular and while it's less user friendly than distributions like Ubuntu, it is great for people who like free and open source software. 


## FFMPEG library versions
This section of output lists the main libraries in FFMPEG and the versions you have installed. 


## FFMPEG input info 

This section shows you details about your video live stream. 

Here's the first line: 

"FFMPEG:Input #0, matroska, webm, from 'pipe:':

This explains how video is being input. We can see here that the multimedia container matroska and the open video format webm is being used to capture video.

"Metadata:
  encoder         : Chrome
 Duration: N/A, start: 0.000000, bitrate: N/A"
 
 Next, there's more information about the streams being input: 
 
 "Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
  Stream #0:1(eng): Video: h264 (Constrained Baseline), yuv420p(progressive),   
	240x240, SAR 1:1 DAR 1:1, 1k tbr, 1k tbn (default)"

### The audio stream
The audio stream is using the opus codec with a sample rate of 48000 Hz, which is standard for digital video files. Mono means the audio is monophonic, meaning all sounds are mixed together in a single channel. And fltp is Planar Floating point format. It's a way of retrieving the sample format for your audio. Sample format information is used to determine how many bits will be in each audio sample. The higher the number of bits per sample in a file, the more data that will be stored in the sample. You can learn more about sample format here: What is Sample Format?

### The video stream
The video stream is using the h264 codec with a constrained baseline. The term constrained baseline describes the video stream profile. A profile determines the methods that an encoder can use to generate streams and also describe the methods a decoder is expected to perform in order to decode the stream later. The term 'yuv420p' describes the type of video signal you're using. 

YUV is a type of video signal where colors are split into three channels, one for luminance (brightness) which includes green, and two for chrominance (color) which include blue and red in separate channels. 420 describes the chroma subsampling used on the video. Normally, you'd notate this as 4:2:0. It's telling you that the color channels have half the resolution of the luminance channel. If you want to learn a bit more about this way of representing color, see the api.video definition for YCbCr. The term progressive means that the video is in a progressive format, which means that the TBR value (coming up in our discussion) is talking about the frame rate rather than the field rate. There's confusion about both these terms. For the purpose of this walkthrough, let's keep things simple. A progressive video format will use frame rate to describe frames per second. An interlaced video format will use field rate. It takes two fields to show you one image. 

240x240 describes the pixels in each frame of video. SAR stands for Sample Aspect Ratio, and it's showing you the ratio between the width and the height of each individual pixel - 1:1. DAR stands for Display Aspect Ratio and that's the ratio between the width and height of a full image in your video. In this case it's also 1:1. 

In the final line, it closes out with "1k tbr, 1k tbn (default)." TBR represents the frame rate baseline. This is guessed by FFMPEG when it looks at the video stream. At the start of a video stream, TBR is low. In our stream, it's 1 at the very start. Sometimes if you have a video format that's interlaced, the TBR will be twice as high as expected, because there are two fields used to create an image. TBN represents the time base in AVStream that comes from the container. TBC is the time base for the codec for the stream.

## Stream mapping 
Next in our information from FFMPEG we have a section about stream mapping that looks like this:

Stream #0:1 is our video stream. This means that the input stream is from the first file '0' and it is the second stream in that file '1.' h264 (native) means that the detected codec of the input stream is H.264 and that FFMPEG is going to use the built-in decoder to decode the stream. It's being mapped to h264 (libx264) meaning that the output stream is going to be H.264 and FFMPEG will use the libx264 encoder to generate the stream. 

Stream #0:0 is our audio stream. This means that the input stream is from the first file '0 and its the first stream in that file '0.' opus (native) means that the detected audio codec of the input stream is opus and as before, native means that FFMPEG is going to use its built-in decoder for opus to decode the stream. aac (native) means the output audio will be in aac, and FFMPEG will use the built in encoder for aac to generate the stream. 

## Configuration details for libx264
We're mapping to libx264, so now we receive some details about how libx264 is configured for our stream. 

You can see the Video Buffering Verifier confirms that there's a buffer size set, but the max bit rate it can handle isn't specified, so it's ignored. Then the SAR is 1:1, as it was to start with. We can see the CPU capabilities available for sending our live stream and that we'll be using the constrained baseline for H.264. 

Next there's the line: 

"FFMPEG:264 - core 164 r3065 ae03d92 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html"

This tells us what version of the H.264 codec FFMPEG uses. It's followed by options that are configured for the codec. 

**cabac=0** 
Cabac stands for context-adaptive binary arithmetic coding. It's a type of entropy encoding, which is a lossless data compression technique. H.264 is lossy, so cabac is used at the end to do a final compression that is lossless. You should not mistake its inclusion in H.264's compression steps as meaning all the steps are lossless. Also, explaining this algorithm in detail would take some time. If you want to learn more about cabac, look here: Context=adaptive binary arithmetic coding

**ref=1**
This indicates which frame will be the first reference frame. Here, it's the first frame.

**deblock=0:0:0**
This is the deblock filter for H.264. It creates pixelation to help improve the appearance of your video. It's useful when macroblocking occurs, which would be a huge block of pixels that look unnatural. Deblocking blurs these blocks to make them look normal again.

**analyse=0:0** 
This is part of deblocking. It helps determine the size of each block you will apply deblocking to. 

**me=dia**
This is saying that the motion estimation method is diamond or dia. Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. Diamond search technique for block motion estimation is based on diamond search patterns. The process of block matching motion estimation is to find out a candidate block, within a search area in the previous frame, which is most similar to the current block in the present frame. So you're saying here use this technique for motion estimation.

**subme=0** 
This is for the sub-pixel motion estimation method. It's important for video coding inter-frame prediction and has high computation complexity. It's set to 0 meaning there will be zero sub-pixel motion estimation done. 

**psy=1 psy_rd=1.00:0.00**
This is also to do with motion-compensated DCT, which is a lossy block compression transform coding technique. If set to 1 it means use psychovisual optimizations. It means optimizing quantization tables based on the human visual system and humans see. It's a complex topic that can't be fully explained here. psy_rd is about setting how strong the psychovisual optimization will be.

**mixed_ref=0** 
This allows you to use one reference per partition as opposed to per macroblock. A macroblock is a large block of pixels and a partition is a smaller, multiple variable-sized prediction block. 

**me_range=16**
Again about motion estimation, but this time about the range for a motion vector. In video compression, a motion vector is the key element in the motion estimation process. It is used to represent a macroblock in a picture based on the position of this macroblock in another picture, called the reference picture. This parameter lets you set the range.

**chroma_me=1** 
Typically when doing motion estimation, only luminance is examined. You can have chroma considered as well by setting chroma_me to be 1.

**trellis=0** 
This lets you set the rate-distortion optimal quantization. This is a method used to improve video quality in video compression. It optimizes the amount of video quality lost per chunk of data by measuring how far an optimization will deviate from the source material and the cost of that deviation. This can be used to decide what choices are useful vs. resulting in the loss of too much data. It's a slow process, so sometimes it's better not to use it, which is what's being done here by setting trellis=0.

**8x8dct=0** 
With this parameter you can enable adaptive spatial transform. This is a way of altering an image or video frame through rotation, flips, scaling and deformation. For our case, where it's for video, it's about ways to manipulate the block of data so it appears less distorted and of higher quality.

**cqm=0** 
Allows the use of a custom quantization matrix to weight frequencies differently in the quantization process. The presets quant matrices are "jvt" and "flat". Recommended only if you know what you're doing.

**deadzone=21,11** 
When trellis isn't activated, deadzone parameters determine how many DCT coefficients are rounded up or down. Rounding up results in higher quality and more detail retention, but costs more bits--so rounding is a balance between quality and bit cost. Lowering these settings will result in more coefficients being rounded up, and raising the settings will result in more coefficients being rounded down. Recommended: keep them at the defaults. This is part of the settings for rate-distortion optimal quantization. This is again fairly complex to explain. If you want to learn more about it conceptually, check out this page: Quantization

**fast_pskip=1** 
By default, x264 will skip macroblocks in P-frames that don't appear to have changed enough between two frames to justify encoding the difference. This considerably speeds up encoding. However, for a slight quality boost, P-skip can be disabled. In this case, the full analysis will be done on all P-blocks, and the only skips in the output stream will be the blocks whose motion vectors happen to match that of the skip vector and motion vectors happen to match that of the skip vector and which have no residual. The speed cost of enabling no-fast-pskip is relatively high, especially with many reference frames. There is a similar B-skip internal to x264, which is why B-frames generally encode much faster than P-frames, but it cannot be disabled on the commandline.

**chroma_qp_offset=0** 
This stands for chroma quantization parameter offset. It is the quantization parameter difference between chroma and luma. 

**threads=3**
FFMPEG is multi-threaded by default. How many threads it will use depends on the codec. By default for libx264 it is cores x1.5 for frame threads and cores x 1 for slice threads. 

**lookahead_threads=3** 
A lookahead thread looks at incoming frames before they are transcoded. Number of lookahead threads can impact performance speed.

**sliced_threads=1** 
Sliced threading decodes multiple parts of a frame at the same time.

**slices=3**
If the codec you're using supports multi-threading, you can decode more than one part of a single frame at once by working on multiple slices. This is only available if the video was encoded using slices.

**nr=0** 
Confusingly, this stands for noise-sensitivity. You can use it with static-thresh to change the threshold on blocks that are examined in an image. If they fall below the specified threshold the encoder skips them.

**decimate=1** 
This allows you to drop duplicated frames at regular intervals. 

**interlaced=0** 
This is a simple interlacing filter from progressive contents. This interleaves upper (or lower) lines from odd frames with lower (or upper) lines from even frames, halving the frame rate and preserving image height.

**bluray_compat=0** 
Configure the encoder to be compatible with the bluray standard. It is a shorthand for setting "bluray-compat=1 force-cfr=1".

**constrained_intra=0** 
You can enable constrained intra prediction, which is required for the base layer of SVC encodes. This is an intra prediction technique where the video encoder limits the use of neighboring blocks as reference blocks in the intra prediction process. If you're using constrained intra prediction, you can configure video to not use neighboring blocks as reference blocks if the neighboring blocks were coded using inter prediction. By not using inter predicted blocks as reference blocks for intra prediction, the video encoder may create an encoded video bitstream that is more error resilient.
 
**bframes=0** 
You can set the maximum number of concurrent B-frames that x264 can use. Without B-frames, a typical x264 stream has frame types like so: IPPPPP...PI. With --bframes 2, up to two consecutive P-frames can be replaced with B-frames, like: IBPBBPBPPPB...PI. B-frames are similar to P-frames, except they can use motion prediction from future frames as well. This can lead to significantly better efficiency in terms of compression ratio. 

**weightp=0** 
Use this to enable use of explicit weighted prediction to improve compression in P-frames. Higher modes are slower.

**keyint=30** 
This sets the maximum interval between IDR-frames (aka keyframes) in x264's output. You can specify "infinite" to never insert non-scenecut IDR-frames. IDR-frames are 'delimiters' in the stream - no frame can reference data from the other side of the IDR-frame. As well as this, IDR-frames are also I-frames, so they don't reference data from any other frame. This means they can be used as seek points in a video.

**keyint_min=16** 
This sets the minimum length between IDR-frames. Very small keyint ranges can cause "incorrect" IDR-frame placement (for example, a strobing scene). This option limits the minimum length in frames after each IDR-frame before another can be placed.

**scenecut=0** 
This sets the threshold for I/IDR frame placement (read: scene change detection). x264 calculates a metric for every frame to estimate how different it is from the previous frame. If the value is lower than scenecut, a 'scenecut' is detected. An I-frame is placed if it has been less than keyint_min frames since the last IDR-frame, otherwise an IDR-frame is placed. Higher values of scenecut increase the number of scenecuts detected.

**intra_refresh=0** 
You can disable IDR-frames, instead H.264 uses intra coding for each macroblock in the frame every keyint_min frames. Blocks are refreshed in a horizontally scrolling column - the 'refresh wave'. This benefits low-latency streaming by making it possible to achieve more constant frame sizes than is possible with standard IDR-frames. It also increases the resilience of the video stream to packet loss. This option does reduce compression efficiency, hence only use it when needed.

**rc=crf**
rc stands for Rate Control. You can adjust the file data rate in several ways, the option shown here is CRF, or Constant Rate Factor. CRF is an encoding mode that adjusts the file data rate up or down to achieve a selected quality level rather than a specific data rate. CRF values range from 0 to 51, with lower numbers delivering higher quality scores.

**mbtree=0** 
This stands for macroblock tree rate control. Macroblock tree rate control improves compression by tracking how long different parts of an image last across frames. For parts of an image that stay the same, you don't need to continually save that data, only make a note that it continues to be there so you can use that as a reference to reconstruct that part of the frame later. 

**crf=25.0** 
This is related to the rc=crf command. CRF is an encoding mode that adjusts the file data rate up or down to achieve a selected quality level rather than a specific data rate. CRF values range from 0 to 51, with lower numbers delivering higher quality scores.

**qcomp=0.60** 
The default setting is 0.60. qcomp stands for quantizer curve compression factor. A setting of 0 will give you a constant bitrate while a setting of 1 will give you a constant quantizer. This can be combined with mbtree, but the higher your qcomp setting, the weaker the effect of the mbtree setting.

**qpmin=0** 
The default setting for this is 0 and raising this minimum is discouraged because it can negatively impact the quality of background areas in your frame. 
qpmin defines the minimum quantizer that H.264 will ever use. The lower the quantizer, the closer the output is to the input. 

**qpmax=51** 
The default setting for qpmax is 51. The default of 51 is the highest quantizer available for use in the H.264 spec, and is extremely low quality. This default disables qpmax. You can set this as low as the 30-40 range if you want to cap the minimum quality x264 can output, but adjusting it is not recommended.

**qpstep=4**
This allows you to set the maximum change in the quantizer between two frames. 

**ip_ratio=1.40** 
The default setting is 1.40. You can change the average decrease in quantizer for B-frames vs. P-frames, with higher values decreasing B-frame quality.

**aq=0**
aq stands for Adaptive Quantization. This helps distribute bits between macroblocks in your video. If you don't use it, it can underallocate bits to sections that lack detail. A setting of 0, as with other FFMPEG settings, means you will not use it at all. 

## Output to api.video
Now that you've got all the configuration settings for your streams and you've mapped them, you'll want to see some details about what will be output to your destination. 

Here you can see the destination, shown by the stream key and then a summary of details about your video and audio streams. The encoders being used are listed as well. We don't need to walk through everything listed here because we covered it in the earlier sections when we mapped the video and audio streams. FPS is frames per second. 

Now we can take a look at the lines that show us details about each chunk of data being streamed. Here's one of the lines: 

"FFMPEG:frame=65 fps= 55 q=12.0 size=29kB time=00:00:05.06 bitrate=  47.1kbits/s speed=4.26x"

The line is a snapshot where everything that is happening at that moment is reported to the terminal. It says that in this moment, you're streaming frame 65. The frames per second is 55, and the quality (q) is 12. The lower the number for quality, the better the video. The higher the number, the worse. The size of the chunk of data being streamed is also listed in kB with size. The time is the time from when the live stream started, and the bitrate is how many kbits per second yu're streaming. speed is how fast everything is streaming. 

## Conclusion
While this guide doesn't tell you everything about all concepts (for example quantization, which is too complicated to include in this post) I hope that this walkthrough helped explain what things are a bit more.
To help you further, you can go on [our Community](https://community.api.video/) or [start building](https://dashboard.api.video/register) now.



What do my video output live stream details from ffmpeg mean ? 

What do my video output live stream details from ffmpeg mean?

large_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

medium_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

small_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

thumbnail_what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

what-do-my-video-output-live-stream-details-from-ffmpeg-mean.png

Video list

doug.jpg

twitter-brands.svg

Doug Twitter

Doug Sillars

large_api-video-blog-tutorials.png

medium_api-video-blog-tutorials.png

small_api-video-blog-tutorials.png

thumbnail_api-video-blog-tutorials.png

api-video-blog-tutorials.png

Tutorials

NodeJS

With your api.video livestream you can have video streaming out to the internet in no time. (If you don't have an api.video livestream yet, here's how to [create a livestream](https://api.video/blog/tutorials/live-stream-tutorial/).) We have demos on how to stream with your [browser](https://api.video/blog/tutorials/live-stream-tutorial/), [Zoom](https://api.video/blog/video-trends/start-remote-work-with-online-video-and-zoom-us/), [OBS](https://api.video/blog/tutorials/live-stream-tutorial/), with a [Raspberry Pi](https://api.video/blog/tutorials/video-streaming-with-a-raspberry-pi/), and you can expect more tutorials in the future!  Every stream that you create has a player that can be embeded on any webpage.  

In this post, we'll create a very simple webpage to host your livestream using NodeJS. This page will show the current livestream (if it is broadcasting), and a few recent recordings from the stream.



## Is the Stream Broadcasting?

When you request the [live stream status](https://docs.api.video/reference#get_live-streams-livestreamid) API endpoint, you'll receive a JSON response with a lot of details about your stream.

Here the response from the live stream:

```json
{
  liveStreamId: 'li400mYKSgQ6xs7taUeSaE',
  streamKey: '30087931-229e-42cf-b5f9',
  name: 'Live Stream',
  broadcasting: false,
  assets: {
    iframe: '<iframe src="https://embed.api.video/live/li400mYKSgQ6xs7taUeSaE" width="100%" height="100%" frameborder="0" scrolling="no" allowfullscreen="true"></iframe>',
    player: 'https://embed.api.video/live/li400mYKSgQ6xs7taUeSaE',
    hls: 'https://live.api.video/li400mYKSgQ6xs7taUeSaE.m3u8',
    thumbnail: 'https://cdn.api.video/live/li400mYKSgQ6xs7taUeSaE/thumbnail.jpg'
  }
}
```


You'll notice the broadcasting parameter.  If this value is true - the stream is live! At the moment I made this request, the livestream was not active.

Our Node application calls the live status, and if the stream is broadcasting, the site is served an iframe with the video, and if not, we serve the thumbnail image:

```
		if(broadcasting){
			liveResult = "Here is your Live Video:";
			videoIframe = "iframe src=\""+liveStream.assets.player+"#autoplay\" width=\"100%\" height=\"50%\" frameborder=\"0\" scrolling=\"no\" allowfullscreen=\true\"";
		}else{
			videoIframe = "img src ="+liveStream.assets.thumbnail +" width=\"100%\"";
		}

```

If you look closely, I don't use the exact iframe returned in the response. I build an iframe with the player url, and I add '#autoplay' to the end of the url. This tells the player to autoplay the video as soon as it loads.

## Video recordings

When the livestream ends, the video is re-encoded as video on demand (VOD). These videos have a 'liveStreamId' parameter linking them back to the original livestream.

If we request the [video search](https://docs.api.video/reference#list-videos) endpoint with the livestreamId, we get a list of all the videos that have been saved from the stream.  In the below code, we sort by newest videos first, and save the first 5 (most recent) videos into an array. This array is used by Pug and Express to create a list of the videos.

```
	//Now we can get the most recent VOD from the stream.
		    let videoList = client.videos.search({'liveStreamId': liveStreamId, sortOrder: 'desc', sortBy: 'publishedAt'});
			videoList.then(function(liveStreamList){
				//console.log(liveStreamList);

				//loop through all the videos, we'll show 5, and if there are more than 20 - delete them
				
				for(i=0; i< liveStreamList.length; i++){
					if(i<5){
						var iframe = "iframe src=\""+liveStreamList[i].assets.player+"\" width=\"50%\" height=\"25%\" frameborder=\"0\" scrolling=\"no\" allowfullscreen=\true\"";
						var name = liveStreamList[i].title;
						var data = {
							"name": name, 
							"iframe":iframe
						};
						recordedVideos.push(data);
					}else if(i>20){
						//delete the videos...
						var videoId = liveStreamList[i].videoId;
						let deleteVid = client.videos.delete(videoId);
						deleteVid.then(function(statusCode) {
		  					console.log(videoId, "deleted");
		  					console.log(statusCode);
						}).catch(function(error) {
		  					console.error(error);
						});	
					}


				}
				//now we have an array with all the videos to show:
				console.log(recordedVideos);
				return res.render('live', {liveResult, videoIframe, recordedVideos});
			}).catch((err) => {
				console.log(err);
			});
```

NB the second part of this if/else.  If there are over 20 videos, we delete all of the oldest videos (ensuring there are never more than 20 stored in my account). This is my *super simplistic* approach to remove older recordings from my account, but is not unlike security systems that delete all videos over X days old.

That's it! Thank you for reading. 

How are you using live video in your projects?  We'd love to know - leave a note over on our [community forum](https://community.api.video). If you haven't created your api.video account yet, you can do that in just a few seconds [here](https://dashboard.api.video/register). 

Watching a Livestream

large_television-40695101920.jpg

medium_television-40695101920.jpg

small_television-40695101920.jpg

thumbnail_television-40695101920.jpg

watch-a-livestream.jpg

Announcements

small_cedric.webp

thumbnail_cedric.webp

cedric.webp

Cédric Montet

Company News

AI, models, assistants, agents and prompts sound and look magical on social networks. But from the countless conversations we have had with startups and Enterprise customers over the last months, we have realized that the gap between existing video workflow and concrete use cases is real.

&nbsp;

Whether it’s getting transcripts in seconds, changing the language of your video or being able to search for items in your content – the benefits, value or potential savings you see and can get with AI are obvious.

&nbsp;

This is why we are thrilled to unveil how we are bringing AI into video, where we are heading and how you can get more value from it!

## What does the future of video streaming look like?

AI will completely upend video consumption as we know it today. We envision a world where video is not a content piece anymore which you can only start and stop – because AI will make videos discoverable and universally understandable. More than ever it resonates fully with our core mission – connecting people through their cameras.

&nbsp;

Since Day 1, we are building api.video to make your use cases, first, a reality and, then, a success.

&nbsp;

Back in the days, the fragmented video streaming market, the number of services you had to aggregate, the lack of innovation by the incumbents was preventing you from joining our video-first world, while your efforts and budgets were consumed by maintaining mashups instead of focusing on what matters. 

&nbsp;

This is why we have built the video backbone and seeing that, now, you host 2.5 millions videos on api.video and deliver dozens of millions videos every month demonstrate why simplicity, exceptional performances and an E2E approach were key.


&nbsp;

What’s happening today with AI, the efforts it requires to benchmark, setup and customize LLMs, to run and maintain them, the Infrastructure you have to build to execute them before getting any value, remind us of the state of the streaming market when we had launched api.video.

&nbsp;

Helping you to benefit from the AI revolution by abstracting all the underlying complexity became obvious and a natural next step in our evolution.

&nbsp;

To get there and to bring you there, here is what we are building, api.video’s very first – **Video Model Hub**.

&nbsp;

Let’s start with the root principles encompassing our work on this Hub:

&nbsp;

- **Open-source models over private ones.** We are long time believers and contributors of the open source community and we have chosen to run only models that are available publicly.

- **Your data is… your data.** You already trust us to host and deliver sensitive content,and this is exactly why we won’t train our own models or sell the data generated from your videos to anyone.

- **Privacy at the core.** We host and execute the models inside our own infrastructure. Your data remains inside the api.video infrastructure. Your videos or data is not sent to external partner, SaaS providers like OpenAI, or others.

- **We believe that every organization will eventually build their own models to leverage their unique data sets**. Next year, alongside the ability to run public and open source models on api.video, we will offer you the ability to host your own.

- **One API to rule them all.** As we have already replaced a dozen of services and vendors under GET/videos to make your videos available worldwide in seconds, we are abstracting all the complex work of executing LLMs behind clear and straightforward endpoints.

&nbsp;

At a glance, api.video brings to you today the power of AI by:

- Leveraging its years of experience in video management and building infrastructure, with GPUs.
- Combining with the most advanced LLMs available on the market.

so that you get more value from your videos in a few lines of code!

&nbsp;

## Here are the first three new features from the Video Model Hub

### 1. Summarization of long-form videos

Upload a video in English, Spanish, French, Portuguese or Chinese and get a text summary of 120 characters with the 3 key takeaways discussed in the video (within seconds!). Ideal to index or preview long form recorded meetings or courses!

&nbsp;

_The video below gives you a glimpse into AI summarization._


### 2. Transcription of your video in seconds

Get a transcript (in .VTTformat) from your uploaded video with the timecodes and offer the captions to your audiences. The best part? The api.video player supports VTT files so there’s nothing new that you need to build on your side!
This also makes your content more inclusive by making it accessible to people with hearing impairments. 


### 3. Translation of subtitles in 5 different languages (and more to come!)

Is your video in English? You can get Spanish and French subtitles. In Chinese? You will get English subtitles and three more languages. In short, this is the perfect solution to spread your message to your international audience!

&nbsp;

_Check out the video below for a glimpse into AI translation._ 


_Try it for yourself now by uploading a video in your [sandbox account](https://dashboard.api.video/overview)._ 

## What’s next

From video classification, alteration, generation, audio alteration, metadata generation, we have identified and plan to implement more than 200 additional use cases and features within the upcoming months!

&nbsp;

From next year, we will let you bring your own models on the infrastructure to get even better and more personalized outcomes.

&nbsp;

This is a long-journey and we’re only getting started. The whole team is thrilled to be at the forefront of this next chapter!


AI  >< Video: The Video Model Hub unveiled

small_AI thought leadership.png

thumbnail_AI thought leadership.png

AI-thought-leadership.png

Arushi Gupta

Product updates

Our mission at api.video has always been to turn your use cases into successful realities. We built the video backbone with this goal in mind, and today, you host 2.5 million videos and deliver tens of millions more!

&nbsp;

Helping you harness the power of AI, while simplifying the problems that come along with, was the next direction to take. 

&nbsp;

To achieve this and to support your journey, we bring to you –  **AI features for your videos.**



## What are the AI video features?

### 1. AI video transcription & captions

With AI transcription, you can convert spoken words in your videos into written text, providing an accurate, searchable record of your content. This means you can easily create subtitles, enhancing accessibility for the hearing impaired or getting a written record of your meetings or events. 






### 2. AI video translation

With AI translation, you can translate your videos into multiple languages, ensuring that viewers from different linguistic backgrounds can understand and engage with your content. This can be useful if you’re hosting an international virtual event, creating multilingual educational materials, or expanding your marketing reach.

&nbsp;

Translation will be available for 5 languages – French, English, Spanish, Portuguese and Chinese.


### 3. AI video summarization

Using AI summarization, you can distill lengthy content into concise, digestible summaries, allowing viewers to quickly grasp the key points without watching the entire video with AI video summarization. This can prove invaluable for educational lectures, business presentations, and lengthy seminars where time is of the essence.

## Do you really need this?

Well, the potential [uses of our AI features](https://api.video/use-cases/generative-ai/) are limitless! From e-learning and e-commerce to online streaming, these [AI features](https://api.video/product/ai-features/) are going to be the difference between ordinary and excellent. What can you do really?

&nbsp;

- Summarize educational lectures for a quick review
- Translate virtual events into multiple languages for global reach
- Transcribe video interviews for easy reference
- And so much more!

## More than just encoding, hosting, and delivery

We’ve gone beyond the basics of video hosting and delivery to give you features that are a step up from the ordinary. Now, not only can you ensure that your customers get video delivery at minimal cost, but also that they get the flexibility to offer content  in their preferred language and in a crisp manner. 

&nbsp;

Additionally, when the features are in production, you may also get the option of adding them via your API. This means that you get complete power to add the features you like without having to go through the dashboard.

## Your videos stay private to you

Your data is yours, and we respect that. You already trust us to host and deliver your most sensitive content, which is why we will never train our own models or sell data generated from your videos. Privacy is at the core of everything we do, and that’s why we host and execute the models within our own infrastructure. Your data stays entirely within the api.video ecosystem and is never sent to external partners, SaaS providers like OpenAI or others.

&nbsp;

_Our CEO and founder, Cedric Montet, speaks more about privacy being our topmost concern in his latest [blog](https://api.video/blog/company/video-model-hub-ai/)._

## Give it a try!

See a glimpse how these AI features look like with a free sandbox account! [Upload a video now in your dashboard](https://dashboard.api.video/overview) and witness AI make a difference to your videos. As this is a Beta release, any feedback, questions, doubts on the features are welcome and much appreciated.

Create a sandbox account now 

Lights, Camera, AI Action! Discover our AI features

Introducing AI features to compliment your videos

small_Introducing AI features to compliment your videos.png

thumbnail_Introducing AI features to compliment your videos.png

AI-features-to-compliment-your-videos.png

AI >< Video: The Video Model Hub unveiled

How to integrate AI facial expression detection to control your video – Javascript 

5 AI tools that will save you 10+ hours on video production

Lights, Camera, AI Action! 

In a world where time is money and creativity reigns supreme, AI tools that convert text to video are game-changers! These AI-powered wonders are designed to save your day—and countless hours. Join us as we unveil the top 5 AI video tools that can dramatically cut down your video creation and editing time. From transforming text into captivating visuals to adding that perfect touch of magic to your edits, these tools are set to become your new best friends in the digital playground!

## **1. CapCut**

At number 1, we have Capcut. 

&nbsp;

[CapCut](https://www.capcut.com/) surged in popularity due to its user-friendly interface and powerful editing features, making it a favorite among video creators. Its integration with TikTok and social media platforms is also reason for its widespread adoption.

&nbsp;

The best part about using CapCut is that most of these important features are available for free, which kicks out budget-constraints out of the window. CapCut uses AI to empower users to create stunning videos without needing any fancy editing experience. With mobile and desktop apps, as well as a web version, CapCut can easily be used on the go. 

&nbsp;

CapCut offers a range of AI-driven features that are very, very impressive. With 'Smart Cuts & Trims', CapCut's AI automatically detects shaky parts, long pauses, or unnecessary bits, suggesting cuts to tighten up your video and eliminate boring footage. 'Effortless Audio Magic' helps isolate voices and enhance audio quality, making your message crystal clear even if there’s noisy background interference. 'Instant Captions' auto-generate captions, making your videos more accessible and saving you time and effort. Moreover, if your filming didn't capture the perfect lighting, 'Light Like a Pro' allows CapCut’s AI to adjust lighting in your videos post-production, creating the ideal mood.

What I most love about CapCut AI video generator is its AI-driven background removal feature which eliminates the need for a green screen, allowing me to replace any background with an image or color of my choice.

&nbsp;

Something to note about Capcut is that the free version includes watermarks on exported videos. which can be removed only with a paid subscription. This may potentially pose a barrier for you if you’re a budget-conscious user.

## **2. Runway**

[Runway](https://runwayml.com/) can be called a virtual sorcerer's wand for video editing! It has the potential to be an entire AI video production studio, so it can basically create an entire movie without filming a single frame — if you have the time, that is. Runway can generate videos and animations from simple text prompts, create animations from still images, generate lip-syncing avatars based on a recording of your voice, and even allow you to train your own AI models. 

&nbsp;

Runway might be a little intimidating in the beginning. That’s because it has an array of features that take time to understand, execute and implement, and that could be slightly time-taking and if I can say, complex.

Another aspect to note is that while Runway offers a range of pricing plans, the advanced features that truly set it apart are locked behind higher-tier subscriptions, making it less accessible for small businesses or individual creators on a tight budget.

## **3. Invideo AI**

[Invideo AI](https://invideo.io/ai/) is an excellent tool for generating a starting point for your videos. It helps you to quickly create a basic structure for your videos, which can then be further customized and polished. The  features allow you to input simple prompts, from which the tool generates an initial draft that includes relevant footage, transitions, and effects. 

&nbsp;

A few standout features of Invideo AI are an extensive stock media library that offers a range of options to choose from, AI-powered script generation which creates a complete script for your videos and, voiceover addition. Their AI voiceover generator is particularly good because it has an option to add your own voice which it then uses to narrate scripts in. The narration is also decent, with multiple accents and focussing well on the feeling, intent, and emotion behind each word.

Overall, the tool is great for YouTubers and people who want to start off their video making journey with minimal effort. 

&nbsp;

One of the shortcomings of Invideo AI is its dependency on a stable internet connection, as the tool is cloud-based. This can be a significant limitation for users in areas with unreliable internet access. I also noticed that rendering videos on Invideo AI can sometimes be slow, especially for longer or high-resolution projects, so keep enough time in hand before you expect a final video. 

## **4. Wondershare Filmora**

[Wondershare Filmora](https://filmora.wondershare.net/de/ad/videobearbeitungsprogramm-brand.html) has been a staple in the video editing world for over 15 years. Despite its long history, it's continuously evolving, with a suite of AI features that enhance video quality without requiring professional training or extensive time investments. 

&nbsp;

Filmora is a popular choice among full-time influencers and video editors for advanced-level video editing. It is also a preference because the tool keeps evolving based on new technological developments, and therefore is easily able to meet the demanding standards of modern digital media.

&nbsp;

Some of Filmora’s features include 'AI smart cutout', which allows you to cut objects out of scenes simply by painting their area on the screen, and 'AI audio denoise', which helps control background noise. 'AI audio stretch' lets you extend music to match scene transitions, while AI motion tracking enables face blurring for privacy or fun effects. They also have AI remove background which allows you to create imaginative scenes without hefty budgets. 

Wondershare also provides detailed explanations of these features on their website, so if you ever face a problem using any of them, be sure to check their dedicated pages. 


&nbsp;

However, do note that Filmora's performance can be sluggish when handling large files or high-resolution videos, leading to slower rendering times and potential crashes at times.

## **5. Canva Magic Studio**

Easy to use. Neat interface. Quick results. 

&nbsp;

Those were the three things I liked best about [Canva Magic Studio](https://www.canva.com/magic/). 

&nbsp;

Canva Magic Studio, though currently in a very basic state, looks very promising as an AI video tool. Having been a fan of Canva for infographics, presentations and social posts since the past few years, I have full confidence that their AI video tool will evolve too. 

&nbsp;

As of now, Canva Magic Studio gives you amazing results when it comes to video quality and is also very quick to generate the final video. The good video quality comes from its integration with [Runway](https://zapier.com/blog/best-ai-video-generator/#runway) (our #2 in the list).

&nbsp;

To show you an example, I used the prompt: "A cute rabbit watching a video on a laptop”. And the [result](https://www.canva.com/design/DAGGCqwwf4A/GS7q42J5C9nx26gxJGR9lA/watch) was surprisingly very apt!

Canva Magic Studio is a great option if you want to bulk-create your social media posts – create a chunk of Instagram reels or Facebook ads in a go or create similar kinds of videos using a single template. The template options Canva offers have a wide range and are decent for beginners to start with.

## Sora – Open AI’s video generator

Open AI sparked a wave of excitement around AI with ChatGPT, demonstrating its incredible potential in natural language processing. Now, it's taking things a step further with [Sora, an innovative AI](https://openai.com/index/sora/) video creator.

Open AI recently shared a series of tweets showing the potential of Sora and I must say, I am amazed. Be it [several giant mammoths](https://x.com/OpenAI/status/1758192960116785459) rushing through a snowy meadow or a [fluffy monster](https://x.com/OpenAI/status/1758192964222988506) behind a melted candle, the kind of videos Sora will be able to create is astounding! It looks like it will be able to take detailed prompts into consideration, just like it did with ChatGPT, to create exactly what the user needs.

&nbsp;

As Sora continues to evolve and improve, it will undoubtedly shape the future of video production, showing way to new forms of storytelling and engagement in the years to come.

## AI is great – but it cannot replace you

While these AI video tools can ease the load for you and save some time, they cannot really take the place of your creativity, intuition, and emotional intelligence. In the end, it is your perspective, personal touch, and experiences that shape the videos you create. 

&nbsp;

_If you’re finding a place to store, host and deliver your [AI generated videos](https://api.video/use-cases/generative-ai/), api.video could be just what you’re looking for. With quick video upload options, flexible hosting capabilities, ability to add watermarks and keep videos private, you can choose api.video for all your video needs. If you would like to test out our features, [sign-up](https://dashboard.api.video/register) for a sandbox account now and you wound’t be disappointed._

small_7 AI Tools for Quick Video Creation.jpg

thumbnail_7 AI Tools for Quick Video Creation.jpg

7-AI-Tools-for-Quick-Video-Creation.jpg

5 ways the human eye impacts how we do video compression

7 eye-opening stats that prove the impact of product videos in e-commerce

5 ways to utilize product videos to boost your e-commerce sales

Yohann Martzolff

React

In today's tech-driven world, enhancing user interaction and creating immersive experiences is crucial. One exciting way to achieve this is by integrating AI facial expression detection into your video content.  In this blog, we'll guide you through the steps to <u>integrate AI facial expression detection to control your videos using React</u>. 

&nbsp;

Let's start.

&nbsp;

_*To learn how to do the integration with Javascript, head to this [blog](https://api.video/blog/tutorials/AI-facial-expression-detection-javascript/).*_

## 1. Create a minimal React app

As the first step, we need to create a new React app. Run the following command for the same. 

Now add the MediaPipe and the api.video React player packages

## 2. Add a video player

You’re going to use the official [api.video’s React player](https://github.com/apivideo/api.video-react-player) in this application, but feel free to use your favorite one and adapt the callbacks to it.

&nbsp;

Replace the code inside `App.tsx` with the following.
The `playerRef` allows us to control the [api.video player](https://api.video/product/player/).

Run your application with the npm run start command and go to `http://localhost:3000/`, you should see a video that you can play, pause or mute. 

## 3. Record your facial expressions

To control the player with your facial expressions, you need to stream your face through your webcam and pass this streamed data to the trained machine-learning model.

&nbsp;

Add a `<video />` HTML tag to `App.tsx`. 
Its `webcamRef` allows the video element to be targeted.
Apply a `display: "none"` inline style to hide the element, and an `autoPlay` attribute so the webcam stream time is updated. 

&nbsp;

**Note:** You can remove the `display: "none"` style and customize your CSS, it doesn’t matter.

## 4. Load the machine-learning model

Before we can use the machine-learning model, we must wait for it to finish loading. These models can be large and take a moment to get everything needed to run.

&nbsp;

Put this logic inside a `useEffect` and store the `faceLandmarker` class as a React stateful value.

## 5. Start detecting the webcam stream

Once the machine-learning model has been loaded and the `faceLandmarker` class has been instantiated, you can start detecting the stream.

&nbsp;

Create a new `useEffect`

Run your application with the `npm run start` command, go to `http://localhost:3000/`, and open your console. You should see a lot of console logs appearing each millisecond.

&nbsp;

Each one of them logs an array of objects of type:

## 6. Use the facial expression detection results

You now have access to a list of facial expression detection results. Use them to do whatever you want and in this case, to control your video player.

Let’s assume that you want to play the video when you smile, and pause it when you do a kiss.

Check the facial expression detection results on each frame and play or pause your video depending on the score of the wanted result by replacing your console.log with the following.

Once again, run your application with the `npm run start` command, go to `http://localhost:3000/`, and try to smile at your webcam. It should trigger the player’s `play` callback and play the video. Then, try to do a kiss ( 😗 ) in front of your webcam, and the video should pause.

## 7. Display the results

It can be helpful to have a visual representation of the detection results. Display these results as a list and highlight the detected facial expressions by adding a `<ul />` HTML tag to your JSX and creating a new ref to target it.

Add the code to update the HTML according to the results.

Now, you should see a list of results displayed in your application, with a green highlight on the ones with more than 60% confidence.

## 8. Code samples

Check the full `App.tsx` code below.

We hope this blog helped you understand the coding behind integrating AI facial expression detection to your videos.

 &nbsp;

_For any questions or doubts you may have, chat with us on our [website](https://api.video/contact/). If you would like to try out api.video for your videos, [sign-up](https://dashboard.api.video/register) for a free sandbox account here._

How to build AI facial expression detection into your videos with React

small_AI face recognition (React) - Version 2.png

thumbnail_AI face recognition (React) - Version 2.png

AI face recognition (React) - Version 2.png

Create a Next.js 13 & React 18 video app

How to create a Next.js 13 & React 18 video app

React Native Livestream Module: building the sample app

How to upload a video with React and NodeJS

Featured article

# api.video blog

api.video Blog

Follow the api.video Blog to explore tutorials, video use cases, and learn about on-demand and live-streaming video updates.

Video on demand

Live streaming

Analytics

Video infrastructure

Video player

AI features

See all features

Online learning & Corporate training

Social & Entertainment

Marketplace & E-commerce

Communication & UGC

Generative AI

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Customer success stories

Help center

Live stream status