Normal

doug.jpg

twitter-brands.svg

Doug Twitter

Doug Sillars

large_api-video-blog-tutorials.png

medium_api-video-blog-tutorials.png

small_api-video-blog-tutorials.png

thumbnail_api-video-blog-tutorials.png

api-video-blog-tutorials.png

Tutorials

ffmpeg

## 🎶🎶🎶 Now here's the story... 🎶🎶🎶

As a child, the Brady Bunch was a regular part of my afternoon. It wasn't a new show, but every day after school there were cartoons and the Brady Bunch reruns on TV.

For those not from the USA (it was interesting that at api.video, it was just the Americans who knew the show), the story revolves around a single dad with 3 boys who marries a single mom with 3 girls. They hired a wacky housekeeper, and together the nine characters were the Brady Bunch. Years of laugh track hijinks followed their adventures.

## The opening

The image above is a short GIF, but each character is looking around and smiling at the other members of the Brady clan. It's an iconic US television look, and I thought it would be fun to re-create a "Brady Bunch" video using FFMPEG to interlace 9 different videos into that familiar 9-grid pattern.


> A huge Thank You! to Anne-Sophie and Cedric for recording all of our colleagues (and of course to my api.video colleagues for going along with this fun project).

## Starting my FFMPEG command

After pitching this idea to the team, I woke up a few days later to a number of videos shared in a Slack channel. Woo hoo! Here we go!

When composing an FFMPEG command, the first step is to input all of the files you will be using. We'll do a **-filter_complex** to resize and build the video layout, and then output the video in **done.mp4**.

```
ffmpeg -i sorin.mp4 -i olia.mp4	-i maria.mp4  -i josephine.mp4 
-i yohann.mp4 -i anne-sophie.mp4 -i brady_erikka.mov 
-i cedric.mov -i pierre.mov  -i T3.mov  -i fancy.mov 
-i romain.mov	-i jb.mov 
-filter_complex '<we'll get here in a second>' 
done.mp4

```

For the filter complex, we'll generate a 4K video.   4K is 3840x2160, and is super convenient as it allows us to place 9 1280x720 videos into the "Brady grid":




large_screenshot-2021-09-10-at-08-46-09.png

medium_screenshot-2021-09-10-at-08-46-09.png

small_screenshot-2021-09-10-at-08-46-09.png

thumbnail_screenshot-2021-09-10-at-08-46-09.png

sample-9-grid-pattern-video.png

So, each of the videos needs to be 1280x720.  And here is where we came across our first complication.  Half of the videos were recorded in landscape mode:


large_cedric-api-video.png

medium_cedric-api-video.png

small_cedric-api-video.png

thumbnail_cedric-api-video.png

cedric-api-video.png

..but half were recorded in portrait mode....

large_screenshot-2021-09-10-at-09-05-07.png

medium_screenshot-2021-09-10-at-09-05-07.png

small_screenshot-2021-09-10-at-09-05-07.png

thumbnail_screenshot-2021-09-10-at-09-05-07.png

screenshot-2021-09-10-at-09-05-07.png

The landsacpe videos are all the right 16:9 proportion - but are 1080p - so will have to be rescaled to 1280x720.

The portrait videos are 1080x1920, so we'll have to do a crop,and then slightly upscale them.  But we're using FFMPEG, so this will be a snap!

large_screenshot-2021-09-10-at-09-08-16.png

medium_screenshot-2021-09-10-at-09-08-16.png

small_screenshot-2021-09-10-at-09-08-16.png

thumbnail_screenshot-2021-09-10-at-09-08-16.png

screenshot-2021-09-10-at-09-08-16.png

### Building the palette

We want our video to be 4k, so we first input a video and resize  it to 4K. It doesn't matter which video, since we will draw on top of it. I'm using the center video - the 5th video (index 4) as the background. It is resized to 4K, and named main.

```
[4]scale=3840:2160[main]
```

### Cropping inside the filter_complex

For each video, we can build something like this:

```
[0:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[0];
```

Let's walk through what this command does:

```[0:v]``` - grab the video from input 0 (thats sorin.mp4). 

```crop=in_w:in_w*9/16:0:in_h*0.2```  - the crop command, which has 4 parameters video width: video height:x start: y start
* video width: in_w - take the width of the input video
* video height: in_w\*9/16 - take the width, and multiply by 9/16 to keep the video  in the 9:16 format. (this will do nothing to the landscape videos, but will crop the portrait ones).
* x_start: 0.  Grab from the far left of the video
* y_start: in_h\*0.2 Get the height of the video, and begin the crop 20% down. The percentage can be modified slightly on each video - as some of our colleagues are taller than others.

Now we've cropped each video.  The landscape videos will be mostly unchanged, but the portrait videos will be cropped into a landscape view around the face.

```scale=1280:720[0]```  Resize the 16:9 video to 720p. 

```[0]```. Rename this modified video [0].  This is a variable that we can call later for placement.

We have to do this for each video, so our **-filter_complex** begins growing:

```
[4]scale=3840:2160[main];
[0:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[0];
[1:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[1];
[2:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[2];
[3:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[3];
[4:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[4];
[5:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[5];
[6:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[6];
[7:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[7];
[8:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[8];
[9:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[9];
[10:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[10];
[11:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[11];
[12:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[12];
[13:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[13];
[14:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[14];
[15:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[15]

```

It took a few tests to lay out each person the way I wanted - mostly changing the **in_h** multiplier to properly center the face.

### Layout of the videos

The next bit of the **filter_complex** lays out each of the videos into the grid.

```
[main][0]overlay=0:0[main0]
```

This tells FFMPEG to take the "main" 4K video, and overlay video "0" at (0,0).  Rename this intermediate video "main0". We now repeat this for each video, updating the variables and the (x,y) coordinates for each video.

Only one video will remain at the center (index 4) - the video of our CEO Cedric.


```
[main][0]overlay=0:0[main0];
[main0][1]overlay=1280:0[main1];
[main1][2]overlay=2560:0[main2];
[main2][3]overlay=0:720[main3];
[main3][4]overlay=1280:720[main4];
[main4][5]overlay=2560:720[main5];
[main5][6]overlay=0:1440[main6];
[main6][7]overlay=1280:1440[main7];
[main7][8]overlay=2560:1440[main8];
[main8][9]overlay=0:0[main9];
[main9][10]overlay=1280:0[main10];
[main10][11]overlay=2560:0[main11];
[main11][12]overlay=0:720[main12];
[main12][13]overlay=2560:720[main13];
[main13][14]overlay=0:1440[main14];
[main14][15]overlay=1280:1440[main15];
[main15][16]overlay=2560:1440
```

Now, those of you with keen eyes will notice that video 9 will sit on top of video 0 (10 on top of 1, etc.). I have more videos to add to the grid of 9 than places for them.  We'll fix that in our next step.

### Video timings

Each video has a slightly different length. When a video reaches an end, the video freezes, while the rest continue playing. I'd like all the videos to move for the entire clip. Looking at all of them, the shortest video is 9 seconds long. (Thanks to Pierre for defining our video's length 🤣).

We want each video to be 9 seconds long. We can fix this by changing the input parameter for each video:

```-t 9 -i  doug.mp4```

```-t 9``` says, "grab only the first 9 seconds of doug.mp4." Adding this to each video - now the overall video is exactly 9 seconds long!

### More video timing - desynchronizing the movements

Having watched these videos a lot, everyone moved their heads in the same pattern of motions, at approximately the same rate. When they are all combined into one video, the results were eerily robotic. Usually, you want all your videos in sync, but in this case, we want to actually artificially de-synchronize our videos a bit.  

To do this, we can grab 9 seconds of each video - but from a different start point for each video:

```
 -ss 5 -t 9 -i olia.mp4
```

```-ss 4``` start offset of 4 seconds
```-t 9``` 9 seconds of video

This command tells FFMpeg: "input 9 seconds of Olia.mp4, starting at 5 seconds into the video,". As long as we are careful and the SS + t values are not longer than the video's length, we'll be fine (I created a spreadsheet). So we input different -ss values for each video.

### Timing part 3: More than 9 videos

As noted, we have more than 9 videos. We could have just stopped at 9, but I wanted to include everyone. The easiest way to do that is to have two rounds of people in the 9 grid. We'll put Cedric in the middle for 18 seconds (his video was over 20s, so we are ok), and array 2 sets of 8 people around him.

```
 -ss 5 -t 9 -i erikka.mp4
 
 -itsoffset 9 -t 9 -i romain.mov
```

Erikka's video will be 9 seconds long (starting at time 5 from her video), and will be inserted into the final video at t=0.  Romain's video will be 9 seconds long as well, but the ```-itsoffset 9``` parameter says "Begin adding this video 9 seconds into the video."

So the result is Erikka is on the video for time 0-9s, and at t=9, Romain's video replaces her for t=9-18.

At this point, I'm super happy with the video. but it is just missing something... and that is (of course) a text overlay.

## The api.video Bunch

There is a [free Brady Bunch font](https://www.dafont.com/brady-bunch.font).  I downloaded it, and placed it in the same directory as the videos.  When you add text, you specify the location, the size and the font. I'll have 3 lines of text. The comma seperate each line, but for readability, I'll add a few lines between each one:

```
drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='The':x=(3840)/2-200:y=(2160)/2-600,

drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='api.video':x=(3840)/2-400:y=(2160)/2-400,

drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='Bunch':x=(3840)/2-200:y=(2160)/2+200
```


* ```drawtext=enable``` Turn on the drawing (I guess)
* ```fontfile=BradBunR.ttf``` Load in the Brady Bunch font!
* ```fontsize=400``` It is a 4K video, so we can go pretty large.
* ```borderw=10:bordercolor=white``` The default font color is black, but we want a nice retro white border around it.
* ```text='api.video'``` The text to output.
* ```x=(3840)/2-400:y=(2160)/2-400``` Finally the (x,y) coordinates to start the text.  This took some trial and error to get the locations in the right spot, but I'm pretty happy with the result:



## The full FFMPEG command

We've walked through all the steps used to create the command, but we cannot avoid the inevitable: we have to show the full FFMPEG command. It follows the basic premise above:

1. Input all the videos (with time = 9, start offsets and playback offsets)
2. filter all the videos
3. resize and or crop all the videos
4. overlay them all on a 4K background
5. add the text
6. output the video

```
ffmpeg  -ss 10 -t 9 -i  sorin.mp4  -ss 5 -t 9 -i olia.mp4	-ss 4 -t 9 -i maria.mp4  -ss 5 -t 9 -i josephine.mp4 -ss 2 -t 18 -i cedric.mov -ss 1 -t 9 -i yohann.mp4 -ss 6 -t 9 -i anne-sophie.mp4 -ss 2 -t 9 -i brady_erikka.mov -ss 0 -t 9  -i pierre.mov  -itsoffset 9 -ss 3 -t 9 -i   T3.mov -itsoffset 9 -ss 6 -t 9  -i fancy.mov -ss 0 -itsoffset 9 -t 9 -i romain.mov	-itsoffset 9 -ss 11 -t 9 -i jb.mov -itsoffset 9 -ss 0 -t 9 -i doug.mp4 -itsoffset 9 -ss 0 -t 9 -i stephanie.mov -ss 10 -itsoffset 9 -t 9 -i  sorin.mp4  -ss 5 -itsoffset 9 -t 9 -i olia.mp4 -filter_complex '[4]scale=3840:2160[main];[0:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[0];[1:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[1];[2:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[2];[3:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[3];[4:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[4];[5:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[5];[6:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[6];[7:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[7];[8:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[8];[9:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[9];[10:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[10];[11:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[11];[12:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[12];[13:v]crop=in_w:in_w*9/16:0:in_h*0.1,scale=1280:720[13];[14:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[14];[15:v]crop=in_w:in_w*9/16:0:in_h*0.2,scale=1280:720[15];[main][0]overlay=0:0[main0];[main0][1]overlay=1280:0[main1];[main1][2]overlay=2560:0[main2];[main2][3]overlay=0:720[main3];[main3][4]overlay=1280:720[main4];[main4][5]overlay=2560:720[main5];[main5][6]overlay=0:1440[main6];[main6][7]overlay=1280:1440[main7];[main7][8]overlay=2560:1440[main8];[main8][9]overlay=0:0[main9];[main9][10]overlay=1280:0[main10];[main10][11]overlay=2560:0[main11];[main11][12]overlay=0:720[main12];[main12][13]overlay=2560:720[main13];[main13][14]overlay=0:1440[main14];[main14][15]overlay=1280:1440[main15];[main15][16]overlay=2560:1440[finalnotext];[finalnotext]drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='The':x=(3840)/2-200:y=(2160)/2-600,drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='api.video':x=(3840)/2-400:y=(2160)/2-400,drawtext=enable:fontfile=BradBunR.ttf:fontsize=400:borderw=10:bordercolor=white:text='Bunch':x=(3840)/2-200:y=(2160)/2+200'   apibunch.mp4
```

Yeah, it's a long one.  But the results speak for themselves - a fun video showing off the api.video team!

## Conclusion

Install FFMPEG!  Get 9 friends to film a video - and create your own "The <insert name> Bunch."  We hope you'll upload it to api.video and share the results with us!

Using FFMPEG to recreate the Brady Bunch introduction

large_bbtitle2.jpeg

medium_bbtitle2.jpeg

small_bbtitle2.jpeg

thumbnail_bbtitle2.jpeg

bbtitle2.jpeg

A very Brady video: Using FFMPEG to recreate the Brady Bunch introduction

Overlaying two videos with FFMPEG

[FFMPEG](https://api.video/what-is/ffmpeg) is a free and open source software project that offers many tools for video and audio processing. It's designed to run on a command line interface, and has many different libraries and programs to manipulate and handle video files.

## The Question

### How can I best overlay 2 live camera feeds with FFMPEG to produce one live stream to broadcast at api.video? I’m using a raspberry PI, and having a lot of latency issues.

We then began working on the FFMPEG command that he was using to build the livestream.

For testing purposes, I replaced the two livestreams with videos stored locally on my computer.  My goal is to create a video that looks like this: Die Hard with an Indiana Jones (and the Last Crusade) overlay.


video overlay of Diehard and Indiana Jones

large_screenshot-2021-08-23-at-18-20-16.png

medium_screenshot-2021-08-23-at-18-20-16.png

small_screenshot-2021-08-23-at-18-20-16.png

thumbnail_screenshot-2021-08-23-at-18-20-16.png

screenshot-2021-08-23-at-18-20-16.png



My command began with:



```
ffmpeg  -i DieHard.mp4 -i Indianajones.mp4
```

The ```-i``` tells ffmpeg that these are input files, and then the filename.  If you were using a livestream, or a remote video you could use a url.

The end of the ffmpeg command looks like:



```
-f flv rtmp://broadcast.api.video/s/23df9302-5897-4fca-8b27-0eb39ea67ab2
```

This says to FFMPEG, save the video output as flv and send it to my livestream endpoint.  When video begins arriving at this endpoint, your api.video stream will begin broadcasting whatever video is being sent. At this point, it will just take the first input (DieHard.mp4) and stream that file - ignoring Indiana Jones. I need to do some work to add in the Indiana Jones video...

If you want to test locally, you can change the output to:


```
test.mp4
```

To save the video locally to your computer.

### Initial command

```
ffmpeg  -i DieHard.mp4 -i Indianajones.mp4 test.mp4
```

This takes in the 2 videos, outputs just DieHard.mp4 and saves it into test.mp4.  We want to overlay the Indiana Jones video.

The first thing we’ll do is scale the Diehard video to 1080p (it is 1080p, but this will ensure that all future videos fit correctly), and we will give the output of this transformation the name ‘main’

```
 -filter_complex '[0:v]scale=-1:1080[main]’
```

This command says - take the video from input 1, and resize it to -1:1080. The 2nd value is pretty obvious - make the video 1080 high. The ‘-1’ on the horizontal axis says make the height 1080, and scale the width so that the video is not stretched in any way.  If the input video is 16:9,the output will be 1920x1080. If the input video is 4:3, the output will be 1440x1080.

Let’s resize Indiana Jones to 360p (again using the ‘-1’ to ensure that the video is not stretched), and name the resized video “overlay”.  We place a semicolon between the two commands:

```
 -filter_complex '[0:v]scale=-1:1080[main];[1:v]scale=-1:360[overlay]’
```

Now we have 2 correctly sized videos, let’s set the overlay on top of the main video.  In creating the overlay, we could hard code the coordinates, but we can also use variables. In this case, main_w and main_h are for the main video width and height, and overlay_w and overlay_h are assigned to the overlay video.

We'll place the overlay in the bottom right - 25 pixels from the right and 25 pixels from the bottom.  To do this, we’ll subtract 25 and the dimension of the overlay from the dimension of the main video. The command will look like:


```
[main][overlay]overlay=(main_w-overlay_w-25):(main_h-overlay_h-25)
```

The ```[main]``` video is used as the main video, and the ```[overlay]``` as the overlay, and then we use the variables to place the overlay video.

The full command now looks like:

```
ffmpeg  -i DieHard.mp4 -i Indianajones.mp4 -filter_complex '[0:v]scale=-1:720[main];[1:v]scale=-1:240[overlay];[main][overlay]overlay=(main_w-overlay_w-25):(main_h-overlay_h-25)' test.mp4

```

And the video looks exactly how we’d like it to look! The audio, however, is from the main video: Die Hard.  We can remove the audio (making the video silent) by only exporting the video from the output video.  To do this, we can use the ```map``` parameter.  We’ll name the overlay filter [out] and then just map the first track (track 0) to the output:


```
ffmpeg  -i DieHard.mp4 -i Indianajones.mp4 
-filter_complex '[0:v]scale=-1:720[main];[1:v]scale=-1:240[overlay];[main][overlay]overlay=(main_w-overlay_w-25):(main_h-overlay_h-25)[out]' -map '[out]'0:0  test.mp4
```

This results in a silent movie.  We can programmatically pick which audio track (track 1 in both cases) with a second ```-map``` command:

* ```-map 0:1``` selects the Die Hard audio (the original 0 index input, track 1)
* ```-map 1:1``` selects Indiana jones audio (the original 1 index input, track 1)



Here are the final 2 commands (with the stream parameters added in:

```
ffmpeg  -i DieHard.mp4 -i Indianajones.mp4 -filter_complex '[0:v]scale=-1:720[main];[1:v]scale=-1:240[overlay];[main][overlay]overlay=(main_w-overlay_w-25):(main_h-overlay_h-25)[out]' -map '[out]'0:0 -map 0:1 -f flv rtmp://broadcast.api.video/s/23df9302-5897-4fca-8b27-0eb39ea67ab2

ffmpeg  -i DieHard.mp4 -i Indianajones.mp4 -filter_complex '[0:v]scale=-1:720[main];[1:v]scale=-1:240[overlay];[main][overlay]overlay=(main_w-overlay_w-25):(main_h-overlay_h-25)[out]' -map '[out]'0:0 -map 1:1 -f flv rtmp://broadcast.api.video/s/23df9302-5897-4fca-8b27-0eb39ea67ab2
```

Here’s what the final result looks like (Die Hard audio version):



In order to create his live stream, our developer simply changed the input from 2 movie trailers to his 2 live streams, and it all *just worked*.

## Conclusion

As Indiana Jones famously did not say: “What is FFMPEG? Fortune and Glory kid. Fortune and Glory.”

And we are pretty sure John McClain never said “Yippiee Ki Yay FFMPEG!”

But that is because neither of them ever got the chance to play with the tooling that allowed us to create the video above.

In this post, we’ve talked through how to scale the dimensions of multiple input videos, how to overlay them, and then how to apply an audio track to the videos.  Finally, the video created by ffmpeg is streamed to api.video for live streaming consumption by your viewers.


Use FFMPEG to resize and overlay videos and live stream them

Use FFMPEG to resize and overlay two videos, and then live stream them

Announcements

small_cedric.webp

thumbnail_cedric.webp

cedric.webp

Cédric Montet

Company News

AI, models, assistants, agents and prompts sound and look magical on social networks. But from the countless conversations we have had with startups and Enterprise customers over the last months, we have realized that the gap between existing video workflow and concrete use cases is real.

&nbsp;

Whether it’s getting transcripts in seconds, changing the language of your video or being able to search for items in your content – the benefits, value or potential savings you see and can get with AI are obvious.

&nbsp;

This is why we are thrilled to unveil how we are bringing AI into video, where we are heading and how you can get more value from it!

## What does the future of video streaming look like?

AI will completely upend video consumption as we know it today. We envision a world where video is not a content piece anymore which you can only start and stop – because AI will make videos discoverable and universally understandable. More than ever it resonates fully with our core mission – connecting people through their cameras.

&nbsp;

Since Day 1, we are building api.video to make your use cases, first, a reality and, then, a success.

&nbsp;

Back in the days, the fragmented video streaming market, the number of services you had to aggregate, the lack of innovation by the incumbents was preventing you from joining our video-first world, while your efforts and budgets were consumed by maintaining mashups instead of focusing on what matters. 

&nbsp;

This is why we have built the video backbone and seeing that, now, you host 2.5 millions videos on api.video and deliver dozens of millions videos every month demonstrate why simplicity, exceptional performances and an E2E approach were key.


&nbsp;

What’s happening today with AI, the efforts it requires to benchmark, setup and customize LLMs, to run and maintain them, the Infrastructure you have to build to execute them before getting any value, remind us of the state of the streaming market when we had launched api.video.

&nbsp;

Helping you to benefit from the AI revolution by abstracting all the underlying complexity became obvious and a natural next step in our evolution.

&nbsp;

To get there and to bring you there, here is what we are building, api.video’s very first – **Video Model Hub**.

&nbsp;

Let’s start with the root principles encompassing our work on this Hub:

&nbsp;

- **Open-source models over private ones.** We are long time believers and contributors of the open source community and we have chosen to run only models that are available publicly.

- **Your data is… your data.** You already trust us to host and deliver sensitive content,and this is exactly why we won’t train our own models or sell the data generated from your videos to anyone.

- **Privacy at the core.** We host and execute the models inside our own infrastructure. Your data remains inside the api.video infrastructure. Your videos or data is not sent to external partner, SaaS providers like OpenAI, or others.

- **We believe that every organization will eventually build their own models to leverage their unique data sets**. Next year, alongside the ability to run public and open source models on api.video, we will offer you the ability to host your own.

- **One API to rule them all.** As we have already replaced a dozen of services and vendors under GET/videos to make your videos available worldwide in seconds, we are abstracting all the complex work of executing LLMs behind clear and straightforward endpoints.

&nbsp;

At a glance, api.video brings to you today the power of AI by:

- Leveraging its years of experience in video management and building infrastructure, with GPUs.
- Combining with the most advanced LLMs available on the market.

so that you get more value from your videos in a few lines of code!

&nbsp;

## Here are the first three new features from the Video Model Hub

### 1. Summarization of long-form videos

Upload a video in English, Spanish, French, Portuguese or Chinese and get a text summary of 120 characters with the 3 key takeaways discussed in the video (within seconds!). Ideal to index or preview long form recorded meetings or courses!

&nbsp;

_The video below gives you a glimpse into AI summarization._


### 2. Transcription of your video in seconds

Get a transcript (in .VTTformat) from your uploaded video with the timecodes and offer the captions to your audiences. The best part? The api.video player supports VTT files so there’s nothing new that you need to build on your side!
This also makes your content more inclusive by making it accessible to people with hearing impairments. 


### 3. Translation of subtitles in 5 different languages (and more to come!)

Is your video in English? You can get Spanish and French subtitles. In Chinese? You will get English subtitles and three more languages. In short, this is the perfect solution to spread your message to your international audience!

&nbsp;

_Check out the video below for a glimpse into AI translation._ 


_Try it for yourself now by uploading a video in your [sandbox account](https://dashboard.api.video/overview)._ 

## What’s next

From video classification, alteration, generation, audio alteration, metadata generation, we have identified and plan to implement more than 200 additional use cases and features within the upcoming months!

&nbsp;

From next year, we will let you bring your own models on the infrastructure to get even better and more personalized outcomes.

&nbsp;

This is a long-journey and we’re only getting started. The whole team is thrilled to be at the forefront of this next chapter!


AI  >< Video: The Video Model Hub unveiled

small_AI thought leadership.png

thumbnail_AI thought leadership.png

AI-thought-leadership.png

Arushi Gupta

Product updates

Our mission at api.video has always been to turn your use cases into successful realities. We built the video backbone with this goal in mind, and today, you host 2.5 million videos and deliver tens of millions more!

&nbsp;

Helping you harness the power of AI, while simplifying the problems that come along with, was the next direction to take. 

&nbsp;

To achieve this and to support your journey, we bring to you –  **AI features for your videos.**



## What are the AI video features?

### 1. AI video transcription & captions

With AI transcription, you can convert spoken words in your videos into written text, providing an accurate, searchable record of your content. This means you can easily create subtitles, enhancing accessibility for the hearing impaired or getting a written record of your meetings or events. 






### 2. AI video translation

With AI translation, you can translate your videos into multiple languages, ensuring that viewers from different linguistic backgrounds can understand and engage with your content. This can be useful if you’re hosting an international virtual event, creating multilingual educational materials, or expanding your marketing reach.

&nbsp;

Translation will be available for 5 languages – French, English, Spanish, Portuguese and Chinese.


### 3. AI video summarization

Using AI summarization, you can distill lengthy content into concise, digestible summaries, allowing viewers to quickly grasp the key points without watching the entire video with AI video summarization. This can prove invaluable for educational lectures, business presentations, and lengthy seminars where time is of the essence.

## Do you really need this?

Well, the potential [uses of our AI features](https://api.video/use-cases/generative-ai/) are limitless! From e-learning and e-commerce to online streaming, these [AI features](https://api.video/product/ai-features/) are going to be the difference between ordinary and excellent. What can you do really?

&nbsp;

- Summarize educational lectures for a quick review
- Translate virtual events into multiple languages for global reach
- Transcribe video interviews for easy reference
- And so much more!

## More than just encoding, hosting, and delivery

We’ve gone beyond the basics of video hosting and delivery to give you features that are a step up from the ordinary. Now, not only can you ensure that your customers get video delivery at minimal cost, but also that they get the flexibility to offer content  in their preferred language and in a crisp manner. 

&nbsp;

Additionally, when the features are in production, you may also get the option of adding them via your API. This means that you get complete power to add the features you like without having to go through the dashboard.

## Your videos stay private to you

Your data is yours, and we respect that. You already trust us to host and deliver your most sensitive content, which is why we will never train our own models or sell data generated from your videos. Privacy is at the core of everything we do, and that’s why we host and execute the models within our own infrastructure. Your data stays entirely within the api.video ecosystem and is never sent to external partners, SaaS providers like OpenAI or others.

&nbsp;

_Our CEO and founder, Cedric Montet, speaks more about privacy being our topmost concern in his latest [blog](https://api.video/blog/company/video-model-hub-ai/)._

## Give it a try!

See a glimpse how these AI features look like with a free sandbox account! [Upload a video now in your dashboard](https://dashboard.api.video/overview) and witness AI make a difference to your videos. As this is a Beta release, any feedback, questions, doubts on the features are welcome and much appreciated.

Create a sandbox account now 

Lights, Camera, AI Action! Discover our AI features

Introducing AI features to compliment your videos

small_Introducing AI features to compliment your videos.png

thumbnail_Introducing AI features to compliment your videos.png

AI-features-to-compliment-your-videos.png

AI >< Video: The Video Model Hub unveiled

How to integrate AI facial expression detection to control your video – Javascript 

5 AI tools that will save you 10+ hours on video production

Lights, Camera, AI Action! 

Video trends

In a world where time is money and creativity reigns supreme, AI tools that convert text to video are game-changers! These AI-powered wonders are designed to save your day—and countless hours. Join us as we unveil the top 5 AI video tools that can dramatically cut down your video creation and editing time. From transforming text into captivating visuals to adding that perfect touch of magic to your edits, these tools are set to become your new best friends in the digital playground!

## **1. CapCut**

At number 1, we have Capcut. 

&nbsp;

[CapCut](https://www.capcut.com/) surged in popularity due to its user-friendly interface and powerful editing features, making it a favorite among video creators. Its integration with TikTok and social media platforms is also reason for its widespread adoption.

&nbsp;

The best part about using CapCut is that most of these important features are available for free, which kicks out budget-constraints out of the window. CapCut uses AI to empower users to create stunning videos without needing any fancy editing experience. With mobile and desktop apps, as well as a web version, CapCut can easily be used on the go. 

&nbsp;

CapCut offers a range of AI-driven features that are very, very impressive. With 'Smart Cuts & Trims', CapCut's AI automatically detects shaky parts, long pauses, or unnecessary bits, suggesting cuts to tighten up your video and eliminate boring footage. 'Effortless Audio Magic' helps isolate voices and enhance audio quality, making your message crystal clear even if there’s noisy background interference. 'Instant Captions' auto-generate captions, making your videos more accessible and saving you time and effort. Moreover, if your filming didn't capture the perfect lighting, 'Light Like a Pro' allows CapCut’s AI to adjust lighting in your videos post-production, creating the ideal mood.

What I most love about CapCut AI video generator is its AI-driven background removal feature which eliminates the need for a green screen, allowing me to replace any background with an image or color of my choice.

&nbsp;

Something to note about Capcut is that the free version includes watermarks on exported videos. which can be removed only with a paid subscription. This may potentially pose a barrier for you if you’re a budget-conscious user.

## **2. Runway**

[Runway](https://runwayml.com/) can be called a virtual sorcerer's wand for video editing! It has the potential to be an entire AI video production studio, so it can basically create an entire movie without filming a single frame — if you have the time, that is. Runway can generate videos and animations from simple text prompts, create animations from still images, generate lip-syncing avatars based on a recording of your voice, and even allow you to train your own AI models. 

&nbsp;

Runway might be a little intimidating in the beginning. That’s because it has an array of features that take time to understand, execute and implement, and that could be slightly time-taking and if I can say, complex.

Another aspect to note is that while Runway offers a range of pricing plans, the advanced features that truly set it apart are locked behind higher-tier subscriptions, making it less accessible for small businesses or individual creators on a tight budget.

## **3. Invideo AI**

[Invideo AI](https://invideo.io/ai/) is an excellent tool for generating a starting point for your videos. It helps you to quickly create a basic structure for your videos, which can then be further customized and polished. The  features allow you to input simple prompts, from which the tool generates an initial draft that includes relevant footage, transitions, and effects. 

&nbsp;

A few standout features of Invideo AI are an extensive stock media library that offers a range of options to choose from, AI-powered script generation which creates a complete script for your videos and, voiceover addition. Their AI voiceover generator is particularly good because it has an option to add your own voice which it then uses to narrate scripts in. The narration is also decent, with multiple accents and focussing well on the feeling, intent, and emotion behind each word.

Overall, the tool is great for YouTubers and people who want to start off their video making journey with minimal effort. 

&nbsp;

One of the shortcomings of Invideo AI is its dependency on a stable internet connection, as the tool is cloud-based. This can be a significant limitation for users in areas with unreliable internet access. I also noticed that rendering videos on Invideo AI can sometimes be slow, especially for longer or high-resolution projects, so keep enough time in hand before you expect a final video. 

## **4. Wondershare Filmora**

[Wondershare Filmora](https://filmora.wondershare.net/de/ad/videobearbeitungsprogramm-brand.html) has been a staple in the video editing world for over 15 years. Despite its long history, it's continuously evolving, with a suite of AI features that enhance video quality without requiring professional training or extensive time investments. 

&nbsp;

Filmora is a popular choice among full-time influencers and video editors for advanced-level video editing. It is also a preference because the tool keeps evolving based on new technological developments, and therefore is easily able to meet the demanding standards of modern digital media.

&nbsp;

Some of Filmora’s features include 'AI smart cutout', which allows you to cut objects out of scenes simply by painting their area on the screen, and 'AI audio denoise', which helps control background noise. 'AI audio stretch' lets you extend music to match scene transitions, while AI motion tracking enables face blurring for privacy or fun effects. They also have AI remove background which allows you to create imaginative scenes without hefty budgets. 

Wondershare also provides detailed explanations of these features on their website, so if you ever face a problem using any of them, be sure to check their dedicated pages. 


&nbsp;

However, do note that Filmora's performance can be sluggish when handling large files or high-resolution videos, leading to slower rendering times and potential crashes at times.

## **5. Canva Magic Studio**

Easy to use. Neat interface. Quick results. 

&nbsp;

Those were the three things I liked best about [Canva Magic Studio](https://www.canva.com/magic/). 

&nbsp;

Canva Magic Studio looks very promising as an AI video tool. Having been a fan of Canva for infographics, presentations and social posts since the past few years, their AI video tool has quite impressed me. What stands out is that their UI is very, very simple to understand. Plus, their features don't seem to be overwhelming when it comes to video editing, which I often feel is the case with other professional editing tools. 

&nbsp;
 
Canva Magic Studio gives you amazing results when it comes to video quality and is also very quick to generate the final video. The good video quality comes from its integration with Runway (our #2 in the list).

&nbsp;

To show you an example, I used the prompt: "A cute rabbit watching a video on a laptop”. And the [result](https://www.canva.com/design/DAGGCqwwf4A/GS7q42J5C9nx26gxJGR9lA/watch) was surprisingly very apt!

Canva Magic Studio is a great option if you want to bulk-create your social media posts – create a chunk of Instagram reels or Facebook ads in a go or create similar kinds of videos using a single template. The template options Canva offers have a wide range and are decent for beginners to start with.

## Sora – Open AI’s video generator

Open AI sparked a wave of excitement around AI with ChatGPT, demonstrating its incredible potential in natural language processing. Now, it's taking things a step further with [Sora, an innovative AI](https://openai.com/index/sora/) video creator.

Open AI recently shared a series of tweets showing the potential of Sora and I must say, I am amazed. Be it [several giant mammoths](https://x.com/OpenAI/status/1758192960116785459) rushing through a snowy meadow or a [fluffy monster](https://x.com/OpenAI/status/1758192964222988506) behind a melted candle, the kind of videos Sora will be able to create is astounding! It looks like it will be able to take detailed prompts into consideration, just like it did with ChatGPT, to create exactly what the user needs.

&nbsp;

As Sora continues to evolve and improve, it will undoubtedly shape the future of video production, showing way to new forms of storytelling and engagement in the years to come.

## AI is great – but it cannot replace you

While these AI video tools can ease the load for you and save some time, they cannot really take the place of your creativity, intuition, and emotional intelligence. In the end, it is your perspective, personal touch, and experiences that shape the videos you create. 

&nbsp;

_If you’re finding a place to store, host and deliver your [AI generated videos](https://api.video/use-cases/generative-ai/), api.video could be just what you’re looking for. With quick video upload options, flexible hosting capabilities, ability to add watermarks and keep videos private, you can choose api.video for all your video needs. If you would like to test out our features, [sign-up](https://dashboard.api.video/register) for a sandbox account now and you wound’t be disappointed._

small_7 AI Tools for Quick Video Creation.jpg

thumbnail_7 AI Tools for Quick Video Creation.jpg

7-AI-Tools-for-Quick-Video-Creation.jpg

5 ways the human eye impacts how we do video compression

7 eye-opening stats that prove the impact of product videos in e-commerce

5 ways to utilize product videos to boost your e-commerce sales

Yohann Martzolff

React

In today's tech-driven world, enhancing user interaction and creating immersive experiences is crucial. One exciting way to achieve this is by integrating AI facial expression detection into your video content.  In this blog, we'll guide you through the steps to <u>integrate AI facial expression detection to control your videos using React</u>. 

&nbsp;

Let's start.

&nbsp;

_*To learn how to do the integration with Javascript, head to this [blog](https://api.video/blog/tutorials/AI-facial-expression-detection-javascript/).*_

## 1. Create a minimal React app

As the first step, we need to create a new React app. Run the following command for the same. 

Now add the MediaPipe and the api.video React player packages

## 2. Add a video player

You’re going to use the official [api.video’s React player](https://github.com/apivideo/api.video-react-player) in this application, but feel free to use your favorite one and adapt the callbacks to it.

&nbsp;

Replace the code inside `App.tsx` with the following.
The `playerRef` allows us to control the [api.video player](https://api.video/product/player/).

Run your application with the npm run start command and go to `http://localhost:3000/`, you should see a video that you can play, pause or mute. 

## 3. Record your facial expressions

To control the player with your facial expressions, you need to stream your face through your webcam and pass this streamed data to the trained machine-learning model.

&nbsp;

Add a `<video />` HTML tag to `App.tsx`. 
Its `webcamRef` allows the video element to be targeted.
Apply a `display: "none"` inline style to hide the element, and an `autoPlay` attribute so the webcam stream time is updated. 

&nbsp;

**Note:** You can remove the `display: "none"` style and customize your CSS, it doesn’t matter.

## 4. Load the machine-learning model

Before we can use the machine-learning model, we must wait for it to finish loading. These models can be large and take a moment to get everything needed to run.

&nbsp;

Put this logic inside a `useEffect` and store the `faceLandmarker` class as a React stateful value.

## 5. Start detecting the webcam stream

Once the machine-learning model has been loaded and the `faceLandmarker` class has been instantiated, you can start detecting the stream.

&nbsp;

Create a new `useEffect`

Run your application with the `npm run start` command, go to `http://localhost:3000/`, and open your console. You should see a lot of console logs appearing each millisecond.

&nbsp;

Each one of them logs an array of objects of type:

## 6. Use the facial expression detection results

You now have access to a list of facial expression detection results. Use them to do whatever you want and in this case, to control your video player.

Let’s assume that you want to play the video when you smile, and pause it when you do a kiss.

Check the facial expression detection results on each frame and play or pause your video depending on the score of the wanted result by replacing your console.log with the following.

Once again, run your application with the `npm run start` command, go to `http://localhost:3000/`, and try to smile at your webcam. It should trigger the player’s `play` callback and play the video. Then, try to do a kiss ( 😗 ) in front of your webcam, and the video should pause.

## 7. Display the results

It can be helpful to have a visual representation of the detection results. Display these results as a list and highlight the detected facial expressions by adding a `<ul />` HTML tag to your JSX and creating a new ref to target it.

Add the code to update the HTML according to the results.

Now, you should see a list of results displayed in your application, with a green highlight on the ones with more than 60% confidence.

## 8. Code samples

Check the full `App.tsx` code below.

We hope this blog helped you understand the coding behind integrating AI facial expression detection to your videos.

 &nbsp;

_For any questions or doubts you may have, chat with us on our [website](https://api.video/contact/). If you would like to try out api.video for your videos, [sign-up](https://dashboard.api.video/register) for a free sandbox account here._

How to build AI facial expression detection into your videos with React

small_AI face recognition (React) - Version 2.png

thumbnail_AI face recognition (React) - Version 2.png

AI face recognition (React) - Version 2.png

Create a Next.js 13 & React 18 video app

How to create a Next.js 13 & React 18 video app

React Native Livestream Module: building the sample app

How to upload a video with React and NodeJS

Featured article

# api.video blog

api.video Blog

Follow the api.video Blog to explore tutorials, video use cases, and learn about on-demand and live-streaming video updates.

Video on demand

Live streaming

Analytics

Video infrastructure

Video player

AI features

See all features

Online learning & Corporate training

Social & Entertainment

Marketplace & E-commerce

Communication & UGC

Generative AI

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Customer success stories

Help center

ffmpeg