Progressively upload large video files without compromising on speed

If you work with video, you know that when you have a gigantic video file, you must break it into chunks to send it to the server. We at api.video already offer a way for you to break up your video into chunks and send it to us, but today we are introducing a new way of uploading large files: Progressive Upload.

Instead of spending time worrying about getting the bytes right for the header, or relying on our API clients to handle it for you (still a great choice if you have a client in use) you can simplify the header for a chunk to be about what part of your video you're sending. Here's an example of what I mean:

Old header: Content-Range: bytes 0-1000000/200000000

Here you can see the Content-Range header shows the bytes being sent from the video and then the total number of bytes. But there's a couple of issues with this. First, it's tricky setting up the header correctly (which makes the clients great to use since they handle this for you) and second, what if you want to upload a video where you don't know its size? It'd also be great if you could upload video chunks concurrently and have that handled for you. Progressive Upload does this, and the new header looks like this:

New header: Content-Range: part 1/*

Every video chunk just receives a part number. The only video chunk that can be smaller than 5MB is the last chunk. Everything else must be larger. And with this explained, we're ready to prepare and try a code sample.

Prerequisites

For this project, you'll need:

• (Optional) Familiarity with other ways to upload (it might make this tutorial easier, but this is written so you don't need the other tutorials)
• An api.video account, you can sign up here
• Python or a Python virtual environment to work with (recommended)
• Requests library for Python installed

Progressive Upload code sample

The code sample we're going to walk through looks like this:

In the code sample, you're doing the following things:

1. Retrieving a token using your API key.

2. Setting up a video container to upload to. You upload in two parts - first you create a container with metadata about your video. Then you retrieve the videoID for your video container, and reference it so you can upload your chunks to the right container. You can only upload once to a container, so make sure you have everything set up correctly. Otherwise you'll have to start over with a new container.

3. We set a variable CHUNK_SIZE to represent the size of each chunk of video we'll send. You can make this bigger if you like to speed up the process.

4. We write a little function read_in_chunks that breaks a chunk of data off our video file to send, then remembers the spot it was in so when we come back to the function again, it gives us the next chunk of data. This is accomplished by using yield in the function.

5. In the upload function, we get ready to loop through our video file, sending a chunk of data each time with the appropriately labeled header. For Content-Range we just increase the part number by 1 each time, and leave it set as unknown, like: part 1/* If you know your video size ahead of time, you can change the asterisk to the final part number. This code sample assumes you don't know the size. What you do per chunk is check if it's smaller than 5 MB, since this will signify we've reached the final piece of the video. For this last piece of video, we change the total number of parts to match the part we're on. This signals to api.video that we aren't going to send any more chunks and that the upload is complete.

And we're done! To start building right now, check our docs and sign up for a free account.