Uploading large files with JavaScript: File.slice() to the rescue!

api.video enables developers to build, scale and operate video in their own apps and platforms in minutes, with just a few lines of code. The service handles the end-to-end workflow, from video ingestion to worldwide video delivery. You can test it for free right away and start building.

August 2021 update Since this post was written, we've published a library to simplify JavaScript upload of videos read the blog post to learn more.

17 February 2021 update The example code has been added to extract the filename, and then append it to the first upload. Without this step, your video title will be 'blob.'

You can view the API reference documentation for the file upload endpoint here: Upload a video

As our presentations, PDFs, files, and videos get larger and larger, we are stretching remote servers’ ability to accept our files. With just a few lines of JavaScript, we can ensure that this error goes away, no matter what you are trying to upload. Keep reading to learn more.

The most common error with large uploads is the server response: HTTP 413: Request Entity too Large. Since the server is configured only to accept files to a specific size, it will reject any file larger than that limit.

One possible resolution would be to edit your server settings to allow for larger uploads, but sometimes this is not possible for security or other reasons. (If the server limit gets raised to 2GB for videos, imagine the images that might get uploaded!)

Further, if a large file fails during upload, you may have to start the upload all over again. How many times have you gotten an “upload failed” at 95% complete? Utterly frustrating!

Segments/Chunks

When you watch a streaming video from api.video, Netflix or YouTube, the large video files are broken into smaller segments for transmission. Then the player on your device reassembles the video segments to play back in the correct order. What if we could do the same with our large file uploads? Break the large file into smaller segments and upload each one separately? We can, and even better, we can do it in a way that is seamless to our users!

Baked into JavaScript are the File API and the Blob API, with full support across the browser landscape:

This API lets us accept a large file from our customer, and use the browser locally to break it up into smaller segments, with our customers being none the wiser!

Let’s walk through how you might use this to upload a large video to api.video.
To follow along, the code is available on Github, so feel free to clone the repo and run it locally.

To build your own uploader like this, you’ll need a free api.video account. Use this to create a delegated upload token. It takes just 3 steps to create using CURL and a terminal window.

A delegated token is a public upload key, and anyone with this key can upload videos into your api.video account. We recommend that you place a TTL (time to live) on your token, so that it expires as soon as the video is uploaded.

Now that you're back, we'll begin the process of uploading large files.

Markup

The HTML for our page is basic (we could pretty it up with CSS, but it's a demo 😛):

There is an input field for a video file, and then there are 2 divs where we will output information as the video uploads.

Next on the page is the <script> section - and here's where the heavy lifting will occur.

We begin by creating some JavaScript variables:

• input: the file input interface specified in the HTML.
• url: the delegated upload url to api.video. The token in the code above (and on Github) points to a sandbox instance, so videos will be watermarked and removed automatically after 24-72 hours. If you've created a delegated token, replace the url parameter 'to1R5LOYV0091XN3GQva27OS' with your token.
• chunkCounter: Number of chunks that will be created.
• chunkSize: each chunk will be 6,000,000 bytes - just above the 5 MB minimum. For production, we can increase this to 100MB or similar.
• videoId: the delegated upload will assign a videoId on the api.video service. This is used on subsequent uploads to identify the segments, ensuring that the video is identified properly for reassembly at the server.
• playerUrl: Upon successful upload, this will output the playback url for the api.video player.

Next, we create an EventListener on the input - when a file is added, split up the file and begin the upload process:

We name the file uploaded as 'file'. To determine the number of chunks to upload, we divide the file size by the chunk size. We round the number round up, as any 'remainder' less than 6M bytes will be the final chunk to be uploaded. This is then written onto the page for the user to see. (In a real product, your users probably do not care about this, but for a demo, it is fun to see).

Slicing up the file

The function createChunk slices up the file.

Next, we begin to break the file into chunks. Since the file is zero indexed, you might think that the last byte of the chunk we create should be chunkSize -1, and you would be correct. However, we do not subtract one from the chunkSize.

the first byte that will not be included in the new Blob (i.e. the byte exactly at this index is not included).

So, we must use chunkSize, as it will be the first byte NOT included in the new Blob.

In the createChunk function, we determine which chunk we are uploading by incrementing the chunkCounter, and again calculate the end of the chunk (recall that the last chunk will be smaller than chunkSize, and only needs to go to the end of the file).

In the first chunk uploaded, we append in the filename to name the file (if we omit this, the file will be named 'blob.'

The actual slice command

The file.slice breaks up the video into a 'chunk' for upload. We've begun the process of cutting up the file!

We then create a form to upload the video segment to the API. After the first segment is uploaded, the API returns a videoId that must be included in subsequent segments (so that the backend knows which video to add the segment to).

On the first upload, the videoId has length zero, so this is ignored. We add the chunk to the form, and then call the uploadChunk function to send this file to api.video. On subsequent uploads, the form will have both the videoId and the video segment.

Uploading the chunk

Let's walk through the uploadChunk function:

We kick off the upload by creating a XMLHttpRequest to handle the upload. We add a listener so we can track the upload progress.

adding a byterange header

When doing a partial upload, you need to tell the server which 'bit' of the file you are sending - we use the byterange header to do this.

We add a header to this request with the byterange of the chunk being uploaded.

Note that in this case, the end of the byterange should be the last byte of the segment, so this value is one byte smaller than the slice command we used to create the chunk.

The header will look something like this:

Upload progress updates

While the video chunk is uploading, we can update the upload progress on the page, so our user knows that everything is working properly. We created the progress listener at the beginning of the uploadChunk function. Now we can define what it does:

First, we do a little bit of math to compute the progress. For each chunk we can calculate the percentage uploaded (percentComplete). Again, a fun value for the demo, but not useful for real users.

What our users want is the totalPercentComplete, a sum of the existing chunks uploaded, but the amount currently being uploaded.

For the sake of this demo, all of these values are written to the 'chunk-information' div on the page.

Chunk upload complete

Once a chunk is fully uploaded, we run the following code (in the onload event).

When the file segment is uploaded, the API returns a JSON response with the VideoId. We add this to the videoId variable, so it can be included in subsequent uploads.

To upload the next chunk, we increment the bytrange start variable by the chunkSize. If we have not reached the end of the file, we call the createChunk function with the videoId and the start. This will recursively upload each subsequent slice of the large file, continuing until we reach the end of the file.

Upload complete

When start > file.size, we know that the file has been completely uploaded to the server, and our work is complete! In this example, we know that the server can accept 5 MB files, so we break up the video into many smaller segments to fit uder the server size maximum.

When the last segment is uploaded, the api.video response contains the full video response (similar to the get video endpoint). This response includes the player url that is used to watch the video. We add this value to the playerUrl variable, and add a link on the page so that the user can see their video. And with that, we've done it!

Conclusion

In this post, we use a form to accept a large file from our user. To prevent any 413: file too large upload errors, we use the file.slice API in the users' browser to break up the file locally. We can subsequently upload each segment until the entire file has been completely uploaded to the server. All of this is done without any work from the end user. No more "file too large" error messages, improving the customer experience by abstracting a complex problem with an invisible solution!

When building a video uploading infrastructure, it us great to know that browser APIs can make your job building upload tools easy and painless for your users.

Are you using the File and Blob APIs in your upload service? Let us know how! If you'd like to try it out, your can create a free account and use the Sandbox environment for your tests.

If this has helped you, leave a comment in our community forum.