Video list

Video Status

Normal

thumbnail_aya-modified.png

aya-modified.png

github-brands.svg

Github Aya Bochman

Aya Bochman

Tutorials

JavaScript

NodeJS

Next.js

Typescript

In this tutorial we will create a Udemy-like learning platform with [api.video](http://api.video) using Next.js, React, and Typescript. Udemy is an open online learning and teaching marketplace where you can have access to a variety of courses.

We’ll focus on how to easily add Udemy’s notes feature to your video with the help of [api.video’s player SDK](https://github.com/apivideo/api.video-player-sdk), which allows us to control and interact with our player. We’ll also see how to show a progress bar on the videos preview and resume a video where it was last paused.

You can see the demo [here](https://udemy-clone-next-typescript.vercel.app/) and the complete code [here](https://github.com/apivideo/udemy-clone-next-typescript).

video-note-demo.gif

## 1. Quick Setup

Let's start to build a new Next.js application using **Create Next App** and `--ts` for **Typescript**

Run these commands in the terminal:

Once inside our project, install api.video’s Node.js client and player SDK  with some style and icon packages:

Note: We'll use styled-components and react-icons to style our example application, but you can use any styling libraries you want. 

## 2. Video list preview

First, on our main page, we will display a list of our videos. You can follow the 3rd step, “How to fetch data,” in our [Youtube clone tutorial](https://api.video/blog/use-cases/rebuilding-youtube-with-api-video-next-js/) to achieve that. 

After you fetch the videos data in `pages/index.tsx`, add a component that you’ll pass the `videos` result to.

## Video preview

Once we have our array of videos in a designated component, we want to create a simple video preview with a title and thumbnail.  

We’ll map the list, and in each video object, we can access the thumbnail CDN with `assets.thumbnail` and show the `title`. Pass it to your desired preview component, style it, and you can get a result like this:

large_videos-preview-empty.png

medium_videos-preview-empty.png

small_videos-preview-empty.png

thumbnail_videos-preview-empty.png

videos-preview-empty.png

## 3. Video page view

### Setting up the player SDK

I’ve created a `components/Video/index.tsx` component which will display the video on my `pages/video/[videoId]` page. Then let’s set up the player SDK in 2 simple steps:

**Step 1 - Get video details**

Create an API route in `pages/api/video.ts` for getting video details by Id with our [Node.js client](https://github.com/apivideo/api.video-nodejs-client).

Now in the front end, get the `videoId` from the query, send it to the API, and you will get a video object. 

**Step 2 - Create player SDK**

Now the fun part begins 😄 after we get the `videoId`, we can create our [api.video player SDK](https://github.com/apivideo/api.video-player-sdk#documentation). The SDK will help us to get a bunch of information from our video, and to control and customize the player as we want. 

After these 2 steps, our code should look like this:

Go to the [player SDK documentation](https://github.com/apivideo/api.video-player-sdk#documentation) to read more about the params `id, hideTitle, hidePoster` 

You’ll notice that we’re getting an element by id called `myVideo`. We need it to connect the SDK to the actual HTML5 player. So let’s create that iframe element in our render:

large_player-img.png

medium_player-img.png

small_player-img.png

thumbnail_player-img.png

player-img.png

Set the player SDK in the state, and once it’s added to the state for use, we want to add a few more features when the player is ready with `addEventListener('ready', callback)`. `ready` means the player is loaded and ready to play, so here we can customize the theme to Udemy’s player colors, and get the total duration of the video for the Overview tab display.

We can do that with the SDK’s `setTheme()` and `getDuration()` methods. I’ve put the methods in separate functions to keep the code clean.

Read more about the player SDK methods and event listeners [here](https://github.com/apivideo/api.video-player-sdk#methods).

The duration we get from the SDK is in seconds, so I’ve added a conversion to hours function in `utils/functions.ts`

large_duration-preview.png

medium_duration-preview.png

small_duration-preview.png

thumbnail_duration-preview.png

duration-preview.png

### Bookmarking a video - How to listen to a player’s timestamp?

In this part, we'll focus on how to add a note to a video’s specific timestamp! We can achieve that by using [api.video player SDK](https://github.com/apivideo/api.video-player-sdk) that we set in the previous step.

new-note-click.gif

To create a bookmark in a video, we need to constantly listen to the video’s playback. The idea is that once we click on ‘Add Note,’ the video will pause, and we can write and save a note on its timestamp. 

So our next step is to add a `timeupdate` event listener, then set it to the state called `currTimestamp`. Make it an object of `minutesFormat` and `seconds` because we want to have a formatted minutes display but also a seconds value for some of the player SDK methods.

Save the video’s timestamp in the localStorage to keep track of the video’s playback. We’re going to use it to show progress later.

I chose to format our seconds to `mm:ss` display, but you can choose whatever format you like, just add it in our `utils/functions.ts`

And we’re set! 👏  At this point, we have our `currTimestamp` in the state, and we can now create a note button that will pause the video when we click on it with `playerSdk.pause()` method, then open the Notes tab with an input on the current timestamp.

large_note-preview-filled.png

medium_note-preview-filled.png

small_note-preview-filled.png

thumbnail_note-preview-filled.png

note-preview-filled.png

*I’m saving the notes in an object state where the key is our formatted `mm:ss` timestamp, and the value is an object of `note` and `seconds`. For the simplicity of the demo, the notes are saved in the local state.*

Suppose you go straight to the Notes tab without deciding which timestamp you want or after saving a timestamp. In that case, Udemy has a ‘Create a new note at’ button at the top, which has a changing timestamp value corresponding with the player’s playback. 

This is easy to achieve since we’ve already set our `currTimestamp` in the state with a `minutesFormat` value. So when displayed in the button, it will change with the playback time, like this:

note-timestamp.gif

Now, after we have some saved notes, we want to be able to jump to the timestamp by clicking on the note’s timestamp button.

That’s going to be very simple - since we’re saving `seconds` in every note object we create, let’s map our `notesList` for display, then send `notesList[key].seconds` to our player SDK method `playerSdk.setCurrentTime(seconds)`. This will set the player’s playback time to our desired seconds.

jump-to-timestamp.gif

### Display video progress

Remember we saved the video’s timestamp in our localStorage? Now it’s time to use it for creating a progress bar like this:

large_filled-progress.png

medium_filled-progress.png

small_filled-progress.png

thumbnail_filled-progress.png

filled-progress.png

In our video preview component, create this function:

First, we’re calling [api.video](http://api.video) [GET video status API](https://docs.api.video/reference/api/Videos#retrieve-video-status-and-details) from which we can extract the total duration of our video. 

Then we get the video’s last timestamp we saved in our localStorage with `videoId`. 

Once you have the total duration and paused seconds, all you’ve got left is to choose how you want to draw the progress on the preview. I used [Radix UI progress](https://www.radix-ui.com/docs/primitives/components/progress) which is very easy to customize. 

The same goes for when we click on a video and want to set it to the last paused timestamp. Extract the timestamp from the local storage, then use our player SDK `setCurrentTime(seconds)` method. 

And voila! 🥳 Now we can display a nice progress preview on our videos and set a video to its last timestamp.

## Conclusion

This tutorial taught us how to recreate a Udemy-like learning platform, focusing on their notes feature and video progress. 

We achieved amazing, functioning results with the power of api.video’s player SDK, and Node.js client. The entire code is available for you [here](https://github.com/apivideo/udemy-clone-next-typescript).

That’s it! 💪 [Start building](https://dashboard.api.video/register) now and don’t hesitate to ask questions or share your project with us on the [Community](https://community.api.video/)!

See you in our following tutorials 👋

### Part 2 is available: Add AI-generated summary, topics, and transcript to your videos using [symbl.ai](https://symbl.ai/blog/accessibility-automated-content-management-e-learning-platforms/)

How to create a Udemy-like learning platform

large_udemy-preview-blog.png

medium_udemy-preview-blog.png

small_udemy-preview-blog.png

thumbnail_udemy-preview-blog.png

udemy-preview-blog.png

Video upload

doug.jpg

twitter-brands.svg

Doug Twitter

Doug Sillars


Here at [api.video](https://api.video), we love making video delivery easy for our customers. We take care of doing all the video transcoding and [video streaming](https://api.video/streaming/) so that you can focus on all the other facets of your product. Using our APIs (and our SDKs) is a straightforward and easy way to add video to your service. In this post, we’ll walk through the steps to create a website where you can upload videos.

> Note: Use NodeJS in the middle results in a "double upload" - the video is uploaded to the Node server - then to api.video.  You can avoid this with a [delegated token](https://api.video/blog/endpoints/delegated-upload/) and our [JS library](https://api.video/blog/tutorials/new-video-uploader-javascript-library/).

You can also check out our API reference documentation here: [API Reference documentation](https://docs.api.video/reference)

## Overall Architecture

![Overall architecture api.video](https://cdn-images-1.medium.com/max/800/1*Vje7Cz76uC0VTHLkrAmGFQ.png)

1.  Starting with a web form, a video is submitted to the Node server.
2.  The video is immediately uploaded to api.video for processing
3.  api.video acknowledges the video upload, returns the videoId, and begins encoding the video.
4.  The node server polls api.video to obtain the playable status (confirming that encoding is completed).
5.  As long as the result is “false”, the polling continues.
6.  When the video is playable, the Node server returns a page with the video. The video autoplays to show that the video is ready to play. The page also returns the timings for upload & for video processing.
7.  The video link is e-mailed to the user (optional).

### Getting Started

### Node & Dependencies

There are a number of required Node dependencies:

``` 
npm install express dotenv @api.video/nodejs-sdk fs pug serve-favicon formidable email-validator
```

This tutorial also optionally connects to Intercom to send emails to your users. This will require:

``` 
npm install intercom-client
```

### Building the Site

### Starting index.js

You can clone the GitHub repository to follow along, but here is the code that builds your video uploader:

In the src/index.js file, copy the following code:

In this snippet, Node imports Express (and dotenv for environmental variables). The first big chunk of code creates and sets up an express app. It will use Pug to build the views. Static files are saved in the ‘public/’ directory, and serve-favicon adds the favicon to the site. You can use the default favicon in the public directory on GitHub, or create your own.

Next we have app.get — on the default entry point of the site. If a request occurs at the “/upload-demo” directory of the web server, we’ll deliver the start.pug file that is saved in the views directory.

Finally, the app listens on port 3000 for requests. If you run npm start from the terminal, you should get a response of “Example app listening on port 3000!” You can now enter “localhost:3000/upload-demo” into your browser, and if you have added the views directory — you’ll see the form. If you have not yet added the view, you’ll probably get an error.

### Start.pug

The landing page is simply an introductory text describing the upload and a form to select the file. This page is created with Pug. Pug is really persnickety about indentation spaces, so it might be best to copy the code (or just use the code) from GitHub rather than copy and paste from this post.

In the head tag, we load some CSS and the favicon.

medium_capture-d-ecran-2020-07-29-a-16-54-48.png

small_capture-d-ecran-2020-07-29-a-16-54-48.png

thumbnail_capture-d-ecran-2020-07-29-a-16-54-48.png

capture-d-ecran-2020-07-29-a-16-54-48.png

The body loads descriptive text on api.video, and the submission form. The form takes 4 inputs: video name, e-mail, true/false mp4 support and the video file. You may notice that the form action will reload the root directory of the server. Since the data submission is a POST, it will be easy to identify the 2nd request.

### POST Form Processing

Add these additional lines to the index.js. At the top of the file, we need to add dependencies:

api.video is our backend where uploaded videos are transcoded into video streams. We add the SDK, and then I store my API key in a .env file. I have added the .env file to my .[gitignore](https://git-scm.com/docs/gitignore) file so that when I post my code on GitHub, my key is not uploaded and remains secure.

We also add formidable to read the form data and save the video locally on the Node server. The e-mail validator does what it sounds like it might do, and finally I add three variables to measure how quickly the service handles the video processing.

In the body of index.js, add the following:

We are now building the POST response. Formidable saves the video in the directory specified in the .env variable videoDir (this doesn’t have to be private, but it is nice to save all configurable variables in one place). When the upload requests mp4 support, the variable must be a boolean, so we use the string from the response to create the corresponding boolean value. The email validator validates the email, and if the email is valid  -  we use production credentials at api.video (permanent video hosting). If there is no e-mail, or it is not valid, the video is instead processed in the Sandbox, meaning that it will be watermarked & deleted after 24 hours. For your use — you probably just want to post to production, so you can simplify this logic.

Finally, we begin the upload of the video to api.video (and capture the time of the upload start).

If you try to run this code now, it will hang — we are not yet listening for the response from api.video. However, adding:

```
console.log(result);
```

Will show the JSON response from api.video:

The videoID is important to display and playback the video, as are the assets. We also can see useful metadata about the video — Mp4Support, Panoramic, public/private, etc. Note that there is no mp4 url — the video must be encoded to receive that url, so we’ll grab it later.

### Video Upload Response

After the “let result = client.videos.upload(files.source.path, {title: fields.title, mp4Support: mp4Support});” paste in the lines below.

We listen for the result from the upload. This allows us to mark upload completed (to measure the elapsed upload time). If the video has successfully been uploaded to api.video, we now have 2 copies of the video: one on the node server, and the other at api.video. Fs.unlink deletes the file on the node server. The JSON response gives us the videoId at api.video, and links to the iframe and player video urls (which we assign to variables). The mp4 url is only presented in this response when encoding is finished.

Even though we have the urls to the videos, they have not been processed yet. To determine if the video is playable, we call the videoStatus function. This sends a request to api.video to obtain the video status. The response is a JSON with lots of details about upload status, file size, and metadata like the dimensions and bitrate (you can learn more about these parameters in the [documentation](https://docs.api.video/reference/api/Videos#retrieve-video-status-and-details)):

Reading the videoStats.encoding.playable value tells us if it is ready to play or not. If it is not ready, we call videoStatus again recursively until we get the affirmative response. When we receive that ‘playable’ is true (as in the example above), the function can then query for the MP4 url, and return the view to the user: return res.render(‘video’, {iframe, player, uploadSeconds,processSeconds});

This means that video.pug will be displayed to the end user with some variables (the urls of the video, and timings on upload and processing). We’ll discuss that view in the next section.

### Displaying the Video

The Video.pug file is a simple html file that simply adds an iframe for the video, and reports the elapsed times for upload and processing of the video. The variables are ensconced in #{} tags. One other interesting thing to note is adding #autoplay to the url tells the video player to autoplay the video (you can also add #loop to loop the video).

And there we have it, a working video upload tool, where the response page shows the encoded video, along with the time for upload and for processing! Pretty cool!



medium_1uie4dyqvwgeeyfd3visag.jpg

small_1uie4dyqvwgeeyfd3visag.jpg

thumbnail_1uie4dyqvwgeeyfd3visag.jpg

1uie4dyqvwgeeyfd3visag.jpg

### Optional Step: Email integration

In the video upload response section, there is a line of code that calls the intercom integration:

We first create a lead (or a contact) inside Intercom. This will come back with an id that uniquely IDs the user. If the email address already exists, Intercom returns an error message that says “There is already a user with the id=<idnumber>.” So, if the user already exists, we can parse the error message to grab the user id.

Then we can add a tag to Intercom to show that the user has used the video upload tool, and we send an email to the user’s email address with links to the videos, and the time it took to upload and process the videos. The text of the email is right in the code, so easy to modify.


### Update (2022)
	
We have a new version of a video uploader that uses a delegated token. It is live at [upload.a.video](https://upload.a.video), and you can read a post on [delegated token upload](https://api.video/blog/tutorials/uploading-large-files-with-javascript/). Delegated tokens remove the "double upload" to Node and then to api.video: videos are directly uploaded to api.video.
	
	
### Conclusion

We’ve built a video uploader using Node.js and api.video. The easy to build tool now gives us the ability to easily share our videos across our internet properties. Using Node and api.video, the entire process creates an easy to use interface to take regular videos, and convert them into video streams. If this tutorial has inspired you (or if you have any questions), let us know by posting in our [community forum](https://community.api.video/). 

Building a video upload demo with Node.js and api.video

medium_githubsocialnodejs.png

small_githubsocialnodejs.png

thumbnail_githubsocialnodejs.png

githubsocialnodejs.png

captions

Delegated Upload

Video captions/subtitles make it easier to follow along with your video. Whether your users have hearing issues or not, sometimes they may not want to have a loud speaker playing the video's audio, and adding captions allows them to understand the video.


medium_screenshot-2021-02-26-at-15-02-16-1.jpg

small_screenshot-2021-02-26-at-15-02-16-1.jpg

thumbnail_screenshot-2021-02-26-at-15-02-16-1.jpg

screenshot-2021-02-26-at-15-02-16-1.jpg




When adding captions to a video, you must create a WebVTT file. We've [written on the process of creating WebVTT files](https://api.video/blog/tutorials/how-to-add-captions-to-your-videos/). The format is not difficult to understand, but they are not simple to create manually, you need to break up a transcription into segments, and add start and stop times for each section.  


```

3
00:00:07.519 --> 00:00:11.759
The little train rumbled over the tracks.

4
00:00:10.880 --> 00:00:15.279
She was a happy little train for She had such

5
00:00:13.439 --> 00:00:16.719
a jolly load to carry.

6
00:00:15.599 --> 00:00:20.480
Her cars were filled full of good things for boys

7
00:00:18.879 --> 00:00:21.199
and girls.

```



In this post, we'll show how using the [Authot](https://www.authot.com/) transcription APIs can help you automatically add video captions to your videos, without the fuss of manually creating the VTT caption file.

Authot will receive the video file, make a transcription of the audio file, and then return a complete webVTT file. api.video can then ingest this file into the same videoID as the video, and the captions are added to the video.

It sounds like magic! To prove that it actually works, we've built a demo application [caption.a.video](https://caption.a.video) that utilizes api.video and Authot to host, stream and auto-caption your videos.  

## How it works

The first step is to upload your video to api.video.  This demo is built on top of a [delegated token upload](https://api.video/blog/product-updates/video-access-management-how-to-create-deliver-and-manage-private-videos-and-what/), meaning that a temporary public token was created that lets us upload right from the browser.



### Upload complete

Once the video is uploaded, api.video begins transcoding into a HLS sstream and an MP4 version of the movie.  To create a transcription, Authot requires the mp4 file, so the application uses the [video status](https://docs.api.video/reference#get-video-status/) endpoint to determine when the mp4 video has been encoded. This endpoint tells us when each video stream is ready for playback. Once the mp4 is encoded, we can submit the file to Authot for transcription.

### Transcription

Once the video has been submitted to Authot, the API returns the ID number of the transcription.  The Status endpoint tells us what state the transcription is in.  

The initial state is 0: Uploading

The final state is 10: transcoded

There are some states in between, including 2: transcribing

### Getting the WebVTT

Once the transcription is complete, (when Authot returns the status "10"),  the Node server requests the VTT file, and then uploads the file to the existing video at api.video.

### Completed

The webpage finally updates the text, saying "success".  Clicking the video link - opens the video, and the user can enable captions and watch them play back.


## Automatic captions!

Of course, automatic captions are not 100% accurate.  You may have a few words incorrectly transcribed.  However, in my testing the Authot API is pretty good, and the captions are of a very high quality.

On the other hand - manually creating captions will either take a lot of time, or cost a lot of money (to pay someone else to do the work).  The ease of use, and low cost makes auto-captioning a compelling choice


## Conclusion

We've described an application that allows you to upload a video and automatically add captions to the video.  Try it out today at [caption.a.video](https://caption.a.video). If you're curious on how we built this app, the code is available on [Github](https://github.com/dougsillars/captionavideo).  Let us know what you think by posting on our [community forum](https://community.api.video/).

Add captions, and watch your video watch time increase!

Auto Caption a video

In this post, we'll walk through the steps to upload a video. We've extended the application to not only upload the video to api.video, but to also post it to a Discord channel!

The sample code is on [Github](https://github.com/apivideo/videoUploadToDiscord).

## Connecting to Discord

Before we can upload the video and send the file to Discord, we need to prep our channel to receive the messages. To this end, I created the apivideo_upload bot.  To add this bot to your channel, simply visit: [https://discord.com/oauth2/authorize?client_id=780911671153000479&scope=bot
](https://discord.com/oauth2/authorize?client_id=780911671153000479&scope=bot) to add it to your server (this requires "Manage Server" permissions.)

## Uploading the video

>NOTE: Since this post was written, we've published a library to simplify JavaScript upload of videos [read the blog post to learn more](https://api.video/blog/tutorials/new-video-uploader-javascript-library/).

As discussed in the last post, the browser uses the File Slice API to break the video into 1MB segments and uploads them to api.video, using byte range requests and a videoId to identify the segments. Upon completion, the video is reassembled and encoded. The final API response lists all the details of the video.

The form to upload the video now also asks for a video name, description and a channelId. 

The channelId is the Id of the Discord channel that will receive the video.  To obtain the channelId, you must have developer mode enabled (user settings: appearance: advanced.

large_screenshot-2020-11-27-at-22-56-41.png

medium_screenshot-2020-11-27-at-22-56-41.png

small_screenshot-2020-11-27-at-22-56-41.png

thumbnail_screenshot-2020-11-27-at-22-56-41.png

screenshot-2020-11-27-at-22-56-41.png

Once developer mode is enabled, you can right click the channel name to copty the ID number:

large_screenshot-2020-11-27-at-22-58-38.png

medium_screenshot-2020-11-27-at-22-58-38.png

small_screenshot-2020-11-27-at-22-58-38.png

thumbnail_screenshot-2020-11-27-at-22-58-38.png

screenshot-2020-11-27-at-22-58-38.png

Once the video has been uploaded and transcoded, we send a POST request to NodeJS. server. The Node JS server updates the video name and description (and tags the video as a Discord upload) using the [update video](https://docs.api.video/reference/api/Videos#update-a-video-object) endpoint.

```
let result = client.videos.update(videoId, {	title: videoName, 
																							description: videoDesc,					
																					tags: [tag]
																	});
```

We're not quite ready to send the video to Discord yet - for the embed to play properly in the Discord application - the video must have at least one video quality transcoded, and be able to be played. So we use the [video status](https://docs.api.video/reference/api/Videos#retrieve-video-status-and-details), and ping it every 2 seconds until the video is playable.

Once the video is playable - we can send the link to Discord. We identify the channel using the channelID provided in the form, and simply send the video description and the url as a message.

```
function checkPlayable(videoId) {
			console.log("checking encoding status");
			let status = client.videos.getStatus(videoId);
			status.then(function(videoStats){
			 // console.log(videoStats);
			  let playable = videoStats.encoding.playable;
			  let qualitylist = videoStats.encoding.qualities;
			  console.log("is video playable?", playable);
			  //only look for the mp4 if the video is playable
			  //when still encoding, sometimes the mp4 status does not appear immediately
			  if(playable){
				 console.log("video is playable");
				//send to discord
				//send 200 back to page
				var channel = discordClient.channels.cache.get(discordChannel);
				channel.send( videoDesc + playerUrl);
				res.sendStatus(200);

			  }else{
				  setTimeout(checkPlayable,2000,videoId);
			  }
		  }).catch((error) => {
				console.log(error);
			});	
		}
```


medium_screenshot-2020-11-27-at-23-00-31.jpg

small_screenshot-2020-11-27-at-23-00-31.jpg

thumbnail_screenshot-2020-11-27-at-23-00-31.jpg

screenshot-2020-11-27-at-23-00-31.jpg

Bam! or "bloop bloop"!  The video is posted right into Discord.

Simply adding a few lines of code has allowed us to create a video uploader that will automatically post the video to Discord.

[Try it out](https://dashboard.api.video/register) and let us know what you think on our [community page](https://community.api.video).

Upload a video: Discord

large_upload-a-video-discord.png

medium_upload-a-video-discord.png

small_upload-a-video-discord.png

thumbnail_upload-a-video-discord.png

upload-a-video-discord.png

Video tagging

Video update

# Video Content Moderation

User generated content (UGC) is taking over the internet. From sites offering video product reviews to vlogging and online education - video is being created at the fastest pace yet (and shows no sign of stopping). Many companies are looking for ways to easily incorporate UGC video on their sites.  

large_screenshot-2020-10-23-at-12-17-19.png

medium_screenshot-2020-10-23-at-12-17-19.png

small_screenshot-2020-10-23-at-12-17-19.png

thumbnail_screenshot-2020-10-23-at-12-17-19.png

screenshot-2020-10-23-at-12-17-19.png

All of our customers who allow UGC to be posted on their site worry about a few bad actors working to ruin their brand by posting inappropriate content. In order to protect their brand and the content on their site - evaluating any user generated content before it is allowed to be placed on the website is an important step. Traditional content moderation requires human moderators (which takes time and is expensive).

In this post, I'll walk through an alternative: using machine learning to moderate videos. Before publishing, it is scanned using machine learning for content, tested against rules, and then either accepted or rejected based on those rules.  Super fast, works 24 hours a day, and there will be no human error in the categorisation of the video.

As a demonstration, we've built a site with [api.video](https://api.video) for video hosting, and using Hive AI to power the video moderation.

## The Basics

## api.video
api.video is a full service API-based video hosting solution.  You can use our APIs to upload videos, modify and serve streaming video.  Every video that is uploaded is transcoded into a video stream, and can be delivered in a custom video player. If you haven't created an api.video account yet, you can do that [here](https://dashboard.api.video/register). 

In this solution, we'll use the [delegated upload](https://docs.api.video/reference/api/Videos) (used like a public key) for uploading the videos.  We'll use the video [tagging](https://api.video/blog/tutorials/video-tagging-best-practices/) function to label each video based on the moderation results. We can search all of the tagged videos to deliver the videos (and custom player) to users based on their moderation state.

### Hive AI

Once the videos are uploaded, we need to run moderation before allowing them to appear on the website.  

Hive AI has several moderation suites. We've set up our API to use 2 endpoints. For short videos (under 25s) - we scan 1 frame every second, and for long videos, one frame every 5 seconds. (The demo above only works for 25s long videos).

The API is trained to identify several subjects that, depending on the context of your application, might be important to moderate. In this post, we will use a small subset of the trained models:

* Safe for work/Not Safe for Work
* Yes/No: Female nudity
* Yes/No: Male nudity
* Yes/No: Female swimwear
* Yes/No: Shirtless male
* Yes/No: guns
* Yes/No: smoking

After each frame is analyzed, a JSON file with data for each frame analyzed is returned for analysis.  

After analysis, the video is tagged with moderation value, ensuring that it only appears on pages appropriate for the video.

### Code

This application is available on [GitHub](https://github.com/apivideo/videoModeration). The general flow is:

1. On Upload, a video is tagged "needs moderation" (and will not appear on the site).
2. Hive AI performs frame analysis.
3. Based on the analysis, the video is tagged as "SFW" or "NSFW" (etc.).

moderation workflow, ai moderator, automated moderator

large_screenshot-2020-10-23-at-12-49-40.png

medium_screenshot-2020-10-23-at-12-49-40.png

small_screenshot-2020-10-23-at-12-49-40.png

thumbnail_screenshot-2020-10-23-at-12-49-40.png

screenshot-2020-10-23-at-12-49-40.png

large_screenshot-2020-10-22-at-15-17-28.png

medium_screenshot-2020-10-22-at-15-17-28.png

small_screenshot-2020-10-22-at-15-17-28.png

thumbnail_screenshot-2020-10-22-at-15-17-28.png

screenshot-2020-10-22-at-15-17-28.png

On entering the site, users are presented with an upload form.  As we walk through this example, we'll be following the upload of the intro theme to the classic TV show Baywatch.

With api.video, we have created a delegated upload key - which allows us to place the code publicly on the webpage without exposing our API private key. The form takes the video, and uploads the video to api.video.  The uploader uses the [Blob API](https://api.video/blog/tutorials/uploading-large-files-with-javascript/) to break the videos into 50MB segments for uploading.

For the purposes of the demo, we show how many chunks are created, and update the progress of each chunk, in addition to the total video upload:


small_screenshot-2020-10-23-at-13-34-15.png

thumbnail_screenshot-2020-10-23-at-13-34-15.png

screenshot-2020-10-23-at-13-34-15.png

The response from the upload provides the api.video ```videoId```, which is used to identify the video at api.video.  This is then sent to the NodeJS backend as a POST (along with the video's name).

On the Node server, we begin the process of moderating the video.  First, we call the [update video](https://docs.api.video/reference/api/Videos#update-a-video-object) endpoint to add the video's name and to tag the video "needsScreening" to indicate that it has entered the moderation queue.

### Transcoding

When the video is uploaded, api.video's servers begin the process of creating different size/bitrate videos to provide adaptive bitrate streaming.  We also create a mp4 version of the video. For content moderation, we need to submit the mp4 to Hive AI. The Node server pings api.video's [video status](https://docs.api.video/reference/api/Videos#retrieve-video-status-and-details) endpoint every 2 seconds to determine when the mp4 is ready. Initially, the API will indicate that the video is not playable (transcoding has not yet started). Once transcoding has started, the API lists the encoding status of every format being created, so we can monitor the encoding status of the mp4.

video status, video upload, upload a video

medium_screenshot-2020-10-23-at-13-36-55.jpg

small_screenshot-2020-10-23-at-13-36-55.jpg

thumbnail_screenshot-2020-10-23-at-13-36-55.jpg

screenshot-2020-10-23-at-13-36-55.jpg

Once the mp4 is created, we can create our connection to Hive AI, and make the request for content moderation.

The request looks like this:
```JSON
{
  method: 'POST',
  url: 'https://api.thehive.ai/api/v2/task/sync',
  headers: {
    accept: 'application/json',
    authorization: 'token {API TOKEN}'
  },
  form: {
    image_url: 'https://cdn.api.video/vod/vi1iHWIy6Doy0LBJl3ajaED0/mp4/1080/source.mp4'
  }
}
```

and a few seconds later, a huge JSON response comes back - 39 categories * x frames analysed.  Let's look at a snip of one frame to see the sort of information we get (for brevity, I've only included the first 5 categories that are returned):

```JSON
	{
		"class": "general_not_nsfw_not_suggestive",
		"score": 0.00460230773187999
	},
	{
		"class": "general_nsfw",
		"score": 5.180850871024288e-06
	},
	{
		"class": "general_suggestive",
		"score": 0.995392511417249
	},
	{
		"class": "no_female_underwear",
		"score": 0.9998768576722025
	},
	{
		"class": "yes_female_underwear",
		"score": 0.00012314232779748514
	}

```

"Not NSFW" means "Not Not Safe for Work" which, removing the double negative is "safe for work." This score is combined with "not suggestive", and is scored at 0.004.  Since 0 is a low score, and 1 is a high score, this means that the API has determined that this frame is not considered appropriate for work.  Looking at the next 2 values, general NSFW is also very small, but the "general suggestive" is 99.9%. Since the yes:no scores will always add up to one, this means that the general suggestiveness is what makes this frame not safe for work.

video moderator, automated content moderator, ai moderation

large_screenshot-2020-10-24-at-19-46-39.png

medium_screenshot-2020-10-24-at-19-46-39.png

small_screenshot-2020-10-24-at-19-46-39.png

thumbnail_screenshot-2020-10-24-at-19-46-39.png

screenshot-2020-10-24-at-19-46-39.png

### Compiling the scores

Hive AI gives you scores for each category on each frame, but it is up to us to define the pass/fail criteria for our videos.  

In order to do this, I take all the scores for each category, and place them in an array. With the data sequestered for each category, I can calculate the min, max, average and median of each score. I also calculate how many frames appear over 0.9 as a 90% certainty ("yes_smoking") and how many frames appear under 0.1 (90% certainty of "no_smoking.")  


medium_screenshot-2020-10-22-at-15-25-19.jpg

small_screenshot-2020-10-22-at-15-25-19.jpg

thumbnail_screenshot-2020-10-22-at-15-25-19.jpg

screenshot-2020-10-22-at-15-25-19.jpg

In the above "SFW" array (really the 'not not safe for work' response, but that is a mouthful), of 22 frames, I finde:
* min score: 0 (not safe for work)
* max score: 1 (safe)
* average:  0.55 right in the middle!
* median: 0.75 
* count of frames > 0.9: 9
* count of frames <0.1: 7

As you can see, 9 frames are over 0.9 (safe!), but 7 are below 0.1 (not safe!). If the median falls below 0.9, than means at least 50% of the frames are not "certain" to be safe for work ( which I place at the 90% threshold).  Based on these numbers, my rudimentary pass/fail algorithm deems the Baywatch intro "NSFW."  

If you wanted to prevent videos with smoking from appearing to your audience, just one frame with a 90% certainty of smoking would be enough to cause the video to be categorised as "yes_smoking." I used this same threshold for guns, nudity, underwear and swimwear. 

>Interestigly, this does push the Schindler's List trailer into the "yes smoking" category.

Based on these metrics, it is not terribly surprising that the Baywatch video is flagged for "yes female swimwear" and "yes shirtless male."  Also unsurprisingly, the algorithm found an absence of smoking and guns.

large_screenshot-2020-10-22-at-15-25-26.jpg

medium_screenshot-2020-10-22-at-15-25-26.jpg

small_screenshot-2020-10-22-at-15-25-26.jpg

thumbnail_screenshot-2020-10-22-at-15-25-26.jpg

screenshot-2020-10-22-at-15-25-26.jpg

Once we have the categories measured for each application, we can remove the tag ```needsScreening``` and add in the new tags from the moderation. This is done with the [Video Update](https://docs.api.video/reference/api/Videos#update-a-video-object) endpoint.  

And that ends the moderation process for the video.

### Video Categories

Now that each video has been categorised, it is easy to display each video category. Each video's categories have been added to the video as a tag and the [List videos](https://docs.api.video/reference/api/Videos#list-all-video-objects) endpoint allows us to search by tag - returning every video that does not have smoking, or is safe for work, etc.  Using Node, we can generate the list of videos (sorted newest first), and send them to the client:

```Javascript
app.get('/no_smoking', (req, res) => {
	//get list of no smoking videos
	client = new apiVideo.Client({ apiKey: apiVideoKey});
	let recordedList = client.videos.search({"tags":'no_smoking', "sortBy":"publishedAt","sortOrder":"desc"});
	recordedList.then(function(list) {
		console.log("list of tagged videos");
		console.log(list);
		return res.render('videos',{list});  
	}).catch((error) => {
	    console.log(error);
	});
		
});

```

The API returns the metadata for each video (including links for the video), so creating a page with each video iFrame is a pretty simple task.  On each page in the sample app, you can display (and watch) each of the videos assigned to a category.  In the demo, I use Pug for the rendering

```
 each video in list
    p #{video.title}
    iframe(type="text/html", src=video.assets.player, width = "960", height="540",frameborder="0", scrollling="no")
    p #{video.publishedAt}
   
```

For example, the "yes guns" page has 3 movie trailers (as I write this post): Indiana Jones and the Last Crusade, Die Hard and the latest James Bond,  

medium_screenshot-2020-10-24-at-19-51-28.jpg

small_screenshot-2020-10-24-at-19-51-28.jpg

thumbnail_screenshot-2020-10-24-at-19-51-28.jpg

screenshot-2020-10-24-at-19-51-28.jpg

## Conclusion

There we have it - we have uploaded a video to api.video, and before it is displayed on the site, it is moderated by Hive AI for several categories of inappropriateness.  Based on the analysis for each frame, the video is categorised into buckets and displayed on the appropriate page on the website.


If you have questions about using content moderation with your videos at api.video, feel free to reach out on our [community forum](https://community.api.video) or comment on the [GitHub repo](https://github.com/apivideo/videoModeration).  We'd love to see how you are using moderation to sort and categorise your videos, and the rubrics you utilise to decide what categories a video might fall into. If you haven't created your api.video account yet, you can do that in just a fewmomnets by follwoing [this link](https://dashboard.api.video/register). Happy building! 

Video moderation with machine learning

video content, machine learning, moderator, AI moderator

Video on demand

Live streaming

Analytics

Video infrastructure

See all features

Online learning

Short-form videos

Marketplace & E-commerce

Video messaging and recording

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Help center

Blog

Huge pricing update! Introducing video encoding at zero cost