Analytics

Video Sessions (Analytics)

Normal

doug.jpg

twitter-brands.svg

Doug Twitter

Doug Sillars

large_api-video-blog-tutorials.png

medium_api-video-blog-tutorials.png

small_api-video-blog-tutorials.png

thumbnail_api-video-blog-tutorials.png

api-video-blog-tutorials.png

Tutorials

JavaScript

NodeJS

How much of your video is being watched?  With api.video analytics, we can tell you exactly how much of the video was watched.  In this post, we'll walk through how to parse video session data in order to add up how much of the video was watched in a session.

## A video Session

A video session occurs when an api.video player is opened in a browser.  As long as the same window is used, each viewing of the video will be added to that one session.

You can get a list of your video sessions by using the [analytics endpoint](https://docs.api.video/analytics/overview) for a given videoId. In my code (using NodeJS), it looks something like this:

```
var params = { videoId, metadata, currentPage, pageSize};
  
const result = client.rawStatistics.listVideoSessions(params);
  result.then(function(videos) {
        var sessionList = videos.data;
        console.log("Number of sessions to test " , sessionList.length);

          for(i=0;i<sessionList.length;i++){
            //get each session ID, and figure out how long each session watched the video
            var sessionId = sessionList[i].session.sessionId;
            pageSize = '100';
            console.log(sessionId);

```


The Node endpoint takes my parameters (including the videoId I wish to analyze) and sends it to the ```client.rawStatistics.listVideoSessions``` endpoint.  This returns an array of the sessions that can be analyzed. 

The log will list a count of sessionIds, and then list each sessionId. Here is an example output from this code snippet:

```
Number of sessions to test  8
ps5orCkYsuaMw08btQmsW8vp
ps24iGEi011m9xKPqdTnIOvZ
ps7ODtZR04dfRBQCcarOIhv7
ps19mQmv5CTQwdeb75n8lLWr
ps1UdYeHbp6tQ21vrGUI8gXw
ps6xqBySKoSQfmG4zjcyZtWU
ps3cLLchT0Os75lzXovlCqTe
ps3MpNgw5Ax5EQ4GSAVyODRh
```

## get Session Events

In each session, the video player records each event, and sends it back to api.video.  Using the [Session event](https://docs.api.video/analytics/overview) API endpoint, we can retrieve each of these events.

```
 var sessionParams = {sessionId, currentPage, pageSize}; const sessionData = client.rawStatistics.listSessionEvents(sessionParams);
    sessionData.then( function(lastSessionData) {
      //this is the session data
      console.log("session events", lastSessionData);

```


The response of this endpoint looks something like this:

```
session events RawStatisticsListPlayerSessionEventsResponse {
  data: [
    PlayerSessionEvent {
      type: 'ready',
      emittedAt: 2021-06-17T19:00:54.000Z,
      at: 0,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'play',
      emittedAt: 2021-06-17T19:00:55.000Z,
      at: 0,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'end',
      emittedAt: 2021-06-17T19:02:27.000Z,
      at: 91,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'pause',
      emittedAt: 2021-06-18T10:59:03.000Z,
      at: 91,
      from: undefined,
      to: undefined
    }
  ],

```

In this case, the video is ready at time =0, begins playing, and then reaches the end at 91s.  The window is closed a day later, sending a pause event.  (this is just evidence that your author has a browser tab problem).

Using this data (and with the knowledge that the video is 91 seconds long), it's pretty easy to determine that the entire video was watched in this session.  from ready to the first pause is 91s, so the video must have been completed. But what about more complicated viewing patterns?  Let's look at an example:


```
  data: [
    PlayerSessionEvent {
      type: 'ready',
      emittedAt: 2021-06-21T12:33:06.000Z,
      at: 0,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'play',
      emittedAt: 2021-06-21T12:36:13.000Z,
      at: 0,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'pause',
      emittedAt: 2021-06-21T12:36:18.000Z,
      at: 81,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'resume',
      emittedAt: 2021-06-21T12:36:19.000Z,
      at: 81,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'seek.forward',
      emittedAt: 2021-06-21T12:36:19.000Z,
      at: undefined,
      from: 5,
      to: 81
    },
    PlayerSessionEvent {
      type: 'end',
      emittedAt: 2021-06-21T12:37:19.000Z,
      at: 91,
      from: undefined,
      to: undefined
    },
    PlayerSessionEvent {
      type: 'pause',
      emittedAt: 2021-06-21T13:10:18.000Z,
      at: 91,
      from: undefined,
      to: undefined
    }
  ],
```


In this session, we again start playing at time = 0, but the next set of events occur at 81s (pause/resume). So the video stopped and restarted at t=81. The next event tells us how we got to t-81 - the view skipped ahead (noted as a 'seek.forward' event) at t=5, to t=81. The video paused because the video had to load, and then from t=81, it played until the end at t=91.  

In this case, even though the playback started at t=0, and ended at t-91, no one would consider this video fully watched, as the viewer skipped 76s of the video.

## Calculating the time watched

To calculate the time watched, we'll add up the time between every play/resume event and every pause/end event.  Then we'll subtract any time in a 'seek.forward' event.

```
 for(j=0; j< numberOfEvents; j++){
        if(lastSessionData.data[j].type == "play" || lastSessionData.data[j].type == "resume"){
          //start time measure
          starttime = lastSessionData.data[j].at;
          counting = true;
          console.log ("start" , starttime);
        }else if(lastSessionData.data[j].type == "pause" || lastSessionData.data[j].type == "end"){
          if(counting){
            endtime = lastSessionData.data[j].at;
            counting=false;
            console.log ("stop" , endtime);
            console.log ("added time" , endtime-starttime);
            summedtime += endtime-starttime;
            
          }
        }else if(lastSessionData.data[j].type == "seek.forward"){
          console.log("seeked forward", (lastSessionData.data[j].to - lastSessionData.data[j].from));
          summedtime -= (lastSessionData.data[j].to - lastSessionData.data[j].from);
          //if there was a seekforward, the max time previously calculated previously is no longer valid and needs to be recalculated.
          maxWatchedTime = 0;
        }
        console.log("summedtime: ", summedtime);
				
				
}

```

We can log the math that happens as we iterate through the events:

```
summedtime:  0
start 0
summedtime:  0
stop 81
added time 81
summedtime:  81
start 81
summedtime:  81
seeked forward 76
summedtime:  5
stop 91
added time 10
summedtime:  15

```

Comparing these events - we can see that from the first start to the first pause is 81seconds, but then we encounter the seeked forward of 76 seconds - leading to a summed time of 5 seconds.  After the last 10 seconds are watched, we have a total of 15 seconds of the 91 second video watched.


## Playback totals

When watching required training videos, it is often mandatory that the time watched is logged and and recorded. With the api.video session data, we can track each user's viewing to know if they have completed a video, or if they must re-watch the video.

In a future post, we will combine this video watching algorithm with code to 'unlock' subsequent videos in a training series.

Video event analysis using api.video sessions

large_screenshot-2021-06-21-at-13-06-24.png

medium_screenshot-2021-06-21-at-13-06-24.png

small_screenshot-2021-06-21-at-13-06-24.png

thumbnail_screenshot-2021-06-21-at-13-06-24.png

screenshot-2021-06-21-at-13-06-24.png

Dynamic Metadata

When I read a book, I fold a corner over the page I am reading to remember my place.  Sometimes I fall asleep while reading, and it takes a few minutes to figure out where I had finished the night before, and I wish to myself that I had folded over the corner.

Watching video online is the same way - if you're interrupted while watching a long video (Amazon delivery at the door, child is hungry, cat needs to be let out... you know) it would be nice to resume right where you left off.

## Resuming videos

YouTube recently remembered where I had left off on a knitting tutorial.. that I had watched 3 years ago.  I have small children, so we often have to pause movies for days before we finish them, and it is great that Netflix recalls that we turned off the TV at 42:11 into the feature.  No more scanning and seeing potential spoilers a scene or two ahead.

We can do the same with api.video's videos.  We've released a demo at [resume.a.video](https://resume.a.video) where you can test this feature out.



## Resuming Videos with api.video

### Identifying users

For each video that you'd like to resume, you should create a [dynamic metadata](https://api.video/blog/tutorials/dynamic-metadata/) attribute that you will use to identify each user. For the videos in the demo, I created a dynamic metadata with key "userName"

screenshot showing metadata setting for video

large_screenshot-2021-01-11-at-15-46-51.png

medium_screenshot-2021-01-11-at-15-46-51.png

small_screenshot-2021-01-11-at-15-46-51.png

thumbnail_screenshot-2021-01-11-at-15-46-51.png

screenshot-2021-01-11-at-15-46-51.png

>You can create a Dynamic metadata with the [Update video](https://docs.api.video/reference#patch-video) endpoint.  For the 'key' use the key.  For 'value' add two'_' before and after the term. The API will know that this value can have any potential value.

> You can also use the GUI at go.api.video to add the metadata.

We can use this to uniquely identify users who watch the video.  I would recommend using a unique non-identifiable string for each user, but for simplicity, I'll use first names in this demo. In this way, we can identify who is watching the video:

https://embed.api.video/vod/vi4fvaDG9nqfdMTqRnmZuvlJ?metadata[userName]=Doug

This video will record its session with the metadata userName: Doug.  

Awesome - assuming we know the user's name, we can insert this URL paramater and now identify all of their sessions with the video.


### Identifying user sessions

When you use the api.video player, each session has video analytics recorded (and each session is based on a cookie set in the browser). Using the [analytics video session](https://docs.api.video/reference#get_analytics-videos-videoid) endpoint, you can search each session for a video played.  You can limit this search with the metadata term. In NodeJS, this looks like:

```
var firstSessionQuery = {
			method: 'GET',
			url: 'https://ws.api.video/analytics/videos/'+videoId+'?currentPage=1&pageSize=1&metadata[userName]='+userName,
			headers: {
				accept: 'application/json',
				authorization: 'Bearer ' +authToken
			}
		}
		request(firstSessionQuery, function (error, response, body) {
			if (error) throw new Error(error);
			body = JSON.parse(body);
		}

```

The request url uses the videoId and userName to get just the sessions for the user.  The response can return from 1 to 100 results, with the most recent session appearing at the end. In this query, I only request the first session:

```
data {
  session: {
    sessionId: 'ps5qU6t436lzoDftbAfuJPzP',
    loadedAt: '2021-01-05T12:52:21+00:00',
    endedAt: '2021-01-05T12:52:53+00:00',
    metadata: [ [Object] ]
  },
  location: { country: 'United Kingdom', city: 'Watford' },
  referrer: {
    url: null,
    medium: 'unknown',
    source: 'unknown',
    searchTerm: 'unknown'
  },
  device: { type: 'computer', vendor: 'unknown', model: 'Other' },
  os: { name: 'Mac OS X', shortname: 'Mac OS X', version: '11.0.1' },
  client: { type: 'unknown', name: 'Chrome', version: '87.0.4280' }
}
pagination {
  currentPage: 1,
  currentPageItems: 1,
  pageSize: 1,
  pagesTotal: 23,
  itemsTotal: 23,
  links: [
    {
      rel: 'self',
      uri: '/analytics/videos/vi4fvaDG9nqfdMTqRnmZuvlJ?currentPage=1&pageSize=1&metadata%5BuserName%5D=Doug'
    },
    {
      rel: 'first',
      uri: '/analytics/videos/vi4fvaDG9nqfdMTqRnmZuvlJ?currentPage=1&pageSize=1&metadata%5BuserName%5D=Doug'
    },
    {
      rel: 'next',
      uri: '/analytics/videos/vi4fvaDG9nqfdMTqRnmZuvlJ?currentPage=2&pageSize=1&metadata%5BuserName%5D=Doug'
    },
    {
      rel: 'last',
      uri: '/analytics/videos/vi4fvaDG9nqfdMTqRnmZuvlJ?currentPage=23&pageSize=1&metadata%5BuserName%5D=Doug'
    }
  ]
}

```
 
Ok, so when I created this video in January 2021, I watched it, to ensure that it worked.  You can see my approximate location (it's really where my ISP exits to the internet), and I was on a Mac running Chrome.

So, why just one entry?  We want to get the last session that was viewed, and that is always the last session in the list.

```
var sessionCount = body.pagination.itemsTotal;
```

There are really only three possibilities to worry about here:

* sessionCount = 0.  This is a new viewer of the video, and there are no sessions saved.  If there are no sessions for the user - we can immediately serve the page, and start the video at the beginning:

```
				//no sessions for this user
				//return the video at time =0
				var videoUrl = 'https://embed.api.video/vod/'+videoId+'?metadata[userName]='+userName+'#autoplay';
				return res.render('video', {videoUrl});
```
This will load the page with the video autoplaying.

* sessionCount=1. The viewer has watched the video once.  If that is the case, the sessionId returned in the response will be the session we want.  You can now skip ahead to the "parsing session data" section.
* sessionCount >1.  There is more than one session for this user. In the example above, the itemsTotal parameter indicates that there are 23 sessions for userName=Doug.

The API has a limit of 100 sessions in the video analytics response. In general, one user is unlikely to have >100 sessions for a single video, so one query would probably be sufficient. However, this is a demo app with a default user - it is possible that we might exceed 100 sessions.  If I queried for 100 sessions, the demo would appear to fail - we'd only ever see the 100th session. So, we request one, and then to ensure that we always get the last session, we can now make a second query for the last session (conveniently in the pagination.links.last.uri).

```
var lastUri = decodeURI(body.pagination.links[linksLength-1].uri);
					console.log("number of sessions with this username:", lastSessionCounter);
					console.log("uri", lastUri);
					//request that last session
					var lastSessionQuery = {
						method: 'GET',
						url: 'https://ws.api.video' +lastUri,
						headers: {
							accept: 'application/json',
							authorization: 'Bearer ' +authToken
						}
					}
					request(lastSessionQuery, function (error, response, body) {
						if (error) throw new Error(error);
						body = JSON.parse(body);
						console.log("body", body);
						console.log(body.data[0]);
						var sessionId = body.data[0].session.sessionId;
						console.log(sessionId);
						//now we can access this sessionId, and get the last time in the session
						getSession(authToken, videoId, sessionId, userName);

```

This response now tells me the last session for Doug and this video:

```
 session: {
    sessionId: 'pshSgwO44hBbc0nYKz1OZsz',
    loadedAt: '2021-01-11T19:02:43+00:00',
    endedAt: '2021-01-12T11:35:49+00:00',
    metadata: [ [Object] ]
  },
  location: { country: 'United Kingdom', city: 'Slough' },
  referrer: {
    url: 'http://localhost:3006/',
    medium: 'unknown',
    source: 'unknown',
    searchTerm: 'unknown'
  },
  device: { type: 'computer', vendor: 'unknown', model: 'Other' },
  os: { name: 'Mac OS X', shortname: 'Mac OS X', version: '11.0.1' },
  client: { type: 'unknown', name: 'Chrome', version: '87.0.4280' }
}

```

Again - on my Mac - but this time, I am in Slough, UK (and I was testing this app locally on my computer).  The sessionId I want is 'pshSgwO44hBbc0nYKz1OZsz'

### Parsing session data

If there was a previous session for this user - we now have the last session (and more importantly the sessionId for that session). It may have just been one request - or it may have taken two.

I use the [session events](https://docs.api.video/reference#get_analytics-sessions-sessionid-events) endpoint to find out what happened in the session:

```
	function getSession (authToken, videoId, sessionId, userName) {
		//lets get 100 session activities - i doubt there will ever be that many - (or more than 100)
		var getSession = {
			method: 'GET',
			url: 'https://ws.api.video/analytics/sessions/'+sessionId+'/events?currentPage=1&pageSize=100',
			headers: {
				accept: 'application/json',
				authorization: 'Bearer ' +authToken
			}
		}
		request(getSession, function (error, response, body) {
			if (error) throw new Error(error);
			body = JSON.parse(body);
			console.log("sessions", body.data);
			var numberofEvents = body.data.length;
			console.log("num events", numberofEvents);
			var lastTimeRecorded = body.data[numberofEvents-1].at;
			console.log("last event time", lastTimeRecorded);
	
			var videoUrl = 'https://embed.api.video/vod/'+videoId+'?metadata[userName]='+userName+'#autoplay;t='+lastTimeRecorded;
					return res.render('video', {videoUrl});
	
		});
	
	}
	
	  
});

```

So the API calls the session endpoint with the sessionId we discovered in the last section.  We get a response (write to console log) and we extract how many events were in the last session, and find out what the last event was. 

>You'll note that in the comments I make an assumption that there will never be 100 events (play, pause, seek, etc.) in a single session. I think this is safe  to make :)

The session response looks like this:

```
sessions [
  { type: 'ready', emittedAt: '2021-01-12T11:59:42+00:00', at: 0 },
  { type: 'play', emittedAt: '2021-01-12T11:59:42+00:00', at: 0 },
  { type: 'pause', emittedAt: '2021-01-12T11:59:45+00:00', at: 2 },
  {
    type: 'seek.forward',
    emittedAt: '2021-01-12T11:59:46+00:00',
    from: 2,
    to: 163
  },
  { type: 'resume', emittedAt: '2021-01-12T11:59:47+00:00', at: 163 },
  { type: 'pause', emittedAt: '2021-01-12T11:59:49+00:00', at: 165 }
]
num events 6
last event time 165

```

So I started the video at time =0, and then at time t=2, I jumped ahead "seek.forward" to 163s.  Then I paused at 165s.

**So the last time viewed was at 165s.**

### Starting the video

We have now found all the sessions for the user.  We grabbed the last (most recent) session, and then found the last event in that session - to give us the last time that the user saw the video.

In HTML5, if you add #t=X, where X is an integer to the end of a video URL, the video playback will begin at that time. Simply returning the url with this parameter will start the video at the last time the user viewed the video.

## Summary

With Dynamic metadata, we are able to assign video watching sessions to specific customers.  We can then search a specific user's sessions for the most recent session, and inside that session, parse all the events that occured during playback. 

Using the last session, and the last event, we get the last time that the viewer interacted with the video.  We can now use this time as the "resume time" for the video, and stat the video at that time.

Don't forget to try it out: [resume.a.video](https://resume.a.video)

This is a nice feature that makes watching longer videos easier - your viewers know they can come back and quickly pick up where they left off.  

Are you thinking about using 'resume' in your videos?  The code is all up on [Github](https://github.com/dougsillars/resumeAVideo) Let us know - we'd love to see if it improves the engagement of your viewers! Leave us a comment in the [community](https://community.api.video) forum.

Resume a video

medium_screenshot-2021-01-11-at-13-29-09.jpg

small_screenshot-2021-01-11-at-13-29-09.jpg

thumbnail_screenshot-2021-01-11-at-13-29-09.jpg

screenshot-2021-01-11-at-13-29-09.jpg

You have created the perfect video, uploaded it to api.video, and embedded the player on your website.  Are you curious about how many people are looking at your video?  How about where they are in the world?

Maybe you want to get even more granular, and know how much of the video is being watched?  Are they rewinding and watching parts again?  Are they skipping ahead?  With api.video’s analytics, you can find out.

In this post, we’ll take a look at how to extract this information for one video, and in future posts, we will go into deeper depth into building analysis tools.


## My Video





I’ve used this video of Tuba players playing the Beatles in other tutorial posts.  So, we know it will have a lot of views. But, how many?  Using the [List Video sessions](https://docs.api.video/analytics/overview) endpoint, we can get information about each time this video was played with the api.video player.

To use this API, you need 2 items:  a ```videoId``` of the video you want to measure, and an authentication token.  In order to [authenticate your session](https://docs.api.video/reference/basic-authentication), you need your api.video token.  Read the details in the [authentication tutorial](https://api.video/blog/tutorials/authentication-tutorial/), and then come back with your token (it will be valid for one hour).

In this example, we’ll use curl to access the video session endpoint.  The videoId is ```vi4blUQJFrYWbaG44NChkH27```.  The curl request looks like:

```
curl --request GET \
  --url 'https://ws.api.video/analytics/videos/vi4blUQJFrYWbaG44NChkH27?currentPage=5&pageSize=100' --header 'authorization: Bearer {token}'   --header 'accept: application/vnd.api.video+json'
```

I am requesting page 5 (currentPage in the url), with pageSize=100 (100 sessions per page, since I knew that this video had between 400 and 500 views.  For simplicity, I’ll show just the 2 most recent sessions:

```json
{
	"data": [
		{
			"session": {
				"sessionId": "ps4mPLq8bZzjXlxwJG4cg4vl",
				"loadedAt": "2020-10-12T11:21:09+00:00",
				"endedAt": "2020-10-12T11:27:20+00:00"
			},
			"location": {
				"country": "United Kingdom",
				"city": "Glasgow"
			},
			"referrer": {
				"url": null,
				"medium": "unknown",
				"source": "unknown",
				"searchTerm": "unknown"
			},
			"device": {
				"type": "computer",
				"vendor": "unknown",
				"model": "Other"
			},
			"os": {
				"name": "Mac OS X",
				"shortname": "Mac OS X",
				"version": "10.15.6"
			},
			"client": {
				"type": "unknown",
				"name": "Chrome",
				"version": "85.0.4183"
			}
		},
		{
			"session": {
				"sessionId": "ps42gcSLNv8hSQxKOWWgxeNj",
				"loadedAt": "2020-10-12T11:43:56+00:00",
				"endedAt": "2020-10-12T11:44:19+00:00"
			},
			"location": {
				"country": "Netherlands",
				"city": "Uithoorn"
			},
			"referrer": {
				"url": "https://t.co",
				"medium": "social",
				"source": "Twitter",
				"searchTerm": "unknown"
			},
			"device": {
				"type": "computer",
				"vendor": "unknown",
				"model": "Other"
			},
			"os": {
				"name": "Mac OS X",
				"shortname": "Mac OS X",
				"version": "10.15.7"
			},
			"client": {
				"type": "unknown",
				"name": "Safari",
				"version": "14.0"
			}
		}
	],
	"pagination": {
		"currentPage": 5,
		"currentPageItems": 22,
		"pageSize": 100,
		"pagesTotal": 5,
		"itemsTotal": 422,
		"links": [
			{
				"rel": "self",
				"uri": "/analytics/videos/vi4blUQJFrYWbaG44NChkH27?currentPage=5&pageSize=100"
			},
			{
				"rel": "first",
				"uri": "/analytics/videos/vi4blUQJFrYWbaG44NChkH27?currentPage=1&pageSize=100"
			},
			{
				"rel": "previous",
				"uri": "/analytics/videos/vi4blUQJFrYWbaG44NChkH27?currentPage=4&pageSize=100"
			},
			{
				"rel": "last",
				"uri": "/analytics/videos/vi4blUQJFrYWbaG44NChkH27?currentPage=5&pageSize=100"
			}
		]
	}
}
```
Let’s start with the information at the bottom.  We can get links for each page of sessions, and itemsTotal is 422 - meaning there have been 422 sessions watching this video.  

Let’s look at the 2 sessions in the logs.  The first session has sessionId = ps4mPLq8bZzjXlxwJG4cg4vl.  This user watched the video for about 6 minutes based on the session length:
				"loadedAt": "2020-10-12T11:21:09+00:00",
				"endedAt": "2020-10-12T11:27:20+00:00"

We also know that they are in Glasgow, UK, using Chrome 85 on a Mac..  It just so happens that my ISP enters the internet in Scotland (all my advertising is Scotland based) even though my physical location is England, so this session is from my computer.

The 2nd session is interesting:

	"location": {
				"country": "Netherlands",
				"city": "Uithoorn"
			},
			"referrer": {
				"url": "https://t.co",
				"medium": "social",
				"source": "Twitter",
				"searchTerm": "unknown"
			},
			
I accidentally tweeted the url to this video, and deleted the tweet within 30 seconds,  Yet, someone on Twitter in the Netherlands still got to see the video before the tweet disappeared.

> Did you know that the api.video player works in Twitter?  Now you do! Your analytics will show video plays from any share on Twitter!

### Applications

With this data - you could build a map of where the video is being watched by country, and in some cases by city (with the caeat that this is not 100% accurate - my ISP says I am in Scotland).  You can also understand the referring urls: Twitter, blogs, Linkedin, etc.  Finally, you can also monitor the traffic patterns by day of week, time of day, etc.


## Going even deeper 

It is great to know that there was a video view at 11:21 GMT - but we can learn even more about this video view.  Using the [session events](https://docs.api.video/reference/api/Analytics#get_analytics-sessions-sessionid-events) endpoint and the sessionId, we can actually describe the exact usage of that session.  Making a request using a sessionId (note this is a different - more interesting - session):

 ```
--request GET \
   --url 'https://ws.api.video/analytics/sessions/ps42gcSLNv8hSQxKOWWgxeNj/events?currentPage=1&pageSize=100' --header 'accept: application/vnd.api.video+json' --header 'authorization: Bearer {token}'  --header 'accept: application/vnd.api.video+json'
```

This request will return all of the video player events in that session: so we can see any play, pause or seek events that occurred in playback:

```json
{
	"data": [
		{
			"type": "ready",
			"emittedAt": "2020-10-11T20:43:32+00:00",
			"at": 0
		},
		{
			"type": "play",
			"emittedAt": "2020-10-11T20:43:32+00:00",
			"at": 0
		},
		{
			"type": "pause",
			"emittedAt": "2020-10-11T20:43:57+00:00",
			"at": 2
		},
		{
			"type": "resume",
			"emittedAt": "2020-10-11T20:44:00+00:00",
			"at": 18
		},
		{
			"type": "seek.forward",
			"emittedAt": "2020-10-11T20:44:00+00:00",
			"from": 2,
			"to": 18
		},
		{
			"type": "pause",
			"emittedAt": "2020-10-11T20:44:01+00:00",
			"at": 19
		},
		{
			"type": "resume",
			"emittedAt": "2020-10-11T20:44:04+00:00",
			"at": 2
		},
		{
			"type": "seek.backward",
			"emittedAt": "2020-10-11T20:44:04+00:00",
			"from": 19,
			"to": 2
		},
		{
			"type": "pause",
			"emittedAt": "2020-10-11T20:44:11+00:00",
			"at": 9
		}
	],
	"pagination": {
		"currentPage": 1,
		"currentPageItems": 9,
		"pageSize": 100,
		"pagesTotal": 1,
		"itemsTotal": 9,
		"links": [
			{
				"rel": "self",
				"uri": "/analytics/sessions/psyd0zH6RWUCxP7xnWqcnKm/events?currentPage=1&pageSize=100"
			},
			{
				"rel": "first",
				"uri": "/analytics/sessions/psyd0zH6RWUCxP7xnWqcnKm/events?currentPage=1&pageSize=100"
			},
			{
				"rel": "last",
				"uri": "/analytics/sessions/psyd0zH6RWUCxP7xnWqcnKm/events?currentPage=1&pageSize=100"
			}
		]
	}
}
```

The video was ready to play (entry 0) and started to play (entry 1) at ```2020-10-11T20:43:32+00:00```.  (That is expected, as the [player](https://api.video/blog/tutorials/tutorial-video-player-customisation/) associated with this video is set to autoplay as soon as it is ready). The video is also set to loop, which is evident in the next entry.

The third entry shows that the video was paused at ```2020-10-11T20:43:57+00:00```.  The video timestamp shows “2” meaning 2 seconds into the video.  About 25 seconds elapsed since the video started, so how does that work?  The video is about 20 seconds long, meaning that the video played to the end and started again before the pause occurred.  

The next entries show that there is a seek.forward event from 2->18s, and the video resumed for one second before rewinding back to 2 and then playing to 9 seconds into the video.

Aggregating these session stats across all sessions can give you an idea of when users rewind - perhaps when an important point is raised  - indicating that in future videos that data should be highlighted for your users.  You can also see if there is early abandonment, or jumps ahead - as users look for the content that they are interested in.

### Fast analytics

These results are available as soon as the session is completed.  To see how quickly, I recorded a video where I watched the video, and then retrieved the analytics immediately afterward.  




## Conclusion

When you use the api.video player, there are a number of interesting analytics collected. Beyond knowing the number of views, the geo-location of the views and the time that the videos were viewed, you can also dig into the analytics of each session to better understand how your videos are being played. If you have questions, please reach out in the [community forum](https://community.api.video)


Video analytics: A primer

medium_abacus6.jpg

small_abacus6.jpg

thumbnail_abacus6.jpg

abacus6.jpg

In the classic movie *Field of Dreams*, a mysterious voice whispers to Kevin Costner “if you build it, they will come.”  


<video autoplay muted loop>
		<source src="https://media.giphy.com/media/gZuxOq7zSL5DO/giphy.mp4">
</video><br/><br/>


His belief of this statement completely transformed his life and drove him to build a baseball stadium in the middle of a cornfield in Iowa… and they came…<br/>


<video autoplay muted loop>
		<source src="https://media.giphy.com/media/z3WMjg3hu3ce4/giphy.mp4">
</video><br/><br/>


## If you Stream it...

You've created some awesome videos, and uploaded them to api.video.  Great work - it looks awesome.  

Here is a video that we posted in our blog post announcing our new video player:

<iframe src="https://embed.api.video/vod/viTLpKB231ATWOWjkHgi9bf#autoplay#loop" frameborder="0" scrolling="no" allowfullscreen=""></iframe>
<br/><br/>

## ...Who will come?

Eventually you want to know, how many people came to see my video?  What can I learn about them? Enter api.video’s powerful [video analytics](https://docs.api.video/reference/api/Analytics) API. Using the API, we can transfer all of the view data form our videos into our favourite database, and learn about the viewers of our videos.

Let’s start with grabbing the viewing details for a video. In this case, I’ll use this Planets video that my daughter recorded for her class on astronomy:

<iframe src="https://embed.api.video/vod/vi16NDdXmGxk5QSggW54qw0v#autoplay#loop"  frameborder="0" scrolling="no" allowfullscreen=""></iframe>

The url for this video is: https://embed.api.video/vod/vi16NDdXmGxk5QSggW54qw0v

First I must [authenticate](https://api.video/blog/tutorials/authentication-tutorial/) myself at API.video, and I receive an access token that is good for an hour.

With the token, I can use the videoId to request all of the view information about my video with the [analytics API](https://docs.api.video/reference/api/Analytics).  The videoId for the video is the value at the end of the url (vi16NDdXmGxk5QSggW54qw0v):




In this case, I am requesting 25 results per page, and page one of the data.  That’s actually fine, since I know there are only 19 views on this video.  

The JSON result is pretty long, but here is what one entry looks like:


To me, this is super cool.  A person in Vietnam, using Firefox on Ubuntu, watched this video in June!  And there are 18 more of these to look at and to parse.

### Demo

We have created a demo that takes a videoId and counts the country, device and browser for each view, and places the results on a page. You can try it out at [whoswatched.a.video](https://whoswatched.a.video).

In order to look at the data, I am going to suck the JSON file into Couchbase - a database that uses JSON to create a "table" format similar to SQL. This will allow me to run queries on the JSON much like one would run a database query.

### Creating a database in Couchbase:

Download and install [Couchbase](https://www.couchbase.com/) on your computer.  When the install is complete, you’ll have an admin dashboard open in your browser (on my Mac it is running on localhost:8901).  We’d like to add in our api.video JSON file into a data bucket.  To create the bucket - navigate to the Bucket section and click the "ADD BUCKET" button. I have the memory set at the minimum 100 MB (there are all of 19 entries after all!).


large_screenshot-2020-08-13-at-13-01-16.png

medium_screenshot-2020-08-13-at-13-01-16.png

small_screenshot-2020-08-13-at-13-01-16.png

thumbnail_screenshot-2020-08-13-at-13-01-16.png

screenshot-2020-08-13-at-13-01-16.png

Great - now we have our data bucket added.  Now we’ll want to import the JSON from api.video, so we can do our analysis.  This is done in the terminal using the ```cbimport``` command:

My username is Administrator with password 123456 (Google Chrome is mighty peeved about my use of an insecure password on localhost), adding a json file into the videoStats bucket.  Each video view has a sessionId, so I use that as a unique key.

I get a success message in my terminal, and looking back at the Couchbase admin panel under Documents, I can see each entry from the JSON has been added:


medium_screenshot-2020-08-13-at-13-12-54.jpg

small_screenshot-2020-08-13-at-13-12-54.jpg

thumbnail_screenshot-2020-08-13-at-13-12-54.jpg

screenshot-2020-08-13-at-13-12-54.jpg

Now we must index the data. In the Query Editor, enter:

Now we can do data analysis of our video views. For example, to understand the number of views per country:

small_screenshot-2020-08-13-at-13-14-20.png

thumbnail_screenshot-2020-08-13-at-13-14-20.png

screenshot-2020-08-13-at-13-14-20.png

That’s pretty cool!  


## Live Stream Analytics

We can do exactly the same thing with live stream analytics. Note: these only show “views” of a live stream.  If you run a live stream and no one watches - there will be no analytics reported.  Further - if your live streams are saved as video on demand (VOD) for later use - any VOD analytics are saved with the newly created VOD video, and will have no connection to the original live stream.

Let’s look at a live stream that has a number of views.  On the [api.video](https://api.video) homepage, we have a demo that allows you to livestream right from the homepage.  The live streamer then “watches” their stream in the browser at the same time that they are broadcasting - guaranteeing that every live stream gets at least one viewer (in the same location as the live streamer!)

The API call is similar to that of the VOD request, I just use the 'live-streams' endpoint as opposed to the 'videos' endpoint I used above.

### GeoLocation  


The live stream demo is hosted in North America and in Europe, and we use geolocation to direct our users to the correct server. Let’s see how well our load balancers are doing. Traffic directed to North America is on the left, and European traffic is on the right:


large_screenshot-2020-08-13-at-13-16-21.png

medium_screenshot-2020-08-13-at-13-16-21.png

small_screenshot-2020-08-13-at-13-16-21.png

thumbnail_screenshot-2020-08-13-at-13-16-21.png

screenshot-2020-08-13-at-13-16-21.png

It looks like our load balancing is doing a bang up job. :)


Combining the two sets of data (I just created a new bucket with all the data added) allows us to see that api.video’s home page has been live streaming all around the world:


Couchbase results in Google Sheets to create this chart.

medium_screenshot-2020-08-13-at-13-17-03.jpg

small_screenshot-2020-08-13-at-13-17-03.jpg

thumbnail_screenshot-2020-08-13-at-13-17-03.jpg

screenshot-2020-08-13-at-13-17-03.jpg

### Devices & browsers

Most of the streams are from desktop machines. All 28 phone instances correspond directly to 28 Android devices.(This is due to a limitation in Safari that prevents the demo from working on any iOS device.)  Of our Android users, we see 25 using Chrome, 1 Samsung Internet, 1 Opera (and one hidden variant.)

Desktop users are mostly Windows and Mac (and pretty evenly distributed), but also a good number on Linux/Ubuntu).


small_screenshot-2020-08-13-at-13-33-04.png

thumbnail_screenshot-2020-08-13-at-13-33-04.png

screenshot-2020-08-13-at-13-33-04.png

thumbnail_screenshot-2020-08-13-at-13-33-10.png

screenshot-2020-08-13-at-13-33-10.png

thumbnail_screenshot-2020-08-13-at-13-33-15.png

screenshot-2020-08-13-at-13-33-15.png

## Conclusion


The video analytics offered by api.video allow you to better understand who is consuming your content. In this post, we've walked through the steps to extract the data using the Analytics API in JSON format, and then ingested the data into a database for fast and easy queries.  

We used this data to see how the live stream demo on the api.video home page was used in July 2020.

How are you using api.video's analytics? Drop us a note in the [community forum](https://community.api.video) with your use cases!

Analytics with api.video

Announcements

small_cedric.webp

thumbnail_cedric.webp

cedric.webp

Cédric Montet

Company News

AI, models, assistants, agents and prompts sound and look magical on social networks. But from the countless conversations we have had with startups and Enterprise customers over the last months, we have realized that the gap between existing video workflow and concrete use cases is real.

&nbsp;

Whether it’s getting transcripts in seconds, changing the language of your video or being able to search for items in your content – the benefits, value or potential savings you see and can get with AI are obvious.

&nbsp;

This is why we are thrilled to unveil how we are bringing AI into video, where we are heading and how you can get more value from it!

## What does the future of video streaming look like?

AI will completely upend video consumption as we know it today. We envision a world where video is not a content piece anymore which you can only start and stop – because AI will make videos discoverable and universally understandable. More than ever it resonates fully with our core mission – connecting people through their cameras.

&nbsp;

Since Day 1, we are building api.video to make your use cases, first, a reality and, then, a success.

&nbsp;

Back in the days, the fragmented video streaming market, the number of services you had to aggregate, the lack of innovation by the incumbents was preventing you from joining our video-first world, while your efforts and budgets were consumed by maintaining mashups instead of focusing on what matters. 

&nbsp;

This is why we have built the video backbone and seeing that, now, you host 2.5 millions videos on api.video and deliver dozens of millions videos every month demonstrate why simplicity, exceptional performances and an E2E approach were key.


&nbsp;

What’s happening today with AI, the efforts it requires to benchmark, setup and customize LLMs, to run and maintain them, the Infrastructure you have to build to execute them before getting any value, remind us of the state of the streaming market when we had launched api.video.

&nbsp;

Helping you to benefit from the AI revolution by abstracting all the underlying complexity became obvious and a natural next step in our evolution.

&nbsp;

To get there and to bring you there, here is what we are building, api.video’s very first – **Video Model Hub**.

&nbsp;

Let’s start with the root principles encompassing our work on this Hub:

&nbsp;

- **Open-source models over private ones.** We are long time believers and contributors of the open source community and we have chosen to run only models that are available publicly.

- **Your data is… your data.** You already trust us to host and deliver sensitive content,and this is exactly why we won’t train our own models or sell the data generated from your videos to anyone.

- **Privacy at the core.** We host and execute the models inside our own infrastructure. Your data remains inside the api.video infrastructure. Your videos or data is not sent to external partner, SaaS providers like OpenAI, or others.

- **We believe that every organization will eventually build their own models to leverage their unique data sets**. Next year, alongside the ability to run public and open source models on api.video, we will offer you the ability to host your own.

- **One API to rule them all.** As we have already replaced a dozen of services and vendors under GET/videos to make your videos available worldwide in seconds, we are abstracting all the complex work of executing LLMs behind clear and straightforward endpoints.

&nbsp;

At a glance, api.video brings to you today the power of AI by:

- Leveraging its years of experience in video management and building infrastructure, with GPUs.
- Combining with the most advanced LLMs available on the market.

so that you get more value from your videos in a few lines of code!

&nbsp;

## Here are the first three new features from the Video Model Hub

### 1. Summarization of long-form videos

Upload a video in English, Spanish, French, Portuguese or Chinese and get a text summary of 120 characters with the 3 key takeaways discussed in the video (within seconds!). Ideal to index or preview long form recorded meetings or courses!

&nbsp;

_The video below gives you a glimpse into AI summarization._


### 2. Transcription of your video in seconds

Get a transcript (in .VTTformat) from your uploaded video with the timecodes and offer the captions to your audiences. The best part? The api.video player supports VTT files so there’s nothing new that you need to build on your side!
This also makes your content more inclusive by making it accessible to people with hearing impairments. 


### 3. Translation of subtitles in 5 different languages (and more to come!)

Is your video in English? You can get Spanish and French subtitles. In Chinese? You will get English subtitles and three more languages. In short, this is the perfect solution to spread your message to your international audience!

&nbsp;

_Check out the video below for a glimpse into AI translation._ 


_Try it for yourself now by uploading a video in your [sandbox account](https://dashboard.api.video/overview)._ 

## What’s next

From video classification, alteration, generation, audio alteration, metadata generation, we have identified and plan to implement more than 200 additional use cases and features within the upcoming months!

&nbsp;

From next year, we will let you bring your own models on the infrastructure to get even better and more personalized outcomes.

&nbsp;

This is a long-journey and we’re only getting started. The whole team is thrilled to be at the forefront of this next chapter!


AI  >< Video: The Video Model Hub unveiled

small_AI thought leadership.png

thumbnail_AI thought leadership.png

AI-thought-leadership.png

Arushi Gupta

Product updates

Our mission at api.video has always been to turn your use cases into successful realities. We built the video backbone with this goal in mind, and today, you host 2.5 million videos and deliver tens of millions more!

&nbsp;

Helping you harness the power of AI, while simplifying the problems that come along with, was the next direction to take. 

&nbsp;

To achieve this and to support your journey, we bring to you –  **AI features for your videos.**



## What are the AI video features?

### 1. AI video transcription & captions

With AI transcription, you can convert spoken words in your videos into written text, providing an accurate, searchable record of your content. This means you can easily create subtitles, enhancing accessibility for the hearing impaired or getting a written record of your meetings or events. 






### 2. AI video translation

With AI translation, you can translate your videos into multiple languages, ensuring that viewers from different linguistic backgrounds can understand and engage with your content. This can be useful if you’re hosting an international virtual event, creating multilingual educational materials, or expanding your marketing reach.

&nbsp;

Translation will be available for 5 languages – French, English, Spanish, Portuguese and Chinese.


### 3. AI video summarization

Using AI summarization, you can distill lengthy content into concise, digestible summaries, allowing viewers to quickly grasp the key points without watching the entire video with AI video summarization. This can prove invaluable for educational lectures, business presentations, and lengthy seminars where time is of the essence.

## Do you really need this?

Well, the potential [uses of our AI features](https://api.video/use-cases/generative-ai/) are limitless! From e-learning and e-commerce to online streaming, these [AI features](https://api.video/product/ai-features/) are going to be the difference between ordinary and excellent. What can you do really?

&nbsp;

- Summarize educational lectures for a quick review
- Translate virtual events into multiple languages for global reach
- Transcribe video interviews for easy reference
- And so much more!

## More than just encoding, hosting, and delivery

We’ve gone beyond the basics of video hosting and delivery to give you features that are a step up from the ordinary. Now, not only can you ensure that your customers get video delivery at minimal cost, but also that they get the flexibility to offer content  in their preferred language and in a crisp manner. 

&nbsp;

Additionally, when the features are in production, you may also get the option of adding them via your API. This means that you get complete power to add the features you like without having to go through the dashboard.

## Your videos stay private to you

Your data is yours, and we respect that. You already trust us to host and deliver your most sensitive content, which is why we will never train our own models or sell data generated from your videos. Privacy is at the core of everything we do, and that’s why we host and execute the models within our own infrastructure. Your data stays entirely within the api.video ecosystem and is never sent to external partners, SaaS providers like OpenAI or others.

&nbsp;

_Our CEO and founder, Cedric Montet, speaks more about privacy being our topmost concern in his latest [blog](https://api.video/blog/company/video-model-hub-ai/)._

## Give it a try!

See a glimpse how these AI features look like with a free sandbox account! [Upload a video now in your dashboard](https://dashboard.api.video/overview) and witness AI make a difference to your videos. As this is a Beta release, any feedback, questions, doubts on the features are welcome and much appreciated.

Create a sandbox account now 

Lights, Camera, AI Action! Discover our AI features

Introducing AI features to compliment your videos

small_Introducing AI features to compliment your videos.png

thumbnail_Introducing AI features to compliment your videos.png

AI-features-to-compliment-your-videos.png

AI >< Video: The Video Model Hub unveiled

How to integrate AI facial expression detection to control your video – Javascript 

5 AI tools that will save you 10+ hours on video production

Lights, Camera, AI Action! 

Video trends

In a world where time is money and creativity reigns supreme, AI tools that convert text to video are game-changers! These AI-powered wonders are designed to save your day—and countless hours. Join us as we unveil the top 5 AI video tools that can dramatically cut down your video creation and editing time. From transforming text into captivating visuals to adding that perfect touch of magic to your edits, these tools are set to become your new best friends in the digital playground!

## **1. CapCut**

At number 1, we have Capcut. 

&nbsp;

[CapCut](https://www.capcut.com/) surged in popularity due to its user-friendly interface and powerful editing features, making it a favorite among video creators. Its integration with TikTok and social media platforms is also reason for its widespread adoption.

&nbsp;

The best part about using CapCut is that most of these important features are available for free, which kicks out budget-constraints out of the window. CapCut uses AI to empower users to create stunning videos without needing any fancy editing experience. With mobile and desktop apps, as well as a web version, CapCut can easily be used on the go. 

&nbsp;

CapCut offers a range of AI-driven features that are very, very impressive. With 'Smart Cuts & Trims', CapCut's AI automatically detects shaky parts, long pauses, or unnecessary bits, suggesting cuts to tighten up your video and eliminate boring footage. 'Effortless Audio Magic' helps isolate voices and enhance audio quality, making your message crystal clear even if there’s noisy background interference. 'Instant Captions' auto-generate captions, making your videos more accessible and saving you time and effort. Moreover, if your filming didn't capture the perfect lighting, 'Light Like a Pro' allows CapCut’s AI to adjust lighting in your videos post-production, creating the ideal mood.

What I most love about CapCut AI video generator is its AI-driven background removal feature which eliminates the need for a green screen, allowing me to replace any background with an image or color of my choice.

&nbsp;

Something to note about Capcut is that the free version includes watermarks on exported videos. which can be removed only with a paid subscription. This may potentially pose a barrier for you if you’re a budget-conscious user.

## **2. Runway**

[Runway](https://runwayml.com/) can be called a virtual sorcerer's wand for video editing! It has the potential to be an entire AI video production studio, so it can basically create an entire movie without filming a single frame — if you have the time, that is. Runway can generate videos and animations from simple text prompts, create animations from still images, generate lip-syncing avatars based on a recording of your voice, and even allow you to train your own AI models. 

&nbsp;

Runway might be a little intimidating in the beginning. That’s because it has an array of features that take time to understand, execute and implement, and that could be slightly time-taking and if I can say, complex.

Another aspect to note is that while Runway offers a range of pricing plans, the advanced features that truly set it apart are locked behind higher-tier subscriptions, making it less accessible for small businesses or individual creators on a tight budget.

## **3. Invideo AI**

[Invideo AI](https://invideo.io/ai/) is an excellent tool for generating a starting point for your videos. It helps you to quickly create a basic structure for your videos, which can then be further customized and polished. The  features allow you to input simple prompts, from which the tool generates an initial draft that includes relevant footage, transitions, and effects. 

&nbsp;

A few standout features of Invideo AI are an extensive stock media library that offers a range of options to choose from, AI-powered script generation which creates a complete script for your videos and, voiceover addition. Their AI voiceover generator is particularly good because it has an option to add your own voice which it then uses to narrate scripts in. The narration is also decent, with multiple accents and focussing well on the feeling, intent, and emotion behind each word.

Overall, the tool is great for YouTubers and people who want to start off their video making journey with minimal effort. 

&nbsp;

One of the shortcomings of Invideo AI is its dependency on a stable internet connection, as the tool is cloud-based. This can be a significant limitation for users in areas with unreliable internet access. I also noticed that rendering videos on Invideo AI can sometimes be slow, especially for longer or high-resolution projects, so keep enough time in hand before you expect a final video. 

## **4. Wondershare Filmora**

[Wondershare Filmora](https://filmora.wondershare.net/de/ad/videobearbeitungsprogramm-brand.html) has been a staple in the video editing world for over 15 years. Despite its long history, it's continuously evolving, with a suite of AI features that enhance video quality without requiring professional training or extensive time investments. 

&nbsp;

Filmora is a popular choice among full-time influencers and video editors for advanced-level video editing. It is also a preference because the tool keeps evolving based on new technological developments, and therefore is easily able to meet the demanding standards of modern digital media.

&nbsp;

Some of Filmora’s features include 'AI smart cutout', which allows you to cut objects out of scenes simply by painting their area on the screen, and 'AI audio denoise', which helps control background noise. 'AI audio stretch' lets you extend music to match scene transitions, while AI motion tracking enables face blurring for privacy or fun effects. They also have AI remove background which allows you to create imaginative scenes without hefty budgets. 

Wondershare also provides detailed explanations of these features on their website, so if you ever face a problem using any of them, be sure to check their dedicated pages. 


&nbsp;

However, do note that Filmora's performance can be sluggish when handling large files or high-resolution videos, leading to slower rendering times and potential crashes at times.

## **5. Canva Magic Studio**

Easy to use. Neat interface. Quick results. 

&nbsp;

Those were the three things I liked best about [Canva Magic Studio](https://www.canva.com/magic/). 

&nbsp;

Canva Magic Studio, though currently in a very basic state, looks very promising as an AI video tool. Having been a fan of Canva for infographics, presentations and social posts since the past few years, I have full confidence that their AI video tool will evolve too. 

&nbsp;

As of now, Canva Magic Studio gives you amazing results when it comes to video quality and is also very quick to generate the final video. The good video quality comes from its integration with [Runway](https://zapier.com/blog/best-ai-video-generator/#runway) (our #2 in the list).

&nbsp;

To show you an example, I used the prompt: "A cute rabbit watching a video on a laptop”. And the [result](https://www.canva.com/design/DAGGCqwwf4A/GS7q42J5C9nx26gxJGR9lA/watch) was surprisingly very apt!

Canva Magic Studio is a great option if you want to bulk-create your social media posts – create a chunk of Instagram reels or Facebook ads in a go or create similar kinds of videos using a single template. The template options Canva offers have a wide range and are decent for beginners to start with.

## Sora – Open AI’s video generator

Open AI sparked a wave of excitement around AI with ChatGPT, demonstrating its incredible potential in natural language processing. Now, it's taking things a step further with [Sora, an innovative AI](https://openai.com/index/sora/) video creator.

Open AI recently shared a series of tweets showing the potential of Sora and I must say, I am amazed. Be it [several giant mammoths](https://x.com/OpenAI/status/1758192960116785459) rushing through a snowy meadow or a [fluffy monster](https://x.com/OpenAI/status/1758192964222988506) behind a melted candle, the kind of videos Sora will be able to create is astounding! It looks like it will be able to take detailed prompts into consideration, just like it did with ChatGPT, to create exactly what the user needs.

&nbsp;

As Sora continues to evolve and improve, it will undoubtedly shape the future of video production, showing way to new forms of storytelling and engagement in the years to come.

## AI is great – but it cannot replace you

While these AI video tools can ease the load for you and save some time, they cannot really take the place of your creativity, intuition, and emotional intelligence. In the end, it is your perspective, personal touch, and experiences that shape the videos you create. 

&nbsp;

_If you’re finding a place to store, host and deliver your [AI generated videos](https://api.video/use-cases/generative-ai/), api.video could be just what you’re looking for. With quick video upload options, flexible hosting capabilities, ability to add watermarks and keep videos private, you can choose api.video for all your video needs. If you would like to test out our features, [sign-up](https://dashboard.api.video/register) for a sandbox account now and you wound’t be disappointed._

small_7 AI Tools for Quick Video Creation.jpg

thumbnail_7 AI Tools for Quick Video Creation.jpg

7-AI-Tools-for-Quick-Video-Creation.jpg

5 ways the human eye impacts how we do video compression

7 eye-opening stats that prove the impact of product videos in e-commerce

5 ways to utilize product videos to boost your e-commerce sales

Yohann Martzolff

React

In today's tech-driven world, enhancing user interaction and creating immersive experiences is crucial. One exciting way to achieve this is by integrating AI facial expression detection into your video content.  In this blog, we'll guide you through the steps to <u>integrate AI facial expression detection to control your videos using React</u>. 

&nbsp;

Let's start.

&nbsp;

_*To learn how to do the integration with Javascript, head to this [blog](https://api.video/blog/tutorials/AI-facial-expression-detection-javascript/).*_

## 1. Create a minimal React app

As the first step, we need to create a new React app. Run the following command for the same. 

Now add the MediaPipe and the api.video React player packages

## 2. Add a video player

You’re going to use the official [api.video’s React player](https://github.com/apivideo/api.video-react-player) in this application, but feel free to use your favorite one and adapt the callbacks to it.

&nbsp;

Replace the code inside `App.tsx` with the following.
The `playerRef` allows us to control the [api.video player](https://api.video/product/player/).

Run your application with the npm run start command and go to `http://localhost:3000/`, you should see a video that you can play, pause or mute. 

## 3. Record your facial expressions

To control the player with your facial expressions, you need to stream your face through your webcam and pass this streamed data to the trained machine-learning model.

&nbsp;

Add a `<video />` HTML tag to `App.tsx`. 
Its `webcamRef` allows the video element to be targeted.
Apply a `display: "none"` inline style to hide the element, and an `autoPlay` attribute so the webcam stream time is updated. 

&nbsp;

**Note:** You can remove the `display: "none"` style and customize your CSS, it doesn’t matter.

## 4. Load the machine-learning model

Before we can use the machine-learning model, we must wait for it to finish loading. These models can be large and take a moment to get everything needed to run.

&nbsp;

Put this logic inside a `useEffect` and store the `faceLandmarker` class as a React stateful value.

## 5. Start detecting the webcam stream

Once the machine-learning model has been loaded and the `faceLandmarker` class has been instantiated, you can start detecting the stream.

&nbsp;

Create a new `useEffect`

Run your application with the `npm run start` command, go to `http://localhost:3000/`, and open your console. You should see a lot of console logs appearing each millisecond.

&nbsp;

Each one of them logs an array of objects of type:

## 6. Use the facial expression detection results

You now have access to a list of facial expression detection results. Use them to do whatever you want and in this case, to control your video player.

Let’s assume that you want to play the video when you smile, and pause it when you do a kiss.

Check the facial expression detection results on each frame and play or pause your video depending on the score of the wanted result by replacing your console.log with the following.

Once again, run your application with the `npm run start` command, go to `http://localhost:3000/`, and try to smile at your webcam. It should trigger the player’s `play` callback and play the video. Then, try to do a kiss ( 😗 ) in front of your webcam, and the video should pause.

## 7. Display the results

It can be helpful to have a visual representation of the detection results. Display these results as a list and highlight the detected facial expressions by adding a `<ul />` HTML tag to your JSX and creating a new ref to target it.

Add the code to update the HTML according to the results.

Now, you should see a list of results displayed in your application, with a green highlight on the ones with more than 60% confidence.

## 8. Code samples

Check the full `App.tsx` code below.

We hope this blog helped you understand the coding behind integrating AI facial expression detection to your videos.

 &nbsp;

_For any questions or doubts you may have, chat with us on our [website](https://api.video/contact/). If you would like to try out api.video for your videos, [sign-up](https://dashboard.api.video/register) for a free sandbox account here._

How to build AI facial expression detection into your videos with React

small_AI face recognition (React) - Version 2.png

thumbnail_AI face recognition (React) - Version 2.png

AI face recognition (React) - Version 2.png

Create a Next.js 13 & React 18 video app

How to create a Next.js 13 & React 18 video app

React Native Livestream Module: building the sample app

How to upload a video with React and NodeJS

Featured article

# api.video blog

api.video Blog

Follow the api.video Blog to explore tutorials, video use cases, and learn about on-demand and live-streaming video updates.

Video on demand

Live streaming

Analytics

Video infrastructure

Video player

AI features

See all features

Online learning & Corporate training

Social & Entertainment

Marketplace & E-commerce

Communication & UGC

Generative AI

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Customer success stories

Help center

Video Sessions (Analytics)