The Web APIs of record.a.video

We’ve just released record.a.video, a web application that lets you record and share videos. If that were not enough, you can also livestream. The page works in Chrome, Edge, Firefox Safari (14 and up), and on Android devices. This means that the application will work for about 75% of people using the web today. That’s not great, but since there are several new(ish) APIs in the application, it also isn’t that bad!

Here are a list of the major APIs used in record.a.video (don't forget to try it out!).

Video API

The most important API in record.a.video is api.video. api.video handles all of the video encoding, hosting and delivery. The app simply uploads the recorded video (more on this later), and api.video takes care of the rest!

For live streaming - again api.video accepts the live stream, and converts it into a live HLS stream for immediate live viewing.

Without this backbone to handle all of the videos, record.a.video could not exist. But, before the video is uploaded and processed, we are doing a lot of cool stuff in the browser that I find really fascinating. I thought that identifying the APIs and how they work would be of interest to many web developers.

Browser APIs

Here are the media APIs I am using to capture video:

MediaDevices.getUserMedia API can connect to the device camera & microphone to record video. I’ve already built a live stream video (read all about it in the api.video blog.

ScreenCapture API has excellent desktop support, but no support on mobile devices.

With these two APIs I can grab the screen and the camera. For my video recording, that’s nearly half the battle. In livestream.a.video app, we simply took the camera video and streamed it. But now, I want to combine these two feeds and record it for upload. In order to do that, I draw the 2 video feeds to a canvas. Then we record from the canvas using:

Media Recorder API reads from the canvas, and creates a new video instance that we can save, or stream to the live streaming server.

WebSpeech API does instantaneous speech recognition - enabling the possibility to add captions to your recording. It is experimental, and in Chrome only. But what I played with it, it was really cool, so I added it anyway. Since it is only Chrome, I built the app to gracefully turns off captioning on all the other browsers.

Web API Browser support:

With web APIs, it is crucial to understand the landscape of browser support. How well will my app work across the browser landscape? If a critical API is not supported on a brwoser - that means my application will not work (I'm looking at you MediaRecorder...)

getUserMedia: getUserDevices works in all devices, so cameras and microphones are fair play for recording!

ScreenCapture API: Excellent desktop browser support, but zero mobile support. This kind of makes sense - sharing a mobile screen would be weird. So on mobile, the app will only share the camera.

MediaRecorder/MediaSource: This is where Safari has issues: Desktop: only supported in the newest Safari 14. iOS: no support. Without the ability to record from the canvas - the application just cannot work. I spent a lot of time debugging this on desktop Safari. :(

WebSpeech recognition: As previously stated: Chrome only.

So, The full sharing screen and camera will work on desktops (and Chrome & Edge can add captions).

On mobile - we can only share the camera. I thought it would be cool to share both the front & rear camera at the same time on mobile...but that does not work. (You can try it yourself at record.a.video/index1.html.) If that ever becomes available, it will become another really cool application for live video.

The future

If Media Recorder support appears in iOS Safari, we can perhaps support iOS devices. But, until that time, the application will not work.

Conclusion

The webAPIs in record.a.video are super powerful, but some of them required a slightly different browser implementation. In the coming week, I will be writing up sample implementations, and the workarounds I had to utilize to make these APIs work across the supported browsers.