api.video lets you quickly and securely deliver on-demand and live-stream videos directly from your website, software, or app. 

api.video

Glossary-ingest.svg

Definition of ingestion
- Food: When you ingest food, you are taking food into your body by swallowing it. Your body then processes the food into energy.
- Videos: Ingestion of a video is the process of uploading the file, and processing the video for playback.

## What is video ingestion?

After the video is uploaded, a video encoder will convert the uploaded file into versions that are available for playback.  This step is important as not all video formats can be played on every type of device. For example, iPhone videos are often recorded in h.265 HVEC - which is not widely supported outside of Safari and Apple products. In the conversion step, the video is created in enough formats to support universal playback (oftentimes this is h.264 - not the best compression, but the widest playback support).
 
 ## Video ingest at api.video
 
At api.video - video ingestion occurs when a video is created, and then the video file is uploaded into the video.  Once the file has been uploaded, api.video converts the video into 6 HLS video streams of differing sizes and bitrates, and a h.264 [mp4](https://api.video/what-is/mp4/) video file.

When the video is being ingested, you can query the ingestion process using the [video status](https://docs.api.video/vod/show-video-upload-status) endpoint:

```
"encoding":{
"playable":true
"qualities":[
{
"quality":"360p"
"status":"encoded"
}
{
"quality":"480p"
"status":"encoded"
}
{
"quality":"720p"
"status":"encoded"
}
{
"quality":"1080p"
"status":"encoding"
}
{
"quality":"2160p"
"status":"waiting"
}
```


You can also use the ```video.encoding.quality.completed``` [webhook](https://docs.api.video/reference/api/Webhooks) to understand when the video ingestion has completed for a given video at every quality it is encoded into.





Glossary-Instant-replay.svg

## What is an Instant Replay?

Instant replay is the ability to retransmit a segment of video that was initially transmitted live.

The most common uses for instant replay is in sporting matches where a spectacular play or score is retransmitted after the fact.  Often different angles and speeds are used to watch the replay of the action - providing more information and insight into how the action unfolded.

In many sporting events, if the call by the on-field referee or judge is contested, instant replay can be utilized.  A judge with access to the myriad views and camera angles can replay the action in question to see if the call was warranted or if the video evidence shows that the call should be overturned.

medium_tomwashington-montymccutchen-brentbarnakyonmarch30-2011.jpeg

small_tomwashington-montymccutchen-brentbarnakyonmarch30-2011.jpeg

thumbnail_tomwashington-montymccutchen-brentbarnakyonmarch30-2011.jpeg

tomwashington-montymccutchen-brentbarnakyonmarch30-2011.jpeg

## DVR/Instant replay at api.video

All of our live streams have 5 minutes of video stored on the server.  The api.video player has a rewind button that allows viewers to 'rewind' the live stream to rewatch any content that they would like to see again - essentially allowing for 'instant replay' opf anything being recorded on out platform.
- Read more about [instant replay/DVR](https://api.video/blog/product-updates/dvr-features-in-live-streaming/)
- See how we watched the [DVR replay of our snowman falling over](https://api.video/blog/video-trends/do-you-want-to-stream-a-snowman/)

Glossary-Inter-frame-compression.svg

## What is inter-frame compression? 
Inter-frame compression is a type of compression that is based on what happens across frames. By taking advantage of the fact that sometimes very little changes from frame-to-frame, you can store less information and reduce the overall size of a video. A simple example of interframe compression is taking one frame and saving the entire frame. This frame becomes a keyframe that is used as a reference.

Moving forward in a video, an interframe compression algorithm would only make note of changes that differ from the keyframe. This is all that's stored. Later, when the video is decompressed, all that has to be added is the changes that occur in the video from the keyframe you are comparing against. If there is a dramatic change in the video, for example a new scene begins, then a new keyframe would be selected and used for the next block of frames as the reference point. More keyframes means smoother playback, but a bigger video file. 

## I, P and B Frames 
In the simple example I went over in the earlier section I talked about keyframes. There are several types of frames that are common in video compression algorithms that use inter-frame compression. These are called I, P and B frames. 

### What are I-Frames?
I-frame is short for intra-coded frame. For this type of frame, compression techniques are limited to those which act only upon that frame. These frames are also referred to as keyframes, because these are the frames that are completely stored as-is and used as reference points for the other frames in the video. 

### What are P-Frames? 
P-frame is short for prediction frame. With inter-frame prediction, the frame is divided into blocks. The encoder then tries to find blocks that match or are similar to those of a previously encoded frame (which would be an I-frame or keyframe).

For blocks that match, then the block is encoded by pointing to the reference block it matches. This pointer can later be used to restore that matching block to the frame. In cases where the block doesn't match exactly the encoder computes the difference and this information is saved for use when decoding later. The P-frame must have an I-frame to reference in order to perform the calculations and block matching.

### What are B-Frames?
A B-frame does something similar to what P-frames are doing, however these are predicted frames that can be created based on information in either direction. B-frames are predicted from information contained in P and I-frames. 

## S-Frames
S-frame is short for switching frame. These are relatively new, and are part of the AV1 codec. It's a P-frame that only uses an I-frame, all other information must be provided by the video stream. This type of frame is used to allow Adaptive Bit Rate (ABR) to stream smaller segments of frames. A segment can start with an S-frame copied from a previous I-frame and then the decoder knows it can decode from this frame without worrying about whether it references data that comes before this frame.

Something unique about an S-frame is it can be a different resolution from the original I-frame based on what's available for streaming. If you're interested in learning more about S-frames and how they're used in AV1 to improve video compression, check out this talk: [Tarek Amara - S-Frame in AV1: Enabling better compression for low latency live streaming](https://www.youtube.com/watch?v=o5sJX6VA34o). If you're interested in learning more about the history of video compression, check out [The History of Video Compression Standards, From 1929 Until Now](https://api.video/blog/video-trends/the-history-of-video-compression-starts-in-1929/).

Glossary-Large-Language-Model.svg

## What is Large Language Model (LLM)?

A Large Language Model (LLM) is a type of artificial intelligence system designed to understand, process, and generate human-like text based on vast amounts of training data. In the context of video and AI, LLMs play a crucial role in understanding and generating text-based content related to video, such as scripts, captions, and metadata. These models use [deep learning](https://api.video/what-is/deep-learning/) techniques to process and produce language at a scale and complexity that mimics human-like understanding.

## How Large Language Models Work
LLMs are based on complex neural network architectures, typically using transformer models. They are trained on enormous datasets, often containing billions of words from diverse sources. Key aspects of their functionality include:
1. Tokenization: Breaking down input text into smaller units (tokens) for processing.
2. Attention Mechanisms: Allowing the model to focus on relevant parts of the input when generating output.
3. Contextual Understanding: Grasping the nuances and context of language beyond simple word-for-word translation.  

&nbsp;  

In video applications, LLMs can process textual information associated with video content, enabling sophisticated analysis and generation tasks.

## Applications in Video Production and AI
Large Language Models (LLMs) have several important applications in the video and AI industry, spanning various aspects of content creation and management. In the realm of creative development, LLMs can assist in script generation, producing dialogue or entire storylines based on given prompts or themes. They also enhance accessibility by processing audio transcripts to generate accurate and context-aware captions for videos. For content management, LLMs excel at video summarization, creating concise text summaries useful for cataloging or quick reference.

These models contribute to improved searchability and organization through content tagging and metadata generation. In terms of user interaction, LLMs enable sophisticated natural language interfaces, allowing for intuitive voice commands and interactions with video editing software and AI-powered production tools. Furthermore, when integrated with video analysis AI, LLMs facilitate cross-modal understanding by providing text descriptions of visual content, effectively bridging the gap between visual and textual information. This diverse range of applications demonstrates the versatility and potential of LLMs in revolutionizing various aspects of video production and AI-driven content creation.

## Challenges and Limitations
While powerful, LLMs face several challenges in the video and AI domain:
1. Context Limitation: LLMs may struggle with very long-form content or maintaining consistency over extended narratives.
2. Bias and Accuracy: Models can perpetuate biases present in their training data, potentially leading to unfair or inaccurate representations in video content.
3. Computational Resources: Running and fine-tuning LLMs requires significant computational power, which can be costly.
4. Integration with Visual AI: Seamlessly combining language understanding with visual processing remains a complex challenge.

## Future of Large Language Models in Video and AI

The future of LLMs in video and AI looks promising:
1. Multimodal Models: Future LLMs may integrate more closely with visual and audio processing, enabling more holistic understanding of video content.
2. Real-time Script Adaptation: LLMs could enable dynamic script adjustments during live productions based on real-time feedback or changing conditions.
3. Enhanced Creative Assistance: More sophisticated models could serve as creative partners in the video production process, offering ideas and solutions to complex narrative challenges.
4. Improved Localization: Advanced LLMs could provide more nuanced and culturally appropriate translations and adaptations of video content for global audiences.
5. AI Directors: Combined with other AI technologies, LLMs could potentially guide entire video productions, making high-level creative decisions based on narrative understanding.  

&nbsp;  

As Large Language Models continue to evolve, their integration with video technology is likely to create new possibilities for content creation, analysis, and interaction, potentially transforming the landscape of video production and consumption.

## What is video latency? 
Video latency is a measure of how long it takes between when a video stream is requested and when transfer of the video stream starts. Latency is used when you are working with a live stream. In the smallest unit of measurement, its how long it takes one frame of video to move from where it's being captured by a camera to the player you want it to appear on. Sometimes this is referred to as glass-to-glass.


## When is low video latency most important?
There are some situations where latency might not matter as much. For example, if you're watching a playback of an event, latency might not be a big deal, you can wait a few moments to get started watching. But if you're having a conversation, it might become difficult to have your conversation if there's drag between video of you and the other participants reaching one another. The most common use cases where you'd want low latency are:

* **Two-way communication** - meetings and broadcasts where you want to speak with your audience real time.
* **Betting and auctions** - anything where people need to place a bid or make a bet, you're going to need everything to be as up-to-date and realtime as possible.
* **Video games** - you need to be able to respond to what's happening in your game real-time.

## What causes latency?
When live streaming, your latency is affected by many things: 

* **Device** - you are limited by what your device is capable of.
* **Distance to travel** - the further your video is from its destination, the greater the chance of latency.
* **Video codec and vide size** - bigger files take longer to transmit, and different codecs will handle the files at different speeds, resulting in higher or lower latency.
* **Bandwidth** - the higher the bandwidth the faster you can stream video, resulting in lower latency.

## How is latency categorized?
You may have heard about different types of streaming described with different latency terms like standard, low and others. But how much latency occurs for each type? Here's a quick list: 

* **Standard broadcast latency** - 5-18 seconds, TV and some VOD online have this much latency. 
* **Low latency** - 1-5 seconds, cable TV and some VOD have this much latency.
* **Ultra-low latency** - Less than 1 second, when you have this much latency, you're at the start of what can be used for real-time streaming. It's suitable for two-way communication, though you will sometimes notice a pause if you're trying to do something complex, like play music together across a two-way connection. 
* **Real-time latency** - Milliseconds, you can't tell that there's a delay in most cases. This can still be noticeable for things that require precision, again the best example would be if you're trying to do something like play music, where the timing needs to be close to perfect for everyone to be together. 

## Video latency and api.video
At api.video, enjoy [low latency live streaming](https://api.video/product/live-stream/), stream your videos with a latency of 3 seconds in ideal conditions.




Glossary-image-to-video.svg

## What is Image to Video?

Image to Video is an AI-powered technology that transforms static images into dynamic video content. This innovative application combines [Computer Vision](https://api.video/what-is/computer-vision/) techniques with video generation algorithms to create motion and temporal progression from still visual inputs. It's a subset of [Generative AI](https://api.video/what-is/generative-ai/) that bridges the gap between static and moving imagery, enabling the creation of video content from single or multiple image sources.

## Key Components of Image to Video Systems
Image to Video systems typically involve several key components:
1. Image Analysis: Understanding the content and structure of the input image(s)
2. Motion Estimation: Predicting plausible movements for elements in the image
3. Frame Generation: Creating intermediate frames to produce smooth motion
4. Background Reconstruction: Extending or recreating background elements as needed
5. Temporal Consistency: Ensuring coherent movement and changes across frames

## Techniques Used in Image to Video
Several AI techniques are crucial for Image to Video technology. Convolutional Neural Networks (CNNs) play a key role in analyzing and understanding the content of input images. Generative Adversarial Networks (GANs) are employed to create new video frames based on the input image.

Optical Flow Estimation is used to predict the motion of objects between frames, while Inpainting techniques fill in missing parts of the background revealed by object movement. Video Interpolation is also utilized to generate intermediate frames for smoother motion. These techniques work in concert to transform static images into dynamic video sequences, each contributing to different aspects of the conversion process.

## Applications of Image to Video
Image to Video technology has numerous potential applications across various fields. In social media, it can be used for content enhancement, bringing still photos to life for more engaging posts. Historical visualization benefits from this technology by animating old photographs to create more immersive historical experiences. In education, it serves as a powerful tool for creating dynamic visualizations from static diagrams or illustrations.

The advertising industry can leverage this technology to transform product images into eye-catching video advertisements. For tourism, it enables the generation of flythrough videos from landscape or architectural photos, offering virtual tours. In the film and animation industry, Image to Video technology assists in pre-visualization or creates dynamic backgrounds from still images. These diverse applications demonstrate the versatility and potential impact of Image to Video technology across multiple sectors.

## Challenges and Considerations
Image to Video technology faces several challenges:
- Realistic Motion Generation: Creating natural and plausible movements for objects in the image.
- Handling Complex Scenes: Accurately animating images with multiple objects or intricate details.
- Temporal Consistency: Maintaining coherent object appearance and movement across generated frames.
- Background Reconstruction: Convincingly filling in areas of the background revealed by object movement.
- Ethical Considerations: Addressing concerns about the authenticity of generated video content.

## The Future of Image to Video
As Image to Video technology continues to advance, we can expect several exciting developments:
- Improved Realism: Generation of increasingly natural and complex motions from still images.
- Enhanced User Control: More precise control over the type and extent of motion in the generated videos.
- Multi-Image Input: Creating more complex videos by intelligently combining multiple input images.
- Real-time Processing: Faster algorithms enabling on-the-fly video generation from images.

&nbsp;  

Integration with Other AI Technologies: Combining with [text to image](https://api.video/what-is/text-to-image/) or audio generation for more comprehensive content creation.

As Image to Video technology evolves, it has the potential to revolutionize how we interact with and create visual content. From bringing personal photos to life to creating dynamic visual effects for film and television, this technology opens up new possibilities for creative expression and storytelling. However, it also raises important questions about the nature of visual media and the need for transparency in content creation.

## What is HTTP live streaming (HLS)? 

HTTP Live Streaming (HLS) is a type of video streaming created by Apple. Despite the name, HLS streams can be recorded video ([video on demand](https://api.video/what-is/vod-video-on-demand/)) or live streamed.  HLS is the primary form of video streaming used on the web today, followed by MPEG-DASH.
 
Video streaming is different from a regular video in that the video is broken into smaller segments - typically 2-10s each, and are downloaded to a video player for playback.  
 
The segments are defined by a manifest file - which tells the player which files to request and how long each segment is - so the player can queue up and make the requests in a way that ensures that the video will always have enough video downloaded  - preventing the video from stalling.
 
HLS streaming can also support adaptive bitrate streaming. In this case, the same video is encoded at many different sizes and bitrates (smaller sizes have lower bitrates), and the player can adapt the video playback for the screen and for the available network speed.
 
All videos uploaded to api.video are encoded into a HLS video stream for playback. For more detail into HLS, and how each manifest file works, please read [What is HLS Video Streaming and How Does it Work](https://api.video/blog/video-trends/what-is-hls-video-streaming-and-how-does-it-work/).

Glossary-HTTP-Live-Streaming-HLS.svg

Glossary-Hosting.svg

## What is Video Hosting?

Hosting definition: In the summer of 2021, Tokyo and Japan *hosted* the 2020 Summer Olympics. The country created facilities for thousands of athletes to come visit and compete in their respective events. The hosts of the Olympics made sure that everything was working, operational and ran smoothly. 

## Video Hosting

The concept of video hosting is similar. A video host [ingests](https://api.video/what-is/ingest/) videos, and hosts the files on their servers, ensuring that the videos are ready for playback. 

There are a number of options for video hosting - using a service, or self-hosting your videos.

### Self hosting video

Self hosting is the process of placing the videos on your own server for playback.  The advantage is that you control all of the aspects of your server.  The disadvantage is that videos are large - taking up a lot of server storage, and the transmission can also become quite expensive. You also cannot leverage the experience of a team of video delivery experts (who all generally work at video hosting services.)

### Video hosting services

Video hosting services include YouTube, Vimeo and (of course) [api.video](https://api.video). When working with a video hosting service, you can leverage the experience of video delivery experts to deliver your video - not only saving your bandwidth and storage, but also engineering and development time - since you do not have to create a pipeline for video upload, storage and delivery.

Advantages of video hosting services - Your video is typically converted into the best formats for a large number of viewers. Additionally, the video host will have optimized their infrastructure for the delivery of video. This typically includes low latency drives and memory to quickly deliver the data to the network. The videos are also cached around the world with e Content Distribution Network (CDN) to further lower the video delivery time. 

Some free services will watermark your video, insert ads or place other limitations on the video playback. A paid service like api.video will host your video in the optimal streaming formats, and deliver it for fast, low latency, no buffering playback to all of your users around the world.

## Hosting Summary

A video host is a server on the internet that is used to deliver video. There are specialized video delivery teams that have worked to optimize all aspects of video delivery - making your content shine, while simplifying your workflow.




## What does HDR stand for?

High dynamic range, normally abbreviated to “HDR”, is a term that originated in photography but quite quickly found its way into filmmaking. As an overview, “dynamic range” relates to the ratio between the smallest and largest values that an image or sound can be reliably transmitted. Within imagery (including video), it is essentially the extent to which its brightest and darkest regions can differ while still looking good.

HDR then extends that range to a much further extent. It is used to heighten contrasts between the darkest blacks and the brightest whites in order to bring an image’s color closer to its real-life details.

## What is HDR video?

In an attempt to provide a more photorealistic video-watching experience, HDR video technology has been developed to achieve that experience. HDR video is the next step in reproducing exactly what tends to be perceived by the naked eye into video form.

The process of creating photorealistic video begins, naturally, from the camera and will follow through to the post-production process of color correction before finally being shown on HDR-capable displays.


Video on demand

Live streaming

Analytics

Video infrastructure

Video player

AI features

See all features

Online learning & Corporate training

Social & Entertainment

Marketplace & E-commerce

Communication & UGC

Generative AI

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Customer success stories

Help center

Ingest

What is video ingestion?

Video ingest at api.video