api.video lets you quickly and securely deliver on-demand and live-stream videos directly from your website, software, or app. 

api.video

Glossary-H-261.svg

## What is the H.261 video codec?
The H.261 codec is the first codec in the H.26x family. Today, it's still the most widely used international compression standard for video conferencing (though it isn't the main part of video conferencing anymore - it's used only for backwards compatibility these days). It was developed by the International Telecommunications Union and released in 1988. There were other codecs before this one, for example H.120, but this was the first one to really be usable. All the other H.26 series of codecs use the design principles that first appeared in this codec. 

## What's unique about the H.261 vieo codec - macroblocks
The H.261 codec is notable for being the first video codec to use macroblocks. Each macroblock in H.261 consisted of a 16x16 array of luma samples and two corresponding 8x8 arrays of chroma samples using 4:2:0 sampling and YCbCr color space.

This codec uses 8x8 discrete cosine transform (DCT) to reduce spatial redundancy. Something unique about this standard is it only specifies how to decode video. So all kinds of different motion estimation algorithms and other concepts were applied to compress and encode the video. Encoders can pre-process in any way they want and decoders can post-process any way they want so long as they decode the video according to the standard. A popular technique still employed today was deblocking filtering. Because this codec uses blocks of pixels to do compression and because it's the first of its kind, the blocks end up looking...well blocky. So to offset that, people use tricks like deblocking filtering or dithering to try to reduce the appearance of the blocks in the final decompressed video. 

H.261 as people know it today is very different from how it was back in 1988 -- it went through several revisions to make it a complete standard. One of the more unique features added to it was a backwards-compatible way to transmit still images. To achieve this, images were subsampled horizontally and vertically, dividing the picture into four sub pictures that were sent sequentially to form the complete picture. 

## H.261 codec today
The H.261 codec is becoming obsolete with newer members of this codec family. However, it's still one of the most popular international standards for video conferencing, so you're likely to see it in use a bit longer while everything converts to using newer codecs. And you'll likely see the basics of the design for H.261 in the rest of the family in the use of DCT and macroblocking. 



Glossary-H-263.svg

## What is the H.263 video codec?

The H.263 video codec was designed for teleconferencing video compression. It could be compressed and sent at a low bitrate. The video quality wasn’t great, but it was efficient for videoconferencing. Today, while this codec is obsolete, it’s still included in videoconferencing systems along with H.261 for occasions when it might be required. H.263 was revamped several times, so that in addition to videoconferencing, it ended up being used in the early days to display Flash video on the internet. 

## How is H.263 different from H.261 and H.262? 

The H.263 video codec is an evolutionary improvement on the algorithms and techniques that were started with H.261. A key difference is that it handles low bit-rates better than previous codecs in the H.26x family. It keeps a lot of the main design features like the discrete cosine transform (DCT), use of transform coding for intra-frames and predictive coding for inter-frames. It uses more advanced techniques to compress macroblocks. It also does a better job of smoothing out compressed blocks so they are smoother when decompressed for playback. Something unique is the introduction of the B frame. 

Previously, video frames were compressed using techniques that involve I and P frames. An I-frame is a type of compression where everything references only the frame you're looking at. It's a key frame that other frames might reference. A Prediction frame (P-frame) is a frame that predicts where objects and things may have moved to from the I-frame. Up until H.263, these are the types of frames used. H.263 adds the B or Bi-Directional frame. This type of frame is similar to a P-frame, but they can predict where an object will be by looking into the past frames or into the future frames. They're efficient because they have more options for how to compress data temporally and spatially but they're resource heavy because it takes more work to calculate them. If a frame needs details from a frame that comes before and after it in order to encode, you have to wait to get those frames before encoding. So it's a bit more complex since now the frames aren't processed in the order they appear in. 

## Conclusion
H.263 went through three major revisions and was widely used for video conferencing and internet videos until the creation of H.264. Now H.263 is included just in case it's still needed for old systems, but most of the time H.264 or higher is going to be used. 

Glossary-H-264.svg

## What is the H.264 codec?
H.264 is a video compression standard that's currently the most widely used codec on the planet. It uses advanced algorithms for estimating motion, inter-prediction, spatial intra-prediction and discrete cosine transfer (DCT) coding. It can be used for discs like Blu-ray (and others), broadcast and video streaming. This codec is also compatible with a lot of different container formats like MP4, MOV, 3GP, MPEG and F4V. Audio is compressed separately, usually using the [AAC (advanced audio coding)](https://api.video/what-is/aac-advanced-audio-coding/) standard. 

## How does the H.264 codec work? 
H.264 has an encoder and a decoder as part of the codec. The encoder handles prediction, transform and encoding tasks in order to efficiently compress a video file. 

Prediction is where the encoder takes a single frame of a video at a time and breaks it into blocks. Each block is called a macroblock, and these can vary in size by codec or method used to handle the blocks. Typically for H.264, the encoder breaks blocks into groups of between 16x16 and 4x4 pixels, but it could be different. The encoder then guesses what's going to be in the block, using these techniques: 
* **Intra prediction** - the encoder uses the way things are set up directly in the image to guess what other pixels in the image might be. For example if you're in a shadowy part of the image, the encoder might guess that there are more dark pixels in that area. Blocks for this type of prediction are either 4x4 or 16x6 pixels high and wide.
* **Inter prediction** - the encoder looks at a previous frame, then subtracts whatever is the same and keeps only the differences in the new frame for storage. This is called forming the residual. Blocks for this type of prediction can be anything from 4x4 to 16x16.

small_h-264.jpg

thumbnail_h-264.jpg

h-264.jpg

The transform task is where the encoder takes the information its saved and transforms it into a matrix of values where the positions in the matrix correspond to pixels at specific positions in the image. To do this, it uses a Fourier-transform. Since this article is about H.264, we won't get into too much detail. Suffice it to say the transform is going from an image to a matrix that represents the pixels to a wave. You're transforming the information into something that can be modeled using waves. This is easier to store and transfer, because once you can model your image using waves, you can come up with a way to represent the signal as the sum of combinations of cosine waves.  

## What is the difference between an MP4 and an H.264 file?
MP4 is a container format and stands for MPEG-4 Part 14. It's the most popular video format in the world because it allows you to combine all sorts of media - video, audio, subtitles and images. Most devices will play an MP4 and today many sites choose MP4 above other formats, making it everyone's go to choice. MP4 is different from H.264 because MP4 is a file format and H.264 is a codec. A file format describes what kinds of information can contain and how it will be organized. A codec describes how video or audio (or both) will be compressed and decompressed later for playback.

## Which is better: H.264 or H.265?
To determine whether [H.264 or H.265](https://api.video/blog/video-trends/H264-vs-H265/) is better, you must choose your criteria. For quality and speed, [H.265](https://api.video/what-is/h-265/) is better. For price, H.264 is free. There are more instances where you may use the H.264 codec free of charge for your video content. H.265 on the other hand, must always be paid for. Because of this, there is less support for H.265. If you want good quality and speed, and the ability to play back your video almost anywhere for free, choose H.264. Companies like Netflix and Hulu earn enough money that they can benefit from paying for H.265 to deliver a higher quality more efficient video experience. 

New codecs are under development to replace both of these, with the goal of the codecs being more efficient, providing higher quality, and being free. 

Glossary-H-265.svg

## What is the H.265 video codec?

H.265 is a video compression standard that offers 25-50% better data compression than H.264 while maintaining the same level of video quality. To achieve this incredible improvement, it extends on concepts found in the [H.264 codec](https://api.video/what-is/h-264/). A key difference from AVC (or H.264) is that H.265 uses more types of transforms and block sizes to achieve compression. H.265 is a fantastic choice if you want to stream and display high resolution video like 4K or higher. 

## How does the H.265 video codec work?

The H.265 video compression codec works by improving upon techniques that made H.264 so good. Recall that for the H.264 codec, video frames are broken into blocks. Each block is referred to as a macroblock, and these can vary in size. In the H.264 codec, blocks can range from 16x16 to 4x4 depending on the part of the compression process. The encoder makes a guess about what will happen in each block using intra and inter frame prediction Intra frame prediction happens within the same frame of video. The encoder tries to guess from surrounding pixels what will be present. Inter frame prediction looks at surrounding frames to determine what the difference is between the current frame and surrounding frames. 

Whether intra or inter prediction is being used at a given time, the encoder always breaks information into blocks. A big deal for the H.264 codec was the encoder being able to choose blocks of different sizes. For inter frame prediction, the encoder can choose blocks ranging from 16x16 to 4x4. For intra frame prediction, blocks are usually either 4x4 or 16x16 pixels wide. 

All of this changes with the introduction of the H.265 codec. In the H.265 codec, discrete cosine transform (DCT) is used for pixel blocks of 4x4 and 8x8. However, this codec ups the ante by ALSO using discrete SINE transforms (DST). It transforms block sizes from 4x4 to 32x32. This is for inter prediction. For INTRA prediction the H.265 codec can handle blocks of up to 64x64 pixels! These are more efficient ways of compressing the data, resulting in an up to 50% better data compression rate while still maintaining the same level of video quality or better. 

h265.jpeg

## What is the difference between H.265 and H.264?

There are a number of [differences between H.265 and H.264](https://api.video/blog/video-trends/H264-vs-H265/) codecs, here's a list: 
* H.264 is used almost everywhere in the world for video and can play on almost any browser. H.265 on the other hand only plays in about 18% of browsers. 
* For inter prediction, H.264 uses blocks ranging from 16x16 to 4x4. H.265 uses blocks that range from 32x32 to 4x4. 
* For intra prediction, H.264 uses blocks that are either 16x16 or 4x4 pixels wide. H.265 can use a complete range of blocks from 64x64 pixels all the way to 4x4. 
* H.265 can compress data up to 50% as much as H.264 without reducing quality.
* H.265 uses less bandwidth than H.264, allowing for higher quality video.


## Which is better: H.264 or H.265?

If you want to be sure your video plays when you send it out across the internet, you choose H.264. While it's not as efficient a codec, it's free and browsers nearly everywhere support the decoder for H.264. If you can afford to use H.265 (it's not free) and you know you're going to use a player that can handle H.265, then H.265 is the best choice. You will run into an issue with browsers still, because many browsers don't support H.265 due to the cost. While H.265 is great, it's also patent encumbered, driving up its cost of use.

Over time, a new codec will replace both these options. There are a few options in the works that will allow for a super efficient, high quality codec that's free and plays on every device. Some examples are H.266, VP9 and AV1, but one has not yet emerged as the leader. VP1 and AV1 are competitors with one another and H.266 is the next generation of H.265. All promise to be less patent encumbered and free. We'll just have to wait and see! 


## What does HDR stand for?

High dynamic range, normally abbreviated to “HDR”, is a term that originated in photography but quite quickly found its way into filmmaking. As an overview, “dynamic range” relates to the ratio between the smallest and largest values that an image or sound can be reliably transmitted. Within imagery (including video), it is essentially the extent to which its brightest and darkest regions can differ while still looking good.

HDR then extends that range to a much further extent. It is used to heighten contrasts between the darkest blacks and the brightest whites in order to bring an image’s color closer to its real-life details.

## What is HDR video?

In an attempt to provide a more photorealistic video-watching experience, HDR video technology has been developed to achieve that experience. HDR video is the next step in reproducing exactly what tends to be perceived by the naked eye into video form.

The process of creating photorealistic video begins, naturally, from the camera and will follow through to the post-production process of color correction before finally being shown on HDR-capable displays.


Glossary - Cloud video streaming (2).svg

## What is GPT (Generative Pre-trained Transformer) ?

GPT, which stands for Generative Pre-trained Transformer, is a type of [Large Language Model](https://api.video/what-is/large-language-model/) based on the transformer architecture. While primarily designed for natural language processing tasks, GPT models have significant implications for video technology, particularly in areas where text generation or understanding is crucial.

## How GPT Works
GPT models use a transformer architecture, which includes:
1. Self-attention mechanisms
2. Feed-forward neural networks
3. Positional encodings
4. Layer normalization

&nbsp;  

These components allow the model to process and generate text by understanding context and relationships between words.

## Applications in Video Technology

Although GPT is primarily a text model, it has several applications in video technology. In the pre-production phase, GPT can assist content creators by generating scripts or dialogue for videos. It can also enhance accessibility by creating detailed captions or subtitles through video captioning. For efficient content management, GPT can produce text summaries of video content, which are useful for cataloging or quick reference. The model's capabilities extend to metadata generation, where it can create descriptive tags and categories for video content based on textual descriptions or transcripts.

Furthermore, GPT can power natural language interfaces, enabling voice commands and intuitive interactions for video editing software. These diverse applications demonstrate how GPT, despite its text-based nature, can significantly contribute to various aspects of video production and management.

## Challenges and Considerations

Integrating GPT in video technology presents several challenges:
- Context Limitation: GPT may struggle with very long-form content or maintaining consistency over extended narratives.
- Visual Understanding: As a text-based model, GPT lacks direct visual comprehension, requiring integration with other AI systems for full video understanding.
- Computational Resources: Running and fine-tuning GPT models requires significant computational power.
- Bias and Accuracy: Like other AI models, GPT can perpetuate biases present in its training data.

## Future of GPT in Video Technology

As GPT and similar models evolve, we can anticipate exciting developments in their application to video technology. Future versions may achieve closer multimodal integration with visual processing, enabling a more holistic understanding of video content. This could lead to innovations such as real-time script adaptation during live productions based on immediate feedback.

As GPT technology continues to advance, its integration with video production and processing tools is likely to create new possibilities for content creation, analysis, and interaction, potentially transforming various aspects of the video industry.

## What is Generative AI?

Generative AI refers to artificial intelligence systems capable of creating new content, including images, text, and most relevantly for our context, video. In the realm of video production and processing, Generative AI algorithms can produce, manipulate, and enhance visual content in ways previously unimaginable. These systems learn patterns from existing data and use that knowledge to generate new, original content that didn't exist before.

## How Generative AI Works in Video
Generative AI in video applications typically relies on [deep learning](https://api.video/what-is/deep-learning/) techniques, particularly advanced neural networks. These networks are trained on vast datasets of video content, learning to understand and replicate patterns in motion, color, texture, and even narrative structure. The [algorithms](https://api.video/what-is/algorithm/) powering Generative AI in video often use architectures like Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs).

For instance, in video generation, the AI might be trained on thousands of hours of footage, learning to predict how scenes typically unfold. It can then use this knowledge to generate entirely new video sequences, complete with realistic motion and consistent visual elements.

## Applications in Video Production
Generative AI is revolutionizing various aspects of video production:
1. Video Creation: AI can generate entire video sequences from text descriptions or rough storyboards, saving time in pre-production and conceptualization.
2. Video Editing: Generative AI can automate tasks like color grading, scene transitions, and even suggest edits based on content and style preferences.
3. Visual Effects: AI can generate complex visual effects, create realistic 3D models, or even digitally age or de-age actors.
4. Video Upscaling: Generative techniques can enhance low-resolution footage, adding detail and improving quality.
5. Content Personalization: AI can generate personalized video content for individual viewers, adapting storylines or product placements in real-time.

## Challenges and Limitations
Despite its potential, Generative AI in video faces several challenges. The computational demands of generating high-quality video require significant processing power, which can be costly and time-consuming. While improving rapidly, AI-generated video can sometimes fall into the "uncanny valley," appearing almost but not quite realistic. The ability to generate realistic fake videos raises important ethical concerns about misinformation and consent.

Additionally, questions arise about who owns the rights to AI-generated content and how it relates to the training data, complicating issues of copyright and ownership. These challenges highlight the complex landscape in which Generative AI for video is developing, balancing technological advancements with practical, ethical, and legal considerations.

## Future of Generative AI in Video
The future of Generative AI in video is promising and potentially transformative:
1. Real-time Video Generation: As processing power increases, we may see AI generating high-quality video in real-time, enabling new forms of interactive entertainment.
2. Automated Video Production: Entire video productions could be generated from scripts, reducing the need for physical sets and potentially democratizing video creation.
3. Enhanced Personalization: Videos could adapt in real-time to viewer preferences, creating truly personalized viewing experiences.
4. Integration with [Large Language Models](https://api.video/what-is/large-language-model/): This could lead to more sophisticated narrative generation and even AI-driven directorial decisions.  

&nbsp;  

As [Generative AI](https://api.video/use-cases/generative-ai/) continues to evolve, it's poised to redefine what's possible in video production, opening new avenues for creativity and efficiency in the industry.

Glossary-Generative-AI.svg

Glossary-Gamma-correction.svg

## What is gamma correction? 

Definition: Gamma correction is a crucial nonlinear operation in digital imaging and video processing that adjusts the luminance (brightness) of images or video content. It consists of two main processes: gamma encoding and gamma decoding. This technique optimizes how brightness is represented in digital media and displayed on screens, accounting for differences between how cameras capture light and how human eyes perceive it.

Graphic depicting perceived linear brightness vs. physical linear brightness

gamma-correction-brightness.png

## Key Aspects:

### Purpose:
- Ensures accurate brightness representation on digital displays
- Optimizes data storage and transmission by focusing on perceptually important information

### Human vs. Camera Vision:
- Cameras have a linear response to light intensity
- Human vision has a nonlinear response, more sensitive to changes in darker areas
- Gamma correction bridges this gap, making digital images appear more natural to human eyes

### Encoding Process:
- Redistributes tonal information to match human perception
- Allocates more data to shadow details and less to highlights
- Compensates for the overemphasis on brightness in camera sensors

### Decoding Process:
- Adjusts stored digital information for proper display on various screens
- Accounts for different display technologies and their characteristics

### Practical Applications:
- Improves overall image quality and detail visibility
- Ensures consistent appearance across different devices and displays
- Essential for accurate color reproduction in digital media

### Considerations:
- Different displays may interpret gamma values differently
- Requires careful balancing to preserve important visual information while optimizing data usage

Gamma correction is fundamental in digital imaging, ensuring that what we see on our screens closely matches our natural visual perception, despite the limitations of both capture and display technologies.

## What is Video Framerate?

**Framerate**: how frequently the images in a video update. Each image in a video is a frame,and the images update many times per second. The framerate is measured in **frames per second (fps)** and the FPS tells you how many images appear in that second.

In the 'flip book" image above, each page is a frame, and the 'video' updates as fast as the user is able to flip through the pages.  The higher the framerate, the more 'fluid' the motion will appear.

## What are common framerates used today?

* Film is typically recorded at 24 fps. 
* TV is typically 29.97 FPS or 59.94 FPS.  The values were initially 30/60 - based on the common electrical frequency of 60Hz, but an artifact in black and white televisions was removed by lowering the frame rate by 0.1%, and as a result, these framerates are still common today.
* Video games are typically played at 60 fps.  With fast motion, it is important to not have any jank or lag in the visuals, and it was found that 60 FPS was best for gaming.
* Cartoons: Cartoons are often 'inked' at 8 ("on threes") or 12 ("on twos") frames per second, and then repeated on screen multiple times to reach the 24 fps expected by viewers. The term "on threes" comes from how often each frame is repeated to get to the 24 fps that is ultimately broadcast.

## Human perception

Until recently, it was assumed that humans cannot see faster than 60 fps.

Most TVs and monitors refresh at 60 Hz (and since Hz has the unit 'per second', you can think of that as 60 refreshes per second).  Some gaming monitors refresh at 90 or even 120 Hz.

A [recent study at MIT](https://mollylab-1.mit.edu/sites/default/files/documents/FastDetect2014withFigures.pdf) showed the brain can process an image in as low as 13ms which comes to ~ 75 fps, indicating that our eye and brain can process motion at a faster rate, if only subconsciously.

## FPS at api.video

Each video uploaded to api.video is encoded into multiple sizes for streaming, and we have optimized the framerate for each stream. The frames per second varies by the size of the video, and as the video dimensions get larger, so does the fps of the video:

* 240p: 18 fps
* 360p: 24 fps
* 480p: 24 fps
* 720p: 24 fps
* 1080p: 30 fps
* 4k:    60 fps





Video on demand

Live streaming

Analytics

Video infrastructure

Video player

AI features

See all features

Online learning & Corporate training

Social & Entertainment

Marketplace & E-commerce

Communication & UGC

Generative AI

Quickstart guides

Documentation

Ecosystem catalog

Clients & SDKs

Community

Blog

Demos

Customer success stories

Help center

H.261

What is the H.261 video codec?

What's unique about the H.261 vieo codec - macroblocks

H.261 codec today