You don't need to know much about the human eye to use video compression standards. However, if you want to understand why they are designed the way they are, or tune your compression, knowing a little bit about the human eye goes a long way. Here are 5 facts about the human eye that directly influence our compression standards.
1. When it comes to colors, the human eye sees green best, red second best, and a little bit of blue.
Impact: Algorithms and video gear that focus on capturing green more than other colors. Humans process brightness and color differently. Because we are most sensitive to green, the way we see brightness is determined by how much green we see in something. As a result, cameras capture more green than other colors. Part of why we use green screens is because they appear so bright for people, they don't require as much lighting compared to other choices. We also use green screens because the green channel has more information, allowing for a cleaner key around a subject. The way we process color not only impacts what the camera captures, but how we set up color encoding. We tend to create algorithms that preserve green more than the other colors, because that's what we're most sensitive to!
2. The highest resolution the human eye is capable of seeing is 576 megapixels.
Impact: Researchers and developers are working to create 8K TV and a codec to compress it. While your first thought might be that the 576 megapixels is a lot, it's not the resolution we see with most of the time. That's what you see with a very small area in the center of your eye called the fovea. The fovea only covers about 1% of the retina and you can only see about 1/60th of a degree with it. In the real world, this would be like looking at a dot about .07 inches wide from a distance of 20 feet. Most of the time the resolution you see at is between 5 and 15 megapixels. Because of this, it's believed by many that 8K TV will be the end of the line in terms of quality. If you can't see past this level why create it when it's harder to capture, compress and deliver?
3. Your brain fills in missing information for the blind spot caused by your optic nerve.
Impact: Video compression algorithms use their own version of 'filling in' with predictive coding. Many video compression algorithms use keyframes. A keyframe contains complete information for that frame. Frames that come after (and sometimes before) are stored based on the difference between them and the key frame. When motion occurs, many compression algorithms work to predict what that motion will be and store information based on predictions. This is similar to how your brain fills in and predicts data.
4. The human eye is more sensitive to luminance (brightness) than chrominance (color).
Impact: The YUV color encoding system is built based on how human sight works. The system takes into account that humans are less sensitive to color, allowing reduced bandwidth for color components. If there are transmission errors or compression artifacts they are less obvious than they would be with the RGB color system. What to keep and what to throw away in video compression that uses YUV is based on what the human eye is most and least sensitive to.
5. The human eye can see about 30-60 frames per second (FPS).
Impact: Most content is streamed at 24, 30 or 60 fps.
There is an ongoing debate about how many frames per second the human eye can see. Gamers in particular will claim people are able to see more. Many people confuse fps with Hz. Hz is short for Hertz and is a unit of frequency. When applied to a monitor or projector, it represents the refresh rate for your screen or display, not the number of frames displayed per second. In any case, while there may be a debate regarding fps, the limitation of the human eye, or beliefs regarding its limitation result in most content streaming at 60 fps or under. Video games may stream faster, but it's still debatable as to whether people can see past 60 fps or beyond 90 Hz.
Resources
- Blue Screen Vs. Green Screen: Which One Do You Need? - https://www.premiumbeat.com/blog/blue-screen-vs-green-screen/
- Can the Human Eye See in 8K? - https://www.lasikmd.com/blog/can-the-human-eye-see-in-8k
- The Brain Adapts in a Blink to Compensate for Missing Information - https://www.scientificamerican.com/article/brain-adapts-in-a-blink/
- YUV - https://en.wikipedia.org/wiki/YUV
- How Many Frames Per Second Can the Human Eye See? - https://www.healthline.com/health/human-eye-fps#:~:text=In%20the%20past%2C%20experts%20maintained,see%20topped%20out%20around%2060.
