The comeback of specialized hardware

The comeback of specialized hardware and its impact on the video industry

Over the past few decades, the trend in computing has been toward general-purpose computing systems that can run a wide variety of applications. This has led to the widespread use of CPUs and GPUs. However, there has been a recent resurgence of interest in specialized hardware, which is designed specifically to perform one or a few tasks extremely efficiently. In this blog post, we will explore the reasons behind this trend and some of the critical examples of specialized hardware being used in the video industry today.

Why is specialized hardware making a comeback in computing?

Specialized hardware in the computing industry refers to computer components or devices designed for a specific purpose, such as application-specific integrated circuits (ASICs) or systems-on-a-chip (SoCs) . These hardware components are optimized for their specific task, resulting in faster and more efficient performance than general-purpose hardware.

Several factors have contributed to the recent resurgence of specialized hardware in computer science:

• High efficiency : Why we want it quick and ressource-efficient

One key driver is the increasing demand for high-performance solutions. Artificial intelligence, machine learning and video processing are domains where CPUs are far from being the right solution. Regarding AI workloads, training, and running models are more efficient on GPU and even better on specialized hardware like Tensor processing units (TPUs). Google has been the first to design TPUs. These ASICs (Application Specific Integrated Circuits) outperform the CPUs.

Why are CPUs outperformed ? In early 2023 commodity servers come with 1 or 2 sockets and maximum 96 cores per socket for the x86 microarchitecture (128 for ARM). On the other hand, a server with a mid range CPUs can receive several ASICs/TPUs on the PCIe ports. Those boosted systems outperforms by a lot the CPUs based system. The TPUv4, in terms of Tflops is 2000x faster than a laptop according to Google.

The benefits of a specialized hardware is that it provides better performance, higher density and a lower energy consumption than the classic CPUs.

In most cases, TPU/ASIC > GPU > CPU on the 3 criteria : performance, density, and energy savings.

• Total Cost of Ownership (TCO): Why It's Worth the cost

Specialized hardware exhibits smaller size, higer speed, and greater electrical efficiency. As a result the density increases and the number of computing units decreases. At the end, the TCO is improved. After the switch to ASICs based VCUs (Video coding unit), Youtube is reported to have saved millions of CPUs ! At this scale you also save data centers 💰

How specialized hardware is changing the game in the video industry

In the video industry, one of the most computationally intensive processes is video transcoding, especially for high-quality videos. Transcoding decodes a video from one codec and encodes it to another, such as from H.264 to H.265. This process can also involve changing other video parameters like resolution, bit rate, frame rate, etc. This has led to a need for fast and efficient video transcoding solutions without sacrificing the video quality output.

Transcoding is a service api.video provides in addition to hosting and delivery. Via an API call, a video is uploaded, and then the URL of the video is returned immediately. Note that high resolution and new codecs increase the computational cost.

Similarly to Artificial Intelligence (AI), several types of hardware are commonly used in video transcoding applications:

1. ASICs (Application Specific Integrated Circuits): Video ASICs are specialized hardware accelerators designed specifically for video processing tasks, such as video encoding/ decoding / transcoding. They are sometimes called VPU (Video Processing Unit) or VCU (Video coding Unit). They significantly improve performance over traditional CPUs, particularly for high-resolution or high-frame-rate video. Alphabet, the mother company of Google and YouTube has designed its own ASICs : codename Argos. For a system with 20 VCUs, the TCO has decreased by a 33x factor for VP9 encoding, and throughput increased by a 100x factor!

Facebook/ Meta made the same move to specific hardware. They partnered with silicon vendors to design custom ASICs.

The electric consumption of an ASIC is around a few watts.

2. FPGAs (Field-programmable gate arrays): FPGAs are programmable hardware devices that can be customized for specific applications, including video transcoding. They can provide significant performance improvements over traditional CPUs for certain types of video transcoding tasks, such as simultaneously encoding or decoding multiple video streams. Twitch is using an FPGA from Xilinx.

The electric consumption is around 25 W for a PCIe card

3. Graphics processing units (GPUs): GPUs are another type of hardware. This hardware is more generic than FPGA and ASIC, but specialized video encoders/decoders are added to these cards. Nvidia was the first company to build a datacenter GPU for transcoding. Intel enters the market in late 2022.

The electric consumption is generally around 70W.

Regarding energy consumption ASICs > (are better than) FPGAs > GPU > CPU. This order is the same for the transcoding speed.

Reducing the carbon footprint of video transcoding : api.video’s approach

At api.video we are really focused on the energy consumption of our transcoding pipeline. We want to minimize the ecological impact of video transcoding while keeping the transcoding speed high. The infrastructure team has performed multiple benchmarks, and here are the results.

Those numbers are based on public information (marketing sheet) for the GPUs and FPGAs and real benchmarks for the CPU and ASICs. The quality of the video's output could change the results slightly, but we clearly see a winner regarding power efficiency. In a future blog post, we will share precise benchmarks about the ASICs vs. CPU battle with you. Stay tuned.

Along this article, we have seen the resurgence of dedicated hardware in computer science is driven by a growing need for high-performance computing, particularly in fields like artificial intelligence, machine learning, and video transcoding. Dedicated hardware can provide better performance and energy efficiency than traditional general-purpose computing units, making it an attractive option for many applications. As computing continues to evolve, we will likely see more specialized hardware being developed for specific tasks, driving innovation and progress in the field.

Our video API allows you to add videos to your products while prioritizing the reduction of your carbon footprint. Join us and take part of the transition to an environmentally responsible video infrastructure.