Graphics Composited Video

Description

Modern graphics cards like the Radeon HD don't generally have overlay hardware. Hardware overlay is an old and rather obsolete method to accelerate video playback by allowing video frames to be displayed directly in their native YUV format.

Rather than try to emulate overlay using the GPU, AmigaOS supports YUV formats directly via the CompositeTags() function. This is called composited video which combines the advantages of textured video (the overlay replacement on other OSes) with all the power of compositing.

Some key features/advantages of composited video are:

• Accelerates video playback by enabling planar YUV video frames to be rendered directly to a bitmap.
• No limit to how many videos can be displayed simultaneously (overlay is restricted to 1 or 2 hardware surfaces).
• Can be rendered anywhere from full-screen to multiple videos in a webpage (on AmigaOS, overlay is restricted to a window).
• Supports both SD and HD YUV video standards and even custom YUV to RGB transformation matrices (NOTE: you can exploit this to get brightness, contrast & saturation adjustment for free).
• Can use alpha blending for combining video with other graphics; this allows subtle anti-aliased blending, unlike overlay's on/off colour-keying.
• Can use video frames directly with the full power of CompositeTags(); this includes alpha blending, rotation, warping, cross-fades, vertex-arrays, etc.

The last item in the list above introduces a whole set of unique possibilities that previously weren't possible. It could be used for much more than just faster video display; it could be used for real-time video effects, from cross-fades to 3D transitions and more.

Performance

Composited video (and overlay) improve performance in two ways:

1. It reduces the bandwidth required to copy video frames to the graphics card (YUV420p bitmaps are 37.5% the size of an equivalent 32-bit RGBA bitmap).
2. It shifts the task of converting from YUV to RGB (the graphics card's native format) from the CPU to the GPU. The GPU is better suited to this task and it frees the CPU up to work on other things like decoding the next frame.

The net result is that it takes less processing power and bus bandwidth to display the same video.

What is YUV?

Basically, the Y channel stores a pixel's brightness/luminance, while the U & V channels store the colour information. Video files use this format instead of RGB because humans see brightness differences at a higher resolution than gray scale. So, we can get away with storing the colour channels at a lower resolution than the brightness without people noticing it. Effectively, we've compressed the video frame while maintaining high visual quality.

Much more information about YUV is available on Wikipedia.