withPlanes
Reads a video frame's planes without copying them.
block receives one pointer and one row pitch per plane. Both are valid only until it returns: they point into the frame's own buffers, and the frame may be closed straight afterwards.
The row pitch is not the width. It is almost never the width. A renderer that assumes otherwise produces an image that skews diagonally, which is the most common first bug in every new video pipeline. The pitch is given here rather than left to be computed for exactly that reason.
The alternative is Frame.copyPlanesToByteArray, which is correct and costs a full copy of the frame: 3.11 MB for 1080p and 24.9 MB for 4K 10-bit, so between 187 MB/s and 1.5 GB/s at 60 frames a second, plus one allocation per frame. That is right for a thumbnail and unusable for playback.
Throws
when the frame lives in hardware memory, which has no readable planes. Check FrameInfo.isHardware first, and use Frame.hardwareSurface instead.