How much memory would decoding every frame take?

All frames decoded
Current + next frame

Counts frame bitmaps at 4 bytes per pixel, with a separate buffer for each view.