Hi there,
We've been struggling with a similar issue in the Multimedia team. If we want Extension:MultimediaViewer to scale properly, we should probably bucket our requests for thumbnails to a specified list of sizes. This may be a way to test different solutions to this problem without necessarily figuring out a canonical solution in core.
Basically the idea would be to try bucketing the thumbnail requests to a set of 10-20 predetermined sizes, then either requesting the nearest one above the ideal size and scaling down on the client, or requesting the nearest one below and showing the image in a container with a frame.
If we can be of any use in testing possible solutions, we'd love suggestions as to implementation details.