Extension:TimedMediaHandler/Test media

See copy on testwiki at https://test.wikipedia.org/wiki/TimedMediaHandler/Test_media

I've made a couple of high-resolution sample video clips of a bubbling fountain, which may be freely used for testing TimedMediaHandler's ingest, transcoding, and playback. The scripts for making various formats and sizes from the source clips will be checked into git shortly; because the files are large they're being placed on-wiki here to avoid cluttering up the git repos. --Brooke Vibber (WMF) (talk) 18:22, 17 August 2023 (UTC)

Source clips

Two 20-second video clips, one at 1920x1080 59.94fps HDR, and the other at 3840x2160 29.97fps HDR. (These are the highest resolution options available on my camera at each frame rate.)

Currently our transcoding doesn't correctly pass through HDR settings to output and it ends up with the wrong colorspace without any mapping. This is fixable in TMH but needs some fancy ffmpeg calls, and isn't yet finished.

• TimedMediaHandler_test_source_HDR_1080p60.webm (todo: finish tone mapping)
• TimedMediaHandler_test_source_HDR_2160p30.webm (todo: finish tone mapping)

Derived clips

• SDR down-conversions of the two HDR sources at full resolution
  • TimedMediaHandler_test_source_SDR_1080p60.webm
  • TimedMediaHandler_test_source_SDR_2160p30.webm
• "codec" tests converting the 30fps sources to 360p VP8, VP9, Theora, MPEG-1, MPEG-2, MPEG-4, H.264/AVC, and HEVC. Note that MPEG-4, H.264, and HEVC should not pass through uploads at present as those codecs have not been approved for WMF usage, but they should be recognized so that can be enforced - thus they are not included in this archive but can be regenerated from scripts.
  • TimedMediaHandler_test_codec_MPEG-1_360p30.mpg
  • TimedMediaHandler_test_codec_MPEG-2_360p30.mpg
  • TimedMediaHandler_test_codec_Theora_360p30.ogv
  • TimedMediaHandler_test_codec_VP8_360p30.webm
  • TimedMediaHandler_test_codec_VP9_360p30.webm
  • TimedMediaHandler_test_codec_VP9_HDR_360p30.webm (todo: finish tone mapping)
• "small" test with a tiny 160x90 frame size to confirm playback isn't confused
  • TimedMediaHandler_test_small_90p30.webm
• "crop" tests with 4:3 and 2.39 aspect ratios to confirm playback isn't confused
  • TimedMediaHandler_test_crop_narrow_720p30.webm
  • TimedMediaHandler_test_crop_wide_720p30.webm
• "legacy" format conversions from 60fps SDR sources to VideoCD (MPEG-1) DVD (MPEG-2 progressive and interlaced), and ATSC-style HD video (MPEG-2 progressive and interlaced). These should all upload and work. Note that currently our file type detection requires an MPEG-1 program stream, but it may contain MPEG-2 video:
  • TimedMediaHandler_test_legacy_VCD_NTSC_240p30.mpg
  • TimedMediaHandler_test_legacy_VCD_PAL_276p30.mpg
  • TimedMediaHandler_test_legacy_DVD_widescreen_480p24.mpg
  • TimedMediaHandler_test_legacy_DVD_NTSC_480i30.mpg
  • TimedMediaHandler_test_legacy_DVD_PAL_576i25.mpg
  • TimedMediaHandler_test_legacy_HDTV_720p60.mpg
  • TimedMediaHandler_test_legacy_HDTV_1080i30.mpg
• "chroma" subsampling tests comparing 4:2:0, 4:2:2, and 4:4:4 input in WebM VP9 (all should be converted to 4:2:0 output)
  • TimedMediaHandler_test_chroma_420_240p30.webm
  • TimedMediaHandler_test_chroma_422_240p30.webm
  • TimedMediaHandler_test_chroma_444_240p30.webm

Automation

Given the two source files, all derived files can be regenerated with a Makefile that wraps ffmpeg commands for each file. This will get checked into source control soon.

Currently there's no automation for the upload process, or fetching the files for local testing. This may be added, but preserving the artifacts and their creation scripts is the most important thing!

Production testing

The files here on mediawiki.org can serve as canonical examples of the test files in the production environment. They will also be mirrored to test.wikipedia.org which will sometimes run an "advance configuration" with features not yet enabled live on the other wikis -- we'll be using this for doing some more serious checking for regressions and edge cases on the iOS-compatible HLS media generation.

See User:Brion Vibber (WMF)/Mobile video playback 2023

Things to check are working on the new transcode settings

General updates will go live everywhere once the patch merges and deploys, per usual staging. Will land to test first, though, so check the mirrors over there.

• interlaced MPEG-2 files should now output one frame per field, preserving temporal resolution: so 480i30 MPEG-2 input -> 480p60, 360p60, 240p60 VP9 output
• bitrates should be more consistent, and scaled according to framerate: so 24 fps videos may use lower bandwidth than 30 fps or 60 fps
• high-frame rate media should convert reasonably

Things to check on the new HLS output

(Expected to go live October 2023!)

This will go live on test before it's enabled elsewhere to shake out any remaining bugs before we throw peoples random iPhones at this.

• most newer iPhones (check version requirement) will see the full range of resolutions of VP9 video, auto-selected by the HLS player
• some older iPhones back to iOS 12 (double-check this) will see a low-res Motion-JPEG video instead because they don't grok the VP9 and we haven't changed our policy about H.264 yet.
• audio tracks should "just work", but iOS 17 will use Opus and iOS 16 and earlier will use MP3
  • confirm nothing breaks if you attempt to change audio or anything like that...
• there should be no surprises like double-playback
• Picture-in-Picture and AirPlay should work as expected, with both VP9 and Motion-JPEG (though the latter will be very blurry on AirPlay!)
  • note AirPlay is not enabled yet, but we can turn it on later (it needs room in the UI!)