video: Add support for playing F4V (MP4) files

[torokati44]

Similar to the somewhat related #14654, this is in a really early stage.

TODOs:

• Don't hardcode track index
• Precompute and store all frame ("sample" in MP4 lingo) offsets and lengths for all tracks?
• Handle audio
• Handle all supported video codecs
• Seeking
• Consider whether mp4 would be a better fit than mp4parse
  • Not really: It's larger (according to crates.io, and it includes writing as well as reading), and is really geared towards working with files on disk - unlike mp4parse, which is made for a really similar use case in Firefox.
• How about Symphonia?
  • Nope, sadly it won't even give me the codec type or extradata of video tracks.
• Resolve inevitable conflicts with Progressive download of video files #14647
• Consider adding the ability to play MP4 files directly (as is on https://z0r.de/3099 - WARNING: content is not nice!)
  • Out of scope for this PR. Some other kinds of files (maybe images?) could also be opened this way.
• Cleanup and testing, as usual

[danielhjacobs]

Seems to now be called max_time:

ruffle/core/src/streams.rs

Line 1110 in df0ac43

let max_time = write.stream_time + dt;

[danielhjacobs]

I know this is still a draft, so I don't know if it's supposed to be panic-free yet, but on http://new.weedtowonder.org/, when click the play button on the video, this caused the panic :

panicked at core/src/streams.rs:850:63:
called `Result::unwrap()` on an `Err` value: InvalidData(CheckParserStateErr)

[danielhjacobs]

Every video on that site has that same result.

[torokati44]

Is this on desktop for you? Or with the extension?

[torokati44]

Aand now the site seems to have changed...?

[danielhjacobs]

On web after building the extension, and I changed the site to add https. It's now https://new.weedtowonder.org/.

[torokati44]

Just found a cool web based MP4 dissector tool: https://gpac.github.io/mp4box.js/test/filereader.html
EDIT: This might also be useful: https://mpeggroup.github.io/FileFormatConformance/

[danielhjacobs]

Also note, this time testing with the attached files and a directory named mp4-1200 containing the files from https://github.com/danielhjacobs/danielhjacobs.github.io/tree/main/mp4-1200, I get this error when opening anastasia2017-teacher.swf with a Desktop app built from this PR and clicking one of the videos:

panicked at core/src/streams.rs:1384:26:
called `Result::unwrap()` on an `Err` value: DecoderError(DecoderError(MiddleOfBitstream))

anastasia.zip

[danielhjacobs]

Replacing https://github.com/ruffle-rs/ruffle/pull/14655/files#diff-894b995744e59b9d8803f92eb6b185191b9018cd571feeaeb4a7c53dbd9a0bc5R1380-R1385 with this removes the panic:

                match context.video.decode_video_stream_frame(
                    video_handle,
                    encoded_frame,
                    context.renderer,
                ) {
                    Ok(bitmap) => {
                        write.last_decoded_bitmap = Some(bitmap);
                    }
                    Err(e) => {
                        tracing::error!(
                            "Decoding video frame {} failed: {}",
                            frame_id.unwrap(),
                            e
                        );
                        return;
                    }
                }

But that doesn't fix the underlying issue, as then there is the message ruffle_core::streams: Decoding video frame 0 failed: Decoder error and the video won't open in browser anymore as a workaround as it attempts to be decoded (and fails).