MFCopy produces incorrect duration when start and duration are specified

[DougRogers]

These options create a video with five seconds of video, but the duration is 24 seconds.

-s 20000 -d 5000 inputVideo.mp4 testOut.mp4

[DougRogers]

Using the existing sample and just changing the date stamp is the issue. There is some data in the sample that reflects the sample's current position, even though it is set via SetSampleTime. You have to copy the sample, but without calling CopyAllItems. Unfortunately, none of this is documented.

[roman380]

So, do you happen to have a solution by any chance? If you submit a patch or pull request I will update the sample app.

[DougRogers]

I am working on a solution. When CopyAllItems is not called, the duration is correct, but it introduces jittering in playback. I wish WMF was open source.

[DougRogers]

It looks like the jitter is a different issue. If I take the original code and add a start position, I get jitter in playback on the resulting video.

[DougRogers]

The attribute MFSampleExtension_DecodeTimestamp is the issue. This also causes the jitter because the decode time is not correct. Still working on the fix.

[DougRogers]

This works for me. I copy the sample, except the DTS. The DTS for the destination sample is derived from the source sample time stamp and DTS.

Add:

         IMFSample *destSample = NULL;
         hr                    = CreateAndCopySingleBufferIMFSample(sourceSample, &destSample, dstTimeStamp);

to _ProcessSamples.

Release the destination sample when done.

Derived from:
https://github.com/sipsorcery/mediafoundationsamples/blob/74c2f7cfcca2dc585a44f78931cc6a2630c0e106/Common/MFUtility.h#L1131
and
https://github.com/sipsorcery/mediafoundationsamples/blob/74c2f7cfcca2dc585a44f78931cc6a2630c0e106/Common/MFUtility.h#L1160

#define IFC(val)                                                                                                                                     \\
    {                                                                                                                                                \\
        hr = val;                                                                                                                                    \\
        if (hr != S_OK)                                                                                                                              \\
        {                                                                                                                                            \\
            goto done;                                                                                                                               \\
        }                                                                                                                                            \\
    }

HRESULT CopyAttributesExceptDecodeTimestamp(IMFSample *pSrcSample, IMFSample *pDstSample)
{

    UINT32 count = 0;
    pSrcSample->GetCount(&count);

    HRESULT hr = S_OK;

    for (UINT32 i = 0; i < count; ++i)
    {
        PROPVARIANT var;
        PropVariantInit(&var);
        GUID guidId;
        hr = pSrcSample->GetItemByIndex(i, &guidId, &var);

        if (hr == S_OK)
        {
            if (guidId == MFSampleExtension_DecodeTimestamp)
            {
                continue;
            }

            IFC(pDstSample->SetItem(guidId, var));
        }
    }
done:
    return hr;
}

/**
 * Creates a new single buffer media sample.
 * @param[in] bufferSize: size of the media buffer to set on the create media sample.
 * @param[out] pSample: pointer to the create single buffer media sample.
 * @@Returns S_OK if successful or an error code if not.
 */
HRESULT CreateSingleBufferIMFSample(DWORD bufferSize, IMFSample **pSample)
{
    IMFMediaBuffer *pBuffer = NULL;

    HRESULT hr = S_OK;

    IFC(MFCreateSample(pSample));

    // Adds a ref count to the pBuffer object.
    IFC(MFCreateMemoryBuffer(bufferSize, &pBuffer));

    // Adds another ref count to the pBuffer object.
    IFC((*pSample)->AddBuffer(pBuffer));

done:
    // Leave the single ref count that will be removed when the pSample is released.
    SAFE_RELEASE(pBuffer);
    return hr;
}

/**
 * Creates a new media sample and copies the first media buffer from the source to it.
 * @param[in] pSrcSample: size of the media buffer to set on the create media sample.
 * @param[out] pDstSample: pointer to the media sample created.
 * @@Returns S_OK if successful or an error code if not.
 */
HRESULT CreateAndCopySingleBufferIMFSample(IMFSample *pSrcSample, IMFSample **ppDstSample, LONGLONG destTimeSample)
{
    IMFMediaBuffer *pDstBuffer = NULL;
    DWORD srcBufLength;

    HRESULT hr = S_OK;

    // Gets total length of ALL media buffer samples. We can use here because it's only a
    // single buffer sample copy.
    IFC(pSrcSample->GetTotalLength(&srcBufLength));

    IFC(CreateSingleBufferIMFSample(srcBufLength, ppDstSample));

    IMFSample *pDstSample = *ppDstSample;

    // We do not use CopyAllItems since we want to manage Decode Timestamp
    // hr = pSrcSample->CopyAllItems(pDstSample);

    IFC(CopyAttributesExceptDecodeTimestamp(pSrcSample, pDstSample));

    LONGLONG sampleTime;
    hr = pSrcSample->GetSampleTime(&sampleTime);

    PROPVARIANT var;
    PropVariantInit(&var);
    hr = pSrcSample->GetItem(MFSampleExtension_DecodeTimestamp, &var);
    if (S_OK == hr)
    {
        // how early to decode the sample before the presentation time
        auto delta = sampleTime - var.hVal.QuadPart;

        // back the desintation time up that to set the decode time.
        auto destinationDecodeTime = destTimeSample - delta;

        if (destinationDecodeTime >= 0)
        {
            // only set decode time legal time
            var.hVal.QuadPart = destinationDecodeTime;
            hr                = pDstSample->SetItem(MFSampleExtension_DecodeTimestamp, var);
        }
    }
    PropVariantClear(&var);

    hr = pDstSample->GetBufferByIndex(0, &pDstBuffer);

    hr = pSrcSample->CopyToBuffer(pDstBuffer);

    LONGLONG sampleDuration;

    hr = pSrcSample->GetSampleDuration(&sampleDuration);
    hr = pDstSample->SetSampleDuration(sampleDuration);
    hr = pDstSample->SetSampleTime(destTimeSample);
done:

    SAFE_RELEASE(pDstBuffer);
    return hr;
}

[roman380]

To be honest, I am not convinced it's exactly what needs to be done. That is, that the problem is with MFSampleExtension_DecodeTimestamp. I quickly checked against one media file and the copy worked correctly. Indeed, the problem might be that sart poisition falls onto either middle of video GOP or onto frame reordering somehow. Maybe you can attach input file to reproduce this so that it's possible to see things under debugger.

[DougRogers]

Here is a link to an example.
https://www.dropbox.com/s/edjsg3nlc62t943/BootySwing.mp4?dl=0

The arguments:

-s 20000  -d 5000 BootySwing.mp4 testOut.mp4

produce a video that has the incorrect duration and where the video is jittery.

[roman380]

With this command line parameters and the file MFCopy's remuxing is incorrect because it does not take prerolling into consideration. As image below shows video processing should start at 18.143s, then frames up to 20s need to be discarded (let me omit details for brevity).

Instead the app converts 18.143s into 0.000s of output file.

Audio track is also affected. So essentially the app will only work properly (more or less) if start time falls into IDR.

The trimming function overall is pretty much broken in the sample, that is.