[DRM][Playready] Initial implementation

[TheDaChicken]

Description

Implement PlayReady Support using seemly new (?) MediaFoundation API CDM.
MediaFoundationCDM was added to Window's API in Windows 20H2¹.

Examples I am going off:

1. https://github.com/microsoft/media-foundation/tree/master/samples/MediaEngineEMEUWPSample
2. https://github.com/chromium/chromium/blob/fd8a8914ca0183f0add65ae55f04e287543c7d4a/media/cdm/win/media_foundation_cdm.cc#L277
3. https://github.com/chromium/chromium/blob/master/media/cdm/win/media_foundation_cdm_module.cc

Motivation and context

PlayReady DRM content in inputstream.adaptive will improve ISA plugins long-term for Windows.

I've been hiding in the shadows & using Kodi (randomly appeared because @CastagnaIT 's netflix plugin broke), loving ISA, and am willing/wanting to try and get something working.

How has this been tested?

This requires Windows above 20H2 for testing.

The code I am started out with was tested using an external test program I wrote.

Screenshots (if appropriate):

Types of change

• Bug fix (non-breaking change which fixes an issue)
• Clean up (non-breaking change which removes non-working, unmaintained functionality)
• Improvement (non-breaking change which improves existing functionality)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that will cause existing functionality to change)
• Cosmetic change (non-breaking change that doesn't touch code)
• None of the above (please explain below)

Checklist:

• I have read the Contributing document
• My code follows the Code Guidelines of this project
• My change requires a change to the Wiki documentation
• I have updated the documentation accordingly

What I have

Currently, I have somewhat of a "mock cdm" class.

Current State of the PR:

• I was able to generate a challenge and print it out after creating a CDM context.

And, this PR is not 100% complete, not everything I have here is set in stone. Ex: license header under source/headers files and bad file structure.

Footnotes

1. Important Note under README.md ↩

[CastagnaIT]

Hi,
first of all big thanks to this contribution!
if it works well it is great thing for all

some basic initial considerations

i invite you to temporary stop your development
because currently as you can see on GH roadmap @glennguy is working to completely rewrite the ssd_wv project,
this implies that your work will be subject to many changes that may potentially force you to rewrite many part of your code,
for this, i think @glennguy can give us some advice on his current progress status with the ssd_wv project merging

I just had a very quick look at the code without going in depth

at glance i see that there are some code that dont follow our standards, suggest you to give a quick look at our guidelines:
https://github.com/xbmc/inputstream.adaptive/wiki/Contributing-and-code-guidelines#code-guidelines
https://github.com/xbmc/xbmc/blob/master/docs/CODE_GUIDELINES.md
and where needed adapt your code

new files rules: they must have always our SPDX license:

/*
 *  Copyright (C) 2023 Team Kodi
 *  This file is part of Kodi - https://kodi.tv
 *
 *  SPDX-License-Identifier: GPL-2.0-or-later
 *  See LICENSES/README.md for more information.
 */

and then no custom licenses or custom credits
specifics credits are linked to github PR's and commits for who interested
for this reason can be used also the Co-authored-by: git declaration for multiple authors

new files that are licensed by thirdy parties like the "Chromium" one
can keep their license header or can also be converted to our SPDX format,
the required license definition file for the thirdy parties files, if missing must be add to "licenses" folder

ifdeffing like this #ifndef PLAYREADY_API_TEST_ERROR_H must be replaced by #pragma once on all files

in case not done, i also remember you that the new code must be Clang-formatted, on VS you can use CTRL+K+F shortcut keys to automatically format the selected code lines in easy way

as per guidelines, no "funny" commit titles but descriptive and shorts, no censured PR titles
in your case should appears like
[DRM][Playready] Initial implementation or so

i have a question this will works also for UWP platform?

[TheDaChicken]

Thank you so much for the reply and also for telling me what's going on currently!

I'll pause currently until I'm told otherwise or if there is any new information.

I want to get this working and up to code guidelines. I mean, this was a very rough draft if you call cause it that, hahaha.

[glennguy]

Hi @TheDaChicken !

Thanks so much for your interest in this project. As @CastagnaIT mentioned I'm doing a big refactor on wvdecrypter to accomplish several things:

• Break up the monolithic wvdecrypter.cpp into separate header and source files for each class
• Build all into 1 single library instead of having a separate ssd_wv one
• Still maintain the common interface to avoid any tight coupling in the main IA components
• More probably, can't think exactly right now

I'm around 50% through I think, hoping to PR and start the review process within the next 2 weeks.

Your work looks very exciting! It seems like this would work on Xbox which would make a lot of users on that platform happy since Widevine could never work outside of building your own package with bundled CDM and loading in dev mode.

Please monitor this repo for my progress

[TheDaChicken]

It seems like this would work on Xbox

I didn't even think about that! That would be great

Also something interesting: This CDM allows debugging. I don't know if using the renderer to get decoded frames allows that but we'll see!

EDIT: (I missed this question)

this will works also for UWP platform?

Yes. There are examples of Media Foundation being used under UWP platform.

[glennguy]

Thanks for your patience @TheDaChicken! The above mentioned refactor has been merged in so if you're keen please continue the development!

[TheDaChicken]

Thanks for your patience @TheDaChicken! The above mentioned refactor has been merged in so if you're keen please continue the development!

All right just synced the fork and stuff,

Where should I create the "MediaFoundationAdapter" aka classes for initializing MF? Does it look like the bast place would be "lib/mf"? lib/cdm/win? chromium places theirs at media/cdm/win.

That way I can separate an "MFDecrypter" inheriting IDecrypter to src/decrypters/mediafoundation without so many files in the folder.

[CastagnaIT]

despite "cdm" have a generic meaning, for existing "lib/cdm" folder we refer to it for widevine only,
imo its better make a separate folder for a "MediaFoundation CDM" interface, something like lib/mfcdm or lib/mf_cdm

i remember you that when you add an external library to lib/** must be unrelated to the ISA project files
so the external library dont have to contains includes to any of the ISA project files

[TheDaChicken]

i remember you that when you add an external library to lib/** must be unrelated to the ISA project files

Oh my gosh I should have asked this aswell:

I also was using cdm::SessionType etc. from cdm_library. I could use I am unsure if linking to cdm_library would be a great idea? I was creating a similar mode to CdmAdapter.

[CastagnaIT]

I also was using cdm::SessionType etc. from cdm_library.

mmh now i dont have a full background of MF but i would have expected that MF is an independent CDM, so the question is:
MF require some source code of widevine interface in order to work? Or are you just doing this for convenience?

if you start use headers of WV-CDM folder, and in the future widevine change interface or the wv interface require some refactor/rework, this will lead us to a big problem with MF, which will be broken for a long time due to possible changed headers and code, and will be needed also someone that will have time/competence to rework it

I could use I am unsure if linking to cdm_library would be a great idea?

linking CDM library to MF library? as my response above, atm this sound a very bad idea
it would probably make more sense to copy/take ispiration from WV-CDM sources and not depends form it

side note: please note that files on "cdm" folder dont follow our code guidelines, this because come from external sources, so if you take inspiration from them, please do not copy/paste that code directly but adapt it by respecting our code guidelines

[TheDaChicken]

Or are you just doing this for convenience?

Basically convenience-ish? I would have to expose "mfapi.h" outside lib/mfcdm unless I make my own implementation of an enum etc

Because logically, it'd be great for mfcdm to be an independent piece of code w/ sources only talking to MF.

this will lead us to a big problem with MF, which will be broken for a long time due to possible changed headers and code, and will be needed also someone that will have time/competence to rework it

Gosh! that's true. The whole reason I am asking these questions because my main goal is to go through and make MF access painless and an "interface" similar to Widevine. That way, it is painless to read.
It would look like a mess if I didn't create an "interface" to access MF. winrt::com_ptrs, AccessFactories horrible to deal with.

please note that files on "cdm" folder dont follow our code guidelines

Yes, 100% noted.

I asked to prevent repeat code when classes already technically existed?
Though... also adding ... UWP platform... not needing cdm_library aswell..

[CastagnaIT]

Basically convenience-ish? I would have to expose "mfapi.h" outside lib/mfcdm unless I make my own implementation of an enum etc

i agree, add its own interface in to ML library its correct in order to avoid expose propertary components of windows api's directly to the client (ISAdaptive sources)

[TheDaChicken]

I feel a bit stuck again. I still feel the Widevine decrypter implementation is still a little quirky trying to work with and painful. I feel like I am going to have to repeat a bunch of code.

Could/Should I open a PR for that? It's very hard since I am unsure about the codebase. Should probably wait for "Reintroduce CDM aarch64 support" though.

[CastagnaIT]

I feel a bit stuck again. I still feel the Widevine decrypter implementation is still a little quirky trying to work with and painful. I feel like I am going to have to repeat a bunch of code.
Should probably wait for "Reintroduce CDM aarch64 support" though.

i dont know exactly what you are thinking to do, but this makes me think that again you want to bring changes to widevine,
if so, as said on my previous comment the widevine interface source code come from external sources, if now you rework/change code on cdm-wv folder, for your purposes, you will cause to us more headaches to maintain the code from the official sources,
and future issues related to possible widevine interface changes

maybe you are looking at things from a wrong perspective?
At the end it should be important just to have a way to make happy the methods in the `/decrypters/IDecrypter.h' interface to give them the data that they require, this dont limit you to following what the widevine interface does and how they implement all things to works with his proprietary binary, because allow you to create all implementation as you wish

maybe @glennguy have time to give also his opinion,
from my point of view mediafoundation/playready cdm should live totally independently although parts of interfaces can be similar to wv

[TheDaChicken]

if so, as said on my previous comment the widevine interface source code come from external sources

I am talking about the WVDecrypter implementation. An example would be in "WVCencSingleSampleDecrypter." Contains same code for parsing license key & response in both android and non-android. PlayReady wouldn't need that but I am just saying as an example.

If you think it's fine then it's okay. I'll just go with it. I don't wanna mess with much.

I agree that cdm_library is fine and I don't think really needs to be touched. It does what it needs to do.

[CastagnaIT]

ah ok now its more clear, yeah there are duplicate code between android/non-android wv implementation
like mentioned WVCencSingleSampleDecrypter
currently as you have seen we have just refactor decrypter code, mainly to remove the separate project,
so the code has been re-adapted but without cleanup to avoid possible regressions as possible

That's fine for now, don't worry too much about repeated/duplicated code
there is a lot of code to clean where some parts require also some rework,
it will be done in future, but if you want to propose some cleanup where this PR can benefit from it,
yes on separate PR so that we can evaluate it avoiding to mixing cleanup with this PR implementation

if you are uncertain about your cleanups you think to do, you can also open an Issue thread
to explain your changes, e.g. also linking examples from a your GH branch, so we can discuss them

[TheDaChicken]

I also tried a test stream:

#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_key=|
#KODIPROP:inputstream.adaptive.license_type=com.microsoft.playready
https://test.playready.microsoft.com/media/dash/APPLEENC_CBCS_BBB_1080p/1080p.mpd

IA tries to look for PSSH in MOOV even when PR tag was parsed.
Plus, I think MF wants the straight pssh from mspr:pro tag. MF parses it and handles all of that.

[CastagnaIT]

hi nice work thank you for your perseverance,
you will be asked to adjust different code over the time, so be patient and take your time

atm i dont have much time to take a deep look on this weekend, just some fast points,

1. please dont use merging Merge branch 'Omega' into, so remove that commit and instead rebase your local branch vs the upstream Omega branch
2. i think its more appropriate use the windows macro define WIN32 instead of
   if(MSVC), #if _MSC_VER, please replace
3. DrmFactory.cpp has been reintroduced wrong return chars (i think a typo due to missing rebase)
   
4. on the "file changed" PR view, you can see that many files are missing of endline, see warning icon
   

[TheDaChicken]

1. please dont use merging Merge branch 'Omega' into, so remove that commit and instead rebase your local branch vs the upstream Omega branch

I saw there were conflicts on the PR. GitHub did straight things and did some weird merging. I'm never doing that again... sorry about that.

2. i think its more appropriate use the windows macro define WIN32 instead of
   if(MSVC), #if _MSC_VER, please replace

Yep, _MSC_VER is a very bad macro for this use case.
I think MF is only available on MSVC. Something like mingw64 may be considered "win32."

I could do if(WIN32) and then check if mf headers exist. Though that'd require complier definition so it knows that those headers exists..

4. on the "file changed" PR view, you can see that many files are missing of endline, see warning icon
   

Oop, okay, I'll fix that.

atm i dont have much time to take a deep look on this weekend, just some fast points

Take your time as well!

you will be asked to adjust different code over the time, so be patient and take your time

No problem!

[CastagnaIT]

Yep, _MSC_VER is a very bad macro for this use case.
I think MF is only available on MSVC. Something like mingw64 may be concerned "win32."
I could do if(WIN32) and then check if mf headers exist. Though that'd require complier definition so it knows that those headers exists..

thanks for explanation,
_WIN32 should provided by msvc can you try verify?
https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-170&viewFallbackFrom=vs-2019
it could be better replacement also for _MSC_VER used to get the version number

MediaFoundationCDM was added to Window's API in Windows 20H2

we need also to think a way to limit this feature from the specified windows version at runtime

[CastagnaIT]

I think i will stop here for now for review, since with my old previous comment and these new ones will require a significant code changes

[CastagnaIT]

I also tried a test stream:

#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_key=|
#KODIPROP:inputstream.adaptive.license_type=com.microsoft.playready
https://test.playready.microsoft.com/media/dash/APPLEENC_CBCS_BBB_1080p/1080p.mpd

IA tries to look for PSSH in MOOV even when PR tag was parsed. Plus, I think MF wants the straight pssh from mspr:pro tag. MF parses it and handles all of that.

i have not tested it
but i think that the MPD pssh is not parsed for playready case
https://github.com/xbmc/inputstream.adaptive/blob/21.3.0-Omega/src/parser/DASHTree.cpp#L1293C10-L1302
also its not implemented for HLS, but for now go well make works for DASH case only (should works also for ISM), HLS can be done in future

[TheDaChicken]

but i think that the MPD pssh is not parsed for playready case
https://github.com/xbmc/inputstream.adaptive/blob/21.3.0-Omega/src/parser/DASHTree.cpp#L1293C10-L1302
also its not implemented for HLS, but for now go well make works for DASH case only (should works also for ISM), HLS can be done in future

Yeah that is basically what I am talking about. I am confused because PRProtectionParser exists but it's not being used for anything other than printing?

[CastagnaIT]

I am confused because PRProtectionParser exists but it's not being used for anything other than printing?

its used to extract kid, depends on the data provided with manifest
now this code was reworked but kept same behaviour from the old isa versions,
i don't know exactly for what use cases it was thought to extract the kid from PR,
perhaps in the past it was intended to use in conjunction with license_data property that allow to provide custom pssh

[CastagnaIT]

maybe im wrong but this condition seem weird, for widevine this is not mandatory
why should be on playready? take in account that we dont allow to store encrypted content to device storage for later uses nor store license

[TheDaChicken]

Yep, _MSC_VER is a very bad macro for this use case.
I think MF is only available on MSVC. Something like mingw64 may be concerned "win32."
I could do if(WIN32) and then check if mf headers exist. Though that'd require complier definition so it knows that those headers exists..

thanks for explanation, _WIN32 should provided by msvc can you try verify? https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-170&viewFallbackFrom=vs-2019 it could be better replacement also for _MSC_VER used to get the version number

MediaFoundationCDM was added to Window's API in Windows 20H2

we need also to think a way to limit this feature from the specified windows version at runtime

From my testings:
_WIN32 definition includes both mingw64 and msvc.
MF headers are actually available on mingw64.

It would be better to check for Windows's SDK 20H2 in CMake & compiler time to include lib/mfcdm headers.

MediaFoundation did exist before Windows 20H2 BUT headers didn't exist. During runtime, MF will just fail to initialize if it doesn't exist. that's okay.

[TheDaChicken]

Is something like this okay?

Using _WIN32 macro and also using VER_PRODUCTBUILD to make sure MF headers exists.

#if _WIN32
#include <ntverp.h>
#if VER_PRODUCTBUILD >= 1001 // Windows SDK higher than Windows 20H2
#include "mediafoundation/MFDecrypter.h"
#endif
#endif

[CastagnaIT]

i see code improvements, i will take a look at weekend or next week
please stop use git merge to update your local branch,
instead use git fetch <upstream name> && git rebase <upstream name/branch name>
because "git merge" generates messes in the existing changes that are persistents
an example is DrmFactory.cpp that has been overwrited, you need to restore it to its original state and after readd your changes, you can also squash all your commits in a single one to make easier rebases

#if VER_PRODUCTBUILD >= 1001

should be 10011 ?
seem that ms api use same value on all sdk versions, but im not aware of better things

[TheDaChicken]

please stop use git merge to update your local branch,
instead use git fetch <upstream name> && git rebase <upstream name/branch name>

OOPS! I didn't correlate GitHub merge vs Git merge.

should be 10011 ? seem that ms api use same value on all sdk versions, but im not aware of better things

Yeah, I think you are right.

Hmmm, maybe this is better? NTDDI_WIN10_VB is Windows 20H2.

#if _WIN32
#include <SdkDdkVer.h>
#if WDK_NTDDI_VERSION >= NTDDI_WIN10_VB // Windows SDK higher than Windows 20H2
#include "mediafoundation/MFDecrypter.h"
#endif
#endif

[CastagnaIT]

I tried what you said. rebase still didn't want to properly work

assuming locally you have just one commit, because you followed my command list, with same order,
to push changed to PR, since now your local branch is now diverged from your remote, you need to "force" push -f:
git push -f TheDaChicken/playready-mediafoundation to "overwrite" your remote branch

git rebase fails every time there is a code conflict, means that someonelse has changed some code around your changes,
and you need to recheck whats happened to readapt your code and avoid mistakes,
so when this happen with git status you can see the list of files with code conflicts to be resolved manually
after you fixed manually files with your code editor you can continue the rebase

yeah if you have never used git you need a bit of time to understand how works most used commands, there are a lot of easy tutorials on web, often i seen users use git merge maybe because it initially seems easier but is not so... maybe a small project could be ok use it

if you are unable to fix your branch by yourself, i can try help you then you have just to make a copy of my fixed branch
but it would be nice for you to learn how to use rebase that fundamental
in case let me know

[TheDaChicken]

git rebase fails every time there is a code conflict, means that someonelse has changed some code around your changes,
and you need to recheck whats happened to readapt your code and avoid mistakes,

Git thinks that my local is updated.... it definitely isn't.

What do you think I should do - Github automatically closed the commit because I tried resetting to the original commit- it didn't like it at all.

I could reset to 315ea9b. Am unsure if that would be a great commit to hard rest too. Then add my code back.

[CastagnaIT]

you have done really too much mess in your branch...
i have cleanup your branch, so you can fetch playready_branch from my repo:
https://github.com/CastagnaIT/inputstream.adaptive/tree/playready_branch
i have merged all old commits into one single commit "wip"

to reset your branch with my branch:
assuming you have already add and fetch my repo (git remote add.. git fetch...)
set your branch playready-mediafoundation as current working branch,
then, reset your branch with commit of my branch:
git reset --hard castangnaitUpstream/playready_branch

check if success:
git log
you have to see as last commit "wip"

now you have sync the branch to last upstream changes:
git rebase yourkodiupstreamname/Omega

now try build and fix remaining things...
when you have to save your new changes to existing "wip" commit:

git reset --soft HEAD^
git add .
git commit

at the end to update your PR you need to do a forced push

[TheDaChicken]

Oh my lord. Thank you so much.

[CastagnaIT]

i tried test the code and i see that needs futher developments to be able to play a drm video
let we know when the PR will be more stable

[TheDaChicken]

i tried test the code and i see that needs futher developments to be able to play a drm video

Yep. The latest commit should now create an MF session. It writes MF's session message to 9A04F079-9840-4286-AB92-E65BE0885F95.message for debugging. The message contains the challenge and HTTP headers (Yes, PlayReady is that quirky) in XML format. I find the base64 decoded challenge very interesting. Also, the session ID is a base64 decoded message as well.

[TheDaChicken]

This is the current .strm file I am using now:

#KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.manifest_type=mpd
#KODIPROP:inputstream.adaptive.license_key=https://test.playready.microsoft.com/service/rightsmanager.asmx?cfg=(persist:false,sl:150)|Origin=https://testweb.playready.microsoft.com&Referer=https://testweb.playready.microsoft.com/&User-Agent=Mozilla%2F5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F118.0.0.0%20Safari%2F537.36%20Edg%2F118.0.2088.57||
#KODIPROP:inputstream.adaptive.license_type=com.microsoft.playready
#KODIPROP:inputstream.adaptive.license_flags=persistent_storage
https://test.playready.microsoft.com/media/dash/APPLEENC_CBCS_BBB_1080p/1080p.mpd

The MPD pssh contains an old LR_URL. If license_key is present with a URL, it uses that instead.

[TheDaChicken]

Here is an update on what I've learned:

I started working on MFMediaEngine. This allows decoding (if you can call it that) from created MF Samples. It's very tedious. You have to create a virtual window etc to enable output protection. All of that is in chromium & a bit in the MF EME sample.

I figured out MediaEngine has 2 ways to output decoded frames. One using what's called "frame-server" and one called DirectComposition. "frame-server" outputs direct11x audio frames (which seemed like the best option originally) or MediaEngine outputs a "surface" for DirectComposition.

According to Microsoft's documentation and Chromium's comment messages, protected content is only available on DirectComposition or you have to set an application cert in SetApplicationCertificate (mf doc) frame-server.

DirectComposition outputs only a surface, not a frame. This means it requires a bit of Kodi ends.... I don't know enough about Kodi's codebase..... that's problematic

Edit: I also want to mention that direct11x audio frames can be converted to regular frames. It requires reading and allowing CPU access to the frame. Though, Chromium does output a "staging GPU frame" so the memory stays on the GPU and no CPU reads required which may reduce performance?

[CastagnaIT]

set an application cert in SetApplicationCertificate for frame-server.

i dont know if we have a certificate and if its possible have a way to pass it here
i have asked to the team hoping someone give us some info/help

DirectComposition outputs only a surface

unfurnately im not able to help you on "surface" things
but i can try ping some windows dev @thexai @CrystalP,
maybe they are able to provide you some kind of help

[CrystalP]

Hi, thanks for your interest and efforts!

Frame-server seems "easier" but it seems unlikely that Kodi could get one of the certificates to unlock that mode. I didn't see the comment in Chromium or MS doc for details.
Direct composition then looks like the way to go but with protected content it's not clear what kind of interop with D3D is allowed.

In that mode however MediaEngine behaves like a complete player (with the exception of displaying the video). I don't know what the addon interface used by ISA allows?

Using mediaengine could be interesting also for protected files, not only internet streams.

[TheDaChicken]

Hi, thanks for your interest and efforts!

Yes, thank you haha.

Frame-server seems "easier" but it seems unlikely that Kodi could get one of the certificates to unlock that mode. I didn't see the comment in Chromium or MS doc for details.

It's not very clear if SetApplicationCertificate enables frame-server mode. Chromium says nothing about it in the comments and MF documentation says "Call this method to access protected video content in frame-server mode."

Direct composition then looks like the way to go but with protected content it's not clear what kind of interop with D3D is allowed.

In the eme sample, all I could understand was some surface handle that is gathered during a MediaEngine event. line
I'm sadly not very familiar with stuff like that.

Just now, I went through a rabbit hole on chromium and found this. It mentions d3d from direct composion. ¯\(ツ)/¯ Chromium took the Direct Composition handle and created lots of wrappers around it.

In that mode however MediaEngine behaves like a complete player (with the exception of displaying the video). I don't know what the addon interface used by ISA allows?

For protected content, MF needs IMFTrustedInput. That can be taken from MFContentDecryptionModule. That needs to be passed somehow to MediaEngine.

I do want to mention that IMFTrustedInput can only be inserted into MediaEngine by creating a custom IMFMediaSource.. A custom IMFMediaSource requires a custom IMFMediaEngineExtension to insert a custom MediaSource.... this is the kind of tedious stuff I was talking about.

For ISA, StreamCryptoSession gets sent across to Kodi for crypto stuff for Kodi to handle it. It's used for example, Android's Mediacodec.. There is no way currently to pass anything related to MF down to Kodi easily I think.

Using mediaengine could be interesting also for protected files, not only internet streams.

Never thought about that!

If you have any questions please let me know. I did start working on MediaEngine so I know somewhat what I am saying... it's something...

[CrystalP]

Based on MS docs and eme sample there seem to be three ways to get video out in "rendering" mode:
IMFMediaEngineClassFactory::CreateInstance with MF_MEDIA_ENGINE_PLAYBACK_HWND > direct to a window
IMFMediaEngineClassFactory::CreateInstance with MF_MEDIA_ENGINE_PLAYBACK_VISUAL > to supplied IDCompositionVisual
IMFMediaEngineEx::EnableWindowlessSwapchainMode + GetVideoSwapchainHandle for a swap chain handle to be bound to a direct composition visual

Kodi is written in C++ with Win32 / D3D API and MediaEngineDCompWin32Sample (same place as eme sample) is closer to what things would look like as far as video output goes. Adding the missing pieces in Kodi to accept a swap chain handle like in the sample looks doable.

Kodi doesn't use Direct Show / Media Foundation like other players on Windows so it's not only the crypto (or whatever special sauce needed to unlock protected content in MF) that's needed.

[TheDaChicken]

Based on MS docs and eme sample there seem to be three ways to get video out in "rendering" mode:
IMFMediaEngineClassFactory::CreateInstance with MF_MEDIA_ENGINE_PLAYBACK_HWND > direct to a window
IMFMediaEngineClassFactory::CreateInstance with MF_MEDIA_ENGINE_PLAYBACK_VISUAL > to supplied IDCompositionVisual
IMFMediaEngineEx::EnableWindowlessSwapchainMode + GetVideoSwapchainHandle for a swap chain handle to be bound to a direct composition visual

Chromium creates a virtual window only for output protection, and Chromium writes it to MF_MEDIA_ENGINE_OPM_HWND. It seems MF_MEDIA_ENGINE_PLAYBACK_HWND is an option too with same HWND argument. It needs some sort of window for crypto. It doesn't need to be real for it to work. Chromium uses that AND the GetVideoSwapchainHandle method.

Kodi is written in C++ with Win32 / D3D API and MediaEngineDCompWin32Sample (same place as eme sample) is closer to what things would look like as far as video output goes. Adding the missing pieces in Kodi to accept a swap chain handle like in the sample looks doable.

Ah, I see. I'm glad it looks doable!

Kodi doesn't use Direct Show / Media Foundation like other players on Windows so it's not only the crypto (or whatever special sauce needed to unlock protected content in MF) that's needed.

Yeah, I knew Kodi wouldn't have used Media Foundation at all. In my brain, it's just a weird windows api.

Thank you so much for looking more into this. I really appreciated it.

Hopefully, I'm providing some insight.

[CastagnaIT]

i have little knownledge on how works interaction with protected content between Kodi/ISA
also because has been implemented by old devs no longer here and our old ancient commits have no descriptions

for example for widevine case (to work on systems different from android) use kodi interface callbacks methods:
CVideoCodecAdaptive::AddData
CVideoCodecAdaptive::GetPicture
that you can see from main.cpp
these methods that are overridden from API
https://github.com/xbmc/xbmc/blob/master/xbmc/addons/kodi-dev-kit/include/kodi/addon-instance/VideoCodec.h#L309-L330

AddData is used to decrypt and decode a packet of data into a frame
GetPicture is used to convert the frame to a kodi picture format

from here seem kodi API inputstream interface require frames,
ofc if required its however possible extent it for other use cases

[CrystalP]

I'm new to this and could be missing things (and MS docs don't explain very well how the pieces fit), but it looks like Media Foundation (MF) is a complete pipeline that doesn't give access to the intermediate steps. It can do everything from opening a URL to buffering, demuxing, decrypting, decoding, rendering audio & video, and it's also a player with play/pause/speed/stop type of controls. The caller has to provide a visual surface for the output, to be composited with Kodi UI.

OTOH MF also provides a lot of classes that suggest that maybe it's possible to isolate some pieces. It could be that forcing software rather than hardware handling offers more options (like frame-server mode), but I didn't find clear explanations and it looks like some testing would be needed.

[TheDaChicken]

MS docs don't explain very well how the pieces fit

I didn't find clear explanations

Welcome to hell :D I wish I could say I didn't feel at all the same.

also provides a lot of classes that suggest that maybe it's possible to isolate some pieces

From what it looks like (for me), it is possible but is a pain. I mean if it helps any better, you know that chromium figured it out and uses it as a video renderer. ¯\(ツ)/¯

Edge somehow does something else too? no clue what they use at all since closed source...

You can pass Samples using a custom IMFMediaStream that implements RequestSample function. That somehow leads to chromium's reading their custom demuxer ????. Since it allows more than one stream, it is contained in IMFMediaSource ... which is again passed through IMFMediaEngineExtension

I had to look at ffmpeg's source code for how to deal with directx11 since I don't know much before I even figured out "frame-server" could not be an option... IMFMediaEngine::TransferVideoFrame/IMFMediaEngineProtectedContent::TransferVideoFrame function fills a directx11 frame.

wtf I even found IMFMediaEngineEME ???? by looking at MF documentation while trying to write comment.
Did github search... couldn't find anyone who used it..

It looks like some testing would be needed.

Sadly will have to go through your testing. Could figure out more things than I have which would be great.

I don't know what protection looks like with Playready. I don't know if disabling hardware acceleration and somehow using software causes decrypting to not work. Funny that it would still have key access due to how this works.

also because has been implemented by old devs no longer here and our old ancient commits have no descriptions

During that time, I did a lot of lurking... so I know smol bit. That's really how Netflix previously worked on Windows for Widevine because it required decoding with the CDM (probably hardware decoding) and couldn't be just decrypted. I assume you already know a bit about that. I was originally thinking with "frame-server" to implement OpenVideoDecoder in decrypters if it would be possible to grab frames in software.