obs-virtualcam-module is unloaded prematurely

[Pehrsons]

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

30.1.2

OBS Studio Version (Other)

No response

OBS Studio Log URL

n/a

OBS Studio Crash Log URL

No response

Expected Behavior

obs-virtualcam-module is not unloaded until all managed objects have been released

Current Behavior

obs-virtualcam-module is unloaded while an outstanding IBaseFilter instance with a non-zero refcount exists

Steps to Reproduce

Firefox dev here. This report originates in Firefox bug 1707060.

In Firefox crash-stats there is a crash pattern where in our DirectShow camera capture backend we release an IBaseFilter instance which was managed by a DLL that has been unloaded. Looking at the unloaded DLL, the largest volume consists of obs-virtualcam-module64.dll.

People are understandably frustrated with Firefox that it crashes again and again, but we may not be to blame.

Looking at DirectShow docs on DllCanUnloadNow I see that:

DllCanUnloadNow should return S_FALSE if there are any existing references to objects that the DLL manages.

Looking at the implementation of DllCanUnloadNow in obs-virtualcam-module it does

STDAPI DllCanUnloadNow()
{
	return os_atomic_load_long(&locks) ? S_FALSE : S_OK;
}

and locks is only mutated here:

STDMETHODIMP VCamFactory::LockServer(BOOL lock)
{
	if (lock) {
		os_atomic_inc_long(&locks);
	} else {
		os_atomic_dec_long(&locks);
	}

	return S_OK;
}

LockServer is a method on IClassFactory whose docs do say under Notes to Implementers:

If the lock count is zero, there are no more objects in use, and the application is not under user control, the server can be closed.

but also

When the lock count and the class object reference count are both zero, the class object can be freed.

Under Notes to Callers they say:

Most clients do not need to call this method. It is provided only for those clients that require special performance in creating multiple instances of their objects.

I do think you need to make DllCanUnloadNow aware of managed object refcounts, or at least the number of managed objects alive.

Anything else we should know?

No response