Test, debug, and better understand Safari limitations

[ryanrolds]

Investigate, correct, and document Safari limitations. After adding the Event Target shim in #27, I'm seeing a black video box if I open multiple tabs to http://localhost:3001/ in multiple tabs in Safari.

We need to better understand Safari's media device, Web RTC, and Web Audio limitations.

Once we a better understanding of the limits we can decide how much support we can offer at this time.

[tsmada]

@ryanrolds Are you seeing 2x black boxes on each localhost when observing this? I'm suspect of #27 causing this. Otherwise I may need to merge #27 in to get this to produce these black video boxes. I've noticed my MediaStreamTrack objects to be set to the muted state (video black/audio paused) since the beginning of the project.

This is due to the implementation of the MediaStream allows the operating system to control the state of the MediaStreamTracks. Unplugging mics etc cause the mute event to be fired. Safari 13 browser implements this by firing the mute Event when one tab starts a new MediaStream. Safari can also use the browser tab control (speaker icon/camera icon) to fire the unmute Event to the MediaStream. This unmute Event targets the original MediaStreamTrack and sets MediaStreamTrack.muted to false. Chrome allows you to send the same userDevice to multiple srcObject targets without controlling their mute state. From the w3c:

There are two dimensions related to the media flow for a "live" MediaStreamTrack : muted / not muted, and enabled / disabled. Muted refers to the input to the MediaStreamTrack. If live samples are not made available to the MediaStreamTrack it is muted. Muted is out of control for the application, but can be observed by the application by reading the muted attribute and listening to the associated events mute and unmute. There can be several reasons for a MediaStreamTrack to be muted: the user pushing a physical mute button on the microphone, the user toggling a control in the operating system, the user clicking a mute button in the browser chrome, the User Agent (on behalf of the user) mutes, etc.

Whenever the User Agent initiates such a change, it MUST queue a task, using the user interaction task source, to set a track's muted state to the state desired by the user.

To set a track's muted state to newState, the User Agent MUST run the following steps:

1. Let track be the MediaStreamTrack in question.
2. If track.muted is already newState, then abort these steps.
3. Set track.muted to newState.
4. If newState is true let eventName be mute, otherwise unmute.
5. Fire an event named eventName on track.

[ryanrolds]

My understanding is that without the Event Target changes in #27 that the current examples fail earlier. Please either wait until #27 is merged or check it out when evaluating Safari.

It sounds like some additional logic/controls needs to be added to allow us to test on Safari with multiple tabs or if it's not possible then we just need to understand it's not possible. Do the audio example work?

[tsmada]

It looks like the w3c allows the browsers to determine whether or not they will allow non-visible (background) tabs to continue broadcasting video/audio when a new tab opens. Take a look at the closed issue here on the w3c spec. It appears the consensus is that (sic) it's creepy to broadcast from background.

The audio static example suffers the same fate in Safari. Opening a new MediaStream while an existing MediaStream is active in the background will cause the browser to fire the mute event.

[ryanrolds]

Ok. So the method of opening multiple tabs won't work in Safari. We will need to open a tab in Chrome and one in Safari to test. What about the other examples, like audio, do those work in Safari?

[DoctorBud]

One thing that shows up in the various examples and in the React frontend is the issue of EventTarget, which is a class that is being extended in the code. However, Safari's security policy (or out of date implementation) prevents extension of this class. But there is an @ungap/event-target polyfill that handles this:

• https://github.com/ungap/event-target

In the case of plain HTML files, the best way to add this is via a script tag that is included before any reference to EventTarget. For example:

    <script src="https://unpkg.com/@ungap/event-target"></script>
    <script src="/js/localmedia.js"></script>
    <script src="/js/peering.js"></script>
    <script src="/js/signaling.js"></script>

In the case of preprocessed JS or ES6 Modules, then the polyfill should be import-ed before EventTarget is referenced (basically, the top of the file that extends EventTarget):

import EventTarget from '@ungap/event-target'

class SignalingServer extends EventTarget {
...

I'm mentioning this here mostly to be able to point to it so that we can fix up any existing places where this is still an issue.

BTW, the symptom that this is happening is usually a console error of the form:

TypeError: function is not a constructor (evaluating 'super()')