Trying to just send an RTP message from libdatachannel to libdatachannel, why won't it work?

[wusspuss]

I've been successfully using libdatachannel to send and receive RTP with another piece of software, but then I realized I'm unable to receive frames to libdatachannel from libdatachannel. I made a mock up to reproduce the problem locally, in Python for brevity.
Verbose libdatachannel log
What am I doing wrong? I'm probably just stupidly missing some small detail but I've been stuck on this for days now...
What I would expect with the following code is that "message received" be printed, or some error be produced, but message_callback is simply never called. And it's not a bindings issue - I have successfully re-implemented the media_receiver and media_sender examples with these bindings.

import libdatachannel
libdatachannel.init_logger(libdatachannel.LogLevel.LOG_ERROR)

sender=libdatachannel.PeerConnection()
receiver=libdatachannel.PeerConnection()

track = sender.add_track(
    b"\n".join(
        [b"audio 9 UDP/TLS/RTP/SAVPF 0",
         b"c=IN IP4 0.0.0.0",
         b"a=rtpmap:0 PCMU/8000",
         b"a=mid:audio",
         b"a=sendrecv",
         b"a=rtcp:9 IN IP4 0.0.0.0",
        ]
    )
)

def on_sender_gathering_state_change(state):
    if state == libdatachannel.GatheringState.GATHERING_COMPLETE:
        def on_receiver_gathering_state_change(state):
            if state == libdatachannel.GatheringState.GATHERING_COMPLETE:
                sender.set_remote_description(receiver.local_description.encode(), b'answer')
        def on_b_track(track):
            track.message_callback=lambda x: print("message received", x)
            
        receiver.gathering_state_change_callback = on_receiver_gathering_state_change
        receiver.track_callback=on_b_track
        receiver.set_remote_description(sender.local_description.encode(), b'offer')

        def on_sender_track_open():
            example_rtp_packet = b"\x80\x80\n\xa8\xbd.\x0b\x1d/(\xfe\x9cbY[TV_`ZWWZ[[\\YXXW]_^do\xf8\xf7mge]^biknkfg_\\Z^bWOONLLKHHHHIJNQNMLLOPRTWTOSZagillf__]["
            track.send_message(example_rtp_packet, len(example_rtp_packet))
            print("message sent")
        track.open_callback=on_sender_track_open
        

sender.gathering_state_change_callback = on_sender_gathering_state_change
sender.set_local_description()
input()

[paullouisageneau]

You have to set the SSRC in the track description, otherwise RTP packets have unknown SSRC and are ignored as a result. This would definitely require some logging as the silent failure can be confusing.

[wusspuss]

Yes, that's why we should check that the track is indeed the only one before forwarding a packet to it. Wouldn't something like this #1023 make sense? (I'm not familiar with C++ so I may be doing something incredibly stupid there)

[paullouisageneau]

It works for a single track but it's merely a quick hack to fix a specific use case. For instance, other tracks may be added later in the session which may seemingly break the first track. Or developers will report that video stops working as soon as they add audio. The proper fix is to implement SDES headers like the specification says.

[wusspuss]

@paullouisageneau do I understand correctly that an SDP with multiple media but no SSRCSs like the one below would flat out not be supported by libdatachannel? Is there currently any hack around this? Maybe with two PeerConnections somehow?

v=0
o=- 8770656990916039506 2 IN IP4 127.0.0.1
s=-
t=0 0
a=msid-semantic: WMS my-media-stream
m=audio 9 UDP/TLS/RTP/SAVPF 111 112 113 114 9 0 8
c=IN IP4 0.0.0.0
a=ice-ufrag:sG3i
a=ice-pwd:nExSgJUZ7JWYOBPG9IHLf9
a=ice-options:trickle
a=fingerprint:sha-256 01:3C:AB:A0:D1:67:F2:1D:E4:52:AF:E6:6E:15:9A:0A:62:58:18:91:04:B4:35:D9:78:B9:82:DF:CC:BD:F5:A7
a=setup:actpass
a=rtpmap:111 opus/48000/2
a=rtpmap:112 speex/32000
a=rtpmap:113 speex/16000
a=rtpmap:114 speex/8000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=fmtp:111 useinbandfec=1
a=mid:audio
a=sendrecv
a=rtcp-mux
a=rtcp:9 IN IP4 0.0.0.0
m=video 9 UDP/TLS/RTP/SAVPF 98
c=IN IP4 0.0.0.0
a=ice-ufrag:27wi
a=ice-pwd:zLztPiGMCOTfqXtGPY7wVl
a=ice-options:trickle
a=fingerprint:sha-256 01:3C:AB:A0:D1:67:F2:1D:E4:52:AF:E6:6E:15:9A:0A:62:58:18:91:04:B4:35:D9:78:B9:82:DF:CC:BD:F5:A7
a=setup:actpass
a=rtpmap:98 VP8/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=extmap:1 urn:3gpp:video-orientation
a=mid:video
a=sendrecv
a=rtcp-mux
a=rtcp:9 IN IP4 0.0.0.0

[paullouisageneau]

There is currently no way to receive media in tracks in that case (sending would still work), there is no hack around it. Such a use case requires SDES headers support #1020, provided the remote implementation supports it.

Of course you can always receive all media packets by setting a media handler on the peer connection itself to intercept everything, but streams won't be sorted out and tracks won't receive anything.

[paullouisageneau]

It works for a single track but it's merely a quick hack to fix a specific use case. For instance, other tracks may be added later in the session which may seemingly break the first track. Or developers will report that video stops working as soon as they add audio. The proper fix is to implement SDES headers like the specification says.

As expected, they do, for instance #1209