SeqManager generate discontinue sequence number in h264 svc mode

[saufwy]

Environment :

L1T3 h264 svc mode
I frame with sequence range [11, 15], TemporalLayer : 0, SpatialLayer : 0
P1 frame with sequence range [16, 20], TemporalLayer : 2, SpatialLayer : 0
P2 frame with sequence range[21, 25], TemporalLayer : 1, SpatialLayer : 0

SVC consumer TargetTemporalLayer: 1, TargetSpatialLayer 0

Issue description

svc consumer receive video rtp packet as follow:

seqManager.Sync(11 -1); // first packet is keyframe
rtp packet 11 -----------> seqManager output seq 1
rtp packet 12 ------------> seqManager output seq 2
rtp packet 13 ------------> seqManager output seq 3
rtp packet 14 ------------> seqManager output seq 4
rtp packet 15 ------------> seqManager output seq 5

rtp packet 16 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 17 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 18 ------------> The packet is lost, server did not receive it
rtp packet 19 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 20 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer

rtp packet 21 ------------> seqManager output seq 7, but expected value should be 6

There is a sequence gap when server deliver I frame(with seq [15,20]) and P2 frame (with seq [21, 25]) to consumer.

[jmillan]

Hi,

I don't see any proper solution here since the 6 sequence number should be assigned to the input packet 18 when it arrives.

If we assigned the 6 value to the input 21 and later arrived the 18, then we would have an inconsistence and we would be forced to drop it.

[saufwy]

Hi，
consumer TargetTemporalLayer is 1, packet 18 is part of P1 frame，P1 frame with TargetTemporalLayer 2 should be all droped.

we should only deliver I frame(with sequence [11, 15]) and P2 frame(with sequence [21, 25]) to consumer . The situation above (packet 18 is lost) generate discontinue sequence when we deliver I farme(with sequence [11, 15]) and P2 frame(with sequence [21, 25]) to consumer. webrtc client can't handle correctly discontinue sequence in h264.

Please bear with me, I'm not a native English speaker, so I hope i have made it clear.

[jmillan]

I understand the problem.

It is only the codec H264_SVC.cpp the one that knows about it. But the codec implementation knows nothing about RTP (and thus RTP seq number). Meaning we have no current way to know if a previous non received packet should also be dropped in the SeqManager.

I'm open to proposals.

[notorca]

In the example provided:

rtp packet 16 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 17 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 18 ------------> The packet is lost, server did not receive it
rtp packet 19 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer
rtp packet 20 ------------> The packet was droped , TemporalLayer > TargetTemporalLayer

Rtp packets 16-20 belongs to the same frame so they must have same RTP timestamp. That makes possible to deduce that packet 18 also belongs to the same frame and can be dropped.