Reduce backoff spinning/usage

[ibraheemdev]

This PR is a first step to reducing a lot of the spinning-related issues that have cropped up. It does two things:

• Remove optimistic spinning completely from Context::wait_until. There are already spin loops in recv/send, so another set of spins after entering the park queue is excessive.
• Reduce the amount of spinning in Backoff::snooze. As seen in crossbeam-channel uses pure userspace spinlocks, which perform very poorly in some situations #821 (comment), the amount of PAUSE instructions issued is quite high. This change reduces it from 6 to 3, similar to parking-lot. It also reduces the amount of total amount of spinning, reducing YIELD_LIMIT to 6. This differs from parking-lot as it is common for a channel, unlike a mutex, to remain empty/full for long periods of times, and so if yielding doesn't allow progress after a few times it is likely better to park properly.

[ibraheemdev]

I'll try to post some relevant benchmark/profiling results to see how much this helps.

[taiki-e]

Thanks! As for SPIN_LIMIT, I remember that 4 looked good when I looked into it before. taiki-e@0362ffe

That said, either is fine if performance is not that different.

[codehobbyist06]

Hi @ibraheemdev , any update on this PR? When are the chances that this PR could be merged?

[taiki-e]

Thanks!

[ibraheemdev]

I ran the benchmarks locally and there are some large performance regressions, especially on the spsc benchmarks. However, I'm not we should be optimizing for those benchmarks as opposed to the real world cases where excessive spinning is problematic. That said I'd like to experiment a bit more, maybe just removing the double use of Backoff and the yield limit is enough?

[taiki-e]

The removal of double use of Backoff is definitely a non-controversial change. I think the others probably have different optimal solutions depending on the nature of the benchmark and the platform. I think splitting this PR and merging less controversial changes first is one of the reasonable solutions to progress this.