Merging target branch for group of PRs lead to different outcomes (retargetted vs closed)

[JanKrivanek]

Select Topic Area

Bug

Body

Situation

• PR: Analyzers prototype proposal dotnet/msbuild#9735 for 'exp/build-analyzers' -> 'main'
• Couple dependant PRs targetting the 'exp/build-analyzers':
  • [BuildCheck] Editor config support dotnet/msbuild#9811
  • Analyzers acquisition experience dotnet/msbuild#9977
  • Instrumentation module for BuildCheck dotnet/msbuild#9890
  • BuildCheck does not run on restore dotnet/msbuild#9907

After merging the inital PR, only the first dependant PR got auto-retargetted, all other were auto closed. To make it worse - the closed PRs cannot be reopened.

Expectation

Based on https://github.blog/changelog/2020-05-19-pull-request-retargeting/ - all PRs targetting branch B that gets merged&deleted as part of other PR merge should be auto retargeted.