WIP: Additional Test Events for OneTimeSetUp / OneTimeTearDown #4652
Conversation
…or OneTimeSetUp / OneTimeTearDown.
Solution builds now with dotnet build command.
Event for OneTimeTearDown shall only be fired if there was a corresponding OneTimeSetUp. Additionally the events shall only be fired if there are corresponding methods in the SetUpTearDownItem objects.
…Down methods. Reused test fixtures from NUnit.TestData.OneTimeSetUpTearDownData
…e class and one in derived class).
… level (no inheritage). Test is failing since only one event for both events is raised. Tests is valid though.
In order to be non-breaking, changes done to the ITestListener interface have been reverted. Instead, a new interface ITestListenerExt has been intrtoduced. Also many changes done to implementations of ITestListener are reverted. This is the first part of changes done coming from review comments by Terje.
The changes were coming from previously trying out how to add new test cases. However the changes are not needed. Feedback from Terje was that MockAssembly should not be altered anyway. Therefore reverting.
…rDownItem. Events should be raised as close as possible to the method call. Thias was a feedback from Terje.
…w the test events arrive at a custom event listener that implements the ITestEventListener from NUnit.Engine.
| @@ -323,6 +323,10 @@ private TestExecutionContext CreateTestExecutionContext(ITest loadedTest, ITestL | |||
| // Set the listener - overriding runners may replace this | |||
| context.Listener = listener; | |||
|
|
|||
| // Set the listener for extended test events like OneTimeSetUp / OneTimeTearDown | |||
| if (listener is ITestListenerExt testListenerExt) | |||
There was a problem hiding this comment.
Mixing behaviour. Avoid this.
|
|
||
| private static void RaiseStartedEvent(TestExecutionContext context, MethodInfoCache.TestMethodMetadata methodInfo) | ||
| { | ||
| if (methodInfo.HasOneTimeSetUpAttribute) |
There was a problem hiding this comment.
Why do you check this? You should not be here if you were not in a Setup/Teardown method.
I assume you try to separate between pure OneTimeSetup/TD and the test Setup/TD. That is checked earlier. It might be necessary to state that information inside the SetUpTearDownItem. You find this in the TestContext that is passed along, there the test type is specified. If it is a test fixture it is onetimesetup, if it is a test it is a setup.
There was a problem hiding this comment.
Right. Updated in latest commit.
Changed to distinguish OneTimeSetUp and SetUp with the TestContext.
| else if (methodInfo.HasOneTimeTearDownAttribute) | ||
| context.ListenerExt.OneTimeTearDownStarted(context.CurrentTest); | ||
| } | ||
|
|
There was a problem hiding this comment.
Do you need this if inside here, you know when you call what you have, you can just pass that along, or even better drop this method all together and just call in in-place.
There was a problem hiding this comment.
Right. Updated in latest commit.
| private static void RaiseFinishedEvent(TestExecutionContext context, MethodInfoCache.TestMethodMetadata methodInfo) | ||
| { | ||
| if (methodInfo.HasOneTimeSetUpAttribute) | ||
| context.ListenerExt.OneTimeSetUpFinished(context.CurrentTest); |
There was a problem hiding this comment.
The ListenerExt may not exist, thus be null, and if it is null, it should not be called.
context.ListenerExt?.OneTimeSetUpFinished(context.CurrentTest);
There was a problem hiding this comment.
Right. Updated in latest commit.
Reverted original QueuingEventListener to be unchanged and added a separate QueuingEventListenerExt for custom events.
Used constrained model for collection asserts.
Differentiation between OneTimeSetUp and SetUp is now done with context.CurrentTest.IsSuite. That enabled the revert of all previous changes in MethodInfoCache. Added also a new test that checks that [SetUp] methods do not trigger OneTimeSetUp events.
|
Will close the PR since we will continue with another, more leightweight proposal in #4744 |
We implemented a first draft of a solution that provides additional test events for OneTimeSetUp and OneTimeTearDown.
The main uncertainty during the implementation was to find the right place where the events should be raised.
Please find more information in the corresponding issue.
This PR should serve more as a base for a technical discussion rather than being "production-ready".