Add flag to execute_steps to print sub-steps

[rittneje]

Currently, if a step implementation calls execute_steps, the resulting sub-steps are not printed, because it calls step.run with quiet=True. Even if that were fixed, it would need to turn off capturing (if it is on), or else those prints would be captured too.

Please add an optional parameter to execute_steps to allow sub-steps to be printed normally. (Or better yet, a context attribute so we don't have to ensure all test authors call it correctly.) Without this, it can be difficult to understand where exactly a test failed.

As a nice-to-have, sub-steps should be indented relative to their parent step when printed.

[jenisys]

Could you explain why you need such a functionality?

[rittneje]

As I mentioned, right now it is difficult to tell where exactly a scenario failed, because the nested sub-steps don't get printed. You also don't really get "progress updates" while the sub-steps are running.

Really I feel it makes little sense for the sub-steps to be treated differently at all.

[jenisys]

Context.execute_steps() provides a level of indirection. It is sometimes useful if you need to provide an abstraction over basic/technical steps. You should not overuse it. At call depth of more than 2 or 3 levels is a "smell" that your doing something wrong (you overuse: context.execute_steps()).

If you call a function, you also do not see what other functions are called inside the function body implementation.
If you need progress, one can provide log records. This can be done by your step implementation or by behave.

BUT: Your main pain point is, when an encapsulated step fails.
Have you looked at the stack-trace of the failing step ?

In my opinion, that should be sufficient to understand your own steps where the fault location is.

SEE ALSO:

• Issue exception of a failed test is not catched if context.execute_steps is used #981 (which has such an example of a stack-trace).

SIDE NOTE:
You can always provide your own formatter (and use it) that implements the specified behavior that you suggested.

[rittneje]

If you need progress, one can provide log records. This can be done by your step implementation or by behave.

No, this does not work because of output capture.

Have you looked at the stack-trace of the failing step ?

Yes, and it is of no use. First, because execute_steps mangles the exception message from the sub-step. And second, because this doesn't work for any steps that are common/generic. As an aside, the code has a comment here referencing #96, and what I am suggesting here would improve upon that.

You can always provide your own formatter (and use it) that implements the specified behavior that you suggested.

No, because you are passing quiet=True so the formatter is never called, hence this issue.

[jenisys]

Mmh, after reading this, I think your main problem is probably that all your step-functions have the name step_impl() ?!?
If that is correct, why do not give these functions real, readable names?
That is what I normally do for my step-functions (hint: step_impl() is the default, but you do not need to use it).
If you do that, you should get a more readable stack-trace.
But even without it, besides the function name, each stack frame contains the filename of the file where the step-function is located and the line number.

NOTE: ctx.execute_steps() does not change your stack-trace. It just uses the one that you provided as step implementer. It is just plain Python execution: your steps and the behave runtime functions needed to execute your steps (in between).

OTHERWISE:
I think about your other points.
But currently, you seem to want something that is normally not needed.

[rittneje]

No, all our step functions have different names.

[jenisys]

Out of curiosity, how deep is your callstack with ctx.execute_step(...), meaning:

• First level step uses ctx.execute_step(...)
• Called second level step uses ctx.execute_step(...)
• Called third level step uses ctx.execute_step(...)
• ...

[rittneje]

We don't normally call execute_steps recursively, if that's what you mean. But sometimes it is called in a loop.

[Anurag-Pola]

@rittneje Is there a manual way of doing this, I have a use case and need the same?

[Anurag-Pola]

@jenisys is there a flag added for the same?

[jenisys]

@Anurag-Pola
As you can see from this issue, there are no changes related to this issue. Therefore, there is also no option or userdata flag that exists to enable the requested functionality.

[Anurag-Pola]

@jenisys Thank You for your quick reply, is there a possibility to tweak the configuration on my side to do that?

[jenisys]

@Anurag-Pola
You would need to patch the implementation first (before you can make it configurable).

[Feko-Karels]

@jenisys I would also love to have that feature. My use case is the following:

I have two file with inputs I want to test. For every input of the first file I want to test all inputs of the second file. Lets say the first file has x inputs and the second one y. For every input from for first file i need a expensive setup. That means Scenario Outline does not work since it would create x* y Scenarios, and the expensive setup would run x*y times.
By printing the substeps I could use scenario outline for the first file and context.execute_steps for the second one. That way it would create x Scenarios and the setup is only needed x times. Inside every Scenario would be y steps then.

Using the before_feature or background does not work either, because every input from file1 requires a different setup.