Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce verbosity on skipped testcases' output #2809

Open
paulo-ferraz-oliveira opened this issue Jul 11, 2023 · 3 comments · May be fixed by #2821
Open

Reduce verbosity on skipped testcases' output #2809

paulo-ferraz-oliveira opened this issue Jul 11, 2023 · 3 comments · May be fixed by #2821

Comments

@paulo-ferraz-oliveira
Copy link
Contributor

paulo-ferraz-oliveira commented Jul 11, 2023 •
edited

Pre-Check

I didn't find anything related to this (potential) feature request in the issues or pull requests.
I am willing to implement a solution to this if it's decided to move forward with it.

Environment

$ rebar3 version
rebar 3.22.0 on Erlang/OTP 25 Erts 13.2.2.2

Current behaviour

At the moment, running rebar3 ct over a SUITE that has skipped testcases, will output something like

===> Running Common Test suites...
%%% test_SUITE: ....
%%% test_SUITE ==> test_skipped: SKIPPED
%%% test_SUITE ==> {tc_user_skip,my_skipped_reason}

Skipped 1 (1, 0) tests. Passed 18 tests.

which is readable but could be less verbose.

Expected behaviour

Allow for a flag to reduce verbosity to e.g.

===> Running Common Test suites...
%%% test_SUITE: ....S

with the appropriate skipped summary.

Note

In Elixir/Mix, for example, a skip is flagged as *. We could even use the same symbol.
@tsloughter
Copy link
Collaborator

A flag would be good. I personally like getting the output when the test fails. The default to skipped is debatable.

@paulo-ferraz-oliveira
Copy link
Contributor Author

paulo-ferraz-oliveira commented Jul 11, 2023 •
edited

Thanks, @tsloughter.

Yeah, I feel the failure should stay as-is too (even if behind a flag, I probably would keep using the default: otherwise we probably would have to come up with a summary of sorts).

On the other hand, yeah, skip is a consumer choice, so having it there while having made the choice is a bit noisy. Lemme know what you think the default should be.

I'll update the title and initial description accordingly...

@paulo-ferraz-oliveira paulo-ferraz-oliveira changed the title Reduce verbosity on skipped/failed testcases' output Reduce verbosity on skipped testcases' output Jul 11, 2023
@garazdawi garazdawi mentioned this issue Jul 12, 2023
Closed
@paulo-ferraz-oliveira
Copy link
Contributor Author

@garazdawi beat me to it, since I was going to mention the ongoing open pull request for Erlang/OTP. I'll see about implementing it here, with a verbose flag, and making the default the less verbose option implemented, but once it's done on Erlang/OTP it should be transparent to everybody.

Merged
@paulo-ferraz-oliveira paulo-ferraz-oliveira linked a pull request Feb 18, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow for less verbose test execution output on SKIPPED paulo-ferraz-oliveira/rebar3
2 participants
@tsloughter @paulo-ferraz-oliveira