Forgotten set -e when run <my_function>

[andras-tim]

Describe the bug
The run changed resets the set -e that change the behaviors of the function.

To Reproduce

1. Here is an example script that should fail without print anything to console:

set -e

function main() {
    my_command
}

function my_command() {
    false # failing command
    echo 'FOOBAR'
}

if [ "${BASH_SOURCE[0]}" == "$0" ]; then
    main
fi

2. But this test will fail, because the set -e is not set as the source :

setup() {
    source my_script.sh
}

@test "test my_command" {
    run my_command
    [ "$output" != '' ]
}

Expected behavior
The sourced set flags have to be set under testing too.

Environment (please complete the following information):

• Bats version: 1.8.2
• OS: Linux (nixos)
• Bash version: 5.1.16

Additional context
I use run to have $status and $output.

I have read the https://bats-core.readthedocs.io/en/stable/writing-tests.html#when-not-to-use-run part of the documentation, that mentions the code will run with set -e in a subshell. This would be right for me, but my function works differently.

[martin-schulze-vireso]

I have read the https://bats-core.readthedocs.io/en/stable/writing-tests.html#when-not-to-use-run part of the documentation, that mentions the code will run with set -e in a subshell. This would be right for me, but my function works differently.

The documentation says that set -e is active outside of run.

The run function suppresses set -e internally. Probably for historic reasons. (I don't know why.)

I think we could allow run to not do this but changing the default behavior would likely break existing code.

[andras-tim]

I think we could allow run to not do this but changing the default behavior would likely break existing code.

Maybe you should add a command line flag, or introduce a new run command (e.g. check_run) to avoid breaking the existing codes.

[TheHillBright]

check_run is definitely needed. IIRC, I had to write helper function for now that set -e then call the actual function when I want to use run in situation like this.

[TheHillBright]

I tried to implement this but I quickly realised that set -e doesn't even work in the command executed by run. Take the original my_command as example. Even if we set -e inside my_command before the first statement false, the false will not trigger errexit. Any idea why this happens (note in run's definition, we set +E before executing the actual command under test)? IMO, this is a bug because run should allow testing under different shell options including errexit - run should not interfere with the shell options being set in the executed command. Besides errexit, other options like -x seem to work fine.

[x-yuri]

@martin-schulze-vireso Can you possibly tell where/how run suppresses set -e? A link to the code?

[martin-schulze-vireso]

Yes, here:

bats-core/lib/bats-core/test_functions.bash

Line 368 in e9fd17a

set +eET

You can provide your own wrapper function that would set -e again (warning, untested adhoc code):

with_set_e() {
    set -eE
    "$@"
}

@test "a test" {
    run with_set_e <your command>
}

However, be warned that this could interfere with run's internals.

[TheHillBright]

@x-yuri Does the code snippet above work for you? I think it does more or less the same as #815 and #748. It didn't work for me on bats c041f32 with bash 5.0 (see my previous comment in this issue above). @martin-schulze-vireso would you like to try it and see if we missed anything?

[martin-schulze-vireso]

Oh boy, that's what you get for just working off the top of your head. I just tried my own code and as you said it does not work.
Investigating further, I think I found the culprits:

bats-core/lib/bats-core/test_functions.bash

Line 378 in e9fd17a

)" && status=0 || status=$?

bats-core/lib/bats-core/test_functions.bash

Line 383 in e9fd17a

output="$("$pre_command" "$@")" && status=0 || status=$?

These boolean operations suppress the errexit. I think I put them there to prepare removing set +e from run but inadvertantly disabled set -e under run that way.

UPDATE
I thought this would be easy to fix but the problems are deeper and more subtle than expected. When restructuring the code like this

output=$(function to run)
status=$?

A test started to fail. I cannot really pinpoint it yet, but the set -x trace tells me, that our ERR trap gets executed inside the $() when it fails and it somehow seems to interfere with the external code. I don't have time to investigate further right now.