Advice for reducing test binary size

[troibe]

We are considering committing the generated testcases into our repository as the generation and compilation takes quite some time.
Currently the 256bit test cases for 64bit and 32bit are around 2G each.
Is this expected?
Are there any special flags enabled that especially increase the binary size?
Do you have any suggestions with regards to reducing binary size?

[jerryz123]

My stage2 Zvl256b_Zve64d test cases are only 56 M. I'm guessing its because I am running in "cosim" mode.

This is my build script: https://github.com/ucb-bar/saturn-vectors/blob/master/build-tests.sh

This script takes 5 minutes on a 72 thread machine.

[jerryz123]

Cosim mode assumes you have lockstep cosimulation of your DUT, so the binary no longer needs to contain the reference results.

[troibe]

Thanks for the data points that sounds really promising.
I guess one reason I was considering committing the binaries was getting around the Spike/Go installs.
So with cosim and the required Spike install I'm not sure how much I'd be saving in terms of compute time/space.

I'll definitely try out the cosim mode though for some experiments.
Is cosim mode more compute intensive when running the tests themselves?

[ksco]

(Sorry for the late reply, I missed this one)

The binary size is mainly contributed by the self-checking stuff, which contains scalar-checking code and a random data section in every elf file, this makes the tests self-contained and easy to use but sacrifices the binary size.

The "cosim" mode removes all the self-checking and relies on the user to check via a Co-simulator, so it can be much smaller in size.

as the generation and compilation takes quite some time

Did you integrate this in the CI? If so, might be a good idea to cache the build outputs.