GitHub actions fails intermittently due to an issue with the cache

[nwagenbrenner]

Since we added the caching feature, our tests sometimes fail and it appears to be an issue with the cache. We need to look closer at how the caching works.

[latwood]

Took a stab at trying to see what was going on, as my last push failed to build 3 times retrying the build, while another push ran fine, 30 min or so from each other.

The closest thing that I can see is messages claiming that grib2 is not a supported file format, right before the errors. This implies gdal somehow was not quite installed right. But looking at the logs, and reviewing the process, I cannot find what could be getting messed up. The only thing that I can think of is if one or more of the paths for the sudo make install step, from the configure --prefix=/usr/local, is somehow getting confused or messed up between builds, but I am not finding anything that can easily point to something like that happening. I did think about maybe just running configure again in the build from cache step, to see if it would enhance stability, but that doesn't seem to quite make sense since configure is usually run BEFORE make and maybe it could throw off the build from the make step to squeeze it in there an additional time after make but before sudo make install. And the whole point of cacheing in the first place is to avoid dealing with the make step additional times.

But yeah, either some random path is somehow not holding true between builds, even builds of 30 min apart, which doesn't make sense to me. Or maybe is it possible that the test is just giving this message for a file not existing that it is trying to read with that format, so a bad test message for a failed download, for just files of that type? Only way I can think of to test that without adding grib2 files to the repo is to maybe cache one of the working last time downloaded grib2 files, and then just try opening it on the next run as an additional test.

Anyhow, it sounds like the github actions cacheing is getting quite close to out of date anyways, sounds like actions/checkout@v3 and actions/cache@v2 need to be updated to later versions, something about moving from actions that use older nodes (12, 16) to actions that use Node.js 20. For all we know, migrating to newer versions of these actions may improve the stability and help get rid of this problem, or may make it more of a pain. Either way, newer versions may have some changes to the commands they use, means studying up the methods for these actions somewhat from scratch again.

Sounds like actions/cache@v2 is now v4, the latest version is v4. In the comments for the v3 upgrade, it mentions a fix that may be related to our troubles: "Fixed download issue for files > 2GB during restore." The gdal-3.2.1 folder we are cacheing IS 3.3 GB on my home machine, so is it possible that the cacheing was maybe just dropping a few files here or there for this folder before? Though then why no error messages during the build?
https://github.com/actions/cache

Looks like actions/cache@v3 is now v4. I haven't studied that page and corresponding changes much yet.
https://github.com/actions/checkout

[nwagenbrenner]

Appears to be fixed with 4b85a34.

[nwagenbrenner]

I think there may still be an issue here. See the log files from a recent failure for some details. There are warnings about deprecated actions. https://github.com/firelab/windninja/actions/runs/9507098420