Opaque logging when "make check" tests fail

[liquidaty]

Describe the bug
Test failure provides insufficient logging. Any suggestion as to how the Makefile or other scripts could be modified to produce useful logging would be much appreciated.

This issue is related to #1846, but this issue is asking for better logging, and using the windows tests as an example for reproducing broken tests, whereas #1846 is asking for a solution to the broken tests.

To Reproduce
Build via MSYS-2 using the below build yml. Below is the log, which is not very useful.

make  check-TESTS
make[3]: Entering directory ...
make[4]: Entering directory ...
FAIL: tests/jqtest
FAIL: tests/mantest
FAIL: tests/optionaltest
PASS: tests/onigtest
PASS: tests/base64test
PASS: tests/utf8test
FAIL: tests/shtest

Expected behavior
Test failure should provide some useful output, at least for the first failure, if not for all fails. At the least:

• what command was run, along with any related inputs / parameters
• what the expected output was
• it would also be nice, though not 100% necessary, to include what the actual output was, that differed from the expected output

It's also not clear whether NO_VALGRIND in the Makefile is doing anything at all.

Environment (please complete the following information):

• Windows msys-- see below yml
• latest jq repo clone

Additional context

see below YML (modified from the the non-working one that is in the repo):

name: Windows Build

on:
  push:
    branches:
    - 'main'
    - 'windows-build'
    paths:
    - '**.[chly]'
    - '**/Makefile.am'
    - '.github/workflows/windows.yml'

  pull_request:
    paths:
    - '**.[chly]'
    - '**/Makefile.am'
    - '.github/workflows/windows.yml'

jobs:
  windows:
    runs-on: windows-latest
    env:
      APPVER: '10.0'
      CODESIGN_PKT: 0000000000000000
      INSTALL_DIR: C:\jq
      WINSDKVER: '10.0.22000.0'
      WIXDIR: 'c:\Program Files (x86)\Windows Installer XML v3.5'
    steps:
    - name: Clone repository
      uses: actions/checkout@v1
    - name: Find MSVC and run vcvarsall.bat
      uses: ilammy/msvc-dev-cmd@v1
      with:
        arch: amd64
    - uses: msys2/setup-msys2@v2
      with:
        update: true
        install: >-
          base-devel
          git
          mingw-w64-x86_64-toolchain

    - name: Install dependencies
      shell: msys2 {0}
      run: |
        pacman --noconfirm -S autoconf automake libtool
        pacman --noconfirm -S flex bison

    - name: Build and Test
      shell: msys2 {0}
      run: |
        git submodule update --init --recursive
        mkdir %INSTALL_DIR%
        export PATH=$PATH:/c/msys64/usr/bin:/c/cygwinbin
        export PATH=$PATH:/c/tools/cygwin/bin
        echo PATH=$PATH

        # Make
        autoreconf -fi
        ./configure --with-oniguruma=builtin --disable-shared --enable-static --enable-all-static
        make
        # make SUBDIRS= "TESTS=tests/mantest tests/jqtest tests/onigtest tests/base64test" check # can't get these to pass with latest repo on osx
        # Valgrind doesn't exist on Windows
        make -j4 NO_VALGRIND=1 check

[liquidaty]

also related: #2550

[wader]

Sorry i missed that you had already worked on a windows build workflow. But i noticed also that logging on failure would be very nice, i did .. || (cat tests/*.log; exit 1) when i was iterating on it.