HIP and MPI+HIP updates

[ohearnk]

NOTE: do not merge this until the issues below are resolved.

This MR ports the updated CUDA and MPI+CUDA codes (including recently merged f-function optimizations) to HIP and MPI+HIP versions, respectively. Also, this MR begins to unify the CUDA and HIP versions to simpily future GPU code maintainence.

Closes #344.

[ohearnk]

As noted above, all tests (full test suite) are passing for the CUDA and MPI+CUDA (1 GPU) versions. However, some tests are failing for the HIP and MPI+HIP versions. See the logs below from tests on the MI210s on the AMD AAC. Interestingly, the test failures are slightly different between the HIP and MPI+HIP versions.

This is a bit difficult to debug at least when comparing against the working HIP / MPI+HIP versions from the 23.08b release as there are also a number of test failures there. It may be better to pick a commit before the f-function optimizations and run tests there for comparison.

Test configuration on the AMD AAC:

• RHEL9 partition (1CN128C8G2H_2IB_MI210_RHEL9)
• ROCm v5.7.1, UCX v1.15.0, OpenMPI v4.1.6, GCC v11.3.1 (gfortran)
• f-function support disabled
• CMake configuration (HIP version):

cmake .. -DCOMPILER=MANUAL -DCMAKE_C_COMPILER=hipcc -DCMAKE_CXX_COMPILER=hipcc -DCMAKE_Fortran_COMPILER=gfortran -DMPI= -DHIP=TRUE -DQUICK_USER_ARCH=gfx90a -DENABLEF= -DCMAKE_INSTALL_PREFIX=${PWD}/../install_rhel9_hip_gfx90a_rocm5.7.1_ucx1.15.0_ompi4.1.6 -DHIP_TOOLKIT_ROOT_DIR=/shared/apps/rhel9/opt/rocm-5.7.1

HIP test summary and diffs:
runtest_hip.log
hip_test_diffs.log

MPI+HIP test summary and diffs:
runtest_mpi_hip_1gpu.log
mpi_hip_test_diffs.log