Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase the default value of MAXXGRID for serial fregid ? #260

Open
ngs333 opened this issue Dec 5, 2023 · 1 comment
Open

Increase the default value of MAXXGRID for serial fregid ? #260

ngs333 opened this issue Dec 5, 2023 · 1 comment
Milestone

Comments

@ngs333
Copy link
Contributor

ngs333 commented Dec 5, 2023

The current default value for MAXXGRID is 1e6. This precludes the running of serail fregrid under some circumstances (when a weights file is not already available) for the size of grids used today at GFDL. Since fregrid_parallel is available for extreme regridding,
in has in the past been the goto solution when regridding with large grids. But one problem with fregrid_parallel is that sometimes time or manpower is required to get it to run successfully, and the frequency of this is increasing as the lab moves to larger grids. A related problem is that this issue may be the only reason to move or copy some files from Analysis, workstations, or home computers onto Gaea visible filesystems.

This feature request is for MAXXGRID to be increased for fregrid, but not necessarily for fregrid_parallel or any of the other of the NCTools tools. MAXXGRID is set to a low 1e6 probably because fregrid_parallel is run on many-core supercomputer nodes,
and so the memory used per node may be large when the core count per node is high. But even if that is true, it probably can be increased somewhat for fregrid_parallel. In any case it can probably be increased for the fregrid tool by itself despite the common code.

The problems with this request are:
a) Short of a clever solution, fregrid may be required to be built by itself e.g. to allow for a re-definition of MAXXGRID.
b) Any changes require extensive testing, particularly If MAXXGRID is changed for fregrid_parallel.
c) If the new C++ optimal fregrid becomes part of the baseline, then the issue is mute.

@ngs333
Copy link
Contributor Author

ngs333 commented Dec 14, 2023

Note that if MAXXGRID is increased significantly for fregrid serial, then the run times may be exceedingly high and therefore end user time may be wasted. It may also be beneficial to estimate and display the expected run time to the user, and
possibly even recommend to the user to use fregrid_parallel (at least for weights generation). Note the information in #202 may be helpful for this.

@underwoo underwoo added this to the 2024.02 milestone May 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants