You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm donwloading wind forecast data from GFS in GRIB2 format. I'm only getting two variables (u and v 10m), and one GRIB file for each forecast time, which means I am downloading apprx. 150 files. Each file is apprx. 1.5 MB, so in total its around 200 MB.
I tried converting into a zarr store by reading each file into xarray and then write to the zarr store like this: ds.to_zarr('test.zarr', append_dim = 'time') , but noticed that the zarr store is 2+ GB on disk, meaning more than 10x size of original files. Am I doing something wrong here or are GRIB files just really well compressed? Is there anything I can do to reduce size?
For the record, this is the xarray representation of a single GRIB files:
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'm donwloading wind forecast data from GFS in GRIB2 format. I'm only getting two variables (u and v 10m), and one GRIB file for each forecast time, which means I am downloading apprx. 150 files. Each file is apprx. 1.5 MB, so in total its around 200 MB.
I tried converting into a zarr store by reading each file into xarray and then write to the zarr store like this:
ds.to_zarr('test.zarr', append_dim = 'time')
, but noticed that the zarr store is 2+ GB on disk, meaning more than 10x size of original files. Am I doing something wrong here or are GRIB files just really well compressed? Is there anything I can do to reduce size?For the record, this is the xarray representation of a single GRIB files:
And this is the xarray representation when loading the whole zarr store into xarray:
Beta Was this translation helpful? Give feedback.
All reactions