Skip to content
This repository has been archived by the owner on Apr 30, 2021. It is now read-only.

esmlab.resample with freq='mon' yields unexpected results when called with monthly data #135

Open
andersy005 opened this issue May 14, 2019 · 2 comments
Open

esmlab.resample with freq='mon' yields unexpected results when called with monthly data #135

andersy005 opened this issue May 14, 2019 · 2 comments
Labels
bug Something isn't working

Comments

@andersy005
Copy link
Contributor

andersy005 commented May 14, 2019 •
edited
Loading

Calling:

esmlab.resample(dset(), freq='mon')

where dset() is already monthly resolution

Expected:To do nothing. However, it returns a dataset with mucked up time axis and 12 not 24 values!

Minimal Example:

In [1]: import esmlab

In [2]: ds = esmlab.datasets.open_dataset(
'cmip6_fgco2_Omon_CESM2_1pctCO2_r1i1p1f1_gn_000101-005012')

In [3]: ds
Out[3]: 
<xarray.Dataset>
Dimensions:    (d2: 2, nlat: 384, nlon: 320, time: 24, vertices: 4)
Coordinates:
    lat        (nlat, nlon) float64 ...
    lon        (nlat, nlon) float64 ...
  * nlat       (nlat) int32 1 2 3 4 5 6 7 8 ... 377 378 379 380 381 382 383 384
  * nlon       (nlon) int32 1 2 3 4 5 6 7 8 ... 313 314 315 316 317 318 319 320
  * time       (time) object 0001-01-15 12:45:00 ... 0002-12-15 12:00:00
Dimensions without coordinates: d2, vertices
Data variables:
    fgco2      (time, nlat, nlon) float32 ...
    time_bnds  (time, d2) object ...
    lat_bnds   (nlat, nlon, vertices) float32 ...
    lon_bnds   (nlat, nlon, vertices) float32 ...
Attributes:
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   0.0
    branch_time_in_parent:  182500.0
    case_id:                58
    cesm_casename:          b.e21.B1PCT.f09_g17.CMIP6-1pctCO2.001
    contact:                cesm_cmip6@ucar.edu
    creation_date:          2019-03-09T17:23:49Z
    data_specs_version:     01.00.29
    experiment:             1 percent per year increase in CO2
    experiment_id:          1pctCO2
    external_variables:     areacello
    forcing_index:          1
    frequency:              mon
    further_info_url:       https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.1...
    grid:                   native gx1v7 displaced pole grid (384x320 latxlon)
    grid_label:             gn
    initialization_index:   1
    institution:            National Center for Atmospheric Research, Climate...
    institution_id:         NCAR
    license:                CMIP6 model data produced by <The National Center...
    mip_era:                CMIP6
    model_doi_url:          https://doi.org/10.5065/D67H1H0V
    nominal_resolution:     100 km
    parent_activity_id:     CMIP
    parent_experiment_id:   piControl
    parent_mip_era:         CMIP6
    parent_source_id:       CESM2
    parent_time_units:      days since 0001-01-01 00:00:00
    parent_variant_label:   r1i1p1f1
    physics_index:          1
    product:                model-output
    realization_index:      1
    realm:                  ocnBgchem
    source:                 CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite v...
    source_id:              CESM2
    source_type:            AOGCM BGC AER
    sub_experiment:         none
    sub_experiment_id:      none
    table_id:               Omon
    tracking_id:            hdl:21.14100/1ccc0d3e-97b1-4392-b607-e037670617ef
    variable_id:            fgco2
    variant_info:           CMIP6 CESM2 1pctCO2 experiment with interactive a...
    variant_label:          r1i1p1f1

In [4]: ds_mon = esmlab.resample(ds, freq='mon')
/Users/abanihi/opt/miniconda3/envs/devel/lib/python3.6/site-packages/xarray/core/nanops.py:159: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

In [5]: ds_mon
Out[5]: 
<xarray.Dataset>
Dimensions:    (d2: 2, nlat: 384, nlon: 320, time: 12, vertices: 4)
Coordinates:
  * nlat       (nlat) int64 1 2 3 4 5 6 7 8 ... 377 378 379 380 381 382 383 384
  * nlon       (nlon) int64 1 2 3 4 5 6 7 8 ... 313 314 315 316 317 318 319 320
  * time       (time) object 0001-07-18 00:22:30 ... 0002-06-17 00:00:00
    lat        (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon        (nlat, nlon) float64 320.6 321.7 322.8 ... 318.9 319.4 319.8
Dimensions without coordinates: d2, vertices
Data variables:
    fgco2      (time, nlat, nlon) float32 nan nan nan nan ... nan nan nan nan
    time_bnds  (d2, time) object 0001-01-01 01:30:00 ... 0003-01-01 00:00:00
    lat_bnds   (nlat, nlon, vertices) float32 -79.48714 -79.48714 ... 72.41355
    lon_bnds   (nlat, nlon, vertices) float32 320.0 321.125 ... 320.0 319.586
Attributes:
    history:  \n2019-06-09 10:51:42.697515 esmlab.resample(<DATASET>, freq="m...
@andersy005
Copy link
Contributor Author

andersy005 commented Jun 9, 2019 •
edited
Loading

@matt-long

Incidentally, calling esmlab.resample with freq='ann' yields unexpected results when called with yearly data as well:

In [6]: ds = esmlab.datasets.open_dataset(
'cmip6_fgco2_Omon_CESM2_1pctCO2_r1i1p1f1_gn_000101-005012',
decode_times=True).resample({'time': '1Y'}).nearest()

In [7]: ds
Out[7]: 
<xarray.Dataset>
Dimensions:    (d2: 2, nlat: 384, nlon: 320, time: 2, vertices: 4)
Coordinates:
  * time       (time) object 0001-12-31 00:00:00 0002-12-31 00:00:00
    lat        (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon        (nlat, nlon) float64 320.6 321.7 322.8 ... 318.9 319.4 319.8
  * nlat       (nlat) int32 1 2 3 4 5 6 7 8 ... 377 378 379 380 381 382 383 384
  * nlon       (nlon) int32 1 2 3 4 5 6 7 8 ... 313 314 315 316 317 318 319 320
Dimensions without coordinates: d2, vertices
Data variables:
    fgco2      (time, nlat, nlon) float32 ...
    time_bnds  (time, d2) object ...
    lat_bnds   (nlat, nlon, vertices) float32 -79.48714 -79.48714 ... 72.41355
    lon_bnds   (nlat, nlon, vertices) float32 320.0 321.125 ... 320.0 319.586
Attributes:
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   0.0
    branch_time_in_parent:  182500.0
    case_id:                58
    cesm_casename:          b.e21.B1PCT.f09_g17.CMIP6-1pctCO2.001
    contact:                cesm_cmip6@ucar.edu
    creation_date:          2019-03-09T17:23:49Z
    data_specs_version:     01.00.29
    experiment:             1 percent per year increase in CO2
    experiment_id:          1pctCO2
    external_variables:     areacello
    forcing_index:          1
    frequency:              mon
    further_info_url:       https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.1...
    grid:                   native gx1v7 displaced pole grid (384x320 latxlon)
    grid_label:             gn
    initialization_index:   1
    institution:            National Center for Atmospheric Research, Climate...
    institution_id:         NCAR
    license:                CMIP6 model data produced by <The National Center...
    mip_era:                CMIP6
    model_doi_url:          https://doi.org/10.5065/D67H1H0V
    nominal_resolution:     100 km
    parent_activity_id:     CMIP
    parent_experiment_id:   piControl
    parent_mip_era:         CMIP6
    parent_source_id:       CESM2
    parent_time_units:      days since 0001-01-01 00:00:00
    parent_variant_label:   r1i1p1f1
    physics_index:          1
    product:                model-output
    realization_index:      1
    realm:                  ocnBgchem
    source:                 CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite v...
    source_id:              CESM2
    source_type:            AOGCM BGC AER
    sub_experiment:         none
    sub_experiment_id:      none
    table_id:               Omon
    tracking_id:            hdl:21.14100/1ccc0d3e-97b1-4392-b607-e037670617ef
    variable_id:            fgco2
    variant_info:           CMIP6 CESM2 1pctCO2 experiment with interactive a...
    variant_label:          r1i1p1f1

In [8]: esmlab.resample(ds, freq='ann')
/Users/abanihi/opt/miniconda3/envs/devel/lib/python3.6/site-packages/xarray/core/nanops.py:159: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)
Out[8]: 
<xarray.Dataset>
Dimensions:    (d2: 2, nlat: 384, nlon: 320, time: 1, vertices: 4)
Coordinates:
  * nlat       (nlat) int64 1 2 3 4 5 6 7 8 ... 377 378 379 380 381 382 383 384
  * nlon       (nlon) int64 1 2 3 4 5 6 7 8 ... 313 314 315 316 317 318 319 320
  * time       (time) object 0002-07-02 12:00:00
    lat        (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon        (nlat, nlon) float64 320.6 321.7 322.8 ... 318.9 319.4 319.8
Dimensions without coordinates: d2, vertices
Data variables:
    fgco2      (time, nlat, nlon) float64 nan nan nan nan ... nan nan nan nan
    time_bnds  (d2, time) object 0002-01-01 00:00:00 0003-01-01 00:00:00
    lat_bnds   (nlat, nlon, vertices) float32 -79.48714 -79.48714 ... 72.41355
    lon_bnds   (nlat, nlon, vertices) float32 320.0 321.125 ... 320.0 319.586
Attributes:
    history:  \n2019-06-09 11:31:50.773175 esmlab.resample(<DATASET>, freq="a...

@andersy005
Copy link
Contributor Author

andersy005 commented Jun 9, 2019 •
edited
Loading

@matt-long, I am currently trying to fix this issue and I am wondering whether we didn't complicate our current implementations for esmlab.resample(ds, freq='ann') and esmlab.resample(ds,freq='mon').

Could you help me understand whether I am missing anything by assuming that the following (with additional steps to handle weights, etc..) would fix our issues?

Let's say you have the following dataset:

In [14]: ds = esmlab.datasets.open_dataset(
         'cmip6_fgco2_Omon_CESM2_1pctCO2_r1i1p1f1_gn_000101-005012', 
           decode_times=True).resample({'time': '7D'}).nearest()
 

In [15]: ds
Out[15]: 
<xarray.Dataset>
Dimensions:    (d2: 2, nlat: 384, nlon: 320, time: 100, vertices: 4)
Coordinates:
  * time       (time) object 0001-01-15 00:00:00 ... 0002-12-09 00:00:00
    lat        (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon        (nlat, nlon) float64 320.6 321.7 322.8 ... 318.9 319.4 319.8
  * nlat       (nlat) int32 1 2 3 4 5 6 7 8 ... 377 378 379 380 381 382 383 384
  * nlon       (nlon) int32 1 2 3 4 5 6 7 8 ... 313 314 315 316 317 318 319 320
Dimensions without coordinates: d2, vertices
Data variables:
    fgco2      (time, nlat, nlon) float32 ...
    time_bnds  (time, d2) object ...
    lat_bnds   (nlat, nlon, vertices) float32 -79.48714 -79.48714 ... 72.41355
    lon_bnds   (nlat, nlon, vertices) float32 320.0 321.125 ... 320.0 319.586
Attributes:
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   0.0
    branch_time_in_parent:  182500.0
    case_id:                58
    cesm_casename:          b.e21.B1PCT.f09_g17.CMIP6-1pctCO2.001
    contact:                cesm_cmip6@ucar.edu
    creation_date:          2019-03-09T17:23:49Z
    data_specs_version:     01.00.29
    experiment:             1 percent per year increase in CO2
    experiment_id:          1pctCO2
    external_variables:     areacello
    forcing_index:          1
    frequency:              mon
    further_info_url:       https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.1...
    grid:                   native gx1v7 displaced pole grid (384x320 latxlon)
    grid_label:             gn
    initialization_index:   1
    institution:            National Center for Atmospheric Research, Climate...
    institution_id:         NCAR
    license:                CMIP6 model data produced by <The National Center...
    mip_era:                CMIP6
    model_doi_url:          https://doi.org/10.5065/D67H1H0V
    nominal_resolution:     100 km
    parent_activity_id:     CMIP
    parent_experiment_id:   piControl
    parent_mip_era:         CMIP6
    parent_source_id:       CESM2
    parent_time_units:      days since 0001-01-01 00:00:00
    parent_variant_label:   r1i1p1f1
    physics_index:          1
    product:                model-output
    realization_index:      1
    realm:                  ocnBgchem
    source:                 CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite v...
    source_id:              CESM2
    source_type:            AOGCM BGC AER
    sub_experiment:         none
    sub_experiment_id:      none
    table_id:               Omon
    tracking_id:            hdl:21.14100/1ccc0d3e-97b1-4392-b607-e037670617ef
    variable_id:            fgco2
    variant_info:           CMIP6 CESM2 1pctCO2 experiment with interactive a...
    variant_label:          r1i1p1f1
  • For esmlab.resample(ds, freq='mon'): we would just need to resample to monthly frequency, and then interpolate the data regardless of the original time frequency:
In [16]: ds_mon = ds.resample({'time': '1M'}).interpolate('nearest')
/Users/abanihi/opt/miniconda3/envs/devel/lib/python3.6/site-packages/xarray/core/nanops.py:159: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

In [17]: ds_mon
Out[17]: 
<xarray.Dataset>
Dimensions:   (nlat: 384, nlon: 320, time: 24, vertices: 4)
Coordinates:
  * time      (time) object 0001-01-31 00:00:00 ... 0002-12-31 00:00:00
    lat       (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon       (nlat, nlon) float64 320.6 321.7 322.8 323.9 ... 318.9 319.4 319.8
  * nlat      (nlat) int32 1 2 3 4 5 6 7 8 9 ... 377 378 379 380 381 382 383 384
  * nlon      (nlon) int32 1 2 3 4 5 6 7 8 9 ... 313 314 315 316 317 318 319 320
Dimensions without coordinates: vertices
Data variables:
    fgco2     (time, nlat, nlon) float32 nan nan nan nan nan ... nan nan nan nan
    lat_bnds  (time, nlat, nlon, vertices) float32 -79.48714 ... 72.41355
    lon_bnds  (time, nlat, nlon, vertices) float32 320.0 321.125 ... 319.586
Attributes:
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   0.0
    branch_time_in_parent:  182500.0
    case_id:                58
    cesm_casename:          b.e21.B1PCT.f09_g17.CMIP6-1pctCO2.001
    contact:                cesm_cmip6@ucar.edu
    creation_date:          2019-03-09T17:23:49Z
    data_specs_version:     01.00.29
    experiment:             1 percent per year increase in CO2
    experiment_id:          1pctCO2
    external_variables:     areacello
    forcing_index:          1
    frequency:              mon
    further_info_url:       https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.1...
    grid:                   native gx1v7 displaced pole grid (384x320 latxlon)
    grid_label:             gn
    initialization_index:   1
    institution:            National Center for Atmospheric Research, Climate...
    institution_id:         NCAR
    license:                CMIP6 model data produced by <The National Center...
    mip_era:                CMIP6
    model_doi_url:          https://doi.org/10.5065/D67H1H0V
    nominal_resolution:     100 km
    parent_activity_id:     CMIP
    parent_experiment_id:   piControl
    parent_mip_era:         CMIP6
    parent_source_id:       CESM2
    parent_time_units:      days since 0001-01-01 00:00:00
    parent_variant_label:   r1i1p1f1
    physics_index:          1
    product:                model-output
    realization_index:      1
    realm:                  ocnBgchem
    source:                 CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite v...
    source_id:              CESM2
    source_type:            AOGCM BGC AER
    sub_experiment:         none
    sub_experiment_id:      none
    table_id:               Omon
    tracking_id:            hdl:21.14100/1ccc0d3e-97b1-4392-b607-e037670617ef
    variable_id:            fgco2
    variant_info:           CMIP6 CESM2 1pctCO2 experiment with interactive a...
    variant_label:          r1i1p1f1
  • For esmlab.resample(ds, freq='ann'): we would just need to resample to yearly frequency, and then interpolate the data regardless of the original time frequency:
In [18]: ds_ann = ds.resample({'time': '1Y'}).interpolate('nearest')
/Users/abanihi/opt/miniconda3/envs/devel/lib/python3.6/site-packages/xarray/core/nanops.py:159: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

In [19]: ds_ann
Out[19]: 
<xarray.Dataset>
Dimensions:   (nlat: 384, nlon: 320, time: 2, vertices: 4)
Coordinates:
  * time      (time) object 0001-12-31 00:00:00 0002-12-31 00:00:00
    lat       (nlat, nlon) float64 -79.22 -79.22 -79.22 ... 72.2 72.19 72.19
    lon       (nlat, nlon) float64 320.6 321.7 322.8 323.9 ... 318.9 319.4 319.8
  * nlat      (nlat) int32 1 2 3 4 5 6 7 8 9 ... 377 378 379 380 381 382 383 384
  * nlon      (nlon) int32 1 2 3 4 5 6 7 8 9 ... 313 314 315 316 317 318 319 320
Dimensions without coordinates: vertices
Data variables:
    fgco2     (time, nlat, nlon) float32 nan nan nan nan nan ... nan nan nan nan
    lat_bnds  (time, nlat, nlon, vertices) float32 -79.48714 ... 72.41355
    lon_bnds  (time, nlat, nlon, vertices) float32 320.0 321.125 ... 319.586
Attributes:
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   0.0
    branch_time_in_parent:  182500.0
    case_id:                58
    cesm_casename:          b.e21.B1PCT.f09_g17.CMIP6-1pctCO2.001
    contact:                cesm_cmip6@ucar.edu
    creation_date:          2019-03-09T17:23:49Z
    data_specs_version:     01.00.29
    experiment:             1 percent per year increase in CO2
    experiment_id:          1pctCO2
    external_variables:     areacello
    forcing_index:          1
    frequency:              mon
    further_info_url:       https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.1...
    grid:                   native gx1v7 displaced pole grid (384x320 latxlon)
    grid_label:             gn
    initialization_index:   1
    institution:            National Center for Atmospheric Research, Climate...
    institution_id:         NCAR
    license:                CMIP6 model data produced by <The National Center...
    mip_era:                CMIP6
    model_doi_url:          https://doi.org/10.5065/D67H1H0V
    nominal_resolution:     100 km
    parent_activity_id:     CMIP
    parent_experiment_id:   piControl
    parent_mip_era:         CMIP6
    parent_source_id:       CESM2
    parent_time_units:      days since 0001-01-01 00:00:00
    parent_variant_label:   r1i1p1f1
    physics_index:          1
    product:                model-output
    realization_index:      1
    realm:                  ocnBgchem
    source:                 CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite v...
    source_id:              CESM2
    source_type:            AOGCM BGC AER
    sub_experiment:         none
    sub_experiment_id:      none
    table_id:               Omon
    tracking_id:            hdl:21.14100/1ccc0d3e-97b1-4392-b607-e037670617ef
    variable_id:            fgco2
    variant_info:           CMIP6 CESM2 1pctCO2 experiment with interactive a...
    variant_label:          r1i1p1f1

Also, wouldn't this completely fix #55 ??

Comments would be helpful from @kmpaul , @jukent , @alperaltuntas, @klindsay28 and any others.

@andersy005 andersy005 added bug Something isn't working and removed bug labels Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@andersy005