Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect pieces for SSP 2024/ScenarioMIP #235

Draft
wants to merge 159 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
159 commits
Select commit Hold shift + click to select a range
46d8e2c
Add ScenarioMIP doc stub; link to #234
khaeru Sep 26, 2024
631abbc
Create `/project/ssp/script` directory and add `__init__.py`
measrainsey Oct 2, 2024
905d6fc
Create `/project/ssp/script/util` directory and add `__init__.py`
measrainsey Oct 2, 2024
de99fe2
Add functions
measrainsey Oct 2, 2024
02b27df
Add script to modify Low scenarios
measrainsey Oct 2, 2024
efe55a7
Add script to modify Very Low scenarios
measrainsey Oct 2, 2024
124af07
Add UE shares files
measrainsey Oct 2, 2024
0b57f47
Change to using `package_data_path()`
measrainsey Oct 2, 2024
54f0031
Remove manual setting of path
measrainsey Oct 2, 2024
4672c82
Clean up scripts and remove commented sections
measrainsey Oct 2, 2024
68ae94c
Add new materials and industry reporting
macflo8 Nov 5, 2024
05d6f00
Correct syntax for materials reporting to change units
OFR-IIASA Nov 7, 2024
1e0ca4b
Extend and rename variable mappings
macflo8 Nov 7, 2024
b2f7606
Update docs
macflo8 Aug 27, 2024
356a23d
Revise materials documentation
macflo8 Aug 29, 2024
34373b5
Update cement furnaces efficiency
macflo8 May 26, 2024
addac4f
Update clinker CCS diffusion constraint
macflo8 May 26, 2024
daee3cf
Update cement and aluminum demand SSP5
macflo8 May 27, 2024
830d1b2
Add SSP specific input steel/cement data
macflo8 May 27, 2024
cd948dd
Add SSP changes to data_steel
macflo8 May 27, 2024
28bd1cc
Remove unused low temp cement furnaces
macflo8 May 28, 2024
6540bc7
Update base year steel and cement demands
macflo8 May 28, 2024
5ecd37b
Update data read functions to new SSP argument
macflo8 May 28, 2024
102e056
Keep solar and biomass base year industry bounds
macflo8 Jun 3, 2024
50a97c1
Adjust base year cement demand of CHN region
macflo8 Jun 3, 2024
61db510
Fix low temp cement furnace ACT to 0
macflo8 Jun 3, 2024
f9ffba2
Update non-metallic minerals base year calibration
macflo8 Jun 3, 2024
675515b
Add cement furnace base year bounds
macflo8 Jun 3, 2024
8599a23
Silence pandas FutureWarnings in reporting
macflo8 Jun 3, 2024
22ff4ea
Improve materials reporting
macflo8 Jun 3, 2024
7feff1b
Add non-CO2 emissions factors to industry furnaces
macflo8 Jun 5, 2024
8e84d9d
Remove deprecated Materials gas heat pumps
macflo8 Jun 5, 2024
9d6d6c2
Remove duplicated technology in set and format
macflo8 Jun 5, 2024
9acdc6f
Fix non-CO2 furnace build bugs
macflo8 Jun 5, 2024
f6a2dda
Fix bug in refactored materials reporting function
macflo8 Jun 6, 2024
1393c4c
Add furnaces to existing GAINS link relation
macflo8 Jun 6, 2024
980ed4e
Fix missing years error caused by data_generic.py
macflo8 Jun 18, 2024
5c41598
Exclude chemicals flow from i_spec calibration
macflo8 Jul 29, 2024
a2f53ed
Reorganize materials commands 2
macflo8 Jul 29, 2024
d4316b8
Add updated cement calibration stats
macflo8 Jul 30, 2024
e60a614
Run TE-tool with "gdp" method by default
macflo8 Aug 1, 2024
f5b8e75
Add cost updates from costs/new_inputs branch
macflo8 Aug 1, 2024
c854549
Clean up new custom SSP build helper
macflo8 Aug 1, 2024
2aa0df4
Add filter for relations in methanol build
macflo8 Aug 1, 2024
a911b8c
Add technologies to remove section to set.yaml
macflo8 Aug 1, 2024
503a110
Add extra MACRO calibration CLI command
macflo8 Aug 1, 2024
9251c96
Add updated aluminum demands
macflo8 Aug 1, 2024
821e7c9
Revert to SSP2 for all SSPs for steel/cement
macflo8 Aug 1, 2024
5b98d74
Add industry non-CO2 emission data file
macflo8 Aug 6, 2024
acdc2a9
Add share constraint generator
macflo8 Aug 14, 2024
61b2756
Move "other industry" utilities to module
macflo8 Aug 14, 2024
c16f2b1
Extend docstrings and type hints
macflo8 Aug 16, 2024
7926310
Split calibration function
macflo8 Aug 16, 2024
4aad596
Move CCS parametrization to input files
macflo8 Aug 16, 2024
b957ae2
Move set modification to set.yaml
macflo8 Aug 16, 2024
dc420c3
Add filename to iea_data_path cli option
macflo8 Aug 19, 2024
bdefd47
Add new residual industry calibration
macflo8 Aug 19, 2024
860c3f8
Generalize coal_i share constraint helper
macflo8 Aug 19, 2024
507f067
Remove deprecated CCS function from build
macflo8 Aug 19, 2024
22e6fdf
Fix new "other industry" calibration workflow
macflo8 Aug 21, 2024
80eee14
Update data_generic to sdmx Code set
macflo8 Sep 2, 2024
37a8a1c
Extend type hints and docstrings to data_util
macflo8 Sep 2, 2024
8a31501
Add water sets missing in SSP_dev
macflo8 Sep 2, 2024
d40a1ee
Add MTO_petro 0 bound_activity_up for 2025
macflo8 Sep 2, 2024
4d11090
Move co2_trans_disp parametrization to file
macflo8 Sep 2, 2024
269abb6
Reorganize parametrization scripts
macflo8 Sep 2, 2024
9edf4bd
Add new demand and activity data to other industry
macflo8 Sep 2, 2024
75916ac
Add low temp heat share constraint modifier
macflo8 Sep 2, 2024
e02f014
Integrate parametrization updates in build
macflo8 Sep 2, 2024
6dd38aa
Add 0 bounds for low temp furnace modes
macflo8 Sep 2, 2024
9e2ac40
Fix fix_cost timeseries break for i_therm tecs
macflo8 Sep 3, 2024
40fb326
Update SSP MACRO calibration workflow
macflo8 Sep 3, 2024
1a36ab6
Extend SSP 2020 activity calibration procedure
macflo8 Sep 3, 2024
969ec78
Remove unused option from material build command
macflo8 Sep 5, 2024
22cb5ac
Rename share constraint generator function
macflo8 Sep 5, 2024
375369d
Remove unused context from materials reporting
macflo8 Sep 5, 2024
e09d43e
Fix in/output unit of furnaces input file
macflo8 Oct 5, 2024
21e1eff
Add new plastics CO2_Emission calculator
macflo8 Oct 12, 2024
80b263e
Extend methanol fs downstream supply chain
macflo8 Oct 15, 2024
29fb1d4
Add missing SSP industry input files
macflo8 Oct 17, 2024
d9c36b7
Fix material balance of MTO_petro technology
macflo8 Oct 22, 2024
848619d
Add new CO2_ind calculator for chemicals
macflo8 Oct 23, 2024
ab0c9a3
Add embodied emissions for ethanol to ethylene
macflo8 Oct 27, 2024
cb638f5
Remove "World" meth_ind_fs parameter regions
macflo8 Oct 28, 2024
ed99590
Fix biomass_nh3_ccs CO2_Emission coefficient
macflo8 Oct 28, 2024
27c3a20
Add ethanol_to_ethylene CO2_ind emi factor
macflo8 Oct 28, 2024
185125f
Replace get_nodes with tested utility
macflo8 Nov 1, 2024
4fc2cf3
Add CO2_cc coefficient for cokeoven
macflo8 Nov 5, 2024
b4d9c23
Fix merge duplication
macflo8 Nov 11, 2024
9e80e73
Migrate ssp cost updates
macflo8 Nov 11, 2024
49cf031
Delete unused tab in cement input xlsx
macflo8 Nov 13, 2024
f5d20df
Update High scenario cost reductions
GamzeUnlu95 Nov 13, 2024
685d1b3
Add DAC to cost projection tool
ywpratama Nov 6, 2024
7834922
Add add_dac tool and related data
ywpratama Nov 6, 2024
a1afaf3
Add draft of CCS and DAC implementation script
ywpratama Nov 6, 2024
54b7567
Update DAC parameters
ywpratama Nov 7, 2024
a0b2c86
Fix bug and improve year_act parameter cut
ywpratama Nov 7, 2024
879c2d7
Add low overshoot scenarios script
ywpratama Nov 11, 2024
327e27f
Add CO2_cc for leakages
ywpratama Nov 15, 2024
2c2d401
Add add_ccs_setup function
ywpratama Nov 15, 2024
86f238f
Update data reading in add_tech function
ywpratama Nov 18, 2024
eacf176
Add macro calibration files for all SSPs
macflo8 Nov 19, 2024
8c3df0a
Fix methanol final energy allocation bug
macflo8 Nov 21, 2024
f1023ca
Add macro calibration files for SSPs
OFR-IIASA Dec 2, 2024
6644f68
Change default region config in model/config from R14 to R12
OFR-IIASA Dec 2, 2024
cee323f
Change data_methanol syntax for reading file
OFR-IIASA Dec 2, 2024
d424be5
Add scenario transact statement to DAC setup workflow
OFR-IIASA Dec 2, 2024
86b6f88
Add diffusion constraint changes
GamzeUnlu95 Nov 27, 2024
2dc2eca
Differentitate growth_new_capacity_up for SSPs
GamzeUnlu95 Dec 2, 2024
e2961cf
Differentiate the scalar used for initial_capacity_new for SSPs
GamzeUnlu95 Dec 2, 2024
92b28fa
Update steel diffusion constraints for steel in baselines
GamzeUnlu95 Dec 4, 2024
412dd17
Add custom cost differentiation for materials
GamzeUnlu95 Dec 4, 2024
9f18740
Add steel recycling changes
GamzeUnlu95 Dec 5, 2024
b2c9ee7
Fix 2025 material demands for all SSPs
macflo8 Nov 25, 2024
5c1c0de
Add updated aluminum capacity calibration
macflo8 Nov 25, 2024
cd08c9b
Add updated aluminum activity calibration
macflo8 Nov 25, 2024
4521c1f
Remove aluminum timeseries growth constraints
macflo8 Nov 27, 2024
0c625d5
Update aluminum sector furnace efficiencies
macflo8 Nov 28, 2024
9cf605f
Add typing to ssp functions to resolve warning
macflo8 Dec 2, 2024
b7afa42
Remove unneeded rows from parametrization dfs
macflo8 Dec 2, 2024
bbc147d
Remove power sector module from build
macflo8 Dec 2, 2024
4c031f7
Add aluminum updates
macflo8 Dec 2, 2024
ad9c263
Set fuel constraints for aluminum refining
macflo8 Dec 2, 2024
63162be
Limit soderberg aluminum new capacity
macflo8 Dec 2, 2024
5d2a91f
Add regionally differentiated alumina heat inputs
macflo8 Dec 4, 2024
dfe2e08
Fix 2025 industry energy demands for all SSPs
macflo8 Dec 5, 2024
46602d3
Remove whitespaces in materials cost csv
macflo8 Dec 6, 2024
e2886b2
Differentiate aluminum recycling
GamzeUnlu95 Dec 6, 2024
b4f53a3
Remove operation reserve for DAC
ywpratama Dec 6, 2024
442424d
Update CO2 injection rates using literature values
ywpratama Dec 6, 2024
2fa9893
Remove CO2 outputs at useful level
ywpratama Dec 6, 2024
d1670f3
Update clinker CCS heat demand
macflo8 Dec 6, 2024
8ddff7e
Add exception for LED methanol reporting
macflo8 Dec 9, 2024
639e93b
Fix nuc_lc set config
adrivinca Dec 10, 2024
b203380
Pre-filter remove.set to speed up the build process
adrivinca Dec 10, 2024
88ffaec
Exclude pre-firstmodelyear years from material rep
macflo8 Dec 12, 2024
319d13e
Add dac module to function type hints
measrainsey Dec 17, 2024
91bb4ea
Move `dac` cost reduction assumptions to module folder
measrainsey Dec 17, 2024
310fbea
Merge pull request #271 from iiasa/mm/ssp-dev-dac
measrainsey Jan 27, 2025
dbaa5a9
Add reduction_year as parameter in Config
measrainsey Jan 27, 2025
9f55313
Switch from hardcoding reduction year to using Config.reduction_year
measrainsey Jan 27, 2025
79552b6
Add attributes for Config.final_projection_year and Config.final_mode…
measrainsey Jan 27, 2025
0c8b247
Update projections code to use new year parameters
measrainsey Jan 27, 2025
1c80de0
Update docstrings and comments
measrainsey Jan 27, 2025
577f432
Edit missing column name
measrainsey Jan 27, 2025
9631092
Edit config comments
measrainsey Jan 27, 2025
2da144a
Add functions to modify pre-base year costs values
measrainsey Jan 27, 2025
7e3aac0
Add example of how to use update_scenario_costs()
measrainsey Jan 27, 2025
ea8477a
Merge pull request #285 from iiasa/costs/years
measrainsey Jan 27, 2025
2105d4c
Start changelog for SSP development
macflo8 Dec 20, 2024
cce6167
Refine formatting of SSP changelog
macflo8 Jan 7, 2025
cd9afe7
Fix missing MEA alumina refining input
macflo8 Jan 7, 2025
6eb12e9
Fix materials reporting bugs
macflo8 Jan 19, 2025
20fc97e
Add cost to alumina trade
macflo8 Jan 24, 2025
b3785c4
Delete deprecated input data
macflo8 Jan 25, 2025
37f69e5
Refactor aluminum input data
macflo8 Jan 26, 2025
bdcfac7
Refactor cement input data
macflo8 Jan 26, 2025
2fc41e6
Refactor steel input data
macflo8 Jan 27, 2025
317a23a
Merge pull request #287 from iiasa/ssp-dev-industry-cleanup
macflo8 Jan 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ Commonly used classes may be imported directly from :mod:`message_ix_models`.
project/circeular
project/edits
project/gea
project/scenariomip
project/shape
project/ssp

Expand Down
138 changes: 138 additions & 0 deletions doc/material/whatsnew.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
What's new
**********

Next release
============

SSP 3.0 development
---------------

By topic:

.. contents::
:local:
:backlinks: none


Non-model changes
^^^^^^^^^^^^^^^^^

- the ``--datafile`` option for the the material-ix build command was removed since it is not used anymore.
- a ``material-ix calibrate`` command has been added that can be used to run the MACRO calibration step in isolation.
- values from model parameter data that have a year_act and year_vtg columns greater than the technology lifetime are dropped since they are inflating the model size unnecessarily.
- a module :mod:`.model.material.share_constraints` for commodity share constraints was created, but is not fully used in the build yet.
.. todo:: consider differentiating the constraints by SSP

Demand
""""""

- the demand commodity ``level`` of methanol has been changed from ``final_material`` to ``demand``
- an additional demand projection setting for aluminum, steel and cement has been introduced called "highest" which SSP5 scenarios are mapped to
- the 2025 demands are fixed to the projected SSP2 values for each SSP
- Aluminum base year demand has been updated based on IAI MFA output for 2020
- Cement base year demand has been updated based on GlobBulk Consulting data for 2020
- Steel base year demand has been updated based on worldsteel association data for 2020


Post processing
"""""""""""""""

- the new industry reporter has been implemented :mod:`.model.material.report`, with mapping files in ``data/material/reporting``

Model development
^^^^^^^^^^^^^^^^^

.. _general-changes:

General
"""""""

- the power sector module has been deactivated for the SSP builds
- coal_i ``technology`` is share constrained in each region based on 2020 IEA statistics
- the low temperature share constraint for other industry is updated to reflect explicit modelling of heavy industry

.. todo:: compile low temperature literature and set up references

- the .tools.cost module has been updated to run in "gdp" mode when called by the materials build
- the .tools.cost module is called twice if ``--update_costs`` option is :any:`True` in ``material-ix build`` command to be able to get the correct cost projections for the non-MESSAGEix-Materials industry technologies (e.g ``coal_i``, ``sp_el_I`` etc.)
- the following utility functions were added/updated in utils.py
- for mapping country names with `ISO 3166-1 alpha-3 <https://en.wikipedia.org/wiki/ISO_3166-1_alpha-3>`_ codes with `pycountry <https://pypi.org/project/pycountry/>`_.

.. todo:: integrate in :mod:`.util.pycountry`

- adding a ``R12`` column with region values mapped based on a iso column of the same dataframe.
- the MACRO calibration excel input file updater is now also updating the ``demand_ref`` tab based on the given scenario ``demand`` parameter values.
- the MACRO calibration excel input file updater can be run in "extrapolation" mode or just take values from the scenario directly.


Aluminum
""""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the aluminum build reads SSP differentiated input data by reading from :file:`data/materials/aluminum/<SSPX>/aluminum_techno_economic.xlsx`
- the historical capacity of smelters is calibrated until 2020 using the genisim dataset.
- the historical activity of smelters is calibrated until 2020 using British Geological Survey data (Soderberg activity is calibrated assuming capacity shares computed from genisim dataset).
- the historical activity of smelters is calibrated until 2020 using British Geological Survey data.
- the historical activity of alumina refining is calibrated until 2020 using British Geological Survey data.
- a trade model for alumina is introduced to be able to calibrate refining and smelting activity.
- the process heat fuel consumption is calibrated using IAI data.
- the regional alumina refining process heat intensities are calibrated using IAI data.

Ammonia
"""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- a unit issue in the emission factor of ``biomass_NH3_ccs`` ``technology`` was fixed.

Cement
""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the cement build reads SSP differentiated input data by reading from :file:`data/materials/cement/<SSPX>/Global_cement_MESSAGE.xlsx`.
- selected regional ``CEMENT`` values in :file:`residual_industry_2019.csv` were updated since they seemed wrong when comparing the data to the IEA cement report.
.. todo:: TODO: paste link to IEA cement report here

- the heat input for clinker CCS addons was lowered a lot.
.. todo:: insert US NREL publication reference here

Other industry
""""""""""""""

- 2025 ``demand`` SSP2 fixing as mentioned under :ref:`general-changes` changes.
- the demands for the other industry are generated based on IEA historical data of the non-explicitly modelled sectors

.. note:: the new demand model still sits in a private repository

- the sector furnaces were missing non CO2 emission factors of their original MESSAGE counterpart (e.g ``coal_i``). Thus, they were copied from the originals and scaled with the input coefficients.
- the furnaces are now writing into the ``IndThermDemLink`` relation, that is required for the MESSAGE-GAINS linkage.

.. _methanol-changes:

Methanol
""""""""

- the model structure was slightly updated to simplify and correct the carbon emission balance accounting

- the negative emission coefficients that represent the carbon stored in long lived products, were moved from ``meth_t_d`` technology to a new technology ``meth_ind_fs``.

- the carbon balance of ``MTO_petro`` was not correct. The process emissions and the ``input``/``output`` parameters were updated based on new literature.

.. todo:: compile literature list and summarize in a paragraph

Petrochemicals
""""""""""""""

- together with the changes in :ref:`methanol-changes` the carbon accounting was improved:

- the negative emission coefficients that represent the carbon stored in long lived products, were moved from ``steam_cracker_petro`` ``technology`` to the ``production_HVC`` ``technology``.

- since carbon capture in plastics was moved upstream, ``ethanol_to_ethylene_petro`` ``technology`` needed a positive emission factor to represent the combusted chemicals part produced with ethylene feedstock from ethanol.

Steel
"""""

- the steel build reads SSP differentiated input data by reading from :file:`data/materials/steel/<SSPX>/Global_steel_MESSAGE.xlsx`.
- the maximum recycling relation was updated.
- the minimum recycling relation was changed for SSP1.
- the ``cokeoven_steel`` ``technology`` got its own bottom up CO2 emission accounting coefficient, to separate that from industrial emissions and move it to transformation emissions.
- the cost changes done in the last PR (insert steel hydrogen PR link) were also transferred to the :mod:`tools.cost` module and custom reduction rates were assigned.
12 changes: 12 additions & 0 deletions doc/project/scenariomip.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
ScenarioMIP
***********

There is currently no dedicated code in :mod:`message_ix_models` associated with this project.
This page is a place-holder.

See also :doc:`/project/ssp`.

Project information
===================

- Website: https://wcrp-cmip.org/mips/scenariomip/
10 changes: 10 additions & 0 deletions doc/project/ssp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,16 @@
Shared Socioeconomic Pathways (:mod:`.project.ssp`)
***************************************************

.. warning::

The 2024 SSP update is **under development**.
For details, see:

- Tracking issue :issue:`234`.
- `Issues and PRs labeled 'p:SSP-2024' <https://github.com/iiasa/message-ix-models/issues?q=label%3Ap:SSP-2024>`_ on the :mod:`.message_ix_models` GitHub repository.

See also :doc:`/project/scenariomip`.

Structure
=========

Expand Down
Loading
Loading