diff --git a/project_plan_and_summary/model_updates.qmd b/project_plan_and_summary/model_updates.qmd index 8d8f4f9..aa959ab 100644 --- a/project_plan_and_summary/model_updates.qmd +++ b/project_plan_and_summary/model_updates.qmd @@ -1,268 +1,311 @@ ---- -title: "Model updates" -order: 5 ---- - -# Update 2024/06/06 - -## v.1.2 of NHP model, inputs and outputs apps - -All scenarios on the inputs app that are currently on v.1.1 will be automatically upgraded to v.1.2. Users who have run a scenario on v.1.1 may wish to re-run their scenario on v.1.2. - -Users on v.1.0 will need to create a new scenario from scratch in order to utilise the newest available version of the model. - -**How can I see which version of the model and inputs app is being used?** - -Check the URL: - -- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-0/ then it is v.1.0. -- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-1/ then it is v.1.1. -- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-2/ then it is v.1.2. - -**Capacity conversion removed** - -We have removed bed capacity from the inputs app, and calculation of beds from the model calculations and outputs. We are working on an improved capacity calculation which utilises queueing theory, which will replace this. - -**More granular results** - -On the Excel download from the outputs app, model results are now available broken down by treatment specialty and length of stay. - -**Bug fixes** - -- In the inputs app, the diagnosis tables for outpatients mitigators were showing all attendances, not just those without procedures. The strategies in the model only apply to activity with no procedures. The diagnosis tables should now match the activity which will be affected in the model. - -- In the inputs app, the Outpatient Convert to Tele-Attendance mitigator charts were including attendances with procedures. The strategy itself in the model only applies to activity with no procedures. The tables and charts for these mitigators should now match the activity which will be affected in the model. - -- The A&E Patients Left Before Being Treated mitigators now utilise the SNOMED codes "1066301000000103", "1066311000000101" and "1066321000000107". Previously, they only used the SNOMED code "1066321000000107". - -## Planned changes to waterfall / impact of changes chart and principal projection - -**Waterfalls / impact of changes charts** - -We are updating our evidence-based approach to visualising the shifts in activity in the outputs app. This will be available in July 2024. In the meantime, occasionally when the points of delivery (PODs) are small, we find the current approach doesn't work in the way we envisaged (resulting in the bars on the waterfall chart that illustrate the effect of individual modelling factors showing incorrect data. The underlying data within the downloadable Excel and JSON of results is similarly affected). Please note and be assured that the baseline and final model estimate figures are correct. - -**Principal projection** - -We are currently implementing an improved method to generate the principal projection. Again this will be available in July 2024. The new approach delivers a more stable estimate of future activity that is less affected by the randomness inherent in the probabilistic methods used within the model engine. The key impact of this improvement will be to ensure that the principal projection will always sit in the middle of the activity distributions resulting from the Monte Carlo simulation. At present, the principal projection can vary from the centre of the distribution and in some instances this variation can be material. - -In the meantime, if you note that any of the activity distribution charts provided in the outputs app show a principal projection that does not sit close to the middle of the distribution (as shown by the 50th percentile in the S-curve chart) AND if that variation is considered to be material (i.e. it would result in materially more or fewer beds than if the central (50th percentile) estimate were used) then we would advise that the current principal projection data is not used as the basis for your capacity conversions and instead wait for the updated model outputs before making the final capacity calculations. - - -# Update 2024/05/10 - -## Open source repositories - -We are pleased to announce that some repositories relating to the NHP model are now public and open source. -These are: - -- [nhp_project_information](https://github.com/The-Strategy-Unit/nhp_project_information): The repository for this website -- [nhp_sql](https://github.com/The-Strategy-Unit/nhp_sql): The repository containing the SQL code for identifying activity relating to each of the [activity mitigators](https://connect.strategyunitwm.nhs.uk/nhp/project_information/user_guide/mitigators_lookup.html). -- [nhp_demogr_module_inputs](https://github.com/The-Strategy-Unit/nhp_demogr_module_inputs): The repository containing the analysis underpinning the various [demographic adjustments](../modelling_methodology/demographic_modelling/demographic_modelling.qmd) in the model. - -A full list of the repositories relating to the NHP model is available [here](../project_plan_and_summary/components-overview.qmd), -although some are still private. We are currently in discussions about open sourcing the entire model. - -## v1.1.1 of NHP inputs and outputs apps - -### Inputs - -The non-demographic adjustment section of the inputs app was updated to: - -* set Variant 2 as the default for non-elective inpatients (for more information about Variant 2 see [the modelling methodology](../modelling_methodology/non-demographic_growth.html)) -* show the 10 and 90% confidence intervals in the the table of parameter values - -### Outputs - -The outputs app was updated to: - -* provide results by bed days in the summary-by-year table -* let you select any combination of points-of-delivery in the activity distribution and activity-in-detail tabs -* add a tabset in the activity summary tab to let you toggle between median and principal values - -# Update 2024/04/24 - -## v.1.1 of NHP model, inputs and outputs apps - -The changes listed below are available in v.1.1 of the model. If continuing work on an existing v.1.0 scenario (from before 26th April 2024), or creating a new scenario based on an existing v.1.0 scenario, users will use v.1.0 of the model. -If starting a new scenario from 26th April 2024, users will default to the v.1.1 version of the inputs app and model. - -Due to conflicts between the mitigators in v.1.0 and v.1.1, it is not possible to copy a v.1.0 scenario into v.1.1, or to create a new v.1.1 scenario based on an existing v.1.0 scenario. - -**How can I see which version of the model and inputs app is being used?** - -Check the URL: - -- If it starts with `https://connect.strategyunitwm.nhs.uk/nhp/v1-0/` then it is v.1.0. -- If it starts with `https://connect.strategyunitwm.nhs.uk/nhp/v1-1/` then it is v.1.1. - -## Numbering of mitigators - -Following feedback from schemes, we have now added identifying codes for each mitigator. These are all listed on [this page](../user_guide/mitigators_lookup.html) and will be seen on the inputs and outputs apps v.1.1 as well. - -## New Virtual Ward mitigators - -Virtual wards allow patients to receive the care they need at home, safely and conveniently rather than in hospital. They also provide systems with a significant opportunity to narrow the gap between demand and capacity for secondary care beds, by providing an alternative to admission and/or early discharge. There are four new Virtual Ward activity mitigators now available in the model v.1.1: - -- [Virtual Wards Admission Avoidance (Acute Respiratory Infection) (IP-AA-030)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-admission-avoidance) -- [Virtual Wards Admission Avoidance (Heart Failure) (IP-AA-031)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-admission-avoidance) -- [Virtual Wards LoS Reduction (Acute Respiratory Infection) (IP-EF-026)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-los-reduction) -- [Virtual Wards LoS Reduction (Heart Failure) (IP-EF-027)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-los-reduction) - -## New Day procedures mitigators - -Day surgery is now established practice with rates continuing to increase. Due to advances in anaesthesia and surgical techniques, it is the standard pathway of care for many complex patients and procedures traditionally treated through inpatient pathways. There are four new day procedures mitigators now available in the model v.1.1. Further information is [available here](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#day-procedures). - -- Day Procedures: Occasionally performed as a Daycase (IP-EF-005) -- Day Procedures: Usually performed as a Daycase (IP-EF-007) -- Day Procedures: Occasionally performed in Outpatients (IP-EF-006) -- Day Procedures: Usually performed in Outpatients (IP-EF-008) - -## Site level information now available for A&E - -A&E activity can now be split by site in the model results. We have accomplished this by moving to a different dataset. There may be slight differences in the amount of A&E activity recorded for each trust in the baseline year 2019/20 in v1.1 of the model as compared with v.1.0, but [our analysis](../resources/nhp_ae_datasets.html) has shown that the differences are minimal for NHP schemes. This change also ensures that we will be better prepared for a move to the 2022/23 baseline year, when the data is made available. - -## Downloadable reports from outputs app - -Users can now download two reports in HTML format from the outputs app v.1.1: - -- Parameters report containing the input parameters for the selected model run -- Outputs report containing the charts and tables for the selected model run and selected sites. - -## Fix to General LoS reduction mitigator - -We have fixed a bug causing the General LoS Reduction mitigator to apply to too many rows of patient data. This mitigator should only be applied if the row of activity is not impacted by any other LoS reduction mitigators. - -# Update 2024/04/04 - -## Inputs app 1.0.1 - -- Users who have submitted their initial model parameters and attended their first refinement workshops will be granted the ability to run the model themselves. Once given this functionality, these users can now select another variant of non-demographic growth. Further details and the analysis underpinning this alternative variant are forthcoming and this page will be updated when it is available. - -- Fixed an error causing the incorrect number of procedures to show for each mitigator. - -- For the Outpatient Convert to Tele-Attendance mitigators, the diagnosis/procedures tables and charts were previously showing the amount of activity for tele-appointments. However, the area of opportunity is in the reduction of face-to-face appointments; therefore, the tables and charts have been updated to show the face-to-face appointment numbers instead. - -## Outputs app 1.0.1 - -- A tab showing length-of-stay summary is now available. This breaks down the model results into the following categories, allowing for more nuanced interpretation. - - - 0-day - - - 1-7 days - - - 8-14 days - - - 15-21 days - - - 22+ days - -- Other smaller changes have also been implemented, including more explanatory text for bed days, better sorting of values in the Excel download, and more robust URL links to results. - -## Planned developments - -- Introducing new activity mitigators, including virtual ward and day procedures. - -- Using the Emergency Care Data Set (ECDS) for our Accident and Emergency (A&E) data, instead of the older HES AE dataset. This will allow us to split A&E activity by site. - -- Introducing numbering of mitigators to aid their discussion and reduce confusion. - -# Update 2024/03/07 - -Version 1.0 of the NHP demand and capacity model and its associated inputs and outputs apps has been released. Thank you for your patience during the downtime. - -## What has changed? - -1. Improved impact of changes / waterfall charts. These charts are based on simulations and are never 100% accurate - but we are doing more computation now to make them more accurate and detailed. These results can now be filtered by site, and by point of delivery (POD). - -2. We can now have more than one version of the app released at the same time. This helps with maintenance and enables us to roll out new features without disrupting previous work. Users will be directed to the correct version of the app automatically. Please note that old URLs for specific model run results will no longer work and users must manually select the scenario they wish to view from the outputs app using the dropdown boxes. The URLs for the inputs and outputs apps are the same as before: - -- Inputs app: -- Outputs app: - -3. The day case mitigators have been temporarily removed. This will enable us to improve their analytical function. The description of the mitigators will remain unchanged and we hope to reintroduce them after 18th March. The mitigators affected are: - -- Elective surgery - recommended mainly day case -- Elective surgery - recommended occasional day case -- Elective surgery - recommended mainly outpatient procedure -- Elective surgery - recommended mainly outpatient procedure or day case - -### If your scheme has not yet run the model - -- Please enter your model parameters as usual using the [NHP model inputs app](https://connect.strategyunitwm.nhs.uk/nhp/inputs/). Scenarios that you were working on prior to the update should still be viewable and editable. Note that day case mitigators have been removed temporarily; we hope to reintroduce them from March 18th. - -### If your scheme has already run the model - -- Please enter your model parameters as usual using the [NHP model inputs app](https://connect.strategyunitwm.nhs.uk/nhp/inputs/). Scenarios that you were working on prior to the update should still be viewable and editable. Note that day case mitigators have been removed temporarily; we hope to reintroduce them from March 18th. Any schemes that have completed their refinement workshops should also now have access to the Run Model option. - -- All scenarios have now been rerun with model v.1.0. These have more detailed impact of changes / waterfall diagrams. Day case mitigators have also been removed, so the results will be slightly different. These results can be viewed using the [NHP model outputs app](https://connect.strategyunitwm.nhs.uk/nhp/outputs/). The results of old model runs with day case mitigators and less detailed impact of changes / waterfall diagrams are available on request. - -## Planned future developments - -We hope to introduce downloadable reports from the outputs app soon, summarising model results. We are also working on showing the impact of changes / waterfall diagrams with 0-day LOS and day case bed days split out. - -# Update 2024/03/01 - -There is a **temporary shutdown of the model between 5pm Friday 1st March and 9am Thursday 7th March**. Users will not be able to access the model during this time. This includes both the NHP inputs and NHP outputs apps. The shutdown will enable us to make several model upgrades. - -Users should save the scenarios they are working on before 5pm on Friday 1st March, by stopping work on them and closing the app. Their work will remain available to access when the model goes live again at 9am on Thursday 7th March. - -We will be **pausing use of the day case mitigators from 5pm Friday 1st March until 18th March**. This will enable us to improve their analytical function. The description of the mitigators will remain unchanged. The mitigators affected are: - -- Elective surgery - recommended mainly day case -- Elective surgery - recommended occasional day case -- Elective surgery - recommended mainly outpatient procedure -- Elective surgery - recommended mainly outpatient procedure or day case - -For models run before Friday 1st March, outputs will remain unchanged and will include these mitigators. Trusts that have used these mitigators (avoiding 30-50% of activity) have found that their impact on inpatient bed days is approximately 1% of the total. - -For models run between 7th and 18th March, day case mitigators will not be included in the model parameters. - -The improved day case mitigators will be reintroduced to the inputs app and model after 18th March. - -**What will be new after March 7th?** - -- Users will be able to switch between different versions of the inputs and outputs apps. This is to ensure continuity for the different tranches, as development of the apps and model is ongoing. There will be a slight change to the app interfaces, with an additional selection page added to both the inputs and outputs apps. -- Following user feedback, we are providing more detailed step counts calculations within the model, which show the impact of the different model parameters on model results. This has required a complete recoding of how step counts are calculated within the model. This change will also result in more detailed "impact of changes" waterfall diagrams in the outputs app. - -# Update 2024/02/20 - -## Inputs app 0.6.8 - -- Added a "Supporting rationale" text box to the "Model Run" page to enable users of the app to provide further context for their model parameters. -- We have removed the "upload parameters" option temporarily as we work on a more stable solution to enable users to build on another user's scenario. -- Fixed a minor bug affecting the download of the baseline data CSV file. - -## Outputs app 0.6.3 - -To prepare for the technical and refinement workshops for Tranche 1, a lot of work has been done to improve the outputs app. This work aims to facilitate understanding of the model results through improvements to the labelling, ordering, and visualisation of data. Minor changes to the graphics and tables may be observed by users of the outputs app during this period. Despite these changes, the underlying model results data remains the same. - -## Project information site - -- Addition of ECDF explanation on Glossary and definitions page -- Analysis underpinning the Non-Demographic Growth model parameters has been added -- More information added to FAQ page - -## Planned developments currently in progress: - -We are currently working on the ability to release different versions of the input and output apps simultaneously. This is particularly important as different tranches are at different stages of their progress through the NHP D&C model process, and we want to ensure consistency for each organisation. - -We are also working on the creation of a downloadable file from the outputs app, and on the creation of a more user-friendly "landing page" when accessing the inputs app. - -# Update 2024/02/08 - -## Inputs app - -- Baseline data for each trust is now available to download as a CSV, following feedback from participating trusts. -- The 2018-19 baseline year has been removed, as it is arguably of little analytical value. Its removal will make model development and data extraction a smoother process moving forward. -- Non-demographic growth will no longer be applied for inpatients maternity activity. The parameter is set at 1 which represents no change. This assumption is currently under review and may be revised in later versions of the model. -- A General Length of Stay (LOS) mitigator has been added, which will allow trusts to estimate how much they will reduce the average length of stay for any admissions greater than 0 days in length. This is for emergency and elective admissions. - -## Project information site - -- Information about the new general length of stay mitigator has been added. -- An FAQ page has been added to help highlight questions asked by trusts that might be useful to other trusts. -- A Glossary page has been added to help explain commonly used terms throughout the NHP model and its associated apps. - -## Planned developments currently in progress - -- We are currently working on the model outputs app, to ensure that it meets the needs of end users. +--- +title: "Model updates" +order: 5 +--- + +# Update 2024/07/04 + +## v.2.0 of NHP model, inputs and outputs apps + +All scenarios on the inputs app that are currently on v.1.2 will be automatically upgraded to v.2.0. Users who have run a scenario on v.1.2 may wish to re-run their scenario on v.2.0. + +Users on v.1.1 of the model or earlier will need to create a new scenario from scratch in order to utilise the newest available version of the model. + +**How can I see which version of the model and inputs app is being used?** + +Check the URL: + +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-0/ then it is v.1.0. +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-1/ then it is v.1.1. +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-2/ then it is v.1.2. +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v2-0/ then it is v.2.0. + +## New definition of principal projection + +The key outputs of the model are estimates of future hospital activity. The principal projection can be thought of as the 'best guess' or most likely estimate based on the parameters set. + +For model v2.0 onwards, the principal projection is the average (mean) of all the 256 Monte Carlo simulations which have been run for a scenario. In the outputs, users will see a principal projection figure for all the different types of activity which the model forecasts. This approach to determining the principal projection ensures that the it will sit in the middle of the activity distribution (that is, it will be very close to the 50th percentile or median). + +For model versions up to and including v1.2, the principal projection was determined by a single model run which took the midpoint of all the parameter confidence intervals set by the user. However, given the probabilistic nature of the model, we have found that using the average of all the simulations is a more stable method of determining the principal projection. + +This is not to be confused with the ONS principal population projection. + +## New waterfall charts + +We have updated the methodology for determining the impact of each of the parameters in the model on the actual results. The "waterfall charts" seen in the outputs app should therefore be a more accurate, although still highly approximate, indication of the amount of activity that has been added or removed due to each parameter. + +The NHP model is multiplicative. The interaction between the various factors in the model has in the past been redistributed across all the steps. However, we found this created inconsistencies in specific scenarios. To address this, we now show the Model Interaction Term as a separate value in the waterfall charts. + +## Other changes + +**Virtual Ward Efficiencies mitigators: bug fixed** + +A bug was identified in the data where the Virtual Wards efficiency mitigators were wrongly flagged as activity avoidance mitigators. This meant that if any Virtual Wards efficiency mitigators were set, these were not having any impact on the model results. This error was corrected and the data re-extracted. + +**Time profiles temporarily removed** + +The time profiles, or year-by-year estimates of activity in the years between the baseline and horizon year, were based on the old definition of the principal model run. As we have moved away from this definition, we will need to revise how time profiles are approached in the model. This work will be completed in August 2024. + +# Update 2024/06/06 + +## v.1.2 of NHP model, inputs and outputs apps + +All scenarios on the inputs app that are currently on v.1.1 will be automatically upgraded to v.1.2. Users who have run a scenario on v.1.1 may wish to re-run their scenario on v.1.2. + +Users on v.1.0 will need to create a new scenario from scratch in order to utilise the newest available version of the model. + +**How can I see which version of the model and inputs app is being used?** + +Check the URL: + +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-0/ then it is v.1.0. +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-1/ then it is v.1.1. +- If it starts with https://connect.strategyunitwm.nhs.uk/nhp/v1-2/ then it is v.1.2. + +**Capacity conversion removed** + +We have removed bed capacity from the inputs app, and calculation of beds from the model calculations and outputs. We are working on an improved capacity calculation which utilises queueing theory, which will replace this. + +**More granular results** + +On the Excel download from the outputs app, model results are now available broken down by treatment specialty and length of stay. + +**Bug fixes** + +- In the inputs app, the diagnosis tables for outpatients mitigators were showing all attendances, not just those without procedures. The strategies in the model only apply to activity with no procedures. The diagnosis tables should now match the activity which will be affected in the model. + +- In the inputs app, the Outpatient Convert to Tele-Attendance mitigator charts were including attendances with procedures. The strategy itself in the model only applies to activity with no procedures. The tables and charts for these mitigators should now match the activity which will be affected in the model. + +- The A&E Patients Left Before Being Treated mitigators now utilise the SNOMED codes "1066301000000103", "1066311000000101" and "1066321000000107". Previously, they only used the SNOMED code "1066321000000107". + +## Planned changes to waterfall / impact of changes chart and principal projection + +**Waterfalls / impact of changes charts** + +We are updating our evidence-based approach to visualising the shifts in activity in the outputs app. This will be available in July 2024. In the meantime, occasionally when the points of delivery (PODs) are small, we find the current approach doesn't work in the way we envisaged (resulting in the bars on the waterfall chart that illustrate the effect of individual modelling factors showing incorrect data. The underlying data within the downloadable Excel and JSON of results is similarly affected). Please note and be assured that the baseline and final model estimate figures are correct. + +**Principal projection** + +We are currently implementing an improved method to generate the principal projection. Again this will be available in July 2024. The new approach delivers a more stable estimate of future activity that is less affected by the randomness inherent in the probabilistic methods used within the model engine. The key impact of this improvement will be to ensure that the principal projection will always sit in the middle of the activity distributions resulting from the Monte Carlo simulation. At present, the principal projection can vary from the centre of the distribution and in some instances this variation can be material. + +In the meantime, if you note that any of the activity distribution charts provided in the outputs app show a principal projection that does not sit close to the middle of the distribution (as shown by the 50th percentile in the S-curve chart) AND if that variation is considered to be material (i.e. it would result in materially more or fewer beds than if the central (50th percentile) estimate were used) then we would advise that the current principal projection data is not used as the basis for your capacity conversions and instead wait for the updated model outputs before making the final capacity calculations. + + +# Update 2024/05/10 + +## Open source repositories + +We are pleased to announce that some repositories relating to the NHP model are now public and open source. +These are: + +- [nhp_project_information](https://github.com/The-Strategy-Unit/nhp_project_information): The repository for this website +- [nhp_sql](https://github.com/The-Strategy-Unit/nhp_sql): The repository containing the SQL code for identifying activity relating to each of the [activity mitigators](https://connect.strategyunitwm.nhs.uk/nhp/project_information/user_guide/mitigators_lookup.html). +- [nhp_demogr_module_inputs](https://github.com/The-Strategy-Unit/nhp_demogr_module_inputs): The repository containing the analysis underpinning the various [demographic adjustments](../modelling_methodology/demographic_modelling/demographic_modelling.qmd) in the model. + +A full list of the repositories relating to the NHP model is available [here](../project_plan_and_summary/components-overview.qmd), +although some are still private. We are currently in discussions about open sourcing the entire model. + +## v1.1.1 of NHP inputs and outputs apps + +### Inputs + +The non-demographic adjustment section of the inputs app was updated to: + +* set Variant 2 as the default for non-elective inpatients (for more information about Variant 2 see [the modelling methodology](../modelling_methodology/non-demographic_growth.html)) +* show the 10 and 90% confidence intervals in the the table of parameter values + +### Outputs + +The outputs app was updated to: + +* provide results by bed days in the summary-by-year table +* let you select any combination of points-of-delivery in the activity distribution and activity-in-detail tabs +* add a tabset in the activity summary tab to let you toggle between median and principal values + +# Update 2024/04/24 + +## v.1.1 of NHP model, inputs and outputs apps + +The changes listed below are available in v.1.1 of the model. If continuing work on an existing v.1.0 scenario (from before 26th April 2024), or creating a new scenario based on an existing v.1.0 scenario, users will use v.1.0 of the model. +If starting a new scenario from 26th April 2024, users will default to the v.1.1 version of the inputs app and model. + +Due to conflicts between the mitigators in v.1.0 and v.1.1, it is not possible to copy a v.1.0 scenario into v.1.1, or to create a new v.1.1 scenario based on an existing v.1.0 scenario. + +**How can I see which version of the model and inputs app is being used?** + +Check the URL: + +- If it starts with `https://connect.strategyunitwm.nhs.uk/nhp/v1-0/` then it is v.1.0. +- If it starts with `https://connect.strategyunitwm.nhs.uk/nhp/v1-1/` then it is v.1.1. + +## Numbering of mitigators + +Following feedback from schemes, we have now added identifying codes for each mitigator. These are all listed on [this page](../user_guide/mitigators_lookup.html) and will be seen on the inputs and outputs apps v.1.1 as well. + +## New Virtual Ward mitigators + +Virtual wards allow patients to receive the care they need at home, safely and conveniently rather than in hospital. They also provide systems with a significant opportunity to narrow the gap between demand and capacity for secondary care beds, by providing an alternative to admission and/or early discharge. There are four new Virtual Ward activity mitigators now available in the model v.1.1: + +- [Virtual Wards Admission Avoidance (Acute Respiratory Infection) (IP-AA-030)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-admission-avoidance) +- [Virtual Wards Admission Avoidance (Heart Failure) (IP-AA-031)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-admission-avoidance) +- [Virtual Wards LoS Reduction (Acute Respiratory Infection) (IP-EF-026)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-los-reduction) +- [Virtual Wards LoS Reduction (Heart Failure) (IP-EF-027)](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#virtual-wards-los-reduction) + +## New Day procedures mitigators + +Day surgery is now established practice with rates continuing to increase. Due to advances in anaesthesia and surgical techniques, it is the standard pathway of care for many complex patients and procedures traditionally treated through inpatient pathways. There are four new day procedures mitigators now available in the model v.1.1. Further information is [available here](../modelling_methodology/activity_mitigators/inpatient_activity_mitigators.html#day-procedures). + +- Day Procedures: Occasionally performed as a Daycase (IP-EF-005) +- Day Procedures: Usually performed as a Daycase (IP-EF-007) +- Day Procedures: Occasionally performed in Outpatients (IP-EF-006) +- Day Procedures: Usually performed in Outpatients (IP-EF-008) + +## Site level information now available for A&E + +A&E activity can now be split by site in the model results. We have accomplished this by moving to a different dataset. There may be slight differences in the amount of A&E activity recorded for each trust in the baseline year 2019/20 in v1.1 of the model as compared with v.1.0, but [our analysis](../resources/nhp_ae_datasets.html) has shown that the differences are minimal for NHP schemes. This change also ensures that we will be better prepared for a move to the 2022/23 baseline year, when the data is made available. + +## Downloadable reports from outputs app + +Users can now download two reports in HTML format from the outputs app v.1.1: + +- Parameters report containing the input parameters for the selected model run +- Outputs report containing the charts and tables for the selected model run and selected sites. + +## Fix to General LoS reduction mitigator + +We have fixed a bug causing the General LoS Reduction mitigator to apply to too many rows of patient data. This mitigator should only be applied if the row of activity is not impacted by any other LoS reduction mitigators. + +# Update 2024/04/04 + +## Inputs app 1.0.1 + +- Users who have submitted their initial model parameters and attended their first refinement workshops will be granted the ability to run the model themselves. Once given this functionality, these users can now select another variant of non-demographic growth. Further details and the analysis underpinning this alternative variant are forthcoming and this page will be updated when it is available. + +- Fixed an error causing the incorrect number of procedures to show for each mitigator. + +- For the Outpatient Convert to Tele-Attendance mitigators, the diagnosis/procedures tables and charts were previously showing the amount of activity for tele-appointments. However, the area of opportunity is in the reduction of face-to-face appointments; therefore, the tables and charts have been updated to show the face-to-face appointment numbers instead. + +## Outputs app 1.0.1 + +- A tab showing length-of-stay summary is now available. This breaks down the model results into the following categories, allowing for more nuanced interpretation. + + - 0-day + + - 1-7 days + + - 8-14 days + + - 15-21 days + + - 22+ days + +- Other smaller changes have also been implemented, including more explanatory text for bed days, better sorting of values in the Excel download, and more robust URL links to results. + +## Planned developments + +- Introducing new activity mitigators, including virtual ward and day procedures. + +- Using the Emergency Care Data Set (ECDS) for our Accident and Emergency (A&E) data, instead of the older HES AE dataset. This will allow us to split A&E activity by site. + +- Introducing numbering of mitigators to aid their discussion and reduce confusion. + +# Update 2024/03/07 + +Version 1.0 of the NHP demand and capacity model and its associated inputs and outputs apps has been released. Thank you for your patience during the downtime. + +## What has changed? + +1. Improved impact of changes / waterfall charts. These charts are based on simulations and are never 100% accurate - but we are doing more computation now to make them more accurate and detailed. These results can now be filtered by site, and by point of delivery (POD). + +2. We can now have more than one version of the app released at the same time. This helps with maintenance and enables us to roll out new features without disrupting previous work. Users will be directed to the correct version of the app automatically. Please note that old URLs for specific model run results will no longer work and users must manually select the scenario they wish to view from the outputs app using the dropdown boxes. The URLs for the inputs and outputs apps are the same as before: + +- Inputs app: +- Outputs app: + +3. The day case mitigators have been temporarily removed. This will enable us to improve their analytical function. The description of the mitigators will remain unchanged and we hope to reintroduce them after 18th March. The mitigators affected are: + +- Elective surgery - recommended mainly day case +- Elective surgery - recommended occasional day case +- Elective surgery - recommended mainly outpatient procedure +- Elective surgery - recommended mainly outpatient procedure or day case + +### If your scheme has not yet run the model + +- Please enter your model parameters as usual using the [NHP model inputs app](https://connect.strategyunitwm.nhs.uk/nhp/inputs/). Scenarios that you were working on prior to the update should still be viewable and editable. Note that day case mitigators have been removed temporarily; we hope to reintroduce them from March 18th. + +### If your scheme has already run the model + +- Please enter your model parameters as usual using the [NHP model inputs app](https://connect.strategyunitwm.nhs.uk/nhp/inputs/). Scenarios that you were working on prior to the update should still be viewable and editable. Note that day case mitigators have been removed temporarily; we hope to reintroduce them from March 18th. Any schemes that have completed their refinement workshops should also now have access to the Run Model option. + +- All scenarios have now been rerun with model v.1.0. These have more detailed impact of changes / waterfall diagrams. Day case mitigators have also been removed, so the results will be slightly different. These results can be viewed using the [NHP model outputs app](https://connect.strategyunitwm.nhs.uk/nhp/outputs/). The results of old model runs with day case mitigators and less detailed impact of changes / waterfall diagrams are available on request. + +## Planned future developments + +We hope to introduce downloadable reports from the outputs app soon, summarising model results. We are also working on showing the impact of changes / waterfall diagrams with 0-day LOS and day case bed days split out. + +# Update 2024/03/01 + +There is a **temporary shutdown of the model between 5pm Friday 1st March and 9am Thursday 7th March**. Users will not be able to access the model during this time. This includes both the NHP inputs and NHP outputs apps. The shutdown will enable us to make several model upgrades. + +Users should save the scenarios they are working on before 5pm on Friday 1st March, by stopping work on them and closing the app. Their work will remain available to access when the model goes live again at 9am on Thursday 7th March. + +We will be **pausing use of the day case mitigators from 5pm Friday 1st March until 18th March**. This will enable us to improve their analytical function. The description of the mitigators will remain unchanged. The mitigators affected are: + +- Elective surgery - recommended mainly day case +- Elective surgery - recommended occasional day case +- Elective surgery - recommended mainly outpatient procedure +- Elective surgery - recommended mainly outpatient procedure or day case + +For models run before Friday 1st March, outputs will remain unchanged and will include these mitigators. Trusts that have used these mitigators (avoiding 30-50% of activity) have found that their impact on inpatient bed days is approximately 1% of the total. + +For models run between 7th and 18th March, day case mitigators will not be included in the model parameters. + +The improved day case mitigators will be reintroduced to the inputs app and model after 18th March. + +**What will be new after March 7th?** + +- Users will be able to switch between different versions of the inputs and outputs apps. This is to ensure continuity for the different tranches, as development of the apps and model is ongoing. There will be a slight change to the app interfaces, with an additional selection page added to both the inputs and outputs apps. +- Following user feedback, we are providing more detailed step counts calculations within the model, which show the impact of the different model parameters on model results. This has required a complete recoding of how step counts are calculated within the model. This change will also result in more detailed "impact of changes" waterfall diagrams in the outputs app. + +# Update 2024/02/20 + +## Inputs app 0.6.8 + +- Added a "Supporting rationale" text box to the "Model Run" page to enable users of the app to provide further context for their model parameters. +- We have removed the "upload parameters" option temporarily as we work on a more stable solution to enable users to build on another user's scenario. +- Fixed a minor bug affecting the download of the baseline data CSV file. + +## Outputs app 0.6.3 + +To prepare for the technical and refinement workshops for Tranche 1, a lot of work has been done to improve the outputs app. This work aims to facilitate understanding of the model results through improvements to the labelling, ordering, and visualisation of data. Minor changes to the graphics and tables may be observed by users of the outputs app during this period. Despite these changes, the underlying model results data remains the same. + +## Project information site + +- Addition of ECDF explanation on Glossary and definitions page +- Analysis underpinning the Non-Demographic Growth model parameters has been added +- More information added to FAQ page + +## Planned developments currently in progress: + +We are currently working on the ability to release different versions of the input and output apps simultaneously. This is particularly important as different tranches are at different stages of their progress through the NHP D&C model process, and we want to ensure consistency for each organisation. + +We are also working on the creation of a downloadable file from the outputs app, and on the creation of a more user-friendly "landing page" when accessing the inputs app. + +# Update 2024/02/08 + +## Inputs app + +- Baseline data for each trust is now available to download as a CSV, following feedback from participating trusts. +- The 2018-19 baseline year has been removed, as it is arguably of little analytical value. Its removal will make model development and data extraction a smoother process moving forward. +- Non-demographic growth will no longer be applied for inpatients maternity activity. The parameter is set at 1 which represents no change. This assumption is currently under review and may be revised in later versions of the model. +- A General Length of Stay (LOS) mitigator has been added, which will allow trusts to estimate how much they will reduce the average length of stay for any admissions greater than 0 days in length. This is for emergency and elective admissions. + +## Project information site + +- Information about the new general length of stay mitigator has been added. +- An FAQ page has been added to help highlight questions asked by trusts that might be useful to other trusts. +- A Glossary page has been added to help explain commonly used terms throughout the NHP model and its associated apps. + +## Planned developments currently in progress + +- We are currently working on the model outputs app, to ensure that it meets the needs of end users. diff --git a/project_plan_and_summary/plan_and_summary.qmd b/project_plan_and_summary/plan_and_summary.qmd index 9042934..7c49524 100644 --- a/project_plan_and_summary/plan_and_summary.qmd +++ b/project_plan_and_summary/plan_and_summary.qmd @@ -3,14 +3,10 @@ title: "Project objectives" order: 1 --- -The government has committed to building more than 40 new hospitals by 2030. The New Hospital Programme (NHP), a partnership between DHSC and NHS E&I, aims to ensure that this new hospital infrastructure will meet the future needs of the population and that the required investment represents value for money. +The government has committed to building more than 40 new hospitals by 2030. The New Hospital Programme (NHP), a partnership between DHSC and NHSE, aims to ensure that this new hospital infrastructure will meet the future needs of the population and that the required investment represents value for money. -Estimating future activity levels represents a critical early step in the process of scaling and designing new hospital infrastructure. The Activity and Demand Modelling Tool has been developed by the Strategy Unit to support DHSC and its stakeholders to make robust and auditable assessments of activity that hospitals may need to accommodate in future. +Estimating future activity levels represents a critical early step in the process of scaling and designing new hospital infrastructure. The Demand and Capacity Modelling Tool has been developed by the Strategy Unit to support DHSC and its stakeholders to make robust and auditable assessments of activity that hospitals may need to accommodate in future. -This document describes this model from a technical perspective. It is designed to be used by individuals engaging directly with the model, to ensure that they have a clear understanding of the model's operations, and by those involved in quality assuring the model. +This document describes this model from a technical perspective. It is designed to be used by individuals engaging directly with the model, to ensure that they have a clear understanding of the model's operations, and by those involved in quality assuring the model. -The model will aid in the forecasting of the following: - -* Required activity levels the hospital will need to service. -* Estimations of high-level accommodation/clinical facility requirements (number of beds, rooms, wards, theatres), to be discussed and iterated with other NHP workstreams. -* The degree of flexibility to build into hospital designs, and where this flexibility should be focused, to account for uncertainties and limitations in the modelling. +The model will aid in the forecasting of the required activity levels the hospital will need to service, as well as the degree of flexibility to build into hospital designs to account for uncertainties and limitations in the modelling. diff --git a/user_guide/NHP_model_summary.qmd b/user_guide/NHP_model_summary.qmd index 8fe2d16..b22035d 100644 --- a/user_guide/NHP_model_summary.qmd +++ b/user_guide/NHP_model_summary.qmd @@ -5,14 +5,20 @@ order: 1 # What does the NHP model do? -In a nutshell, the NHP model estimates future hospital activity. To do this, it uses as its basis activity from a baseline year (2019/20). +Essentially, the NHP model estimates future hospital activity. It is a probabilistic model which factors in the inherent uncertainty associated with making forecasts about future demand. As such, some of the inputs must be given in the form of ranges (80% confidence intervals) and the outputs provide a [principal projection](/user_guide/glossary.qmd#principal-projection) as well as a description of the variation in the estimated future activity. The main [outputs](/user_guide/FAQ.qmd#how-are-outputs-provided) are high-level summary statistics such as counts of admissions, bed-days, outpatient and A&E attendances as well as more detailed information providing a breakdown by site, activity type, age group, and treatment specialty. -Users provide input parameters, estimating the probability of whether specific types of activity will increase or decrease. Other change factors are also taken into consideration, such as demographic and non-demographic growth. +In addition to the forecasting of hospital activity, schemes will be offered the use of tools to assist them in their task of converting the forecasted demand into capacity. -The model runs a '**principal model run**' which takes as its parameters the midpoint of all the parameter estimates. So for example if the estimated expatriation of inpatient elective opthalmology activity is 80% - 100%, the model will take the value of 90% for this parameter for the principal run. +# What does the model NOT do? -It will then run as many Monte Carlo simulations as have been selected by the user (usually 256). During these Monte Carlo simulations, the parameters will be randomly sampled from the confidence intervals provided, assuming a normal distribution. So for example if the estimated expatriation of inpatient elective opthalmology activity is 80% - 100%, some runs might take 80% as the value for this parameter, and some might take 100% as the value for this parameter. Most runs will take values somewhere in the middle, however. +The model does **not** provide users with estimations of accommodation requirements such as the number of beds, rooms, wards, theatres etc. The task of converting the projected demand into capacity is the responsibility of the scheme. -More detail on how the parameters are applied to individual records from the baseline data is [available here](../modelling_methodology/modelling_uncertainty.qmd). +# How does the model generate its outputs? -After each model run we get estimated future hospital activity. The numerous runs that happen during the Monte Carlo simulations are summarised to give an overall estimation of likely future activity. +The model uses HES data and handles discrete hospital events rather than aggregate summaries of hospital activity. It currently uses the [baseline year 2019/20](/user_guide/FAQ.qmd#why-do-we-use-a-201920-baseline-year) and uses this as the basis to model future hospital activity up to the [horizon year](/user_guide/FAQ.qmd#which-year-should-be-selected-as-the-horizon-year) chosen by the scheme. Users provide input parameters, estimating the extent to which specific types of activity will increase or decrease. These assumptions are given in the form of 80% confidence intervals. Other change factors are also taken into consideration, such as demographic and non-demographic growth. Some parameters can not be adjusted by the user because they have been determined nationally. + +The model uses Monte Carlo simulations – it runs as many simulations as have been selected by the user, with the default being 256. In these simulations, values are sampled from each of the input parameters – these values are taken from anywhere within the confidence intervals the user has given. However, it assumes normally distributed parameters, which means most simulations take values around the middle of the confidence interval. Each simulation provides an estimation of a possible future of hospital activity at the horizon year. + +From model v2.0, it is the average (mean) of all the simulations which produces the principal projection – the most likely estimate of activity in the horizon year. It is the figures of the principal projection which will be the starting point for schemes converting this projected demand into capacity. The model outputs also inform the user of the potential variation in future activity, as displayed in beeswarm plots and S-curve charts. For many of the outputs, the values of the lower and upper confidence intervals are also available. As such, the outputs of the model reflect the uncertainty in the underpinning inputs, assumptions and estimates. + +More detail on how the parameters are applied to individual records from the baseline data is [available here](/modelling_methodology/modelling_uncertainty.qmd). diff --git a/user_guide/glossary.qmd b/user_guide/glossary.qmd index a416043..64982ce 100644 --- a/user_guide/glossary.qmd +++ b/user_guide/glossary.qmd @@ -25,9 +25,10 @@ The s-curve shows the range of results (or values) obtained from all the model r ## Impact of changes -High-level estimates of the number of rows that are added/removed due to each parameter. These should be regarded as very rough estimates, and treated with a pinch of salt. +High-level estimates of the number of rows that are added/removed due to each parameter. These should be regarded as very rough estimates. This is sometimes referred to as the "waterfall diagram" in the outputs app. -Detailed technical explanation: The “step counts” in the model are created during the phase of the model where we create the factors. When we add a step to the model we mutate a column, then take the mean of this mutate column to get a scalar value. We compare this scalar to the prior scalar to work out the effect of this step (the step count). However, the parameters are just the values that are used for the lambda parameter of a random Poisson distribution. Here, we try to convert factors of lambda to the number of rows that were added/removed due to that factor. However, because it is a random process, there is no direct relationship. We end up with an error term which is redistributed across all the steps to make the end result be consistent. +Detailed technical explanation: The "step counts" in the model are created during the phase of the model where we create the factors. When we add a "step" to the model we work out which rows of activity are affected by the step, creating a new column per step with the factors for that row. We then take the weighted mean of each new column to get a scalar value, grouped by point of delivery and site of treatment. +We take this weighted mean, subtract 1, and multiply by the baseline to get the estimated simple effect of the step in isolation from all other steps. When these simple effects are summed, they do not equal the total effect, as in the model the factors are actually multiplied together and not summed. Therefore, we include a model interaction term to describe the effect of multiplying the factors together as well as the residual from random sampling. ## Inpatient admissions @@ -46,7 +47,13 @@ CASE ## Principal projection -A single model run which takes as its parameters the midpoint of all the parameter estimates. So for example if the estimated expatriation of inpatient elective opthalmology activity is 80% - 100%, the model will take the value of 90% for this parameter for the principal run. Not to be confused with the ONS principal population projection. +The key outputs of the model are estimates of future hospital activity. The principal projection can be thought of as the 'best guess' or most likely estimate based on the parameters set. + +For model v2.0 onwards, the principal projection is the average (mean) of all the 256 Monte Carlo simulations which have been run for a scenario. In the outputs, users will see a principal projection figure for all the different types of activity which the model forecasts. This approach to determining the principal projection was adopted in July 2024. It ensures that the principal projection will sit in the middle of the activity distribution (i.e. it will be very close to the 50th percentile or median). + +For model versions up to and including v1.2, the principal projection was determined by a single model run which took the midpoint of all the parameter confidence intervals set by the user. However, given the probabilistic nature of the model, we have found that using the average of all the simulations is a more stable method of determining the principal projection. + +This is not to be confused with the ONS principal population projection. ## Summary by year