Google Floods Part 1: Creating gauge points and tooltip graphs; COUNTRY=cambodia #1328

lowriech · 2024-08-08T22:11:17Z

Description

This fulfills step 1 and 2 of 3 for #1317 (see plan).

In this PR, we're fetching flood status by gauge from Google's Floods API for the entire country (10-100 gauges per country). We're displaying these gauges as points on Prism. They're color coded by flood status and provide past and future flooding in a tooltip when clicked.

Forecasts for each gauge were added here: #1330 (merged into this branch for review)

How to test the feature:

Open the Mozambique or cambodia deployment
Enable the Google Flood layer
Click on a gauge point

Checklist - did you ...

Test your changes with

REACT_APP_COUNTRY=rbd yarn start
REACT_APP_COUNTRY=cambodia yarn start
REACT_APP_COUNTRY=mozambique yarn start
Add / update necessary tests?
Add / update outdated documentation?

Screenshot/video of feature:

382087325-f448a602-a406-4f64-b6c5-861a8bc25c49

github-actions · 2024-08-08T22:18:28Z

Build succeeded and deployed at https://prism-1328.surge.sh
(hash 51defcf deployed at 2024-12-18T23:57:24)

ericboucher · 2024-08-09T20:30:39Z

api/app/main.py

@@ -412,3 +413,10 @@ def post_raster_geotiff(raster_geotiff: RasterGeotiffModel):
    return JSONResponse(
        content={"download_url": presigned_download_url}, status_code=200
    )
+
+
+@app.get("/google-floods-gauges")


If we're gonna add a bunch of integrations, it will be worthwile to think about a standardized approach and common architecture. And maybe expose them through a sub endpoint of the API to keep things cleaner. wdyt @lowriech ?

I think that's a good question, @ericboucher. Do you see other APIs we're already leveraging fitting into this integration category (kobo, hdc stats, others)? Or are you thinking just for new integrations?

I'm wary of premature abstraction and trying to determine if I think now is the right time to align on this standardized approach.

It seems like low-hanging fruit to abstract once we decide on a pattern. So yes, agree we should think about it, but not sure about the velocity of adding entirely new providers. Would vote to wait until we see more pattern around this.

frontend/src/config/mozambique/layers.json

gislawill · 2024-08-12T22:10:28Z

@lowriech and @ericboucher, there are a couple outstanding issues (noted in the PR description) but this ready for initial review

gislawill · 2024-08-12T22:20:23Z

frontend/src/components/MapView/utils.ts

+      if (
+        itemProps.visibility === FeatureInfoVisibility.IfDefined &&
+        !properties[item]
+      ) {
+        return obj;
+      }


We have some feature properties that may or may not be defined by the google api (for example: site name, river name). This allows us to hide the property from the pop up if it's not defined

gislawill · 2024-08-12T22:22:53Z

api/app/googleflood.py

+            "river": (
+                data["river"] if "river" in data and len(data["river"]) > 1 else None
+            ),


"river" can come in as a string with just 1 space (" "). This ensures there's a real value

frontend/src/config/mozambique/layers.json

gislawill · 2024-08-16T15:39:28Z

Note: I'm adding pytest-recording to store our google flood responses in local tests and use them in CI/CD. That will fix the failing API test in this PR

…api key to docker compose

…AM/prism-app into feature/google-floods/1317

wadhwamatic · 2024-11-09T01:10:57Z

Looks good, thanks Will! I created an issue for the other (pre-existing) bug I mentioned: #1365. If it's simple to address here, please do so otherwise we can address in a separate PR

…ties

gislawill · 2024-11-12T16:32:36Z

frontend/src/components/MapView/MapTooltip/index.tsx

@@ -42,11 +49,13 @@ const useStyles = makeStyles(() =>
      marginBottom: '4px',
    },
    popup: {
+      // Overrides the default maxWidth of 240px set by react-map-gl
+      maxWidth: 'none !important',


Heads up @wadhwamatic, this fixes #1365

wadhwamatic · 2024-11-14T15:10:33Z

@gislawill - I noticed today that the charts module has a new bug on this branch. If I go to charts, select an indicator, and then drag the time slider, the chart is not updating to show the new date range even though the axis labels change.

Also, if I choose an indicator that uses a line chart (ex: rainfall anomaly), the thickness of the lines has increased compared to what was previously there.

gislawill · 2024-11-14T16:53:26Z

If I go to charts, select an indicator, and then drag the time slider, the chart is not updating to show the new date range even though the axis labels change. Also, if I choose an indicator that uses a line chart (ex: rainfall anomaly), the thickness of the lines has increased compared to what was previously there.

Thanks for catching these issues! I'll check it out.

For the second issue (thickness of the line increasing), that's a result of me purposefully making the line thicker for the flood graphs (easier to interpret imo). @wadhwamatic could you confirm you prefer the thinner line in the charts module? Do you have a stance on the weight of the floods line?

wadhwamatic · 2024-11-15T10:52:07Z

frontend/src/components/Common/Chart/index.tsx

@@ -178,7 +193,7 @@ const Chart = memo(
          fill: config.fill || false,
          backgroundColor: colors[i],
          borderColor: colors[i],
-          borderWidth: 2,
+          borderWidth: 4,


@gislawill a value of 3 works well from my perspective. let me know what you think

Looks good to me, updated

wadhwamatic · 2024-11-15T10:54:06Z

For the second issue (thickness of the line increasing), that's a result of me purposefully making the line thicker for the flood graphs (easier to interpret imo). @wadhwamatic could you confirm you prefer the thinner line in the charts module? Do you have a stance on the weight of the floods line?

@gislawill - I thought that might be the case. I personally prefer a bit thinner. I added a comment, suggesting borderWidth: 3 instead of 4. Sound / look ok to you?

It looks like this:

gislawill · 2024-11-15T22:24:02Z

@wadhwamatic, line thickness updated and time selection in the charts module updated

wadhwamatic · 2024-11-16T02:32:49Z

Thanks @gislawill. The line thickness looks good to me and the time slider issue has been resolved

gislawill · 2024-11-18T16:10:06Z

Thanks @wadhwamatic, is this good to go as far as you're concerned?

Some thoughts on potential remaining steps:

How does the copy look?
Is the layer set up optimal in each deployment (should this be in it's own layer group as currently implemented or put in an existing layer group)
We'll need translations once the copy and layer organization is complete

I'll note for context that the 502 errors from Google in RBD is not an issue on this branch. It seems to only crop up in #1332 (where I've added a cache-based workaround)

wadhwamatic · 2024-12-17T23:01:16Z

@ericboucher - can you make a final pass on this PR? It's gtg from my side. Let's please remove the google flood layers from prism.json in the demo countries before merging into main though but bring it back subsequently in #1332

ericboucher · 2024-12-19T10:03:03Z

api/app/googleflood.py

+            ],
+        },
+        "properties": {
+            "gaugeId": data["gaugeId"],


How confident are we that all the keys will be present? Should ze use a getter with a default None to be safe?
Or make sure that we have proper error handling for missing keys.

ericboucher · 2024-12-19T10:06:27Z

api/app/utils.py

@@ -38,3 +40,29 @@ def filter(self, record):
            self.warning_count += 1

        return False
+
+
+def make_request_with_retries(


Do we need something specific, or could we rely on the Retry util directly?
from requests.adapters import Retry

ericboucher · 2024-12-19T10:11:20Z

frontend/src/config/rbd/layers.json

@@ -6531,5 +6531,77 @@
      }
    ],
    "legend_text": "Joint Food Security & Nutrition Hotspot Analysis conducted by WFP, UNICEF, and partners based on SMART surveys and Cadre Harmonisé (CH) & Integrated Food Security Phase Classification (IPC) analyses"
+  },
+  "google_flood_status_at_gauges": {


Should this layer be shared ans then just override the necessary param?

ericboucher · 2024-12-19T10:12:39Z

frontend/src/utils/fetch-with-timeout.ts

@@ -8,7 +8,7 @@ interface FetchWithTimeoutOptions extends RequestInit {

 export const ANALYSIS_REQUEST_TIMEOUT = 60000;

-const DEFAULT_REQUEST_TIMEOUT = 15000;
+const DEFAULT_REQUEST_TIMEOUT = 30000;


Still needed? Is it to account for the retries?

ericboucher · 2024-12-19T10:16:45Z

frontend/src/components/Common/Chart/index.tsx

@@ -208,8 +223,21 @@ const Chart = memo(
          fill: false,
        }));
      }
+      if (data.GoogleFloodConfig) {


This PR is introducing a lot of extra complexity to this file. I'll open an issue recommending to split it by functions/services

ericboucher

Nice work overall. Let's make sure to QA all different tooltips inc. EWS.
I left a few comments in line but this looks gtm overall !

Adding Google Floods Backend

c5c1215

lowriech marked this pull request as draft August 8, 2024 22:11

ericboucher reviewed Aug 9, 2024

View reviewed changes

gislawill added 2 commits August 11, 2024 22:26

Viewing the gauges on the frontend, adding error handling to backend

b9722d3

Update comment

bc92d38

gislawill reviewed Aug 12, 2024

View reviewed changes

frontend/src/config/mozambique/layers.json Outdated Show resolved Hide resolved

Updates to support tooltip content

08a4499

gislawill changed the title ~~Adding Google Floods Backend~~ Adding Google Floods Backend + Creating a Gauge Points Aug 12, 2024

gislawill added 3 commits August 12, 2024 14:05

copy update

0653f1e

Refactor title state management

7cd866d

Fixing and adding server tests

94e1588

gislawill changed the title ~~Adding Google Floods Backend + Creating a Gauge Points~~ Adding Google Floods Backend + Creating a Gauge Points; COUNTRY=mozambique Aug 12, 2024

gislawill reviewed Aug 12, 2024

View reviewed changes

ericboucher reviewed Aug 13, 2024

View reviewed changes

frontend/src/config/mozambique/layers.json Outdated Show resolved Hide resolved

ericboucher and others added 4 commits August 13, 2024 14:19

update set_env

6ac3aad

Merge branch 'master' into feature/google-floods/1317

eb29b6e

update snapshots

6be26d7

Merge branch 'master' into feature/google-floods/1317

2fe481a

gislawill linked an issue Aug 19, 2024 that may be closed by this pull request

[Feature Request]: Google Flood Hub integration #1317

Open

gislawill added 3 commits August 20, 2024 17:29

Adding support for multi country deployments and adding google flood …

8e84cbb

…api key to docker compose

Use interpolation for popup titles

efdf2c9

Adding pytest-recording

8fa327c

gislawill marked this pull request as ready for review August 21, 2024 01:21

ericboucher added 3 commits August 21, 2024 10:38

Merge branch 'master' into feature/google-floods/1317

f5ffbf6

Update test_google_floods_api.py

438ee04

Merge branch 'feature/google-floods/1317' of https://github.com/WFP-V…

e369704

…AM/prism-app into feature/google-floods/1317

gislawill mentioned this pull request Nov 11, 2024

[Bug]: Chart tooltip window resizes in error #1365

Open

gislawill added 2 commits November 12, 2024 10:07

Fixing tooltip collapsing

66cba35

Updating api with requested changes: test comments, abstracting utili…

05188b3

…ties

gislawill linked an issue Nov 12, 2024 that may be closed by this pull request

[Bug]: Chart tooltip window resizes in error #1365

Open

gislawill reviewed Nov 12, 2024

View reviewed changes

gislawill requested a review from ericboucher November 12, 2024 16:33

wadhwamatic reviewed Nov 15, 2024

View reviewed changes

gislawill added 2 commits November 15, 2024 16:02

Update border width

daaf86c

Updates to chart line thickness and time selection

439a542

gislawill mentioned this pull request Nov 25, 2024

Google Floods Part 3: Adding Inundation Maps; COUNTRY=mozambique #1332

Open

wadhwamatic added 3 commits December 3, 2024 21:56

update text for river discharge layer

455a69e

Merge branch 'master' into feature/google-floods/1317

733efcf

Merge branch 'master' into feature/google-floods/1317

5b4e17d

wadhwamatic added 2 commits December 18, 2024 08:07

Merge branch 'master' into feature/google-floods/1317

8cf36fd

Merge branch 'master' into feature/google-floods/1317

51defcf

ericboucher reviewed Dec 19, 2024

View reviewed changes

ericboucher approved these changes Dec 19, 2024

View reviewed changes

ericboucher mentioned this pull request Dec 19, 2024

Cleanup / Split Chart/index file #1399

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Google Floods Part 1: Creating gauge points and tooltip graphs; COUNTRY=cambodia #1328

Google Floods Part 1: Creating gauge points and tooltip graphs; COUNTRY=cambodia #1328

lowriech commented Aug 8, 2024 •

edited by gislawill

Loading

github-actions bot commented Aug 8, 2024 •

edited

Loading

ericboucher Aug 9, 2024

gislawill Aug 12, 2024

lowriech Aug 13, 2024

gislawill commented Aug 12, 2024

gislawill Aug 12, 2024

gislawill Aug 12, 2024

gislawill commented Aug 16, 2024

wadhwamatic commented Nov 9, 2024

gislawill Nov 12, 2024

wadhwamatic commented Nov 14, 2024

gislawill commented Nov 14, 2024

wadhwamatic Nov 15, 2024

gislawill Nov 15, 2024

wadhwamatic commented Nov 15, 2024

gislawill commented Nov 15, 2024

wadhwamatic commented Nov 16, 2024

gislawill commented Nov 18, 2024

wadhwamatic commented Dec 17, 2024

ericboucher Dec 19, 2024

ericboucher Dec 19, 2024 •

edited

Loading

ericboucher Dec 19, 2024

ericboucher Dec 19, 2024

ericboucher Dec 19, 2024

ericboucher left a comment

Google Floods Part 1: Creating gauge points and tooltip graphs; COUNTRY=cambodia #1328

Are you sure you want to change the base?

Google Floods Part 1: Creating gauge points and tooltip graphs; COUNTRY=cambodia #1328

Conversation

lowriech commented Aug 8, 2024 • edited by gislawill Loading

Description

How to test the feature:

Checklist - did you ...

Screenshot/video of feature:

github-actions bot commented Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gislawill commented Aug 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gislawill commented Aug 16, 2024

wadhwamatic commented Nov 9, 2024

Choose a reason for hiding this comment

wadhwamatic commented Nov 14, 2024

gislawill commented Nov 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wadhwamatic commented Nov 15, 2024

gislawill commented Nov 15, 2024

wadhwamatic commented Nov 16, 2024

gislawill commented Nov 18, 2024

wadhwamatic commented Dec 17, 2024

Choose a reason for hiding this comment

ericboucher Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericboucher left a comment

Choose a reason for hiding this comment

lowriech commented Aug 8, 2024 •

edited by gislawill

Loading

github-actions bot commented Aug 8, 2024 •

edited

Loading

ericboucher Dec 19, 2024 •

edited

Loading