Skip to content

Commit

Permalink
Merge branch 'development' of github.com:hotosm/fmtm into fix/design-…
Browse files Browse the repository at this point in the history
…word-changes
  • Loading branch information
NSUWAL123 committed Dec 16, 2024
2 parents 483b8ef + 259c256 commit c011531
Show file tree
Hide file tree
Showing 41 changed files with 828 additions and 486 deletions.
100 changes: 100 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,105 @@
# Changelog

## 2024.5.0 (2024-12-11)

### Feat

- update auth cookie logic for both backend and frontend (#1974)
- **mapper**: add navigation mode to map (pitch angle / rotate) (#1973)
- **debug**: add cloudflare tunnel config to simple configurable frontend button
- **frontend**: add 'Start Mapping' call to action button on homepage cards (#1968)
- **mapper**: add new point geom in maplibre terradraw, inject to ODK collect (#1966)
- **projects**: add organisation_logo field to ProjectSummary model (#1956)
- **mapper**: add zoom to extent button to map (#1947)
- **frontend**: all uploaded submission features/geometries on submission instance page (#1931)
- **mapper**: frontend pmtile basemap management pt2 (#1925)
- **mapper**: set entity (feature) color based on it's mapping status (#1921)
- **mapper**: basemap control component + opfs pmtile support (pt1) (#1922)
- **mapper**: frontend login + remove temp auth from React frontend (#1903)
- **splitTasks**: additional entities feature count add to split tasks section (#1906)
- **mapper**: basemap layer switcher integration for maplibre (#1835)
- **submission**: add project_contributors dependency for permission … (#1873)
- **splitTasks**: add total number of features on split tasks section (#1880)
- **mapper**: map new feature in odk btn (#1879)
- **frontend**: visualise submission photos via slider (#1857)
- **backend**: API to delete user account #1661 (#1848)
- **mapper**: start improved mapper flow (#1854)
- **mapper**: flatgeobuf maplibre component for loading features (#1851)
- update API to use events and task states + fixes to backend refactor (#1838)
- **backend**: remove SQLAlchemy and replace with async psycopg db driver (#1834)
- **frontend**: mapper UI frontend refactor (#1830)
- **frontend**: mapper frontend continuation (#1823)
- mapper frontend using ElectricSQL ShapeStream (live updates) (#1760)

### Fix

- **auth**: final fixes to temp/osm auth across frontends
- finalise auth setup between frontends (#1981)
- **backend**: correctly return JSONResponse content on refresh endpoints
- apply pagination after fetch submissions from odk (#1971)
- **mapper**: remove extra semicolon from new_feature odk field injection
- **mapper**: correctly inject username into qrcode from logged in details
- **frontend**: update token refresh calls to match updates in PR #1948
- **backend**: ambiguous project_id reference in project summaries .all()
- **frontend**: update min height of project home page card, move start mapping button
- **backend**: including num_contribotors and total_tasks to the project response
- **frontend**: default do not show map on home page
- **mapper**: attempt fixing missing basemap download buttons on mobile
- **backend**: login enforced for management, additional temp login option for mappers (#1948)
- **+page**: remove user details from localStorage in case of session expiration (#1965)
- **backend**: polygon geometries; remove holes, fix right hand rule (#1961)
- Submission Instance Page, Create Project (#1958)
- odk credentials are passed without encrypting at first to avoid double encryption (#1957)
- **mapper**: better distinguish create project layer colours (#1936)
- **frontend**: remove trailing slash from activity endpoint (#1932)
- **backend**: use pydantic SecretStr for all sensitive env vars
- edit multi additional entities names replacing space with '_' (#1926)
- **backend**: resolve route ambiguity for /tasks/activity endpoint (#1924)
- **frontend**: update the body of generate-project-data;parse null as a json instead of form data (#1919)
- **mapper**: geolocation layerswitch bug (#1914)
- **mapper**: frontend enhancements, add more instructions (#1905)
- **mapper**: mobile broser map control placement fix (#1899)
- **project**: allow users to view project cards without login (#1902)
- **frontend**: project creation wording form --> survey (#1858)
- **frontend**: minor fixes including bottom sheet drag up (#1895)
- trailing slashes from endpoints, fix check_access for backend roles (#1893)
- **backend**: update check_access logic to allow checking for org_manager to a project (#1892)
- **backend**: create and delete organisation (#1867)
- **backend**: fix permissions for backend project routes (#1885)
- **backend**: building of extra cors origins, type mismatch
- **backend**: update project xlsform SQL after backend refactor (#1872)
- **mapper**: disable mobile browsers to pull-to-refresh functionality (#1878)
- **mapper**: loading of electric shapestreams, api call ordering
- **backend**: cors origin list building when DEBUG not set on localhost
- **mapper**: task comment events for mapper frontend (#1871)
- **backend**: remove auth bypass in for HOTOSM org #1785 (#1845)
- **formUpdateTab**: xFormId undefined issue fix on form update (#1863)
- **createProjectService**: replace all spaces with underscore (#1862)
- **frontend**: pass additional entity file name to backend (#1860)
- **additional-entity**: allow custom properties to create entities list (#1861)
- **backend**: s3 upload and db insertion for submission photos (#1856)
- **frontend**: after backend refactoring to events (#1844)
- mapper frontend task display and event POSTs (#1842)
- add temp ?sslmode=disable to electric url due to electric#1792
- **backend**: allow empty task id in entity statuses for new geopoint (#1822)
- dockerfile warning such as Casing and whitespace separator
- updated osm-fieldwork -> 0.16.8 (#1814)
- **files**: cleanup function remove
- **QrcodeComponent**: reduce qrcode skeleton size
- **dialogTaskActions**: add task id filter on odk redirect
- **qrcode**: increase qrcode size
- **createProject**: xlsform key update, fix customForm upload issue
- **editor**: solve editor empty issue on manageProject on initial render
- **submissionsTable**: on filter reset, set task_id to null

### Refactor

- **mapper**: improve logic for rendering basemap action buttons
- **logging**: enhance error logging with stack information across multiple modules (#1887)
- **mapper**: upgrade mapper frontend to Svelte 5 Runes (#1846)
- remove unnecessary db injection where not needed
- update link to custom ODK build (renamed to -FMTM)

## 2024.4.1 (2024-10-24)

### Fix
Expand Down
2 changes: 1 addition & 1 deletion Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ bump:
&& git config --global user.name svcfmtm \
&& git config --global user.email [email protected] \
&& cd src/backend \
&& cz bump --check-consistency'
&& cz bump --check-consistency --no-verify'

# Run docs website locally
docs:
Expand Down
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,15 +118,20 @@ Alternatively see the [docs](https://docs.fmtm.dev) for various deployment guide
|| 🖥️ view mapper submissions in the FMTM dashboard |
|| 📢 Beta Release |
|| 🖥️ & 📱 basic user tutorials and usage guides |
|⚙️| 📱 open ODK Collect with feature already selected |
|⚙️| 📱 live updates during mapping (if online) |
| | 🖥️ export (+merge) the final data to OpenStreetMap |
| | 🖥️ organization creation |
|| 📱 open ODK Collect with feature already selected |
|| 📱 live updates during mapping (if online) |
|| 📱 features turn green once mapped |
|⚙️| 📱 better support for mapping **new** points, lines, polygons |
|⚙️| 📱 navigation and capability for routing to map features |
| | 📱 integrate ODK Web Forms (to avoid switching apps) |
| | 📱 fully offline field mapping |
| | 🖥️ organization creation and management |
| | 🖥️ simplify project creation with basic / advanced workflows |
| | 🖥️ refinements to task splitting algorithm |
| | 🖥️ improvements to the validation criteria and workflow |
| | 🖥️ export (+merge) the final data to OpenStreetMap |
| | 🖥️ better data visualisation and export options |
| | 🖥️ Field Admin role |
| | 📱 fully offline FMTM (± small field-based device) |
<!-- prettier-ignore-end -->

> [!Note]
Expand Down
2 changes: 1 addition & 1 deletion chart/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ type: application
name: fmtm
description: Field Mapping Tasking Manager - coordinated field mapping.
version: "0.1.0"
appVersion: "2024.4.1"
appVersion: "2024.5.0"
maintainers:
- email: [email protected]
name: Sam Woodcock
Expand Down
1 change: 0 additions & 1 deletion docker-compose.development.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,6 @@ services:
APP_VERSION: ${GIT_BRANCH}
VITE_API_URL: https://${FMTM_API_DOMAIN:-api.${FMTM_DOMAIN}}
VITE_SYNC_URL: https://${FMTM_SYNC_DOMAIN:-sync.${FMTM_DOMAIN}}
NODE_ENV: development
volumes:
- fmtm_frontend:/frontend
network_mode: none
Expand Down
1 change: 0 additions & 1 deletion docker-compose.main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,6 @@ services:
APP_VERSION: main
VITE_API_URL: https://${FMTM_API_DOMAIN:-api.${FMTM_DOMAIN}}
VITE_SYNC_URL: https://${FMTM_SYNC_DOMAIN:-sync.${FMTM_DOMAIN}}
NODE_ENV: production
volumes:
- fmtm_frontend:/frontend
network_mode: none
Expand Down
46 changes: 45 additions & 1 deletion docs/about/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ into your language.
### What technologies are used in FMTM?

FMTM is built using several technologies, including FastAPI,
Postgres, React, and web components. The codebase is written in
Postgres, React, Svelte, and web components. The codebase is written in
Python & TypeScript mostly.

---
Expand All @@ -288,6 +288,50 @@ Please follow the instructions in the setup guide on this site.

---

### Help! Cloning the repo doesn't work for me

If you are receiving errors cloning the repo, it may be an unstable internet
connection causing issues.

First, please check your internet connection, or use another connection if
possible.

One option to reduce the amount of content that needs to be downloaded
is to do a shallow clone:

```bash
# Shallow clone only the `development` branch
git clone --depth 1 --no-single-branch https://github.com/hotosm/fmtm.git
```

To pull additional branches as you need them (for example, to work on them):

```bash
git fetch origin branch-name
git checkout branch-name
```

As an **absolute last resort**, you could try the following:

```bash
# Set HTTP version to 1.1 for stability (multiple connections vs multiplexing)
git config --global http.version HTTP/1.1

# Increase timeout to 3 minutes
git config --global http.lowSpeedTime 180

# Allow retries in case of failure
git config --global http.retry 3

# Disable integrity checks for easier resume
git config --global fetch.fsckObjects false
git config --global fetch.writeCommitGraph false

# Your clone command here...
```

---

### How can I report a bug or suggest a new feature for FMTM?

You can report bugs or suggest new features by opening an issue on
Expand Down
27 changes: 27 additions & 0 deletions docs/manuals/OData Connection for Submission Analysis
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
If you want to visualise the submissions and create custom charts
and diagrams using FMTM submissions, then you can follow the steps
below and load the submissions on any data analysis platform using OData.
OData endpoints are a standardised way to ingest
this data into other tools: PowerBI, Tableau, Redash, Grafana

Why PowerBI? You can use other tools too like Tableau, Redash, Grafana or even
Excel. However, PowerBI is free to use, very powerful, and user friendly though its Proprietary Microsoft.

The steps shows how to use powerBI and create custom visualisations.
ODK already has good docs on this which you can refer to.
https://docs.getodk.org/tutorial-mapping-households/

Step 1: Start a new project, add a new data source 'OData Feed'
https://private-user-images.githubusercontent.com/78538841/366426499-376de20e-61e5-452e-ab76-669bf1940c72.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzM5MDgwMTUsIm5iZiI6MTczMzkwNzcxNSwicGF0aCI6Ii83ODUzODg0MS8zNjY0MjY0OTktMzc2ZGUyMGUtNjFlNS00NTJlLWFiNzYtNjY5YmYxOTQwYzcyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDEyMTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMjExVDA5MDE1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThiMWYzOTBiMDQyNGEzNDI1ZjBjZDg1Mjg2ZmE0NGU4Yzg1MGQyZWIyN2Q5YjQ4MGJmZmRlYzYzMWI3OWMxZWQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.riSVz4Y2tiFXqjkaSWsmsOvLjhx8M_5YiA-LSQ7FBys
Step 2: Use the OData URLs shown in the ODK docs:
a. Submission data: /v1/projects/{projectId}/forms/{xmlFormId}.svc
e.g. https://odk.hotosm.org/v1/projects/86/forms/df9135c8-84b1-4753-b348-e8963a8b4088.svc
a. Entity data: /v1/projects/{projectId}/datasets/{name}.svc
e.g. https://odk.hotosm.org/v1/projects/86/datasets/features.svc

Step 3: Enter your credentials using Basic Auth

Step 4: Preview the data

Step 5: Transform data as needed, load into charts, graphs, etc, to create the dashboard you need.

10 changes: 10 additions & 0 deletions docs/manuals/field-mapping-examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Field Mapping Examples

Users often write their own guides for the team they are working with.

This page provides some links to user-generated guides of FMTM over time.

## Know Your City Africa: Ghana Slum Mapping

[Link](https://docs.google.com/document/d/1i6LPj3Ah860BaSQCLvmxqbLmcQB3fM0_W8n15NEeZPo/edit?tab=t.0)
to project document, including detail on the mapping workflow.
109 changes: 109 additions & 0 deletions docs/manuals/mapping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Mapper Frontend Documentation

The **Mapper Frontend** was developed to provide a simpler, more intuitive
mapping experience.

## Part (A): Setup For Mapping

### Step 1: Install The Custom ODK Collect Mobile App

- The first time you load the project, you should be prompted to download
the custom `.apk` from the sidebar.
- Once downloaded, you should install the custom ODK Collect application.

!!! tip

If you already have ODK Collect installed, you may have to uninstall
it first.

You may have to enable installing from unknown sources in your device
settings too.

### Step 2: Access the Mapper Frontend

- **Option 1:** Click the **Start Mapping** button on the project cards of
explore project page.
- **Option 2:** Click the **Start Mapping** button on the project details page.
- **Option 3:** Go to `https://fmtm.hotosm.org/mapnow/<project_id>` to open
the Mapper Frontend for a specific project.

!!! note

This functionality is designed for mappers in the field, so it is
recommended to use a mobile device to access it.

### Step 3: Configure ODK Collect (once only)

- **Option 1:** Scan the QR code displayed on the Mapper Frontend using
the custom ODK Collect mobile application.

- **Option 2:** Download the QR code and import it into ODK Collect to
load the project. Follow the steps below:

1. Open ODK Collect.
2. Click the project name in the top right corner.
3. Tap the **menu** icon (three horizontal dots).
4. Select **Import QR Code**.
5. Browse to the downloaded QR image and load the project.

!!! tip

For a demonstration of the process above, click the **i** info icon
on the QR Code tab of the mapper frontend.

### Step 4: Load Imagery (optional)

- You can load the reference base imagery by clicking on offline mode option
(second tab on bottom navigation bar).
- Choose the layer you want to load and click on **Show on Map** to visualise
the layer on map.
- The layer can also be stored for offline mapping by clicking
**Store Offline**.

## Part (B): Mapping Features

### Select An Existing Feature

In most cases we are submitting a survey about a feature that already exists.

- Click on a task area: the features within will be displayed.
- Now click on a feature you wish to map: a popup will appear.
- Now click 'Map Feature In ODK': ODK Collect will open, with the
feature pre-selected in the survey (no need to open the ODK map!).
- Complete the survey and submit.

!!! note

You may manually click 'Start Mapping' for a task area.

However, the task will be automatically locked for you
if you map a feature within the task area.

### Mapping A New Feature

Sometimes the feature does not exist on the map yet!

- Click on a task area: a popup will display.
- At the top right, there is a button **Map New Feature**.
- Click on the map to create a new geometry.
- ODK Collect will be opened automatically to fill out the survey
data for the newly created feature.

### Repeat The Process

- Once the form is submitted, you have to get back to FMTM and select the
another feature for mapping.
- You also also have to click the **Sync Status** button to see the feature
turn green.
- Select another feature to map (repeat from step 5)!

!!! note

You need internet access to update the feature status.

In a future release, this process will be handled seamlessly
without user interaction.

## Upcoming improvements

Refer the milestone: <https://github.com/hotosm/fmtm/milestone/50>
Loading

0 comments on commit c011531

Please sign in to comment.