Skip to content

Releases: hotosm/fmtm

2024.5.0

12 Dec 01:24
@spwoodcock spwoodcock
2024.5.0
53dd8a4
Compare
Choose a tag to compare
2024.5.0 Latest
Latest

FMTM 2024.5.0 Mapper Frontend

3 months in the making, we are excited to release our new mapper frontend!

The frontend is specifically targeted at a streamlined and more intuitive mapping workflow, on mobile devices, with significant improvements to the mapping experience such as real-time data syncing between users and colored features based on status.

To access this feature, click the 'Start Mapping' button on the home page card for a project.
Alternatively, a project manager may distribute the 'mapping' URL to users, via any means such as email and messaging apps.

We would also like to announce @Anuj-Gupta4 joining the FMTM dev team, working on improvements to the backend, and features into the future 🚀

🌟 Highlight: Mapper Frontend

  • New UI: A fresh UI has been implemented for easier navigation, with clearly defined call to action for field users.

  • Colored features based on status: A long awaited feature, task areas and map features have specific colors based on their status. For example, mapped buildings turn green, in progress is yellow, and if there is a mapping issue red.

  • Add new map features: often the feature you wish to survey does not exist on the map yet! We now allow the user to collect new point features as part of a survey.

  • Near real time update of feature status: to facilitate better coordination between mappers, the status of tasks or features mapped is updated seamlessly in the background (without a manual refresh), informing the user of ongoing mapping activities in the project.

  • Better handling of generated basemap imagery: Better handle basemaps and geolocation for easier mapper experience: We’ve optimized basemap and geolocation handling on mapper frontend for a more seamless mapping experience.

  • Workflow prompts: On first load, the user is prompted to scan the QR Code, click on a task area, and get mapping. This should help to ease the learning curve for the mapper in the field.

🚀 Coming soon for the Mapper Frontend

  • Enhanced Navigation and Routing: We are working on improving navigation and adding routing options to assist mappers in the field.
  • Offline Support: In the longer term, we are planning to support offline functionality, enabling field mappers to update feature statuses while offline.

🛠️ Improvements

  • Make submission data downloads only possible for project contributors: Submission data downloads are now restricted to project contributors, enhancing data security.
  • Associated geometry selection during survey: Normally a single map feature is selected to survey. But some workflows require the selection of 'associated geometries' too. During project creation additional geojson datasets can be uploaded to facilitate this.
  • Removed the 'temporary login' feature from the main application. This is now available only on the mapper frontend, and is handled seamlessly as the default login mechanism (no user interaction required). To have edits attributed to your OSM user, login is still required.
  • Injecting mandatory FMTM fields into custom XLSForms: A few questions related to feature verification have been added at the end of the survey, for easier data conflation later.
  • Huge refactor of the backend code: For any developer contributors out there, this is a major change! Hopefully resulting in a more stable and easily maintainable backend into the future. We also added extra test cases, particularly around the submission workflow.
  • Fully activate user roles: Previously role based authentication was only partially enabled, with a bypass in place for the HOTOSM organization during the Beta. It is now fully active for all organizations.

🐛 Bugs Fixed

  • Unable to Create or Delete Organisation: Resolved an issue where users encountered the error "Failed to create organisation" while attempting to create an organisation. Organisation deletion issues from API documentation have also been fixed.
  • Task Split Page Crash: Fixed a crash issue that occurred when users selected the "Divide on Square" option for task splitting.
  • Addressed a problem where uploading a custom form resulted in the default form on custom ODK collect instead of the intended custom form.
  • Resolved issue while updating XLSForm in the manage project section.
  • Adding a new geopoint in the survey, now correctly creates an ODK Entity, as previously this would fail.
Technical Summary

What's Changed

Read more

Contributors

spwoodcock, NSUWAL123, and 3 other contributors
Assets 2
Loading

2024.4.1

24 Oct 13:31
@spwoodcock spwoodcock
2024.4.1
342e0ed
Compare
Choose a tag to compare
2024.4.1

FMTM 2024.4.1 Patch Release

Various bugfixes built on the 2024.4.0 release.

🐛 Bugs Fixed

  • The custom XLSForm upload wasn't being used (default form was loaded instead).

  • Fixes to the functionality to add new geometries during mapping.

  • Minor QRCode tweaks & fixes on the submission table.

Technical Summary

What's Changed

Full Changelog: 2024.4.0...2024.4.1

Contributors

NSUWAL123 and Sujanadh
Assets 2
Loading

2024.4.0

24 Sep 10:45
@spwoodcock spwoodcock
2024.4.0
7d39a97
Compare
Choose a tag to compare
2024.4.0

FMTM 2024.4.0 Launch

Hey everyone, we’re excited to announce that FMTM has moved out of its beta release and is now available as a fully functional product!

The FMTM Public Beta organisation has now been renamed to HOTOSM.

While the core functionalities are solid, we continue to work on smaller bug fixes, feature improvements, and enhancements to ensure a smoother user experience.

🌟 New Features

  • Flexible custom XLSForm upload: previously uploading a custom XLSForm with custom survey questions was a hassle. Now any uploaded XLSForm should work, with the specific questions required by FMTM being injected into the form automatically. Simply create an XLSForm survey as normal, upload, and see the magic happen.

  • Add new geometries from ODK Collect: if a building has not been mapped in the field, it is now possible to add a new 'geopoint' for the feature location in ODK Collect. This feature will be displayed in the submissions on FMTM, along with survey data collected.

    signal-2024-10-16-103655

  • Select multiple geometries: during project creation it is now possible to upload an additional geometry set, for later selection in ODK Collect. For example, uploading buildings as the main geometries, alongside a road dataset as an additional geometry. In ODK Collect it is then possible to select the building, then also select the adjacent road to the building.

    image

🛠️ Improvements

  • Added a 'Project Not Found' page when the user navigates incorrectly:

    image

  • Improve GeoJSON handling of different types on upload: Multi-geometries, FeatureCollections, single geometries, anything you want!

  • Make user uploaded images from ODK Collect more accessible from within FMTM. In our next release you should be able to browse the photos from the FMTM frontend 😄

  • Added comprehensive testing to the UI and backend, so hopefully you should see a more stable FMTM with less errors introduced in future releases!

🐛 Bugs Fixed

  • Resolved an issue where the Infographics page displayed the logged-in account as the contributor instead of the actual contributor.

  • Updated user-facing content by rewording 'Data Extract' to 'Map Features' for clarity.

  • Improved validation for inputting TMS URLs when adding custom imagery.

Technical Summary

What's Changed

New Contributors

Full Changelog: 2024.3.2...2024.4.0

Contributors

emi420, azharcodeit, and 4 other contributors
Assets 2
Loading

2024.3.2

30 Jul 23:16
@spwoodcock spwoodcock
2024.3.2
a5ba928
Compare
Choose a tag to compare
2024.3.2

FMTM 2024.3.2 Patch Release

A minor release stabilising FMTM prior to webinar and wider public communication.

🛠️ Improvements

  • Faster project creation: we significantly improved the project creation workflow to optimise for speed. Creation should complete in under 10 seconds for most projects of any size.

  • Layer opacity: reduce the opacity of the task overlay with satellite base: The high opacity of task colour hindered field mappers to view the base layers clearly. Reducing the opacity has greatly improved the user's experience.
    image

  • Form question ordering: we moved the digitisation related questions to the end of all forms, based on field-user feedback. Typically a mapper will interact with the owner / ask questions, prior to assessing the building footprint and trespassing on land.

  • Project generation failure: if project generation fails, the remaining steps are now halted and the steps so far are reverted. This allows the user to correct their input and try again.

  • Form updates: form category and uploaded form should be pre populated while editing project form: While editing the form, the form category was not being populated causing confusion for users on form category. However, the form category can’t be edited because it has some serious impact on task splitting and data extraction activities.
    image

  • ODK token refresh: submitting data from ODK Collect uses a special authentication token. This token can now be refreshed if needed, if the token is lost of suspected to be compromised.

🐛 Bugs Fixed

  • 'Map Feature in ODK' button is enabled if the task area is locked: For some field mappers who work on groups may require mapping single building within locked tasks too. FMTM now offers users to map a few features within locked tasks to be mapped from ODK.
  • Error message during submission download when there is no submission on that project: The vague error message saying “something went wrong” or “The data can’t be downloaded” used to be shown to the user when there is no submission within that project. This is now fixed by providing a clear error message.
  • Parsing of hashtags was sometimes incorrect on varied input. It should now be consistent.
  • Improved GeoJSON parsing, allowing for more flexible input without failures: MultiPolygon, Feature, FeatureCollection, etc.
  • Fixed MBTile / PMTile generation when using a custom TMS URL - this was not always working previously.
  • S3-based objects related to a project were not deleted on project deletion - they are now.
Technical Summary

What's Changed

Full Changelog: 2024.3.0...2024.3.2

Contributors

azharcodeit, pre-commit-ci, and 4 other contributors
Assets 2
Loading

2024.3.1

24 Jun 12:01
@spwoodcock spwoodcock
2024.3.1
f182207
Compare
Choose a tag to compare
2024.3.1

FMTM 2024.3.1 Patch Release

A minor release fixing a few minor issues on 2024.3.0.

🛠️ Improvements

  • Upgrade to React 19 for performance improvements and better code maintainability.

  • Added a cookie acceptance banner to comply with GDPR:

  • Addition of Healthcare category: We have added a healthcare category alongside buildings for users to perform field mapping of healthcare too. More categories will be added back in with future releases.

  • Fixed page redirection after successful login: Users are now redirected to the page they requested to go after signing in to FMTM. Previously, users would be redirected to the homepage after signing in no matter where they were requesting to reach.

  • Fixes to loading a pre-selected feature in ODK Collect directly from the FMTM frontend. Previously this functionality would not always load the correct form, but now it works well with the custom ODK Collect build (until agreed and merged into the official app)

🐛 Bugs Fixed

  • Display task card even if no features in a task: Users are now able to see the list of tasks even though the tasks contains no features.
    image

  • Colour of the tasks are changed according to the tasks status: The problem of tasks colour not being updated after refreshing the page has been addressed to change them with respect to legend.

Technical Summary

New Contributors

Full Changelog: 2024.3.0...2024.3.1

Contributors

azharcodeit, pre-commit-ci, and 2 other contributors
Assets 2
Loading

2024.3.0

28 May 10:14
@spwoodcock spwoodcock
2024.3.0
9fac3fe
Compare
Choose a tag to compare
2024.3.0

FMTM 2024.3.0: ODK Entities

Three months since the last release, this is a major FMTM increment.

There has been a complete re-write of the underlying link to ODK
(ODK Entities were released at the end of 2023, with rapid development since then).

🌟 New Features

  • Loading a feature in ODK Collect by clicking a button:

    1. First the custom build of ODK Collect that allows this must be installed (first uninstall your current version):

    2. Then ensure your project QR Code is loaded in ODK Collect at least once:

    3. Each map feature that is clicked on will display the 'Map Feature in ODK' button:

    4. Clicking this button will open ODK Collect, with the feature pre-selected in the form:

  • Caching offline basemaps directly in the browser:
    The PMTiles format can be viewed when the user is offline or in the field, if they cache the basemap by clicking the ‘bolt’ icon below:

  • Mapping is now possible without an OSM account
    We found this was a barrier for some mappers, so removed the dependency entirely.
    FMTM is still based on OSM and uses OSM accounts for project management.
    A ‘temporary login’ method was added for mappers wishing to just get started and map!

  • Extendable survey forms
    Options to download and edit the the survey (XLSForm) interactively have been added.
    Interactive editing allows for a spreadsheet like interface on the web.

🛠️ Improvements

  • Easier basemap configuration
    For project managers, they can specify a TMS URL for basemap generation during project generation, which will be pre-filled when the user attempts to download a basemap.
    This avoids the need to distribute the TMS URL to mappers via other communication channels.

    image

    image

  • Integration of Concept of entities in FMTM Workflow
    This enables loading ODK Form with features pre-selected.

  • Refinements to the FMTM splitting algorithm
    Better error handling in various scenarios and more accurate splitting.

  • Validate features being mapped sufficiently before task completion is possible

    image

  • Replaced OSM login popup with a login within the same web page, reducing possible issues with popup blockers.

  • Technical feature for self-hosted FMTM: added option for users to self-host FMTM with Kubernetes (via Helm Chart).

Technical Summary

Feat

  • add link for interactive xlsform editing during project creation (#1480)
  • update PWA config with 2023 icon requirements and caching (#1474)
  • download submissions in geojson (#1517)
  • add OpenTelemetry configuration for backend API (monitoring) (#1496)
  • prompt user if task area is not fully mapped on mark complete (#1493)
  • use task index as an user facing task_id (#1470)
  • frontend: login options to frontend, OSM or temp auth (#1458)
  • frontend buttons to load Entities in ODK Collect by intent (#1449)
  • distinguish between tasks locked by the current user and tasks locked by others (#1469)
  • append extra hashtag to projects with domain and project id identifier (#1454)
  • endpoints for getting Entity data & updating Entity mapping status (#1445)
  • temporary authentication login for mappers (svcfmtm) (#1410)
  • add /projects/features endpoint for project FeatureCollection (disaster.ninja integration) (#1442)
  • Playwright integration with test cases to be written (#1433)
  • implement ODK Entities for project creation (#1383)

Fix

  • lock map feature if 'Map Feature In ODK' clicked (#1516)
  • pmtile basemap generation and remove temp workarounds (#1535)
  • backend: generate location_str in model_validator instead of computed_field (#1534)
  • backend: validation_exception_handler pass all required params
  • valid matomo tracking script
  • pass project aoi as a outline geojson (#1533)
  • typo error in project name prefix in submission-download-geojson (#1523)
  • download submissions(csv,json) & refactor submission endpoints (#1519)
  • populate task_id correctly in feature properties (#1515)
  • refactored project dashboard and submission graph (#1509)
  • dialogTaskActions: same name variable conflict solve (#1506)
  • backend: use task ids to count validated and mapped status (#1485)
  • backend: get contributors count in project summary (#1484)
  • task hover popup & disable splitting algo without linestring (#1481)
  • login methods after temp cookie auth (#1471)
  • frontend: do not call introspect endpoint on /osmauth callback
  • basemap tile download, refactor to use /{project_id}/name (#1467)
  • frontend: invalidate login if mismatch between existing and new user login (#1462)
  • role mapper to login required in data extract (#1450)
  • fix list-forms endpoint logic to return list of dicts
  • activity comment api calls in frontend (#1435)
  • handle multipolygon geometries for project area (#1430)
  • XLSForm template download endpoint for specified categories (#1441)
  • filter task_history endpoint using task_id (#1436)
  • update logic for more flexible submission json --> geojson
  • add optional auth to raw-data-api calls, plus folder structure for persistence (#1431)
  • project details mobile UI, user details in header (#1407)
  • backend: allow missing odk_token for project, but log warning
  • add odk_token from projectInfo to qrcode creation
  • project name editing validation (#1416)
  • return proper error message if form edit is invalid (#1415)
  • default UNDERPASS_API_URL no trailing slash
  • more informative browser tab titles/details (#1411)
  • update task activity endpoint (#1406)
  • project edit form update validation (#1397)

Refactor

  • remove missed warning log from update_survey_form
  • remove broken submission code no longer used
  • use HTTPStatus enum for helper route response codes
  • fix ambiguous typos preventing pre-commit codespell passing
  • run codespell spelling fixer
  • replace unecessary outline_centroid for tasks with ol.extent.getCenter (#1447)
  • merged multi polygon to single polygon (#1426)
  • task comment response and schema (#1423)
  • response of update task status and added user info in task history (#1419)

Full Changelog: 0.1.0...2024.3.0

Assets 2
Loading

2024.2.0

03 Apr 16:33
@spwoodcock spwoodcock
2024.2.0
04825eb
Compare
Choose a tag to compare
2024.2.0

FMTM 2024.2.0: Mobile Mapper Experience

🌟 New Features

  • Significant improvements to the mobile experience.

    • Geolocation and orientation of the user on the map.
    • Downloading basemaps into offline storage in your browser, enabling offline navigation.
      image

  • Add comments to tasks.
    image

  • Add instructions to projects.
    image

  • Review state integration for validators.
    image

🛠️ Improvements

  • Editing of an uploaded or drawn project AOI.

  • Mobile layout for various components.

  • Edit existing ODK form & validate.

  • Improved documentation for backup and restore of production FMTM data.

  • Project creation ODK form upload: replace radiobutton with checkbox for better user experience.
    image

  • Performance and maintainability improvements by using raw SQL queries.

🐛 Bugs Fixed

  • Refinements to splitting algorithm to handle edge cases.
  • Use centroids for geometry division amongst tasks, to ensure all are uploaded to ODK Central (if they fall on boundaries).
  • Profile image not updating in user profile after login.
  • Page crashes in various scenarios, in different web browsers.
  • Determine login state on first page load, to invalidate user session if required.
Technical Summary

What's Changed

Full Changelog: 2024.1.0...2024.2.0

Contributors

varun2948, allcontributors, and 3 other contributors
Assets 2
Loading

2024.1.0

05 Mar 17:53
@spwoodcock spwoodcock
2024.1.0
658a430
Compare
Choose a tag to compare
2024.1.0

INTRODUCTION

Hello everyone!

With the first release of the year, we are thrilled to present you with the Beta Release of FMTM which incorporates many refinements.

UI/UX Enhancements

Redesign of UI for easier usage and better engagement
The user interface for project creation, project pages, and submission pages has been revamped to enhance the overall user experience.
image
image

NEW FEATURES

VIEW PROJECT SUMMARY
The infographics page has been added for project managers to track the status of the project.
The submissions are visualised in the form of various charts and graphs. You can filter submissions by submission date. . Visualise charts on project progress, number of tasks mapped and validated per project infographics for better visualisation. .
Screenshot from 2024-03-06 11-25-05

Screenshot from 2024-03-06 11-24-56

Screenshot from 2024-03-06 11-25-11

VIEW SUBMISSIONS
The list of individual submissions in a project can be viewed in a tabular view.
image

REGISTER YOUR ORGANIZATION
You can register your organization and create your own projects.
image
image

EDIT PROJECT DETAILS
You can update your survey form in a project and modify basic project details to realign with your needs.

INTEGRATION OF BACKEND MODULARIZED CODE

  • Refinement of our backend module code that underpins FMTM.
  • Integration of osm-rawdata, which uses HOT's raw-data-api service underneath.
  • Minor improvements of fmtm-splitter to account for railways and airstrips as linear splitting features.
  • Replacing HTTP BasicAuth in ODK Central connections from osm-fieldwork with session token authentication.

BUG FIXES:

  • Optimization of task splitting algorithm: Split tasks based on polygon and line features for eg: split road network for road survey
  • Project creation workflow has been made simpler and smoother with introduction of better UI/UX improvements and setup processes.

Upcoming Features:

We are planning to incorporate user roles and permissions for the next release with many more functionalities and bug fixes.

Stay tuned!

New Contributors

Full Changelog: 0.1.0...2024.3.0

Your contributions will undoubtedly enhance the project, and we're excited to have you on board. 🤝

Contributors

cordovez, manjitapandey, and Prajwalism
Assets 2
Loading

v0.1.0

08 Nov 05:03
@spwoodcock spwoodcock
0.1.0
5e5c4ee
Compare
Choose a tag to compare
v0.1.0

Field Mapping Tasking Manager v1.0 - "First Release 🚀 2023"

Release Date: November 7, 2023

Introduction
We are pleased to present our first consolidated release note for "Field Mapping Tasking Manager". Although this isn't our first deployment, we have gathered the key changes and updates made in previous deployments into this comprehensive release note.

It's wonderful to have a dedicated and expanding team 🌟 working on the development of Field Mapping Tasking Manager.

@spwoodcock as Tech Lead for FMTM
@susmina94 as Project Manager

@varun2948 as Senior Frontend Developer
@nrjadkry as Senior Backend Developer

@robsavoye as Senior Tech Lead at HOT

They have been unwavering in their commitment to the development, playing integral roles in its success.
Together, as a team, everybody's collective efforts are greatly appreciated. 🚀👥🌟

In the ~2 months since the beta release, we have the following updates:

🌟 New Features:

  1. Upgraded database management capabilities
  • Finalising the first version of the database schema.
  • Automated schema migrations on FMTM startup.
  • New migration files are located in src/backend/migrations.
  1. Added frontend and backend test cases for the following functionalities:
  • Project creation with various parameters.
  • ODK Central project creation.
  • Converting database project objects to application-specific representations.
  • Uploading custom data extracts for project tasks.
  1. Download imageries from different sources such as (ESRI, Google, OSM, Bing, Topo) in different output formats: (PMTiles, OSMAnd, MBTiles)
    You may use these files as offline basemaps in ODKCollect or any other supported medium.

image

  1. FMTM also supports input of custom TMS url and its conversion into the different file formats (PMTiles, OSMAnd, MBTiles)

image

  1. You can now view the clusters for each project location on map interface

image

🐛 Bug Fixes:

  1. Optimized Mobile UI for Project Info, Home and Organization page

image

image

  1. FMTM uses osm-rawdata instead of osm-fieldwork for extracting OSM data extracts

🛠️ Improvements/Enhancements:

  1. Introduced a user-friendly UI workflow for creating projects.

Features already included in the beta release:

  • Login to FMTM via OSM account

  • Create new projects which include:

  • Upload/draw custom AOI

  • Use XLSforms compatible with ODK

  • Upload custom/OSM data extracts

  • Split project area into tasks via three different algorithms:

    • Divide as square,
    • Choose area as task,
    • Use splitting algorithm

  • Create and register new organization

We are excited about this release and look forward to your feedback and further improvements in future versions.

NEW CONTRIBUTORS

@Sujanadh @JoltCode @NSUWAL123
Your contributions will undoubtedly enhance the project, and we're excited to have you on board. 🤝

Contributors

varun2948, nrjadkry, and 6 other contributors
Assets 2
Loading