Make XML map config primary #138
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Since package.json commands have to be single-line, the deploy and deploy-new commands are unreadably long and threatening to get longer. This moves them into scripts within the tiler source tree.
Since this is a bunch of operations sequential, a promise chain turns out to be easier to read than a series of `await` calls and intermediate variables.
Changes how map config is handled: - Makes the Mapnik XML canonical, and tracked. - Moves the Carto MML/MSS map config files into a subdirectory. They are now useful as working files, but aren't used by the running code. - Removes the compilation step from the `deploy` and `local` yarn commands. Config file compilation can still be run using the `build-xml` script, but won't be run automatically. - Removes the 'template-vars' function that was being used during XML compilation, moving the logic into a function in tiler.js. The XML files will now be templates to be filled with variables from the environment, like the Carto map config files used to be, and the substitutions will be made at runtime. - Changes the variables and how they're provided to docker-compose to support the new setup of filling them in at runtime. This includes adding commands to the `deploy` scripts to add the variables to the Lambda function's environment.
Specifies the services to be started by scripts/server so it won't keep running a terraform container that just prints errors. Also removes some vestigial code.
Adds a few parameters that speed up the PostGIS interaction. - `srid` saves Mapnik having to do a preliminary query to check - `persist_connection=false` should reduce the chance of saturating the database connections by releasing them quicker - `extent_from_subquery` saves another preliminary query by adding a query that Mapnik does to get a rough extent into the main data query as a subquery. Query.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Changes the way map configuration is handled so that rather than maintaining Carto styles that get compiled to Mapnik XML during build/deploy, we'll maintain actual Mapnik XML. Based on making the same move in the People For Bikes version, as recommended on issue 129.
Maintaining the config as XML has the major advantage that it allows adding configuration/optimization parameters to the layer config that aren't supported by Carto. The parameters added here helped improve the performance of PFB's tiles from many seconds to the 300-700ms range. (Different situations will differ, but the ability to tune the database parameters is likely to be important in most or all cases.)
It also reduces the number of moving parts that might do something unexpected. And it doesn't preclude working with Carto layer styling, since the
build-all-xmlcommand will compile the Carto style files into XML just as before. The difference being that now the XML files are tracked and the generated file will generally be the input to a manual process of validating and improving it.Since compiling the config is no longer required and the XML will be tracked in the repo, filling in environment variables during config compilation won't work. So this also transitions to using environment variables in the Mapnik XML and replacing them with values from the environment at runtime. This should also make it easier to support deployment across multiple environments (e.g. local, staging, production).
Notes
I originally had more changes in this branch, toward facilitating deployment across multiple environments as mentioned above, but I decided to do that separately.
Testing Instructions
scripts/deploy) should also still work as before.