Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable all platforms again #976

Draft
wants to merge 99 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
7e93421
proj6 - WIP
artemp Feb 10, 2021
f308203
Merge branch 'master' into proj6
artemp Mar 22, 2021
0f0dcbc
Update test data to use proj6 format
artemp Apr 17, 2021
12b2adc
add vcpkg
mathisloge Jul 2, 2021
3d685ee
getting started
mathisloge Jul 2, 2021
0e7bcc1
enable compilation
mathisloge Jul 2, 2021
cce4830
first successful build
mathisloge Jul 2, 2021
4041d9d
remove deps replaced by vcpkg
mathisloge Jul 2, 2021
4c3124e
remove lock
mathisloge Jul 2, 2021
9a84454
remove wrong var
mathisloge Jul 2, 2021
31bd956
update deps
mathisloge Jul 2, 2021
1855a94
add some deployment
mathisloge Jul 2, 2021
27ae60b
generate mapnik_settings.js
mathisloge Jul 2, 2021
6ba2b19
correct settings paths
mathisloge Jul 2, 2021
9bf9199
fix prebuild
mathisloge Jul 2, 2021
88b0a2a
add publish binary
mathisloge Jul 4, 2021
13feaa8
add some debug output
mathisloge Jul 4, 2021
869fdf6
add proj copy
mathisloge Jul 4, 2021
76c9062
copy dlls on windows
mathisloge Jul 5, 2021
0875547
copy dlls from executables
mathisloge Jul 5, 2021
cae1ae2
remove not needed script
mathisloge Jul 5, 2021
ca07cf6
update vcpkg
mathisloge Jul 7, 2021
1a2ecdf
fix cmake
mathisloge Jul 7, 2021
c055b4e
some fixes
mathisloge Jul 7, 2021
f2d7231
only use vcpkg on windows
mathisloge Jul 8, 2021
c510aed
update vcpkg
mathisloge Jul 25, 2021
fda4399
update cmake-js
mathisloge Jul 25, 2021
75c16f2
update vcpkg to microsoft repo
mathisloge Sep 17, 2021
548175e
update modules
mathisloge Sep 17, 2021
b3d9b3d
comment out proj
mathisloge Sep 17, 2021
c1df593
use correct define for MAPNIK_USE_PROJ
mathisloge Sep 19, 2021
999290a
update some deps
mathisloge Oct 6, 2021
9648c83
update
mathisloge Oct 11, 2021
c8be847
add preinstall step to clone submodules
mathisloge Oct 11, 2021
759cd30
remove install step
mathisloge Oct 11, 2021
b5e8c6a
Merge branch 'master' into proj6
artemp Oct 13, 2021
3ea68bc
update vcpkg
mathisloge Oct 13, 2021
f304f81
Merge branch 'proj6' of https://github.com/mapnik/node-mapnik
mathisloge Oct 13, 2021
ce68212
update vcpkg
mathisloge Oct 13, 2021
f72328e
use generator expr. to generate the settings file
mathisloge Oct 13, 2021
af857e9
better copy deps
mathisloge Oct 14, 2021
9fcf8ac
update vcpkg
mathisloge Oct 14, 2021
d78d0e4
prevent UB
mathisloge Oct 15, 2021
f21d963
node module is a module not a shared
mathisloge Oct 15, 2021
044f019
add threads
mathisloge Oct 19, 2021
dcc0517
add dlls to search path
mathisloge Oct 19, 2021
4717d5c
update vcpkg
mathisloge Oct 19, 2021
91c9edc
update vcpkg to latest master
mathisloge Oct 20, 2021
678ed28
DEBUG: use docker
mathisloge Nov 15, 2021
62d6473
Merge branch 'master' of https://github.com/mapnik/node-mapnik
mathisloge Nov 15, 2021
970e2fc
docker: delete build dir
mathisloge Nov 15, 2021
eb26634
update vcpkg
mathisloge Nov 15, 2021
26ec085
better docker image
mathisloge Nov 16, 2021
068944b
better docker file
mathisloge Nov 16, 2021
545a2b8
remove `cmake-js`
mathisloge Nov 19, 2021
7f5b2e3
strip whitespaces
mathisloge Nov 19, 2021
392b624
build with node-pre-gyp
mathisloge Nov 19, 2021
a43e670
update package.json
mathisloge Nov 19, 2021
bca8a1b
linux support
mathisloge Nov 19, 2021
35be137
wip
mathisloge Nov 19, 2021
c6d6648
use cmake version of mapnik-vector-tile
mathisloge Nov 21, 2021
2b2146f
update remote mapnik_vector_tile
mathisloge Nov 21, 2021
49b02c6
fix
mathisloge Nov 21, 2021
d5e99bc
remove make as gen
mathisloge Nov 21, 2021
6700106
update vcpkg
mathisloge Jan 18, 2022
3597553
add targets to package.json
mathisloge Jan 18, 2022
a8d064f
extract post build generation file into an actual file
mathisloge Jan 18, 2022
4b0f0a4
formatting
mathisloge Jan 18, 2022
868443c
remove console.log
mathisloge Jan 18, 2022
21f28c2
add npm publish and test workflow
mathisloge Jan 18, 2022
c20e503
set name for workflow
mathisloge Jan 19, 2022
51230f4
version bump
mathisloge Jan 19, 2022
753bf7d
use latest vcpkg
mathisloge Jan 19, 2022
0fe5e8f
dont run the tests for now
mathisloge Jan 19, 2022
f60f23f
use old baseline
mathisloge Jan 19, 2022
391319a
use older vcpkg
mathisloge Jan 19, 2022
d495a7d
newer vcpkg
mathisloge Jan 21, 2022
025274c
remove vcpkg
mathisloge Jan 21, 2022
47cb33b
use newer node-pre-gyp-github
mathisloge Jan 21, 2022
829f168
remove baseline
mathisloge Jan 21, 2022
ca23f42
collect logs
mathisloge Jan 21, 2022
029a626
remove duplicate input-geojson in manifest
mathisloge Jan 21, 2022
e29ac46
readd vcpkg for development
mathisloge Jan 22, 2022
815cbd9
Revert "readd vcpkg for development"
mathisloge Jan 22, 2022
f360272
test vcpkg gdal fix
mathisloge Jan 22, 2022
ff62aeb
run only windows for debugging
mathisloge Jan 22, 2022
4cec3f3
windows has not enough disk space. disable
mathisloge Jan 22, 2022
9762ed5
bump version
mathisloge Jan 22, 2022
6b8b28b
bump version
mathisloge Jan 22, 2022
a937665
fix publish script
mathisloge Jan 22, 2022
c444664
fix publish
mathisloge Jan 22, 2022
ba62427
remove publish
mathisloge Jan 22, 2022
d1100be
unix improvements
mathisloge Jan 22, 2022
f5fa7a2
windows fixes
mathisloge Jan 22, 2022
e6a8021
wrong toolchain
mathisloge Jan 22, 2022
8e0300b
use vcpkg
mathisloge Feb 14, 2022
9366943
remove rpath
mathisloge Feb 15, 2022
641f560
ubuntu install sys deps
mathisloge Feb 15, 2022
2e63b33
install make
mathisloge Feb 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.vscode
.git
build
build-tmp*
node_modules
package-lock.json
vcpkg
lib/binding
81 changes: 81 additions & 0 deletions .github/workflows/npm-publish-github-packages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: Node.js Package

on:
release:
types: [created]

env:
VCPKG_SHA: 5ef52b5b75887fb150711f5effb221dd98b99e6f,
NUGET_REGISTRY: https://nuget.pkg.github.com/mathisloge/index.json
NUGET_USERNAME: mathisloge
VCPKG_BINARY_SOURCES: "clear;nuget,GitHub,readwrite"
VCPKG_NUGET_REPOSITORY: https://github.com/mathisloge/vcpkg-nuget.git
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
#- os: windows-latest
# architecture: x64
# node_version: 16

- os: ubuntu-latest
mono: mono
architecture: x64
node_version: 16

#- os: macos-latest
# mono: mono
# architecture: x64
# node_version: 16

permissions:
contents: read
packages: write

steps:
- uses: ilammy/msvc-dev-cmd@v1

- uses: actions/checkout@v2

- name: checkout vcpkg
uses: actions/checkout@v2
with:
repository: "dg0yt/vcpkg"
ref: gdal-paths
path: vcpkg

- name: "Setup vcpkg"
shell: bash
run: ./vcpkg/bootstrap-vcpkg.sh

- name: "Setup NuGet Credentials"
shell: "bash"
run: >
${{ matrix.mono }} `./vcpkg/vcpkg fetch nuget | tail -n 1`
sources add
-source "${{ env.NUGET_REGISTRY }}"
-storepasswordincleartext
-name "GitHub"
-username "${{ env.NUGET_USERNAME }}"
-password "${{ secrets.NUGET_REGISTRY_PAT }}"

- name: "Setup NuGet apikey"
shell: "bash"
run: >
${{ matrix.mono }} `./vcpkg/vcpkg fetch nuget | tail -n 1`
setapikey "${{ secrets.NUGET_REGISTRY_PAT }}" -Source "${{ env.NUGET_REGISTRY }}"

- uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node_version }}
architecture: ${{ matrix.architecture }}
registry-url: 'https://registry.npmjs.org'

- run: npm install

- name: upload release assets
run: npm run postpublish
env:
NODE_PRE_GYP_GITHUB_TOKEN: ${{secrets.NUGET_REGISTRY_PAT}}
83 changes: 83 additions & 0 deletions .github/workflows/npm-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Node.js Test

on:
push:

env:
VCPKG_SHA: 5ef52b5b75887fb150711f5effb221dd98b99e6f
NUGET_REGISTRY: https://nuget.pkg.github.com/mathisloge/index.json
NUGET_USERNAME: mathisloge
VCPKG_BINARY_SOURCES: "clear;nuget,GitHub,readwrite"
VCPKG_NUGET_REPOSITORY: https://github.com/mathisloge/vcpkg-nuget.git
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
#- os: windows-latest
# architecture: x64
# node_version: 16

- os: ubuntu-latest
mono: mono
architecture: x64
node_version: 16

#- os: macos-latest
# mono: mono
# architecture: x64
# node_version: 16

steps:
- name: Install required system packages
shell: "bash"
run: |
sudo apt-get install -y gperf libxxf86vm-dev ninja-build make
- uses: ilammy/msvc-dev-cmd@v1

- uses: actions/checkout@v2

- name: checkout vcpkg
uses: actions/checkout@v2
with:
repository: "mathisloge/vcpkg"
ref: test/mapnik-static
path: vcpkg

- name: "Setup vcpkg"
shell: bash
run: ./vcpkg/bootstrap-vcpkg.sh

- name: "Setup NuGet Credentials"
shell: "bash"
run: >
${{ matrix.mono }} `./vcpkg/vcpkg fetch nuget | tail -n 1`
sources add
-source "${{ env.NUGET_REGISTRY }}"
-storepasswordincleartext
-name "GitHub"
-username "${{ env.NUGET_USERNAME }}"
-password "${{ secrets.NUGET_REGISTRY_PAT }}"

- name: "Setup NuGet apikey"
shell: "bash"
run: >
${{ matrix.mono }} `./vcpkg/vcpkg fetch nuget | tail -n 1`
setapikey "${{ secrets.NUGET_REGISTRY_PAT }}" -Source "${{ env.NUGET_REGISTRY }}"

- uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node_version }}
architecture: ${{ matrix.architecture }}
registry-url: 'https://registry.npmjs.org'

- run: npm install --build-from-source
#- run: npm test

- uses: actions/upload-artifact@v2
if: failure()
with:
name: vcpkg-logs
path: vcpkg/buildtrees/**/*.log
retention-days: 2
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
.DS_Store
archive
build
build-tmp*
build-cmake
lib/binding
lib/mapnik_settings.js
node_modules
Expand All @@ -27,3 +29,5 @@ mason-postgis-config.env
local-postgres/
postgres.log
local-unix-socket/
package-lock.json
prebuilds/
6 changes: 3 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[submodule "deps/wagyu"]
path = deps/wagyu
url = https://github.com/mapbox/wagyu.git
[submodule "deps/geometry"]
path = deps/geometry
url = https://github.com/mapbox/geometry.hpp.git
[submodule "deps/protozero"]
path = deps/protozero
url = https://github.com/mapbox/protozero.git
[submodule "deps/wagyu"]
path = deps/wagyu
url = https://github.com/mapbox/wagyu.git
3 changes: 3 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ bench
node_modules
lib/binding
build
build-tmp*
test
benchmark
configure
Expand Down Expand Up @@ -37,3 +38,5 @@ deps/protozero/test
mason-postgis-config.env
local.env
postgres.log
vcpkg
.vscode
7 changes: 7 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"files.associations": {
"iosfwd": "cpp"
},
"C_Cpp.default.configurationProvider": "ms-vscode.cmake-tools",
"cmake.configureOnEdit": false
}
143 changes: 143 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
cmake_minimum_required(VERSION 3.15)
set(CMAKE_TOOLCHAIN_FILE "${CMAKE_CURRENT_LIST_DIR}/vcpkg/scripts/buildsystems/vcpkg.cmake")
project(node-mapnik)

set(MAPBOX_WAGYU_INCLUDE_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/deps/wagyu/include" CACHE INTERNAL "wagyu include dir")
set(BUILD_TESTING OFF)
set(MAPNIK_VECTOR_TILE_ENABLE_INSTALL OFF)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(Threads REQUIRED)
find_package(PkgConfig)
pkg_check_modules(Cairo REQUIRED IMPORTED_TARGET cairo)
pkg_check_modules(PROJ REQUIRED IMPORTED_TARGET proj)
find_package(Protobuf REQUIRED)
find_package(mapnik CONFIG REQUIRED COMPONENTS shapeindex mapnik-index)

include(FetchContent)
FetchContent_Declare(
napi_modules
GIT_REPOSITORY https://github.com/mathisloge/cmake-napi.git
GIT_TAG cdc5203c12f7e44ccb260b315256590d54b574af
)
FetchContent_Declare(
mapnik_vector_tile
GIT_REPOSITORY https://github.com/mathisloge/mapnik-vector-tile.git
GIT_TAG 4ad96069d5241a592d326ee0eca1ce89cf33372f
GIT_SUBMODULES "src" # https://gitlab.kitware.com/cmake/cmake/-/issues/20579
)
FetchContent_MakeAvailable(mapnik_vector_tile napi_modules)
FetchContent_GetProperties(napi_modules SOURCE_DIR napi_src)
list(APPEND CMAKE_MODULE_PATH "${napi_src}/modules")
include(napi-gyp)


execute_process(COMMAND node -p "require('node-addon-api').include"
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
OUTPUT_VARIABLE NODE_ADDON_API_DIR
)
string(REPLACE "\n" "" NODE_ADDON_API_DIR ${NODE_ADDON_API_DIR})
string(REPLACE "\"" "" NODE_ADDON_API_DIR ${NODE_ADDON_API_DIR})

add_subdirectory(deps)
mapnik_find_plugin_dir(MAPNIK_PLUGINS_DIR)
message(STATUS "Using plugins from ${MAPNIK_PLUGINS_DIR}")

add_library(node-mapnik MODULE
src/mapnik_logger.cpp
src/node_mapnik.cpp
src/blend.cpp
src/mapnik_map.cpp
src/mapnik_map_load.cpp
src/mapnik_map_from_string.cpp
src/mapnik_map_render.cpp
src/mapnik_map_query_point.cpp
src/mapnik_color.cpp
src/mapnik_geometry.cpp
src/mapnik_feature.cpp
src/mapnik_image.cpp
src/mapnik_image_encode.cpp
src/mapnik_image_open.cpp
src/mapnik_image_fill.cpp
src/mapnik_image_save.cpp
src/mapnik_image_from_bytes.cpp
src/mapnik_image_from_svg.cpp
src/mapnik_image_solid.cpp
src/mapnik_image_multiply.cpp
src/mapnik_image_clear.cpp
src/mapnik_image_copy.cpp
src/mapnik_image_resize.cpp
src/mapnik_image_compositing.cpp
src/mapnik_image_filter.cpp
src/mapnik_image_view.cpp
src/mapnik_grid.cpp
src/mapnik_grid_view.cpp
src/mapnik_palette.cpp
src/mapnik_projection.cpp
src/mapnik_layer.cpp
src/mapnik_datasource.cpp
src/mapnik_featureset.cpp
src/mapnik_expression.cpp
src/mapnik_cairo_surface.cpp
src/mapnik_vector_tile.cpp
src/mapnik_vector_tile_data.cpp
src/mapnik_vector_tile_query.cpp
src/mapnik_vector_tile_json.cpp
src/mapnik_vector_tile_info.cpp
src/mapnik_vector_tile_simple_valid.cpp
src/mapnik_vector_tile_render.cpp
src/mapnik_vector_tile_clear.cpp
src/mapnik_vector_tile_image.cpp
src/mapnik_vector_tile_composite.cpp
)
set_target_properties(node-mapnik PROPERTIES PREFIX "" OUTPUT_NAME "mapnik" SUFFIX ".node")
target_include_directories(node-mapnik PRIVATE
${NODE_ADDON_API_DIR}
)

target_compile_definitions(node-mapnik PRIVATE
_USE_MATH_DEFINES
MAPNIK_GIT_REVISION=1
)

if(MSVC)
target_compile_definitions(node-mapnik PRIVATE /wd4068)
endif()

target_link_libraries(node-mapnik
mapnik::core
mapnik::json
mapnik::wkt
mapnik::mapnik
mapbox-geometry
mapbox-wagyu
mapbox-protozero
node::napi
mapnik::mapnik-vector-tile
)

# post build
file(GENERATE
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/$<CONFIG>_copy.cmake"
INPUT "${CMAKE_CURRENT_SOURCE_DIR}/cmake/post_build.cmake"
)

# installation
macro(install_variable var)
install(CODE "set(${var} \"${${var}}\")")
endmacro()

install(TARGETS node-mapnik
RUNTIME_DEPENDENCY_SET node-mapnik-deps
RUNTIME DESTINATION "${CMAKE_INSTALL_PREFIX}"
ARCHIVE DESTINATION "${CMAKE_INSTALL_PREFIX}"
LIBRARY DESTINATION "${CMAKE_INSTALL_PREFIX}"
)
install_variable(MAPNIK_FONTS_DIR)
install_variable(MAPNIK_PLUGINS_DIR)
install_variable(VCPKG_INSTALLED_DIR)
install_variable(VCPKG_TARGET_TRIPLET)
install(CODE "set(SOURCE_DIR \"${CMAKE_CURRENT_SOURCE_DIR}\")")
install(SCRIPT ${CMAKE_CURRENT_BINARY_DIR}/$<CONFIG>_copy.cmake)
9 changes: 9 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM ghcr.io/mathisloge/mapnik:docker
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a Dockerfile for this base image?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's build from this branch mapnik/mapnik#4267

ARG DEBIAN_FRONTEND=noninteractive

RUN apt update && apt install -y nodejs npm git libprotobuf-dev protobuf-compiler && mkdir /nodemapnik

WORKDIR /nodemapnik
COPY . .
RUN npm install && rm -rf build*
RUN npm run test
Loading