Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Docker image build #6

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
581b827
Merge pull request #1 from hotosm/feat/set_up
katporks Nov 18, 2023
48e89a3
Updated README
katporks Nov 18, 2023
40a320a
Switched to correct worklflow: image_build
katporks Nov 18, 2023
75256c2
Merge pull request #2 from hotosm/feat/set_up
katporks Nov 18, 2023
42602d9
Added custom user for future extensbility
katporks Nov 18, 2023
72e73c1
Added pytest-factoryboy dependency
katporks Nov 22, 2023
df0112d
Added User factory and tests
katporks Nov 23, 2023
c9a55b4
Fix: Added self to test methods
katporks Nov 23, 2023
d45e34f
Fix: Corrected failing test from typo
katporks Nov 23, 2023
cdffe93
Modified Dockerfile to be able to run pytest locally
katporks Nov 25, 2023
08e5824
build: remove requirements-dev install to separate stage
spwoodcock Nov 27, 2023
284f1c2
Smoothed out simple pytest warnings
katporks Nov 28, 2023
a7147e6
Removed package related pytest warnings
katporks Nov 28, 2023
7e0820f
Simplified Makefile commands to default to dev env
katporks Nov 28, 2023
260ebda
Merge pull request #3 from hotosm/feature/add-custom-user-model
katporks Nov 28, 2023
6e4ebc6
Update README.md for adjusted Makefile
katporks Nov 28, 2023
cbea54c
Configured postcss + compressor
katporks Dec 3, 2023
47b5952
Configured django-browser-reload
katporks Dec 3, 2023
472795a
Merge pull request #4 from hotosm/feat/set-up-frontend
katporks Dec 5, 2023
bef5484
Created frontend folder for npm + tailwind config
katporks Dec 6, 2023
a8c9278
Adjusted Dockerfile for frontend folder
katporks Dec 6, 2023
ebc3953
Merge pull request #5 from hotosm/feat/set-up-frontend
katporks Dec 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/docker_image_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,9 @@ on:
- 'dev'

jobs:
pytest:
uses: hotosm/gh-workflows/.github/workflows/test_pytest.yml@main
build:
uses: hotosm/gh-workflows/.github/workflows/image_build.yml@main
secrets: inherit
with:
image_name: ghcr.io/${{ github.repository }}
build_target: prod
environment: dev
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
**/__pycache__/
**/.pytest_cache/
**/.venv/
node_modules
**/dist/
42 changes: 30 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
ARG PYTHON_IMG_TAG=3.11
ARG NODE_IMG_TAG=20.5.1

FROM node:${NODE_IMG_TAG}-bookworm-slim as frontend-base
WORKDIR /app
COPY ./frontend ./frontend
RUN cd frontend && npm install
RUN cd frontend && npm run build

# Define the base stage
FROM docker.io/python:${PYTHON_IMG_TAG}-slim-bookworm as base
Expand Down Expand Up @@ -35,7 +41,7 @@ RUN pip install --no-cache-dir --upgrade pip \
poetry==1.7.1 poetry-plugin-export==1.6.0 \
&& poetry config warnings.export false
RUN poetry export --without dev --output requirements.txt
RUN poetry export --only dev --output requirements-dev.txt
RUN poetry export --with dev --output requirements-dev.txt


# Define build stage (install deps)
Expand All @@ -51,13 +57,17 @@ RUN set -ex \
"libjpeg62-turbo-dev" \
"zlib1g-dev" \
"libwebp-dev" \
"nodejs" \
"npm" \
&& rm -rf /var/lib/apt/lists/*
COPY --from=extract-deps \
/opt/python/requirements.txt /opt/python/
# Copy the built CSS from the Node.js stage
# COPY --from=frontend-base /dist/ /dist/
# Install dependencies
RUN pip install --user --no-warn-script-location \
--no-cache-dir -r /opt/python/requirements.txt


# Define run stage
FROM base as runtime
ARG PYTHON_IMG_TAG
Expand Down Expand Up @@ -88,10 +98,15 @@ COPY --chown=wagtail:wagtail container-entrypoint.sh /
COPY --from=build \
/root/.local \
/home/wagtail/.local
# Copy compiled css from frontend stage
COPY --from=frontend-base \
/app/frontend/dist \
/app/frontend/dist
# Use /app folder as a directory where the source code is stored.
WORKDIR /app
# Copy project
COPY . /app/

# Add non-root user, permissions
RUN useradd -u 1001 -m -c "hotosm account" -d /home/wagtail -s /bin/false wagtail \
&& chown -R wagtail:wagtail /app /home/wagtail \
Expand All @@ -102,28 +117,31 @@ USER wagtail
ENTRYPOINT ["/container-entrypoint.sh"]


# Define dev-deps stage (install requirements-dev)
FROM runtime as dev-deps
COPY --from=extract-deps --chown=wagtail \
/opt/python/requirements-dev.txt /home/wagtail/
RUN pip install --user --no-warn-script-location \
--no-cache-dir -r /home/wagtail/requirements-dev.txt \
&& rm -r /home/wagtail/requirements-dev.txt


# Define test (ci) stage
FROM runtime as test
FROM dev-deps as test
USER root
ARG PYTHON_IMG_TAG
COPY --from=extract-deps \
/opt/python/requirements-dev.txt /opt/python/
# Copy packages from user to root dirs (run ci as root)
# && install dev dependencies (pytest)
RUN mv /home/wagtail/.local/bin/* /usr/local/bin/ \
&& mv /home/wagtail/.local/lib/python${PYTHON_IMG_TAG}/site-packages/* \
&& cp -R /home/wagtail/.local/lib/python${PYTHON_IMG_TAG}/site-packages/* \
/usr/local/lib/python${PYTHON_IMG_TAG}/site-packages/ \
&& pip install --upgrade --no-warn-script-location \
--no-cache-dir -r \
/opt/python/requirements-dev.txt \
&& rm -r /opt/python \
&& rm -rf /home/wagtail/.local/ \
# Pre-compile packages to .pyc (init speed gains)
&& python -c "import compileall; compileall.compile_path(maxlevels=10, quiet=1)"
CMD ["pytest"]


# Define debug (development) stage
FROM runtime as debug
FROM dev-deps as debug
# Add Healthcheck
HEALTHCHECK --start-period=10s --interval=5s --retries=20 --timeout=5s \
CMD curl --fail http://localhost:8000 || exit 1
Expand Down
10 changes: 4 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: build-dev build-prod test up-dev up-prod down-dev down-prod

build-dev:
build:
@docker compose -f docker-compose.dev.yml build
@docker compose -f docker-compose.dev.yml up -d

Expand All @@ -9,17 +9,15 @@ build-prod:
@docker compose -f docker-compose.prod.yml up -d

test:
@docker build --target test -t myproject:test -f Dockerfile .
@docker run --rm myproject:test
@docker rmi myproject:test
@docker compose -f docker-compose.dev.yml run --rm web pytest

up-dev:
up:
@docker compose -f docker-compose.dev.yml up -d

up-prod:
@docker compose -f docker-compose.prod.yml up

down-dev:
down:
@docker compose -f docker-compose.dev.yml down

down-prod:
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,18 @@ Remember to keep your `.env` file secure and do not commit it to your version co

## Dependency Management

This project uses Pipenv for dependency management. Pipenv is a Python tool that combines the best features of pip (Python's package installer), Pipfile (a manifest file format for Python package requirements), and virtualenv (a tool for creating isolated Python environments).
This project uses Poetry for dependency management. Poetry is a Python tool that helps to handle dependency installation, building packages, and versioning. It simplifies package management and deployment by adding a layer of abstraction over the standard setup tools.

Dependencies are specified in the `Pipfile`, and the exact versions of the dependencies are locked in the `Pipfile.lock`. To install the dependencies, run `pipenv install`. To add a new dependency, run `pipenv install {dependency}`.
Dependencies are specified in the `pyproject.toml`, and the exact versions of the dependencies are locked in the `poetry.lock`. To install the dependencies, run `poetry install`. To add a new dependency, run `poetry add {dependency}`.

## Using the Makefile

Ensure you have the superuser credentials you want in your .env.dev file prior to running `make build-dev`.
The Makefile provides several commands for building and running the project:

- `make build-dev`: Builds and starts the Docker images for the development environment.
- `make up-dev`: Starts the Docker containers for the development environment.
- `make down-dev`: Stops the Docker containers for the development environment.
- `make build`: Builds and starts the Docker images for the development environment.
- `make up`: Starts the Docker containers for the development environment.
- `make down`: Stops the Docker containers for the development environment.
- `make build-prod`: Builds the Docker images for the production environment.
- `make up-prod`: Starts the Docker containers for the production environment.
- `make down-prod`: Stops the Docker containers for the production environment.
Expand Down
Loading
Loading