Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: hot server reload & debugging over docker #2718

Merged
merged 18 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions localenv/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,28 @@ Note that you have to go through an additional "login" step by providing you IPv

After stopping the script it is necessary to clear the environment using the command described in [Shutting down](#Shutting-down). This is necessary as on a new run of the scripts (with autopeering or not) the wallet address url will differ.

### Debugging
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add a section with the launch.json configuration for VSCode as well?

Also, did you want to keep it just in this readme, or should we also add it to the documentation site?

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added the vscode section and added to the documentation site: 02a5e6b


Debuggers for the services are exposed on the following ports:

| IP and Port | Services |
| -------------- | ----------------------- |
| 127.0.0.1:9229 | Cloud Nine Backend |
BlairCurrey marked this conversation as resolved.
Show resolved Hide resolved
| 127.0.0.1:9230 | Cloud Nine Auth |
| 127.0.0.1:9231 | Happy Life Bank Backend |
| 127.0.0.1:9232 | Happy Life Bank Auth |

To use these debuggers with a chromium browser:

- go to `chrome://inspect` in chromium browser
- Click "Configure" add the IP addresses and ports detailed above
- start docker containers
- click "inspect" on the service you want to debug to open the chromium debugger

You can either trigger the debugger by adding `debugger` statements in code and restarting, or by adding breakpoints directly in the chromium debugger after starting the docker containers.

For more ways to connect to these debuggers, see the Node docs for debugging: https://nodejs.org/en/learn/getting-started/debugging

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to provide specific instructions for a basic way to use the debugger despite the fact that everyone is liable to have a different setup (vscode, jetbrains, text editor, etc.). Thus I provided instructions for setting up the chromium debugging client and left a link to the node docs if people want to tune it more for their own setup.

Of course this does require a chromium browser, but it's still the most developer-environment-agnostic option. Note that it's not possible to use safari, firefox, or any non-chromium browser because node uses the v8 engine, and those browsers do not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let me know if you can follow these instructions to start using the debugger successfully.

### Shutting down

```
Expand Down
18 changes: 15 additions & 3 deletions localenv/cloud-nine-wallet/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,19 @@ services:
image: rafiki-backend
build:
context: ../..
dockerfile: ./packages/backend/Dockerfile
dockerfile: ./packages/backend/Dockerfile.dev
volumes:
- type: bind
source: ../../packages/backend/src
target: /home/rafiki/packages/backend/src
read_only: true
restart: always
privileged: true
ports:
- '3000:80'
- '3001:3001'
- '3002:3002'
- "9229:9229"
networks:
- rafiki
environment:
Expand Down Expand Up @@ -73,13 +79,19 @@ services:
image: rafiki-auth
build:
context: ../..
dockerfile: ./packages/auth/Dockerfile
dockerfile: ./packages/auth/Dockerfile.dev
volumes:
- type: bind
source: ../../packages/auth/src
target: /home/rafiki/packages/auth/src
read_only: true
restart: always
networks:
- rafiki
ports:
- '3003:3003'
- '3006:3006'
- "9230:9229"
environment:
NODE_ENV: ${NODE_ENV:-development}
TRUST_PROXY: ${TRUST_PROXY}
Expand Down Expand Up @@ -110,7 +122,7 @@ services:
image: rafiki-frontend
build:
context: ../..
dockerfile: ./packages/frontend/devDockerfile
dockerfile: ./packages/frontend/Dockerfile.dev
Copy link
Contributor Author

@BlairCurrey BlairCurrey May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

went ahead and renamed this for consistency. I went with Dockerfile.dev because that seemed to be a more standard convention from examples I saw a variety of places. VSCode also recognizes its a dockerfile from name alone, whereas it did not with devDockerfile.

volumes:
- type: bind
source: ../../packages/frontend/app
Expand Down
12 changes: 12 additions & 0 deletions localenv/happy-life-bank/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,17 @@ services:
hostname: happy-life-bank-backend
image: rafiki-backend
pull_policy: never
volumes:
- type: bind
source: ../../packages/backend/src
target: /home/rafiki/packages/backend/src
read_only: true
restart: always
privileged: true
ports:
- "4000:80"
- "4001:3001"
- '9231:9229'
networks:
- rafiki
environment:
Expand Down Expand Up @@ -64,11 +70,17 @@ services:
image: rafiki-auth
pull_policy: never
restart: always
volumes:
- type: bind
source: ../../packages/auth/src
target: /home/rafiki/packages/auth/src
read_only: true
networks:
- rafiki
ports:
- '4003:3003'
- '4006:3006'
- '9232:9229'
environment:
NODE_ENV: development
AUTH_DATABASE_URL: postgresql://happy_life_bank_auth:happy_life_bank_auth@shared-database/happy_life_bank_auth
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ async function callWithRetry(fn: () => any, depth = 0): Promise<void> {
try {
return await fn()
} catch (e) {
if (depth > 7) {
if (depth > 10) {
Copy link
Contributor Author

@BlairCurrey BlairCurrey May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new startup command in the dockerfile takes longer because we have to rebuild and copy over gql/openapi schemas. 9 was all that was required but figured I'd bump it to 10 for some headroom.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to change this with ts-node-dev?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no we dont, changed back to 7

throw e
}
await wait(2 ** depth * 30)
Expand Down
26 changes: 26 additions & 0 deletions packages/auth/Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM node:20-alpine3.18

WORKDIR /home/rafiki

RUN corepack enable
RUN corepack prepare [email protected] --activate

COPY pnpm-lock.yaml package.json pnpm-workspace.yaml .npmrc tsconfig.json tsconfig.build.json ./

RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm fetch \
| grep -v "cross-device link not permitted\|Falling back to copying packages from store"

COPY . ./

RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm install \
--recursive \
--offline \
--frozen-lockfile

RUN pnpm --filter auth build:deps

EXPOSE 9229

CMD pnpm --filter auth dev
File renamed without changes.
4 changes: 3 additions & 1 deletion packages/auth/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
"prepack": "pnpm build",
"postinstall": "pnpm copy-op-schemas",
"copy-op-schemas": "cp ./node_modules/@interledger/open-payments/dist/openapi/specs/auth-server.yaml ./node_modules/@interledger/open-payments/dist/openapi/specs/schemas.yaml ./src/openapi/specs/",
"copy-files": "cp src/graphql/schema.graphql dist/graphql/ && cp -r ./src/openapi ./dist/"
"copy-files": "cp src/graphql/schema.graphql dist/graphql/ && cp -r ./src/openapi ./dist/",
"dev": "nodemon --watch src --ext ts --exec \"tsc && pnpm copy-files && node --inspect=0.0.0.0:9229 dist/index.js\""
Copy link
Contributor Author

@BlairCurrey BlairCurrey May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is run from the dev dockerfile. In plain english, it is:

rebuilding, copying over the gql/openapi schema files, and starting the sever with the debugger. Then on the docker container's src file change, it is repeating this rebuild/copy/server start. The docker container's /src dir mirrors the host machine's /src dir via a bind mount. Thus saving changes on the host machine triggers the server restart in docker.

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatives I considered included:

  • using ts-node instead of explicit build step. Never really liked the idea but its the more common use-case with nodemon. Tried it briefly but ran into some typescript errors and didn't pursue it further.
  • Docker Compose Watch. This could be used to varying extents. Effectively replacing nodemon, or replacing nodemon and bind mount. However we dont really take advantage of the advantages over bind mounts listed in the docs and it would create more overheard on restart (restarts entire container instead of just server process).
  • I had a sort of vague idea about doing a bind mount on /dist instead and adding something to watch on the hostmachine (tacked onto localenv:compose and localenv:compose:psql probably), but then we'd still need to watch the /dist in docker as well. Overall I felt like this was more complicated.

},
"dependencies": {
"@adonisjs/fold": "^8.2.0",
Expand Down Expand Up @@ -65,6 +66,7 @@
"jest-openapi": "^0.14.2",
"nock": "^13.5.4",
"node-mocks-http": "^1.14.1",
"nodemon": "^3.1.0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"nodemon": "^3.1.0",

Copy link
Contributor Author

@BlairCurrey BlairCurrey May 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, committed locally and updated the lockfile

"openapi-types": "^12.1.3",
"pino-pretty": "^11.0.0",
"testcontainers": "^10.7.2"
Expand Down
26 changes: 26 additions & 0 deletions packages/backend/Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM node:20-alpine3.18

WORKDIR /home/rafiki

RUN corepack enable
RUN corepack prepare [email protected] --activate

COPY pnpm-lock.yaml package.json pnpm-workspace.yaml .npmrc tsconfig.json tsconfig.build.json ./

RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm fetch \
| grep -v "cross-device link not permitted\|Falling back to copying packages from store"

COPY . ./

RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm install \
--recursive \
--offline \
--frozen-lockfile

RUN pnpm --filter backend build:deps

EXPOSE 9229

CMD pnpm --filter backend dev
File renamed without changes.
4 changes: 3 additions & 1 deletion packages/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
"copy-files": "cp src/graphql/schema.graphql dist/graphql/ && cp -r ./src/openapi ./dist/",
"copy-op-schemas": "cp ./node_modules/@interledger/open-payments/dist/openapi/specs/schemas.yaml ./src/openapi/specs/",
"prepack": "pnpm build",
"postinstall": "pnpm copy-op-schemas"
"postinstall": "pnpm copy-op-schemas",
"dev": "nodemon --watch src --ext ts --exec \"tsc && pnpm copy-files && node --inspect=0.0.0.0:9229 dist/index.js\""
},
"devDependencies": {
"@apollo/client": "^3.9.9",
Expand All @@ -38,6 +39,7 @@
"jest-openapi": "^0.14.2",
"nock": "^13.5.4",
"node-mocks-http": "^1.14.1",
"nodemon": "^3.1.0",
"openapi-types": "^12.1.3",
"react": "~18.2.0",
"rosie": "^2.1.1",
Expand Down
File renamed without changes.
File renamed without changes.
Loading
Loading