Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daemon re-runs "start" task when restarting after "on-failure: shutdown" #527

Open
benhoyt opened this issue Nov 27, 2024 · 0 comments
Open
Labels
Bug An undesired feature ;-)

Comments

@benhoyt
Copy link
Contributor

benhoyt commented Nov 27, 2024

If a service is configured with on-failure: shutdown and it fails, pebble will shut down, but then when pebble runs again it'll try to redo that "start" task and immediately shut down again. The task is in status 3 (DoingStatus). For example

services:
  svc1:
    override: replace
    command: sleep x
    on-failure: shutdown
# First run (on "pebble start")
$ go run ./cmd/pebble/ run
2024-11-27T22:57:35.035Z [pebble] Started daemon.
2024-11-27T22:57:35.036Z [pebble] POST /v1/services 244.475µs 400
2024-11-27T22:57:35.036Z [pebble] Cannot start default services: no default services
# "pebble start" executed on another termina
2024-11-27T22:57:37.853Z [pebble] POST /v1/services 33.159477ms 202
2024-11-27T22:57:37.871Z [pebble] Service "svc1" starting: sleep x
2024-11-27T22:57:37.873Z [pebble] Service "svc1" stopped unexpectedly with code 1
2024-11-27T22:57:37.873Z [pebble] Service "svc1" on-failure action is "shutdown", triggering failure shutdown
2024-11-27T22:57:37.873Z [pebble] Server exiting! Reason: <nil>
exit status 10

# Second run
$ go run ./cmd/pebble/ run
2024-11-27T22:57:59.137Z [pebble] Started daemon.
2024-11-27T22:57:59.154Z [pebble] Service "svc1" starting: sleep x
2024-11-27T22:57:59.154Z [pebble] POST /v1/services 15.91363ms 400
2024-11-27T22:57:59.154Z [pebble] Cannot start default services: no default services
2024-11-27T22:57:59.156Z [pebble] Service "svc1" stopped unexpectedly with code 1
2024-11-27T22:57:59.156Z [pebble] Service "svc1" on-failure action is "shutdown", triggering failure shutdown
2024-11-27T22:57:59.156Z [pebble] Server exiting! Reason: <nil>
exit status 10

# Third run
$ go run ./cmd/pebble/ run
2024-11-27T22:58:19.848Z [pebble] Started daemon.
2024-11-27T22:58:19.873Z [pebble] Service "svc1" starting: sleep x
2024-11-27T22:58:19.873Z [pebble] POST /v1/services 23.920321ms 400
2024-11-27T22:58:19.873Z [pebble] Cannot start default services: no default services
2024-11-27T22:58:19.874Z [pebble] Service "svc1" stopped unexpectedly with code 1
2024-11-27T22:58:19.874Z [pebble] Service "svc1" on-failure action is "shutdown", triggering failure shutdown
2024-11-27T22:58:19.874Z [pebble] Server exiting! Reason: <nil>
exit status 10

This does not seem great. Pebble should probably transition the task to done or error.

@benhoyt benhoyt added the Bug An undesired feature ;-) label Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug An undesired feature ;-)
Projects
None yet
Development

No branches or pull requests

1 participant