Commit 14564e7
committed
ci(deploy): pause watchdog timer during container swap
The systemd watchdog (/etc/systemd/system/ocaml-lob-watchdog.timer)
races the deploy script's stop/rm/run sequence: the timer fires
every 30s, and if it ticks between [docker stop] and [docker rm]
it sees an unhealthy container and does [docker restart ocaml_lob],
which brings the container back up. The subsequent [docker rm]
then either fails (without -f) or leaves a stale container, and
[docker run --name ocaml_lob] hits a name conflict.
Saw this fire on the SSE deploy: build succeeded, deploy died
with "container name already in use" while watchdog logs showed
a restart firing in the same minute. Fix:
1. [systemctl stop ocaml-lob-watchdog.timer] before the swap,
[systemctl start] via trap on EXIT so liveness coverage resumes
even if the deploy fails mid-way.
2. [docker rm -f] for defense in depth — handles a running
container regardless of how it got there.1 parent adfeb77 commit 14564e7
1 file changed
Lines changed: 14 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
141 | 152 | | |
| 153 | + | |
| 154 | + | |
142 | 155 | | |
143 | | - | |
| 156 | + | |
144 | 157 | | |
145 | 158 | | |
146 | 159 | | |
| |||
0 commit comments