fix(s6): make web services wait for their config oneshots (fixes root-mode startup race) - #689
Open
LorenzoRogai wants to merge 1 commit into
Open
Conversation
…-mode startup race When a container built on the s6 images runs as root, php-fpm and the web server (nginx/apache2) are brought up in parallel with the entrypoint oneshots that configure them, because the long-running services have no dependency on those oneshots. As root this races: - php-fpm reads its pool before `5-fpm-pool-user` appends `user`/`group`, failing with "ALERT: [pool www] user has not been defined" -> "ERROR: FPM initialization failed". - the web server starts before `10-init-webserver-config` renders its config (e.g. nginx: open() "/etc/nginx/nginx.conf" failed). s6 restarts the crashed services so the container eventually recovers, which is why the failure is intermittent and hard to reproduce (see discussion serversideup#425), but it emits alarming errors, slows startup, and leaves a brief window with no service. docker-php-serversideup-s6-init now adds a dependency from each web service to the entrypoint oneshot that configures it, appending to the existing flat `dependencies` file. The oneshots are chained in alphabetical order, so depending on one transitively waits for all earlier ones (php-fpm -> 5-fpm-pool-user; nginx/apache2 -> 10-init-webserver-config). Entries are de-duplicated and appended newline-safely (nginx's shipped `dependencies` has no trailing newline). Dependencies are only added when both the service and the oneshot exist, so cli/fpm/frankenphp images and images that remove a script are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
LorenzoRogai
force-pushed
the
fix/root-mode-fpm-nginx-startup-race
branch
from
July 22, 2026 14:05
46d9ba2 to
c7befe0
Compare
mbrodala
suggested changes
Aug 20, 2026
Comment on lines
+97
to
+98
| dependencies_file="${service_dir}/dependencies" | ||
| [ -e "$dependencies_file" ] || : > "$dependencies_file" |
There was a problem hiding this comment.
Suggested change
| dependencies_file="${service_dir}/dependencies" | |
| [ -e "$dependencies_file" ] || : > "$dependencies_file" | |
| dependencies_file="${service_dir}/dependencies" | |
| touch "$dependencies_file" |
No need to check for file existence, touch is idempotent.
| add_startup_dependency() { | ||
| # $1 = long-running service that must wait, $2 = entrypoint oneshot it needs | ||
| service_dir="${S6_HOME}/s6-rc.d/$1" | ||
| [ -d "$service_dir" ] && [ -d "${S6_HOME}/s6-rc.d/$2" ] || return 0 |
There was a problem hiding this comment.
Maybe add an explanatory comment for this condition.
Comment on lines
+82
to
+91
| # Make the long-running services wait for the entrypoint oneshots that configure | ||
| # them. When the container runs as root, php-fpm and the web server otherwise | ||
| # start in parallel with these oneshots and can lose the race: php-fpm reads the | ||
| # pool before "5-fpm-pool-user" adds "user = www-data" (ALERT: [pool www] user | ||
| # has not been defined -> FPM initialization failed), and the web server starts | ||
| # before "10-init-webserver-config" renders its config. s6 restarts the crashed | ||
| # services so the container recovers, but it produces alarming errors, a slower | ||
| # start, and a brief window with no service. The entrypoint oneshots are chained | ||
| # in alphabetical order, so depending on one transitively waits for all earlier | ||
| # ones. Each dependency is only added when both the service and the oneshot exist. |
There was a problem hiding this comment.
IMO this should be in Git/PR description only to keep the code compact.
| [ -e "$dependencies_file" ] || : > "$dependencies_file" | ||
|
|
||
| # Skip if the dependency is already declared | ||
| grep -qxF "$2" "$dependencies_file" 2>/dev/null && return 0 |
There was a problem hiding this comment.
Suggested change
| grep -qxF "$2" "$dependencies_file" 2>/dev/null && return 0 | |
| grep --quiet --line-regexp --fixed-strings "$2" "$dependencies_file" 2>/dev/null && return 0 |
IMO CLI arguments should always be long form unless you are typing in a terminal. This makes it a lot easier to understand the command.
Comment on lines
+103
to
+108
| # Ensure existing content ends with a newline before appending (some shipped | ||
| # dependency files, e.g. nginx's, have no trailing newline) | ||
| if [ -s "$dependencies_file" ] && [ -n "$(tail -c 1 "$dependencies_file")" ]; then | ||
| printf '\n' >> "$dependencies_file" | ||
| fi | ||
| printf '%s\n' "$2" >> "$dependencies_file" |
There was a problem hiding this comment.
Suggested change
| # Ensure existing content ends with a newline before appending (some shipped | |
| # dependency files, e.g. nginx's, have no trailing newline) | |
| if [ -s "$dependencies_file" ] && [ -n "$(tail -c 1 "$dependencies_file")" ]; then | |
| printf '\n' >> "$dependencies_file" | |
| fi | |
| printf '%s\n' "$2" >> "$dependencies_file" | |
| printf '\n%s\n' "$2" >> "$dependencies_file" |
The file doesn't need to be pretty. If you need a newline before, just always output it. :-)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a container built on the s6 images runs as root,
php-fpmand the web server (nginx/apache2) are brought up in parallel with the entrypoint oneshots that configure them — the long-running services declare no dependency on those oneshots.As root this is a race:
php-fpmreads its pool config before5-fpm-pool-userappendsuser/group, failing with:10-init-webserver-configrenders its config, e.g.nginx: [emerg] open() "/etc/nginx/nginx.conf" failed (2: No such file or directory).s6restarts the crashed longruns, so the container eventually recovers — which is exactly why this is intermittent and hard to reproduce (see #425). But it emits alarming errors on every boot that loses the race, slows startup, and leaves a brief window with no service.Fix
docker-php-serversideup-s6-initnow adds a dependency from each web-facing longrun to the entrypoint oneshot that configures it, after the oneshots are created:The entrypoint oneshots are chained in alphabetical order, so depending on one transitively waits for all earlier ones (
php-fpm→5-fpm-pool-user; the web servers →10-init-webserver-config, which already sits after5-*). Each dependency is added only when both the service and the oneshot exist, socli/fpm/frankenphpimages — and images where a user removes a script — are unaffected. It does not make services wait on later app oneshots such as50-laravel-automations, so a failing app hook won't block the web server from starting.Testing
8.4-fpm-nginx-alpine: first boot shows the fpm + nginx errors, then self-heals.php-fpm/nginxreach "ready to handle connections" on the first attempt.5-fpm-pool-useris a no-op when not root; the dependency just orders startup).Refs #425