Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions checks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@
| [check-php-legacy-backport.py](check-php-legacy-backport.py) | 旧 PHP 源码包校验、补丁应用与构建失败处理 | Python 3、Shell、patch;可传入 PHP 7.3.33 源码包 |
| [check-php-pecl.sh](check-php-pecl.sh) | 容器内下载、编译、加载 APCu 并验证缓存读写 | Bash、Docker、已构建的 PHP 镜像、网络 |
| [check-php-legacy-runtime.py](check-php-legacy-runtime.py) | PHP CLI、FastCGI、OOM 恢复、扩展及上传行为 | Python 3、Docker、已构建的 PHP 镜像 |
| [check-mysql-cluster.sh](check-mysql-cluster.sh) | MySQL 默认客户端、备份恢复、全量与增量复制、只读及重启恢复 | Bash、Docker、已构建的 MySQL 主从镜像 |
| [check-postgresql-cluster.sh](check-postgresql-cluster.sh) | PostgreSQL 流复制、历史认证规则保留、配置、目录权限及数据版本保护 | Bash、Docker、已构建的 PostgreSQL 主从镜像 |
| [check-redis-cluster.sh](check-redis-cluster.sh) | Redis 主从复制、复制延迟等待、配置优先级、密码及 RDB/AOF 恢复 | Bash、Docker、已构建的 Redis 主从镜像 |
| [check-mysql-cluster.sh](check-mysql-cluster.sh) | MySQL 默认客户端、备份恢复、全量与增量复制、只读、重启恢复及容器日志 | Bash、Docker、已构建的 MySQL 主从镜像 |
| [check-postgresql-cluster.sh](check-postgresql-cluster.sh) | PostgreSQL 流复制、历史认证规则保留、配置、目录权限、数据版本保护及容器日志 | Bash、Docker、已构建的 PostgreSQL 主从镜像 |
| [check-redis-cluster.sh](check-redis-cluster.sh) | Redis 主从复制、复制延迟等待、配置优先级、密码、RDB/AOF 恢复及容器日志 | Bash、Docker、已构建的 Redis 主从镜像 |

## 何时执行

Expand Down
46 changes: 46 additions & 0 deletions checks/check-mysql-cluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,34 @@ wait_count() {
return 1
}

check_logs() {
local container="$1" role="$2" phase="$3"
shift 3
local log_file="$test_dir/$role-$phase.log" message started_at missing='required MySQL startup logs'
started_at="$(docker inspect --format '{{.State.StartedAt}}' "$container")"
local deadline=$((SECONDS + 60))
while (( SECONDS < deadline )); do
docker logs --since "$started_at" "$container" > "$log_file" 2>&1
missing=''
for message in "$@"; do
if ! grep -Fq -- "[INFO] MySQL cluster ($role): $message" "$log_file"; then
missing="$message"
break
fi
done
if [[ -z "$missing" ]] && ! grep -Eq "\\[INFO\\] MySQL cluster \\($role\\): Starting MySQL $role with server ID [0-9]+" "$log_file"; then
missing='MySQL startup stage'
fi
if [[ -z "$missing" ]] && ! grep -Eq '\[MY-010931\].*ready for connections.*port: 3306([[:space:]]|$)' "$log_file"; then
missing='MySQL engine readiness on port 3306'
fi
[[ -n "$missing" ]] || return 0
sleep 1
done
echo "Log check timed out after 60s: $container ($phase), waiting for $missing" >&2
return 1
}

run_replica() {
docker run -d --name "$replica" --network "$network" \
-e MYSQL_REPLICATION_MODE=slave -e MYSQL_REPLICATION_USER=replicator \
Expand Down Expand Up @@ -94,6 +122,8 @@ docker run -d --name "$master" --network "$network" \
-v "$master_volume:/bitnami/mysql/data" \
-v "$test_dir/my.cnf:/opt/bitnami/mysql/conf/my.cnf" "$image" >/dev/null
wait_healthy "$master"
check_logs "$master" master initial \
'Configuring root authentication' 'Creating replication account' 'Initialization complete'
[[ "$(docker exec "$master" id -u)" == 1001 ]]
docker exec -e MYSQL_PWD="$root_password" "$master" mysqladmin --user=root ping >/dev/null
[[ "$(query "$master" "SELECT plugin FROM mysql.user WHERE User='root' AND Host='localhost'")" == mysql_native_password ]]
Expand All @@ -107,6 +137,10 @@ docker exec -i -e MYSQL_PWD="$root_password" "$master" mysql --user=root ha_rest
run_replica
wait_healthy "$replica"
wait_count 3
check_logs "$replica" slave initial \
'Configuring root authentication' "Waiting for master $master:3306" 'Master connection established' \
'Creating initial dump from master' 'Restoring initial dump' "Configuring replication from $master:3306" \
'Replication configuration applied' 'Initialization complete'
[[ "$(query "$replica" 'SELECT content FROM ha_test.records WHERE id=2')" == 'existing data' ]]
[[ "$(query "$replica" 'SELECT @@super_read_only')" == 1 ]]
if query "$replica" "INSERT INTO ha_test.records VALUES (99,'must fail')" >/dev/null 2>&1; then
Expand All @@ -123,6 +157,7 @@ docker rm -v "$replica" >/dev/null
run_replica
wait_healthy "$replica"
wait_count 5
check_logs "$replica" slave persisted 'Using existing data'
[[ "$(query "$replica" 'SELECT @@server_id')" == "$server_id" ]]
if docker run --rm -e MYSQL_REPLICATION_MODE=slave -e MYSQL_REPLICATION_USER=replicator \
-e MYSQL_REPLICATION_PASSWORD="$replication_password" -e MYSQL_ROOT_PASSWORD="$root_password" \
Expand All @@ -135,6 +170,7 @@ grep -q 'different replication role' "$test_dir/wrong-role.log"
docker exec "$master" rm /bitnami/mysql/data/.1panel-replication-mode
docker restart "$master" >/dev/null
wait_healthy "$master"
check_logs "$master" master persisted 'Using existing data'
query "$master" "INSERT INTO ha_test.records VALUES (6,'after master restart');" >/dev/null
wait_count 6
[[ "$(query "$replica" "SELECT COUNT(*) FROM performance_schema.replication_applier_status WHERE SERVICE_STATE='ON'")" == 1 ]]
Expand All @@ -151,4 +187,14 @@ if docker run --rm -e MYSQL_REPLICATION_MODE=master -e MYSQL_REPLICATION_USER=re
exit 1
fi
grep -q 'previous initialization did not finish' "$test_dir/unfinished-init.log"
for log_file in "$test_dir/"*.log; do
for password in "$root_password" "$replication_password" unused-on-dump-replica; do
escaped_password="${password//\\/\\\\}"
escaped_password="$(printf '%s' "$escaped_password" | sed "s/'/''/g")"
if grep -Fq -- "$password" "$log_file" || grep -Fq -- "$escaped_password" "$log_file"; then
echo "Configured password found in container output: $(basename "$log_file")" >&2
exit 1
fi
done
done
echo "MySQL cluster runtime checks passed: $image"
51 changes: 51 additions & 0 deletions checks/check-postgresql-cluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,46 @@ wait_rows() {
return 1
}

check_log_secrets() {
if [[ "$1" == *"$password"* || "$1" == *"$replication_password"* ]]; then
printf >&2 'PostgreSQL logs exposed a configured password\n'
return 1
fi
}

check_startup_logs() {
local container="$1"
shift
local started_at logs message attempt
started_at="$(docker inspect --format '{{.State.StartedAt}}' "$container")"
for (( attempt=0; attempt<90; attempt++ )); do
logs="$(docker logs --since "$started_at" "$container" 2>&1)"
check_log_secrets "$logs"
if awk '
/\[INFO\] PostgreSQL cluster \((master|slave)\): Starting PostgreSQL on port / { started = 1; ready = 0; next }
started && /database system is ready to accept/ { ready = 1 }
END { exit !ready }
' <<< "$logs"; then
break
fi
sleep 1
done
if (( attempt == 90 )); then
printf >&2 'PostgreSQL engine readiness log is missing after the current startup: %s\n' "$container"
return 1
fi
if ! grep -Eq '^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z \[INFO\] PostgreSQL cluster \((master|slave)\): ' <<< "$logs"; then
printf >&2 'Structured PostgreSQL startup log is missing: %s\n' "$container"
return 1
fi
for message in "$@"; do
if ! grep -Fq "$message" <<< "$logs"; then
printf >&2 'PostgreSQL startup log is missing for %s: %s\n' "$container" "$message"
return 1
fi
done
}

docker image inspect "$image" >/dev/null
major="$(docker run --rm "$image" bash -ec 'printf "%s" "$PG_MAJOR"')"
case "$major" in
Expand Down Expand Up @@ -87,6 +127,7 @@ docker run -d --name "$master" --network "$network" --network-alias pg-master \
-e "POSTGRESQL_REPLICATION_PASSWORD=$replication_password" \
-v "${volumes[0]}:/bitnami/postgresql" -v "${volumes[1]}:/bitnami/postgresql/conf" "$image" >/dev/null
wait_healthy "$master"
check_startup_logs "$master" 'Initializing master database' 'Configuring primary replication account' 'Startup configuration complete'
[[ "$(sql "$master" 55432 'SHOW shared_buffers')" == 32MB ]]
[[ "$(docker exec "$master" id -u)" == 1001 ]]
[[ "$(sql "$master" 55432 "SELECT current_setting('server_version_num')::integer / 10000")" == "$major" ]]
Expand All @@ -109,6 +150,7 @@ docker exec "$master" bash -ec '
'
docker restart "$master" >/dev/null
wait_healthy "$master"
check_startup_logs "$master" 'Using existing master data; skipping initialization and base backup' 'Configuring primary replication account'
[[ "$(sql "$master" 55432 'SELECT system_identifier FROM pg_control_system()')" == "$master_id" ]]
[[ "$(docker exec -e "PGPASSWORD=$password" "$master" psql -h 127.0.0.1 -p 55432 -U postgres -d postgres -Atqc 'SELECT 1')" == 1 ]]
if docker exec -e "PGPASSWORD=$password" "$master" psql -h "$master_ip" -p 55432 -U postgres -d postgres -Atqc 'SELECT 1' >/dev/null 2>&1; then
Expand All @@ -123,6 +165,7 @@ docker run -d --name "$slave" --network "$network" \
-e "POSTGRESQL_REPLICATION_PASSWORD=$replication_password" \
-v "${volumes[2]}:/bitnami/postgresql" -v "${volumes[3]}:/bitnami/postgresql/conf" "$image" >/dev/null
wait_healthy "$slave"
check_startup_logs "$slave" 'Cloning standby database from primary; waiting for base backup' 'Standby base backup complete' 'Configuring standby replication connection'
[[ "$(sql "$slave" 55433 'SELECT pg_is_in_recovery()')" == t ]]
[[ "$(sql "$slave" 55433 'SHOW shared_buffers')" == 48MB ]]
[[ "$(sql "$slave" 55433 'SELECT system_identifier FROM pg_control_system()')" == "$master_id" ]]
Expand All @@ -145,9 +188,11 @@ docker stop "$slave" >/dev/null
sql "$master" 55432 "INSERT INTO cluster_probe VALUES (3, 'offline-replica');" >/dev/null
docker start "$slave" >/dev/null
wait_healthy "$slave"
check_startup_logs "$slave" 'Using existing slave data; skipping initialization and base backup' 'Configuring standby replication connection'
wait_rows 3
docker restart "$master" >/dev/null
wait_healthy "$master"
check_startup_logs "$master" 'Using existing master data; skipping initialization and base backup' 'Configuring primary replication account'
sql "$master" 55432 "INSERT INTO cluster_probe VALUES (4, 'restarted-primary');" >/dev/null
wait_rows 4
[[ "$(docker exec "$slave" stat -c '%u:%g:%a' /bitnami/postgresql/conf/replication.pgpass)" == 1001:1001:600 ]]
Expand All @@ -173,6 +218,10 @@ if docker run --name "$prefix-failed" --network "$network" \
printf >&2 'Clone unexpectedly succeeded against a closed port\n'
exit 1
fi
failed_started_at="$(docker inspect --format '{{.State.StartedAt}}' "$prefix-failed")"
failed_logs="$(docker logs --since "$failed_started_at" "$prefix-failed" 2>&1)"
check_log_secrets "$failed_logs"
grep -Fq '[ERROR] PostgreSQL cluster (slave): Standby base backup failed' <<< "$failed_logs"
[[ "$(docker run --rm -v "${volumes[7]}:/conf" "$image" cat /conf/user-file)" == 'preserve configuration' ]]
[[ -z "$(docker run --rm -v "${volumes[6]}:/bitnami/postgresql" "$image" find /bitnami/postgresql -maxdepth 1 -name '.clone.*' -print)" ]]

Expand All @@ -187,6 +236,7 @@ fi
[[ "$(docker run --rm -v "${volumes[0]}:/bitnami/postgresql" "$image" cat /bitnami/postgresql/data/PG_VERSION)" == "$major" ]]
docker start "$master" >/dev/null
wait_healthy "$master"
check_startup_logs "$master" 'Using existing master data; skipping initialization and base backup'
[[ "$(sql "$master" 55432 'SELECT system_identifier FROM pg_control_system()')" == "$master_id" ]]
[[ "$(sql "$master" 55432 'SELECT count(*) FROM cluster_probe')" == 4 ]]

Expand All @@ -204,6 +254,7 @@ docker run -d --user root --name "$master" --network "$network" --network-alias
-e "POSTGRESQL_REPLICATION_PASSWORD=$replication_password" \
-v "${volumes[0]}:/bitnami/postgresql" -v "${volumes[1]}:/bitnami/postgresql/conf" "$image" >/dev/null
wait_healthy "$master"
check_startup_logs "$master" 'Preparing directory permissions and switching to the postgres user' 'Using existing master data; skipping initialization and base backup' '[WARN] PostgreSQL cluster (master): Authentication configuration is missing'
[[ "$(docker exec "$master" stat -c '%u:%g' /bitnami/postgresql)" == 1001:1001 ]]
[[ "$(docker exec "$master" stat -c '%u' /proc/1)" == 1001 ]]
[[ "$(docker exec "$master" grep -Fxc 'local all all peer' /bitnami/postgresql/conf/pg_hba.conf)" == 1 ]]
Expand Down
38 changes: 38 additions & 0 deletions checks/check-redis-cluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,37 @@ wait_for() {
return 1
}

check_logs() {
local container="$1" role="$2"
shift 2
local started_at logs message password missing
started_at="$(docker inspect --format '{{.State.StartedAt}}' "$container")"
local deadline=$((SECONDS + 60))
while (( SECONDS < deadline )); do
logs="$(docker logs --since "$started_at" "$container" 2>&1)"
for password in "$master_password" "$replica_password" config-password; do
if grep -Fq -- "$password" <<< "$logs"; then
printf 'Configured password found in container output: %s\n' "$container" >&2
return 1
fi
done
missing=''
for message in 'Validating replication settings' 'Preparing data and configuration directories' 'Starting Redis server' "$@"; do
if ! grep -Fq -- "[INFO] Redis cluster ($role): $message" <<< "$logs"; then
missing="$message"
break
fi
done
if [[ -z "$missing" && "$logs" != *'Ready to accept connections'* ]]; then
missing='Redis engine accepting connections'
fi
[[ -n "$missing" ]] || return 0
sleep 1
done
printf 'Log check timed out after 60s: %s, waiting for %s\n' "$container" "$missing" >&2
return 1
}

containers+=("$prefix-invalid" "$prefix-init")
if docker run --rm --name "$prefix-invalid" -e REDIS_PASSWORD=unused -e REDIS_REPLICATION_MODE=sentinel "$IMAGE" >/dev/null 2>&1; then
printf '%s\n' 'Unsupported replication mode was accepted' >&2
Expand Down Expand Up @@ -95,6 +126,8 @@ containers+=("$replica")
docker start "$master" "$replica" >/dev/null
wait_for PONG master_cli PING
wait_for PONG replica_cli PING
check_logs "$master" master 'Using mounted Redis configuration' 'Applying additional Redis arguments'
check_logs "$replica" slave 'Generating default Redis configuration with AOF enabled' 'Applying replica connection settings' 'Applying additional Redis arguments'

[[ "$(master_cli INFO server | tr -d '\r' | sed -n 's/^redis_version://p')" == "$redis_version" ]]
[[ "$(docker exec "$master" id -u)" == 1001 ]]
Expand Down Expand Up @@ -125,6 +158,7 @@ docker exec "$replica" test -s "$aof_file"
docker kill --signal KILL "$master" >/dev/null
docker start "$master" >/dev/null
wait_for PONG master_cli PING
check_logs "$master" master 'Using mounted Redis configuration'
[[ "$(master_cli GET counter)" == 40 ]]
[[ "$(master_cli GET entry:40)" == 'value 40' ]]

Expand All @@ -143,11 +177,15 @@ docker stop --time 10 "$master" >/dev/null
docker kill --signal KILL "$replica" >/dev/null
docker start "$replica" >/dev/null
wait_for PONG replica_cli PING
check_logs "$replica" slave 'Generating default Redis configuration with AOF enabled' 'Applying replica connection settings'
[[ "$(replica_cli GET counter)" == 80 ]]
[[ "$(replica_cli GET entry:80)" == 'value 80' ]]

docker start "$master" >/dev/null
wait_for PONG master_cli PING
check_logs "$master" master 'Using mounted Redis configuration'
[[ "$(master_cli INCR counter)" == 81 ]]
wait_for 81 replica_cli GET counter
check_logs "$master" master 'Using mounted Redis configuration'
check_logs "$replica" slave 'Applying replica connection settings'
printf 'Redis %s replication, configuration, persistence and reconnect checks passed.\n' "$redis_version"
3 changes: 2 additions & 1 deletion database-ha/mysql/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ COPY healthcheck.sh /opt/bitnami/scripts/mysql/healthcheck.sh

RUN chmod 0755 /usr/local/bin/1panel-mysql-entrypoint /opt/bitnami/scripts/mysql/healthcheck.sh && \
chown mysql:mysql /opt/bitnami/mysql/conf/my.cnf && \
ln -sf /opt/bitnami/mysql/conf/my.cnf /etc/my.cnf
ln -sf /opt/bitnami/mysql/conf/my.cnf /etc/my.cnf && \
ln -sf /proc/1/fd/1 /opt/bitnami/mysql/logs/mysqld.log

USER 1001:1001

Expand Down
10 changes: 9 additions & 1 deletion database-ha/mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,20 @@ bash checks/check-mysql-cluster.sh 1panel/mysql-cluster:8.4.13
- 配置:`/opt/bitnami/mysql/conf/my.cnf`,支持只读挂载
- 健康检查:`/opt/bitnami/scripts/mysql/healthcheck.sh`
- socket、PID:`/opt/bitnami/mysql/tmp/`
- 错误日志:`/opt/bitnami/mysql/logs/mysqld.log`
- 错误日志:`/opt/bitnami/mysql/logs/mysqld.log`,镜像内链接到 `/proc/1/fd/1`

官方客户端默认配置路径 `/etc/my.cnf` 关联到上述兼容配置文件,因此面板直接调用 `mysql`、`mysqladmin`、`mysqldump` 时也会使用正确的 socket;兼容配置仍支持只读挂载。

入口覆盖配置中的数据库程序目录、插件目录、数据目录、socket、PID、错误日志、复制日志与 server ID,以兼容现有 Bitnami 布局的配置文件。从库以 `read_only`、`super_read_only` 启动。选择 `mysql_native_password` 时会显式启用 MySQL 8.4 的该认证插件。

## 日志

MySQL 引擎错误日志通过上述链接写入 PID 1 的标准输出,初始化和正常运行期间都可以通过 `docker logs <容器名>` 或面板容器日志查看。日志文件路径保留用于兼容配置,不再是镜像内独立累积的普通文件。

入口同时输出新库初始化、已有数据启动、等待主库、全量备份与导入、复制配置等阶段的进度。等待主库期间按间隔报告重试进度;阶段与重试消息不输出密码或完整认证命令。

初始化配置完成表示入口已完成当前节点的初始化和配置步骤,不代表复制 IO、SQL 线程均已运行或数据已经追平。健康检查仅验证本地 SQL 连接;复制状态仍需通过面板主从状态或 `SHOW REPLICA STATUS\G` 检查,包括 `Replica_IO_Running`、`Replica_SQL_Running` 及对应错误字段。

## 初始化与重启

首次启动主库时创建 root 与复制账户。从库全量初始化通过 `mysqldump --single-transaction --source-data=2` 获取数据与对应 binlog 位点,导入后配置复制,从该位点继续接收增量。主库已有数据时也使用该流程。
Expand Down
Loading
Loading