问题描述
正在尝试在Docker容器中引导一个MariaDB Galera集群。当设置network_mode: host时，以下配置可以正常工作，但是无法直接从其他容器内部访问MariaDB，并且他更希望使用Docker发布的端口而不是直接绑定到主机接口。每个容器都在单独的主机上运行。这不是Docker Swarm或Kubernetes设置，只是三个独立的带有Docker的主机。
基本上，我一直在将Bitnami Galera镜像文档中的命令转换为Ansible playbook。
tasks:
- name: Deploy mysql bootstrap container
docker_container:
name: mariadb
image: "bitnami/mariadb-galera:10.5"
pull: yes
restart_policy: "unless-stopped"
# network_mode: host
ports:
- "0.0.0.0:3306:3306"
- "0.0.0.0:4567:4567"
- "0.0.0.0:4567:4567/udp"
- "0.0.0.0:4568:4568"
- "0.0.0.0:4444:4444"
volumes:
- "{{ mariadb_datadir }}:/bitnami/mariadb"
env:
MARIADB_DATABASE: "{{ nextcloud_db_name }}"
MARIADB_USER: "{{ nextcloud_db_username }}"
MARIADB_PASSWORD: "{{ nextcloud_db_password }}"
MARIADB_ROOT_PASSWORD: "{{ mariadb_root_password }}"
MARIADB_GALERA_CLUSTER_NAME: "{{ mariadb_galera_cluster_name }}"
MARIADB_GALERA_MARIABACKUP_PASSWORD: "{{ mariadb_galera_mariabackup_password }}"
MARIADB_GALERA_CLUSTER_BOOTSTRAP: "{{ mariadb_galera_cluster_bootstrap }}"
MARIADB_REPLICATION_PASSWORD: "{{ mariadb_replication_password }}"
MARIADB_GALERA_CLUSTER_ADDRESS: "gcomm://"
when: inventory_hostname == groups[mariadb_galera_cluster_name][0]
- name: Wait for Galera Cluster status
shell:
cmd: "docker exec -i mariadb mysql -u root -p{{ mariadb_root_password }} -s -e \"SELECT variable_value FROM information_schema.global_status WHERE variable_name='WSREP_LOCAL_STATE_COMMENT'\""
register: galera_status
retries: 60
delay: 10
until: galera_status.stdout == 'Synced'
when: inventory_hostname == groups[mariadb_galera_cluster_name][0]
- name: Deploy mysql cluster containers
docker_container:
name: mariadb
image: "bitnami/mariadb-galera:10.5"
pull: yes
restart_policy: "unless-stopped"
# network_mode: host
ports:
- "0.0.0.0:3306:3306"
- "0.0.0.0:4567:4567"
- "0.0.0.0:4567:4567/udp"
- "0.0.0.0:4568:4568"
- "0.0.0.0:4444:4444"
volumes:
- "{{ mariadb_datadir }}:/bitnami/mariadb"
env:
MARIADB_ROOT_PASSWORD: "{{ mariadb_root_password }}"
MARIADB_GALERA_CLUSTER_NAME: "{{ mariadb_galera_cluster_name }}"
MARIADB_GALERA_MARIABACKUP_PASSWORD: "{{ mariadb_galera_mariabackup_password }}"
MARIADB_REPLICATION_PASSWORD: "{{ mariadb_replication_password }}"
MARIADB_GALERA_CLUSTER_ADDRESS: "gcomm://{{ groups[mariadb_galera_cluster_name] | map('extract', hostvars, ['ansible_default_ipv4', 'address']) | join(':4567,') }}:4567"
when: inventory_hostname != groups[mariadb_galera_cluster_name][0]
当我注释掉network_mode并直接发布端口时，容器可以启动，但节点无法加入集群，我收到很多这样的消息：
2021-07-28 9:31:51 0 [Warning] WSREP: Member 0.0 (f38e8cff30b6) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
以及以下内容：
WSREP_SST: [ERROR] Possible timeout in receiving first data from donor in gtid stage: exit codes: 124 0 (20210728 09:31:22.716)
以下是其中一个节点的完整日志：
“`
mariadb 09:26:21.27mariadb 09:26:21.28 Welcome to the Bitnami mariadb-galera containermariadb 09:26:21.28 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-mariadb-galeramariadb 09:26:21.29 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-mariadb-galera/issuesmariadb 09:26:21.29mariadb 09:26:21.29 INFO ==> ** Starting MariaDB setup mariadb 09:26:21.31 INFO ==> Validating settings in MYSQL_/MARIADB_ env varsmariadb 09:26:21.34 INFO ==> Initializing mariadb databasemariadb 09:26:21.36 INFO ==> Updating ‘my.cnf’ with custom configurationmariadb 09:26:21.37 INFO ==> Setting wsrep_node_name optionmariadb 09:26:21.38 INFO ==> Setting wsrep_node_address optionmariadb 09:26:21.39 INFO ==> Setting wsrep_cluster_name optionmariadb 09:26:21.42 INFO ==> Setting wsrep_cluster_address optionmariadb 09:26:21.43 INFO ==> Setting wsrep_sst_auth optionmariadb 09:26:21.45 INFO ==> ** MariaDB setup finished! mariadb 09:26:21.49 INFO ==> ** Starting MariaDB **mariadb 09:26:21.49 INFO ==> Setting previous boot2021-07-28 9:26:21 0 [Note] /opt/bitnami/mariadb/sbin/mysqld (mysqld 10.5.11-MariaDB-log) starting as process 1 …2021-07-28 9:26:21 0 [Note] WSREP: Loading provider /opt/bitnami/mariadb/lib/libgalera_smm.so initial position: 00000000-0000-0000-0000-000000000000:-12021-07-28 9:26:21 0 [Note] WSREP: wsrep_load(): loading provider library ‘/opt/bitnami/mariadb/lib/libgalera_smm.so’2021-07-28 9:26:21 0 [Note] WSREP: wsrep_load(): Galera 4.8(rXXXX) by Codership Oy info@codership.com loaded successfully.2021-07-28 9:26:21 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration.2021-07-28 9:26:21 0 [Warning] WSREP: Could not open state file for reading: ‘/bitnami/mariadb/data//grastate.dat’2021-07-28 9:26:21 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 12021-07-28 9:26:21 0 [Note] WSREP: GCache DEBUG: opened preamble:Version: 0UUID: 00000000-0000-0000-0000-000000000000Seqno: -1 – -1Offset: -1Synced: 02021-07-28 9:26:21 0 [Note] WSREP: Skipped GCache ring buffer recovery: could not determine history UUID.2021-07-28 9:26:21 0 [Note] WSREP: Passing config to GCS: base_dir = /bitnami/mariadb/data/; base_host = 172.17.0.2; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /bitnami/mariadb/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc2021-07-28 9:26:21 0 [Note] WSREP: Start replication2021-07-28 9:26:21 0 [Note] WSREP: Connecting with bootstrap option: 02021-07-28 9:26:21 0 [Note] WSREP: Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-12021-07-28 9:26:21 0 [Note] WSREP: protonet asio version 02021-07-28 9:26:21 0 [Note] WSREP: Using CRC-32C for message checksums.2021-07-28 9:26:21 0 [Note] WSREP: backend: asio2021-07-28 9:26:21 0 [Note] WSREP: gcomm thread scheduling priority set to other:02021-07-28 9:26:21 0 [Warning] WSREP: access file(/bitnami/mariadb/data//gvwstate.dat) failed(No such file or directory)2021-07-28 9:26:21 0 [Note] WSREP: restore pc from disk failed2021-07-28 9:26:21 0 [Note] WSREP: GMCast version 02021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) listening at tcp://0.0.0.0:45672021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) multicast: , ttl: 12021-07-28 9:26:21 0 [Note] WSREP: EVS version 12021-07-28 9:26:21 0 [Note] WSREP: gcomm: connecting to group ‘nextcloud_int’, peer ‘10.0.28.26:4567,10.0.28.31:4567,10.0.28.32:4567’2021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) Found matching local endpoint for a connection, blacklisting address tcp://10.0.28.31:45672021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) connection established to d9e457ac-8a0d tcp://10.0.28.26:45672021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) turning message relay requesting on, nonlive peers:2021-07-28 9:26:21 0 [Note] WSREP: (def73391-b951, ‘tcp://0.0.0.0:4567’) connection established to df17c022-aae4 tcp://10.0.28.32:45672021-07-28 9:26:22 0 [Note] WSREP: EVS version upgrade 0 -> 12021-07-28 9:26:22 0 [Note] WSREP: declaring d9e457ac-8a0d at tcp://10.0.28.26:4567 stable2021-07-28 9:26:22 0 [Note] WSREP: declaring df17c022-aae4 at tcp://10.0.28.32:4567 stable2021-07-28 9:26:22 0 [Note] WSREP: PC protocol upgrade 0 -> 12021-07-28 9:26:22 0 [Note] WSREP: Node d9e457ac-8a0d state prim2021-07-28 9:26:22 0 [Note] WSREP: view(view_id(PRIM,d9e457ac-8a0d,2) memb { d9e457ac-8a0d,0 def73391-b951,0 df17c022-aae4,0} joined {} left {} partitioned {})2021-07-28 9:26:22 0 [Note] WSREP: save pc into disk2021-07-28 9:26:22 0 [Note] WSREP: gcomm: connected2021-07-28 9:26:22 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 326362021-07-28 9:26:22 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)2021-07-28 9:26:22 0 [Note] WSREP: Opened channel ‘nextcloud_int’2021-07-28 9:26:22 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 32021-07-28 9:26:22 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.2021-07-28 9:26:22 1 [Note] WSREP: Starting rollbacker thread 12021-07-28 9:26:22 0 [Note] WSREP: STATE EXCHANGE: sent state msg: df444d26-ef85-11eb-8936-be401295de772021-07-28 9:26:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: df444d26-ef85-11eb-8936-be401295de77 from 0 (e4a7a8af0b78)2021-07-28 9:26:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: df444d26-ef85-11eb-8936-be401295de77 from 2 (cd8aaaad506b)2021-07-28 9:26:22 2 [Note] WSREP: Starting applier thread 22021-07-28 9:26:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: df444d26-ef85-11eb-8936-be401295de77 from 1 (f38e8cff30b6)2021-07-28 9:26:22 0 [Note] WSREP: Quorum results: version = 6, component = PRIMARY, conf_id = 1, members = 1/3 (joined/total), act_id = 15, last_appl. = 14, protocols = 2/10/4 (gcs/repl/appl), vote policy= 0, group UUID = d6aa7099-ef85-11eb-9fbe-dbb821267db32021-07-28 9:26:22 0 [Note] WSREP: Flow-control interval: [28, 28]2021-07-28 9:26:22 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 16)2021-07-28 9:26:22 2 [Note] WSREP: ####### processing CC 16, local, ordered2021-07-28 9:26:22 2 [Note] WSREP: Process first view: d6aa7099-ef85-11eb-9fbe-dbb821267db3 my uuid: def73391-ef85-11eb-b951-cb71f524bed32021-07-28 9:26:22 2 [Note] WSREP: Server f38e8cff30b6 connected to cluster at position d6aa7099-ef85-11eb-9fbe-dbb821267db3:16 with ID def73391-ef85-11eb-b951-cb71f524bed32021-07-28 9:26:22 2 [Note] WSREP: Server status change disconnected -> connected2021-07-28 9:26:22 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.2021-07-28 9:26:22 2 [Note] WSREP: ####### My UUID: def73391-ef85-11eb-b951-cb71f524bed32021-07-28 9:26:22 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes2021-07-28 9:26:22 0 [Note] WSREP: Service thread queue flushed.2021-07-28 9:26:22 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -12021-07-28 9:26:22 2 [Note] WSREP: State transfer required: Group state: d6aa7099-ef85-11eb-9fbe-dbb821267db3:16 Local state: 00000000-0000-0000-0000-000000000000:-12021-07-28 9:26:22 2 [Note] WSREP: Server status change connected -> joiner2021-07-28 9:26:22 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.2021-07-28 9:26:22 0 [Note] WSREP: Joiner monitor thread started to monitor2021-07-28 9:26:22 0 [Note] WSREP: Running: ‘wsrep_sst_mariabackup –role ‘joiner’ –address ‘172.17.0.2’ –datadir ‘/bitnami/mariadb/data/’ –defaults-file ‘/opt/bitnami/mariadb/conf/my.cnf’ –parent ‘1’ –binlog ‘mysql-bin’ –mysqld-args –defaults-file=/opt/bitnami/mariadb/conf/my.cnf –basedir=/opt/bitnami/mariadb –datadir=/bitnami/mariadb/data –socket=/opt/bitnami/mariadb/tmp/mysql.sock –pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid’
WSREP_SST: [INFO] SSL configuration: CA=”, CERT=”, KEY=”, MODE=’DISABLED’, encrypt=’0′ (20210728 09:26:22.594)
WSREP_SST: [INFO] Streaming with mbstream (20210728 09:26:22.684)
WSREP_SST: [INFO] Using socat as streamer (20210728 09:26:22.688)
WSREP_SST: [INFO] Evaluating timeout -k 310 300 socat -u TCP-LISTEN:4444,reuseaddr stdio | ‘/opt/bitnami/mariadb//bin/mbstream’ -x; RC=( ${PIPESTATUS[@]} ) (20210728 09:26:22.709)
2021-07-28 9:26:23 2 [Note] WSREP: ####### processing CC 16, local, ordered
2021-07-28 9:26:23 2 [Note] WSREP: Process first view: d6aa7099-ef85-11eb-9fbe-dbb821267db3 my uuid: def73391-ef85-11eb-b951-cb71f524bed3
2021-07-28 9:26:23 2 [Note] WSREP: Server f38e8cff30b6 connected to cluster at position d6aa7099-ef85-11eb-9fbe-dbb821267db3:16 with ID def73391-ef85-11eb-b951-cb71f524bed3
2021-07-28 9:26:23 2 [Note] WSREP: Server status change disconnected -> connected
2021-07-28 9:26:23 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2021-07-28 9:26:23 2 [Note] WSREP: ####### My UUID: def73391-ef85-11eb-b951-cb71f524bed3
2021-07-28 9:26:23 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes
2021-07-28 9:26:23 2 [Note] WSREP: State transfer required:
Group state: d6aa7099-ef85-11eb-9fbe-dbb821267db3:16
Local state: 00000000-0000-0000-0000-000000000000:-1
2021-07-28 9:26:23 2 [Note] WSREP: Server status change connected -> joiner
2021-07-28 9:26:23 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2021-07-28 9:26:23 0 [Note] WSREP: Joiner monitor thread started to monitor
2021-07-28 9:26:23 0 [Note] WSREP: Running: ‘wsrep_sst_mariabackup –role ‘joiner’ –address ‘172.17.0.2’ –datadir ‘/bitnami/mariadb/data/’ –defaults-file ‘/opt/bitnami/mariadb/conf/my.cnf’ –parent ‘1’ –binlog ‘mysql-bin’ –mysqld-args –defaults-file=/opt/bitnami/mariadb/conf/my.cnf –basedir=/opt/bitnami/mariadb –datadir=/bitnami/mariadb/data –socket=/opt/bitnami/mariadb/tmp/mysql.sock –pid-file=/opt/bitnami/mariadb/tmp/mysqld.pid’
WSREP_SST: [INFO] SSL configuration: CA=”, CERT=”, KEY=”, MODE=’DISABLED’, encrypt=’0′ (20210728 09:26:22.594)
WSREP_SST: [INFO] Streaming with mbstream (20210728 09:26:22.684)
WSREP_SST: [INFO] Using socat as streamer (20210728 09:26:22.688)
WSREP_SST: [INFO] Evaluating timeout -k 310 300 socat -u TCP-LISTEN:4444,reuseaddr stdio | ‘/opt/bitnami/mariadb//bin/mbstream’ -x; RC=( ${PIPESTATUS[@]}