配置Docker多台宿主机间的容器互联-原创手记-慕课网

如果要在生产和测试环境大规模采用docker技术，首先就需要解决不同物理机建的docker容器互联问题。本文介绍在centos7环境下采用open vswitch实现不同物理服务器上的docker容器互联的配置。

环境介绍：

Server1: 192.168.115.5/24

Server2: 192.168.115.6/24

一：在Server1和Server2上分别用rpm方式安装docker并启动服务

# yum list *docker*

# yum -y install docker

# service docker start

Redirecting to /bin/systemctl start docker.service

wKiom1S-I_Hh2bgsAAGXa1qq8Rw744.jpg

二: 在Server1和Server2上分别安装open vswitch

# yum -y install wget openssl-devel kernel-devel

# yum groupinstall "Development Tools"

# adduser ovswitch

# su - ovswitch

$ wget http://openvswitch.org/releases/openvswitch-2.3.0.tar.gz

$ tar -zxvpf openvswitch-2.3.0.tar.gz

$ mkdir -p ~/rpmbuild/SOURCES

$ sed 's/openvswitch-kmod, //g' openvswitch-2.3.0/rhel/openvswitch.spec > openvswitch-2.3.0/rhel/openvswitch_no_kmod.spec

$ cp openvswitch-2.3.0.tar.gz rpmbuild/SOURCES/

$ rpmbuild -bb --without check ~/openvswitch-2.3.0/rhel/openvswitch_no_kmod.spec

$ exit

# yum localinstall /home/ovswitch/rpmbuild/RPMS/x86_64/openvswitch-2.3.0-1.x86_64.rpm

# mkdir /etc/openvswitch

# setenforce 0

# systemctl start openvswitch.service

# systemctl status openvswitch.service -l

wKioL1S-JPCgVO1JAA_49emGvvo761.jpg

三：在Server1和Server2上建立OVS Bridge并配置路由

Server1: docker容器内网ip网段172.17.1.0/24

# cat /proc/sys/net/ipv4/ip_forward

# ovs-vsctl add-br obr0

# ovs-vsctl add-port obr0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.115.5

# brctl addbr kbr0

# brctl addif kbr0 obr0

# ip link set dev docker0 down

# ip link del dev docker0

# vi /etc/sysconfig/network-scripts/ifcfg-kbr0

ONBOOT=yes

BOOTPROTO=static

IPADDR=172.17.1.1

NETMASK=255.255.255.0

GATEWAY=172.17.1.0

USERCTL=no

TYPE=Bridge

IPV6INIT=no

# cat /etc/sysconfig/network-scripts/route-ens32

172.17.2.0/24 via 192.168.115.6 dev ens32

# systemctl restart network.service

wKiom1S-JGLztcVOAAkVfvfiakg521.jpgwKioL1S-JUKhpM98AAMPTzyeXDA526.jpg

Server2: docker容器内网ip网段172.17.2.0/24

# cat /proc/sys/net/ipv4/ip_forward

# ovs-vsctl add-br obr0

# ovs-vsctl add-port obr0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.115.6

# brctl addbr kbr0

# brctl addif kbr0 obr0

# ip link set dev docker0 down

# ip link del dev docker0

# vi /etc/sysconfig/network-scripts/ifcfg-kbr0

ONBOOT=yes

BOOTPROTO=static

IPADDR=172.17.2.1

NETMASK=255.255.255.0

GATEWAY=172.17.2.0

USERCTL=no

TYPE=Bridge

IPV6INIT=no

# cat /etc/sysconfig/network-scripts/route-ens32

172.17.1.0/24 via 192.168.115.5 dev ens32

# systemctl restart network.service

wKiom1S-JJ3DpEGhAAk8rJcGxhs651.jpgwKioL1S-JYSi2UT_AAM9JNqom84252.jpg

四:启动容器测试

Server1和Server2上修改docker启动的虚拟网卡绑定为kbr0，重启docker进程

wKiom1S-JMvzlMcuAAGyypVRvSs783.jpg

Server1:

# docker run -idt --name test1 registry.fjhb.cn/centos6 /bin/bash

wKioL1S-JbCSshcMAAYrxbYbJzU457.jpg

Server2:

# docker run -idt --name test2 registry.fjhb.cn/centos6 /bin/bash

wKiom1S-JRujzHceAAffwv6cFsU079.jpg

Server1:

wKioL1S-JgKi_26TAALMRW9LYKQ999.jpg

open vswitchdockercentos7Docker