在某些场景中, 当一个服务器的一块网卡上面配置了多个IP时, 例如虚拟IP, 可能想指定虚拟IP地址作为出口地址.
例如对这个虚拟IP有鉴权要求的场景. 如PostgreSQL的pg_hba.conf.
如在集中的流复制standby场景, 当集中的主机DOWN掉的话, 希望把虚拟IP切走, 同时生产机也只允许这个虚拟IP来访问的情况.
那么需要改写集中流复制的主机的路由表, 让其出口为虚拟IP.
默认情况下当有多个IP时, 路由是primary优先的.
如下, 默认的出口是
inet 172.16.3.111/24 brd 172.16.3.255 scope global eth0 而不是eth0:1
[root@db-172-16-3-111 network-scripts]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether b8:ca:3a:6d:fe:b0 brd ff:ff:ff:ff:ff:ff
inet 172.16.3.111/24 brd 172.16.3.255 scope global eth0
inet 172.16.3.1/24 brd 172.16.1.255 scope global secondary eth0:1
为了达到指定出口IP的目的, 可以通过修改路由表来实现.
[root@db-172-16-3-111 ~]# which ip
/sbin/ip
在没有创建eth0:1的情况下的默认路由如下 :
[root@db-172-16-3-111 ~]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0
新增一个IP地址
[root@db-172-16-3-111 ~]# cd /etc/sysconfig/network-scripts/
[root@db-172-16-3-111 network-scripts]# cp ifcfg-eth0 ifcfg-eth0:1
[root@db-172-16-3-111 network-scripts]# vi ifcfg-eth0:1
DEVICE="eth0:1"
BOOTPROTO="static"
IPADDR="172.16.3.105"
NETMASK="255.255.255.0"
HWADDR="B8:CA:3A:6D:FE:B0"
IPV6INIT="no"
MTU="1500"
NM_CONTROLLED="no"
ONBOOT="no"
TYPE="Ethernet"
UUID="37fb27ad-4293-43ba-a3b7-8e94fa563384"
[root@db-172-16-3-111 network-scripts]# ifup eth0:1
Determining if ip address 172.16.3.105 is already in use for device eth0...
[root@db-172-16-3-111 network-scripts]# ifconfig
eth0 Link encap:Ethernet HWaddr B8:CA:3A:6D:FE:B0
inet addr:172.16.3.111 Bcast:172.16.3.255 Mask:255.255.255.0
inet6 addr: fe80::baca:3aff:fe6d:feb0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:536391 errors:0 dropped:0 overruns:0 frame:0
TX packets:88849 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:718980563 (685.6 MiB) TX bytes:4883966 (4.6 MiB)
Memory:dcb00000-dcc00000
eth0:1 Link encap:Ethernet HWaddr B8:CA:3A:6D:FE:B0
inet addr:172.16.3.105 Bcast:172.16.3.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Memory:dcb00000-dcc00000
然后添加路由, 目的是增加一条metric=1的路由, 使得去整个广播域和网关走新增的这个地址出去.
[root@db-172-16-3-111 network-scripts]# ip route add default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
[root@db-172-16-3-111 network-scripts]# ip route add 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
然后删除原路由条目
[root@db-172-16-3-111 network-scripts]# ip route del default via 172.16.3.1 dev eth0
[root@db-172-16-3-111 network-scripts]# ip route del 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
注意, 最好增加2个条目, 如果虚拟IP切走的话, 还能正常走eth0出去的路由. 只是metric改大一点.
[root@db-172-16-3-111 network-scripts]# ip route add default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
[root@db-172-16-3-111 network-scripts]# ip route add 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
添加完后的路由表
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 metric 100
看看路由有没有生效, 已经使用了我们配置的metric=1的路由条目.
[root@db-172-16-3-111 network-scripts]# ip route get 192.168.1.1
192.168.1.1 via 172.16.3.1 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.1
172.16.3.1 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
关闭这个虚拟IP, 走metric=100的路由条目
[root@db-172-16-3-111 network-scripts]# ifdown eth0:1
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.1
172.16.3.1 dev eth0 src 172.16.3.111
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 192.168.1.1
192.168.1.1 via 172.16.3.1 dev eth0 src 172.16.3.111
cache mtu 1500 advmss 1460 hoplimit 64
关闭eth0:1后, 路由条目自动消失.
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0 metric 100
[root@db-172-16-3-111 network-scripts]# ifup eth0:1
Determining if ip address 172.16.3.105 is already in use for device eth0...
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0
default via 172.16.3.1 dev eth0 metric 100
如果虚拟IP切回来, 又要重新写路由, 比较土的方法是重启network服务, 然后执行前面一样的步骤添加路由.
[root@db-172-16-3-111 network-scripts]# service network restart
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: Determining if ip address 172.16.3.111 is already in use for device eth0...
[ OK ]
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0
[root@db-172-16-3-111 network-scripts]# ip route add default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
[root@db-172-16-3-111 network-scripts]# ip route add 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
[root@db-172-16-3-111 network-scripts]# ip route del default via 172.16.3.1 dev eth0
[root@db-172-16-3-111 network-scripts]# ip route del 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
[root@db-172-16-3-111 network-scripts]# ip route add default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
[root@db-172-16-3-111 network-scripts]# ip route add 172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
比较方便的方法是写配置文件, 或者是使用iptables .
配置文件比较多, 参考
/usr/share/doc/initscripts-9.03.40/sysconfig.txt
这里用到ifup-post调用的ifup-routes
/etc/sysconfig/network-scripts/route-<interface-name>
Contains lines that specify additional routes that should be added when the
associated interface is brought up.
The files are processed by the ifup-routes script and uses the /sbin/ipcalc
utility for all network masks and numbers. Routes are specified using the
syntax:
ADDRESSn=<network>
NETMASKn=<network/prefix mask>
GATEWAYn=<next-hop router/gateway IP address>
The "n" is expected to be consecutive positive integers starting from 0.
For example:
ADDRESS0=192.168.2.0
NETMASK0=255.255.255.0
GATEWAY0=192.168.1.1
adds a network route to the 192.168.2.0 network via the gateway at
192.168.1.1. Since you must already have a route to the network of the
gateway, there is no need to specify a device.
Note: The ifup-routes script also supports an older syntax designed to be
used directly as an argument to "/sbin/ip route add".
If no "ADDRESSn" lines are found the following will still
work:
192.168.2.0/24 dev ppp0
adds a network route to the 192.168.2.0 network through ppp0.
所以需要给绑定了多个IP的网络设备写一个自定义路由配置文件
/etc/sysconfig/network-scripts/route-eth0
内容如下 :
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
这个配置文件是在ifup-post后调用ip route add来添加路由.
但是仅此还有问题, 因为默认的话还是会添加2条路由, 网关和本网段的路由.
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
default via 172.16.3.1 dev eth0
[root@db-172-16-3-111 ~]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111 metric 100
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
为了去除这两条路由, 需要调整一下配置文件.
[最终的配置 : ]
把网关去掉
vi /etc/sysconfig/network
and
vi /etc/sysconfig/network-scripts/ifcfg-eth0
vi /etc/sysconfig/network-scripts/ifcfg-eth0:1
把GATEWAY=172.16.3.1删除
同时调整
/etc/sysconfig/network-scripts/route-eth0
最终的内容如下, 优先匹配小的子网掩码 :
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
172.16.3.0/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.128/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
重启network服务后, 路由表如下 :
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.128/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
ifup , ifdown, service network等都会自动读取这个配置文件, 所以不需要在手工修改了.
另外注意, 不要直接使用ip命令对IP进行配置.
[root@db-172-16-3-111 network-scripts]# ip route get 192.168.1.1
192.168.1.1 via 172.16.3.1 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.1
172.16.3.1 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.0
172.16.3.0 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.127
172.16.3.127 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ip route get 172.16.3.128
172.16.3.128 dev eth0 src 172.16.3.105
cache mtu 1500 advmss 1460 hoplimit 64
[root@db-172-16-3-111 network-scripts]# ifdown eth0:1
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
[root@db-172-16-3-111 network-scripts]# ifup eth0:1
Determining if ip address 172.16.3.105 is already in use for device eth0...
RTNETLINK answers: File exists
[root@db-172-16-3-111 network-scripts]# ip route
172.16.3.0/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.128/25 dev eth0 proto kernel scope link src 172.16.3.105 metric 1
172.16.3.0/24 dev eth0 proto kernel scope link src 172.16.3.111
169.254.0.0/16 dev eth0 scope link metric 1002
default via 172.16.3.1 dev eth0 src 172.16.3.105 metric 1
default via 172.16.3.1 dev eth0 src 172.16.3.111 metric 100
[参考]
1. /usr/share/doc/initscripts-9.03.40/sysconfig.txt
2. man ip
3. man ifup
