一、架构
二、唠叨一会原理：
1、nginx
Nginx进程基于于Master+Slave(worker)多进程模型，自身具有非常稳定的子进程管理功能。在Master进程分配模式下，Master进程永远不进行业务处理，只是进行任务分发，
从而达到Master进程的存活高可靠性，Slave(worker)进程所有的业务信号都 由主进程发出，Slave(worker)进程所有的超时任务都会被Master中止，属于非阻塞式任务模型。
2、keepalived
Keepalived是Linux下面实现VRRP 备份路由的高可靠性运行件。基于Keepalived设计的服务模式能够真正做到主服务器和备份服务器故障时IP瞬间无缝交接,作用:
主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现
3、单点故障
Nginx有很强代理功能，但是一台nginx就形成了单点，现在使用keepalived来解决这个问题，keepalived的故障转移时间很短.
Nginx+keepalived双机实现nginx反向代理服务的高可用,一台nginx挂掉之后不影响应用也不影响内网访问外网.
4、此架构需要考虑的问题
1) Master没挂，则Master占有vip且nginx运行在Master上
2) Master挂了，则backup抢占vip且在backup上运行nginx服务
3) 如果master服务器上的nginx服务挂了，则vip资源转移到backup服务器上
4) 检测后端服务器的健康状态
5、叙述
Master和Backup两边都开启nginx服务，无论Master还是Backup，当其中的一个keepalived服务停止后，vip都会漂移到keepalived服务还在的节点上，
如果要想使nginx服务挂了，vip也漂移到另一个节点，则必须用脚本或者在配置文件里面用shell命令来控制。
首先必须明确后端服务器的健康状态检测keepalived在这种架构上是无法检测的，后端服务器的健康状态检测是有nginx来判断的，但是nginx的检测机制有一定的缺陷，后端服务器某一个宕机之后，nginx还是会分发请求给它，在一定的时间内后端服务响应不了，nginx则会发给另外一个服务器，然后当客户的请求来了，nginx会一段时间内不会把请求分发给已经宕机的服务器，但是过一段时间后，nginx还是会把分发请求发给宕机的服务器上。
三、准备工作
|
1
2
3
4
5
6
7
8
9
|
1、环境
系统版本：CentOS 6.4 64bit
nginx版本：nginx-1.4.7.tar.gz
keepalived版本：keepalived-1.2.7.tar.gz
2、地址规划
nginx+keepalived主 192.168.0.189 master
nginx+keepalived辅 192.168.0.200 badkup
vip 192.168.0.250 vip
|
四、安装nginx和keepalived服务
1、安装nginx服务[分别在master机器和backup机器安装nginx服务]
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
|
#！/bin/bash
# time 2014-08-18
# Allentuns
#安装依赖环境包
yum -y install gcc gcc-c++ make unzip wget vim openssh-clients
#支持重写rewrite,正则表达式
unzip pcre-8.33.zip
cd pcre-8.33
./configure --prefix=/usr/local/pcre
make && make install
cd ..
#支持网页压缩
tar xf zlib-1.2.8.tar.gz
cd zlib-1.2.8
./configure --prefix=/usr/local/zlib
make && make install
cd ..
#支持https
tar xf openssl-1.0.0l.tar.gz
cd openssl-1.0.0l
./config --prefix=/usr/local/openssl
make && make install
cd ..
#安装mp4和flv模块
tar xf nginx_mod_h264_streaming-2.2.7_update.tar.gz
#创建运行nginx服务的账号和组
groupadd -r nginx
useradd -g nginx -r -s /sbin/nologin nginx
#安装nginx
tar xf nginx-1.4.7.tar.gz
cd nginx-1.4.7
#启用flv和mp4功能
./configure \
--user=nginx \
--group=nginx \
--prefix=/usr/local/nginx \
--with-pcre=../pcre-8.33 \
--with-zlib=../zlib-1.2.8 \
--with-openssl=../openssl-1.0.0l \
--with-http_stub_status_module \
--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--with-http_gzip_static_module \
--http-proxy-temp-path=/var/tmp/nginx/proxy/ \
--http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \
--http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
--http-scgi-temp-path=/var/tmp/nginx/scgi \
--with-ipv6 \
--with-http_flv_module \
--with-http_mp4_module
make && make install
cd ..
#提供nginx启动脚本
cat > /etc/init.d/nginx << EOF
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemin
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /usr/local/nginx/conf/nginx.conf
# pidfile: /usr/local/nginx/logs/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[
"\$NETWORKING"
=
"no"
] && exit 0
nginx=
"/usr/local/nginx/sbin/nginx"
prog=\$(basename \$nginx)
NGINX_CONF_FILE=
"/usr/local/nginx/conf/nginx.conf"
lockfile=/var/lock/subsys/nginx
start() {
[ -x \$nginx ] || exit 5
[ -f \$NGINX_CONF_FILE ] || exit 6
echo -n \$
"Starting \$prog: "
daemon \$nginx -c \$NGINX_CONF_FILE
retval=\$?
echo
[ \$retval -eq 0 ] && touch \$lockfile
return \$retval
}
stop() {
echo -n \$
"Stopping \$prog: "
killproc \$prog -QUIT
retval=\$?
echo
[ \$retval -eq 0 ] && rm -f \$lockfile
return \$retval
}
restart() {
configtest || return \$?
stop
start
}
reload() {
configtest || return \$?
echo -n \$
"Reloading \$prog: "
killproc \$nginx -HUP
RETVAL=\$?
echo
}
force_reload() {
restart
}
configtest() {
\$nginx -t -c \$NGINX_CONF_FILE
}
rh_status() {
status \$prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case
"\$1"
in
start)
rh_status_q && exit 0
\$1
;;
stop)
rh_status_q || exit 0
\$1
;;
restart|configtest)
\$1
;;
reload)
rh_status_q || exit 7
\$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo \$
"Usage: \$0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
EOF
##赋予nginx执行的权限
chmod +x /etc/init.d/nginx
chkconfig --add nginx
#创建文件
mkdir -p /var/tmp/nginx/proxy
#启动nginx服务
/etc/init.d/nginx restart
#查看nginx状态
/bin/ps -ef |grep nginx
|
2、分别在两台机器上创建不同的测试页面[为了测试]
|
1
2
3
4
|
#在master机器创建index.html页面
echo
"Welcome To 192.168.0.189"
> /usr/local/nginx/html/index.html
#在backup机器创建index.html页面
echo
"Welcome To 192.168.0.200"
> /usr/local/nginx/html/index.html
|
3、安装keepalived
#至于keepalived的安装可以选择源码编译安装或者yum源安装，都可以；这里选择源码编译安装
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
#安装keepalived源码安装的依赖包
yum -y install openssl openssl-devel
yum -y install popt-devel
#源码编译安装keepalived
tar xf keepalived-1.2.7.tar.gz
cd keepalived-1.2.7
./configure --prefix=/usr/local/keepalived
make && make install
#拷贝keepavlied的一些文件到相关目录中
mkdir /etc/keepalived
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
|
修改keepalived配置文件
keepalived的文件路径/etc/keepalived/keepalived.conf
#####【Master-keepalived】#####
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
! Configuration File for keepalived #全局定义
global_defs {
notification_email { #指定keepalived在发生事件时(比如切换),需要发送的email对象,可以多个,每行一个
zhengyansheng@cdvcloud.com
}
notification_email_from admin@cdvcloud.com
smtp_server 127.0.0.1 #指定发送email的smtp服务器
smtp_connect_timeout 30
router_id LVS_DEVEL #运行keepalived的机器的一个标识
}
vrrp_instance VI_1 {
state MASTER #为主服务器
interface eth1 #监听的本地网卡接口
virtual_router_id 51 #主辅virtual_router_id号必须相同
mcast_src_ip=192.168.0.189 #主nginx的ip地址
priority 100 #优先级为100，此值越大优先级越大 就为master 权重值
advert_int 1 #VRRP Multicast 广播周期秒数；心跳检测时间，单位秒
authentication {
auth_type PASS #vrrp认证方式
auth_pass 1111 #vrrp口令
}
virtual_ipaddress { #VRRP HA 虚拟地址 如果有多个VIP，继续换行填写
192.168.0.250/24 dev eth1
}
}
|
#####【Backup-keepalived】#####
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
! Configuration File for keepalived
global_defs {
notification_email {
zhengyansheng@cdvcloud.com
}
notification_email_from admin@cdvcloud.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 51
mcast_src_ip=192.168.0.200
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.250/24 dev eth1
}
}
|
启动keepalived服务[所有的服务开机启动]
/etc/init.d/keepalived start
chkconfig nginx on
chkconfig keepalived on
查看vip状态
#首先在master节点上查看vip的状态
|
1
2
3
4
5
6
7
8
9
10
11
12
|
[root@master ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:59:45:de brd ff:ff:ff:ff:ff:ff
inet 192.168.0.189/24 brd 192.168.0.255 scope global eth1
inet 192.168.0.250/32 scope global eth1 #vip地址已经加入到master节点上
inet6 fe80::20c:29ff:fe59:45de/64 scope link
valid_lft forever preferred_lft forever
|
#其次在backup节点上查看vip的状态
|
1
2
3
4
5
6
7
8
9
10
11
|
[root@backup ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:61:70:d3 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.200/24 brd 192.168.0.255 scope global eth1
inet6 fe80::20c:29ff:fe61:70d3/64 scope link
valid_lft forever preferred_lft forever
|
学会看日志输出
一部分：当启动keepalived服务的时候，会根据配置文件的优先级来竞选谁为master，从日志来看192.168.0.189竞选master
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
|
[root@master ~]# /etc/init.d/keepalived start
正在启动 keepalived： [确定]
[root@master ~]# tail -f /var/log/messages
Aug 16 11:09:48 localhost Keepalived[25707]: Starting Keepalived v1.2.7 (08/16,2014)
Aug 16 11:09:48 localhost Keepalived[25708]: Starting Healthcheck child process, pid=25710
Aug 16 11:09:48 localhost Keepalived[25708]: Starting VRRP child process, pid=25711
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]:
Interface
queue is empty
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Netlink reflector reports IP 192.168.0.189 added
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Netlink reflector reports IP fe80::20c:29ff:fe59:45de added
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Registering Kernel netlink reflector
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Registering Kernel netlink command channel
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Registering gratuitous ARP shared channel
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]:
Interface
queue is empty
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Netlink reflector reports IP 192.168.0.189 added
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Netlink reflector reports IP fe80::20c:29ff:fe59:45de added
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Registering Kernel netlink reflector
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Registering Kernel netlink command channel
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Opening file
'/etc/keepalived/keepalived.conf'.
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Configuration is using : 62766 Bytes
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: Using LinkWatch kernel netlink reflector...
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Opening file
'/etc/keepalived/keepalived.conf'.
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Configuration is using : 7073 Bytes
Aug 16 11:09:48 localhost Keepalived_vrrp[25711]: VRRP sockpool: [ifindex(2), proto(112), fd(11,12)]
Aug 16 11:09:48 localhost Keepalived_healthcheckers[25710]: Using LinkWatch kernel netlink reflector...
Aug 16 11:09:51 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 16 11:09:56 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 16 11:09:56 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 16 11:09:56 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 192.168.0.250
Aug 16 11:09:56 localhost Keepalived_healthcheckers[25710]: Netlink reflector reports IP 192.168.0.250 added
Aug 16 11:10:01 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 192.168.0.250
[root@backup ~]# tail -f /var/log/messages
Aug 16 11:09:51 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Received higher prio advert
Aug 16 11:09:51 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Entering BACKUP STATE
Aug 16 11:09:51 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) removing protocol VIPs.
Aug 16 11:09:51 localhost Keepalived_healthcheckers[26135]: Netlink reflector reports IP 192.168.0.250 removed
[root@master ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:59:45:de brd ff:ff:ff:ff:ff:ff
inet 192.168.0.189/24 brd 192.168.0.255 scope global eth1
inet 192.168.0.250/32 scope global eth1
inet6 fe80::20c:29ff:fe59:45de/64 scope link
valid_lft forever preferred_lft forever
二部分：当我们停掉主keepalived服务的时候，主master的vip会自动切换到辅机器上，看日志
停止主keepalived服务
[root@master ~]# /etc/init.d/keepalived stop
停止 keepalived： [确定]
[root@master ~]# tail -f /var/log/messages
Aug 16 11:16:14 localhost Keepalived[25708]: Stopping Keepalived v1.2.7 (08/16,2014)
Aug 16 11:16:14 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) sending 0 priority
Aug 16 11:16:14 localhost Keepalived_vrrp[25711]: VRRP_Instance(VI_1) removing protocol VIPs.
[root@backup ~]# tail -f /var/log/messages
Aug 16 11:09:51 localhost Keepalived_healthcheckers[26135]: Netlink reflector reports IP 192.168.0.250 removed
Aug 16 11:16:14 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 16 11:16:19 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 16 11:16:19 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 16 11:16:19 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 192.168.0.250
Aug 16 11:16:19 localhost Keepalived_healthcheckers[26135]: Netlink reflector reports IP 192.168.0.250 added
Aug 16 11:16:24 localhost Keepalived_vrrp[26136]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 192.168.0.250
[root@backup ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:61:70:d3 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.200/24 brd 192.168.0.255 scope global eth1
inet 192.168.0.250/32 scope global eth1 辅节点已经成功接管vip
inet6 fe80::20c:29ff:fe61:70d3/64 scope link
valid_lft forever preferred_lft forever
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
测试1：nginx主备切换的时间
C:\Users\Allentuns>ping 192.168.0.250 -t
正在 Ping 192.168.0.250 具有 32 字节的数据:
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
请求超时。
请求超时。
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.0.250 的回复: 字节=32 时间<1ms TTL=64
192.168.0.250 的 Ping 统计信息:
数据包: 已发送 = 28，已接收 = 26，丢失 = 2 (7% 丢失)，
往返行程的估计时间(以毫秒为单位):
最短 = 0ms，最长 = 0ms，平均 = 0ms
Control-C
^C
C:\Users\Allentuns>
|
以上keepalived的配置文件并不能满足整个架构：
因为默认情况下，keepalived工作模式并不能直接监控nginx服务，只有当keepalived服务挂掉后才能主备切换，nginx服务挂掉后不能实现主备服务器的切换，但是我们的目的就是要实现nginx服务keepalived挂掉后，都要主备切换。
以上有两种方法可以实现
-
keepalived配置文件中可以支持shell脚本，写个监听nginx服务的脚本就可以了
-
单独写个脚本来监听nginx和keepalived服务
基于第一种情况
#####【Master-keepalived】#####
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
|
! Configuration File for keepalived #全局定义
global_defs {
notification_email { #指定keepalived在发生事件时(比如切换),需要发送的email对象,可以多个,每行一个
zhengyansheng@cdvcloud.com
}
notification_email_from admin@cdvcloud.com
smtp_server 127.0.0.1 #指定发送email的smtp服务器
smtp_connect_timeout 30
router_id LVS_DEVEL #运行keepalived的机器的一个标识
}
vrrp_script
chk_nginx { #检测nginx服务是否在运行有很多方式，比如进程，用脚本检测等等
script
"killall -0 nginx"
#用shell命令检查nginx服务是否存在
interval 1 #时间间隔为1秒检测一次
weight -2 #当nginx的服务不存在了，就把当前的权重-2
fall 2 #测试失败的次数
rise 1 #测试成功的次数
}
vrrp_instance VI_1 {
state MASTER #为主服务器
interface eth1 #监听的本地网卡接口
virtual_router_id 51 #主辅virtual_router_id号必须相同
mcast_src_ip=192.168.0.189 #主nginx的ip地址
priority 100 #优先级为100，此值越大优先级越大 就为master 权重值
advert_int 1 #VRRP Multicast 广播周期秒数；心跳检测时间，单位秒
authentication {
auth_type PASS #vrrp认证方式
auth_pass 1111 #vrrp口令
}
virtual_ipaddress { #VRRP HA 虚拟地址 如果有多个VIP，继续换行填写
192.168.0.250/24 dev eth1
}
track_script {
chk_nginx #引用上面的vrrp_script定义的脚本名称
}
}
|
#####【Backup-keepalived】#####
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
|
! Configuration File for keepalived
global_defs {
notification_email {
zhengyansheng@cdvcloud.com
}
notification_email_from admin@cdvcloud.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script
chk_nginx { #检测nginx服务是否在运行有很多方式，比如进程，用脚本检测等等
script
"killall -0 nginx"
#用shell命令检查nginx服务是否存在
interval 1 #时间间隔为1秒检测一次
weight -2 #当nginx的服务不存在了，就把当前的权重-2
fall 2 #测试失败的次数
rise 1 #测试成功的次数
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 51
mcast_src_ip=192.168.0.200
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.250/24 dev eth1
}
track_script {
chk_nginx #引用上面的vrrp_script定义的脚本名称
}
}
#这种情况的监听脚本是2秒中监听一次nginx服务，如果进程不存在，则将keepalived的权重减少2，为98，那么权重为99的为主 自动切换到备用服务器上
|
基于第二种情况
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
#!/bin/bash
while :
do
nginxpid=`ps -C nginx --no-header | wc -l`
if [ $nginxpid -eq 0 ];then
/usr/local/nginx/sbin/nginx
sleep 5
nginxpid=`ps -C nginx --no-header | wc -l`
echo $nginxpid
if [ $nginxpid -eq 0 ];then
/etc/init.d/keepalived stop
fi
fi
sleep 5
done
#这个脚本确实很好用，来自酒哥的博文
|
至于后端tomcat的负载均衡和主备切换在这里就不多说了！很简单。。。。 网上有很多这方面的教程
我会在下周末之前把nginx的双主架构和hadoop的源码编译贡献给博客上。
博文的笔记和所有的软件都已经上传！期待和大家交流更多linux运维 QQ 467754239
http://yunpan.cn/QaSn4mzCd65YP 访问密码 06b3
----------补充时间 2015-01-14----------
主Nginx机器之一的Keepalived.conf配置文件如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
! Configuration File
for
keepalived
global_defs {
notification_email {
yuhongchun027@163.com
}
notification_email_from keepalived@chtopnet.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1paituan.com
}
virtual_ipaddress {
192.168.1.8
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 52
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1paituan.com
}
virtual_ipaddress {
192.168.1.9
}
}
|
主Nginx之二的keepalivd.conf配置文件如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
! Configuration File
for
keepalived
global_defs {
notification_email {
yuhongchun027@163.com
}
notification_email_from keepalived@chtopnet.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1paituan
}
virtual_ipaddress {
192.168.1.8
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1paituan
}
virtual_ipaddress {
192.168.1.9
}
}
|
二台机器的监控Nginx的进程脚本，脚本内容如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
#!/bin/bash
while
:
do
nginxpid=`
ps
-C nginx --no-header |
wc
-l`
if
[ $nginxpid -
eq
0 ];
then
/usr/local/nginx/sbin/nginx
sleep
5
nginxpid=`
ps
-C nginx --no-header |
wc
-l`
echo
$nginxpid
if
[ $nginxpid -
eq
0 ];
then
/etc/init
.d
/keepalived
stop
fi
fi
sleep
5
done
|
|
1
|
nohup
sh
/root/nginxpid
.sh &
|
此脚本我是直接从生产服务器上下载的，大家不要怀疑它会引起死循环和有效性的问题，我稍为解释一下，这是一个无限循环的脚本，放在主Nginx机器上（因为目前主要是由它提供服务），每隔5秒执行一次，用ps -C 命令来收集nginx的PID值到底是否为0，如果是0的话（即Nginx进程死掉了），尝试启动nginx进程；如果继续为0，即nginx启动失改， 则关闭本机的Keeplaived进程，VIP地址则会由备机接管，当然了，整个网站就会由备机的Nginx来提供服务了，这样保证Nginx进程的高可用。
最近工作不是很忙，花时间做了个实验，来验证vip能不能是公网地址中的一个或者多个。
验证的结果是这样的：如果按照上图来分配地址，使用公网地址117.25.36.41和117.25.36.43中的任意一个来充当vip的后果是，物理网卡上eth1上的地址就会失效，导致无法使用xshell的ssh连接服务器，因为找不到地址了。只有vip浮动到那个服务器上才能远程登录到服务器上。
建议：vip最好不使用服务器上绑定的物理网卡，以免影响不必要的麻烦。
时间：2015-04-13 补充
|
1
2
3
4
5
6
|
Apr
13
17
:
08
:
07
localhost Keepalived_vrrp[
2186
]: bogus VRRP packet received on eth0 !!!
Apr
13
17
:
08
:
07
localhost Keepalived_vrrp[
2186
]: VRRP_Instance(VI_1) ignoring received advertisment...
Apr
13
17
:
08
:
08
localhost Keepalived_vrrp[
2186
]: ip address associated with VRID
not
present
in
received packet :
192.168
.
0.251
Apr
13
17
:
08
:
08
localhost Keepalived_vrrp[
2186
]: one
or
more VIP associated with VRID mismatch actual MASTER advert
Apr
13
17
:
08
:
08
localhost Keepalived_vrrp[
2186
]: bogus VRRP packet received on eth0 !!!
Apr
13
17
:
08
:
08
localhost Keepalived_vrrp[
2186
]: VRRP_Instance(VI_1) ignoring received advertisment...
|
解决办法：
改变配置文件/etc/keepalived/keepalived.conf 中virtual_route_id的值
比如virtual_router_id 60 主从方都要改，默认为51
本文转自zys467754239 51CTO博客，原文链接：http://blog.51cto.com/467754239/1541421，如需转载请自行联系原作者
