环境说明:
操作系统: Redhat 6.5 x64,本文采用rpm方式安装haproxy,keepalived。
如果采用原码安装:可以参考基础CentOS5.9 Haproxy+Keepalived+Jboss集群实施架构一例
app1: 192.168.0.24
app1: 192.168.0.25
VIP : 192.168.0.26
http1:192.168.0.24:8080 主机配置LAP环境。
http2:192.168.0.25:8080 主机配置LAP环境。
一、双机Keepalived配置
实现一个VIP出现,出例采用VIP地址是192.168.0.26。
1.安装keepalived软件
说明:RHEL/CentOS/OEL6.X安装版本中已经集成了keepalive-1.2.7,以及haproxy软件包经过安装没有问题。
在app1,app2两个节点上直接采用RPM包安装。
1
2
3
4
5
6
7
8
9
10
|
# rpm -ivh keepalived-1.2.7-3.el6.x86_64.rpm net-snmp-utils-5.5-49.el6.x86_64.rpm net-snmp-libs-5.5-49.el6.x86_64.rpm lm_sensors-3.1.1-17.el6.x86_64.rpm lm_sensors-
devel-3.1.1-17.el6.x86_64.rpm lm_sensors-libs-3.1.1-17.el6.x86_64.rpm
Preparing...
########################################### [100%]
1:lm_sensors-libs
########################################### [ 17%]
2:net-snmp-libs
########################################### [ 33%]
3:keepalived
########################################### [ 50%]
4:net-snmp-utils
########################################### [ 67%]
5:lm_sensors
########################################### [ 83%]
6:lm_sensors-devel
########################################### [100%]
#
|
2. 创建keepalived配置文件
1) 在app1 节点一上配置文件
说明: 修改配置文件, 绑定的网卡是eth0,从机就是优先级与本机IP不一样外,其它都是一样,相比之前的示例增加了监控脚本。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
|
# vi /etc/keepalived/keepalived.conf
! Configuration File
for
keepalived
global_defs {
notification_email {
xxx@126.com
}
notification_email_from service@abc.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_haproxy {
script
"killall -0 haproxy"
interval 1
weight -2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
mcast_src_ip 192.168.0.24
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 876543
}
virtual_ipaddress {
192.168.0.26
/24
dev eth0 label eth0:1
}
track_interface {
eth0
}
track_script {
chk_haproxy
}
}
|
2) 在app2节点二上配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
|
# vi /etc/keepalived/keepalived.conf
! Configuration File
for
keepalived
global_defs {
notification_email {
xxx@126.com
}
notification_email_from service@abc.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_haproxy {
script
"killall -0 haproxy"
interval 1
weight -2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
mcast_src_ip 192.168.0.25
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 876543
}
virtual_ipaddress {
192.168.0.26
/24
dev eth0 label eth0:1
}
track_interface {
eth0
}
track_script {
chk_haproxy
}
}
|
3 .在app1,app2两节点上启动与创建keepalived服务
1) 启动服务并加为开机启动:
1
2
|
service keepalived start
chkconfig keepalived on
|
2) 测试并观察VIP漂移情况
(1) VIP地址切换观察
说明:两种方式切换VIP, 一个是关闭haproxy代理服务,一个是关闭相关节点的Keepalived服务。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
[root@app1 /]
# service keepalived start
Starting keepalived: [ OK ]
[root@app1 keepalived]
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link
/loopback
00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1
/8
scope host lo
inet6 ::1
/128
scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link
/ether
00:0c:29:4c:39:43 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.24
/24
brd 192.168.0.255 scope global eth0
inet 192.168.0.26
/24
scope global secondary eth0:1
inet6 fe80::20c:29ff:fe4c:3943
/64
scope link
valid_lft forever preferred_lft forever
[root@app1 keepalived]
#
[root@app2 keepalived]
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link
/loopback
00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1
/8
scope host lo
inet6 ::1
/128
scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link
/ether
00:0c:29:cf:05:99 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.25
/24
brd 192.168.0.255 scope global eth0
inet6 fe80::20c:29ff:fecf:599
/64
scope link
valid_lft forever preferred_lft forever
[root@app2 keepalived]
#
注:可以关闭keepalived服务,以及关闭haproxy服务,通过
tail
-f
/var/log/messages
观察VIP移动情况。
Jan 11 12:54:19 app2 Keepalived_vrrp[26911]: VRRP_Instance(VI_1) Received higher prio advert
Jan 11 12:54:19 app2 Keepalived_vrrp[26911]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 11 12:54:19 app2 Keepalived_vrrp[26911]: VRRP_Instance(VI_1) removing protocol VIPs.
Jan 11 12:54:19 app2 Keepalived_healthcheckers[26910]: Netlink reflector reports IP 192.168.0.26 removed
|
二、HAproxy反向代理配置
app1, app2配置操作
1. 添加非本机IP邦定支持
# vi /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1
# sysctl -p
2. 安装haproxy软件
# rpm -ivh haproxy-1.4.24-2.el6.x86_64.rpm
3. 创建配置文件
1)app1上创建配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
# vi /usr/local/haproxy/conf/haproxy.cfg
global
log 127.0.0.1 local0
maxconn 4000
chroot
/var/lib/haproxy
pidfile
/var/run/haproxy
.pid
user haproxy
group haproxy
daemon
nbproc 1
stats socket
/var/lib/haproxy/stats
defaults
log global
mode http
option httplog
option dontlognull
option redispatch
option httpclose
option forwardfor
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
timeout check 1s
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
listen stats
mode http
bind 0.0.0.0:91
stats
enable
stats uri
/admin
stats realm
"Admin console"
stats auth admin:123456
stats hide-version
stats refresh 10s
stats admin
if
TRUE
frontend web_proxy
bind *:80
mode http
acl url_dynamic path_end -i .php
use_backend phpserver
if
url_dynamic
default_backend webservers
backend webservers
balance roundrobin
option httpchk GET
/test
.html HTTP
/1
.0\r\nHost:192.168.0.26
server node01 192.168.0.24:8080 weight 3 check inter 2000 rise 2 fall 1
server node02 192.168.0.25:8080 weight 3 check inter 2000 rise 2 fall 1
backend phpserver
balance roundrobin
option httpchk GET
/test
.php
server node01 192.168.0.24:8080 weight 3 check inter 2000 rise 2 fall 1
|
2)app2上创建配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
# vi /usr/local/haproxy/conf/haproxy.cfg
global
log 127.0.0.1 local0
maxconn 4000
chroot
/var/lib/haproxy
pidfile
/var/run/haproxy
.pid
user haproxy
group haproxy
daemon
nbproc 1
stats socket
/var/lib/haproxy/stats
defaults
log global
mode http
option httplog
option dontlognull
option redispatch
option httpclose
option forwardfor
retries 3
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
timeout check 1s
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
listen stats
mode http
bind 0.0.0.0:91
stats
enable
stats uri
/admin
stats realm
"Admin console"
stats auth admin:123456
stats hide-version
stats refresh 10s
stats admin
if
TRUE
frontend web_proxy
bind *:80
mode http
acl url_dynamic path_end -i .php
use_backend phpserver
if
url_dynamic
default_backend webservers
backend webservers
balance roundrobin
option httpchk GET
/test
.html HTTP
/1
.0\r\nHost:192.168.0.26
server node01 192.168.0.24:8080 weight 3 check inter 2000 rise 2 fall 1
server node02 192.168.0.25:8080 weight 3 check inter 2000 rise 2 fall 1
backend phpserver
balance roundrobin
option httpchk GET
/test
.html
server node01 192.168.0.24:8080 weight 3 check inter 2000 rise 2 fall 1
|
说明:两节点互为主备模式,均优化将本机的节点应用做为主节点,也可以为负载均衡模式, 主例也配置了一个动静分离的模式。
4. app1,app2上配置HAproxy日志文件
Haproxy日志配置,否则默认是不记haproxy日志的,注意与RHEL/CentOS5.x版本的区别。
1
2
3
4
5
6
7
8
9
|
# vi /etc/rsyslog.conf
$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 127.0.0.1
local0.*
/var/log/haproxy
.log
*.info;mail.none;authpriv.none;
cron
.none;local0.none
/var/log/messages
说明: 第五行是去掉在
/var/log/message
再记录haproxy.log日志的功能的。
直接手动执行
service rsyslog restart
|
注:Redhat/Centos6.X 默认haproxy服务运行采用haproxy用户,RPM包安装haproxy,系统已自动配置日录文件轮询功能。
三、配置Keepalived注意点小结:
1,是否有多VIP地址,如果有的话,需要配置多个vrrp_instance实例,双机互备。
2,监控脚本,用于监控nginx,或haproxy代理服务器的存活状态,用于切换VIP地址。
3,两种方式切换VIP,一个是关闭代理服务,一个是关闭相关节点的Keepalived服务器。
本文转自 koumm 51CTO博客,原文链接:http://blog.51cto.com/koumm/1733880,如需转载请自行联系原作者