keeplive+haproxy+nginx

  1. 云栖社区>
  2. 博客>
  3. 正文

keeplive+haproxy+nginx

技术小阿哥 2017-11-27 13:40:00 浏览1574
展开阅读全文

Nginx ("engine x") 是一个高性能的 HTTP 和 反向代理 服务器,也是一个 IMAP/POP3/SMTP 代理服务器。 Nginx 是由 Igor Sysoev 为俄罗斯访问量第二的 Rambler.ru 站点开发的,第一个公开版本0.1.0发布于2004年10月4日。其将源代码以类BSD许可证的形式发布,因它的稳定性、丰富的功能集、示例配置文件和低系统资源的消耗而闻名


Keepalivedt做高可用,其高可用,都是站在服务器脚本去说的高可用,而不是服务的角度,也就是说,如果服务器DOWN机或者网络出现故障,高可用是可以实现自动切换的。如果运行的服务,比如Nginx挂掉,这些高可用软件是意识不到的,需要自己写脚本去实现服务的切换



HAProxy是一个使用C语言编写的自由及开放源代码软件[1],其提供高可用性负载均衡,以及基于TCPHTTP的应用程序代理

HAProxy特别适用于那些负载特大的web站点,这些站点通常又需要会话保持或七层处理。HAProxy运行在当前的硬件上,完全可以支持数以万计的并发连接。并且它的运行模式使得它可以很简单安全的整合进您当前的架构中, 同时可以保护你的web服务器不被暴露到网络上。

HAProxy实现了一种事件驱动, 单一进程模型,此模型支持非常大的并发连接数。多进程或多线程模型受内存限制 、系统调度器限制以及无处不在的锁限制,很少能处理数千并发连接。事件驱动模型因为在有更好的资源和时间管理的用户空间(User-Space) 实现所有这些任务,所以没有这些问题。此模型的弊端是,在多核系统上,这些程序通常扩展性较差。这就是为什么他们必须进行优化以 使每个CPU时间片(Cycle)做更多的工作。



master        nginx+keeplive+haproxy  192.168.1.108
master/slave    nginx+keeplive+haproxy  192.168.1.103
vip          192.168.1.200


实验环境

centos_6.8(x64)


实验软件

pcre-8.10.tar.gz

nginx-1.1.1.tar.gz

keepalived-1.2.1.tar.gz

haproxy-1.4.8.tar.gz


软件安装

yum install -y gcc gcc-c++* make* cmake* zlib* openssl* popt* nmap* ntp* lrzsz* autoconf* automake* libxml*  ncurses* libmcrypt* ipvsadm* libnl-devel* kernel-devel


/etc/rc.d/init.d/ntpd start

ntpdate 服务器IP

clock -w        

chkconfig --level 35 ntpd on

 

tar zxvf pcre-8.10.tar.gz

cd pcre-8.10

./configure

make && make install


tar zxvf nginx-1.1.1.tar.gz

cd nginx-1.1.1

./configure --prefix=/usr/local/nginx --with-http_stub_status_module

make && make install

/usr/local/nginx/sbin/nginx -t  检查nginx配置

nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successfu

如果看到这行提示,证明nginx配置没有错误

/usr/local/nginx/sbin/nginx -s reload

nginx: [error] invalid PID number "" in "/usr/local/nginx/logs/nginx.pid"

/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

/usr/local/nginx/sbin/nginx -s reload        nginx平滑重启

cp /usr/local/nginx/sbin/nginx /etc/init.d/

chmod +x /etc/init.d/nginx 

/etc/rc.d/init.d/nginx                  启动nginx

/usr/local/nginx/sbin/nginx &             也可以使用这种办法启动nginx

cp /usr/local/nginx/html/index.html /root/    

echo > /usr/local/nginx/html/index.html

echo web1 > /usr/local/nginx/html/index.html  master端192.168.1.108

tail /usr/local/nginx/html/index.html 

web1

echo web2 > /usr/local/nginx/html/index.html  slave端192.168.1.103操作

tail /usr/local/nginx/html/index.html 

web2


tar zxvf keepalived-1.2.1.tar.gz

cd keepalived-1.2.1

ll /usr/src/kernels/

total 4

drwxr-xr-x 22 root root 4096 May  8 18:01 2.6.32-696.1.1.el6.x86_64

./configure --prefix=/usr/local/keeplived 

--with-kernel-dir=/usr/src/kernels/2.6.32-696.1.1.el6.x86_64


Keepalived configuration

------------------------

Keepalived version       : 1.2.1

Compiler                 : gcc

Compiler flags           : -g -O2

Extra Lib                : -lpopt -lssl -lcrypto 

Use IPVS Framework       : Yes

IPVS sync daemon support : Yes

Use VRRP Framework       : Yes  3个必须是yes

Use Debug flags          : No


make  &&  make install

cp /usr/local/keeplived/etc/rc.d/init.d/keepalived   /etc/rc.d/init.d/

chmod +x /etc/rc.d/init.d/keepalived

cp /usr/local/keeplived/etc/sysconfig/keepalived /etc/sysconfig/

cp /usr/local/keeplived/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/keeplived/sbin/keepalived /usr/sbin/

chmod +x /usr/sbin/keepalived

echo "/usr/local/nginx/sbin/nginx" >>  /etc/rc.local

echo "/etc/init.d/keepalived start" >>  /etc/rc.local  开机启动服务


/etc/init.d/keepalived start

Starting keepalived: [  OK  ]


tar zxvf haproxy-1.4.8.tar.gz 

cd haproxy-1.4.8

uname -a

Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

make TARGET=linux26 PREFIX=/usr/local/haproxy

make install PREFIX=/usr/local/haproxy

useradd -s /sbin/nologin haproxy

chown -R haproxy.haproxy /usr/local/haproxy


ll /usr/local/haproxy/

total 12

drwxr-xr-x 3 haproxy haproxy 4096 May  8 16:36 doc

drwxr-xr-x 2 haproxy haproxy 4096 May  8 16:36 sbin

drwxr-xr-x 3 haproxy haproxy 4096 May  8 16:36 share   默认没有.cfg 配置文件

touch /usr/local/haproxy/haproxy.cfg


vim  /usr/local/haproxy/haproxy.cfg


global

log 127.0.0.1 local0

maxconn 5120

chroot /usr/local/haproxy

user haproxy

group haproxy

daemon

quiet

nbproc  1

pidfile /usr/local/haproxy/haproxy.pid

debug

defaults

log 127.0.0.1 local3

mode http

option httplog

option httpclose

option  dontlognull

#option  forwardfor

option  redispatch

retries 2

maxconn 2000

balance source

contimeout      5000

clitimeout      50000

srvtimeout      50000


listen web_proxy :8080  web页面配置 nginx默认端口80造成IP冲突 修改为8080

     server www1  nginx_master服务器IP:80  weight 5 check inter 2000 rise 2 fall 5

     server www2  nginx_slave服务器IP:80  weight 5 check inter 2000 rise 2 fall 5



listen  mysql

      bind 0.0.0.0:7306  7306为代理数据库虚拟端口

      mode tcp

      server mysql 后端数据库IP:3306


mode http

#transparent

stats uri / haproxy-stats

stats realm Haproxy \ statisti

                                

/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg & 启动服务

pkill haproxy  杀死进程


touch  /etc/init.d/haproxy

vim   /etc/init.d/haproxy

-----------------------------------------------------------------------

#!/bin/bash

#

# haproxy

#

# chkconfig: 35 85 15

# description: HAProxy is a free, very fast and reliable solution \

# offering high availability, load balancing, and \

# proxying for TCP and HTTP-based applications

# processname: haproxy

# config: /usr/local/haproxy/haproxy.cfg  haproxy配置文件目录

# pidfile: /usr/local/haproxy/haproxy.pid


# Source function library.

. /etc/rc.d/init.d/functions


# Source networking configuration.

. /etc/sysconfig/network


# Check that networking is up.

[ "$NETWORKING" = "no" ] && exit 0


config="/usr/local/haproxy/haproxy.cfg"  haproxy 配置文件目录

exec="/usr/local/haproxy/sbin/haproxy"

prog=$(basename $exec)


[ -e /etc/sysconfig/$prog ] && . /etc/sysconfig/$prog


lockfile=/var/lock/subsys/haproxy


check() {

    $exec -c -V -f $config

}


start() {

    $exec -c -q -f $config

    if [ $? -ne 0 ]; then

        echo "Errors in configuration file, check with $prog check."

        return 1

    fi

 

    echo -n $"Starting $prog: "

    # start it up here, usually something like "daemon $exec"

    daemon $exec -D -f $config -p /var/run/$prog.pid

    retval=$?

    echo

    [ $retval -eq 0 ] && touch $lockfile

    return $retval

}


stop() {

    echo -n $"Stopping $prog: "

    # stop it here, often "killproc $prog"

    killproc $prog 

    retval=$?

    echo

    [ $retval -eq 0 ] && rm -f $lockfile

    return $retval

}


restart() {

    $exec -c -q -f $config

    if [ $? -ne 0 ]; then

        echo "Errors in configuration file, check with $prog check."

        return 1

    fi

    stop

    start

}


reload() {

    $exec -c -q -f $config

    if [ $? -ne 0 ]; then

        echo "Errors in configuration file, check with $prog check."

        return 1

    fi

    echo -n $"Reloading $prog: "

    $exec -D -f $config -p /var/run/$prog.pid -sf $(cat /var/run/$prog.pid)

    retval=$?

    echo

    return $retval

}


force_reload() {

    restart

}


fdr_status() {

    status $prog

}


case "$1" in

    start|stop|restart|reload)

        $1

        ;;

    force-reload)

        force_reload

        ;;

    checkconfig)

        check

        ;;

    status)

        fdr_status

        ;;

    condrestart|try-restart)

      [ ! -f $lockfile ] || restart

    ;;

    *)

        echo $"Usage: $0 {start|stop|status|checkconfig|restart|try-restart|reload|force-reload}"

        exit 2

esac

-----------------------------------------------------------------------------

chmod +x /etc/init.d/haproxy  使用脚本方式重启服务

 

ps -aux | grep haporxy

Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ

haproxy   9165  0.1  0.1  14072  1856 pts/0    S    17:01   0:00 /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg

root      9167  0.0  0.0 103316   836 pts/0    S+   17:05   0:00 grep haproxy


netstat -tuplna | grep nginx

tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      11343/nginx


ps -aux | grep nginx

Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ

root     38686  0.0  0.1  20160  1180 ?        Ss   10:48   0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

nobody   38691  0.0  0.1  20544  1284 ?        S    10:49   0:00 nginx: worker process                                          

root     38711  0.0  0.0 103260   836 pts/1    S+   10:51   0:00 grep nginx


以上所有操作 master端点 和slave端相同



http://服务器IP:8080/haproxy-stats


wKiom1kQNcnQ9CdRAAEp0omKgAI046.jpg-wh_50



wKioL1kLVZKSQJR0AABwO5RzDis462.jpg-wh_50



wKioL1kLRUCxzYQ6AAB3aZ5FiwA285.jpg-wh_50



http://192.168.1.200 刷新,停止master端nginx 刷新看看是不是切换到了slave主机



vim /etc/keepalived/keepalived.conf    master端 192.168.1.108配置

! Configuration File for keepalived


global_defs {

   notification_email {

     xxxx@126.com  通告邮箱

   }

   notification_email_from  xxx@126.com

   smtp_server smtp.126.com

   smtp_connect_timeout 30

   router_id master

}

vrrp_script chk_http_port {

    script "/root/chk_nginx.sh"

    interval 2

    weight 2

}

track_script {

    chk_http_port

}

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 101

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.1.200  虚拟IP

    }

}


vim /etc/keepalived/keepalived.conf  slave端 192.168.1.103 配置


! Configuration File for keepalived


global_defs {

   notification_email {

     xxx@126.com  通告邮箱

   }

   notification_email_from  xxxx@126.com

   smtp_server smtp.126.com

   smtp_connect_timeout 30

   router_id backup

}

vrrp_script chk_http_port {

    script "/root/chk_nginx.sh"

    interval 2

    weight 2

}

track_script {

    chk_http_port

}

vrrp_instance VI_1 {

    state MASTER/BACKUP    master为主主模式 backup为主从模式

    interface eth0

    virtual_router_id 101   主主 主从 模式route_id相同

    priority 100/99       主主 100/100优先级相同 主从100/99从优先级比主小

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.1.200     虚拟IP

    }

}


/etc/rc.d/init.d/keepalived restart


touch chk_nginx.sh

chmod +x chk_nginx.sh

vim chk_nginx.sh

#!/bin/sh

# check nginx server status

NGINX=/usr/local/nginx/sbin/nginx

PORT=80


nmap localhost -p $PORT | grep "$PORT/tcp open"

#echo $?

if [ $? -ne 0 ];then

    $NGINX -s stop

    $NGINX

    sleep 3

    nmap localhost -p $PORT | grep "$PORT/tcp open"

    [ $? -ne 0 ] && /etc/init.d/keepalived stop

fi


sh chk_nginx.sh 

80/tcp open  http


vim /etc/crontab

*/5  *  *  *  * sh /root/root/chk_nginx.sh

service crond restart


vim /etc/sysctl.conf

net.ipv4.ip_forward = 1 修改为1 打开路由转发功能

sysctl -p

net.ipv4.ip_forward = 1



[root@centos6 ~]# ip a                master端192.168.1.108配置

 eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:45:17:69 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.108/24 brd 192.168.1.255 scope global eth0

    inet 192.168.1.200/32 scope global eth0      虚拟ip启动成功

    inet6 fe80::20c:29ff:fe45:1769/64 scope link 

    valid_lft forever preferred_lft forever

[root@centos6-1 ~]# ip a               slave端192.168.1.103配置

  eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:3d:5b:1b brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.103/24 brd 192.168.1.255 scope global eth0

    inet 192.168.1.200/24 scope global secondary eth0  虚拟ip启动成功

    inet6 fe80::20c:29ff:fe3d:5b1b/64 scope link 

       valid_lft forever preferred_lft forever



本文转自 mailfile 51CTO博客,原文链接:http://blog.51cto.com/mailfile/1329362,如需转载请自行联系原作者

网友评论

登录后评论
0/500
评论
技术小阿哥
+ 关注