5

Nginx + Keepalived 高可用集群部署 - 上古南城

 1 year ago
source link: https://www.cnblogs.com/zhangwencheng/p/17182896.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Nginx + Keepalived 高可用集群部署

正文

负载均衡技术对于一个网站尤其是大型网站的web服务器集群来说是至关重要的！做好负载均衡架构，可以实现故障转移和高可用环境，避免单点故障，保证网站健康持续运行。在使用 Nginx 做反向代理或者负载均衡的时候，都是以 Nginx 为入口，如果 Nginx 宕机了，那么所有的服务都无法正常提供，影响非常严重。

为了避免负载均衡服务器宕机造成严重影响，就需要建立一个备份机。主服务器和备份机上都运行高可用（High Availability）监控程序，通过传送诸如“I am alive”这样的信息来监控对方的运行状况。当备份机不能在一定的时间内收到这样的信息时，它就接管主服务器的服务IP并继续提供负载均衡服务；当备份管理器又从主管理器收到“I am alive”这样的信息时，它就释放服务IP地址，这样的主服务器就开始再次提供负载均衡服务。

高可用（High Availability）是分布式系统架构设计中必须考虑的因素之一，它通常是指，通过设计减少系统不能提供服务的时间。如果一个系统能够一直提供服务，那么这个可用性则是百分之百，但是我们不能保证一个系统能永远不出问题，所以我们只能通过设计来尽可能的去减少由于系统的故障所造成的影响。

由于业务扩展，网站的访问量不断加大，负载越来越高。现需要在web前端放置nginx负载均衡,同时结合keepalived对前端nginx实现HA高可用。

前文分享了《Linux下Nginx基础应用》，《Linux下实现高可用软件-Keepalived基础知识梳理》；今天简单分享Nginx + Keepalived 高可用集群部署。

主从集群架构图

1080590-20230309121807900-1301868474.png

环境说明

hostname ip 说明
Client-01 172.16.70.171 客户端测试机
KeepMaster 172.16.70.181 keepalived 主服务器 (nginx 主负载均衡器)
KeepBackup 172.16.70.182 keepalived 备服务器 (nginx 备负载均衡器)
VIP 172.16.70.183 vrrp HA 虚拟地址，可有多个IP
Web1 172.16.70.191 后端web主服务器1 (nginx 站点)
Web2 172.16.70.192 后端web备服务器2 (nginx 站点)

准备环境

# 本次所有部署服务器都配置
# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
# uname -r
3.10.0-1160.83.1.el7.x86_64
# systemctl stop firewalld
# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/sysconfig/selinux
# setenforce 0
# ntpdate 0.centos.pool.ntp.org
# yum install net-tools vim wget curl -y

 搭建后端web服务器

  • 主备一样操作
# 这里以 web-01 为例
[root@web-01 ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# rpm -ivh nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# nginx -v
nginx version: nginx/1.20.2
[root@web-01 ~]# #echo "`hostname` `ifconfig ens33 |sed -n 's#.*inet \(.*\)netmask.*#\1#p'`" > /usr/share/nginx/html/index.html
[root@web-01 ~]# cat /usr/share/nginx/html/index.html
web-01 172.16.70.191
[root@web-01 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@web-01 ~]# systemctl start nginx
[root@web-01 ~]# systemctl enable nginx
[root@web-01 ~]# netstat -ntupl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      10687/nginx: master
[root@web-01 ~]# ps -ef | grep nginx
root      10687      1  0 16:36 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     10688  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10689  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10690  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10691  10687  0 16:36 ?        00:00:00 nginx: worker process
root      10761  10586  0 16:45 pts/1    00:00:00 grep --color=auto nginx
# Client-01 测试访问
[root@Client-01 ~]# curl 172.16.70.191
web-01 172.16.70.191

 浏览器测试访问 http://ip/

 

1080590-20230306164933642-1366737072.png

Keep服务器上部署nginx负载均衡器

  • 主备一样操作
# 这里以 KeepMaster 为例
# 安装部署nginx
[root@KeepMaster ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# rpm -vih nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# nginx -v
nginx version: nginx/1.18.0
# 新建
[root@KeepMaster ~]# cat /etc/nginx/conf.d/web.conf
upstream web {
server 172.16.70.191:80 weight=1 max_fails=3 fail_timeout=20s;
server 172.16.70.192:80 weight=2 max_fails=3 fail_timeout=20s;
}
# weight(权重)和访问比率成正比，默认值为1
# max_fails 为允许失败的次数，默认值为1
# fail_timeout 当max_fails次失败后，暂停将请求分发到该后端服务器的时间
server {
listen 80;
server_name www.zhangwencheng.org;
location / {
proxy_pass http://web;
proxy_set_header HOST $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
[root@KeepMaster ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@KeepMaster ~]# systemctl start nginx
[root@KeepMaster ~]# systemctl enable nginx
[root@KeepMaster ~]# ps -ef | grep nginx
root       1677      1  0 17:28 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx      1678   1677  0 17:28 ?        00:00:00 nginx: worker process
root       1708   1444  0 17:33 pts/0    00:00:00 grep --color=auto nginx
[root@KeepMaster ~]# netstat -tnpl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1677/nginx: master

测试机Client-01验证负载均衡

  • Keep主备服务器上的nginx负载均衡
# 在测试机上添加host解析, KeepMaster/KeepBackup主机IP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.181   www.zhangwencheng.org
172.16.70.182   www.zhangwencheng.org
# 测试时候轮流关闭Keep节点，关闭后还是能够访问并看到轮循效果即表示nginx负载均衡器集群搭建成功。
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191

Keep服务器上部署keepalived

  • 主备一样操作
[root@KeepMaster ~]# yum install -y openssl openssl-devel libnl libnl-devel gcc
[root@KeepMaster ~]# mkdir /data/apps/keepalived -p
[root@KeepMaster ~]# wget --no-check-certificate  http://www.keepalived.org/software/keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# tar -xf keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# cd keepalived-2.2.4
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4  autogen.sh   build-aux    ChangeLog  configure.ac  COPYING  INSTALL     keepalived.spec.in  m4           Makefile.in  snap  tools
AUTHOR      bin_install  build_setup  configure  CONTRIBUTORS  doc      keepalived  lib                 Makefile.am  README.md    TODO
[root@KeepMaster keepalived-2.2.4]# ./configure --prefix=/data/apps/keepalived
....
....
# 最后编译正常输出如下
Keepalived configuration
------------------------
Keepalived version       : 2.2.4
Compiler                 : gcc gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Preprocessor flags       : -D_GNU_SOURCE
Compiler flags           : -g -g -O2 -Wextra -Wunused -Wstrict-prototypes -Wabi -Wbad-function-cast -Wcast-align -Wcast-qual -Wdisabled-optimization -Wdouble-promotion \
-Wfloat-equal -Wframe-larger-than=5120 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs \
-Wmissing-prototypes -Wnested-externs -Wold-style-definition -Woverlength-strings -Wpointer-arith -Wredundant-decls -Wshadow -Wstack-protector -Wstrict-overflow=4 -Wsuggest-attribute=format \
-Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsafe-loop-optimizations -Wunsuffixed-float-constants -Wvariadic-macros \
-Wwrite-strings -fPIE -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -O2
Linker flags             : -pie -Wl,-z,relro -Wl,-z,now
Extra Lib                : -lm -lcrypto -lssl -lnl
Use IPVS Framework       : Yes
IPVS use libnl           : Yes
IPVS syncd attributes    : No
IPVS 64 bit stats        : No
HTTP_GET regex support   : No
fwmark socket support    : Yes
Use VRRP Framework       : Yes
Use VRRP VMAC            : Yes
Use VRRP authentication  : Yes
With track_process       : Yes
With linkbeat            : Yes
Use BFD Framework        : No
SNMP vrrp support        : No
SNMP checker support     : No
SNMP RFCv2 support       : No
SNMP RFCv3 support       : No
DBUS support             : No
Use JSON output          : No
libnl version            : 1
Use IPv4 devconf         : No
Use iptables             : No
Use nftables             : No
init type                : systemd
systemd notify           : No
Strict config checks     : No
Build documentation      : No
Default runtime options  : -D
[root@KeepMaster keepalived-2.2.4]# make -j 4 && make install
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4  bin          build_setup  config.status  CONTRIBUTORS  INSTALL          keepalived.spec.in  Makefile     README     TODO
AUTHOR      bin_install  ChangeLog    configure      COPYING       keepalived       lib                 Makefile.am  README.md  tools
autogen.sh  build-aux    config.log   configure.ac   doc           keepalived.spec  m4                  Makefile.in  snap
[root@KeepMaster keepalived-2.2.4]# cp keepalived/keepalived /usr/local/sbin/ -a
[root@KeepMaster keepalived-2.2.4]# keepalived -v
Keepalived v2.2.4 (08/21,2021)
Copyright(C) 2001-2021 Alexandre Cassen, <[email protected]>
Built with kernel headers for Linux 3.10.0
Running on Linux 3.10.0-1160.83.1.el7.x86_64 #1 SMP Wed Jan 25 16:41:43 UTC 2023
Distro: CentOS Linux 7 (Core)
configure options: --prefix=/data/apps/keepalived
Config options:  LVS VRRP VRRP_AUTH VRRP_VMAC OLD_CHKSUM_COMPAT INIT=systemd
System options:  VSYSLOG LIBNL1 RTA_ENCAP RTA_EXPIRES RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTA_VIA IFA_FLAGS \
NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE SO_MARK
[root@KeepMaster keepalived-2.2.4]# cd /data/apps/keepalived/
[root@KeepMaster keepalived]# ls
bin  etc  sbin  share
[root@KeepMaster keepalived]# mv etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
[root@KeepMaster keepalived]# vim etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script chk_nginx {    
script "/data/apps/keepalived/chk_nginx.sh" 
interval 2                 
weight -5                                  
}
vrrp_instance VI_1 {
state MASTER　　　　# 备服务器这为 BACKUP
interface ens33
virtual_router_id 51
priority 110　　　　# 备服务器这小于110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
}

优先级不会不断的提高或者降低，最终优先级的范围是在[1,254]，不会出现优先级小于等于0或者优先级大于等于255的情况。
在MASTER节点的vrrp_instance中配置nopreempt，当它异常恢复后，即使它prio更高也不会抢占，这样可以避免正常情况下做无谓的切换。

nginx检测脚本

编写脚本来判断本机nginx是否正常，如果发现NginX不正常，自重启nginx。等待2秒再次校验，仍然失败则不再尝试，关闭keepalived，让其他主机此时会接管VIP。
此脚本必须在keepalived服务运行的前提下才有效！如果在keepalived服务先关闭的情况下，那么nginx服务关闭后就不能实现自启动了。

[root@KeepMaster keepalived]# cat chk_nginx.sh
#!/bin/bash
chk=$(ps -C nginx --no-heading|wc -l)
if [ "${chk}" = "0" ]; then
systemctl start nginx
sleep 2
chk=$(ps -C nginx --no-heading|wc -l)
if [ "${chk}" = "0" ]; then
systemctl stop keepalived
fi
fi
[root@KeepMaster keepalived]# chmod +x chk_nginx.sh

测试机Client-01验证VIP

  • Keep服务器上的VIP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.183   www.zhangwencheng.org
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192

故障转移测试

#手动关闭Master机器上的nginx服务，最多2秒钟后就会自启动
[root@KeepMaster ~]# systemctl stop nginx
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root      57266      1  0 16:21 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      57267  57266  0 16:21 ?        00:00:01 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      60019      1  0 16:42 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60027   1444  0 16:42 pts/0    00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
#手动关闭Master机器上的keepalived服务，已经发现没VIP了
[root@KeepMaster ~]# systemctl stop keepalived
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root      60019      1  0 16:42 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60348   1444  0 16:45 pts/0    00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@KeepMaster keepalived]# tail /var/log/messages
Mar  7 17:01:01 Keepalived-01 systemd: Started Session 27 of user root.
Mar  7 17:01:28 Keepalived-01 Keepalived[60703]: Stopping
Mar  7 17:01:28 Keepalived-01 systemd: Stopping LVS and VRRP High Availability Monitor...
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) sent 0 priority
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) removing VIPs.
Mar  7 17:01:28 Keepalived-01 NetworkManager[570]: <info>  [1678179688.0176] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepMaster'
Mar  7 17:01:29 Keepalived-01 Keepalived_vrrp[60704]: Stopped - used (self/children) 0.005506/0.797936 user time, 0.172766/0.818969 system time
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: CPU usage (self/children) user: 0.000000/0.803442 system: 0.001394/0.994146
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: Stopped Keepalived v2.2.4 (08/21,2021)
Mar  7 17:01:29 Keepalived-01 systemd: Stopped LVS and VRRP High Availability Monitor
  • KeepBackup查看，已经接管VIP
[root@KeepBackup keepalived]# ps -ef | egrep 'nginx|keepalived'
root      65036      1  0 17:01 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      65037  65036  0 17:01 ?        00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root      65067      1  0 17:01 ?        00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     65068  65067  0 17:01 ?        00:00:00 nginx: worker process
root      65122   1514  0 17:01 pts/0    00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepBackup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
[root@KeepBackup keepalived]# tail /var/log/messages
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 NetworkManager[573]: <info>  [1678179688.8137] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepBackup'
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183

双机高可用一般是通过虚拟IP（飘移IP）方法来实现的，基于Linux/Unix的IP别名技术，目前分为两种：

  • 双机主从模式：即前端使用两台服务器，一台主服务器和一台热备服务器，正常情况下，主服务器绑定一个公网虚拟IP，提供负载均衡服务，热备服务器处于空闲状态；当主服务器发生故障时，热备服务器接管主服务器的公网虚拟IP，提供负载均衡服务；但是热备服务器在主机器不出现故障的时候，永远处于浪费状态，对于服务器不多的网站，该方案不经济实惠。
  • 双机主主模式：即前端使用两台负载均衡服务器，互为主备，且都处于活动状态，同时各自绑定一个公网虚拟IP，提供负载均衡服务；当其中一台发生故障时，另一台接管发生故障服务器的公网虚拟IP（这时由非故障机器一台负担所有的请求）。这种方案，经济实惠，非常适合于当前架构环境。

主主模式集群架构图

1080590-20230309121854600-1731074856.png

当了解主备模式后，双主模式就容易配置多了。只需要在每台keepalived配置文件，加上一个vrrp_instance命名vrrp_instance VI_2即可，更改几个参数，设置另一个VIP：172.16.70.184

  • KeepMaster：state BACKUP ，priority 100， virtual_router_id 52
  • KeepBackup：state MASTER ，priority 110， virtual_router_id 52

KeepMaster上的keepalived.conf

[root@KeepMaster keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script chk_nginx {    
script "/data/apps/keepalived/chk_nginx.sh" 
interval 2                 
weight -5                  
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state BACKUP
interface ens33
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.184
}
track_script {
chk_nginx
}
}
[root@KeepMaster keepalived]# systemctl restart keepalived

 KeepBackup上的keepalived.conf

[root@KeepBackup keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script chk_nginx {    
script "/data/apps/keepalived/chk_nginx.sh" 
interval 2                 
weight -5                  
}
vrrp_instance VI_1 {
state BAKCUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state MASTER
interface ens33
virtual_router_id 52
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.184
}
track_script {
chk_nginx
}
}
[root@KeepBackup keepalived]# systemctl restart keepalived

 测试机Client-01验证VIP2

  • Keep服务器上的VIP2
# 注释VIP1解析，此时仅测试VIP2；测试成功后再取消VIP1注释
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#172.16.70.183  www.zhangwencheng.org
172.16.70.184   www.zhangwencheng.org
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192 
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
  •  查看VIP情况
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33　　　　# VIP1
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@KeepBackup ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.184/32 scope global ens33　　　　# VIP2
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
  •  故障转移测试类同，不再赘述。

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK