2
fail2ban 防止WEB攻击
source link: https://blog.51cto.com/waringid/5811979
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
fail2ban 防止WEB攻击
精选 原创fail2ban简单来说就是通过不断读取设定的日志文件，并通过正则校验每条日志是否符合规则。一旦符合，则提取日志中的IP地址与时间戳，然后写入到fail2ban的数据库中。写库的同时进行计数，如果该IP在设定的时间间隔内被匹配的次数超过阈值，则调用iptables，将其的请求reject。
Linux 系统生产环境配置指南 https://blog.51cto.com/waringid/5782872
Linux 系统防火墙 Firewall-cmd 日常操作指南 https://blog.51cto.com/waringid/5806004
利用 ipset 封禁大量 IP https://blog.51cto.com/waringid/5806004
前期的准备操作请参考“ Linux 系统生产环境配置指南”。Fail2ban 结构如下：
/etc/fail2ban ## fail2ban 服务配置目录
/etc/fail2ban/action.d ## iptables 、mail 等动作文件目录
/etc/fail2ban/filter.d ## 条件匹配文件目录，过滤日志关键内容
/etc/fail2ban/jail.conf ## fail2ban 防护配置文件
/etc/fail2ban/fail2ban.conf ## fail2ban 配置文件，定义日志级别、日志、sock 文件位置等
2. 检查防火墙
#如果您已经安装iptables建议先关闭
service iptables stop
#查看Firewalld状态
firewall-cmd --state
#启动firewalld
systemctl start firewalld
#设置开机启动
systemctl enable firewalld.service
#放行22端口
firewall-cmd --zone=public --add-port=80/tcp --permanent
#重载配置
firewall-cmd --reload
#查看已放行端口
firewall-cmd --zone=public --list-ports
service iptables stop
#查看Firewalld状态
firewall-cmd --state
#启动firewalld
systemctl start firewalld
#设置开机启动
systemctl enable firewalld.service
#放行22端口
firewall-cmd --zone=public --add-port=80/tcp --permanent
#重载配置
firewall-cmd --reload
#查看已放行端口
firewall-cmd --zone=public --list-ports
3. 安装配置
3.1. 安装
yum -y install fail2ban
安装成功后fail2ban配置文件位于/etc/fail2ban，其中jail.conf为主配置文件，相关的匹配规则位于filter.d目录，其它目录/文件一般很少用到，如果需要详细了解可自行搜索。
3.2. 配置jail
cp jail.conf jail.local
vim jail.local
vim jail.local
banaction = firewallcmd-ipset
action = %(action_mwl)s
#将默认的执行动作由iptables改为firewallcmd-ipset
[waf-blockip]
enabled = true
filter = waf-blockip
port = http,https
logpath = /tmp/2019*_waf.log
blocktype = DROP
action = %(action_mwl)s
datapattern = %Y-%m-%d %H:%M:%S
bantime = 120
maxretry = 1
findtime = 1
[nginx-cc]
enabled = true
filter = nginx-cc
port = http,https
logpath = /tmp/cc.log
blocktype = DROP
action = %(action_mwl)s
bantime = 120
maxretry = 1
findtime = 1
action = %(action_mwl)s
#将默认的执行动作由iptables改为firewallcmd-ipset
[waf-blockip]
enabled = true
filter = waf-blockip
port = http,https
logpath = /tmp/2019*_waf.log
blocktype = DROP
action = %(action_mwl)s
datapattern = %Y-%m-%d %H:%M:%S
bantime = 120
maxretry = 1
findtime = 1
[nginx-cc]
enabled = true
filter = nginx-cc
port = http,https
logpath = /tmp/cc.log
blocktype = DROP
action = %(action_mwl)s
bantime = 120
maxretry = 1
findtime = 1
以上每行内容的大致意义如下：
[nginxx-cc]：定义jail名称
- enabled：是否启用该jail，默认的所有规则都没有该项，需要手动添加
- port：指定封禁的端口，默认为0:65535，也就是所有端口，但可以在jail中设定
- filter：指定过滤器名称
- logpath：日志路径
- action：达到阈值后的动作
- maxretry：阈值
- findtime：时间间隔
- bantime：封禁时长
- ignoreip：忽略的IP
注意
logpath与action可以有多行，如action中的设定：调用iptables-multiport封禁目标IP访问的多个端口，调用sendmail发送告警邮件
findtime不是检查日志的时间间隔，日志的检查是实时的。因为fail2ban自带数据库，所以可以在设定的时间内统计匹配次数
ignoreip添加后端服务器的IP或CDN的IP
3.3. 配置filter
vim waf-blockip.conf
[Definition]
failregex = \"<HOST>\",\"rule_tag\"
ignoreregex =
[Definition]
failregex = \"<HOST>\",\"rule_tag\"
ignoreregex =
日志内容如下所示：
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 12:00:13","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_age
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 12:00:13","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_age
vim nginx-cc.conf
[Definition]
failregex = <HOST> - - \[.*\] .*GET \/\?=*
#failregex = <HOST> - - \[.*\] \".*(403|400).*\" #匹配403或404的错误
ignoreregex =
[Definition]
failregex = <HOST> - - \[.*\] .*GET \/\?=*
#failregex = <HOST> - - \[.*\] \".*(403|400).*\" #匹配403或404的错误
ignoreregex =
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
50.195.104.171 - - [13/Jun/2019:18:18:47 +0800] "GET /?=13253 HTTP/1.1" 504 905"https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; U; NetBSD amd64; en-US; rv:1.9.2.15) Gecko/20110308 Namoroka/3.6.15"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
192.168.13.43 - - [06/Jun/2019:18:25:58 +0800] "GET /ngx_status HTTP/1.1" 301 16"-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Ge
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
50.195.104.171 - - [13/Jun/2019:18:18:47 +0800] "GET /?=13253 HTTP/1.1" 504 905"https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; U; NetBSD amd64; en-US; rv:1.9.2.15) Gecko/20110308 Namoroka/3.6.15"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
192.168.13.43 - - [06/Jun/2019:18:25:58 +0800] "GET /ngx_status HTTP/1.1" 301 16"-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Ge
4. 常用配置
#查看被ban IP，其中ssh-iptables为名称，比如上面的[ssh-iptables]和[nginx-dir-scan]
fail2ban-client status ssh-iptables
#添加白名单
fail2ban-client set ssh-iptables addignoreip IP地址
#删除白名单
fail2ban-client set ssh-iptables delignoreip IP地址
#查看状态
fail2ban-client status
#查看日志
tail /var/log/fail2ban.log
#查看规则
firewall-cmd --direct --get-all-rules
ipset list
fail2ban-client status ssh-iptables
#添加白名单
fail2ban-client set ssh-iptables addignoreip IP地址
#删除白名单
fail2ban-client set ssh-iptables delignoreip IP地址
#查看状态
fail2ban-client status
#查看日志
tail /var/log/fail2ban.log
#查看规则
firewall-cmd --direct --get-all-rules
ipset list
4.1. 定时清理日志
vi /root/del_waf_log.sh
#! /bin/bash
cat /dev/null > /tmp/`date +"%Y-%m-%d"`-waf.log
chmod +x /root/del_waf_log.sh
crontab -e
30 3 * * 0 sh /root/del_waf_log.sh
#! /bin/bash
cat /dev/null > /tmp/`date +"%Y-%m-%d"`-waf.log
chmod +x /root/del_waf_log.sh
crontab -e
30 3 * * 0 sh /root/del_waf_log.sh
5. 内核优化
针对TIME_WAIT的问题，可以通过调整内核的方式进行优化。
#修改超时时间为30秒，某些情况可以进一步降低该值
net.ipv4.tcp_fin_timeout = 30
#将keepalive的发送频率降低为20分钟一次
net.ipv4.tcp_keepalive_time = 1200
#开启SYN Cookies，当SYN队列溢出时启用Cookies
net.ipv4.tcp_syncookies = 1
#开启TIME-WAIT sockets重用功能
net.ipv4.tcp_tw_reuse = 1
#开启TIME-WAIT sockets快速回收功能
net.ipv4.tcp_tw_recycle = 1
#加大SYN队列
net.ipv4.tcp_max_syn_backlog = 8192
#最大TIME_WAIT保持数，超过将全部清除
net.ipv4.tcp_max_tw_buckets = 5000
#修改超时时间为30秒，某些情况可以进一步降低该值
net.ipv4.tcp_fin_timeout = 30
#将keepalive的发送频率降低为20分钟一次
net.ipv4.tcp_keepalive_time = 1200
#开启SYN Cookies，当SYN队列溢出时启用Cookies
net.ipv4.tcp_syncookies = 1
#开启TIME-WAIT sockets重用功能
net.ipv4.tcp_tw_reuse = 1
#开启TIME-WAIT sockets快速回收功能
net.ipv4.tcp_tw_recycle = 1
#加大SYN队列
net.ipv4.tcp_max_syn_backlog = 8192
#最大TIME_WAIT保持数，超过将全部清除
net.ipv4.tcp_max_tw_buckets = 5000
6. 过滤规则
6.1. nginx的验证
fail2ban-regex /tmp/2019-07-10_waf.log /etc/fail2ban/filter.d/waf-blockip.conf
failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$
^ \[error\] \d+#\d+: \*\d+ no user/password was provided for basic authentication, client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$
failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$
^ \[error\] \d+#\d+: \*\d+ no user/password was provided for basic authentication, client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$
6.2. 检查fail2ban启动问题
fail2ban-client -x start
6.3. modsecurity 日志过滤
利用 modsecurity 结合 nginx 并整合到 微服务网关实现 web 安全防护。
kong 网关结合 modsecurity 输出的日志格式如下
112.119.33.91 - - [18/Dec/2020:09:21:58 +0800] "GET /socket.io/?__sails_io_sdk_version=0.13.8&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=polling&t=NPpS75x&sid=o6ZI8VoIHqBUgvFMAACb HTTP/2.0" 200 4 "https://dashboard.waringid.me/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
2020/12/18 09:23:10 [error] 25142#0: *659 [client 113.78.65.148] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/owasp-modsecurity-crs-3.2.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "79"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "172.19.0.5"] [uri "/.env"] [unique_id "1608254590"] [ref ""], client: 113.78.65.148, server: kong, request: "GET /.env HTTP/1.1", host: "dashboard.waringid.me"
113.78.65.148 - - [18/Dec/2020:09:23:10 +0800] "GET /.env HTTP/1.1" 403 150 "-" "curl/7.29.0"
51.159.23.43 - - [18/Dec/2020:09:28:46 +0800] "GET / HTTP/1.1" 404 48 "-" "-"
89.97.157.10 - - [18/Dec/2020:09:43:52 +0800] "GET / HTTP/1.1" 404 48 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
2020/12/18 09:23:10 [error] 25142#0: *659 [client 113.78.65.148] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/owasp-modsecurity-crs-3.2.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "79"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "172.19.0.5"] [uri "/.env"] [unique_id "1608254590"] [ref ""], client: 113.78.65.148, server: kong, request: "GET /.env HTTP/1.1", host: "dashboard.waringid.me"
113.78.65.148 - - [18/Dec/2020:09:23:10 +0800] "GET /.env HTTP/1.1" 403 150 "-" "curl/7.29.0"
51.159.23.43 - - [18/Dec/2020:09:28:46 +0800] "GET / HTTP/1.1" 404 48 "-" "-"
89.97.157.10 - - [18/Dec/2020:09:43:52 +0800] "GET / HTTP/1.1" 404 48 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
注意2种不同的日志格式类型，2中不同日志类型匹配不同的正则式。
同时注意不同日志时间格式也不一样。
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK