7
Freelancer任务之四squid查询用户浏览记录
source link: https://bajie.dev/posts/20240124-freelancer_4/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Freelancer任务之四squid查询用户浏览记录
2024-01-24
3 分钟阅读
这个需求也比较简单：
User Browsing Log for Open VPN server
简单说就是用户连到他的openvpn服务器，通过上面的squid代理来浏览其他网站，比较特别的是需要查看用户http和https的浏览记录。
squid做透明代理，这样就可以截取浏览记录并且提供加速了
服务器是Ubuntu，缺省安装的的squid是不支持SSL的，所以需要重新编译一个
安装依赖包：
sudo apt-get install build-essential fakeroot devscripts gawk gcc-multilib dpatch
sudo apt-get build-dep squid3
sudo apt-get build-dep openssl
sudo apt-get install libssl-dev
sudo apt-get source squid3
下载到squid的源代码，以及ubuntu的修改包，解压并释放：
tar zxvf squid3_3.5.12.orig.tar.gz
cd squid3-3.5.12
tar xf ../squid3_3.5.12-1ubuntu7.5.debian.tar.xz
修改参数增加对ssl的支持：
vi debian/rules
Add --with-openssl --enable-ssl --enable-ssl-crtd under the DEB_CONFIGURE_EXTRA_FLAGS section.
DEB_CONFIGURE_EXTRA_FLAGS := BUILDCXXFLAGS="$(CXXFLAGS) $(LDFLAGS)" \
...
--with-default-user=proxy \
--with-openssl \
--enable-ssl \
--enable-ssl-crtd
...
编译，会生成7个deb包
debuild -us -uc -b
cd ..
ls -1 *.deb
squid3_3.5.12-1ubuntu7.5_all.deb
squid_3.5.12-1ubuntu7.5_amd64.deb
squid-cgi_3.5.12-1ubuntu7.5_amd64.deb
squidclient_3.5.12-1ubuntu7.5_amd64.deb
squid-common_3.5.12-1ubuntu7.5_all.deb
squid-dbg_3.5.12-1ubuntu7.5_amd64.deb
squid-purge_3.5.12-1ubuntu7.5_amd64.deb
安装，先装语言包，然后安装三个自己生成的包
sudo apt-get install squid-langpack
sudo dpkg -i squid_3.5.12-1ubuntu7.5_amd64.deb squid-common_3.5.12-1ubuntu7.5_all.deb squid-dbg_3.5.12-1ubuntu7.5_amd64.deb
检查一下新的squid是否支持ssl了
squid -v|grep ssl
configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,smb_lm' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--with-openssl' '--enable-ssl' '--enable-ssl-crtd' '--enable-build-info=Ubuntu linux' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security'
cd /usr/lib/squid
ls ssl_crtd
Gen出ssl的证书和密钥并拷贝到正确的位置，更新ca-certificates：
openssl genrsa -out squid.key 2048
openssl req -new -key squid.key -out squid.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Rendoumi.com
Organizational Unit Name (eg, section) []:Rendoumi.com
Common Name (e.g. server FQDN or YOUR name) []:159.89.116.192
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
openssl x509 -req -days 3650 -in squid.csr -signkey squid.key -out squid.crt
Signature ok
subject=/C=CN/ST=Beijing/L=Beijing/O=Rendoumi.com/OU=Rendoumi.com/CN=159.89.116.192
Getting Private key
sudo cp squid.crt /usr/local/share/ca-certificates
sudo /usr/sbin/update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
sudo cp squid.pem /etc/squid
修改/etc/squid.conf配置文件
cd /etc/squid
cat squid.conf|grep -v ^# | grep -v ^$
sudo vi /etc/squid/squid.conf
----------------------------------------
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl localnet src 10.8.0.0/16
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost
http_access allow localnet
http_access deny all
coredump_dir /var/spool/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
# example lin deb packages
#refresh_pattern (\.deb|\.udeb)$ 129600 100% 129600
refresh_pattern . 0 20% 4320
shutdown_lifetime 3
http_port 3128 intercept
https_port 3129 intercept ssl-bump generate-host-certificates=on version=1 options=NO_SSLv2,NO_SSLv3,SINGLE_DH_USE dynamic_cert_mem_cache_size=4MB cert=/etc/squid/squid.pem
always_direct allow all
ssl_bump none localhost
ssl_bump server-first all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
----------------------------------------
初始化ssl_db
sudo /usr/lib/squid/ssl_crtd -c -s /var/lib/ssl_db/
chown -R proxy /var/lib/ssl_db
重启squid
sudo systemctl restart squid.service
特别的一点，雇主写了巨多的ufw的规则，导致IPTABLE爆满，居然无法手动清除所有的规则，这也是第一次遇到这样的，只能写脚本清除，方法如下：
vi cl.txt
-----------------------
# Empty the entire filter table
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
-----------------------
sudo iptables-restore < cl
最后修改IPTABLES，把80和443的请求都送到squid去
sudo vi /etc/rc.local
iptables -t nat -A PREROUTING -p tcp -s 10.8.0.0/24 --dport 80 -j REDIRECT --to-ports 3128
iptables -t nat -A PREROUTING -p tcp -s 10.8.0.0/24 --dport 443 -j REDIRECT --to-ports 3129
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 ! -d 10.8.0.0/24 -j SNAT --to 159.89.116.192
用代理访问，验证一下：
sudo tail -f /var/log/squid/access.log
这样做有个问题，就是https实际是被劫持代理了，所以客户端会弹出个是否信任证书，除非在每个客户端中预埋，这样才能解决。
That is all.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK