Overview of Cache Agents

How web proxies work:
Cache Web page objects to reduce duplicate requests

Basic types of agents

Traditional Proxy: For Internet use, specify the server explicitly
 Transparent proxy: Clients do not need to specify the proxy server's address and port. The firewall redirects the web to the proxy via the default route

Benefits of using proxies

Improving web access speed
 Hide the client's true IP address

Experimental environment

squid server: 192.168.52.134
 web server: 192.168.52.135
 client server: 192.168.52.138

1. Install squid proxy server on squid server

[root@squid ~]# mkdir /abc
[root@squid ~]# mount.cifs //192.168.100.3/LNMP-C7/abc/ ##Mount
[root@squid ~]# cd /abc/
[root@squid abc]# tar zxvf squid-3.4.6.tar.gz -C /opt  ##decompression
[root@squid abc]# yum install gcc gcc-c++ make -y  ##Install Environment Components
[root@squid abc]# cd /opt/squid-3.4.6
[root@squid squid-3.4.6]# ./configure \
--prefix=/usr/local/squid \  ##Installation Path
--sysconfdir=/etc \   ##Profile directory
--enable-arp-acl \   ##Support acl access control list
--enable-linux-netfilter \   ##Supports network filtering
--enable-linux-tproxy \   ##Supports transparency
--enable-async-io=100 \   ##io optimization
--enable-err-language="Simplify_Chinese" \   ##Error Display Simplified Chinese
--enable-underscore \
--enable-poll \
--enable-gnuregex   ##Supports regular expression
[root@squid squid-3.4.6]# make && make install   ##Compile Installation
[root@squid squid-3.4.6]# ln -s /usr/local/squid/sbin/* /usr/local/sbin/  ##Easy system identification
[root@squid squid-3.4.6]# useradd -M -s /sbin/nologin squid   ##Create System User
[root@squid squid-3.4.6]# chown -R squid.squid /usr/local/squid/var/  ##Give directory all files ownership group permissions

2. Modify squid configuration file and optimize startup

[root@squid squid-3.4.6]# vim /etc/squid.conf   ##Modify squid configuration file
# And finally deny all other access to this proxy
http_access allow all   ##Add this item
#http_access deny all ##Comments, allowing terminal access

# Squid normally listens to port 3128
http_port 3128
cache_effective_user squid   ##Specify user squid
cache_effective_group squid ##Specify Group
[root@squid squid-3.4.6]# squid -k parse ##Check profile syntax
[root@squid squid-3.4.6]# squid -z  ##Initialize Cache Directory
[root@squid squid-3.4.6]# squid  ##Open Service
[root@squid squid-3.4.6]# netstat -ntap | grep 3128  ##View squid port
[root@squid squid-3.4.6]# cd /etc/init.d/
[root@squid init.d]# vim squid    ##Edit the service startup squid script
#!/bin/bash
#chkconfig: 2345 90 25
PID="/usr/local/squid/var/run/squid.pid"   ##PID file process number
CONF="/etc/squid.conf"   ##Main Profile
CMD="/usr/local/squid/sbin/squid"   ##Start Command

case "$1" in
start)
                netstat -ntap | grep squid &> /dev/null
                if [ $? -eq 0 ]
                then 
                 echo "squid is running"
                 else
                 echo "Starting squid...." 
                 $CMD
                fi
                ;;
stop)
                $CMD -k kill &> /dev/null   ##Close squid
                rm -rf $PID &> /dev/null    ##Delete PID file
                ;;
status)
                [ -f $PID ] &> /dev/null
                 if [ $? -eq 0 ]
                                then
                                 netstat -ntap | grep squid
                                else
                                 echo "squid is not running"
                fi
                ;;
restart)
                $0 stop &> /dev/null
                echo "Closing squid..."
                $0 start &> /dev/null
                echo "Starting squid..."
                ;;
reload)
                $CMD -k reconfigure  ##Overload Profile
                ;;
check)
                $CMD -k parse   ##Check grammar
                ;;
*)
                echo "usage:$0{start|stop|reload|status|check|restart}"
                ;;
esac
[root@squid init.d]# chmod +x squid   ##Give Execution Permission
[root@squid init.d]# chkconfig --add squid   ##Add to service management
[root@squid init.d]# chkconfig --level 35 squid on  ##Start-Up Self-Starting

3. Setting up traditional proxy configuration

[root@squid init.d]# vim /etc/squid.conf  ##Modify Main Profile
# Squid normally listens to port 3128
http_port 3128
cache_mem 64 MB   ##Memory space size
reply_body_max_size 10 MB  ##Allow download of maximum file size
maximum_object_size 4096 KB   ##Allow maximum object size to save cache space
[root@squid init.d]# service squid restart
[root@squid init.d]# iptables -L  ##View Table Contents
[root@squid init.d]# iptables -F  ##Empty table cache
[root@squid init.d]# setenforce 0
[root@squid init.d]# iptables -I INPUT -p tcp --dport 3128 -j ACCEPT ##Allow port 3128
[root@squid init.d]# service squid reload  ##Overload Profile

4. Install http service on web server

[root@web ~]# systemctl stop firewalld.service   ##Close Firewall
[root@web ~]# setenforce 0
[root@web ~]# yum install httpd -y  ##Install web Services
[root@web ~]# systemctl start httpd.service

5. Test Agent Service

(1) Direct access to web pages using client tester (clear cache after access)

(2) Set up proxy access again

(3) View the httpd service log files of the web host

[root@localhost logs]# head -1 access_log 
192.168.52.138 - - [04/Dec/2019:17:09:15 +0800] "GET / HTTP/1.1" 403 4897 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"
#You can see that it is accessed by the IP address of the client host
[root@localhost logs]#

[root@localhost logs]# tail -1 access_log 
192.168.52.134 - - [04/Dec/2019:17:26:55 +0800] "GET /browserconfig.xml HTTP/1.1" 404 215 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"
#You can see that it is accessed by the IP address of the squid server
[root@localhost logs]#

2. Transparent Agent

Experimental environment:

squid: 192.168.100.1  Intranet: ens33
            12.0.0.1           External network: ens36
web: 12.0.0.12
client: 192.168.100.50

1. Add a network card to the squid service and set the ip address

ens33 Network Card Profile:
[root@squid ~]# cd /etc/sysconfig/network-scripts/
[root@squid network-scripts]# vim ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=8ecd53ce-afdb-46f8-b7ff-b2f428a3bc8f
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.100.1
NETMASK=255.255.255.0

ens36 Network Card Profile:
[root@squid network-scripts]# cp -p ifcfg-ens33 ifcfg-ens36
[root@squid network-scripts]# vim ifcfg-ens36
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens36
DEVICE=ens36
ONBOOT=yes
IPADDR=12.0.0.1
NETMASK=255.255.255.0
[root@squid network-scripts]# systemctl restart network
[root@squid network-scripts]# ifconfig 
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.100.1  netmask 255.255.255.0  broadcast 192.168.100.255
        inet6 fe80::c776:9d00:618:88f2  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:eb:34:07  txqueuelen 1000  (Ethernet)
        RX packets 226  bytes 31111 (30.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 84  bytes 10776 (10.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 12.0.0.1  netmask 255.255.255.0  broadcast 12.0.0.255
        inet6 fe80::55bc:65c1:7046:e2d6  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:eb:34:11  txqueuelen 1000  (Ethernet)
        RX packets 216  bytes 30613 (29.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 76  bytes 9847 (9.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[root@squid network-scripts]# service network restart   ##service network restart
[root@squid network-scripts]# vim /etc/sysctl.conf   ##Turn on routing forwarding
net.ipv4.ip_forward=1
[root@squid network-scripts]# sysctl -p   ##Load

2. Set up transparent proxy on squid server

[root@squid network-scripts]# vim /etc/squid.conf   #Set Profile
http_port 192.168.100.1:3128    transparent   #Set Transparent Proxy

[root@squid ~]# iptables -F   #Empty table cache
[root@squid ~]# iptables -t nat -F    #Empty NAT table cache
root@squid ~]# setenforce 0    #Turn off enhanced security features
[root@squid network-scripts]# iptables -t nat -I PREROUTING -i ens33 -s 192.168.100.0/24 -p tcp --dport 80 -j REDIRECT --to 3128
#Define rule entry ens33, port 80 redirected to 328
[root@squid network-scripts]# iptables -t nat -I PREROUTING -i ens33 -s 192.168.100.0/24 -p tcp --dport 443 -j REDIRECT --to 3128
#https443 port
[root@squid network-scripts]# iptables -I INPUT -p tcp --dport 3128 -j ACCEPT    #Allow port 3128 access
[root@squid network-scripts]# service squid stop    #Shut down services
[root@squid network-scripts]# service squid start    #Open Service
//Starting squid...
[root@squid network-scripts]# 

3. Change network mode of web host and client test host to host only, respectively

4. Configure the fixed IP address of the web host

[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33 
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=3ceed540-b04c-48d6-a4f7-79951f09ea1d
DEVICE=ens33
ONBOOT=yes
IPADDR=12.0.0.12
NETMASK=255.255.255.0
GATEWAY=12.0.0.1
[root@localhost ~]# systemctl restart network
[root@localhost ~]# ifconfig 
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 12.0.0.12  netmask 255.255.255.0  broadcast 12.0.0.255
        inet6 fe80::3e1d:31ba:f66a:6f80  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:95:9b:1b  txqueuelen 1000  (Ethernet)
        RX packets 189  bytes 26901 (26.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 83  bytes 10980 (10.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

5. Configure client test host fixed IP address and close proxy server

6. Test Agent Service

View the httpd service access log on the web server

[root@localhost ~]# tail -1 /etc/httpd/logs/access_log 
12.0.0.1 - - [04/Dec/2019:19:57:53 +0800] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240"
#You can see that it is accessed by the IP address of the proxy server
[root@localhost ~]#