Deploying the Squid proxy server on Linux

Ian gives an example of the installation, configuration and maintenance of this multi-tentacled invertebrate proxy server.

To provide Internet access for users in the SAS Institute Europe, Middle East and Africa (EMEA), a number of proxy servers have been installed both at the country office level and centrally at SAS European Headquarters in Heidelberg, Germany.

These servers run the Squid proxy server software; this software is available under the GNU general public license. In brief, Squid provides for caching and/or forwarding requests for internet objects such as the data available via HTTP, FTP and gopher protocols. Web browsers can then use the local Squid cache server as a proxy HTTP server, reducing access time as well as bandwidth consumption. Squid keeps these objects in RAM or on local disk. Squid servers can be installed in hierarchies to allow central servers to build large caches of data available for servers lower in the hierarchy.

Squid has been in use for some time around SAS EMEA and is performing very well; the software is extremely stable and is delivering seamless access to the Internet for connected clients.

The original proxy servers were installed on HP workstations running release 10.20 of HPUX and Squid version 2.1. This was run on a mix of hardware but typically HP9000/720 workstations with 64MB of memory and about 4GB of disk. This configuration is difficult to support; the hardware has reached an age where failures are becoming common and the increased use of the Internet coupled with growth in the offices has left the configuration under-powered. Our main problem of late has been disk space management; the increased access patterns have left our existing log areas looking undersized at 100MB and our actual cache directories are looking rather small at 2GB.

As a result, we began researching some alternatives in order to maintain the service. Since we were happy with the Squid software itself, and we already had a good understanding of the configurations, we decided to continue using Squid but to review the hardware base.

Since Squid is an open-source project and well supported under Linux, it seemed a good idea to explore the possibility of using a Linux-based solution using a standard SAS EMEA Intel PC. This configuration is a Dell desktop PC with 256MB of RAM, 500MHz Intel Pentium and internal 20GB IDE disk. As Dell has a relationship with Red Hat, it made sense to their distribution. Also, SAS has recently released versions of the SAS product in partnership with Red Hat.

Architecture

The original architecture in SAS EMEA used three central parent Squid caches with direct access to the Internet and child Squid servers in many of the country offices. Some of the smaller countries' operations still connect to the central headquarter caches, and we felt that using less expensive hardware would give us the opportunity to install proxies in these offices. Further, in many of the country operations the SAS presence is split among several offices connected via WAN links; again the less expensive hardware gives us the opportunity to install proxies in these offices. These deployments should improve the response times for web traffic and hopefully reduce the overhead on our WAN links.

Finally, we had some reservations about the resilience and availability of the original infrastructure, and we felt that with revised client and server configurations we could improve the service level of our internal customers.

Our new architecture is not much altered in principle; we still have three central servers, but they now run Linux. We are deploying more child proxies, and we require a three-level hierarchy in some offices. For example, some countries have satellite offices that only connect to the SAS Intranet via a single WAN link to the country headquarters; in these cases we will install proxies at the satellite office with a preferred parent cache in the country headquarters rather than European headquarters.

A new addition to our architecture has been the Trend Interscan Virus Wall product for HTTP virus-scanning. We have installed three virus scanning systems also running Red Hat Linux; these systems are positioned behind the current Squid parent caches, providing a virus-scanning layer between the Squid cache hierarchy and the external Internet. Since the virus scanners are simply pass-through in nature, we simply configure our top-level Squid servers to “round-robin” between them.

Installation

The original HP-UX servers were installed by duplicating a disk image from a known configuration. This was a totally unsatisfactory method for several reasons, not the least of which was that it was difficult to make provisions for maintenance of this image for patches or version updates for Squid, etc.

Our goal was a scripted and automated installation that could be performed quickly by local office staff. We have been pleased with the implementation of this concept, and it has some useful benefits with regard to recovery and configuration management (see below).

We produced a KickStart configuration to build machines for us. KickStart is a tool from Red Hat to automate system installations. Basically we can tell the install how to partition a disk, which packages (RPMs) to install and include some local configuration steps via shell commands. Our KickStart configuration is placed on a floppy disk along with normal Linux boot utilities, and we instruct the KickStart to perform installation from a CD.

This means that for a new proxy server we can arrange shipment of a PC that looks similar to our expected hardware configuration and ship a CD and floppy disk for the remote office to complete the configuration.

The installation process has been automated with three exceptions: users will be prompted for the hostname, IP information and the keyboard type (some of our offices use different keyboards for the local language). The KickStart hard-codes all other choices; for example the installation language is always English, the choice of packages are always the same and the disk always partitions in the same way.

The basic installation from placing the disk in the drive until the reboot with a freshly installed OS takes under ten minutes. This is much quicker than we could do and a huge decrease in the time it would take to perform a HP-UX installation. This obviously has some implications for our backup and recovery procedures (see below).

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

evalesco

Remote Monitoring's picture

Evalesco is one of the leading network monitoring companies with the product SysOrb. It is born to perform scalable yet very straigt forward and easy to use network monitoring. http://www.evalesco.com/

9 years on and people still asking for help.

No Comment's picture

Still a great article though..

Step by step install Linux Debian server

Masud hasan's picture

I need help, how to step by step setup on Debian, please provide good process.
thanks

Proxys server

zeeeg's picture

hey every 1
can you please help me i want to make a proxy server to make socks so please some help me with that best regards

zeeg

Reply for squid

Bhupendra singh ranawat's picture

Starting Squid
Use the chkconfig configure Squid to start at boot::
[root@bigboy tmp]# chkconfig squid on
Use the service command to start, stop, and restart Squid after booting:
[root@bigboy tmp]# service squid start
[root@bigboy tmp]# service squid stop
[root@bigboy tmp]# service squid restart
You can test whether the Squid process is running with the pgrep command:
[root@bigboy tmp]# pgrep squid
You should get a response of plain old process ID numbers.
The /etc/squid/squid.conf File
The main Squid configuration file is squid.conf, and, like most Linux applications, Squid needs to be restarted for changes to the configuration file can take effect.

Restricting Access to specific Web sites
Squid is also capable of reading files containing lists of web sites and/or domains for use in ACLs. In this example we create to lists in files named /usr/local/etc/allowed-sites.squid and /usr/local/etc/restricted-sites.squid.
# File: /usr/local/etc/allowed-sites.squid
www.openfree.org
linuxhomenetworking.com

# File: /usr/local/etc/restricted-sites.squid
www.porn.com
illegal.com
These can then be used to always block the restricted sites and permit the allowed sites during working hours. This can be illustrated by expanding our previous example slightly.
#
# Add this to the bottom of the ACL section of squid.conf
#
acl home_network src 192.168.1.0/24
acl business_hours time M T W H F 9:00-17:00
acl GoodSites dstdomain "/usr/local/etc/allowed-sites.squid"
acl BadSites dstdomain "/usr/local/etc/restricted-sites.squid"

#
# Add this at the top of the http_access section of squid.conf
#
http_access deny BadSites
http_access allow home_network business_hours GoodSites

Restricting Web Access By IP Address
You can create an access control list that restricts Web access to users on certain networks. In this case, it's an ACL that defines a home network of 192.168.1.0.
#
# Add this to the bottom of the ACL section of squid.conf
#
acl home_network src 192.168.1.0/255.255.255.0
You also have to add a corresponding http_access statement that allows traffic that matches the ACL:
#
# Add this at the top of the http_access section of squid.conf
#
http_access allow home_network

By these step you able to configure squid easily

Thanks
Bhupendra singh ranawat
System administrator
Linux / Window 2003
Picasso Animation
jaipur

Thanks, Need further assistance.

Masud hasan's picture

I am masud hasan,i have completed MCSE. I live in Bangladesh, working in largest ISP company. I have to say very good suggest, i use this command now squid working smoothly. Again thanks Bhupendra singh ranawat, please give me your mail ID need for further assistance. My mail ID masud.shohel@gmx.com

DNS

Misafir's picture

DNS is basically used for resolving FQDN into IP address .

about useradmin

veera's picture

How to create multiple users at a time on linux machine.

Bandwitdth managment with squid proxy sever....

Anonymous's picture

Hi, i m punit and working on load balancing server ....my problem is as...I have squid proxy server sq1,sq2,sq3.Large no. of request are coming to the sq1 and after then sq1 take the bandwidth consumption information from Load balancing sever and based upon this result it redirect the request on least loaded server.I want to know how squid check the bandwidth information that is available to some other server ...can any one help me..

ipttraf

Zakir's picture

u can install iptraf in ur linux server. then u can see bandith info.

how configure squid linux box

sephiwe's picture

Hi i m jnr admin of linux the mentioned above artical is so good but i need step by step configuration of squid coz i have background of windows not in linux

how configure squid linux box

sephiwe's picture

Hi i m jnr admin of linux the mentioned above artical is so good but i need step by step configuration of squid coz i have background of windows not in linux

Delay Pools

Abdul's picture

i have deployed Delay pools. it implements bandwidth on the whole network. how can we asign a allow ful bandwith to single user/admin uder this senario.
thanks

geniousinit@yahoo.co.uk

Squid help

waqas ahmed's picture

Hi i m asst.sys admin of linux the mentioned above artical is so good but i need step by step configuration of squid coz i have background of windows not in linux

regrad

waqas
w_ahmed@thuraya.com

New to Linux

Ronald's picture

Hi!
I am a system administrator working on windows environment and i want to change my system to LINUX environment.

Can you please help me how to setup a linux server which will act as a firewall server and also a SQUID and DANSGUARDIAN for restriction of access.
I would appreciate your help.

Email address: ronald.abellar@yahoo.com.ph

Thanks!

help me

pankaj chauhan's picture

i have a squid proxy server it is working fine.
my client ip rang is 192.168.5.1 to 192.168.5.254
i want that first 20 ip get more fast speed then other ips

can it is possible with squid server.
plz tell me wat chang i will do on squid.conf file for this.

thanks
pankaj.chauhan.fbd@gmail.com

Squid proxy slow

jayi's picture

Hi,
I installed squid proxy one year back , now l am facing the problem that, my proxy server is very slow, l have enough space id system, if restart the squid services it work fine for one week after that it become very slow browsing ( but server is not slow)
Squid is running on RHEL core 3

First check the size of with

Murali's picture

First check the size of with squid directory with following cmd du -sh /var/spool/squid/, if it is in GBs then do the following .......
remove the caching directories in /var/spool/squid/
then type squid -z command then it creates the caching directories in that location, next restart the squid and check it

Slow proxy

Anonymous's picture

I too facing the same problem, As you said I have cleared the cache and recreated. Still its slow, any other solution is there?
If any fine tuning is needed please let me know.

Regds,
Ramachandran

Same problem

Anonymous's picture

Same problem here, strange thing is that the server itself is getting slow as well when connected through putty, although when you look at the running processes and the current use of the processor and the memory its like its almost standing idle...

Changing the contents of the packets

Harshit's picture

I want to block the user from seeing certain "words" on various webpages.
Does Squid also provides any help for changing the html contents.

Need exact command line &

jame's picture

Need exact command line & description to add rule Bypassing Cache for Some Sites in Linux.

thanks

clark's picture

just want to say thank you here, this is a great article, I just setup my own proxy server minutes ago, it is working ... thanks

configuring squid

navraj's picture

i want to use a public ip in network working under proxy server. please suggest me the configuration for squid proxy server

squid proxy configuration

Anonymous's picture

i want to configure squid on my fedora , how can i

how to configure the sqid proxy server in linux

sachin's picture

Dear Sir

I m also working as a sysytem admin will u please tell me datails the how to configure the sqid proxy server in linux.

Need help to setup Squid proxy Server

Anonymous's picture

I need to setup a Squid proxy server, I have the full details of the squid conf file. Is there anyone who can send me the ready made squid.conf file. I will do the necessary settings myself.

Need help to scratch to setup step by step Squid proxy Server

Yasir Shafiq's picture

Hello I am a new user of linux, This is my first attempt to install linux 9.0. So Please If possible send me detailed installation guide of squid. I am a user of visual desktop If possible send to graphical interface to configure squid mail.

Linux Red hat?? HELP!!

Patrick Hi's picture

Hi All,

Maybe someone could help me how to use linux, I'am a green apple in using Linux. My new office has a new setup with linux as the proxy and mail server. So i got here a problem cause I haven't use a linux before. So, to have my responsibility going I need to to research and self study on Linux. And indeed I have found this site maybe all you linuxs wizz out here could help me how to use, improve and manage a Linux red hat proxy and mail server. Maybe someone can give me a brief and welcome explanation with linux. please give your feedbacks on patrickhilmanabat@yahoo.com thanks in advance for all the warm hearted surfers out there...

Thank you very much happy New Year!

Patrick

linux as a proxy server

Shahzad Qadri's picture

hope u will be doin fine
i have configured my proxy server through the following links
http://www.linuxpakistan.net/forum/viewtopic.php?t=414
http://www.linuxpakistan.net/wiki/index.php?pagename=InstallingAndConfig...

take care
Shahzad Qadri

Transparent Proxy and Hotmail

Nahuel Ramos's picture

I got the blank screen just after login on my Hotmail account using Squid Transparent Proxy and IE6 and I fix it with:

http://www.squid-cache.org/bugs/show_bug.cgi?id=1169

adding one of these lines to my squid.conf

header_access Accept-Encoding deny all (2.5STABLE)
anonymize_headers deny Accept-Encoding (2.4STABLE)

I hope this post helps someone.

Nahuel Ramos.

problem

tahir's picture

yahoo did not work with transparent proxy or transparent proxy did not work

Cant Login in to HOTMAIL Through Squid Transparent Proxy

Devang Patel's picture

I setup squid server on RedHat Linux 9.0 (Transparent Proxy)
Now everything working fine else then Hotmail website.
I can access the website but when i am trying to login into www.hotmail.com its stuck up on below mentioned url.....

"http://by17fd.bay17.hotmail.msn.com/cgi-bin/hmhome?fti=yes&curmbox=F000000001&a=2b2fabe5e0e9d58d59775c0f17754bb1"

No result and just stuckin up with the above mentioned url.

Plz..suggest me what to do if someone is familiar with this prob.
Awaited for your favourable reply.

Devang Patel.

Need same help

harry's picture

If u got helped pls provide me with your solution that helped u.

hi, im just tryin to configur

Anonymous's picture

hi, im just tryin to configure my squid but seems it aint workin, can u pls gimme ur squid.conf file so that i may have a look over it.. thanks
Hussu;)

i am trying to configure the

ravi's picture

i am trying to configure the squid but i am not getting it plz help me out

A workaround is simply to add

surazzz's picture

A workaround is simply to add the following three lines to /etc/squid/squid.conf:

acl hotmail_domains dstdomain .hotmail.msn.com
acl ie6 browser MSIE[[:space:]]6
header_access Accept-Encoding deny ie6 hotmail_domains

A workaround is simply to add : Working on Debian + Squid

jmascarua's picture

The workaround surazz commented is working on Debian SID + Squid 2.5.7.3

Best regards
Joaquín

bypass squid cache

Anonymous's picture

Hi,

I want to bypass caching of one perticular site in squid caching , how to do that ,

we are updating the contents of one website , That updates are not taking effect when we open the website , it is showing the same old ocntents , but after 24 hours or after cleaning the squid cache the changes can be seen in that web site , what might be the problem , Please help me in this regard.

Please mail me on venu.jill@gmail.com
ASAP

Regards
Venugopal

workaround works very well

Gill shamir's picture

thank you surazz i'm using freebsd+Squid 2.5 stable and its works perfectly thanks again Gill

check this link it maybe useful

wael_awni's picture

Pls how can I configure my

arimsam's picture

Pls how can I configure my Redhat Linux as a proxy server.The commands are need also.

Also how to configure redhat as a Nat server.

Thanks

Sam

how can I configure my Redhat Linux as a proxy server

Pravin's picture

how can I configure my Redhat Linux as a proxy server.
Also how to configure redhat as a Nat server.

Hi there is problem with IE using squid try to use other browser

Sachin's picture

Hi,

We too are also facing the same problem using IE with squid but when we use same proxy server by Mozilla FireFox and NN it works fine.May be there is a problem in IE browsers.

So Check it out

There is a problem in squid when using hotmail account

Sachin's picture

I too am facing the same problem but it works fine when using outlook with http as transport protocol .

Check it out

We also have the same problem

Anonymous's picture

We also have the same problem starting just a week ago. We already run our proxy for a year now without any problems but suffer now from a hotmail problem. Anyone with a solution is welcome. Current proxy: Squid 2.5Stable7

Clearing temp files in squid

Anonymous's picture

Hi Ian,

My proxy server stops responding whenever the temp file size goes upto 100MBso i want to know how to clear the temp files as well i want to know how to start proxy if it has stopped and how to check whether the proxy has started or not

Re: Deploying the Squid proxy server on Linux

Anonymous's picture

hi
i am mahesh i will try to linux squid server these are perfect ful windows ,,, i have last 4 year running windows 2002,xp,98 but these windows are not squre linux squid proxy is much better is windows

maheshkumar198131@india.com
mahesh@prodcomtech.com

Mahesh Arora

RE: Deploying ....

Raghu's picture

Hey Mahesh,

On what planet did you learn such incoherent English?!?

NEW LINUX USER

Anonymous's picture

Hello!
i am working on WINDOWS 2000 with ISA SERVER 2000 i want to upgrade my proxy server on Linux SQUID can anybody guide me and help me in this matter..
my email address is
barjeesshees@hotmail.com
thanks
bye

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState