INCREASING THE SERVICE CAPABILITY FOR MULTIUSERS FOR VNU WEB PROXY WITH HAPROXY

ACKNOWLEDGEMENT First of all, I would like to express my sincere gratitude to my supervisor Ass.Prof. Ho Si Dam, and M.S Doan Minh Phuong, University of Engineering and Technology, Viet Nam National University (VNU), Hanoi for his enthusiastic guidance, warm encouragement and helpful research experiences. I would like to also thank the Center Application Internet Technology – CAIT for all their careful guidance and preparing facility to my application. I am grateful to thank all the teachers in University of Engineering and Technology, VNU who provide invaluable knowledge and life skills for me during the four academic years. I would like to also thank my friends in K53CA class whom help me during the four academic years. Last, but not least, my family is really the biggest motivation for me. My parents always encourage me when I have stress and difficulty. I would like to send them great love and gratefulness. Ha Noi, May …, 2012 Le Tri Thai ABSTRACT Our thesis has been completed objective and satisfied the requirement of subjective: Increasing the service capability for multiusers for VNU web proxy with Haproxy. We have successful integrated haproxy server load balancer into present system. We also modified old server to compatible with new scenario successfully. Lastly, all system was tuned up with higher capability and higher availability. Our work is to solve the problem of management servers. haproxy is stable, and does not any problem with all old system. The maintenance, upgrade backend servers, squid servers is easier than ever. Nowadays, the more servers is installed, the more works of administrators. But haproxy can save their time and working enforce. Squid proxy is stronger than before. The error request is reduce by 40%. The session rate is increate more than three times without deceasing of quality. The advantage is outweighing the disadvantage of difficult in installing and boosting system. Keyword: haproxy, squid proxy, capability List of Figures Figure 1.1. The University Campus Network. 5 Figure 2.1. Testing model of haproxy in support website. 8 Figure 2.2. Deployment model of haproxy load balancer. 10 Figure 4.1. System status generated by top command. 24 Figure 4.2. Filedescriptors in haproxy, server, and maxfile. 25 Figure 4.3. Haproxy server webpage statistics. 26 Figure 4.4. Filedescriptor shortage in squid proxy shown in cache.log. 29 List of Tables Table 3.2 Basic haproxy configuration 14 Table 3.3 Detail haproxy configuration 15 Table 3.4 Configuration of rsyslog for haproxy log 17 Table 3.5 Disable selinux in etcselinuxconfig 18 Table 3.6 Disable iptables service 18 Table 3.7 Restart haproxy. 18 Table 3.8 FD configuration. 20 Table 3.9. Statistic network by crontab. 20 Table 3.10. Calamaris analysis squid access log. 21 Table 4.1. Statistic of resource log in crond service. 23 Table 4.2. Incoming TCP request by status untuned 27 Table 4.4. Distribution histogram of request untuned 28 Table 4.5. Incoming TCPrequest by status tuned 29 Table 4.6. Requestdestination by 2ndleveldomain tuned. 30 Table 4.7. Distribution histogram of request tuned. 32 Table 4.8. Frequency load time website. 34 List of Acronyms CentOS : The Community ENTerprise Operating System VNU: Vietnam National University VNUnet: Vietnam National University network Table of Contents ..SemiFinalThai_thesis_v0.4.1.docx _Toc325917965 ABSTRACT iv List of Tables v List of Acronyms vi Chapter 1 1 INTRODUCTION 1 1.1 Motivation 1 1.2 Contributions and thesis overview 2 Chapter 2 3 INCREASING THE SERVICE CAPABILITY FOR MULTIUSERS FOR VNU WEB PROXY WITH HAPROXY 3 2.1 Squid web cache proxy 3 2.2 Server load balance 7 2.2.1 haproxy server load balancer 7 2.2.2 Haproxy specification 8 2.3 Operating system CentOS 10 2.3.1 File descriptors and maxfile 11 2.3.2 Sysctl.conf 11 2.3.3 SElinux 11 2.3.4 Service rsyslog 11 2.3.5 Service iptables 12 Chapter 3 13 DEPLOYMENT 13 3.1 Haproxy server configuration 13 3.2 Squid proxy server configuration 19 3.3 Logging tool. 20 3.3.1 Crond Service 20 3.3.2 Analysis squid log with calamaris 21 Chapter 4 22 EXPERIENCES AND RESULTS 22 4.1 Haproxy server 22 4.1.1 Uptime 22 4.1.2 Resource consuming 23 4.1.3 Web statistics and administration: 25 4.1.4 Haproxy conclusion 26 4.2 Squid proxy servers 27 4.2.1 Untuned proxy static 27 4.2.2 Tuned up proxy static 29 4.2.3 Squid conclusion 32 4.3 Users experience 33 Chapter 5 36 CONCLUSION AND FUTURE WORK 36 Reference 37

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY

Le Tri Thai

INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH

HAPROXY Major: Computer Science

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY

Supervisor: Prof Ho Si Dam

M.S Doan Minh Phuong

SUPERVISOR’S APPROVAL

“I hereby approve that the thesis in its current form is ready for committee examination

as a requirement for the Bachelor of Computer Science degree at the University of Engineering and Technology.”

Signature:………

First of all, I would like to express my sincere gratitude to my supervisor Ass.Prof Ho Si Dam, and M.S Doan Minh Phuong, University of Engineering and Technology, Viet Nam National University (VNU), Hanoi for his enthusiastic guidance, warm

encouragement and helpful research experiences

I would like to also thank the Center Application Internet Technology – CAIT for all their careful guidance and preparing facility to my application

I am grateful to thank all the teachers in University of Engineering and Technology, VNU who provide invaluable knowledge and life skills for me during the four academic years

I would like to also thank my friends in K53CA class whom help me during the four academic years

Last, but not least, my family is really the biggest motivation for me My parents always encourage me when I have stress and difficulty I would like to send them great love and gratefulness

Ha Noi, May …, 2012

Le Tri Thai

Our thesis has been completed objective and satisfied the requirement of subjective:

Increasing the service capability for multi-users for VNU web proxy with Haproxy We have successful integrated haproxy server load balancer into present system We also

modified old server to compatible with new scenario successfully Lastly, all system wastuned up with higher capability and higher availability

Our work is to solve the problem of management servers haproxy is stable, and does not

any problem with all old system The maintenance, upgrade backend servers, squidservers is easier than ever Nowadays, the more servers is installed, the more works of

administrators But haproxy can save their time and working enforce

Squid proxy is stronger than before The error request is reduce by 40% The session rate

is increate more than three times without deceasing of quality The advantage isoutweighing the disadvantage of difficult in installing and boosting system

Keyword: haproxy, squid proxy, capability

List of Figures

Figure 1.1 The University Campus Network 5

Figure 2.1 Testing model of haproxy in support website 8

Figure 2.2 Deployment model of haproxy load balancer 10

Table 3.2 Basic haproxy configuration 14

Table 3.3 Detail haproxy configuration 16

Table 3.4 Configuration of rsyslog for haproxy log 18

Table 3.5 Disable selinux in /etc/selinux/config 19

Table 3.6 Disable iptables service 19

Table 3.7 Restart haproxy script 19

Table 3.8 FD configuration 21

Table 3.9 Statistic network by crontab 21

Table 3.10 Calamaris analysis squid access log 22

Table 4.1 Statistic of resource log in crond service 24

Figure 4.1 System status generated by top command 25

Figure 4.2 File-descriptors in haproxy, server, and max-file 26

Figure 4.3 Haproxy server webpage statistics 27

Table 4.2 Incoming TCP request by status un-tuned 28

Table 4.4 Distribution histogram of request un-tuned 29

Figure 4.4 File-descriptor shortage in squid proxy shown in cache.log 30

Table 4.5 Incoming TCP-request by status tuned 30

Table 4.6 Request-destination by 2nd-level-domain tuned 31

Table 4.7 Distribution histogram of request tuned 33

Table 4.8 Frequency load time website 35

List of Tables

Figure 1.1 The University Campus Network 5

Figure 2.1 Testing model of haproxy in support website 8

Figure 2.2 Deployment model of haproxy load balancer 10

Table 3.2 Basic haproxy configuration 14

Table 3.3 Detail haproxy configuration 16

Table 3.4 Configuration of rsyslog for haproxy log 18

Table 3.5 Disable selinux in /etc/selinux/config 19

Table 3.6 Disable iptables service 19

Table 3.7 Restart haproxy script 19

Table 3.8 FD configuration 21

Table 3.9 Statistic network by crontab 21

Table 3.10 Calamaris analysis squid access log 22

Table 4.1 Statistic of resource log in crond service 24

Figure 4.1 System status generated by top command 25

Figure 4.2 File-descriptors in haproxy, server, and max-file 26

Figure 4.3 Haproxy server webpage statistics 27

Table 4.2 Incoming TCP request by status un-tuned 28

Table 4.4 Distribution histogram of request un-tuned 29

Figure 4.4 File-descriptor shortage in squid proxy shown in cache.log 30

Table 4.5 Incoming TCP-request by status tuned 30

Table 4.6 Request-destination by 2nd-level-domain tuned 31

Table 4.7 Distribution histogram of request tuned 33

Table 4.8 Frequency load time website 35

List of Acronyms

CentOS : The Community ENTerprise Operating System

VNU: Vietnam National University

VNUnet: Vietnam National University network

Table of Contents

Le Tri Thai 1

Le Tri Thai 2

ABSTRACT iv

List of Tables vi

List of Acronyms vii

Chapter 1 1

INTRODUCTION 1

1.1 Motivation 1

1.2 Contributions and thesis overview 2

Chapter 2 3

INCREASING THE SERVICE CAPABILITY FOR MULTI-USERS FOR VNU WEB PROXY WITH HAPROXY 3

2.1 Squid web cache proxy 3

2.2 Server load balance 7

2.2.1 haproxy - server load balancer 7

2.2.2 Haproxy specification 8

2.3 Operating system - CentOS 10

2.3.1 File descriptors and max-file 11

2.3.2 Sysctl.conf 11

2.3.3 SElinux 11

2.3.4 Service rsyslog 11

2.3.5 Service iptables 12

Chapter 3 13

DEPLOYMENT 13

3.1 Haproxy server configuration 13

3.3 Logging tool 21

3.3.1 Crond Service 21

3.3.2 Analysis squid log with calamaris 22

Chapter 4 23

EXPERIENTAL RESULT 23

4.1 Haproxy server 23

4.1.1 Uptime 23

4.1.2 Resource consuming 24

4.1.3 Web statistics and administration: 26

4.1.4 Haproxy conclusion 27

4.2 Squid proxy servers 28

4.2.1 Un-tuned proxy static 28

4.2.2 Tuned up proxy static 30

4.2.3 Squid conclusion 33

4.3 Users experience 34

Chapter 5 36

CONCLUSION AND FUTURE WORK 36

Reference 38

At Hanoi, National University, the university has students facilitate Internet use forlearning Students can access internet easily with a laptop from the phone or Internetconnection Notice is very easily being sent via email, IM, Facebook Documents areeasy to download from the course website Electronic documents can be easilytransferred to other portable devices such as laptops, cell phones, or tablets Students inthe class are a pioneer in using new technology to life, studying and working.

Meeting the increasing needs of students and staff on campus, the center VNUnetmanaging the network and server system for all faculties university Hanoi continuousimprovement, improving the quality of services to meet user’s need Recently, the Centrehas upgraded the Internet connection; specifically speed of international connection boostfrom 10 up to 15 Mbps and 10% increasing of domestic Internet from 100 to 110Mbps.The number of frequency users generally at 800 - 1000 people, distributed in a ForeignLanguage student dormitory, Me Tri dormitory, Hanoi University Sciences, HanoiUniversity Social Sciences and Humanities, libraries, University of Engineering andTechnology Internet has helped many students in their daily work and life.

Although the system and internet connection speed have been upgraded, Internet servicesremain some problems Students and officers are clustered in server by account andspecific proxy server There is an unbalance user sharing for each proxy in this approach

As the number of Internet users is increasing, it is requires processing capabilities of theweb proxy cache must also increase

1.2 Contributions and thesis overview

The purpose of this project is to increase capacity to serve users for web proxies system in HanoiNational University with Haproxy load balancer With load balancing, the jobs of upgrade,maintain squid proxy do not interrupt of Internet service, and the upgrade is also easier Inaddition, the deployment of Haproxy also achieves greater or equal performance compared withthat system before

In chapter 2, we present related work and background information needed for a better

understanding of this paper We describe our approach and outline our implementation in

chapter 3 We present our experiments and results in chapter 4 Finally, we conclude and offer suggestions for future works in chapter 5.

2.1 Squid web cache proxy

Proxy server is a server that is intermediary for requests form clients surfing resources fromInternet or other servers Clients have to connect to server, requesting some resource such as afile, connection, web page, or may be other resources from different servers Today, most of theproxies are web proxy, providing access to content on the World Wide Web [2]

There are many purposes of using proxy server:

• Security, keep client behind proxy server

• Accelerate access to resources (caching proxy).\

• Building access policy to network services or content.

• Accessing prohibited sites by ISP

• Logging clients Internet usage

Reverse proxy is an direct assess Internet proxy used as a gateway to control and protect access

to a server on a private network, and also acting tasks as load-balancing, authentication,decryption or caching [1]

Squid is a caching proxy for the Web With Squid, we can use less bandwidth on our Internetconnection when surfing the Web, reduce the client latency for a web pages, protect the hosts byproxying their web traffic, block unwanted website to users, control users by authenticate toaccess Internet, decrease the load to web server [10]

Squid job is to be a proxy and a cache also As a proxy, Squid control Internet access It receivesrequest from clients, processes that requests, and then forwards the request to the origin server.The requests may be rejected, logged, modified before be forwarded The responses may becached for latter usage and sent back to clients also By caching content in its own memory,instead of fetching contents from origin server, Squid directly return to user the cached copy ofthat request This makes reduce bandwidth and latency for a web pages

Squid – web caching do the jobs of storing the web resources for future reuse For example,when a user visits a site, the Squid cache stores the HTML page and other resources in that site.Latter, others visits that site again, the validate cache which stored in squid memory should beresponse for them It is a “cache hit”, when a request is responded to client by fetching fromsquid caches A “cache miss” occurs when Squid cannot get the validate response from its cache.The cache miss take place when the request may be uncatchable, first requested (not have anycopy of that request)

Squid runs on popular Linux systems, and Microsoft Windows, too Squid’s hardware requiresmost on memory The shortage of memory causes significant degradation in performance Diskspace is another important factor More capacity disk space have, more cached copy can bestored, and higher hit ratios Because of more request memory, high speed CPU is not asimportant as high speed memory, disks Squid uses a small amount of memory for every cachedresponse We have a relationship between amount of memory and amount of disk space Each32MB of memory is needs for each GB of disk space

The amount of disk space is one of factor decide high hit-ratio Disk space is enough if it canhold 3-7 days of worth web traffic [3] For example, the VNUnet use up 80Mbps for 10 hoursper day, we need space between 600GB and 1200GB This number device into 4 squid proxy inVNUnet, we have to prepare 150 GB – 300 GB for each cache proxy.

Objective of Squid proxy in VNUnet

The VNUnet is complexity network include up to hundreds of servers, providing all internetactivity for student, staff office, and others There are webservers, portal webs, blackboardcourse, authentication server, resources, and so on They are together supporting for all people inVietnam National University The overview of VNUnet is shown in Figure 1.1, which showsserver status, how serves are connected, connection status, etc

Figure 1.1 The University Campus Network.

With 800-1000 frequency users every day, performance is the most important factor of thisnetwork In common, each student who owns a computer with Internet connection has Internet

speed of, at least, 2Mbps Therefore, we must have a 1600Mbps – 2000Mbps Internetconnection The numbers are extremely high, and it is impossible to set a high speed connectionlike that In case of possible, the high speed connection costs a great of money However, theproblem is able to solve by using squid Not only because of squid possibility but also because ofuser habitat are together partly to be solved.

Most of the students have a trend of visiting a small number of frequency websites Facebook isthe most visited sites with 30% total number of request follow by others popular website such asDantri.com.vn, Vnexpress, Zing.vn But Youtube.com, the most bandwidth consumer’s site,takes around 40 percent of total bandwidth By cache most request resource in Squid, we save alot of time to directly forward to remote address A cache is fetched from very fast memory anddisk drivers The results for caching frequency site is that hit-ratio is somewhere between25.36% and 85.05%

Moreover, we also did not find any time that all students with internet use Time key is to startfrom 2pm until 2am of the following day Accesses surge usually begins from 8 pm andgradually reduces after midnight We prefer to serve student who read document in the web SoSquid also is configured with the priority for that purpose The popular website where studentsoften access to the social news sites, total access to 10 pages is more than total most of all 30thousand addresses that we recorded in the log of the squid Ten pages there are also the cacheobject is most effective with over 50% hit-rate ratio The sites usually contain elements asimages, css, js, html, in which picture has most portion in terms of size and quantity

Currently, the VNUnet central uses two servers to install and run the squid proxy service.Students are also advised to set up proxy for their computer to access Internet However, wecannot share all of the users for each server should have occurred while the status of this server isserved multiple users, the other is unoccupied We measured the number of simultaneousconnections to each server is limited to 500 connections, the browsing speed significantlyreduced, as can be clearly seen on the web browser through the proxy in this situation, siteslatency increased For this reason, we also limit the number to connect to server at 6-10concurrent connections for each client However, the current popular browsers can open up to 6

to connect simultaneously the same domain, and with opening multiple tabs, the number ofconcurrent connections increases there each time Normally, each user typically open more fourtabs when browsing, or we have to serve about 30 simultaneous connections for each client [6]

We also realize that the size of request objects is from 100bytes to 1MB The request comesfrom the images, html files, and other components of the site as css, js, swf If these objects are

cached on the squid server, we will save bandwidth and requests outside, and speed up pageloads for clients.

As far as we know, people have used squid cache over the world and have achieved many timesbetter results than we do Although the deployment context is difference, we will also study toimprove the ability of squid proxy

2.2 Server load balance

As efforts to upgrade the server hardware has reached a limit on processor speed, memorycapacity, in addition, the ability to connect people start thinking about the cheapest solution, easy

to use, easy to upgrade, more manageable, easy to maintain One more problem posed is how todivide the load to achieve the system work better before upgrading that system Load balancing

is a computer networking methodology to distribute workload to multiple computers, networklinks, disk drivers, or other resources, to achieve optimum efficiency, maximum throughput, and

minimum response time and prevent overload We investigated haproxy as free open-source server load balancer in our work This session will focus on haproxy in application and

specification

2.2.1 haproxy - server load balancer

To be recommended, we have a look at how to deploy Haproxy as temporary solution for theissue of load balancing in Vnunet HAProxy is a free, fast and reliable solution offering highavailability, load balancer, and proxy for TCP and HTTP-based applications It is particularlysuited for web sites crawling under very high loads while needing persistence or Layer7processing Supporting thousands of connections is clearly realistic with today’s hardware Itsmode of operation makes its integration into existing architectures very easy and riskless, whilestill offering the possibility not to expose fragile web servers to the Net The sample model ofhaproxy in its support page is shown in Figure 2.1 We decide to apply this production astemporary solution for server load balancing

Figure 2.1 Testing model of haproxy in support website.

2.2.2 Haproxy specification

In order to measure haproxy performance, we focus on 3 important factors They are session

rate, session concurrency, and data rate

The session rate

This factor is very important It determines when the load balancer will not be able to distributethe entire request it receives The factor is mostly depending on the CPU This factor is measuredwith varying objects sizes The fastest response is coming from empty objects Session rate issimilar with requests/s or hits/s It also is the same as sessions/s in HTTP/1.0 or HTTP/1.1 withkeep-alive disabled

The session concurrency

The session concurrent is measured as follow The situation is that a load balancer receives

10000 sessions per second and the servers respond in 100ms, then the load balancer will have

1000 concurrent sessions The result is limited by the amount of memory and the amount of descriptors the system can handle Haproxy take about 16kB per session, in ideal, we will havearound 60000 sessions per GB of RAM But socket buffers in system also need some memory

file-The data rate

The data rate is at the opposite of the session rate It is demonstrated in Megabytes/s, orMegabits/s High data rates are achieved with large objects to minimize the overhead caused bysession setup and close Large objects increase session concurrency

As a load balancer, haproxy is gateway of all network traffic In the backend, in this case, there are some servers to response request through haproxy haproxy is depending on load balancing

algorithm to make a decision of what server to be forwarded that requests [11]

The logging is optional haproxy in default not generate any log because of it is not built up with When enabled, haproxy send UDP log packets to the specific address to other logging service as syslog or rsyslog Both haproxy and rsyslog must be configured to log could generate

and be cached exactly

We decide to configure haproxy as load balancer in HTTP layers (layer 7) Clients have Intranet

IP address of 5.0.0.0/8 and 10.0.0.0/8 Servers have an interface with private IP address of172.0.0.0/8 and another Interface with public Internet address We, however, only utilize the

private eth interface Clients set proxy in their network configuration to haproxy server Haproxy

receives client requests and forward that requests to other squid proxies in the backend Whenrequest come to squid proxy, requests, depending on squid cache and requests, will directforward to remote host or fetched from squid cache Responses are travel in the path requestcame to clients The model is described in Figure 2.2

Figure 2.2 Deployment model of haproxy load balancer.

2.3 Operating system - CentOS

The last thing this paper deals with is tuning the server performance up There are many systemdefault value which decrease overall system ability We are going to introduce file descriptor,

2.3.1 File descriptors and max-file

File descriptors are the numbers that identify each file and socket that a process has opened Bydefault, number of file descriptors in CentOS is 1024 Therefore, total number of concurrent file

to be open by each user or process is limited by 1024 minus the reserve for operating system.The number is around 1000 file descriptors

This number could be set by function ulimit –n [number-file-descriptor] for temporary log in session The current system limit for file descriptors could check in /proc/sys/fs/file-max By

increase the number in that file, the limit would be changed immediately However, the

configuration should be persistent in system Each restart time, the number was set by ulimit,

which is stored temporary in /proc/sys/fs/file-max and /proc/sys/fs/file-nr To do the persistentchanges, we are going to increase the maximum number of open file on system by editing the/etc/security/limits.conf configuration file

2.3.2 Sysctl.conf

sysctl is used to modify kernel parameters at runtime The parameters available are listed

under /proc/sys/ directory There are a number of configuration options in sysctl.conf In thescope of this thesis, we have changed the following the session of tcp and fs Each alterative intcp configuration will take a modification of TCP connection In fs session, the file system isdefined in fs.file-max option Hence, the bigger number file-max is, the more number of filesystem can use [14]

2.3.3 SElinux

SElinux is stand for Security-Enhanced Linux It is a mandatory access control (MAC) securitymechanism implemented in the kernel SElinux was introduced in CentOS4 first, and have asignificantly enhancement in CentOS 5 We do not focus much on this service, so simply disable

it [12]

2.3.4 Service rsyslog

Rsyslog is an upgrade of syslog service It is an open-source software It implements the basic

syslog protocol, extends syslog with content-based filtering, rich filtering capabilities, flexible

configuration alterations and adds important feature such as TCP, UPD for transport In default,

CentOS is not configured for UPD, which is using by haproxy log generate [13].

2.3.5 Service iptables

Iptables is building blocks of a framework inside the Linux 2.4 and 2.6 Iptables uses the concept

of IP addresses, protocols (tcp, udp, icmp) and ports also Iptables places rules into predefinedchains (INPUT, FORWARD, and OUPUT) that are checked against any network traffic (IPpackets) suitable to those chains and making a decision about what to do with each packet based

on those rules: accepting, dropping the packet [5]

Chapter 3

DEPLOYMENT

To deployment haproxy for load balance purpose, we concentrate on three factors Firstly, preparing system to be able compile haproxy source code needs gcc compiler installed To capture haproxy log, we used rsyslog service which listens UDP packet log that is generated by haproxy Moreover, both iptables and SElinux should be closed After preparing system, we compile haproxy and configure it The third step is tune haproxy’s performance up and tune

capability of operating system up, and making changes policy in Squid proxy servers

3.1 Haproxy server configuration

Change IP address of haproxy machine to 172.16.1.204 and 172.16.1.205 That IP addresses are

IP addresses of Squid proxy servers which is set in client in case of surfing the Internet

Haproxy is also to be set to listen on port 8080, the same as the request port is set in client web browsers By change Haproxy IP address, and listen port in haproxy, users will not have any

trouble and disturb in using Internet We also do not need to notice the new configure about newproxy IP or port This will make all users have to change, as a result, it waits time and makenoisy