고가용성.
Client - Web Proxy Server - Web Server X 3 - DB Server
위 구조에서 프록시 서버는 웹서버를 정해진 규칙에 따라 요청. 웹서버는 동일 DB 서버를 사용.
로드밸런싱 방법
* 사설 IP에서 Name 서버를 따로 설정하였음.
Apache
vim /etc/httpd/conf.d/loadbalancing.conf
<VirtualHost [IP/Domain]:[Port]>
ServerName [Domain]
DocumentRoot "[Directroy Path]"
<Directory [Directory Path]>
AllowOverride All
</Directory>
<Proxy "balancer://tiredcluster">
BalancerMember [IP/Domain]:[Port]
ProxySet stickysession=[Loadblancing Method]
</Proxy>
ProxyPass "/" "balancer://tiredcluster/"
ProxyPassReverse "/" "balancer://tiredcluster/"
ErrorLog [Log Path]
CustomLog [Log Path] [Log Format]
</VirtualHost>
Ex :
<VirtualHost 172.16.20.6:80>
ServerName www.tired.com
DocumentRoot "/var/www/html"
<Directory /var/www/html>
AllowOverride All
</Directory>
<Proxy "balancer://tiredcluster">
BalancerMember http://w1.tired.com
BalancerMember http://w2.tired.com
BalancerMember http://w3.tired.com
BalancerMember http://w4.tired.com
ProxySet stickysession=ROUTEID
</Proxy>
ProxyPass "/" "balancer://tiredcluster/"
ProxyPassReverse "/" "balancer://tiredcluster/"
ErrorLog logs/w1.tired.com-error_log
CustomLog logs/w1.tired.com-access_log common
</VirtualHost>
<VirtualHost *:80>
ServerName www.tired.com
DocumentRoot "/var/www/html"
<Directory /var/www/html>
AllowOverride All
</Directory>
</VirtualHost>
XFF Log
LogFormat "%h \"%{X-Forwarded-For}i\" %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
Loadbalancing Method
- Round Robin ( roundrobin )
: 기본옵션. 돌아가면서 요청 - Least Connections ( leastconn )
: 가장 적은 요청을 받은 서버를 요청 - Source IP Hashing ( source )
: 특정 IP를 매핑하여 요청 - Random
: 무작위 - By Traffic ( bytraffic )
: 가장 적은 트래픽 서버 요청 - By Requests ( byrequests )
: 가장 적은 현재 요청 수를 가진 서버를 요청 - Sticky Session
: 클라이언트와 서버의 세션을 연결하여 동일 서버로 요청
Setting : ProxySet stickysession=JSESSIONID
Log Format
- common: 기본적인 로그 형식
- combined: common 로그에 추가적인 정보 (예: Referer, User-Agent 등)를 포함
- %h: 클라이언트 IP 주소
- %l: 사용자 식별 (사용되지 않음, 항상 -)
- %u: 인증된 사용자 (로그인한 경우)
- %t: 요청 시간 (클라이언트 시간대)
- %r: 요청 라인 (예: GET /index.html HTTP/1.1)
- %>s: HTTP 상태 코드
- %b:헤더 제외 응답 크기 (바이트 단위, -는 크기가 없음을 의미)
- %{Referer}i: Referer 헤더 값
- %{User-Agent}i: User-Agent 헤더 값
- %D: 요청 처리 시간 (마이크로초 단위)
- %T: 요청 처리 시간 (초 단위)
Nginx
vim /etc/nginx/conf.d/loadblancing.conf
upstream [Upstream Server Name] {
[Loadbalancing Method]
server [IP/Domain]:[Port];
}
server {
listen 80;
location / {
proxy_set_header Host $host;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://tired;
}
}
upstream tired {
server w1.tired.com;
server w2.tired.com;
server w3.tired.com;
server w4.tired.com;
}
Ex :
upstream tired {
server w1.tired.com;
server w2.tired.com;
server w3.tired.com;
server w4.tired.com;
}
server {
listen 80;
location / {
proxy_set_header Host $host;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://tired;
}
access_log /var/log/nginx/proxy_acclog.log;
}
XFF Log
# Nginx의 경우 해당 값이 기본값임
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
Loadbalancing Method
- Round Lobin
: 기본값 - Least Connections ( least_conn; )
- IP Hash ( ip_hash; )
- Least Time ( least_time header; )
- Rendom ( random two least_time=last_byte; )
Proxy Header
- proxy_set_header Host $host;
: Host 헤더를 Client의 host로 설정 - proxy_set_header Connection "";
: Connection을 빈 값으로 설정. 프록시 서버의 경우 만약 해당 헤더가 close일 경우 클라이언트와의 연결을 끊을 수 있기 때문. - proxy_set_header X-Real-IP $remote_addr;
: X-Real-IP를 client의 IP로 설정 - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
: X-Forwarded-For 헤더에 현제 Proxy서버 IP를 넣음. 넣기 전 빈 값이면 client의 IP를 넣고 프록시 서버 IP를 넣음. - proxy_set_header X-Forwarded-Proto $scheme;
: X-Forwarded-Proto 헤더 값을 일정하게 유지. HTTP/HTTPS 연결을 계속해서 동일한 연결로 이어지도록 함.
ha-proxy
Log Sample
Apache
1. Access_log
172.16.1.254 - - [02/Dec/2024:14:29:44 +0900] "HEAD / HTTP/1.0" 200 -
172.16.20.1 - - [02/Dec/2024:14:30:03 +0900] "GET /owncloud/status.php HTTP/1.1" 404 196
172.16.1.254 - - [02/Dec/2024:14:30:14 +0900] "HEAD / HTTP/1.0" 200 -
172.16.1.254 - - [02/Dec/2024:14:29:14 +0900] "HEAD / HTTP/1.0" 200 -
172.16.1.254 - - [02/Dec/2024:14:30:44 +0900] "HEAD / HTTP/1.0" 200 -
172.16.20.1 - - [02/Dec/2024:14:31:05 +0900] "GET /owncloud/status.php HTTP/1.1" 404 196
172.16.1.254 - - [02/Dec/2024:14:31:14 +0900] "HEAD / HTTP/1.0" 200 -2. Error_log
[Mon Dec 02 14:03:40.001307 2024] [proxy_http:error] [pid 727464:tid 727507] [client 172.16.20.1:52755] AH01114: HTTP: failed to make connection to backend: w2.tired.com
[Mon Dec 02 14:03:50.988403 2024] [proxy:error] [pid 727464:tid 727513] (70007)The timeout specified has expired: AH00957: http: attempt to connect to 172.16.13.19:80 (w2.tired.com:80) failed
[Mon Dec 02 14:03:50.988816 2024] [proxy_http:error] [pid 727464:tid 727513] [client 172.16.13.15:50122] AH01114: HTTP: failed to make connection to backend: w2.tired.com
3. Client_access_log
172.16.20.6 "172.16.20.1" - - [02/Dec/2024:03:19:32 -0500] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.1.254 "-" - - [02/Dec/2024:03:19:41 -0500] "HEAD / HTTP/1.0" 302 - "-" "-"
172.16.1.254 "-" - - [02/Dec/2024:03:19:45 -0500] "HEAD / HTTP/1.0" 200 - "-" "-"
172.16.20.6 "172.16.1.254" - - [02/Dec/2024:03:19:45 -0500] "HEAD / HTTP/1.1" 200 - "-" "-"
172.16.1.254 "-" - - [02/Dec/2024:03:20:11 -0500] "HEAD / HTTP/1.0" 302 - "-" "-"
172.16.1.254 "-" - - [02/Dec/2024:03:20:15 -0500] "HEAD / HTTP/1.0" 200 - "-" "-"
172.16.20.6 "172.16.20.1" - - [02/Dec/2024:03:20:37 -0500] "GET /owncloud/status.php HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Windows) mirall/5.3.1.14018 (ownCloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
172.16.1.254 "-" - - [02/Dec/2024:03:20:41 -0500] "HEAD / HTTP/1.0" 302 - "-" "-"
172.16.1.254 "-" - - [02/Dec/2024:03:20:45 -0500] "HEAD / HTTP/1.0" 200 - "-" "-"
172.16.20.6 "172.16.11.188" - - [02/Dec/2024:03:21:03 -0500] "GET / HTTP/1.1" 200 7 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0"
Nginx
1. Access_log
172.16.1.254 - - [02/Dec/2024:14:58:23 +0900] "HEAD / HTTP/1.0" 200 0 "-" "-"
172.16.20.1 - - [02/Dec/2024:14:58:28 +0900] "GET / HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.20.6 - - [02/Dec/2024:14:58:28 +0900] "GET / HTTP/1.0" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.20.6 - - [02/Dec/2024:14:58:28 +0900] "GET / HTTP/1.0" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.8.1 - - [02/Dec/2024:14:58:28 +0900] "GET / HTTP/1.1" 200 9 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.20.1 - - [02/Dec/2024:14:58:29 +0900] "GET / HTTP/1.1" 200 17 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.20.1 - - [02/Dec/2024:14:58:40 +0900] "GET / HTTP/1.1" 200 9 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
172.16.20.1 - - [02/Dec/2024:14:58:41 +0900] "GET / HTTP/1.1" 200 17 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"2. Error_log
2024/12/02 14:58:28 [error] 729166#729166: *7 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.8.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.16.20.6:80/", host: "172.16.20.6"
2024/12/02 14:58:28 [error] 729166#729166: *11 upstream timed out (110: Connection timed out) while connecting to upstream, client: 172.16.20.6, server: , request: "GET / HTTP/1.0", upstream: "http://172.16.13.19:80/", host: "172.16.20.6"
2024/12/02 14:58:57 [error] 729166#729166: *17 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.16.20.1, server: , request: "GET /owncloud/status.php HTTP/1.1", upstream: "http://172.16.20.6:80/owncloud/status.php", host: "172.16.20.6"
2024/12/02 14:58:57 [error] 729166#729166: *19 upstream timed out (110: Connection timed out) while connecting to upstream, client: 172.16.20.6, server: , request: "GET /owncloud/status.php HTTP/1.0", upstream: "http://172.16.13.19:80/owncloud/status.php", host: "172.16.20.6"
RR dns
www IN A 172.16.0.1
www IN A 172.16.0.2
www IN A 172.16.0.3
www IN A 172.16.0.4
'네트워크 및 서버 > 서버' 카테고리의 다른 글
| 비동기식 DB 백업(Mysql/Mariadb) (0) | 2024.12.04 |
|---|---|
| Web Server 로드밸런싱 구현 (2) (0) | 2024.12.03 |
| Clamav / Amavis (0) | 2024.11.25 |
| Web(Apache) SSL 인증서 발급 및 적용 (0) | 2024.11.15 |
| 서버 실습 (0) | 2024.11.15 |
