略去导航.
主页
LVS中文站点
Google
 
Web LinuxVirtualServer.org
首页 | 论坛 | 博客 | 手册 | LVS英文站点
主页 » 论坛 » LVS集群

请教LVS/TUN方式的配置问题

由 linuxcn.com 在 周二, 2006-07-04 14:06 提交

问题描述:
Loadbalance:192.168.0.10----kernel支持LVS
Loadbalance1:192.168.0.11----kernel支持LVS
Realserver1:192.168.0.12----kernel打过避免ARP问题的补丁
Realserver2:192.168.0.13----kernel打过避免ARP问题的补丁

前面的准备工作已经做完了,并且,在这4台机器上,我LVS/DR方式能够正常工作,但我现在把LVS的参数改到LVS/TUN的方式却不能够工作,为什么呢?
是不是前端调度机器Loadbalance也要打避免ARP问题的补丁?

具体的配置我是这样做的,在调度机Loadbalance上
ifconfig eth0:0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20 up
echo "1" >/proc/sys/net/ipv4/ip_forward
IPVSADM -A -t 192.168.0.20:80 -s rr
IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.12 -i
IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.13 -i

在Realserver1和Realserver2上:
ifconfig tunl0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20
route add -host 192.168.0.20 dev tunl0
echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/conf/all/hidden
echo "1" > /proc/sys/net/ipv4/conf/tunl0/hidden

192.168.0.20是虚拟IP,但我不能够通过虚拟IP访问到Realserver的web服务

‹ LVS集群会话同步问题 关于配置lvs-tun时，在Director与Realserver在不同网段无法运行的疑惑 ›
» 发表新评论 | 5748 次阅读

我的配置

由 tscck 在 周五, 2007-05-25 17:42 提交

client-->192.168.200.166(VIP:192.168.200.165)-->PIX FW-->Real1(192.168.215.170)
|

|-->Real2(192.168.215.171)

LVS資料
lb1:~ # ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.200.165:http wlc
-> 192.168.215.170:http Tunnel 1 0 1
-> 192.168.215.171:http Tunnel 1 0 0

Real1資料
srv1:~ # netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.200.165 0.0.0.0 255.255.255.255 UH 0 0 0 tunl0
192.168.215.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.215.1 0.0.0.0 UG 0 0 0 eth1
srv1:~ # ifconfig -a
eth1 Link encap:Ethernet HWaddr 00:50:56:85:0D:CB
inet addr:192.168.215.170 Bcast:192.168.215.255 Mask:255.255.255.0
inet6 addr: fe80::250:56ff:fe85:dcb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:168776 errors:0 dropped:0 overruns:0 frame:0
TX packets:136183 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15446564 (14.7 Mb) TX bytes:24834436 (23.6 Mb)
Interrupt:177 Base address:0x1400

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:176 errors:0 dropped:0 overruns:0 frame:0
TX packets:176 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13614 (13.2 Kb) TX bytes:13614 (13.2 Kb)

sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.200.165 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

srv1:~ # sysctl -p
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth1.arp_ignore = 1
net.ipv4.conf.eth1.arp_announce = 2
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
srv1:~ #

Real2資料
srv2:~ # netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.200.165 0.0.0.0 255.255.255.255 UH 0 0 0 tunl0
192.168.215.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.215.1 0.0.0.0 UG 0 0 0 eth1
srv2:~ # ifconfig -a
eth1 Link encap:Ethernet HWaddr 00:50:56:85:6B:EC
inet addr:192.168.215.171 Bcast:192.168.215.255 Mask:255.255.255.0
inet6 addr: fe80::250:56ff:fe85:6bec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:160196 errors:0 dropped:0 overruns:0 frame:0
TX packets:103515 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:14809996 (14.1 Mb) TX bytes:12402100 (11.8 Mb)
Interrupt:177 Base address:0x1400

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:178 errors:0 dropped:0 overruns:0 frame:0
TX packets:178 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13367 (13.0 Kb) TX bytes:13367 (13.0 Kb)

sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.200.165 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

srv2:~ # sysctl -p
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth1.arp_ignore = 1
net.ipv4.conf.eth1.arp_announce = 2
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0

就是連不到Real1及Real2的HTTP服務, TCPDUMP看到VIP連接RIP的活動, 但看不到RIP經TUNL0回復VIP或CIP的活動, 請章博士指教!!

» 回复

也遇到同樣問題

由 tscck 在 周五, 2007-05-25 17:03 提交

我成功實驗了LVS/DR,但就是架設LVS/TUN總是失敗,我機乎己搜索中英文文章, 設定也如上述兩位人兄差不多, 好像這裏還未有人能回答關於設定LVS/TUN的成功例子,真的很暈 :(

» 回复

我也碰到同样的的问

由 ddxxkk 在 周日, 2007-05-13 17:48 提交

我也碰到同样的的问题LVS/TUN试验不成功,用tcpdump检查,发现是客户机把信息给了LB,LB信息也给了RS,但RS没传给客户机.
VIP 172.16.33.44
服务器配置
/sbin/ifconfig tunl0 172.16.33.44 broadcast 172.16.33.44 netmask 255.255.255.255 up
/sbin/route add -host 172.16.33.44 dev tunl0
ipvsadm -C
ipvsadm -A -t 172.16.33.44:8088 -s rr
ipvsadm -a -t 172.16.33.44:8088 -r 172.16.33.237:8088 -i -w 1

eth0 Link encap:Ethernet HWaddr 00:0C:29:C3:25:B8
inet addr:172.16.33.146 Bcast:172.16.33.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:112228 errors:0 dropped:0 overruns:0 frame:0
TX packets:1087 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7431035 (7.0 Mb) TX bytes:126080 (123.1 Kb)
Interrupt:10 Base address:0x1080

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:700 (700.0 b) TX bytes:700 (700.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:172.16.33.44 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

[root@PC-0095 boot]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.33.44 * 255.255.255.255 UH 0 0 0 tunl0
172.16.33.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
default 172.16.33.254 0.0.0.0 UG 0 0 0 eth0

真实服务器配置
ifconfig tunl0 172.16.33.44 netmask 255.255.255.255 broadcast 172.16.33.44 up
/sbin/route add -host 172.16.33.44 dev tunl0
echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

eth0 Link encap:Ethernet HWaddr 00:0F:EA:52:4C:C4
inet addr:172.16.33.237 Bcast:172.16.33.255 Mask:255.255.255.0
inet6 addr: fe80::20f:eaff:fe52:4cc4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7393925 errors:0 dropped:0 overruns:0 frame:0
TX packets:2874675 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4232092397 (3.9 GiB) TX bytes:208128177 (198.4 MiB)
Interrupt:201 Memory:e1000000-0

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:12026 errors:0 dropped:0 overruns:0 frame:0
TX packets:12026 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:637835 (622.8 KiB) TX bytes:637835 (622.8 KiB)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:172.16.33.44 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

root@OASERVER bin]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.33.44 * 255.255.255.255 UH 0 0 0 tunl0
172.16.33.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
default 172.16.33.254 0.0.0.0 UG 0 0 0 eth0

客户机是
192.16.33.157

» 回复

VIP

由 jhxhj 在 周二, 2010-05-11 22:23 提交

VIP 172.16.33.44
服务器配置
/sbin/ifconfig tunl0 172.16.33.44 broadcast 172.16.33.44 netmask 255.255.255.255 up
/sbin/route add -host 172.16.33.44 dev tunl0
这个设置只在real server上设置就可以了，在服务器上把vip设置在你的虚拟网卡上就可以，如eth0:1(外网网卡）
同时把IP转发开起
echo "1" >/proc/sys/net/ipv4/ip_forward

» 回复

确认RS收到了IPIP包了

由 because 在 周二, 2007-06-26 10:49 提交

确认RS收到了IPIP包了没？

我这里的情况是IPIP包被防火墙挡着了（开始拿安式的防火墙当路由器用的，后来拿2000的系统做路由，发现一样被挡着），执行服务器没有收到IPIP包。

还有路由器上的rp_filter可能需要改成0，不进行反向路径过滤。举例而言，RS的IP为172.16.27.155.VIP为65.28.41.23，那么在172.16.27网段出现了源地址为65网段的包很可能会被路由器丢掉。

» 回复

自己顶一下,章博士来

由 linuxcn.com 在 周四, 2006-07-06 14:52 提交

自己顶一下,章博士来指导一下吗

» 回复

章博士进来看看啊

由 linuxcn.com 在 周三, 2006-07-12 09:53 提交

章博士进来看看啊

» 回复

在RS上执行如下命令看看

由 denielcheng 在 周五, 2010-05-21 09:59 提交

echo "1" > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

##如下是为了解决源地址验证问题
echo "0" > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter

sysctl -p

» 回复

LB服务器端[root@HA1 ~]#

由 winsonzhou1981 在 周四, 2010-06-10 17:38 提交

LB服务器端
[root@HA1 ~]# more lvs_tun.sh
#!/bin/bash
ifconfig eth0:100 192.168.1.254 netmask 255.255.255.255 broadcast 192.168.1.254 up
route add -host 192.168.1.254 dev eth0:100
ipvsadm -C
ipvsadm -A -t 192.168.1.254:80 -s rr
ipvsadm -a -t 192.168.1.254:80 -r 192.168.1.222 -i
ipvsadm -a -t 192.168.1.254:80 -r 192.168.1.223 -i
ipvsadm
echo "0" >/proc/sys/net/ipv4/ip_forward
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth0/send_redirects

RS01和RS02设置如下：
[root@RS01 ~]# more lvs_tun.sh
ifconfig tunl0 192.168.1.254 netmask 255.255.255.255 broadcast 192.168.1.254 up
route add -host 192.168.1.254 dev tunl0
echo "0" > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
关闭RS 防火墙，可以实现

» 回复

关闭了RS和LS的防火墙

由 Aaron 在 周四, 2011-06-23 15:31 提交

关闭了RS和LS的防火墙，可还是不通~~~，请章博士解答下~~

» 回复