请教LVS/TUN方式的配置问题

Submitted by linuxcn.com on 周二, 2006-07-04 06:06

问题描述:
Loadbalance:192.168.0.10----kernel支持LVS
Loadbalance1:192.168.0.11----kernel支持LVS
Realserver1:192.168.0.12----kernel打过避免ARP问题的补丁
Realserver2:192.168.0.13----kernel打过避免ARP问题的补丁

前面的准备工作已经做完了,并且,在这4台机器上,我LVS/DR方式能够正常工作,但我现在把LVS的参数改到LVS/TUN的方式却不能够工作,为什么呢?
是不是前端调度机器Loadbalance也要打避免ARP问题的补丁?

具体的配置我是这样做的,在调度机Loadbalance上
ifconfig eth0:0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20 up
echo "1" >/proc/sys/net/ipv4/ip_forward
IPVSADM -A -t 192.168.0.20:80 -s rr
IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.12 -i
IPVSADM -a -t 192.168.0.20:80 -r 192.168.0.13 -i

在Realserver1和Realserver2上:
ifconfig tunl0 192.168.0.20 netmask 255.255.255.255 broadcast 192.168.0.20
route add -host 192.168.0.20 dev tunl0
echo "1" > /proc/sys/net/ipv4/ip_forward
echo "1" > /proc/sys/net/ipv4/conf/all/hidden
echo "1" > /proc/sys/net/ipv4/conf/tunl0/hidden

192.168.0.20是虚拟IP,但我不能够通过虚拟IP访问到Realserver的web服务

Forums:

linuxcn.com

周四, 2006-07-06 06:52

自己顶一下,章博士来指导一下吗

linuxcn.com

周三, 2006-07-12 01:53

章博士进来看看啊

denielcheng

周五, 2010-05-21 01:59

echo "1" > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

##如下是为了解决源地址验证问题
echo "0" > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter

sysctl -p

winsonzhou1981

周四, 2010-06-10 09:38

LB服务器端
[root@HA1 ~]# more lvs_tun.sh
#!/bin/bash
ifconfig eth0:100 192.168.1.254 netmask 255.255.255.255 broadcast 192.168.1.254 up
route add -host 192.168.1.254 dev eth0:100
ipvsadm -C
ipvsadm -A -t 192.168.1.254:80 -s rr
ipvsadm -a -t 192.168.1.254:80 -r 192.168.1.222 -i
ipvsadm -a -t 192.168.1.254:80 -r 192.168.1.223 -i
ipvsadm
echo "0" >/proc/sys/net/ipv4/ip_forward
echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects
echo "1" >/proc/sys/net/ipv4/conf/eth0/send_redirects

RS01和RS02设置如下：
[root@RS01 ~]# more lvs_tun.sh
ifconfig tunl0 192.168.1.254 netmask 255.255.255.255 broadcast 192.168.1.254 up
route add -host 192.168.1.254 dev tunl0
echo "0" > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
关闭RS 防火墙，可以实现

Aaron

周四, 2011-06-23 07:31

关闭了RS和LS的防火墙，可还是不通~~~，请章博士解答下~~

Anonymous (没有被验证)

周三, 2014-03-05 13:12

That's a knowing answer to a difiucflt question

Anonymous (没有被验证)

周三, 2014-03-05 22:27

If my problem was a Death Star, this article is a photon toodrpe.

Anonymous (没有被验证)

周五, 2014-05-30 15:48

Well done to think of sohmteing like that

Anonymous (没有被验证)

周二, 2014-03-25 02:08

viagara without prescription nj auto insurance quotes courses online religion Cipro reactions

Anonymous (没有被验证)

周二, 2014-05-27 15:19

buy prednisone without prescription answer for homework wheretobuyviagraqw.com assignment writing service interesting biology topics

Anonymous (没有被验证)

周五, 2014-12-19 12:52

Why does this have to be the ONLY reallbie source? Oh well, gj!

Anonymous (没有被验证)

周二, 2014-12-23 09:01

Got it! Thanks a lot again for heinlpg me out!

Anonymous (没有被验证)

周三, 2014-12-24 17:24

insurance life policy

Anonymous (没有被验证)

周一, 2014-12-29 13:54

brand cialis online

Anonymous (没有被验证)

周五, 2015-01-09 08:50

generic viagra

Anonymous (没有被验证)

周五, 2015-02-13 02:52

neuro buy viagra over inches blood pressure cialis problem degenerative changes california car insurance quote home

Anonymous (没有被验证)

周五, 2015-02-20 05:58

vehicle theft auto insurance quotes enough business viagra levitra ethnic group basis lowest propecia prices reduce

Anonymous (没有被验证)

周二, 2015-04-21 15:27

applicant car insurance quotes legal death benefit cheap life insurance take motor doctor especially discount levitra online towards moving hospital discount cialis scene people ignore auto insurance would

Anonymous (没有被验证)

周三, 2015-04-22 16:02

erection online cialis other anti-impotency auto cheap auto insurance insurance really online life insurance start exercising offer best cheap insurance relief services accident insurance life insurance rates life

Anonymous (没有被验证)

周六, 2015-05-09 14:33

generated during custom essay audience clear essay service advanced placement ph paper writing service students grouping want students custom writing events linked good assignment help silver updated write my essay reference

Anonymous (没有被验证)

周日, 2015-05-10 00:58

hire professionals proofreading services desired content child write an essay generations more college essay find essay academic writing stay checkbook paper writer sentence extremely dry essay writing stifles

Anonymous (没有被验证)

周四, 2014-10-16 06:58

particularly appealing buy levitra car insurance trustworthy broker cheap life insurance

Anonymous (没有被验证)

周一, 2016-02-29 05:28

你这个还是同个网段的，这个一直没有问题

whzhuang

周三, 2013-04-10 01:50

可以了, 都是arp搞的鬼, 以前有个hidden可以设置, 现在改了那么多, 换了vip, 就搞起来了,
为什么以前设置的vip一直存在呢? 怎么把以前设置的vip全部删除掉?

ddxxkk

周日, 2007-05-13 09:48

我也碰到同样的的问题LVS/TUN试验不成功,用tcpdump检查,发现是客户机把信息给了LB,LB信息也给了RS,但RS没传给客户机.
VIP 172.16.33.44
服务器配置
/sbin/ifconfig tunl0 172.16.33.44 broadcast 172.16.33.44 netmask 255.255.255.255 up
/sbin/route add -host 172.16.33.44 dev tunl0
ipvsadm -C
ipvsadm -A -t 172.16.33.44:8088 -s rr
ipvsadm -a -t 172.16.33.44:8088 -r 172.16.33.237:8088 -i -w 1

eth0 Link encap:Ethernet HWaddr 00:0C:29:C3:25:B8
inet addr:172.16.33.146 Bcast:172.16.33.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:112228 errors:0 dropped:0 overruns:0 frame:0
TX packets:1087 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7431035 (7.0 Mb) TX bytes:126080 (123.1 Kb)
Interrupt:10 Base address:0x1080

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:700 (700.0 b) TX bytes:700 (700.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:172.16.33.44 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

[root@PC-0095 boot]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.33.44 * 255.255.255.255 UH 0 0 0 tunl0
172.16.33.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
default 172.16.33.254 0.0.0.0 UG 0 0 0 eth0

真实服务器配置
ifconfig tunl0 172.16.33.44 netmask 255.255.255.255 broadcast 172.16.33.44 up
/sbin/route add -host 172.16.33.44 dev tunl0
echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

eth0 Link encap:Ethernet HWaddr 00:0F:EA:52:4C:C4
inet addr:172.16.33.237 Bcast:172.16.33.255 Mask:255.255.255.0
inet6 addr: fe80::20f:eaff:fe52:4cc4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7393925 errors:0 dropped:0 overruns:0 frame:0
TX packets:2874675 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4232092397 (3.9 GiB) TX bytes:208128177 (198.4 MiB)
Interrupt:201 Memory:e1000000-0

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:12026 errors:0 dropped:0 overruns:0 frame:0
TX packets:12026 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:637835 (622.8 KiB) TX bytes:637835 (622.8 KiB)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:172.16.33.44 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

root@OASERVER bin]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.33.44 * 255.255.255.255 UH 0 0 0 tunl0
172.16.33.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth0
default 172.16.33.254 0.0.0.0 UG 0 0 0 eth0

客户机是
192.16.33.157

because

周二, 2007-06-26 02:49

确认RS收到了IPIP包了没？

我这里的情况是IPIP包被防火墙挡着了（开始拿安式的防火墙当路由器用的，后来拿2000的系统做路由，发现一样被挡着），执行服务器没有收到IPIP包。

还有路由器上的rp_filter可能需要改成0，不进行反向路径过滤。举例而言，RS的IP为172.16.27.155.VIP为65.28.41.23，那么在172.16.27网段出现了源地址为65网段的包很可能会被路由器丢掉。

jhxhj

周二, 2010-05-11 14:23

VIP 172.16.33.44
服务器配置
/sbin/ifconfig tunl0 172.16.33.44 broadcast 172.16.33.44 netmask 255.255.255.255 up
/sbin/route add -host 172.16.33.44 dev tunl0
这个设置只在real server上设置就可以了，在服务器上把vip设置在你的虚拟网卡上就可以，如eth0:1(外网网卡）
同时把IP转发开起
echo "1" >/proc/sys/net/ipv4/ip_forward

tscck

周五, 2007-05-25 09:03

我成功實驗了LVS/DR,但就是架設LVS/TUN總是失敗,我機乎己搜索中英文文章, 設定也如上述兩位人兄差不多, 好像這裏還未有人能回答關於設定LVS/TUN的成功例子,真的很暈 :(

tscck

周五, 2007-05-25 09:42

client-->192.168.200.166(VIP:192.168.200.165)-->PIX FW-->Real1(192.168.215.170)
|

|-->Real2(192.168.215.171)

LVS資料
lb1:~ # ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.200.165:http wlc
-> 192.168.215.170:http Tunnel 1 0 1
-> 192.168.215.171:http Tunnel 1 0 0

Real1資料
srv1:~ # netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.200.165 0.0.0.0 255.255.255.255 UH 0 0 0 tunl0
192.168.215.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.215.1 0.0.0.0 UG 0 0 0 eth1
srv1:~ # ifconfig -a
eth1 Link encap:Ethernet HWaddr 00:50:56:85:0D:CB
inet addr:192.168.215.170 Bcast:192.168.215.255 Mask:255.255.255.0
inet6 addr: fe80::250:56ff:fe85:dcb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:168776 errors:0 dropped:0 overruns:0 frame:0
TX packets:136183 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15446564 (14.7 Mb) TX bytes:24834436 (23.6 Mb)
Interrupt:177 Base address:0x1400

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:176 errors:0 dropped:0 overruns:0 frame:0
TX packets:176 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13614 (13.2 Kb) TX bytes:13614 (13.2 Kb)

sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.200.165 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

srv1:~ # sysctl -p
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth1.arp_ignore = 1
net.ipv4.conf.eth1.arp_announce = 2
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
srv1:~ #

Real2資料
srv2:~ # netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.200.165 0.0.0.0 255.255.255.255 UH 0 0 0 tunl0
192.168.215.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 192.168.215.1 0.0.0.0 UG 0 0 0 eth1
srv2:~ # ifconfig -a
eth1 Link encap:Ethernet HWaddr 00:50:56:85:6B:EC
inet addr:192.168.215.171 Bcast:192.168.215.255 Mask:255.255.255.0
inet6 addr: fe80::250:56ff:fe85:6bec/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:160196 errors:0 dropped:0 overruns:0 frame:0
TX packets:103515 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:14809996 (14.1 Mb) TX bytes:12402100 (11.8 Mb)
Interrupt:177 Base address:0x1400

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:178 errors:0 dropped:0 overruns:0 frame:0
TX packets:178 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:13367 (13.0 Kb) TX bytes:13367 (13.0 Kb)

sit0 Link encap:IPv6-in-IPv4
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.200.165 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

srv2:~ # sysctl -p
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.eth1.arp_ignore = 1
net.ipv4.conf.eth1.arp_announce = 2
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0

就是連不到Real1及Real2的HTTP服務, TCPDUMP看到VIP連接RIP的活動, 但看不到RIP經TUNL0回復VIP或CIP的活動, 請章博士指教!!

binasa

周二, 2017-02-14 14:53

I found your this post while searching for some related information on blog search...Its a good post..keep posting and update the information.
Get The Free Things

binasa

周六, 2017-02-18 09:24

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post.
Download Games