求助。LVS/TUN,realserver 找client端问题。

困扰3天了。一直找不到原因。

是这样的,我用tcpdump看到director确实收到了client的包。而且real server也成功避免对Virtual IP的arp reply.
在real server端,也收到了来自director的包。但是real server找不到回到client端的路。发出很多对client的arp request,client也应答了,但是似乎收不到。一直在问。
值得一提的是,client端, realserver和Director都在同一网端。
Client IP: 172.25.21.189

Vitual IP: 172.25.21.231
Director IP: 172.25.21.177
real server 172.25.21.21

抓包内容如下:
Director: (收到了来自client端的http请求)
[]# tcpdump dst 172.25.21.231
tcpdump: listening on eth0
12:21:33.038797 arp who-has 172.25.21.231 tell 172.25.21.189
12:21:33.039335 172.25.21.189.1163 > 172.25.21.231.http: S 3271125168:3271125168(0) win 65535 (DF)
12:26:54.600168 arp who-has 172.25.21.231 tell 172.25.21.189
12:26:54.600786 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF)
12:26:57.588094 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF)
12:27:03.603497 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF)
12:27:29.595063 172.25.21.189.1185 > 172.25.21.231.http: S 3262471317:3262471317(0) win 65535 (DF)
12:27:32.586584 172.25.21.189.1185 > 172.25.21.231.http: S 3262471317:3262471317(0) win 65535 (DF)
12:27:38.601938 172.25.21.189.1185 > 172.25.21.231.http: S 3262471317:3262471317(0) win 65535 (DF)

realserver端:(收到了来自Director的包)
[]# tcpdump src 172.25.21.177
tcpdump: listening on eth0
11:23:06.957349 172.25.21.177 > 172.25.21.21: 172.25.21.189.1163 > 172.25.21.231.http: S 3271125168:3271125168(0) win 65535 (DF) (ipip-proto-4)
11:28:28.522120 172.25.21.177 > 172.25.21.21: 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF) (ipip-proto-4)
11:28:31.509335 172.25.21.177 > 172.25.21.21: 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF) (ipip-proto-4)
11:28:37.524806 172.25.21.177 > 172.25.21.21: 172.25.21.189.1179 > 172.25.21.231.http: S 368996646:368996646(0) win 65535 (DF) (ipip-proto-4)

[]# tcpdump dst 172.25.21.189 (realserver一直在找client,我从client端确实看到了reply,但是毫无作用,依然一直问。)
tcpdump: listening on eth0
11:23:06.957419 arp who-has 172.25.21.189 tell 172.25.21.231
11:23:07.948609 arp who-has 172.25.21.189 tell 172.25.21.231
11:23:08.948605 arp who-has 172.25.21.189 tell 172.25.21.231
11:23:15.926128 arp who-has 172.25.21.189 tell 172.25.21.231
11:23:16.918605 arp who-has 172.25.21.189 tell 172.25.21.231
11:23:17.918603 arp who-has 172.25.21.189 tell 172.25.21.231

请教各位,给指点一下。如何才能让realserver找到client端?

Forums:

你能否罗列一下在真实服务器作的所有设置,如IP地址和网络路由。

我猜是在真实服务器上没有将VIP地址的掩码设置为255.255.255.255。

echo 0 > /proc/sys/net/ipv4/ip_forward
ifconfig tunl0 172.25.21.231 netmask 255.255.255.255 broadcast 172.25.21.231
arptables -F
arptables -A IN -d 172.25.21.231 -j DROP
arptables -A OUT -d 172.25.21.231 -j mangle --mangle-ip-s 172.25.21.177

route add -host 172.25.21.231 dev tunl0

I've been lokiong for a post like this for an age

You've really helped me untdasernd the issues. Thanks.

现在我在一个HUB上连接了所有的机器。
按照realserver的如上配置,已经没有问题了。

但是如果把realserver放在另一个地方,即realserver和director之间有一个switch,那依然是上面的问题。不知道是不是因为这个原因呢?

注明一下:
第一种情况下,client端和realserver, director中间隔了一个switch.
第二种情况下,client端和一台realserver在switch一端。另一台realserver和director用hub相连。见图。奇怪的是,client和rs2连接没问题,和rs1确有问题。同样是收不到arp reply.

rs1, client --switch -- rs2, DR

randomness