GRE 实验1 同网段GRE实验
简介
本文描述我为了学习GRE tunnel自己做的几个实验.
包括：
- ipv4 over GRE over IPv4
- ipv6 over GRE over IPv4
- ipv4/ipv6 over GRE over IPv4
- ipv4 over IP6GRE over IPv6
- ipv6 over IP6GRE over IPv6
- ipv4/ipv6 over IP6GRE over IPv6
- ipv4 over GRE over IPsecv4
- ipv6 over GRE over IPsecv4
- ipv4/ipv6 over GRE over IPsecv4
- ipv4 over IP6GRE over IPsecv6
- ipv6 over IP6GRE over IPsecv6
- ipv4/ipv6 over IP6GRE over IPsecv6
准备
添加国内lxc-images镜像
lxc remote add mirror-images https://mirrors.tuna.tsinghua.edu.cn/lxc-images/ --protocol=simplestreams --public
创建arch1、arch2
lxc launch mirror-images:archlinux/current/amd64 arch1
lxc launch mirror-images:archlinux/current/amd64 arch2
设置时区
ln -sf /usr/share/zoneinfo/Asia/Chongqing /etc/localtime
设置国内源 /etc/pacman.d/mirrorlist
Server = http://mirrors.tencent.com/archlinux/$repo/os/$arch
设置loopback网卡 /etc/systemd/network/25-dummy.netdev
[NetDev]
Name=lo1
Kind=dummy
设置地址 /etc/systemd/network/25-dummy.network
arch1
[Match]
Name=lo1
[Network]
Address=192.168.101.1/24
Address=fd00:0:0:1::1/64
arch2
[Match]
Name=lo1
[Network]
Address=192.168.102.1/24
Address=fd00:0:0:2::1/64
重启服务使网络生效
systemctl restart systemd-networkd
lxc 虚拟机配置
lxc list
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| arch1 | RUNNING | 192.168.101.1 (lo1) | fd42:a497:a975:8f4d:216:3eff:fe48:c4db (eth0) | CONTAINER | 0 |
| | | 10.165.103.239 (eth0) | fd00:0:0:1::1 (lo1) | | |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| arch2 | RUNNING | 192.168.102.1 (lo1) | fd42:a497:a975:8f4d:216:3eff:fe23:7b9f (eth0) | CONTAINER | 0 |
| | | 10.165.103.163 (eth0) | fd00:0:0:2::1 (lo1) | | |
+-------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
lxc 网络配置
# lxc network list
+--------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+--------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| eth0 | physical | NO | | | | 0 | |
+--------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
| lxdbr0 | bridge | YES | 10.165.103.1/24 | fd42:a497:a975:8f4d::1/64 | | 3 | CREATED |
+--------+----------+---------+-----------------+---------------------------+-------------+---------+---------+
手工设置IPv4路由进行测试
arch1
ip route add 192.168.102.0/24 via 10.165.103.163
arch2
ip route add 192.168.101.0/24 via 10.165.103.239
实验从arch1上ping 192.168.102.1能通,从arch2上ping 192.168.101.1能通
然后删除路由
arch1
ip route delete 192.168.102.0/24
arch2
ip route delete 192.168.101.0/24
手工设置IPv6路由进行测试
arch1
ip -6 route add fd00:0:0:2::0/64 via fd42:a497:a975:8f4d:216:3eff:fe23:7b9f
arch2
ip -6 route add fd00:0:0:1::0/64 via fd42:a497:a975:8f4d:216:3eff:fe48:c4db
实验从arch1上ping fd00:0:0:2::1能通,从arch2上ping fd00:0:0:1::1能通
然后删除路由
arch1
ip -6 route delete fd00:0:0:2::0/64
arch2
ip -6 route delete fd00:0:0:1::0/64
在宿主机上启用ip_gre、ip6_gre kernel module
modprobe ip_gre
modprobe ip6_gre
实验
ipv4 over GRE over IPv4
参考 ipv4/ipv6 over GRE over IPv4
ipv6 over GRE over IPv4
参考 ipv4/ipv6 over GRE over IPv4
ipv4/ipv6 over GRE over IPv4
arch1
ip tunnel add gre-arch2 local 10.165.103.239 remote 10.165.103.163 mode gre key 42
ip link set gre-arch2 up
ip address add 10.0.3.1 dev gre-arch2 peer 10.0.3.2
ip r a 192.168.102.0/24 dev gre-arch2
ip -6 route add fd00:0:0:2::0/64 dev gre-arch2
arch2
ip tunnel add gre-arch1 local 10.165.103.163 remote 10.165.103.239 mode gre key 42
ip link set gre-arch1 up
ip address add 10.0.3.2 dev gre-arch1 peer 10.0.3.1
ip r a 192.168.101.0/24 dev gre-arch1
ip -6 route add fd00:0:0:1::0/64 dev gre-arch1
实验从arch1上ping 192.168.102.1能通,从arch2上ping 192.168.101.1能通,注意需要像上面倒数第三行一样设置对端ip，不然不通
实验从arch1上ping fd00:0:0:2::1能通,从arch2上ping fd00:0:0:1::1能通,ipv6不需要设置对端IP，我想是因为系统自动为GRE通道产生了link local address,详细如下
arch1
# ip address
...
6: gre-arch2@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1472 qdisc noqueue state UNKNOWN group default qlen 1000
inet6 fe80::aa5:67ef/64 scope link
valid_lft forever preferred_lft forever
...
arch2
# ip address
...
6: gre-arch1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1472 qdisc noqueue state UNKNOWN group default qlen 1000
inet6 fe80::aa5:67a3/64 scope link
valid_lft forever preferred_lft forever
...
如上所示 fd80::0/64网段的地址是系统自动产生的link local address
另外
在arch1
ping fe80::aa5:67ef%gre-arch2
及
ping fe80::aa5:67a3%gre-arch2
都能直接ping通
综上，一个GRE通道可以同时承担IPv4和IPv6流量，不冲突
实验完毕,删除GRE配置
arch1
ip route delete 192.168.102.0/24
ip tunnel delete gre-arch2
arch2
ip route delete 192.168.101.0/24
ip tunnel delete gre-arch1
ipv4 over IP6GRE over IPv6
参见 ipv4/ipv6 over GRE over IPv6
ipv6 over IP6GRE over IPv6
参见 ipv4/ipv6 over GRE over IPv6
ipv4/ipv6 over IP6GRE over IPv6
arch1
ip -6 tunnel add gre6-arch2 mode ip6gre local fd42:a497:a975:8f4d:216:3eff:fe48:c4db remote fd42:a497:a975:8f4d:216:3eff:fe23:7b9f
ip -6 link set gre6-arch2 up
ip address add 10.0.4.1 dev gre6-arch2 peer 10.0.4.2
ip r a 192.168.102.0/24 dev gre6-arch2
ip -6 route add fd00:0:0:2::0/64 dev gre6-arch2
arch2
ip -6 tunnel add gre6-arch1 mode ip6gre local fd42:a497:a975:8f4d:216:3eff:fe23:7b9f remote fd42:a497:a975:8f4d:216:3eff:fe48:c4db
ip -6 link set gre6-arch1 up
ip address add 10.0.4.2 dev gre6-arch1 peer 10.0.4.1
ip r a 192.168.101.0/24 dev gre6-arch1
ip -6 route add fd00:0:0:1::0/64 dev gre6-arch1
实验从arch1上ping 192.168.102.1能通,从arch2上ping 192.168.101.1能通,注意ipv4需要像上面倒数第三行一样设置对端IP，不然不通
实验从arch1上ping fd00:0:0:2::1能通,从arch2上ping fd00:0:0:1::1能通，ipv6不需要设置对端IP，也是因为系统自动为GRE通道产生了link local address
执行以下命名
# ip address
...
9: gre6-arch1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1448 qdisc noqueue state UNKNOWN group default qlen 1000
link/gre6 fd42:a497:a975:8f4d:216:3eff:fe23:7b9f peer fd42:a497:a975:8f4d:216:3eff:fe48:c4db permaddr ce09:8b09:b957::
inet 10.0.3.2 peer 10.0.3.1/32 scope global gre6-arch1
valid_lft forever preferred_lft forever
inet6 fe80::fe23:7b9f/64 scope link
valid_lft forever preferred_lft forever
...
如上,inet6 fe80::fe23:7b9f/64是arch1机上gre6-arch1的link local address
实验完成,删除配置，接口删掉后路由会自动删掉,所以不手工删除路由了
arch1
ip tunnel delete gre6-arch2
arch2
ip tunnel delete gre6-arch1
GRE/IP6GRE over IPv4/6是不加密的，下面我们来实验 GRE/IP6GRE over IPsecv4/v6
ipv4 over GRE over IPsecv4
参照 ipv4/ipv6 over GRE over IPsecv4
ipv6 over GRE over IPsecv4
参照 ipv4/ipv6 over GRE over IPsecv4
ipv4/ipv6 over GRE over IPsecv4
安装strongswan,tcpdump,后者用于抓包分析
pacman -S strongswan tcpdump
把宿主机上的pki基础设置文件传入lxc容器
lxc file push arch1Key.pem arch1/etc/swanctl/private/
lxc file push arch1Cert.pem arch1/etc/swanctl/x509/
lxc file push strongswanCert.pem arch1/etc/swanctl/x509ca/
lxc file push arch2Key.pem arch2/etc/swanctl/private/
lxc file push arch2Cert.pem arch2/etc/swanctl/x509/
lxc file push strongswanCert.pem arch2/etc/swanctl/x509ca/
由于需要在同一个子网上,指10.165.103.0/24,调试IPsec，需要禁用bypass-lan插件
vi /etc/strongswan.d/charon/bypass-lan.conf
bypass-lan {
load = no
}
arch1 /etc/swanctl/swanctl.conf
include conf.d/*.conf
connections {
gw-gw {
local_addrs = 10.165.103.239
remote_addrs = 10.165.103.163
local {
auth = pubkey
certs = arch1Cert.pem
id = arch1.strongswan.org
}
remote {
auth = pubkey
id = arch2.strongswan.org
}
children {
gre {
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
mode = transport
esp_proposals = aes128gcm128-x25519
}
}
version = 2
proposals = aes128-sha256-x25519
}
}
arch2 /etc/swanctl/swanctl.conf
include conf.d/*.conf
connections {
gw-gw {
local_addrs = 10.165.103.163
remote_addrs = 10.165.103.239
local {
auth = pubkey
certs = arch2Cert.pem
id = arch2.strongswan.org
}
remote {
auth = pubkey
id = arch1.strongswan.org
}
children {
gre {
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
mode = transport
esp_proposals = aes128gcm128-x25519
}
}
version = 2
proposals = aes128-sha256-x25519
}
}
启动服务
systemctl enable strongswan
systemctl start strongswan
加载IPsec之前测试
arch1
ping -c 1 -I 192.168.101.1 192.168.102.1
ping -c 1 -I fd00:0:0:1::1 fd00:0:0:2::1
arch1 tcpdump -n -i eth0 not port ssh and not port domain抓包
18:07:17.946445 IP 10.165.103.239 > 10.165.103.163: GREv0, key=0x2a, length 92: IP 192.168.101.1 > 192.168.102.1: ICMP echo request, id 6938, seq 1, length 64
18:07:17.946484 IP 10.165.103.163 > 10.165.103.239: GREv0, key=0x2a, length 92: IP 192.168.102.1 > 192.168.101.1: ICMP echo reply, id 6938, seq 1, length 64
18:07:17.946445 IP 10.165.103.239 > 10.165.103.163: GREv0, key=0x2a, length 92: IP 192.168.101.1 > 192.168.102.1: ICMP echo request, id 6938, seq 1, length 64
18:07:17.946484 IP 10.165.103.163 > 10.165.103.239: GREv0, key=0x2a, length 92: IP 192.168.102.1 > 192.168.101.1: ICMP echo reply, id 6938, seq 1, length 64
如上显示，走的是GRE，并没有加载IPsec
加载IPsec
swanctl --initiate --child gre
加载IPsec之后测试
arch1
ping -c 1 -I 192.168.101.1 192.168.102.1
ping -c 1 -I fd00:0:0:1::1 fd00:0:0:2::1
arch1 tcpdump显示
18:05:49.927884 IP 10.165.103.239 > 10.165.103.163: ESP(spi=0xc3f11d1d,seq=0x2), length 128
18:05:49.928016 IP 10.165.103.163 > 10.165.103.239: ESP(spi=0xcc341cbe,seq=0x2), length 128
18:48:38.443081 IP 10.165.103.239 > 10.165.103.163: ESP(spi=0xc0f4780d,seq=0x2), length 148
18:48:38.443175 IP 10.165.103.163 > 10.165.103.239: ESP(spi=0xc143612a,seq=0x2), length 148
说明已经启用IPsec了 结合上面两个例子即可，一个GRE通道可以同时承担IPv4和IPv6流量，不冲突
ipv4 over IP6GRE over IPsecv6
参考 ipv4/ipv6 over IP6GRE over IPsecv6
ipv6 over IP6GRE over IPsecv6
参考 ipv4/ipv6 over IP6GRE over IPsecv6
ipv4/ipv6 over IP6GRE over IPsecv6
参考 ipv4/ipv6 over IP6GRE over IPv6 , 设置好IP6GRE
arch1 /etc/swanctl/swanctl.conf
include conf.d/*.conf
connections {
gw-gw {
local_addrs = fd42:a497:a975:8f4d:216:3eff:fe48:c4db
remote_addrs = fd42:a497:a975:8f4d:216:3eff:fe23:7b9f
local {
auth = pubkey
certs = arch1Cert.pem
id = arch1.strongswan.org
}
remote {
auth = pubkey
id = arch2.strongswan.org
}
children {
gre {
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
mode = transport
esp_proposals = aes128gcm128-x25519
}
}
version = 2
proposals = aes128-sha256-x25519
}
}
arch2 /etc/swanctl/swanctl.conf
include conf.d/*.conf
connections {
gw-gw {
local_addrs = fd42:a497:a975:8f4d:216:3eff:fe23:7b9f
remote_addrs = fd42:a497:a975:8f4d:216:3eff:fe48:c4db
local {
auth = pubkey
certs = arch2Cert.pem
id = arch2.strongswan.org
}
remote {
auth = pubkey
id = arch1.strongswan.org
}
children {
gre {
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
mode = transport
esp_proposals = aes128gcm128-x25519
}
}
version = 2
proposals = aes128-sha256-x25519
}
}
启动IPsecv6 之前
arch1上 tcpdump ping -c 1 -I 192.168.101.1 192.168.102.1
19:15:02.361347 IP6 fd42:a497:a975:8f4d:216:3eff:fe48:c4db > fd42:a497:a975:8f4d:216:3eff:fe23:7b9f: DSTOPT GREv0, length 88: IP 192.168.101.1 > 192.168.102.1: ICMP echo request, id 62686, seq 1, length 64
19:15:02.361390 IP6 fd42:a497:a975:8f4d:216:3eff:fe23:7b9f > fd42:a497:a975:8f4d:216:3eff:fe48:c4db: DSTOPT GREv0, length 88: IP 192.168.102.1 > 192.168.101.1: ICMP echo reply, id 62686, seq 1, length 64
arch1上 tcpdump ping -c 1 -I fd00:0:0:1::1 fd00:0:0:2::1
19:21:25.361267 IP6 fd42:a497:a975:8f4d:216:3eff:fe48:c4db > fd42:a497:a975:8f4d:216:3eff:fe23:7b9f: DSTOPT GREv0, length 108: IP6 fd00:0:0:1::1 > fd00:0:0:2::1: ICMP6, echo request, id 20892, seq 1, length 64
19:21:25.361303 IP6 fd42:a497:a975:8f4d:216:3eff:fe23:7b9f > fd42:a497:a975:8f4d:216:3eff:fe48:c4db: DSTOPT GREv0, length 108: IP6 fd00:0:0:2::1 > fd00:0:0:1::1: ICMP6, echo reply, id 20892, seq 1, length 64
启动IPsecv6
swanctl --initiate --child gre
启动IPsecv6 之后
arch1上 tcpdump ping -c 1 -I 192.168.101.1 192.168.102.1
19:17:23.152844 IP6 fd42:a497:a975:8f4d:216:3eff:fe48:c4db > fd42:a497:a975:8f4d:216:3eff:fe23:7b9f: DSTOPT ESP(spi=0xc3ae1e5c,seq=0x1), length 124
19:17:23.153003 IP6 fd42:a497:a975:8f4d:216:3eff:fe23:7b9f > fd42:a497:a975:8f4d:216:3eff:fe48:c4db: DSTOPT ESP(spi=0xc18d7056,seq=0x1), length 124
arch1上 tcpdump ping -c 1 -I fd00:0:0:1::1 fd00:0:0:2::1
19:26:48.491735 IP6 fd42:a497:a975:8f4d:216:3eff:fe48:c4db > fd42:a497:a975:8f4d:216:3eff:fe23:7b9f: DSTOPT ESP(spi=0xc4550724,seq=0x2), length 144
19:26:48.491810 IP6 fd42:a497:a975:8f4d:216:3eff:fe23:7b9f > fd42:a497:a975:8f4d:216:3eff:fe48:c4db: DSTOPT ESP(spi=0xc642f708,seq=0x2), length 144
说明流量走IPsec成功
总结
- GRE 有一定overhead,会使MTU减小几个字节
- GRE之上可以跑各种协议，这里只跑了三层协议,没有实验二层协议
更多的研究方向
- modprobe ip_gre后，会多出两个默认interface gre0@NONE 和 gretap0@NONE ，如何去掉？
- gre-arch1@NONE，这个@NONE是什么?
- modprobe ip6_gre后，会多出两个默认interface ip6gre0@NONE 和 ip6tnl0@NONE ，如何去掉？
- 需要手工启动IPsec，是否可以自动启动，防止流量泄漏为明文?
- 如何固化gre配置
- 实验不使用GRE，使用VTI或XFRM设备，能否进行三层转发
下一步
将GRE over IPsec用于实际的场景
参考
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/configuring-ip-tunnels_configuring-and-managing-networking
- https://blog.tianjinkun.com/post/113.html
- https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.tunnel.gre.html
- https://docs.strongswan.org/docs/5.9/features/routeBasedVpn.html#_gre_tunnels