How to : QEMU Networking

Well,
I have posted the original article here.

Copying it here for readers.
------------------------------------------------------
Hi All,
It has been ages since I have written anything over here. One of the main reason is that I have started posting mathematical post at my wordpress blog.

Anyways, Sameer and I were trying to get our hands dirty on Qemu networking. Although there is a fairly good description is given over here, I am trying to fill in here a few missing bits.

Okay, so here is the scenario we want to create.

On one machine ( say host-A ) we want to create many guest machines ( guestA1, guestA2 ) etc. Similarly, there is another machine ( host-B ) having many guest machines ( guestB1, guestB2 ). We want to make guestA* communicate with guestB*.

I am assuming that readers are aware of how to install a guest OS on a guest machine created by qemu.

First we need to configure a ipv4-over-ipv4 tunnel for the host machines.

--host-A configuration is as follows

ip : 172.27.16.1
netmask : 255.255.255.0
default gateway : 172.27.16.254

host-B configuration is
ip : 172.27.16.2

netmask and gateway is the same as above.

--In host-A ( as root ) do the following

root@hostA# ip tunnel add tunl1 mode ipip remote 172.27.16.2 dev eth0

-- this command essentially create one endpoint of the tunnel in host-A. It says that the other end of the tunnel lies at ip 172.27.16.2. Also, the tunnel device will use eth0 as its physical device.

root@hostA# ip link set tunl1 up

-- this command will bring up the tunnel interface

root@hostA#ifconfig tunl1 10.0.1.1 pointopoint 172.27.16.2

-- guestA network will be 10.0.1.0/24 network and tunnel device should be a part of it. This command gives the tunnel device an ip address and it says that other endpoint of the tunnel will be at 172.27.16.2

root@hostA# route

--this will show you the current configuration of routing table. There might be an entry in the routing table

172.27.16.2 * 255.255.255.255 U 0 0 tunl1

--you need to delete this entry using following command :

root@hostA# route del -net 172.27.16.2 netmask 255.255.255.255 dev tunl1

--Also you need to tell routing table that guestB network ( 10.0.2.0/24) should be routed through the tunnel

root@hostA# route add -net 10.0.2.0 netmask 255.255.255.0 dev tunl1

--You need to repeat the same procedure for hostB with suitable ips ( use 10.0.1.* instead of 10.0.2.* and vice versa , replace 172.27.16.1 with 172.27.16.2 and vice versa )

--By now, your tunnel should be working. To check, try

root@hostA# ping 10.0.2.1

root@hostB# ping 10.0.1.1

--If it does not ping, it might be the case that your firewall is coming into the way. If you are expert enough, modify your firewall rules with iptables. If not, lets first save your firewall rules

root@hostA# iptables-save | cat > iptablebackup.bak

--flush the firewall rules now

root@hostA# iptables -F
root@hostA# iptables -t nat -F

--Now, that firewall is completely disabled, the tunnel should work. (Of course, you need to disable your firewall at hostB as well! ) If it still does not work, then there might be some other problem or you might not have done things right.

-- you can restore your firewall ( later ) with

root@hostA# cat iptablebackup.bak | iptables-restore

--Okay, so the tunnel is working now!! Great news!! But what next??
Next we create a tapping device - a virtual network interface - which will be visible to the guest OSes.

root@hostA# tunctl -u username

-- provide a username here. This user will be able to use this tapping device while initializing qemu process. This command would have created a tapping device (tap0) in your system

--assign an ip address of guestA network to tapping device

root@hostA# ip addr add 10.0.1.2 dev tap0

-- bring the tapping device up

root@hostA# ip link set tap0 up

-- Also create a file /etc/qemu-ifup which can look like following :
#!/bin/bash
#echo "bringing up interface $1"

-- Now start the first guestOS

username@hostA$ qemu -net nic,macaddr=macaddr1 -net tap,ifname=tap0 -net socket,mcast=230.0.0.1:9999 ....

--second guestOS can be started with following
username@hostA$ qemu -net nic,macaddr=macaddr2 -net socket, mcast=230.0.0.1:9999 ...

-- multicast socket is needed for mchines in guestA network to talk to each other

--In the guestA1 do the following :

root@guestA1# ifconfig eth0 10.0.1.3 netmask 255.255.255.0
root@guestA1# route add default gw 10.0.1.2 dev eth0

-- Note that we have given the ip address of tapping device as the gateway fo guest OSes
-- second, third etc guest machines can be configured with different ip addresses for their eth0

-- Repeat this procedure on host-B as well with appropriate ip addresses.

-- Now, you should be able to ping machines from guestA network to guestB network and vice versa. However, this does not yet allow you to connect to host network. To do that, you need to enable ip forwardng

root@hostA# echo "1" > /proc/sys/net/ipv4/ip_forward

-- perform source NAT

root@hostA# iptables -t nat -I POSTROUTING 1 -o eth0 -j SNAT --to-source 172.27.16.1

--Now, from guestA network ( 10.0.1.0/24 ) you should be able to connect to ( 10.0.2.0/24 ) as well as host network ( 172.27.16.0/24 )

-- To delete the tunnel

root@hostA# ip link set tunl1 down
root@hostA# ip tunnel del tunl1

--To delte the tapping device

root@hostA# tunctl -d tap0

--Above mentioned setup is tested with host machines running fedora core 9 and guest machines running Arch Linux, Debian etc.

If you have any queries post it as comments, I will be willing to answer if I can.