Commit 4ac65362 authored by Matous Holinka's avatar Matous Holinka

duplicate removed

parent 77ad6899
Birdlab je hypervisor, který vytváří on-demand testovací prostředí pro Birda.
Podle definičního souboru vytvoří a spustí sadu virtuálních strojů, spojí je do
sítí podle definice a nastaví příslušným rozhraním IP adresy.
Celé prostředí sídlí v ROOT=/var/lib/virt, definice prostředí v $ROOT/etc/virtlist.
S ohledem na malý počet členů týmu návrh systému neřeší kolize mezi jednotlivými uživateli.
Ukázková konfigurace:
####################################################################
# Tento host slouží k instalaci a nastavování image OpenBSD 5.8
Host openbsd58-install
Image openbsd58
Memory 512M
VNC 52 # Je přístupný na IP adrese hypervizora na VNC portu 52
Type openbsd
# Tento host je konfigurační template pro vytváření dalších hostů s OpenBSD 5.8
Host openbsd58
Image openbsd58
Memory 512M
Type openbsd
Cow yes # Všechny změny na disku zapisuj do paměti a po ukončení zahoď
Host freebsd10-install
Image freebsd10
Memory 512M
VNC 51
Type freebsd
Host freebsd10
Image freebsd10
Memory 512M
Type freebsd
Cow yes
Host debian8-install
Image debian8
Memory 512M
VNC 11
Type linux
Host debian8
Image debian8
Memory 512M
Type linux
Cow yes
Host debian5-install
Image debian5
Memory 512M
VNC 12
Type linux
Host debian5
Image debian5
Memory 512M
Type linux
Cow yes
# Naklonuj konfiguraci hosta openbsd58 (3x) a vytvoř tak hosty ob1, ob2, ob3
Copy openbsd58 ob1 ob2 ob3
Copy freebsd10 fb1 fb2 fb3
Copy debian8 d1 d2 d3 d4
# Naklonuj hosta debian8 a vytvoř minidebian; uprav nastavení klonu
Copy debian8 minidebian
Host minidebian
Memory 64M
# Vytvoř 6 dalších debianů s 64M paměti
Copy minidebian md1 md2 md3 md4 md5 md6
# Vytvoř síťový segment, připoj na něj uvedené hosty a nastav jim příslušné IP adresy
Net md1 192.168.1.1/24 md2 192.168.1.2/24 md3 192.168.1.3/24 md4 192.168.1.4/24 md5 192.168.1.5/24 md6 192.168.1.6/24
# Vytvoř p2p linky mezi uvedenými dvojicemi hostů
Link fb1 192.168.55.64/32 d1 192.168.55.71/32
Link ob1 192.168.55.77/32 d1 192.168.55.33/32
Link ob1 192.168.55.17/32 fb1 192.168.55.241/32
Link d2 192.168.55.15/32 d3 192.168.55.117/32
Link d3 192.168.93.15/32 d4 192.168.15.93/32
Net fb1 192.168.44.1/24 fb2 192.168.44.2/24 fb3 192.168.44.3/24
Net d1 192.168.66.1/24 d2 192.168.66.2/24 d3 192.168.66.3/24
Net ob1 192.168.77.1/24 ob2 192.168.77.2/24 ob3 192.168.77.3/24
####################### Konec konfigurace ###########################
Manipulaci s virtuály zajišťuje příkaz vctl.
vctl prepare: vytvoř potřebná rozhraní, spusť potřebné daemony
vctl status: vypiš stav všech virtuálů
vctl cleanup: zruš rozhraní a vypni daemony, ukliď po sobě; nevypíná virtuály
vctl start <host> [args...]: nastartuj virtuál <host>; argumenty navíc se předají přímo Qemu.
vctl ssh <host>: připoj se na terminál <host>ovi
vctl mon <host>: otevři monitorovací socket Qemu
vctl stop <host>: vypni virtuál <host>; pokouší se o to několika způsoby, poslední je sigkill.
Virtuály běží v Qemu a jejich síťová rozhraní jsou připojená do openVswitche
(s tím se dá povídat přes ovs-vsctl, základní příkaz je ovs-vsctl show).
Linky a sítě jsou vytvořené jako jednotlivé VLANy v OVS, tag se generuje hashovací funkcí z definice sítě.
Každý virtuál má k dispozici /mnt/nfs, kam se namontuje $ROOT/nfs. Při startu se spustí /mnt/nfs/rc.local.
V základu se tímto skriptem nastaví sítě. Adresář $ROOT/nfs/net obsahuje další
důležité soubory definující nastavení sítě v závislosti na MAC adrese
virtuálního rozhraní.
Je-li tedy třeba po startu také přeložit Birda a spustit jej se správným
configem, $ROOT/nfs/rc.local je správné místo pro zavolání takové věci.
TODO: IPv6
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
ID=$2
if [ -z "$NAME" ] || [ -z "$ID" ]; then
echo "Usage: $0 NAME ID"
exit 2
fi
socat UNIX-CONNECT:$ROOT/run/$NAME.sock - >/dev/null <<EOF
netdev_add type=tap,id=$ID,ifname=$ID,script=no
device_add e1000,netdev=$ID
EOF
#!/bin/bash
. $(dirname $0)/virt-lib
[ -f $ROOT/run/dhcpd.pid ] && kill $(cat $ROOT/run/dhcpd.pid)
iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
iptables -D FORWARD -i eth0 -o $VIRTCTRL -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -D FORWARD -i eth0 -o $VIRTCTRL -j ACCEPT
iptables -D FORWARD -i $VIRTCTRL -o eth0 -j ACCEPT
iptables -D FORWARD -o eth0 -j REJECT
ovs-vsctl del-br $VIRTBR
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NETLIST=$ROOT/run/networks
NET=$1
if [ -z "$NET" ] || [ -z "$2" ]; then
echo "Usage: $0 NET hosts"
exit 2
fi
shift
if grep -q $NET $NETLIST; then
echo "Network $NET exists"
exit 2
fi
MAXID=$(sort -k1,1nr $NETLIST | head -n1|cut -f1 -d' ')
MAXID=$((MAXID+1))
echo "$MAXID $NET" >> $NETLIST
i=0
idprefix=auto-$MAXID
for H in "$@"; do
$ROOT/bin/addnic $H $idprefix-$i
ip link set $idprefix-$i up
ovs-vsctl add-port $VIRTBR $idprefix-$i tag=$MAXID
i=$((i+1))
done
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
ID=$2
if [ -z "$NAME" ] || [ -z "$ID" ]; then
echo "Usage: $0 NAME ID"
exit 2
fi
socat UNIX-CONNECT:$ROOT/run/$NAME.sock - >/dev/null <<EOF
netdev_del $ID
device_del e1000,netdev=$ID
EOF
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
if [ -z "$NAME" ]; then
echo "Usage: $0 NAME"
exit 2
fi
$SOCAT READLINE,history=$ROOT/run/$NAME.mon-history UNIX-CONNECT:$ROOT/run/$NAME.sock
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
set -e
EVENT=$1
shift
idtoip() {
echo 10.255.$(($1 / 254)).$(($1 % 254))
}
idtolink() {
echo 10.254.$(($1 / 254)).$(($1 % 254))
}
if ! brctl show | grep -q br-mpls; then brctl addbr br-mpls; ip link set br-mpls up; fi
case $EVENT in
add-rr|add-p|add-pe)
T=${EVENT##add-}
id=$1
ip=$(idtoip $id)
vctl start -t mpls-auto $T-$id
gethost mpls-auto-$T-$id
vctl serial-mikrotik-fix.pl mpls-auto-$T-$id
vctl ssh mpls-auto-$T-$id -t <<EOF
/interface bridge add name=lo
/ip address add address=$ip/32 interface=lo
/routing ospf instance set distribute-default=never redistribute-connected=as-type-1 router-id=$ip 0
/routing ospf network add network=$ip/32 area=backbone
EOF
if [ "$EVENT" != "add-rr" ]; then
echo "/mpls ldp set enabled=yes lsr-id=$ip transport-address=$ip" | vctl ssh mpls-auto-$T-$id -t
fi
;;
add-c)
id=$1
vpn=$2
pe=$3
ip=$(idtoip $id)
peip=$(idtoip $pe)
linkip=$(idtolink $id)
linkpeip=$(idtolink $pe)
vctl start -t mpls-auto c-$id
gethost mpls-auto-c-$id
cip=$HOSTIP4
gethost mpls-auto-pe-$pe
cpeip=$HOSTIP4
vctl serial-mikrotik-fix.pl mpls-auto-c-$id
vctl ssh mpls-auto-c-$id -t <<EOF
/interface bridge add name=lo
/ip address add address=$ip/32 interface=lo
/interface gre add remote-address=$cpeip name=pe-$pe
/ip address add address=$linkip/32 network=$linkpeip interface=pe-$pe
/routing ospf instance set distribute-default=never redistribute-connected=as-type-1 router-id=$ip 0
/routing ospf network add network=$ip/32 area=backbone
/routing ospf network add network=$linkpeip/32 area=backbone
EOF
vctl ssh mpls-auto-pe-$pe -t <<EOF
/interface gre add remote-address=$cip name=c-$id
/ip address add address=$linkpeip/32 network=$linkip interface=c-$id
/ip route vrf add routing-mark=$vpn interfaces=c-$id route-distinguisher=$vpn import-route-targets=$vpn export-route-targets=$vpn
/routing bgp instance vrf add instance=default routing-mark=$vpn redistribute-connected=yes redistribute-ospf=yes
/routing ospf instance add name=c-$id routing-table=$vpn redistribute-bgp=as-type-1
/routing ospf area add instance=c-$id name=c-$id
/routing ospf network add network=$linkip/32 area=c-$id
EOF
;;
add-link)
from=$1
to=$2
fip=$(idtolink ${from##*-})
tip=$(idtolink ${to##*-})
gethost mpls-auto-$from
cfip=$HOSTIP4
gethost mpls-auto-$to
ctip=$HOSTIP4
vctl ssh mpls-auto-$from -t <<EOF
/interface gre add remote-address=$ctip name=$to
/ip address add address=$fip/32 network=$tip interface=$to
/routing ospf network add network=$tip/32 area=backbone
EOF
vctl ssh mpls-auto-$to -t <<EOF
/interface gre add remote-address=$cfip name=$from
/ip address add address=$tip/32 network=$fip interface=$from
/routing ospf network add network=$fip/32 area=backbone
EOF
case "$from$to" in
*rr*) :
;;
*)
echo "/mpls ldp interface add interface=$to" | vctl ssh mpls-auto-$from -t
echo "/mpls ldp interface add interface=$from" | vctl ssh mpls-auto-$to -t
;;
esac
;;
bgp-rr)
pe=$1
rr=$2
peip=$(idtoip $pe)
rrip=$(idtoip $rr)
echo "/routing bgp peer add remote-address=$rrip remote-as=65530 address-families=vpnv4 update-source=lo" | vctl ssh mpls-auto-pe-$pe -t
echo "/routing bgp peer add remote-address=$peip remote-as=65530 address-families=vpnv4 update-source=lo route-reflect=yes" | vctl ssh mpls-auto-rr-$rr -t
;;
bgp-link)
from=$1
to=$2
fip=$(idtoip ${from})
tip=$(idtoip ${to})
echo "/routing bgp peer add remote-address=$tip remote-as=65530 address-families=vpnv4 update-source=lo" | vctl ssh mpls-auto-rr-$from -t
echo "/routing bgp peer add remote-address=$fip remote-as=65530 address-families=vpnv4 update-source=lo" | vctl ssh mpls-auto-rr-$to -t
;;
*)
echo "Commands: add-rr ID, add-p ID, add-pe ID, add-c ID VPN PE, add-link FROM TO, bgp-rr PE RR"
exit 2
;;
esac
#!/usr/bin/env python
import pypureomapi
import sys
keyname = "dhcp_key"
#secret = "6Qsf2UG4cvSli+Fup3bHI/4lCGO4i3SO6hl6WWBr042vUHktQ/1KiQwI 3MQrNrGs7c7EMDr/fho2Fzr3y1EP/Q=="
secret = "3MQrNrGs7c7EMDr/fho2Fzr3y1EP/Q=="
server = "127.0.0.1"
port = 7911
try:
oma = pypureomapi.Omapi(server, port, keyname, secret)
except pypureomapi.OmapiError, err:
print "OMAPI error: %s" % (err,)
sys.exit(1)
if sys.argv[1] == "add":
oma.add_host_supersede_name(sys.argv[4], sys.argv[3], sys.argv[2])
elif sys.argv[1] == "del":
oma.del_host(sys.argv[2])
else:
print("Usage: omapi (add MAC IP|del MAC)")
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
HOST=$1
if ! gethost $HOST; then
echo "Unknown host: $HOST"
exit 2
fi
case ${HOSTTYPE^^} in
FREEBSD)
INSTALLPKG="pkg install rsync autotools bison flex readline gmake"
FSTAB="192.168.192.1:/var/lib/virt/nfs /mnt/nfs nfs rw 0 0"
DHCPFIX="echo 'request broadcast-address, domain-name, domain-name-servers, domain-search, host-name, routers;' >> /etc/dhclient.conf"
;;
OPENBSD) ;&
NETBSD)
FSTAB="192.168.192.1:/var/lib/virt/nfs /mnt/nfs nfs rw 0 0"
;;
LINUX)
FSTAB="192.168.192.1:/var/lib/virt/nfs /mnt/nfs nfs defaults,intr,_netdev 0 0"
;;
*)
echo "Unknown type of host: $TYPE"
exit 2
;;
esac
AUTH=`mktemp`
cat >$AUTH <<EOF
mkdir -p /root/.ssh
cat >/root/.ssh/authorized_keys <<EOC
EOF
cat /home/birdlab/.ssh/authorized_keys >> $AUTH
cat /root/.ssh/virt-rsa.pub >>$AUTH
echo EOC >>$AUTH
ssh root@$HOSTIP4 bash <$AUTH
SCRIPT=`mktemp`
cat >$SCRIPT <<EOF
if [ ! -x /bin/bash ]; then
ln -s \$(command -v bash) /bin/bash
fi
rm -f /etc/udev/rules.d/70-persistent-net.rules
if egrep -q '/mnt/nfs[[:space:]]+nfs[[:space:]]+' /etc/fstab; then
( egrep -v '/mnt/nfs[[:space:]]+nfs[[:space:]]+' /etc/fstab ; echo "$FSTAB") > /etc/fstab.new
cp /etc/fstab.new /etc/fstab
mkdir -p /mnt/nfs
mount -o remount /mnt/nfs
else
echo "$FSTAB" >> /etc/fstab
mkdir -p /mnt/nfs
mount /mnt/nfs
fi
$DHCPFIX
cat >/etc/rc.local <<EOC
#!/bin/bash
until [ -x /mnt/nfs/rc.local ]; do
sleep 1
done
/mnt/nfs/rc.local > /tmp/rc-local-log 2>/tmp/rc-local-err
EOC
chmod +x /etc/rc.local
$INSTALLPKG
EOF
ssh root@$HOSTIP4 bash <$SCRIPT
rm $SCRIPT
rm $AUTH
#!/bin/bash
. $(dirname $0)/virt-lib
vsctl add-br $VIRTBR
ip link set $VIRTBR up
ip addr add 192.168.10.1/28 dev $VIRTBR
ip link add $VIRTCTRL type veth peer name $VIRTUPLINK
ip link set $VIRTCTRL up
ip link set $VIRTUPLINK up
ip addr add 192.168.192.1/18 dev $VIRTCTRL
ip -6 addr add 2001:1488:A001::1/64 dev $VIRTCTRL
vsctl add-port $VIRTBR $VIRTUPLINK tag=1
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -o $VIRTCTRL -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth0 -o $VIRTCTRL -j ACCEPT
iptables -A FORWARD -i $VIRTCTRL -o eth0 -j ACCEPT
iptables -A FORWARD -o eth0 -j REJECT
dhcpd -cf $ROOT/etc/dhcpd.conf -pf $ROOT/run/dhcpd.pid $VIRTCTRL
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
[ -f $ROOT/run/dhcpd.pid ] && kill $(cat $ROOT/run/dhcpd.pid)
dhcpd -cf $ROOT/etc/dhcpd.conf -pf $ROOT/run/dhcpd.pid $VIRTCTRL
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
if [ -z "$NAME" ]; then
echo -e "Usage: $0 name [custom options for qemu]\n or $0 -m list of names"
exit 2
fi
vctl stop "$@"
vctl start "$@"
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
if [ -z "$NAME" ]; then
echo "Usage: $0 NAME"
exit 2
fi
minicom -D unix\#$ROOT/run/$1.serial
#!/usr/bin/perl -CL
use common::sense;
use Data::Dumper;
use IO::Socket::UNIX;
my $client = IO::Socket::UNIX->new( Type => SOCK_STREAM(), Peer => "/var/lib/virt/run/$ARGV[0].serial");
my $data;
my $debug = 0;
sub D { say @_ if $debug; }
sub S { say "\rmikrotik init ", @_; }
$/ = "";
my $state = "login";
say "";
S "started";
D "init client";
syswrite $client, "\r";
while (1) {
D "";
sleep 1;
sysread $client, $data, 65536;
$data =~ s/[^ -~]/?/g;
D Dumper \$data;
if ($data =~ /MikroTik Login: $/) {
S "sending login";
syswrite $client, "root\r";
$state = "login";
next;
}
if ($data =~ /assword: $/) {
S "sending password";
syswrite $client, "root\r";
next;
}
if ($data =~ /tinue!/) {
S "continue boilerplate";
syswrite $client, "\r";
# $state = "export";
$state = "macreset";
next;
}
if ($data =~ /\[root\@MikroTik\] > $/ && $state eq "export") {
D "export";
syswrite $client, "export\r";
$state = "macreset";
next;
}
if ($data =~ /\[root\@MikroTik\] > $/ && $state eq "macreset") {
D "interface ethernet reset-mac-address numbers=0";
S "reset mac address";
syswrite $client, "interface ethernet reset-mac-address numbers=0\r";
$state = "dhcp";
next;
}
if ($data =~ /\[root\@MikroTik\] > $/ && $state eq "dhcp") {
D "ip dhcp-client renew numbers=0";
S "flush dhcp";
syswrite $client, "ip dhcp-client renew numbers=0\r";
$state = "quit";
next;
}
if ($data =~ /\[root\@MikroTik\] > $/ && $state eq "quit") {
D "quit";
syswrite $client, "quit\r";
sleep 1;
sysread $client, $data, 65536;
$data =~ s/[^ -~]/?/g;
D Dumper \$data;
S "done", " "x50;
exit 0;
}
if ($data =~ /\[root\@MikroTik\] > $/) {
D "Logged in, noop now";
syswrite $client, "\r";
$state = "macreset";
next;
}
D "NOP, strange input";
}
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
if [ -z "$NAME" ]; then
echo "Usage: $0 NAME"
exit 2
fi
shift
gethost $NAME
if [ "${#@}" == "0" ]; then
TERM=rxvt-256color ssh -i /root/.ssh/virt-rsa -t root@$HOSTIP4 bash
else
TERM=rxvt-256color ssh -i /root/.ssh/virt-rsa root@$HOSTIP4 "$@"
fi
#!/bin/bash
. $(dirname $(readlink -f $0))/virt-lib
NAME=$1
shift
usage() {
echo -e "Usage: $0 name [custom options for qemu]\n or $0 -m list of names\n or $0 -t name suffix [custom options for qemu]"
exit 2
}
check_stale() {
local NAME=$1
if [ -e $ROOT/run/$NAME.pid ]; then
echo -n "Host $NAME pidfile found"
if [ -d /proc/$(<$ROOT/run/$NAME.pid) ]; then
echo " running with PID $(<$ROOT/run/$NAME.pid)"
exit 1
else
echo " stale ... removing"
rm $ROOT/run/$NAME.pid
fi
fi
}
[ -z "$NAME" ] && usage
if [ "$NAME" = "-m" ]; then
set -e
for N in "$@"; do
echo Starting "$N"
$0 "$N"
done
exit 0
fi
if [ "$NAME" = "-t" ]; then
NAME=$1
SUFFIX=$2
([ -z "$SUFFIX" ] || [ -z "$NAME" ]) && usage
ID=0
if [ -e $ROOT/run/tmp-$NAME-$SUFFIX ]; then
echo "Host $NAME tmpfile found."
check_stale $NAME-$SUFFIX
rm $(readlink -f $ROOT/run/tmp-$NAME-$SUFFIX) $ROOT/run/tmp-$NAME-$SUFFIX
fi
while [ -e $ROOT/run/tmp-$ID ]; do
ID=$((ID+1))
done
HEXID=$(printf "%02x" $ID)
echo "$SUFFIX" > $ROOT/run/tmp-$ID
ln -s tmp-$ID $ROOT/run/tmp-$NAME-$SUFFIX
shift 2
else
check_stale $NAME
fi
NAME=$NAME${SUFFIX:+-$SUFFIX}
gethost $NAME
if [ "$?" != 0 ]; then
echo "Host $NAME not configured"
exit 2
fi
if [ "$1" = "--install" ]; then
INSTALL="-cdrom $2 -boot d"
shift 2
fi
RCGFILE=$ROOT/nfs/rc/rc-gen-$HOSTNAME
truncate -s0 $RCGFILE
declare -a HOSTNICVLAN
if [ -n "$HOSTNIC" ]; then
declare -a HOSTNIC=( $HOSTNIC )
for N in ${HOSTNIC[@]}; do
IFS=';' read NETNAME MAC TAG IP4 IP4PEER <<<"$N"
TAPID=$(mactotap $MAC)
HOSTNICDEV="$HOSTNICDEV -netdev tap,id=$TAPID,ifname=$TAPID,script=no -device e1000,netdev=$TAPID,mac=$MAC"
echo "N=\$(ifname \${IFBYMAC[$MAC]} $NETNAME)" >>$RCGFILE
echo "ifnameset $MAC \$N" >>$RCGFILE
if [ -n "$IP4PEER" ]; then
echo "ip4ptp \$N $IP4 $IP4PEER" >>$RCGFILE
else
echo "ip4 \$N $IP4" >>$RCGFILE
fi
echo "ifup \$N" >>$RCGFILE
HOSTNICVLAN+=( $TAPID,$TAG )
done
fi
if [ -n "$HOSTVARS" ]; then
declare -a HOSTVARS=( $HOSTVARS )
for V in ${HOSTVARS[@]}; do
echo "export $V" >> $RCGFILE
done
declare -a HOSTVARNAMES
for V in ${HOSTVARS[@]}; do
HOSTVARNAMES+=( ${V%%=*} )
done
echo "declare -a VARNAMES=(${HOSTVARNAMES[@]})" >> $RCGFILE
fi
if [ -n "$HOSTDUMMY" ]; then
declare -a HOSTDUMMY=( $HOSTDUMMY )
for D in ${HOSTDUMMY[@]}; do
IFS=';' read DNAME IP4 IP6 <<<"$D"
echo "N=\$(dummy $DNAME)" >>$RCGFILE
echo "INTERFACES+=(\$N)" >>$RCGFILE
echo "ip4 \$N $IP4" >>$RCGFILE
echo "ip6 \$N $IP6" >>$RCGFILE
done
fi
if [ -n "$HOSTGRE" ]; then
declare -a HOSTGRE=( $HOSTGRE )
for G in ${HOSTGRE[@]}; do
IFS=';' read GRENAME PEER IP4 IP4PEER <<<"$G"
echo "N=\$(gre $GRENAME $HOSTIP4 ${CONF[$PEER;IP4]})" >>$RCGFILE
echo "INTERFACES+=(\$N)" >>$RCGFILE
echo "ip4ptp \$N $IP4 $IP4PEER" >>$RCGFILE
done
fi