Lab 3: OpenFlow & Mininet

實驗描述

OpenFlow 是目前在 SDN 中,負責處理控制層 (controller) 和資料層 (switch, router...etc devices) 的溝通協定。

本次實驗讓大家使用 Mininet 快速佈署模擬 SDN 的網路環境,並且簡單操作 OpenvSwitch 來操作 flow,

最後來對 POX controller programming來達到不同的需求。

實驗環境

理學大樓 1002 教室
Cloud-A01 ~ Cloud-D12
CPU AMD Phenom™ II X6 1065T Processor
Memory 8G
Disk spaces 500G、500G
O.S. Debian Wheezy

虛擬機環境
Virtual Machine
Location 821 機房伺服器
CPU vcpu*2
Memory 8G
Disk spaces 80G (QCOW2 Format)
O.S. Debian Jessie
理學大樓 821 機房
CSIE-Cloud01 ~ CSIE-Cloud06
CPU AMD Opteron™ Processor 6128 * 2
(total 16 cpu cores)
Memory 64G
Disk spaces 500G, 500G, 1T
O.S. Debian wheezy
CSIE-Cloud07, CSIE-Cloud08
CPU AMD Opteron™ Processor 6234 * 2
(total 24 cpu cores)
Memory 128G, 160G
Disk spaces 500G, 500G, 1T
O.S. Debian wheezy

環境參數

為了避免輸入錯誤以及方便區分 IP 使用環境,我們使用下列幾個按鈕來快速取代各個不同 IP。

您的機器參數:

Username : [USERNAME]

Code number : [code-num]

Host IP : [Host-IP]

Gateway : [Gateway-IP]


安裝實作 (Mininet)

Login Server

若您使用 MS Windows, 請參閱說明 Login Server From Windows.

Open a terminal emulator and then type the following commands.

ssh [USERNAME]@cloud.cs.nchu.edu.tw -X -p [PortNUM]

安裝 Mininet 所需套件及準備工作

  1. 安裝需求套件和設定

    sudo cp ~/.Xauthority /root
    sudo aptitude update 
    sudo aptitude install python-setuptools git xterm
    	The following NEW packages will be installed:
      git git-man{a} liberror-perl{a} python-setuptools rsync{a} 
    0 packages upgraded, 5 newly installed, 0 to remove and 198 not upgraded.
    Need to get 5,074 kB of archives. After unpacking 24.9 MB will be used.
    Do you want to continue? [Y/n/?]
    	
  2. 將需要的檔案從 github 上抓下來

     git clone git://github.com/mininet/mininet
    	Cloning into 'mininet'...
    remote: Counting objects: 6722, done.
    remote: Compressing objects: 100% (2542/2542), done.
    remote: Total 6722 (delta 4098), reused 6670 (delta 4058)
    Receiving objects: 100% (6722/6722), 2.26 MiB | 239.00 KiB/s, done.
    Resolving deltas: 100% (4098/4098), done.
    Checking connectivity... done.
    	
    cd mininet

    由於最近剛改版,但我們還是使用上一個穩定的版本,切換到2.1.0

    git checkout -b 2.1.0 2.1.0 
    	Switched to a new branch '2.1.0'
    	
  3. 開始編譯和安裝 mininet

    gcc -o mnexec mnexec.c
    sudo mv mnexec /usr/local/bin/
    sudo python setup.py install
    	...
    
    Adding setuptools 5.5.1 to easy-install.pth file
    Installing easy_install script to /usr/local/bin
    Installing easy_install-2.7 script to /usr/local/bin
    
    Using /usr/lib/python2.7/dist-packages
    Finished processing dependencies for mininet==2.1.0
    	

    檢查有無錯誤

    echo $?
    	0	
    	
  4. 測試 mininet 版本

    sudo mn --version
    	2.1.0	
    	

建立實驗環境

    用 command line 啟動 mininet

    建立 1 台 Switch 和 3 台 Host 的網路拓樸環境,並等待遠端接上 OpenFlow Controller

    sudo mn --topo single,3 --mac --controller remote
    	*** Creating network
    *** Adding controller
    Unable to contact the remote controller at 127.0.0.1:6633
    *** Adding hosts:
    h1 h2 h3 
    *** Adding switches:
    s1 
    *** Adding links:
    (h1, s1) (h2, s1) (h3, s1) 
    *** Configuring hosts
    h1 h2 h3 
    *** Starting controller
    *** Starting 1 switches
    s1 
    *** Starting CLI:
    mininet>
    	

    檢查網路拓樸連線狀況

    net
    	h1 h1-eth0:s1-eth1	
    h2 h2-eth0:s1-eth2
    h3 h3-eth0:s1-eth3
    s1 lo:  s1-eth1:h1-eth0 s1-eth2:h2-eth0 s1-eth3:h3-eth0
    c0
    	

    觀察其中一台 host 的網路狀況,我們選擇 h1

    h1 ifconfig
    	h1-eth0   Link encap:Ethernet  HWaddr 00:00:00:00:00:01  
              inet addr:10.0.0.1  Bcast:10.255.255.255  Mask:255.0.0.0
              inet6 addr: fe80::200:ff:fe00:1/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:38 errors:0 dropped:0 overruns:0 frame:0
              TX packets:68 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000 
              RX bytes:2356 (2.3 KiB)  TX bytes:4048 (3.9 KiB)
    ...
    	

使用指令透過 switch 中的 rule 來觀察 Openflow

  1. 實作 switch 正常行為
  2. 讓底下的 host 互相 ping

    pingall
    	*** Ping: testing ping reachability
    h1 -> X X 
    h2 -> X X 
    h3 -> X X 
    *** Results: 100% dropped (0/6 received)
    	

    查看 switch 中的 flow

    sh ovs-ofctl dump-flows s1
    	NXST_FLOW reply (xid=0x4):
    	

    插入 flow 讓 switch 正常運作

    sh ovs-ofctl add-flow s1 "priority=0,action=normal"
    	

    這時候再觀察 switch 中的 flow

    sh ovs-ofctl dump-flows s1
    	NXST_FLOW reply (xid=0x4):
     cookie=0x0, duration=35.574s, table=0, n_packets=0, n_bytes=0, idle_age=35, priority=0 actions=NORMAL
    	

    讓底下的 host 互相 ping,此時 switch 已經正常運作

    pingall
    	*** Ping: testing ping reachability
    h1 -> h2 h3 
    h2 -> h1 h3 
    h3 -> h1 h2 
    *** Results: 0% dropped (6/6 received)
    	

    清除 switch 中的 flow

    sh ovs-ofctl del-flows s1
    	

    為了確保 switch 中的 flow 為空,我們再 dump 一次

    sh ovs-ofctl dump-flows s1
    	NXST_FLOW reply (xid=0x4):
    	
  3. IP traffic control
  4. 首先讓 switch 能正常運作,因此再重新加入 flow

    sh ovs-ofctl add-flow s1 "priority=0,action=normal"
    	

    再增加一條新的 flow 到 switch 中,讓所有目標 IP 為 10.0.0.1 的封包都作 drop 行為

    sh ovs-ofctl add-flow s1 "priority=100,eth_type=0x800,ip_dst=10.0.0.1,action=drop"
    	

    此時 switch 中的 flow 規則如下

    sh ovs-ofctl dump-flows s1
    	NXST_FLOW reply (xid=0x4):
     cookie=0x0, duration=8.880s, table=0, n_packets=0, n_bytes=0, idle_age=8, priority=0 actions=NORMAL
     cookie=0x0, duration=3.480s, table=0, n_packets=0, n_bytes=0, idle_age=3, priority=100,ip,nw_dst=10.0.0.1 actions=drop
    	

    作 pingall 的動作,可觀察到所有往 h1 的 IP 封包都傳不過去

    pingall
    	*** Ping: testing ping reachability
    h1 -> X X 
    h2 -> X h3 
    h3 -> X h2 
    *** Results: 66% dropped (2/6 received)
    	

    清除 switch 中的 flow

    sh ovs-ofctl del-flows s1
    	
  5. ARP traffic control
  6. 首先讓 switch 能正常運作,因此再重新加入 flow

    sh ovs-ofctl add-flow s1 "priority=0,action=normal"
    	

    這次新增一條 flow 到 switch 中,讓所有目標 MAC address 為 00:00:00:00:00:02 的封包都作 drop 行為

    sh ovs-ofctl add-flow s1 "priority=100,eth_type=0x806,dl_dst=00:00:00:00:00:02,action=drop"
    	

    作 pingall 的動作,可觀察到所有往 h2 的 ARP 封包都傳不過去

    pingall
    	*** Ping: testing ping reachability
    h1 -> X h3 
    h2 -> X X 
    h3 -> h1 X 
    *** Results: 66% dropped (2/6 received)
    	

    關閉 mininet

    exit
    	*** Stopping 1 switches
    s1 ...
    *** Stopping 3 hosts
    h1 h2 h3 
    *** Stopping 1 controllers
    c0 
    *** Done
    completed in 5566.056 seconds
    	

POX controller

在上一段落中,我們使用 mininet 來建立我們需要的 openflow 實驗環境,並且用指令來對 switch 作安裝 flow 的動作。

這段落我們介紹如何透過 SDN controller 達到同樣的效果,並作簡單的操作。

打開新的 terminal 並且登入到所分配的機器

ssh [USERNAME]@cloud.cs.nchu.edu.tw -X -p [PortNUM]
  1. 安裝POX

    換回新開的 terminal,我們從學校伺服器抓取檔案,以節省頻寬

    wget http://www.cs.nchu.edu.tw/~snmlab/CloudMgnt201409/pox.tar

    解壓縮

    tar xvf pox.tar

    在校外可以直接上官網抓

    git clone http://github.com/noxrepo/pox
    cd ~/pox
  2. 測試 POX 範例

    執行 layer 2 learning switch

    ./pox.py forwarding.l2_learning
    	POX 0.2.0 (carp) / Copyright 2011-2013 James McCauley, et al.
    INFO:core:POX 0.2.0 (carp) is up.
    INFO:openflow.of_01:[00-00-00-00-00-01 1] connected
    	
  3. 我們還是使用 mininet 來當作實驗環境

    首先先切回 mininet 的 terminal ,並重新執行 mininet 環境

    sudo mn --topo single,3 --mac --controller remote
    	*** Creating network
    *** Adding controller
    the remote controller at 127.0.0.1:6633
    *** Adding hosts:
    h1 h2 h3 
    *** Adding switches:
    s1 
    *** Adding links:
    (h1, s1) (h2, s1) (h3, s1) 
    *** Configuring hosts
    h1 h2 h3 
    *** Starting controller
    *** Starting 1 switches
    s1 
    *** Starting CLI:
    	

    讓 host 間互 ping

    pingall
    	*** Ping: testing ping reachability
    h1 -> h2 h3 
    h2 -> h1 h3 
    h3 -> h1 h2 
    *** Results: 0% dropped (6/6 received)
    	

    查看 switch 中的 flow

    sh ovs-ofctl dump-flows s1
    	 cookie=0x0, duration=1.540s, table=0, n_packets=1, n_bytes=98, idle_timeout=10, hard_timeout=30, idle_age=1, priority=65535,icmp,in_port=1,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.1,nw_dst=10.0.0.2,nw_tos=0,icmp_type=8,icmp_code=0 actions=output:2
     cookie=0x0, duration=1.522s, table=0, n_packets=1, n_bytes=98, idle_timeout=10, hard_timeout=30, idle_age=1, priority=65535,icmp,in_port=3,vlan_tci=0x0000,dl_src=00:00:00:00:00:03,dl_dst=00:00:00:00:00:01,nw_src=10.0.0.3,nw_dst=10.0.0.1,nw_tos=0,icmp_type=0,icmp_code=0 actions=output:1
    ...
    	
  4. 修改簡單防火牆功能

    對 POX controller 的視窗按下 Crtl + C 暫時中斷 controller

    ^C
    	INFO:core:Going down...
    INFO:openflow.of_01:[00-00-00-00-00-01 3] disconnected
    INFO:core:Down.
    	

    我們用 l2_learning.py 來作修改,首先從同一個檔案複製成不同的檔名

    cd ~/pox/pox/forwarding/
    ls
    	hub.py        l2_flowvisor.py  l2_multi.py             l2_pairs.py
    __init__.py   l2_learning.py   l2_nx.py                l3_learning.py
    __init__.pyc  l2_learning.pyc  l2_nx_self_learning.py  topo_proactive.py
    	
    cp l2_learning.py l2_icmp_block.py
    cp l2_learning.py l2_ip_block.py
    cp l2_learning.py l2_mac_block.py

    檢查檔案都有複製到檢查檔案都有複製到

    ls
    	hub.py           l2_icmp_block.py  l2_mac_block.py         l2_pairs.py
    __init__.py      l2_ip_block.py    l2_multi.py             l3_learning.py
    __init__.pyc     l2_learning.py    l2_nx.py                topo_proactive.py
    l2_flowvisor.py  l2_learning.pyc   l2_nx_self_learning.py
    	
  5. Mininet 除了提供使用 command line 來建立虛擬的網路之外,也可以利用提供的 library 來建立我們想要的環境
  6. 切換到 mininet 的 terminal 視窗,關閉 mininet

    exit
    	*** Stopping 1 switches
    s1 ...
    *** Stopping 3 hosts
    h1 h2 h3 
    *** Stopping 1 controllers
    c0 
    *** Done
    completed in 5566.056 seconds
    	

    前往 mininet 的 example 資料夾

    cd ~/mininet/examples

    建立實驗用的虛擬網路環境

    nano lab3_network_subnets.py
    	#!/usr/bin/python
    
    from mininet.net import Mininet
    from mininet.node import RemoteController,Host,Node
    from mininet.cli import CLI
    from mininet.log import setLogLevel, info
    
    def emptyNet():
    
        net = Mininet( controller=RemoteController )
    
        info( '*** Adding controller\n' )
        net.addController( 'c0', 
                           controller=RemoteController, 
                           ip = "127.0.0.1", 
                           port=6633 )
    
        info( '*** Adding hosts\n' )
        #---------subnet1---------
        h1 = net.addHost( 'h1', 
                          ip='10.0.0.1', 
                          mac='00:00:00:00:00:01',
                          defaultRoute='via 10.0.0.254')
        h2 = net.addHost( 'h2', 
                          ip='10.0.0.2',
                          mac='00:00:00:00:00:02',
                          defaultRoute='via 10.0.0.254')
        h3 = net.addHost( 'h3', 
                          ip='10.0.0.3', 
                          mac='00:00:00:00:00:03',
                          defaultRoute='via 10.0.0.254')
        #---------subnet2---------
        k4 = net.addHost( 'k4', 
                          ip='192.168.0.10', 
                          mac='00:00:00:00:00:10',
                          defaultRoute='via 192.168.0.254')
        k5 = net.addHost( 'k5',
                          ip='192.168.0.11',
                          mac='00:00:00:00:00:11',
                          defaultRoute='via 192.168.0.254')
        k6 = net.addHost( 'k6', 
                          ip='192.168.0.12', 
                          mac='00:00:00:00:00:12',
                          defaultRoute='via 192.168.0.254')
            
        info( '*** Adding switch\n' )
        s1 = net.addSwitch( 's1' )
        s2 = net.addSwitch( 's2' )
        info( '*** Creating links\n' )
        #---------subnet1---------
        net.addLink( h1, s1 )
        net.addLink( h2, s1 )
        net.addLink( h3, s1 )
        #---------subnet2---------
        net.addLink( k4, s2 )
        net.addLink( k5, s2 )
        net.addLink( k6, s2 )
        #---------switch----------
        net.addLink( s1, s2 )
    
        info( '*** Adding gateway\n')
        gateway = Host('gateway')
        net.addLink(s1, gateway)
        net.addLink(s2, gateway)
        gateway.intf("gateway-eth0").setIP('10.0.0.254/24')
        gateway.intf("gateway-eth1").setIP('192.168.0.254/24')
        
        info( '*** Starting network\n')
        net.start()
    
        info( '*** Running CLI\n' )
        CLI( net )
    
        info( '*** Stopping network' )
        net.stop()
    
    if __name__ == '__main__':
        setLogLevel( 'info' )
        emptyNet()
    	

    將檔案轉成可執行檔

    sudo chmod +x lab3_network_subnets.py

    執行檔案並我們需要的網路環境

    sudo ./lab3_network_subnets.py
    	*** Adding controller
    Unable to contact the remote controller at 127.0.0.1:6633
    *** Adding hosts
    *** Adding switch
    *** Creating links
    *** Adding gateway
    *** Starting network
    *** Configuring hosts
    h1 h2 h3 k4 k5 k6 
    *** Starting controller
    *** Starting 2 switches
    s1 s2 
    *** Running CLI
    *** Starting CLI:
    mininet>
    	

    正常的狀況而言,網路中的 switch 因為沒有任何的 flow 在裡面,所以網路不會通,此步驟可跳過,因為機器有點多耗時也多

    pingall
    	*** Ping: testing ping reachability
    h1 -> X X X X X 
    h2 -> X X X X X 
    h3 -> X X X X X 
    k4 -> X X X X X 
    k5 -> X X X X X 
    k6 -> X X X X X 
    *** Results: 100% dropped (0/30 received)	
    	

    切換到原本使用 POX controller 的 terminal,確定目錄為 ~/pox/pox/forwarding

    cd ~/pox/pox/forwarding

    先來測試看看 l3_learning 的範例

    ~/pox/pox.py forwarding.l3_learning --fakeways=10.0.0.254,192.168.0.254
    	POX 0.2.0 (carp) / Copyright 2011-2013 James McCauley, et al.
    INFO:core:POX 0.2.0 (carp) is up.
    INFO:openflow.of_01:[00-00-00-00-00-02 1] connected
    INFO:openflow.of_01:[00-00-00-00-00-01 2] connected
    	

    切換到 mininet 的 terminal 視窗

    pingall
    	*** Ping: testing ping reachability
    h1 -> h2 h3 k4 k5 k6 
    h2 -> h1 h3 k4 k5 k6 
    h3 -> h1 h2 k4 k5 k6 
    k4 -> h1 h2 h3 k5 k6 
    k5 -> h1 h2 h3 k4 k6 
    k6 -> h1 h2 h3 k4 k5 
    *** Results: 0% dropped (30/30 received)	
    	

    我們現在來增加一些簡單的 traffic control,切換回 POX 的視窗,按下 crtl + c 先中斷執行中的 controller

    ^c
    	^CINFO:core:Going down...
    INFO:openflow.of_01:[00-00-00-00-00-01 2] disconnected
    INFO:openflow.of_01:[00-00-00-00-00-02 1] disconnected
    INFO:core:Down.	
    	

    從伺服器上抓已經修改好的檔案

    wget http://www.cs.nchu.edu.tw/~snmlab/CloudMgnt201409/l3_learning_lab3.py

    確認檔案存在

    ls
    	hub.py           l2_icmp_block.py   l2_learning.py    l2_multi.py             l3_learning_lab3.py
    __init__.py      l2_icmp_block.pyc  l2_learning.pyc   l2_nx.py                l3_learning.py
    __init__.pyc     l2_ip_block.py     l2_mac_block.py   l2_nx_self_learning.py  topo_proactive.py
    l2_flowvisor.py  l2_ip_block.pyc    l2_mac_block.pyc  l2_pairs.py
    	

    這份檔案是由剛剛的 l3_learning 作修改,從 203 行的地方開始新增內容,若是想要手動修改的話,請注意空格對齊還有重複的部分

    建議對照 l3_learning_lab3 檔案進行修改

    nano -c l3_learning_lab3.py
    	# 191 行新增
    srcaddr = packet.next.srcip
    # 203 行新增
    if dstaddr == '10.0.0.1':
      msg = of.ofp_flow_mod()
      msg.match.dl_type = 0x800
      msg.match.nw_dst = IPAddr("10.0.0.1")
      event.connection.send(msg.pack())
    elif srcaddr == '10.0.0.1':
      msg = of.ofp_flow_mod()
      msg.match.dl_type = 0x800
      msg.match.nw_src = IPAddr("10.0.0.1")
      event.connection.send(msg.pack())
    else:
      actions = []
      actions.append(of.ofp_action_dl_addr.set_dst(mac))
      actions.append(of.ofp_action_output(port = prt))
      match = of.ofp_match.from_packet(packet, inport)
      match.dl_src = None # Wildcard source MAC
                
      msg = of.ofp_flow_mod(command=of.OFPFC_ADD,
      idle_timeout=FLOW_IDLE_TIMEOUT,
      hard_timeout=of.OFP_FLOW_PERMANENT,
      buffer_id=event.ofp.buffer_id,
      actions=actions,
      match=of.ofp_match.from_packet(packet,
                                     inport))
      event.connection.send(msg.pack())
    	

    來測試剛剛修改的 l3_learning_lab3 controller

    ~/pox/pox.py forwarding.l3_learning_lab3 --fakeways=10.0.0.254,192.168.0.254
    	POX 0.2.0 (carp) / Copyright 2011-2013 James McCauley, et al.
    INFO:core:POX 0.2.0 (carp) is up.
    INFO:openflow.of_01:[00-00-00-00-00-02 1] connected
    INFO:openflow.of_01:[00-00-00-00-00-01 2] connected
    	

    切換到 mininet 的 terminal 視窗,作 pingall 的動作,可以發現通往 h1 的封包都不通

    pingall
    	*** Ping: testing ping reachability
    h1 -> X X X X X 
    h2 -> X h3 k4 k5 k6 
    h3 -> X h2 k4 k5 k6 
    k4 -> X h2 h3 k5 k6 
    k5 -> X h2 h3 k4 k6 
    k6 -> X h2 h3 k4 k5 
    *** Results: 33% dropped (20/30 received)	
    	

    我們再來觀察目前 switch 中的 flow,可以發現來往 h1 機器的 IP 封包都會被 drop

    sh ovs-ofctl dump-flows s1
    	NXST_FLOW reply (xid=0x4):
     cookie=0x0, duration=66.621s, table=0, n_packets=1, n_bytes=98, idle_age=56, ip,nw_dst=10.0.0.1 actions=drop
     cookie=0x0, duration=116.643s, table=0, n_packets=4, n_bytes=392, idle_age=76, ip,nw_src=10.0.0.1 actions=drop	
    	

Assignment#3 (Deadline: before Lab4 12/24 23:59)

    設計一 POX controller program,需有以下功能:

  1. 阻擋所有的 ping 10.0.0.1 的 ICMP 封包。
  2. 建立一台 mac to IP 的唯一機器。
  3. 阻擋特定兩個 IP 互相連線。
  4. 其他的網路功能則維持 L2 learning 之狀態。
  5. 由於 Lab4 實驗環境架設需做調整,故本次作業不會有 Bonus 題目,分數照常算入本次作業,

    但是機器會在 12/23 的下午做更新動作,故上機的實驗機器將不再提供作業,請同學注意時間。

    繳交作業方式有所變更,請寄信到助教信箱( g102056049@mail.nchu.edu.tw ),請附上姓名、 機器代號和檔案位置,並且標明第 2 點和第 3 點的 IP ,

    已經完成作業的同學不好意思,麻煩再寄一份檔案到我信箱。

    可繳交作業時間不會改變,請同學盡速完成作業。

    繳交作業時請關閉POX和Mininet,避免衝突。