Steve's OpenStack Blog

  • Home
  • About

Category Archives: Neutron

DevStack Neutron With ML2, Open VSwitch, VLANs, and Overlay VxLAN Tunnels

18th August, 2014 · sweston · 2 Comments

Greetings, Stackers!  One of the largest barriers preventing the widespread execution, adoption and migration to Neutron is the complexity involved in using such a configurable and powerful abstraction of networking.  This is the first of a series of blog articles intended to decrease this knowledge barrier.

We will focus today’s discussion on DevStack, as it is the most widely used deployment method by Neutron developers.    The choice of VxLAN over GRE as the overlay technology here is purely academic, and the tunneling protocol in use can be easily changed (thus exemplifying the purpose of modular layer 2).  If further education is required by the reader on the subject of tunnel networks as a network overlay, please reference the Neutron ML2 OpenStack wiki, available online here https://wiki.openstack.org/wiki/Neutron/ML2.

Please note that this is a developer’s guide to deploying Neutron in a development configuration using DevStack, and is not meant to be used in a production environment.

Network Abstractions in OpenStack

Ideally, we would like to provide our OpenStack hosts with two networks in order to implement a logical separation between the external network and the OpenStack hosts.  The external network (or provider network) contains the resources which we will to provide to the virtual machines running on the hosts.  The host network will be used in order to provide a separate environment for the OpenStack components to communicate with one another.  In other words, the virtual machines will never see the host network.  If the hosts do not have multiple network interfaces, we can implement the logical separation by creating a separate VLAN interface on the host.  This does, however, require that the system which you are using to access the hosts be able to to tag the VLAN packets, or that there is a router and switch in your infrastructure to tag the packets for your machine.  Additional resources for further explanation are provided here:  https://wiki.ubuntu.com/vlan.

The provider network will be represented by the 10.75.20.0/24 network as the untagged, native VLAN on the first network interface, eth0.  The host network will be tagged on  the same interface as VLAN 10.  A logical diagram of the example infrastructure is provided below:

Blog Diagram

OpenStack Neutron Host Initial Configuration

We will start with a fresh installation of Ubuntu 14.04.1 LTS.  Configure the eth0 interface from the console:

sudo ip addr add 10.75.20.1/24 dev eth0 
sudo ip link set dev eth0 up 
sudo ip route 0/0 via 10.75.20.252 dev eth0

Because of the way in which Neutron performs address translation for external networks, reverse path filtering must be disabled in order for traffic to flow correctly.  Enable packet forwarding and disable reverse path filtering by editing the /etc/sysctl.conf file:

net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0

Execute the sysctl command to make the changes active immediately:

sudo sysctl -p

Make sure that the vlan package is installed by executing the following command:

sudo apt-get install vlan

Next, make sure the 802.1q kernel module is loaded, and instruct the kernel to load it on boot:

sudo modprobe 8021q
sudo sh -c 'echo 8021q >> /etc/modules'

Next, we create a new interface on VLAN 10:  sudo vconfig add eth0 70Configure the ip address and subnet for the interface:

sudo vconfig add eth0 10
sudo ip addr add 172.16.10.1/24 dev eth0.10
sudo ip link set dev eth0.10 up

Verify that the interface is operational and reachable from the host network before continuing.

DevStack Installation

Clone the openstack-dev organization of the devstack repository from github:

sudo apt-get -y install git
git clone https://github.com/openstack-dev/devstack.git

Create the stack user, copy the source code into the new home directory, and switch user to stack:

sudo ./devstack/tools/create-stack-user.sh
sudo usermod -a -G sudo stack
sudo mv devstack /opt/stack
sudo su - stack
sudo chown -R stack:stack devstack

Create the /opt/stack/devstack/local.conf file with your editor of choice:

[[local|localrc]]
ADMIN_PASSWORD=mysupersecretadminpassword
HOST_IP=172.16.10.1
MYSQL_PASSWORD=stackdb
MYSQL_HOST=$HOST_IP
RABBIT_PASSWORD=stackqueue
SERVICE_PASSWORD=$ADMIN_PASSWORD
RABBIT_HOST=$HOST_IP
SERVICE_HOST=$HOST_IP

# Enabled Services
enable_service q-lbaas
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron

# Network
NETWORK_GATEWAY=10.1.0.1
PUBLIC_NETWORK_GATEWAY=10.75.20.2
FLOATING_RANGE=10.75.20.0/24
FIXED_RANGE=10.1.0.0/24
FIXED_NETWORK_SIZE=256

# Neutron - OVS, VxLAN, and ML2
Q_PLUGIN=ml2
Q_AGENT=openvswitch
Q_USE_DEBUG_COMMAND=True
Q_L3_ENABLED=True
Q_L3_ROUTER_PER_TENANT=True
ENABLE_TENANT_TUNNELS=True
TENANT_TUNNEL_RANGE=-1:1000
Q_ML2_TENANT_NETWORK_TYPE=vxlan
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_BRIDGE_MAPPINGS=public:br-eth0
OVS_ENABLE_TUNNELING=True
PUBLIC_BRIDGE=br-eth0

# Nova
disable_service n-net
VIRT_DRIVER=libvirt

# Pip
PIP_USE_MIRRORS=False
USE_GET_PIP=1

# Logging
SCREEN_LOGDIR=/opt/stack/logs

# Tempest
enable_service tempest
TEMPEST_HTTP_IMAGE=127.0.0.1

Note that the PUBLIC_NETWORK_GATEWAY should not have the same IP address as any other device on your provider network, including the address assigned to the eth0 interface.  It is extremely important that the ip address assigned to the eth0 interface and the address used as the public network gateway be distinct, otherwise the DevStack installation will fail and you will lose connectivity to the host.

It is now time to … stack it up!

cd devstack
./stack.sh

(Installation runs … go get some coffee or other beverage of choice)

Horizon is now available at http://172.16.10.1/ 
Keystone is serving at http://172.16.10.1:5000/v2.0/ 
Examples on using novaclient command line is in exercise.sh 
The default users are: admin and demo 
The password: mysupersecretadminpassword
This is your host ip: 172.16.10.1
You are using Q_AGENT_EXTRA_AGENT_OPTS to pass configuration into /etc/neutron/neutron.conf.
Please convert that configuration in localrc to a /etc/neutron/neutron.conf section in local.conf:
2014-08-15 15:41:00.580 | WARNING: Q_AGENT_EXTRA_AGENT_OPTS is used
Q_AGENT_EXTRA_AGENT_OPTS will be removed early in the 'K' development cycle

[[post-config|/$Q_PLUGIN_CONF_FILE]]
[DEFAULT]

tunnel_types=vxlan
2014-08-15 15:41:00.581 | stack.sh completed in 527 seconds.

Excellent!  Devstack has finished the installation.

Execute the following command to stop the Devstack processes while we perform the final network configuration:

script /dev/null
screen -D -RR stack
<ctrl>-a <ctrl>-\ y

OpenStack Neutron Host Final Network Configuration

We will need to add eth0 to the Open VSwitch bridge which devstack has set up for us.  If you are connected via ssh, make sure you are using the host network, otherwise you will be disconnected when you execute the first command:

sudo ip addr flush dev eth0
sudo ovs-vsctl add-port br-eth0 eth0
sudo ip addr add 10.75.20.1/24 dev br-eth0
sudo ip route add 0/0 via 10.75.20.252 dev br-eth0

Make sure that both ip addresses are now pingable on the OVS bridge from the provider network.

Modify the /etc/network/interfaces file to create the final network interfaces configuration:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0

# The external bridge 
auto br-eth0
allow-ovs br-eth0
iface br-eth0 inet static
ovs_type OVSBridge
ovs_ports eth0
address 10.75.20.1
netmask 255.255.255.0
gateway 10.75.20.252
dns-nameservers 75.75.75.75 75.75.75.76
pre-up ovs-vsctl --may-exist add-br br-eth0
pre-up ovs-vsctl --may-exist add-port br-eth0 eth0
pre-up ifconfig br-eth0 up
post-up ip addr add 10.75.20.2/24 dev br-eth0
post-down ovs-vsctl --if-exists del-port br-eth0 eth0
post-down ip addr delete 10.75.20.2/24 dev br-eth0
post-down ifconfig br-eth0 down

# The host VLAN 10
auto eth0.10
iface eth0.10 inet static
address 172.16.10.1
netmask 255.255.255.0

Reboot, and verify connectivity to the host and provider interfaces:

sudo shutdown -ry now

(system reboots)

sudo su - stack

ping -c 1 172.16.10.1 
PING 172.16.10.1 (172.16.10.1): 56 data bytes 
64 bytes from 172.16.10.1: icmp_seq=0 ttl=63 time=1.985 ms 

ping -c 1 10.75.20.1 
PING 10.75.20.1 (10.75.20.1): 56 data bytes 
64 bytes from 10.75.20.1: icmp_seq=0 ttl=63 time=1.841 ms 

ping -c 1 10.75.20.2 
PING 10.75.20.2 (10.75.20.2): 56 data bytes 
64 bytes from 10.75.20.2: icmp_seq=0 ttl=63 time=2.633 ms

Modify the /etc/neutron/dhcp_agent.ini:

dhcp_domain = weston.la
dnsmasq_dns_servers = 75.75.75.75,75.75.76.76

Restart the devstack processes:

script /dev/null
sudo -u stack sh -c 'cd /opt/stack/devstack ; ./rejoin-stack.sh'

Press <ctrl-a> d to detach from the screen session.

Finished

Profit!  And, remember the OpenStack mission statement:

To produce the ubiquitous OpenSource Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.

What’s Up Next?

In the next post we will go over how to expose provider network resources to your OpenStack tenants with Neutron.

 

Posted in Neutron, OpenStack | Tags: DevStack, ML2, Neutron, Open VSwitch, OpenStack, VLAN, VxLAN |

Recent Posts

  • DevStack Neutron With ML2, Open VSwitch, VLANs, and Overlay VxLAN Tunnels

Recent Comments

  • Ramy Asselin on DevStack Neutron With ML2, Open VSwitch, VLANs, and Overlay VxLAN Tunnels
  • Gloria Gu on DevStack Neutron With ML2, Open VSwitch, VLANs, and Overlay VxLAN Tunnels
© Steve Weston's OpenStack Blog
  • About