Greetings, Stackers! One of the largest barriers preventing the widespread execution, adoption and migration to Neutron is the complexity involved in using such a configurable and powerful abstraction of networking. This is the first of a series of blog articles intended to decrease this knowledge barrier.
We will focus today’s discussion on DevStack, as it is the most widely used deployment method by Neutron developers. The choice of VxLAN over GRE as the overlay technology here is purely academic, and the tunneling protocol in use can be easily changed (thus exemplifying the purpose of modular layer 2). If further education is required by the reader on the subject of tunnel networks as a network overlay, please reference the Neutron ML2 OpenStack wiki, available online here https://wiki.openstack.org/wiki/Neutron/ML2.
Please note that this is a developer’s guide to deploying Neutron in a development configuration using DevStack, and is not meant to be used in a production environment.
Network Abstractions in OpenStack
Ideally, we would like to provide our OpenStack hosts with two networks in order to implement a logical separation between the external network and the OpenStack hosts. The external network (or provider network) contains the resources which we will to provide to the virtual machines running on the hosts. The host network will be used in order to provide a separate environment for the OpenStack components to communicate with one another. In other words, the virtual machines will never see the host network. If the hosts do not have multiple network interfaces, we can implement the logical separation by creating a separate VLAN interface on the host. This does, however, require that the system which you are using to access the hosts be able to to tag the VLAN packets, or that there is a router and switch in your infrastructure to tag the packets for your machine. Additional resources for further explanation are provided here: https://wiki.ubuntu.com/vlan.
The provider network will be represented by the 10.75.20.0/24 network as the untagged, native VLAN on the first network interface, eth0. The host network will be tagged on the same interface as VLAN 10. A logical diagram of the example infrastructure is provided below:
OpenStack Neutron Host Initial Configuration
We will start with a fresh installation of Ubuntu 14.04.1 LTS. Configure the eth0 interface from the console:
sudo ip addr add 10.75.20.1/24 dev eth0 sudo ip link set dev eth0 up sudo ip route 0/0 via 10.75.20.252 dev eth0
Because of the way in which Neutron performs address translation for external networks, reverse path filtering must be disabled in order for traffic to flow correctly. Enable packet forwarding and disable reverse path filtering by editing the /etc/sysctl.conf file:
net.ipv4.ip_forward=1 net.ipv4.conf.all.rp_filter=0 net.ipv4.conf.default.rp_filter=0
Execute the sysctl command to make the changes active immediately:
sudo sysctl -p
Make sure that the vlan package is installed by executing the following command:
sudo apt-get install vlan
Next, make sure the 802.1q kernel module is loaded, and instruct the kernel to load it on boot:
sudo modprobe 8021q sudo sh -c 'echo 8021q >> /etc/modules'
Next, we create a new interface on VLAN 10: sudo vconfig add eth0 70Configure the ip address and subnet for the interface:
sudo vconfig add eth0 10 sudo ip addr add 172.16.10.1/24 dev eth0.10 sudo ip link set dev eth0.10 up
Verify that the interface is operational and reachable from the host network before continuing.
DevStack Installation
Clone the openstack-dev organization of the devstack repository from github:
sudo apt-get -y install git git clone https://github.com/openstack-dev/devstack.git
Create the stack user, copy the source code into the new home directory, and switch user to stack:
sudo ./devstack/tools/create-stack-user.sh sudo usermod -a -G sudo stack sudo mv devstack /opt/stack sudo su - stack sudo chown -R stack:stack devstack
Create the /opt/stack/devstack/local.conf file with your editor of choice:
[[local|localrc]] ADMIN_PASSWORD=mysupersecretadminpassword HOST_IP=172.16.10.1 MYSQL_PASSWORD=stackdb MYSQL_HOST=$HOST_IP RABBIT_PASSWORD=stackqueue SERVICE_PASSWORD=$ADMIN_PASSWORD RABBIT_HOST=$HOST_IP SERVICE_HOST=$HOST_IP # Enabled Services enable_service q-lbaas enable_service q-svc enable_service q-agt enable_service q-dhcp enable_service q-l3 enable_service q-meta enable_service neutron # Network NETWORK_GATEWAY=10.1.0.1 PUBLIC_NETWORK_GATEWAY=10.75.20.2 FLOATING_RANGE=10.75.20.0/24 FIXED_RANGE=10.1.0.0/24 FIXED_NETWORK_SIZE=256 # Neutron - OVS, VxLAN, and ML2 Q_PLUGIN=ml2 Q_AGENT=openvswitch Q_USE_DEBUG_COMMAND=True Q_L3_ENABLED=True Q_L3_ROUTER_PER_TENANT=True ENABLE_TENANT_TUNNELS=True TENANT_TUNNEL_RANGE=-1:1000 Q_ML2_TENANT_NETWORK_TYPE=vxlan Q_USE_PROVIDERNET_FOR_PUBLIC=True OVS_BRIDGE_MAPPINGS=public:br-eth0 OVS_ENABLE_TUNNELING=True PUBLIC_BRIDGE=br-eth0 # Nova disable_service n-net VIRT_DRIVER=libvirt # Pip PIP_USE_MIRRORS=False USE_GET_PIP=1 # Logging SCREEN_LOGDIR=/opt/stack/logs # Tempest enable_service tempest TEMPEST_HTTP_IMAGE=127.0.0.1
Note that the PUBLIC_NETWORK_GATEWAY should not have the same IP address as any other device on your provider network, including the address assigned to the eth0 interface. It is extremely important that the ip address assigned to the eth0 interface and the address used as the public network gateway be distinct, otherwise the DevStack installation will fail and you will lose connectivity to the host.
It is now time to … stack it up!
cd devstack ./stack.sh
(Installation runs … go get some coffee or other beverage of choice)
Horizon is now available at http://172.16.10.1/ Keystone is serving at http://172.16.10.1:5000/v2.0/ Examples on using novaclient command line is in exercise.sh The default users are: admin and demo The password: mysupersecretadminpassword This is your host ip: 172.16.10.1 You are using Q_AGENT_EXTRA_AGENT_OPTS to pass configuration into /etc/neutron/neutron.conf. Please convert that configuration in localrc to a /etc/neutron/neutron.conf section in local.conf: 2014-08-15 15:41:00.580 | WARNING: Q_AGENT_EXTRA_AGENT_OPTS is used Q_AGENT_EXTRA_AGENT_OPTS will be removed early in the 'K' development cycle [[post-config|/$Q_PLUGIN_CONF_FILE]] [DEFAULT] tunnel_types=vxlan 2014-08-15 15:41:00.581 | stack.sh completed in 527 seconds.
Excellent! Devstack has finished the installation.
Execute the following command to stop the Devstack processes while we perform the final network configuration:
script /dev/null screen -D -RR stack <ctrl>-a <ctrl>-\ y
OpenStack Neutron Host Final Network Configuration
We will need to add eth0 to the Open VSwitch bridge which devstack has set up for us. If you are connected via ssh, make sure you are using the host network, otherwise you will be disconnected when you execute the first command:
sudo ip addr flush dev eth0 sudo ovs-vsctl add-port br-eth0 eth0 sudo ip addr add 10.75.20.1/24 dev br-eth0 sudo ip route add 0/0 via 10.75.20.252 dev br-eth0
Make sure that both ip addresses are now pingable on the OVS bridge from the provider network.
Modify the /etc/network/interfaces file to create the final network interfaces configuration:
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface allow-hotplug eth0 # The external bridge auto br-eth0 allow-ovs br-eth0 iface br-eth0 inet static ovs_type OVSBridge ovs_ports eth0 address 10.75.20.1 netmask 255.255.255.0 gateway 10.75.20.252 dns-nameservers 75.75.75.75 75.75.75.76 pre-up ovs-vsctl --may-exist add-br br-eth0 pre-up ovs-vsctl --may-exist add-port br-eth0 eth0 pre-up ifconfig br-eth0 up post-up ip addr add 10.75.20.2/24 dev br-eth0 post-down ovs-vsctl --if-exists del-port br-eth0 eth0 post-down ip addr delete 10.75.20.2/24 dev br-eth0 post-down ifconfig br-eth0 down # The host VLAN 10 auto eth0.10 iface eth0.10 inet static address 172.16.10.1 netmask 255.255.255.0
Reboot, and verify connectivity to the host and provider interfaces:
sudo shutdown -ry now
(system reboots)
sudo su - stack ping -c 1 172.16.10.1 PING 172.16.10.1 (172.16.10.1): 56 data bytes 64 bytes from 172.16.10.1: icmp_seq=0 ttl=63 time=1.985 ms ping -c 1 10.75.20.1 PING 10.75.20.1 (10.75.20.1): 56 data bytes 64 bytes from 10.75.20.1: icmp_seq=0 ttl=63 time=1.841 ms ping -c 1 10.75.20.2 PING 10.75.20.2 (10.75.20.2): 56 data bytes 64 bytes from 10.75.20.2: icmp_seq=0 ttl=63 time=2.633 ms
Modify the /etc/neutron/dhcp_agent.ini:
dhcp_domain = weston.la dnsmasq_dns_servers = 75.75.75.75,75.75.76.76
Restart the devstack processes:
script /dev/null sudo -u stack sh -c 'cd /opt/stack/devstack ; ./rejoin-stack.sh'
Press <ctrl-a> d to detach from the screen session.
Finished
Profit! And, remember the OpenStack mission statement:
To produce the ubiquitous OpenSource Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.
What’s Up Next?
In the next post we will go over how to expose provider network resources to your OpenStack tenants with Neutron.