Installing Multi-Node OpenStack Folsom with DevStack

These days we are preparing to release WSO2 Stratos 2.0 PaaS Foundation and we needed to install OpenStack in multiple nodes in our lab environment.

In this guide I'm going to explain how to install OpenStack Folsom using the DevStack script [1] and also how to install additional Compute Nodes.


Installing OpenStack was not an easy task. I spent last 5 days to install and configure the OpenStack and tried various guides.

Earlier we used Damitha's guide to Install OpenStack Essex on a single node [2]. Then we followed different guides. Finally we decided to try the DevStack script, which is a great option to install and run an OpenStack cloud in a local development environment.

Using Nova Network

This guide uses the Nova Network instead of the Quantum Network plugin. I followed a really good guide [3] at NetworkStatic.net, which also use the Nova Network. Quantum Network Plugin is bit complicated and we need two physical network interfaces. I'm planning to install OpenStack with Quantum Plugin soon.

For the Nova Network, one physical network interface is enough.

Why another guide?

Following the exact steps in many guides didn't help me to complete the OpenStack installation successfully. Even the Muti-Node lab guide [4] from DevStack failed. When we run DevStack script according to that guide,  the compute node installation was failing and I figured out one important configuration of setting "SERVICE_HOST" was missing.

Out of all the guides, NetworkStatic.net guide [3] was the best I found for installing OpenStack Folsom with Nova Network. However I couldn't get the OpenStack running successfully.

I think the reason might be that when we clone the DevStack GIT repository, we get the master branch. I tried to use Folsom branch as instructed in DevStack FAQ [5], but that attempt also failed.

Then I found that DevStack GitHub also maintain different branches for OpenStack releases. What I did was to clone the stable Folsom branch and everything worked perfectly!

Setting up the environment

For the multi-node setup, we had two desktop PCs. One for installing OpenStack Controller Node and the other one is for installing OpenStack Compute Node

Step 1: Installing Ubuntu Server

I installed Ubuntu Server 12.04.2 LTS (Precise Pangolin). After installation, we need to comment out existing eth0 configuration and use a static address for eth0 network interface. For that we need to edit /etc/network/interfaces file.

sudo vi /etc/network/interfaces

# The primary network interface
#auto eth0
#iface eth0 inet dhcp

auto eth0
iface eth0 inet static
address 10.100.0.30
netmask 255.255.252.0
gateway 10.100.1.254
dns-nameservers 8.8.8.8

Then upgrade of existing software packages after updating the package list index.

sudo su
apt-get update && apt-get -y upgrade

Step 2: Add "stack" user for DevStack as root user

sudo su

groupadd stack
useradd -g stack -s /bin/bash -d /opt/stack -m stack

echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

Step 3: Clone DevStack stable/folsom branch as stack user

This is the most important step for me!

sudo su stack
cd

sudo apt-get install -y git
git clone git://github.com/openstack-dev/devstack.git -b stable/folsom

cd devstack

Step 4: Installing OpenStack Controller Node

To install Controller node, we need to create a localrc inside devstack directory (/opt/stack/devstack), which is inside the home directory of stack user. 

Following is the content of localrc file I used.

HOST_IP=10.100.0.30
FLAT_INTERFACE=eth0
FIXED_RANGE=192.168.16.0/25
FIXED_NETWORK_SIZE=126
FLOATING_RANGE=10.100.0.128/25
MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=root
RABBIT_PASSWORD=rabbitmq
SERVICE_PASSWORD=supersecrete
SERVICE_TOKEN=token

SCREEN_LOGDIR=/opt/stack/logs

Note that I have used an IP address range accessible from the network for  "FLOATING_RANGE". We can use any private network for "FIXED_RANGE". Subnet Calculator [6] can be a useful tool for deciding on which network ranges to use.

After creating the localrc file, just run the stack.sh.

./stack.sh

Make sure stack.sh completed successfully. Following is the output I get at last.

stack.sh completed in 3168 seconds.



Horizon is now available at http://10.100.0.30/
Keystone is serving at http://10.100.0.30:5000/v2.0/
Examples on using novaclient command line is in exercise.sh
The default users are: admin and demo
The password: labstack
This is your host ip: 10.100.0.30

Step 5: Installing OpenStack Compute Node

We are installing Compute Node in a different desktop PC and therefore we need to make sure Steps 1 to 3 are followed.

For compute node, we need to create following localrc inside devstack directory (/opt/stack/devstack).

SERVICE_HOST=10.100.0.30


HOST_IP=10.100.0.28
FLAT_INTERFACE=eth0
FIXED_RANGE=192.168.16.0/25
FIXED_NETWORK_SIZE=126
FLOATING_RANGE=10.100.0.128/25
MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=root
RABBIT_PASSWORD=rabbitmq
SERVICE_PASSWORD=supersecrete
SERVICE_TOKEN=token
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_HOST=$SERVICE_HOST

ENABLED_SERVICES=n-cpu,n-net,n-api,c-sch,c-api,c-vol,rabbit

SCREEN_LOGDIR=/opt/stack/logs

Please note that the "SERVICE_HOST" points the Controller Node IP.

Now run stack.sh

./stack.sh

The stack.sh should complete successfully.

Note: You can repeat this step to install any number of Compute Nodes.

Step 6: Testing the OpenStack Folsom Setup.

Now you can access the OpenStack Dashboard hosted at Controller Node.

Just open the browser and enter the IP of the Controller Node in address bar. The Login details were displays at the end of stack.sh script for Controller Node.

OpenStack Commands:

There are also various commands to get more information and change configuration. Before running any command we need to run following command from devstack directory.

source openrc <user-name> <tenant-name>

For example:

source openrc demo demo

List services:

nova-manage service list

List images:

glance image-list

Disable nova compute service in controller:

nova-manage service disable --host=s2controller --service=nova-compute

Restarting OpenStack after a reboot

If for some reason you want to reboot the server, then you can start OpenStack again by running the rejoin-stack.sh script.

You might get an error like following when you run ./rejoin-stack.sh 

./rejoin-stack.sh 
Attaching to already started screen session..
Cannot open your terminal '/dev/pts/0' - please check.

To avoid that you can use either one of following options:

• Run script /dev/null to own the shell [7]

script /dev/null

• Change permissions for terminal [8]

sudo chmod o+rw /dev/pts/0

Now you should be able run rejoin-stack.sh.

To exit from screen, press Ctrl-a d or Ctrl-a Ctrl-d [9]

That's it! Now you have an OpenStack cloud environment in your own lab environment!

NetworkStatic.net has also published another great guide to install OpenStack Grizzly [10]. I'm eagerly waiting to try that also with Quantum Network plugin!

References

[1] http://devstack.org/
[2] http://damithakumarage.wordpress.com/2013/03/20/easy-installation-of-openstack-essex-on-a-single-node/
[3] http://networkstatic.net/openstack-multi-node-devstack-nova-network-tutorial/
[4] http://devstack.org/guides/multinode-lab.html
[5] http://devstack.org/faq.html
[6] http://www.subnet-calculator.com/
[7] https://makandracards.com/makandra/2533-solve-screen-error-cannot-open-your-terminal-dev-pts-0-please-check
[8] http://www.linuxquestions.org/questions/linux-general-1/problem-using-screen-cannot-open-your-terminal-dev-pts-0-please-check-338313/
[9] http://stackoverflow.com/questions/4847691/how-do-i-get-out-of-a-screen-without-typing-exit
[10] http://networkstatic.net/installing-openstack-grizzly-with-devstack/