OpenStack Architecture

This page exists to document OpenStack configuration.

Note that the current approach is to use devstack, which is not a good way to learn OpenStack. So much of this document will be about doing a devstack-like configuration for Funtoo.

This document will split OpenStack configuration into each architectural component, describing configuration steps for each component separately.

SQL Database

A number of OpenStack services use a SQL back-end for storing various bits of data.

While DevStack uses MySQL for its SQL deployment, multiple database back-ends are actually supported thanks to SQLAlchemy being used behind the scenes, which is a re-targetable Python database API. Thus, it should be possible to use Postgres, etc, by simply using different connection strings. A list of SQLAlchemy connection types can be found on this SQLAlchemy documentation page.

Using a single root database user account for all services is not a good policy for production deployment. Ideally, each service should have its own restricted user account with only the ability to access its own database.

Let's look at how each service is configured in regards to SQL:

nova

Here's how to set up a MySQL database back-end for nova and tell nova to initialize its database tables:

mysql> create database nova character set latin1;
Query OK, 1 row affected (0.02 sec)

mysql> grant all privileges on nova.* to nova@localhost identified by 'foobar';
Query OK, 0 rows affected (0.00 sec)

Now set the following connection string in /etc/nova/nova.conf:

--sql_connection=mysql://nova:foobar@localhost/nova

Note the use of the latin1 character set when we created the tables in MySQL. This is so the following command will not cause an error due to the default UTF-8 character set creating indexes that are too big for MySQL to handle:

root # nova-manage db sync
2012-03-02 21:31:14 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.migration' from '/usr/lib64/python2.7/site-packages/nova/db/sqlalchemy/migration.pyc'> from (pid=17779) __get_b
ackend /usr/lib64/python2.7/site-packages/nova/utils.py:602

After running the command above, you should now have all the relevant database tables created:

xdev var # mysql -u root -p nova
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 16
Server version: 5.1.61-log Gentoo Linux mysql-5.1.61

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show tables;
+-------------------------------------+
| Tables_in_nova                      |
+-------------------------------------+
| agent_builds                        |
| aggregate_hosts                     |
| aggregate_metadata                  |
| aggregates                          |
| auth_tokens                         |
| block_device_mapping                |
| bw_usage_cache                      |
| certificates                        |
| compute_nodes                       |
| console_pools                       |
...

You have now validated that nova is connecting to your MySQL database correctly.

glance

From glance.openstack.org:

The Glance project provides services for discovering, registering, and retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image.

Glance typically uses a MySQL database called glance, although the name is configurable in the connection string.

SQL connection settings might be stored in a glance configuration file located at /opt/stack/glance/etc/glance-registry.conf. In the devstack installation process, /opt/stack/glance contains a git checkout of the glance software.

The SQL connection configuration string might look something like this:

sql_connection = mysql://glance:yourpassword@192.168.206.130/glance

More info on glance configuration is available here.

keystone

Keystone, the OpenStack identity service, also uses SQL. etc/keystone.conf keystone install/git repo directory is used to store the SQL configuration:

sql_connection = %SQL_CONN%

As everything else, the SQL connection string uses SQLAlchemy syntax.

Quantum and Open VSwitch

Quantum is an incubated OpenStack project to provide "network connectivity as a service" between interface devices (e.g., vNICs) managed by other Openstack services (e.g., nova).

Open VSwitch is described as:

Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag). In addition, it is designed to support distribution across multiple physical servers similar to VMware's vNetwork distributed vswitch or Cisco's Nexus 1000V.

There is an Open VSwitch Plug-in for OpenStack Quantum which can be set up by DevStack. This plug-in uses SQL storage. The SQLAlchemy connection string is stored in (relative to git/install root) etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini. Similarly to everything but nova, the SQL connection string is stored in sql_connection = format.

If the plugin is enabled, the following settings are added to nova.conf:

 --libvirt_vif_type=ethernet
 --libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
 --linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
 --quantum_use_dhcp

Melange

From the Melange site:

Melange is intended to provide network information services for use across OpenStack services. The initial focus will be on IP address management (IPAM) and address discovery (DHCP/dnsmasq) functionality. Melange is intended to be a standalone service with it's own API. However, the initial use case will be to decouple existing IP management and VM address discovery from Nova and support the existing Nova networking capabilities.

Melange also uses a sql_connection = string stored in its etc/melange/melange.conf (relative to install/git root).

RabbitMQ

RabbitMQ is a reliable messaging framework used by OpenStack. Currently, it looks like only nova uses it. Nova is configured to connect to rabbitmq by setting the following lines in /etc/nova/nova.conf:

--rabbit_host=$RABBIT_HOST
--rabbit_password=$RABBIT_PASSWORD

Rabbit's password is configured using the following command, as root:

root # rabbitmqctl change_password guest $RABBIT_PASSWORD

I am not yet completely sure how RabbitMQ fits into the OpenStack architecture. It may be that the supporting services expect it to be running locally, and that Nova compute nodes need to hook into a Nova instance, which would typically be running remotely. (Thus the ability for DevStack to target a remote RabbitMQ host.)

Virtualization Technology

DevStack defaults to configuring OpenStack to use libvirt with KVM, and will fall back to basic QEMU support if the kvm kernel module is not available. It also has support for using libvirt with LXC, in addition to using Xen Server directly (bypassing libvirt.)