After sharing my tutorial with a few friends in the OpenStack community, it seems there was some interest in having these documents published outside of our internal Nimbula wiki. So, here it is:

http://openstack-folsom-install-guide.readthedocs.org/en/latest/ 

If you have any suggestions or find an error, please send an email to openstack-docs (at) nimbula.com

-Zach VanDuyn, Technical Marketing Intern

Requirements

• A Linux host with KVM installed and working. (This host will be used to construct the FreeBSD machineimage. No data on this host will be destroyed.)
• Download the FreeBSD 9.0 install disk from http://www.freebsd.org/where.html
• Download the virtio drivers for FreeBSD 9.0 from http://people.freebsd.org/~kuriyama/virtio/

In these instructions, commands to be run under Linux are shown in blue and commands to be run under the FreeBSD guest OS are shown in red.

Install instructions

Create an empty disk image (10GB)

The basis of a machineimage is a sparse file that is sized to the desired maximum size of the virtual hard drive. In these instructions we will use a 10GB maximum size. With a blocksize of 1024 bytes, a 10GB file consists of 10485760 blocks.

On your Linux host, run the following command to create the sparse file:

dd if=/dev/null of=freebsd-9_0.raw bs=1024 seek=10485760

Boot the FreeBSD installer and perform installation

Launch kvm to boot from the FreeBSD iso file with the sparse virtual hard drive file as the attached virtual hard drive.

On your Linux host, run the following command:

sudo kvm -vnc 0.0.0.0:55 -drive file=freebsd-9_0.raw,if=ide -net nic -net user -cdrom FreeBSD-9.0-RELEASE-amd64-disc1.iso

This will start up KVM exposing the virtual console via VNC port 5955 (vnc base of 5900 + 55 from kvm command).

Connect to the guest OS using a VNC client pointed to port 5955 on your Linux host. (The VNC client may also run on your Linux host.)

Step through the FreeBSD installer:

• Allocate the whole virtual drive to FreeBSD.
• Configure networking to use IPv4 and DHCP.

When the installer finishes, reboot the FreeBSD guest.

Install VirtIO drivers

KVM and Nimbula Director support virtualization-optimized device emulation for hard drives and network adapters known as “virtio”. This virtio device emulation provides significant performance gains over full device emulation. To use virtio, specific device drivers need to be installed in the guest OS.

Download the binary package of virtio drivers for FreeBSD 9.0 from http://people.freebsd.org/~kuriyama/virtio/. Copy the driver file to the guest OS.

Install the package by running the following command in the guest OS:

pkg_add virtio-kmod-9-0.239473.tbz

Configure the VirtIO drivers

Edit the /boot/loader.conf file to add the following lines:

virtio_load="YES"
virtio_pci_load="YES"
virtio_blk_load="YES"
if_vtnet_load="YES"

Edit /etc/fstab in the guest OS to change the device identifiers to /dev/vtbd*. (The following example shows the original device entries commented out. The new entries in bold are the virtio entries.)

# Device Mountpoint FStype Options Dump Pass#
#/dev/ada0p2 / ufs rw 1 1
#/dev/ada0p3 none swap sw 0 0
/dev/vtbd0p2 / ufs rw 1 1
/dev/vtbd0p3 none swap sw 0 0

Edit /etc/rc.conf in the guest OS to add the line shown in bold. (This line configures FreeBSD to use DHCP when bringing up the vtnet0 network interface.):

hostname="freebsd-90"
ifconfig_re0="DHCP"
ifconfig_vtnet0="DHCP"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"


Power off the guest.

Relaunch using virtio for the virtual hard drive and virtual network adapter using the following KVM command on your Linux host:

sudo kvm -vnc 0.0.0.0:54 -drive file=freebsd-9_0.raw,if=virtio -net nic,model=virtio -net user


Verify correct hard drive and network operation in the guest OS.

If the hard drive and network appear to be working correctly then power off the FreeBSD guest.

At this stage you have created a virtual hard drive image with a functioning FreeBSD installation that includes virtio support. The remaining step is to package up the hard drive image as a Nimbula Director machineimage.

Package the machineimage

Tar up the freebsd-9_0.raw file to produce the Nimbula Director machine image using the following command on your Linux host:

tar zcvf freebsd-9_0.tar.gz freebsd-9_0.raw

FreeBSD virtio setup resources

The following resources were used to prepare these instructions:

• http://people.freebsd.org/~kuriyama/virtio/
• http://www.area536.com/projects/freebsd-as-a-kvm-guest-using-virtio/
• http://kdl.nobugware.com/post/2011/10/14/freebsd-90-guest-virtio-support-in-KVM/

In this post I’ll be talking about a fourth method: the Nimbula API Python Bindings. This is a Python library which is distributed alongside our tools (which can be found here). It provides all the functionality available against any Nimbula entity via HTTP API calls to a Nimbula Director endpoint. Let’s look at some examples:

Setting up the environment

To use the bindings we first need to set up the environment to point at the API endpoint we would like to use. The standard way to do this is by setting the NIMBULA_API environment variable (which is also used by our CLI tools). For example:

$ export NIMBULA_API="http://api.example.com/"

Now any subsequent library calls will work against the specified endpoint. If you prefer not to use environment variables it is possible to set the API endpoint directly in code by setting the base_url property of the transport you are using, for example:

from nimbula.api import client
client.DEFAULT_TRANSPORT.base_url = "http://api.example.com/"

Hello Cloud

Now that the endpoint is set we can write our first program. In this example we log in to the cloud as the user ‘project/user’ with the password ‘p4ssw0rd’ and then get some basic details about the site:

from nimbula.api import models as m
m.User.authenticate('project/user', 'p4ssw0rd')
print m.SiteInformation.get()

On my development cluster this produces:

<SiteInformation: {'name': u'usdev150', 'idpname': u'usdev150', 'uri': None, 'licensed': True, 'version': u'2.0.1', 'fingerprint': u'06:7E:67:24:C0:38:2C:F8:6D:97:CE:D2:CC:7F:7B:FF:2C:DC:C0:C0'}>

You may notice that these are all synchronous calls which correspond with REST calls against the API.

Now that we’ve done our first example, let’s look at something more useful. In the following example, we launch two instances (a database and webserver) with two different shapes and two different images. We also set a further placement requirement that these two instances run on the same node. Finally, we wait till the instances have started and then print out their IP addresses.

from nimbula.api import models as m,client as c
from time import sleep
m.User.authenticate('project/user','p4ssw0rd')
lp = m.LaunchPlan()

web_instance = m.Instance()
db_instance = m.Instance()

web_instance.label = 'webserver'
web_instance.imagelist = '/project/webimages/webserver'
web_instance.shape = 'medium'

db_instance.label = 'database'
db_instance.imagelist = '/project/webimages/database'
db_instance.shape = 'large'

lp.instances = [web_instance, db_instance]
lp.relationships = [{ "instances":["database", "webserver" ],
                      "type":"same_node" }  ] # we want the instances to run on the same node

lp.save() # The api launch call happens here

for inst in lp.instances: # The following two lines are necessary for
    inst.bind()           # older versions of the product

def is_running(instance):
    instance.refresh()
    return instance.state == 'running'

while not all(map(is_running, lp.instances)):
    print 'Waiting for instances to start'
    sleep(5)

print 'Instances started'
print '\n'.join(['Instance label: %s , IP: %s' % (i.label, i.ip) for i in lp.instances])

Twisted transport

As I mentioned earlier, the previous examples have all been synchronous calls to the REST API, achieved using a blocking HTTP request. However our Python bindings can also make use of the Twisted library for asynchronous IO. This allows you to make API calls and register callbacks for when they complete. Twisted is a great event-based networking library which we make use of extensively for our internal services. More information can be found here.

There are two ways to make use of theTwisted transport when using our Python bindings. One way is by setting the transport directly on the client model:

from nimbula.api import client
from nimbula.api.transports.http.twisted import PersistentTwistedHTTPTransport
client.DEFAULT_TRANSPORT = PersistentTwistedHTTPTransport

The second way is by importing twisted.internet at the beginning of your script. Our Python bindings will inspect the modules imported and if Twisted is present it will make use of that.

Here is a simple example to delete all instances on the Nimbula deployment using the Twisted transport:

from twisted.internet import reactor, defer
from nimbula.api import models as m

@defer.inlineCallbacks
def delete_instances():
    yield m.User.authenticate('project/user', 'p4ssw0rd')
    instances = yield m.Instance.list()
    dl = [instance.delete() for instance in instances]
    print 'Waiting on callbacks'
    yield defer.DeferredList(dl)
    print 'Done'
    reactor.stop()

delete_instances()
reactor.run()

EC2 API: Using boto with Nimbula Director

In Nimbula Director 2.0 we introduced our AWS EC2 shim which allows users to interact with a Director deployment using any EC2 compatible tools by implementing both the SOAP and Query protocols. One of the popular AWS tools is the boto library, which provides Python bindings to the AWS API.

To use the AWS API you first need to create authentication credentials. To do this you can run:

$ nimbula-api add accesskey /project/user/mykey

This will create an access and secret key pair with the identifier /project/user/mykey. These can then be used against Director’s AWS API shim. Any commands which are then run against the cloud with these credentials will run with the permissions of the user that created the key. Once you have created the authentication credentials, using the Nimbula EC2 API through boto is simple. Here is an example of a simple boto program to print all of the instances which are running:

import boto
from boto.ec2.regioninfo import RegionInfo
hostname = "api.example.com" # the nimbula API endpoint
region = RegionInfo(name='nimbula', endpoint=hostname)
accesskey = '< your access key >'
secretkey = '< your secret key >'

connection = boto.connect_ec2(aws_access_key_id=accesskey,
    aws_secret_access_key=secretkey, is_secure=False,
    region=region, path="/aws/")
print connection.get_all_instances()

Check it out yourself!

So that should all be enough to try it out for yourself. These examples have only shown a few of the objects you can interact with, however there are many more. A great way to explore the Python bindings is with ipython and its tab completion: simply import the models module in nimbula.api.models and see which objects are available.

Alternatively you can read the source distributed alongside the tools (download here), under tools/source/.

Some tutorials that have grown out of these conversations include the following topics:

• How to create Windows Server 2008 based Nimbula compatible images with optimized VirtIO drivers
  http://nimbula.com/forum/topic/26/
• Sample demo PHP and Javascript code of creating your own Web Interface
  http://nimbula.com/forum/topic/43/
• Script to add high availability to running Nimbula Director virtual machines
  http://nimbula.com/forum/topic/41/
• Guide to Converting a Windows Server 2008 VMware Virtual Machine Image to KVM Format
  http://nimbula.com/forum/topic/40/
• Nimbula Director Install Setup Checklist
  http://nimbula.com/forum/topic/1/

We started off small by installing Nimbula Director on a few machines and created our private cloud for the engineering team. Each engineer has a laptop and they utilized the dogfood private cloud for their dev/test work.

The diagram above shows a single site install of our dogfood environment. A Nimbula Director installation is comprised of the concepts of a site, clusters, and nodes.

Site: The site is a single Nimbula Director installation at a single location.

Cluster:  One or more networked collection of computers or nodes is called a cluster.  A cluster usually runs on a single broadcast domain. A site can have one or more clusters. A cluster is uniquely identified by its IP subnet.

Node: A single server, which participates in a cluster.

Getting our network ready for the cloud should ideally be a one time thing, after which the Cloud Operating System gives you the flexibility and elasticity to adapt to change. Example: the Nimbula Director administrator can add and remove clusters, nodes or sites dynamically by using a restful API or the CLI.

Since engineers manage this private cloud, ‘self-service and automation’ has become the essence of the product starting right from the installation stage. Check out how easy it is to install Nimbula Director to create your private cloud in the video here.

Using our own product for our day-to-day engineering operations has accelerated our understanding of customer requirements. This has led to organic growth of innovative ideas and has enhanced our understanding of the importance of usability, quality and documentation of the product. If you can’t eat it, expect no one else to…

In the next few blog posts I will talk about how we are feeding our way to a robustly engineered product by using the features of Nimbula Director in our dogfood environment.

Various members of our engineering team will be contributing posts to this blog. Over time you will find technical dives into aspects of Nimbula Director; rationales for engineering choices; analysis and investigation of available technologies; and other miscellaneous thoughts from the team.