What is LIO target? Linux-OI Target is a Linux SCSI target introduced in a kernel v.2.6.38 and supports different fabrics modules like FibreChannel, iSCSI, iSER, etc. It works in a kernel space, so it’s faster than tgtd which is used in Cinder by default. Why do we still use tgtd instead of more faster LIO in Cinder by default? It’s only because we have to support rolling upgrades and we don’t know how to migrate from TGTd to LIO in a such way and pass Grenade successfully.

We’ve got non-voting gate-tempest-dsvm-full-lio-ubuntu-xenial job for a while. Due to some of my performance tests results it’s really faster than tgtd. So, how can you use it?

It’s pretty easy with LVM + Devstack. Everything you need is to add 'CINDER_ISCSI_HELPER=lioadm' to your localrc/local.conf.

If you have already configured Cinder+LVM it’s easy too to switch to the new target driver. I mean that you don’t have any in-use volume now but you have Cinder with LVM configured and running. Just follow these steps:

1) first of all, you have to install ‘rtslib-fb’ package using pip:

# pip install rtslib-fb

or using OS package manager:

# apt-get install python-rtslib-fb 

2) stop tgt:

# sudo service tgt stop

3) change /etc/cinder/cinder.conf to use LIO driver:

Set ‘iscsi_helper = lioadm' instead of ‘iscsi_helper = tgtadm


4) restart cinder and enjoy it!


OpenStack Cinder provides an API to attach/detach volume to Nova instances. This is public, but not documented API which is used only by Nova now.  In scope of “Attach/detach volumes without Nova” [1] blueprint we introduce new python-cinderclient extension to provide attach/detach API not only for Nova called python-brick-cinderclient-ext. Before Mitaka release everybody who want to use Cinder volumes not only with Nova instances have to create hardening scripts based on python-cinderclient and os-brick [3] projects to make it done.

Since Mitaka, Cinder opens attach/detach API for any users. It will allow to:


  • Attach volume to Ironic instance
  • Attach volume to any virtual/baremetal host which is not provisioned by Nova or Ironic


It means, Cinder becomes stand-alone project that could be used outside OpenStack cloud with one limitation: Keystone is still required.

For now, python-brick-cinderclient-ext has only ‘get-connector’ API. Attach/detach features are under development and any feedback are welcome to get implemented in the best way. I hope, it will be implemented and documented as well in scope of Mitaka release cycle.

I will show you how it works in current proof-of-concept code [4]. Anybody is welcome to review and test it:).

To demonstrate this feature I will use virtual Devstack environment with Ironic+Cinder. Here is my local.conf [5].

Current limitations are:


  • Ironic instance must have access to API and storage networks (it works on Devstack with a default configuration
  • Users inside instance must have root permissions and be able to install required software


Detailed manual how to setup Ironic using Devstack could be found here [6]. Since volumes attach/detach operations require python, open-iscsi, udev and other packages I will use Ubuntu-based image for Ironic instances. You can use Ubuntu cloud image [7] or build your own using ‘disk-image-builder’ tool [8]. I’ve built my Ubuntu image with disk-image-builder:

$  disk-image-create ubuntu vm dhcp-all-interfaces grub2 -o ubuntu-image
$  glance image-create --name ubuntu-image --visibility public \
--disk-format qcow2 \
--container-format bare < ubuntu-image.qcow2

After it we need to run Ironic instance:

#  query the image id of the default cirros image
$  image=$(nova image-list | egrep "ubuntu" | awk '{ print $2 }')
#  create keypair
$  ssh-keygen
$  nova keypair-add default --pub-key ~/.ssh/id_rsa.pub # spawn instance $ prv_net_id=$(neutron net-list | egrep "$PRIVATE_NETWORK_NAME"'[^-]' | awk '{ print $2 }') $ nova boot --flavor baremetal --nic net-id=$prv_net_id --image $image --key-name default testing

Wait until instance is booted and ready [9]:

$  nova list
$ ironic node-list

Now you can connect to the instance using SSH:

$  ssh ubuntu@

By default, in Devstack both Nova and Ironic instances have access to OpenStack APIs.

To attach volume you need to install required packages inside you instance:

$  sudo apt-get install -y open-iscsi udev python-dev python-pip git

NOTE: if you can't acces Internet inside your instance, try the following command on the DevStack host:

$  sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERAD

Clone and install the latest python-cinderclient (the latests version from PyPi will also work but you'll need to pass --os-volume-api-version explicit):

$  git clone https://github.com/openstack/python-cinderclient.git
$  cd python-cinderclient
$  sudo pip install .

Clone the python-brick-cinderclient-ext and apply the patch:

$  git clone https://github.com/openstack/python-brick-cinderclient-ext.git
$  cd python-brick-cinderclient-ext
$  git fetch https://review.openstack.org/openstack/python-brick-cinderclient-ext refs/changes/44/263744/8 && git checkout FETCH_HEAD
$  sudo pip install .

That’s all! Now, you can attach/detach volumes inside your instance. Because it is still PoC implementation you need few additional steps:

$  PATH=$PATH:/lib/udev
$  export PATH

The steps above is needed until python-brick-cinderclient-ext will use oslo.rootwrap or privsep libraries.

Verify, that python-brick-cinderclient-ext works well [10] (you need to setup your own credentiala and auth_url):

$  cat << EOF >> ~/openrc
 #!/usr/bin/env bash
export OS_AUTH_URL=""
export OS_NO_CACHE="1"
export OS_PASSWORD="password"
export OS_REGION_NAME="RegionOne"
export OS_TENANT_NAME="admin"
export OS_USERNAME="admin"
$  source ~/openrc
$  sudo -E cinder get-connector

You should get something this: [11].

Finally, create and attach volume to your Ironic instance:

$ cinder create 1
$ sudo -E PATH=$PATH cinder local-attach 0a946c67-2d5c-4413-b8ec-350240e967d2

You should get something like: [12]

Now you can verify that volume is attached via iSCSI protocol [13]:

$  sudo iscsiadm -m session
$  ls -al /dev/disk/by-path/ip-

Detach is also easy:

$  sudo -E PATH=$PATH cinder local-detach 0a946c67-2d5c-4413-b8ec-350240e967d2

That’s all! You’ve got attached your Cinder volume to an Ironic instance without Nova! You can do the same steps to attach volumes inside Nova instance or your desktop. It will work too. I will show you a demo with Nova instance and cloud config scrips in the next post.


[1] https://github.com/openstack/cinder-specs/blob/master/specs/mitaka/use-cinder-without-nova.rst
[2] https://github.com/openstack/python-brick-cinderclient-ext
[3] https://github.com/openstack/os-brick
[4] https://review.openstack.org/263744
[5] https://gist.github.com/e0ne/2579921aba839322decc
[6] http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack
[7] https://cloud-images.ubuntu.com/
[8] http://docs.openstack.org/developer/ironic/deploy/install-guide.html#image-requirements
[9] http://paste.openstack.org/show/483734/
[11] http://paste.openstack.org/show/483742/
[12] http://paste.openstack.org/show/483743/

Other useful links: