Configuring the block storage (cinder) in openstack - controller part

Cinder is the block storage server in openstack configuration . This setup is designed to consume storage devices either from local storage through  LVM or third party devices for the computer node NOVA.

In our case we will use the default LVM as a storage device which will share to the instances which we created earlier . ( as we don't have separate storage node configured we will use controller node as storage) 

The cinder  architecture 

Cinder-API - This is a wsgi based api which will routes and authenticate the requests to the block storage service. It supports openstack API which will call as a cinder client ( Nova EC2 instances also supports this as an alternate)

Cinder -Scheduler - This will schedule the  requests and route it to appropriate volume service as per your configuration .

Cinder Volume - This is the back end storage devices . typically the devices which are supported are given below

• Ceph RADOS Block Device (RBD)
• Coraid AoE driver configuration
• Dell EqualLogic volume driver
• EMC VMAX iSCSI and FC drivers
• EMC VNX direct driver
• EMC XtremIO OpenStack Block Storage driver guide
• GlusterFS driver
• HDS HNAS iSCSI and NFS driver
• HDS HUS iSCSI driver
• Hitachi storage volume driver
• HP 3PAR Fibre Channel and iSCSI drivers
• HP LeftHand/StoreVirtual driver
• HP MSA Fibre Channel driver
• Huawei storage driver
• IBM GPFS volume driver
• IBM Storwize family and SVC volume driver
• IBM XIV and DS8000 volume driver
• LVM
• NetApp unified driver
• Nexenta drivers
• NFS driver
• ProphetStor Fibre Channel and iSCSI drivers
• Pure Storage volume driver
• Sheepdog driver
• SolidFire
• VMware VMDK driver
• Windows iSCSI volume driver
• XenAPI Storage Manager volume driver
• XenAPINFS
• Zadara
• Oracle ZFSSA iSCSI Driver

Cinder backup - Provides the cinder back up to various targets.

Cinder work flow 

• A volume is created through the cinder create command. This command creates an LV into the volume group (VG) “cinder-volumes.”
• The volume is attached to an instance through the nova volume-attach command. This command creates a unique iSCSI IQN that is exposed to the compute node.
• The compute node, which runs the instance, now has an active iSCSI session and new local storage (usually a /dev/sdX disk).
• Libvirt uses that local storage as storage for the instance. The instance get a new disk, usually a /dev/vdX disk.

While entering to the configuration side we have two parts for the configuration . 

configuration of the controller side and configuration of the storage node side. But as i mentioned earlier we have no storage node configured separately due to the limitation in my lab and we will configure both in controller node  

Configuring the controller node for cinder setup

•  Login to the CTRL node as root
• Create the databases for cinder service 

root@CTRL:~# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 27
Server version: 5.5.44-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE DATABASE cinder;
Query OK, 1 row affected (0.00 sec)

• Provide the proper access to the cinder database and set the password for the cinder DB.

mysql> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY 'Onm0bile';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%'  IDENTIFIED BY 'Onm0bile';
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye
root@CTRL:~#

• Now we need to access the admin commands using source the admin.rc file 

root@CTRL:~# cat admin.rc
export OS_USERNAME=admin
export OS_PASSWORD=password
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://CTRL:35357/v2.0
root@CTRL:~# source admin.rc
root@CTRL:~#

• Create the service credentials for cinder using keystone command . We need to create cinder user

root@CTRL:~# keystone user-create --name cinder --pass Onm0bile
+----------+----------------------------------+
| Property |              Value               |
+----------+----------------------------------+
|  email   |                                  |
| enabled  |               True               |
|    id    | c1791460385745f79015a4ee40f94db8 |
|   name   |              cinder              |
| username |              cinder              |
+----------+----------------------------------+
root@CTRL:~#

• Add admin role to the cinder user 

root@CTRL:~# keystone user-role-add --user cinder --tenant service --role admin
root@CTRL:~#

• Create the cinder service entities for both cinder API V1 and V2 

(currently the block storage API versions are up to 3 and we will only use 2)

root@CTRL:~# keystone service-create --name cinder --type volume --description "OpenStack Block Storage"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |     OpenStack Block Storage      |
|   enabled   |               True               |
|      id     | 6c91s86b3acb23d2b1294171c14fed68 |
|     name    |              cinder              |
|     type    |              volume              |
+-------------+----------------------------------+
root@CTRL:~#
root@CTRL:~# keystone service-create --name cinderv2 --type volumev2 --description "OpenStack Block Storage"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |     OpenStack Block Storage      |
|   enabled   |               True               |
|      id     | 414d7125e8e44314ce58beb8fc4ca781|
|     name    |             cinderv2             |
|     type    |             volumev2             |
+-------------+----------------------------------+
root@OSCTRL-UA:~#

• Create the API storage end points for version1 and version 2

keystone endpoint-create --service-id $(keystone service-list | awk '/ volume / {print $2}') --publicurl http://CTRL:8776/v1/%\(tenant_id\)s --internalurl http://CTRL:8776/v1/%\(tenant_id\)s --adminurl http://CTRL:8776/v1/%\(tenant_id\)s --region regionOne

keystone endpoint-create --service-id $(keystone service-list | awk '/ volume / {print $2}') --publicurl http://CTRL:8776/v2/%\(tenant_id\)s --internalurl http://CTRL:8776/v2/%\(tenant_id\)s --adminurl http://CTRL:8776/v2/%\(tenant_id\)s --region regionOne

root@CTRL:~# keystone endpoint-create --service-id $(keystone service-list | awk '/ volume / {print $2}') --publicurl http://CTRL:8776/v1/%\(tenant_id\)s --internalurl http://CTRL:8776/v1/%\(tenant_id\)s --adminurl http://CTRL:8776/v1/%\(tenant_id\)s --region regionOne
+-------------+----------------------------------------+
|   Property  |                 Value                  |
+-------------+----------------------------------------+
|   adminurl  | http://CTRL:8776/v1/%(tenant_id)s |
|      id     |    6c91s86b3acb23d2b1294171c14fed68   |
| internalurl | http://CTRL:8776/v1/%(tenant_id)s |
|  publicurl  | http://CTRL:8776/v1/%(tenant_id)s |
|    region   |               regionOne                |
|  service_id |    7a90b86b3aab43d2b1194172a14fed79    |
+-------------+----------------------------------------+
root@CTRL:~#
root@CTRL:~# keystone endpoint-create --service-id $(keystone service-list | awk '/ volumev2 / {print $2}') --publicurl http://CTRL:8776/v1/%\(tenant_id\)s --internalurl http://CTRL:8776/v1/%\(tenant_id\)s --adminurl http://CTRL:8776/v1/%\(tenant_id\)s --region regionOne
+-------------+----------------------------------------+
|   Property  |                 Value                  |
+-------------+----------------------------------------+
|   adminurl  | http://CTRL:8776/v1/%(tenant_id)s |
|      id     |       414d7125e8e44314ce58beb8fc4ca781
| internalurl | http://CTRL:8776/v1/%(tenant_id)s |
|  publicurl  | http://CTRL:8776/v1/%(tenant_id)s |
|    region   |               regionOne                |
|  service_id |    716e7125e8e44414ad58deb9fc4ca682    |
+-------------+----------------------------------------+
root@OSCTRL-UA:~#

• Next we have to install the cinder components 

root@CTRL:~# yum install cinder-api cinder-scheduler python-cinderclient
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-cinderclient is already the newest version.
python-cinderclient set to manually installed.
The following extra packages will be installed:
  cinder-common python-barbicanclient python-cinder python-networkx
  python-taskflow
Suggested packages:
  python-ceph python-hp3parclient python-scipy python-pydot
The following NEW packages will be installed:
  cinder-api cinder-common cinder-scheduler python-barbicanclient
  python-cinder python-networkx python-taskflow
0 upgraded, 7 newly installed, 0 to remove and 37 not upgraded.
Need to get 1,746 kB of archives.
After this operation, 14.0 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

• Edit the /etc/cinder/cinder.conf file and configure as below 

Database session 

[database]
connection = mysql://cinder:Onm0bile@CTRL/cinder

Rabbit MQ configuration 

[DEFAULT]
...
rpc_backend = rabbit
rabbit_host = CTRL
rabbit_password = Onm0bile

Update the auth_stratery in default session 

[DEFAULT]
auth_strategy = keystone

Update the keystone credentials 

[keystone_authtoken]
auth_uri = http://CTRL:5000/v2.0
identity_uri = http://CTRL:35357
admin_tenant_name = service
admin_user = cinder
admin_password = Onm0bile

Update the my_ip option to access the management IP of the controller node 

[DEFAULT]
.....
my_ip = 192.168.24.10

Enable the verbose 

[DEFAULT]
.....
verbose = True

Populate the configuration in cinder database 

root@CTRL:~# su -s /bin/sh -c "cinder-manage db sync" cinder
2017-11-07 04:37:00.143 9423 INFO migrate.versioning.api [-] 0 -> 1...
2017-11-07 04:37:00.311 9423 INFO migrate.versioning.api [-] done
2017-11-07 04:37:00.312 9423 INFO migrate.versioning.api [-] 1 -> 2...
2017-11-07 04:37:00.424 9423 INFO migrate.versioning.api [-] done

....out put is omitted....

Restart the cinder services once the database update finishes 

root@CTRL:~# service cinder-scheduler restart
cinder-scheduler stop/waiting
cinder-scheduler start/running, process 9444
root@CTRL:~# service cinder-api restart
cinder-api stop/waiting
cinder-api start/running, process 9466
root@CTRL:~#

I will cover the storage part on next session 

Configuring the neutron in openstack

Neutron is the networking component of the openstack setup which will manage the inward and outward traffic to and from the instances to the external network. The benefits of neutron is it will act as a network connectivity service and supports many L2 and L3 technologies. It is easy to manage as we can deploy in centralised setup also it can be deployed as distributed setup.The advanced technologies which includes in neutron  are like load balancing,  VPN, firewall etc.

The basic neutron process is given below 

1. Boot VM start.
2. Create a port and notify the DHCP of the new port.
3. Create a new device (virtualization library – libvirt).
4. Wire port (connect the VM to the new port).
5. Complete boot.

Neutron server contains mainly 3 components 

• REST API - This is a http based API service which includes methods, url's, media types , responses etc. It also exposes logical resources like subnets & ports 
• Queue - This handles bi directional communication between agents and the neutron server. 
• Plugin - This component will communicate with plugin agents installed in instances to manage vswitch configuration . Also this will help neutron server to access the database persistently using the AMQP protocol.

Basic architecture of a neutron server is below 

Detailed architecture of neutron setup is below

Steps to configure the neutron 

• First we need to create an external network which is called provider network in controller node

command format is below 

neutron net-create <NET-NAME> --provider:physical_network=<LABEL-PHYSICAL-INTERFACE> --provider:network_type=<flat or vlan> --shared --router:external=True




root@CTRL:~# neutron net-create ext-net --router:external --provider:physical_network external --provider:network_type flat
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | f26chf4c-5c46-2881-c0h0-0845918d6536 |
| name                      | ext-net                              |
| provider:network_type     | flat                                 |
| provider:physical_network | external                             |
| provider:segmentation_id  |                                      |
| router:external           | True                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tenant_id                 | c15d8b07h462481348c3f3c4e8d581c7    |
+---------------------------+--------------------------------------+
root@CTRL:~#

• Next step is to assign the IP pool for the external network router and interfaces to avoid the IP conflict . In our case i am assigning the IP pool starting from 192.168.24.10 to 192.168.24.30 and the default gateway is 192.168.24.2 

command format 

neutron subnet-create --name <SUBNET-NAME> <NET-NAME> <SUBNET-CIDR> --gateway <GATEWAY-IP> --allocation-pool start=<STARTING-IP>,end=<ENDING-IP> --dns-nameservers list=true <DNS-1 DNS-2>

root@CTRL:~# neutron subnet-create ext-net --name ext-subnet --allocation-pool start=192.168.24.10,end=192.168.24.30 --disable-dhcp --gateway 192.168.24.2 192.168.24.0/24
Created a new subnet:
+-------------------+--------------------------------------------------------+
| Field             | Value                                                  |
+-------------------+--------------------------------------------------------+
| allocation_pools  | {"start": "192.168.24.10", "end": "192.168.24.30"} |
| cidr              | 192.168.24.0/24                                       |
| dns_nameservers   |                                                        |
| enable_dhcp       | False                                                  |
| gateway_ip        | 192.168.24.2                                          |
| host_routes       |                                                        |
| id                |  f26chf4c-5c46-2881-c0h0-0845918d6536                  
| ip_version        | 4                                                      |
| ipv6_address_mode |                                                        |
| ipv6_ra_mode      |                                                        |
| name              | ext-subnet                                             |
| network_id        | 2d188736-5877-77df-bc8c-eb1964c4a74a                   |
| tenant_id         | c15d8b07h462481348c3f3c4e8d581c7                      |
+-------------------+--------------------------------------------------------+
root@CTRL:~# 

• Next step is to create a Tenant network , we have created a tenant earlier as tuxfixer . We have to source that tuxfixer.rc file 

root@CTRL:~# cat tuxfixer.rc
export OS_USERNAME=tuxfixer
export OS_PASSWORD=tux123
export OS_TENANT_NAME=tuxfixer
export OS_AUTH_URL=http://CTRL:35357/v2.0
root@CTRL:~#
root@CTRL:~# source tuxfixer.rc

command format

neutron net-create <NET-NAME>
neutron subnet-create --name <SUBNET-NAME> <NET-NAME> <SUBNET-CIDR>

root@CTRL:~# neutron net-create tuxfixer-net
Created a new network:
+-----------------+--------------------------------------+
| Field           | Value                                |
+-----------------+--------------------------------------+
| admin_state_up  | True                                 |
| id              | 2c0dh763-3fd4-2f8c-743f-7h0j35cv6cde |
| name            | tuxfixer-net                          |
| router:external | False                                |
| shared          | False                                |
| status          | ACTIVE                               |
| subnets         |                                      |
| tenant_id       | dbe3cf30f46b446fcfe84b205459780d     |
+-----------------+--------------------------------------+

Now create a subnet for the tenant tuxfixer

The tuxfixer tanent can use the ip starting from 192.168.5.2 to 192.168.5.254

root@CTRL:~# neutron subnet-create tuxfixer-net --name tuxfixer-subnet --gateway 192.168.5.1 192.168.5.0/24
Created a new subnet:
+-------------------+--------------------------------------------------+
| Field             | Value                                            |
+-------------------+--------------------------------------------------+
| allocation_pools  | {"start": "192.168.5.2", "end": "192.168.5.254"} |
| cidr              | 192.168.5.0/24                                   |
| dns_nameservers   |                                                  |
| enable_dhcp       | True                                             |
| gateway_ip        | 192.168.5.1                                      |
| host_routes       |                                                  |
| id                | ac05bc74-eade-4811-8e7b-8de021abe0c1             |
| ip_version        | 4                                                |
| ipv6_address_mode |                                                  |
| ipv6_ra_mode      |                                                  |
| name              | tuxfixer-subnet                                   |
| network_id        | 2c0dh763-3fd4-2f8c-743f-7h0j35cv6cde            |
| tenant_id         | dbe3cf30f46b446fcfe84b205459780d                |
+-------------------+--------------------------------------------------+

• We have to create a tanent router and add the internal and external interfaces to that .

command details

neutron router-create <ROUTER-NAME>

neutron router-interface-add <ROUTER-NAME> <SUBNET-NAME>

neutron router-gateway-set <ROUTER-NAME> <NET-NAME>

root@CTRL:~# neutron router-create tuxfixer-router
Created a new router:
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| admin_state_up        | True                                 |
| external_gateway_info |                                      |
| id                    | 1e4g48d3-a9d0-3567-3f1c-29cd8b83345d |
| name                  | tuxfixer-router                       |
| routes                |                                      |
| status                | ACTIVE                               |
| tenant_id             | dbe3cf30f46b446fcfe84b205459780d     |

root@CTRL:~# neutron router-interface-add tuxfixer-router tuxfixer-subnet
Added interface 445d79cb-3dcf-5f88-963c-aa054f7ce758 to router tuxfixer-router.

root@CTRL:~# neutron router-gateway-set tuxfixer-router ext-net
Set gateway for router tuxfixer-router

Now we need to list the newly created router details. We have 2 subnets configured where 1 will use for tanent and other will be for external

root@CTRL:~# neutron router-port-list tuxfixer-router
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                          |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+
|  |1e4g48d3-a9d0-3567-3f1c-29cd8b83345d       | fc:16:4d:13:32:21 | {"subnet_id": "f6523637-7162-449d-b12c-e1f0eda6196d", "ip_address": "192.168.5.1"} |

We can verify the work by pinging to the external and tanent IP from controller node 

First we need to verify the router interfaces which we created for tuxfixer 

root@CTRL:/var/log/neutron# ip netns
efccp-49ff7852-07c4-30d2-82cb-e6f7daf673a4
qrouter-43681237-d673-5e1b-ca04-7e4672274992

Now ping the external IP using below command 

root@CTRL:~# ip netns exec qrouter-43681237-d673-5e1b-ca04-7e4672274992 ping 192.168.24.30

PING 192.168.24.30 (192.168.24.30) 56(84) bytes of data.
64 bytes from 192.168.24.30: icmp_seq=1 ttl=64 time=0.165 ms
64 bytes from 192.168.24.30: icmp_seq=2 ttl=64 time=0.126 ms
64 bytes from 192.168.24.30: icmp_seq=3 ttl=64 time=0.082 ms
^C

Ping the Tanent IP 

root@CTRL:~# ip netns exec qrouter-43681237-d673-5e1b-ca04-7e4672274992 ping 192.168.5.1

PING 192.168.5.1 (192.168.5.1) 56(84) bytes of data.
64 bytes from 192.168.5.1: icmp_seq=1 ttl=64 time=0.165 ms
64 bytes from 192.168.5.1: icmp_seq=2 ttl=64 time=0.126 ms
64 bytes from 192.168.5.1: icmp_seq=3 ttl=64 time=0.082 ms
^C

Basic neutron configuration is completed except the security groups which i will discuss as separately

Creating and launching instances in Horizon

Here i am providing the steps to create and launch instances and tenant using horizon dash board in open stack

• As per the architecture given below we have configured the horizon dash board using the ip 192.168.2.14

• We have to type http://192.168.2.14/dashboard to get the horozon as below and we need to login as per the credentials mentioned in /root/keystonrc file . In this case it is admin/password

• In the dashboard menu go to Identity/projects/create project and type the project name , i am going  to give the sample as tuxfixer here .

• Create a new user and assign him a the new tenant ( here tuxfixer) . Go to : Identity/users/create user assign user as a member to the newly created tenant tuxfixer 

• Logout from the admin and login as a new user (tuxfixer) here and create new networks and routes go to Project/networks/create network 

below details need to be given to create the private & public network


Create following net:
Network Name: priv_net
Admin State: UP

Create following subnet:
Subnet Name: priv_subnet
Network Address: 192.168.20.0/24
IP Version: IPv4
Gateway IP: 192.168.20.1
DHCP: Enable

Now let’s create public network. Click the Create Network button once again:

Create following net:
Network Name: pub_net
Admin State: UP

Create following subnet:
Subnet Name: pub_subnet
Network Address: 192.168.2.0/24
IP Version: IPv4
Gateway IP: 192.168.2.1
DHCP: Disable
Allocation Pools: 192.168.2.6 – 192.168.2.10

• So we should have 2 networks in our dash board priv_net and pub_net 

• Now we have to update the pub_net to public network which can be performed only as admin. So again we need to logout from tuxfixer and login as admin . Then go to dashboard of Admin/networks/ edit the network of pub_net and select the external network check box and save the changes 

Logout from the admin and login as tuxfixer again and now we need to create the router for internal to external connectivity part

Go to Project/network/routers/create router

router name:  router0
Click Set Gateway for router0:
External Network: pub_net
In the dashboard we should see now our router0 with external Network set for pub_net

Click on the router then Add Interface
Subnet: priv_net: 192.168.20.0/24 (priv_subnet)
IP Address: 192.168.20.1

We will get the below diagram while we check the network topology

Now we need to create flavors as per our requirement , which allows user's to select different templates of instances with predefined resources . Flavors are defined as per below criteria
– Virtual CPUs
– RAM
– disk size
– swap size

Open stack comes with many predefined flavors which can be accessed from Admin/System/Flavors

 

• Upload image - Open stack uses different types of images as per the requirement like VDI,QCOW2,ISO,VMDK etc .  I am using cirros_mage due to its small size which is good for cloud environments.

Current Cirros Image (cirros-0.3.4-x86_64-disk.img) available at: http://download.cirros-cloud.net.

Image details:
Name: cirros
Description: cirros image
Image Source: Image File
Image File: cirros-0.3.4-x86_64-disk.img
Architecture: (leave blank)
Minimum Disk: (leave blank)
Minimum RAM: (leave blank)
Public: No
Protected: No

• Create security groups - The basic functionality of the security group is to control the network access inside the tenant . 

Go to: Project / Compute / Access & Security / Create Security Group.

Security Group details:
Name: tuxfixer_sec_group
Description: tuxfixer security group

In the tuxfixer security group row click on manage rules and add the following rules 

Egress IPv4 ICMP – 0.0.0.0/0 (CIDR)
Ingress IPv4 ICMP – 0.0.0.0/0 (CIDR)
Ingress IPv4 TCP 1 – 65535 0.0.0.0/0 (CIDR)
Egress IPv4 TCP 1 – 65535 0.0.0.0/0 (CIDR)

Assign floating IP's to the tenant . Floating IP's are used to access the instances from public network 

To assign new IPs to the Project Tanant go to: Project -> Compute -> Access & Security -> Floating IPs -> Allocate IP to The Project

Allocate 2 Floating IPs:
Pool: pub_net

• Create instances - 

Now when we have all the need components, we can create Instances (virtual machines running inside cloud).
Let’s create 2 Instances, go to: Project -> Compute -> Instances -> Launch Instance

Details Tab:
Availability Zone: nova
Instance Name: cirros_instance
Flavor: m1.tiny
Instance Count: 2
Instance Boot Source: Boot from image
Image Name: cirros

Access & Security Tab:
Key Pair: No key pairs available
Security Groups: tuxfixer_sec_group

Networking Tab:
Selected networks: priv_net

Post-Creation Tab: leave as it is

Advanced Options Tab: leave as it is

Note: Process of creating Instances may take couple of minutes.

View of created instances:

• We have running instances and floating IP's assined to the project . We can assign the floating IP to a particular instance by selecting the Actions column choose Associate Floating IP from drop down menu.

• We can access the console by clicking the console tab 

Login to instance console:
instance login: cirros
instance password: cubswin:)

• Also test the connectivity from controller and compute node using the ssh login 

[root@ctrl ~]# ping 192.168.2.6
PING 192.168.2.6 (192.168.2.6) 56(84) bytes of data.
64 bytes from 192.168.2.6: icmp_seq=1 ttl=63 time=17.0 ms
64 bytes from 192.168.2.6: icmp_seq=2 ttl=63 time=1.08 ms
64 bytes from 192.168.2.6: icmp_seq=3 ttl=63 time=0.707 ms
64 bytes from 192.168.2.6: icmp_seq=4 ttl=63 time=0.666 ms
^C
--- 192.168.2.6 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 0.666/4.872/17.030/7.021 ms
[root@ctrl ~]# ping 192.168.2.7
PING 192.168.2.7 (192.168.2.7) 56(84) bytes of data.
64 bytes from 192.168.2.7: icmp_seq=1 ttl=63 time=14.8 ms
64 bytes from 192.168.2.7: icmp_seq=2 ttl=63 time=1.08 ms
64 bytes from 192.168.2.7: icmp_seq=3 ttl=63 time=0.730 ms
64 bytes from 192.168.2.7: icmp_seq=4 ttl=63 time=0.943 ms
^C
--- 192.168.2.7 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 0.730/4.411/14.886/6.049 ms

2 node open stack installation in CentOS 7

Open Stack is an open source private cloud computing platform developed by joint project of rack space hosting and NASA.Users mainly use it as a infrastructure as a service (Iaas). This plat form uses many services as mariadb,rabbitMQ,Linux KVM ,LVM, iscsi etc.

The main open stack components and its functionalities are given below

1.  Horizon: web browser user interface (dashboard) based on Python Django for creating and managing instances (virtual machines)
2.  Keystone: authentication and authorization framework
3.  Neutron: network connectivity as a service
4.  Cinder: persistent block storage for instances based on LVM
5.  Nova: instances management system based on Linux KVM
6.  Glance: registry for instance images
7.  Swift: file storage for cloud
8.  Ceilometer: metering engine for collecting billable data and analysis.
9.  Heat: orchestration service for template-based instance deployment

The Architecture 

Add caption

we have 2 nodes for the setup one is controller node (ctrl.example.com) and compute node (cmp.example.com). Both servers are configured with two nic cards where one is for external connectivity and other is for internal connectivity . The ip addresses are below

1. ctrl.example.com    - 192.168.2.14  for enp0s3 (nated with external network) and enp0s8 ( for internal connectivity without ip address)
2.cmd.example.com - 192.168.2.15 for enp0s3(nated with external network) and enp0s8(for internal connectivity without ip address)

We are installing the openstack in ctrl node and below are the prerequisites

• change the selinux to permissive or disable in both the nodes 

[root@ctrl ~]# getenforce
Permissive

• Disable the firewall in both the nodes 

[root@ctrl ~]# systemctl status firewalld

 firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

• Configure the ipaddress in both nodes and both interfaces 

[root@ctrl ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:cf:f6:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.14/24 brd 192.168.2.255 scope global dynamic eth0
       valid_lft 42065sec preferred_lft 42065sec
    inet6 fe80::5054:ff:fecf:f6ef/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s8:  mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:c6:92:ee brd ff:ff:ff:ff:ff:ff

[root@cmp ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:4d:fa:06 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.15/24 brd 192.168.2.255 scope global dynamic eth0
       valid_lft 42039sec preferred_lft 42039sec
    inet6 fe80::5054:ff:fe4d:fa06/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s8:  mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:fe:a6:c8 brd ff:ff:ff:ff:ff:ff

********************************************************************************

• Map the hostname & ip to the /etc/hosts file

[root@ctrl ~]# cat /etc/hosts

192.168.2.14 ctrl.example.com

192.168.2.15 cmp.example.com

• Update the system in both the nodes
• create the repository 

•   install the openstack packstack in ctrl node

• Now we need to configure the openstack using packstack automated installation for that first we need to create the answer file which contains all configuration parameters  in ctrl node

[root@ctrl ~]# packstack --gen-answer-file=/root/answers.txt

Packstack changed given value  to required value /root/.ssh/id_rsa.pub

[root@ctrl ~]# cd /root/

[root@ctrl ~]# ll answers.txt 

-rw-------. 1 root root 52366 Oct  5 02:05 answers.txt

• Configure the below parameters in /root/answers.txt maually  ( you may need to edit the values) 

CONFIG_NTP_SERVERS=0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org

CONFIG_CONTROLLER_HOST=192.168.2.14

CONFIG_COMPUTE_HOSTS=192.168.2.15

CONFIG_KEYSTONE_ADMIN_PW=password

CONFIG_NEUTRON_ML2_MECHANISM_DRIVERS=openvswitch

CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vlan

CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vlan

CONFIG_NEUTRON_ML2_VLAN_RANGES=physnet1:1000:2000

CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=physnet:br-ex

• Now start the installation using that answer file and it may take some time (more than 1 hour) 

• Once the installation is completed you will get the link for the Horizon dash board

**** Installation completed successfully ******

Additional information:
 * File /root/keystonerc_admin has been created on OpenStack client host 192.168.2.14. To use the command line tools you need to source the file.
 * To access the OpenStack Dashboard browse to http://192.168.2.14/dashboard .
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * Because of the kernel update the host 192.168.2.14 requires reboot.
 * The installation log file is available at: /var/tmp/packstack/20160320-230116-mT1aV6/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20160320-230116-mT1aV6/manifests

• The credentials which is used to login in the Horizon will be stored in /root/keystonrc file. In our case it will be admin/password

• Now we have to create openswitch bridges and bind to the physical interfaces . After the openstack installation below are the interface details (we can see extra openswitch interfaces are added ) 

[root@ctrl ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:cf:f6:ef brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.14/24 brd 192.168.2.255 scope global dynamic eth0
       valid_lft 41134sec preferred_lft 41134sec
    inet6 fe80::5054:ff:fecf:f6ef/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s81:  mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:c6:92:ee brd ff:ff:ff:ff:ff:ff
4: ovs-system:  mtu 1500 qdisc noop state DOWN 
    link/ether 72:b8:b8:de:3a:f7 brd ff:ff:ff:ff:ff:ff
5: br-int:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 0e:f7:ad:b9:21:48 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::cf7:adff:feb9:2148/64 scope link 
       valid_lft forever preferred_lft forever
6: br-enp0s8:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether f2:d0:68:22:b2:46 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f0d0:68ff:fe22:b246/64 scope link 
       valid_lft forever preferred_lft forever
7: br-ex:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 76:7a:de:52:ec:42 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::747a:deff:fe52:ec42/64 scope link 
       valid_lft forever preferred_lft forever

[root@cmp ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:4d:fa:06 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.15/24 brd 192.168.2.255 scope global dynamic eth0
       valid_lft 40548sec preferred_lft 40548sec
    inet6 fe80::5054:ff:fe4d:fa06/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s8:  mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 52:54:00:fe:a6:c8 brd ff:ff:ff:ff:ff:ff
6: ovs-system:  mtu 1500 qdisc noop state DOWN 
    link/ether 2e:11:a9:be:7b:cc brd ff:ff:ff:ff:ff:ff
7: br-int:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether a2:b9:7e:04:cd:48 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a0b9:7eff:fe04:cd48/64 scope link 
       valid_lft forever preferred_lft forever
8: br-enp0s8:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 36:8c:69:06:42:4b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::348c:69ff:fe06:424b/64 scope link 
       valid_lft forever preferred_lft forever

• Take the configuration back up of original interfaces in ctrl node 

cp /etc/sysconfig/network-scripts/ifcfg-enp0s3 /root/ifcfg-enp0s3.backup
cp /etc/sysconfig/network-scripts/ifcfg-enp0s3 /etc/sysconfig/network-scripts/ifcfg-br-ex
cp /etc/sysconfig/network-scripts/ifcfg-enp0s8 /root/ifcfg-enp0s8.backup

• Modify the ifcfg-enp0s3 file and update as below in ctrl node 

DEVICE=enp0s3
HWADDR=52:54:00:CF:F6:EF
ONBOOT=yes

•   Modify the ifcfg-br-ex file as below 

DEVICE=br-ex
TYPE=Ethernet
BOOTPROTO=static
ONBOOT=yes
NM_CONTROLLED=no
IPADDR=192.168.2.14
PREFIX=24

•       Modify ifcfg-enp0s8 file as below in ctrl node

DEVICE=enp0s8
HWADDR=52:54:00:C6:92:EE
TYPE=Ethernet
BOOTPROTO=none
NM_CONTROLLED=no
ONBOOT=yes

• Now its the time to add both the ports to ovs switch using below command 

ovs-vsctl add-port br-ex enp0s3; systemctl restart network

ovs-vsctl add-port br-enp0s8 enp0s8; systemctl restart network

   The network interface structure is given below after adding the ports

[root@ctrl ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000
    link/ether 52:54:00:cf:f6:ef brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fecf:f6ef/64 scope link 
       valid_lft forever preferred_lft forever
3: enp0s8:  mtu 1500 qdisc pfifo_fast master ovs-system state UP qlen 1000
    link/ether 52:54:00:c6:92:ee brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fec6:92ee/64 scope link 
       valid_lft forever preferred_lft forever
4: ovs-system:  mtu 1500 qdisc noop state DOWN 
    link/ether ea:c6:b3:ff:17:ba brd ff:ff:ff:ff:ff:ff
5: br-enp0s8:  mtu 1500 qdisc noop state DOWN 
    link/ether f2:d0:68:22:b2:46 brd ff:ff:ff:ff:ff:ff
6: br-ex:  mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 76:7a:de:52:ec:42 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.14/24 brd 192.168.2.255 scope global br-ex
       valid_lft forever preferred_lft forever
    inet6 fe80::747a:deff:fe52:ec42/64 scope link 
       valid_lft forever preferred_lft forever
7: br-int:  mtu 1500 qdisc noop state DOWN 
    link/ether 0e:f7:ad:b9:21:48 brd ff:ff:ff:ff:ff:ff

Now we can check the ovs configuration in details

[root@ctrl ~]# ovs-vsctl show
0dcba8a0-bebe-4785-82d6-7c67619874cd
    Bridge "br-enp0s8"
        Port "phy-br-enp0s8"
            Interface "phy-br-enp0s8"
                type: patch
                options: {peer="int-br-enp0s8"}
        Port "enp0s8"
            Interface "enp0s8"
        Port "br-enp0s8"
            Interface "br-enps8"
                type: internal
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "enp0s3"
            Interface "enp0s3"
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "int-br-enp0s8"
            Interface "int-br-enp0s8"
                type: patch
                options: {peer="phy-br-enp0s8"}
    ovs_version: "2.1.3"




• Let us configure the same in computer node. Modify the ifcfg-enp0s8 interface as below in computer node

DEVICE=enp0s8
HWADDR=52:54:00:FE:A6:C8
TYPE=Ethernet
BOOTPROTO=none
NM_CONTROLLED=no
ONBOOT=yes

•   Add the interface to ovs switch using below command in comp node

ovs-vsctl add-port br-enp0s8 enp0s8; systemctl restart network

• Finally the configuration in computer node will look like below

[root@cmp ~]# ovs-vsctl show
cc9e8eff-ea10-40dc-adeb-2d6ee6fc9ed9
    Bridge br-int
        fail_mode: secure
        Port "int-br-enp0s8"
            Interface "int-br-enp0s8"
                type: patch
                options: {peer="phy-br-enp0s8"}
        Port br-int
            Interface br-int
                type: internal
    Bridge "br-enp0s8"
        Port "phy-br-enp0s8"
            Interface "phy-br-enp0s8"
                type: patch
                options: {peer="int-br-enp0s8"}
        Port "enp0s8"
            Interface "enp0s8"
        Port "br-enp0s8"
            Interface "br-enp0s8"
                type: internal
    ovs_version: "2.1.3"




• Each openstack installation will create /root/keystonerc_admin file in each nodes and same will be look like below in ctrl node

[root@ctrl ~]# cat /root/keystonerc_admin 
export OS_USERNAME=admin
export OS_TENANT_NAME=admin
export OS_PASSWORD=password
export OS_AUTH_URL=http://192.168.2.14:5000/v2.0/
export OS_REGION_NAME=RegionOne
export PS1='[\u@\h \W(keystone_admin)]\$ '

•      We can sync these variables to OS variables using below method 

[root@ctrl ~]# source /root/keystonerc_admin
[root@ctrl ~(keystone_admin)]# 

•   Now check the service status in ctrl node to make sure all services are running fine 

[root@ctrl ~(keystone_admin)]# openstack-status
== Nova services ==
openstack-nova-api:                     active
openstack-nova-cert:                    active
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
openstack-nova-scheduler:               active
openstack-nova-conductor:               active
== Glance services ==
openstack-glance-api:                   active
...




• Also verify the services in cloud hosts ( execute in ctrl node)

[root@ctrl ~(keystone_admin)]# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth controller                           internal         enabled       2017-10-09 22:27:24
nova-scheduler   controller                           internal         enabled       2017-10-09 22:27:25
nova-conductor   controller                           internal         enabled       2017-10-09 22:27:24
nova-cert        controller                           internal         enabled       2017-10-09 22:27:21
nova-compute     compute                              nova             enabled       2017-10-09 22:27:24

The basic installation of the Open stack in Centos 7 is completed