How do I extend an OpenStack cluster with extra controller/network nodes?

Scenario:

You have installed OpenStack with one controller. Now you want to add two more controllers to expand the database cluster.

Adding one more controller/network to your OpenStack cluster can be done by following these steps:

1. Decide on the nodes you want to use as controller/network nodes

2. Make sure the interfaces are configured on the network nodes that you want to add

For example :

If you have a node001 with two interfaces -- one interface for the external OpenStack network, and the other interface for the internal network -- then make sure that you match the interfaces configuration on your new nodes.

Now, append both the  OpenStackControllers and OpenStackNetworkNodes configuration overlay to the new nodes:

[am-c-02-08-t-c7u2-1->configurationoverlay]% append openstackcontrollers nodes node002,node003

[am-c-02-08-t-c7u2-1->configurationoverlay*]% append openstacknetworknodes nodes node002,node003

[am-c-02-08-t-c7u2-1->configurationoverlay*]% commit

Make sure you restart both of your new nodes after you commit:

[am-c-02-08-t-c7u2-1->device]% reboot -n node002,node003

While you wait for your two nodes to be rebooted, log into your first controller/network node.

Then install xtradbackup,

https://www.percona.com/doc/percona-xtrabackup/2.2/installation/yum_repo.html

on the controller/network node.

After installation,  execute the following :

On your head node :

#cat /root/.galerarc

#ssh controller-1

[root@node001 ~]# mkdir -p /cm/shared/apps/openstack/backup/

[root@node001 ~]# innobackupex --galera-info --user=root --password=password --socket=/var/lib/mysql/mysql.sock /cm/shared/apps/openstack/backup/

The previous command will create a backup directory under /cm/shared/apps/openstack/backup/

with a name derived from the date and time of the day.

[root@node001 ~]# innobackupex -apply-log --galera-info --user=root --password=2O1711H542KV24Kg --socket=/var/lib/mysql/mysql.sock /cm/shared/apps/openstack/backup/BACKUP DIRECTORY/

First we will add a file to this backup before we stream it to the new nodes

1. [root@node001 2017-02-08_12-03-20]# cat xtrabackup_galera_info

2. #touch grastate.dat

3. #cat  xtrabackup_galera_info
   7c6112b1-ede7-11e6-9afb-fb7093d22da5:6336

4. The content of xtrabackup_galera_info should be used to construct the content of grastate.dat, which should look like this:

# GALERA saved state

version: 2.1

uuid:    7c6112b1-ede7-11e6-9afb-fb7093d22da5

seqno: 6336

safe_to_bootstrap: 0

Notice that 6336 is the number after “:” in xtrabackup_galera_info

Now we need to copy this backup to /var/lib/mysql to both of your new nodes

#ssh node002

#cd /var/lib/mysql

# rm -rf *

#rsync -avzp --progress /cm/shared/apps/openstack/backup/2017-02-08_12-03-20/ .

#chown -R mysql.mysql /var/lib/mysql

Repeat the previous steps to also rsync it over to your third node

When done, go back to your head node and execute the following :

[root@am-c-02-08-t-c7u2-1 ~]# cmsh

[am-c-02-08-t-c7u2-1]% configurationoverlay

[am-c-02-08-t-c7u2-1->configurationoverlay]% append galeranodes nodes node002..node003

[am-c-02-08-t-c7u2-1->configurationoverlay*]% commit

To confirm the controllers are there, please log into any of your new controllers and execute the following :

[root@node001 ~]# mysql -uroot -ppassword

MariaDB [(none)]> show status like 'wsrep_cluster_size';

Your output should be :

+--------------------+-------+

| Variable_name      | Value |

+--------------------+-------+

| wsrep_cluster_size | 3     |

+--------------------+-------+

1 row in set (0.00 sec)