Test Scaling

More slaves can be added to the MySQL Cluster to increase read capacity. This can be done by following command.

kubectl -n mysql scale statefulset mysql --replicas=5

You can see the message that StatefulSet “mysql” scaled.


statefulset "mysql" scaled

Watch the progress of ordered and graceful scaling.

kubectl -n mysql rollout status statefulset mysql


Waiting for 2 pods to be ready...
Waiting for 1 pods to be ready...
partitioned roll out complete: 5 new pods have been updated...

It may take few minutes to launch all the pods.

Open another terminal to check loop if you closed it.

kubectl -n mysql run mysql-client-loop --image=mysql:5.7 -i -t --rm --restart=Never --\
   bash -ic "while sleep 1; do mysql -h mysql-read -e 'SELECT @@server_id,NOW()'; done"

You will see 5 servers are running.


+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         100 | 2020-01-25 02:32:43 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         102 | 2020-01-25 02:32:44 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         101 | 2020-01-25 02:32:45 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         103 | 2020-01-25 02:32:46 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         104 | 2020-01-25 02:32:47 |
+-------------+---------------------+
+-------------+---------------------+
| @@server_id | NOW()               |
+-------------+---------------------+
|         103 | 2020-01-25 02:32:48 |
+-------------+---------------------+

Verify if the newly deployed slave (mysql-3) have the same data set by following command.

kubectl -n mysql run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never --\
 mysql -h mysql-3.mysql -e "SELECT * FROM test.messages"

It will show the same data that master has.


+--------------------------+
| message                  |
+--------------------------+
| hello, from mysql-client |
+--------------------------+

Scale down replicas to 3 by following command.

kubectl -n mysql  scale statefulset mysql --replicas=3

You can see StatefulSet “mysql” scaled


statefulset "mysql" scaled

Note that scale in doesn’t delete the data or PVCs attached to the pods. You have to delete them manually.

Check scale in is completed by following command.

kubectl -n mysql get pods -l app=mysql


NAME      READY     STATUS    RESTARTS   AGE
mysql-0   2/2       Running   0          1d
mysql-1   2/2       Running   0          1d
mysql-2   2/2       Running   0          35m

Check data-mysql-3, data-mysql-4 PVCs still exist by following command.

kubectl -n mysql  get pvc -l app=mysql


NAME           STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-mysql-0   Bound     pvc-83e9dfeb-e721-11e8-86c5-069628ef0c9c   10Gi       RWO            mysql-gp2            1d
data-mysql-1   Bound     pvc-977e7806-e721-11e8-86c5-069628ef0c9c   10Gi       RWO            mysql-gp2            1d
data-mysql-2   Bound     pvc-b3009b02-e721-11e8-86c5-069628ef0c9c   10Gi       RWO            mysql-gp2            1d
data-mysql-3   Bound     pvc-de14acd8-e811-11e8-86c5-069628ef0c9c   10Gi       RWO            mysql-gp2            34m
data-mysql-4   Bound     pvc-e916c3ec-e812-11e8-86c5-069628ef0c9c   10Gi       RWO            mysql-gp2            26m

Challenge

By default, deleting a PersistentVolumeClaim will delete its associated persistent volume. What if you wanted to keep the volume?

Change the reclaim policy of the PersistentVolume associated with PersistentVolumeClaim called “data-mysql-3” to “Retain”. Please see Kubernetes documentation for help

Expand here to see the solution

Delete data-mysql-3 and data-mysql-4 with following commands.

kubectl -n mysql delete pvc data-mysql-3
kubectl -n mysql delete pvc data-mysql-4


persistentvolumeclaim "data-mysql-3" deleted
persistentvolumeclaim "data-mysql-4" deleted