Skip to main content

Manage Kubernetes Storage using Persistent Volume (PV) and Persistent Volume Claim (PVC)

Background

This is the sixth part of the series on building highly scalable multi-container apps using AKS. So far in this series we have covered following topics.

In this post we will learn about a very important concept of Persistent Volumes (PV) and Persistent Volume Claim (PVC). We will use these concepts to manage data for the SQL Server 2017 Linux container.

Why do we need volume in the first place?

Managing data in a containerized scenario is quite tricky. Containers are ephemeral by design. Which means that they can be created, deleted and rescheduled on any host in the cluster. If the container is recreated, what happens to the data stored inside the container? It is oblivious that the data stored locally inside the container will be lost.

This scenario might be ok while testing. But as we start moving to deploying the applications using containers, we would want to persist the data even if the container restarts or a newer version of the image is used to upgrade the older version of container. In such scenario it makes sense to store the data external to the container. It is in such scenario that external volumes play a very important role.

During the course of this post we will focus on following

  • Provision Persistent Volume (PV)  using Azure Disk
  • Create Persistent Volume Claim (PVC) to the provisioned volume

Provision Persistent Volume (PV)  using Azure Disk

Persistent Volume abstracts from users details of how the storage is provided and how it is consumed. It is a cluster level resource provisioned by administrator and available to the pods to use. Persistent volume supports multiple types of storage including network storage, physical disks or cloud storage like Azure Disk. The storage is provisioned using one of the supported Storage Class. Lets look at an example of creating a storage class based on Azure Disk

We will skip the initial metadata, on line 7 we specify the provisioner as azure-disk. We also specify the parameters storage account type as Standard_LRS and kind as Managed. The parameters will be different for different types of provisioner. You can refer to the storage-class provisioner link below to know more about the supported provisioners with Kubernetes.

Lets go and apply this Kubernetes manifest to our cluster. Use kubectl command line to execute the command

kubectl apply –f 01_storage-class.yml

We can browse to the Kubernetes dashboard and see that the storage class is created successfully.

storage class

The creation of Storage Class is an indication of what kind of storage we are going to provision within the Kubernetes cluster. Lets go ahead and create a Persistent Volume Claim.

Create Persistent Volume Claim to the provisioned storage

Lets look at an example of the Kubernetes manifest file to create a Persistent Volume Claim (PVC).

Above we see the definition of PVC. Line 8 links the storage class named azure-disk. Note that this is the name we gave while creating the storage class earlier. From line 9 onwards we specify the requirements for the storage claim. In our example we have specified ReadWriteOnce as the access mode. For the resource request, we specified 1Gb as the disk space. When we apply this manifest, 1Gb of disk space will be provisioned for us. Run the kubectl command

kubectl apply –f 02_PersistentVolumeClaim.yml

The Persistent Volume has different lifecycle. It takes few minutes to provision the storage and bind it to the Kubernetes cluster. Initially the status of the persistent volume claim will be Pending.

PVC Pending

Once the volume is successfully provisioned it is bound to the claim. We can verify this by looking at the bound state.

PVC Bound

At this point we can go to the Persistent Volumes section on the left of the Kubernetes dashboard and find details about the underlying volume.

PVC

In this view we can find details like the name of the volume, its capacity, access modes, status, associated claim etc. Like we had done in earlier parts of this series, we can use the Kubernetes cli to query the PV object to know more details. Execute the following command

kubectl describe pv <<name of the pvc>>

The output will be as shown in the screenshot  below

kubectl describe PVC

It is same as what we see in the Kubernetes dashboard. We will see how to make use of this Persistent Volume in the next part of this series.

References

[1] Kubernetes Storage class provisioner : https://kubernetes.io/docs/concepts/storage/storage-classes/

[2] Persistent Volumes : https://kubernetes.io/docs/concepts/storage/persistent-volumes/

Conclusion

We saw how to provision a Persistent Volume backed by Azure Disk. With the help of 2 Kubernetes Manifest files we were able to get the disk and associate a claim with it.

Persistent Volumes are quite powerful concept in Kubernetes. It allows developers and cluster administrators to work on different parts of the application. Using Persistent volumes, the administrator can provision storage which can be used by developers without knowing the underlying storage mechanism. This can be used to swap the underlying storage without any need to make changes to application code. This may not be very clear right now, but I am sure it will make sense in the next part where we will see how to utilize these volumes

Until next time, Code with Passion and Strive for Excellence.

Comments

Popular Posts

Build custom SQL Server 2017 Linux Docker image

BackgroundIn the earlier series of posts on continuous deployment of multi-container apps, I had used two containers for ASP.Net Core MVC web site and ASP.Net Core Web API. This post is a sort of continuation. In the earlier version we were using in memory database to store the data. This approach is useful in a proof of concept or a quick demo application. In an enterprise scenario we would use some sort  of persistent data store to store the data. It is very easy to containerize the stateless services and web front ends. When we move towards stateful processing, things start to get interesting.Containers by their very nature are supposed to be ephemeral. This means that containers can be stopped and destroyed and a new one built and put in place with minimum setup and configuration. If we destroy the container and replace it with a new one what happens to the data stored inside the container? Can we use containers to run database related processes? How can the stateless containers c…

Docker Compose Tip : How to avoid SQL Server 2017 container exiting when running using docker compose

Problem starting SQL 2017 containerWhile working on recent blog post on Integrating SQL Server 2017 Linux with ASP.Net Core using Docker I cam across a strange behavior. I was able to run SQL Server 2017 Linux container image successfully as demonstrated here. I was able to initialize the database using the command docker run -it -p 1433:1433 \ --name sql2017 \ nileshgule/sqldbThis command runs the custom image in interactive mode based on the –i flag. We also pass the t flag to create a psudo tty terminal. The next step for me was to try the integration with Dotnet Core project. I started with updating the docker-compose file and other changes explained in the post on integration.Before I added the sql2017 as the service inside docker-compose file, I ran the SQL Server 2017 container individually using the above command & verified that the Core MVC and Core Web API we able to communicate to the database inside container.This seemed pretty easy at first. All I had to do was update…

Integrate SQL Server 2017 Linux with ASP.NET Core using Docker

In the previous post, we saw how to initialize a SQL Server 2017 on Linux database running inside Docker container. This post is continuation of my exploration about Docker containers for stateful applications. In this post we will explore how to integrate the SQ Server 2017 on Linux container with ASP.Net Core Web API also running inside a Docker container. We will perform following activitiesUpdate Docker compose file to build the SQL server imageUpdate Web API project to use SQL Server Database instead of In memory databaseBuild and run the application containers using Docker compose commandsQuery Data from SQL container using the MVC web front endUpdate Docker compose file to build the SQL server imageDocker provides Docker Compose as a tool for defining and running multi-container Docker applications. I already have an existing docker compose file which I have been using for my series of post on implementing continuous deployment of multi container apps. In a compose file we defi…