Example: Deploying Cassandra with Stateful Sets

Edit This PageThis tutorial shows you how to develop a native cloud Cassandra deployment on Kubernetes. In this example, a custom Cassandra SeedProvider enables Cassandra to discover new Cassandra nodes as they join the cluster.StatefulSets make it easier to deploy stateful applications within a clustered environment. For more information on the features used in this tutorial, see the StatefulSet documentation.Cassandra on DockerThe Pods in this tutorial use the gcr.io/google-samples/cassandra:v13image from Google’s container registry.The Docker image above is based on debian-baseand includes OpenJDK 8.This image includes a standard Cassandra installation from the Apache Debian repo.By using environment variables you can change values that are inserted into cassandra.yaml.ENV VARDEFAULT VALUECASSANDRA_CLUSTER_NAME'Test Cluster'CASSANDRA_NUM_TOKENS32CASSANDRA_RPC_ADDRESS0.0.0.0ObjectivesBefore you beginCreating a Cassandra Headless ServiceUsing a StatefulSet to Create a Cassandra RingValidating The Cassandra StatefulSetModifying the Cassandra StatefulSetCleaning upWhat's nextObjectivesBefore you beginTo complete this tutorial, you should already have a basic familiarity with Pods, Services, and StatefulSets. In addition, you should: Note: Please read the getting started guides if you do not already have a cluster.Additional Minikube Setup Instructions Caution: Minikube defaults to 1024MB of memory and 1 CPU. Running Minikube with the default resource configuration results in insufficient resource errors during this tutorial. To avoid these errors, start Minikube with the following settings:minikube start --memory 5120 --cpus=4A Kubernetes Service describes a set of Pods that perform the same task.The following Service is used for DNS lookups between Cassandra Pods and clients within the Kubernetes cluster. application/cassandra/cassandra-service.yaml apiVersion: v1kind: Servicemetadata: labels: app: cassandra name: cassandraspec: clusterIP: None ports: - port: 9042 selector: app: cassandra Launch a terminal window in the directory you downloaded the manifest files.Create a Service to track all Cassandra StatefulSet nodes from the cassandra-service.yaml file:kubectl create -f https://k8s.io/examples/application/cassandra/cassandra-service.yamlValidating (optional)Get the Cassandra Service.kubectl get svc cassandraThe response isNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEcassandra ClusterIP None <none> 9042/TCP 45sService creation failed if anything else is returned. Read Debug Services for common issues.The StatefulSet manifest, included below, creates a Cassandra ring that consists of three Pods. Note: This example uses the default provisioner for Minikube. Please update the following StatefulSet for the cloud you are working with.Update the StatefulSet if necessary.Create the Cassandra StatefulSet from the cassandra-statefulset.yaml file:kubectl create -f https://k8s.io/examples/application/cassandra/cassandra-statefulset.yamlGet the Cassandra StatefulSet:kubectl get statefulset cassandraThe response should be:NAME DESIRED CURRENT AGEcassandra 3 0 13sThe StatefulSet resource deploys Pods sequentially.Get the Pods to see the ordered creation status:kubectl get pods -l="app=cassandra"The response should be:NAME READY STATUS RESTARTS AGEcassandra-0 1/1 Running 0 1mcassandra-1 0/1 ContainerCreating 0 8sIt can take several minutes for all three Pods to deploy. Once they are deployed, the same command returns:NAME READY STATUS RESTARTS AGEcassandra-0 1/1 Running 0 10mcassandra-1 1/1 Running 0 9mcassandra-2 1/1 Running 0 8mRun the Cassandra nodetool to display the status of the ring.kubectl exec -it cassandra-0 -- nodetool statusThe response should look something like this:Datacenter: DC1-K8Demo======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns (effective) Host ID RackUN 172.17.0.5 83.57 KiB 32 74.0% e2dd09e6-d9d3-477e-96c5-45094c08db0f Rack1-K8DemoUN 172.17.0.4 101.04 KiB 32 58.8% f89d6835-3a42-4419-92b3-0e62cae1479c Rack1-K8DemoUN 172.17.0.6 84.74 KiB 32 67.1% a6a1e8c2-3dc5-4417-b1a0-26507af2aaad Rack1-K8DemoUse kubectl edit to modify the size of a Cassandra StatefulSet.Run the following command:kubectl edit statefulset cassandraThis command opens an editor in your terminal. The line you need to change is the replicas field. The following sample is an excerpt of the StatefulSet file:# Please edit the object below. Lines beginning with a '#' will be ignored,# and an empty file will abort the edit. If an error occurs while saving this file will be# reopened with the relevant failures.#apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2kind: StatefulSetmetadata: creationTimestamp: 2016-08-13T18:40:58Z generation: 1 labels: app: cassandra name: cassandra namespace: default resourceVersion: "323" selfLink: /apis/apps/v1/namespaces/default/statefulsets/cassandra uid: 7a219483-6185-11e6-a910-42010a8a0fc0spec: replicas: 3Change the number of replicas to 4, and then save the manifest.The StatefulSet now contains 4 Pods.Get the Cassandra StatefulSet to verify:kubectl get statefulset cassandraThe response should beNAME DESIRED CURRENT AGEcassandra 4 4 36mCleaning upDeleting or scaling a StatefulSet down does not delete the volumes associated with the StatefulSet. This setting is for your safety because your data is more valuable than automatically purging all related StatefulSet resources. Warning: Depending on the storage class and reclaim policy, deleting the PersistentVolumeClaims may cause the associated volumes to also be deleted. Never assume you’ll be able to access data if its volume claims are deleted.Run the following commands (chained together into a single command) to delete everything in the Cassandra StatefulSet:grace=$(kubectl get po cassandra-0 -o=jsonpath='{.spec.terminationGracePeriodSeconds}') \ && kubectl delete statefulset -l app=cassandra \ && echo "Sleeping $grace" \ && sleep $grace \ && kubectl delete pvc -l app=cassandraRun the following command to delete the Cassandra Service.kubectl delete service -l app=cassandraWhat's nextLearn how to Scale a StatefulSet.Learn more about the KubernetesSeedProviderSee more custom Seed Provider Configurations