In this post, we will use Ansible to and set-up an Apache Cassandra database cluster. We will use AWS EC2 instances as the nodes for the cluster. Creating a cluster manually is a tedious task. We have to manually configure each node and each node must be correctly configured before starting the cluster.With Ansible, we can automate the task and let Ansible handle the configuration management for us.
First of all, create a directory for storing the files and folders related to the playbook. It helps in keeping our work organized and saves us from the confusion which may arise due to relative and absolute path references in passing the variables in our playbook. Following is the structure of my directory that contains the playbook and the roles:
Steps To Follow While Using AWS:
-
Create 2-3 instances of AWS EC2 which will serve as the nodes in a cluster.
-
Create a security group to allow all connections and add the nodes to that security groups.
-
Create an inventory that has the IP addresses of the nodes.
-
Add the inventory file into the configuration file of the Ansible, i.e. ansible.cfg.
Now, we create a playbook to set up the nodes for us. Following is the playbook for the same.
---
- hosts: aws-webservers
gather_facts: yes
remote_user: ec2-user
become: yes
vars:
cluster_name: Test_Cluster
seeds: 13.xxx.xxx.xxx
roles:
- installation
Then, we define the roles we have created. In the role, installation, the following tasks have been achieved:
-
Installing a JRE
-
Adding and unpacking the Apache Cassandra tar.
-
Replacing the cassandra.yaml having default configurations with cassandra.yaml with our own configurations whose details are given below.
-
Ensuring Cassandra is started
The following is the main.yml file from the roles:
---
- name: Copt Java RPM file
copy:
src: jdk-8_linux-x64_bin.rpm
dest: /tmp/jdk-8_linux-x64_bin.rpm
- name: install JDK via RPM file with yum
yum:
name: /tmp/jdk-8_linux-x64_bin.rpm
state: present
- name: Copy Cassandra tar
copy:
src: apache-cassandra-3.11.2-bin.tar.gz
dest: /tmp/apache-cassandra-3.11.2-bin.tar.gz
- name: Extract Cassandra
command: tar -xvf /tmp/apache-cassandra-3.11.2-bin.tar.gz
- name: override cassandra.yaml file
template: src=cassandra.yaml dest=apache-cassandra-3.11.2/conf/
- name: Run Cassandra from bin folder
command: ./cassandra -fR
args:
chdir: /home/ec2-user/apache-cassandra-3.11.2/bin/
The cassandra.yaml contains most of the Cassandra configuration, such as ports used, file locations and seed node IP addresses. We need to edit this file on each node, so I have created a template for the file. The template cassandra.yaml uses the following variables:
- cluster_name: ‘{{ cluster_name }}’ – can be anything chosen by you to describe the name of the cluster.
- seeds: “{{ seeds }}” – are the IP addresses of the clusters seed servers. Seed nodes are used as known places where cluster information (such as a list of nodes in the cluster) can be obtained.
- listen_address: {{ aws-webservers }} – is the IP address that Cassandra will listen on for internal (Cassandra to Cassandra) communication will occur.
- rpc_address: {{ aws-webservers }} – is the IP address that Cassandra will listen on for client-based communication.
Now, we can run the playbook and our cluster will be up and running. We can add more nodes to the list by simply adding them to the hosts list and Ansible will ensure that Cassandra is installed and the nodes are connected to the cluster and started.
Points To Remember
-
The host IP should be the public IP of a node.
-
Put the Java rpm packages and Cassandra tar file in the files directory of the role created.
-
Use Java 8 as Cassandra is not supported on higher versions of Java. It will throw the following error.
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/home/mmatak/monero/apache-cassandra-3.11.1/logs/gc.log instead. intx ThreadPriorityPolicy=42 is outside the allowed range [ 0 ... 1 ] Improperly specified VM option 'ThreadPriorityPolicy=42' Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.
Thus, Ansible makes it very easy to install distributed systems like Cassandra. The thought of doing it manually is very disheartening. The full source code including templates and directory structure is here.
