1/1/2021

Reading time:6 min

roberd13/Getting-Started-With-DSE-and-Docker

by roberd13

You take the blue pill, the story ends. You wake up in your bed and believe whatever you want to believe about your normal ideas on how to run DSE and your apps. You take the red pill, you stay in Wonderland, I show you how deep the rabbit hole goes and you set sail on the Container ship of the future called DSE on DockerWelcome to the real world!!!Containers are the Future, Google boasts that they run everything in containers. So why not you? Containers are scalable and fast to deploy, you can have a single node or multinode cluster up and running in minutes with just a few commands that is in a consistent, environment, plus you can share them with your teammates and be the hero.There are multiple ways to get DSE on Docker. You can pull DSE from Docker Store, use docker compose or even roll your own from our public Github Repo.Let's not waste any time and get you started on your mastery by creating some DSE, OpsCenter and Studio Containers.This blog assumes you to haveA basic understanding of Docker images and containers.Docker installed on your local system, see Docker Installation Instructions.When building custom images from the DataStax github repository, a DataStax Academy account.The BasicsHere you will see examples on how to start a simple standalone container for the specific workloads by adding flags to the end of the docker run command. No flag for transactional, -g for graph, -k for Analytics, -s for Search workloads. You can use any combination of those flags for the workload you need.Notice the -e flag which is for environmental variable it has DS_LICENSE=accept just after it. This is required or the container will not start.Now for some fun.Create a DSE database containerdocker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5Create a DSE container with Graph enableddocker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -gCreate a DSE container with Analytics (Spark) enableddocker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -kCreate a DSE container with Search enableddocker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -sCongratulations You have now graduated to Advanced training!It’s time to build on the basics you learned and use some of the features that have been added to the DSE Docker containerDataStax provided Docker images include a start up script that swaps DSE configuration files found in the Volume /conf with the configuration file in the default location on the container.To use this feature:Create a directory on your local host.Add the configuration files you want to replace. Use the following links for full list of configuration files:The file name must match a corresponding configuration file in the image and include all the required values, for example cassandra.yaml, dse.yaml, opscenterd.conf.Mount the local directory to the exposed Volume /config.Start the container. For example to start a transactional node:docker run -e DS_LICENSE=accept --name my-dse -d -v /dse/conf:/config store/datastax/dse-server:5.1.5The DSE images also expose the following volumes.For DataStax Enterprise Transactional, Search, Graph, and Analytics workloads:/var/lib/cassandra: Data from Cassandra/var/lib/spark: Data from DSE Analytics w/ Spark/var/lib/dsefs: Data from DSEFS/var/log/cassandra: Logs from Cassandra/var/log/spark: Logs from Spark/conf: Directory to add custom config files for the container to pickup.For OpsCenter: /var/lib/opscenterFor Studio: /var/lib/datastax-studioTo persist data, pre-create directories on the local host and map the directory to the corresponding volume using the docker run -v flag.NOTE: If the volumes are not mounted from the local host, all data is lost when the container is removed.To mount a volume, use the following syntax:docker run -v <local_directory>:<container_volume>Example Mount the host directory /dse/conf to the DSE volume /config to manage configuration files.docker run -e DS_LICENSE=accept --name my-dse -d -v /dse/conf:/config store/datastax/dse-server:5.1.5See Docker docs > Use volumes for more information.If the container is running in the background (using the -d), use the following command to run DSE commands interactively.docker exec -it <container_name> <command_name>For examplenodetool status command:docker exec -it my-dse nodetool statusTo load data we can use a cqlsh prompt:docker exec -it my-dse cqlshIf you want to get a bash prompt where you can do anything just like having a ssh connection to an oldschool nodedocker exec -it <container_name> bashTo exit the shell without stopping the container use ctl P ctl QHere we will learn how to create an Opscenter container and a connected DataStax Enterprise server container on the same Docker host.To create and connect the containers:First create an OpsCenter container.docker run -e DS_LICENSE=accept -d -p 8888:8888 --name my-opscenterSee OpsCenter Docker run options for additional options that persist data or manage configuration.To Create a DataStax Enterprise (DSE) server container that is linked to the OpsCenter container.docker run -e DS_LICENSE=accept --link my-opscenter:opscenter --name my-dse -d store/datastax/dse-server:5.1.5Get the DSE container IP address:docker exec -it my-dse nodetool statusOpen a browser and go to http://DOCKER_HOST_IP:8888.Click Manage existing cluster.In host name, enter the DSE IP address.Click Install agents manually. Note that the agent is already installed on the DSE image; no installation is required.OpsCenter is ready to use with DSE. See the OpsCenter User Guide for detailed usage and configuration instructions.Wait, there's more_ Now dive deeper and create a Studio container_docker run -e DS_LICENSE=accept --link my-dse --name my-studio -p 9091:9091 -d datastax/dse-studioOpen your browser and point to http://DOCKER_HOST_IP:9091, create the new connection using my-dse as the hostname.Check Studio documentation for further instructions.Bootstrapping a multi-node cluster with OpsCenter and Studio can be automated with Docker Compose. To get sample compose.yml files visit the following links.Wow this is so simple3-Node Setupdocker-compose -f docker-compose.yml up -d --scale node=23-Node Setup with OpsCenterdocker-compose -f docker-compose.yml -f docker-compose.opscenter.yml up -d --scale node=23-Node Setup with OpsCenter and Studiodocker-compose -f docker-compose.yml -f docker-compose.opscenter.yml -f docker-compose.studio.yml up -d --scale node=2Single Node Setup with Studiodocker-compose -f docker-compose.yml -f docker-compose.studio.yml up -d --scale node=0In ClosingI hope you have enjoyed learning the basics of creating a DSE container or cluster using Docker. If you would like to learn more and see more advanced configuration options please visit Installing DSE on Docker and our public Github Repo.I will leave you with this"You are the ONE, use your training for good and awaken the rest of your colleagues to the Real World of DSE on Docker."

Read this article if you want to know more about roberd13/Getting-Started-With-DSE-and-Docker

You take the blue pill, the story ends. You wake up in your bed and believe whatever you want to believe about your normal ideas on how to run DSE and your apps. You take the red pill, you stay in Wonderland, I show you how deep the rabbit hole goes and you set sail on the Container ship of the future called DSE on Docker

Welcome to the real world!!!

Containers are the Future, Google boasts that they run everything in containers. So why not you? Containers are scalable and fast to deploy, you can have a single node or multinode cluster up and running in minutes with just a few commands that is in a consistent, environment, plus you can share them with your teammates and be the hero.

There are multiple ways to get DSE on Docker. You can pull DSE from Docker Store, use docker compose or even roll your own from our public Github Repo.

Let's not waste any time and get you started on your mastery by creating some DSE, OpsCenter and Studio Containers.

This blog assumes you to have

A basic understanding of Docker images and containers.
Docker installed on your local system, see Docker Installation Instructions.
When building custom images from the DataStax github repository, a DataStax Academy account.

The Basics

Here you will see examples on how to start a simple standalone container for the specific workloads by adding flags to the end of the docker run command. No flag for transactional, -g for graph, -k for Analytics, -s for Search workloads. You can use any combination of those flags for the workload you need.

Notice the -e flag which is for environmental variable it has DS_LICENSE=accept just after it. This is required or the container will not start.

Now for some fun.

Create a DSE database container

docker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5

Create a DSE container with Graph enabled

docker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -g

Create a DSE container with Analytics (Spark) enabled

docker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -k

Create a DSE container with Search enabled

docker run -e DS_LICENSE=accept --name my-dse -d store/datastax/dse-server:5.1.5 -s

Congratulations You have now graduated to Advanced training!

It’s time to build on the basics you learned and use some of the features that have been added to the DSE Docker container

DataStax provided Docker images include a start up script that swaps DSE configuration files found in the Volume /conf with the configuration file in the default location on the container.

To use this feature:

Create a directory on your local host.
Add the configuration files you want to replace. Use the following links for full list of configuration files:
The file name must match a corresponding configuration file in the image and include all the required values, for example cassandra.yaml, dse.yaml, opscenterd.conf.
Mount the local directory to the exposed Volume /config.
Start the container. For example to start a transactional node:

docker run -e DS_LICENSE=accept --name my-dse -d -v /dse/conf:/config store/datastax/dse-server:5.1.5

The DSE images also expose the following volumes.

For DataStax Enterprise Transactional, Search, Graph, and Analytics workloads:
- /var/lib/cassandra: Data from Cassandra
- /var/lib/spark: Data from DSE Analytics w/ Spark
- /var/lib/dsefs: Data from DSEFS
- /var/log/cassandra: Logs from Cassandra
- /var/log/spark: Logs from Spark
- /conf: Directory to add custom config files for the container to pickup.
For OpsCenter: /var/lib/opscenter
For Studio: /var/lib/datastax-studio

To persist data, pre-create directories on the local host and map the directory to the corresponding volume using the docker run -v flag.

NOTE: If the volumes are not mounted from the local host, all data is lost when the container is removed.

To mount a volume, use the following syntax:

docker run -v <local_directory>:<container_volume>

Example Mount the host directory /dse/conf to the DSE volume /config to manage configuration files.

docker run -e DS_LICENSE=accept --name my-dse -d  -v /dse/conf:/config store/datastax/dse-server:5.1.5

See Docker docs > Use volumes for more information.

If the container is running in the background (using the -d), use the following command to run DSE commands interactively.

docker exec -it <container_name> <command_name>

For example

nodetool status command:

docker exec -it my-dse nodetool status

To load data we can use a cqlsh prompt:

docker exec -it my-dse cqlsh

If you want to get a bash prompt where you can do anything just like having a ssh connection to an oldschool node

docker exec -it <container_name> bash

To exit the shell without stopping the container use ctl P ctl Q

Here we will learn how to create an Opscenter container and a connected DataStax Enterprise server container on the same Docker host.

To create and connect the containers:

First create an OpsCenter container.

docker run -e DS_LICENSE=accept -d -p 8888:8888 --name my-opscenter

See OpsCenter Docker run options for additional options that persist data or manage configuration.

To Create a DataStax Enterprise (DSE) server container that is linked to the OpsCenter container.

docker run -e DS_LICENSE=accept --link my-opscenter:opscenter --name my-dse -d store/datastax/dse-server:5.1.5

Get the DSE container IP address:

docker exec -it my-dse nodetool status

Open a browser and go to http://DOCKER_HOST_IP:8888.
Click Manage existing cluster.
In host name, enter the DSE IP address.
Click Install agents manually. Note that the agent is already installed on the DSE image; no installation is required.

OpsCenter is ready to use with DSE. See the OpsCenter User Guide for detailed usage and configuration instructions.

Wait, there's more

_ Now dive deeper and create a Studio container_

docker run -e DS_LICENSE=accept --link my-dse --name my-studio -p 9091:9091 -d datastax/dse-studio

Open your browser and point to http://DOCKER_HOST_IP:9091, create the new connection using my-dse as the hostname.

Check Studio documentation for further instructions.

Bootstrapping a multi-node cluster with OpsCenter and Studio can be automated with Docker Compose. To get sample compose.yml files visit the following links.

Wow this is so simple

3-Node Setup

docker-compose  -f docker-compose.yml up -d --scale node=2

3-Node Setup with OpsCenter

docker-compose -f docker-compose.yml -f docker-compose.opscenter.yml up -d --scale node=2

3-Node Setup with OpsCenter and Studio

docker-compose -f docker-compose.yml -f docker-compose.opscenter.yml -f docker-compose.studio.yml up -d --scale node=2

Single Node Setup with Studio

docker-compose -f docker-compose.yml -f docker-compose.studio.yml up -d --scale node=0

In Closing

I hope you have enjoyed learning the basics of creating a DSE container or cluster using Docker. If you would like to learn more and see more advanced configuration options please visit Installing DSE on Docker and our public Github Repo.

I will leave you with this

"You are the ONE, use your training for good and awaken the rest of your colleagues to the Real World of DSE on Docker."

Checkout Planet Cassandra

Claim Your Free Planet Cassandra Contributor T-shirt!

Make your contribution and score a FREE Planet Cassandra Contributor T-Shirt!  We value our incredible Cassandra community, and we want to express our gratitude by sending an exclusive Planet Cassandra Contributor T-Shirt you can wear with pride.