Illustration Image

Cassandra.Link

The best knowledge base on Apache Cassandra®

Helping platform leaders, architects, engineers, and operators build scalable real time data platforms.

10/23/2019

Reading time:1 min

Tom-Causs/kafka-cassandra-poc

by John Doe

The app consists of two parts:Kafka Producer - generates random messages and sends them to KafkaKafka Consumer - reads messages from Kafka topic and stores them in CassandraRequirementsJava 8GitMavenVirtualBoxVagrantSetupTo run the app, first execute the following cmd from the project root:vagrant upThis will start up the Vagrant box. The first time will take a while as it has to download the OS image and other dependencies.To start-up the producer app, run the following cmd from the kafka-producer folder:$ mvn spring-boot:runNote: the producer will terminate after the messages have been sentTo start-up the consumer app, run the following cmd from the kafka-consumer folder:$ mvn spring-boot:runNote: the consumer will keep listening for new messages until manually terminatedShutting down the vagrant box can be done by typingvagrant haltMisc. commandsCreating a Kafka topic:./kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myTopicList all Kafka topics:./kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181Run the console producer:./kafka/bin/kafka-console-producer.sh --broker-list 192.168.33.10:9092 --topic myTopicRun the console consumer:./kafka/bin/kafka-console-consumer.sh --zookeeper 192.168.33.10:2181 --topic myTopic --from-beginningRun Cassandra terminal client:./cassandra/bin/cqlshAboutA simple demo to show how Kafka works together with CassandraUsed technologies:Spring BootApache KafkaCassandra

Illustration Image

The app consists of two parts:

  • Kafka Producer - generates random messages and sends them to Kafka
  • Kafka Consumer - reads messages from Kafka topic and stores them in Cassandra

Requirements

Setup

To run the app, first execute the following cmd from the project root:

vagrant up

This will start up the Vagrant box. The first time will take a while as it has to download the OS image and other dependencies.

To start-up the producer app, run the following cmd from the kafka-producer folder:

$ mvn spring-boot:run

Note: the producer will terminate after the messages have been sent

To start-up the consumer app, run the following cmd from the kafka-consumer folder:

$ mvn spring-boot:run

Note: the consumer will keep listening for new messages until manually terminated

Shutting down the vagrant box can be done by typing

vagrant halt

Misc. commands

Creating a Kafka topic:

./kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myTopic

List all Kafka topics:

./kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181

Run the console producer:

./kafka/bin/kafka-console-producer.sh --broker-list 192.168.33.10:9092 --topic myTopic

Run the console consumer:

./kafka/bin/kafka-console-consumer.sh --zookeeper 192.168.33.10:2181 --topic myTopic --from-beginning

Run Cassandra terminal client:

./cassandra/bin/cqlsh

About

A simple demo to show how Kafka works together with Cassandra

Used technologies:

  • Spring Boot
  • Apache Kafka
  • Cassandra

Related Articles

cassandra
event.driven
spark

Build an Event-Driven Architecture with Apache Kafka, Apache Spark, and Apache Cassandra

DataStax

8/3/2024

Checkout Planet Cassandra

Claim Your Free Planet Cassandra Contributor T-shirt!

Make your contribution and score a FREE Planet Cassandra Contributor T-Shirt! 
We value our incredible Cassandra community, and we want to express our gratitude by sending an exclusive Planet Cassandra Contributor T-Shirt you can wear with pride.

Join Our Newsletter!

Sign up below to receive email updates and see what's going on with our company

Explore Related Topics

AllKafkaSparkScyllaSStableKubernetesApiGithubGraphQl

Explore Further

java