Illustration Image

Cassandra.Link

The best knowledge base on Apache Cassandra®

Helping platform leaders, architects, engineers, and operators build scalable real time data platforms.

7/8/2021

Reading time:1 min

yukim/cassandra-cdc-example

by John Doe

For DataStax Enterprise, see dse branchThis repository contains the sample project for reading Apache Cassandra commit log file in CDC location and outputs in JSON format.Build$ ./mvnw package -DskipTestsThis will produce cassandra-cdc-json-VERSION.tar.gz in target directory.RunningMake sure you are running Apache Cassandra with CDC enabled.See https://cassandra.apache.org/doc/latest/operating/cdc.html for enabling CDC.Upload the artifact cassandra-cdc-json-VERSION.tar.gz to you cassandra nodes, expand it to desired directory.Then run the following with user who is running Apache Cassandra.$ cd cassandra-cdc-json-VERSION$ bin/cassandra-cdc.shThe application should work if you install Apache Cassandra with package manager like yum, but if not,set CASSANDRA_INCLUDE environment variable that points to your Apache Cassandra installation's cassandra.in.sh.# CASSANDRA_INCLUDE=/path/to/cassandra/bin/cassandra.in.sh bin/cassandra-cdc.shChangeEventPartitions inside Mutation are first converted to list of ChangeEvents.A ChangeEvent represents an update happened to certain CQL row or deletion criteria at a specific timestamp.For details, see ChangeEvent.java.LimitationSince this program only loads schema at the start up, changes made since the application start up cannot be picked up.The following CQL patterns are not implemented or tested yet:Complex CQL types: set, list, counter and User Defined Type (UDT)Time to live (TTL)INSERT JSON

Illustration Image

For DataStax Enterprise, see dse branch

This repository contains the sample project for reading Apache Cassandra commit log file in CDC location and outputs in JSON format.

Build

$ ./mvnw package -DskipTests

This will produce cassandra-cdc-json-VERSION.tar.gz in target directory.

Running

Make sure you are running Apache Cassandra with CDC enabled. See https://cassandra.apache.org/doc/latest/operating/cdc.html for enabling CDC.

Upload the artifact cassandra-cdc-json-VERSION.tar.gz to you cassandra nodes, expand it to desired directory. Then run the following with user who is running Apache Cassandra.

$ cd cassandra-cdc-json-VERSION
$ bin/cassandra-cdc.sh

The application should work if you install Apache Cassandra with package manager like yum, but if not, set CASSANDRA_INCLUDE environment variable that points to your Apache Cassandra installation's cassandra.in.sh.

# CASSANDRA_INCLUDE=/path/to/cassandra/bin/cassandra.in.sh bin/cassandra-cdc.sh

ChangeEvent

Partitions inside Mutation are first converted to list of ChangeEvents. A ChangeEvent represents an update happened to certain CQL row or deletion criteria at a specific timestamp.

For details, see ChangeEvent.java.

Limitation

  • Since this program only loads schema at the start up, changes made since the application start up cannot be picked up.
  • The following CQL patterns are not implemented or tested yet:
    • Complex CQL types: set, list, counter and User Defined Type (UDT)
    • Time to live (TTL)
    • INSERT JSON

Related Articles

cassandra
cdc

Combine Data at Rest and Data in Motion with CDC for Apache Cassandra

John Doe

4/21/2022

cdc
datastax
cassandra

Checkout Planet Cassandra

Claim Your Free Planet Cassandra Contributor T-shirt!

Make your contribution and score a FREE Planet Cassandra Contributor T-Shirt! 
We value our incredible Cassandra community, and we want to express our gratitude by sending an exclusive Planet Cassandra Contributor T-Shirt you can wear with pride.

Join Our Newsletter!

Sign up below to receive email updates and see what's going on with our company

Explore Related Topics

AllKafkaSparkScyllaSStableKubernetesApiGithubGraphQl

Explore Further

cassandra