Cassandra connector for AWS Kinesis that allows Kinesis records to be stored into Cassandra.
Introduction
Cassandra Kinesis connector is based on Amazon Kinesis Connector Library project. It extends the library with Cassandra-specific transformer and emitter (ITransformer and IEmitter, that is, CassandraTransformer and CassandraEmitter) and provides basic implementations for those. It also includes a connector executor implementation that allows the connector to be run as a stand-alone Java process.
The provided default implementations of CassandraTransformer and CassandraEmitter are JsonCassandraTransformer and DefaultCassandraEmitter.
JsonCassandraTransformer interprets the input records from Kinesis stream as JSON strings and converts them into CassandraRecord instances. Every JSON top-level property is treated as a Cassandra table column. Only JSON properties of type string, boolean or number are processed. The rest is silently ignored.
DefaultCassandraEmitter stores CassandraRecord instances (produced by JsonCassandraTransformer) into the respective Cassandra table (specified by the configuration).
Configuration
Cassandra Kinesis Connector extends Amazon Kinesis Connector Library's configuration and adds additional Cassandra-specific properties.
All default property values can be replaced using an external properties file - cassandra-kinesis-connector.properties.
Usage
Cassandra Kinesis Conector can be either embedded as a library into an application or used as a out-of-the-box connector application.
For embedding, see CassandraKinesisConnectorExecutor implementation.
As a stand-alone, out-of-the-box connector application, Cassandra Kinesis Connector executable JAR (cassandra-kinesis-VERSION-all.jar) should be used. When started, the connector tries to load external properties file cassandra-kinesis-connector.properties from the working directory.
License and development
Cassandra Kinesis is licensed under the liberal and business-friendly Apache Licence, Version 2.0 and is freely available on GitHub. Cassandra Kinesis is further released to the repositories of Maven Central and on JCenter. The project is built using Maven. From your shell, cloning and building the project would go something like this:
git clone https://github.com/smartcat-labs/cassandra-kinesis.git cd cassandra-kinesis mvn package
