{"componentChunkName":"component---src-templates-article-single-page-js","path":"/post/smartcat-labs-berserker","result":{"pageContext":{"obj_id":"bbcb1335-9a12-500a-a6e6-ab15920ef1b1","node":{"content":"

Load generator with modular architecture.

\"Build\n \"Download\"

Berserker is designed to be modular from beginning as illustrated on the following diagram.

\"Core

Rate generator controls the rate at which load generator operates, rate is expressed on per second basis, or better say, number of impulses which will be generated within one second. Each time impulse is generated load generator fetches data from data source and passes it to worker. Since those are simple interfaces, it is easy to add additional module implementing either data source, worker and even rate generator.\nFollowing diagram represents possible modules for Load Generator of which some are already implemented.

\"Architecture\"

Berserker is designed as command line tool, but having modular architecture makes it easy to use it as Java library as well.

Berserker Commons

Berserker Commons holds interface for core and configuration and it provides signature all the modules need to confront to be able to work together.

Berserker Core

Berserker Core contains load generator implementation, and common implementations of data source, rate generator and worker.

Berserker Runner

Berserker Runner represents runnable jar where desired data source, rate generator and worker can be specified within YAML configuration.\nFollowing section illustrates YAML configuration example.

load-generator-configuration:\n  data-source-configuration-name: Ranger\n  rate-generator-configuration-name: default\n  worker-configuration-name: Cassandra\n  metrics-reporter-configuration-name: JMX\n  thread-count: 10\n  queue-capacity: 100000\ndata-source-configuration:\n  values:\n    id: uuid()\n    firstName: random(['Peter', 'Mike', 'Steven', 'Joshua', 'John', 'Brandon'])\n    lastName: random(['Smith', 'Johnson', 'Williams', 'Davis', 'Jackson', 'White', 'Lewis', 'Clark'])\n    age: random(20..45)\n    email: string('{}@domain.com', randomLengthString(5))\n    statement:\n      consistencyLevel: ONE\n      query: string(\"INSERT INTO person (id, first_name, last_name, age, email) VALUES ({}, '{}', '{}', {}, '{}');\", $id, $firstName, $lastName, $age, $email)\n  output: $statement\nrate-generator-configuration:\n  rates:\n    r: 1000\n  output: $r\nworker-configuration:\n  connection-points: 0.0.0.0:32770\n  keyspace: my_keyspace\n  async: false\n  bootstrap-commands:\n    - \"CREATE KEYSPACE IF NOT EXISTS my_keyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};\"\n    - USE my_keyspace;\n    - CREATE TABLE IF NOT EXISTS person (id uuid, first_name text, last_name text, age int, email text, primary key (id));\nmetrics-reporter-configuration:\n  domain: berserker\n  filter:

Main part of configuration is load-generator-configuration where concrete modules which will be used for data source, rate generator and worker need to be specified. After load-generator-configuration section, there should be exactly one section for data source, rate generator and worker.\nEach section is allowed to contain module specific configuration as configuration interpretation will be done by module itself.\nIn order for berserker-runner to be able to find particular module, each module jar must be in classpath.

Rate generator configuration

Documentation on rate generator configuration can be found here.

Modules

List of existing modules:

Berserker Ranger

Berserker Ranger is Ranger data source implementation.

Berserker Kafka

Berserker Kafka is worker implementation which sends messages to Kafka cluster.

Berserker Cassandra

Berserker Cassandra is worker implementation which executes CQL statements on Cassandra cluster.

Berserker HTTP

Berserker HTTP is worker implementation which sends HTTP request on configured endpoint.

Berserker RabbitMQ

Berserker RabbitMQ is worker implementation which sends AMQP messages to RabbitMQ.

Berserker MQTT

Berserker MQTT is worker implementation which publishes messages to MQTT broker.

Usage

Berserker can be used either as a library or as a stand-alone command line tool.

Library usage

Artifact can be fetched from bintray.

Add following repository element to your <repositories> section in pom.xml:

<repository>\n  <id>bintray-smartcat-labs-maven</id>\n  <name>bintray</name>\n  <url>https://dl.bintray.com/smartcat-labs/maven</url>\n</repository>

Add the dependency element to your <dependencies> section in pom.xml depending which artifact and version you need:

<dependency>\n  <groupId>io.smartcat</groupId>\n  <artifactId>artifact</artifactId>\n  <version>version</version>\n</dependency>

Command line tool usage

","id":"bbcb1335-9a12-500a-a6e6-ab15920ef1b1","title":"smartcat-labs/berserker","origin_url":"https://github.com/smartcat-labs/berserker","url":"https://github.com/smartcat-labs/berserker","wallabag_created_at":"2018-05-17T18:23:25+00:00","published_at":null,"published_by":"['']","reading_time":3,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/0349248a3cff661557423583e7a24bbd535f2d52c22df015505774be4f3bbed7/smartcat-labs/berserker","tags":["cassandra","github","testing"],"description":"Load generator with modular architecture.\n Berserker is designed to be modular from beginning as illustrated on the following diagram.Rate generator controls the rate at which load generator operates,..."},"relatedArticles":[{"content":"

{{ message }}

\n
\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
\n\n
","id":"166bfd43-cfec-506b-be68-f384b93820b4","title":"GitHub - apache/cassandra-harry: Apache Cassandra - Harry","origin_url":"https://github.com/apache/cassandra-harry","url":"https://github.com/apache/cassandra-harry","wallabag_created_at":"2022-06-08T20:37:16+00:00","published_at":null,"published_by":"['apache']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/aa15162e4aac1bfa5fbd58f2757de131eaa602ad2bc71025321e6eeb66ff4553/apache/cassandra-harry","tags":["cassandra","testing"],"description":"{{ message }}\n / cassandra-harry PublicNotifications\nFork 12\n\n Star 43 \n\nApache Cassandra - Harrycassandra.apache.org/License Apache-2.0 license 43 stars 12 forks Star Notifications\n\n © 2022 GitHub, ..."},{"content":"

{{ message }}

\n
\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
","id":"d29b900a-7c3e-537e-a25e-6c8caff38e03","title":"GitHub - apache/cassandra-diff: Apache cassandra diff","origin_url":"https://github.com/apache/cassandra-diff","url":"https://github.com/apache/cassandra-diff","wallabag_created_at":"2022-06-08T20:37:07+00:00","published_at":null,"published_by":"['']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/e7a10c615cc491e34c4e3f952c02c227c9b02a49ab4ccf2ebd3bc5832f285203/apache/cassandra-diff","tags":["cassandra","testing"],"description":"{{ message }}\nApache cassandra diffcassandra.apache.org/LicenseApache-2.0 license10 stars 16 forks\n\n© 2022 GitHub, Inc.\n\nYou can’t perform that action at this time.\nYou signed in with another tab or w..."},{"content":"

{{ message }}

\n

/ adelphi Public

Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool

License

Apache-2.0 license

9 stars 4 forks

Star

Notifications

\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
\n\n
","id":"75fb8bf0-c48c-58d3-b5ef-b60dc7508eb1","title":"GitHub - datastax/adelphi: Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool","origin_url":"https://github.com/datastax/adelphi","url":"https://github.com/datastax/adelphi","wallabag_created_at":"2022-06-08T20:36:15+00:00","published_at":null,"published_by":"['datastax']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/afe20d0322f42c980557122f254ed0db1a89239e8a1e4130618ed630ed786ddc/datastax-archive/adelphi","tags":["stress","cassandra","cassandra.stress","testing"],"description":"{{ message }}\n / adelphi PublicNotifications\nFork 4\n\n Star 9 \n\nAutomation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltoolLicense Apache-2.0 license 9 stars 4 forks Star Not..."},{"content":"
\n

This is a sample code of utilizing Gatling testing framework for stress testing with DSE

\n

Pre-requisites

\n

Note: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin 0.0.7 is not compatible with Gatling 2.3.1 yet.

\n
  1. Gatling high charts bundle version 2.2.5:
    2. \n
  1. Gatling CQL plugin v0.0.7 (latest version as of writing)
    2. \n

[NOTES] - as of April 10, 2018, Gatling CQL plugin has released version 0.0.8 which works with the latest Gatling framework 2.3.1. Please feel free to download these latest versions for your test. Please also be aware that Gatling 2.3.1 requires scala 2.1.2 and make sure your scala version is upgraded first.

\n

Notes about the Example Simulation Scenario

\n

The simulation scenario (MyTestSimu.scala) as included in this example simulates a mixed read/write workload. The core steps of the scenario are summarized below and you can follow the same steps when creating your own scenario:

\n
  1. Set up connection to Cassandra/DSE cluster with proper properties, such as \"contact points\", \"load balancing policy\", etc.
    2. \n
  2. Create the application keyspace and table schema
    3. \n
  3. Define the (random) value generator for table columns based on their types
    4. \n
  4. Define the Read/Write statements to be used in the simulation, including the Consistency Level associated with the statements
    5. \n
  5. Set up the user simulation behavior for Read and Write. The key parts include:
    6. \n

Procedure

\n
  1. Set up Gatling framework and the CQL plug-in (just unzip, as per description found here)
    2. \n
  2. Download the MyTestSimu.scala file and put it under folder <GATLING_HOME>/user-files/simulations/
    3. \n
  3. Execute the Gatling simulation (stress-testing) scenario by running command: <GATLING_HOME>/bin/gatling.sh. Follow the instructions on the command-line output.
    4. \n

NOTE: The simulation scenario (MyTestSimu.scala) is tested against a DSE cluster (version 5.1.6) with UserName/Password authentication. Please adjust accordingly for your case.

\n

An example is as below. Simulation number 1 is the Cassandra stres-testing scenario as defined by this example.

\n
$ bin/gatling.sh\nGATLING_HOME is set to /home/automaton/gatling-charts-highcharts-bundle-2.2.5\nChoose a simulation number:\n     [0] cassandra.CassandraSimulation\n     [1] cassandra.MyTestSimu\n     [2] computerdatabase.BasicSimulation\n     [3] computerdatabase.advanced.AdvancedSimulationStep01\n     [4] computerdatabase.advanced.AdvancedSimulationStep02\n     [5] computerdatabase.advanced.AdvancedSimulationStep03\n     [6] computerdatabase.advanced.AdvancedSimulationStep04\n     [7] computerdatabase.advanced.AdvancedSimulationStep05\n1\nSelect simulation id (default is 'mytestsimu'). Accepted characters are a-z, A-Z, 0-9, - and _\nSelect run description (optional)\nSimulation cassandra.MyTestSimu started...\n================================================================================\n2018-04-04 15:33:43                                           5s elapsed\n---- Requests ------------------------------------------------------------------\n> Global                                                   (OK=232    KO=0     )\n> upsertStmt                                               (OK=93     KO=0     )\n> readStmt                                                 (OK=139    KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[#                                                                         ]  1%\n          waiting: 8861   / active: 0      / done:139\n---- Write Workload Scenario ---------------------------------------------------\n[                                                                          ]  0%\n          waiting: 20907  / active: 0      / done:93\n================================================================================\n... ...\n================================================================================\n2018-04-04 15:43:39                                         600s elapsed\n---- Requests ------------------------------------------------------------------\n> Global                                                   (OK=30000  KO=0     )\n> upsertStmt                                               (OK=21000  KO=0     )\n> readStmt                                                 (OK=9000   KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:9000\n---- Write Workload Scenario ---------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:21000\n================================================================================\nSimulation cassandra.MyTestSimu completed in 600 seconds\nParsing log file(s)...\nParsing log file(s) done\nGenerating reports...\n================================================================================\n---- Global Information --------------------------------------------------------\n> request count                                      30000 (OK=30000  KO=0     )\n> min response time                                      0 (OK=0      KO=-     )\n> max response time                                    224 (OK=224    KO=-     )\n> mean response time                                     2 (OK=2      KO=-     )\n> std deviation                                          5 (OK=5      KO=-     )\n> response time 50th percentile                          1 (OK=1      KO=-     )\n> response time 75th percentile                          2 (OK=2      KO=-     )\n> response time 95th percentile                          3 (OK=3      KO=-     )\n> response time 99th percentile                          7 (OK=7      KO=-     )\n> mean requests/sec                                     50 (OK=50     KO=-     )\n---- Response Time Distribution ------------------------------------------------\n> t < 800 ms                                         30000 (100%)\n> 800 ms < t < 1200 ms                                   0 (  0%)\n> t > 1200 ms                                            0 (  0%)\n> failed                                                 0 (  0%)\n================================================================================\nReports generated in 4s.\nPlease open the following file: /home/automaton/gatling-charts-highcharts-bundle-2.2.5/results/mytestsimu-1522856018600/index.html\n
\n
","id":"22b64c72-6b88-5842-8e08-ebff60cc319b","title":"yabinmeng/cassgatling","origin_url":"https://github.com/yabinmeng/cassgatling","url":"https://github.com/yabinmeng/cassgatling","wallabag_created_at":"2018-11-14T17:34:32+00:00","published_at":null,"published_by":null,"reading_time":3,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/ba96401352e04a099abef2d795d350d978caf9070e2f7cf302b70025f16fcffd/yabinmeng/cassgatling","tags":["stress","cassandra","gatling","testing"],"description":"\nThis is a sample code of utilizing Gatling testing framework for stress testing with DSE\nPre-requisites\nNote: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin ..."},{"content":"
\n

Methodology

\n

Breakdown the switches!

\n

I told you that this would be educational!

\n

First test

\n

cassandra-stress user profile=stress.yaml n=1000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 104.196.140.126 -rate threads\\>=121

\n

Second test

\n

I am not going to go over the flags/arguments that I went over in the previous example. Instead, I'll go over the new ones or changes.

\n

nohup cassandra-stress user profile=stress.yaml n=1000000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 40.118.149.27 -rate threads=300 -log file=output.txt -errors ignore

\n

Results

\n

to be continued... maybe, if I can.

\n\n

Thanks for taking a look at this repo and let me know if you have any questions.

\n
","id":"c7ef4299-9aed-5814-830c-152a2c71eee3","title":"justinbreese/dse-cassandra-stress","origin_url":"https://github.com/justinbreese/dse-cassandra-stress","url":"https://github.com/justinbreese/dse-cassandra-stress","wallabag_created_at":"2018-11-13T20:12:27+00:00","published_at":null,"published_by":null,"reading_time":5,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/ad3b3ac83d6bdca681e5fa455a46e248f827c4ef3fed9166b05113d5ea996ee6/justinbreese/dse-cassandra-stress","tags":["stress","cassandra","dse","testing","aws","google.cloud","azure"],"description":"\nI wanted to run some cassandra-stress testing on the three main cloud providers (Google, Amazon, and Microsoft) on similarly sized DataStax Enterprise clusters and see what the results would be\nWhy d..."},{"content":"

Developing software is fun. Developing software that is well tested, where those tests run quickly is even more fun.

Stubbed Cassandra is an open source tool that enables you to test applications that use Cassandra in a quick, deterministic way.

It is especially aimed edge case testing such as read and write timeouts.

It acts as a real Cassandra instance and can be primed to respond with results or with exceptions like read timeouts. It does this by implementing the server side of the CQL binary protocol.

It is separated into two components:

Scassandra, currently v1.1.0, is aimed at Java developers so most of the information is on the Java Client section of the website. It also may be used as a standalone jar.

Release v1.1.1

Release v1.1.0

Release v1.0.10

Release v1.0.9

Release v1.0.8

Release v1.0.7

Release v1.0.6

Release v1.0.5

Release v1.0.4

Release v1.0.3

Release v1.0.2

Release v1.0.1

Release v1.0.0

Release v0.11.0

Release v0.10.0

Release v0.9.1

Release v0.8.0

Release v0.7.0

Release v0.6.0

Release v0.5.0

Release v0.4.1

Release v0.3.0

Release v0.2.0

Release v0.1

Feature backlog

For feature requests and bug reports raise an issue at the Github issues page

Any questions ping me on twitter: @chbatey

","id":"f25dfc52-d2f5-5804-8a88-de850ba485a3","title":"Stubbed Cassandra","origin_url":"http://www.scassandra.org/","url":"http://www.scassandra.org/","wallabag_created_at":"2018-10-08T20:04:55+00:00","published_at":null,"published_by":null,"reading_time":4,"domain_name":"www.scassandra.org","preview_picture":null,"tags":["cassandra","testing"],"description":"Developing software is fun. Developing software that is well tested, where those tests run quickly is even more fun.Stubbed Cassandra is an open source tool that enables you to test applications that ..."},{"content":"

Support: \"Join
Documentation: User Guide, JavaDocs\n
A/B Testing Overview: \"A/B \"Blog \"Blog
Continuous Integration: \"Build\n\"Coverage\n\"Maven
License: \"Apache

Project

Wasabi A/B Testing Service is a real-time, enterprise-grade, 100% API driven project. Users are empowered to own their own data, and run experiments across web, mobile, and desktop. It’s fast, easy to use, it’s chock full of features, and instrumentation is minimal.

Learn more about how Wasabi can empower your team to move from hunches to actionable, data-driven user insights with our simple, flexible, and scalable experimentation platform.

Features

User Interface

Get Started

The following steps will help you install the needed tools, then build and run a complete Wasabi stack.

Bootstrap Your Environment

Mac OS
% /usr/bin/ruby \\\n  -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n% brew install git\n% git clone https://github.com/intuit/wasabi.git\n% cd wasabi\n% ./bin/wasabi.sh bootstrap

Installed tools include: homebrew 0.9, git 2,\nmaven 3, java 1.8,\ndocker 1.12, node 6 and python 2.7.

Ubuntu

Bootstrapping on Ubuntu requires sudo privileges to install all the required dependencies. You will be prompted to enter your password. Currently only 16.04 (x64) is supported.

% sudo apt-get install git\n% git clone https://github.com/intuit/wasabi.git\n% cd wasabi\n% ./bin/wasabi.sh bootstrap\n% sudo reboot

NOTE: A reboot is required after running the bootstrap command on Ubuntu.

For all other processes (build, start etc.) the commands are same for Ubuntu and Mac OS.

Installed tools include: git 2,\nmaven 3, OpenJdk 8,\ndocker 1.12, node 6 and python 2.7

Similar tooling will work for Windows. Contribute a patch :)

Start Wasabi

Now that we have the necessary tools in place, let's move on to build and start Wasabi, followed by issuing a ping\ncommand to verify the build:

% ./bin/wasabi.sh build start\n...\nwasabi is operational:\n  ui: % open http://localhost:8080     note: sign in as admin/admin\n  ping: % curl -i http://localhost:8080/api/v1/ping\n  debug: attach to localhost:8180\n% curl -i http://localhost:8080/api/v1/ping\nHTTP/1.1 200 OK\nDate: Wed, 25 May 2016 00:25:47 GMT\n...\nX-Application-Id: wasabi-api-20151215171929-SNAPSHOT-development\nContent-Type: application/json\nTransfer-Encoding: chunked\nServer: Jetty(9.3.z-SNAPSHOT)\n{\n  \"componentHealths\":[\n    {\n      \"componentName\":\"Experiments Cassandra\",\n      \"healthy\":true\n    },\n    {\n      \"componentName\":\"MySql\",\"healthy\":true\n    }\n  ],\n  \"wasabiVersion\":\"wasabi-api-20151215171929-SNAPSHOT-development\"\n}

Congratulations! You are the proud owner of a newly minted Wasabi instance. :)

Running Wasabi with remote storage

Set Mysql and Cassandra credentials
Download Cassandra migration tool https://oss.sonatype.org/content/repositories/public/com/builtamont/cassandra-migration/0.9/cassandra-migration-0.9-jar-with-dependencies.jar
Set up your environment variables
export CASSANDRA_MIGRATION=/location/of/cassandra-migration-0.9-jar-with-dependencies.jar
export MIGRATION_SCRIPT=/location/of/modules/repository-datastax/src/main/resources/com/intuit/wasabi/repository/impl/cassandra/migration
Set up Cassandra tables
CQLSH_VERSION=<version> CQLSH_USERNAME=<username> CQLSH_PASSWORD=<pwd> CQLSH_HOST=<host> bin/docker/migration.sh
Run Wasabi with env variables for remote storage hosts
MYSQL_HOST=<mysql_host> NODE_HOST=<cassandra_host> ./bin/wasabi.sh start:wasabi
Run Wasabi outside of docker with WASABI_CONFIGURATION for remote storage hosts
WASABI_CONFIGURATION=\"\n  -Ddatabase.url.host=$MYSQL_HOST\\\n  -Ddatabase.url.port=$MYSQL_PORT\\\n  -Ddatabase.url.dbname=$MYSQL_DATABASE\\\n  -Ddatabase.user=$MYSQL_USER\\\n  -Ddatabase.password=$MYSQL_PASSWORD\\\n  -Ddatabase.pool.connections.min=$MYSQL_MIN_CONNECTIONS\\\n  -Ddatabase.pool.connections.max=$MYSQL_MAX_CONNECTIONS\\\n  -Dusername=$CASSANDRA_USER\\\n  -Dpassword=$CASSANDRA_PASSWORD\\\n  -DnodeHosts=$CASSANDRA_HOST\\\n  -DtokenAwareLoadBalancingLocalDC=$CASSANDRA_DATACENTER\\\n  -Dapplication.http.port=$PORT\" bash usr/local/wasabi-main-*/bin/run

Troubleshooting

% ./bin/wasabi.sh status\nCONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                     NAMES\n8c12458057ef        wasabi-main              \"entrypoint.sh wasabi\"   25 minutes ago      Up 25 minutes       0.0.0.0:8080->8080/tcp, 0.0.0.0:8090->8090/tcp, 0.0.0.0:8180->8180/tcp    wasabi-main\n979ecc885239        mysql:5.6                \"docker-entrypoint.sh\"   26 minutes ago      Up 26 minutes       0.0.0.0:3306->3306/tcp                                                    wasabi-mysql\n2d33a96abdcb        cassandra:2.1            \"/docker-entrypoint.s\"   27 minutes ago      Up 27 minutes       7000-7001/tcp, 0.0.0.0:9042->9042/tcp, 7199/tcp, 0.0.0.0:9160->9160/tcp   wasabi-cassandra
% ./bin/wasabi.sh start:cassandra\n% ./bin/wasabi.sh start:wasabi

Call Wasabi

These are the 3 common REST endpoints that you will use to instrument your client application with Wasabi.

Let's assume that you've created and started an experiment, 'BuyButton,' in the 'Demo_App' application with the following buckets:

You can assign a user with a unique ID (e.g. 'userID1') to the experiment by making this HTTP request:

\n

Assign a user to experiment and bucket:

\n
% curl -H \"Content-Type: application/json\" \\\n    http://localhost:8080/api/v1/assignments/applications/Demo_App/experiments/BuyButton/users/userID1\n{  \n   \"cache\":true,\n   \"payload\":\"green\",\n   \"assignment\":\"BucketA\",\n   \"context\":\"PROD\",\n   \"status\":\"NEW_ASSIGNMENT\"\n}

Now the 'userID1' user is assigned into the 'BucketA' bucket. Let's further record an impression, meaning the user has seen a given experience:

\n

Record an impression:

\n
% curl -H \"Content-Type: application/json\" \\\n    -d \"{\\\"events\\\":[{\\\"name\\\":\\\"IMPRESSION\\\"}]}\" \\\n    http://localhost:8080/api/v1/events/applications/Demo_App/experiments/BuyButton/users/userID1

If the 'userID1' user performs an action such as clicking the Buy button, you'd record that action with the following request:

\n

Record an action:

\n
% curl -H \"Content-Type: application/json\" \\\n    -d \"{\\\"events\\\":[{\\\"name\\\":\\\"BuyClicked\\\"}]}\" \\\n    http://localhost:8080/api/v1/events/applications/Demo_App/experiments/BuyButton/users/userID1

Explore Various Resources

The following developer resources are available:

\n

API: Swagger API playground

\n
% ./bin/wasabi.sh resource:api
\n

Javadoc

\n
% ./bin/wasabi.sh resource:doc
\n

Wasabi UI

\n
% ./bin/wasabi.sh resource:ui
\n

Cassandra: cqlsh shell

\n
% ./bin/wasabi.sh resource:cassandra
\n

MySQL: mysql shell

\n
% ./bin/wasabi.sh resource:mysql
\n

Java Debugger: Remote attach configuration

\n
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8180

Stop Wasabi

Alas, all good things must come to an end. Let's clean things up a bit stop the newly created Wasabi stack:

At this point in time, we now have all the requisite tools installed, and subsequent invocations of Wasabi will\nstart up much more quickly.

% ./bin/wasabi.sh stop

Get Familiar with wasabi.sh

Further, there are a number of additional wasabi.sh options available you should become familiar with:

% ./bin/wasabi.sh --help\n  usage: wasabi.sh [options] [commands]\n  options:\n    -e | --endpoint [ host:port ]          : api endpoint; default: localhost:8080\n    -v | --verify [ true | false ]         : verify installation configuration; default: false\n    -s | --sleep [ sleep-time ]            : sleep/wait time in seconds; default: 30\n    -h | --help                            : help message\n  commands:\n    bootstrap                              : install dependencies\n    build                                  : build project\n    start[:cassandra,mysql,wasabi]         : start all, cassandra, mysql, wasabi\n    test                                   : test wasabi\n    stop[:wasabi,cassandra,mysql]          : stop all, wasabi, cassandra, mysql\n    resource[:ui,api,doc,cassandra,mysql]  : open resource api, javadoc, cassandra, mysql\n    status                                 : display resource status\n    remove[:wasabi,cassandra,mysql]        : remove all, wasabi, cassandra, mysql\n    package                                : build deployable packages\n    release[:start,finish]                 : promote release

Develop

Build and Run Wasabi Server

% mvn package\n% ./bin/wasabi.sh start:cassandra,mysql\n% (cd modules/main/target; \\\n    WASABI_CONFIGURATION=\"-DnodeHosts=localhost -Ddatabase.url.host=localhost\" ./wasabi-main-*-SNAPSHOT-development/bin/run) &\n% curl -i http://localhost:8080/api/v1/ping\n...

The runtime logs can be accessed executing the following command in a another shell:

\n

Viewing runtime logs:

\n
% tail -f modules/main/target/wasabi-main-*-SNAPSHOT-development/logs/wasabi-main-*-SNAPSHOT-development.log

Build and Run Wasabi UI

% cd modules/ui\n% grunt build
% grunt serve

Stop Wasabi

% ./bin/wasabi.sh stop

Now while that was fun, in all likelihood you will be using an IDE to work on Wasabi. In doing so, you need only\nadd the configuration information above to the JVM commandline prior to startup:

\n

Wasabi runtime configuration:

\n
-DnodeHosts=localhost -Ddatabase.url.host=localhost

Run Integration Tests

Code changes can readily be verified by running the growing collection of included integration tests:

% ./bin/wasabi.sh start test stop
Troubleshooting

Integration tests might fail intermittently due to a time drift issue in docker containers on Mac OSX.

When the Mac sleeps and wakes back up, there is a lag created between the clock in the Mac vs the\nrunning docker containers. This is a known issue in Docker for Mac.

This can be fixed by running the following command:

% docker run --rm --privileged alpine hwclock -s

The above command will need to be run every time when there is a time drift.

To automatically run this command and update the time each time the Mac wakes up, you could install\nthe following agent:

% curl https://raw.githubusercontent.com/arunvelsriram/docker-time-sync-agent/master/install.sh | bash

You can read more about this at: quick-tip-fixing-time-drift-issue-on-docker-for-mac

Package and Deploy at Scale

Wasabi can readily be packaged as installable rpm or deb distributions and deployed at scale as follows:

\n

Package by running integration tests 1st:

\n
% ./bin/wasabi.sh start package
\n

Package without integration tests, if needed:

\n
% ./bin/wasabi.sh -t false package
\n

Find generated package files:

\n
% find . -type f \\( -name \"*.rpm\" -or -name \"*.deb\" \\)

Note: Java 8 is a runtime dependency

Integrate

Wasabi is readily embeddable via the following maven dependency GAV family:

<dependency>\n    <groupId>com.intuit.wasabi</groupId>\n    <artifactId>wasabi</artifactId>\n    <version>1.0.20160627213750<build_timestamp></version>\n</dependency>

Contribute

We greatly encourage contributions! You can add new features, report and fix existing bugs, write docs and\ntutorials, or any of the above. Feel free to open issues and/or send pull requests.

The master branch of this repository contains the latest stable release of Wasabi, while snapshots are published to the develop branch. In general, pull requests should be submitted against develop by forking this repo into your account, developing and testing your changes, and creating pull requests to request merges. See the Contributing to a Project\narticle for more details about how to contribute in general and find more specific information on how to write code for Wasabi in our user guide.

Extension projects such as browser plugins, client integration libraries, and apps can be contributed under the contrib directory.

Steps to contribute:

  1. Fork this repository into your account on Github
    2. \n
  2. Clone your forked repository (not our original one) to your hard drive with git clone https://github.com/YOURUSERNAME/wasabi.git
    3. \n
  3. Design and develop your changes
    4. \n
  4. Add/update unit tests
    5. \n
  5. Add/update integration tests
    6. \n
  6. Add/update documentation on gh-pages branch
    7. \n
  7. Create a pull request for review to request merge
    8. \n
  8. Obtain 2 approval squirrels before your changes can be merged
    9. \n

Thank you for your contribution!

","id":"965b9cd4-097c-560f-94b6-00ce67a88391","title":"intuit/wasabi","origin_url":"https://github.com/intuit/wasabi","url":"https://github.com/intuit/wasabi","wallabag_created_at":"2018-09-24T16:18:17+00:00","published_at":null,"published_by":"['']","reading_time":10,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/6b71518933139be280b839b74b254bf883d8ad401517eabea88c5f1d20fc4cc9/intuit/wasabi","tags":["open.source","cassandra","testing"],"description":"Support: Documentation: User Guide, JavaDocs\nA/B Testing Overview: Continuous Integration: \n\n License: ProjectWasabi A/B Testing Service is a real-time, enterprise-grade, 100% API driven project...."},{"content":"

In the previous two posts of this series (Part 1 and Part 2) I covered some of the basic commands of cassandra-stress. In this post I will start looking at the use of the stress YAML file for more advanced stress scenarios, particularly where you want to run stress against a schema that matches one you are planning to use for your application.

It’s worth noting in the intro that cassandra-stress with a YAML file use a significantly (80%?) different set of code to the standard read/write/mixed commands. So, some assumptions and learnings from the standard commands won’t hold for YAML-driven stress. To cite one example, when running based on YAML, cassandra-stress does not validate that data returned from a select has the expected values as it does with read or mixed.

For this article, I’ll reference the following YAML specification file:

Before explaining the contents here, let’s see what happens when we run it with the following simple scenario:
cassandra-stress user profile=file:///eg-files/stressprofilemixed.yaml no-warmup ops(insert=1) n=100 -rate threads=1 -node x.x.x.x

After running this on an empty cluster, I ran select count(*) from eventsrawtest;. The result? 345 rows – probably not you would have guessed. Here’s how cassandra-stress gets to that:

Let’s look at some of the other column settings:
We covered most of the insert settings in the introductory points but here’s a recap:
The final section the yaml file that bears some explanation is the queries section. For each query, you specify:
The ops command specified as part of the command line controls the mix of different operations to run. Take for example the following command:

cassandra-stress user profile=file:///eg-files/stressprofilemixed.yaml ops(insert=1, pull_for_rollup=1, get-value=10) n=120 -node x.x.x.x

This will execute insert batches, pull_for_rollup queries and get-value queries in the ratio 1:1:10. So for this specific example, we’d get 10 inserts, 10 pull_for_rollup queries and 100 get-value queries.

Hopefully that’s explained the key information you need to use a YAML profile for running cassandra stress. In future instalments I’ll take a look at some of the remaining command line options and walk through a full end-to-end example of designing and executing a test.

Click here for Part One: Deep Diving into Cassandra Stress
Click here for Part Two: Mixed Command

","id":"d7cce620-feeb-572c-bc3f-3a4fd803fdf6","title":"Deep Diving cassandra-stress - Part 3 (Using YAML Profiles) - Instaclustr","origin_url":"https://www.instaclustr.com/deep-diving-cassandra-stress-part-3-using-yaml-profiles/","url":"https://www.instaclustr.com/deep-diving-cassandra-stress-part-3-using-yaml-profiles/","wallabag_created_at":"2018-09-13T14:57:14+00:00","published_at":"2016-08-24T05:03:19+00:00","published_by":"['']","reading_time":5,"domain_name":"www.instaclustr.com","preview_picture":"https://www.instaclustr.com/wp-content/uploads/2021/10/cs-article3.png","tags":["stress","cassandra","cassandra.stress","testing"],"description":"In the previous two posts of this series (Part 1 and Part 2) I covered some of the basic commands of cassandra-stress. In this post I will start looking at the use of the stress YAML file for more adv..."},{"content":"

This is the second of a series of articles explain the operations of cassandra-stress. See here for Part 1.

This series of blogs has been created as part of my prep for my Cassandra summit talk ‘Load Testing Cassandra Applications’.

In this post we’ll examine the operation of the mixed command in Cassandra stress and along with that some option for changing the schema used by stress (without using the yaml configuration for a complete custom run).

Firstly, let’s look at the most simple invocation of the mixed command:

cassandra-stress mixed n=100 -node x.x.x.x

First thing to note about this command is that for it to work you need to have run a write command to create the schema and populate data before the run. The write command should be run with at least the same number of operations as the mixed command to ensure that read commands will all retrieve data. For example, to prepare for the above command you should run:

cassandra-stress write n=100 -node x.x.x.x

If you haven’t written the right data before you run the mixed test then you will receive errors like the following:

java.io.IOException: Operation x0 on key(s) [38374d394b504d353430]: Data returned was not validated

This is because cassandra-stress validates the data returned from the select operations against the generated data set expected for the row (and in this case no data was returned).

So, once you’re set up correctly, what will the command actually do? The key default settings that come into play are:

The rate setting illustrates an interesting point in using cassandra-stress: generally, increasing the number of client threads will increase the load (throughput) you are placing on the cluster. This is because all operations are executed synchronously (in order to time them) and so more threads allow more operations to be simultaneously thrown at the cluster. Of course, like any multi-threaded system, there is an overhead incurred managing threads on the stress client and a limit to how much work the client can do before it becomes a bottleneck to the test. So, increase threads to increase the load on your cluster but check when you hit the limit that it’s actually your cluster reaching its limit not your stress client.

That’s it for the mixed command and associated info. Future installments will look at some of the other key options and the yaml configuration file.

Click here for Part One: Deep Diving into Cassandra Stress
Click here for Part Three: Using YAML Profiles

","id":"8514554d-37af-5231-90d2-40e59dc0280b","title":"Deep Diving into cassandra-stress - Part 2 (Mixed Command) - Instaclustr","origin_url":"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-2/","url":"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-2/","wallabag_created_at":"2018-09-13T14:57:04+00:00","published_at":"2016-08-23T04:16:17+00:00","published_by":"['']","reading_time":3,"domain_name":"www.instaclustr.com","preview_picture":"https://www.instaclustr.com/wp-content/uploads/Insta_Website_Social-Share-Thumbnail_rebrand.png","tags":["stress","cassandra","cassandra.stress","testing"],"description":"This is the second of a series of articles explain the operations of cassandra-stress. See here for Part 1.This series of blogs has been created as part of my prep for my Cassandra summit talk ‘Load T..."},{"content":"

Overview

This is the first in a series of blog post I’m planning to create as part of my prep for my Cassandra summit talk ‘Load Testing Cassandra Applications’.

cassandra-stress is a great utility for stress testing Cassandra. However, available documentation is a little sparse and it not always entirely clear what load cassandra-stress will generate in a given situation. In this series of blog posts, I plan to walk through a number of cassandra stress scenarios examining exactly how cassandra-stress behaves.

For this series, I will be using the latest 3.x version of cassandra-stress. If I notice any differences related to a particular point version of Cassandra I will call them out.

In this first post, I will look at what is about the most basic cassandra-stress command you can run:
cassandra-stress write n=10 -node x.x.x.x

I chose this command for two reasons: firstly, using a simple command will allow us to look at some of the basic functions that apply across any cassandra-stress command and secondly, in almost all scenarios, you will want to execute a write to populate a cluster with data before running a read or mixed scenario.

cassandra-stress

Let’s start by looking at that components of the command itself:

Filling in with defaults, here’s all the settings cassandra-stress will actually use for the run (from my in-progress implementation of CASSANDRA-11914):

Step by Step

As you can see, there is a lot going on behind the scenes. So. let’s walk-through step by step what cassandra-stress actually does when you execute this command:

    1. The options provided through the command line are parsed and filled in with a whole range of default options as necessary (will touch on important defaults below and more in future articles). Interestingly, at this stage a valid cassandra.yaml will need to be loaded. However, as far as I could tell this is just a side effect of cassandra-stress using some core Cassandra classes and the contents of the cassandra.yaml have no effect on the actual cassandra-stress operations.
      2. \n
    2. The Cassandra Java driver is used to connect to the node specified in the command line. From this node, the driver retrieves the node list and token map for the cluster and then initiates a connection to each node in the cluster.
      3. \n
    3. A create keyspace (if one doesn’t already exist) command is executed with the following definition:
      While this definition is a reasonable choice for a simple default, it’s important to note that this is unlikely to be representative of a keyspace you would want to run in production. By far the most common production scenario would be to use NetworkTopologyStrategy and a replication factor of 3. To have cassandra-stress create a keyspace with this strategy you would need to drop any existing keyspace and add the following parameters to the cassandra-stress command line:
      -schema replication(strategy=NetworkTopologyStrategy,DC_NAME=3)
      Replace DC_NAME with the actual name of your Cassandra data center. On some systems you may also need to escape the brackets ie. replication\\(...\\)
      4. \n
    4. Once the keyspace is created, cassandra-stress creates two tables in the keyspace: standard1 and counter1. We’ll ignore counter1 for now as it’s not used in this test. The definition of the standard1 table created is as follows:
      While, again, this is a reasonable choice for a simple default, there are a few characteristics to keep in mind if you are trying to draw conclusions from performance using this table definition:\n
      • There is no clustering key, so 1 row per partition – potentially very different performance to a scenario with many rows per partition.
        • \n
      • Compression is disabled – the overhead of compression is typically not huge but could be significant.
        • \n
      • Compact Storage is enabled – this is not enabled by default and will result in smaller representation of the data on disk (although minimal difference with Cassandra 3.x).
        • \n

      I’ll cover options for using different schemas in a later installment of this series.

      5. \n
    5. cassandra-stress will attempt to make a jmx connection to the nodes in the cluster to collect garbage collection stats. If the attempt fails, the run will proceed without collection garbage collection stats.
      6. \n
    6. Next, cassandra-stress will run a warmup. This is a standard practice in load testing to reduce variation from start-up variation such as code being loaded to memory and JVM hotspot compilers. The number of warm-up iterations is the lesser 25% of the target of operations or 50k – 2 operations in this trivial example. The warm-up operations are basically the same as the test operations except not timed so I won’t go into them in detail.
      7. \n
    7. We’ve entered the actual load test phase. cassandra-stress creates 200 client threads and begins executing the target number of operations. In a real test, using the -rate option to control the number of client threads is a good way to control load on the cluster.
      The first attempted operation will create a CQL prepared statement as follows:
      UPDATE \"standard1\" SET \"C0\" = ?,\"C1\" = ?,\"C2\" = ?,\"C3\" = ?,\"C4\" = ? WHERE KEY=?
      Although we were probably expecting an INSERT statement, updates and inserts are identical in terms of Cassandra implementation so we can expect performance to be the same.This prepared statement will then be will be executed 10 times with different, random data generated for each execution. The statement will be executed with consistent level LOCAL_ONE.cassandra-stress seeds the random generation with a static string plus the column name and a seed number which for the write command defaults to sequentially used numbers from 1 to the number of operations. That means that each column will get different values but the set of values generated will be the same over multiples runs. Generating a static set of values is necessary for read tests but does have the side effect that if you were to run our sample operation (write n=10) 1000 times the end result would still be just 10 rows of data in the table.
      8. \n
    8. Finally, cassandra-stress prints its results. Here’s an example from a run of this command:\n

      Many of these results are self explanatory but some bear further explanation:
      Op rate is the rate of execution commands. Partition rate is that rate that partitions were visited (updated or read) by those commands and Row rate is that rate that rows were visited. For simple, single-row commands all three rates will be equal. The rates will vary in more complex scenarios where a single operation might visit multiple partitions and rows.
      Similarly, Total partitions is the total number of partitions visited during the test. It’s worth noting that this is not unique partitions so even in some write-only scenarios it may not reflect the total number of partitions created by the test.
      The GC statistics report on garbage collection and are zero in this case as JMX ports were blocked to the test cluster.

      9. \n
    2. \n

Conclusion

Well, that’s a lot to write about a simple test that inserts 10 rows into a table. Putting it together has helped improved my understanding of cassandra-stress, I hope it’s useful for you too. In future installments, I’ll look some more into the different data generation operations, mixed test and customers schemas using the YAML configuration file. Let me know in the comments if there are any particular areas of interest for the future articles.

Click here for Part Two: Mixed Command
Click here for Part Three: Using YAML Profiles

","id":"25e43042-097b-5768-9b7d-2c8713b3e074","title":"Deep Diving into cassandra-stress (Part 1) - Instaclustr","origin_url":"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-1/","url":"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-1/","wallabag_created_at":"2018-09-13T14:56:56+00:00","published_at":"2016-08-19T01:20:07+00:00","published_by":"['']","reading_time":6,"domain_name":"www.instaclustr.com","preview_picture":"https://www.instaclustr.com/wp-content/uploads/Insta_Website_Social-Share-Thumbnail_rebrand.png","tags":["stress","cassandra","cassandra.stress","testing"],"description":"OverviewThis is the first in a series of blog post I’m planning to create as part of my prep for my Cassandra summit talk ‘Load Testing Cassandra Applications’.cassandra-stress is a great utility for ..."}],"tagSets":[{"tag":"cassandra","articles":[{"content":"

This is the second post in my series on improving node density and lowering costs with Apache Cassandra. In the previous post, I examined how streaming performance impacts node density and operational costs. In this post, I’ll focus on compaction throughput, and a recent optimization in Cassandra 5.0.4 that significantly improves it, CASSANDRA-15452.

This post assumes some familiarity with Apache Cassandra storage engine fundamentals. The documentation has a nice section covering the storage engine if you’d like to brush up before reading this post.

The Compaction Bottleneck

Compaction in Cassandra is the process of merging multiple SSTables and writing out new ones, discarding tombstones, resolving overwrites, and generally organizing data for efficient reads. It’s an I/O intensive background operation that directly competes with foreground operations for system resources. In a later post I’ll look at how compaction strategies impact node density, but for now, I’ll just focus on throughput.

Why Compaction Throughput Matters for Node Density

As we continue to increase the amount of data we store per node, compaction performance becomes increasingly important. It affects:

Simply put: as your data volume and write throughput increase, compaction throughput must as well. If it doesn’t, you’ll hit a performance wall that effectively caps your maximum practical node density.

Despite the significant improvements to compaction throughput over the years, there are some circumstances where compaction performance is inadequate. Let’s take a look at the reason why, then dive into what can be done about it.

When doing any performance evaluation, it’s important to understand how to measure where your time is spent. A lot of folks make incorrect assumptions, and then waste a lot of time trying to optimize something that doesn’t matter. I’ve written several posts about how useful profiling with the async-profiler can be from an application perspective. For looking at the OS and hardware, the eBPF based toolkit bcc-tools can help you identify process bottlenecks. I’ve used these tools extensively over the years, and in this post I’ll show how they’ve helped identify two major performance bottlenecks in compaction. My easy-cass-lab software includes all these tools, as well as integration with AxonOps for Cassandra dashboards and operational tooling.

Being 10x Smarter With Our Disk Access

When investigating compaction behavior, I discovered an major inefficiency in how Cassandra was accessing disk. The problem was especially severe in cloud environments with disaggregated storage like AWS EBS, where IOPS (Input/Output Operations Per Second) are both limited and expensive when used improperly.

When Cassandra would read in data during compaction, it would read individual compressed chunks off disk, one small read at a time. Using bcc-tools, we can monitor every filesystem operation. Here I’m using xfsslower to record every read operation on the filesystem (original headers back in for clarity):

$ sudo /usr/share/bcc/tools/xfsslower 0 -p 26988 | awk '$4 == \"R\" { print $0 }'\nTracing XFS operations\nTIME     COMM           PID    T BYTES   OFF_KB   LAT(ms) FILENAME\n22:27:38 CompactionExec 26988  R 4096    0           0.01 nb-7-big-Statistics.db\n22:27:38 CompactionExec 26988  R 4096    4           0.00 nb-7-big-Statistics.db\n22:27:38 CompactionExec 26988  R 2062    8           0.00 nb-7-big-Statistics.db\n22:27:38 CompactionExec 26988  R 14907   0           0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14924   14          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14896   29          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14844   43          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14923   58          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14931   72          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14905   87          0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14891   101         0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14919   116         0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14965   130         0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14918   145         0.01 nb-7-big-Data.db\n22:27:38 CompactionExec 26988  R 14930   160         0.01 nb-7-big-Data.db\n

The above is showing we’re reading about 14KB at a time. That’s the size of the compressed page. This pattern is terrible for performance on cloud storage systems like EBS, where:

  1. Each read operation, no matter how small, counts against your provisioned IOPS
    2. \n
  2. Small reads waste IOPS quota while delivering minimal data
    3. \n
  3. You pay for IOPS allocation whether you use it efficiently or not
    4. \n

Looking at a wall clock performance profile, we can see compaction is spending a LOT of time waiting on disk, in the really wide column with the pread call at the top:

\"wall-clock-profile-compaction.png\"

Readahead is a disk optimization strategy where the operating system reads a larger block of data than was requested into memory. The objective is reduce latency and improve performance for sequential read operations. Unfortunately, when you don’t need the data it’s reading, it can be the source of major performance problems. In my experience, read ahead is one of the worst culprits in the world of Cassandra performance. It’s especially terrible for lightweight transactions and counters, where we perform a read before write.

My advice to Cassandra operators is to reduce readahead to 4KB to avoid unnecessary read amplification on the read path.

Readahead does have one place, however, where it can benefit performance. You may have already guessed that it’s compaction. Let’s take a step back and look at how the size of our reads impacts our throughput in a simple benchmark. Larger reads, initiated either from read ahead or the user, should deliver improved throughput, especially when we’re dealing with a quota on our IOPS (EBS), our drives have higher latency (SAN), or both.

Benchmarking

I ran benchmark tests with sequential fio workloads using different request sizes on a 3K IOPS GP3 EBS volume. Here’s the configuration used:

[global]\nrw=read\ndirectory=data\ndirect=1\ntime_based=1\nfile_service_type=normal\nstonewall\nsize=100M\nnumjobs=12\ngroup_reporting\n[bs4]\nstonewall\nruntime=60s\nblocksize=4k\n[bs8]\nstonewall\nruntime=60s\nblocksize=8k\n[bs16]\nstonewall\nruntime=60s\nblocksize=16k\n[bs32]\nstonewall\nruntime=60s\nblocksize=32k\n[bs64]\nstonewall\nruntime=60s\nblocksize=64k\n[bs128]\nstonewall\nblocksize=128k\nruntime=60s\n[bs256]\nstonewall\nruntime=60s\nblocksize=256k\n

When reviewing the results, the benefits of using larger request sizes were evident:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Request SizeIOPSThroughput
4K304911.9 MB/s
8K301223 MB/s
16K301347 MB/s
32K301394 MB/s
64K1938121 MB/s
128K957120 MB/s
256K478120 MB/s

The data shows that using 256KB reads instead of 16KB reads would deliver almost 3x the throughput while using only 1/6th of the provisioned IOPS. That’s a massive efficiency improvement. Rather than chewing through all our IOPS to deliver a paltry 47MB/s of throughput, we’re only using about 500 for 120MB/s. That means if we can see these gains in the database, we’ll be able to compact faster, put more data on each node, and lower our total cost.

The Solution: Internally Buffering Sequential Reads

In CASSANDRA-15452, I worked with my fellow Cassandra committer Jordan West to implement a solution: an efficient, internal read-ahead buffer for bulk reading operations. Here’s how it works:

  1. Instead of reading tiny chunks, we use a 256KB off-heap buffer
    2. \n
  2. Each read operation pulls in a full 256KB of data at once
    3. \n
  3. Compressed chunks are extracted from this buffer as needed
    4. \n
  4. The buffer is refilled only when necessary
    5. \n

This approach maximizes IOPS efficiency by using larger reads during compaction (as well as repair and range reads) that deliver more data per operation. For cloud environments, it’s a game-changer that directly aligns with storage provider recommendations. AWS EBS, for instance, considers any I/O operation up to 256KB as a single operation, so by using the largest possible size we should get optimal performance.

Real-World Impact: A Major Improvement in Compaction Throughput

When Jordan and I tested the implementation using easy-cass-lab on EBS, the results were nothing short of spectacular. The 10.0.2.171 node is running our patched version, the other two nodes are running an unpatched release. The graphs clearly show a 2-3x improvement to throughput and a 3x reduction in IOPS.

\"15452-bytes-read.png\"

\"Compaction

You can see the results in the flamegraph as well. The calls to pread take up significantly less time.

\"wall-clock-profile-compaction-after-15452.png\"

We can use xfsslower from bcc-tools again to watch the filesystem access:

$ sudo /usr/share/bcc/tools/xfsslower 0 -p $(cassandra-pid) | awk '$4 == \"R\" { print $0 }'\nTIME     COMM           PID    T BYTES   OFF_KB   LAT(ms) FILENAME\n14:40:29 CompactionExec 1782   R 262144  256         0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  512         0.06 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  768         0.06 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1024        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1280        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1536        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 241123  1792        0.07 nb-4-big-Data.db\n

This is a lot better, now we’re fetching 256KB at a time using way fewer requests.

The EBS test configuration used a GP3 volume with 3K IOPS and 256MB throughput. With the existing code, compaction was bottlenecked by IOPS, peaking at exactly 3K IOPS but achieving only about 51MB/s throughput. With our optimization, the same operation used only ~500 IOPS to achieve around 106MB/s—a more than 2x improvement in throughput with 1/3IOPS.

In our most aggressive testing, we actually hit the EBS throughput limit rather than the IOPS limit. That’s a significant transformation in Cassandra’s resource utilization profile.

The patch also has the benefit of applying to anti-compaction, repair, and range reads. We can see a significant reduction in range reads, aka table scans:

\"15452-range-reads.png\"

If you’re running Spark jobs using the Cassandra connector, you should see an improvement in performance, and your repair times should decrease.

What’s next? Can we do more?

Yes, absolutely! There’s several more improvements to IO that will help improve things. I’ll cover them here very quickly, and if there’s interest I’ll write about them in detail in a future post.

Avoid Reading the Statistics

When compacting, we read data out of the Statistics.db file before reading the data itself. This is completely unnecessary, as it’s stats about the data we’re about to read. Skipping this can reduce IO even further. Looking at a compaction’s IO activity, I see about 30% of the filesystem access is reading from Statistics.db:

14:40:29 CompactionExec 1782   R 4096    0           0.00 nb-3-big-Statistics.db\n14:40:29 CompactionExec 1782   R 701     4           0.00 nb-3-big-Statistics.db\n14:40:29 CompactionExec 1782   R 4096    0           0.00 nb-4-big-Statistics.db\n14:40:29 CompactionExec 1782   R 4096    4           0.00 nb-4-big-Statistics.db\n14:40:29 CompactionExec 1782   R 1962    8           0.00 nb-4-big-Statistics.db\n14:40:29 CompactionExec 1782   R 262144  0           0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 2115    0           0.01 nb-3-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  256         0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  512         0.06 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  768         0.06 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1024        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1280        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 262144  1536        0.07 nb-4-big-Data.db\n14:40:29 CompactionExec 1782   R 241123  1792        0.07 nb-4-big-Data.db\n

This has already been fixed in trunk by Branimir Lambov in CASSANDRA-20092 and is being backported to 5.0 by Jordan.

Direct I/O for Compaction

Let’s talk more about page cache. Since we go through the Linux page cache when doing reads, we want to make sure it’s working optimally. Page cache lets us avoid going to disk! Unfortunately we also use it when reading for compaction. This is a problem because we’re pulling data into the page cache that we plan on deleting. To make room for the new data, other data will be evicted. If we compact 10GB of data, we’re pushing out a lot of valuable data from the page cache, meaning it needs to be fetched back into memory later on. Using Direct I/O we can bypass the page cache entirely, which will prevent data from being evicted. This can be a huge help in latency sensitive systems or systems where IOPS are limited like EBS.

I’ve filed CASSANDRA-19987 to look at this.

Non Blocking Compression

Next, compression. When we’re writing to disk, we fill a buffer, sized by the chunk_length_in_kb table setting, compress, and write to disk. The compression here is a blocking call, which means we can spend a lot of time waiting on compression to finish, when we could be reading and merging the next chunk in parallel This can show up as a performance bottleneck, so I’ve filed CASSANDRA-20085 to look into it.

Better Memory Management

When a system is not bottlenecked on disk I/O, such as when using NVMe, the main issue we’ll run into is our heap allocation rate. I’ll go into details in a future post, but for now, it’s enough to know that the more memory we allocate, the worse our performance. Being smart about memory allocations can make a big difference in overall time spent, as allocations aren’t free. It also reduces both the frequency and duration of Garbage Collection. Big wins all around.

I recently profiled an instance where the row size was about 2KB (not out of the ordinary) and found that a single call was accounting for roughly 50% of memory allocated. Fixing this one thing has the potential to deliver a massive performance improvement, especially in workloads where we have either lots of fields, or large fields like serialized blobs.

Reaching again for async-profiler, this time we run it with -e alloc to track allocations and --reverse to reverse the stacks. I do this because the same underlying call comes from the read path and compaction, and I want to see the time in aggregate.

\"allocation-profile-compaction.png\"

Addressing this single allocation won’t just deliver faster compaction, but will reduce pressure on the heap, which in turn reduces GC overhead. As part of this series I’ll also be covering GC, as a lot’s changed since I wrote about it last.

I’ve filed CASSANDRA-20428 and there’s already a fair bit of discussion about different approaches to solving the problem.

Conclusion

Maximizing compaction throughput is critical for achieving higher node density with Apache Cassandra. The improvements in CASSANDRA-15452 have removed one of the primary bottlenecks that previously limited practical node size in a lot of clusters.

By upgrading to Cassandra 5.0.4 (or later) you can:

  1. Dramatically improve compaction throughput
    2. \n
  2. Reduce IOPS consumption significantly
    3. \n
  3. Improve overall system stability during write-heavy workloads
    4. \n
  4. Increase the maximum practical data density per node
    5. \n
  5. Significantly reduce your cloud storage costs
    6. \n

This improvement, combined with the streaming optimizations discussed in the previous post, creates a multiplier effect on your ability to increase node density. Each optimization removes a bottleneck, allowing you to push your hardware further and achieve more with less.

In my next post, I’ll be discussing how and why compaction strategies affect node density. Picking the right strategy can have a significant impact on your cluster’s performance and cost efficiency. Make sure you sign up for my mailing list if you’re interested in getting notified when it’s released!

If you found this post helpful, please consider sharing to your network. I'm also available to help you be successful with your distributed systems! Pleasereach outif you're interested in working with me, and I'll be happy to schedule a free one-hour consultation.","id":"716f9c80-d6d7-54c8-afdd-766f03897674","title":"Cassandra Compaction Throughput Performance Explained","origin_url":"https://rustyrazorblade.com/post/2025/04-compaction-throughput/","url":"https://rustyrazorblade.com/post/2025/04-compaction-throughput/","wallabag_created_at":"2025-04-24T12:03:02+00:00","published_at":"2025-04-16T00:00:00+00:00","published_by":"['']","reading_time":14,"domain_name":"rustyrazorblade.com","preview_picture":"https://rustyrazorblade.com/images/2025/wall-clock-profile-compaction.png","tags":["cassandra","performance"],"description":"This is the second post in my series on improving node density and lowering costs with Apache Cassandra. In the previous post, I examined how streaming performance impacts node density and operational..."},{"content":"

Welcome to the Awesome Accord repository! This guide provides resources and examples for implementing ACID transactions in Apache Cassandra. Learn how to leverage distributed transactions for building reliable applications.

Accord is in active development and still a feature branch in the Apasche Cassandra® Repo. You will find bug. What we ask is that you help with a contribution of a bug report.

You can use the Github discussions bug report forum for this or use the Planet Cassandra Discord channel for accord listed below. A bug report should have the folowing:

If you have suggestions about syntax or improving the overall developer expirience, we want to hear about that to! Add it as a suggestion or feature request using Github discussions or let us know in the Planet Cassandra Discord.

Now, on to the fun!

docker pull pmcfadin/cassandra-accord\ndocker run -d --name cassandra-accord -p 9042:9042 pmcfadin/cassandra-accord
brew tap rustyrazorblade/rustyrazorblade\nbrew install easy-cass-lab
/\n├── docker/              # Docker configuration and setup\n├── easy-cass-lab/      # Multi-node testing environment\n├── examples/           # Implementation examples\n│   ├── banking/       # Financial transaction examples\n│   ├── inventory/     # Stock management examples\n│   └── user-mgmt/     # User operations examples\n└── docs/              # Guides and documentation\n

Our documentation includes:

  1. Choose your deployment option:\n
    2. \n
  2. Follow the Quick Start Guide
    3. \n
  3. Explore example implementations
    4. \n
  4. Connect with our Discord community
    5. \n
  5. Feedback! Github Discussions
    6. \n
BEGIN TRANSACTION\n    LET fromBalance = (SELECT account_balance \n                      FROM ks.accounts \n                      WHERE account_holder='alice');\n    IF fromBalance.account_balance >= 20 THEN\n        UPDATE ks.accounts \n        SET account_balance -= 20 \n        WHERE account_holder='alice';\n        UPDATE ks.accounts \n        SET account_balance += 20 \n        WHERE account_holder='bob';\n    END IF\nCOMMIT TRANSACTION;

Apache License 2.0

","id":"19f1d3b7-37d9-5bb4-a0f6-b15fa788df22","title":"GitHub - pmcfadin/awesome-accord: Repository of all kinds of things to help you get up and running with ACID transactions on Apache Cassandra®","origin_url":"https://github.com/pmcfadin/awesome-accord","url":"https://github.com/pmcfadin/awesome-accord","wallabag_created_at":"2025-01-16T16:28:31+00:00","published_at":null,"published_by":"['pmcfadin']","reading_time":1,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/3e477fb2dd2b1ded1c5b53477f4848297badc75ece00c5b49bad1476fdb76167/pmcfadin/awesome-accord","tags":["acid","open.source","cassandra","accord"],"description":"Welcome to the Awesome Accord repository! This guide provides resources and examples for implementing ACID transactions in Apache Cassandra. Learn how to leverage distributed transactions for building..."},{"content":"

Visual Flow is an ETL tool designed for effective data manipulation via convenient and user-friendly interface. The tool has the following capabilities:

Visual Flow application is divided into the following repositories:

Check the official guide.

Visual flow is an open-source software licensed under the Apache-2.0 license.

","id":"988e1911-975f-50b5-849a-7e9d4dd5fe35","title":"GitHub - ibagroup-eu/Visual-Flow: Visual-Flow main repository","origin_url":"https://github.com/ibagroup-eu/Visual-Flow","url":"https://github.com/ibagroup-eu/Visual-Flow","wallabag_created_at":"2024-12-02T13:34:31+00:00","published_at":null,"published_by":"['ibagroup-eu']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/9187fdecad3a37939c1971bcdec19ffed4090307ee508b009f47c7bcd49a7f8d/ibagroup-eu/Visual-Flow","tags":["mongo","nocode","elasticsearch","open.source","cassandra","data.pipeline","elastic","aws.s3","etl","low.code","postgres"],"description":"Visual Flow is an ETL tool designed for effective data manipulation via convenient and user-friendly interface. The tool has the following capabilities:Can integrate data from heterogeneous sources:\nA..."},{"content":"

\"GitHub \"Go

\"cql-proxy\"

cql-proxy is designed to forward your application's CQL traffic to an appropriate database service. It listens on a local address and securely forwards that traffic.

The cql-proxy sidecar enables unsupported CQL drivers to work with DataStax Astra. These drivers include both legacy DataStax drivers and community-maintained CQL drivers, such as the gocql driver and the rust-driver.

cql-proxy also enables applications that are currently using Apache Cassandra or DataStax Enterprise (DSE) to use Astra without requiring any code changes. Your application just needs to be configured to use the proxy.

If you're building a new application using DataStax drivers, cql-proxy is not required, as the drivers can communicate directly with Astra. DataStax drivers have excellent support for Astra out-of-the-box, and are well-documented in the driver-guide guide.

Use the -h or --help flag to display a listing all flags and their corresponding descriptions and environment variables (shown below as items starting with $):

$ ./cql-proxy -h\nUsage: cql-proxy\nFlags:\n  -h, --help                                              Show context-sensitive help.\n  -b, --astra-bundle=STRING                               Path to secure connect bundle for an Astra database. Requires '--username' and '--password'. Ignored if using the\n                                                          token or contact points option ($ASTRA_BUNDLE).\n  -t, --astra-token=STRING                                Token used to authenticate to an Astra database. Requires '--astra-database-id'. Ignored if using the bundle path\n                                                          or contact points option ($ASTRA_TOKEN).\n  -i, --astra-database-id=STRING                          Database ID of the Astra database. Requires '--astra-token' ($ASTRA_DATABASE_ID)\n      --astra-api-url=\"https://api.astra.datastax.com\"    URL for the Astra API ($ASTRA_API_URL)\n      --astra-timeout=10s                                 Timeout for contacting Astra when retrieving the bundle and metadata ($ASTRA_TIMEOUT)\n  -c, --contact-points=CONTACT-POINTS,...                 Contact points for cluster. Ignored if using the bundle path or token option ($CONTACT_POINTS).\n  -u, --username=STRING                                   Username to use for authentication ($USERNAME)\n  -p, --password=STRING                                   Password to use for authentication ($PASSWORD)\n  -r, --port=9042                                         Default port to use when connecting to cluster ($PORT)\n  -n, --protocol-version=\"v4\"                             Initial protocol version to use when connecting to the backend cluster (default: v4, options: v3, v4, v5, DSEv1,\n                                                          DSEv2) ($PROTOCOL_VERSION)\n  -m, --max-protocol-version=\"v4\"                         Max protocol version supported by the backend cluster (default: v4, options: v3, v4, v5, DSEv1, DSEv2)\n                                                          ($MAX_PROTOCOL_VERSION)\n  -a, --bind=\":9042\"                                      Address to use to bind server ($BIND)\n  -f, --config=CONFIG                                     YAML configuration file ($CONFIG_FILE)\n      --debug                                             Show debug logging ($DEBUG)\n      --health-check                                      Enable liveness and readiness checks ($HEALTH_CHECK)\n      --http-bind=\":8000\"                                 Address to use to bind HTTP server used for health checks ($HTTP_BIND)\n      --heartbeat-interval=30s                            Interval between performing heartbeats to the cluster ($HEARTBEAT_INTERVAL)\n      --idle-timeout=60s                                  Duration between successful heartbeats before a connection to the cluster is considered unresponsive and closed\n                                                          ($IDLE_TIMEOUT)\n      --readiness-timeout=30s                             Duration the proxy is unable to connect to the backend cluster before it is considered not ready\n                                                          ($READINESS_TIMEOUT)\n      --idempotent-graph                                  If true it will treat all graph queries as idempotent by default and retry them automatically. It may be\n                                                          dangerous to retry some graph queries -- use with caution ($IDEMPOTENT_GRAPH).\n      --num-conns=1                                       Number of connection to create to each node of the backend cluster ($NUM_CONNS)\n      --proxy-cert-file=STRING                            Path to a PEM encoded certificate file with its intermediate certificate chain. This is used to encrypt traffic\n                                                          for proxy clients ($PROXY_CERT_FILE)\n      --proxy-key-file=STRING                             Path to a PEM encoded private key file. This is used to encrypt traffic for proxy clients ($PROXY_KEY_FILE)\n      --rpc-address=STRING                                Address to advertise in the 'system.local' table for 'rpc_address'. It must be set if configuring peer proxies\n                                                          ($RPC_ADDRESS)\n      --data-center=STRING                                Data center to use in system tables ($DATA_CENTER)\n      --tokens=TOKENS,...                                 Tokens to use in the system tables. It's not recommended ($TOKENS)

To pass configuration to cql-proxy, either command-line flags, environment variables, or a configuration file can be used. Using the docker method as an example, the following samples show how the token and database ID are defined with each method.

docker run -p 9042:9042 \\\n  --rm datastax/cql-proxy:v0.1.5 \\\n  --astra-token <astra-token> --astra-database-id <astra-datbase-id>
docker run -p 9042:9042  \\\n  --rm datastax/cql-proxy:v0.1.5 \\\n  -e ASTRA_TOKEN=<astra-token> -e ASTRA_DATABASE_ID=<astra-datbase-id>

Proxy settings can also be passed using a configuration file with the --config /path/to/proxy.yaml flag. This can be mixed and matched with command-line flags and environment variables. Here are some example configuration files:

contact-points:\n  - 127.0.0.1\nusername: cassandra\npassword: cassandra\nport: 9042\nbind: 127.0.0.1:9042\n# ...

or with a Astra token:

astra-token: <astra-token>\nastra-database-id: <astra-database-id>\nbind: 127.0.0.1:9042\n# ...

All configuration keys match their command-line flag counterpart, e.g. --astra-bundle is astra-bundle:, --contact-points is contact-points: etc.

Multi-region failover with DC-aware load balancing policy is the most useful case for a multiple proxy setup.

When configuring peers: it is required to set --rpc-address (or rpc-address: in the yaml) for each proxy and it must match is corresponding peers: entry. Also, peers: is only available in the configuration file and cannot be set using a command-line flag.

Here's an example of configuring multi-region failover with two proxies. A proxy is started for each region of the cluster connecting to it using that region's bundle. They all share a common configuration file that contains the full list of proxies.

Note: Only bundles are supported for multi-region setups.

cql-proxy --astra-bundle astra-region1-bundle.zip --username token --password <astra-token> \\\n  --bind 127.0.0.1:9042 --rpc-address 127.0.0.1 --data-center dc-1 --config proxy.yaml
cql-proxy ---astra-bundle astra-region2-bundle.zip --username token --password <astra-token> \\\n  --bind 127.0.0.2:9042 --rpc-address 127.0.0.2 --data-center dc-2 --config proxy.yaml

The peers settings are configured using a yaml file. It's a good idea to explicitly provide the --data-center flag, otherwise; these values are pulled from the backend cluster and would need to be pulled from the system.local and system.peers table to properly setup the peers data-center: values. Here's an example proxy.yaml:

peers:\n  - rpc-address: 127.0.0.1\n    data-center: dc-1\n  - rpc-address: 127.0.0.2\n    data-center: dc-2

Note: It's okay for the peers: to contain entries for the current proxy itself because they'll just be omitted.

There are three methods for using cql-proxy:

  1. \n

    Build cql-proxy.

    \n
    go build
    \n
    2. \n
  2. \n

    Run with your desired database.

    \n
    3. \n
  1. \n

    Run with your desired database.

    \n
    2. \n

If you wish to have the docker image removed after you are done with it, add --rm before the image name datastax/cql-proxy:v0.1.5.

Using Kubernetes with cql-proxy requires a number of steps:

  1. \n

    Generate a token following the Astra instructions. This step will display your Client ID, Client Secret, and Token; make sure you download the information for the next steps. Store the secure bundle in /tmp/scb.zip to match the example below.

    \n
    2. \n
  2. \n

    Create cql-proxy.yaml. You'll need to add three sets of information: arguments, volume mounts, and volumes. A full example can be found here.

    \n
    3. \n
  1. \n

    Create a configmap. Use the same secure bundle that was specified in the cql-proxy.yaml.

    \n
    kubectl create configmap cql-proxy-configmap --from-file /tmp/scb.zip
    \n
    2. \n
  2. \n

    Check the configmap that was created.

    \n
    kubectl describe configmap cql-proxy-configmap\n  Name:         cql-proxy-configmap\n  Namespace:    default\n  Labels:       <none>\n  Annotations:  <none>\n  Data\n  ====\n  BinaryData\n  ====\n  scb.zip: 12311 bytes
    \n
    3. \n
  3. \n

    Create a Kubernetes deployment with the YAML file you created:

    \n
    kubectl create -f cql-proxy.yaml
    \n
    4. \n
  4. \n

    Check the logs:

    \n
    kubectl logs <deployment-name>
    \n
    5. \n

Drivers that use token-aware load balancing may print a warning or may not work when using cql-proxy. Because cql-proxy abstracts the backend cluster as a single endpoint this doesn't always work well with token-aware drivers that expect there to be at least \"replication factor\" number of nodes in the cluster. Many drivers print a warning (which can be ignored) and fallback to something like round-robin, but other drivers might fail with an error. For the drivers that fail with an error it is required that they disable token-aware or configure the round-robin load balancing policy.

","id":"46e6ccfe-7ae3-5b01-9b45-db01f8f06c62","title":"GitHub - datastax/cql-proxy: A client-side CQL proxy/sidecar.","origin_url":"https://github.com/datastax/cql-proxy","url":"https://github.com/datastax/cql-proxy","wallabag_created_at":"2024-11-01T17:26:01+00:00","published_at":null,"published_by":"['datastax']","reading_time":8,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/c2528e3426d98910ed27819e048b4c1081fab2ed2c7adbea6e6a3b1872deb30a/datastax/cql-proxy","tags":["migration","proxy","cassandra","cql"],"description":" cql-proxy is designed to forward your application's CQL traffic to an appropriate database service. It listens on a local address and securely forwards that traffic.The cql-proxy sidecar enables unsu..."}]},{"tag":"github","articles":[{"content":"

A new blog post covering each of the main components of this project can be found here:

\n

http://thelastpickle.com/blog/2018/01/23/docker-meet-cassandra.html

\n
git clone git@github.com:thelastpickle/docker-cassandra-bootstrap.git\ncd docker-cassandra-bootstrap\ncp .env.template .env\ndocker-compose build
\n

If you would like to see a hosted log service interact seemlessly with this Docker Compose stack, sign up for Papertrail.

\n

Then find your specific port number by looking at your Log Destinations and update your .env setting accordingly.

\n
# turn off all running Docker containers\ndocker-compose down\n# delete any persistent data\nrm -rf data/\n# rebuild the images\ndocker-compose build
\n

Start our Docker-integrated logging connector:

\n
# start Docker logging connector\ndocker-compose up logspout\n# view logging HTTP endpoint\ncurl http://localhost:8000/logs
\n

Start Cassandra and setup the required schema:

\n
# start Cassandra\ndocker-compose up cassandra\n# view cluster status\ndocker-compose run nodetool status\n# create schema\ndocker-compose run cqlsh -f /schema.cql\n# confirm schema\ndocker-compose run cqlsh -e \"DESCRIBE SCHEMA;\"
\n

Start Reaper for Apache Cassandra and monitor your new cluster:

\n
# start Reaper for Apache Cassandra\ndocker-compose up cassandra-reaper\nopen http://localhost:8080/webui/\n# add one-off repair\n# add scheduled repair
\n

Start Prometheus and become familiar with the UI:

\n
# start Prometheus\ndocker-compose up prometheus\nopen http://localhost:9090
\n

Start Grafana, connect it to the Prometheus data source, and upload the TLP Dashboards.

\n
# start Grafana\ndocker-compose up grafana\n# create \n./grafana/bin/create-data-sources.sh\n# user/pass: admin/admin\nopen http://localhost:3000\n# upload dashboards\n./grafana/bin/upload-dashboards.sh
\n

Generate fake workforce and activity:

\n
docker-compose run pickle-factory
\n

Sample timesheets:

\n
docker-compose run pickle-shop
","id":"f950326f-a534-5983-9f77-7ebbc14f932b","title":"thelastpickle/docker-cassandra-bootstrap","origin_url":"https://github.com/thelastpickle/docker-cassandra-bootstrap","url":"https://github.com/thelastpickle/docker-cassandra-bootstrap","wallabag_created_at":"2018-05-25T21:15:40+00:00","published_at":null,"published_by":"['']","reading_time":1,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/3e8c3c57c7b7abfb9ce896d1ccf7a27ce34b383eec2d79d3b8ffc4ab6ef4ffef/thelastpickle/docker-cassandra-bootstrap","tags":["cassandra","github","docker"],"description":"A new blog post covering each of the main components of this project can be found here:\nhttp://thelastpickle.com/blog/2018/01/23/docker-meet-cassandra.html\ngit clone git@github.com:thelastpickle/docke..."},{"content":"

Theory of Operation

Tablesnap is a script that uses inotify to monitor a directory for IN_MOVED_TO\nevents and reacts to them by spawning a new thread to upload that file to\nAmazon S3, along with a JSON-formatted list of what other files were in the\ndirectory at the time of the copy.

When running a Cassandra cluster, this behavior can be quite useful as it\nallows for automated point-in-time backups of SSTables. Theoretically,\ntablesnap should work for any application where files are written to some\ntemporary location, then moved into their final location once the data is\nwritten to disk. Tablesnap also makes the assumption that files are immutable\nonce written.

Installation

The simplest way to install tablesnap is from the Python Package Index, PyPI.\nhttps://pypi.python.org/pypi/tablesnap

pip install tablesnap\n

This distribution provides a debian/ source directory, allowing it to be built\nas a standard Debian/Ubuntu package and stored in a repository. The Debian\npackage includes an init script that can run and daemonize tablesnap for you.\nTablesnap does not daemonize itself. This is best left to tools like\ninit, supervisord, daemontools, etc.

We do not currently maintain binary packages of tablesnap. To build the debian\npackage from source, assuming you have a working pbuilder environment:

git checkout debian\ngit-buildpackage --git-upstream-branch=master --git-debian-branch=debian --git-builder='pdebuild'\n

The daemonized version of the Debian/Ubuntu package uses syslog for logging.\nThe messages are sent to the DAEMON logging facility and tagged with\ntablesnap. If you want to redirect the log output to a log file other than\n/var/log/daemon.log you can filter by this tag. E.g. if you are using\nsyslog-ng you could add

# tablesnap\nfilter f_tablesnap { filter(f_daemon) and match(\"tablesnap\" value(\"PROGRAM\")); };\ndestination d_tablesnap { file(\"/var/log/tablesnap.log\"); };\nlog { source(s_src); filter(f_tablesnap); destination(d_tablesnap); flags(final); };\n

to /etc/syslog-ng/syslog-ng.conf.

If you are not a Debian/Ubuntu user or do not wish to install the tablesnap\npackage, you may copy the tablesnap script anywhere you'd like and run it from\nthere. Tablesnap depends on the pyinotify and boto Python packages. These are\navailable via \"pip install pyinotify; pip install boto;\", or as packages from\nmost common Linux distributions.

Configuration

All configuration for tablesnap happens on the command line. If you are using\nthe Debian package, you'll set these options in the DAEMON_OPTS variable in\n/etc/default/tablesnap.

usage: tablesnap [-h] -k AWS_KEY -s AWS_SECRET [-r] [-a] [-B] [-p PREFIX]\n                 [--without-index] [--keyname-separator KEYNAME_SEPARATOR]\n                 [-t THREADS] [-n NAME] [-e EXCLUDE | -i INCLUDE]\n                 [--listen-events {IN_MOVED_TO,IN_CLOSE_WRITE}]\n                 [--max-upload-size MAX_UPLOAD_SIZE]\n                 [--multipart-chunk-size MULTIPART_CHUNK_SIZE]\n                 bucket paths [paths ...]\nTablesnap is a script that uses inotify to monitor a directory for events and\nreacts to them by spawning a new thread to upload that file to Amazon S3,\nalong with a JSON-formatted list of what other files were in the directory at\nthe time of the copy.\npositional arguments:\n  bucket                S3 bucket\n  paths                 Paths to be watched\noptional arguments:\n  -h, --help            show this help message and exit\n  -k AWS_KEY, --aws-key AWS_KEY\n  -s AWS_SECRET, --aws-secret AWS_SECRET\n  -r, --recursive       Recursively watch the given path(s)s for new SSTables\n  -a, --auto-add        Automatically start watching new subdirectories within\n                        path(s)\n  -B, --backup          Backup existing files to S3 if they are not already\n                        there\n  -p PREFIX, --prefix PREFIX\n                        Set a string prefix for uploaded files in S3\n  --without-index       Do not store a JSON representation of the current\n                        directory listing in S3 when uploading a file to S3.\n  --keyname-separator KEYNAME_SEPARATOR\n                        Separator for the keyname between name and path.\n  -t THREADS, --threads THREADS\n                        Number of writer threads\n  -n NAME, --name NAME  Use this name instead of the FQDN to identify the\n                        files from this host\n  -e EXCLUDE, --exclude EXCLUDE\n                        Exclude files matching this regular expression from\n                        upload.WARNING: If neither exclude nor include are\n                        defined, then all files matching \"-tmp\" are excluded.\n  -i INCLUDE, --include INCLUDE\n                        Include only files matching this regular expression\n                        into upload.WARNING: If neither exclude nor include\n                        are defined, then all files matching \"-tmp\" are\n                        excluded.\n  --listen-events {IN_MOVED_TO,IN_CLOSE_WRITE,IN_CREATE}\n                        Which events to listen on, can be specified multiple\n                        times. Values: IN_MOVED_TO, IN_CLOSE_WRITE, IN_CREATE\n                        (default: IN_MOVED_TO, IN_CLOSE_WRITE)\n  --max-upload-size MAX_UPLOAD_SIZE\n                        Max size for files to be uploaded before doing\n                        multipart (default 5120M)\n  --multipart-chunk-size MULTIPART_CHUNK_SIZE\n                        Chunk size for multipart uploads (default: 256M or 10%\n                        of free memory if default is not available)\n

For example:

$ tablesnap -k AAAAAAAAAAAAAAAA -s BBBBBBBBBBBBBBBB me.synack.sstables /var/lib/cassandra/data/GiantKeyspace\n

This would cause tablesnap to use the given Amazon Web Services credentials to\nbackup the SSTables for my GiantKeyspace to the S3 bucket named\nme.synack.sstables.

Questions, Comments, and Help

The fine folks in #cassandra-ops on irc.freenode.net are an excellent\nresource for getting tablesnap up and running, and also for solving more\ngeneral Cassandra issues.

","id":"cd740b21-3dd8-5171-abb4-4e789c17dbd9","title":"JeremyGrosser/tablesnap","origin_url":"https://github.com/JeremyGrosser/tablesnap","url":"https://github.com/JeremyGrosser/tablesnap","wallabag_created_at":"2018-04-03T23:50:04+00:00","published_at":null,"published_by":"['']","reading_time":4,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/8f43224982503de627f3b471708d935faa34ee4b6e58bb53b81a9ce7d0c66e7d/JeremyGrosser/tablesnap","tags":["aws","cassandra","github"],"description":"Theory of OperationTablesnap is a script that uses inotify to monitor a directory for IN_MOVED_TO\nevents and reacts to them by spawning a new thread to upload that file to\nAmazon S3, along with a JSON..."},{"content":"

\n \n README.md\n

\n

A reference application for Node.js developers looking to learn more about using\nApache Cassandra and DataStax Enterprise in their applications and\nservices. Learn more at killrvideo.github.io.

\n

Running Locally

\n

Use these guides to get started running KillrVideo locally on your development machine:

\n

Contributing, Requests for More Examples

\n

This project will continue to evolve along with Cassandra and you can expect that as\nCassandra, DSE, and the drivers add new features, this application will try and provide\nexamples of those. We gladly accept any pull requests for bug fixes, new features, etc. and\nif you have a request for an example that you don't see in the code currently, feel free to\nopen an issue here on GitHub or send a message to @LukeTillman on Twitter.

\n
","id":"fc49c54e-9796-516d-85ef-2f0b661c8bca","title":"KillrVideo/killrvideo-nodejs","origin_url":"https://github.com/KillrVideo/killrvideo-nodejs","url":"https://github.com/KillrVideo/killrvideo-nodejs","wallabag_created_at":"2018-03-14T22:27:46+00:00","published_at":null,"published_by":null,"reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/4f50ab463beb119ee95e16cec253627bc046097a770ec753fa46bdf8a735602a/KillrVideo/killrvideo-nodejs","tags":["cassandra","node","github"],"description":"\n \n README.md\n \nA reference application for Node.js developers looking to learn more about using\nApache Cassandra and DataStax Enterprise in their applications and\nservices. Learn more at..."},{"content":"

\n \n README.md\n

\n

A reference application for .NET developers looking to learn more about using Apache Cassandra and\nDataStax Enterprise in their applications and services. Learn more at killrvideo.github.io.

\n

Running Locally

\n

Use these guides to get started running KillrVideo locally on your development machine:

\n

Pull Requests, Requests for More Examples

\n

This project will continue to evolve along with Cassandra and you can expect that as Cassandra and the DataStax driver add new features, this sample application will try and provide examples of those. I'll gladly accept any pull requests for bug fixes, new features, etc. and if you have a request for an example that you don't see in the code currently, send me a message @LukeTillman on Twitter or open an issue here on GitHub.

\n

License

\n

Copyright 2016 Luke Tillman

\n

Licensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at

\n

http://www.apache.org/licenses/LICENSE-2.0

\n

Unless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.

\n
","id":"59f4ae26-bf0e-54ce-ba3a-d90e00ea31ad","title":"LukeTillman/killrvideo-csharp","origin_url":"https://github.com/LukeTillman/killrvideo-csharp","url":"https://github.com/LukeTillman/killrvideo-csharp","wallabag_created_at":"2018-03-14T22:27:36+00:00","published_at":null,"published_by":null,"reading_time":1,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/c1cf548592976495837a26647aae2acf25f30bf82d0d7271025a6fae57195cea/LukeTillman/killrvideo-csharp","tags":["cassandra","csharp","github"],"description":"\n \n README.md\n \nA reference application for .NET developers looking to learn more about using Apache Cassandra and\nDataStax Enterprise in their applications and services. Learn more at ki..."}]},{"tag":"testing","articles":[{"content":"

{{ message }}

\n
\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
\n\n
","id":"166bfd43-cfec-506b-be68-f384b93820b4","title":"GitHub - apache/cassandra-harry: Apache Cassandra - Harry","origin_url":"https://github.com/apache/cassandra-harry","url":"https://github.com/apache/cassandra-harry","wallabag_created_at":"2022-06-08T20:37:16+00:00","published_at":null,"published_by":"['apache']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/aa15162e4aac1bfa5fbd58f2757de131eaa602ad2bc71025321e6eeb66ff4553/apache/cassandra-harry","tags":["cassandra","testing"],"description":"{{ message }}\n / cassandra-harry PublicNotifications\nFork 12\n\n Star 43 \n\nApache Cassandra - Harrycassandra.apache.org/License Apache-2.0 license 43 stars 12 forks Star Notifications\n\n © 2022 GitHub, ..."},{"content":"

{{ message }}

\n
\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
","id":"d29b900a-7c3e-537e-a25e-6c8caff38e03","title":"GitHub - apache/cassandra-diff: Apache cassandra diff","origin_url":"https://github.com/apache/cassandra-diff","url":"https://github.com/apache/cassandra-diff","wallabag_created_at":"2022-06-08T20:37:07+00:00","published_at":null,"published_by":"['']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/e7a10c615cc491e34c4e3f952c02c227c9b02a49ab4ccf2ebd3bc5832f285203/apache/cassandra-diff","tags":["cassandra","testing"],"description":"{{ message }}\nApache cassandra diffcassandra.apache.org/LicenseApache-2.0 license10 stars 16 forks\n\n© 2022 GitHub, Inc.\n\nYou can’t perform that action at this time.\nYou signed in with another tab or w..."},{"content":"

{{ message }}

\n

/ adelphi Public

Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool

License

Apache-2.0 license

9 stars 4 forks

Star

Notifications

\n
\n

You can’t perform that action at this time.

\n

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

\n
\n\n
","id":"75fb8bf0-c48c-58d3-b5ef-b60dc7508eb1","title":"GitHub - datastax/adelphi: Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool","origin_url":"https://github.com/datastax/adelphi","url":"https://github.com/datastax/adelphi","wallabag_created_at":"2022-06-08T20:36:15+00:00","published_at":null,"published_by":"['datastax']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/afe20d0322f42c980557122f254ed0db1a89239e8a1e4130618ed630ed786ddc/datastax-archive/adelphi","tags":["stress","cassandra","cassandra.stress","testing"],"description":"{{ message }}\n / adelphi PublicNotifications\nFork 4\n\n Star 9 \n\nAutomation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltoolLicense Apache-2.0 license 9 stars 4 forks Star Not..."},{"content":"
\n

This is a sample code of utilizing Gatling testing framework for stress testing with DSE

\n

Pre-requisites

\n

Note: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin 0.0.7 is not compatible with Gatling 2.3.1 yet.

\n
  1. Gatling high charts bundle version 2.2.5:
    2. \n
  1. Gatling CQL plugin v0.0.7 (latest version as of writing)
    2. \n

[NOTES] - as of April 10, 2018, Gatling CQL plugin has released version 0.0.8 which works with the latest Gatling framework 2.3.1. Please feel free to download these latest versions for your test. Please also be aware that Gatling 2.3.1 requires scala 2.1.2 and make sure your scala version is upgraded first.

\n

Notes about the Example Simulation Scenario

\n

The simulation scenario (MyTestSimu.scala) as included in this example simulates a mixed read/write workload. The core steps of the scenario are summarized below and you can follow the same steps when creating your own scenario:

\n
  1. Set up connection to Cassandra/DSE cluster with proper properties, such as \"contact points\", \"load balancing policy\", etc.
    2. \n
  2. Create the application keyspace and table schema
    3. \n
  3. Define the (random) value generator for table columns based on their types
    4. \n
  4. Define the Read/Write statements to be used in the simulation, including the Consistency Level associated with the statements
    5. \n
  5. Set up the user simulation behavior for Read and Write. The key parts include:
    6. \n

Procedure

\n
  1. Set up Gatling framework and the CQL plug-in (just unzip, as per description found here)
    2. \n
  2. Download the MyTestSimu.scala file and put it under folder <GATLING_HOME>/user-files/simulations/
    3. \n
  3. Execute the Gatling simulation (stress-testing) scenario by running command: <GATLING_HOME>/bin/gatling.sh. Follow the instructions on the command-line output.
    4. \n

NOTE: The simulation scenario (MyTestSimu.scala) is tested against a DSE cluster (version 5.1.6) with UserName/Password authentication. Please adjust accordingly for your case.

\n

An example is as below. Simulation number 1 is the Cassandra stres-testing scenario as defined by this example.

\n
$ bin/gatling.sh\nGATLING_HOME is set to /home/automaton/gatling-charts-highcharts-bundle-2.2.5\nChoose a simulation number:\n     [0] cassandra.CassandraSimulation\n     [1] cassandra.MyTestSimu\n     [2] computerdatabase.BasicSimulation\n     [3] computerdatabase.advanced.AdvancedSimulationStep01\n     [4] computerdatabase.advanced.AdvancedSimulationStep02\n     [5] computerdatabase.advanced.AdvancedSimulationStep03\n     [6] computerdatabase.advanced.AdvancedSimulationStep04\n     [7] computerdatabase.advanced.AdvancedSimulationStep05\n1\nSelect simulation id (default is 'mytestsimu'). Accepted characters are a-z, A-Z, 0-9, - and _\nSelect run description (optional)\nSimulation cassandra.MyTestSimu started...\n================================================================================\n2018-04-04 15:33:43                                           5s elapsed\n---- Requests ------------------------------------------------------------------\n> Global                                                   (OK=232    KO=0     )\n> upsertStmt                                               (OK=93     KO=0     )\n> readStmt                                                 (OK=139    KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[#                                                                         ]  1%\n          waiting: 8861   / active: 0      / done:139\n---- Write Workload Scenario ---------------------------------------------------\n[                                                                          ]  0%\n          waiting: 20907  / active: 0      / done:93\n================================================================================\n... ...\n================================================================================\n2018-04-04 15:43:39                                         600s elapsed\n---- Requests ------------------------------------------------------------------\n> Global                                                   (OK=30000  KO=0     )\n> upsertStmt                                               (OK=21000  KO=0     )\n> readStmt                                                 (OK=9000   KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:9000\n---- Write Workload Scenario ---------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:21000\n================================================================================\nSimulation cassandra.MyTestSimu completed in 600 seconds\nParsing log file(s)...\nParsing log file(s) done\nGenerating reports...\n================================================================================\n---- Global Information --------------------------------------------------------\n> request count                                      30000 (OK=30000  KO=0     )\n> min response time                                      0 (OK=0      KO=-     )\n> max response time                                    224 (OK=224    KO=-     )\n> mean response time                                     2 (OK=2      KO=-     )\n> std deviation                                          5 (OK=5      KO=-     )\n> response time 50th percentile                          1 (OK=1      KO=-     )\n> response time 75th percentile                          2 (OK=2      KO=-     )\n> response time 95th percentile                          3 (OK=3      KO=-     )\n> response time 99th percentile                          7 (OK=7      KO=-     )\n> mean requests/sec                                     50 (OK=50     KO=-     )\n---- Response Time Distribution ------------------------------------------------\n> t < 800 ms                                         30000 (100%)\n> 800 ms < t < 1200 ms                                   0 (  0%)\n> t > 1200 ms                                            0 (  0%)\n> failed                                                 0 (  0%)\n================================================================================\nReports generated in 4s.\nPlease open the following file: /home/automaton/gatling-charts-highcharts-bundle-2.2.5/results/mytestsimu-1522856018600/index.html\n
\n
","id":"22b64c72-6b88-5842-8e08-ebff60cc319b","title":"yabinmeng/cassgatling","origin_url":"https://github.com/yabinmeng/cassgatling","url":"https://github.com/yabinmeng/cassgatling","wallabag_created_at":"2018-11-14T17:34:32+00:00","published_at":null,"published_by":null,"reading_time":3,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/ba96401352e04a099abef2d795d350d978caf9070e2f7cf302b70025f16fcffd/yabinmeng/cassgatling","tags":["stress","cassandra","gatling","testing"],"description":"\nThis is a sample code of utilizing Gatling testing framework for stress testing with DSE\nPre-requisites\nNote: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin ..."}]}]}},"staticQueryHashes":[]}