{"componentChunkName":"component---src-templates-tags-js","path":"/tags/testing/1","result":{"pageContext":{"tag":"testing","urlTag":"testing","chunk":[{"content":"<div data-pjax-replace=\"\" id=\"js-flash-container\"><div class=\"flash flash-full {{ className }} px-2\"><p>{{ message }}</p></div>\n</div><div class=\"application-main\" data-commit-hovercards-enabled=\"\" data-discussion-hovercards-enabled=\"\" data-issue-and-pr-hovercards-enabled=\"\"><main id=\"js-repo-pjax-container\" data-pjax-container=\"\"><div id=\"repository-container-header\" class=\"pt-3 hide-full-screen c3\" data-pjax-replace=\"\"><div class=\"d-flex mb-3 px-3 px-md-4 px-lg-5\"><p> / <strong itemprop=\"name\" class=\"mr-2 flex-self-stretch\"><a data-pjax=\"#repo-content-pjax-container\" href=\"https://github.com/apache/cassandra-harry\">cassandra-harry</a></strong> Public</p><ul class=\"pagehead-actions flex-shrink-0 d-none d-md-inline c1\"><li><a href=\"https://github.com/login?return_to=%2Fapache%2Fcassandra-harry\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;notification subscription menu watch&quot;,&quot;repository_id&quot;:null,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/apache/cassandra-harry&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"9cf17cf30bb042224fd5877004d34f4e99e95f6488041ba1ea9a11a0bab36f47\" aria-label=\"You must be signed in to change notification settings\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn\">Notifications</a></li>\n<li><a href=\"https://github.com/login?return_to=%2Fapache%2Fcassandra-harry\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;repo details fork button&quot;,&quot;repository_id&quot;:292627306,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/apache/cassandra-harry&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"16cc71ae0543724bd3934f99a05d0a8a4c0dad8342599c07e6c15e6e968b5552\" aria-label=\"You must be signed in to fork a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn\">Fork 12</a></li>\n<li>\n<p><a href=\"https://github.com/login?return_to=%2Fapache%2Fcassandra-harry\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;star button&quot;,&quot;repository_id&quot;:292627306,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/apache/cassandra-harry&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"f371aced4cf7d947b7473514455df8a59812332fa68e0591cfbcbabb5876773e\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn BtnGroup-item\"> Star 43</a> </p>\n</li>\n</ul></div><div class=\"d-block d-md-none mb-2 px-3 px-md-4 px-lg-5\" id=\"responsive-meta-container\" data-pjax-replace=\"\"><p class=\"f4 mb-3\">Apache Cassandra - Harry</p><p><a title=\"https://cassandra.apache.org/\" role=\"link\" target=\"_blank\" class=\"text-bold\" rel=\"noopener noreferrer\" href=\"https://cassandra.apache.org/\">cassandra.apache.org/</a></p><h3 class=\"sr-only\">License</h3><p><a href=\"https://github.com/apache/cassandra-harry/blob/trunk/LICENSE.txt\" class=\"Link--muted\" data-analytics-event=\"{&quot;category&quot;:&quot;Repository Overview&quot;,&quot;action&quot;:&quot;click&quot;,&quot;label&quot;:&quot;location:sidebar;file:license&quot;}\"> Apache-2.0 license</a></p><p><a class=\"Link--secondary no-underline mr-3\" href=\"https://github.com/apache/cassandra-harry/stargazers\"> 43 stars</a> <a class=\"Link--secondary no-underline\" href=\"https://github.com/apache/cassandra-harry/network/members\"> 12 forks</a></p><div class=\"d-flex\"><p><a href=\"https://github.com/login?return_to=%2Fapache%2Fcassandra-harry\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;star button&quot;,&quot;repository_id&quot;:292627306,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/apache/cassandra-harry&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"f371aced4cf7d947b7473514455df8a59812332fa68e0591cfbcbabb5876773e\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn btn-block BtnGroup-item\"> Star</a> </p><p><a href=\"https://github.com/login?return_to=%2Fapache%2Fcassandra-harry\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;notification subscription menu watch&quot;,&quot;repository_id&quot;:null,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/apache/cassandra-harry&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"9cf17cf30bb042224fd5877004d34f4e99e95f6488041ba1ea9a11a0bab36f47\" aria-label=\"You must be signed in to change notification settings\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn btn-block\">Notifications</a></p></div></div></div>\n</main></div>\n<footer class=\"footer width-full container-xl p-responsive\" role=\"contentinfo\"><div class=\"position-relative d-flex flex-items-center pb-2 f6 color-fg-muted border-top color-border-muted flex-column-reverse flex-lg-row flex-wrap flex-lg-nowrap mt-6 pt-6\"><ul class=\"list-style-none d-flex flex-wrap col-0 col-lg-2 flex-justify-start flex-lg-justify-between mb-2 mb-lg-0\"><li class=\"mt-2 mt-lg-0 d-flex flex-items-center\"> © 2022 GitHub, Inc.</li>\n</ul></div>\n</footer><p> You can’t perform that action at this time.</p>\n<p> You signed in with another tab or window. <a href=\"\">Reload</a> to refresh your session. You signed out in another tab or window. <a href=\"\">Reload</a> to refresh your session.</p>\n<details class=\"details-reset details-overlay details-overlay-dark lh-default color-fg-default hx_rsm\" open=\"open\">\n\n</details>","id":"166bfd43-cfec-506b-be68-f384b93820b4","title":"GitHub - apache/cassandra-harry: Apache Cassandra - Harry","origin_url":"https://github.com/apache/cassandra-harry","url":"https://github.com/apache/cassandra-harry","wallabag_created_at":"2022-06-08T20:37:16+00:00","published_at":null,"published_by":"['apache']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/aa15162e4aac1bfa5fbd58f2757de131eaa602ad2bc71025321e6eeb66ff4553/apache/cassandra-harry","tags":["cassandra","testing"],"description":"{{ message }}\n / cassandra-harry PublicNotifications\nFork 12\n\n Star 43 \n\nApache Cassandra - Harrycassandra.apache.org/License Apache-2.0 license 43 stars  12 forks Star Notifications\n\n © 2022 GitHub, ..."},{"content":"<div class=\"flash flash-full {{ className }} px-2\" data-pjax-replace=\"\" id=\"js-flash-container\"><p>{{ message }}</p></div>\n<div class=\"application-main\" data-commit-hovercards-enabled=\"\" data-discussion-hovercards-enabled=\"\" data-issue-and-pr-hovercards-enabled=\"\"><main id=\"js-repo-pjax-container\" data-pjax-container=\"\"><div id=\"repository-container-header\" class=\"pt-3 hide-full-screen c3\" data-pjax-replace=\"\"><div class=\"d-block d-md-none mb-2 px-3 px-md-4 px-lg-5\" id=\"responsive-meta-container\" data-pjax-replace=\"\"><p class=\"f4 mb-3\">Apache cassandra diff</p><p><a title=\"https://cassandra.apache.org/\" role=\"link\" target=\"_blank\" class=\"text-bold\" rel=\"noopener noreferrer\" href=\"https://cassandra.apache.org/\">cassandra.apache.org/</a></p><h3 class=\"sr-only\">License</h3><p><a href=\"https://github.com/apache/cassandra-diff/blob/trunk/LICENSE.txt\" class=\"Link--muted\" data-analytics-event=\"{&quot;category&quot;:&quot;Repository Overview&quot;,&quot;action&quot;:&quot;click&quot;,&quot;label&quot;:&quot;location:sidebar;file:license&quot;}\">Apache-2.0 license</a></p><p><a class=\"Link--secondary no-underline mr-3\" href=\"https://github.com/apache/cassandra-diff/stargazers\">10 stars</a> <a class=\"Link--secondary no-underline\" href=\"https://github.com/apache/cassandra-diff/network/members\">16 forks</a></p></div></div>\n</main></div>\n<footer class=\"footer width-full container-xl p-responsive\" role=\"contentinfo\"><div class=\"position-relative d-flex flex-items-center pb-2 f6 color-fg-muted border-top color-border-muted flex-column-reverse flex-lg-row flex-wrap flex-lg-nowrap mt-6 pt-6\"><ul class=\"list-style-none d-flex flex-wrap col-0 col-lg-2 flex-justify-start flex-lg-justify-between mb-2 mb-lg-0\"><li class=\"mt-2 mt-lg-0 d-flex flex-items-center\">© 2022 GitHub, Inc.</li>\n</ul></div>\n</footer><p>You can’t perform that action at this time.</p>\n<p>You signed in with another tab or window. <a href=\"\">Reload</a> to refresh your session. You signed out in another tab or window. <a href=\"\">Reload</a> to refresh your session.</p>\n<details class=\"details-reset details-overlay details-overlay-dark lh-default color-fg-default hx_rsm\" open=\"open\"></details>","id":"d29b900a-7c3e-537e-a25e-6c8caff38e03","title":"GitHub - apache/cassandra-diff: Apache cassandra diff","origin_url":"https://github.com/apache/cassandra-diff","url":"https://github.com/apache/cassandra-diff","wallabag_created_at":"2022-06-08T20:37:07+00:00","published_at":null,"published_by":"['']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/e7a10c615cc491e34c4e3f952c02c227c9b02a49ab4ccf2ebd3bc5832f285203/apache/cassandra-diff","tags":["cassandra","testing"],"description":"{{ message }}\nApache cassandra diffcassandra.apache.org/LicenseApache-2.0 license10 stars 16 forks\n\n© 2022 GitHub, Inc.\n\nYou can’t perform that action at this time.\nYou signed in with another tab or w..."},{"content":"<div data-pjax-replace=\"\" id=\"js-flash-container\"><div class=\"flash flash-full {{ className }} px-2\"><p>{{ message }}</p></div>\n</div><div class=\"application-main\" data-commit-hovercards-enabled=\"\" data-discussion-hovercards-enabled=\"\" data-issue-and-pr-hovercards-enabled=\"\"><main id=\"js-repo-pjax-container\" data-pjax-container=\"\"><div id=\"repository-container-header\" class=\"pt-3 hide-full-screen c3\" data-pjax-replace=\"\"><div class=\"d-flex mb-3 px-3 px-md-4 px-lg-5\"><p> / <strong itemprop=\"name\" class=\"mr-2 flex-self-stretch\"><a data-pjax=\"#repo-content-pjax-container\" href=\"https://github.com/datastax/adelphi\">adelphi</a></strong> Public</p><ul class=\"pagehead-actions flex-shrink-0 d-none d-md-inline c1\"><li><a href=\"https://github.com/login?return_to=%2Fdatastax%2Fadelphi\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;notification subscription menu watch&quot;,&quot;repository_id&quot;:null,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/datastax/adelphi&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"dd7eb365e4b5d3f98a22bdd8d18a4af0fbd166a0491e1f620c51d20d647aec28\" aria-label=\"You must be signed in to change notification settings\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn\">Notifications</a></li>\n<li><a href=\"https://github.com/login?return_to=%2Fdatastax%2Fadelphi\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;repo details fork button&quot;,&quot;repository_id&quot;:280185818,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/datastax/adelphi&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"1997e3a5189659d5f9c924425df984a1798c6158f289de62ec3cf960385f554d\" aria-label=\"You must be signed in to fork a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn\">Fork 4</a></li>\n<li>\n<p><a href=\"https://github.com/login?return_to=%2Fdatastax%2Fadelphi\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;star button&quot;,&quot;repository_id&quot;:280185818,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/datastax/adelphi&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"f054bc7eba043e2fe1b97fb786319ed96ea23960ae6311a7994628481c32ae93\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn BtnGroup-item\"> Star 9</a> </p>\n</li>\n</ul></div><div class=\"d-block d-md-none mb-2 px-3 px-md-4 px-lg-5\" id=\"responsive-meta-container\" data-pjax-replace=\"\"><p class=\"f4 mb-3\">Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool</p><h3 class=\"sr-only\">License</h3><p><a href=\"https://github.com/datastax/adelphi/blob/master/LICENSE.txt\" class=\"Link--muted\" data-analytics-event=\"{&quot;category&quot;:&quot;Repository Overview&quot;,&quot;action&quot;:&quot;click&quot;,&quot;label&quot;:&quot;location:sidebar;file:license&quot;}\"> Apache-2.0 license</a></p><p><a class=\"Link--secondary no-underline mr-3\" href=\"https://github.com/datastax/adelphi/stargazers\"> 9 stars</a> <a class=\"Link--secondary no-underline\" href=\"https://github.com/datastax/adelphi/network/members\"> 4 forks</a></p><div class=\"d-flex\"><p><a href=\"https://github.com/login?return_to=%2Fdatastax%2Fadelphi\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;star button&quot;,&quot;repository_id&quot;:280185818,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/datastax/adelphi&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"f054bc7eba043e2fe1b97fb786319ed96ea23960ae6311a7994628481c32ae93\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn btn-block BtnGroup-item\"> Star</a> </p><p><a href=\"https://github.com/login?return_to=%2Fdatastax%2Fadelphi\" rel=\"nofollow\" data-hydro-click=\"{&quot;event_type&quot;:&quot;authentication.click&quot;,&quot;payload&quot;:{&quot;location_in_page&quot;:&quot;notification subscription menu watch&quot;,&quot;repository_id&quot;:null,&quot;auth_type&quot;:&quot;LOG_IN&quot;,&quot;originating_url&quot;:&quot;https://github.com/datastax/adelphi&quot;,&quot;user_id&quot;:null}}\" data-hydro-click-hmac=\"dd7eb365e4b5d3f98a22bdd8d18a4af0fbd166a0491e1f620c51d20d647aec28\" aria-label=\"You must be signed in to change notification settings\" data-view-component=\"true\" class=\"tooltipped tooltipped-s btn-sm btn btn-block\">Notifications</a></p></div></div></div>\n</main></div>\n<footer class=\"footer width-full container-xl p-responsive\" role=\"contentinfo\"><div class=\"position-relative d-flex flex-items-center pb-2 f6 color-fg-muted border-top color-border-muted flex-column-reverse flex-lg-row flex-wrap flex-lg-nowrap mt-6 pt-6\"><ul class=\"list-style-none d-flex flex-wrap col-0 col-lg-2 flex-justify-start flex-lg-justify-between mb-2 mb-lg-0\"><li class=\"mt-2 mt-lg-0 d-flex flex-items-center\"> © 2022 GitHub, Inc.</li>\n</ul></div>\n</footer><p> You can’t perform that action at this time.</p>\n<p> You signed in with another tab or window. <a href=\"\">Reload</a> to refresh your session. You signed out in another tab or window. <a href=\"\">Reload</a> to refresh your session.</p>\n<details class=\"details-reset details-overlay details-overlay-dark lh-default color-fg-default hx_rsm\" open=\"open\">\n\n</details>","id":"75fb8bf0-c48c-58d3-b5ef-b60dc7508eb1","title":"GitHub - datastax/adelphi: Automation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltool","origin_url":"https://github.com/datastax/adelphi","url":"https://github.com/datastax/adelphi","wallabag_created_at":"2022-06-08T20:36:15+00:00","published_at":null,"published_by":"['datastax']","reading_time":null,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/afe20d0322f42c980557122f254ed0db1a89239e8a1e4130618ed630ed786ddc/datastax-archive/adelphi","tags":["stress","cassandra","cassandra.stress","testing"],"description":"{{ message }}\n / adelphi PublicNotifications\nFork 4\n\n Star 9 \n\nAutomation tool for testing C* OSS that assembles cassandra-diff, nosqlbench, fqltoolLicense Apache-2.0 license 9 stars  4 forks Star Not..."},{"content":"<article class=\"markdown-body entry-content\" itemprop=\"text\">\n<p>This is a sample code of utilizing <a href=\"https://gatling.io/\" rel=\"nofollow\">Gatling</a> testing framework for stress testing with DSE</p>\n<h3><a id=\"user-content-pre-requisites\" class=\"anchor\" aria-hidden=\"true\" href=\"#pre-requisites\"></a>Pre-requisites</h3>\n<p>Note: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin 0.0.7 is not compatible with Gatling 2.3.1 yet.</p>\n<ol><li>Gatling high charts bundle version 2.2.5:</li>\n</ol><ul><li><a href=\"https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/2.2.5/\" rel=\"nofollow\">https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/2.2.5/</a></li>\n</ul><ol start=\"2\"><li>Gatling CQL plugin v0.0.7 (latest version as of writing)</li>\n</ol><p><strong>[NOTES]</strong> - as of April 10, 2018, Gatling CQL plugin has released version 0.0.8 which works with the latest Gatling framework 2.3.1. Please feel free to download these latest versions for your test. Please also be aware that Gatling 2.3.1 requires scala 2.1.2 and make sure your scala version is upgraded first.</p>\n<h3><a id=\"user-content-notes-about-the-example-simulation-scenario\" class=\"anchor\" aria-hidden=\"true\" href=\"#notes-about-the-example-simulation-scenario\"></a>Notes about the Example Simulation Scenario</h3>\n<p>The simulation scenario (MyTestSimu.scala) as included in this example simulates a mixed read/write workload. The core steps of the scenario are summarized below and you can follow the same steps when creating your own scenario:</p>\n<ol><li>Set up connection to Cassandra/DSE cluster with proper properties, such as \"contact points\", \"load balancing policy\", etc.</li>\n<li>Create the application keyspace and table schema</li>\n<li>Define the (random) value generator for table columns based on their types</li>\n<li>Define the Read/Write statements to be used in the simulation, including the Consistency Level associated with the statements</li>\n<li>Set up the user simulation behavior for Read and Write. The key parts include:</li>\n</ol><ul><li>How many concurrent users (can be constant or some variance) to be simulated per second.</li>\n<li>How long does the simulation executes</li>\n</ul><h3><a id=\"user-content-procedure\" class=\"anchor\" aria-hidden=\"true\" href=\"#procedure\"></a>Procedure</h3>\n<ol><li>Set up Gatling framework and the CQL plug-in (just unzip, as per description found <a href=\"https://github.com/gatling-cql/GatlingCql\">here</a>)</li>\n<li>Download the MyTestSimu.scala file and put it under folder &lt;GATLING_HOME&gt;/user-files/simulations/</li>\n<li>Execute the Gatling simulation (stress-testing) scenario by running command: &lt;GATLING_HOME&gt;/bin/gatling.sh. Follow the instructions on the command-line output.</li>\n</ol><p>NOTE: The simulation scenario (MyTestSimu.scala) is tested against a DSE cluster (version 5.1.6) with UserName/Password authentication. Please adjust accordingly for your case.</p>\n<hr /><p>An example is as below. Simulation number 1 is the Cassandra stres-testing scenario as defined by this example.</p>\n<pre>$ bin/gatling.sh\nGATLING_HOME is set to /home/automaton/gatling-charts-highcharts-bundle-2.2.5\nChoose a simulation number:\n     [0] cassandra.CassandraSimulation\n     [1] cassandra.MyTestSimu\n     [2] computerdatabase.BasicSimulation\n     [3] computerdatabase.advanced.AdvancedSimulationStep01\n     [4] computerdatabase.advanced.AdvancedSimulationStep02\n     [5] computerdatabase.advanced.AdvancedSimulationStep03\n     [6] computerdatabase.advanced.AdvancedSimulationStep04\n     [7] computerdatabase.advanced.AdvancedSimulationStep05\n1\nSelect simulation id (default is 'mytestsimu'). Accepted characters are a-z, A-Z, 0-9, - and _\nSelect run description (optional)\nSimulation cassandra.MyTestSimu started...\n================================================================================\n2018-04-04 15:33:43                                           5s elapsed\n---- Requests ------------------------------------------------------------------\n&gt; Global                                                   (OK=232    KO=0     )\n&gt; upsertStmt                                               (OK=93     KO=0     )\n&gt; readStmt                                                 (OK=139    KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[#                                                                         ]  1%\n          waiting: 8861   / active: 0      / done:139\n---- Write Workload Scenario ---------------------------------------------------\n[                                                                          ]  0%\n          waiting: 20907  / active: 0      / done:93\n================================================================================\n... ...\n================================================================================\n2018-04-04 15:43:39                                         600s elapsed\n---- Requests ------------------------------------------------------------------\n&gt; Global                                                   (OK=30000  KO=0     )\n&gt; upsertStmt                                               (OK=21000  KO=0     )\n&gt; readStmt                                                 (OK=9000   KO=0     )\n---- Read Workload Scenario ----------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:9000\n---- Write Workload Scenario ---------------------------------------------------\n[##########################################################################]100%\n          waiting: 0      / active: 0      / done:21000\n================================================================================\nSimulation cassandra.MyTestSimu completed in 600 seconds\nParsing log file(s)...\nParsing log file(s) done\nGenerating reports...\n================================================================================\n---- Global Information --------------------------------------------------------\n&gt; request count                                      30000 (OK=30000  KO=0     )\n&gt; min response time                                      0 (OK=0      KO=-     )\n&gt; max response time                                    224 (OK=224    KO=-     )\n&gt; mean response time                                     2 (OK=2      KO=-     )\n&gt; std deviation                                          5 (OK=5      KO=-     )\n&gt; response time 50th percentile                          1 (OK=1      KO=-     )\n&gt; response time 75th percentile                          2 (OK=2      KO=-     )\n&gt; response time 95th percentile                          3 (OK=3      KO=-     )\n&gt; response time 99th percentile                          7 (OK=7      KO=-     )\n&gt; mean requests/sec                                     50 (OK=50     KO=-     )\n---- Response Time Distribution ------------------------------------------------\n&gt; t &lt; 800 ms                                         30000 (100%)\n&gt; 800 ms &lt; t &lt; 1200 ms                                   0 (  0%)\n&gt; t &gt; 1200 ms                                            0 (  0%)\n&gt; failed                                                 0 (  0%)\n================================================================================\nReports generated in 4s.\nPlease open the following file: /home/automaton/gatling-charts-highcharts-bundle-2.2.5/results/mytestsimu-1522856018600/index.html\n</pre>\n</article>","id":"22b64c72-6b88-5842-8e08-ebff60cc319b","title":"yabinmeng/cassgatling","origin_url":"https://github.com/yabinmeng/cassgatling","url":"https://github.com/yabinmeng/cassgatling","wallabag_created_at":"2018-11-14T17:34:32+00:00","published_at":null,"published_by":null,"reading_time":3,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/ba96401352e04a099abef2d795d350d978caf9070e2f7cf302b70025f16fcffd/yabinmeng/cassgatling","tags":["stress","cassandra","gatling","testing"],"description":"\nThis is a sample code of utilizing Gatling testing framework for stress testing with DSE\nPre-requisites\nNote: At the moment, the latest version of Gatling framework is 2.3.1. However, the CQL plugin ..."},{"content":"<article class=\"markdown-body entry-content\" itemprop=\"text\">\n<ul><li>I wanted to run some <code>cassandra-stress</code> testing on the three main cloud providers (Google, Amazon, and Microsoft) on similarly sized DataStax Enterprise clusters and see what the results would be</li>\n<li>Why did I want to do this? When running on a cluster of similarly-sized VMs, networking starts to come into play for replication, storage, and stronger consistency levels. This type of an exercise exposes the network so you can understand what you may have to contend with for a specific cloud provider versus another</li>\n<li>This is also a tutorial on some of the fun switches that you can use to tune your <code>cassandra-stress</code> test</li>\n<li>This is not a tutorial on how to deploy the VMs, if you want that information, please go over to another repo of mine as I repurposed it. It is 99% similar; I just didn't enable SSL and installed <code>dse-full</code>: <a href=\"https://github.com/justinbreese/dse-multi-cloud-demo\">https://github.com/justinbreese/dse-multi-cloud-demo</a></li>\n</ul><h2><a id=\"user-content-methodology\" class=\"anchor\" aria-hidden=\"true\" href=\"#methodology\"></a>Methodology</h2>\n<ul><li>Setup 4 VMs on each of the cloud providers; VM size was as close as possible with similar storage provisioned: i3.4xl, Standard_DS14_v2, and n1-highmem-16</li>\n<li>3 of the VMs will be running DataStax Enterprise as their own cluster; again each cloud is their own cluster</li>\n<li>The 4th VM runs DataStax OpsCenter and acts as a <code>cassandra-stress</code> client\n<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/justinbreese/dse-cassandra-stress/blob/master/architecture.png?raw=true\"><img src=\"https://github.com/justinbreese/dse-cassandra-stress/raw/master/architecture.png?raw=true\" alt=\"Screenshot\" /></a></li>\n<li>Next, I ran this test on each of the clients 35 times: <code>cassandra-stress user profile=stress.yaml n=1000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 104.196.140.126 -rate threads\\&gt;=121</code>\n<ul><li>I then captured the results into an Excel file</li>\n</ul></li>\n<li>Then I decided to really turn it up and do 1B operations; and it led to this test that was run once on each of the clients: <code>nohup cassandra-stress user profile=stress.yaml n=1000000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 40.118.149.27 -rate threads=300 -log file=output.txt -errors ignore</code>\n<ul><li>Again, I captured the results in an Excel file</li>\n</ul></li>\n</ul><h2><a id=\"user-content-breakdown-the-switches\" class=\"anchor\" aria-hidden=\"true\" href=\"#breakdown-the-switches\"></a>Breakdown the switches!</h2>\n<p>I told you that this would be educational!</p>\n<h3><a id=\"user-content-first-test\" class=\"anchor\" aria-hidden=\"true\" href=\"#first-test\"></a>First test</h3>\n<p><code>cassandra-stress user profile=stress.yaml n=1000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 104.196.140.126 -rate threads\\&gt;=121</code></p>\n<ul><li><code>profile=stress.yaml</code> --&gt; take a look at the one that I created in the root of this repo. I went to a colleague's website and was able to build it in a few minutes (source below). Do this to create the actual data model that you want to use in production!</li>\n<li><code>n=1000000</code> --&gt; the amount of times that you want to run operations</li>\n<li><code>ops\\(insert=3,likelyquery1=1\\)</code> --&gt; the ratio of writes:reads that you want done for your test. This is a 3:1 ratio. You can be specific about the reads that you want done. In <code>stress.yaml</code>, towards the bottom, there is a query called <code>likelyquery</code> that I want to run for my reads. You can customize this to read whatever query you prefer; super useful!</li>\n<li><code>cl=QUORUM</code> --&gt; consistency level. I see way too many stress tests done just at a consistency level of <code>CL=ONE</code>. The reality is that most don't run with a <code>CL=ONE</code> when they're in production; so why would they test against it now. Run your test against the consistency level that you want to run in production!</li>\n<li><code>-mode native cql3 user=cassandra password=datastax</code> --&gt; you need the client to be able to access DataStax Enterprise, so ensure it has the correct username and password to do so</li>\n<li><code>-node 104.196.140.126</code> --&gt; as I am running on a client, I need to be able to connect to the cluster. This IP address was of one of the seed nodes for the cluster that I was targeting. Once I point it towards one node, the client will become aware of all of the other nodes via gossip.</li>\n<li><code>-rate threads\\&gt;=121</code> --&gt; this runs the test at thread levels that are greater than or equal to 121. You can run at less than that, but I wanted to run at realistic levels.\n<ul><li>A great first test is to run at <code>-rate auto</code> which will will start at a single digit thread count and work its way up as high as it can go. This is great for understanding what is going on so you can understand the relationship for latency, operations/second, threadcount, etc.</li>\n</ul></li>\n</ul><h2><a id=\"user-content-second-test\" class=\"anchor\" aria-hidden=\"true\" href=\"#second-test\"></a>Second test</h2>\n<p>I am not going to go over the flags/arguments that I went over in the previous example. Instead, I'll go over the new ones or changes.</p>\n<p><code>nohup cassandra-stress user profile=stress.yaml n=1000000000 ops\\(insert=3,likelyquery1=1\\) cl=QUORUM -mode native cql3 user=cassandra password=datastax -node 40.118.149.27 -rate threads=300 -log file=output.txt -errors ignore</code></p>\n<ul><li><code>nohup</code> --&gt; I nohup'd because I wanted this to run as its own process. The advantage here is that it will run after I exit the session, computer goes to sleep, etc.</li>\n<li><code>n=1000000000</code> --&gt; increased the ops to 1B</li>\n<li><code>-rate threads=300</code> --&gt; set the thread count to an even 300. This is generally a point at which we start to see saturation, contention, etc.; so I wanted to stay in that range for an extended period of time</li>\n<li><code>-log file=output.txt</code> --&gt; as this would be a long running test, I wanted to be able to save the output all into a file that way I could <code>SCP</code> it back to my laptop for analysis in Excel. Otherwise it will just print to the screen; and after 1B transactions, that gets messy!</li>\n<li><code>-errors ignore</code> --&gt; after you run 1B transactions, you will get errors eventually. Cassandra handles them just fine, but you don't want it to stop the test. Do this for your tests! Otherwise, you could have your test stop half-way through; ask me how I know...</li>\n</ul><h2><a id=\"user-content-results\" class=\"anchor\" aria-hidden=\"true\" href=\"#results\"></a>Results</h2>\n<p>to be continued... maybe, if I can.</p>\n<ul><li>If you want to learn more about <code>cassandra-stress</code>, then look no further than the excellent work of my colleague: <a href=\"https://www.datastax.com/dev/blog/data-modeler\" rel=\"nofollow\">https://www.datastax.com/dev/blog/data-modeler</a></li>\n<li>The actual tool that builds out the <code>stress.yaml</code>: <a href=\"https://www.sestevez.com/sestevez/CassandraDataModeler/\" rel=\"nofollow\">https://www.sestevez.com/sestevez/CassandraDataModeler/</a></li>\n</ul>\n<p>Thanks for taking a look at this repo and let me know if you have any questions.</p>\n</article>","id":"c7ef4299-9aed-5814-830c-152a2c71eee3","title":"justinbreese/dse-cassandra-stress","origin_url":"https://github.com/justinbreese/dse-cassandra-stress","url":"https://github.com/justinbreese/dse-cassandra-stress","wallabag_created_at":"2018-11-13T20:12:27+00:00","published_at":null,"published_by":null,"reading_time":5,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/ad3b3ac83d6bdca681e5fa455a46e248f827c4ef3fed9166b05113d5ea996ee6/justinbreese/dse-cassandra-stress","tags":["stress","cassandra","dse","testing","aws","google.cloud","azure"],"description":"\nI wanted to run some cassandra-stress testing on the three main cloud providers (Google, Amazon, and Microsoft) on similarly sized DataStax Enterprise clusters and see what the results would be\nWhy d..."},{"content":"<p>Developing software is fun. Developing software that is well tested, where those tests run quickly is even more fun.</p><p>Stubbed Cassandra is an open source tool that enables you to test applications that use Cassandra in a quick, deterministic way.</p><p>It is especially aimed edge case testing such as read and write timeouts.</p><p>It acts as a real Cassandra instance and can be primed to respond with results or with exceptions like read timeouts. It does this by implementing the server side of the CQL binary protocol.</p><p>It is separated into two components:</p><ul><li><a href=\"http://scassandra-docs.readthedocs.org/en/latest/java/overview/\">Java Client</a>: Java client for Scassandra. A thin Java wrapper around Scassandra that allows Java projects to depend on Scassandra via maven dependency and have a programmatic interface for starting/stopping and priming.</li>\n  <li><a href=\"http://scassandra-docs.readthedocs.org/en/latest/standalone/overview/\">Scassandra Server</a>: Stubbed Scassandra server. Only go here if you’re insterested in running Stubbed Cassandra without a build tool such as maven or gradle. Implemented in Scala, can be run as a standalone server or depended on via the Java client. Doesn’t have an embedded Cassandra, is a standalone implementation of the server side of the Cassandra native protocol. You can prime the server to return rows, read timeout and write timeout via a REST API.</li>\n</ul><p>Scassandra, currently v1.1.0, is aimed at Java developers so most of the information is on the Java Client section of the website.  It also may be used as a standalone jar.</p><h3 id=\"release-v111\">Release v1.1.1</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/issues/181\">#181</a> - Record ‘USE ' queries in activity log</li>\n</ul><h3 id=\"release-v110\">Release v1.1.0</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/171\">#171</a> - Protocol 3 and 4 support.  Migrate cql-antlr into tree as submodule.</li>\n</ul><h3 id=\"release-v1010\">Release v1.0.10</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/167\">#167</a> - Provide CORS support to Scassandra endpoints</li>\n</ul><h3 id=\"release-v109\">Release v1.0.9</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/157\">#157</a> - Share ActorSystem between Scassandra instances</li>\n</ul><h3 id=\"release-v108\">Release v1.0.8</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/155\">#155</a> - Add getters for thens in multi-prime requests</li>\n</ul><h3 id=\"release-v107\">Release v1.0.7</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/151\">#151</a> - Allow priming of config using config object</li>\n  <li><a href=\"https://github.com/scassandra/scassandra-server/pull/154\">#154</a> - Provide custom configuration for batch result</li>\n</ul><h3 id=\"release-v106\">Release v1.0.6</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/149\">#149</a> - Introduce alias for Then to avoid reserved word issues</li>\n  <li><a href=\"https://github.com/scassandra/scassandra-server/pull/150\">#150</a> - Handle invalid types with an informative error message</li>\n</ul><h3 id=\"release-v105\">Release v1.0.5</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/148\">#148</a> - Use shapeless 2</li>\n</ul><h3 id=\"release-v104\">Release v1.0.4</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/147\">#147</a> - Akka configuration fix</li>\n</ul><h3 id=\"release-v103\">Release v1.0.3</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/141\">#141</a> - Add ability to retrieve prepared multi primes</li>\n</ul><h3 id=\"release-v102\">Release v1.0.2</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/140\">#140</a> - Support for clearing prepared multi primes</li>\n</ul><h3 id=\"release-v101\">Release v1.0.1</h3><ul><li><a href=\"https://github.com/scassandra/scassandra-server/pull/139\">#139</a> - Delay support for batch statements</li>\n</ul><h3 id=\"release-v100\">Release v1.0.0</h3><ul><li>Multi prime API.</li>\n</ul><h3 id=\"release-v0110\">Release v0.11.0</h3><ul><li>Log better error message for timeout binding to ports</li>\n  <li><a href=\"https://github.com/scassandra/scassandra-server/issues/112\">#112</a> - Prime Result that closes connection</li>\n  <li><a href=\"https://github.com/scassandra/scassandra-server/pull/115\">#115</a> - Archive standalone jar</li>\n</ul><h3 id=\"release-v0100\">Release v0.10.0</h3><ul><li>Batch support</li>\n  <li>Ability to see currently connected clients and kill connections</li>\n</ul><h3 id=\"release-v091\">Release v0.9.1</h3><ul><li>Java 6 suppport for those still sadly using it</li>\n  <li>Verification for when a prepared statement is prepared</li>\n  <li>Fat jar support</li>\n  <li>Capturing of query parameters for non-prepared statements</li>\n</ul><h3 id=\"release-v080\">Release v0.8.0</h3><ul><li>Noving to a single build for server, client and tests.</li>\n</ul><h3 id=\"release-v070\">Release v0.7.0</h3><ul><li>Support for priming specific errors e.g the number of replicas responded rather than just that it wsa a read/write timeout</li>\n  <li>Removed support for version 1.* of the Java DataStax driver due to classpath issues (you can still use 2.* with Cassandra 1.2 and it will work)</li>\n</ul><h3 id=\"release-v060\">Release v0.6.0</h3><ul><li>Support for maps of any type</li>\n  <li>New method for priming types: CqlType (the ColumnTypes enum has been deprecated)</li>\n  <li>Prepared statement matcher now handles maps</li>\n</ul><h3 id=\"release-v050\">Release v0.5.0</h3><ul><li>Support for lists and sets of any type</li>\n  <li>Prepared statement matcher that handles matching the varialbe list correctly</li>\n</ul><h3 id=\"release-v041\">Release v0.4.1</h3><ul><li>[Feature #50] Support adding a fixed delay to both queries and prepared statements</li>\n  <li>[Feature #52] Cassandra 2.1 support</li>\n  <li>[Feature #53] JUnit matchers for queries and prepared statements</li>\n</ul><h3 id=\"release-v030\">Release v0.3.0</h3><ul><li>Text map maps support: varchar, ascii and text</li>\n  <li>Can use a queryPattern rather than a query for priming, making knowing the exact query the application will execute no longer necessary</li>\n  <li>[Bug #49] QueryPattern for prepared statements for error scenarios do not work</li>\n</ul><h3 id=\"release-v020\">Release v0.2.0</h3><ul><li>Lists and sets of the character types: varchar, ascii and text</li>\n  <li>JUnit rule for Java Client</li>\n</ul><h3 id=\"release-v01\">Release v0.1</h3><ul><li>Priming of queries with columns of all the primitive types (no suport for collections/custom tyes).</li>\n  <li>Priming of prepared statements. The variable (?s) types and response types can be any of the primitive types.</li>\n  <li>Retrieval of a list of all recorded queries.</li>\n  <li>Retrieval of a list of all the recorded executed prepared statements. If the prepared statement has been primed then the variable values are also visible.</li>\n</ul><h3 id=\"feature-backlog\">Feature backlog</h3><ul><li>User defined types</li>\n  <li>Batches</li>\n  <li>Retrieval of a list of all prepared statements even if they haven’t been executed.</li>\n  <li>Priming of tables rather than queries. Currently Scassandra does not parse the query and compares an executed query with all the primes query field. This would be very useful for priming the system keyspace as certain drivers expect the same thing to be in system.local but do slightly different queries to retireve it.</li>\n</ul><p>For feature requests and bug reports raise an issue at the <a href=\"https://github.com/scassandra/scassandra-server/issues\">Github issues page</a></p><p>Any questions ping me on twitter: @chbatey</p><ul><li>Christopher Batey (@chbatey)</li>\n  <li>Andrew Tolbert</li>\n  <li>Dogan Narinc</li>\n</ul>","id":"f25dfc52-d2f5-5804-8a88-de850ba485a3","title":"Stubbed Cassandra","origin_url":"http://www.scassandra.org/","url":"http://www.scassandra.org/","wallabag_created_at":"2018-10-08T20:04:55+00:00","published_at":null,"published_by":null,"reading_time":4,"domain_name":"www.scassandra.org","preview_picture":null,"tags":["cassandra","testing"],"description":"Developing software is fun. Developing software that is well tested, where those tests run quickly is even more fun.Stubbed Cassandra is an open source tool that enables you to test applications that ..."},{"content":"<p><strong>Support:</strong> <a href=\"https://gitter.im/intuit/wasabi?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=pr-badge&amp;utm_content=badge\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/13115004d1de8c8debb6d9636ed770ceb3cd44ea/68747470733a2f2f6261646765732e6769747465722e696d2f696e747569742f7761736162692e737667\" alt=\"Join the chat at https://gitter.im/intuit/wasabi\" data-canonical-src=\"https://badges.gitter.im/intuit/wasabi.svg\" /></a> <br /><strong>Documentation:</strong> <a href=\"https://intuit.github.io/wasabi/v1/guide/index.html\" rel=\"nofollow\">User Guide</a>, <a href=\"https://intuit.github.io/wasabi/v1/javadocs/latest/\" rel=\"nofollow\">JavaDocs</a>\n<br /><strong>A/B Testing Overview:</strong> <a href=\"https://www.youtube.com/watch?v=_HtvJwBPUqk&amp;feature=youtu.be\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/7d0321a6112c23f202f9852f1db1d4587db17d85/687474703a2f2f696d672e736869656c64732e696f2f62616467652f766964656f2d412532464225323054657374696e672532304f766572766965772d7265642e737667\" alt=\"A/B Testing Overview\" data-canonical-src=\"http://img.shields.io/badge/video-A%2FB%20Testing%20Overview-red.svg\" /></a> <a href=\"https://medium.com/blueprint-by-intuit/open-sourcing-wasabi-the-a-b-testing-platform-by-intuit-a8d5abc958d\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/31955dbe1b8474ed4a4ab59e38a26e05b0ac0622/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f626c6f672d4d6565742532305761736162692d627269676874677265656e2e737667\" alt=\"Blog Meet Wasabi\" data-canonical-src=\"https://img.shields.io/badge/blog-Meet%20Wasabi-brightgreen.svg\" /></a> <a href=\"https://medium.com/blueprint-by-intuit/the-architecture-behind-wasabi-an-open-source-a-b-testing-platform-b52430d3fd80\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/f1f2ee9250c9a93017c1c82c4dc7fa0013d61401/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f626c6f672d417263686974656374757265253230426568696e642532305761736162692d6f72616e67652e737667\" alt=\"Blog Architecture Behind Wasabi\" data-canonical-src=\"https://img.shields.io/badge/blog-Architecture%20Behind%20Wasabi-orange.svg\" /></a> <br /><strong>Continuous Integration:</strong> <a href=\"https://travis-ci.org/intuit/wasabi\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/c02cc709a1c51c0107101c3d1a96e28b7f10dd1c/68747470733a2f2f6170692e7472617669732d63692e6f72672f696e747569742f7761736162692e7376673f6272616e63683d646576656c6f70\" alt=\"Build Status\" data-canonical-src=\"https://api.travis-ci.org/intuit/wasabi.svg?branch=develop\" /></a>\n<a href=\"https://coveralls.io/github/intuit/wasabi?branch=develop\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/21a3f18f699685c7ac4db1f687158a9bcfe1470f/68747470733a2f2f636f766572616c6c732e696f2f7265706f732f6769746875622f696e747569742f7761736162692f62616467652e737667\" alt=\"Coverage Status\" data-canonical-src=\"https://coveralls.io/repos/github/intuit/wasabi/badge.svg\" /></a>\n<a href=\"https://maven-badges.herokuapp.com/maven-central/com.intuit.wasabi/wasabi\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/6da12cf851ba62b357771ca25a0e3561339330c5/68747470733a2f2f6d6176656e2d6261646765732e6865726f6b756170702e636f6d2f6d6176656e2d63656e7472616c2f636f6d2e696e747569742e7761736162692f7761736162692f62616467652e737667\" alt=\"Maven Central\" data-canonical-src=\"https://maven-badges.herokuapp.com/maven-central/com.intuit.wasabi/wasabi/badge.svg\" /></a> <br /><strong>License:</strong> <a href=\"http://www.apache.org/licenses/LICENSE-2.0\" rel=\"nofollow\"><img src=\"https://camo.githubusercontent.com/cb7ed4d28c8df19eb3b269fe0cad477d17621500/687474703a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d417061636865253230322d627269676874677265656e2e737667\" alt=\"Apache 2\" data-canonical-src=\"http://img.shields.io/badge/license-Apache%202-brightgreen.svg\" /></a> <br /></p><h2>Project</h2><p>Wasabi A/B Testing Service is a real-time, enterprise-grade, 100% API driven project. Users are empowered to own their own data, and run experiments across web, mobile, and desktop. It’s fast, easy to use, it’s chock full of features, and instrumentation is minimal.</p><p>Learn more about how Wasabi can empower your team to move from hunches to actionable, data-driven user insights with our simple, flexible, and scalable experimentation platform.</p><h3>Features</h3><ul><li><strong>Own your own data</strong> - Wasabi runs on your servers, in the cloud or on-premise, where you have complete control over your data.</li>\n<li><strong>Proven</strong> - Wasabi is battle-tested in production at Intuit, Inc., a financial technology company. Wasabi is the experimentation platform for TurboTax, QuickBooks, Mint.com, and other Intuit offerings.</li>\n<li><strong>High Performance</strong> - Consistent server-side response times for user assignments within 30ms.</li>\n<li><strong>100% API-Driven</strong> - The Wasabi REST API is compatible with any language and environment.</li>\n<li><strong>Platform Agnostic</strong> - Uniform, consistent testing across Web, mobile, desktop. Also supports front-end, back-end integrations.</li>\n<li><strong>Real-time user assignments</strong> - Assign users into experiments in real time, to preserve traffic for other parallel A/B tests.</li>\n<li><strong>Cloud and on-premise</strong> - Designed to live in the cloud or in your own data center.</li>\n<li><strong>Analytics</strong> - Core experiment analytics and metrics visualization out of the box, as well as the ability to send data to your existing analytics infrastructure.</li>\n<li><strong>Pluggable</strong> - Well-defined interfaces for plugging in your own access control, sending data to data pipelines, and providing fully custom bucket allocations.</li>\n<li><strong>Experiment Management UI</strong> - Setup and manage experiments via a modern Web interface. Management via REST API is also possible.</li>\n<li><strong>Dockerized</strong> - Spin up a Wasabi Docker instance in 5 minutes and be in production with the platform, instrumentation, and experiments within a day.</li>\n</ul><h3>User Interface</h3><ul><li><strong>Create an experiment and its buckets:</strong>\n<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://camo.githubusercontent.com/2e4a1b3b58c231a9b2ee753131d28b8982f751c5/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4372656174654275636b65742e706e67\"><img src=\"https://camo.githubusercontent.com/2e4a1b3b58c231a9b2ee753131d28b8982f751c5/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4372656174654275636b65742e706e67\" alt=\"\" data-canonical-src=\"https://intuit.github.io/wasabi/v1/guide/images/readme/CreateBucket.png\" /></a></li>\n<li><strong>Filter which customers are considered for your experiment:</strong>\n<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://camo.githubusercontent.com/86693dfd1865dc803f40a0a61786a1f573903968/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f5365676d656e746174696f6e52756c65732e706e67\"><img src=\"https://camo.githubusercontent.com/86693dfd1865dc803f40a0a61786a1f573903968/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f5365676d656e746174696f6e52756c65732e706e67\" alt=\"\" data-canonical-src=\"https://intuit.github.io/wasabi/v1/guide/images/readme/SegmentationRules.png\" /></a></li>\n<li><strong>Follow your currently running experiments:</strong>\n<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://camo.githubusercontent.com/f0372ebe376391b70abe344675870d8830e32621/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4578706572696d656e744c6973742e706e67\"><img src=\"https://camo.githubusercontent.com/f0372ebe376391b70abe344675870d8830e32621/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4578706572696d656e744c6973742e706e67\" alt=\"\" data-canonical-src=\"https://intuit.github.io/wasabi/v1/guide/images/readme/ExperimentList.png\" /></a></li>\n<li><strong>Track your experiment results in real-time:</strong>\n<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://camo.githubusercontent.com/26570e8c8a76366b25111ee208473c5f85b057ee/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4578706572696d656e7444657461696c732e706e67\"><img src=\"https://camo.githubusercontent.com/26570e8c8a76366b25111ee208473c5f85b057ee/68747470733a2f2f696e747569742e6769746875622e696f2f7761736162692f76312f67756964652f696d616765732f726561646d652f4578706572696d656e7444657461696c732e706e67\" alt=\"\" data-canonical-src=\"https://intuit.github.io/wasabi/v1/guide/images/readme/ExperimentDetails.png\" /></a></li>\n</ul><h2>Get Started</h2><p>The following steps will help you install the needed tools, then build and run a complete Wasabi stack.</p><h4>Bootstrap Your Environment</h4><h5>Mac OS</h5><div class=\"highlight highlight-source-shell\"><pre>% /usr/bin/ruby \\\n  -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n% brew install git\n% git clone https://github.com/intuit/wasabi.git\n% cd wasabi\n% ./bin/wasabi.sh bootstrap</pre></div><p>Installed tools include: <a href=\"http://brew.sh\" rel=\"nofollow\">homebrew 0.9</a>, <a href=\"https://git-scm.com\" rel=\"nofollow\">git 2</a>,\n<a href=\"https://maven.apache.org\" rel=\"nofollow\">maven 3</a>, <a href=\"http://www.oracle.com/technetwork/java/javase/overview/index.html\" rel=\"nofollow\">java 1.8</a>,\n<a href=\"https://docker.com\" rel=\"nofollow\">docker 1.12</a>, <a href=\"https://nodejs.org/en\" rel=\"nofollow\">node 6</a> and <a href=\"https://www.python.org\" rel=\"nofollow\">python 2.7</a>.</p><h5>Ubuntu</h5><p>Bootstrapping on Ubuntu requires sudo privileges to install all the required dependencies. You will be prompted to enter your password. Currently only 16.04 (x64) is supported.</p><div class=\"highlight highlight-source-shell\"><pre>% sudo apt-get install git\n% git clone https://github.com/intuit/wasabi.git\n% cd wasabi\n% ./bin/wasabi.sh bootstrap\n% sudo reboot</pre></div><p>NOTE: A reboot is required after running the bootstrap command on Ubuntu.</p><p>For all other processes (build, start etc.) the commands are same for Ubuntu and Mac OS.</p><p>Installed tools include: <a href=\"https://git-scm.com\" rel=\"nofollow\">git 2</a>,\n<a href=\"https://maven.apache.org\" rel=\"nofollow\">maven 3</a>, <a href=\"http://openjdk.java.net/projects/jdk8/\" rel=\"nofollow\">OpenJdk 8</a>,\n<a href=\"https://docker.com\" rel=\"nofollow\">docker 1.12</a>, <a href=\"https://nodejs.org/en\" rel=\"nofollow\">node 6</a> and <a href=\"https://www.python.org\" rel=\"nofollow\">python 2.7</a></p><p>Similar tooling will work for Windows. Contribute a patch :)</p><h4>Start Wasabi</h4><p>Now that we have the necessary tools in place, let's move on to build and start Wasabi, followed by issuing a <em>ping</em>\ncommand to verify the build:</p><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh build start\n...\nwasabi is operational:\n  ui: % open http://localhost:8080     note: sign in as admin/admin\n  ping: % curl -i http://localhost:8080/api/v1/ping\n  debug: attach to localhost:8180\n% curl -i http://localhost:8080/api/v1/ping\nHTTP/1.1 200 OK\nDate: Wed, 25 May 2016 00:25:47 GMT\n...\nX-Application-Id: wasabi-api-20151215171929-SNAPSHOT-development\nContent-Type: application/json\nTransfer-Encoding: chunked\nServer: Jetty(9.3.z-SNAPSHOT)\n{\n  \"componentHealths\":[\n    {\n      \"componentName\":\"Experiments Cassandra\",\n      \"healthy\":true\n    },\n    {\n      \"componentName\":\"MySql\",\"healthy\":true\n    }\n  ],\n  \"wasabiVersion\":\"wasabi-api-20151215171929-SNAPSHOT-development\"\n}</pre></div><p>Congratulations! You are the proud owner of a newly minted Wasabi instance. :)</p><h3>Running Wasabi with remote storage</h3><h5>Set Mysql and Cassandra credentials</h5><ul><li>Modify /pom.xml to set the values that apply to your environment</li>\n</ul><h5>Download Cassandra migration tool <a href=\"https://oss.sonatype.org/content/repositories/public/com/builtamont/cassandra-migration/0.9/cassandra-migration-0.9-jar-with-dependencies.jar\" rel=\"nofollow\">https://oss.sonatype.org/content/repositories/public/com/builtamont/cassandra-migration/0.9/cassandra-migration-0.9-jar-with-dependencies.jar</a></h5><h5>Set up your environment variables</h5><ul><li>Set location of the migration tool</li>\n</ul><div class=\"highlight highlight-source-shell\"><pre>export CASSANDRA_MIGRATION=/location/of/cassandra-migration-0.9-jar-with-dependencies.jar</pre></div><ul><li>Set location of migration scripts within your project</li>\n</ul><div class=\"highlight highlight-source-shell\"><pre>export MIGRATION_SCRIPT=/location/of/modules/repository-datastax/src/main/resources/com/intuit/wasabi/repository/impl/cassandra/migration</pre></div><h5>Set up Cassandra tables</h5><div class=\"highlight highlight-source-shell\"><pre>CQLSH_VERSION=&lt;version&gt; CQLSH_USERNAME=&lt;username&gt; CQLSH_PASSWORD=&lt;pwd&gt; CQLSH_HOST=&lt;host&gt; bin/docker/migration.sh</pre></div><h5>Run Wasabi with env variables for remote storage hosts</h5><div class=\"highlight highlight-source-shell\"><pre>MYSQL_HOST=&lt;mysql_host&gt; NODE_HOST=&lt;cassandra_host&gt; ./bin/wasabi.sh start:wasabi</pre></div><h5>Run Wasabi outside of docker with WASABI_CONFIGURATION for remote storage hosts</h5><div class=\"highlight highlight-source-shell\"><pre>WASABI_CONFIGURATION=\"\n  -Ddatabase.url.host=$MYSQL_HOST\\\n  -Ddatabase.url.port=$MYSQL_PORT\\\n  -Ddatabase.url.dbname=$MYSQL_DATABASE\\\n  -Ddatabase.user=$MYSQL_USER\\\n  -Ddatabase.password=$MYSQL_PASSWORD\\\n  -Ddatabase.pool.connections.min=$MYSQL_MIN_CONNECTIONS\\\n  -Ddatabase.pool.connections.max=$MYSQL_MAX_CONNECTIONS\\\n  -Dusername=$CASSANDRA_USER\\\n  -Dpassword=$CASSANDRA_PASSWORD\\\n  -DnodeHosts=$CASSANDRA_HOST\\\n  -DtokenAwareLoadBalancingLocalDC=$CASSANDRA_DATACENTER\\\n  -Dapplication.http.port=$PORT\" bash usr/local/wasabi-main-*/bin/run</pre></div><h4>Troubleshooting</h4><ul><li>\n<p>While starting Wasabi, if you see an error when the docker containers are starting up, you could do the following:</p>\n<ul><li>Look at the current docker containers that have been successfully started.</li>\n</ul></li>\n</ul><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh status\nCONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                     NAMES\n8c12458057ef        wasabi-main              \"entrypoint.sh wasabi\"   25 minutes ago      Up 25 minutes       0.0.0.0:8080-&gt;8080/tcp, 0.0.0.0:8090-&gt;8090/tcp, 0.0.0.0:8180-&gt;8180/tcp    wasabi-main\n979ecc885239        mysql:5.6                \"docker-entrypoint.sh\"   26 minutes ago      Up 26 minutes       0.0.0.0:3306-&gt;3306/tcp                                                    wasabi-mysql\n2d33a96abdcb        cassandra:2.1            \"/docker-entrypoint.s\"   27 minutes ago      Up 27 minutes       7000-7001/tcp, 0.0.0.0:9042-&gt;9042/tcp, 7199/tcp, 0.0.0.0:9160-&gt;9160/tcp   wasabi-cassandra</pre></div><ul><li>The above shell output shows a successful start of 3 docker containers needed by Wasabi: wasabi-main (the Wasabi server),\nwasabi-mysql, and wasabi-cassandra. If any of these are not running, try starting them individually. For example, if the\nMySQL container is running, but Cassandra and Wasabi containers failed to start (perhaps due to a network timeout docker\ncould not download the Cassandra image), do the following:</li>\n</ul><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh start:cassandra\n% ./bin/wasabi.sh start:wasabi</pre></div><h4>Call Wasabi</h4><p>These are the 3 common REST endpoints that you will use to instrument your client application with Wasabi.</p><p>Let's assume that you've created and started an experiment, 'BuyButton,' in the 'Demo_App' application with the following buckets:</p><ul><li>'BucketA': green button (control bucket)</li>\n<li>'BucketB': orange button bucket</li>\n</ul><p>You can assign a user with a unique ID (e.g. 'userID1') to the experiment by making this HTTP request:</p><blockquote>\n<p>Assign a user to experiment and bucket:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% curl -H \"Content-Type: application/json\" \\\n    http://localhost:8080/api/v1/assignments/applications/Demo_App/experiments/BuyButton/users/userID1\n{  \n   \"cache\":true,\n   \"payload\":\"green\",\n   \"assignment\":\"BucketA\",\n   \"context\":\"PROD\",\n   \"status\":\"NEW_ASSIGNMENT\"\n}</pre></div><p>Now the 'userID1' user is assigned into the 'BucketA' bucket. Let's further record an impression, meaning the user has seen a given experience:</p><blockquote>\n<p>Record an impression:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% curl -H \"Content-Type: application/json\" \\\n    -d \"{\\\"events\\\":[{\\\"name\\\":\\\"IMPRESSION\\\"}]}\" \\\n    http://localhost:8080/api/v1/events/applications/Demo_App/experiments/BuyButton/users/userID1</pre></div><p>If the 'userID1' user performs an action such as clicking the Buy button, you'd record that action with the following request:</p><blockquote>\n<p>Record an action:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% curl -H \"Content-Type: application/json\" \\\n    -d \"{\\\"events\\\":[{\\\"name\\\":\\\"BuyClicked\\\"}]}\" \\\n    http://localhost:8080/api/v1/events/applications/Demo_App/experiments/BuyButton/users/userID1</pre></div><h4>Explore Various Resources</h4><p>The following developer resources are available:</p><blockquote>\n<p>API: Swagger API playground</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh resource:api</pre></div><blockquote>\n<p>Javadoc</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh resource:doc</pre></div><blockquote>\n<p>Wasabi UI</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh resource:ui</pre></div><blockquote>\n<p>Cassandra: cqlsh shell</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh resource:cassandra</pre></div><blockquote>\n<p>MySQL: mysql shell</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh resource:mysql</pre></div><blockquote>\n<p>Java Debugger: Remote attach configuration</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8180</pre></div><h4>Stop Wasabi</h4><p>Alas, all good things must come to an end. Let's clean things up a bit stop the newly created Wasabi stack:</p><p>At this point in time, we now have all the requisite tools installed, and subsequent invocations of Wasabi will\nstart up much more quickly.</p><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh stop</pre></div><h4>Get Familiar with wasabi.sh</h4><p>Further, there are a number of additional wasabi.sh options available you should become familiar with:</p><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh --help\n  usage: wasabi.sh [options] [commands]\n  options:\n    -e | --endpoint [ host:port ]          : api endpoint; default: localhost:8080\n    -v | --verify [ true | false ]         : verify installation configuration; default: false\n    -s | --sleep [ sleep-time ]            : sleep/wait time in seconds; default: 30\n    -h | --help                            : help message\n  commands:\n    bootstrap                              : install dependencies\n    build                                  : build project\n    start[:cassandra,mysql,wasabi]         : start all, cassandra, mysql, wasabi\n    test                                   : test wasabi\n    stop[:wasabi,cassandra,mysql]          : stop all, wasabi, cassandra, mysql\n    resource[:ui,api,doc,cassandra,mysql]  : open resource api, javadoc, cassandra, mysql\n    status                                 : display resource status\n    remove[:wasabi,cassandra,mysql]        : remove all, wasabi, cassandra, mysql\n    package                                : build deployable packages\n    release[:start,finish]                 : promote release</pre></div><h2>Develop</h2><h4>Build and Run Wasabi Server</h4><div class=\"highlight highlight-source-shell\"><pre>% mvn package\n% ./bin/wasabi.sh start:cassandra,mysql\n% (cd modules/main/target; \\\n    WASABI_CONFIGURATION=\"-DnodeHosts=localhost -Ddatabase.url.host=localhost\" ./wasabi-main-*-SNAPSHOT-development/bin/run) &amp;\n% curl -i http://localhost:8080/api/v1/ping\n...</pre></div><p>The runtime logs can be accessed executing the following command in a another shell:</p><blockquote>\n<p>Viewing runtime logs:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% tail -f modules/main/target/wasabi-main-*-SNAPSHOT-development/logs/wasabi-main-*-SNAPSHOT-development.log</pre></div><h4>Build and Run Wasabi UI</h4><div class=\"highlight highlight-source-shell\"><pre>% cd modules/ui\n% grunt build</pre></div><div class=\"highlight highlight-source-shell\"><pre>% grunt serve</pre></div><h4>Stop Wasabi</h4><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh stop</pre></div><p>Now while that was fun, in all likelihood you will be using an IDE to work on Wasabi. In doing so, you need only\nadd the configuration information above to the JVM commandline prior to startup:</p><blockquote>\n<p>Wasabi runtime configuration:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>-DnodeHosts=localhost -Ddatabase.url.host=localhost</pre></div><h4>Run Integration Tests</h4><p>Code changes can readily be verified by running the growing collection of included integration tests:</p><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh start test stop</pre></div><h5>Troubleshooting</h5><p>Integration tests might fail intermittently due to a time drift issue in docker containers on Mac OSX.</p><p>When the Mac sleeps and wakes back up, there is a lag created between the clock in the Mac vs the\nrunning docker containers. This is a known issue in Docker for Mac.</p><p>This can be fixed by running the following command:</p><div class=\"highlight highlight-source-shell\"><pre>% docker run --rm --privileged alpine hwclock -s</pre></div><p>The above command will need to be run every time when there is a time drift.</p><p>To automatically run this command and update the time each time the Mac wakes up, you could install\nthe following agent:</p><div class=\"highlight highlight-source-shell\"><pre>% curl https://raw.githubusercontent.com/arunvelsriram/docker-time-sync-agent/master/install.sh | bash</pre></div><p>You can read more about this at: <a href=\"https://blog.shameerc.com/2017/03/quick-tip-fixing-time-drift-issue-on-docker-for-mac\" rel=\"nofollow\">quick-tip-fixing-time-drift-issue-on-docker-for-mac</a></p><h2>Package and Deploy at Scale</h2><p>Wasabi can readily be packaged as installable <em>rpm</em> or <em>deb</em> distributions and deployed at scale as follows:</p><blockquote>\n<p>Package by running integration tests 1st:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh start package</pre></div><blockquote>\n<p>Package without integration tests, if needed:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% ./bin/wasabi.sh -t false package</pre></div><blockquote>\n<p>Find generated package files:</p>\n</blockquote><div class=\"highlight highlight-source-shell\"><pre>% find . -type f \\( -name \"*.rpm\" -or -name \"*.deb\" \\)</pre></div><p>Note: <a href=\"http://www.oracle.com/technetwork/java/javase/overview/index.html\" rel=\"nofollow\">Java 8</a> is a runtime dependency</p><h2>Integrate</h2><p>Wasabi is readily embeddable via the following <em>maven</em> dependency GAV family:</p><div class=\"highlight highlight-text-xml\"><pre>&lt;dependency&gt;\n    &lt;groupId&gt;com.intuit.wasabi&lt;/groupId&gt;\n    &lt;artifactId&gt;wasabi&lt;/artifactId&gt;\n    &lt;version&gt;1.0.20160627213750&lt;build_timestamp&gt;&lt;/version&gt;\n&lt;/dependency&gt;</pre></div><h2>Contribute</h2><p>We greatly encourage contributions! You can add new features, report and fix existing bugs, write docs and\ntutorials, or any of the above. Feel free to open issues and/or send pull requests.</p><p>The <code>master</code> branch of this repository contains the latest stable release of Wasabi, while snapshots are published to the <code>develop</code> branch. In general, pull requests should be submitted against <code>develop</code> by forking this repo into your account, developing and testing your changes, and creating pull requests to request merges. See the <a href=\"https://guides.github.com/activities/contributing-to-open-source/\">Contributing to a Project</a>\narticle for more details about how to contribute in general and find more specific information on how to write code for Wasabi in our <a href=\"https://intuit.github.io/wasabi/v1/guide/index.html#developing-wasabi\" rel=\"nofollow\">user guide</a>.</p><p>Extension projects such as browser plugins, client integration libraries, and apps can be contributed under the <code>contrib</code> directory.</p><p>Steps to contribute:</p><ol><li>Fork this repository into your account on Github</li>\n<li>Clone <em>your forked repository</em> (not our original one) to your hard drive with <code>git clone https://github.com/YOURUSERNAME/wasabi.git</code></li>\n<li>Design and develop your changes</li>\n<li>Add/update unit tests</li>\n<li>Add/update integration tests</li>\n<li>Add/update documentation on <code>gh-pages</code> branch</li>\n<li>Create a pull request for review to request merge</li>\n<li>Obtain 2 approval <em>squirrels</em> before your changes can be merged</li>\n</ol><p>Thank you for your contribution!</p>","id":"965b9cd4-097c-560f-94b6-00ce67a88391","title":"intuit/wasabi","origin_url":"https://github.com/intuit/wasabi","url":"https://github.com/intuit/wasabi","wallabag_created_at":"2018-09-24T16:18:17+00:00","published_at":null,"published_by":"['']","reading_time":10,"domain_name":"github.com","preview_picture":"https://opengraph.githubassets.com/6b71518933139be280b839b74b254bf883d8ad401517eabea88c5f1d20fc4cc9/intuit/wasabi","tags":["open.source","cassandra","testing"],"description":"Support:  Documentation: User Guide, JavaDocs\nA/B Testing Overview:    Continuous Integration: \n\n License:  ProjectWasabi A/B Testing Service is a real-time, enterprise-grade, 100% API driven project...."},{"content":"<p>In the previous two posts of this series (<a href=\"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-1/\">Part 1</a> and <a href=\"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-2/\">Part 2</a>) I covered some of the basic commands of <a href=\"https://www.instaclustr.com/apache-cassandra/\">cassandra</a>-stress. In this post I will start looking at the use of the stress YAML file for more advanced stress scenarios, particularly where you want to run stress against a schema that matches one you are planning to use for your application.</p><p>It’s worth noting in the intro that cassandra-stress with a YAML file use a significantly (80%?) different set of code to the standard read/write/mixed commands. So, some assumptions and learnings from the standard commands won’t hold for YAML-driven stress. To cite one example, when running based on YAML, cassandra-stress does not validate that data returned from a select has the expected values as it does with read or mixed.</p><h5>For this article, I’ll reference the following YAML specification file:</h5><p>Before explaining the contents here, let’s see what happens when we run it with the following simple scenario:<br /><code>cassandra-stress user profile=file:///eg-files/stressprofilemixed.yaml no-warmup ops(insert=1) n=100 -rate threads=1 -node x.x.x.x</code></p><p>After running this on an empty cluster, I ran <code>select count(*) from eventsrawtest;</code>. The result? 345 rows – probably not you would have guessed. Here’s how cassandra-stress gets to that:</p><ul><li>n=100 counts number of insert batches, not number of individual insert operations</li>\n<li>Each batch will contain 1 partition’s data (due to partitions=fixed(1) setting) and all 15 of the rows in the partition. There are 15 rows in every partition as the single cluster key (time) has a cluster setting of fixed(15). All the rows in the partition will be included in the batch due to the select: fixed(10)/10 setting (ie changing this to say fixed(5)/10 would result in half the rows from the partition being include in any given batch).</li>\n<li>100 batches of 15 rows each gets you to 1500 rows so how did we end up with 345? This is due (primarily, in this case) to the relatively small range of potential values for the bucket_time. This results in a high overlap in the partition key values that end up getting generated by the uniform distributions. To demonstrate, changing the population of bucket_time to uniform(1..1288) results in 540 rows. In most cases, you want to initially insert data with no overlap to build up a base data set for testing. To facilitate this, I’ve recently submitted a cassandra-stress enhancement that provides sequential generation of seed values the same as used with the write command (<a href=\"https://issues.apache.org/jira/browse/CASSANDRA-12490\">https://issues.apache.org/jira/browse/CASSANDRA-12490</a>). Changing the uniform() distribution to seq() results in the expected 1500 rows being inserted by this command.</li>\n</ul><h5>Let’s look at some of the other column settings:</h5><ul><li><strong>population</strong> – determines the distribution of seed values used in the random data generation. By controlling the distribution of the seed values you control the distribution of the actual inserted values. So, for example uniform(1..100) will allow for up 100 different values each with the same chance of being selected. guassian(1..100) will also allow for up to 100 different values but as they will follow a guassian (otherwise known as normal or bell-curve) distribution, the values around the middle will have a much higher chance of being selected than the values at the extremes (so there will be a set of values more likely to get repeated and some which will occur very infrequently).</li>\n<li><strong>size</strong> – determines the size (length in bytes) of the of the values created for the field.</li>\n<li><strong>cluster</strong> – only applies to clustering columns, specifies the number of values for the column appearing in a single partition. The maximum number of rows in a partition is therefore the product of the maximum number of row of each clustering column (eg max(row1) * max(row 2) * max(row3)).</li>\n</ul><h5>We covered most of the insert settings in the introductory points but here’s a recap:</h5><ul><li><strong>partitions</strong>: the number of different partitions to include in each generated insert batch. Once a partition is chosen for inclusion in a batch, all rows in the partition will become eligible for inclusion and then be filtered according to the select setting. Using, uniform(1..5) would result in each batches containing between 1 and 5 partitions worth of data (with an equal chance of each number in the range).</li>\n<li><strong>batchtype</strong>: logged or unlogged – determines the cassandra batch type to use</li>\n<li><strong>select</strong>: select determines the portion of rows from a partition to select (at random) for inclusion in particular batch. So, for example, fixed(5)/10 would include 50% of rows from the selected partition in each batch. uniform(1..10)/10 would result in between 10% and 100% of rows in the partition being included in the batch with a different select percentage being randomly picked for each partition in each batch.</li>\n</ul><h5>The final section the yaml file that bears some explanation is the queries section. For each query, you specify:</h5><ul><li>A name for the query (pull-for-rollup, get-a-value) which are used to refers to the queries when specifying the mix of operations through the cassandra-stress command line.</li>\n<li><strong>cql</strong> – The actual query with ? characters where values from the population will be substituted in.</li>\n<li><strong>fields</strong> – either samerow or multirow. For samerow, the key values to use for the select will be picked at random (following the same general population rules as for insert) for the list of row keys that has been generated for inserting. For multirow, each of the column values making up the key will be independently randomly selected so there is a chance of generating keys for the selection parameters that don’t exist in the set of data that will/could be inserted according the the population settings.</li>\n</ul><h5>The ops command specified as part of the command line controls the mix of different operations to run. Take for example the following command:</h5><p><code>cassandra-stress user profile=file:///eg-files/stressprofilemixed.yaml ops(insert=1, pull_for_rollup=1, get-value=10) n=120 -node x.x.x.x</code></p><p>This will execute insert batches, pull_for_rollup queries and get-value queries in the ratio 1:1:10. So for this specific example, we’d get 10 inserts, 10 pull_for_rollup queries and 100 get-value queries.</p><p>Hopefully that’s explained the key information you need to use a YAML profile for running cassandra stress. In future instalments I’ll take a look at some of the remaining command line options and walk through a full end-to-end example of designing and executing a test.</p><p>Click here for <a href=\"https://www.instaclustr.com/blog/2016/08/19/deep-diving-into-cassandra-stress-part-1/\">Part One: </a><a href=\"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-1/\">Deep Diving into Cassandra Stress</a><br />Click here for <a href=\"https://www.instaclustr.com/deep-diving-into-cassandra-stress-part-2/\">Part Two: Mixed Command</a></p>","id":"d7cce620-feeb-572c-bc3f-3a4fd803fdf6","title":"Deep Diving cassandra-stress - Part 3 (Using YAML Profiles) - Instaclustr","origin_url":"https://www.instaclustr.com/deep-diving-cassandra-stress-part-3-using-yaml-profiles/","url":"https://www.instaclustr.com/deep-diving-cassandra-stress-part-3-using-yaml-profiles/","wallabag_created_at":"2018-09-13T14:57:14+00:00","published_at":"2016-08-24T05:03:19+00:00","published_by":"['']","reading_time":5,"domain_name":"www.instaclustr.com","preview_picture":"https://www.instaclustr.com/wp-content/uploads/2021/10/cs-article3.png","tags":["stress","cassandra","cassandra.stress","testing"],"description":"In the previous two posts of this series (Part 1 and Part 2) I covered some of the basic commands of cassandra-stress. In this post I will start looking at the use of the stress YAML file for more adv..."}],"skip":0,"limit":8,"totalCount":12,"currentPage":1,"url":"/tags/testing"}},"staticQueryHashes":[]}