DataStax-Examples/query-diagnostics

This is a simple example of reading and writing to an Apache Cassandra™ databasecluster while giving a window into the internal routing and execution tracing withinthe cluster. It utilizes basic logging, Cassandra query tracing, and events fromthe driver's connection to the cluster. The driver events can includenotifications that members of the clusters go down or come back up along with connectionsto local and remote data centers.There are often questions about why the server throws certain exceptions to theclient application. For example, why do I get NoNodeAvailableExceptionerrors when I know nodes in my cluster are available? Diagnosis of a fault in adistributed system is tricky. The application and driver have limited visibilityof the status of the network and members of the database cluster. Each exceptionemanates from a limited knowledge of the state of all components in the system. See theByzantine Generals problemfor a general explanation. However with more information about what is seen at each stagethrough tracing and logging, you will hopefully arrive at a diagnosis more efficiently.Not considered directly for this example are multi-data center connections and data center failover. However,the logging in this example shows what connections are made from the driver to nodes in the cluster. Best practicesfor failover, especially in a multi-data center environment, are discussed at length in thiswhite paperand in this webinarabout designing fault tolerant applications. There is also an accompanying demoto show best practices for data center failover with the latest 4.x Java driver.Contributors: Jeremy HannaObjectivesUnderstand how queries interact with and within a Cassandra clusterLearn about tools to help diagnose problems such as logging, tracing, and those in Additional ResourcesProject LayoutThe project has a standard Apache Maven project layout with a single Java class: QueryDiagnostics.Setup and RunningPrerequisitesApache Maven 3 should be installed and in the pathJDK 14An Apache Cassandra™ cluster is running and accessible through the contacts points and data center identified in application.conf.The program will create a keyspace with NetworkToplogyStrategy with replication in dc1, the Apache Cassandra default when using the GossipingPropertyFileSnitch.RunningBuildingAt the project root level, executemvn clean packageConfiguration changesThe main configuration file is src/main/resources/application.conf.Consider the following changes from defaults for your cluster and environment:Change basic.contact-points = ["127.0.0.1:9042"] to the address and cql port of your Cassandra cluster.Change basic.local-datacenter = dc1 to connect to a data center in your Cassandra cluster.Change basic.request.consistency = LOCAL_ONE to your preferred consistency level.Change replication = {'class': 'NetworkTopologyStrategy', 'dc1' : 1} in QueryDiagnostics.javato your preferred replication settings.Running the programTo execute the program, run the following:mvn exec:java -D"exec.mainClass"="com.datastax.examples.QueryDiagnostics"Additional ResourcesNote in this repo that we've enabled debug logging of com.datastax.oss.driver.internal.core.channel in the logback configuration to log connection updates with different nodes in the clusterJava driver documentation on query tracingcqlsh documentation on query tracingDataStax Studio developer notebooks have execution configurations that can perform query tracesInteractive request tracingProbabilistic tracingPacket capture for dynamic tracingUsing Wireshark for dynamic cql tracingReplacing Cassandra tracing with Zipkin (this method is not yet possible with DataStax Enterprise)