Securing Cassandra for Compliance

Successfully reported this slideshow.

Securing Cassandra for Compliance
Securing Cassandra
for Compliance (or Paranoia)
Hi, I'm Nate.
@zznate
https://www.linkedin.com/in/zznate
http://www.slideshare.net/zznate/
Co-Founder, CTO
The Last Pickle...
Security presentations can be scary.
Here's a cat.
First, how did we get here and why is
securing Cassandra important?
"Target CEO Gregg Steinhafel Resigns In
Data Breach Fallout"
http://www.forbes.com/sites/clareoconnor/2014/05/05/target-ce...
I have
your
personal
information
Customers place a lot of trust
in technology companies
LOL! Me too!
Sometimes too much.
Ease of scalability comes with a price
HA! A bin-packed
message format with no source
verification!*
Ease of scalability comes with a price
* <currently reading o...
nmap -Pn -p7000 
-oG logs/cass.gnmap 54.88.0.0/14
I'm publicly
discussing your
technical
shortcomings
Then you end up in this situation.
Meanwhile, at the FCC...
We have to require two
factor, secure socket transport
encryption, something something...
ZZZzzzz...
We did a regulation!
My staffers still print
out my email :)
Why
are we doing
this again?
Sssshhhh.
I'm AES'ing...
...even though the traffic
never leaves a backplane.
Some industries ...
1. Encrypting data at rest
2. Encrypting data on the wire
3. Authentication and authorization
4. Management and tooling
Fo...
1. Encryption at rest
No matter what:
understand the failure modes
bit rot, entropy, etc.
Horrible things can happen with on disk encryption.
Don't mind me, I'm just
your key server.
Haha! Later!
x
What's on this
disk again?
Shrug.
...but you may not have a choice.
Because we said "at rest"
dmcrypt, eCryptFS
Open source options:
Vormetric, Gazzang
Commercial options:
DSE Encryption
CREATETABLE users
...
WITH compression_parameters:sstable_compression = 'Encryptor'
and compression_paramet...
DSE Encryption
CREATETABLE users
...
WITH compression_parameters:sstable_compression = 'Encryptor'
and compression_paramet...
EBS Encryption
(a.k.a "not my problem")
(Looks like this)
EBS Encryption
(a.k.a "not my problem")
http://www.slideshare.net/AmazonWebServices/bdt323-amazon-ebs-ca...
Maybe Client Side?
The Java Driver now has custom codecs
which would make this easy to implement
https://github.com/datast...
Maybe Client Side?
The Java Driver now has custom codecs
which would make this easy to implement
https://github.com/datast...
New in Cassandra 3.4
(DSE 5.1?):
Commitlog Encryption: CASSANDRA-6018
Hint File Encryption: CASSANDRA-11040
https://issues...
2. Encryption on the wire
Because:
It is really easy to attack
an un-protected cluster
It takes a single Message
to insert an admin account
into the system table
-Dcassandra.write_survey=true
How to steal writes in real time:
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Bo...
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Bo...
When you are done it should look like:
Things to note:
Use "dc" or "rack" to limit encryption to
connections between racks and data centers
Thanks for that!!
Huzzah!
(But AES on modern hardware
will not be a bottleneck)
Things to note:
Keystore and key password must match
(artifact of JDK X.509 Impl complexity)
Things to note:
256 bit means export restrictions
(requires JCE provider JAR)
http://www.oracle.com/technetwork/java/javas...
Don't forget this part or else...
Things to note:
Hahaha!
Now I'm hacking you over SSL.
*Still* vulnerable AND you can't see what the
attacker is doing.
Client to Server SSL
Client to Server SSL
(see slides 30 to 35)
Client to Server SSL
(see slides 30 to 35)
Now with NO downtime!!!
https://issues.apache.org/jira/browse/CASSANDRA-10559
A...
Need to Debug SSL?
-Djavax.net.debug=ssl
http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/ReadDebug.html
Certs are hard :(
Netflix Lemur:
x.509 Certificate Orchestration Framework
http://techblog.netflix.com/2015/09/introducing-le...
Certs are hard :(
Hashicorp Vault
"secures, stores, and tightly controls access to
tokens, passwords, certificates, API key...
2. Encryption on the wire
But wait! There's more!
The internode authentication API:
BYO identity verification
Looks like this:
3. Authentication and Authorization
Best practices should not be new to you.
user segmentation
schema access limitation
etc.
(Everything we did with an RDBMS)
Best practices should not be new to you.
user segmentation
schema access limitation
etc.
Best practices should not be new to you.
user segmentation
schema access limitation
etc.
(Everything we did with an RDBMS)...
An Example
An Example
An Example
An Example
An Example
buzzword compliant!
An Example
An Example
Turning it all on
authenticator: PasswordAuthenticator
Tip: keep your read-only cqlsh credentials in
$HOME/.cassandra/cqls...
Turning it all on
authorizer: CassandraAuthorizer
Turning it all on
role_manager: CassandraRoleManager
Turning it all on
authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
...
authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
...
authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
...
authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
...
Turning it all on
authorizer: TransitionalAuthorizer
authenticator: TransitionalAuthenticator
DSE plugins to avoid downtime
Turning it all on
system.schema_keyspace
system.schema_columns
system.schema_columnfamilies
system.local
system.peers
Thes...
Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace
replication factor if you use ...
Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace
replication factor if you use ...
4. Management and tooling
4. Management and tooling
Securing JMX
nmap -Pn -p7199 
-oG logs/cass.gnmap 54.88.0.0/14
Always a few suckers that
TL,DR'ed
Why do I need to secure JMX?
Works as Advertised!
also
good for
some
LOLs
Securing JMX
SSL setup is like node to node and client to server
http://docs.oracle.com/javase/8/docs/technotes/guides/man...
Securing JMX
JMX Authentication is straightforward
and well documented
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAV...
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now...
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now...
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now...
Thanks!@zznate
Securing Cassandra for Compliance

Upcoming SlideShare

Loading in …5

×

  1. 1. Securing Cassandra for Compliance (or Paranoia)
  2. 2. Hi, I'm Nate. @zznate https://www.linkedin.com/in/zznate http://www.slideshare.net/zznate/ Co-Founder, CTO The Last Pickle Cassandra user since 2009 (v0.4) Austin, Texas
  3. 3. Security presentations can be scary. Here's a cat.
  4. 4. First, how did we get here and why is securing Cassandra important?
  5. 5. "Target CEO Gregg Steinhafel Resigns In Data Breach Fallout" http://www.forbes.com/sites/clareoconnor/2014/05/05/target-ceo-gregg-steinhafel-resigns-in-wake-of-data-breach-fallout/ First, how did we get here and why is securing Cassandra important?
  6. 6. I have your personal information Customers place a lot of trust in technology companies
  7. 7. LOL! Me too! Sometimes too much.
  8. 8. Ease of scalability comes with a price
  9. 9. HA! A bin-packed message format with no source verification!* Ease of scalability comes with a price * <currently reading o.a.c.net.MessageIn#read>
  10. 10. nmap -Pn -p7000 -oG logs/cass.gnmap 54.88.0.0/14
  11. 11. I'm publicly discussing your technical shortcomings Then you end up in this situation.
  12. 12. Meanwhile, at the FCC... We have to require two factor, secure socket transport encryption, something something... ZZZzzzzzzzZZZzz
  13. 13. We did a regulation! My staffers still print out my email :)
  14. 14. Why are we doing this again? Sssshhhh. I'm AES'ing... ...even though the traffic never leaves a backplane. Some industries will require node to node SSL
  15. 15. 1. Encrypting data at rest 2. Encrypting data on the wire 3. Authentication and authorization 4. Management and tooling Focusing our Discussion: Architecture
  16. 16. 1. Encryption at rest
  17. 17. No matter what: understand the failure modes
  18. 18. bit rot, entropy, etc. Horrible things can happen with on disk encryption.
  19. 19. Don't mind me, I'm just your key server.
  20. 20. Haha! Later! x What's on this disk again? Shrug.
  21. 21. ...but you may not have a choice. Because we said "at rest"
  22. 22. dmcrypt, eCryptFS Open source options:
  23. 23. Vormetric, Gazzang Commercial options:
  24. 24. DSE Encryption CREATETABLE users ... WITH compression_parameters:sstable_compression = 'Encryptor' and compression_parameters:cipher_algorithm = 'AES/ECB/ PKCS5Padding' and compression_parameters:secret_key_strength = 128;
  25. 25. DSE Encryption CREATETABLE users ... WITH compression_parameters:sstable_compression = 'Encryptor' and compression_parameters:cipher_algorithm = 'AES/ECB/ PKCS5Padding' and compression_parameters:secret_key_strength = 128; WARNING: commitlog not included* *eCryptFS would work fine for this
  26. 26. EBS Encryption (a.k.a "not my problem")
  27. 27. (Looks like this) EBS Encryption (a.k.a "not my problem") http://www.slideshare.net/AmazonWebServices/bdt323-amazon-ebs-cassandra-1-million-writes-per-second See Crowdstrike's presentation on Cassandra GP2 performance (with encryption):
  28. 28. Maybe Client Side? The Java Driver now has custom codecs which would make this easy to implement https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs
  29. 29. Maybe Client Side? The Java Driver now has custom codecs which would make this easy to implement https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs Column-level encryption!
  30. 30. New in Cassandra 3.4 (DSE 5.1?): Commitlog Encryption: CASSANDRA-6018 Hint File Encryption: CASSANDRA-11040 https://issues.apache.org/jira/browse/CASSANDRA-6018 https://issues.apache.org/jira/browse/CASSANDRA-11040
  31. 31. 2. Encryption on the wire
  32. 32. Because: It is really easy to attack an un-protected cluster
  33. 33. It takes a single Message to insert an admin account into the system table
  34. 34. -Dcassandra.write_survey=true How to steal writes in real time:
  35. 35. The fix is straight forward: node to node encryption and SSL client certificate authentication to cluster traffic
  36. 36. Awwwwww. The fix is straight forward: node to node encryption and SSL client certificate authentication to cluster traffic
  37. 37. Awwwwww. The fix is straight forward: node to node encryption and SSL client certificate authentication to cluster traffic Bonus: can be done with NO downtime!!!
  38. 38. Awwwwww. The fix is straight forward: node to node encryption and SSL client certificate authentication to cluster traffic Bonus: can be done with NO downtime!!! How-to guide: http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to- server.html
  39. 39. When you are done it should look like:
  40. 40. Things to note: Use "dc" or "rack" to limit encryption to connections between racks and data centers
  41. 41. Thanks for that!! Huzzah! (But AES on modern hardware will not be a bottleneck)
  42. 42. Things to note: Keystore and key password must match (artifact of JDK X.509 Impl complexity)
  43. 43. Things to note: 256 bit means export restrictions (requires JCE provider JAR) http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#importlimits
  44. 44. Don't forget this part or else... Things to note:
  45. 45. Hahaha! Now I'm hacking you over SSL. *Still* vulnerable AND you can't see what the attacker is doing.
  46. 46. Client to Server SSL
  47. 47. Client to Server SSL (see slides 30 to 35)
  48. 48. Client to Server SSL (see slides 30 to 35) Now with NO downtime!!! https://issues.apache.org/jira/browse/CASSANDRA-10559 Available in: 2.1.12, 2.2.4, 3.0.0
  49. 49. Need to Debug SSL? -Djavax.net.debug=ssl http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/ReadDebug.html
  50. 50. Certs are hard :( Netflix Lemur: x.509 Certificate Orchestration Framework http://techblog.netflix.com/2015/09/introducing-lemur.html https://github.com/Netflix/lemur
  51. 51. Certs are hard :( Hashicorp Vault "secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. " https://www.vaultproject.io/
  52. 52. 2. Encryption on the wire But wait! There's more!
  53. 53. The internode authentication API: BYO identity verification
  54. 54. Looks like this:
  55. 55. 3. Authentication and Authorization
  56. 56. Best practices should not be new to you. user segmentation schema access limitation etc.
  57. 57. (Everything we did with an RDBMS) Best practices should not be new to you. user segmentation schema access limitation etc.
  58. 58. Best practices should not be new to you. user segmentation schema access limitation etc. (Everything we did with an RDBMS) New in 2.2: Role-based access control!
  59. 59. An Example
  60. 60. An Example
  61. 61. An Example
  62. 62. An Example
  63. 63. An Example buzzword compliant!
  64. 64. An Example
  65. 65. An Example
  66. 66. Turning it all on authenticator: PasswordAuthenticator Tip: keep your read-only cqlsh credentials in $HOME/.cassandra/cqlshrc of the system's admin account
  67. 67. Turning it all on authorizer: CassandraAuthorizer
  68. 68. Turning it all on role_manager: CassandraRoleManager
  69. 69. Turning it all on authorizer: CassandraAuthorizer authenticator: PasswordAuthenticator role_manager: CassandraRoleManager WARNING: potential downtime!
  70. 70. authorizer: CassandraAuthorizer authenticator: PasswordAuthenticator role_manager: CassandraRoleManager Turning it all on WARNING: potential downtime! WARNING: stupid defaults
  71. 71. authorizer: CassandraAuthorizer authenticator: PasswordAuthenticator role_manager: CassandraRoleManager Turning it all on WARNING: potential downtime! WARNING: stupid defaults TIP: turn these WAY UP: permissions_validity_in_ms roles_validity_in_ms Also: use permissions_update_interval_in_ms for async refresh if needed
  72. 72. authorizer: CassandraAuthorizer authenticator: PasswordAuthenticator role_manager: CassandraRoleManager Turning it all on WARNING: potential downtime! WARNING: stupid defaults NEW in 3.4:credentials_validity_in_ms* * https://issues.apache.org/jira/browse/CASSANDRA-7715
  73. 73. Turning it all on authorizer: TransitionalAuthorizer authenticator: TransitionalAuthenticator DSE plugins to avoid downtime
  74. 74. Turning it all on system.schema_keyspace system.schema_columns system.schema_columnfamilies system.local system.peers These tables have default read permissions for every authenticated user:
  75. 75. Turning it all on IMPORTANT cassandra.yaml line note: "Please increase system_auth keyspace replication factor if you use this..." Tip: replication factor for the system_auth keyspace should be the same as the number of nodes in the data center
  76. 76. Turning it all on IMPORTANT cassandra.yaml line note: "Please increase system_auth keyspace replication factor if you use this..." Tip: replication factor for the system_auth keyspace should be the same as the number of nodes in the data center WARNING: stupid defaults* *https://issues.apache.org/jira/browse/CASSANDRA-11340
  77. 77. 4. Management and tooling
  78. 78. 4. Management and tooling
  79. 79. Securing JMX
  80. 80. nmap -Pn -p7199 -oG logs/cass.gnmap 54.88.0.0/14 Always a few suckers that TL,DR'ed
  81. 81. Why do I need to secure JMX?
  82. 82. Works as Advertised!
  83. 83. also good for some LOLs
  84. 84. Securing JMX SSL setup is like node to node and client to server http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
  85. 85. Securing JMX JMX Authentication is straightforward and well documented $JAVA_HOME/jre/lib/management/jmxremote.access $JAVA_HOME/jre/lib/management/ jmxremote.password.template http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
  86. 86. Securing JMX $JAVA_HOME/jre/lib/management/jmxremote.access $JAVA_HOME/jre/lib/management/ jmxremote.password.template Now you can: nodetool -u admin -pw secret compactionstats http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html JMX Authentication is straightforward and well documented
  87. 87. Securing JMX $JAVA_HOME/jre/lib/management/jmxremote.access $JAVA_HOME/jre/lib/management/ jmxremote.password.template Now you can: nodetool -u admin -pw secret compactionstats Tip: -pwf option will read the password from a file http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html JMX Authentication is straightforward and well documented
  88. 88. Securing JMX $JAVA_HOME/jre/lib/management/jmxremote.access $JAVA_HOME/jre/lib/management/ jmxremote.password.template Now you can: nodetool -u admin -pw secret compactionstats JMX Authentication is straightforward and well documented THIS JUST IN!!! RBAC for JMX Authentication and Authorization https://issues.apache.org/jira/browse/CASSANDRA-10091
  89. 89. Thanks!@zznate