Encryption: Data in Transit Client to Node

Follow the procedures below to enable a client to node encryption.
Each Scylla node needs to be enabled for SSL encryption separately. Repeat this procedure for each node.
If you are working on a new cluster skip steps 1-2.
Steps 1-2 must be executed on every Scylla node you wish to enable SSL encryption mode. These steps can be done on all nodes together or one-by-one.
Steps 3-6 should be performed one node at a time (one by one), so to allow the cluster to continue serving requests.
Steps 7-9 are client-side validation steps to demonstrate that clients can now connect in SSL encryption mode.

Procedure

  1. Run nodetool drain.
  2. Stop Scylla.

CentOS, RHEL or Ubuntu 16.04

sudo systemctl stop scylla-server

Ubuntu 14.04 or Debian

sudo service scylla-server stop
  1. Edit /etc/scylla/scylla.yaml to modify the client_encryption_options.

Available options are:

  • enabled (default - false)
  • certificate - A PEM format certificate, either self-signed, or provided by a certificate authority (CA).
  • keyfile - The corresponding PEM format key for the certificate
  • truststore - Optional path to a PEM format certificate store of trusted CA:s. If not provided, Scylla will attempt to use the system truststore to authenticate certificates.

Note - If using a self-signed certificate, the “truststore” parameter need to be set to a PEM format container with the private authority.

For example:

client_encryption_options:
    enabled: true
    certificate: /etc/scylla/db.crt
    keyfile: /etc/scylla//db.key
    truststore: <optional path to PEM encoded trust store>

Note: params (enabled, certificate, keyfile) must be 4 spaces indent to the header (client_encryption_options:)

  1. Start Scylla:

CentOS, RHEL or Ubuntu 16.04

sudo systemctl start scylla-server

Ubuntu 14.04 or Debian

sudo service scylla-server start
  1. To validate that encrypted connection to the node is enabled, check the logs using journalctl _COMM=scylla. You should see the following message: storage_service - Enabling encrypted CQL connections between client and node.

Client side validation steps

6. In order for cqlsh to work in client to node encryption SSL mode, you need to generate cqlshrc file: vi ~/.cassandra/cqlshrc

Example:
[authentication]
username = myusername
password = mypassword
[cql]
; Substitute for the version of Cassandra you are connecting to.
version = 3.3.1
[connection]
hostname = 127.0.0.1
port = 9042
factory = cqlshlib.ssl.ssl_transport_factory
[ssl]
certfile = /etc/scylla/db.crt
; Note: If validate = true then the certificate name must match the machine's hostname
validate = true
; If using client authentication (require_client_auth = true in cassandra.yaml) you'll also need to point to your userkey and usercert.
; SSL client authentication is only supported via cqlsh on C* 2.1 and greater.
; This is disabled by default on all Instaclustr-managed clusters.
userkey = /etc/scylla/db.key
usercert = /etc/scylla/db.crt
  1. Copy the following created files (db.key, db.crt, cadb.key, cadb.pem) to your client/s, from which you run cassandra-stress
  2. In order to run cassandra-stress using SSL you need to create java key store (jks) from the .pem file on the every client that runs cassandra-stress
  • Generate the Java keystore for the node certs
openssl pkcs12 -export -out keystore.p12 -inkey /home/scylla/server_files/db.key -in /home/scylla/server_files/db.crt

keytool -importkeystore -destkeystore keystore.jks -srcstoretype PKCS12 -srckeystore keystore.p12
  • Generate the Java truststore for the trust provider
openssl pkcs12 -export -out truststore.p12 -inkey /home/scylla/server_files/cadb.key -in /home/scylla/server_files/cadb.pem

keytool -importkeystore -destkeystore truststore.jks -srcstoretype PKCS12 -srckeystore truststore.p12
  • Download and install the Java security providers:

Install to <jre>/lib/security

Note: make sure you have the latest version from this location.

  1. Run Cassandra stress with the parameters below:
cassandra-stress write n=1000000 cl=ONE -node 10.240.0.48 -transport keystore=keystore.jks keystore-password=[password] truststore=truststore.jks truststore-password=[password] -mode native cql3 -pop -rate threads=50

Note: when running cassandra-stress you may encounter an exception, if some nodes are still not in client to node SSL encrypted mode, yet the cassandra-stress will continue to run and connect only to the nodes it can.

Note2: This procedure works as-is for v1.7 or higher. When using Scylla v1.6.x you will need a dummy keystore in the default (conf/.keystore) location with password “cassandra” to run. The contents is irrelevant. Also, it only pertains to cassandra-stress. It has no impact/relation to using the normal java driver connection or cqlsh.

  1. Enable encryption on the client application.