Scylla Documentation Logo Documentation
  • Server
    • Scylla Open Source
    • Scylla Enterprise
    • Scylla Alternator
  • Cloud
    • Scylla Cloud
    • Scylla Cloud Docs
  • Tools
    • Scylla Manager
    • Scylla Monitoring Stack
    • Scylla Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
Download
Menu
Scylla The CQL Optimization

The CQL Optimization¶

The CQL Optimization is part of the CQL Dashboard and is a tool to help identify potentials issues with queries, data models, and drivers.

../../../../_images/cql_optimization_master4.png

The CQL Dashboard¶

The upper part of the dashboard holds CQL related metrics.

The lower parts hold gauges and graphs. When inspecting the system, we like the gauge to be near zero and the graphs as low as possible.

Note

Besides your queries, there are queries generated by the cql driver and internal queries to the system tables, which can be misleading when testing with low traffic.

The following sections describe each of the dashboard’s panel

Prepared Statements¶

Prepared statements are queries that are first defined as a template with place holders for the values, and then that template is used multiple times with different values.

Using prepared statements has the following benefits:

  • The database only needs to parse the query once

  • The driver can route the query to the right node

  • Using place-holders and values is safer and prevents CQL-Injection

The CQL Non-Prepared Queries Gauge shows the percentage of queries that are not prepared.

The CQL Non-Prepared Queries Graph shows the rate of the queries. Make sure both are low.

Token Aware¶

Scylla is a distributed database, with each node contains only part of the data - a range of the token ring. Ideally, a query would reach the node that holds the data (one of the replicas), failing to do so would mean the coordinator will need to send the query internally to a replica, result with higher latency, and more resources usage.

Typically, your driver would know how to route the queries to a replication node, but using non-prepared statements, non-token-aware driver or load-balance can cause the query to reach a node that is not a replica.

The Non-Token Aware Gauge shows the percentage of queries that reached a node that does not hold that data (a node that is not a replica-node).

The Non-Token Aware Queries Graph shows the rate of the queries that did not reach a replica-node. Make sure both are low.

Paged Queries¶

By default, read queries are paged. This means that Scylla will break the results into multiple chunks limiting the reply size. Non-Paged queries require all results to be returned in one result, increasing the overall load of the system and clients and should be avoided.

The Non-Paged CQL Reads Gauge shows the percentage of non-paged read queries that did not use paging.

The Non-Paged CQL Reads Graph shows the rate of the non-paged queries. Make sure both are low.

Reversed CQL Reads¶

Scylla supports compound primary keys with a clustering column, this kind of primary keys allows an efficient way to return sorted results that are sorted by the clustering column.

Querying with an order different than the order the CLUSTERING ORDER BY was defined is inefficient and should be avoided.

For example, look at the following table:

CREATE TABLE ks1.table_demo (
   category text,
   type int,
   PRIMARY KEY (category, type))
WITH CLUSTERING ORDER BY (type DESC);

The following query uses reverse order:

select * from ks1.table_demo where category='cat1' order by type ASC;

The Reversed CQL Reads Gauge shows the percentage of read queries that use ORDER BY that is different than the CLUSTERING ORDER BY.

The Reversed CQL Reads Graph shows the rate of the read queries that use ORDER BY that is different than the CLUSTERING ORDER BY. Make sure both are low.

ALLOW FILTERING¶

Scylla supports server-side data filtering that is not based on the primary key. This means Scylla would read data and then filter and return part of it to the user. Data that is read and then filtered is an overhead to the system.

These kinds of queries can create a big load on the system and should be used with care.

The CQL optimization dashboard, check for two things related to queries that use ALLOW FILTERING how many such queries exist and how much of the data that was read was dropped before returning to the client.

The ALLOW FILTERING CQL Reads Gauge shows the percentage of read queries that use ALLOW FILTERING.

The ALLOW FILTERING CQL Reads Graph shows the rate of the read queries that use ALLOW FILTERING. Make sure both are low.

The ALLOW FILTERING Filtered Rows Gauge shows the percentage of rows that were read and then filtered. This is an indication of the additional overhead to the system.

The ALLOW FILTERING Filtered Rows Graph shows multiple graphs: the rows that were read, the rows that match, and the rows that were dropped. Rows that were dropped are an additional overhead to the system.

Consistency Level¶

Typically data in Scylla is duplicated into multiple replicas for availability reasons. A coordinator node would get the request and will send it to the nodes holding the replicas.

A query Consistency Level determines at what point the coordinator will reply to the client with regards to the number of replied it got from the replicas. The most common case is to use QUORUM, which means that when the coordinator gets a majority of the replies from the replicas, it will return success to the client.

Two consistency levels hold a potential problem and should be used with care ANY and ALL.

The CQL ANY Queries Gauge shows the percentage of queries that use Consistency Level ANY. Using consistency level ANY in a query may hurt persistency. If the node receiving the request will fail, the data may be lost.

The CQL ANY CL Queries Graph shows the rate of the queries that use Consistency Level ANY. Make sure both are low.

The CQL ALL CL Queries Gauge shows the percentage of queries that use the Consistency Level ALL. Using consistency level ALL in a query may hurt availability. If a node is unavailable, the operations will fail.

The CQL ALL CL Queries Graph shows the rate of the queries that use Consistency Level ALL. Make sure both are low.

Cross DC¶

Cross DC traffic is usually more expensive in terms of latencies and cost. This metric reports on such traffic in situations were it could be avoided.

Cross DC Consistency Level¶

Using consistency level QUORUM or consistency level ONE in a query when there is more than one DC may hurt performance, as queries may end in the non-local DC. Use LOCAL_QUORUM and LOCAL_ONE instead.

Cross DC read requests¶

Note

The CQL Optimization Dashboard relies on the definition of nodes per Data Center in the Monitoring Stack (prometheus/scylla_servers.yml) to match the Data Center names used in Scylla Cluster. If this is not the case, you will see the wrong result.

In a typical situation, a client performs a read from the nearest data-center, and that query is performed locally to the data-center. A read request that ends up causing traffic between data-centers adds additional overhead to the system.

The Cross DC read requests Gauge shows the percentage of read queries that caused a request to an external data-center. Make sure it is low or zero.

  • Getting Started
    • Install Scylla
      • Scylla Unified Installer (relocatable executable)
      • Air-gapped Server Installation
      • What is in each RPM
      • Scylla Housekeeping and how to disable it
      • Scylla Developer Mode
      • Scylla Configuration Reference
    • Configure Scylla
    • Scylla Requirements
      • System Requirements
      • OS Support by Platform and Version
      • Scylla in a Shared Environment
    • Cassandra Query Language (CQL)
      • CQLSh the CQL shell
      • Data Definition
      • Data Manipulation
      • Expiring Data with Time to Live (TTL)
      • Additional Information
      • Security
      • Data Types
      • Appendices
      • Definitions
      • Materialized Views
      • Functions
      • JSON
      • Global Secondary Indexes
      • Additional Information
      • Compaction
      • Consistency Levels
      • Reserved Keywords
      • Non-reserved Keywords
    • CQLSh: the CQL shell
    • Scylla Drivers
      • Scylla CQL Drivers
      • Scylla DynamoDB Drivers
    • Migrate to Scylla
      • Migration Process from Cassandra to Scylla
      • Scylla and Apache Cassandra Compatibility
      • Migration Tools Overview
    • Integration Solutions
      • Integrate Scylla with Spark
      • Integrate Scylla with KairosDB
      • Integrate Scylla with Presto
      • Integrate Scylla with Elasticsearch
      • Integrate Scylla with Kubernetes
      • Integrate Scylla with the JanusGraph Graph Data System
      • Integrate Scylla with DataDog
      • Integrate Scylla with Kafka
      • Integrate Scylla with IOTA Chronicle
      • Integrate Scylla with Spring
      • Shard-Aware Kafka Connector for Scylla
      • Install Scylla with Ansible
      • Integrate Scylla with Databricks
    • Tutorials
  • Scylla for Administrators
    • Administration Guide
    • Procedures
      • Cluster Management
      • Backup & Restore
      • Change Configuration
      • Maintenance
      • Best Practices
      • Benchmarking Scylla
      • Migrate from Cassandra to Scylla
      • Disable Housekeeping
    • Security
      • Scylla Security Checklist
      • Enable Authentication
      • Enable and Disable Authentication Without Downtime
      • Generate a cqlshrc File
      • Reset Authenticator Password
      • Enable Authorization
      • Grant Authorization CQL Reference
      • Role Based Access Control (RBAC)
      • Scylla Auditing Guide
      • Encryption: Data in Transit Client to Node
      • Encryption: Data in Transit Node to Node
      • Generating a self-signed Certificate Chain Using openssl
      • Encryption at Rest
      • LDAP Authentication
      • LDAP Authorization (Role Management)
    • Admin Tools
      • Nodetool Reference
      • CQLSh
      • REST
      • Tracing
      • scylla-sstable
      • SSTableLoader
      • cassandra-stress
      • SSTabledump
      • SSTable2json
      • SSTable Index
      • Scylla Logs
      • Seastar Perftune
    • Scylla Manager
      • Scylla Manager Docs
      • Upgrade Scylla Manager
      • Monitoring Support Matrix
    • Scylla Monitoring Stack
      • Latest Version
      • Upgrade Scylla Monitoring Stack
      • Monitoring Support Matrix
    • Scylla Operator
    • Upgrade Procedures
      • Scylla Enterprise
      • Scylla Open Source
      • Scylla Open Source to Scylla Enterprise
      • Scylla Manager
      • Scylla Monitoring
      • Scylla AMI
    • System Configuration
      • System Configuration Guide
      • scylla.yaml
      • Scylla Snitches
    • Benchmarking Scylla
  • Scylla for Developers
    • Learn To Use Scylla
      • Scylla University
      • Course catalog
      • Scylla Essentials
      • Basic Data Modeling
      • Advanced Data Modeling
      • MMS - Learn by Example
      • Care-Pet an IoT Use Case and Example
    • CQLSh
    • Apache Cassandra Query Language (CQL)
    • Scylla Alternator
    • Scylla Features
      • Scylla Open Source Features
      • Scylla Enterprise Features
    • Scylla Drivers
      • Scylla CQL Drivers
      • Scylla DynamoDB Drivers
  • Scylla Architecture
    • Scylla Ring Architecture
    • Scylla Fault Tolerance
    • Consistency Level Console Demo
    • Scylla Anti-Entropy
      • Scylla Hinted Handoff
      • Scylla Read Repair
      • Scylla Repair
    • SSTable
      • Scylla SSTable - 2.x
      • Scylla SSTable - 3.x
    • Compaction Strategies
  • Troubleshooting Scylla
    • Errors and Support
      • Report a Scylla problem
      • Error Messages
      • Change Log Level
    • Scylla Startup
      • Ownership Problems
      • Scylla will not Start
      • Scylla Python Script broken
    • Cluster and Node
      • Failed Decommission Problem
      • Cluster Timeouts
      • Node Joined With No Data
      • SocketTimeoutException
      • NullPointerException
    • Data Modeling
      • Scylla Large Partitions Table
      • Scylla Large Rows and Cells Table
      • Large Partitions Hunting
    • Data Storage and SSTables
      • Space Utilization Increasing
      • Disk Space is not Reclaimed
      • SSTable Corruption Problem
      • Pointless Compactions
      • Limiting Compaction
    • CQL
      • Time Range Query Fails
      • COPY FROM Fails
      • CQL Connection Table
      • Reverse queries fail
    • Scylla Monitor and Manager
      • Manager and Monitoring integration
      • Manager lists healthy nodes as down
  • Knowledge Base
    • Upgrading from experimental CDC
    • Compaction
    • Counting all rows in a table is slow
    • CQL Query Does Not Display Entire Result Set
    • When CQLSh query returns partial results with followed by “More”
    • Run Scylla and supporting services as a custom user:group
    • Decoding Stack Traces
    • Snapshots and Disk Utilization
    • DPDK mode
    • Debug your database with Flame Graphs
    • How to Change gc_grace_seconds for a Table
    • Gossip in Scylla
    • Increase Permission Cache to Avoid Non-paged Queries
    • How does Scylla LWT Differ from Apache Cassandra ?
    • Map CPUs to Scylla Shards
    • Scylla Memory Usage
    • NTP Configuration for Scylla
    • POSIX networking for Scylla
    • Scylla consistency quiz for administrators
    • Recreate RAID devices
    • How to Safely Increase the Replication Factor
    • Scylla and Spark integration
    • Increase Scylla resource limits over systemd
    • Scylla Seed Nodes
    • How to Set up a Swap Space
    • Scylla Snapshots
    • Stopping a local repair
    • System Limits
    • How to flush old tombstones from a table
    • Time to Live (TTL) and Compaction
    • Scylla Nodes are Unresponsive
    • Update a Primary Key
    • Using the perf utility with Scylla
    • Configure Scylla Networking with Multiple NIC/IP Combinations
  • Scylla University
  • Scylla FAQ
  • Contribute to Scylla
  • Glossary
  • Create an issue

On this page

  • The CQL Optimization
    • Prepared Statements
    • Token Aware
    • Paged Queries
    • Reversed CQL Reads
    • ALLOW FILTERING
    • Consistency Level
  • Cross DC
    • Cross DC Consistency Level
    • Cross DC read requests
Logo
Docs Contact Us About Us
Mail List Icon Slack Icon
© 2022, ScyllaDB. All rights reserved.
Last updated on 10 May 2022.
Powered by Sphinx 4.3.2 & ScyllaDB Theme 1.2.1