Scylla Documentation Logo Documentation
  • Server
    • Scylla Open Source
    • Scylla Enterprise
    • Scylla Alternator
  • Cloud
    • Scylla Cloud
    • Scylla Cloud Docs
  • Tools
    • Scylla Manager
    • Scylla Monitoring Stack
    • Scylla Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
Download
Menu
Scylla Adding and Modifying Dashboards

Adding and Modifying Dashboards¶

The following document explains how to update or create Grafana dashboard for Scylla Monitoring stack.

It will explain about Scylla monitoring templated dashboards and how to modify them.

General Limitations¶

Scylla Monitoring Stack uses Grafana for its dashboards. As of Grafana version 5.0, there are limitations to the changes you can make to dashboards.

You can create your own dashboard and modify it with the GUI, but you will face a consistency issue (keeping the dashboard between grafana container restart).

At large, we suggest maintaining your dashboards as files, as Scylla Monitoring Stack does.

Note

You cannot save a dashboard that was created as a file when you modify it with the GUI.

Using templated Dashboards¶

Why Scylla Monitoring uses templated dashboards?

We found the Grafana dashboards Json format too verbose to be maintainable.

Each element in the dashboard file (Each Json object) contains all of its attributes and values.

For example a typical graph panel would look like:

{
    "aliasColors": {},
    "bars": false,
    "datasource": "prometheus",
    "editable": true,
    "error": false,
    "fill": 0,
    "grid": {
        "threshold1": null,
        "threshold1Color": "rgba(216, 200, 27, 0.27)",
        "threshold2": null,
        "threshold2Color": "rgba(234, 112, 112, 0.22)"
    },
    "gridPos": {
        "h": 6,
        "w": 10,
        "x": 0,
        "y": 4
    },
    "id": 2,
    "isNew": true,
    "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": false,
        "total": false,
        "values": false
    },
    "lines": true,
    "linewidth": 2,
    "links": [],
    "nullPointMode": "connected",
    "percentage": false,
    "pointradius": 5,
    "points": false,
    "renderer": "flot",
    "seriesOverrides": [
        {}
    ],
    "span": 5,
    "stack": false,
    "steppedLine": false,
    "targets": [
        {
            "expr": "sum(node_filesystem_avail) by (instance)",
            "intervalFactor": 1,
            "legendFormat": "",
            "refId": "A",
            "step": 1
        }
    ],
    "timeFrom": null,
    "timeShift": null,
    "title": "Available Disk Size",
    "tooltip": {
        "msResolution": false,
        "shared": true,
        "sort": 0,
        "value_type": "cumulative"
    },
    "transparent": false,
    "type": "graph",
    "xaxis": {
        "show": true
    },
    "yaxes": [
        {
            "format": "percent",
            "logBase": 1,
            "max": 101,
            "min": 0,
            "show": true
        },
        {
            "format": "short",
            "logBase": 1,
            "max": null,
            "min": null,
            "show": true
        }
    ]
}

As you can imagine, most panels would have similar values.

To reduce the redundancy of Grafana JSON format, we added templated dashboards.

The template classes system¶

Scylla Monitoring templates use class attribute that can be added to any JSON object in a template file. The different classes are defined in a file.

The class system resembles CSS classes. It is hierarchical, so a class type definition can have a class attribute and it would inherit that class attributes, the inherit class can add or override inherited attributes.

In the template file, you can also add or override attributes.

Scylla Monitoring Stack generation script, uses the types.json file and a template file and creates a dashboard.

When generating dashboards, each class will be replaced by its definition.

For example row in the type.json defined

{
 "base_row": {
     "collapse": false,
     "editable": true
 },
 "row": {
     "class": "base_row",
     "height": "250px"
 }
 }

In a Template it will be used like

{
     "class": "row",
     "height": "150px",
     "panels": [
     ]

The output will be

{
     "class": "row",
     "collapse": false,
     "editable": true,
     "height": "150px",
     "panels": [

     ]
}

We can see that the template added the panels attribute and override the height attribute.

Panel example¶

Look at the following example that defines a row inside a dashboard with a graph panel for the available disk size.

{
     "class": "row",
     "panels": [
         {
             "class": "bytes_panel",
             "span": 3,
             "targets": [
                 {
                     "expr": "sum(node_filesystem_avail) by (instance)",
                     "intervalFactor": 1,
                     "legendFormat": "",
                     "metric": "",
                     "refId": "A",
                     "step": 1
                 }
             ],
             "title": "Available Disk Size"
         }
     ]
}

In the example we used the bytes_panel class that generate a graph with bytes as units (that would mean that your Y axis unit would adjust themselves to make the graph readable (i.e. GB, MB, bytes, etc’)

You can also see that we override the span attribute to set the panel size.

To get a grasp of the difference, take a look at the Grafana panel example and see how it looks originally.

Grafana Formats and Layouts¶

Grafana layout used to be based on rows, each contains multiple panels. Each row would have a total span of 12, if the total span of the panels be larger than 12, it would break the lines into multiple lines.

Starting from Grafana version 5.0 and later, rows are no longer supported, it was replaced with a layout that uses absolute positions (ie. X,Y, height, width).

The server should be backward compatible, but we’ve found it had issues with parsing it correctly. More so, absolute positions are impossible to get right when done by hand.

To overcome these issues, the dashboard generation script will generate the dashboards in the Grafana version 5.0 format. In the transition, rows will be replaced with a calculated absolute position.

Panels height will be taken from their row. The span attribute is stil supported like you expect it to work, so does row height.

you can use the gridPos attribute which is a Grafana 5.0 format, but not like Grafana, you can use partial attributes.

gridPos has the following attributes:

{
   "x": 0,
   "y": 0,
   "w": 24,
   "h": 4
 }

When using Scylla’s template you don’t need to supply them all, so for example to specify that a row is 2 units height you can use:

{
   "gridPos": {
      "h": 2
    }
}

Generating the dashboards from templates (generate-dashboards.sh)¶

prerequisite¶

Python 2.7

make_dashboards.py is a utility that generates dashboards from templates or help you update the template when working in reverse mode (the -r flag).

Use the -h flag to get help information.

You can use the make_dashboards.py to generate a single dashboard, but it’s usually easier to use the generate-dashboards.sh wrapper.

When you’re done changing an existing dashboard template, run the generate-dashboards.sh with the current version, this will replace your existing dashboards.

For example, if you are changing a dashboard in Scylla Enterprise version 2018.1 run:

.\generate-dashboards.sh -v 2018.1

Note

generate-dashboards.sh will update the dashboards in place, no need for a restart for the changes to take effect, just refresh the dashboard.

Validation¶

After making changes to a template, run the generate_generate-dashboards.sh you should see that it run without any errors.

Refresh your browser for changes to take effect. Make sure that your panels contain data, if not, maybe there is something wrong with your expr attribute.

  • Getting Started
    • Install Scylla
      • Scylla Unified Installer (relocatable executable)
      • Air-gapped Server Installation
      • What is in each RPM
      • Scylla Housekeeping and how to disable it
      • Scylla Developer Mode
      • Scylla Configuration Reference
    • Configure Scylla
    • Scylla Requirements
      • System Requirements
      • OS Support by Platform and Version
      • Scylla in a Shared Environment
    • Cassandra Query Language (CQL)
      • CQLSh the CQL shell
      • Data Definition
      • Data Manipulation
      • Expiring Data with Time to Live (TTL)
      • Additional Information
      • Security
      • Data Types
      • Appendices
      • Definitions
      • Materialized Views
      • Functions
      • JSON
      • Global Secondary Indexes
      • Additional Information
      • Compaction
      • Consistency Levels
      • Reserved Keywords
      • Non-reserved Keywords
    • CQLSh: the CQL shell
    • Scylla Drivers
      • Scylla CQL Drivers
      • Scylla DynamoDB Drivers
    • Migrate to Scylla
      • Migration Process from Cassandra to Scylla
      • Scylla and Apache Cassandra Compatibility
      • Migration Tools Overview
    • Integration Solutions
      • Integrate Scylla with Spark
      • Integrate Scylla with KairosDB
      • Integrate Scylla with Presto
      • Integrate Scylla with Elasticsearch
      • Integrate Scylla with Kubernetes
      • Integrate Scylla with the JanusGraph Graph Data System
      • Integrate Scylla with DataDog
      • Integrate Scylla with Kafka
      • Integrate Scylla with IOTA Chronicle
      • Integrate Scylla with Spring
      • Shard-Aware Kafka Connector for Scylla
      • Install Scylla with Ansible
      • Integrate Scylla with Databricks
    • Tutorials
  • Scylla for Administrators
    • Administration Guide
    • Procedures
      • Cluster Management
      • Backup & Restore
      • Change Configuration
      • Maintenance
      • Best Practices
      • Benchmarking Scylla
      • Migrate from Cassandra to Scylla
      • Disable Housekeeping
    • Security
      • Scylla Security Checklist
      • Enable Authentication
      • Enable and Disable Authentication Without Downtime
      • Generate a cqlshrc File
      • Reset Authenticator Password
      • Enable Authorization
      • Grant Authorization CQL Reference
      • Role Based Access Control (RBAC)
      • Scylla Auditing Guide
      • Encryption: Data in Transit Client to Node
      • Encryption: Data in Transit Node to Node
      • Generating a self-signed Certificate Chain Using openssl
      • Encryption at Rest
      • LDAP Authentication
      • LDAP Authorization (Role Management)
    • Admin Tools
      • Nodetool Reference
      • CQLSh
      • REST
      • Tracing
      • scylla-sstable
      • SSTableLoader
      • cassandra-stress
      • SSTabledump
      • SSTable2json
      • SSTable Index
      • Scylla Logs
      • Seastar Perftune
    • Scylla Manager
      • Scylla Manager Docs
      • Upgrade Scylla Manager
      • Monitoring Support Matrix
    • Scylla Monitoring Stack
      • Latest Version
      • Upgrade Scylla Monitoring Stack
      • Monitoring Support Matrix
    • Scylla Operator
    • Upgrade Procedures
      • Scylla Enterprise
      • Scylla Open Source
      • Scylla Open Source to Scylla Enterprise
      • Scylla Manager
      • Scylla Monitoring
      • Scylla AMI
    • System Configuration
      • System Configuration Guide
      • scylla.yaml
      • Scylla Snitches
    • Benchmarking Scylla
  • Scylla for Developers
    • Learn To Use Scylla
      • Scylla University
      • Course catalog
      • Scylla Essentials
      • Basic Data Modeling
      • Advanced Data Modeling
      • MMS - Learn by Example
      • Care-Pet an IoT Use Case and Example
    • CQLSh
    • Apache Cassandra Query Language (CQL)
    • Scylla Alternator
    • Scylla Features
      • Scylla Open Source Features
      • Scylla Enterprise Features
    • Scylla Drivers
      • Scylla CQL Drivers
      • Scylla DynamoDB Drivers
  • Scylla Architecture
    • Scylla Ring Architecture
    • Scylla Fault Tolerance
    • Consistency Level Console Demo
    • Scylla Anti-Entropy
      • Scylla Hinted Handoff
      • Scylla Read Repair
      • Scylla Repair
    • SSTable
      • Scylla SSTable - 2.x
      • Scylla SSTable - 3.x
    • Compaction Strategies
  • Troubleshooting Scylla
    • Errors and Support
      • Report a Scylla problem
      • Error Messages
      • Change Log Level
    • Scylla Startup
      • Ownership Problems
      • Scylla will not Start
      • Scylla Python Script broken
    • Cluster and Node
      • Failed Decommission Problem
      • Cluster Timeouts
      • Node Joined With No Data
      • SocketTimeoutException
      • NullPointerException
    • Data Modeling
      • Scylla Large Partitions Table
      • Scylla Large Rows and Cells Table
      • Large Partitions Hunting
    • Data Storage and SSTables
      • Space Utilization Increasing
      • Disk Space is not Reclaimed
      • SSTable Corruption Problem
      • Pointless Compactions
      • Limiting Compaction
    • CQL
      • Time Range Query Fails
      • COPY FROM Fails
      • CQL Connection Table
      • Reverse queries fail
    • Scylla Monitor and Manager
      • Manager and Monitoring integration
      • Manager lists healthy nodes as down
  • Knowledge Base
    • Upgrading from experimental CDC
    • Compaction
    • Counting all rows in a table is slow
    • CQL Query Does Not Display Entire Result Set
    • When CQLSh query returns partial results with followed by “More”
    • Run Scylla and supporting services as a custom user:group
    • Decoding Stack Traces
    • Snapshots and Disk Utilization
    • DPDK mode
    • Debug your database with Flame Graphs
    • How to Change gc_grace_seconds for a Table
    • Gossip in Scylla
    • Increase Permission Cache to Avoid Non-paged Queries
    • How does Scylla LWT Differ from Apache Cassandra ?
    • Map CPUs to Scylla Shards
    • Scylla Memory Usage
    • NTP Configuration for Scylla
    • POSIX networking for Scylla
    • Scylla consistency quiz for administrators
    • Recreate RAID devices
    • How to Safely Increase the Replication Factor
    • Scylla and Spark integration
    • Increase Scylla resource limits over systemd
    • Scylla Seed Nodes
    • How to Set up a Swap Space
    • Scylla Snapshots
    • Stopping a local repair
    • System Limits
    • How to flush old tombstones from a table
    • Time to Live (TTL) and Compaction
    • Scylla Nodes are Unresponsive
    • Update a Primary Key
    • Using the perf utility with Scylla
    • Configure Scylla Networking with Multiple NIC/IP Combinations
  • Scylla University
  • Scylla FAQ
  • Contribute to Scylla
  • Glossary
  • Create an issue

On this page

  • Adding and Modifying Dashboards
    • General Limitations
    • Using templated Dashboards
      • The template classes system
      • Panel example
      • Grafana Formats and Layouts
      • Generating the dashboards from templates (generate-dashboards.sh)
        • prerequisite
      • Validation
Logo
Docs Contact Us About Us
Mail List Icon Slack Icon
© 2022, ScyllaDB. All rights reserved.
Last updated on 22 September 2021.
Powered by Sphinx 4.3.2 & ScyllaDB Theme 1.2.1