Data Storage and Retention

The retention period for historical data is configured through the system settings in the WebGUI. This includes metric data, events, notifications, and web transaction results. Some tables are partitioned. Partitions older than the configured retention period are automatically removed by the DB Manager of the Bloonix Server.

Metrics

The metric values of all checks (with the exception of web transactions) are stored in high resolution in the stat table using daily partitions in the datastore. The Archive Manager of the Bloonix Server aggregates the metric values of each hour into the values MIN, MAX, and AVG and stores them in the stat_hourly table, which is partitioned by month.

A special case applies to checks that are executed through satellites. During aggregation, the metric values of all satellites assigned to a service are combined. The mapping to individual satellites is not preserved in the aggregated data. As a result, metrics can only be analyzed per satellite for the period configured by the metric retention setting.

Storage Requirements for Metrics

The storage requirements of a check depend on various factors, in particular the number of metrics, the check interval, the number of status changes with debug information, and other influences. As a general guideline, a single metric stored in the stat table requires approximately 82 bytes of storage. With a check interval of 60 seconds, this results in a storage requirement of approximately 41 MB per year. Due to hourly aggregation, the data stored in the stat_hourly table requires only a fraction of this storage.

For example, the check-linux-cpu check collects 11 metrics. With a check interval of 60 seconds and a retention period of one year, this results in a storage requirement of approximately 451 MB (11 × 41 MB) in the stat table and approximately 9 MB in the stat_hourly table.

By default, the retention period for metrics is 365 days, while aggregated metrics are retained for 10 years. If the available storage space is insufficient, it is recommended to reduce the metric retention period or move the datastore to a server with sufficient storage capacity.

To determine the current storage usage of the individual partitions, execute the following command on the datastore:

SELECT
    c.relname AS partition,
    pg_size_pretty(pg_total_relation_size(c.oid)) AS total,
    pg_size_pretty(pg_relation_size(c.oid)) AS heap,
    pg_size_pretty(pg_total_relation_size(c.reltoastrelid)) AS toast_total
FROM pg_class c
JOIN pg_inherits i ON i.inhrelid = c.oid
JOIN pg_class p ON p.oid = i.inhparent
WHERE p.relname = 'stat'
ORDER BY pg_total_relation_size(c.oid) DESC;

To determine the approximate storage requirements of a check in the stat table, the check can be run with the —plugin-dstat option. The following example shows the check-linux-cpu check:

check-linux-cpu --plugin-dstat --pretty
{
    "bytes_per_interval": 902,
    "num_metrics": 11
}

Bitte beachte

Events

Events store the historical status changes of a service check. They are particularly important for availability reports. Events are stored in the event table using daily partitions.

Web Transactions

The results of web transactions are stored in the result table using daily partitions in the datastore.

Notifications

Notifications sent by email, SMS, or other messaging services are stored in the notification table.