Monitoring metrics

This article provides a list of data source monitoring metrics and descriptions.

Data source metrics

MetricDescription

upsolver.data_source_failed-items

Data source parse errors

upsolver.data_source_handled-bytes

Number of bytes handled by the data source

upsolver.data_source_handled-items

The number of items (records) handled by the data source

upsolver.data_source.ingest_delay

Data source ingestion delay

upsolver.data_source.parse_delay

Data source parse delay

upsolver.data_source.parse_min-delay

Data source minimum delay. This can be used as a filter for the delay metric

upsolver.data_source.errors_errors

Errors encountered when ingesting data

upsolver.binlog-delay

Read delay from the binlog of the database for CDC inputs

Lookup table metrics

MetricDescription

upsolver.materialized_view_estimated_rows

Number of rows currently in the lookup table (available if the lookup table runs on a query cluster)

upsolver.materialized_view.delay_delay

Lookup table delay

upsolver.materialized_view.query_hits

The number of queries that resulted in a lookup table hit (available if the lookup table runs on a query cluster)

upsolver.materialized_view.query_misses

The number of queries that resulted in a lookup table miss (available if the lookup table runs on a query cluster)

upsolver.materialized_view.query_queries

Number of queries sent to the lookup table (available if the lookup table runs on a query cluster)

upsolver.materialized_view.errors_errors

Errors encountered while running the lookup table

Output metrics

MetricDescription

upsolver.output_handled-bytes

Number of bytes handled by the output

upsolver.output_handled-items

Number of items (records) handled by the data source

upsolver.output_written-items

Number of items written to the output destination

upsolver.output_written-bytes

The number of bytes written to the output destination

upsolver.output.delay_delay

Output delay

upsolver.output.delay_min-delay

Output minimum delay. This can be used as a filter for the delay metric

upsolver.output.errors_errors

Errors encountered when outputting data

upsolver.output.data-transformation_handled-items

Number of items (records) handled during output processing (only relevant to database and stream outputs)

upsolver.output.data-transformation_handled-bytes

Number of bytes handled during output processing (only relevant to database and stream outputs)

upsolver.output.data-transformation-delay_delay

Delay during output processing (only relevant to database and stream outputs)

upsolver.output.hive.partition_manager_last-success-diff

The amount of time passed since the last successful execution of the partition manager in Athena/Qubole/Redshift Spectrum outputs

upsolver.output.hive.partition_manager_partitions-count

The number of partitions in the table (for Athena/Qubole/Redshift Spectrum outputs)

upsolver.output.hive.partition_manager_partitions-delay

When using PARTITION_TIME, this metric calculates the delay between now and the latest partition time (for Athena/Qubole/Redshift Spectrum outputs)

upsolver.output.hive.partition_manager_data-loading-delay

When creating partitions, this metric shows the delay of the data loading used to calculate the partitions (for Athena/Qubole/Redshift Spectrum outputs)

upsolver.output.hive.partition_manager_wal-size

When using Upserts, this shows the number of rows in the WAL (Write Ahead Log) partition (for Athena/Qubole/Redshift Spectrum outputs)

upsolver.output.hive.partition_manager_wal-files-count

When using Upserts, this shows the number of files in the WAL (Write Ahead Log) partition (for Athena/Qubole/Redshift Spectrum outputs)

upsolver.output.hive.partition_manager_compaction-delay

Shows the delay of the compaction process of Data Lake outputs (Athena/Spectrum/etc). Delayed compactions affect query speeds but they do not affect data freshness.

Compute cluster and environment metrics

MetricDescription

upsolver.upsolver_compute_units_value

The amount of Upsolver compute units used in a cluster

upsolver.environment_utilization_value

The average CPU load of all the servers in a cluster

upsolver.memory_memory

Memory usage per server

Last updated