Jobs

The jobs system table includes information about the jobs that run within your organization.

From a worksheet within Upsolver, run the following query:

SELECT * 
FROM system.monitoring.jobs;

The system.monitoring.jobs table includes the following columns:

Column	Type	Description
avg_file_size_today	bigint	The average size of the files discovered by the job today.
avg_rows_scanned_per_execution_today	bigint	The average number of rows scanned per job execution.
bytes_written_today	bigint	The size of the data written by the job.
catalog_error	string	Errors returned by the catalog today.
columns_written_to_today	bigint	The number of columns written to by the job. This value can change over time if the query uses * in the select clause.
completed_executions_today	bigint	The number of job executions completed today.
discovered_bytes_today	bigint	The number of bytes to load discovered in the source stream.
discovered_files_today	bigint	The number of files to load discovered by the job today.
discovered_records_today	bigint	The number of records to load discovered by the job.
drop_expectations_triggered_today	bigint	The number of expectations with a DROP action that was triggered by the job today.
execution_failure_reason	string	The error message detailing why the job failed.
executions_retrying_after_failure	bigint	The number of job executions that encountered an error and are currently retrying.
executions_waiting_for_dependencies	bigint	The number of job executions that are waiting for a dependency to complete.
input_id	string	The unique identifier for the ingestion job input.
job_id	string	The unique identifier for the job.
job_name	string	The text name of the job.
load_failure_reason	string	The error message as to why the data failed to load.
max_bytes_written_to_single_partition_today	bigint	The maximum number of bytes written to a single partition by the job today.
max_rows_scanned_in_execution_today	bigint	The maximum number of rows scanned in a single job execution today.
next_run_time	string	The date and time when the job will next run.
parse_errors_today	bigint	The number of items that failed to parse. This value represents a lower bound as malformed items may corrupt subsequent items in the same file as well.
parse_errors_lifetime	bigint	The total items that failed to parse. Parse errors before the configured METADATA_RETENTION of the job will not be counted.
parquet_files_written_today	bigint	The number of Parquet files written to by the job today.
partitions_written_to_today	bigint	The number of partitions written to by the job today.
queued_executions	bigint	The number of queued job executions pending.
running_executions	bigint	The number of currently running job executions.
rows_filtered_by_dedup_clause_today	bigint	The number of rows filtered out by a deduplication clause in the job today.
rows_filtered_by_expectations_today	bigint	The number of rows filtered out by an expectation today.
rows_filtered_by_having_clause_today	bigint	The number of rows that were filtered out because they didn’t pass the HAVING clause predicate defined in the job.
rows_filtered_by_missing_partition_today	bigint	The number of rows that were filtered out because some or all of the partition columns were NULL or empty string.
rows_filtered_by_missing_primary_key_today	bigint	The number of rows that were filtered out because some or all of the primary key columns were NULL.
rows_pending_processing	bigint	The number of rows in the source table that have not been processed yet. Only rows that have been committed to the source table are included.
rows_scanned_by_cancelled_tasks_today	bigint	The number of rows scanned by tasks that were cancelled today.
rows_scanned_by_completed_tasks_today	bigint	The total number of rows scanned by completed executions today. This is a measure of rows that were processed successfully.
rows_scanned_by_failed_tasks_today	bigint	The number of rows scanned by tasks that failed in the job today.
rows_filtered_by_where_clause_today	bigint	The number of rows that were filtered out because they didn’t pass the WHERE clause predicate defined in the job.
rows_scanned_today	bigint	The number of rows scanned by the job today.
rows_written_today	bigint	The number of rows written to the target by the job today.
running_executions	bigint	The number of currently running job executions.
sparse_columns_written_to_today	bigint	The number of sparse columns written to today. A sparse column is a column that appears in less than 0.01% of all rows.
tasks_failing_to_load	bigint	The number of tasks in the job that failed to load today.
total_completed_executions	bigint	The number of job executions completed over the lifetime of the job.
total_tasks_to_load	bigint	The total number of tasks that were loaded.
warn_expectations_triggered_today	bigint	The number of expectations with a WARN action that was triggered in the job today.