Job monitoring
Job monitoring summary page
Access to job monitoring's summary page. From this page, you can access each job to monitor or debug.
Name of the job that you've created. Click on each job to access a detailed job page.
Status of each job. Whether the job is running or in other phases.
Backlog of events being processed.
Number of errors in the last 24 hours.
A graph of the events processed since the job started. Hover over the graph for exact number of events processed at a certain point in time.
How long ago the job was created.
Where the data is being read from and loaded into.
Filter the jobs summary table by a value within a column.
COPY job monitoring (page 1)
Shows where the data is being copied from and where it is being loaded into. The example shows reading from S3 and loading into an Athena table.
The name of the job defined when it was created.
Whether the job is running or not.
The compute cluster that the job is running on.
Whether there are unsolved errors within the past hour. You can click on it to get more details on the errors.
How many events were parsed in the past hour.
The rate of events being processed in the past hour.
The timestamp waiting to be processed relative to the timestamp now.
For example, seeing a processing backlog of 3 hours means the events that are being processed have a timestamp that's 3 hours earlier than the current timestamp.
Allows graphs below to change based on the time range that you pick.
Shows the backlog being processed over time based on the time picker range.
Name of the current work a job is performing and the backlog in relation to that work.
COPY job monitoring (page 2)
Number of events processed over time based on the time picker.
CPU utilization of a cluster. You can see the resources the job is consuming relative to the entire cluster utilization.
How many errors occurred over time based on the time picker.
Job monitoring (page 3)
This table gives you a view of the errors.
You may filter the table by Unresolved Errors, Resolved Errors, or All Errors over a period of time.
Details of each error that were generated by your job. This can help you to troubleshoot your job.
For example, if there are any permission errors, it will show up in this table and you will know to adjust your permissions accordingly.
You may click on Query System Table to start writing SQL against the system table. The system table is very powerful and can help you to debug your jobs and their related tasks.
C
The
COPY FROM jobs have an additional panel to help you debug parsing errors. For example, if your JSON has formatting errors, the object will show up on the parsing error panel. Only
COPY FROM jobs have the parsing panel on the bottom of the monitoring page.
Last modified 6mo ago