Data Violations
The Data Violations tab applies to source datasets, and displays the expectations within jobs that ingest to the dataset. Expectations enable you to define how you expect the data in a column to arrive, for example, values should be within a range, a set number of characters, or not NULL. By applying expectations during ingestion to capture data quality issues in-flight, you can prevent unwanted values reaching your analytics target.
Expectations
The count of expectations monitoring the data that is ingested into the dataset is displayed in the tab header, and the Expectations table details all expectations monitoring the data flowing into your dataset:
The Expectations table includes the following information:
| Column | Description |
|---|---|
Expectation | The name given to the expectation when it was created. |
Action | The action that should be taken when the expectation is met: either warn or drop. |
Job | The name of the job in which the expectation is included. Multiple jobs may be writing to the dataset. Click on the job name to view the Job Status page. |
Violations Today | The count of violations that have been captured today (UTC time). |
Total Violations | The total number of violations since the expectation was created. Violations prior to the jobs METADATA_RETENTION will not be counted. |
You can filter the expectations that are visible on this tab, by switching between All, Warn, and Drop. Furthermore, use the Search textbox to filter expectations by name.
Learn More
To discover how expectations can define quality conditions on your data, please see the article Managing data quality - ingesting data with expectations.
Last updated
