Data Health Dashboard
This page describes the Data Health dashboard page, its components, and how to get the most out of it.
To monitor the health of your data on an ongoing basis, Telmai provides you with a business dashboard about all your data sources and their respective health metrics. This dashboard consists of three sections:
Page Components
Summary Section
This section summarized the datasets that are connected to Telmai. You will see:
Total count of data source(s).
Total count of monitored attribute(s).
A tile for each data source type currently connected. Each tile will have:
Count of datasets monitored.
Count of attributes monitored.
Detections Overtime
This component allows you to visualize detections (or alerts) that were identified overtime across all your datasources. The detections are grouped by their severity (High, Med, Low).
This is done by showing:
A graph that summarizes the count of alerts every day by severity.
A table that lists the top 100 alert based on current selection.
Clicking anywhere in the graph or the table selector will allow exploring more these alerts. That is:
You can click on the graph to see alerts on a given time.
You can use the dropdown selector to look at a specific data source.
You can use the dropdown selector to look at a specific alert.
You can use a combination of these filters as you will.
Finally, clicking on any attribute alert, will navigate you to the trends section to get more details on the corresponding alert.
DQ KPIs Statistics
This components summarized the data upload status, as well as, health KPIs across all datasets.
Data Upload Status:
This section describes when was the last time the corresponding dataset was uploaded and if a schedule is setup. You will be able to see columns:
Last upload: Time since last upload with status green checkmark that means success, and red alert icon for processing failure.
Schedule: Upload schedule if setup
Data Health KPIs
Total Total Record Count
This KPI helps to track changes in the total size of the monitored table and is only available when CDC is enabled (i.e., the “delta only” flag is on). If the flag is off or the monitored source is not an SQL database, this field will be set to N/A.
Record Count
This KPI helps to track changes in the size of the monitored data. If CDC is enabled it reflects the size of the delta, otherwise, it's the size of the entire set of data.
Completeness
This KPI helps to track the percentage of null/missing/placeholder values. Completeness is tracked both at the data source level as well as the attribute level:
Attribute level: percent of records where the attribute value is not null, not empty, or not one of the user-defined placeholders like N/A.
Data Source level: the compounded average of all attribute level completeness within a data source.
Correctness
This KPI helps to track the validity of values, based on expectations set by a user. Correctness is tracked both at the data source level as well as the attribute level. Correctness is calculated only for attributes where Expectations are set, otherwise, the default is 100%.
Attribute level: percent of records where the attribute value meets all expectations, set for the attribute.
Source level: the compounded average of all attribute level correctness within a data source
Freshness
This KPI helps to track record level freshness. Record level freshness is defined by setting an expectation on a timestamp attribute within the data source (e.g.,. “Record Update Date” attribute to be no more than 1 month from now). To mark an attribute as a timestamp use the Advanced section of the Edit Connection menu. If the timestamp attribute is not configured this KPI will be N/A.
Uniqueness
This KPI helps to track the uniqueness of the records based on an ID attribute. The KPI measures the ratio of records with unique id versus the total number of records. For example, if out of 10 records, there are 2 records sharing the same value the uniqueness is 80%.
An attribute can be marked as an ID attribute in the Advanced section of the Edit Connection menu. If ID attribute is not configured this KPI will be N/A
Accuracy
This KPI tracks the accuracy of the values, based on historical data analysis, and detects when current values don’t match predictions. For example, if the revenue for the company Acme Inc was slowly growing from $4M to $5M over the past year, but in today’s observation it’s 20M, then such a value is considered inaccurate. Accuracy is only calculated for attributes that have “Business Metrics” Configured, otherwise, the default is 100%.
Attribute level: the ratio between business metrics dimensions where no drift was detected to the total number of dimensions. If multiple business metrics are defined for an attribute, then the minimum accuracy is picked for this attribute.
Data Source level: the compounded average of all attribute level accuracy within a data source.
Last updated
