Hi Team,
Please help me with an example or pointer to documentation to properly set requested metrics for “Column Pair Map Expectation” type like “expect_column_pair_values_to_be_equal”.
I was successful to set it up for “Column Map Expectation” like “expect_column_values_to_not_be_null”.
Documentation mentioned use of:
expectation_suite_name:
metric_name:
- metric_kwargs_id
Where:
metric_kwargs_id is a string representation of the Metric Kwargs that can be used as a database key.
For simple cases, it could be easily readable, such as column=Age, but when there are multiple keys and values or complex values,
it will most likely be an md5 hash of key/value pairs.
I can not find example how to create md5 hash of key/value pairs.
Here is part of my “action_list” dictionary for checkpoint.
{
"name": "store_metrics",
"action": {
"class_name": "StoreMetricsAction",
"target_store_name": "metric_store",
"requested_metrics": {
"TRANSACTION_FACT Referential Integrity Tests": [
{
"column": {
"DW_SECURITY_ID": [
"expect_column_values_to_not_be_null.result.element_count",
"expect_column_values_to_not_be_null.result.unexpected_index_list",
"expect_column_values_to_not_be_null.result.missing_count",
"expect_column_values_to_not_be_null.result.missing_percent",
"expect_column_values_to_not_be_null.result.unexpected_count",
"expect_column_values_to_not_be_null.result.unexpected_percent",
"expect_column_values_to_not_be_null.result.unexpected_percent_nonmissing",
"expect_column_values_to_not_be_null.result.unexpected_list",
"expect_column_values_to_not_be_null.result.unexpected_index_query"
]
}
},
**{ **
** “column_A,column_B”: {**
** “DW_SECURITY_ID,DIM_SECURITY_ID”: **
** [**
** “expect_column_pair_values_to_be_equal.table.row_count”,**
** "expect_column_pair_values_to_be_equal.column_pair_values.equal.filtered_row_count,**
** “expect_column_pair_values_to_be_equal.column_pair_values.equal.unexpected_count”,**
** “expect_column_pair_values_to_be_equal.column_pair_values.equal.unexpected_values”**
** ]**
** }**
},
“statistics.successful_expectations”,
“statistics.evaluated_expectations”,
“statistics.success_percent”,
“statistics.unsuccessful_expectations”
]
}
}
}
Any help is greatly appreciated. Thanks.